Efficient LLM Inference with SGLang, Lianmin Zheng, xAI

AMD Developer Central • December 18, 2024

AMD Developer Central

About

This channel is dedicated to technology, education, demos, and discussions created by and for developers, with a focus on topics across AI, HPC, gaming and adaptive computing. High performance, open-source solutions, optimized to help you advance at the forefront of technology. Join us to stay updated on the latest developer resources and insights.

Efficient LLM Inference with SGLang, Lianmin Zheng, xAI

AMD Developer Central

About

Latest Posts

Run GPT-OSS-120B Locally with Lemonade on AMD ROCm™ Software

Empowering Developers with AMD ROCm Software Everywhere

Utilizing Low-precision datatypes in PyTorch and Beyond

Blazing-Fast GenAI: How Fireworks AI and Crusoe Are Unleashing Performance with AMD Instinct GPUs

You May Also Like

Boost Your AI Workstation

GEEKOM A8 AI Mini PC (3-Year Quality Support) with AMD Ryzen 9 8945HS Mini Computers, 32GB DDR5 RAM&amp;2TB M.2 2280 NVMe Gen4*4 SSD, Windows 11 Pro Mini Desktop, WiFi 6E/BT5.2/USB4/8K/Quad Display

GMKtec K11 Mini PC, AMD Ryzen 9 8945HS(8C/16T, Up to 5.2GHz) W11 Pro, 32GB DDR5 RAM 1TB Mini Desktop Computer, HDMI/DisplayPort/USB4*2

MINISFORUM AMD Ryzen 9 9955HX MS-A2 Mini PC (16C/32T, up to 5.4GHz), 64GB DDR5 1TB SSD, PCIe×16, HDMI/2x USB-C (8K@60Hz), 2X SFP+ 10G, 2X 2.5G LAN, 3X SSD M.2 (2280/22110/U.2)

AtomMan G7 Pt Mini PC AMD Ryzen 9 7945HX(16C/32T, up to 5.4GHz) 32GB DDR5 1TB PCIe4.0 SSD Micro Computer, HDMI+DP+USB-C Output, 2.5G LAN, WiFi7, BT5.4, 4xUSB AMD Radeon RX 7600M XT Graphics Gaming PC

Seasonic Focus V4 GX-1000 (ATX3) - 1000W - 80+ Gold - ATX 3.0 &amp; PCIe 5.1 Ready -Full-Modular -ATX Form Factor -Premium Japanese Capacitor -10 Year Warranty -Nvidia RTX 30/40 Super &amp; AMD GPU Compatible

Skytech Legacy Gaming PC Desktop, Ryzen 7 9800X3D 4.7 GHz (5.2 GHz Turbo Boost), NVIDIA RTX 5080 16GB GDDR7, 1TB Gen4 SSD, 32GB DDR5 RAM 6000 RGB, 1000W Gold ATX 3.0 PSU, 360 ARGB AIO, Wi-Fi, Win 11

HP Victus 15.6&quot; 144Hz FHD Gaming Laptop, Intel i5-12450H, 32GB RAM, 1TB PCIe SSD, NVIDIA GeForce RTX 3050, Backlit Keyboard, HD Webcam, Win 11, Blue, 256GB Docking Station Set

Corsair Vengeance LPX 32GB (2 X 16GB) DDR4 3200 (PC4-25600) C16 1.35V Desktop Memory - Black

SABRENT Rocket 5 M.2 NVMe SSD 4TB PCIe GEN 5, 14Gbps X4 Internal Solid State Drive, 2280 Gaming Performance Hard Drive, Backward Compatible (SB-RKT5-4TB)

Asus ROG Strix 1000W Gold Aura Edition (Fully Modular Power Supply, 80+ Gold Certified, ATX 3.0, Cybenetics Lambda A+ Certification, PCIe Gen 5.0 Ready, Axial-tech Fan, Aura Sync, 10-Year Warranty)

Corsair RM1000e Fully Modular Low-Noise ATX Power Supply - Dual EPS12V Connectors - 105°C-Rated Capacitors - 80 Plus Gold Efficiency - Modern Standby Support - Black

Logitech G305 LIGHTSPEED Wireless Gaming Mouse, Hero 12K Sensor, 12,000 DPI, Lightweight, 6 Programmable Buttons, 250h Battery Life, On-Board Memory, PC/Mac - Black

Logitech M185 Wireless Mouse, 2.4GHz with USB Mini Receiver, 12-Month Battery Life, 1000 DPI Optical Tracking, Ambidextrous, Compatible with PC, Mac, Laptop - Black

Logitech G203 Wired Gaming Mouse, 8,000 DPI, Rainbow Optical Effect LIGHTSYNC RGB, 6 Programmable Buttons, On-Board Memory, Screen Mapping, PC/Mac Computer and Laptop Compatible - Black

Logitech G435 Lightspeed and Bluetooth Wireless Gaming Headset - Lightweight Over-Ear Headphones, Built-in mics, 18h Battery, Compatible with Dolby Atmos, PC, Mobile - Black

4K Portable Monitor, MAGICRAVEN 15.6inch UHD 3840×2160 100% sRGB Portable Gaming Monitor, USB-C Mini HDMI External Second Monitor Portable with Speakers for Laptop MacBook Switch PS5/4 Xbox

JUNEBOX 4K Portable Monitor 15.6&quot; 3840 * 2160 HDR Display, 500nits 8bit QHD USB-C HDMI Laptop Monitor, 16:9 IPS Eye Care External Second Screen for Mac PC Phone PS4/5 Xbox Switch

USB C Docking Station Dual Monitor for Dell Hp,15-in-1 Laptop Docking Station 3 Monitors USB C Hub with Dual 4K HDMI,8K DP,Button,PD Charging,Ethernet,6 USB A&amp;C,SD/TF, Audio USB-C Multiport Adapter

Loading...

GEEKOM A8 AI Mini PC (3-Year Quality Support) with AMD Ryzen 9 8945HS Mini Computers, 32GB DDR5 RAM&2TB M.2 2280 NVMe Gen4*4 SSD, Windows 11 Pro Mini Desktop, WiFi 6E/BT5.2/USB4/8K/Quad Display

Seasonic Focus V4 GX-1000 (ATX3) - 1000W - 80+ Gold - ATX 3.0 & PCIe 5.1 Ready -Full-Modular -ATX Form Factor -Premium Japanese Capacitor -10 Year Warranty -Nvidia RTX 30/40 Super & AMD GPU Compatible

HP Victus 15.6" 144Hz FHD Gaming Laptop, Intel i5-12450H, 32GB RAM, 1TB PCIe SSD, NVIDIA GeForce RTX 3050, Backlit Keyboard, HD Webcam, Win 11, Blue, 256GB Docking Station Set

JUNEBOX 4K Portable Monitor 15.6" 3840 * 2160 HDR Display, 500nits 8bit QHD USB-C HDMI Laptop Monitor, 16:9 IPS Eye Care External Second Screen for Mac PC Phone PS4/5 Xbox Switch

USB C Docking Station Dual Monitor for Dell Hp,15-in-1 Laptop Docking Station 3 Monitors USB C Hub with Dual 4K HDMI,8K DP,Button,PD Charging,Ethernet,6 USB A&C,SD/TF, Audio USB-C Multiport Adapter