Running Deepseek-R1 671B without a GPU

Video Description

We ran a giant AI model, the Deepseek-R1 671B FP16 model, on an AMD EPYC 9965 server to see if the CPU server could handle what many GPU servers cannot. What is more, we ran the large model alongside traditional virtualization workloads to see how that impacted performance. We even have a how-to so you can get started running AI in your virtualization cluster using Open WebUI and Ollama. STH Main Site Article: https://www.servethehome.com/running-... Substack: https://axautikgroupllc.substack.com/ STH Top 5 Weekly Newsletter: https://eepurl.com/dryM09 ---------------------------------------------------------------------- Become a STH YT Member and Support Us ---------------------------------------------------------------------- Join STH YouTube membership to support the channel:    / @servethehomevideo   Professional Users Substack: https://axautikgroupllc.substack.com/ ---------------------------------------------------------------------- Where to Find STH ---------------------------------------------------------------------- STH Forums: https://forums.servethehome.com Follow on Twitter:   / servethehome   Follow on LinkedIn:   / servethehome-com   Follow on Facebook:   / servethehome   Follow on Instagram:   / servethehome   ---------------------------------------------------------------------- Other STH Content Mentioned in this Video ---------------------------------------------------------------------- Inside a 100K GPU AI Cluster:    • Inside the World's Largest AI Supercluster...   AMD EPYC 9005 Turin:    • 768 Threads Per Server AMD EPYC 9005 Turin...   Buying the most popular server on Newegg:    • We Bought the Most Popular Server on Neweg...   128GB AI mini PC:    • A 128GB AMD Ryzen AI Mini PC is Here   Most unique 8x GPU AI server:    • The most UNIQUE 8x GPU AI Server from ASRo...   Supermicro NVIDIA H200:    • Inside the SUPER NVIDIA H200 Server From S...   Aivres NVIDIA H200:    • Inside a MEGA AI GPU Server with the NVIDI...   ---------------------------------------------------------------------- Timestamps ---------------------------------------------------------------------- 00:00 Introduction 01:43 Why we are running a AI on a virtualization server 04:07 Taking a look at the AMD Volcano 2P AMD EPYC 9965 System 06:10 How-to Setup Open WebUI and Ollama with Deepseek-R1 671b and FP16 10:00 AMD EPYC 9965 Performance on Deepseek-R1 671B FP16 virtualized and bare metal 14:21 Tip for 1P Operation and Not Hurting Performance 16:43 Key Lessons Learned Running LLMs on a Virtualization Host