Knowledge Distillation: How LLMs train each other
Julia Turc
•
April 30, 2025
Julia Turc
View ChannelAbout
AI explainer videos from a former Google Research engineer, now startup founder. Expect American optimism with a solid dose of Eastern European cynicism. Find (free) supplemental materials (slide decks, paper reading lists, Colab notebooks) on Patreon 👉 https://www.patreon.com/c/JuliaTurc
Latest Posts
Essential LLM Training Tools
AI-recommended products based on this video
Loading...
Seasonic Focus V4 GX-1000 (ATX3) - 1000W - 80+ Gold - ATX 3.0 & PCIe 5.1 Ready -Full-Modular -ATX Form Factor -Premium Japanese Capacitor -10 Year Warranty -Nvidia RTX 30/40 Super & AMD GPU Compatible
(22)
$423.35
FREE delivery Oct 8 - 10
Loading...
Seasonic Focus V4 GX-1000 (ATX3) - 1000W - 80+ Gold - ATX 3.0 & PCIe 5.1 Ready -Full-Modular -ATX Form Factor -Premium Japanese Capacitor -10 Year Warranty -Nvidia RTX 30/40 Super & AMD GPU Compatible
(22)
$423.35
FREE delivery Oct 8 - 10
Loading...
【DDR3 RAM Laptop Only】 GIGASTONE 16GB Kit (2x8GB) DDR3/DDR3L 1600MHz (1333MHz) PC3-12800 (PC3-10600) CL11 1.35V/1.5V 2Rx8 SODIMM 204 Pin Unbuffered Non ECC High Performance Notebook Memory Upgrade
(1,599)
$32.99
FREE delivery Sun, Jun 15 on your first order
50+ bought in past month
Loading...
TEAMGROUP Elite DDR4 16GB Kit (2 x 8GB) 3200MHz PC4-25600 CL22 Unbuffered Non-ECC 1.2V SODIMM 260-Pin Laptop Notebook PC Computer Memory Module Ram Upgrade - TED416G3200C22DC-S01
(3,629)
$57.46
$7.05 delivery Jun 17 - 20Only 3 left in stock.
Loading...
Laplink PCmover Ultimate 11 - Migration of your Applications, Files and Settings from an Old PC to a New PC - Data Transfer Software - With Optional High Speed Ethernet Cable - 1 License
(4,609)
$54.95
PrimeFREE delivery Sat, Jun 14
Loading...
Best Seller
Syntech Link Cable Compatible with Meta/Oculus Quest 3S/Quest 3/Quest2/Pro/Pico 4/Ultra and PC/SteamVR, 16FT Upgraded Type C Cable with USB 3.0 Adapter, High Speed Data Transfer Cord for VR Headset
(1,405)
$26.99
FREE delivery Tue, Jun 17 on your first order
100+ bought in past month
Loading...
![Short USB C Cable [4-Pack 8 inch], SUMPK 3A Fast Charging USB 2.0 Data Transfer to Type C Phone Charger Cord Nylon Braided Compatible with Samsung Galaxy S10 S9, Pixel 3, LG V30, Charging Station](https://m.media-amazon.com/images/I/71GPEzylAaS._AC_UL960_FMwebp_QL65_.jpg)
Short USB C Cable [4-Pack 8 inch], SUMPK 3A Fast Charging USB 2.0 Data Transfer to Type C Phone Charger Cord Nylon Braided Compatible with Samsung Galaxy S10 S9, Pixel 3, LG V30, Charging Station
(1,702)
$11.99
FREE delivery Tue, Jun 17 on your first order
Loading...
Best Seller
Syntech Link Cable 16FT for PCVR Compatible with Oculus/Quest 3/Meta Quest 3S, Quest2/Pro/Pico 4/Ultra Accessories and PC/SteamVR, High Speed PC Data Transfer, USB 3.0 to USB C Cable for VR Headset
(19,554)
$24.99
FREE delivery Tue, Jun 17 on your first order
300+ bought in past month
Loading...
Corsair MP600 PRO LPX 4TB M.2 NVMe PCIe x4 Gen4 SSD - Optimized for PS5 (Up to 7,100MB/sec Sequential Read & 6,800MB/sec Sequential Write Speeds, High-Speed Interface, Compact Form Factor) Black
(8,521)
$449.99
FREE delivery Tue, Jun 17
Loading...
LEGO City Mobile Police Dog Training 60369, SUV Toy Car with Trailer, Obstacle Course and Puppy Figures, Animal Playset for Boys and Girls Ages 5 Plus
(1,256)
$33.98
FREE delivery Sat, Jun 14 on your first order
200+ bought in past month