How DeepSeek Rewrote the Transformer [MLA]

Welch Labs • March 5, 2025

Welch Labs

View Channel

About

No channel description available.

Latest Posts

Why Deep Learning Works Unreasonably Well [How Models Learn Part 3]

Welch Labs

The Misconception that Almost Stopped AI [How Models Learn Part 1]

Welch Labs

ChatGPT is made from 100 million of these [The Perceptron]

Welch Labs

The Dark Matter of AI [Mechanistic Interpretability]

Welch Labs

Transform Your AI Learning Today

AI-recommended products based on this video

LEGO City Mobile Police Dog Training 60369, SUV Toy Car with Trailer, Obstacle Course and Puppy Figures, Animal Playset for Boys and Girls Ages 5 Plus

(1,256)

$33.98

FREE delivery Sat, Jun 14 on your first order

200+ bought in past month

soundcore by Anker P20i True Wireless Earbuds, 10mm Drivers with Big Bass, Bluetooth 5.3, 30H Long Playtime, IPX5 Water-Resistant, 2 Mics for AI Clear Calls, 22 Preset EQs, Customization via App

(61,759)

$29.99

FREE delivery Sat, Jun 14 on your first order

4K+ bought in past month

The Pandy - Emotional Healing on Demand, Realistic Panda Plush for Emotional Support, Mimics Natural Movements for Realistic Comfort, Pandy Ai Plush with Ultra-Soft Fur, The Comfort Hug You Have

(0)

$19.88

$4.99 delivery Aug 27 - Sep 9

BASESAILOR iPhone 16 15 Pro Max Charger Cord,USB to USB C Cable 6.6FT/2Pack,Type C Charging Power Cord for Apple iPad 10,Samsung Galaxy AI S25 S24 S23 S22 S20 Plus Ultra Flip,A52 A53 A54,Google Pixel

(29,760)

$12.59

FREE delivery Sat, Jun 14 on your first order

500+ bought in past month

Spigen Case for Galaxy S25 Case, Tough Armor (Ai) MagFit [Wireless Charging] Compatible with MagSafe - Black

(2,126)

$28.99

FREE delivery Sat, Jun 14 on your first order

500+ bought in past month

Skytech Legacy Gaming PC Desktop, Ryzen 7 9800X3D 4.7 GHz (5.2 GHz Turbo Boost), NVIDIA RTX 5080 16GB GDDR7, 1TB Gen4 SSD, 32GB DDR5 RAM 6000 RGB, 1000W Gold ATX 3.0 PSU, 360 ARGB AIO, Wi-Fi, Win 11

(9)

$3,950.00$3,752.50

FREE delivery Wed, Jun 18

HP Victus 15.6" 144Hz FHD Gaming Laptop, Intel i5-12450H, 32GB RAM, 1TB PCIe SSD, NVIDIA GeForce RTX 3050, Backlit Keyboard, HD Webcam, Win 11, Blue, 256GB Docking Station Set

(117)

$1,149.00

FREE delivery Jun 26 - Jul 8

Acer Swift X 14" FHD Laptop, AMD Ryzen 7 5825U, NVIDIA GeForce RTX 3050Ti, 16GB RAM, 512GB SSD, Windows 11 Home

(247)

$1,804.11

Ailltopd Portable Carpet & Upholstery Spot Cleaner, Powerful Suction and Spot Remover, Deep Stain Cleaning for Carpet, Pets, Stairs, Car Seat, Couch, with Dual Size BrushHead and Cleaning Solutions

(21)

$559.96

PrimeFREE delivery Sat, Jun 14

Quiet Ear Plugs – Ultra-Comfy Reusable Noise-Reducing Earplugs for Sleep, Concerts, Deep Focus, Travel, Noise Sensitivity | Flexible Hearing Protection | Customizable Fit | 30dB (SNR) Noise Reduction

(121,527)

$13.99

FREE delivery Tue, Jul 1 on your first order

2K+ bought in past month

Dell S2722DC Monitor - 27-inch WQHD (2560x1440) 75Hz 4Ms Display, AMD FreeSync, Built in Speaker, USB Type-C/USB 3.2 Gen 1/HDMI Connectivity, Height/Pivot/Swivel/Tilt Adjustability - Platinum Silver

(3,971)

$299.99

FREE delivery Mon, Aug 11

100+ bought in past month

Kado 15.6 Inch Portable Monitor Case Sleeve Bag Made for Sceptre Kado MNN KYY InnoView Yxk ForHelp AYY VisionOwl Yodoit Uperfect ViewSonic ASUS Dell Lenovo 15.6" Portable Display Monitors 15.55"x9.25"

(15)

$61.81

FREE delivery Aug 12 - 14

USB C Hub Triple Monitor Adapters 8 in 1 USB C Docking Station with Dual HDMI Displayport VGA, USB C 100w PD,3USB2.0 Multiport Dongle for Dell XPS 17/15/13,Lenovo Lenovo Thinkpad,HP Envy,MacBook

(2,293)

$39.99

400+ bought in past month

Best Seller

Logitech G502 HERO High Performance Wired Gaming Mouse, HERO 25K Sensor, 25,600 DPI, RGB, Adjustable Weights, 11 Programmable Buttons, On-Board Memory, PC / Mac, Black

(53,722)

$49.99$39.99

FREE delivery Tue, Jul 1

1K+ bought in past month

Anker 332 USB-C Hub (5-in-1) with 4K HDMI Display, 5Gbps - and 2 5Gbps USB-A Data Ports and for MacBook Pro, MacBook Air, Dell XPS, Lenovo Thinkpad, HP Laptops and More

(11,639)

$23.99

FREE delivery Sat, Jun 14 on your first order

1K+ bought in past month

How DeepSeek Rewrote the Transformer [MLA]

Welch Labs

About

Latest Posts

Why Deep Learning Works Unreasonably Well [How Models Learn Part 3]

The Misconception that Almost Stopped AI [How Models Learn Part 1]

ChatGPT is made from 100 million of these [The Perceptron]

The Dark Matter of AI [Mechanistic Interpretability]

You May Also Like

Transform Your AI Learning Today

LEGO City Mobile Police Dog Training 60369, SUV Toy Car with Trailer, Obstacle Course and Puppy Figures, Animal Playset for Boys and Girls Ages 5 Plus

soundcore by Anker P20i True Wireless Earbuds, 10mm Drivers with Big Bass, Bluetooth 5.3, 30H Long Playtime, IPX5 Water-Resistant, 2 Mics for AI Clear Calls, 22 Preset EQs, Customization via App

The Pandy - Emotional Healing on Demand, Realistic Panda Plush for Emotional Support, Mimics Natural Movements for Realistic Comfort, Pandy Ai Plush with Ultra-Soft Fur, The Comfort Hug You Have

BASESAILOR iPhone 16 15 Pro Max Charger Cord,USB to USB C Cable 6.6FT/2Pack,Type C Charging Power Cord for Apple iPad 10,Samsung Galaxy AI S25 S24 S23 S22 S20 Plus Ultra Flip,A52 A53 A54,Google Pixel

Spigen Case for Galaxy S25 Case, Tough Armor (Ai) MagFit [Wireless Charging] Compatible with MagSafe - Black

Skytech Legacy Gaming PC Desktop, Ryzen 7 9800X3D 4.7 GHz (5.2 GHz Turbo Boost), NVIDIA RTX 5080 16GB GDDR7, 1TB Gen4 SSD, 32GB DDR5 RAM 6000 RGB, 1000W Gold ATX 3.0 PSU, 360 ARGB AIO, Wi-Fi, Win 11

HP Victus 15.6&quot; 144Hz FHD Gaming Laptop, Intel i5-12450H, 32GB RAM, 1TB PCIe SSD, NVIDIA GeForce RTX 3050, Backlit Keyboard, HD Webcam, Win 11, Blue, 256GB Docking Station Set

Acer Swift X 14&quot; FHD Laptop, AMD Ryzen 7 5825U, NVIDIA GeForce RTX 3050Ti, 16GB RAM, 512GB SSD, Windows 11 Home

Ailltopd Portable Carpet &amp; Upholstery Spot Cleaner, Powerful Suction and Spot Remover, Deep Stain Cleaning for Carpet, Pets, Stairs, Car Seat, Couch, with Dual Size BrushHead and Cleaning Solutions

Quiet Ear Plugs – Ultra-Comfy Reusable Noise-Reducing Earplugs for Sleep, Concerts, Deep Focus, Travel, Noise Sensitivity | Flexible Hearing Protection | Customizable Fit | 30dB (SNR) Noise Reduction

Dell S2722DC Monitor - 27-inch WQHD (2560x1440) 75Hz 4Ms Display, AMD FreeSync, Built in Speaker, USB Type-C/USB 3.2 Gen 1/HDMI Connectivity, Height/Pivot/Swivel/Tilt Adjustability - Platinum Silver

Kado 15.6 Inch Portable Monitor Case Sleeve Bag Made for Sceptre Kado MNN KYY InnoView Yxk ForHelp AYY VisionOwl Yodoit Uperfect ViewSonic ASUS Dell Lenovo 15.6&quot; Portable Display Monitors 15.55&quot;x9.25&quot;

USB C Hub Triple Monitor Adapters 8 in 1 USB C Docking Station with Dual HDMI Displayport VGA, USB C 100w PD,3USB2.0 Multiport Dongle for Dell XPS 17/15/13,Lenovo Lenovo Thinkpad,HP Envy,MacBook

Logitech G502 HERO High Performance Wired Gaming Mouse, HERO 25K Sensor, 25,600 DPI, RGB, Adjustable Weights, 11 Programmable Buttons, On-Board Memory, PC / Mac, Black

Anker 332 USB-C Hub (5-in-1) with 4K HDMI Display, 5Gbps - and 2 5Gbps USB-A Data Ports and for MacBook Pro, MacBook Air, Dell XPS, Lenovo Thinkpad, HP Laptops and More

Loading...

HP Victus 15.6" 144Hz FHD Gaming Laptop, Intel i5-12450H, 32GB RAM, 1TB PCIe SSD, NVIDIA GeForce RTX 3050, Backlit Keyboard, HD Webcam, Win 11, Blue, 256GB Docking Station Set

Acer Swift X 14" FHD Laptop, AMD Ryzen 7 5825U, NVIDIA GeForce RTX 3050Ti, 16GB RAM, 512GB SSD, Windows 11 Home

Ailltopd Portable Carpet & Upholstery Spot Cleaner, Powerful Suction and Spot Remover, Deep Stain Cleaning for Carpet, Pets, Stairs, Car Seat, Couch, with Dual Size BrushHead and Cleaning Solutions

Kado 15.6 Inch Portable Monitor Case Sleeve Bag Made for Sceptre Kado MNN KYY InnoView Yxk ForHelp AYY VisionOwl Yodoit Uperfect ViewSonic ASUS Dell Lenovo 15.6" Portable Display Monitors 15.55"x9.25"