Signin

Mixtral of Experts (Paper Explained)

Yannic Kilcher • May 1, 2024

Video Thumbnail

You May Also Like

Video Thumbnail

Safety Alignment Should be Made More Than Just a Few Tokens Deep (Paper Explained)

Yannic Kilcher

Video Thumbnail

Were RNNs All We Needed? (Paper Explained)

Yannic Kilcher

Video Thumbnail

Mistral / Mixtral Explained: Sliding Window Attention, Sparse Mixture of Experts, Rolling Buffer

Umar Jamil

Video Thumbnail

On the Biology of a Large Language Model (Part 1)

Yannic Kilcher

Video Thumbnail

3D Gaussian Splatting! - Computerphile

Computerphile

Video Thumbnail

[GRPO Explained] DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

Yannic Kilcher

Video Thumbnail

DeepSeek R1 Theory Overview | GRPO + RL + SFT

Deep Learning with Yacine

Video Thumbnail

Richard Wolff: Trump, Hitler, and the End of the American Empire

Robinson Erhardt

Video Thumbnail

The unexpected logic behind rolling multiple dice and picking the highest.

Stand-up Maths

Video Thumbnail

DeepSeek-V3

Gabriel Mongaras

Video Thumbnail

Mixture of Experts Explained: From Vowel Recognition to Trillion-Parameter LLMs (Llama 4, DeepSeek)

Julia Turc

Video Thumbnail

Transformers (how LLMs work) explained visually | DL5

3Blue1Brown

Video Thumbnail

Everything We Know About 'Oumuamua

Astrum

Video Thumbnail

The Dark Matter of AI [Mechanistic Interpretability]

Welch Labs

Video Thumbnail

V-JEPA: Revisiting Feature Prediction for Learning Visual Representations from Video (Explained)

Yannic Kilcher

Video Thumbnail

A Visual Guide to Mixture of Experts (MoE) in LLMs

Maarten Grootendorst

Video Thumbnail

Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters (Paper)

Yannic Kilcher

Video Thumbnail

AlphaGeometry: Solving olympiad geometry without human demonstrations (Paper Explained)

Yannic Kilcher

Video Thumbnail

Stanford CS25: V4 I Demystifying Mixtral of Experts

Stanford Online

Video Thumbnail

1 Million Tiny Experts in an AI? Fine-Grained MoE Explained

bycloud

Yannic Kilcher

About

No channel description available.

Latest Posts

Video Thumbnail

All about AI Accelerators: GPU, TPU, Dataflow, Near-Memory, Optical, Neuromorphic & more (w/ Author)

Yannic Kilcher

Video Thumbnail

xLSTM: Extended Long Short-Term Memory

Yannic Kilcher

Video Thumbnail

Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention

Yannic Kilcher

Video Thumbnail

[GRPO Explained] DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

Yannic Kilcher