Sign In

Variational Autoencoder - Model, ELBO, loss function and maths explained easily!

Umar Jamil • June 7, 2023

Video Thumbnail

View Channel

About

No channel description available.

Latest Posts

Titans: Learning to Memorize at Test Time

Titans: Learning to Memorize at Test Time

Umar Jamil1 year ago

13559

Paper: DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

Paper: DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

Umar Jamil1 year ago

58775

Flash Attention derived and coded from first principles with Triton (Python)

Flash Attention derived and coded from first principles with Triton (Python)

Umar Jamil1 year ago

70025

Coding a Multimodal (Vision) Language Model from scratch in PyTorch with full explanation

Coding a Multimodal (Vision) Language Model from scratch in PyTorch with full explanation

Umar Jamil1 year ago

113998

Video Description

A complete explanation of the Variational Autoencoder, a key component in Stable Diffusion models. I will show why we need it, the idea behind the ELBO, the problems in maximizing the ELBO, the loss function and explain the math derivations step by step. Link to the slides: https://github.com/hkproj/vae-from-scratch-notes Chapters 00:00 - Introduction 00:41 - Autoencoder 02:35 - Variational Autoencoder 04:20 - Latent Space 06:06 - Math introduction 08:45 - Model definition 12:00 - ELBO 16:05 - Maximizing the ELBO 19:49 - Reparameterization Trick 22:41 - Example network 23:55 - Loss function

You May Also Like

ML Interpretability: feature visualization, adversarial example, interp. for language models

ML Interpretability: feature visualization, adversarial example, interp. for language models

Umar Jamil1 year ago

11544

Kolmogorov-Arnold Networks: MLP vs KAN, Math, B-Splines, Universal Approximation Theorem

Kolmogorov-Arnold Networks: MLP vs KAN, Math, B-Splines, Universal Approximation Theorem

Umar Jamil1 year ago

38331

Direct Preference Optimization (DPO) explained: Bradley-Terry model, log probabilities, math

Direct Preference Optimization (DPO) explained: Bradley-Terry model, log probabilities, math

Umar Jamil1 year ago

33076

Reinforcement Learning from Human Feedback explained with math derivations and the PyTorch code.

Reinforcement Learning from Human Feedback explained with math derivations and the PyTorch code.

Umar Jamil2 years ago

60282

Mamba and S4 Explained: Architecture, Parallel Scan, Kernel Fusion, Recurrent, Convolution, Math

Mamba and S4 Explained: Architecture, Parallel Scan, Kernel Fusion, Recurrent, Convolution, Math

Umar Jamil2 years ago

58115

Mistral / Mixtral Explained: Sliding Window Attention, Sparse Mixture of Experts, Rolling Buffer

Mistral / Mixtral Explained: Sliding Window Attention, Sparse Mixture of Experts, Rolling Buffer

Umar Jamil2 years ago

39155

Distributed Training with PyTorch: complete tutorial with cloud infrastructure and code

Distributed Training with PyTorch: complete tutorial with cloud infrastructure and code

Umar Jamil2 years ago

30816

How diffusion models work - explanation and code!

How diffusion models work - explanation and code!

Umar Jamil2 years ago

30368

Quantization explained with PyTorch - Post-Training Quantization, Quantization-Aware Training

Quantization explained with PyTorch - Post-Training Quantization, Quantization-Aware Training

Umar Jamil2 years ago

47464

Retrieval Augmented Generation (RAG) Explained: Embedding, Sentence BERT, Vector Database (HNSW)

Retrieval Augmented Generation (RAG) Explained: Embedding, Sentence BERT, Vector Database (HNSW)

Umar Jamil2 years ago

80853

BERT explained: Training, Inference, BERT vs GPT/LLamA, Fine tuning, [CLS] token

BERT explained: Training, Inference, BERT vs GPT/LLamA, Fine tuning, [CLS] token

Umar Jamil2 years ago

72005

Coding Stable Diffusion from scratch in PyTorch

Coding Stable Diffusion from scratch in PyTorch

Umar Jamil2 years ago

209397

Coding LLaMA 2 from scratch in PyTorch - KV Cache, Grouped Query Attention, Rotary PE, RMSNorm

Coding LLaMA 2 from scratch in PyTorch - KV Cache, Grouped Query Attention, Rotary PE, RMSNorm

Umar Jamil2 years ago

61776

LLaMA explained: KV-Cache, Rotary Positional Embedding, RMS Norm, Grouped Query Attention, SwiGLU

LLaMA explained: KV-Cache, Rotary Positional Embedding, RMS Norm, Grouped Query Attention, SwiGLU

Umar Jamil2 years ago

106892

Segment Anything - Model explanation with code

Segment Anything - Model explanation with code

Umar Jamil2 years ago

36363

LoRA: Low-Rank Adaptation of Large Language Models - Explained visually + PyTorch code from scratch

LoRA: Low-Rank Adaptation of Large Language Models - Explained visually + PyTorch code from scratch

Umar Jamil2 years ago

45831

AiChat Products Saved Account