Generative AI 101: Tokens, Pre-training, Fine-tuning, Reasoning — With SemiAnalysis CEO Dylan Patel

Dylan Patel is the founder and CEO of SemiAnalysis. He joins Big Technology Podcast to explain how generative AI work, explaining the inner workings of tokens, pre-training, fine-tuning, open source, and reasoning. We also cover DeepSeek’s efficiency breakthrough, the race to build colossal AI data centers, and what GPT-5’s hybrid training approach could unlock. Hit play for a masterclass you’ll want to send to every friend puzzled (or excited) about the future of AI. Chapters: 00:00 - Introduction to Generative AI with Dylan Patel 02:00 - Basics of AI Model Training 04:30 - Understanding Tokens and Word Representation 07:00 - How Models Process Language Patterns 10:00 - Attention Mechanisms and Context Understanding 13:00 - Pre-Training: Learning from Internet Data 16:00 - Loss Minimization and Learning Processes 19:00 - Why GPUs Are Perfect for AI Computation 22:00 - Post-Training and Model Personalities 25:00 - Reasoning: How Modern AI Models Think 28:00 - The Growing Efficiency of AI Models 31:00 - Data Center Build-Outs Despite Increasing Efficiency 34:00 - The Future of GPT-5 and AI Development