Coding a ChatGPT Like Transformer From Scratch in PyTorch

Video Description

In this StatQuest we walk through the code required to code your own ChatGPT like Transformer in PyTorch and we do it one step at a time, with every little detail clearly explained. NOTE: This StatQuest assumes that you are already familiar with the concepts behind... Decoder-Only Transformers: https://youtu.be/bQ5BoolX9Ag The Essential Matrix Algebra for Neural Networks: https://youtu.be/ZTt9gsGcdDo The Matrix Math Behind Transformers: https://youtu.be/KphmOJnLAdI You can get the code here: https://github.com/StatQuest/decoder_transformer_from_scratch The full Neural Networks playlist, from the basics to AI, is here: https://www.youtube.com/watch?v=CqOfi41LfDw&list=PLblh5JKOoLUIxGDQs4LFFD--41Vzf-ME1 Learn more about GiveInternet.org: https://giveinternet.org/StatQuest NOTE: Donations up to $30 will be matched by an Angel Investor - so a $30 donation would give $60 to the organization. DOUBLE BAM!!! For a complete index of all the StatQuest videos, check out: https://statquest.org/video-index/ If you'd like to support StatQuest, please consider... Patreon: https://www.patreon.com/statquest ...or... YouTube Membership: https://www.youtube.com/channel/UCtYLUTtgS3k1Fg4y5tAhLbw/join ...buying one of my books, a study guide, a t-shirt or hoodie, or a song from the StatQuest store... https://statquest.org/statquest-store/ ...or just donating to StatQuest! https://www.paypal.me/statquest Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter: https://twitter.com/joshuastarmer 0:00 Awesome song and introduction 1:12 Loading the modules 2:04 Creating the training dataset 6:17 Coding Position Encoding 14:09 Coding Attention 21:04 Coding a Decoder-Only Transformer 26:39 Running the model (untrained) 29:18 Training and using the model #StatQuest #PyTorch #chatgpt