Transformer Neural Networks, ChatGPT's foundation, Clearly Explained!!!
StatQuest with Josh Starmer
@statquestAbout
Statistics, Machine Learning, Data Science, and AI seem like very scary topics, but since each technique is really just a combination of small and simple steps, they are actually quite simple. My goal with StatQuest is to break down the major methodologies into easy-to-understand pieces. That said, I don't dumb down the material. Instead, I build up your understanding so that you are smarter. Contact, Video Index, Etc: https://statquest.org
Video Description
Transformer Neural Networks are the heart of pretty much everything exciting in AI right now. ChatGPT, Google Translate and many other cool things, are based on Transformers. This StatQuest cuts through all the hype and shows you how a Transformer works, one-step-at-a time. NOTE: If you're interested in learning more about Backpropagation, check out these 'Quests: The Chain Rule: https://youtu.be/wl1myxrtQHQ Gradient Descent: https://youtu.be/sDv4f4s2SB8 Backpropagation Main Ideas: https://youtu.be/IN2XmBhILt4 Backpropagation Details Part 1: https://youtu.be/iyn2zdALii8 Backpropagation Details Part 2: https://youtu.be/GKZoOHXGcLo If you're interested in learning more about the SoftMax function, check out: https://youtu.be/KpKog-L9veg If you're interested in learning more about Word Embedding, check out: https://youtu.be/viZrOnJclY0 If you'd like to learn more about calculating similarities in the context of neural networks and the Dot Product, check out: Cosine Similarity: https://youtu.be/e9U0QAFbfLI Attention: https://youtu.be/PSs6nxngL6k For a complete index of all the StatQuest videos, check out: https://statquest.org/video-index/ If you'd like to support StatQuest, please consider... Patreon: https://www.patreon.com/statquest ...or... YouTube Membership: https://www.youtube.com/channel/UCtYLUTtgS3k1Fg4y5tAhLbw/join ...buying one of my books, a study guide, a t-shirt or hoodie, or a song from the StatQuest store... https://statquest.org/statquest-store/ ...or just donating to StatQuest! https://www.paypal.me/statquest Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter: https://twitter.com/joshuastarmer 0:00 Awesome song and introduction 1:26 Word Embedding 7:30 Positional Encoding 12:53 Self-Attention 23:37 Encoder and Decoder defined 23:53 Decoder Word Embedding 25:08 Decoder Positional Encoding 25:50 Transformers were designed for parallel computing 27:13 Decoder Self-Attention 27:59 Encoder-Decoder Attention 31:19 Decoding numbers into words 32:23 Decoding the second token 34:13 Extra stuff you can add to a Transformer #StatQuest #Transformer #ChatGPT
Master AI Today
AI-recommended products based on this video
Neck Massager, Electric Cervical Neck Massager with Fingers, Simulated Hands Necks and Shoulder Massagers for Back Upper Body Muscle Relaxation, Rechargeable USB-C Gifts for Family Friends
CIVO Womens Watch Stainless Steel Waterproof Analog Quartz Wrist Watch Fashion Elegant Designer Unique Two Hands with Diamond Time Scale Dial
Thunelec Bluetooth 5.3 Headset, 90Hrs Dual Noise Cancelling Mics, Single-Ear Wireless Earpiece, Hands Free Headset with APTX HD Audio & Mute Button for Trucker/Office/Business, IPX7 Waterproof, Gray
Miracase Phone Holders for Your Car, Multifunctional Car Phone Holder with Suction Cup Base, Hands Free Universal Cell Phone Car Mount, Automobile Phone Mount Car for iPhone Android Smartphones(Base)
Skytech Archangel Gaming PC Desktop – AMD Ryzen 5 3600 3.6 GHz, NVIDIA RTX 3060, 1TB NVME SSD, 16GB DDR4 RAM 3200, 600W Gold PSU, 11AC Wi-Fi, Windows 11 Home 64-bit
Skytech Blaze 3.0 Gaming PC Desktop – Intel Core i5 12400F 2.5 GHz, NVIDIA RTX 3060, 500GB NVME SSD, 16GB DDR4 RAM 3200, 600W Gold PSU, 11AC Wi-Fi, Windows 11 Home 64-bit
MSI NVIDIA GeForce RTX 3050 Ventus 2X XS 8G OC Graphics Card - 8 GB GDDR6, 1807 MHz, PCI Express Gen 4, 128 Bits, DP v 1.4a, DL DVI-D, HDMI 2.1 (Supports 4K at 120Hz)
