How to Create a Neural Network (and Train it to Identify Doodles)

Video Description

Exploring how neural networks learn by programming one from scratch in C#, and then attempting to teach it to recognize various doodles and images. Support my work (and get early access to new videos and source code) on Patreon or Nebula * Patreon: https://www.patreon.com/SebastianLague * Nebula: https://go.nebula.tv/sebastianlague Source code and Demo: * GitHub: https://github.com/SebLague/Neural-Network-Experiments * Demo: https://sebastian.itch.io/neural-network-experiment Training Data: * Digits: http://yann.lecun.com/exdb/mnist/ * Fashion: https://github.com/zalandoresearch/fashion-mnist * Doodles: https://github.com/googlecreativelab/quickdraw-dataset * Cifar10: https://www.cs.toronto.edu/~kriz/cifar.html Chapters 0:00 Introduction 2:39 The decision boundary 3:49 Weights 5:42 Biases 6:45 Hidden layers 7:45 Programming the network 9:57 Activation functions 12:42 Cost 15:07 Gradient descent example 18:22 The cost landscape 19:55 Programming gradient descent 21:10 It's learning! (slowly) 23:21 Calculus example 27:34 The chain rule 29:50 Some partial derivatives 33:14 Backpropagation 39:25 Digit recognition 43:56 Drawing our own digits 47:37 Fashion 48:25 Doodles 52:00 The final challenge Music: Cosmic Waves - Michael FK Amber - The Stolen Orchestra Beyond the Horizon - Sounds Like Sander Air - Assaf Ayalon Purest Form - Sounds Like Sander Hear Wide Open - Sounds Like Sander Universal Wonder - Moments Roman P - Moments All In Good Time - Shimmer It Will Come Back - The Stolen Orchestra Frontier - Shimmer New Moon - Cloud Wave Sunflower - Cody Martin Inner Peace - Moments Enchanted - Cody Martin Just Around The Corner - Shimmer