LLM Tokenizer in C
Tsoding Daily
•
March 9, 2025
Tsoding Daily
@tsodingdailyAbout
"Daily" Development Log of Tsoding. If you want to repost a clip from this channel, feel free to do so, but please credit the channel and provide the link to the original video. Thank you!
Latest Posts
Video Description
Streamed Live on Twitch: https://twitch.tv/tsoding Enable Subtitles for Twitch Chat Other Episodes: https://www.youtube.com/playlist?list=PLpM-Dvs8t0VaIVKmfGBztiqaIyMHIH-3Y Chapters: - 00:00:00 - Main Session - 01:53:00 - Footnote References: - https://github.com/tsoding/bpe - https://en.wikipedia.org/wiki/Byte_pair_encoding - https://github.com/rexim/bnfuzzer Topic: First naive implementation of Byte-Pair-Encoding. Episode 1 of BPE series. Support: - https://github.com/tsoding/donate#support-tsoding
You May Also Like
Master C Programming with These Tools
AI-recommended products based on this video
Loading...
10.1 Inch Touch Portable Monitor IPS Screen 1366x768P 60Hz 400 Brightness 99% sRGB HDMI USB-C Monitors Switch for Xbox PS3/4/5 Laptop Compatible with Raspberry Pi, Mini Touch Screen
(1)
$99.99
FREE delivery Mon, Aug 11
Loading...
ELECROW 8 Inch Portable Monitor, 1280x800 Mini HD Display with Built-in Speakers, USB Powered, Non-Touch LCD Screen for Raspberry Pi, PC, Laptop, Jetson Nano, Game Consoles
(49)
$117.29$102.21
$6.86 delivery Thu, Aug 14
Loading...
7 Inch Portable Monitor Touchscreen HD 1024x600 LED Display Dual HDMI Port Small Monitor for PC Raspberry Pi Laptop Computer Xbox PS4/5 Switch Built-in Speakers
(61)
$68.99
FREE delivery Tue, Aug 12
Loading...
BrosTrend 1800Mbps WiFi 6 Linux WiFi Adapter for PC and Raspberry Pi 2+, Long Range USB WiFi Dongle Linux for Ubuntu, Mint, Debian, Kubuntu, Lubuntu, Zorin, Windows 11/10, Dual Band Wireless Antenna
(288)
$54.14
FREE delivery Sun, Jun 22
Loading...
New Slim 65W USB C Laptop Charger for Lenovo ThinkPad Yoga 7i 7 9i 9 2-in-1 Yoga Book 9i Gen 10 ThinkBook 16 15 14 Gen 8/7 13s 13x 14s IdeaPad Slim/Pro 5X 5i 5 3X 3i 3 2-in-1 Type C Power Cord
(13)
$24.99
FREE delivery Sat, Jun 21 on your first order
Loading...
135W USB C AC Adapter Charger for Lenovo ThinkPad Z16 P16s P15s P14s E16 E15 E14 T16 Yoga/Slim Pro 9 7 6 7i 9i Book 9 ThinkBook 16 14p 14 15 14s IdeaPad Slim/Pro 9 7 5 Legion 9 Laptop ADL135YSDC3A
(1)
$59.99
FREE delivery Sun, Jun 22
Loading...
2-Pack Lutein Eye Serum with Roller Ball – Anti-Wrinkle, Dark Circles Removal | Lutein Eye Essence Oil with Vitamin C for Hydration, Fast Absorption | Under Eye Care for Fatigue, Light Protection & Brightening
(2)
$15.77
$2.99 delivery Aug 27 - Sep 9
200+ bought in past month
Loading...
12V/24V 100W Car Laptop Charger USB C for Lenovo Thinkpad MacBook HP Dell Acer Asus,Google,MSI, LG, Samsung,Huawei,Surface pro 9 8, Razer, Chromebook, ROG Ally, Steam Deck and All Type-C Devices
(6)
$29.99
FREE delivery Tue, Jun 17 on your first order
50+ bought in past month
Loading...
DHT11 modules Digital Temperature and Humidity Temperature Sensor for arduino DIY KIT 5pcs
(1)
$15.99
FREE delivery Sat, Jun 14 on your first order
Loading...
Magnetic Nasal Strips Starter Kit: Comfortable Nasal Breathing Support for Sleep, Helps Reduce Snoring Noise, Includes 60 Tabs (30 Uses) with 4 Sizes
(40)
$17.99
FREE delivery Wed, Mar 18 on your first orderOnly 7 left in stock.
1K+ bought in past month
Loading...
Environet Hydroponic Growing Kit, Self-Watering Mason Jar Herb Garden Starter Kit Indoor, Windowsill Herb Garden, Grow Your Own Herbs from Organic Seeds (Basil)
(786)
$69.15$33.51
FREE delivery Jun 13 - 17
Loading...
Herb Garden Planter Indoor Kit 21Pcs Kitchen Herb Garden Starter Kit Growing Kit Including Wooden Box Burlap Pots Soil Discs Gardening Tools Unique Easter Birthday Christmas Gift Ideas for Women Mom
(0)
$30.99
FREE delivery Jun 19 - Jul 3
Loading...
Best Seller
Bonsai Starter Kit – 1x Bonsai Tree | Complete Indoor Starter Kit for Growing Plants with Bonsai Seeds, Tools & Planters – Gardening Gifts for Women & Men
(243)
$19.99
FREE delivery Sat, Jun 14 on your first order
300+ bought in past month
Loading...
KOORUI 24 Inch IPS Gaming Monitor, FHD 1920×1080p, 165Hz 200Hz Monitors, 1ms with Adaptive Sync, DCI-P3 90% Color Gamut, HDR400, 2 x HDMI 2.0, DisplayPort 1.4, Tilt Adjustable, G2411P
(4,083)
$139.99$126.47
FREE delivery Tue, Aug 12
400+ bought in past month
Loading...
ViewSonic VX1755 17 Inch 1080p Portable IPS Gaming Monitor with 144Hz, AMD FreeSync Premium, 2 Way Powered 60W USB C, Mini HDMI, and Built-in Stand with Smart Cover
(385)
$468.70
FREE delivery Aug 18 - 27
Loading...
ViewSonic VG1656N 16 Inch 1200p IPS Wireless Portable Monitor with 2 Way Powered 60W USB C, Built-in 4000 mAh Battery and Stand with Smart Cover
(1,543)
$426.22
FREE delivery Tue, Aug 12
Loading...
11.6 Inch Portable Monitor, FHD 1920x1080p IPS Screen Small HDMI Monitor with USB-C & HDMI Port, Built in Speaker Travel Monitor for Laptop, PC, Mac, Phone, PS4/5, Xbox
(4)
$99.99
FREE delivery Tue, Aug 12
Loading...
Monster N-Lite 211 Wireless Earbuds, Bluetooth 6.0 Stereo Headphones with Powerful Bass, Built-in Noise-Cancelling Mic for Clear Calls, LED Display, 36H Playtime, IPX5 Waterproof Design for Sports
(40)
$39.99
FREE delivery Thu, Nov 13
Loading...
Monster N-Lite 211 Wireless Earbuds， Bluetooth 6.0 Stereo Headphones with Noise-Cancelling Mic and LED Display, Built-in Mic for Clear Calls, Waterproof Design for Sports
(8)
$39.99
FREE delivery Tue, Nov 4