Ultimate Guide To Web Scraping - Node.js & Python (Puppeteer & Beautiful Soup)

Traversy Media • November 24, 2024

Traversy Media

About

Traversy Media features the best online web development and programming tutorials for all of the latest web technologies from the building blocks of HTML, CSS & JavaScript to frontend frameworks like React and Vue to backend technologies like Node.js, Python and PHP

Ultimate Guide To Web Scraping - Node.js & Python (Puppeteer & Beautiful Soup)

Traversy Media

About

Latest Posts

This is what I&#39;ve been working on

What happened to Brad Traversy?

NextJS + Postgres and Error Monitoring - Full Stack Support Ticketing System

Node.js Crash Course

You May Also Like

PC Gaming Upgrade Essentials

Dell 15.6&quot; FHD Touchscreen Student and Business Laptop – AMD Ryzen 7 7730U, Radeon Graphics, 16GB DDR4 RAM, 512GB SSD, USB-C, HDMI, SD Card Reader, Wi-Fi 5, Copilot Ready, Windows 11 Pro

ACEMAGIC 2025 Laptop AMD Ryzen 7 5825U (8C/16T), 15.6“FHD Display,16GB RAM 512GB NVMe SSD, WiFi 6, Numeric Keyboard, Type_C, 54.72Wh Battery | Windows 11 Pro Laptop Computer for Work &amp; Play

BOSGAME P3 Lite Mini PC, AMD Ryzen 7 6800H Mini Computer (8C/16T, up to 4.7GHz), 24GB DDR4 1TB M.2 NVMe SSD, Triple Display Desktop Gaming PC, Pre-in Win 11 Pro, Dual LAN, WiFi 6E, BT 5.2

BOSGAME M4 Mini PC Oculink, AMD Ryzen 7 8745HS 32GB DDR5 RAM 1TB PCIe4.0 SSD,Radeon 780M Quad Displays,USB4,Dual LAN 2.5GbE LAN Micro Desktop Computer

Dell G16 Gaming Laptop 7630-16-inch QHD+ 240Hz 3ms Display, Intel Core i9-13900HX, 32GB DDR5 RAM, 1TB SSD, NVIDIA GeForce RTX 4060 8GB GDDR6, Windows 11 Home, Onsite Service - Metallic Nightshade

HP Victus 15.6&quot; 144Hz FHD Gaming Laptop, Intel i5-12450H, 32GB RAM, 1TB PCIe SSD, NVIDIA GeForce RTX 3050, Backlit Keyboard, HD Webcam, Win 11, Blue, 256GB Docking Station Set

Acer Nitro V 15.6 FHD 144Hz Gaming Laptop, Intel i7-13620H, 32GB DDR5, 1TB SSD, NVIDIA GeForce RTX 4060, Keyboard Backlight, Wi-Fi 6, HD Webcam, Windows 11 Home, Black, 256GB Docking Station Set

MSI Ultra-Slim Thin 15 VR-Ready High FPS Gaming Laptop, 15.6 FHD 144Hz, Intel Core i5-13420H, NVIDIA GeForce RTX 4060, 32GB RAM, 2TB SSD, Backlit KB, Wi-Fi 6, Bundle with PCO Notebook Fold Radiator

Asus ROG Strix 1000W Gold Aura Edition (Fully Modular Power Supply, 80+ Gold Certified, ATX 3.0, Cybenetics Lambda A+ Certification, PCIe Gen 5.0 Ready, Axial-tech Fan, Aura Sync, 10-Year Warranty)

Corsair Vengeance LPX 32GB (2 X 16GB) DDR4 3200 (PC4-25600) C16 1.35V Desktop Memory - Black

Samsung 990 EVO Plus - 4TB PCIe Gen4. X4, Gen5. X2 NVMe 2.0 - M.2 Internal SSD, Speed Up to 7,250 MBs, Upgrade Storage for PC-Laptops, HMB Technology and Intelligent Turbowrite (MZ-V9S4T0B/AM)

SAMSUNG EVO Select Micro SD-Memory-Card + Adapter, 128GB microSDXC 160MB/s Full HD &amp; 4K UHD, UHS-I, U3, A2, V30, for Android Smartphones, Tablets, Nintendo-Switch (MB-ME128SA/AM) [Canada Version]

Samsung EVO Select Micro SD Memory Card + Adapter, 256GB microSDXC 160MB/s Full HD &amp; 4K UHD, UHS-I, U3, A2, V30, for Android Smartphones, Tablets, Nintendo-Switch (MB-ME256SA/AM) [Canada Version]

AINOPE Phone Holder for Car Vent 2025 Gravity Air Vent Car Phone Holder Mount for iPhone 16 Pro Max with Vent Clip Hand Free Cell Phone Car Mount for iPhone 16 Plus Samsung S25 Ultra Car Accesorries

Corsair RM1000e Fully Modular Low-Noise ATX Power Supply - Dual EPS12V Connectors - 105°C-Rated Capacitors - 80 Plus Gold Efficiency - Modern Standby Support - Black

Corsair RM1200x Shift Fully Modular ATX Power Supply - Modular Side Interface - ATX 3.1 &amp; PCIe 5.1 Compliant - Zero RPM Fan Mode - 105°C-Rated Capacitors - 80 Plus Gold Efficiency - Black, 1200W

Corsair SF1000 (2024) Fully Modular Low Noise 80 Plus Platinum ATX Power Supply – ATX 3.1 Compliant – PCIe 5.1 Ready – SFX-to-ATX Bracket Included – Black

Corsair CP-9020140-UK HX1200 1200 W 80+ Platinum Fully Modular Power Supply Unit - Black

Logitech M185 Wireless Mouse, 2.4GHz with USB Mini Receiver, 12-Month Battery Life, 1000 DPI Optical Tracking, Ambidextrous, Compatible with PC, Mac, Laptop - Black

Logitech G305 LIGHTSPEED Wireless Gaming Mouse, Hero 12K Sensor, 12,000 DPI, Lightweight, 6 Programmable Buttons, 250h Battery Life, On-Board Memory, PC/Mac - Black

Logitech K400 Plus Wireless Touch TV Keyboard With Easy Media Control and Built-in Touchpad, HTPC Keyboard for PC-connected TV, Windows, Android, Chrome OS, Laptop, Tablet - Black

Logitech G435 Lightspeed and Bluetooth Wireless Gaming Headset - Lightweight Over-Ear Headphones, Built-in mics, 18h Battery, Compatible with Dolby Atmos, PC, Mobile - Black

CORSAIR iCUE Link XD5 RGB Elite LCD Pump-Reservoir Unit - D5 PWM Pump - 480x480 IPS LCD Screen - 22 Addressable RGB LEDs - 440ml Nylon Reservoir - White

CORSAIR iCUE Link XC7 RGB Elite CPU Water Block - Transparent Flow Chamber - 24 RGB LEDs - Fits Intel® LGA 1700, AMD® AM5 and Older - White

CORSAIR Hydro X Series iCUE Link XH405i Custom Cooling Kit – Hardline Water Cooling Loop – XC7 Elite CPU Water Block – XD5 Elite D5 Pump Res – XR5 360mm Radiator – 3X QX120 RGB Fans

Loading...

This is what I've been working on

Dell 15.6" FHD Touchscreen Student and Business Laptop – AMD Ryzen 7 7730U, Radeon Graphics, 16GB DDR4 RAM, 512GB SSD, USB-C, HDMI, SD Card Reader, Wi-Fi 5, Copilot Ready, Windows 11 Pro

ACEMAGIC 2025 Laptop AMD Ryzen 7 5825U (8C/16T), 15.6“FHD Display,16GB RAM 512GB NVMe SSD, WiFi 6, Numeric Keyboard, Type_C, 54.72Wh Battery | Windows 11 Pro Laptop Computer for Work & Play

HP Victus 15.6" 144Hz FHD Gaming Laptop, Intel i5-12450H, 32GB RAM, 1TB PCIe SSD, NVIDIA GeForce RTX 3050, Backlit Keyboard, HD Webcam, Win 11, Blue, 256GB Docking Station Set

SAMSUNG EVO Select Micro SD-Memory-Card + Adapter, 128GB microSDXC 160MB/s Full HD & 4K UHD, UHS-I, U3, A2, V30, for Android Smartphones, Tablets, Nintendo-Switch (MB-ME128SA/AM) [Canada Version]

Samsung EVO Select Micro SD Memory Card + Adapter, 256GB microSDXC 160MB/s Full HD & 4K UHD, UHS-I, U3, A2, V30, for Android Smartphones, Tablets, Nintendo-Switch (MB-ME256SA/AM) [Canada Version]

Corsair RM1200x Shift Fully Modular ATX Power Supply - Modular Side Interface - ATX 3.1 & PCIe 5.1 Compliant - Zero RPM Fan Mode - 105°C-Rated Capacitors - 80 Plus Gold Efficiency - Black, 1200W