Apache Spark Core—Deep Dive—Proper Optimization Daniel Tomes Databricks

Databricks • November 26, 2019

Databricks

About

Databricks is the Data and AI company. More than 20,000 organizations worldwide — including adidas, AT&T, Bayer, Block, Mastercard, Rivian, Unilever, and over 60% of the Fortune 500 — rely on Databricks to build and scale data and AI apps, analytics and agents. Headquartered in San Francisco with 30+ offices around the globe, Databricks offers a unified Data Intelligence Platform that includes Agent Bricks, Lakeflow, Lakehouse, Lakebase and Unity Catalog.

Apache Spark Core—Deep Dive—Proper Optimization Daniel Tomes Databricks

Databricks

About

Latest Posts

Data Intelligence Day Bangkok 2025 - Highlights

Databricks + Microsoft: Live Data, Unified Governance, and the Future of AI Agents

Turn Genie Into an Agent Using Conversation APIs

Deploying Databricks Asset Bundles (DABs) at Scale

You May Also Like

Boost Your Spark Learning

Intel NUC 13 Pro Mini PC, NUC13ANKI7 Intel Core i7-1360P (12C/16T &amp; Up to 5.0GHz), 32GB 3200Mhz DDR4 RAM &amp; 1TB PCIe 4.0 NVMe SSD, Support 8K/HDR/WiFi6E/BT5.3/Windows 11 Pro/Office/Business

GEEKOM IT13 2025 Mini PC 2025 Edition with 13th Gen Intel i9-13900HK, NUC 13 Mini Computers(14C/20T), 32GB RAM&amp;1T M.2 2280 NVMe Gen4*4 SSD, Windows 11 Pro/Wi-Fi 6E/Bluetooth 5.2/USB 4.0/2.5G LAN

Crucial X9 4TB Portable SSD, Up to 1050MB/s, USB 3.2 USB-C, External Solid State Drive, Compatible with Windows, Mac, &amp; Android, Reliable Storage for Games, Files, &amp; Backups, Black - CT4000X9SSD902

Crucial X10 Pro 4TB Portable SSD, Up to 2100MB/s Read, 2000MB/s Write, 3.2 USB-C, External Solid State Drive, Durable Storage for PC &amp; Mac, for Professional Creators, Black - ‎CT4000X10PROSSD902

Crucial X9 Pro 4TB Portable SSD, Up to 1050MB/s Read &amp; Write, 3.2 USB-C, External Solid State Drive, Durable Storage for PC and Mac, for Professional Creators, Silver - ‎CT4000X9PROSSD902

Crucial T705 4TB PCIe Gen5 NVMe M.2 SSD - Up to 14,100 MB/s - Game Ready - Internal Solid State Drive (PC) - +1mo Adobe CC - CT4000T705SSD3

Samsung 990 EVO Plus - 4TB PCIe Gen4. X4, Gen5. X2 NVMe 2.0 - M.2 Internal SSD, Speed Up to 7,250 MBs, Upgrade Storage for PC-Laptops, HMB Technology and Intelligent Turbowrite (MZ-V9S4T0B/AM)

SAMSUNG EVO Select Micro SD-Memory-Card + Adapter, 128GB microSDXC 160MB/s Full HD &amp; 4K UHD, UHS-I, U3, A2, V30, for Android Smartphones, Tablets, Nintendo-Switch (MB-ME128SA/AM) [Canada Version]

Samsung EVO Select Micro SD Memory Card + Adapter, 256GB microSDXC 160MB/s Full HD &amp; 4K UHD, UHS-I, U3, A2, V30, for Android Smartphones, Tablets, Nintendo-Switch (MB-ME256SA/AM) [Canada Version]

AINOPE Phone Holder for Car Vent 2025 Gravity Air Vent Car Phone Holder Mount for iPhone 16 Pro Max with Vent Clip Hand Free Cell Phone Car Mount for iPhone 16 Plus Samsung S25 Ultra Car Accesorries

HDMI Switch 4K 60Hz, WARRKY HDMI Switcher 2 in 1 Out [Aluminum Case, Fixed Braided Cable] Bidirectional HDMI Splitter 2x1 Support HDR, 3D, 4K, 1080P Compatible for PS5/4, Xbox, Roku, Fire Stick

10.1 Inch Touch Portable Monitor IPS Screen 1366x768P 60Hz 400 Brightness 99% sRGB HDMI USB-C Monitors Switch for Xbox PS3/4/5 Laptop Compatible with Raspberry Pi, Mini Touch Screen

USB-C Portable Monitor - 15.6 Inch FHD HDR FreeSync Zero Frame USB-C Computer Display with Dual Type-C Mini HDMI for Laptop PC Phone Mac Surface Xbox PS5 Switch, with Cover VESA Mountable

cocopar Portable Monitor 18.5 inch 1080P 100Hz 72% NTSC USB-C HDMI Large Portable Monitor for Laptop MacBook Xbox PS4/5 Switch

KOORUI 27 Inch 4K Monitor, UHD(3840 x 2160) IPS Computer Monitors, HDR10, 90% DCI-P3, Frameless, 2X HDMI 2.0 &amp; DisplayPort 1.4, VESA Mountable, Eye Care, N07

Logitech G203 Wired Gaming Mouse, 8,000 DPI, Rainbow Optical Effect LIGHTSYNC RGB, 6 Programmable Buttons, On-Board Memory, Screen Mapping, PC/Mac Computer and Laptop Compatible - Black

Logitech G305 LIGHTSPEED Wireless Gaming Mouse, Hero 12K Sensor, 12,000 DPI, Lightweight, 6 Programmable Buttons, 250h Battery Life, On-Board Memory, PC/Mac - Black

10.1 Inch Touch Portable Monitor IPS Screen 1366x768P 60Hz 400 Brightness 99% sRGB HDMI USB-C Monitors Switch for Xbox PS3/4/5 Laptop Compatible with Raspberry Pi, Mini Touch Screen

ELECROW 8 Inch Portable Monitor, 1280x800 Mini HD Display with Built-in Speakers, USB Powered, Non-Touch LCD Screen for Raspberry Pi, PC, Laptop, Jetson Nano, Game Consoles

7 Inch Portable Monitor Touchscreen HD 1024x600 LED Display Dual HDMI Port Small Monitor for PC Raspberry Pi Laptop Computer Xbox PS4/5 Switch Built-in Speakers

BrosTrend 1800Mbps WiFi 6 Linux WiFi Adapter for PC and Raspberry Pi 2+, Long Range USB WiFi Dongle Linux for Ubuntu, Mint, Debian, Kubuntu, Lubuntu, Zorin, Windows 11/10, Dual Band Wireless Antenna

Loading...

Intel NUC 13 Pro Mini PC, NUC13ANKI7 Intel Core i7-1360P (12C/16T & Up to 5.0GHz), 32GB 3200Mhz DDR4 RAM & 1TB PCIe 4.0 NVMe SSD, Support 8K/HDR/WiFi6E/BT5.3/Windows 11 Pro/Office/Business

GEEKOM IT13 2025 Mini PC 2025 Edition with 13th Gen Intel i9-13900HK, NUC 13 Mini Computers(14C/20T), 32GB RAM&1T M.2 2280 NVMe Gen4*4 SSD, Windows 11 Pro/Wi-Fi 6E/Bluetooth 5.2/USB 4.0/2.5G LAN

Crucial X9 4TB Portable SSD, Up to 1050MB/s, USB 3.2 USB-C, External Solid State Drive, Compatible with Windows, Mac, & Android, Reliable Storage for Games, Files, & Backups, Black - CT4000X9SSD902

Crucial X10 Pro 4TB Portable SSD, Up to 2100MB/s Read, 2000MB/s Write, 3.2 USB-C, External Solid State Drive, Durable Storage for PC & Mac, for Professional Creators, Black - ‎CT4000X10PROSSD902

Crucial X9 Pro 4TB Portable SSD, Up to 1050MB/s Read & Write, 3.2 USB-C, External Solid State Drive, Durable Storage for PC and Mac, for Professional Creators, Silver - ‎CT4000X9PROSSD902

SAMSUNG EVO Select Micro SD-Memory-Card + Adapter, 128GB microSDXC 160MB/s Full HD & 4K UHD, UHS-I, U3, A2, V30, for Android Smartphones, Tablets, Nintendo-Switch (MB-ME128SA/AM) [Canada Version]

Samsung EVO Select Micro SD Memory Card + Adapter, 256GB microSDXC 160MB/s Full HD & 4K UHD, UHS-I, U3, A2, V30, for Android Smartphones, Tablets, Nintendo-Switch (MB-ME256SA/AM) [Canada Version]

KOORUI 27 Inch 4K Monitor, UHD(3840 x 2160) IPS Computer Monitors, HDR10, 90% DCI-P3, Frameless, 2X HDMI 2.0 & DisplayPort 1.4, VESA Mountable, Eye Care, N07