From Kafka to a Spreadsheet - A Step by Step Python Tutorial

Quix β€’ June 12, 2024
Video Thumbnail

About

Quix Streams is an open source Python library for processing data in Apache Kafka. Designed around the principle of DataFrames (tabular representation of streaming data), it helps teams build real-time data pipelines for ML and analytics, by simplifying the transition from batch-based processing using libraries like Pandas to stream-based processing Best in class Python developer experience with pure Python, no JVM, no wrappers around other languages, and no cross-language debugging. It allows developers to utilize the full Python ecosystem. It’s stateful, fault-tolerant and promotes best practices to quickly deploy and scale out for production. To learn more you can 1. Follow/star the repo: https://github.com/quixio/quix-streams 2. Join the Slack community: https://quix.io/slack-invite 3. Visit: https://quix.io/

Video Description

Sooner or later, someone will ask you to get data into a spreadsheet. It's a universal truth of the software business. So let's create a project that does exactly that! In this Python coding walkthrough, we'll show you how to build a real-time data-streaming pipeline that takes data from Apache Kafka and streams it into a live Google Spreadsheet. You see every step of the code - minus our API keys - as we build up the solution step-by-step. From empty directory to working, live-updating sheet. πŸ“₯ This project's source code: https://github.com/quixio/simple-kafka-python πŸ‘¨β€πŸ’» Quix Streams Source on Github: https://github.com/quixio/quix-streams πŸ““ Quix Streams API Docs: https://quix.io/docs/quix-streams/api-reference/quixstreams.html πŸ“Ί Watch all the videos in this series: https://www.youtube.com/playlist?list=PL5gMntduShmyJd2fsflN1jwLW9XtDMFAX βš™οΈ Google Developer Console: https://console.cloud.google.com/ -- 00:00 Intro 00:27 Weather data 01:00 Data preprocessing 06:47 Hourly data grouping 08:10 Summarize hourly data 15:10 Connect the data to google sheets 22:44 Summary 23:10 Outro

You May Also Like