Introduction to Dataform in Google Cloud Platform

Video Description

This tutorial shows the overview of the Dataform service in the Google Cloud Platform. Link to the GitHub repo with workflow: https://github.com/rafaello9472/dataform-demo/branches/stale Dataform documentation: https://cloud.google.com/dataform/docs/overview 00:00 Introduction 00:42 What is Dataform? 01:10 Key Features and Benefits 02:30 Why Dataform? 02:43 Key Concepts 04:19 From SQL to SQLX 04:57 Version Control and Collaboration 06:03 Dependency Management 07:30 Javascript in Dataform 09:50 Workflow Execution Scheduling Options 10:35 Creating Dataform Repository 11:41 Creating Necessary GitHub Assets 13:41 Adding Access Token to Secret Manager 14:18 Adding IAM Roles to Service Account 15:46 Creating Development Workspace 17:52 Intro to Demo Examples 18:36 Overview of Repository | 1 20:18 Running First Workflow & Adding IAM Role | 2 25:22 Dependencies as DAGs | 3 27:48 Custom Operations & Tags | 4 32:43 Assertions | 5 36:23 Skipping Pipeline Step Execution with Assertions | 6 38:55 Reuse JavaScript Variables in SQLX Files | 7