Uber End-To-End Data Engineering Project (2026) | Azure Databricks Streaming Project

Databricks | Azure Data Factory | PySpark | End To End Data Engineering Project What you will learn? This is a complete 6+ hours End-To-End Data Engineering Project built using Databricks, Azure Data Factory, Azure Event Hubs, PySpark, Spark Structured Streaming, and Spark Declarative Pipelines. This is a beginner-friendly, real-world project where you’ll build everything step by step - from data ingestion to transformation to real-time processing. Timestamps: 0:00 Introduction 01:08 Data Engineering Project Architecture 15:14 Apache Kafka Overview 28:50 Azure Free Account 43:04 Azure Event Hub Resource 52:02 WebApp To Event Hub Pipeline 1:22:13 Azure Data Factory Overview 1:37:00 Ingestion Pipelines with Azure Data Factory 2:19:38 Databricks Overview 2:39:21 PySpark Structured Streaming For Events 2:56:19 Load Data From Azure Data Lake To Databricks 3:21:22 Metadata Driven PySpark Streaming 5:08:27 Slowly Changing Dimensions 6:07:28 STAR Schema Data Model My UDEMY Courses: Apache Spark 4.0 Bootcamp https://www.udemy.com/course/databricks-certified-associate-developer-for-apache-spark-4/?referralCode=1A5672B3342F3B2D498D Databricks Data Engineer https://www.udemy.com/course/databricks-certified-data-engineer-associate-bootcamp/?couponCode=DATAFAMFEB Fabric Data Engineer https://www.udemy.com/course/dp-700-exam-prep-fabric-data-engineer-associate/?couponCode=DATAFAMMARCH Watch My Previous Videos - Databricks Full Course https://youtu.be/7pee6_Sq3VY?si=sXBMulukb6hPGeZ4 - PySpark Full Course https://youtu.be/94w6hPk7nkM?si=_rDJJLXuSz_KzdLF DBT Full Course https://youtu.be/B8uwFmVt4sU?si=Cz6r_66HqCKNPMJ8 ADF Full Course https://youtu.be/8zIVOdKyoDA?si=t_iTomGrxO0BQjTJ SDP Full Course https://youtu.be/EJJNoUKSc9w?si=M9ZWotUWax806u9D Code Repo https://github.com/anshlambagit/Uber_Data_Engineer_Project Data Engineer Roadmap (With Resources) https://github.com/anshlambagit/Data_Engineer_Roadmap ======================================= Join this channel to SUPPORT MY HARDWORK: https://www.youtube.com/channel/UCu7lQE-L5gzt8aD7zuuufjw/join ======================================= Connect with ME LinkedIn - https://www.linkedin.com/in/ansh-lamba-793681184/ Telegram - https://t.me/anshlambadatafam ========================= For COLLABORATION 👇 anshlambacollab@gmail.com ========================= ⭐Hashtags⭐ #databricks #azure #azuredatabricks #azuredataengineer #pyspark #azuredatafactory #dataengineer #datalake #datamodeling