Master end-to-end data engineering with Azure Databricks and Unity Catalog. This course moves from foundational setup to production deployment, covering environment configuration and enterprise-grade governance. Learn to build robust ingestion pipelines, implement security with Unity Catalog, and deploy optimised workloads.

The target audience is data engineers who have fundamental knowledge of data analytics concepts, a basic understanding of cloud storage, and familiarity with data organisation principles.

Before attending this course, learners should:

• Have experience working with SQL databases and writing SQL queries.
• Be comfortable using Python for data engineering tasks, including working with notebooks.
• Understand how Azure Databricks workspaces and Unity Catalog are used to manage data.
• Have a basic understanding of data engineering and data warehousing concepts.
• Be familiar with common data access and integration methods.
• Have foundational knowledge of Azure security, including Microsoft Entra ID.
• Understand the basics of Git and version control.

This course is intended for learners who already have hands-on experience with modern data platforms and are looking to build AI-enabled database solutions using Microsoft SQL technologies.

What you'll learn

• Set up and configure an Azure Databricks environment
• Secure and govern Unity Catalog objects
• Prepare and process data
• Deploy and maintain data pipelines and workloads

Set up and configure an Azure Databricks environment

• Explore Azure Databricks
• Understand Azure Databricks architecture
• Understand Azure Databricks Integrations
• Select and Configure Compute in Azure Databricks
• Create and organise objects in Unity Catalog

Secure and govern Unity Catalog objects in Azure Databricks

• Secure Unity Catalog objects
• Govern Unity Catalog objects

Prepare and process data with Azure Databricks

• Design and implement data modeling with Azure Databricks
• Ingest data into Unity Catalog
• Cleanse, transform, and load data into Unity Catalog
• Implement and manage data quality constraints with Azure Databricks

Deploy and maintain data pipelines and workloads with Azure Databricks

• Design and implement data pipelines with Azure Databricks
• Implement Lakeflow Jobs with Azure Databricks
• Implement development lifecycle processes in Azure Databricks
• Monitor, troubleshoot and optimise workloads in Azure Databricks