2 Real World Azure Data Engineer Project End to End

2 Real World Azure Data Engineer Project End to End

This course will help you in preparing and mastering your Azure Data engineering Concepts.

It is not like any random project like covid, or twitter analysis. These project is real world projects on which I personally worked and developed it for big clients.

Highlights of the Course:

• Designed to keep only précised information no beating around the bush. (To save your time).

• Real time implementation, no dummy use case.

• Can be added as part of your resume.

• It will help you to showcase your experience in interviews and discussion.

• Involve complex architecture solution which is aligned with industry best practices.

• Single projects involve various component integration like ADF, ADLS, Databricks, Azure SQL DB, Key Vault.

• Solves the problem of real time experience for new Data engineers.

  
  

This course has been developed in mind to keep all the best practices followed in the Industry as an data engineering project and solution.

Technologies involved:

1. Azure Data Lake Storage Gen 2

2. Azure Data Factory

3. Data Factory Pipeline

4. Azure Functions

5. Azure Key Vault

6. Azure SQL DB

7. SSMS

8. AWS S3 Bucket

9. Connect ADF to Databricks

10. Connect Databricks to SQL Server

11. Connect Databricks to ADLS

12. Connect S3 to Azure Cloud

13. Triggers

14. SAS token

15. Create Secrets scope in Databricks

16. Store secretes in Key Vault and access them

What you will learn after this course:

• How to think, design and develop the solution in the data engineering world.

• How to create the architecture diagram for data engineering  projects.

• How to Create Azure Data Factory Account

• How to Create Azure Data Lake Storage Gen 2 account.

• How to Create Azure Databricks Workspace.

• How to create S3 storage account.

• How to create Azure Function.

• How to implement logic in the Databricks notebook using pyspark.

• How to connect ADF to Databricks.

• How to chain the multiple pieces together in project.

• How to create Azure SQL Server.

• How to load the data from file to Azure SQL server.

• How to connect Databricks notebook with Azure SQL Server.

• How to Store secrets in the Azure Key Vault.

  
  

Engineering Project Building from scratch including designing, architecting, implementing solution and overall testing.

Url: View Details

What you will learn

• You will learn how to Architect, Design and build a real-world enterprise level data platform solution including multiple services.
• You will learn design solution using ADF, Azure Function, Databricks, pyspark, Azure Data lake storage Gen 2 (ADLS), Azure SQL Server
• You will learn how to build a real-world data pipeline in Azure Data Factory (ADF). This course has been taught using 2 real world use case scenarios.

Rating: 4.16026

Level: All Levels

Duration: 7 hours

Instructor: Deepak Goyal