Data Engineer (Azure) Curriculum
Azure Fundamentals:
Cloud Services
IAAS/SAAS/PAAS
Public/Private/Hybrid cloud
Azure Components
Azure Data Engineer Solutions
Spark(Scala)/Databricks:
Hadoop Ecosystem and HDFS
YARN
HDFS Commands
Scala for Spark
Spark Architecture
Spark client tools
RDDs
SQL Basics(optional)
Spark SQL(Dataframes/Datasets)
Monitoring and debugging/performance tuning Spark job
Databricks and its Delta Lake
Mini Project involving ETL on Spark
Azure Data Factory:
Introduction to ADF
Pipelines
Activities
Linked Services
Datasets
Triggers
Parameters
Integration Runtime
-----------------ADF Scenarios in Real time Projects ---------------------------
Copy data from blob store and store another location within blob store.
Copy data from blob store to Azure SQL server
Copy data from on premise SQL to Bolb store
Copy data from Azure SQL to blob
Copy data in bulk load/ full load
Copy data in incemental load
Run/Execute SSIS package from ADF
Secure credentials in Azure Key Vault
Performance tuning
ADF Pricng
Azure Analysis Services:
SSAS Architecture
Build tabular model from scratch
Relationships
Work with Measures
Work with Hierarchies
Work with calculated columns
Deploy Tabular model
Scale up/down & Scale out replicas
Overview of Azure pricing for Tabular model
Partitions
Aggregations
Perspectives
Security
Introduction to Dax
Dax caluculations - Simple & more Examples
DAX Related() Function
DAX Text type functions
DAX Time series functions
DAX Logical type functions
DAX Aggeregate functions
DAX Calculate function
DAX Virtual Relationship functions
Use cases/ Demos in Real time projects
Power BI:
Data Sources
DAX
Visualizations
Security