This project demonstrates the end-to-end process of building a data pipeline using Azure Synapse Analytics, Azure Data Factory (ADF), Databricks, and Delta Lake to ingest, clean, transform, and store data.
data-transformation keyvault databricks data-cleaning azuredatafactory etl-pipeline delta-lake medallion-architecture azuresynapseanalytics secrets-scopes service-principal-authentication mount-unmount-storageaccount increamental-load external-tables medill
-
Updated
Nov 15, 2024 - Jupyter Notebook