This is an End to End Azure Data Engineering project copying data from Rest API to Azure cloud.
-
Updated
Dec 21, 2024 - Python
#
adlsgen2
Here are 5 public repositories matching this topic...
AirBnB CDC Ingestion Pipeline: Near Real-Time Change Data Capture (CDC) Pipeline on Azure for Seamless Integration of Continuous Data Streams
-
Updated
Sep 21, 2024 - Python
"Explore Formula 1 data analytics with this project. Leveraging the Ergast API, it utilizes Databricks Spark for ingestion, transformation, and analysis. ADLS acts as the storage layer, while Power BI visualizes the ADLS presentation layer. Uncover insights in the world of Formula 1 through powerful data analytics."
formula1 powerbi databricks datalake azure-data-factory ergast-api azure-databricks delta-lake adlsgen2
-
Updated
Jul 6, 2023 - Python
Implemented Azure Databricks for real-time data processing and governance using Unity Catalog, Spark Structured Streaming, Delta Lake features, Medallion Architecture, and end-to-end CI/CD pipelines. Focused on incremental loading, compute cluster management, maintaining data quality, and creating workflows.
-
Updated
Aug 2, 2024 - Python
Improve this page
Add a description, image, and links to the adlsgen2 topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the adlsgen2 topic, visit your repo's landing page and select "manage topics."