Shield:
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
https://www.udemy.com/course/data-lake-in-aws/?referralCode=CED92E2DBC4FCE825B96
In this course, we will begin by gaining an understanding of the fundamental concepts of a data lake and when it is the appropriate solution as opposed to a data warehouse
We will then delve into the various components that make up a data lake solution, including the ability to query files directly using SQL for rapid ad hoc analysis of datasets
During the course, we will cover the topic of handling changes to the structure of the files in the data lake. We will delve into the various scenarios, such as new fields, changes in data types, and missing data, and discuss the techniques on how to handle them effectively. We will also delve into Glue Catalog Management and the evolution of schemas, with a focus on minimizing disruption to downstream systems
We will also delve into different data formats, such as CSV, Parquet, Avro, and ORC, and examine their respective strengths and weaknesses. Following that, we will delve into Glue ETL, a robust Apache Spark-based solution for data transformation.
This course is filled with hands-on exercises and projects.
To showcase the scalability of Athena, we will query the large Amazon Customer Reviews dataset containing over 130 million reviews. Finally, we will construct a serverless application using Kinesis Firehose, Lambda, Comprehend AI, Glue, Athena, and S3, which can process an unlimited number of customer reviews, perform sentiment analysis, and store the results in the data lake for querying.
I am excited to meet you soon!
Thank you!
Chandra Lingam
Compute With Cloud Inc