Data Lake, Firehose, Glue, Athena, S3 and AWS SDK for .NET
Leverage AWS Kinesis Data Firehose, AWS Glue, S3, Athena and the AWS SDK to build a Data Lake.
Description
The purpose of this class is to demonstrate a proof of concept using a series of lab exercise's (in the AWS Console using AWS Kinesis Data Firehose, AWS Glue, S3, Athena and the AWS SDK, with C# code using the AWS SDK) of building a Data Lake in the AWS ecosystem.
In this class, we will be sending data from a local SQL Server database to AWS RDS securely, and automatically.
I am utilizing the .NET SDK for AWS, however, this could easily be migrated to your language of choice once you understand the concepts that I am teaching.
I will be providing the full working source code of this proof of concept.
I will also provide a working example of creating Parquet files and sending to S3 (without using Firehose).
What You Will Learn!
- How to build a Data Lake using Firehose API for .NET, S3, AWS Glue and Athena
Who Should Attend!
- AWS developers interested in learning about building a data lake using AWS services and the AWS SDK.