awesome-kubernetes/docs/aws-data.md

AWS Big Data

1. Introduction
2. AWS Data Lake
3. AWS Data Pipeline (aka Big Data Pipelines or Data Streams)

Introduction

• aws.amazon.com/big-data
• blogs.aws.amazon.com/bigdata
• Querying Amazon Kinesis Streams Directly with SQL and Spark Streaming
• Using Spark SQL for ETL
• whizlabs.com: AWS Kinesis vs Kafka Apache

AWS Data Lake

• Building a Data Lake on AWS AWS provides a highly scalable, flexible, secure, and cost-effective solution for your organization to build a Data Lake – a data repository for both structured and unstructured data that is designed to be easily accessible for on-demand data analytics enabling you to answer questions as they arise.

AWS Data Pipeline (aka Big Data Pipelines or Data Streams)

• AWS Data Pipeline
• AWS Data Pipeline Documentation
• medium: No-Code Data Collect API on AWS A No-Code Data Collections mechanism for Big Data Pipelines on AWS.
• AWS Big Data Blog: Category - AWS Data Pipeline