# AWS Big Data

!!! info "Architectural Context"
    Detailed reference for AWS Big Data in the context of Cloud Providers (Hyperscalers).

## Table of Contents

1. [Architectural Foundations](#architectural-foundations)
  - [Kubernetes Tools](#kubernetes-tools)
    - [General Reference](#general-reference)
1. [Cloud Infrastructure](#cloud-infrastructure)
  - [AWS](#aws)
    - [Big Data](#big-data)
    - [ETL](#etl)
    - [Event Streaming](#event-streaming)
  - [Event Streaming](#event-streaming-1)
    - [Comparison](#comparison)

## Architectural Foundations

### Kubernetes Tools

#### General Reference

  - [Using Spark SQL for ETL](https://blogs.aws.amazon.com/bigdata/post/Tx2D93GZRHU3TES/Using-Spark-SQL-for-ETL)  <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — A curated technical resource and architectural guide covering blogs.aws.amazon.com in the Kubernetes Tools ecosystem.
  - [medium: No-Code Data Collect API on AWS](https://medium.com/@dima.statz_89242/no-code-data-collect-api-on-aws-d79e3681d204)  <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — A curated technical resource and architectural guide covering medium: No-Code Data Collect API on AWS in the Kubernetes Tools ecosystem.
## Cloud Infrastructure

### AWS

#### Big Data

  - **(2025)** [**aws.amazon.com/big-data**](https://aws.amazon.com/big-data) <span class='md-tag md-tag--warning'>[NONE CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--primary'>[DOCUMENTATION]</span> 🌟🌟🌟🌟 <span class='md-tag md-tag--success'>[ENTERPRISE-STABLE]</span> — Official AWS portal showcasing enterprise managed data services. Covers analytical platform integrations utilizing Amazon EMR, AWS Glue, Athena, and cloud-native Amazon Redshift.
  - **(2025)** [**blogs.aws.amazon.com/bigdata**](https://blogs.aws.amazon.com/bigdata) <span class='md-tag md-tag--warning'>[NONE CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--primary'>[DOCUMENTATION]</span> 🌟🌟🌟🌟 <span class='md-tag md-tag--success'>[ENTERPRISE-STABLE]</span> — AWS Big Data blog feed publishing architectural insights, streaming pipeline blueprints, lake house migrations, and optimized resource configurations on top of AWS analytics stacks.
#### ETL

  - **(2025)** [==AWS Data Pipeline==](https://aws.amazon.com/glue) <span class='md-tag md-tag--warning'>[NONE CONTENT]</span> <span class='md-tag md-tag--primary'>[DOCUMENTATION]</span> 🌟🌟🌟🌟🌟 <span class='md-tag md-tag--success'>[DE FACTO STANDARD]</span> — Official page of AWS Glue, a fully managed serverless data integration service. Identifies structural discovery using AWS Glue Data Catalog, PySpark ETL execution, and schema registry controls.
  - **(2025)** [AWS Data Pipeline Documentation](https://docs.aws.amazon.com/data-pipeline) <span class='md-tag md-tag--warning'>[NONE CONTENT]</span> <span class='md-tag md-tag--primary'>[DOCUMENTATION]</span> 🌟🌟 <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — Classic developer documentation for AWS Data Pipeline. Details the scheduling of batch workflows across EC2 and EMR resources (largely superseded by modern AWS Glue and Step Functions setups).
  - **(2024)** [AWS Big Data Blog: Category - AWS Data Pipeline](https://aws.amazon.com/blogs/big-data/category/analytics/aws-data-pipeline) <span class='md-tag md-tag--warning'>[NONE CONTENT]</span> <span class='md-tag md-tag--primary'>[DOCUMENTATION]</span> 🌟🌟 <span class='md-tag md-tag--critical'>[LEGACY]</span> — Technical blog category detailing legacy deployment setups for AWS Data Pipeline, providing structural templates for data movements before cloud-native ETL tools gained dominance.
#### Event Streaming

  - **(2016)** [**Querying Amazon Kinesis Streams Directly with SQL and Spark Streaming**](https://aws.amazon.com/blogs/big-data/querying-amazon-kinesis-streams-directly-with-sql-and-spark-streaming) <span class='md-tag md-tag--warning'>[SCALA CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> 🌟🌟🌟🌟 <span class='md-tag md-tag--success'>[ENTERPRISE-STABLE]</span> — Technical blueprint detailing Spark Streaming integration with Amazon Kinesis streams. Discusses record-processing window optimizations, checkpoint configurations, and SQL querying on live event pipelines.
### Event Streaming (1)

#### Comparison

  - **(2021)** [**whizlabs.com: AWS Kinesis vs Kafka Apache**](https://www.whizlabs.com/blog/kinesis-vs-kafka) <span class='md-tag md-tag--warning'>[NONE CONTENT]</span> 🌟🌟🌟🌟 <span class='md-tag md-tag--success'>[ENTERPRISE-STABLE]</span> — Comparative architectural review between AWS Kinesis and Apache Kafka. Analyzes data retention policies, throughput capabilities, scaling overheads, and total cost of ownership (TCO) profiles.

---
💡 **Explore Related:** [Googlecloudplatform](./GoogleCloudPlatform.md) | [AWS Pricing](./aws-pricing.md) | [AWS Spain](./aws-spain.md)