awesome-kubernetes/v2-docs/aws-data.md

AWS Big Data

!!! info "Architectural Context" Detailed reference for AWS Big Data in the context of Cloud Providers (Hyperscalers).

Table of Contents

1. Architectural Foundations

• Kubernetes Tools
  • General Reference

1. Cloud Infrastructure

• AWS
  • Big Data
  • ETL
  • Event Streaming
• Event Streaming
  • Comparison

Architectural Foundations

Kubernetes Tools

General Reference

• Using Spark SQL for ETL [COMMUNITY-TOOL] — A curated technical resource and architectural guide covering blogs.aws.amazon.com in the Kubernetes Tools ecosystem.
• medium: No-Code Data Collect API on AWS [COMMUNITY-TOOL] — A curated technical resource and architectural guide covering medium: No-Code Data Collect API on AWS in the Kubernetes Tools ecosystem.

Cloud Infrastructure

AWS

Big Data

• (2025) aws.amazon.com/big-data [NONE CONTENT] [ADVANCED LEVEL] [DOCUMENTATION] 🌟🌟🌟🌟 [ENTERPRISE-STABLE] — Official AWS portal showcasing enterprise managed data services. Covers analytical platform integrations utilizing Amazon EMR, AWS Glue, Athena, and cloud-native Amazon Redshift.
• (2025) blogs.aws.amazon.com/bigdata [NONE CONTENT] [ADVANCED LEVEL] [DOCUMENTATION] 🌟🌟🌟🌟 [ENTERPRISE-STABLE] — AWS Big Data blog feed publishing architectural insights, streaming pipeline blueprints, lake house migrations, and optimized resource configurations on top of AWS analytics stacks.

ETL

• (2025) ==AWS Data Pipeline== [NONE CONTENT] [DOCUMENTATION] 🌟🌟🌟🌟🌟 [DE FACTO STANDARD] — Official page of AWS Glue, a fully managed serverless data integration service. Identifies structural discovery using AWS Glue Data Catalog, PySpark ETL execution, and schema registry controls.
• (2025) AWS Data Pipeline Documentation [NONE CONTENT] [DOCUMENTATION] 🌟🌟 [COMMUNITY-TOOL] — Classic developer documentation for AWS Data Pipeline. Details the scheduling of batch workflows across EC2 and EMR resources (largely superseded by modern AWS Glue and Step Functions setups).
• (2024) AWS Big Data Blog: Category - AWS Data Pipeline [NONE CONTENT] [DOCUMENTATION] 🌟🌟 [LEGACY] — Technical blog category detailing legacy deployment setups for AWS Data Pipeline, providing structural templates for data movements before cloud-native ETL tools gained dominance.

Event Streaming

• (2016) Querying Amazon Kinesis Streams Directly with SQL and Spark Streaming [SCALA CONTENT] [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE] — Technical blueprint detailing Spark Streaming integration with Amazon Kinesis streams. Discusses record-processing window optimizations, checkpoint configurations, and SQL querying on live event pipelines.

Event Streaming (1)

Comparison

• (2021) whizlabs.com: AWS Kinesis vs Kafka Apache [NONE CONTENT] 🌟🌟🌟🌟 [ENTERPRISE-STABLE] — Comparative architectural review between AWS Kinesis and Apache Kafka. Analyzes data retention policies, throughput capabilities, scaling overheads, and total cost of ownership (TCO) profiles.

---

💡 Explore Related: Googlecloudplatform | AWS Pricing | AWS Spain