github/awesome-kubernetes

Fork 0

mirror of https://github.com/nubenetes/awesome-kubernetes.git synced 2026-07-16 11:52:59 +00:00

Files

inafev 2211ec1715 feat: sync V2 elite curated edition from partial cache [skip ci]

2026-05-21 21:01:58 +00:00

148 KiB

Raw Blame History

Message Queue

!!! info "Architectural Context" Detailed reference for Message Queue in the context of Data & Advanced Analytics.

Application Integration

Cloud Managed Services

Pub-Sub Pattern

(2026) ==Google Cloud Platform Pub/Sub== [DOCUMENTATION] 🌟🌟🌟🌟🌟 [DE FACTO STANDARD] [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" Comprehensive documentation for GCP Pub/Sub, an enterprise-grade, globally distributed, fully-managed asynchronous messaging service. It provides consistent sub-second latencies at arbitrary scale. It features seamless integrations with Google Cloud's data analytics stacks.

💡 Explore Related: Yaml | Databases | Newsql

Enterprise Integration Patterns

Cloud Managed Services

(2024) ibm.com: iPaaS (Integration-Platform-as-a-Service) 🌟🌟🌟 [GUIDE] [LEGACY]

??? info "Technical Deep-Dive" A strategic and architectural overview of Integration Platform as a Service (iPaaS). It reviews how cloud-native middleware bridges legacy on-premise systems and modern SaaS ecosystems. Highly useful for enterprise digital integration roadmaps.

💡 Explore Related: Yaml | Databases | Newsql

Event-Driven Systems

(2021) developers.redhat.com: Design event-driven integrations with Kamelets and Camel K 🌟🌟🌟 [EMERGING] [GUIDE]

??? info "Technical Deep-Dive" Introduces "Kamelets" (Camel Route Snippets), which act as reusable cloud-native integration building blocks. It explains how non-developers or low-code frameworks can plug Kamelets into serverless topologies for immediate data flow orchestration on Kubernetes.

💡 Explore Related: Yaml | Databases | Newsql

Middleware

(2025) Red Hat Fuse [ADVANCED LEVEL] [DOCUMENTATION] 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" Red Hat's enterprise-grade, distributed integration platform, heavily utilizing Apache Camel, ActiveMQ, and CXF. It provides a highly stable middleware environment designed to bind heterogeneous enterprise workloads and APIs under unified orchestration rules.

💡 Explore Related: Yaml | Databases | Newsql

Serverless Integration

(2021) developers.redhat.com: Integrating systems with Apache Camel and Quarkus on Red Hat OpenShift [ADVANCED LEVEL] 🌟🌟🌟 [ENTERPRISE-STABLE] [GUIDE] [LEGACY]

??? info "Technical Deep-Dive" Practical walk-through highlighting the integration of legacy enterprise environments using Apache Camel Quarkus extensions (Camel K) on OpenShift. It demonstrates how Quarkus' sub-second startup times enable serverless, cloud-native integration routes.

💡 Explore Related: Yaml | Databases | Newsql

Event Streaming

Enterprise Integration Patterns

(2022) kai-waehner.de: When to use Apache Camel vs. Apache Kafka? 🌟 [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE] [GUIDE]

??? info "Technical Deep-Dive" Examines the complementarity and core differences between Apache Camel (an integration framework implementing Enterprise Integration Patterns) and Apache Kafka (a distributed streaming platform). It outlines architectures where Camel acts as a producer/consumer or edge connector for Kafka pipelines.

💡 Explore Related: Yaml | Databases | Newsql

Kafka Connectors

(2020) developers.redhat.com: Extending Kafka connectivity with Apache Camel Kafka connectors [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE] [GUIDE]

??? info "Technical Deep-Dive" Outlines how the Camel Kafka Connector framework allows developers to utilize Camel's extensive component suite as standard Kafka Connect sources or sinks. This simplifies ingestion and delivery to hundreds of external enterprise systems without custom code.

💡 Explore Related: Yaml | Databases | Newsql

Local Development

Containerization

(2021) geshan.com.np: How to use RabbitMQ and Node.js with Docker and Docker-compose 🌟🌟🌟 [COMMUNITY-TOOL] [GUIDE]

??? info "Technical Deep-Dive" A hands-on tutorial outlining the setup of a localized asynchronous worker pipeline using Node.js, RabbitMQ, and Docker Compose. It serves as an accessible entry point to grasp queue-based application decoupling. Includes configuration templates ready for development workflows.

💡 Explore Related: Yaml | Databases | Newsql

Low-Code Integration

Enterprise Integration Patterns

(2022) Syndesis open source integration platform [DOCUMENTATION] 🌟🌟🌟 [COMMUNITY-TOOL] [LEGACY]

??? info "Technical Deep-Dive" Syndesis is an open-source, cloud-native low-code integration platform designed to run natively on Kubernetes and OpenShift. It enables drag-and-drop connections between diverse business APIs and internal databases, utilizing Apache Camel under the hood. Note: The project has recently transitioned to legacy status.

💡 Explore Related: Yaml | Databases | Newsql

Microservices

(2020) developers.redhat.com: Low-code microservices orchestration with Syndesis 🌟🌟🌟 [COMMUNITY-TOOL] [GUIDE]

??? info "Technical Deep-Dive" Detailed demonstration of leveraging Syndesis for visual, low-code orchestration of enterprise microservices. It highlights quick deployment cycles, declarative configuration models, and integration with Red Hat OpenShift resources.

💡 Explore Related: Yaml | Databases | Newsql

Message Brokers

Clustering

(2021) developers.redhat.com: Implementing Apache ActiveMQ-style broker meshes with Apache Artemis [ADVANCED LEVEL] 🌟🌟🌟 [COMMUNITY-TOOL] [GUIDE]

??? info "Technical Deep-Dive" Focuses on establishing distributed, multi-broker network configurations (broker meshes) using Apache Artemis. It highlights migration techniques from classic ActiveMQ network-of-brokers architectures. It explains target configuration profiles to optimize reliability across complex enterprise regions.

💡 Explore Related: Yaml | Databases | Newsql

Evaluation Frameworks

(2022) kai-waehner.de: Comparison: JMS Message Queue vs. Apache Kafka [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE] [GUIDE]

??? info "Technical Deep-Dive" Details the technical tradeoffs, design limitations, and complementary features of JMS broker specifications versus Apache Kafka. It assists system engineers in distinguishing transaction-heavy classic queuing requirements from massive event streaming workloads.
(2020) developers.redhat.com: Choosing the right asynchronous-messaging infrastructure for the job [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE] [GUIDE]

??? info "Technical Deep-Dive" Lays out a decision framework for choosing between broker-based messaging (e.g., AMQP, ActiveMQ), event-streaming (e.g., Apache Kafka), and cloud-native serverless event routing. It evaluates criteria like throughput, ordering guarantees, consumer groups, and message preservation. This is an essential architectural comparative reference.
(2023) kubemq.io: Kafka VS KubeMQ 🌟 [DOCUMENTATION] 🌟🌟🌟 [COMMUNITY-TOOL] [GUIDE]

??? info "Technical Deep-Dive" Provides a detailed comparison between Apache Kafka and KubeMQ, focusing on memory footprint, container resource demands, and operational complexity. It presents KubeMQ as a highly localized, easy-to-manage container broker, contrasting it with Kafka's robust, distributed cluster topology.

💡 Explore Related: Yaml | Databases | Newsql

Event Streaming

(2021) blog.rabbitmq.com: First Application With RabbitMQ Streams 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" Introduces RabbitMQ Streams, a high-throughput, log-append-only streaming protocol introduced in RabbitMQ 3.9. It compares RabbitMQ Streams' sub-millisecond latencies and message retention directly with traditional AMQP queues and Apache Kafka. The walkthrough showcases a complete consumer-producer application setup.

💡 Explore Related: Yaml | Databases | Newsql

High-Performance Messaging

(2026) ==Apache Artemis JMeter== ⭐ 1017 [ADVANCED LEVEL] 🌟🌟🌟🌟🌟 [DE FACTO STANDARD] [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" The official GitHub mirror of Apache ActiveMQ Artemis, housing the high-performance non-blocking asynchronous message broker. It provides native support for AMQP, MQTT, STOMP, and OpenWire. It delivers ultra-low latency and scalable message distribution under extreme workloads.

💡 Explore Related: Yaml | Databases | Newsql

JMS

(2024) ActiveMQ 5.x "classic" [DOCUMENTATION] 🌟🌟🌟🌟 [ENTERPRISE-STABLE] [LEGACY]

??? info "Technical Deep-Dive" The classic implementation of Apache ActiveMQ, continuing to power millions of production enterprise nodes. It offers rich support for JMS client specifications alongside robust clustering and persistence. Ideal for traditional integration architecture, though increasingly superseded by Artemis.
(1999) Apache ActiveMQ [DOCUMENTATION] 🌟🌟🌟🌟 [ENTERPRISE-STABLE] [LEGACY]

??? info "Technical Deep-Dive" An iconic, mature open-source multi-protocol message broker supporting JMS 1.1 and 2.0, AMQP, MQTT, and STOMP. Known for enterprise-grade reliability and complex message routing patterns. It remains a foundational asset in legacy integration environments globally.

💡 Explore Related: Yaml | Databases | Newsql

Kubernetes Native

(2026) KubeMQ.io: Kubernetes Native Message Queue Broker [DOCUMENTATION] 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" KubeMQ is an enterprise-grade, ultra-lightweight message broker engineered specifically for Kubernetes container ecosystems. Delivered in a minimal footprint, it supports pub/sub, queues, and streams with native GRPC and REST support. It avoids external operational dependencies.
(2024) github.com/kubemq-io/kubemq-community 🌟 ⭐ 668 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" The community-driven core repository for KubeMQ. It offers a lightweight, high-performance messaging interface for microservices on Kubernetes. Supports standard asynchronous protocols and integrates natively with Kubernetes patterns.
(2019) devops.com: Best of 2019: Implementing Message Queue in Kubernetes 🌟🌟🌟 [COMMUNITY-TOOL] [GUIDE]

??? info "Technical Deep-Dive" Evaluates the operational paradigms, stateful challenges, and strategies when setting up distributed message brokers natively inside Kubernetes environments. Discusses dynamic volume allocations, stateful sets, and persistent cloud networking protocols.

💡 Explore Related: Yaml | Databases | Newsql

Pub-Sub Pattern

(2026) Redis Pub/sub [DOCUMENTATION] 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" Official developer documentation detailing Redis' built-in Pub/Sub and Streams features. It provides technical blueprints for lightweight, fire-and-forget message passing and log-append streaming. This allows developers to construct fast messaging queues without setting up heavy broker architectures.

💡 Explore Related: Yaml | Databases | Newsql

Orchestration

Kubernetes Operators

(2024) Apache Camel K [ADVANCED LEVEL] [DOCUMENTATION] 🌟🌟🌟🌟 [DE FACTO STANDARD] [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" Core documentation for Apache Camel K, a lightweight cloud-native integration platform built on Kubernetes. It utilizes the Operator Pattern to run integration DSL routes serverlessly. It drastically simplifies deploying complex integration patterns across cloud-native domains.
(2021) thenewstack.io: Camel K Brings Apache Camel to Kubernetes for Event-Driven Architectures 🌟🌟🌟 [COMMUNITY-TOOL] [GUIDE]

??? info "Technical Deep-Dive" A comprehensive review of Camel K's architecture, analyzing its integration with Knative and Kubernetes-native messaging patterns. It describes how Camel K reduces traditional ESB resource consumption to support high-density container layouts.

💡 Explore Related: Yaml | Databases | Newsql

Reference Architecture

(2022) github.com/osa-ora/camel-k-samples 🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" A curated collection of practical code templates and sample deployment topologies demonstrating Camel K in action. Covers integrations with relational databases, message queues, and cloud endpoints. This repository is a valuable tool for accelerated prototyping.

💡 Explore Related: Yaml | Databases | Newsql

Serverless Integration

(2020) developers.redhat.com: Six reasons to love Camel K 🌟🌟🌟 [COMMUNITY-TOOL] [GUIDE]

??? info "Technical Deep-Dive" Outlines six key architectural advantages of Camel K, including fast deployment loops, native Quarkus optimization, low memory footprints, and serverless scale-to-zero capabilities via Knative. Highly useful for architects modernizing traditional ESBs.

💡 Explore Related: Yaml | Databases | Newsql

Cloud Native Architecture

Domain-Driven Design

Messaging Architectures

(2019) verraes.net: DDD and Messaging Architectures 🌟 [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE] [GUIDE]

??? info "Technical Deep-Dive" Synthesizes core concepts of Domain-Driven Design (DDD) with message-oriented middleware patterns. It examines bounded contexts, aggregate boundaries, and the strategic distribution of domain events. It provides deep conceptual clarity on decoupling enterprise service boundaries using asynchronous message paths.

💡 Explore Related: Yaml | Databases | Newsql

Event-Driven Systems

Foundations

(2021) thenewstack.io: The Rise of Event-Driven Architecture 🌟🌟🌟 [COMMUNITY-TOOL] [GUIDE]

??? info "Technical Deep-Dive" Traces the industry shift from request-response synchronous APIs to asynchronous event-driven models. It outlines the architectural advantages regarding system resilience, temporal decoupling, and scalability. The analysis evaluates standard broker technologies that enable reactive cloud-native systems.

💡 Explore Related: Yaml | Databases | Newsql

Patterns

(2021) codeopinion.com: Event Sourcing vs Event Driven Architecture [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE] [GUIDE]

??? info "Technical Deep-Dive" Clarifies the critical distinctions and synergies between Event Sourcing (capturing state transitions as events) and Event-Driven Architecture (broadcasting state changes). It uses architectural examples to prevent common integration anti-patterns. This assists architects in deciding when to combine or isolate these patterns.

💡 Explore Related: Yaml | Databases | Newsql

Standards

(2022) salaboy.com: Event-Driven applications with CloudEvents on Kubernetes [ADVANCED LEVEL] 🌟🌟🌟🌟 [EMERGING] [ENTERPRISE-STABLE] [GUIDE]

??? info "Technical Deep-Dive" Explains how the CNCF CloudEvents specification standardizes event metadata format across distinct systems. It integrates CloudEvents within Kubernetes architectures using tools like Knative Eventing. This provides an excellent overview of building vendor-neutral, highly reactive event mesh fabrics.

💡 Explore Related: Yaml | Databases | Newsql

Foundations

Introductory Patterns

(2024) ibm.com: Event-driven cloud-native applications (microservices) [DOCUMENTATION] 🌟🌟 [COMMUNITY-TOOL] [GUIDE]

??? info "Technical Deep-Dive" Explains core principles of cloud-native architecture, including containerization, microservices, and reactive behaviors. It outlines the foundational tenets necessary to design robust applications optimized for public and private clouds. It serves as a high-level conceptual reference for infrastructure modernization.

💡 Explore Related: Yaml | Databases | Newsql

Inter-Service Communication

Performance

(2020) particular.net: RPC vs. Messaging – which is faster? 🌟🌟🌟 [COMMUNITY-TOOL] [GUIDE]

??? info "Technical Deep-Dive" Provides a detailed comparative benchmark of Remote Procedure Call (RPC) protocols versus messaging-based asynchronous protocols. It highlights how latency, queue depths, network overhead, and decoupling impact application performance under high load. It concludes that throughput gains in asynchronous messaging often outweigh synchronous RPC latency benefits.

💡 Explore Related: Yaml | Databases | Newsql

Microservices

Change Data Capture CDC

(2020) developers.redhat.com: Change data capture for microservices without writing any code 🌟🌟🌟🌟 [ENTERPRISE-STABLE] [GUIDE]

??? info "Technical Deep-Dive" Walkthrough detailing how to set up out-of-the-box Change Data Capture architectures using Debezium without custom application-level code. It demonstrates immediate real-time synchronization from database transactions straight to Kafka-enabled microservices.
(2019) developers.redhat.com: Decoupling microservices with Apache Camel and Debezium [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE] [GUIDE]

??? info "Technical Deep-Dive" Explains how to decouple distributed database structures in microservices by employing a combination of Debezium (for Change Data Capture) and Apache Camel (for integration and transformation pipelines). It ensures low latency, resilient state updates.

💡 Explore Related: Yaml | Databases | Newsql

Distributed Transactions

(2021) developers.redhat.com: Distributed transaction patterns for microservices compared [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE] [GUIDE]

??? info "Technical Deep-Dive" Analyzes and contrasts critical transactional strategies for microservice boundaries, including 2PC (Two-Phase Commit), Sagas, and Outbox patterns. It highlights how asynchronous message-passing mitigates the failure modes of distributed transactions. Practical implementation guidelines focus on maintaining eventual consistency without tight coupling.

💡 Explore Related: Yaml | Databases | Newsql

Event Sourcing

(2021) blog.bitsrc.io: Why Microservices Should use Event Sourcing 🌟 🌟🌟🌟 [COMMUNITY-TOOL] [GUIDE]

??? info "Technical Deep-Dive" Argues the case for event sourcing as a primary mechanism to store state in distributed microservice topologies. It highlights capabilities such as complete audit trails, high-performance writes, and historical state reconstruction. The post warns of common pitfalls including schema evolution complexity and read projection overhead.

💡 Explore Related: Yaml | Databases | Newsql

Cloud Native Infrastructure

High Availability

Kafka on Kubernetes

(2022) ==learnk8s.io/kafka-ha-kubernetes: Designing and testing a highly available Kafka cluster on Kubernetes 🌟== [ADVANCED LEVEL] 🌟🌟🌟🌟🌟 [DE FACTO STANDARD] [GUIDE]

??? info "Technical Deep-Dive" Curator Insight: Outstanding reference tutorial for validating HA setups in production. Live Grounding: Outlines designing multi-AZ deployments, tuning pod disruption budgets (PDBs), and performing simulated chaos engineering tests (network partitions, node restarts) to prove zero-data-loss capabilities.

💡 Explore Related: Yaml | Databases | Newsql

Kafka on Kubernetes

Application Integration

(2021) itnext.io: Sending Messages to Kafka in Kubernetes [COMMUNITY-TOOL] [GUIDE]

??? info "Technical Deep-Dive" Curator Insight: Practical post on establishing low-latency, secure client connections to Kafka brokers inside a Kubernetes network boundary. Live Grounding: Reviews internal DNS routing, ingress endpoints, and SASL authentication configs to safely bridge containerized publishers and consumer workloads.

💡 Explore Related: Yaml | Databases | Newsql

Deployments

(2023) thelinuxnotes.com: How to deploy Kafka in Kubernetes with Helm chart + kafdrop [COMMUNITY-TOOL] [GUIDE]

??? info "Technical Deep-Dive" Curator Insight: Deployment tutorial utilizing established community Helm packages. Live Grounding: Guides deployment of a functional Kafka test cluster integrated directly with Kafdrop as a companion visual administration interface.

💡 Explore Related: Yaml | Databases | Newsql

Guides

(2022) linkedin.com: Kafka Cluster Setup on Kubernetes 🌟🌟🌟🌟 [ENTERPRISE-STABLE] [GUIDE]

??? info "Technical Deep-Dive" Curator Insight: Step-by-step technical guide for provisioning Kafka on Kubernetes using direct manifests. Live Grounding: Covers statefulsets, headless service definitions, volume claim templates, and environment variables targeting manual multi-broker cluster creation.

💡 Explore Related: Yaml | Databases | Newsql

Local Development

(2021) dev.to: Running Kafka on kubernetes for local development [COMMUNITY-TOOL] [GUIDE]

??? info "Technical Deep-Dive" Curator Insight: Practical setup workflow for running local Kafka configurations under Minikube or Docker Desktop. Live Grounding: Explains minimal YAML profiles using Helm or lightweight operators to quickly spin up development broker instances for sandboxed microservices validation.

💡 Explore Related: Yaml | Databases | Newsql

Kubernetes Strategy

Infrastructure Decisions

(2023) thenewstack.io: Kafka on Kubernetes: Should You Adopt a Managed Solution? 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" Curator Insight: Strategic evaluation of managed SaaS Kafka setups versus DIY operator approaches on Kubernetes. Live Grounding: Compares total cost of ownership (TCO), maintenance scaling, day-2 operations complexity, and custom flexibility demands.

💡 Explore Related: Yaml | Databases | Newsql

Security

Amazon EKS

(2021) itnext.io: Securely Decoupling Kubernetes-based Applications on Amazon EKS using Kafka with SASL/SCRAM [COMMUNITY-TOOL] [GUIDE]

??? info "Technical Deep-Dive" Curator Insight: Hardening guide detailing connection security between AWS EKS-based consumers and Kafka clusters. Live Grounding: Demonstrates configuring Kafka users with SASL/SCRAM credentials, managing secrets within Kubernetes natively, and establishing encrypted TLS connectivity over VPC boundaries.

💡 Explore Related: Yaml | Databases | Newsql

Serverless Data Platforms

Elastic Kafka

(2021) confluent.io: Making Apache Kafka Serverless: Lessons From Confluent Cloud [ADVANCED LEVEL] [CASE STUDY] [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Curator Insight: Architectural retrospective on how Confluent engineered a multi-tenant, elastic serverless Kafka platform. Live Grounding: Explores storage-compute decoupling, automated partition rebalancing, and custom multi-tenant billing-aware resource allocators.

💡 Explore Related: Yaml | Databases | Newsql

Stateful Workloads

Kafka on Kubernetes

(2021) phoenixnap.com: How to Set Up and Run Kafka on Kubernetes 🌟 🌟🌟🌟🌟 [ENTERPRISE-STABLE] [GUIDE]

??? info "Technical Deep-Dive" Curator Insight: Comprehensive guide to running stateful Kafka clusters on Kubernetes platforms. Live Grounding: Outlines deploying Kafka utilizing statefulsets, configuring persistent volumes, and handling network routing. Explores the advantages of operator-managed setups versus standard manual deployments.

💡 Explore Related: Yaml | Databases | Newsql

Strimzi Operator

==strimzi.io== [ADVANCED LEVEL] 🌟🌟🌟🌟🌟 [DE FACTO STANDARD]

??? info "Technical Deep-Dive" Curator Insight: The leading open-source CNCF sandbox operator platform for running Kafka on Kubernetes. Live Grounding: Orchestrates secure topologies, cluster expansion, user management, and seamless rolling upgrades using fully declarative Kubernetes Custom Resources (CRDs).
(2020) developers.redhat.com: Introduction to Strimzi: Apache Kafka on Kubernetes (KubeCon Europe 2020) 🌟 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" Curator Insight: Detailed breakdown of the Strimzi operator architectural internals from KubeCon. Live Grounding: Evaluates how the operator automates bootstrap, health monitoring, protocol configurations, TLS generation, and storage management for Kafka on Kubernetes.
(2021) strimzi.io: Kafka upgrade improvements [ADVANCED LEVEL] [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Curator Insight: Direct technical update from the Strimzi maintainers regarding Kafka upgrade orchestration. Live Grounding: Details the architectural improvements in Strimzi's reconciliation loops, enabling automated, zero-downtime rolling upgrades of stateful Kafka pods with strict schema protection.

💡 Explore Related: Yaml | Databases | Newsql

Tooling and UI

(2024) pepy.tech/project/strimzi-kafka-cli 🌟 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Curator Insight: Download analytics and overview of the Strimzi Kafka CLI. Live Grounding: Provides python-based CLI tools to interactively administer Strimzi-managed custom resources, simplifying manual deployment, topic configuration, and user creation operations.

💡 Explore Related: Yaml | Databases | Newsql

Cloud-Native Infrastructure

Event Streaming

GitOps Practices

(2021) confluent.io: DevOps for Apache Kafka with Kubernetes and GitOps 🌟 [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" Examines GitOps models for coordinating declarative configurations of schemas, partitions, ACLs, and topics across multiple Kubernetes-hosted Kafka environments using automated pipelines.

💡 Explore Related: Yaml | Databases | Newsql

Infrastructure Operations

(2018) tecmint: How to Install Apache Kafka in CentOS/RHEL 7 🌟🌟 [COMMUNITY-TOOL] [GUIDE]

??? info "Technical Deep-Dive" A technical operational guide detailing the installation and configuration of Zookeeper and Apache Kafka services directly on bare-metal or VM instances running CentOS and RHEL 7.

💡 Explore Related: Yaml | Databases | Newsql

Kafka Connect Operators

(2021) developers.redhat.com: Improve your Kafka Connect builds of Debezium. 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" A practical walk-through of the Strimzi Operator build processes for deploying optimized Kafka Connect container environments on Kubernetes. Illustrates declarative custom resource setups to bundle custom Debezium connector packages safely.

💡 Explore Related: Yaml | Databases | Newsql

Kubernetes Operators

(2021) openshift.com: How to Orchestrate Data Pipelines with Applications Deployed on OpenShift [ADVANCED LEVEL] 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Demonstrates deployment patterns for co-locating high-throughput data processing pipelines (Argo, Apache Spark, Strimzi) directly alongside backend application microservices inside Red Hat OpenShift clusters.
(2021) thenewstack.io: Beyond the Quickstart: Running Apache Kafka as a Service on Kubernetes [ADVANCED LEVEL] 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Provides architectural guidelines on hosting stable, production-grade Apache Kafka clusters on Kubernetes. Explores state persistence requirements, advanced container network setups, and operational recovery pipelines.
(2020) containerjournal.com: Red Hat Platform Brings Kafka Closer to Kubernetes 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Discusses Strimzi operators on Red Hat OpenShift, evaluating how declaratively defined operators streamline stateful deployments and coordinate safe, automated rolling updates of Kafka nodes on Kubernetes.

💡 Explore Related: Yaml | Databases | Newsql

Hybrid Cloud Platforms

Anthos Deployments

(2020) confluent.fr: Infrastructure Modernization with Google Anthos and Apache Kafka [FRENCH CONTENT] [ADVANCED LEVEL] [CASE STUDY] 🌟🌟🌟 [CASE STUDY] [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Provides architectural guidelines for deploying federated Confluent Kafka setups across local datacenters and public Google Cloud regions using Google Anthos configuration models [FRENCH CONTENT].

💡 Explore Related: Yaml | Databases | Newsql

Modernization Strategy

(2021) kai-waehner.de: App Modernization and Hybrid Cloud Architectures with Apache Kafka 🌟🌟🌟 [LEGACY]

??? info "Technical Deep-Dive" Examines legacy migration models utilizing Apache Kafka as an integration buffer. Discusses streaming events between on-prem mainframe systems and agile cloud-native microservices with zero downtime.

💡 Explore Related: Yaml | Databases | Newsql

Infrastructure as Code IaC

Event-Driven Provisioning

(2021) daily.dev: Building a fault-tolerant event-driven architecture with Google Cloud, Pulumi and Debezium 🌟🌟🌟 [COMMUNITY-TOOL] [GUIDE]

??? info "Technical Deep-Dive" An infrastructure guide detailing the automated configuration of a fault-tolerant event-driven backend. Demonstrates the step-by-step deployment of Google Cloud resources and Debezium instances using Pulumi for stateful, declarative management.

💡 Explore Related: Yaml | Databases | Newsql

Serverless Computing

Knative Eventing

(2021) piotrminkowski.com: Knative Eventing with Kafka and Quarkus [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" Illustrates the deployment of Knative serverless application endpoints coordinated with Apache Kafka event feeds. Utilizes Quarkus microservices to demonstrate scale-to-zero configurations that adapt automatically to stream ingestion.

💡 Explore Related: Yaml | Databases | Newsql

Data Architecture

Data Lakehouse

Iceberg Integration

(2021) debezium.io: Using Debezium to Create a Data Lake with Apache Iceberg [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" Explains how to feed streaming transaction logs directly into Apache Iceberg storage using Debezium CDC and Kafka Connect. Outlines strategies for supporting dynamic schema evolution and ensuring transactional ACID-level safety on cheap cloud object stores.

💡 Explore Related: Yaml | Databases | Newsql

Data Mesh

Cloud-Native Platforms

(2021) mrpaulandrew.com: BUILDING A DATA MESH ARCHITECTURE IN AZURE – PART 2 [ADVANCED LEVEL] 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" A platform implementation guide focusing on assembling a production-ready Data Mesh within Microsoft Azure. Explores multi-workspace configurations utilizing Azure Synapse, Azure Purview, and Data Factory within enterprise environments.

💡 Explore Related: Yaml | Databases | Newsql

Domain-Driven Design

(2021) towardsdatascience.com: Data Domains and Data Products 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Focuses on building discrete, discoverable, and governed domain-centric data products. Reviews core responsibilities for product engineering teams and logical boundaries required to achieve seamless interoperability within a Data Mesh.

💡 Explore Related: Yaml | Databases | Newsql

Foundational Principles

(2020) ==martinfowler.com: Data Mesh Principles and Logical Architecture== [ADVANCED LEVEL] 🌟🌟🌟🌟🌟 [DE FACTO STANDARD]

??? info "Technical Deep-Dive" The seminal architectural blueprint by Zhamak Dehghani introducing Data Mesh principles. Focuses on the core four pillars: domain-driven decentralized data ownership, data-as-a-product, self-serve data infrastructure platforms, and federated computational governance.

💡 Explore Related: Yaml | Databases | Newsql

Migration Strategies

(2019) ==martinfowler.com: How to Move Beyond a Monolithic Data Lake to a Distributed Data Mesh== [ADVANCED LEVEL] 🌟🌟🌟🌟🌟 [DE FACTO STANDARD]

??? info "Technical Deep-Dive" The pioneering analysis introducing the Data Mesh framework. Outlines failure modes of traditional centralized databases and enterprise data lakes, presenting a distributed, domain-driven data topology as a scalable alternative.

💡 Explore Related: Yaml | Databases | Newsql

Strategic Overview

(2020) infoq.com: Data Mesh Principles and Logical Architecture Defined 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" An executive summary analyzing Zhamak Dehghani's foundational Data Mesh concepts. Contemplates the operational and architectural pivot from centralized monolithic data pools to distributed, domain-centric, and governed team landscapes.

💡 Explore Related: Yaml | Databases | Newsql

Data Science Platform

Real-Time Machine Learning

(2018) confluent.io: How to Build and Deploy Scalable Machine Learning in Production with Apache Kafka [ADVANCED LEVEL] 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Reviews the architecture designs required for deploying, evaluating, and monitoring analytical machine learning models against fast event-streams. Utilizes Apache Kafka as the backbone for scalable ingestion.

💡 Explore Related: Yaml | Databases | Newsql

Event Streaming

Architectural Patterns

(2021) davidxiang.com: Kafka As A Database? Yes Or No [ADVANCED LEVEL] 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Evaluates the controversial 'Kafka as a database' design model. Analyzes the trade-offs of using Kafka for data persistence, explaining limits on random queries and index lookups relative to typical relational/NoSQL setups.

💡 Explore Related: Yaml | Databases | Newsql

Audio Curation

(2020) softwareengineeringdaily.com: Kafka Applications with Tim Berglund (podcast) 🌟 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" A podcast conversation detailing real-world patterns for building highly decoupled event-driven systems. Explores streaming architecture choices, data evolution, and microservice communication patterns with Tim Berglund.

💡 Explore Related: Yaml | Databases | Newsql

Cluster Management

(2026) ==AKHQ (previously known as KafkaHQ) 🌟== ⭐ 3804 🌟🌟🌟🌟🌟 [DE FACTO STANDARD]

??? info "Technical Deep-Dive" A powerful, feature-rich web console for administering Kafka cluster resources. Supports direct topic data browsing, consumer group rebalancing monitoring, schema registry integrations, and multi-tenant ACL audits.
(2021) confluent.io: Simplifying Apache Kafka Multi-Cluster Management Using Control Center and Cluster Registry [ADVANCED LEVEL] 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Explains methods for operating federated or geographically-dispersed Kafka clusters. Details patterns for maintaining centralized visibility and configuring multi-cluster pipelines using Confluent Control Center.

💡 Explore Related: Yaml | Databases | Newsql

Consumer Coordination

(2021) blog.cloudera.com: Scalability of Kafka Messaging using Consumer Groups 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Analyzes the scaling behaviors of consumer groups in Apache Kafka. Outlines dynamic partition rebalancing algorithms, protocol designs, and strategies to minimize processing lag in active networks.

💡 Explore Related: Yaml | Databases | Newsql

Data Pipelines

(2019) Single Message Transformations - The Swiss Army Knife of Kafka Connect 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" An deep-dive breakdown of Single Message Transformations (SMTs) within Kafka Connect. Shows how to filter, modify, anonymize, and restructure record payloads on-the-fly without requiring customized stream computing logic.

💡 Explore Related: Yaml | Databases | Newsql

Development Tutorials

(2021) ==kafka-tutorials.confluent.io 🌟== 🌟🌟🌟🌟🌟 [DE FACTO STANDARD] [GUIDE]

??? info "Technical Deep-Dive" The premier tutorial index hosted by Confluent. Provides a rich set of runnable recipes demonstrating microservice streaming actions, temporal joins, window operations, and message transformations using ksqlDB and Kafka Streams.
(2021) kafka-tutorials.confluent.io: How to count messages in a Kafka topic 🌟🌟🌟 [COMMUNITY-TOOL] [GUIDE]

??? info "Technical Deep-Dive" A precise development recipe outlining how to count incoming records inside Apache Kafka topics using ksqlDB. Details the construction of stateful materialized views for monitoring live volumes.

💡 Explore Related: Yaml | Databases | Newsql

Foundational Principles

(2021) Confluent.io: Intro to Apache Kafka: How Kafka Works 🌟 [DOCUMENTATION] 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" A foundational, highly descriptive reference for Kafka basics. Explains structural layouts of partitions, records, offsets, log retention, and replication, ensuring developers master core broker fundamentals.

💡 Explore Related: Yaml | Databases | Newsql

IoT Telemetry Integration

(2021) kai-waehner.de: Apache Kafka and MQTT (Part 1 of 5) – Overview and Comparison 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" A systematic deep dive comparing MQTT brokers to Kafka. Reviews how to build end-to-end telemetry systems, deploying MQTT at the edge for lightweight transport and Kafka at the core for analytical streaming.

💡 Explore Related: Yaml | Databases | Newsql

Message Brokers

(2026) ==Apache Kafka== [DOCUMENTATION] 🌟🌟🌟🌟🌟 [DE FACTO STANDARD]

??? info "Technical Deep-Dive" The main portal for Apache Kafka, the industry de facto standard distributed event streaming engine. It outlines critical capabilities including partition clustering, transactional controls, offset management, and high-performance ingestion designs.

💡 Explore Related: Yaml | Databases | Newsql

Metadata Management KRaft

(2021) devclass.com: Apache Kafka 2.8.0 previews life without ZooKeeper 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Analyzes the operational and administrative benefits of ZooKeeper removal. Reviews how KRaft architecture improves cluster limits, simplifies administrator overhead, and accelerates recovery speeds during node failures.

💡 Explore Related: Yaml | Databases | Newsql

Resource Indexes

(2026) Awesome Streaming 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" A highly curated meta-resource listing frameworks, engine architectures, academic publications, and database connectors within the streaming data ecosystem. Covers key analytical and event-driven technologies.
(2026) Awesome Kafka 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" A rich community collection of operational utilities, libraries, and GUI packages optimized for developers and administrators deploying and scaling Apache Kafka systems.

💡 Explore Related: Yaml | Databases | Newsql

Video Tutorials

(2020) youtube playlist: Kafka Connect Tutorials | Kafka Connect 101: REST API 🌟 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" An educational video series illustrating the programmatic administration of Kafka Connect connectors. Details the layout of the Connect REST API for creating, validating, scaling, and debugging stateful data integrations.

💡 Explore Related: Yaml | Databases | Newsql

Event-Driven Data

Change Data Capture CDC

(2021) shopify.engineering: Capturing Every Change From Shopify’s Sharded Monolith [ADVANCED LEVEL] [CASE STUDY] 🌟🌟🌟🌟 [CASE STUDY] [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" This architectural case study highlights Shopify's high-throughput solution for real-time mutation extraction from sharded MySQL clusters. By combining Debezium with customized Apache Kafka configurations, the system secures sub-second delivery while safely preserving transaction-order invariants at massive scale.
(2020) vladmihalcea.com: A beginner’s guide to CDC (Change Data Capture) 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" A comprehensive structural overview of Change Data Capture (CDC) design patterns. It details transaction log parsing, dual-writes mitigation, and the key architectural differences between query-based and log-based CDC solutions. This acts as an essential primer for development groups transitioning from monolith DB schemas to real-time event streaming systems.
(2020) developers.redhat.com: Capture database changes with Debezium Apache Kafka connectors 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" A hands-on manual detailing the implementation of Debezium to safely convert relational database modifications into real-time Kafka event feeds. Outlines event formatting, partition strategy, and recovery procedures.
(2020) Build a simple cloud-native change data capture pipeline 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Illustrates how to engineer a low-latency, cloud-native CDC pipeline utilizing Debezium connectors alongside open database architectures. Explores data serialization optimizations and horizontal scale metrics.

💡 Explore Related: Yaml | Databases | Newsql

Compliance Systems

(2018) infoq.com: Building a SQL Database Audit System using Kafka, MongoDB and Maxwell's Daemon [ADVANCED LEVEL] 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Examines a robust architectural design for database auditing. Uses Maxwell's Daemon to ingest database binlogs, routing changes through Kafka into MongoDB to form a tamper-resistant historical record.

💡 Explore Related: Yaml | Databases | Newsql

Data Federation

(2020) Event streaming and data federation: A citizen integrator’s story 🌟🌟 [LEGACY]

??? info "Technical Deep-Dive" Examines low-code patterns for connecting real-time streaming architectures with legacy enterprise databases. Explains how federation tools bridge information gaps between non-technical users and distributed message networks.

💡 Explore Related: Yaml | Databases | Newsql

Debezium Connectors

(2021) developers.redhat.com: Db2 and Oracle connectors coming to Debezium 1.4 GA 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" A technical release breakdown detailing the production-grade integration of IBM Db2 and Oracle connectors within the Debezium 1.4 ecosystem. It reviews performance benchmarks, log-mining mechanisms, and setup procedures critical for cloud-native enterprise migrations.

💡 Explore Related: Yaml | Databases | Newsql

Schema Governance

Event-Driven Governance

(2021) developers.redhat.com: Event-driven APIs and schema governance for Apache Kafka: Get ready for Kafka Summit Europe 2021 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Synthesizes key themes of Kafka Summit Europe 2021, detailing patterns in distributed schema governance, central API catalogs, and microservice integration design protocols.

💡 Explore Related: Yaml | Databases | Newsql

Microservices Design Patterns

(2021) redhat.com: Using a schema registry to ensure data consistency between microservices 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Analyzes the operational role of schema registries in maintaining system stability. Highlights how decoupled producers and consumers leverage registries for backward and forward schema compatibility, protecting distributed microservices from payload parsing errors.

💡 Explore Related: Yaml | Databases | Newsql

Service Registry

(2026) Apicurio Registry ⭐ 806 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" Apicurio Registry is a high-performance, open-source centralized schema registry. It enables teams to maintain and store OpenAPI, AsyncAPI, Avro, Protobuf, and JSON schemas, supporting real-time validation layers in high-throughput microservice pipelines.
(2019) Red Hat Integration service registry 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" An introductory architecture guide describing the capabilities of the Red Hat Integration Service Registry. Reviews standard patterns for managing API schemas (Avro, JSON Schema, Protobuf) to guarantee strong message-contract enforcement in decoupled broker networks.

💡 Explore Related: Yaml | Databases | Newsql

Stream Processing

Evolutionary Topologies

(2020) thenewstack.io: Part 1: The Evolution of Data Pipeline Architecture 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Traces the structural progress of big data analytics pipelines. Focuses on the architectural evolution from high-latency batch map-reduce jobs to real-time Kappa and Lambda messaging topologies.

💡 Explore Related: Yaml | Databases | Newsql

Managed Pipelines

(2021) cloudblog.withgoogle.com: Turn any Dataflow pipeline into a reusable template 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" A technical look at creating and managing GCP Dataflow templates. Details packaging patterns that permit modular, parameter-driven run execution of streaming data pipelines across multi-tenant infrastructures.

💡 Explore Related: Yaml | Databases | Newsql

Microservices Frameworks

(2020) Build a data streaming pipeline using Kafka Streams and Quarkus 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Demonstrates the construction of microsecond-responsive streams using the Kafka Streams API paired with Quarkus. Explores native execution compilation patterns to reduce JVM memory overhead and launch latency.

💡 Explore Related: Yaml | Databases | Newsql

Data Engineering

Change Data Capture CDC

Cloud Managed Services

(2020) debezium.io: Lessons Learned from Running Debezium with PostgreSQL on Amazon RDS [ADVANCED LEVEL] 🌟🌟🌟🌟 [CASE STUDY] [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" A highly valuable technical case study sharing performance profiles, optimization constraints, and gotchas when operating Debezium alongside Amazon RDS PostgreSQL. It details replication slot configurations, WAL storage management, and handling heavy transaction volumes under AWS limitations.

💡 Explore Related: Yaml | Databases | Newsql

PostgreSQL

(2021) info.crunchydata.com: PostgreSQL Change Data Capture With Debezium [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE] [GUIDE]

??? info "Technical Deep-Dive" A comprehensive operational manual focused on establishing Debezium CDC connectors specifically for enterprise-grade PostgreSQL deployments. It details WAL level adjustments, logical replication slot configuration, and the extraction of mutation events for consumer engines.

💡 Explore Related: Yaml | Databases | Newsql

Real-Time Data Streaming

(2026) ==Debezium:== [ADVANCED LEVEL] [DOCUMENTATION] 🌟🌟🌟🌟🌟 [DE FACTO STANDARD] [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" The industry-leading, open-source distributed platform for Change Data Capture (CDC). Built on top of Apache Kafka, it taps database transaction logs in real-time, streaming row-level mutations downstream without querying databases. Essential for low-latency event-driven microservices.

💡 Explore Related: Yaml | Databases | Newsql

Stream Processing

(2021) noti.st: Change Data Capture with Flink SQL and Debezium 🌟 [ADVANCED LEVEL] 🌟🌟🌟🌟 [EMERGING] [ENTERPRISE-STABLE] [GUIDE]

??? info "Technical Deep-Dive" A visual presentation sharing architectural strategies to integrate Debezium with Apache Flink SQL for high-speed continuous stream processing. Explains patterns for building real-time materialized views, continuous aggregations, and live analytics directly from database mutation logs.

💡 Explore Related: Yaml | Databases | Newsql

Data Culture

Real-Time Data Streaming

(2021) linkedin.com: How to Move From a “Wait for it...” Batch-Processing Culture to a “Get It Now” Real-Time Data Culture 🌟🌟 [COMMUNITY-TOOL] [GUIDE]

??? info "Technical Deep-Dive" Discusses the cultural and systemic paradigm shift required for enterprises moving from batch scheduling to real-time event-driven insights. It touches on organizational friction, architectural changes, and immediate business advantages. It serves as a non-technical strategic guide for data transformation.

💡 Explore Related: Yaml | Databases | Newsql

Data Pipelines

Cloud Native Architectures

(2020) towardsdatascience.com: Architecture for High-Throughput Low-Latency Big Data Pipeline on Cloud 🌟 [ADVANCED LEVEL] 🌟🌟🌟 [COMMUNITY-TOOL] [GUIDE]

??? info "Technical Deep-Dive" Evaluates design principles for high-throughput, low-latency cloud-native big data architectures. The guide details how to integrate ingestion layers with stream processing engines and distributed analytical databases. It presents structured architectural templates for unified analytical and machine learning workloads.

💡 Explore Related: Yaml | Databases | Newsql

Data on Kubernetes

Orchestration

(2021) thenewstack.io: The Path to Getting the Full Data Stack on Kubernetes [ADVANCED LEVEL] 🌟🌟🌟 [EMERGING] [GUIDE]

??? info "Technical Deep-Dive" Explores the evolutionary path of running complex, stateful database and data streaming systems natively on Kubernetes. It addresses the maturity of operators, storage classes, and orchestrators that facilitate the deployment of the complete data pipeline. The article details challenges regarding resource management and high availability.

💡 Explore Related: Yaml | Databases | Newsql

Real-Time Data Streaming

Data Stack

(2022) thenewstack.io: Streaming Data and the Modern Real-Time Data Stack 🌟🌟🌟 [COMMUNITY-TOOL] [GUIDE]

??? info "Technical Deep-Dive" Discusses the components constituting the modern real-time data stack, emphasizing continuous streaming over traditional batch ETL. It explores the roles of message logs, stream processors, and real-time OLAP databases. This provides a blueprint for engineering low-latency analytics systems.

💡 Explore Related: Yaml | Databases | Newsql

Foundations

(2022) thenewstack.io: How to Get Started with Data Streaming 🌟🌟🌟 [COMMUNITY-TOOL] [GUIDE]

??? info "Technical Deep-Dive" A beginner-to-intermediate guide outlining initial workflows for setting up real-time stream ingestion and processing pipelines. It reviews primary tooling such as Apache Kafka and Apache Flink. It offers guidance on mapping traditional batch datasets into real-time pipelines.

💡 Explore Related: Yaml | Databases | Newsql

Data Infrastructure

Data Architecture

Data as a Service

Integrations

(2020) mongodb.com: DaaS with MongoDB and Confluent [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" An architecture case study exploring how to design a modern Data-as-a-Service (DaaS) paradigm using MongoDB and Confluent. Focuses on real-time CDC synchronization mechanisms and state persistence across high-throughput microservices.

💡 Explore Related: Yaml | Databases | Newsql

Event Streaming

Apache Kafka

Enterprise Distribution

(2026) ==confluent.io== 🌟🌟🌟🌟🌟 [DE FACTO STANDARD]

??? info "Technical Deep-Dive" The enterprise cloud-native streaming data platform built on top of Apache Kafka. Confluent provides fully managed SaaS offerings, enterprise schema management, cloud-to-local replication, and declarative connectors for data warehouses.

💡 Explore Related: Yaml | Databases | Newsql

Integrations

(2021) strimzi.io: Using HTTP Bridge as a Kubernetes sidecar 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" An exploration of deploy-time design patterns using the Strimzi HTTP Bridge as a Kubernetes sidecar container. This integration simplifies microservices communications by providing standard HTTP REST endpoints to interact with underlying Kafka event-driven pipelines.

💡 Explore Related: Yaml | Databases | Newsql

Management Tools

(2026) ==conduktor.io 🌟== 🌟🌟🌟🌟🌟 [DE FACTO STANDARD]

??? info "Technical Deep-Dive" An enterprise-grade desktop and cloud management platform for Apache Kafka that simplifies queue monitoring, schema registry auditing, and multi-cluster testing. It features advanced user security, performance monitoring, and message generation tools.

💡 Explore Related: Yaml | Databases | Newsql

Monitoring

(2021) strimzi/strimzi-canary ⭐ 42 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" A deployment-ready diagnostic tool that acts as a canary monitor within Kafka clusters. It helps ops teams measure round-trip message latency, validation success, and consumer group responsiveness under realistic workloads.
(2020) confluent.io: Monitoring Your Event Streams: Integrating Confluent with Prometheus and Grafana 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" A guide showing how to set up robust monitoring patterns for Apache Kafka cluster metrics using Prometheus and Grafana. Details exact exporter configurations and provides ready-to-use visualizations of critical performance telemetry.

💡 Explore Related: Yaml | Databases | Newsql

Operators

(2024) Banzai Kafka Operator ⭐ 792 [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" The Banzai Cloud Koperator simplifies Apache Kafka operations on top of Kubernetes clusters. It implements granular auto-scaling, Cruise Control-assisted broker load rebalancing, and self-healing systems directly within the cluster scheduler.

💡 Explore Related: Yaml | Databases | Newsql

Security

(2020) strimzi.io: Using Open Policy Agent with Strimzi and Apache Kafka [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" A detailed security-focused guide illustrating the integration of Open Policy Agent (OPA) with Strimzi and Apache Kafka. Explains how to enforce centralized, declarative, and fine-grained access control policies across streaming clusters.

💡 Explore Related: Yaml | Databases | Newsql

Strimzi Operators

(2022) strimzi/kafka-kubernetes-config-provider: Kubernetes Configuration Provider for Apache Kafka ⭐ 30 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" A specialized Kubernetes configuration provider for Apache Kafka that enables Kafka applications to read configuration data dynamically from Kubernetes Secrets and ConfigMaps. It simplifies the secure mounting of TLS certificates and broker credentials directly in client workloads.
(2020) blog.jromanmartin.io: How to upgrade Strimzi Operator using the CLI 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Provides a practical, command-line step-by-step procedure to upgrade Strimzi Operator versions on live Kubernetes/OpenShift clusters while avoiding message pipeline downtime.
(2021) strimzi.io: Using Kubernetes Configuration Provider to load data from Secrets and Config Maps 🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" This blog post details the implementation of the Strimzi Kubernetes Config Provider. It demonstrates how to decouple configurations from code by directly mapping Kafka properties to Kubernetes-managed infrastructure configurations.

💡 Explore Related: Yaml | Databases | Newsql

Architectural Patterns

Comparisons

(2021) softkraft.co: WS Kinesis vs Kafka comparison: Which is right for you? 🌟 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" An objective comparative analysis contrasting Amazon Kinesis and Apache Kafka across parameters like performance, architecture, pricing, and infrastructure overhead. Helps architects select the ideal event engine for specific scaling targets.
(2020) Pulsar vs Kafka – Comparison and Myths Explored [ADVANCED LEVEL] 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" A detailed technical breakdown comparing Apache Kafka and Apache Pulsar. Evaluates performance benchmarks, architecture complexities, replication topologies, and real-world deployment challenges.

💡 Explore Related: Yaml | Databases | Newsql

Business Ecosystem

Partnerships

(2021) confluent.io: Confluent and Microsoft Announce Strategic Alliance 🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Analysis of the strategic alignment between Microsoft and Confluent. Describes integrations for native resource provisioning, unified billing portals, and security optimizations within the Azure cloud environment.

💡 Explore Related: Yaml | Databases | Newsql

Cloud-Native Streaming

AWS

(2026) ==AWS Kinesis== [DOCUMENTATION] 🌟🌟🌟🌟🌟 [DE FACTO STANDARD]

??? info "Technical Deep-Dive" Official AWS documentation for Kinesis Data Streams. Highly resilient, fully managed cloud service for real-time data streaming at scale, designed for seamless integrations within the AWS ecosystem and serverless application designs.

💡 Explore Related: Yaml | Databases | Newsql

Modern Alternatives

Apache Pulsar

(2026) ==Apache Pulsar== [ADVANCED LEVEL] 🌟🌟🌟🌟🌟 [DE FACTO STANDARD]

??? info "Technical Deep-Dive" A highly scalable cloud-native event streaming model that separates compute (Apache Pulsar brokers) from state/storage (Apache BookKeeper). Ideal for multi-tenant, geographically distributed messaging workloads that require decoupled horizontal scaling.

💡 Explore Related: Yaml | Databases | Newsql

Interviews

(2021) softwareengineeringdaily.com: Redpanda: Kafka Alternative with Alexander Gallego 🌟 [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" An analytical podcast interview featuring Redpanda founder Alexander Gallego. Delves into the C++ implementation details, thread-per-core architectures, and the structural decisions that differentiate Redpanda from JVM-based event processors.

💡 Explore Related: Yaml | Databases | Newsql

Licensing

(2021) Redpanda is now Free & Source Available 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" The official blog post detailing Redpanda's pivot to the Business Source License (BSL). It provides a high-level corporate and architectural perspective regarding license changes and global open-source resource sustainability.

💡 Explore Related: Yaml | Databases | Newsql

Redpanda

(2026) ==Redpanda 🌟== 🌟🌟🌟🌟🌟 [DE FACTO STANDARD]

??? info "Technical Deep-Dive" An ultra-fast, C++ based, Seastar engine implementation of Kafka API protocols. Redpanda acts as a direct, lightweight replacement for Apache Kafka that removes heavy JVM tuning and ZooKeeper/KRaft runtimes, significantly lowering hardware footprints.

💡 Explore Related: Yaml | Databases | Newsql

Red Hat AMQ Streams

Components

(2019) Understanding Red Hat AMQ Streams components for OpenShift and Kubernetes 🌟 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Part 1 of an analytical breakdown detailing the components of Red Hat AMQ Streams (built on Strimzi). Explains operators, ZooKeeper configurations, and Kafka broker deployment patterns within enterprise Kubernetes clusters.

💡 Explore Related: Yaml | Databases | Newsql

Security

(2020) Set up Red Hat AMQ Streams custom certificates on OpenShift [ADVANCED LEVEL] 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Demonstrates replacing auto-generated certificates with custom enterprise CA certs to implement secured TLS and mTLS configurations inside Strimzi-managed AMQ Streams.

💡 Explore Related: Yaml | Databases | Newsql

Slides

(2020) speakerdeck.com: Apache Kafka with Red Hat AMQ Streams 🌟 🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" An informative slide presentation charting Apache Kafka deployments on OpenShift via Red Hat AMQ Streams. Visualizes operator behaviors and declarative infrastructure patterns.

💡 Explore Related: Yaml | Databases | Newsql

In-Memory Computing

Distributed Compute

Hazelcast

(2021) devops.com: Hazelcast Simplifies Streaming for Extremely Fast Event Processing in IoT, Edge and Cloud Environments 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" This article highlights Hazelcast's integration of stream processing and in-memory data store capabilities. Focuses on low-latency streaming applications for edge environments, high-throughput IoT networks, and real-time analytical portals.

💡 Explore Related: Yaml | Databases | Newsql

IoT Messaging

Mosquitto

OpenShift

(2021) developers.redhat.com: Deploying the Mosquitto MQTT message broker on Red Hat OpenShift, Part 1 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" A developer-focused walkthrough on deploying Eclipse Mosquitto, an open-source MQTT message broker, onto Red Hat OpenShift. Ideal for scaling lightweight telemetry components adjacent to containerized enterprise layers.

💡 Explore Related: Yaml | Databases | Newsql

Protocols

MQTT

(2026) ==mqtt.org== [DOCUMENTATION] 🌟🌟🌟🌟🌟 [DE FACTO STANDARD]

??? info "Technical Deep-Dive" The main specification portal for MQTT, an ISO standard lightweight publish-subscribe network protocol. Widely adopted for edge environments, remote telemetry, and machine-to-machine integrations requiring minimal memory footprint and network load.

💡 Explore Related: Yaml | Databases | Newsql

Message Brokers

ActiveMQ

Artemis

(2026) ==Apache ActiveMQ Artemis broker== 🌟🌟🌟🌟🌟 [DE FACTO STANDARD]

??? info "Technical Deep-Dive" Apache ActiveMQ Artemis provides a non-blocking, multi-protocol, highly performant asynchronous message broker designed for enterprise messaging. It supports advanced queue architectures, JMS/AMQP protocols, and cloud cluster deployments.

💡 Explore Related: Yaml | Databases | Newsql

High Availability

(2017) developers.redhat.com: JDBC Master-Slave Persistence setup with Activemq using Postgresql database [ADVANCED LEVEL] 🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Guides system administrators through building a JDBC Master-Slave active/passive high-availability messaging layer in ActiveMQ backed by a PostgreSQL cluster.

💡 Explore Related: Yaml | Databases | Newsql

Enterprise Middleware

Red Hat AMQ

(2026) Red Hat AMQ 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" Official product home of Red Hat AMQ, an enterprise-grade messaging suite. Delivers highly-available JMS, AMQP, and MQTT engines along with robust Strimzi Kafka integration for complex enterprise data layers.

💡 Explore Related: Yaml | Databases | Newsql

Red Hat AMQ

OpenShift Routing

(2020) developers.redhat.com: Connecting external clients to Red Hat AMQ Broker on Red Hat OpenShift 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" A walkthrough explaining how to securely configure ingress and network protocols to route client applications external to OpenShift directly into containerized AMQ brokers.

💡 Explore Related: Yaml | Databases | Newsql

Message Queues

Alternative Architectures

PostgreSQL

(2021) dagster.io: Postgres: a better message queue than Kafka? 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" An architectural exploration evaluating the use of PostgreSQL as a highly concurrent transactional queue using FOR UPDATE SKIP LOCKED. Suggests a lightweight operational alternative to Apache Kafka for low-to-medium scale applications.

💡 Explore Related: Yaml | Databases | Newsql

Stream Processing

Architectural Patterns

Comparisons

(2025) Kafka Streams and ksqlDB Compared – How to Choose [DOCUMENTATION] 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" An extensive comparison guide from Confluent mapping out when to use the lightweight Kafka Streams Java client library versus ksqlDB database abstraction layers. Analyzes development environments, deployment scales, and infrastructure constraints.

💡 Explore Related: Yaml | Databases | Newsql

Distributed Processing

Apache Flink

(2026) ==Apache Flink== [ADVANCED LEVEL] 🌟🌟🌟🌟🌟 [DE FACTO STANDARD]

??? info "Technical Deep-Dive" A highly performant distributed processing framework designed for stateful stream processing over bounded and unbounded data structures. Features sub-millisecond execution latencies and robust exactly-once transaction guarantees.

💡 Explore Related: Yaml | Databases | Newsql

Kubernetes Native

(2021) flink.apache.org: How to natively deploy Flink on Kubernetes with High-Availability (HA) [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" Provides architectural guidelines on natively deploying Apache Flink on Kubernetes clusters with robust High Availability (HA) configurations. Covers resource scheduling, Zookeeper integrations, and Kubernetes-native active scaling strategies.

💡 Explore Related: Yaml | Databases | Newsql

SQL Engines

ksqlDB

(2026) ksqlDB 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" Official product home of ksqlDB, an event-streaming database tailored to construct stream-processing platforms on top of Apache Kafka. Translates complex Java/Scala stream pipelines into standard SQL definitions.

💡 Explore Related: Yaml | Databases | Newsql

Data Platform

Customer Data

iPaaS

(2026) rudderstack.com iPaaS [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" An enterprise-grade Customer Data Platform (CDP) designed specifically for developers, serving as a specialized iPaaS for telemetry and event streaming.
```
  *   Built to run securely on top of existing cloud data warehouses (Snowflake, BigQuery).
  *   Enables real-time event routing, transformation, and identity resolution with strict privacy controls.
```

💡 Explore Related: Yaml | Databases | Newsql

Data Engineering

Event Streaming

(2018) ==O'Really: Streaming data== [ADVANCED LEVEL] 🌟🌟🌟🌟🌟 [DE FACTO STANDARD]

??? info "Technical Deep-Dive" The definitive conceptual companion to the Apache Beam and Google Cloud Dataflow models of stream processing.
```
  *   Details critical patterns of out-of-order data handling.
  *   Explains event-time vs. processing-time, windowing, and triggering paradigms crucial for building resilient stream processing pipelines.
```

💡 Explore Related: Yaml | Databases | Newsql

Data and Databases

Stream Processing

Streaming Databases

(2020) thenewstack.io: The Rise of the Event Streaming Database 🌟 [ADVANCED LEVEL] 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" An analytical piece exploring the convergence of databases and stream processing systems to create unified event-streaming databases. It addresses how modern architectures require real-time log computation. Grounding tracks its evolution toward modern systems like ksqlDB and Materialize.

💡 Explore Related: Yaml | Databases | Newsql

Event-Driven Architecture

API Management

Schema Governance

(2021) developers.redhat.com: Managing the API life cycle in an event-driven architecture: A practical approach 🌟 [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" Curator Insight: Best practices on governing async APIs and schema definitions inside microservice ecosystems. Live Grounding: Focuses on AsyncAPI specifications, Schema Registry integration, and decoupling publisher/consumer evolution paths to ensure backward compatibility and operational maturity.

💡 Explore Related: Yaml | Databases | Newsql

Apache Kafka

Architecture Evolution

(2021) confluent.io: Apache Kafka Made Simple: A First Glimpse of a Kafka Without ZooKeeper [ADVANCED LEVEL] [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Curator Insight: Announcement and architectural breakdown of Kafka's transition away from ZooKeeper in favor of KRaft (Kafka Raft metadata mode). Live Grounding: Discusses the architectural simplification, metadata scalability improvements, and decreased operational footprint of removing the external ZooKeeper dependency.

💡 Explore Related: Yaml | Databases | Newsql

Fundamentals

(2022) freecodecamp.org: The Apache Kafka Handbook – How to Get Started Using Kafka 🌟 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" Curator Insight: Comprehensive handbook targeting developers getting started with event streams. Live Grounding: Explains underlying storage patterns, consumers, producers, and practical command-line exercises, making it an excellent onboarding guide.
(2021) gentlydownthe.stream [COMMUNITY-TOOL] [GUIDE]

??? info "Technical Deep-Dive" Curator Insight: A highly acclaimed visual, interactive introduction to Apache Kafka and stream processing. Live Grounding: Leverages hand-drawn diagrams and narrative storytelling to explain complex streaming concepts such as replication, consumer offsets, and transaction semantics in an exceptionally digestible manner.

💡 Explore Related: Yaml | Databases | Newsql

Learning Resources

==developer.confluent.io 🌟🌟== 🌟🌟🌟🌟🌟 [DE FACTO STANDARD] [GUIDE]

??? info "Technical Deep-Dive" Curator Insight: Confluent's central education portal containing exhaustive learning paths for Apache Kafka. Live Grounding: Houses premium-grade technical videos, tutorials, sample applications, and comprehensive courses covering stream processing, Kafka streams, and event-driven architecture patterns.
==conduktor.io/kafka: Learn Apache Kafka like never before== 🌟🌟🌟🌟🌟 [DE FACTO STANDARD] [GUIDE]

??? info "Technical Deep-Dive" Curator Insight: Conduktor's comprehensive learning catalog targeting advanced Kafka operations. Live Grounding: Step-by-step guides covering schema evolution, security architectures (SASL/mTLS), custom interceptors, and stream processing with Kafka Streams and ksqlDB.

💡 Explore Related: Yaml | Databases | Newsql

Local Development

(2024) github.com/lensesio/fast-data-dev (Lenses Box) ⭐ 2079 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" Curator Insight: Highly popular, all-in-one Docker image comprising Kafka, ZooKeeper, Schema Registry, and REST Proxy. Live Grounding: Excellent for local developer validation and integration pipelines needing a pre-wired, enterprise-ready playground instance.

💡 Explore Related: Yaml | Databases | Newsql

Performance Optimization

(2021) newrelic.com: Effective Strategies for Kafka Topic Partitioning 🌟 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" Curator Insight: Deep-dive tutorial on optimizing Kafka throughput via smart partitioning schemes. Live Grounding: Analyzes consumer group balancing, message ordering requirements, and custom partitioning algorithms. Provides architectural guidelines for sizing partition counts to balance throughput and rebalance overhead.

💡 Explore Related: Yaml | Databases | Newsql

Performance Testing

(2023) KLoadGen - Kafka + (Avro/Json Schema) Load Generator 🌟 ⭐ 218 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Curator Insight: Purpose-built CLI and tool to simulate heavy load scenarios utilizing Schemas. Live Grounding: Streamlines load testing of schema-validated topics by generating synthetic Avro or JSON messages at target event rates.

💡 Explore Related: Yaml | Databases | Newsql

Tooling and UI

(2026) ==Kafdrop – Kafka Web UI 🌟== ⭐ 6132 🌟🌟🌟🌟🌟 [DE FACTO STANDARD]

??? info "Technical Deep-Dive" Curator Insight: Highly popular, lightweight web UI for monitoring and managing Apache Kafka. Live Grounding: Renders cluster info, brokers, topics, partition offsets, consumer group lag, and allows active JSON/protobuf message payload inspection.
(2026) ==redpanda-data/kowl== 🌟🌟🌟🌟🌟 [DE FACTO STANDARD]

??? info "Technical Deep-Dive" Curator Insight: Excellent web UI (now Redpanda Console) designed for debugging and exploring event streams. Live Grounding: Outstanding user experience presenting topology, schema registry mapping, consumer tracking, and high-performance message search.
(2024) github.com/sauljabin/kaskade ⭐ 1013 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" Curator Insight: Modern Terminal User Interface (TUI) client for Apache Kafka. Live Grounding: Employs an elegant console layout allowing engineering teams to navigate topics, inspect raw schema properties, and watch streaming events dynamically right from the terminal.
(2021) dev.to: Learn how to use Kafkacat – the most versatile Kafka CLI client 🌟 🌟🌟🌟🌟 [ENTERPRISE-STABLE] [GUIDE]

??? info "Technical Deep-Dive" Curator Insight: Guide to Kafkacat (now rebranded as kcat), the developer's favorite Swiss Army knife CLI. Live Grounding: Walks through real-world piping, consuming from dynamic offsets, producing raw file contents, and query configurations using the command line.
(2021) towardsdatascience.com: Overview of UI Tools for Monitoring and Management of Apache Kafka Clusters [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Curator Insight: Comparative review of leading open-source and commercial administration portals for Kafka. Live Grounding: Compares visual management capabilities, schema registration support, and partition offset visualization across tools like AKHQ, Kafdrop, and Lenses.

💡 Explore Related: Yaml | Databases | Newsql

Application Integration

Java Spring Boot

(2023) piotrminkowski.com: Concurrency with Kafka and Spring Boot [COMMUNITY-TOOL] [GUIDE]

??? info "Technical Deep-Dive" Curator Insight: Optimization guide for Spring Boot engineers processing high-throughput event logs. Live Grounding: Examines concurrent message listener configurations, partition distribution strategies, and thread-safe processing to fully maximize JVM resources.

💡 Explore Related: Yaml | Databases | Newsql

Architectural Evaluation

Anti-Patterns

(2022) ==kai-waehner.de: When NOT to use Apache Kafka?== [ADVANCED LEVEL] 🌟🌟🌟🌟🌟 [DE FACTO STANDARD]

??? info "Technical Deep-Dive" Curator Insight: Essential architectural review pointing out Kafka anti-patterns. Live Grounding: Evaluates hard constraints of Kafka, comparing it against traditional message queues (RabbitMQ), data warehouses, and API gateways. Ideal for teams auditing if Kafka is the appropriate fit.

💡 Explore Related: Yaml | Databases | Newsql

Architectural Patterns

Resiliency

(2021) developers.redhat.com: Building resilient event-driven architectures with Apache Kafka [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Curator Insight: A practical guide from Red Hat engineers on building resilient EDA systems using Kafka. Live Grounding: Explains foundational patterns such as retries, dead-letter queues (DLQ), and stateful stream processing to prevent message loss and maintain system availability during downstream failures.

💡 Explore Related: Yaml | Databases | Newsql

Disaster Recovery

High Availability

(2021) tech.ebayinc.com: Resiliency and Disaster Recovery with Kafka [ADVANCED LEVEL] [CASE STUDY] [CASE STUDY] [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Curator Insight: Real-world operational strategies for disaster recovery from eBay's engineering team. Live Grounding: Focuses on multi-region active-passive and active-active Kafka setups, addressing replication lag, mirror maker configurations, and failover automation challenges at extreme scale.

💡 Explore Related: Yaml | Databases | Newsql

Integration Patterns

Transactional Outbox

(2021) developers.redhat.com: The outbox pattern with Apache Kafka and Debezium 🌟 [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" Curator Insight: Deep technical analysis of resolving dual-write problems using CDC and the Outbox Pattern. Live Grounding: Uses Debezium and Apache Kafka to stream database transaction events reliably, ensuring strict eventual consistency across decoupled microservices without 2PC overhead.

💡 Explore Related: Yaml | Databases | Newsql

Multi-Cluster Strategy

Governance

(2022) developers.redhat.com: Which is better: A single Kafka cluster to rule them all, or many? [ADVANCED LEVEL] [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Curator Insight: Comparative design analysis evaluating consolidated vs. decentralized cluster strategies. Live Grounding: Evaluates multi-tenancy, risk blast radius, organizational boundaries, billing allocation, and schema maintenance overhead across both topological models.

💡 Explore Related: Yaml | Databases | Newsql

Performance Optimization

Architectural Patterns

(2022) redhat.com: How we use Apache Kafka to improve event-driven architecture performance 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" Curator Insight: Analysis of leveraging Kafka's performance characteristics within complex corporate environments. Live Grounding: Covers tuning throughput and reducing processing latency in microservices by optimization of batch sizes, compression parameters, and consumer allocation.

💡 Explore Related: Yaml | Databases | Newsql

Broker Operations

(2021) strimzi.io: Optimizing Kafka consumers 🌟 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" Curator Insight: In-depth study on maximizing consumer ingestion performance. Live Grounding: Analyzes consumer fetch sizes, commit mechanisms, partition assignments, and session timeout options to prevent unneeded offset rebalancing in enterprise settings.
(2020) strimzi.io: Optimizing Kafka producers [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Curator Insight: Diagnostic guide to fine-tuning publisher performance. Live Grounding: Explains structural impacts of compression types (lz4, zstd), batch.size configurations, linger.ms, and broker request limits on latency and message pipeline delivery.

💡 Explore Related: Yaml | Databases | Newsql

Scale Operations

Automation

(2021) slack.engineering: Building Self-driving Kafka clusters using open source components [ADVANCED LEVEL] [CASE STUDY] [CASE STUDY] [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Curator Insight: Insightful deep dive from Slack engineers on automating cluster maintenance. Live Grounding: Analyzes their usage of LinkedIn's Cruise Control to automate cluster balancing, partition reassignment, and self-healing under heavy operational scaling pressures.

💡 Explore Related: Yaml | Databases | Newsql

Case Studies

(2022) thenewstack.io: LinkedIn Layered Architecture Minimizes Kafka Scaling Issues [ADVANCED LEVEL] [CASE STUDY] [CASE STUDY] [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Curator Insight: Case study detailing how LinkedIn redesigned their backend streaming pipeline layers. Live Grounding: Explains how a layered model decouples the client APIs from physical clusters, mitigating client-induced connection bloat and simplifying routing management.
(2021) analyticsindiamag.com: How Uber is Leveraging Apache Kafka For More Than 300 Micro Services [ADVANCED LEVEL] [CASE STUDY] [CASE STUDY] [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Curator Insight: High-level overview of Uber's multi-cluster global event bus setup. Live Grounding: Discusses operating trillions of daily messages over 300+ microservices, highlighting custom proxying layers, dead-letter routing structures, and regional backpressure mitigation strategies.

💡 Explore Related: Yaml | Databases | Newsql

Schema Governance

Security

(2021) developers.redhat.com: How to secure Apache Kafka schemas with Red Hat Integration Service Registry 2.0 [ADVANCED LEVEL] [COMMUNITY-TOOL] [GUIDE]

??? info "Technical Deep-Dive" Curator Insight: Security-focused guide for configuring and shielding the API schema registry. Live Grounding: Details RBAC integration, TLS communication, and schema evolution restrictions using Red Hat Integration Service Registry 2.0 (based on Apicurio Registry) to protect message payloads.

💡 Explore Related: Yaml | Databases | Newsql

Security

Data Compliance

(2022) developers.redhat.com: End-to-end field-level encryption for Apache Kafka Connect [ADVANCED LEVEL] [COMMUNITY-TOOL] [GUIDE]

??? info "Technical Deep-Dive" Curator Insight: Security implementation guide on field-level crypto for data pipelines. Live Grounding: Addresses PCI/GDPR requirements by demonstrating Cryptographic SMTs (Simple Message Transforms) within Kafka Connect, ensuring data is encrypted before hitting log segments.

💡 Explore Related: Yaml | Databases | Newsql

Kafka Connect

(2020) developers.redhat.com: Using secrets in Kafka Connect configuration [COMMUNITY-TOOL] [GUIDE]

??? info "Technical Deep-Dive" Curator Insight: Security patterns for avoiding plain-text secrets inside Kafka Connect configurations. Live Grounding: Outlines setting up native SecretProviders (such as directory or file-based providers) inside properties files to map dynamic environment secrets securely.

💡 Explore Related: Yaml | Databases | Newsql

Zero Trust

(2022) engineering.grab.com: Zero trust with Kafka [ADVANCED LEVEL] [CASE STUDY] [CASE STUDY] [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Curator Insight: Production study from Grab's engineering team on implementing zero-trust network boundaries for messaging. Live Grounding: Covers mutual TLS (mTLS) for broker-client transport, fine-grained ACL authorization, and automating credential lifecycle rotation.

💡 Explore Related: Yaml | Databases | Newsql

Observability

Monitoring

Grafana Integration

(2021) grafana.com: Get comprehensive monitoring for your Apache Kafka ecosystem instances quickly with Grafana Cloud [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Curator Insight: Overview of cloud-managed observability templates for Kafka. Live Grounding: Details the installation of preconfigured Grafana dashboards targeting JVM metrics, consumer group lag, broker performance, and broker/under-replicated partition alerts.

💡 Explore Related: Yaml | Databases | Newsql

Performance Metrics

(2021) datadoghq.com: Monitoring Kafka performance metrics [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Curator Insight: The gold-standard diagnostic reference for key Kafka metrics. Live Grounding: Breaks down critical under-replicated partition counts, active controller counts, consumer lag, and I/O network thread usage, offering concrete troubleshooting actions for operational stability.

💡 Explore Related: Yaml | Databases | Newsql

Orchestration and Workflow

BPMN Orchestration

Architectural Patterns

Comparisons

(2019) infoq.com: Event Streams and Workflow Engines – Kafka and Zeebe 🌟 [ADVANCED LEVEL] 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Discusses integrating Apache Kafka's distributed streaming logs with Zeebe's stateful workflow management. Analyzes patterns to maintain reliable, long-running saga transactions across microservices.

💡 Explore Related: Yaml | Databases | Newsql

Zeebe

Camunda

(2026) Zeebe workflow engine 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" Camunda's cloud-native workflow engine, Zeebe. Built specifically to orchestrate distributed microservices, Zeebe parses BPMN 2.0 structures and implements high-throughput, horizontally scalable state machines directly on top of Kubernetes.

💡 Explore Related: Yaml | Databases | Newsql

Data Pipelines

Apache Airflow

Advanced Patterns

(2025) docs.astronomer.io: Dynamically generating DAGs in Airflow [ADVANCED LEVEL] [DOCUMENTATION] 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" Technical documentation illustrating design patterns to construct dynamically generated DAG pipelines in Apache Airflow. Covers generation templates, dynamic parameters, and runtime optimization.

💡 Explore Related: Yaml | Databases | Newsql

Architecture

(2020) towardsdatascience.com: Apache Airflow Architecture 🌟 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" A structured architectural deep dive explaining how Apache Airflow schedules and executes pipelines. Outlines relationships between scheduler loops, state synchronization databases, and executors.

💡 Explore Related: Yaml | Databases | Newsql

Basics

(2021) dev.to: Get started with Apache Airflow 🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" A step-by-step introduction to Apache Airflow design patterns. Covers the core orchestration concepts including DAG definitions, basic Python operators, scheduler parameters, and task execution workflows.

💡 Explore Related: Yaml | Databases | Newsql

Configuration

(2024) airflow.apache.org: Add Owner Links to DAG [DOCUMENTATION] 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" A practical guide showing how to map ownership contacts, support channels, and documentation links to pipeline owners within Airflow dashboards for rapid operations management.

💡 Explore Related: Yaml | Databases | Newsql

Deployments

(2026) ==Apache Airflow official helm chart 🌟== [DOCUMENTATION] 🌟🌟🌟🌟🌟 [DE FACTO STANDARD]

??? info "Technical Deep-Dive" The official Apache Airflow community Helm Chart. Provides pre-configured, modular, and enterprise-hardened templates for deploying schedulers, webservers, worker nodes, and scalable Celery or Kubernetes executors.
(2021) youtube: Airflow Helm Chart : Quick Start For Beginners in 10mins 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" A concise introductory video training illustrating how to deploy Apache Airflow quickly using standard Helm commands. Walks through default configurations, worker provisioning, and web interface verification.

💡 Explore Related: Yaml | Databases | Newsql

Kubernetes Native

(2026) ==airflow.apache.org: KubernetesPodOperator 🌟🌟🌟== [ADVANCED LEVEL] [DOCUMENTATION] 🌟🌟🌟🌟🌟 [DE FACTO STANDARD]

??? info "Technical Deep-Dive" Official engineering reference for the KubernetesPodOperator. Explains how to spin up isolated, dedicated pods within a target Kubernetes namespace dynamically for each individual Airflow DAG task execution.
(2020) towardsdatascience.com: Apache Airflow for containerized data-pipelines 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Explores patterns to deploy containerized data processing networks via Apache Airflow. Focuses on orchestrating individual pipeline stages inside isolated runtime structures on top of cloud infrastructure.

💡 Explore Related: Yaml | Databases | Newsql

Monitoring

(2021) redhat.com: Monitoring Apache Airflow using Prometheus 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" A technical tutorial on integrating Apache Airflow orchestration endpoints with Prometheus. Illustrates how to pull scheduler workloads, active runner pools, and pipeline errors into centralized monitoring systems.

💡 Explore Related: Yaml | Databases | Newsql

Machine Learning Orchestration

Data Platforms

Open Data Hub

(2026) Open Data Hub [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" The main portal for Open Data Hub, an AI/ML platform reference architecture on Red Hat OpenShift. Orchestrates tools like Kubeflow, Spark, and Kafka into a standardized workspace for ML operations.

💡 Explore Related: Yaml | Databases | Newsql

Releases

(2020) Open Data Hub 0.6 brings component updates and Kubeflow architecture 🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" Highlights components within Open Data Hub v0.6 release, evaluating the integrated Kubeflow-aligned architectural updates for containerized machine learning pipelines.

💡 Explore Related: Yaml | Databases | Newsql

Roadmaps

(2020) A development roadmap for Open Data Hub 🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" A development roadmap overview outlining core development visions and tooling tracks designed for the Open Data Hub analytics platform.

💡 Explore Related: Yaml | Databases | Newsql

Python SDKs

Couler

(2023) Couler ⭐ 945 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" An open-source Python SDK focused on orchestrating workloads on Kubernetes. Simplifies constructing declarative workflows across native schedulers like Argo or Tekton using programmatic expressions.

💡 Explore Related: Yaml | Databases | Newsql

Serverless

Knative

Declarative Configuration

(2021) itnext.io: Configuring Kafka Sources and Sinks declaratively in Kubernetes using Knative [COMMUNITY-TOOL] [GUIDE]

??? info "Technical Deep-Dive" Curator Insight: Hands-on exploration of declarative serverless ingestion pipelines on Kubernetes. Live Grounding: Focuses on setting up Knative Eventing Kafka sources and sinks, showcasing how to abstract underlying broker complexities into native Kubernetes custom resource definitions (CRDs).

💡 Explore Related: Yaml | Databases | Newsql

Event-Driven Integration

(2021) piotrminkowski.com: Knative Eventing with Quarkus, Kafka and Camel [ADVANCED LEVEL] [COMMUNITY-TOOL] [GUIDE]

??? info "Technical Deep-Dive" Curator Insight: A step-by-step implementation guide showing serverless integration patterns using Knative, Quarkus, Kafka, and Apache Camel. Live Grounding: Demonstrates how to build efficient, fast-booting containerized JVM microservices that react dynamically to Kafka events routed via Knative's eventing framework.

💡 Explore Related: Yaml | Databases | Newsql

Python Microservices

(2023) rogulski.it: Consume Kafka events with Knative service and FastAPI on kubernetes 🌟 🌟🌟🌟🌟 [ENTERPRISE-STABLE] [GUIDE]

??? info "Technical Deep-Dive" Curator Insight: Practical reference implementation for python-based serverless consumers. Live Grounding: Illustrates setting up Knative Eventing with a KafkaSource trigger to dynamically scale a FastAPI container from zero to process inbound streaming records.

💡 Explore Related: Yaml | Databases | Newsql

Software Architecture

Case Studies

Event Delivery

(2016) engineering.atspotify.com: Spotify’s Event Delivery – The Road to the Cloud (Part I) [ADVANCED LEVEL] [CASE STUDY] 🌟🌟🌟🌟 [CASE STUDY] [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" Part one of Spotify's highly detailed case study documenting their massive shift from on-premise infrastructure to Google Cloud Platform event infrastructure. It details scaling to deliver billions of events daily without data loss. Grounding validates this as an classic, essential read for distributed systems architects.

💡 Explore Related: Yaml | Databases | Newsql

Event-Driven Architecture

Application Design

(2020) stackoverflow.blog: How event-driven architecture solves modern web app problems 🌟 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" A highly clear explanation of how event-driven patterns resolve high concurrency and cross-system latency challenges. Grounding demonstrates its continued popularity as a primary training introduction to decoupled publishing and subscribing paradigms.

💡 Explore Related: Yaml | Databases | Newsql

Infrastructure Design

(2021) redhat.com: Event-driven architecture: Understanding the essential benefits 🌟 🌟🌟🌟 [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" A deep dive by Red Hat explaining how event-driven designs foster agility, decoupling, and high horizontal scalability. It discusses integration paths with Kubernetes, Apache Kafka, and Knative. Grounding shows its essential role for platform engineers planning enterprise application modernized routes.

💡 Explore Related: Yaml | Databases | Newsql

Integration Patterns

iPaaS

(2026) ==Mulesoft== [ADVANCED LEVEL] 🌟🌟🌟🌟🌟 [DE FACTO STANDARD] [LEGACY]

??? info "Technical Deep-Dive" The industry-standard enterprise integration platform (Anypoint Platform) providing high-density API management, ESB capabilities, and iPaaS routing. Mulesoft is highly suited for large-scale legacy modernization and hybrid-cloud orchestration, though it introduces significant runtime complexity and enterprise licensing costs.
(2023) quandarycg.com: Everything You Need To Know About System Integration (And IPaaS) 🌟 🌟🌟🌟🌟 [ENTERPRISE-STABLE] [LEGACY]

??? info "Technical Deep-Dive" A comprehensive architectural primer outlining the foundational concepts of Enterprise Application Integration (EAI) and Integration Platform as a Service (iPaaS).
```
  *   Details the transition from legacy point-to-point connections to modern hub-and-spoke models.
  *   Provides evaluation frameworks for cloud-native middleware alternatives.
```
(2024) blog.hubspot.com: The 22 Best iPaaS Vendors for Any Budget [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" A commercial and technical overview of the top 22 Integration Platform as a Service (iPaaS) vendor solutions. Useful for architectural selection phases to compare enterprise offerings like MuleSoft, Workato, and Zapier across cloud compatibility, throughput limits, and ease of orchestration.

💡 Explore Related: Yaml | Databases | Newsql

Java Ecosystem

Microservices

(2020) adambien.blog - 75th airhacks.tv Questions and Answers: Kafka, JAX-RS, MicroProfile, JSON-B, GSON, JWT, VSC, NetBeans, Java Fullstack [COMMUNITY-TOOL]

??? info "Technical Deep-Dive" A technical Q&A session highlighting architectural strategies using Kafka, JAX-RS, MicroProfile, and JSON-B. This resource offers pragmatic patterns for decoupling enterprise Java applications and migrating monolithic structures to cloud-native, microservice-based runtimes.

💡 Explore Related: Yaml | Databases | Newsql

Microservices

Event-Driven Architecture

(2022) confluent.io: Event-Driven Microservices Architecture (white paper) 🌟 [ADVANCED LEVEL] [DOCUMENTATION] 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" Confluent's authoritative white paper on designing and scaling event-driven microservices around Apache Kafka log segments. It addresses CQRS, Event Sourcing, and transactional schemas. Grounding solidifies this as a core reference for large-scale enterprise data mesh topologies.

💡 Explore Related: Yaml | Databases | Newsql

Monolith Migration

Event-Driven Architecture

(2021) infoq.com: From Monolith to Event-Driven: Finding Seams in Your Future Architecture [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE]

??? info "Technical Deep-Dive" An InfoQ guide detailing how to use Domain-Driven Design (DDD) to isolate domain boundaries and discover 'seams' within large-scale monoliths. Grounding confirms its position as a primary methodology for refactoring to decoupled, event-driven pipelines.

💡 Explore Related: Yaml | Databases | Newsql

148 KiB Raw Blame History Unescape Escape

Message Queue

Application Integration

Cloud Managed Services

Pub-Sub Pattern

Enterprise Integration Patterns

Cloud Managed Services

Event-Driven Systems

Middleware

Serverless Integration

Event Streaming

Enterprise Integration Patterns

Kafka Connectors

Local Development

Containerization

Low-Code Integration

Enterprise Integration Patterns

Microservices

Message Brokers

Clustering

Evaluation Frameworks

Event Streaming

High-Performance Messaging

JMS

Kubernetes Native

Pub-Sub Pattern

Orchestration

Kubernetes Operators

Reference Architecture

Serverless Integration

Cloud Native Architecture

Domain-Driven Design

Messaging Architectures

Event-Driven Systems

Foundations

Patterns

Standards

Foundations

Introductory Patterns

Inter-Service Communication

Performance

Microservices

Change Data Capture CDC

Distributed Transactions

Event Sourcing

Cloud Native Infrastructure

High Availability

Kafka on Kubernetes

Kafka on Kubernetes

Application Integration

Deployments

Guides

Local Development

Kubernetes Strategy

Infrastructure Decisions

Security

Amazon EKS

Serverless Data Platforms

Elastic Kafka

Stateful Workloads

Kafka on Kubernetes

Strimzi Operator

Tooling and UI

Cloud-Native Infrastructure

Event Streaming

GitOps Practices

Infrastructure Operations

Kafka Connect Operators

Kubernetes Operators

Hybrid Cloud Platforms

Anthos Deployments

Modernization Strategy

Infrastructure as Code IaC

Event-Driven Provisioning

Serverless Computing

Knative Eventing

Data Architecture

Data Lakehouse

Iceberg Integration

Data Mesh

148 KiB

Raw Blame History