github/awesome-kubernetes
1
0
Fork 0
mirror of https://github.com/nubenetes/awesome-kubernetes.git synced 2026-07-12 18:00:37 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
94ac0b73a20707fdb08e9cd0743a7c2f2c7ea9aa
awesome-kubernetes/v2-docs/message-queue.md

505 lines
57 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Cloud Based Integration and Messaging. Data Processing and Streaming (aka Data Pipeline). Open Data Hub
!!! tip "Nubenetes V2 Elite Portal"
You are browsing the AI-Curated V2 Elite Edition. Looking for the exhaustive list of references? Check out the [**V1 Historical Archive**](/v1/message-queue/).
!!! info "Architectural Context"
Detailed reference for Cloud Based Integration and Messaging. Data Processing and Streaming (aka Data Pipeline). Open Data Hub in the context of Data & Advanced Analytics.
## Table of Contents
1. [Architecture](#architecture)
- [Data Mesh](#data-mesh)
- [Foundations](#foundations)
- [Hybrid Cloud](#hybrid-cloud)
- [App Modernization](#app-modernization)
- [Google Anthos](#google-anthos)
- [Infrastructure as Code](#infrastructure-as-code)
- [Event-Driven](#event-driven)
- [IoT](#iot)
- [Protocols](#protocols)
- [Microservices Patterns](#microservices-patterns)
- [Decoupling](#decoupling)
- [No-Code CDC](#no-code-cdc)
- [Schema Governance](#schema-governance)
1. [Cloud Native Serverless](#cloud-native-serverless)
- [Knative](#knative)
- [Eventing Integration](#eventing-integration)
1. [Data Engineering](#data-engineering)
- [Change Data Capture](#change-data-capture)
- [Debezium](#debezium)
- [Pipelines](#pipelines)
- [Data Pipelines](#data-pipelines)
- [OpenShift](#openshift)
- [Event Streaming](#event-streaming)
- [Apache Kafka](#apache-kafka)
- [Kubernetes Operators](#kubernetes-operators)
- [Machine Learning](#machine-learning)
- [Podcasts](#podcasts)
- [Schema Registry](#schema-registry)
- [Apicurio](#apicurio)
- [Red Hat Integration](#red-hat-integration)
- [Stream Processing](#stream-processing)
- [Quarkus](#quarkus)
1. [Enterprise Integration](#enterprise-integration)
- [Data Pipelines](#data-pipelines-1)
- [RudderStack](#rudderstack)
- [Customer Data Platform](#customer-data-platform)
1. [Event-Driven Systems](#event-driven-systems)
- [Apache Kafka](#apache-kafka-1)
- [Client Development](#client-development)
- [Kubernetes Deployment](#kubernetes-deployment)
- [Resiliency and Patterns](#resiliency-and-patterns)
- [Security](#security)
- [Case Studies](#case-studies)
- [Scale and Infrastructure](#scale-and-infrastructure)
- [Design Patterns](#design-patterns)
- [Transactional Outbox](#transactional-outbox)
1. [Infrastructure](#infrastructure)
- [Cloud Native Integration](#cloud-native-integration)
- [ActiveMQ Artemis](#activemq-artemis)
- [Networking](#networking)
- [Persistence](#persistence)
- [Enterprise Messaging](#enterprise-messaging)
- [AMQ Streams](#amq-streams)
- [ActiveMQ Artemis](#activemq-artemis-1)
- [Red Hat AMQ](#red-hat-amq)
- [Kubernetes Operators](#kubernetes-operators-1)
- [Koperator](#koperator)
- [Strimzi](#strimzi)
- [Strimzi](#strimzi-1)
- [Configuration](#configuration)
- [Introduction](#introduction)
- [Security](#security-1)
- [Sidecar Patterns](#sidecar-patterns)
- [Data Streaming](#data-streaming)
- [Architectural Patterns](#architectural-patterns)
- [Comparisons](#comparisons)
- [Integrations](#integrations)
- [MongoDB](#mongodb)
- [Performance Tuning](#performance-tuning)
- [Kafka Consumers](#kafka-consumers)
- [Kafka Producers](#kafka-producers)
- [Stream Processing](#stream-processing-1)
- [Architectural Patterns](#architectural-patterns-1)
- [ksqlDB](#ksqldb)
- [Enterprise Integration](#enterprise-integration-1)
- [Camel Quarkus](#camel-quarkus)
- [IoT and Edge Messaging](#iot-and-edge-messaging)
- [Brokers](#brokers)
- [Mosquitto](#mosquitto)
- [Protocols](#protocols-1)
- [MQTT](#mqtt)
- [Kubernetes Native](#kubernetes-native)
- [Camel K](#camel-k)
- [Kamelets](#kamelets)
- [Message Brokers](#message-brokers)
- [Docker](#docker)
- [Stream Processing](#stream-processing-2)
- [Flink](#flink)
- [Kubernetes Deployment](#kubernetes-deployment-1)
- [Stateful Computations](#stateful-computations)
- [Flink](#flink-1)
1. [Integration](#integration)
- [Data Federation](#data-federation)
- [Citizen Integration](#citizen-integration)
- [Enterprise Service Bus](#enterprise-service-bus)
- [Red Hat Fuse](#red-hat-fuse)
- [Low-Code Integration](#low-code-integration)
- [Syndesis](#syndesis)
- [Tutorials](#tutorials)
1. [Microservices](#microservices)
- [Cloud Native](#cloud-native)
- [Event-Driven Architecture](#event-driven-architecture)
- [Decomposition](#decomposition)
- [Event-Driven Architecture](#event-driven-architecture-1)
- [Distributed Transactions](#distributed-transactions)
- [Patterns](#patterns)
- [Domain-Driven Design](#domain-driven-design)
- [Patterns](#patterns-1)
- [Enterprise Integration](#enterprise-integration-2)
- [Event-Driven Architecture](#event-driven-architecture-2)
- [Event-Driven Architecture](#event-driven-architecture-3)
- [Industry Trends](#industry-trends)
- [Kafka](#kafka)
- [Inter-Service Communication](#inter-service-communication)
- [Comparison](#comparison)
- [Kubernetes](#kubernetes)
- [CloudEvents](#cloudevents)
- [Patterns](#patterns-2)
- [Event Sourcing](#event-sourcing)
- [Web Development](#web-development)
- [Event-Driven Architecture](#event-driven-architecture-4)
1. [Orchestration](#orchestration)
- [Workflow Engines](#workflow-engines)
- [Camunda](#camunda)
- [Zeebe](#zeebe)
- [Patterns](#patterns-3)
- [Event-Driven Orchestration](#event-driven-orchestration)
- [Workflows](#workflows)
- [Apache Airflow](#apache-airflow)
- [Dynamic DAGs](#dynamic-dags)
- [Kubernetes Integration](#kubernetes-integration)
1. [Software Engineering](#software-engineering)
- [Backend Development](#backend-development)
- [Java Enterprise](#java-enterprise)
- [MicroProfile](#microprofile)
## Architecture
### Data Mesh
#### Foundations
- **(2020)** [martinfowler.com: Data Mesh Principles and Logical Architecture](https://martinfowler.com/articles/data-mesh-principles.html) <span class='md-tag md-tag--warning'>[N/A CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> <span class='md-tag md-tag--secondary'>[GUIDE]</span> — The seminal architectural document by Zhamak Dehghani outlining Data Mesh principles: decentralized domain ownership, data as a product, self-serve data platforms, and federated computational governance. It details how to break down monolithic data lake infrastructures into domain-driven microservices.
### Hybrid Cloud
#### App Modernization
- **(2021)** [kai-waehner.de: App Modernization and Hybrid Cloud Architectures with Apache Kafka](https://www.kai-waehner.de/blog/2021/03/10/apache-kafka-app-modernization-legacy-hybrid-cloud-native-architecture) <span class='md-tag md-tag--warning'>[N/A CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--secondary'>[GUIDE]</span> <span class='md-tag md-tag--critical'>[LEGACY]</span> — This architectural essay covers application modernization using Apache Kafka as an integration plane. It outlines how to isolate legacy monoliths, construct strangler-fig pattern migrations, and enable clean, continuous cloud-native stream pipelines.
#### Google Anthos
- **(2021)** [confluent.fr: Infrastructure Modernization with Google Anthos and Apache Kafka](https://www.confluent.io/fr-fr/blog/modernize-apps-and-infrastructure-with-anthos-confluent-kafka) <span class='md-tag md-tag--warning'>[N/A CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> <span class='md-tag md-tag--secondary'>[GUIDE]</span> — This architectural study outlines app modernization paradigms using Google Anthos alongside Confluent Kafka. It covers cross-cloud synchronization models, data residency strategies, and how to maintain high availability for hybrid event-driven systems.
### Infrastructure as Code
#### Event-Driven
- **(2021)** [daily.dev: Building a fault-tolerant event-driven architecture with Google Cloud, Pulumi and Debezium](https://daily.dev/blog/building-a-fault-tolerant-event-driven-architecture-with-google-cloud-pulumi-and-debezium) <span class='md-tag md-tag--warning'>[GO CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> <span class='md-tag md-tag--secondary'>[GUIDE]</span> — A practical guide demonstrating how to build a fault-tolerant, event-driven architecture using Google Cloud services, Pulumi as Infrastructure as Code (IaC), and Debezium. It focuses on declarative environment setups for Change Data Capture pipelines, ensuring easy replication and scaling.
### IoT
#### Protocols
- **(2021)** [kai-waehner.de: Apache Kafka and MQTT (Part 1 of 5) – Overview and Comparison](https://www.kai-waehner.de/blog/2021/03/15/apache-kafka-mqtt-sparkplug-iot-blog-series-part-1-of-5-overview-comparison) <span class='md-tag md-tag--warning'>[N/A CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> <span class='md-tag md-tag--secondary'>[GUIDE]</span> — This comparison details the synergy between MQTT and Apache Kafka inside industrial IoT platforms. It outlines how MQTT excels at edge device connectivity, while Kafka functions as the analytical and storage core for downstream services.
### Microservices Patterns
#### Decoupling
- **(2019)** [developers.redhat.com: Decoupling microservices with Apache Camel and Debezium](https://developers.redhat.com/blog/2019/11/19/decoupling-microservices-with-apache-camel-and-debezium) <span class='md-tag md-tag--warning'>[JAVA CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> <span class='md-tag md-tag--secondary'>[GUIDE]</span> — This guide covers the integration of Apache Camel and Debezium to decouple microservice database dependencies. By leveraging Camel's rich Enterprise Integration Patterns (EIP) to consume and route Debezium change event logs, organizations can eliminate dual-write risks and ensure resilient distributed transactions.
#### No-Code CDC
- **(2020)** [developers.redhat.com: Change data capture for microservices without writing any code](https://developers.redhat.com/blog/2020/05/15/change-data-capture-for-microservices-without-writing-any-code) <span class='md-tag md-tag--warning'>[N/A CONTENT]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> <span class='md-tag md-tag--secondary'>[GUIDE]</span> — This article demonstrates how to establish a low-maintenance, zero-code Change Data Capture (CDC) pipeline using Debezium and Kafka Connect. It explains how to decouple microservice databases using declarative configurations, bypassing custom transactional outbox implementation code entirely.
#### Schema Governance
- **(2021)** [redhat.com: Using a schema registry to ensure data consistency between microservices](https://www.redhat.com/en/blog/schema-registry) <span class='md-tag md-tag--warning'>[N/A CONTENT]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> <span class='md-tag md-tag--secondary'>[GUIDE]</span> — A strategic whitepaper discussing the foundational role of schema registries in ensuring runtime compatibility and message consistency across distributed microservice systems. It details forward/backward compatibility models and best practices for automated API version upgrades.
## Cloud Native Serverless
### Knative
#### Eventing Integration
- **(2022)** [rogulski.it: Consume Kafka events with Knative service and FastAPI on kubernetes 🌟](https://rogulski.it/blog/kafka-consumer-knative-fastapi) <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — A hands-on implementation guide showing how to connect Knative serverless triggers with Python-based FastAPI services on Kubernetes. Demonstrates configuring custom event subscriptions to feed incoming Kafka payloads directly to serverless worker containers.
- **(2021)** [piotrminkowski.com: Knative Eventing with Kafka and Quarkus](https://piotrminkowski.com/2021/03/31/knative-eventing-with-kafka-and-quarkus) <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — Walks through the configuration of Knative Eventing infrastructure coupled with Apache Kafka topics using Quarkus-based microservices. It illustrates how to leverage the low memory footprint of GraalVM-compiled Quarkus microservices to handle event-driven workloads.
- **(2021)** [piotrminkowski.com: Knative Eventing with Quarkus, Kafka and Camel](https://piotrminkowski.com/2021/06/14/knative-eventing-with-quarkus-kafka-and-camel) <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — Demonstrates the integration of Apache Camel integrations, Quarkus microservices, and Knative serverless platforms connected via Apache Kafka brokers. Details how to design reactive pipelines that auto-scale based on incoming Kafka topic load.
- **(2021)** [itnext.io: Configuring Kafka Sources and Sinks declaratively in Kubernetes using Knative](https://itnext.io/configuring-kafka-sources-and-sinks-in-kubernetes-271e3757b208) <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — An operational guide focusing on declarative source and sink bindings within Kubernetes using Knative Eventing components. Demonstrates how to write custom resources (CRDs) to map Kafka topics directly to serverless HTTP endpoints without writing broker plumbing.
## Data Engineering
### Change Data Capture
#### Debezium
- **(2026)** [==**Debezium**:==](https://debezium.io) <span class='md-tag md-tag--warning'>[JAVA CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> 🌟🌟🌟🌟🌟 <span class='md-tag md-tag--success'>[DE FACTO STANDARD]</span> — Debezium is the industry-standard distributed platform for log-based Change Data Capture (CDC). Built on top of Apache Kafka Connect, it translates row-level database changes into real-time event streams with minimal database overhead. This ensures strict transactional consistency across decoupled microservice architectures.
#### Pipelines
- **(2020)** [Build a simple cloud-native change data capture pipeline](https://developers.redhat.com/blog/2020/07/02/build-a-simple-cloud-native-change-data-capture-pipeline) <span class='md-tag md-tag--warning'>[YAML CONTENT]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> <span class='md-tag md-tag--secondary'>[GUIDE]</span> — A developer tutorial illustrating how to compile a cloud-native Change Data Capture pipeline. It utilizes Strimzi (AMQ Streams) and Debezium on Kubernetes to propagate database updates instantly into reactive microservice topologies.
### Data Pipelines
#### OpenShift
- **(2021)** [openshift.com: How to Orchestrate Data Pipelines with Applications Deployed on OpenShift](https://www.redhat.com/en/blog/how-to-orchestrate-data-pipelines-with-applications-deployed-on-openshift) <span class='md-tag md-tag--warning'>[N/A CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> <span class='md-tag md-tag--secondary'>[GUIDE]</span> — This guide reviews techniques for deploying and orchestrating resilient data pipelines within Red Hat OpenShift. It outlines utilizing Kubernetes-native orchestration patterns and operators to manage high-throughput ETL/ELT tasks alongside standard microservice applications.
### Event Streaming
#### Apache Kafka
- **(2026)** [==Apache Kafka==](https://kafka.apache.org) <span class='md-tag md-tag--warning'>[JAVA CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> 🌟🌟🌟🌟🌟 <span class='md-tag md-tag--success'>[DE FACTO STANDARD]</span> — Apache Kafka is the de facto industry-standard distributed event streaming platform. Operating on a partitioned, append-only log model, Kafka handles millions of messages per second with fault-tolerant durability, acting as the centralized real-time nervous system for microservices.
#### Kubernetes Operators
- **(2021)** [containerjournal.com: Red Hat Platform Brings Kafka Closer to Kubernetes](https://cloudnativenow.com/topics/cloudnativeplatforms/red-hat-platform-brings-kafka-closer-to-kubernetes) <span class='md-tag md-tag--warning'>[YAML CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> <span class='md-tag md-tag--secondary'>[GUIDE]</span> — This article highlights Red Hat AMQ Streams, based on the Strimzi project, and its approach to managing Kafka on OpenShift/Kubernetes. It details how GitOps and custom resource definitions (CRDs) streamline broker, topic, and user management.
#### Machine Learning
- **(2021)** [confluent.io: How to Build and Deploy Scalable Machine Learning in Production with Apache Kafka](https://www.confluent.io/blog/build-deploy-scalable-machine-learning-production-apache-kafka) <span class='md-tag md-tag--warning'>[JAVA CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> <span class='md-tag md-tag--secondary'>[GUIDE]</span> — This architectural blueprint covers deploying machine learning models in production using Apache Kafka. It outlines real-time stream scoring patterns using Kafka Streams and how to architect reliable event structures for online model evaluation pipelines.
#### Podcasts
- **(2020)** [softwareengineeringdaily.com: Kafka Applications with Tim Berglund (podcast) 🌟](https://softwareengineeringdaily.com/2020/12/16/kafka-applications-with-tim-berglund-repeat) <span class='md-tag md-tag--warning'>[N/A CONTENT]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — A podcast discussing real-world Kafka application patterns with Tim Berglund. The conversation covers key design trade-offs of log-based systems, stream-table dualities, and shifting from synchronous request-response models to event-driven architectures.
### Schema Registry
#### Apicurio
- **(2026)** [****Apicurio** Registry**](https://github.com/apicurio/apicurio-registry) <span class='md-tag md-tag--info'>⭐ 814</span> <svg class="v2-sparkline" width="50" height="15" viewBox="0 0 50 15" style="vertical-align: middle; display: inline-block; margin-left: 6px;" title="Activity Trend"><defs><linearGradient id="spark-grad-8ec6a20b" x1="0" y1="0" x2="1" y2="0"><stop offset="0%" stop-color="rgba(34, 211, 238, 0.2)" /><stop offset="100%" stop-color="var(--md-accent-fg-color)" /></linearGradient></defs><path class="v2-sparkline-path" d="M 0 12 L 10 9 L 20 6 L 30 11 L 40 2 L 50 10" fill="none" stroke="url(#spark-grad-8ec6a20b)" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round" /><circle cx="50" cy="10" r="2" fill="var(--md-accent-fg-color)" /></svg> <span class='md-tag md-tag--warning'>[JAVA CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> 🌟🌟🌟🌟 <span class='md-tag md-tag--success'>[ENTERPRISE-STABLE]</span> — Apicurio Registry is an open-source, high-performance centralized schema registry. It manages API contracts, OpenAPI designs, AsyncAPI definitions, Avro, and Protobuf structures, enforcing real-time payload validations over high-throughput microservice pipelines while offering direct Kubernetes operator integrations.
#### Red Hat Integration
- **(2019)** [Red Hat Integration service registry](https://developers.redhat.com/blog/2019/12/16/getting-started-with-red-hat-integration-service-registry) <span class='md-tag md-tag--warning'>[N/A CONTENT]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> <span class='md-tag md-tag--secondary'>[GUIDE]</span> — An introductory guide to Red Hat's Service Registry, based on the Apicurio Registry upstream. It outlines configuration steps for maintaining schema formats (Avro, Protobuf, JSON) inside enterprise messaging pipelines, ensuring API contract governance in decoupled distributed architectures.
### Stream Processing
#### Quarkus
- **(2020)** [Build a data streaming pipeline using Kafka Streams and Quarkus](https://developers.redhat.com/blog/2020/09/28/build-a-data-streaming-pipeline-using-kafka-streams-and-quarkus) <span class='md-tag md-tag--warning'>[JAVA CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> <span class='md-tag md-tag--secondary'>[GUIDE]</span> — A hands-on implementation guide for building stream-processing applications using Quarkus and the Kafka Streams API. By leveraging GraalVM native compilation, developers can achieve fast startup times and tiny footprints for event-driven microservices.
## Enterprise Integration
### Data Pipelines (1)
#### RudderStack
##### Customer Data Platform
- **(2021)** [rudderstack.com iPaaS](https://www.rudderstack.com) <span class='md-tag md-tag--warning'>[GO CONTENT]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — RudderStack is a warehouse-first, developer-focused Customer Data Platform (CDP) and event-streaming pipeline engine. Architected as a secure, open-source alternative to Segment, it allows enterprises to route customer telemetry directly to cloud data warehouses without compromising privacy or incurring high third-party SaaS fees.
## Event-Driven Systems
### Apache Kafka (1)
#### Client Development
- **(2023)** [piotrminkowski.com: Concurrency with Kafka and Spring Boot](https://piotrminkowski.com/2023/04/30/concurrency-with-kafka-and-spring-boot) <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — Examines advanced concurrency paradigms when developing high-throughput event consumers inside Spring Boot applications. Focuses on tuning consumer threads, partition assignments, off-loop processing patterns, and transactional commit strategies.
#### Kubernetes Deployment
- **(2021)** [itnext.io: Sending Messages to Kafka in Kubernetes](https://itnext.io/sending-messages-to-kafka-cfb5a246f5eb) <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — A configuration-focused guide showing how to reliably publish message event payloads from Kubernetes application workloads to external Kafka clusters. Details setup considerations for internal DNS, headless service mappings, and environment variables.
#### Resiliency and Patterns
- **(2021)** [developers.redhat.com: Building resilient event-driven architectures with Apache Kafka](https://developers.redhat.com/blog/2021/05/05/building-resilient-event-driven-architectures-with-apache-kafka) <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — Explores patterns for engineering highly resilient, decoupled event-driven systems with Apache Kafka. Details the implementation of error-handling loops, dead-letter queues (DLQs), retry topics, and transaction configurations to prevent loss of critical state.
#### Security
- **(2021)** [itnext.io: Securely Decoupling Kubernetes-based Applications on Amazon EKS using Kafka with SASL/SCRAM](https://itnext.io/securely-decoupling-applications-on-amazon-eks-using-kafka-with-sasl-scram-48c340e1ffe9) <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — Details secure-connectivity configurations for microservices running inside Amazon EKS communicating with Apache Kafka clusters. Focuses on setting up SASL/SCRAM authentication, certificate management, and Kubernetes namespace access bounds.
### Case Studies
#### Scale and Infrastructure
- **(2021)** [analyticsindiamag.com: How Uber is Leveraging Apache Kafka For More Than 300 Micro Services](https://analyticsindiamag.com/how-uber-is-leveraging-apache-kafka-for-more-than-300-micro-services) <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--secondary'>[CASE STUDY]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — Details Uber's massive deployment topology where Apache Kafka acts as the backbone communication engine linking over 300 discrete microservices. Examines regional and global replication configurations, dispatch routing, and stream optimization techniques under load.
### Design Patterns
#### Transactional Outbox
- **(2021)** [developers.redhat.com: The outbox pattern with Apache Kafka and Debezium 🌟](https://developers.redhat.com/articles/2021/09/01/outbox-pattern-apache-kafka-and-debezium) <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — An implementation walkthrough of the Transactional Outbox pattern utilizing Debezium and Apache Kafka. This pattern guarantees reliable, dual-write-free message propagation from internal microservice datastores directly to downstream Kafka clusters.
## Infrastructure
### Cloud Native Integration
#### ActiveMQ Artemis
##### Networking
- **(2020)** [developers.redhat.com: Connecting external clients to Red Hat AMQ Broker on Red Hat OpenShift](https://developers.redhat.com/blog/2020/08/26/connecting-external-clients-to-red-hat-amq-broker-on-red-hat-openshift) <span class='md-tag md-tag--warning'>[YAML CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> 🌟🌟🌟 <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — Provides step-by-step methods to expose internal ActiveMQ brokers deployed inside OpenShift cluster structures to external clients. It explains how to deploy route maps, configure node ports, and terminate TLS certificates to secure outside access.
##### Persistence
- **(2017)** [developers.redhat.com: JDBC Master-Slave Persistence setup with Activemq using Postgresql database](https://developers.redhat.com/blog/2017/10/05/jdbc-master-slave-persistence-setup-activemq-using-postgresql-database) <span class='md-tag md-tag--warning'>[YAML CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> 🌟🌟🌟 <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — An older but architecturally important guide addressing the configuration of JDBC Master-Slave persistence topologies in ActiveMQ using a PostgreSQL database. It outlines database locking strategies to coordinate high-availability failover configurations.
#### Enterprise Messaging
##### AMQ Streams
- **(2019)** [Understanding Red Hat AMQ Streams components for OpenShift and Kubernetes 🌟](https://developers.redhat.com/blog/2019/12/04/understanding-red-hat-amq-streams-components-for-openshift-and-kubernetes-part-1) <span class='md-tag md-tag--warning'>[N/A CONTENT]</span> 🌟🌟🌟 <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — Explains the underlying architectural parts of AMQ Streams (Red Hat's enterprise packaging of the Strimzi operator). It walks engineers through utilizing operator mechanisms to deploy highly-secure, production-ready Kafka instances inside OpenShift environments.
##### ActiveMQ Artemis (1)
- **(2026)** [**Apache ActiveMQ Artemis broker**](https://artemis.apache.org/components/artemis) <span class='md-tag md-tag--warning'>[JAVA CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> 🌟🌟🌟🌟 <span class='md-tag md-tag--success'>[ENTERPRISE-STABLE]</span> — Apache ActiveMQ Artemis is the next-generation messaging broker featuring a high-performance, asynchronous non-blocking execution model. Supporting AMQP, MQTT, STOMP, and JMS, it represents the primary engine under Red Hat AMQ deployments.
##### Red Hat AMQ
- **(2026)** [**Red Hat AMQ**](https://www.redhat.com/en/technologies/jboss-middleware/amq) <span class='md-tag md-tag--warning'>[JAVA CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> 🌟🌟🌟🌟 <span class='md-tag md-tag--success'>[ENTERPRISE-STABLE]</span> <span class='md-tag md-tag--critical'>[LEGACY]</span> — Red Hat AMQ is an enterprise message-brokering platform supporting traditional queue protocols (AMQP, JMS, MQTT) and high-throughput streaming patterns via integrated Kafka streams. It forms the core transactional backbone for legacy-to-modern hybrid cloud transformations.
#### Kubernetes Operators (1)
##### Koperator
- **(2024)** [**Banzai Kafka Operator**](https://github.com/banzaicloud/koperator) <span class='md-tag md-tag--info'>⭐ 790</span> <svg class="v2-sparkline" width="50" height="15" viewBox="0 0 50 15" style="vertical-align: middle; display: inline-block; margin-left: 6px;" title="Activity Trend"><defs><linearGradient id="spark-grad-f12e5be8" x1="0" y1="0" x2="1" y2="0"><stop offset="0%" stop-color="rgba(34, 211, 238, 0.2)" /><stop offset="100%" stop-color="var(--md-accent-fg-color)" /></linearGradient></defs><path class="v2-sparkline-path" d="M 0 6 L 10 11 L 20 5 L 30 7 L 40 7 L 50 3" fill="none" stroke="url(#spark-grad-f12e5be8)" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round" /><circle cx="50" cy="3" r="2" fill="var(--md-accent-fg-color)" /></svg> <span class='md-tag md-tag--warning'>[GO CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> 🌟🌟🌟🌟 <span class='md-tag md-tag--success'>[ENTERPRISE-STABLE]</span> — Originally engineered by Banzai Cloud, Koperator is a highly automated operator framework designed to manage Kafka on Kubernetes with Cruise Control integrations. While mostly superseded by Strimzi, its historical innovations in granular scaling and fine-grained rebalancing influenced modern stateful Kubernetes abstractions.
##### Strimzi
- **(2026)** [==strimzi.io==](https://strimzi.io) <span class='md-tag md-tag--warning'>[JAVA CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> 🌟🌟🌟🌟🌟 <span class='md-tag md-tag--success'>[DE FACTO STANDARD]</span> — Strimzi represents the premier CNCF project for deploying and managing Apache Kafka clusters natively inside Kubernetes. By leveraging the Operator pattern, Strimzi automates node scaling, security certificate provisioning, cluster balancing, and configuration drift-correction, making it the industry blueprint for stateful distributed streaming systems.
#### Strimzi (1)
##### Configuration
- **(2021)** [strimzi/kafka-kubernetes-config-provider: Kubernetes Configuration Provider' for Apache Kafka](https://github.com/strimzi/kafka-kubernetes-config-provider) <span class='md-tag md-tag--info'>⭐ 30</span> <svg class="v2-sparkline" width="50" height="15" viewBox="0 0 50 15" style="vertical-align: middle; display: inline-block; margin-left: 6px;" title="Activity Trend"><defs><linearGradient id="spark-grad-09374b4f" x1="0" y1="0" x2="1" y2="0"><stop offset="0%" stop-color="rgba(34, 211, 238, 0.2)" /><stop offset="100%" stop-color="var(--md-accent-fg-color)" /></linearGradient></defs><path class="v2-sparkline-path" d="M 0 6 L 10 7 L 20 4 L 30 13 L 40 6 L 50 10" fill="none" stroke="url(#spark-grad-09374b4f)" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round" /><circle cx="50" cy="10" r="2" fill="var(--md-accent-fg-color)" /></svg> <span class='md-tag md-tag--warning'>[JAVA CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> 🌟🌟 <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — A specialized provider class allowing Kafka applications to read operational properties directly from Kubernetes Secrets and ConfigMaps. This architectural utility simplifies TLS certificate mount mappings and broker credential provisioning, eliminating redundant file sync code in application containers.
##### Introduction
- **(2020)** [developers.redhat.com: Introduction to Strimzi: Apache Kafka on Kubernetes (KubeCon Europe 2020) 🌟](https://developers.redhat.com/blog/2020/08/14/introduction-to-strimzi-apache-kafka-on-kubernetes-kubecon-europe-2020) <span class='md-tag md-tag--warning'>[N/A CONTENT]</span> 🌟🌟🌟 <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — A foundational KubeCon session overview exploring Strimzi's architectural foundations. The piece highlights Custom Resource Definitions (CRDs) for brokers, topics, and users, highlighting how the Operator pattern abstracts the inherent friction of stateful cluster administration on Kubernetes.
##### Security (1)
- **(2021)** [strimzi.io: Using Kubernetes Configuration Provider to load data from Secrets and Config Maps](https://strimzi.io/blog/2021/07/22/using-kubernetes-config-provider-to-load-data-from-secrets-and-config-maps) <span class='md-tag md-tag--warning'>[YAML CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> 🌟🌟🌟 <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — A tutorial guiding developers through the integration of the Kubernetes Configuration Provider inside running client workloads. It provides actual deployment manifests showcasing how dynamic secret rotation is accomplished without requiring a full cluster restart.
- **(2020)** [developers.redhat.com: how easy to deploy and configure a Kafka Connect on Kubernetes through strimziio operator and use secrets](https://developers.redhat.com/blog/2020/02/14/using-secrets-in-apache-kafka-connect-configuration) <span class='md-tag md-tag--warning'>[YAML CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> 🌟🌟🌟 <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — A deep-dive technical guide addressing secret configuration in Apache Kafka Connect deployments on Kubernetes. It explains how to declare properties and read externalized secrets through configuration providers, avoiding cleartext passwords inside GitOps repositories.
- **(2020)** [strimzi.io: Using Open Policy Agent with Strimzi and Apache Kafka](https://strimzi.io/blog/2020/08/05/using-open-policy-agent-with-strimzi-and-apache-kafka) <span class='md-tag md-tag--warning'>[REGO CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> 🌟🌟🌟 <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — A technical implementation guide showing how to externalize Kafka authorization rules using Open Policy Agent (OPA). By combining Strimzi with Rego policies, platform engineers can replace static ACL models with dynamic, declarative, attributes-based access controls.
##### Sidecar Patterns
- **(2021)** [strimzi.io: Using HTTP Bridge as a Kubernetes sidecar](https://strimzi.io/blog/2021/08/18/using-http-bridge-as-a-kubernetes-sidecar) <span class='md-tag md-tag--warning'>[YAML CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> 🌟🌟🌟 <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — Architectural breakdown of deploying the Strimzi HTTP Bridge as a sidecar alongside non-Java microservices. This pattern allows lightweight containers to interact with Kafka endpoints via standard HTTP REST APIs, avoiding massive native SDK dependencies.
### Data Streaming
#### Architectural Patterns
##### Comparisons
- **(2021)** [dagster.io: Postgres: a better message queue than Kafka?](https://dagster.io/blog/skip-kafka-use-postgres-message-queue) <span class='md-tag md-tag--warning'>[SQL CONTENT]</span> 🌟🌟🌟 <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — A detailed, practical analysis investigating whether using Postgres with 'SKIP LOCKED' mechanisms is a more appropriate and less complex message-queue architecture than deploying heavy systems like Kafka. It provides explicit guidelines for making decisions based on data scale and operational overhead.
#### Integrations
##### MongoDB
- **(2021)** [mongodb.com: DaaS with MongoDB and Confluent](https://www.mongodb.com/company/blog/technical/daa-s-with-mongo-db-and-confluent) <span class='md-tag md-tag--warning'>[N/A CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> 🌟🌟🌟 <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — Details the construction of a low-latency Data-as-a-Service (DaaS) layer combining MongoDB's document-based storage engine with Confluent's real-time messaging pipeline. This architecture provides microservices with immediate, synchronized access to transactional and analytics database endpoints.
#### Performance Tuning
##### Kafka Consumers
- **(2021)** [strimzi.io: Optimizing Kafka consumers 🌟](https://strimzi.io/blog/2021/01/07/consumer-tuning) <span class='md-tag md-tag--warning'>[N/A CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--primary'>[DOCUMENTATION]</span> 🌟🌟🌟 <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — A comprehensive playbook for tuning Kafka consumers to prevent head-of-line blocking and partition rebalance storms in high-throughput clusters. It details proper session timeout windows, fetch size parameters, and threading behaviors crucial for maintaining consistent low-latency ingestion pipelines.
##### Kafka Producers
- **(2020)** [strimzi.io: Optimizing Kafka producers 🌟](https://strimzi.io/blog/2020/10/15/producer-tuning) <span class='md-tag md-tag--warning'>[N/A CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--primary'>[DOCUMENTATION]</span> 🌟🌟🌟 <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — An analytical guide focused on hardening Kafka message producers against data loss while maintaining performance levels. This resource covers client-side retry architectures, delivery timeouts, and buffer allocation metrics to ensure reliable transport in Kubernetes networks.
#### Stream Processing (1)
##### Architectural Patterns (1)
- **(2021)** [Kafka Streams and ksqlDB Compared – How to Choose](https://docs.confluent.io/platform/current/streams/overview.html) <span class='md-tag md-tag--warning'>[N/A CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--primary'>[DOCUMENTATION]</span> 🌟🌟🌟 <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — A comparative guide contrasting the application patterns of using ksqlDB with writing custom Java code via the Kafka Streams library. It provides engineers with logical decision paths based on pipeline scale, deployment models, and development team specializations.
##### ksqlDB
- **(2026)** [**ksqlDB**](https://www.confluent.io/product/ksqldb) <span class='md-tag md-tag--warning'>[JAVA CONTENT]</span> 🌟🌟🌟🌟 <span class='md-tag md-tag--success'>[ENTERPRISE-STABLE]</span> — An event-streaming database engineered specifically to build stream-processing applications on top of Apache Kafka. By translating familiar SQL queries into stateful Kafka Streams topologies, ksqlDB enables microservices to construct real-time materialized views and joins with minimal code.
### Enterprise Integration (1)
#### Camel Quarkus
- **(2021)** [developers.redhat.com: Integrating systems with Apache Camel and Quarkus on Red Hat OpenShift](https://developers.redhat.com/articles/2021/05/17/integrating-systems-apache-camel-and-quarkus-red-hat-openshift) <span class='md-tag md-tag--warning'>[JAVA CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — An enterprise integration guide illustrating how running Apache Camel on Quarkus on Red Hat OpenShift delivers sub-second cold starts and minimal memory consumption, ideal for resource-constrained Kubernetes environments.
### IoT and Edge Messaging
#### Brokers
##### Mosquitto
- **(2021)** [developers.redhat.com: Deploying the Mosquitto MQTT message broker on Red Hat OpenShift, Part 1](https://developers.redhat.com/blog/2021/04/16/deploying-the-mosquitto-mqtt-message-broker-on-red-hat-openshift-part-1) <span class='md-tag md-tag--warning'>[YAML CONTENT]</span> 🌟🌟🌟 <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — A multi-part deployment sequence demonstrating how to launch and secure the Eclipse Mosquitto MQTT broker inside Red Hat OpenShift. This setup is highly applicable for hybrid architectures where edge devices stream data into Kubernetes-hosted processing grids.
#### Protocols (1)
##### MQTT
- **(2026)** [**mqtt.org**](https://mqtt.org) <span class='md-tag md-tag--warning'>[N/A CONTENT]</span> 🌟🌟🌟🌟 <span class='md-tag md-tag--success'>[ENTERPRISE-STABLE]</span> — The home of MQTT, the industry-standard lightweight publish-subscribe transport protocol designed specifically for extreme remote locations and low-bandwidth channels. It constitutes the primary communication format for edge nodes and mobile endpoints bridging into central event-streaming backbones.
### Kubernetes Native
#### Camel K
- **(2023)** [Apache Camel K](https://camel.apache.org/camel-k/2.10.x) <span class='md-tag md-tag--warning'>[GO CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--primary'>[DOCUMENTATION]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — The homepage for Apache Camel K, a lightweight integration framework optimized for Kubernetes. Built on Knative, Camel K runs integration code natively, using custom operators to automate building and scaling processes.
- **(2021)** [thenewstack.io: Camel K Brings Apache Camel to Kubernetes for Event-Driven Architectures](https://thenewstack.io/camel-k-brings-apache-camel-to-kubernetes-for-event-driven-architectures) <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — This article documents the architectural impact of Camel K, explaining how it extends Kubernetes to support enterprise integration workflows. It highlights its runtime environment and integration with Knative and serverless architectures.
- **(2020)** [developers.redhat.com: Six reasons to love Camel K](https://developers.redhat.com/blog/2020/05/12/six-reasons-to-love-camel-k) <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — This Red Hat article highlights six benefits of adopting Camel K. It details its low memory footprints, sub-second startup times, Serverless integration paths, and how it uses Kamelets to connect external APIs.
#### Kamelets
- **(2021)** [developers.redhat.com: Design event-driven integrations with Kamelets and Camel K](https://developers.redhat.com/blog/2021/04/02/design-event-driven-integrations-with-kamelets-and-camel-k) <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — This article demonstrates how to build integration paths utilizing Kamelets (Camel Route Snippets) and Camel K. It explains how Kamelets allow developers to configure complex system integrations via standard Kubernetes Custom Resources (CRDs).
### Message Brokers
#### Docker
- **(2021)** [geshan.com.np: How to use RabbitMQ and Node.js with Docker and Docker-compose](https://geshan.com.np/blog/2021/07/rabbitmq-docker-nodejs) <span class='md-tag md-tag--warning'>[JAVASCRIPT CONTENT]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — A practical tutorial for orchestrating RabbitMQ alongside Node.js microservices using Docker Compose. It outlines steps to build local development environments and configure basic exchange-bound AMQP workflows.
### Stream Processing (2)
#### Flink
##### Kubernetes Deployment (1)
- **(2021)** [**flink.apache.org: How to natively deploy Flink on Kubernetes with High-Availability (HA)**](https://flink.apache.org/2021/02/10/native-k8s-with-ha.html) <span class='md-tag md-tag--warning'>[YAML CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--primary'>[DOCUMENTATION]</span> 🌟🌟🌟🌟 <span class='md-tag md-tag--success'>[ENTERPRISE-STABLE]</span> — A detailed technical guide explaining how to deploy stateful Flink jobs natively on Kubernetes with High Availability (HA). It details integration patterns using ZooKeeper or Kubernetes API endpoints to coordinate active leader election and prevent split-brain states.
#### Stateful Computations
##### Flink (1)
- **(2026)** [==Apache Flink==](https://flink.apache.org) <span class='md-tag md-tag--warning'>[JAVA CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> 🌟🌟🌟🌟🌟 <span class='md-tag md-tag--success'>[DE FACTO STANDARD]</span> — Apache Flink is the industry-standard distributed framework designed for stateful stream computations on real-time event logs. Offering sub-millisecond execution times and robust exactly-once state processing, Flink handles large-scale stream processing workloads with high efficiency.
## Integration
### Data Federation
#### Citizen Integration
- **(2020)** [Event streaming and data federation: A citizen integrator’s story](https://developers.redhat.com/blog/2020/06/12/event-streaming-and-data-federation-a-citizen-integrators-story) <span class='md-tag md-tag--warning'>[N/A CONTENT]</span> <span class='md-tag md-tag--secondary'>[CASE STUDY]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — A narrative-style case study exploring how visual integration tools and event-streaming pipelines enable citizen integrators to aggregate disparate database models. It maps real-world patterns for democratization of data engineering and integration tasks across departments.
### Enterprise Service Bus
#### Red Hat Fuse
- **(2026)** [**Red Hat Fuse**](https://www.redhat.com/en/products/application-foundations) <span class='md-tag md-tag--warning'>[JAVA CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--critical'>[LEGACY]</span> — Historically a distributed integration platform based on Apache Camel, Red Hat Fuse has transitioned into the Red Hat Application Foundations suite. It provides enterprise-level connectivity for hybrid clouds, routing APIs, and legacy applications. Contemporary architectures deploy Camel Extensions for Quarkus to achieve high performance on Kubernetes.
### Low-Code Integration
#### Syndesis
- **(2026)** [**Syndesis** open source integration platform](https://syndesis.io) <span class='md-tag md-tag--warning'>[JAVA CONTENT]</span> <span class='md-tag md-tag--critical'>[LEGACY]</span> — Syndesis was an open-source, cloud-native low-code integration platform built natively for Kubernetes. Though currently archived, it historically facilitated rapid microservice orchestration and API visual design with prebuilt connectors. Its architectural concepts paved the way for modern cloud-native iPaaS systems.
#### Tutorials
- **(2020)** [developers.redhat.com: Low-code microservices orchestration with Syndesis](https://developers.redhat.com/blog/2020/03/25/low-code-microservices-orchestration-with-syndesis) <span class='md-tag md-tag--warning'>[N/A CONTENT]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> <span class='md-tag md-tag--secondary'>[GUIDE]</span> — This architectural guide demonstrates how to construct and orchestrate low-code microservices integrations using the Syndesis platform on OpenShift. It highlights developer productivity pathways, showcasing visual data mapping and cloud-native connector deployments that bypass traditional integration boilerplate.
## Microservices
### Cloud Native
#### Event-Driven Architecture
- **(2023)** [ibm.com: Event-driven cloud-native applications (microservices)](https://www.ibm.com/think/topics/cloud-native) <span class='md-tag md-tag--primary'>[DOCUMENTATION]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — This IBM resource details how event-driven applications scale natively inside Kubernetes clusters. It focuses on isolating boundaries and implementing lightweight message-driven scaling paths for complex enterprise systems.
### Decomposition
#### Event-Driven Architecture (1)
- **(2020)** [infoq.com: From Monolith to Event-Driven: Finding Seams in Your Future Architecture](https://www.infoq.com/articles/event-driven-finding-seams) <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — This article outlines methodologies for finding boundaries within tight-knit monolithic structures to facilitate migration. It contrasts synchronous runtime calls with asynchronous eventing boundaries, demonstrating how to isolate transactional domains using Domain-Driven Design (DDD) aggregates.
### Distributed Transactions
#### Patterns
- **(2021)** [developers.redhat.com: Distributed transaction patterns for microservices compared](https://developers.redhat.com/articles/2021/09/21/distributed-transaction-patterns-microservices-compared) <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — This article analyzes patterns for distributed transaction management in decoupled architectures. It contrasts two-phase commit (2PC) limitations with the Saga pattern (both orchestrated and choreographed styles), providing a practical guide on maintaining transactional state.
### Domain-Driven Design
#### Patterns (1)
- **(2019)** [verraes.net: DDD and Messaging Architectures 🌟](https://verraes.net/2019/05/ddd-msg-arch) <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — This resource maps Domain-Driven Design (DDD) concepts onto messaging architectures. It explores how to structure messaging channels and aggregate roots to avoid distributed monolith structures and optimize data routing.
### Enterprise Integration (2)
#### Event-Driven Architecture (2)
- **(2021)** [redhat.com: Event-driven architecture: Understanding the essential benefits 🌟](https://www.redhat.com/en/blog/event-driven-architecture-essentials) <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — This Red Hat architectural summary documents the enterprise benefits of Event-Driven Architectures (EDA). It focuses on asynchronous communication, decoupling execution contexts, and enabling real-time analytics integrations.
### Event-Driven Architecture (3)
#### Industry Trends
- **(2021)** [thenewstack.io: The Rise of Event-Driven Architecture](https://thenewstack.io/the-rise-of-event-driven-architecture) <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — This article documents the architectural factors that made event-driven integration standard in modern cloud-native enterprises. It explains how synchronous HTTP calls cause cascade failures and presents asynchronous patterns as the default design choice for complex topologies.
#### Kafka
- **(2021)** [confluent.io: Event-Driven Microservices Architecture (white paper) 🌟](https://www.confluent.io/resources/white-paper/event-driven-microservices) <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--secondary'>[CASE STUDY]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — A comprehensive Confluent white paper establishing design principles for event-driven microservices. It highlights Apache Kafka as an immutable commit log, detailing exact execution models for Event Sourcing and Command Query Responsibility Segregation (CQRS).
### Inter-Service Communication
#### Comparison
- **(2021)** [particular.net: RPC vs. Messaging – which is faster?](https://particular.net/blog/rpc-vs-messaging-which-is-faster) <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — This performance analysis evaluates the trade-offs of RPC-style communication patterns against broker-mediated messaging. It details the impact of synchronous blocking calls on microservice performance and explains how message queues improve reliability.
### Kubernetes
#### CloudEvents
- **(2022)** [salaboy.com: Event-Driven applications with CloudEvents on Kubernetes](https://www.salaboy.com/2022/01/29/event-driven-applications-with-cloudevents-on-kubernetes) <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — This article explores deploying CloudEvents inside Kubernetes ecosystems to build standardized event schemas. It shows how the CloudEvents standard, combined with serverless tools like Knative, drives event-driven microservice integration.
### Patterns (2)
#### Event Sourcing
- **(2021)** [codeopinion.com: Event Sourcing vs Event Driven Architecture](https://codeopinion.com/event-sourcing-vs-event-driven-architecture) <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — This guide highlights the architectural differences between Event Sourcing (rebuilding state via a series of domain events) and Event-Driven Architecture (routing state transitions between services). It prevents common microservice anti-patterns.
- **(2020)** [blog.bitsrc.io: Why Microservices Should use Event Sourcing 🌟](https://blog.bitsrc.io/why-microservices-should-use-event-sourcing-9755a54ebfb4) <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — An in-depth analysis advocating for Event Sourcing inside microservice frameworks. It details how recording every event state change enables historical auditability and decouples read queries from primary transaction engines via CQRS.
### Web Development
#### Event-Driven Architecture (4)
- **(2020)** [stackoverflow.blog: How event-driven architecture solves modern web app problems 🌟](https://stackoverflow.blog/2020/03/16/how-event-driven-architecture-solves-modern-web-app-problems) <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — This study from StackOverflow contrasts traditional request-response architectures with asynchronous event structures. It explains how shifting to non-blocking patterns resolves high-concurrency web app bottlenecks, increasing system fault tolerance.
## Orchestration
### Workflow Engines
#### Camunda
##### Zeebe
- **(2026)** [**Zeebe workflow engine**](https://camunda.com/platform/zeebe) <span class='md-tag md-tag--warning'>[JAVA CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> 🌟🌟🌟🌟 <span class='md-tag md-tag--success'>[ENTERPRISE-STABLE]</span> — Zeebe is Camunda's highly available, horizontally scalable workflow orchestration engine designed specifically for microservices architectures. Relying on event-sourced execution loops, Zeebe manages complex BPMN process flows across thousands of servers with built-in partition tolerance.
#### Patterns (3)
##### Event-Driven Orchestration
- **(2019)** [infoq.com: Event Streams and Workflow Engines – Kafka and Zeebe 🌟](https://www.infoq.com/news/2019/05/kafka-zeebe-streams-workflows) <span class='md-tag md-tag--warning'>[N/A CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> 🌟🌟🌟 <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — An analytical study contrasting event-driven choreography with workflow orchestration. It shows how combining Kafka's decoupled event model with Zeebe's stateful execution engine resolves typical observability and error-handling bottlenecks in microservice topologies.
### Workflows
#### Apache Airflow
##### Dynamic DAGs
- **(2026)** [**docs.astronomer.io: Dynamically generating DAGs in Airflow**](https://www.astronomer.io/docs/learn/dynamically-generating-dags) <span class='md-tag md-tag--warning'>[PYTHON CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--primary'>[DOCUMENTATION]</span> 🌟🌟🌟🌟 <span class='md-tag md-tag--success'>[ENTERPRISE-STABLE]</span> — A deep dive on building dynamically-generated DAGs in Airflow. This blueprint showcases how to dynamically compile hundreds of different workflows from external JSON or YAML configurations, dramatically reducing redundant code in large-scale platform teams.
##### Kubernetes Integration
- **(2026)** [==airflow.apache.org: KubernetesPodOperator 🌟🌟🌟==](https://airflow.apache.org/docs/apache-airflow-providers-cncf-kubernetes/stable/operators.html) <span class='md-tag md-tag--warning'>[PYTHON CONTENT]</span> <span class='md-tag md-tag--critical'>[ADVANCED LEVEL]</span> <span class='md-tag md-tag--primary'>[DOCUMENTATION]</span> 🌟🌟🌟🌟🌟 <span class='md-tag md-tag--success'>[DE FACTO STANDARD]</span> — The KubernetesPodOperator allows Airflow tasks to execute dynamically inside isolated, single-use Kubernetes Pods. By isolating runtime dependencies, it lets developers execute pipeline tasks of any language or version without changing parent worker system environments.
## Software Engineering
### Backend Development
#### Java Enterprise
##### MicroProfile
- **(2020)** [adambien.blog - 75th **airhacks.tv** Questions and Answers: Kafka, JAX-RS, MicroProfile, JSON-B, GSON, JWT, VSC, NetBeans, Java Fullstack](https://adambien.blog/roller/abien/entry/kafka_jax_rs_microprofile_json) <span class='md-tag md-tag--warning'>[JAVA CONTENT]</span> <span class='md-tag md-tag--info'>[COMMUNITY-TOOL]</span> — An edition of Adam Bien's 'airhacks.tv' Q&A series focusing on modern enterprise Java backend architectures. Key engineering discussions cover reactive Kafka messaging integration using MicroProfile, JAX-RS REST endpoint implementations, and a comparison of JSON serialization libraries (JSON-B vs GSON).
---
💡 **Explore Related:** [Yaml](./yaml.md) | [Databases](./databases.md) | [Crunchydata](./crunchydata.md)
🔗 **See Also:** [About](./about.md) | [Postman](./postman.md)
Reference in New Issue View Git Blame Copy Permalink