Files
awesome-kubernetes/v2-docs/chaos-engineering.md

24 KiB
Raw Blame History

Chaos Engineering

!!! info "Architectural Context" Detailed reference for Chaos Engineering in the context of Platform & Site Reliability.

Table of Contents

  1. Architectural Foundations
  1. Curated Playbooks
  1. Resilience

Architectural Foundations

Kubernetes Tools

General Reference

Curated Playbooks

Awesome Lists

  • (2023) ==Awesome Chaos Engineering== 6589 🌟🌟🌟🌟🌟 [DE FACTO STANDARD] — The premier curated directory of resources, tools, and papers dedicated to the practice of Chaos Engineering. It indexes tools for simulating network latency, injecting resource stress, and terminating instances across various platforms, with a strong focus on cloud-native environments. This is a must-have reference for engineering teams building self-healing, fault-tolerant distributed systems.

Resilience

Chaos Engineering (1)

Cloud Architecture

  • (2021) aws.amazon.com: Verify the resilience of your workloads using Chaos Engineering [ADVANCED LEVEL] [COMMUNITY-TOOL] — An enterprise guide from AWS demonstrating resilience validation of distributed workloads using chaos principles. Maps failure injection patterns to the AWS Well-Architected Framework, emphasizing targeted, automated disruption sequences that verify high-availability topologies and recovery mechanisms.
  • (2021) Chaos engineering on Amazon EKS using AWS Fault Injection Simulator [ADVANCED LEVEL] [COMMUNITY-TOOL] — A technical walkthrough demonstrating how to orchestrate chaos experiments on Amazon EKS using AWS Fault Injection Simulator (FIS). Highlights configuring managed cluster actions to trigger node terminations, API failures, and container termination within isolated namespaces.
  • (2021) aws.amazon.com: Chaos Engineering with LitmusChaos on Amazon EKS [ADVANCED LEVEL] [COMMUNITY-TOOL] — Provides an architectural blueprint for integrating LitmusChaos with Amazon EKS. Walks through installing custom resources, setting up experiment workflows for container and node disruptions, and verifying application resilience with AWS native CloudWatch metrics.
  • (2021) Azure Chaos Studio [COMMUNITY-TOOL] — Provides an overview of Azure Chaos Studio, Microsoft's managed chaos orchestration platform. Explains how to configure fault injection pipelines against virtual machines, AKS clusters, and key-value stores directly inside the Azure portal.
  • (2021) aws.amazon.com: Automating and Scaling Chaos Engineering using AWS Fault Injection Simulator [ADVANCED LEVEL] [COMMUNITY-TOOL] — Details structural blueprints for scaling and automating chaos engineering across multiple accounts using AWS FIS. Illustrates how to integrate fault tests with CI/CD systems and automated event alarms to prevent unintended outages.

Continuous Integration

  • (2022) thenewstack.io: Operationalizing Chaos Engineering with GitOps [ADVANCED LEVEL] [COMMUNITY-TOOL] — Proposes the paradigm of GitOps-driven chaos engineering. By declaring chaos configurations alongside standard application manifests in Git, engineering teams achieve strict auditability, versioning, automated cleanups, and predictable pipeline integration.
  • (2021) pingcap.com: chaos-mesh-action: Integrate Chaos Engineering into Your CI [COMMUNITY-TOOL] — Demonstrates the implementation of Chaos Mesh within GitHub Actions CI/CD workflows using chaos-mesh-action. Allows developers to continuously assert the resilience of code changes by spinning up test clusters, injecting faults, and validating outputs on pull requests.

Curated Resources

DevOps Culture

  • (2021) thenewstack.io: Chaos Engineering Is Not Just for Ops [COMMUNITY-TOOL] — Advocates for moving failure simulation from an operational security phase directly into the developer workflow. Demonstrates how validating application response limits and fallback boundaries earlier in the software development lifecycle reduces mean-time-to-resolution (MTTR) and builds robust container services.
  • (2021) thenewstack.io: Why Chaos Engineering Isnt Just for Operations [COMMUNITY-TOOL] — Explores why resilience engineering is a collaborative software development responsibility rather than an operational afterthought. Discusses using simulated outages to expose application logic flaws, API dependency assumptions, and downstream cascading failures in production.

Enterprise Platforms

Failure Simulation

  • (2021) BuggyApp: Simulate performance problems [JAVA / ENGLISH CONTENT] [COMMUNITY-TOOL] — An application suite designed to simulate real-world system errors within test environments. Enables deliberate injection of CPU spikes, memory leaks, thread locks, and database blockages, providing a safe practice space for telemetry alerts and diagnostic playbooks.

Fundamentals

  • (2022) opsmx.com: What is Chaos Engineering? [COMMUNITY-TOOL] — An introductory overview defining chaos engineering principles and lifecycle. Explains the process of forming a steady-state hypothesis, orchestrating failure experiments (such as resource spikes or network drops), and analyzing output telemetry to continuously harden cloud workloads.
  • (2021) thenewstack.io: Chaos Engineering Made Simple [COMMUNITY-TOOL] — Simplifies chaos engineering for development teams, presenting an incremental approach to failure testing. Details how minimal disruption sequences in staging help debug configuration fallbacks before attempting large-scale production game days.

Kubernetes Tools (1)

  • (2025) ==chaosblade== 6352 [GO CONTENT] [ADVANCED LEVEL] 🌟🌟🌟🌟🌟 [DE FACTO STANDARD] — Alibaba's multi-platform chaos engineering tool designed to inject faults across various levels of a system. Specifically targets OS resource exhaustion, network degradation, disk I/O bottlenecks, and deep application-layer faults for languages like Java, Go, and C++.
  • (2024) ==GitHub: kube-monkey== 3064 [GO CONTENT] 🌟🌟🌟🌟🌟 [DE FACTO STANDARD] — A Go-based Kubernetes implementation of Netflix's Chaos Monkey. kube-monkey runs inside k8s clusters to systematically schedule and delete random Pod instances within designated namespaces, forcing development teams to architect highly redundant and self-healing services.
  • (2024) ==PowerfulSeal== 1975 [PYTHON CONTENT] 🌟🌟🌟🌟🌟 [DE FACTO STANDARD] — A Python-based chaos engine designed for Kubernetes. PowerfulSeal operates interactively or via declarative policy configurations to systematically delete pods, shut down infrastructure nodes, and disrupt networking to reveal platform design flaws.
  • (2020) openshift.com: Introduction to Kraken, a Chaos Tool for OpenShift/Kubernetes [COMMUNITY-TOOL] — An introduction to Kraken, Red Hat's open-source chaos engineering engine tailored for OpenShift and Kubernetes. Kraken enables automated node disruptions, namespace-level resource starvation, and API-level faults to locate architecture bottlenecks.

Operations Strategy

  • (2021) thenewstack.io: Chaos Engineering Progressively Moves to Production [ADVANCED LEVEL] [COMMUNITY-TOOL] — Discusses the progression of chaos testing directly into production clusters. Focuses on minimizing the blast radius using progressive deployment gates, automated circuit breakers, canary releases, and deep observability to safely capture real-world dependency issues.
  • (2021) opensource.com: 5 lessons I learned about chaos engineering for Kubernetes [COMMUNITY-TOOL] — Distills key lessons from executing chaos experiments on live Kubernetes clusters. Discusses critical parameters such as understanding container restart policies, the impact of DNS connection caching, resource limit thresholds, and balancing false alert metrics.
  • (2021) thenewstack.io: Use Chaos Engineering to Strengthen Your Incident Response [COMMUNITY-TOOL] — Explores how structured chaos experiments improve an organization's incident response systems. By executing planned simulations, teams refine alert routing, on-call page distributions, incident runbooks, and cross-functional post-mortems.

Serverless Systems

  • (2021) thenewstack.io: Breaking Serverless on Purpose with Chaos Engineering [ADVANCED LEVEL] [COMMUNITY-TOOL] — Explores the challenges and methods of injecting faults into ephemeral, serverless environments (e.g., AWS Lambda). Discusses techniques like wrapper-based latency injection, API response mocking, and runtime variable tampering to validate failure paths.

Stateful Systems

Telemetry Systems


💡 Explore Related: DevOps | Developerportals | SRE