Current Kraken integration with Cerberus monitors the cluster as well as the
application health post chaos and pass/fails if they are not healthy after chaos.
This commit adds ability to monitor the user application health during the chaos
and fails the run in case of downtime as it's potentially a downtime in case of
customers environment as well. It is especially useful in case of control plane
failure scenarios including API server, Etcd, Ingress etc.
This commit:
- Adds timeout to avoid operations hanging for long durations.
- Improves exception handling and exits wherever needed.
- Sets KUBECONFIG env var globoally to access the cluster.
This commit:
- Adds support to automate the infrastructure pieces leveraged by Kraken
including Cerberus and Elasticsearch
- Adds a Kraken config that can be used to discover all the infra pieces
automatically without having to tweak the configuration.
This commit enables alerting in Kraken based on the Prometheus queries defined
by the user and modifies the return code of the run to determine pass/fail for
the run.
This commit:
- Enables Kraken to leverage kube-burner to scrape metrics from
Prometheus and index them into Elasticsearch. This way we can
take a look at the metrics in Grafana long term even after the
cluster is terminated.
- Enables separation of operations based on distribution with
OpenShift as the default option. One of the use cases is to
capture Prometheus instance details as it's installed by default
while it's optional for Kubernetes.
This commit:
- Refactors the code base to be more modular by moving functions
into respective modules to make it lean and reusable.
- Uses black to reformat the code to follow PEP 8 practices.
This commit:
- Adds a pointer to a new blog which emphasizes the importance
of making chaos part of Perf/Scale test runs.
- Bumps up the allowed max-line-length for the linters.
This commit enables performance monitoring on the cluster when
running Kraken to be able to observe how cluster reacts to failures
as it's important to make sure the cluster is healthy in terms of
both recovery as well as performance.
This commit:
- Converts various sections in the readme into individual documents.
- Adds pointers to the public blogs.
- Updates workflow/architecture diagram.
- Adds community info and contributing guidelines.
This commit:
- Adds a node scenario to stop and start an instance
- Adds a node scenario to terminate an instance
- Adds a node scenario to reboot an instance
- Adds a node scenario to stop the kubelet
- Adds a node scenario to crash the node
This commit adds support to run the tool as a container on the host
with access to kubeconfig for better portability. The plan is to
trigger regular image builds on quay.io to make sure it has the
latest code.
This commit updates readme with the following:
- Information on how to use the tool.
- Information on adding new scenarios.
- Information on using Cerberus tool for pass/fail.
