mirror of
https://github.com/lucky-sideburn/kubeinvaders.git
synced 2026-05-17 22:26:56 +00:00
fix readme
This commit is contained in:
Eugenio Marzo
157
README.md
157
README.md
@@ -2,32 +2,29 @@
|
||||
|
||||
**Gamified Chaos Engineering and Educational Tool for Kubernetes**
|
||||
|
||||
<img src="./doc_images/1750875811732.jpg" alt="Alt Text" style="width:50%;">
|
||||
<img src="./doc_images/1750875811732.jpg" alt="KubeInvaders game screenshot" style="width:50%;">
|
||||
|
||||
This project, recommended by the CNCF (https://github.com/cncf/sandbox/issues/124), has a strong following and significant educational value. It's a chaos engineering tool, but it's also recommended for studying Kubernetes and resilience topics.
|
||||
This project is [recommended by the CNCF](https://github.com/cncf/sandbox/issues/124) and has significant educational value. It is a chaos engineering tool, but it is also used for studying Kubernetes and resilience topics.
|
||||
|
||||
It is part of the Cloud Native Computing Foundation's (CNCF) landscape in the Observability and Analysis - Chaos Engineering section (https://landscape.cncf.io/).
|
||||
It is part of the Cloud Native Computing Foundation's (CNCF) landscape in the [Observability and Analysis - Chaos Engineering](https://landscape.cncf.io/) section.
|
||||
|
||||
Some companies use it for marketing at tech conferences in DevOps & SRE. For example at [𝗗𝗲𝗰𝗼𝗺𝗽𝗶𝗹𝗲𝗗 𝟮𝟬𝟮𝟱](https://www.linkedin.com/posts/cloud-%26-heat-technologies-gmbh_kubeinvaders-onpremise-managedkubernetes-activity-7293538807906258946-YtKV?utm_source=share&utm_medium=member_desktop&rcm=ACoAAAkOMNYBK7j_raLIIJBfs2RBA94_sK4Yeyg)
|
||||
Some companies use it for marketing at tech conferences in DevOps & SRE. For example at [Decompiled 2025](https://www.linkedin.com/posts/cloud-%26-heat-technologies-gmbh_kubeinvaders-onpremise-managedkubernetes-activity-7293538807906258946-YtKV?utm_source=share&utm_medium=member_desktop&rcm=ACoAAAkOMNYBK7j_raLIIJBfs2RBA94_sK4Yeyg).
|
||||
|
||||
The teams at Platform Engineering (https://platformengineering.it/) and GDT - Garanti Del Talento ([https://www.garantideltalento.it/](https://www.garantideltalento.it)) back this project. They provide enterprise-grade features and SRE experts to help customers verify the resilience of their Kubernetes infrastructure.
|
||||
The teams at [Platform Engineering](https://platformengineering.it/) and [GDT - Garanti Del Talento](https://www.garantideltalento.it) back this project. They provide enterprise-grade features and SRE experts to help customers verify the resilience of their Kubernetes infrastructure.
|
||||
|
||||
Here are the slides (https://www.slideshare.net/EugenioMarzo/kubeinvaders-chaos-engineering-practices-for-kubernetes1pdf) from the Chaos Engineering speech I prepared for FOSDEM 2023. Unfortunately, I couldn't be present at my talk, but I still wanted to share them with the community.
|
||||
Here are the [slides](https://www.slideshare.net/EugenioMarzo/kubeinvaders-chaos-engineering-practices-for-kubernetes1pdf) from the Chaos Engineering talk I prepared for FOSDEM 2023. Unfortunately, I could not attend in person, but I still wanted to share them with the community.
|
||||
|
||||
# Table of Contents
|
||||
|
||||
1. [Description](#Description)
|
||||
2. [Installation](#Installation)
|
||||
3. [Example using Podman + MiniKube](#Example-using-Podman--MiniKube)
|
||||
4. [Usage](#Usage)
|
||||
5. [URL Monitoring During Chaos Session](#URL-Monitoring-During-Chaos-Session)
|
||||
6. [Persistence](#Persistence)
|
||||
7. [Generic Troubleshooting & Known Problems](#Generic-Troubleshooting-and-Known-Problems)
|
||||
8. [Troubleshooting Unknown Namespace](#Troubleshooting-Unknown-Namespace)
|
||||
9. [Prometheus Metrics](#Prometheus-Metrics)
|
||||
10. [Community](#Community)
|
||||
11. [Community blogs and videos](#Community-blogs-and-videos)
|
||||
12. [License](#License)
|
||||
4. [URL Monitoring During Chaos Session](#URL-Monitoring-During-Chaos-Session)
|
||||
5. [Troubleshooting Unknown Namespace](#Troubleshooting-Unknown-Namespace)
|
||||
6. [Prometheus Metrics](#Prometheus-Metrics)
|
||||
7. [Community](#Community)
|
||||
8. [Community blogs and videos](#Community-blogs-and-videos)
|
||||
9. [License](#License)
|
||||
|
||||
## Description
|
||||
|
||||
@@ -157,7 +154,7 @@ EOF
|
||||
Extract the token:
|
||||
|
||||
```bash
|
||||
TOKEN=$(k get secret -n kubeinvaders -o go-template='{{.data.token | base64decode}}' kinv-sa-token)
|
||||
TOKEN=$(kubectl get secret -n kubeinvaders -o go-template='{{.data.token | base64decode}}' kinv-sa-token)
|
||||
```
|
||||
|
||||
**Important:** Use a valid Kubernetes token. If the token is missing, invalid, or expired, KubeInvaders cannot call the Kubernetes API and game actions will fail.
|
||||
@@ -177,7 +174,8 @@ kubectl create namespace namespace2
|
||||
|
||||
## Example using Podman + MiniKube
|
||||
|
||||
Install MiniKube
|
||||
Install MiniKube:
|
||||
|
||||
```bash
|
||||
luckysideburn:~ >> % minikube start
|
||||
😄 minikube v1.38.1 on Darwin 26.2 (arm64)
|
||||
@@ -199,44 +197,29 @@ luckysideburn:~ >> % minikube start
|
||||
▪ Want kubectl v1.35.1? Try 'minikube kubectl -- get pods -A'
|
||||
```
|
||||
|
||||
Take MiniKube IP
|
||||
Get the MiniKube API server address using one of these commands:
|
||||
|
||||
```bash
|
||||
luckysideburn:~ >> % cat /Users/eugenio/.kube/config | grep server | grep $(minikube ip)
|
||||
server: https://192.168.64.2:8443
|
||||
```
|
||||
|
||||
OR
|
||||
|
||||
```bash
|
||||
luckysideburn:~ >> % kubectl cluster-info
|
||||
|
||||
Kubernetes control plane is running at https://192.168.64.2:8443
|
||||
CoreDNS is running at https://192.168.64.2:8443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
|
||||
```
|
||||
|
||||
Take MiniKube CA
|
||||
Get the MiniKube CA certificate (you will need its content when configuring KubeInvaders):
|
||||
|
||||
```bash
|
||||
luckysideburn:~ >> % cat ~/.minikube/ca.crt
|
||||
-----BEGIN CERTIFICATE-----
|
||||
MIIDBjCCAe6gAwIBAgIBATANBgkqhkiG9w0BAQsFADAVMRMwEQYDVQQDEwptaW5p
|
||||
a3ViZUNBMB4XDTI2MDQyMzA2MTQwMVoXDTM2MDQyMTA2MTQwMVowFTETMBEGA1UE
|
||||
AxMKbWluaWt1YmVDQTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBANuB
|
||||
SAmidikCwaaCYW5wuTzdrtSv/8plGenF5EOh95c01YZgfd9nE/w5fFqDLbbiZ6sm
|
||||
qJI9JzomVI/Dhc5E+GLFsX+Ij0FPEb1AXvM0UEcLnfue9vhVLmR6bOQ8XFolOfb4
|
||||
gijD7V05nyMxMeWU+txRBJeSCNuckvnKzSb9+8l/8CtYSnqZI4pbdpQtjWg2G/De
|
||||
1b3xzxTMLPcWL9s8EnX9S5tfWB41ADlz2r4fVZanW3FiT7jTOC+Kh7oCPfaMmpVj
|
||||
gNDJCXvevrRtp1kztdl+UqMTt2JOi2xd6SCT9njYc1jvTM/JrK1YN1cH69x+LhVR
|
||||
jzvrtiIYWT3aqwt0bCMCAwEAAaNhMF8wDgYDVR0PAQH/BAQDAgKkMB0GA1UdJQQW
|
||||
MBQGCCsGAQUFBwMCBggrBgEFBQcDATAPBgNVHRMBAf8EBTADAQH/MB0GA1UdDgQW
|
||||
BBTvgRYoLtGLZEO1XuEHC55vWc1nMzANBgkqhkiG9w0BAQsFAAOCAQEAdmZBj4Nm
|
||||
GHsqKztFAWrNMtu9SPxzCnPJ/tIJQxBSupRFWkdsv65xxAQZWqunpxZ/iDj7A7qd
|
||||
M6E/xbQc5Df6PpSntgagZesW//xNIXXFWOkiLCH2jaxrj7PkC86TLQRV1phKdVEX
|
||||
2DHOTwrrEo62iKYSlV9pZHpG2VH6HGYIcyqCMFeDRVZGEqPJKgWD+xEoMBO/yHe5
|
||||
gf9pcpaGe4hpj1esitonng+92HwIDldgkRfNipVYgmwqnoeLwMdYhfb+4erjjwyU
|
||||
gQNOiYSIKnBc4A3VwI6oeom8w0aMTikIo9/ljQwNpQgAjgrg+9C0kZfV7BzHtU0o
|
||||
RGlvZDpBZQTHPQ==
|
||||
-----END CERTIFICATE-----
|
||||
cat ~/.minikube/ca.crt
|
||||
```
|
||||
```
|
||||
|
||||
Create Namespace, Service Account and Token
|
||||
Create the namespace, service account, and token:
|
||||
|
||||
```bash
|
||||
luckysideburn:~ >> % kubectl create ns kubeinvaders
|
||||
namespace/kubeinvaders created
|
||||
@@ -249,10 +232,11 @@ luckysideburn:~ >> % kubectl create clusterrolebinding kubeinvaders-cluster-admi
|
||||
--serviceaccount=kubeinvaders:kubeinvaders-sa
|
||||
|
||||
luckysideburn:~ >> % kubectl create token kubeinvaders-sa -n kubeinvaders --duration=24h
|
||||
eyJhbGciOiJSUzI1NiIsImtpZCI6Imh4MGs0WXk3ZXE1eHk4M2pMWHZsRFducmR2d0xQeFJrQ2xzdnlNaDVYcVEifQ.eyJhdWQiOlsiaHR0cHM6Ly9rdWJlcm5ldGVzLmRlZmF1bHQuc3ZjLmNsdXN0ZXIubG9jYWwiXSwiZXhwIjoxNzc3MDk3ODE5LCJpYXQiOjE3NzcwMTE0MTksImlzcyI6Imh0dHBzOi8va3ViZXJuZXRlcy5kZWZhdWx0LnN2Yy5jbHVzdGVyLmxvY2FsIiwianRpIjoiOWZiNjA3ZGEtYjM5NS00ZTI4LTk5MmEtODkyYzY5OWE0OWFlIiwia3ViZXJuZXRlcy5pbyI6eyJuYW1lc3BhY2UiOiJrdWJlaW52YWRlcnMiLCJzZXJ2aWNlYWNjb3VudCI6eyJuYW1lIjoia3ViZWludmFkZXJzLXNhIiwidWlkIjoiYzVhMTkwZjYtYjZiZi00NTFiLWIzMTktODg2YmZkZjJlZGZkIn19LCJuYmYiOjE3NzcwMTE0MTksInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlaW52YWRlcnM6a3ViZWludmFkZXJzLXNhIn0.j1D8EZOPxIvtTL3ydtuFLDt9GVD1s8phj59dJI9AI6LSGJllCF-hmJmT0o-1p2imrS7FYKpq8NAly5IQqlBBKbIzliU84l8gD-DcsBxiuFcqyOqpckMC-ogxvfjcfhc-AB08ROrm9IM7xaporkRBiiC60Q3F0mlqw2_rmLlcZc_CFd2xo515gG56BAA6JJZ-mRLNYsBbMrYbI-TTs0jaIRWk-S-sQXOVhVf_asWt9pqnJ09-t_vevPSkbHGjAWoUp6PQElI4WalZZQqikmxsMdRANpimVne4vZf0HIcueqU4clYkfZdrsRtEhhswd_LDoMz6u6tGL1C5AkWvBwkzhA
|
||||
<your-token>
|
||||
```
|
||||
|
||||
Run KubeInvaders
|
||||
Run KubeInvaders:
|
||||
|
||||
```bash
|
||||
podman run -p 8080:8080 --network=host kubeinvaders:latest
|
||||
```
|
||||
@@ -260,87 +244,28 @@ podman run -p 8080:8080 --network=host kubeinvaders:latest
|
||||
If you are on macOS, you may encounter issues due to Podman Machine networking.
|
||||
|
||||
|
||||
|
||||
## Usage
|
||||
|
||||
### Start The Chaos Experiment
|
||||
|
||||
Press the "Start" button to initiate the automatic pilot (the button changes to "Stop" to disable this feature).
|
||||
|
||||
### Enable Shuffle :joystick:
|
||||
|
||||
Press the "Enable Shuffle" button to randomly rearrange the positions of pods or K8s nodes (the button changes to "Disable Shuffle" to deactivate this feature).
|
||||
|
||||
### Enable Auto Jump Between Namespace :joystick:
|
||||
|
||||
Press the "Auto NS Switch" button to randomly switch between namespaces (the button changes to "Disable Auto NS Switch" to deactivate this feature).
|
||||
|
||||
### Show / Hide Pods Name :joystick:
|
||||
|
||||
Press the "Hide Pods Name" button to conceal the names of the pods beneath the aliens (the button changes to "Show Pods Name" to deactivate this feature).
|
||||
|
||||
### Information about Current Status and Events :joystick:
|
||||
|
||||
As shown below, on the game screen near the spaceship, there are details about the current cluster, namespace, and some configurations.
|
||||
|
||||
|
||||
|
||||
Under the + and - buttons, a bar appears with the latest game events.
|
||||
|
||||
|
||||
|
||||
### Show Special Keys :joystick:
|
||||
|
||||
Press 'h' or select 'Show Special Keys' from the menu.
|
||||
|
||||
### Zoom In / Out :joystick:
|
||||
|
||||
Press the + or - buttons to increase or decrease the game screen.
|
||||
|
||||
### Chaos Containers for Master and Worker Nodes
|
||||
|
||||
- Select "Show Current Chaos Container for Nodes" from the menu to see which container starts when you attack a worker node (not an alien, they are pods).
|
||||
|
||||
- Select "Set Custom Chaos Container for Nodes" from the menu to use your preferred image or configuration against nodes.
|
||||
|
||||
## URL Monitoring During Chaos Session
|
||||
|
||||
During a chaos engineering session, you can monitor the behavior of an HTTP call exposed by an Ingress.
|
||||
|
||||
Use the flag "Add HTTP check & Chaos Report" and add the URL to monitor
|
||||
|
||||
Use the flag "Add HTTP check & Chaos Report" and add the URL to monitor.
|
||||
|
||||
Follow real-time charts during the experiment
|
||||
|
||||
|
||||
|
||||
Follow real-time charts during the experiment.
|
||||
|
||||
|
||||
|
||||
## Persistence
|
||||
|
||||
K-inv uses Redis to save and manage data. Redis is configured with "appendonly."
|
||||
|
||||
The legacy Helm chart does not support PersistentVolumes.
|
||||
|
||||
## Generic Troubleshooting and Known Problems
|
||||
- If you don't see aliens, please follow these steps: [see issue #100](https://github.com/lucky-sideburn/kubeinvaders/issues/100#event-18433067619)
|
||||
- It seems that KubeInvaders does not work with EKS due to problems with ServiceAccount.
|
||||
- Currently, the installation of KubeInvaders into a namespace that is not named "kubeinvaders" is not supported.
|
||||
- I have only tested KubeInvaders with a Kubernetes cluster installed through KubeSpray.
|
||||
- If you don't see aliens, please follow these steps:
|
||||
1. Open a terminal and run "kubectl logs <pod_of_kubeinvader> -n kubeinvaders -f"
|
||||
2. Execute the following command from another terminal: `curl "https://<your_kubeinvaders_url>/kube/pods?action=list&namespace=namespace1" -k`
|
||||
3. Open an issue with attached logs.
|
||||
- If you use route_host instead of ingress, please also specify the port, e.g. route_host: "kubeinvaders.example.com:8080". The port must match the NodePort service port.
|
||||
|
||||
## Troubleshooting Unknown Namespace
|
||||
|
||||
- Check if the namespaces configured in the UI (for example: namespace1,namespace2) exist and contain pods.
|
||||
- Check if the namespaces configured in the UI (for example: `namespace1`, `namespace2`) exist and contain pods.
|
||||
- Check your browser's developer console for any failed HTTP requests (send them to luckysideburn[at]gmail[dot]com or open an issue on this repo).
|
||||
- Try using latest_debug and send logs to luckysideburn[at]gmail[dot]com or open an issue on this repo.
|
||||
- Try using `latest_debug` and send logs to luckysideburn[at]gmail[dot]com or open an issue on this repo.
|
||||
|
||||
## Prometheus Metrics
|
||||
|
||||
KubeInvaders exposes metrics for Prometheus through the standard endpoint /metrics.
|
||||
KubeInvaders exposes metrics for Prometheus through the standard endpoint `/metrics`.
|
||||
|
||||
Here is an example of Prometheus configuration:
|
||||
|
||||
@@ -358,14 +283,14 @@ Example of metrics:
|
||||
| ---------------------------------------------------------- | ------------------------------------------------------------ |
|
||||
| chaos_jobs_node_count{node=workernode01} | Total number of chaos jobs executed per node |
|
||||
| chaos_node_jobs_total | Total number of chaos jobs executed against all worker nodes |
|
||||
| deleted_pods_total 16 | Total number of deleted pods |
|
||||
| deleted_pods_total | Total number of deleted pods |
|
||||
| deleted_namespace_pods_count{namespace=myawesomenamespace} | Total number of deleted pods per namespace |
|
||||
|
||||
[Download Grafana dashboard](./confs/grafana/KubeInvadersDashboard.json)
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Community
|
||||
|
||||
@@ -375,7 +300,8 @@ Please reach out for news, bugs, feature requests, and other issues via:
|
||||
- New features are published on YouTube too in [this channel](https://www.youtube.com/channel/UCQ5BQ8R2fDL_WkNAllYRrpQ)
|
||||
|
||||
## Community blogs and videos
|
||||
|
||||
|
||||
|
||||
|
||||
- [The Kubernetes ecosystem is a candy store](https://opensource.googleblog.com/2024/06/the-kubernetes-ecosystem-is-candy-store.html)
|
||||
- [ AdaCon Norway Live Stream ](https://www.youtube.com/watch?v=rt_eM_KRfK4)
|
||||
@@ -403,5 +329,4 @@ Please reach out for news, bugs, feature requests, and other issues via:
|
||||
|
||||
## License
|
||||
|
||||
KubeInvaders is licensed under the Apache 2.0 License.
|
||||
See [LICENSE](./LICENSE) for the full license text.
|
||||
KubeInvaders is licensed under the GNU General Public License v3.0. See [LICENSE](./LICENSE) for the full license text.
|
||||
|
||||
Reference in New Issue
Block a user