Merge branch 'more-info-on-labels-and-rollouts' into avril2018

This commit is contained in:
Jerome Petazzoni
2018-04-10 06:05:33 -05:00
2 changed files with 56 additions and 0 deletions

View File

@@ -460,3 +460,39 @@ The timestamps should give us a hint about how many pods are currently receiving
kubectl get pods -l run=rng -o name |
xargs kubectl patch -p "$PATCH"
```
---
## Labels and debugging
- When a pod is misbehaving, we can delete it: another one will be recreated
- But we can also change its labels
- It will be removed from the load balancer (it won't receive traffic anymore)
- Another pod will be recreated immediately
- But the problematic pod is still here, and we can inspect and debug it
- We can even re-add it to the rotation if necessary
(Very useful to troubleshoot intermittent and elusive bugs)
---
## Labels and advanced rollout control
- Conversely, we can add pods matching a service's selector
- These pods will then receive requests and serve traffic
- Examples:
- one-shot pod with all debug flags enabled, to collect logs
- pods created automatically, but added to rotation in a second step
<br/>
(by setting their label accordingly)
- This gives us building blocks for canary and blue/green deployments

View File

@@ -94,6 +94,26 @@ That rollout should be pretty quick. What shows in the web UI?
---
## Give it some time
- At first, it looks like nothing is happening (the graph remains at the same level)
- According to `kubectl get deploy -w`, the `deployment` was updated really quickly
- But `kubectl get pods -w` tells a different story
- The old `pods` are still here, and they stay in `Terminating` state for a while
- Eventually, they are terminated; and then the graph decreases significantly
- This delay is due to the fact that our worker doesn't handle signals
- Kubernetes sends a "polite" shutdown request to the worker, which ignores it
- Eventually, Kubernetes gets impatient and kills the container
---
## Rolling out a boo-boo
- What happens if we make a mistake?