mirror of
https://github.com/jpetazzo/container.training.git
synced 2026-07-18 20:39:17 +00:00
Add rolling upgrade section and whatsnext
This commit is contained in:
@@ -72,6 +72,8 @@ chapters:
|
||||
- ourapponkube.md
|
||||
- - kubectlscale.md
|
||||
- daemonset.md
|
||||
- rollout.md
|
||||
- whatsnext.md
|
||||
- |
|
||||
class: title
|
||||
|
||||
|
||||
197
docs/rollout.md
Normal file
197
docs/rollout.md
Normal file
@@ -0,0 +1,197 @@
|
||||
# Rolling updates
|
||||
|
||||
- By default (without rolling updates), when a scaled resource is updated:
|
||||
|
||||
- new pods are created
|
||||
|
||||
- old pods are terminated
|
||||
|
||||
- ... all at the same time
|
||||
|
||||
- if something goes wrong, ¯\\\_(ツ)\_/¯
|
||||
|
||||
---
|
||||
|
||||
# Rolling updates
|
||||
|
||||
- With rolling updates, when a resource is updated, it happens progressively
|
||||
|
||||
- Two parameters determine the pace of the rollout: `maxUnavailable` and `maxSurge`
|
||||
|
||||
- They can be specified in absolute number of pods, or percentage of the `replicas` count
|
||||
|
||||
- At any given time ...
|
||||
|
||||
- there will always be at least `replicas`-`maxUnavailable` pods available
|
||||
|
||||
- there will never be more than `replicas`+`maxSurge` pods in total
|
||||
|
||||
- there will therefore be up to `maxUnavailable`+`maxSurge` pods being updated
|
||||
|
||||
- We have the possibility to rollback to the previous version
|
||||
<br/>(if the update fails or is unsatisfactory in any way)
|
||||
|
||||
---
|
||||
|
||||
## Rolling updates in practice
|
||||
|
||||
- As of Kubernetes 1.8, we can do rolling updates with:
|
||||
|
||||
`deployments`, `daemonsets`, `statefulsets`
|
||||
|
||||
- Editing one of these resources will automatically result in a rolling update
|
||||
|
||||
- Rolling updates can be monitored with the `kubectl rollout` subcommand
|
||||
|
||||
---
|
||||
|
||||
## Building a new version of the `worker` service
|
||||
|
||||
.exercise[
|
||||
|
||||
- Go to the `stack` directory:
|
||||
```bash
|
||||
cd ~/orchestration-workshop/stacks
|
||||
```
|
||||
|
||||
- Edit `dockercoins/worker/worker.py`, update the `sleep` line to sleep 1 second
|
||||
|
||||
- Build a new tag and push it to the registry:
|
||||
```bash
|
||||
export REGISTRY=localhost:3xxxx TAG=v0.2
|
||||
docker-compose -f dockercoins.yml build
|
||||
docker-compose -f dockercoins.yml push
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
---
|
||||
|
||||
## Rolling out the new version of the `worker` service
|
||||
|
||||
.exercise[
|
||||
|
||||
- Let's monitor what's going on by opening a few terminals, and run:
|
||||
```bash
|
||||
kubectl get pods -w
|
||||
kubectl get replicasets -w
|
||||
kubectl get deployments -w
|
||||
```
|
||||
|
||||
- Update `worker` either with `kubectl edit`, or by running:
|
||||
```bash
|
||||
kubectl set image deploy worker worker=$REGISTRY/worker:$TAG
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
--
|
||||
|
||||
That rollout should be pretty quick. What shows in the web UI?
|
||||
|
||||
---
|
||||
|
||||
## Rolling out a boo-boo
|
||||
|
||||
- What happens if we make a mistake?
|
||||
|
||||
.exercise[
|
||||
|
||||
- Update `worker` by specifying a non-existent image:
|
||||
```bash
|
||||
export TAG=v0.3
|
||||
kubectl set image deploy worker worker=$REGISTRY/worker:$TAG
|
||||
```
|
||||
|
||||
- Check what's going on:
|
||||
```bash
|
||||
kubectl rollout status deploy worker
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
--
|
||||
|
||||
Our rollout is stuck. However, the app is not dead (just 10% slower).
|
||||
|
||||
---
|
||||
|
||||
## Recovering from a bad rollout
|
||||
|
||||
- We could push some `v0.3` image
|
||||
|
||||
(the pod retry logic will eventually catch it and the rollout will proceed)
|
||||
|
||||
- Or we could invoke a manual rollback
|
||||
|
||||
.exercise[
|
||||
|
||||
- Cancel the deployment and wait for the dust to settle down:
|
||||
```bash
|
||||
kubectl rollout undo deploy worker
|
||||
kubectl rollout status deploy worker
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
---
|
||||
|
||||
## Changing rollout parameters
|
||||
|
||||
- We want to:
|
||||
|
||||
- revert to `v0.1`
|
||||
- be conservative on availability (always have desired number of available workers)
|
||||
- be aggressive on rollout speed (update more than one pod at a time)
|
||||
- give some time to our workers to "warm up" before starting more
|
||||
|
||||
The corresponding changes can be expressed in the following YAML snippet:
|
||||
|
||||
.small[
|
||||
```yaml
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
containers:
|
||||
- name: worker
|
||||
image: $REGISTRY/worker:v0.1
|
||||
strategy:
|
||||
rollingUpdate:
|
||||
maxUnavailable: 0
|
||||
maxSurge: 3
|
||||
minReadySeconds: 10
|
||||
```
|
||||
]
|
||||
|
||||
---
|
||||
|
||||
## Applying changes through a YAML patch
|
||||
|
||||
- We could use `kubectl edit deployment worker`
|
||||
|
||||
- But we could also use `kubectl patch` with the exact YAML shown before
|
||||
|
||||
.exercise[
|
||||
|
||||
.small[
|
||||
|
||||
- Apply all our changes and wait for them to take effect:
|
||||
```bash
|
||||
kubectl patch deployment worker -p "
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
containers:
|
||||
- name: worker
|
||||
image: $REGISTRY/worker:v0.1
|
||||
strategy:
|
||||
rollingUpdate:
|
||||
maxUnavailable: 0
|
||||
maxSurge: 3
|
||||
minReadySeconds: 10
|
||||
"
|
||||
kubectl rollout status deployment worker
|
||||
```
|
||||
]
|
||||
|
||||
]
|
||||
17
docs/whatsnext.md
Normal file
17
docs/whatsnext.md
Normal file
@@ -0,0 +1,17 @@
|
||||
## What's next?
|
||||
|
||||
- Stateful services (databases and the like)
|
||||
|
||||
volumes, persistent volume claims, stateful sets
|
||||
|
||||
- Layer 7 constructs (HTTP routing)
|
||||
|
||||
ingress (check traefik!)
|
||||
|
||||
- Logging, metrics
|
||||
|
||||
- Secrets, config maps
|
||||
|
||||
- Developer experience
|
||||
|
||||
- Dashboard
|
||||
Reference in New Issue
Block a user