From e2fd9531ef8bf31506831243aa9abb82951daf56 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=A9r=C3=B4me=20Petazzoni?= Date: Fri, 13 Oct 2017 23:16:02 +0200 Subject: [PATCH] Add rolling upgrade section and whatsnext --- docs/kube.yml | 2 + docs/rollout.md | 197 ++++++++++++++++++++++++++++++++++++++++++++++ docs/whatsnext.md | 17 ++++ 3 files changed, 216 insertions(+) create mode 100644 docs/rollout.md create mode 100644 docs/whatsnext.md diff --git a/docs/kube.yml b/docs/kube.yml index 9ca03d39..26147957 100644 --- a/docs/kube.yml +++ b/docs/kube.yml @@ -72,6 +72,8 @@ chapters: - ourapponkube.md - - kubectlscale.md - daemonset.md + - rollout.md + - whatsnext.md - | class: title diff --git a/docs/rollout.md b/docs/rollout.md new file mode 100644 index 00000000..ff62b7ef --- /dev/null +++ b/docs/rollout.md @@ -0,0 +1,197 @@ +# Rolling updates + +- By default (without rolling updates), when a scaled resource is updated: + + - new pods are created + + - old pods are terminated + + - ... all at the same time + + - if something goes wrong, ¯\\\_(ツ)\_/¯ + +--- + +# Rolling updates + +- With rolling updates, when a resource is updated, it happens progressively + +- Two parameters determine the pace of the rollout: `maxUnavailable` and `maxSurge` + +- They can be specified in absolute number of pods, or percentage of the `replicas` count + +- At any given time ... + + - there will always be at least `replicas`-`maxUnavailable` pods available + + - there will never be more than `replicas`+`maxSurge` pods in total + + - there will therefore be up to `maxUnavailable`+`maxSurge` pods being updated + +- We have the possibility to rollback to the previous version +
(if the update fails or is unsatisfactory in any way) + +--- + +## Rolling updates in practice + +- As of Kubernetes 1.8, we can do rolling updates with: + + `deployments`, `daemonsets`, `statefulsets` + +- Editing one of these resources will automatically result in a rolling update + +- Rolling updates can be monitored with the `kubectl rollout` subcommand + +--- + +## Building a new version of the `worker` service + +.exercise[ + +- Go to the `stack` directory: + ```bash + cd ~/orchestration-workshop/stacks + ``` + +- Edit `dockercoins/worker/worker.py`, update the `sleep` line to sleep 1 second + +- Build a new tag and push it to the registry: + ```bash + export REGISTRY=localhost:3xxxx TAG=v0.2 + docker-compose -f dockercoins.yml build + docker-compose -f dockercoins.yml push + ``` + +] + +--- + +## Rolling out the new version of the `worker` service + +.exercise[ + +- Let's monitor what's going on by opening a few terminals, and run: + ```bash + kubectl get pods -w + kubectl get replicasets -w + kubectl get deployments -w + ``` + +- Update `worker` either with `kubectl edit`, or by running: + ```bash + kubectl set image deploy worker worker=$REGISTRY/worker:$TAG + ``` + +] + +-- + +That rollout should be pretty quick. What shows in the web UI? + +--- + +## Rolling out a boo-boo + +- What happens if we make a mistake? + +.exercise[ + +- Update `worker` by specifying a non-existent image: + ```bash + export TAG=v0.3 + kubectl set image deploy worker worker=$REGISTRY/worker:$TAG + ``` + +- Check what's going on: + ```bash + kubectl rollout status deploy worker + ``` + +] + +-- + +Our rollout is stuck. However, the app is not dead (just 10% slower). + +--- + +## Recovering from a bad rollout + +- We could push some `v0.3` image + + (the pod retry logic will eventually catch it and the rollout will proceed) + +- Or we could invoke a manual rollback + +.exercise[ + +- Cancel the deployment and wait for the dust to settle down: + ```bash + kubectl rollout undo deploy worker + kubectl rollout status deploy worker + ``` + +] + +--- + +## Changing rollout parameters + +- We want to: + + - revert to `v0.1` + - be conservative on availability (always have desired number of available workers) + - be aggressive on rollout speed (update more than one pod at a time) + - give some time to our workers to "warm up" before starting more + +The corresponding changes can be expressed in the following YAML snippet: + +.small[ +```yaml +spec: + template: + spec: + containers: + - name: worker + image: $REGISTRY/worker:v0.1 + strategy: + rollingUpdate: + maxUnavailable: 0 + maxSurge: 3 + minReadySeconds: 10 +``` +] + +--- + +## Applying changes through a YAML patch + +- We could use `kubectl edit deployment worker` + +- But we could also use `kubectl patch` with the exact YAML shown before + +.exercise[ + +.small[ + +- Apply all our changes and wait for them to take effect: + ```bash + kubectl patch deployment worker -p " + spec: + template: + spec: + containers: + - name: worker + image: $REGISTRY/worker:v0.1 + strategy: + rollingUpdate: + maxUnavailable: 0 + maxSurge: 3 + minReadySeconds: 10 + " + kubectl rollout status deployment worker + ``` + ] + +] diff --git a/docs/whatsnext.md b/docs/whatsnext.md new file mode 100644 index 00000000..9503310e --- /dev/null +++ b/docs/whatsnext.md @@ -0,0 +1,17 @@ +## What's next? + +- Stateful services (databases and the like) + + volumes, persistent volume claims, stateful sets + +- Layer 7 constructs (HTTP routing) + + ingress (check traefik!) + +- Logging, metrics + +- Secrets, config maps + +- Developer experience + +- Dashboard \ No newline at end of file