github/container.training
1
0
Fork 0
mirror of https://github.com/jpetazzo/container.training.git synced 2026-07-18 12:29:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

Upgrade the cluster upgrade chapter

Browse Source 
Add information about version skew.
Better explain what's good/bad to do.
This commit is contained in:
Jerome Petazzoni
2020-02-02 05:41:01 -06:00
parent 7f85c645f0
commit ebadc1ca59
4 changed files with 169 additions and 64 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
prepare-vms/setup-admin-clusters.sh
View File

@@ -61,6 +61,6 @@ TAG=$PREFIX-$SETTINGS
--count $((3*$STUDENTS))
./workshopctl deploy $TAG
./workshopctl kube $TAG 1.14.6
./workshopctl kube $TAG 1.16.6
./workshopctl cards $TAG

227
slides/k8s/cluster-upgrade.md
View File

@@ -81,7 +81,7 @@
## What version are we running anyway?
- When I say, "I'm running Kubernetes 1.11", is that the version of:
- When I say, "I'm running Kubernetes 1.16", is that the version of:
- kubectl
@@ -139,6 +139,73 @@
---
## Important questions
- Should we upgrade the control plane before or after the kubelets?
- Within the control plane, should we upgrade the API server first or last?
- How often should we upgrade?
- How long are versions maintained?
- All the answers are in [the documentation about version skew policy](https://kubernetes.io/docs/setup/release/version-skew-policy/)!
- Let's review the key elements together ...
---
## Kubernetes uses semantic versioning
- Kubernetes versions look like MAJOR.MINOR.PATCH; e.g. in 1.17.2:
- MAJOR = 1
- MINOR = 17
- PATCH = 2
- It's always possible to mix and match different PATCH releases
(e.g. 1.16.1 and 1.16.6 are compatible)
- It is recommended to run the latest PATCH release
(but it's mandatory only when there is a security advisory)
---
## Version skew
- API server must be more recent than its clients (kubelet and control plane)
- ... Which means it must always be upgraded first
- All components support a difference of one¹ MINOR version
- This allows live upgrades (since we can mix e.g. 1.15 and 1.16)
- It also means that going from 1.14 to 1.16 requires going through 1.15
.footnote[¹Except kubelet, which can be up to two MINOR behind API server,
and kubectl, which can be one MINOR ahead or behind API server.]
---
## Release cycle
- There is a new PATCH relese whenever necessary
(every few weeks, or "ASAP" when there is a security vulnerability)
- There is a new MINOR release every 3 months (approximately)
- At any given time, three MINOR releases are maintained
- ... Which means that MINOR releases are maintained approximately 9 months
- We should expect to upgrade at least every 3 months (on average)
---
## In practice
- We are going to update a few cluster components
@@ -151,47 +218,6 @@
---
## Updating kubelet
- These nodes have been installed using the official Kubernetes packages
- We can therefore use `apt` or `apt-get`
.exercise[
- Log into node `test3`
- View available versions for package `kubelet`:
```bash
apt show kubelet -a | grep ^Version
```
- Upgrade kubelet:
```bash
sudo apt install kubelet=1.15.3-00
```
]
---
## Checking what we've done
.exercise[
- Log into node `test1`
- Check node versions:
```bash
kubectl get nodes -o wide
```
- Create a deployment and scale it to make sure that the node still works
]
---
## Updating the API server
- This cluster has been deployed with kubeadm
@@ -228,7 +254,7 @@
sudo vim /etc/kubernetes/manifests/kube-apiserver.yaml
```
- Look for the `image:` line, and update it to e.g. `v1.15.0`
- Look for the `image:` line, and update it to e.g. `v1.17.0`
]
@@ -249,9 +275,27 @@
---
## Was that a good idea?
--
**No!**
--
- Remember the guideline we gave earlier:
*To update a component, use whatever was used to install it.*
- This control plane was deployed with kubeadm
- We should use kubeadm to upgrade it!
---
## Updating the whole control plane
- As an example, we'll use kubeadm to upgrade the entire control plane
- Let's make it right, and use kubeadm to upgrade the entire control plane
(note: this is possible only because the cluster was installed with kubeadm)
@@ -264,11 +308,11 @@
]
Note 1: kubeadm thinks that our cluster is running 1.15.0.
Note 1: kubeadm thinks that our cluster is running 1.17.0.
<br/>It is confused by our manual upgrade of the API server!
Note 2: kubeadm itself is still version 1.14.6.
<br/>It doesn't know how to upgrade do 1.15.X.
Note 2: kubeadm itself is still version 1.16.6.
<br/>It doesn't know how to upgrade do 1.17.X.
---
@@ -290,8 +334,8 @@ Note 2: kubeadm itself is still version 1.14.6.
]
Note: kubeadm still thinks that our cluster is running 1.15.0.
<br/>But at least it knows about version 1.15.X now.
Note: kubeadm still thinks that our cluster is running 1.17.0.
<br/>But at least it knows about version 1.17.X now.
---
@@ -307,28 +351,89 @@ Note: kubeadm still thinks that our cluster is running 1.15.0.
- Perform the upgrade:
```bash
sudo kubeadm upgrade apply v1.15.3
sudo kubeadm upgrade apply v1.17.2
```
]
---
## Updating kubelets
## Updating kubelet
- After updating the control plane, we need to update each kubelet
- These nodes have been installed using the official Kubernetes packages
- This requires to run a special command on each node, to download the config
- We can therefore use `apt` or `apt-get`
(this config is generated by kubeadm)
.exercise[
- Log into node `test3`
- View available versions for package `kubelet`:
```bash
apt show kubelet -a | grep ^Version
```
- Upgrade kubelet:
```bash
sudo apt install kubelet=1.17.2-00
```
]
---
## Checking what we've done
.exercise[
- Log into node `test1`
- Check node versions:
```bash
kubectl get nodes -o wide
```
- Create a deployment and scale it to make sure that the node still works
]
---
## Was that a good idea?
--
**Almost!**
--
- Yes, kubelet was installed with distribution packages
- However, kubeadm took care of configuring kubelet
(when doing `kubeadm join ...`)
- We were supposed to run a special command *before* upgrading kubelet!
- That command should be executed on each node
- It will download the kubelet configuration generated by kubeadm
---
## Upgrading kubelet the right way
- The command that we need to run was shown by kubeadm
(after upgrading the control plane)
.exercise[
- Download the configuration on each node, and upgrade kubelet:
```bash
for N in 1 2 3; do
ssh test$N sudo kubeadm upgrade node config --kubelet-version v1.15.3
ssh test$N sudo apt install kubelet=1.15.3-00
ssh test$N sudo kubeadm upgrade node config --kubelet-version v1.17.2
ssh test$N sudo apt install kubelet=1.17.2-00
done
```
]
@@ -337,7 +442,7 @@ Note: kubeadm still thinks that our cluster is running 1.15.0.
## Checking what we've done
- All our nodes should now be updated to version 1.15.3
- All our nodes should now be updated to version 1.17.2
.exercise[
@@ -354,12 +459,12 @@ class: extra-details
## Skipping versions
- This example worked because we went from 1.14 to 1.15
- This example worked because we went from 1.16 to 1.17
- If you are upgrading from e.g. 1.13, you will generally have to go through 1.14 first
- If you are upgrading from e.g. 1.14, you will have to go through 1.15 first
- This means upgrading kubeadm to 1.14.X, then using it to upgrade the cluster
- This means upgrading kubeadm to 1.15.X, then using it to upgrade the cluster
- Then upgrading kubeadm to 1.15.X, etc.
- Then upgrading kubeadm to 1.16.X, etc.
- **Make sure to read the release notes before upgrading!**

2
slides/k8s/daemonset.md
View File

@@ -52,7 +52,7 @@
<!-- ##VERSION## -->
- Unfortunately, as of Kubernetes 1.15, the CLI cannot create daemon sets
- Unfortunately, as of Kubernetes 1.17, the CLI cannot create daemon sets
--

2
slides/k8s/versions-k8s.md
View File

@@ -1,6 +1,6 @@
## Versions installed
- Kubernetes 1.17.1
- Kubernetes 1.17.2
- Docker Engine 19.03.5
- Docker Compose 1.24.1