- instead of using 'kubectl wait nodes', we now use a simpler
'kubectl get nodes -o name' and check if there is anything
in the output. This seems to work better (as the previous
method would sometimes remain stuck because the kubectl
process would never get stopped by SIGPIPE).
- the shpod SSH NodePort is no longer hard-coded to 32222,
which allows us to use e.g. vcluster to deploy multiple
Kubernetes labs on a single 'home' (or 'outer') Kubernetes
cluster.
We now go from 1.22 to 1.23.
Updating to 1.22 was necessary because Kubernetes 1.27
deprecated kubeadm config v1beta2, which forced us to
upgrade to v1beta3, which was only introduced in 1.22.
In other words, our scripts can only install Kubernetes
1.22+ now.
Kube No Trouble (kubent) is a simple tool to check whether you're using any of these API versions in your cluster and therefore should upgrade your workloads first, before upgrading your Kubernetes cluster.
v1beta2 support was removed in Kubernetes 1.27.
Warning, v1beta3 was introduced in Kubernetes 1.22
(I think?) which means that the minimum version for
"old cluster" deployments is now 1.22.
They're not valid anymore, and fixing them would require quite a lot of
work, since we drastically changed the way we provision things. I'm
removing them rather than leaving a completely broken thing.
Break down provider-specific configuration into two files:
- config.tf (actual configuration, e.g. credentials, that cannot be
included in submodules)
- variables.tf (per-provider knobs and settings, e.g. mapping logical
VM size like S/M/L to actual cloud SKUs)
Summary of changes:
- "workshopctl" is now "labctl"
- it can handle deployment of VMs but also of managed
Kubernetes clusters (and therefore, it replaces
the "prepare-tf" directory)
- support for many more providers has been added
Check the README.md, in particular the "directory structure";
it has the most important information.
It was still hard-coded to user 'docker' instead of using
the USER_LOGIN environment variable.
Also add download-retry when wgetting the websocketd deb.
- add support for Digital Ocean (through Terraform)
- add support for per-cluster SSH key (hackish for now)
- pre-load Kubernetes APT GPG key (because of GCS outage)
Using environment variables was a mistake, because they must be set again
manually each time we want to re-apply the Terraform configurations.
Instead, put the variables in a tfvars file.
Instead of Ubuntu 18.04, we should use 22.04 (especially as
18.04 will be EOL soon). This moves a few providers to 22.04
(and more will follow).
We now ship a small containerd configuration file (instead
of defaulting to an empty configuration like we did before)
since it looks like recent versions of containerd cause
infinite crashloops if the cgroups driver isn't set properly.
Also, Linode is now provisioned using Terraform (instead of
the old-style system relying on linode-cli) which should make
instance provisioning faster (thanks to Terraform parallelism).
The "wait" command now tries to log in with both "ubuntu" and
"root", and if it fails with "ubuntu" but succeeds with "root",
it will create the "ubuntu" user and give it full sudo rights.
Finally, a "standardize" action has been created to gather all
the commands that deal with non-standard Ubuntu images.
Note that for completeness, we should check that all providers
work correctly; currently only Linode has been validated.
Jérôme Petazzoni
Julien Girardin
dependabot[bot]
ENIX NOC
MrUtkarsh