From 5a815263874545f8779fe3ea72ca92bb1aae24a1 Mon Sep 17 00:00:00 2001 From: Jerome Petazzoni Date: Mon, 9 Apr 2018 07:57:27 -0500 Subject: [PATCH] Add two chapters: Helm and namespaces In these chapters, we: - show how to install Helm - run the Helm tiller on our cluster - use Helm to install Prometheus - don't do anything fancy with Prometheus (it's just for the sake of installing something) - create a basic Helm chart for DockerCoins - explain namespace concepts - show how to use contexts to hop between namespaces - use Helm to deploy DockerCoins to a new namespace These two chapters go together. --- slides/kube/helm.md | 205 ++++++++++++++++++++++ slides/kube/namespaces.md | 351 +++++++++++++------------------------ slides/swarm/namespaces.md | 236 +++++++++++++++++++++++++ 3 files changed, 565 insertions(+), 227 deletions(-) create mode 100644 slides/kube/helm.md create mode 100644 slides/swarm/namespaces.md diff --git a/slides/kube/helm.md b/slides/kube/helm.md new file mode 100644 index 00000000..4365a66f --- /dev/null +++ b/slides/kube/helm.md @@ -0,0 +1,205 @@ +# Managing stacks with Helm + +- We created our first resources with `kubectl run`, `kubectl expose` ... + +- We have also created resources by loading YAML files with `kubectl apply -f` + +- For larger stacks, managing thousands of lines of YAML is unreasonable + +- These YAML bundles need to be customized with variable parameters + + (E.g.: number of replicas, image version to use ...) + +- It would be nice to have an organized, versioned collection of bundles + +- It would be nice to be able to upgrade/rollback these bundles carefully + +- Helm offers all these things! + +--- + +## Helm concepts + +- `helm` is a CLI tool + +- `tiller` is its companion server-side component + +- A "chart" is an archive containing templatized YAML bundles + +- Charts are versioned + +- Charts can be stored on private or public repositories + +--- + +## Installing Helm + +- We need to install the `helm` CLI; then use it to deploy `tiller` + +.exercise[ + +- Install the `helm` CLI: + ```bash + curl https://raw.githubusercontent.com/kubernetes/helm/master/scripts/get | bash + ``` + +- Deploy `tiller`: + ```bash + helm init + ``` + +] + +--- + +## Fix account permissions + +- Helm permission model requires use to tweak permissions + +- In a more realistic deployment, you might create per-user or per-team + service accounts, roles, and role bindings + +.exercise[ + +- Grant `cluster-admin` role to `kube-system:default` service account: + ```bash + kubectl create clusterrolebinding add-on-cluster-admin \ + --clusterrole=cluster-admin --serviceaccount=kube-system:default + ``` + +] + +(Defining the exact roles and permissions on your cluster requires +a deeper knowledge of Kubernetes' RBAC model. The command above is +fine for personal and development clusters.) + +--- + +## View available charts + +- A public repo is pre-configurd when installing Helm + +- We can view available charts with `helm search` (and an optional keyword) + +.exercise[ + +- View all available charts: + ```bash + helm search + ``` + +- View charts related to `gitlab`: + ```bash + helm search gitlab + ``` + +] + +--- + +## Install a chart + +- Most charts use `LoadBalancer` service types by default + +- Most charts require persistent volumes to store data + +- We need to relax these requirements a bit + +.exercise[ + +- Install the Prometheus metrics collector on our cluster: + ```bash + helm install stable/prometheus \ + --set server.service.type=NodePort \ + --set server.persistentVolume.enabled=false + ``` + +] + +Where do these `--set` options come from? + +--- + +## Inspecting a chart + +- `helm inspect` shows details about a chart (including available options) + +.exercise[ + +- See the metadata and all available options for `stable/prometheus`: + ```bash + helm inspect stable/prometheus + ``` + +] + +The chart's metadata includes an URL to the project's home page. + +(Sometimes it conveniently points to the documentation for the chart.) + +--- + +## Creating a chart + +- We are going to show a way to create a *very simplified* chart + +- In a real chart, *lots of things* would be templatized + + (Resource names, service types, number of replicas...) + +.exercise[ + +- Create a sample chart: + ```bash + helm create dockercoins + ``` + +- Move away the sample templates: + ```bash + mkdir dockercoins/do-not-use + mv dockercoins/templates dockercoins/default-templates + ``` + +] + +--- + +## Exporting the YAML for our application + +- The following section assumes that DockerCoins is currently running + +.exercise[ + +- Create one YAML file for each resource that we need: + .small[ + ```bash + + while read kind name; do + kubectl get -o yaml --export $kind $name > dockercoins/templates/$name-$kind.yaml + done < -It is mediated by capabilities, cgroups, namespaces, seccomp, LSMs... - ---- - -class: namespaces - -## User Namespaces in Docker - -- Optional feature added in Docker Engine 1.10 - -- Not enabled by default - -- Has to be enabled at Engine startup, and affects all containers - -- When enabled, `UID:GID` in containers are mapped to a different range on the host - -- Safer than switching to a non-root user (with `-u` or `USER`) in the container -
- (Since with user namespaces, root escalation maps to a non-privileged user) - -- Can be selectively disabled per container by starting them with `--userns=host` - ---- - -class: namespaces - -## User Namespaces Caveats - -When user namespaces are enabled, containers cannot: - -- Use the host's network namespace (with `docker run --network=host`) - -- Use the host's PID namespace (with `docker run --pid=host`) - -- Run in privileged mode (with `docker run --privileged`) - -... Unless user namespaces are disabled for the container, with flag `--userns=host` - -External volume and graph drivers that don't support user mapping might not work. - -All containers are currently mapped to the same UID:GID range. - -Some of these limitations might be lifted in the future! - ---- - -class: namespaces - -## Filesystem ownership details - -When enabling user namespaces: - -- the UID:GID on disk (in the images and containers) has to match the *mapped* UID:GID - -- existing images and containers cannot work (their UID:GID would have to be changed) - -For practical reasons, when enabling user namespaces, the Docker Engine places containers and images (and everything else) in a different directory. - -As a resut, if you enable user namespaces on an existing installation: - -- all containers and images (and e.g. Swarm data) disappear - -- *if a node is a member of a Swarm, it is then kicked out of the Swarm* - -- everything will re-appear if you disable user namespaces again - ---- - -class: namespaces - -## Picking a node - -- We will select a node where we will enable user namespaces - -- This node will have to be re-added to the Swarm - -- All containers and services running on this node will be rescheduled - -- Let's make sure that we do not pick the node running the registry! - -.exercise[ - -- Check on which node the registry is running: - ```bash - docker service ps registry - ``` - -] - -Pick any other node (noted `nodeX` in the next slides). - ---- - -class: namespaces - -## Logging into the right Engine - -.exercise[ - -- Log into the right node: - ```bash - ssh node`X` - ``` - -] - ---- - -class: namespaces - -## Configuring the Engine - -.exercise[ - -- Create a configuration file for the Engine: - ```bash - echo '{"userns-remap": "default"}' | sudo tee /etc/docker/daemon.json - ``` - -- Restart the Engine: - ```bash - kill $(pidof dockerd) - ``` - -] - ---- - -class: namespaces - -## Checking that User Namespaces are enabled - -.exercise[ - - Notice the new Docker path: - ```bash - docker info | grep var/lib - ``` - - - Notice the new UID:GID permissions: - ```bash - sudo ls -l /var/lib/docker - ``` - -] - -You should see a line like the following: -``` -drwx------ 11 296608 296608 4096 Aug 3 05:11 296608.296608 -``` - ---- - -class: namespaces - -## Add the node back to the Swarm - -.exercise[ - -- Get our manager token from another node: - ```bash - ssh node`Y` docker swarm join-token manager - ``` - -- Copy-paste the join command to the node - -] - ---- - -class: namespaces - -## Check the new UID:GID - -.exercise[ - -- Run a background container on the node: - ```bash - docker run -d --name lockdown alpine sleep 1000000 - ``` - -- Look at the processes in this container: - ```bash - docker top lockdown - ps faux - ``` - -] - ---- - -class: namespaces - -## Comparing on-disk ownership with/without User Namespaces - -.exercise[ - -- Compare the output of the two following commands: - ```bash - docker run alpine ls -l / - docker run --userns=host alpine ls -l / - ``` - -] +- We cannot have two resources with the same name -- -class: namespaces +- We cannot have two resources *of the same type* with the same name -In the first case, it looks like things belong to `root:root`. - -In the second case, we will see the "real" (on-disk) ownership. + (But it's OK to have a `rng` service, a `rng` deployment, and a `rng` daemon set) -- -class: namespaces +- We cannot have two resources of the same type with the same name *in the same namespace* -Remember to get back to `node1` when finished! + (But it's OK to have e.g. two `rng` services in different namespaces) + +-- + +- In other words: the tuple *(type, name, namespace)* needs to be unique + + (In the resource YAML, the type is called `Kind`) + +--- + +## Pre-existing namespaces + +- If we deploy a cluster with `kubeadm`, we have three namespaces: + + - `default` (for our applications) + + - `kube-system` (for the control plane) + + - `kube-public` (contains one secret used for cluster discovery) + +- If we deploy differently, we may have different namespaces + +--- + +## Creating namespaces + +- We can create namespaces with a very minimal YAML, e.g.: + ```bash + kubectl apply -f- < +It is mediated by capabilities, cgroups, namespaces, seccomp, LSMs... + +--- + +class: namespaces + +## User Namespaces in Docker + +- Optional feature added in Docker Engine 1.10 + +- Not enabled by default + +- Has to be enabled at Engine startup, and affects all containers + +- When enabled, `UID:GID` in containers are mapped to a different range on the host + +- Safer than switching to a non-root user (with `-u` or `USER`) in the container +
+ (Since with user namespaces, root escalation maps to a non-privileged user) + +- Can be selectively disabled per container by starting them with `--userns=host` + +--- + +class: namespaces + +## User Namespaces Caveats + +When user namespaces are enabled, containers cannot: + +- Use the host's network namespace (with `docker run --network=host`) + +- Use the host's PID namespace (with `docker run --pid=host`) + +- Run in privileged mode (with `docker run --privileged`) + +... Unless user namespaces are disabled for the container, with flag `--userns=host` + +External volume and graph drivers that don't support user mapping might not work. + +All containers are currently mapped to the same UID:GID range. + +Some of these limitations might be lifted in the future! + +--- + +class: namespaces + +## Filesystem ownership details + +When enabling user namespaces: + +- the UID:GID on disk (in the images and containers) has to match the *mapped* UID:GID + +- existing images and containers cannot work (their UID:GID would have to be changed) + +For practical reasons, when enabling user namespaces, the Docker Engine places containers and images (and everything else) in a different directory. + +As a resut, if you enable user namespaces on an existing installation: + +- all containers and images (and e.g. Swarm data) disappear + +- *if a node is a member of a Swarm, it is then kicked out of the Swarm* + +- everything will re-appear if you disable user namespaces again + +--- + +class: namespaces + +## Picking a node + +- We will select a node where we will enable user namespaces + +- This node will have to be re-added to the Swarm + +- All containers and services running on this node will be rescheduled + +- Let's make sure that we do not pick the node running the registry! + +.exercise[ + +- Check on which node the registry is running: + ```bash + docker service ps registry + ``` + +] + +Pick any other node (noted `nodeX` in the next slides). + +--- + +class: namespaces + +## Logging into the right Engine + +.exercise[ + +- Log into the right node: + ```bash + ssh node`X` + ``` + +] + +--- + +class: namespaces + +## Configuring the Engine + +.exercise[ + +- Create a configuration file for the Engine: + ```bash + echo '{"userns-remap": "default"}' | sudo tee /etc/docker/daemon.json + ``` + +- Restart the Engine: + ```bash + kill $(pidof dockerd) + ``` + +] + +--- + +class: namespaces + +## Checking that User Namespaces are enabled + +.exercise[ + - Notice the new Docker path: + ```bash + docker info | grep var/lib + ``` + + - Notice the new UID:GID permissions: + ```bash + sudo ls -l /var/lib/docker + ``` + +] + +You should see a line like the following: +``` +drwx------ 11 296608 296608 4096 Aug 3 05:11 296608.296608 +``` + +--- + +class: namespaces + +## Add the node back to the Swarm + +.exercise[ + +- Get our manager token from another node: + ```bash + ssh node`Y` docker swarm join-token manager + ``` + +- Copy-paste the join command to the node + +] + +--- + +class: namespaces + +## Check the new UID:GID + +.exercise[ + +- Run a background container on the node: + ```bash + docker run -d --name lockdown alpine sleep 1000000 + ``` + +- Look at the processes in this container: + ```bash + docker top lockdown + ps faux + ``` + +] + +--- + +class: namespaces + +## Comparing on-disk ownership with/without User Namespaces + +.exercise[ + +- Compare the output of the two following commands: + ```bash + docker run alpine ls -l / + docker run --userns=host alpine ls -l / + ``` + +] + +-- + +class: namespaces + +In the first case, it looks like things belong to `root:root`. + +In the second case, we will see the "real" (on-disk) ownership. + +-- + +class: namespaces + +Remember to get back to `node1` when finished!