mirror of
https://github.com/jpetazzo/container.training.git
synced 2026-07-21 22:07:13 +00:00
Add two chapters: Helm and namespaces
In these chapters, we: - show how to install Helm - run the Helm tiller on our cluster - use Helm to install Prometheus - don't do anything fancy with Prometheus (it's just for the sake of installing something) - create a basic Helm chart for DockerCoins - explain namespace concepts - show how to use contexts to hop between namespaces - use Helm to deploy DockerCoins to a new namespace These two chapters go together.
This commit is contained in:
Jerome Petazzoni
205
slides/kube/helm.md
Normal file
205
slides/kube/helm.md
Normal file
@@ -0,0 +1,205 @@
|
||||
# Managing stacks with Helm
|
||||
|
||||
- We created our first resources with `kubectl run`, `kubectl expose` ...
|
||||
|
||||
- We have also created resources by loading YAML files with `kubectl apply -f`
|
||||
|
||||
- For larger stacks, managing thousands of lines of YAML is unreasonable
|
||||
|
||||
- These YAML bundles need to be customized with variable parameters
|
||||
|
||||
(E.g.: number of replicas, image version to use ...)
|
||||
|
||||
- It would be nice to have an organized, versioned collection of bundles
|
||||
|
||||
- It would be nice to be able to upgrade/rollback these bundles carefully
|
||||
|
||||
- Helm offers all these things!
|
||||
|
||||
---
|
||||
|
||||
## Helm concepts
|
||||
|
||||
- `helm` is a CLI tool
|
||||
|
||||
- `tiller` is its companion server-side component
|
||||
|
||||
- A "chart" is an archive containing templatized YAML bundles
|
||||
|
||||
- Charts are versioned
|
||||
|
||||
- Charts can be stored on private or public repositories
|
||||
|
||||
---
|
||||
|
||||
## Installing Helm
|
||||
|
||||
- We need to install the `helm` CLI; then use it to deploy `tiller`
|
||||
|
||||
.exercise[
|
||||
|
||||
- Install the `helm` CLI:
|
||||
```bash
|
||||
curl https://raw.githubusercontent.com/kubernetes/helm/master/scripts/get | bash
|
||||
```
|
||||
|
||||
- Deploy `tiller`:
|
||||
```bash
|
||||
helm init
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
---
|
||||
|
||||
## Fix account permissions
|
||||
|
||||
- Helm permission model requires use to tweak permissions
|
||||
|
||||
- In a more realistic deployment, you might create per-user or per-team
|
||||
service accounts, roles, and role bindings
|
||||
|
||||
.exercise[
|
||||
|
||||
- Grant `cluster-admin` role to `kube-system:default` service account:
|
||||
```bash
|
||||
kubectl create clusterrolebinding add-on-cluster-admin \
|
||||
--clusterrole=cluster-admin --serviceaccount=kube-system:default
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
(Defining the exact roles and permissions on your cluster requires
|
||||
a deeper knowledge of Kubernetes' RBAC model. The command above is
|
||||
fine for personal and development clusters.)
|
||||
|
||||
---
|
||||
|
||||
## View available charts
|
||||
|
||||
- A public repo is pre-configurd when installing Helm
|
||||
|
||||
- We can view available charts with `helm search` (and an optional keyword)
|
||||
|
||||
.exercise[
|
||||
|
||||
- View all available charts:
|
||||
```bash
|
||||
helm search
|
||||
```
|
||||
|
||||
- View charts related to `gitlab`:
|
||||
```bash
|
||||
helm search gitlab
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
---
|
||||
|
||||
## Install a chart
|
||||
|
||||
- Most charts use `LoadBalancer` service types by default
|
||||
|
||||
- Most charts require persistent volumes to store data
|
||||
|
||||
- We need to relax these requirements a bit
|
||||
|
||||
.exercise[
|
||||
|
||||
- Install the Prometheus metrics collector on our cluster:
|
||||
```bash
|
||||
helm install stable/prometheus \
|
||||
--set server.service.type=NodePort \
|
||||
--set server.persistentVolume.enabled=false
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
Where do these `--set` options come from?
|
||||
|
||||
---
|
||||
|
||||
## Inspecting a chart
|
||||
|
||||
- `helm inspect` shows details about a chart (including available options)
|
||||
|
||||
.exercise[
|
||||
|
||||
- See the metadata and all available options for `stable/prometheus`:
|
||||
```bash
|
||||
helm inspect stable/prometheus
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
The chart's metadata includes an URL to the project's home page.
|
||||
|
||||
(Sometimes it conveniently points to the documentation for the chart.)
|
||||
|
||||
---
|
||||
|
||||
## Creating a chart
|
||||
|
||||
- We are going to show a way to create a *very simplified* chart
|
||||
|
||||
- In a real chart, *lots of things* would be templatized
|
||||
|
||||
(Resource names, service types, number of replicas...)
|
||||
|
||||
.exercise[
|
||||
|
||||
- Create a sample chart:
|
||||
```bash
|
||||
helm create dockercoins
|
||||
```
|
||||
|
||||
- Move away the sample templates:
|
||||
```bash
|
||||
mkdir dockercoins/do-not-use
|
||||
mv dockercoins/templates dockercoins/default-templates
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
---
|
||||
|
||||
## Exporting the YAML for our application
|
||||
|
||||
- The following section assumes that DockerCoins is currently running
|
||||
|
||||
.exercise[
|
||||
|
||||
- Create one YAML file for each resource that we need:
|
||||
.small[
|
||||
```bash
|
||||
|
||||
while read kind name; do
|
||||
kubectl get -o yaml --export $kind $name > dockercoins/templates/$name-$kind.yaml
|
||||
done <<EOF
|
||||
deployment worker
|
||||
deployment hasher
|
||||
daemonset rng
|
||||
deployment webui
|
||||
deployment redis
|
||||
service hasher
|
||||
service rng
|
||||
service webui
|
||||
service redis
|
||||
EOF
|
||||
```
|
||||
]
|
||||
|
||||
]
|
||||
|
||||
---
|
||||
|
||||
## Testing our chart
|
||||
|
||||
- We can now install our chart with `helm install dockercoins`
|
||||
|
||||
(In that case, `dockercoins` is the path to the chart)
|
||||
|
||||
- However, since the application is already deployed, this will fail
|
||||
|
||||
- To avoid naming conflicts, we will deploy the application in another *namespace*
|
||||
@@ -1,236 +1,133 @@
|
||||
class: namespaces
|
||||
name: namespaces
|
||||
# Namespaces
|
||||
|
||||
# Improving isolation with User Namespaces
|
||||
|
||||
- *Namespaces* are kernel mechanisms to compartimetalize the system
|
||||
|
||||
- There are different kind of namespaces: `pid`, `net`, `mnt`, `ipc`, `uts`, and `user`
|
||||
|
||||
- For a primer, see "Anatomy of a Container"
|
||||
([video](https://www.youtube.com/watch?v=sK5i-N34im8))
|
||||
([slides](https://www.slideshare.net/jpetazzo/cgroups-namespaces-and-beyond-what-are-containers-made-from-dockercon-europe-2015))
|
||||
|
||||
- The *user namespace* allows to map UIDs between the containers and the host
|
||||
|
||||
- As a result, `root` in a container can map to a non-privileged user on the host
|
||||
|
||||
Note: even without user namespaces, `root` in a container cannot go wild on the host.
|
||||
<br/>
|
||||
It is mediated by capabilities, cgroups, namespaces, seccomp, LSMs...
|
||||
|
||||
---
|
||||
|
||||
class: namespaces
|
||||
|
||||
## User Namespaces in Docker
|
||||
|
||||
- Optional feature added in Docker Engine 1.10
|
||||
|
||||
- Not enabled by default
|
||||
|
||||
- Has to be enabled at Engine startup, and affects all containers
|
||||
|
||||
- When enabled, `UID:GID` in containers are mapped to a different range on the host
|
||||
|
||||
- Safer than switching to a non-root user (with `-u` or `USER`) in the container
|
||||
<br/>
|
||||
(Since with user namespaces, root escalation maps to a non-privileged user)
|
||||
|
||||
- Can be selectively disabled per container by starting them with `--userns=host`
|
||||
|
||||
---
|
||||
|
||||
class: namespaces
|
||||
|
||||
## User Namespaces Caveats
|
||||
|
||||
When user namespaces are enabled, containers cannot:
|
||||
|
||||
- Use the host's network namespace (with `docker run --network=host`)
|
||||
|
||||
- Use the host's PID namespace (with `docker run --pid=host`)
|
||||
|
||||
- Run in privileged mode (with `docker run --privileged`)
|
||||
|
||||
... Unless user namespaces are disabled for the container, with flag `--userns=host`
|
||||
|
||||
External volume and graph drivers that don't support user mapping might not work.
|
||||
|
||||
All containers are currently mapped to the same UID:GID range.
|
||||
|
||||
Some of these limitations might be lifted in the future!
|
||||
|
||||
---
|
||||
|
||||
class: namespaces
|
||||
|
||||
## Filesystem ownership details
|
||||
|
||||
When enabling user namespaces:
|
||||
|
||||
- the UID:GID on disk (in the images and containers) has to match the *mapped* UID:GID
|
||||
|
||||
- existing images and containers cannot work (their UID:GID would have to be changed)
|
||||
|
||||
For practical reasons, when enabling user namespaces, the Docker Engine places containers and images (and everything else) in a different directory.
|
||||
|
||||
As a resut, if you enable user namespaces on an existing installation:
|
||||
|
||||
- all containers and images (and e.g. Swarm data) disappear
|
||||
|
||||
- *if a node is a member of a Swarm, it is then kicked out of the Swarm*
|
||||
|
||||
- everything will re-appear if you disable user namespaces again
|
||||
|
||||
---
|
||||
|
||||
class: namespaces
|
||||
|
||||
## Picking a node
|
||||
|
||||
- We will select a node where we will enable user namespaces
|
||||
|
||||
- This node will have to be re-added to the Swarm
|
||||
|
||||
- All containers and services running on this node will be rescheduled
|
||||
|
||||
- Let's make sure that we do not pick the node running the registry!
|
||||
|
||||
.exercise[
|
||||
|
||||
- Check on which node the registry is running:
|
||||
```bash
|
||||
docker service ps registry
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
Pick any other node (noted `nodeX` in the next slides).
|
||||
|
||||
---
|
||||
|
||||
class: namespaces
|
||||
|
||||
## Logging into the right Engine
|
||||
|
||||
.exercise[
|
||||
|
||||
- Log into the right node:
|
||||
```bash
|
||||
ssh node`X`
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
---
|
||||
|
||||
class: namespaces
|
||||
|
||||
## Configuring the Engine
|
||||
|
||||
.exercise[
|
||||
|
||||
- Create a configuration file for the Engine:
|
||||
```bash
|
||||
echo '{"userns-remap": "default"}' | sudo tee /etc/docker/daemon.json
|
||||
```
|
||||
|
||||
- Restart the Engine:
|
||||
```bash
|
||||
kill $(pidof dockerd)
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
---
|
||||
|
||||
class: namespaces
|
||||
|
||||
## Checking that User Namespaces are enabled
|
||||
|
||||
.exercise[
|
||||
- Notice the new Docker path:
|
||||
```bash
|
||||
docker info | grep var/lib
|
||||
```
|
||||
|
||||
- Notice the new UID:GID permissions:
|
||||
```bash
|
||||
sudo ls -l /var/lib/docker
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
You should see a line like the following:
|
||||
```
|
||||
drwx------ 11 296608 296608 4096 Aug 3 05:11 296608.296608
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
class: namespaces
|
||||
|
||||
## Add the node back to the Swarm
|
||||
|
||||
.exercise[
|
||||
|
||||
- Get our manager token from another node:
|
||||
```bash
|
||||
ssh node`Y` docker swarm join-token manager
|
||||
```
|
||||
|
||||
- Copy-paste the join command to the node
|
||||
|
||||
]
|
||||
|
||||
---
|
||||
|
||||
class: namespaces
|
||||
|
||||
## Check the new UID:GID
|
||||
|
||||
.exercise[
|
||||
|
||||
- Run a background container on the node:
|
||||
```bash
|
||||
docker run -d --name lockdown alpine sleep 1000000
|
||||
```
|
||||
|
||||
- Look at the processes in this container:
|
||||
```bash
|
||||
docker top lockdown
|
||||
ps faux
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
---
|
||||
|
||||
class: namespaces
|
||||
|
||||
## Comparing on-disk ownership with/without User Namespaces
|
||||
|
||||
.exercise[
|
||||
|
||||
- Compare the output of the two following commands:
|
||||
```bash
|
||||
docker run alpine ls -l /
|
||||
docker run --userns=host alpine ls -l /
|
||||
```
|
||||
|
||||
]
|
||||
- We cannot have two resources with the same name
|
||||
|
||||
--
|
||||
|
||||
class: namespaces
|
||||
- We cannot have two resources *of the same type* with the same name
|
||||
|
||||
In the first case, it looks like things belong to `root:root`.
|
||||
|
||||
In the second case, we will see the "real" (on-disk) ownership.
|
||||
(But it's OK to have a `rng` service, a `rng` deployment, and a `rng` daemon set)
|
||||
|
||||
--
|
||||
|
||||
class: namespaces
|
||||
- We cannot have two resources of the same type with the same name *in the same namespace*
|
||||
|
||||
Remember to get back to `node1` when finished!
|
||||
(But it's OK to have e.g. two `rng` services in different namespaces)
|
||||
|
||||
--
|
||||
|
||||
- In other words: the tuple *(type, name, namespace)* needs to be unique
|
||||
|
||||
(In the resource YAML, the type is called `Kind`)
|
||||
|
||||
---
|
||||
|
||||
## Pre-existing namespaces
|
||||
|
||||
- If we deploy a cluster with `kubeadm`, we have three namespaces:
|
||||
|
||||
- `default` (for our applications)
|
||||
|
||||
- `kube-system` (for the control plane)
|
||||
|
||||
- `kube-public` (contains one secret used for cluster discovery)
|
||||
|
||||
- If we deploy differently, we may have different namespaces
|
||||
|
||||
---
|
||||
|
||||
## Creating namespaces
|
||||
|
||||
- We can create namespaces with a very minimal YAML, e.g.:
|
||||
```bash
|
||||
kubectl apply -f- <<EOF
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: blue
|
||||
EOF
|
||||
```
|
||||
|
||||
- If we are using a tool like Helm, it will create namespaces automatically
|
||||
|
||||
---
|
||||
|
||||
## Using namespaces
|
||||
|
||||
- We can pass a `-n` or `--namespace` flag to most `kubectl` commands:
|
||||
```bash
|
||||
kubectl -n blue get svc
|
||||
```
|
||||
|
||||
- We can also use *contexts*
|
||||
|
||||
- A context is a *(user, cluster, namespace)* tuple
|
||||
|
||||
- We can manipulate contexts with the `kubectl config` command
|
||||
|
||||
---
|
||||
|
||||
## Creating a context
|
||||
|
||||
- We are going to create a context for the `blue` namespace
|
||||
|
||||
.exercise[
|
||||
|
||||
- View existing contexts to see the cluster name and the current user:
|
||||
```bash
|
||||
kubectl config get-contexts
|
||||
```
|
||||
|
||||
- Create a new context:
|
||||
```bash
|
||||
kubectl config set-context blue --namespace=blue \
|
||||
--cluster=kubernetes --user=kubernetes-admin
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
We have created a context; but this is just some configuration values.
|
||||
|
||||
The namespace doesn't exist yet.
|
||||
|
||||
---
|
||||
|
||||
## Using a context
|
||||
|
||||
- Let's switch to our new context and deploy the DockerCoins chart
|
||||
|
||||
.exercise[
|
||||
|
||||
- Use the `blue` context:
|
||||
```bash
|
||||
kubectl config use-context blue
|
||||
```
|
||||
|
||||
- Deploy DockerCoins:
|
||||
```bash
|
||||
helm install dockercoins
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
In the last command line, `dockercoins` is just the local path where
|
||||
we created our Helm chart before.
|
||||
|
||||
---
|
||||
|
||||
## Viewing the deployed app
|
||||
|
||||
- Let's see if our Helm chart worked correctly!
|
||||
|
||||
.exercise[
|
||||
|
||||
- Retrieve the port number allocated to the `webui` service:
|
||||
```bash
|
||||
kubectl get svc webui
|
||||
```
|
||||
|
||||
- Point our browser to http://X.X.X.X:3xxxx
|
||||
|
||||
]
|
||||
|
||||
Note: it might take a minute or two for the app to be up and running.
|
||||
|
||||
236
slides/swarm/namespaces.md
Normal file
236
slides/swarm/namespaces.md
Normal file
@@ -0,0 +1,236 @@
|
||||
class: namespaces
|
||||
name: namespaces
|
||||
|
||||
# Improving isolation with User Namespaces
|
||||
|
||||
- *Namespaces* are kernel mechanisms to compartimetalize the system
|
||||
|
||||
- There are different kind of namespaces: `pid`, `net`, `mnt`, `ipc`, `uts`, and `user`
|
||||
|
||||
- For a primer, see "Anatomy of a Container"
|
||||
([video](https://www.youtube.com/watch?v=sK5i-N34im8))
|
||||
([slides](https://www.slideshare.net/jpetazzo/cgroups-namespaces-and-beyond-what-are-containers-made-from-dockercon-europe-2015))
|
||||
|
||||
- The *user namespace* allows to map UIDs between the containers and the host
|
||||
|
||||
- As a result, `root` in a container can map to a non-privileged user on the host
|
||||
|
||||
Note: even without user namespaces, `root` in a container cannot go wild on the host.
|
||||
<br/>
|
||||
It is mediated by capabilities, cgroups, namespaces, seccomp, LSMs...
|
||||
|
||||
---
|
||||
|
||||
class: namespaces
|
||||
|
||||
## User Namespaces in Docker
|
||||
|
||||
- Optional feature added in Docker Engine 1.10
|
||||
|
||||
- Not enabled by default
|
||||
|
||||
- Has to be enabled at Engine startup, and affects all containers
|
||||
|
||||
- When enabled, `UID:GID` in containers are mapped to a different range on the host
|
||||
|
||||
- Safer than switching to a non-root user (with `-u` or `USER`) in the container
|
||||
<br/>
|
||||
(Since with user namespaces, root escalation maps to a non-privileged user)
|
||||
|
||||
- Can be selectively disabled per container by starting them with `--userns=host`
|
||||
|
||||
---
|
||||
|
||||
class: namespaces
|
||||
|
||||
## User Namespaces Caveats
|
||||
|
||||
When user namespaces are enabled, containers cannot:
|
||||
|
||||
- Use the host's network namespace (with `docker run --network=host`)
|
||||
|
||||
- Use the host's PID namespace (with `docker run --pid=host`)
|
||||
|
||||
- Run in privileged mode (with `docker run --privileged`)
|
||||
|
||||
... Unless user namespaces are disabled for the container, with flag `--userns=host`
|
||||
|
||||
External volume and graph drivers that don't support user mapping might not work.
|
||||
|
||||
All containers are currently mapped to the same UID:GID range.
|
||||
|
||||
Some of these limitations might be lifted in the future!
|
||||
|
||||
---
|
||||
|
||||
class: namespaces
|
||||
|
||||
## Filesystem ownership details
|
||||
|
||||
When enabling user namespaces:
|
||||
|
||||
- the UID:GID on disk (in the images and containers) has to match the *mapped* UID:GID
|
||||
|
||||
- existing images and containers cannot work (their UID:GID would have to be changed)
|
||||
|
||||
For practical reasons, when enabling user namespaces, the Docker Engine places containers and images (and everything else) in a different directory.
|
||||
|
||||
As a resut, if you enable user namespaces on an existing installation:
|
||||
|
||||
- all containers and images (and e.g. Swarm data) disappear
|
||||
|
||||
- *if a node is a member of a Swarm, it is then kicked out of the Swarm*
|
||||
|
||||
- everything will re-appear if you disable user namespaces again
|
||||
|
||||
---
|
||||
|
||||
class: namespaces
|
||||
|
||||
## Picking a node
|
||||
|
||||
- We will select a node where we will enable user namespaces
|
||||
|
||||
- This node will have to be re-added to the Swarm
|
||||
|
||||
- All containers and services running on this node will be rescheduled
|
||||
|
||||
- Let's make sure that we do not pick the node running the registry!
|
||||
|
||||
.exercise[
|
||||
|
||||
- Check on which node the registry is running:
|
||||
```bash
|
||||
docker service ps registry
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
Pick any other node (noted `nodeX` in the next slides).
|
||||
|
||||
---
|
||||
|
||||
class: namespaces
|
||||
|
||||
## Logging into the right Engine
|
||||
|
||||
.exercise[
|
||||
|
||||
- Log into the right node:
|
||||
```bash
|
||||
ssh node`X`
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
---
|
||||
|
||||
class: namespaces
|
||||
|
||||
## Configuring the Engine
|
||||
|
||||
.exercise[
|
||||
|
||||
- Create a configuration file for the Engine:
|
||||
```bash
|
||||
echo '{"userns-remap": "default"}' | sudo tee /etc/docker/daemon.json
|
||||
```
|
||||
|
||||
- Restart the Engine:
|
||||
```bash
|
||||
kill $(pidof dockerd)
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
---
|
||||
|
||||
class: namespaces
|
||||
|
||||
## Checking that User Namespaces are enabled
|
||||
|
||||
.exercise[
|
||||
- Notice the new Docker path:
|
||||
```bash
|
||||
docker info | grep var/lib
|
||||
```
|
||||
|
||||
- Notice the new UID:GID permissions:
|
||||
```bash
|
||||
sudo ls -l /var/lib/docker
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
You should see a line like the following:
|
||||
```
|
||||
drwx------ 11 296608 296608 4096 Aug 3 05:11 296608.296608
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
class: namespaces
|
||||
|
||||
## Add the node back to the Swarm
|
||||
|
||||
.exercise[
|
||||
|
||||
- Get our manager token from another node:
|
||||
```bash
|
||||
ssh node`Y` docker swarm join-token manager
|
||||
```
|
||||
|
||||
- Copy-paste the join command to the node
|
||||
|
||||
]
|
||||
|
||||
---
|
||||
|
||||
class: namespaces
|
||||
|
||||
## Check the new UID:GID
|
||||
|
||||
.exercise[
|
||||
|
||||
- Run a background container on the node:
|
||||
```bash
|
||||
docker run -d --name lockdown alpine sleep 1000000
|
||||
```
|
||||
|
||||
- Look at the processes in this container:
|
||||
```bash
|
||||
docker top lockdown
|
||||
ps faux
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
---
|
||||
|
||||
class: namespaces
|
||||
|
||||
## Comparing on-disk ownership with/without User Namespaces
|
||||
|
||||
.exercise[
|
||||
|
||||
- Compare the output of the two following commands:
|
||||
```bash
|
||||
docker run alpine ls -l /
|
||||
docker run --userns=host alpine ls -l /
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
--
|
||||
|
||||
class: namespaces
|
||||
|
||||
In the first case, it looks like things belong to `root:root`.
|
||||
|
||||
In the second case, we will see the "real" (on-disk) ownership.
|
||||
|
||||
--
|
||||
|
||||
class: namespaces
|
||||
|
||||
Remember to get back to `node1` when finished!
|
||||
Reference in New Issue
Block a user