mirror of
https://github.com/jpetazzo/container.training.git
synced 2026-07-28 17:21:11 +00:00
Tons of new chapters! Excitement!
- volumes (general overview) - building with the docker engine (bind-mounting the docker socket) - building with kaniko (and init containers) - managing configuration (configmaps, downward api) Also added a new-content.yml file with just the new content (for easier review), containing my plans for future chapters.
This commit is contained in:
28
k8s/docker-build.yaml
Normal file
28
k8s/docker-build.yaml
Normal file
@@ -0,0 +1,28 @@
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: build-image
|
||||
spec:
|
||||
restartPolicy: OnFailure
|
||||
containers:
|
||||
- name: docker-build
|
||||
image: docker
|
||||
env:
|
||||
- name: REGISTRY_PORT
|
||||
value: #"30000"
|
||||
command: ["sh", "-c"]
|
||||
args:
|
||||
- |
|
||||
apk add --no-cache git &&
|
||||
mkdir /workspace &&
|
||||
git clone https://github.com/jpetazzo/container.training /workspace &&
|
||||
docker build -t localhost:$REGISTRY_PORT/worker /workspace/dockercoins/worker &&
|
||||
docker push localhost:$REGISTRY_PORT/worker
|
||||
volumeMounts:
|
||||
- name: docker-socket
|
||||
mountPath: /var/run/docker.sock
|
||||
volumes:
|
||||
- name: docker-socket
|
||||
hostPath:
|
||||
path: /var/run/docker.sock
|
||||
|
||||
18
k8s/haproxy.cfg
Normal file
18
k8s/haproxy.cfg
Normal file
@@ -0,0 +1,18 @@
|
||||
global
|
||||
daemon
|
||||
maxconn 256
|
||||
|
||||
defaults
|
||||
mode tcp
|
||||
timeout connect 5000ms
|
||||
timeout client 50000ms
|
||||
timeout server 50000ms
|
||||
|
||||
frontend the-frontend
|
||||
bind *:80
|
||||
default_backend the-backend
|
||||
|
||||
backend the-backend
|
||||
server google.com-80 google.com:80 maxconn 32 check
|
||||
server bing.com-80 bing.com:80 maxconn 32 check
|
||||
|
||||
16
k8s/haproxy.yaml
Normal file
16
k8s/haproxy.yaml
Normal file
@@ -0,0 +1,16 @@
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: haproxy
|
||||
spec:
|
||||
volumes:
|
||||
- name: config
|
||||
configMap:
|
||||
name: haproxy
|
||||
containers:
|
||||
- name: haproxy
|
||||
image: haproxy
|
||||
volumeMounts:
|
||||
- name: config
|
||||
mountPath: /usr/local/etc/haproxy/
|
||||
|
||||
29
k8s/kaniko-build.yaml
Normal file
29
k8s/kaniko-build.yaml
Normal file
@@ -0,0 +1,29 @@
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: kaniko-build
|
||||
spec:
|
||||
initContainers:
|
||||
- name: git-clone
|
||||
image: alpine
|
||||
command: ["sh", "-c"]
|
||||
args:
|
||||
- |
|
||||
apk add --no-cache git &&
|
||||
git clone git://github.com/jpetazzo/container.training /workspace
|
||||
volumeMounts:
|
||||
- name: workspace
|
||||
mountPath: /workspace
|
||||
containers:
|
||||
- name: build-image
|
||||
image: gcr.io/kaniko-project/executor:latest
|
||||
args:
|
||||
- "--context=/workspace/dockercoins/rng"
|
||||
- "--insecure-skip-tls-verify"
|
||||
- "--destination=registry:5000/rng-kaniko:latest"
|
||||
volumeMounts:
|
||||
- name: workspace
|
||||
mountPath: /workspace
|
||||
volumes:
|
||||
- name: workspace
|
||||
|
||||
21
k8s/nginx-with-volume.yaml
Normal file
21
k8s/nginx-with-volume.yaml
Normal file
@@ -0,0 +1,21 @@
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: nginx-with-volume
|
||||
spec:
|
||||
volumes:
|
||||
- name: www
|
||||
containers:
|
||||
- name: nginx
|
||||
image: nginx
|
||||
volumeMounts:
|
||||
- name: www
|
||||
mountPath: /usr/share/nginx/html/
|
||||
- name: git
|
||||
image: alpine
|
||||
command: [ "sh", "-c", "apk add --no-cache git && git clone https://github.com/octocat/Spoon-Knife /www" ]
|
||||
volumeMounts:
|
||||
- name: www
|
||||
mountPath: /www/
|
||||
restartPolicy: OnFailure
|
||||
|
||||
15
k8s/registry.yaml
Normal file
15
k8s/registry.yaml
Normal file
@@ -0,0 +1,15 @@
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: registry
|
||||
spec:
|
||||
containers:
|
||||
- name: registry
|
||||
image: registry
|
||||
env:
|
||||
- name: REGISTRY_HTTP_ADDR
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: registry
|
||||
key: http.addr
|
||||
|
||||
156
slides/k8s/build-with-docker.md
Normal file
156
slides/k8s/build-with-docker.md
Normal file
@@ -0,0 +1,156 @@
|
||||
# Building images with the Docker Engine
|
||||
|
||||
- Until now, we have built our images manually, directly on a node
|
||||
|
||||
- We are going to show how to build images from within the cluster
|
||||
|
||||
(by executing code in a container controlled by Kubernetes)
|
||||
|
||||
- We are going to use the Docker Engine for that purpose
|
||||
|
||||
- To access the Docker Engine, we will mount the Docker socket in our container
|
||||
|
||||
- After building the image, we will push it to our self-hosted registry
|
||||
|
||||
---
|
||||
|
||||
## Resource specification for our builder pod
|
||||
|
||||
.small[
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: build-image
|
||||
spec:
|
||||
restartPolicy: OnFailure
|
||||
containers:
|
||||
- name: docker-build
|
||||
image: docker
|
||||
env:
|
||||
- name: REGISTRY_PORT
|
||||
value: "`3XXXX`"
|
||||
command: ["sh", "-c"]
|
||||
args:
|
||||
- |
|
||||
apk add --no-cache git &&
|
||||
mkdir /workspace &&
|
||||
git clone https://github.com/jpetazzo/container.training /workspace &&
|
||||
docker build -t localhost:$REGISTRY_PORT/worker /workspace/dockercoins/worker &&
|
||||
docker push localhost:$REGISTRY_PORT/worker
|
||||
volumeMounts:
|
||||
- name: docker-socket
|
||||
mountPath: /var/run/docker.sock
|
||||
volumes:
|
||||
- name: docker-socket
|
||||
hostPath:
|
||||
path: /var/run/docker.sock
|
||||
```
|
||||
]
|
||||
|
||||
---
|
||||
|
||||
## Breaking down the pod specification (1/2)
|
||||
|
||||
- `restartPolicy: OnFailure` prevents the build from running in an infinite lopo
|
||||
|
||||
- We use the `docker` image (so that the `docker` CLI is available)
|
||||
|
||||
- We rely on the fact that the `docker` image is based on `alpine`
|
||||
|
||||
(which is why we use `apk` to install `git`)
|
||||
|
||||
- The port for the registry is passed through an environment variable
|
||||
|
||||
(this avoids to repeat it in the specification, which would be error-prone)
|
||||
|
||||
.warning[The environment variable has to be a string, so the `"` are mandatory!]
|
||||
|
||||
---
|
||||
|
||||
## Breaking down the pod specification (2/2)
|
||||
|
||||
- The volume `docker-socket` is declared with a `hostPath`, indicating a bind-mount
|
||||
|
||||
- It is then mounted in the container onto the default Docker socket path
|
||||
|
||||
- We show a creative way to specify the commands to run in the container:
|
||||
|
||||
- the command executed will be `sh -c <args>`
|
||||
|
||||
- `args` is a list of strings
|
||||
|
||||
- `|` is used to pass a multi-line string in the YAML file
|
||||
|
||||
---
|
||||
|
||||
## Running our pod
|
||||
|
||||
- Let's try this out!
|
||||
|
||||
.exercise[
|
||||
|
||||
- Check the port used by our self-hosted registry:
|
||||
```bash
|
||||
kubectl get svc registry
|
||||
```
|
||||
|
||||
- Edit `~/container.training/k8s/docker-build.yaml` to put the port number
|
||||
|
||||
- Schedule the pod by applying the resource file:
|
||||
```bash
|
||||
kubectl apply -f ~/container.training/k8s/docker-build.yaml
|
||||
```
|
||||
|
||||
- Watch the logs:
|
||||
```bash
|
||||
stern build-image
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
---
|
||||
|
||||
## What's missing?
|
||||
|
||||
What do we need to change to make this production-ready?
|
||||
|
||||
- Build from a long-running container (e.g. a `Deployment`) triggered by web hooks
|
||||
|
||||
(the payload of the web hook could indicate the repository to build)
|
||||
|
||||
- Build a specific branch or tag; tag image accordingly
|
||||
|
||||
- Handle repositories where the Dockerfile is not at the root
|
||||
|
||||
(or containing multiple Dockerfiles)
|
||||
|
||||
- Expose build logs so that troubleshooting is straightforward
|
||||
|
||||
--
|
||||
|
||||
🤔 That seems like a lot of work!
|
||||
|
||||
--
|
||||
|
||||
That's why systems like Docker Hub are helpful.
|
||||
<br/>
|
||||
They handle the whole "code repository → Docker image" workflow.
|
||||
|
||||
---
|
||||
|
||||
## Things to be aware of
|
||||
|
||||
- This is talking directly to a node's Docker Engine to build images
|
||||
|
||||
- It bypasses ressource allocation mechanisms used by Kubernetes
|
||||
|
||||
(but you can use *taints* and *tolerations* to dedicate builder nodes)
|
||||
|
||||
- Be careful not to introduce conflicts when naming images
|
||||
|
||||
(e.g. do not allow the user to specify the image names!)
|
||||
|
||||
- Your builds are going to be *fast*
|
||||
|
||||
(because they will leverage Docker's caching system)
|
||||
193
slides/k8s/build-with-kaniko.md
Normal file
193
slides/k8s/build-with-kaniko.md
Normal file
@@ -0,0 +1,193 @@
|
||||
# Building images with Kaniko
|
||||
|
||||
- [Kaniko](https://github.com/GoogleContainerTools/kaniko) is an open source tool to build container images within Kubernetes
|
||||
|
||||
- It can build an image using any standard Dockerfile
|
||||
|
||||
- The resulting image can be pushed to a registry or exported as a tarball
|
||||
|
||||
- It doesn't require any particular privilege
|
||||
|
||||
(and can therefore run in a regular container in a regular pod)
|
||||
|
||||
- This combination of features is pretty unique
|
||||
|
||||
(most other tools use different formats, or require elevated privileges)
|
||||
|
||||
---
|
||||
|
||||
## Kaniko in practice
|
||||
|
||||
- Kaniko provides an "executor image", `gcr.io/kaniko-project/executor`
|
||||
|
||||
- When running that image, we need to specify at least:
|
||||
|
||||
- the path to the build context (=the directory with our Dockerfile)
|
||||
|
||||
- the target image name (including the registry address)
|
||||
|
||||
- Simplified example:
|
||||
```
|
||||
docker run \
|
||||
-v ...:/workspace gcr.io/kaniko-project/executor \
|
||||
--context=/workspace \
|
||||
--destination=registry:5000/image_name:image_tag
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Running Kaniko in a Docker container
|
||||
|
||||
- Let's build the image for the DockerCoins `worker` service with Kaniko
|
||||
|
||||
.exercise[
|
||||
|
||||
- Find the port number for our self-hosted registry:
|
||||
```bash
|
||||
kubectl get svc registry
|
||||
PORT=$(kubectl get svc registry -o json | jq .spec.ports[0].nodePort)
|
||||
```
|
||||
|
||||
- Run Kaniko:
|
||||
```bash
|
||||
docker run --net host \
|
||||
-v ~/container.training/dockercoins/worker:/workspace \
|
||||
gcr.io/kaniko-project/executor \
|
||||
--context=/workspace \
|
||||
--destination=127.0.0.1:30448/worker-kaniko:latest
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
We use `--net host` so that we can connect to the registry over `127.0.0.1`.
|
||||
|
||||
---
|
||||
|
||||
## Running Kaniko in a Kubernetes pod
|
||||
|
||||
- We need to mount or copy the build context to the pod
|
||||
|
||||
- For bonus points, we are going to build straight from the git repository
|
||||
|
||||
- We need to `git clone` the repository before running Kaniko
|
||||
|
||||
- We are going to use two containers sharing a volume:
|
||||
|
||||
- a first container to `git clone` the repository to the volume
|
||||
|
||||
- a second container to run Kaniko, using the content of the volume
|
||||
|
||||
- However, we need the first container to be done before running the second one
|
||||
|
||||
🤔 How could we do that?
|
||||
|
||||
---
|
||||
|
||||
## [Init Containers](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/) to the rescue
|
||||
|
||||
- A pod can have a list of `initContainers`
|
||||
|
||||
- `initContainers` are executed in the specified order
|
||||
|
||||
- Each Init Container needs to complete (exit) successfully
|
||||
|
||||
- If any Init Container fails (non-zero exit status) the pod fails
|
||||
|
||||
(what happens next depends on the pod's `restartPolicy`)
|
||||
|
||||
- After all Init Containers have run successfully, normal `containers` are started
|
||||
|
||||
- We are going to execute the `git clone` operation in an Init Container
|
||||
|
||||
---
|
||||
|
||||
## Our Kaniko builder pod
|
||||
|
||||
.small[
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: kaniko-build
|
||||
spec:
|
||||
initContainers:
|
||||
- name: git-clone
|
||||
image: alpine
|
||||
command: ["sh", "-c"]
|
||||
args:
|
||||
- |
|
||||
apk add --no-cache git &&
|
||||
git clone git://github.com/jpetazzo/container.training /workspace
|
||||
volumeMounts:
|
||||
- name: workspace
|
||||
mountPath: /workspace
|
||||
containers:
|
||||
- name: build-image
|
||||
image: gcr.io/kaniko-project/executor:latest
|
||||
args:
|
||||
- "--context=/workspace/dockercoins/rng"
|
||||
- "--insecure-skip-tls-verify"
|
||||
- "--destination=registry:5000/rng-kaniko:latest"
|
||||
volumeMounts:
|
||||
- name: workspace
|
||||
mountPath: /workspace
|
||||
volumes:
|
||||
- name: workspace
|
||||
```
|
||||
]
|
||||
|
||||
---
|
||||
|
||||
## Explanations
|
||||
|
||||
- We define a volume named `workspace` (using the default `emptyDir` provider)
|
||||
|
||||
- That volume is mounted to `/workspace` in both our containers
|
||||
|
||||
- The `git-clone` Init Container installs `git` and runs `git clone`
|
||||
|
||||
- The `build-image` container executes Kaniko
|
||||
|
||||
- We use our self-hosted registry DNS name (`registry`)
|
||||
|
||||
- We add `--insecure-skip-tls-verify` since our registry doesn't have TLS certs
|
||||
|
||||
---
|
||||
|
||||
## Running our Kaniko builder pod
|
||||
|
||||
- The YAML for the pod is in `k8s/kaniko-build.yaml`
|
||||
|
||||
.exercise[
|
||||
|
||||
- Create the pod:
|
||||
```bash
|
||||
kubectl apply -f ~/container.training/k8s/kaniko-build.yaml
|
||||
```
|
||||
|
||||
- Watch the logs:
|
||||
```bash
|
||||
stern kaniko
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
---
|
||||
|
||||
## Discussion
|
||||
|
||||
*What should we use? The Docker build technique shown earlier? Kaniko? Something else?*
|
||||
|
||||
- For starters: the [Docker Hub automated builds](https://docs.docker.com/docker-hub/builds/) are very easy to set up
|
||||
|
||||
- link a GitHub repository with the Docker Hub
|
||||
|
||||
- each time you push to GitHub, an image gets build on the Docker Hub
|
||||
|
||||
- If this doesn't work for you: why?
|
||||
|
||||
- too slow (I'm far from `us-east-1`!) → consider using your cloud provider's registry
|
||||
|
||||
- I'm not using a cloud provider → ok, perhaps you need to self-host then
|
||||
|
||||
- I need fancy features (e.g. CI) → consider something like GitLab
|
||||
529
slides/k8s/configuration.md
Normal file
529
slides/k8s/configuration.md
Normal file
@@ -0,0 +1,529 @@
|
||||
# Managing configuration
|
||||
|
||||
- Some applications need to be configured (obviously!)
|
||||
|
||||
- There are many ways for our code to pick up configuration:
|
||||
|
||||
- command-line arguments
|
||||
|
||||
- environment variables
|
||||
|
||||
- configuration files
|
||||
|
||||
- configuration servers (getting configuration from a database, an API...)
|
||||
|
||||
- ... and more (because programmers can be very creative!)
|
||||
|
||||
- How can we fit that with containers and Kubernetes?
|
||||
|
||||
---
|
||||
|
||||
## Passing configuration to containers
|
||||
|
||||
- There are many ways to pass configuration to code running in a container:
|
||||
|
||||
- baking it in a custom image
|
||||
|
||||
- command-line arguments
|
||||
|
||||
- environment variables
|
||||
|
||||
- injecting configuration files
|
||||
|
||||
- exposing it over the Kubernetes API
|
||||
|
||||
- configuration servers
|
||||
|
||||
- Let's review these different strategies!
|
||||
|
||||
---
|
||||
|
||||
## Baking custom images
|
||||
|
||||
- Put the configuration in the image
|
||||
|
||||
(it can be in a configuration file, but also `ENV` or `CMD` actions)
|
||||
|
||||
- It's easy! It's simple!
|
||||
|
||||
- Unfortunately, it also has downsides:
|
||||
|
||||
- multiplication of images
|
||||
|
||||
- different images for dev, staging, prod ...
|
||||
|
||||
- minor reconfigurations require a whole build/push/pull cycle
|
||||
|
||||
- Avoid doing it unless you don't have the time to figure out other options
|
||||
|
||||
---
|
||||
|
||||
## Command-line arguments
|
||||
|
||||
- Pass options to `args` array in the container specification
|
||||
|
||||
- Example ([source](https://github.com/coreos/pods/blob/master/kubernetes.yaml#L29)):
|
||||
```yaml
|
||||
args:
|
||||
- "--data-dir=/var/lib/etcd"
|
||||
- "--advertise-client-urls=http://127.0.0.1:2379"
|
||||
- "--listen-client-urls=http://127.0.0.1:2379"
|
||||
- "--listen-peer-urls=http://127.0.0.1:2380"
|
||||
- "--name=etcd"
|
||||
```
|
||||
|
||||
- The options can be passed directly to the program that we run ...
|
||||
|
||||
... or to a wrapper script that will use them to e.g. generate a config file
|
||||
|
||||
---
|
||||
|
||||
## Command-line arguments, pros & cons
|
||||
|
||||
- Works great when options are passed directly to the running program
|
||||
|
||||
(otherwise, a wrapper script can work around the issue)
|
||||
|
||||
- Works great when there aren't too many parameters
|
||||
|
||||
(to avoid a 20-lines `args` array)
|
||||
|
||||
- Requires documentation and/or understanding of the underlying program
|
||||
|
||||
("which parameters and flags do I need, again?")
|
||||
|
||||
- Well-suited for mandatory parameters (without default values)
|
||||
|
||||
- Not ideal when we need to pass a real configuration file anyway
|
||||
|
||||
---
|
||||
|
||||
## Environment variables
|
||||
|
||||
- Pass options through the `env` map in the container specification
|
||||
|
||||
- Example:
|
||||
```yaml
|
||||
env:
|
||||
- name: ADMIN_PORT
|
||||
value: "8080"
|
||||
- name: ADMIN_AUTH
|
||||
value: Basic
|
||||
- name: ADMIN_CRED
|
||||
value: "admin:0pensesame!"
|
||||
```
|
||||
|
||||
.warning[`value` must be a string! Make sure that numbers and fancy strings are quoted.]
|
||||
|
||||
🤔 Why this weird `{name: xxx, value: yyy}` scheme? It will be revealed soon!
|
||||
|
||||
---
|
||||
|
||||
## The downward API
|
||||
|
||||
- In the previous example, environment variables have fixed values
|
||||
|
||||
- We can also use a mechanism called the *downward API*
|
||||
|
||||
- The downward API allows to expose pod or container information
|
||||
|
||||
- either through special files (we won't show that for now)
|
||||
|
||||
- or through environment variables
|
||||
|
||||
- The value of these environment variables is computed when the container is started
|
||||
|
||||
- Remember: environment variables won't (can't) change after container start
|
||||
|
||||
- Let's see a few concrete examples!
|
||||
|
||||
---
|
||||
|
||||
## Exposing the pod's namespace
|
||||
|
||||
```yaml
|
||||
- name: MY_POD_NAMESPACE
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: metadata.namespace
|
||||
```
|
||||
|
||||
- Useful to generate FQDN of services
|
||||
|
||||
(in some contexts, a short name is not enough)
|
||||
|
||||
- For instance, the two commands should be equivalent:
|
||||
```
|
||||
curl api-backend
|
||||
curl api-backend.$MY_POD_NAMESPACE.svc.cluster.local
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exposing the pod's IP address
|
||||
|
||||
```yaml
|
||||
- name: MY_POD_IP
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: status.podIP
|
||||
```
|
||||
|
||||
- Useful if we need to know our IP address
|
||||
|
||||
(we could also read it from `eth0`, but this is more solid)
|
||||
|
||||
---
|
||||
|
||||
## Exposing the container's resource limits
|
||||
|
||||
```yaml
|
||||
- name: MY_MEM_LIMIT
|
||||
valueFrom:
|
||||
resourceFieldRef:
|
||||
containerName: test-container
|
||||
resource: limits.memory
|
||||
```
|
||||
|
||||
- Useful for runtimes where memory is garbage collected
|
||||
|
||||
- Example: the JVM
|
||||
|
||||
(the memory available to the JVM should be set with the `-Xmx ` flag)
|
||||
|
||||
- Best practice: set a memory limit, and pass it to the runtime
|
||||
|
||||
(see [this blog post](https://very-serio.us/2017/12/05/running-jvms-in-kubernetes/) for a detailed example)
|
||||
|
||||
---
|
||||
|
||||
## More about the downward API
|
||||
|
||||
- [This documentation page](https://kubernetes.io/docs/tasks/inject-data-application/environment-variable-expose-pod-information/) tells more about these environment variables
|
||||
|
||||
- And [this one](https://kubernetes.io/docs/tasks/inject-data-application/downward-api-volume-expose-pod-information/) explains the other way to use the downward API
|
||||
|
||||
(through files that get created in the container filesystem)
|
||||
|
||||
---
|
||||
|
||||
## Environment variables, pros and cons
|
||||
|
||||
- Works great when the running program expects these variables
|
||||
|
||||
- Works great for optional parameters with reasonable defaults
|
||||
|
||||
(since the container image can provide these defaults)
|
||||
|
||||
- Sort of auto-documented
|
||||
|
||||
(we can see which environment variables are defined in the image, and their values)
|
||||
|
||||
- Can be (ab)used with longer values ...
|
||||
|
||||
- ... You *can* put an entire Tomcat configuration file in an environment ...
|
||||
|
||||
- ... But *should* you?
|
||||
|
||||
(Do it if you really need to, we're not judging! But we'll see better ways.)
|
||||
|
||||
---
|
||||
|
||||
## Injecting configuration files
|
||||
|
||||
- Sometimes, there is no way around it: we need to inject a full config file
|
||||
|
||||
- Kubernetes provides a mechanism for that purpose: `configmaps`
|
||||
|
||||
- A configmap is a Kubernetes resource that exists in a namespace
|
||||
|
||||
- Conceptually, it's a key/value map
|
||||
|
||||
(values are arbitrary strings)
|
||||
|
||||
- We can think about them in (at least) two different ways:
|
||||
|
||||
- as holding entire configuration file(s)
|
||||
|
||||
- as holding individual configuration parameters
|
||||
|
||||
---
|
||||
|
||||
## Configmaps storing entire files
|
||||
|
||||
- In this case, each key/value pair corresponds to a configuration file
|
||||
|
||||
- Key = name of the file
|
||||
|
||||
- Value = content of the file
|
||||
|
||||
- There can be one key/value pair, or as many as necessary
|
||||
|
||||
(for complex apps with multiple configuration files)
|
||||
|
||||
- Examples:
|
||||
```
|
||||
# Create a configmap with a single key, "app.conf"
|
||||
kubectl create configmap my-app-config --from-file=app.conf
|
||||
# Create a configmap with a single key, "app.conf" but another file
|
||||
kubectl create configmap my-app-config --from-file=app.conf=app-prod.conf
|
||||
# Create a configmap with multiple keys (one per file in the config.d directory)
|
||||
kubectl create configmap my-app-config --from-file=config.d/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Configmaps storing individual parameters
|
||||
|
||||
- In this case, each key/value pair corresponds to a parameter
|
||||
|
||||
- Key = name of the parameter
|
||||
|
||||
- Value = value of the parameter
|
||||
|
||||
- Examples:
|
||||
```
|
||||
# Create a configmap with two keys
|
||||
kubectl create cm my-app-config \
|
||||
--from-literal=foreground=red \
|
||||
--from-literal=background=blue
|
||||
|
||||
# Create a configmap from a file containing key=val pairs
|
||||
kubectl create cm my-app-config \
|
||||
--from-env-file=app.conf
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exposing configmaps to containers
|
||||
|
||||
- Configmaps can be exposed as plain files in the filesystem of a container
|
||||
|
||||
- this is achieved by declaring a volume and mounting it in the container
|
||||
|
||||
- this is particularly effective for configmaps containing whole files
|
||||
|
||||
- Configmaps can be exposed as environment variables in the container
|
||||
|
||||
- this is achieved with the downward API
|
||||
|
||||
- this is particularly effective for configmaps containing individual parameters
|
||||
|
||||
- Let's see how to do both!
|
||||
|
||||
---
|
||||
|
||||
## Passing a configuration file with a configmap
|
||||
|
||||
- We will start a load balancer powered by HAProxy
|
||||
|
||||
- We will use the [official `haproxy` image](https://hub.docker.com/_/haproxy/)
|
||||
|
||||
- It expects to find its configuration in `/usr/local/etc/haproxy/haproxy.cfg`
|
||||
|
||||
- We will provide a simple HAproxy configuration, `k8s/haproxy.cfg`
|
||||
|
||||
- It listens on port 80, and load balances connections between Google and Bing
|
||||
|
||||
---
|
||||
|
||||
## Creating the configmap
|
||||
|
||||
.exercise[
|
||||
|
||||
- Create a configmap named `haproxy` and holding the configuration file:
|
||||
```bash
|
||||
kubectl create configmap haproxy --from-file=~/container.training/k8s/haproxy.cfg
|
||||
```
|
||||
|
||||
- Check what our configmap looks like:
|
||||
```bash
|
||||
kuebectl get configmap haproxy -o yaml
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
---
|
||||
|
||||
## Using the configmap
|
||||
|
||||
We are going to use the following pod definition:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: haproxy
|
||||
spec:
|
||||
volumes:
|
||||
- name: config
|
||||
configMap:
|
||||
name: haproxy
|
||||
containers:
|
||||
- name: haproxy
|
||||
image: haproxy
|
||||
volumeMounts:
|
||||
- name: config
|
||||
mountPath: /usr/local/etc/haproxy/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Using the configmap
|
||||
|
||||
- The resource definition from the previous slide is in `k8s/haproxy.yaml`
|
||||
|
||||
.exercise[
|
||||
|
||||
- Create the HAProxy pod:
|
||||
```bash
|
||||
kubectl apply -f ~/container.training/k8s/haproxy.yaml
|
||||
```
|
||||
|
||||
- Check the IP address allocated to the pod:
|
||||
```bash
|
||||
kubectl get pod haproxy -o wide
|
||||
IP=$(kubectl get pod haproxy -o json | jq -r .status.podIP)
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
---
|
||||
|
||||
## Testing our load balancer
|
||||
|
||||
- The load balancer will send:
|
||||
|
||||
- half of the connections to Google
|
||||
|
||||
- the other half to Bing
|
||||
|
||||
.exercise[
|
||||
|
||||
- Access the load balancer a few times:
|
||||
```bash
|
||||
curl -I $IP
|
||||
curl -I $IP
|
||||
curl -I $IP
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
We should see connections served by Google (look for the `Location` header) and others served by Bing (indicated by the `X-MSEdge-Ref` header).
|
||||
|
||||
---
|
||||
|
||||
## Exposing configmaps with the downward API
|
||||
|
||||
- We are going to run a Docker registry on a custom port
|
||||
|
||||
- By default, the registry listens on port 5000
|
||||
|
||||
- This can be changed by setting environment variable `REGISTRY_HTTP_ADDR`
|
||||
|
||||
- We are going to store the port number in a configmap
|
||||
|
||||
- Then we will expose that configmap to a container environment variable
|
||||
|
||||
---
|
||||
|
||||
## Creating the configmap
|
||||
|
||||
.exercise[
|
||||
|
||||
- Our configmap will have a single key, `http.addr`:
|
||||
```bash
|
||||
kubectl create configmap registry --from-literal=http.addr=0.0.0.0:80
|
||||
```
|
||||
|
||||
- Check our configmap:
|
||||
```bash
|
||||
kubectl get configmap regsitry -o yaml
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
---
|
||||
|
||||
## Using the configmap
|
||||
|
||||
We are going to use the following pod definition:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: registry
|
||||
spec:
|
||||
containers:
|
||||
- name: registry
|
||||
image: registry
|
||||
env:
|
||||
- name: REGISTRY_HTTP_ADDR
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: registry
|
||||
key: http.addr
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Using the configmap
|
||||
|
||||
- The resource definition from the previous slide is in `k8s/registry.yaml`
|
||||
|
||||
.exercise[
|
||||
|
||||
- Create the registry pod:
|
||||
```bash
|
||||
kubectl apply -f ~/container.training/k8s/registry.yaml
|
||||
```
|
||||
|
||||
- Check the IP address allocated to the pod:
|
||||
```bash
|
||||
kubectl get pod registry -o wide
|
||||
IP=$(kubectl get pod registry -o json | jq -r .status.podIP)
|
||||
```
|
||||
|
||||
- Confirm that the registry is available on port 80:
|
||||
```bash
|
||||
curl $IP/v2/_catalog
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
---
|
||||
|
||||
## Passwords, tokens, sensitive information
|
||||
|
||||
- For sensitive information, there is another special resource: *Secrets*
|
||||
|
||||
- Secrets and Configmaps work almost the same way
|
||||
|
||||
(we'll expose the differences on the next slide)
|
||||
|
||||
- The *intent* is different, though:
|
||||
|
||||
*"You should use secrets for things which are actually secret like API keys,
|
||||
credentials, etc., and use config map for not-secret configuration data."*
|
||||
|
||||
*"In the future there will likely be some differentiators for secrets like rotation or support for backing the secret API w/ HSMs, etc."*
|
||||
|
||||
(Source: [the author of both features](https://stackoverflow.com/a/36925553/580281
|
||||
))
|
||||
|
||||
---
|
||||
|
||||
## Differences between configmaps and secrets
|
||||
|
||||
- Secrets are base64-encoded when shown with `kubectl get secrets -o yaml`
|
||||
|
||||
- keep in mind that this is just *encoding*, not *encryption*
|
||||
|
||||
- it is very easy to [automatically extract and decode secrets](https://medium.com/@mveritym/decoding-kubernetes-secrets-60deed7a96a3)
|
||||
|
||||
- [Secrets can be encrypted at rest](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/)
|
||||
|
||||
- With RBAC, we can authorize a user to access configmaps, but not secrets
|
||||
|
||||
(since they are two different kinds of resources)
|
||||
184
slides/k8s/volumes.md
Normal file
184
slides/k8s/volumes.md
Normal file
@@ -0,0 +1,184 @@
|
||||
# Volumes
|
||||
|
||||
- Volumes are special directories that are mounted in containers
|
||||
|
||||
- Volumes can have many different purposes:
|
||||
|
||||
- share files and directories between containers running on the same machine
|
||||
|
||||
- share files and directories between containers and their host
|
||||
|
||||
- centralize configuration information in Kubernetes and expose it to containers
|
||||
|
||||
- manage credentials and secrets and expose them securely to containers
|
||||
|
||||
- store persistent data for stateful services
|
||||
|
||||
- access storage systems (like Ceph, EBS, NFS, portworx, and many others)
|
||||
|
||||
---
|
||||
|
||||
## Kubernetes volumes vs. Docker volumes
|
||||
|
||||
- Kubernetes and Docker volumes are very similar
|
||||
|
||||
(the [Kubernetes documentation](https://kubernetes.io/docs/concepts/storage/volumes/) says otherwise ...
|
||||
<br/>
|
||||
but it refers to Docker 1.7, which was released in 2015!)
|
||||
|
||||
- Docker volumes allow to share data between containers running on the same host
|
||||
|
||||
- Kubernetes volumes allow to share data between containers in the same pod
|
||||
|
||||
- Both Docker and Kubernetes volumes allow access to storage systems
|
||||
|
||||
- Kubernetes volumes are also used to expose configuration and secrets
|
||||
|
||||
- Docker has specific concepts for configuration and secrets
|
||||
|
||||
(but under the hood, the technical implementation is similar)
|
||||
|
||||
- If you're not familiar with Docker volumes, you can safely ignore this slide!
|
||||
|
||||
---
|
||||
|
||||
## A simple volume example
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: nginx-with-volume
|
||||
spec:
|
||||
volumes:
|
||||
- name: www
|
||||
containers:
|
||||
- name: nginx
|
||||
image: nginx
|
||||
volumeMounts:
|
||||
- name: www
|
||||
mountPath: /usr/share/nginx/html/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## A simple volume example, explained
|
||||
|
||||
- We define a standalone `Pod` named `nginx-with-volume`
|
||||
|
||||
- In that pod, there is a volume named `www`
|
||||
|
||||
- No type is specified, so it will default to `emptyDir`
|
||||
|
||||
(as the name implies, it will be initialized as an empty directory at pod creation)
|
||||
|
||||
- In that pod, there is also a container named `nginx`
|
||||
|
||||
- That container mounts the volume `www` to path `/usr/share/nginx/html/`
|
||||
|
||||
---
|
||||
|
||||
## A volume shared between two containers
|
||||
|
||||
.small[
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: nginx-with-volume
|
||||
spec:
|
||||
volumes:
|
||||
- name: www
|
||||
containers:
|
||||
- name: nginx
|
||||
image: nginx
|
||||
volumeMounts:
|
||||
- name: www
|
||||
mountPath: /usr/share/nginx/html/
|
||||
- name: git
|
||||
image: alpine
|
||||
command: [ "sh", "-c", "apk add --no-cache git && git clone https://github.com/octocat/Spoon-Knife /www" ]
|
||||
volumeMounts:
|
||||
- name: www
|
||||
mountPath: /www/
|
||||
restartPolicy: OnFailure
|
||||
```
|
||||
]
|
||||
|
||||
---
|
||||
|
||||
## Sharing a volume, explained
|
||||
|
||||
- We added another container to the pod
|
||||
|
||||
- That container mounts the `www` volume on a different path (`/www`)
|
||||
|
||||
- It uses the `alpine` image
|
||||
|
||||
- When started, it installs `git` and clones the `octocat/Spoon-Knife` repository
|
||||
|
||||
(that repository contains a tiny HTML website)
|
||||
|
||||
- As a result, NGINX now serves this website
|
||||
|
||||
---
|
||||
|
||||
## Sharing a volume, in action
|
||||
|
||||
- Let's try it!
|
||||
|
||||
.exercise[
|
||||
|
||||
- Create the pod by applying the YAML file:
|
||||
```bash
|
||||
kubectl apply -f ~/container.training/k8s/nginx-with-volume.yaml
|
||||
```
|
||||
|
||||
- Check the IP address that was allocated to our pod:
|
||||
```bash
|
||||
kubectl get pod nginx-with-volume -o wide
|
||||
IP=$(kubectl get pod nginx-with-volume -o json | jq -r .status.podIP)
|
||||
```
|
||||
|
||||
- Access the web server:
|
||||
```bash
|
||||
curl $IP
|
||||
```
|
||||
|
||||
]
|
||||
|
||||
---
|
||||
|
||||
## The devil is in the details
|
||||
|
||||
- The default `restartPolicy` is `Always`
|
||||
|
||||
- This would cause our `git` container to run again ... and again ... and again
|
||||
|
||||
(with an exponential back-off delay, as explained [in the documentation](https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#restart-policy))
|
||||
|
||||
- That's why we specified `restartPolicy: OnFailure`
|
||||
|
||||
- There is a short period of time during which the website is not available
|
||||
|
||||
(because the `git` container hasn't done its job yet)
|
||||
|
||||
- This could be avoided by using [Init Containers](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/)
|
||||
|
||||
(we will see a live example in a few sections)
|
||||
|
||||
---
|
||||
|
||||
## Volume lifecycle
|
||||
|
||||
- The lifecycle of a volume is linked to the pod's lifecycle
|
||||
|
||||
- This means that a volume is created when the pod is created
|
||||
|
||||
- This is mostly relevant for `emptyDir` volumes
|
||||
|
||||
(other volumes, like remote storage, are not "created" but rather "attached" )
|
||||
|
||||
- A volume survives across container restarts
|
||||
|
||||
- A volume is destroyed (or, for remote storage, detached) when the pod is destroyed
|
||||
59
slides/new-content.yml
Normal file
59
slides/new-content.yml
Normal file
@@ -0,0 +1,59 @@
|
||||
title: New content september 2018
|
||||
chat: "In person!"
|
||||
gitrepo: github.com/jpetazzo/container.training
|
||||
slides: http://container.training/
|
||||
|
||||
exclude:
|
||||
- self-paced
|
||||
|
||||
chapters:
|
||||
- shared/toc.md
|
||||
- - k8s/volumes.md
|
||||
- k8s/build-with-docker.md
|
||||
- k8s/build-with-kaniko.md
|
||||
- k8s/configuration.md
|
||||
- |
|
||||
# Stateful sets
|
||||
|
||||
setting up consul with stateful sets
|
||||
- |
|
||||
# Persistence
|
||||
|
||||
setting up portworx using consul
|
||||
|
||||
losetup trick
|
||||
|
||||
explain install/remove
|
||||
|
||||
in action with PostgreSQL
|
||||
- - |
|
||||
# RBAC
|
||||
|
||||
explain roles and systemaccounts and bindings
|
||||
|
||||
run kubectl in a pod with a view systemaccount
|
||||
|
||||
putting it together with namespaces
|
||||
|
||||
generate namespace + systemaccount for the namespace (for deployment)
|
||||
- |
|
||||
# Ingress
|
||||
|
||||
show traefik running in daemonset mode
|
||||
- |
|
||||
# CI/CD pipeline
|
||||
|
||||
gitlab maybe?
|
||||
- |
|
||||
# metrics
|
||||
|
||||
prometheus
|
||||
|
||||
- |
|
||||
# misc todo
|
||||
|
||||
after build-with-docker and build-with-kaniko, show clean up orphan pods
|
||||
|
||||
healthchecks / liveness
|
||||
|
||||
|
||||
Reference in New Issue
Block a user