diff --git a/slides/k8s/buildshiprun-dockerhub.md b/slides/k8s/buildshiprun-dockerhub.md new file mode 100644 index 00000000..ea949cfc --- /dev/null +++ b/slides/k8s/buildshiprun-dockerhub.md @@ -0,0 +1,40 @@ +## Using images from the Docker Hub + +- For everyone's convenience, we took care of building DockerCoins images + +- We pushed these images to the DockerHub, under the [dockercoins](https://hub.docker.com/u/dockercoins) user + +- These images are *tagged* with a version number, `v0.1` + +- The full image names are therefore: + + - `dockercoins/hasher:v0.1` + + - `dockercoins/rng:v0.1` + + - `dockercoins/webui:v0.1` + + - `dockercoins/worker:v0.1` + +--- + +## Setting `$REGISTRY` and `$TAG` + +- In the upcoming exercises and labs, we use a couple of environment variables: + + - `$REGISTRY` as a prefix to all image names + + - `$TAG` as the image version tag + +- For example, the worker image is `$REGISTRY/worker:$TAG` + +- If you copy-paste the commands in these exercises: + + **make sure that you set `$REGISTRY` and `$TAG` first!** + +- For example: + ``` + export REGISTRY=dockercoins TAG=v0.1 + ``` + + (this will expand `$REGISTRY/worker:$TAG` to `dockercoins/worker:v0.1`) diff --git a/slides/k8s/buildshiprun-selfhosted.md b/slides/k8s/buildshiprun-selfhosted.md new file mode 100644 index 00000000..000dd4f4 --- /dev/null +++ b/slides/k8s/buildshiprun-selfhosted.md @@ -0,0 +1,235 @@ +## Self-hosting our registry + +*Note: this section shows how to run the Docker +open source registry and use it to ship images +on our cluster. While this method works fine, +we recommend that you consider using one of the +hosted, free automated build services instead. +It will be much easier!* + +*If you need to run a registry on premises, +this section gives you a starting point, but +you will need to make a lot of changes so that +the registry is secured, highly available, and +so that your build pipeline is automated.* + +--- + +## Using the open source registry + +- We need to run a `registry` container + +- It will store images and layers to the local filesystem +
(but you can add a config file to use S3, Swift, etc.) + +- Docker *requires* TLS when communicating with the registry + + - unless for registries on `127.0.0.0/8` (i.e. `localhost`) + + - or with the Engine flag `--insecure-registry` + +- Our strategy: publish the registry container on a NodePort, +
so that it's available through `127.0.0.1:xxxxx` on each node + +--- + +## Deploying a self-hosted registry + +- We will deploy a registry container, and expose it with a NodePort + +.exercise[ + +- Create the registry service: + ```bash + kubectl create deployment registry --image=registry + ``` + +- Expose it on a NodePort: + ```bash + kubectl expose deploy/registry --port=5000 --type=NodePort + ``` + +] + +--- + +## Connecting to our registry + +- We need to find out which port has been allocated + +.exercise[ + +- View the service details: + ```bash + kubectl describe svc/registry + ``` + +- Get the port number programmatically: + ```bash + NODEPORT=$(kubectl get svc/registry -o json | jq .spec.ports[0].nodePort) + REGISTRY=127.0.0.1:$NODEPORT + ``` + +] + +--- + +## Testing our registry + +- A convenient Docker registry API route to remember is `/v2/_catalog` + +.exercise[ + + + +- View the repositories currently held in our registry: + ```bash + curl $REGISTRY/v2/_catalog + ``` + +] + +-- + +We should see: +```json +{"repositories":[]} +``` + +--- + +## Testing our local registry + +- We can retag a small image, and push it to the registry + +.exercise[ + +- Make sure we have the busybox image, and retag it: + ```bash + docker pull busybox + docker tag busybox $REGISTRY/busybox + ``` + +- Push it: + ```bash + docker push $REGISTRY/busybox + ``` + +] + +--- + +## Checking again what's on our local registry + +- Let's use the same endpoint as before + +.exercise[ + +- Ensure that our busybox image is now in the local registry: + ```bash + curl $REGISTRY/v2/_catalog + ``` + +] + +The curl command should now output: +```json +{"repositories":["busybox"]} +``` + +--- + +## Building and pushing our images + +- We are going to use a convenient feature of Docker Compose + +.exercise[ + +- Go to the `stacks` directory: + ```bash + cd ~/container.training/stacks + ``` + +- Build and push the images: + ```bash + export REGISTRY + export TAG=v0.1 + docker-compose -f dockercoins.yml build + docker-compose -f dockercoins.yml push + ``` + +] + +Let's have a look at the `dockercoins.yml` file while this is building and pushing. + +--- + +```yaml +version: "3" + +services: + rng: + build: dockercoins/rng + image: ${REGISTRY-127.0.0.1:5000}/rng:${TAG-latest} + deploy: + mode: global + ... + redis: + image: redis + ... + worker: + build: dockercoins/worker + image: ${REGISTRY-127.0.0.1:5000}/worker:${TAG-latest} + ... + deploy: + replicas: 10 +``` + +.warning[Just in case you were wondering ... Docker "services" are not Kubernetes "services".] + +--- + +class: extra-details + +## Avoiding the `latest` tag + +.warning[Make sure that you've set the `TAG` variable properly!] + +- If you don't, the tag will default to `latest` + +- The problem with `latest`: nobody knows what it points to! + + - the latest commit in the repo? + + - the latest commit in some branch? (Which one?) + + - the latest tag? + + - some random version pushed by a random team member? + +- If you keep pushing the `latest` tag, how do you roll back? + +- Image tags should be meaningful, i.e. correspond to code branches, tags, or hashes + +--- + +## Checking the content of the registry + +- All our images should now be in the registry + +.exercise[ + +- Re-run the same `curl` command as earlier: + ```bash + curl $REGISTRY/v2/_catalog + ``` + +] + +*In these slides, all the commands to deploy +DockerCoins will use a $REGISTRY environment +variable, so that we can quickly switch from +the self-hosted registry to pre-built images +hosted on the Docker Hub. So make sure that +this $REGISTRY variable is set correctly when +running the exercises!* \ No newline at end of file diff --git a/slides/k8s/ourapponkube.md b/slides/k8s/ourapponkube.md index f753b50f..8abf1f18 100644 --- a/slides/k8s/ourapponkube.md +++ b/slides/k8s/ourapponkube.md @@ -1,328 +1,3 @@ -# Shipping images with a registry - -- Initially, our app was running on a single node - -- We could *build* and *run* in the same place - -- Therefore, we did not need to *ship* anything - -- Now that we want to run on a cluster, things are different - -- The easiest way to ship container images is to use a registry - ---- - -## How Docker registries work (a reminder) - -- What happens when we execute `docker run alpine` ? - -- If the Engine needs to pull the `alpine` image, it expands it into `library/alpine` - -- `library/alpine` is expanded into `index.docker.io/library/alpine` - -- The Engine communicates with `index.docker.io` to retrieve `library/alpine:latest` - -- To use something else than `index.docker.io`, we specify it in the image name - -- Examples: - ```bash - docker pull gcr.io/google-containers/alpine-with-bash:1.0 - - docker build -t registry.mycompany.io:5000/myimage:awesome . - docker push registry.mycompany.io:5000/myimage:awesome - ``` - ---- - -## The plan - -We are going to: - -- **build** images for our app, - -- **ship** these images with a registry, - -- **run** deployments using these images, - -- expose (with a ClusterIP) the deployments that need to communicate together, - -- expose (with a NodePort) the web UI so we can access it from outside. - ---- - -## Building and shipping our app - -- We will pick a registry - - (let's pretend the address will be `REGISTRY:PORT`) - -- We will build on our control node (`node1`) - - (the images will be named `REGISTRY:PORT/servicename`) - -- We will push the images to the registry - -- These images will be usable by the other nodes of the cluster - - (i.e., we could do `docker run REGISTRY:PORT/servicename` from these nodes) - ---- - -## A shortcut opportunity - -- As it happens, the images that we need do already exist on the Docker Hub: - - https://hub.docker.com/r/dockercoins/ - -- We could use them instead of using our own registry and images - -*In the following slides, we are going to show how to run a registry -and use it to host container images. We will also show you how to -use the existing images from the Docker Hub, so that you can catch -up (or skip altogether the build/push part) if needed.* - ---- - -## Which registry do we want to use? - -- We could use the Docker Hub - -- There are alternatives like Quay - -- Each major cloud provider has an option as well - - (ACR on Azure, ECR on AWS, GCR on Google Cloud...) - -- There are also commercial products to run our own registry - - (Docker EE, Quay...) - -- And open source options, too! - -*We are going to self-host an open source registry because it's the most generic solution for this workshop. We will use Docker's reference -implementation for simplicity.* - ---- - -## Using the open source registry - -- We need to run a `registry` container - -- It will store images and layers to the local filesystem -
(but you can add a config file to use S3, Swift, etc.) - -- Docker *requires* TLS when communicating with the registry - - - unless for registries on `127.0.0.0/8` (i.e. `localhost`) - - - or with the Engine flag `--insecure-registry` - -- Our strategy: publish the registry container on a NodePort, -
so that it's available through `127.0.0.1:xxxxx` on each node - ---- - -## Deploying a self-hosted registry - -- We will deploy a registry container, and expose it with a NodePort - -.exercise[ - -- Create the registry service: - ```bash - kubectl create deployment registry --image=registry - ``` - -- Expose it on a NodePort: - ```bash - kubectl expose deploy/registry --port=5000 --type=NodePort - ``` - -] - ---- - -## Connecting to our registry - -- We need to find out which port has been allocated - -.exercise[ - -- View the service details: - ```bash - kubectl describe svc/registry - ``` - -- Get the port number programmatically: - ```bash - NODEPORT=$(kubectl get svc/registry -o json | jq .spec.ports[0].nodePort) - REGISTRY=127.0.0.1:$NODEPORT - ``` - -] - ---- - -## Testing our registry - -- A convenient Docker registry API route to remember is `/v2/_catalog` - -.exercise[ - - - -- View the repositories currently held in our registry: - ```bash - curl $REGISTRY/v2/_catalog - ``` - -] - --- - -We should see: -```json -{"repositories":[]} -``` - ---- - -## Testing our local registry - -- We can retag a small image, and push it to the registry - -.exercise[ - -- Make sure we have the busybox image, and retag it: - ```bash - docker pull busybox - docker tag busybox $REGISTRY/busybox - ``` - -- Push it: - ```bash - docker push $REGISTRY/busybox - ``` - -] - ---- - -## Checking again what's on our local registry - -- Let's use the same endpoint as before - -.exercise[ - -- Ensure that our busybox image is now in the local registry: - ```bash - curl $REGISTRY/v2/_catalog - ``` - -] - -The curl command should now output: -```json -{"repositories":["busybox"]} -``` - ---- - -## Building and pushing our images - -- We are going to use a convenient feature of Docker Compose - -.exercise[ - -- Go to the `stacks` directory: - ```bash - cd ~/container.training/stacks - ``` - -- Build and push the images: - ```bash - export REGISTRY - export TAG=v0.1 - docker-compose -f dockercoins.yml build - docker-compose -f dockercoins.yml push - ``` - -] - -Let's have a look at the `dockercoins.yml` file while this is building and pushing. - ---- - -```yaml -version: "3" - -services: - rng: - build: dockercoins/rng - image: ${REGISTRY-127.0.0.1:5000}/rng:${TAG-latest} - deploy: - mode: global - ... - redis: - image: redis - ... - worker: - build: dockercoins/worker - image: ${REGISTRY-127.0.0.1:5000}/worker:${TAG-latest} - ... - deploy: - replicas: 10 -``` - -.warning[Just in case you were wondering ... Docker "services" are not Kubernetes "services".] - ---- - -class: extra-details - -## Avoiding the `latest` tag - -.warning[Make sure that you've set the `TAG` variable properly!] - -- If you don't, the tag will default to `latest` - -- The problem with `latest`: nobody knows what it points to! - - - the latest commit in the repo? - - - the latest commit in some branch? (Which one?) - - - the latest tag? - - - some random version pushed by a random team member? - -- If you keep pushing the `latest` tag, how do you roll back? - -- Image tags should be meaningful, i.e. correspond to code branches, tags, or hashes - ---- - -## Catching up - -- If you have problems deploying the registry ... - -- Or building or pushing the images ... - -- Don't worry: you can easily use pre-built images from the Docker Hub! - -- The images are named `dockercoins/worker:v0.1`, `dockercoins/rng:v0.1`, etc. - -- To use them, just set the `REGISTRY` environment variable to `dockercoins`: - ```bash - export REGISTRY=dockercoins - ``` - -- Make sure to set the `TAG` to `v0.1` - - (our repositories on the Docker Hub do not provide a `latest` tag) - ---- - # Running our application on Kubernetes - We can now deploy our code (as well as a redis instance) diff --git a/slides/k8s/shippingimages.md b/slides/k8s/shippingimages.md new file mode 100644 index 00000000..7e7245eb --- /dev/null +++ b/slides/k8s/shippingimages.md @@ -0,0 +1,91 @@ +# Shipping images with a registry + +- Initially, our app was running on a single node + +- We could *build* and *run* in the same place + +- Therefore, we did not need to *ship* anything + +- Now that we want to run on a cluster, things are different + +- The easiest way to ship container images is to use a registry + +--- + +## How Docker registries work (a reminder) + +- What happens when we execute `docker run alpine` ? + +- If the Engine needs to pull the `alpine` image, it expands it into `library/alpine` + +- `library/alpine` is expanded into `index.docker.io/library/alpine` + +- The Engine communicates with `index.docker.io` to retrieve `library/alpine:latest` + +- To use something else than `index.docker.io`, we specify it in the image name + +- Examples: + ```bash + docker pull gcr.io/google-containers/alpine-with-bash:1.0 + + docker build -t registry.mycompany.io:5000/myimage:awesome . + docker push registry.mycompany.io:5000/myimage:awesome + ``` + +--- + +## Running DockerCoins on Kubernetes + +- Create one deployment for each component + + (hasher, redis, rng, webui, worker) + +- Expose deployments that need to accept connections + + (hasher, redis, rng, webui) + +- For redis, we can use the official redis image + +- For the 4 others, we need to build images and push them to some registry + +--- + +## Building and shipping images + +- There are *many* options! + +- Manually: + + - build locally (with `docker build` or otherwise) + + - push to the registry + +- Automatically: + + - build and test locally + + - when ready, commit and push a code repository + + - the code repository notifies an automated build system + + - that system gets the code, builds it, pushes the image to the registry + +--- + +## Which registry do we want to use? + +- There are SAAS products like Docker Hub, Quay ... + +- Each major cloud provider has an option as well + + (ACR on Azure, ECR on AWS, GCR on Google Cloud...) + +- There are also commercial products to run our own registry + + (Docker EE, Quay...) + +- And open source options, too! + +- When picking a registry, pay attention to its build system + + (when it has one) diff --git a/slides/kube-fullday.yml b/slides/kube-fullday.yml index c4a66358..4cbf8c16 100644 --- a/slides/kube-fullday.yml +++ b/slides/kube-fullday.yml @@ -32,7 +32,10 @@ chapters: - k8s/setup-k8s.md - k8s/kubectlrun.md - k8s/kubectlexpose.md -- - k8s/ourapponkube.md +- - k8s/shippingimages.md +# - k8s/buildshiprun-selfhosted.md + - k8s/buildshiprun-dockerhub.md + - k8s/ourapponkube.md # - k8s/kubectlproxy.md # - k8s/localkubeconfig.md # - k8s/accessinternal.md diff --git a/slides/kube-halfday.yml b/slides/kube-halfday.yml index d5df8d95..11b232b1 100644 --- a/slides/kube-halfday.yml +++ b/slides/kube-halfday.yml @@ -35,6 +35,9 @@ chapters: - k8s/setup-k8s.md - - k8s/kubectlrun.md - k8s/kubectlexpose.md + - k8s/shippingimages.md + #- k8s/buildshiprun-selfhosted.md + - k8s/buildshiprun-dockerhub.md - k8s/ourapponkube.md #- k8s/kubectlproxy.md #- k8s/localkubeconfig.md diff --git a/slides/kube-selfpaced.yml b/slides/kube-selfpaced.yml index e776bb1e..422ff9fb 100644 --- a/slides/kube-selfpaced.yml +++ b/slides/kube-selfpaced.yml @@ -32,7 +32,10 @@ chapters: - k8s/setup-k8s.md - k8s/kubectlrun.md - k8s/kubectlexpose.md -- - k8s/ourapponkube.md +- - k8s/shippingimages.md + - k8s/buildshiprun-selfhosted.md + - k8s/buildshiprun-dockerhub.md + - k8s/ourapponkube.md - k8s/kubectlproxy.md - k8s/localkubeconfig.md - k8s/accessinternal.md diff --git a/slides/kube-twodays.yml b/slides/kube-twodays.yml index 7f4eb798..84e2f689 100644 --- a/slides/kube-twodays.yml +++ b/slides/kube-twodays.yml @@ -32,7 +32,10 @@ chapters: - k8s/setup-k8s.md - k8s/kubectlrun.md - k8s/kubectlexpose.md -- - k8s/ourapponkube.md +- - k8s/shippingimages.md + #- k8s/buildshiprun-selfhosted.md + - k8s/buildshiprun-dockerhub.md + - k8s/ourapponkube.md - k8s/kubectlproxy.md - k8s/localkubeconfig.md - k8s/accessinternal.md