diff --git a/.gitignore b/.gitignore index 23e580b7..5097a11c 100644 --- a/.gitignore +++ b/.gitignore @@ -6,3 +6,4 @@ prepare-vms/ips.html prepare-vms/ips.pdf prepare-vms/settings.yaml prepare-vms/tags +docs/*.yml.html diff --git a/docs/Makefile b/docs/Makefile deleted file mode 100644 index 73b68440..00000000 --- a/docs/Makefile +++ /dev/null @@ -1,5 +0,0 @@ -workshop.html: - -%.html: %.yml *.md workshop.html - ./markmaker.py < $< > $@ || rm $@ - # open $@ diff --git a/docs/build.sh b/docs/build.sh new file mode 100755 index 00000000..541b4ae7 --- /dev/null +++ b/docs/build.sh @@ -0,0 +1,11 @@ +#!/bin/sh + +for YAML in *.yml; do + ./markmaker.py < $YAML > $YAML.html || rm $YAML.html +done + +if [ "$1" = "watch" ]; then + while true; do + find . | entr -d $0 && break + done +fi diff --git a/docs/daemonset.md b/docs/daemonset.md new file mode 100644 index 00000000..62c9ce14 --- /dev/null +++ b/docs/daemonset.md @@ -0,0 +1,371 @@ +# Daemon sets + +- Remember: we did all that cluster orchestration business for `rng` + +- We want one (and exactly one) instance of `rng` per node + +- If we just scale `deploy/rng` to 4, nothing guarantees that they spread + +- Instead of a `deployment`, we will use a `daemonset` + +- Daemon sets are great for cluster-wide, per-node processes: + + - `kube-proxy` + - `weave` (our overlay network) + - monitoring agents + - hardware management tools (e.g. SCSI/FC HBA agents) + - etc. + +- They can also be restricted to run [only on some nodes](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/#running-pods-on-only-some-nodes) + +--- + +## Creating a daemon set + +- Unfortunately, as of Kubernetes 1.8, the CLI cannot create daemon sets + +-- + +- More precisely: it doesn't have a subcommand to create a daemon set + +-- + +- But any kind of resource can always be created by providing a YAML description: + ```bash + kubectl apply -f foo.yaml + ``` + +-- + +- How do we create the YAML file for our daemon set? + +-- + + - option 1: read the docs + +-- + + - option 2: `vi` our way out of it + +--- + +## Creating the YAML file for our daemon set + +- Let's start with the YAML file for the current `rng` resource + +.exercise[ + +- Dump the `rng` resource in YAML: + ```bash + kubectl get deploy/rng -o yaml --export >rng.yml + ``` + +- Edit `rng.yml` + +] + +Note: `--export` will remove "cluster-specific" information, i.e.: +- namespace (so that the resource is not tied to a specific namespace) +- status and creation timestamp (useless when creating a new resource) +- resourceVersion and uid (these would cause... *interesting* problems) + +--- + +## "Casting" a resource to another + +- What if we just changed the `kind` field? + + (It can't be that easy, right?) + +.exercise[ + +- Change `kind: Deployment` to `kind: DaemonSet` + +- Save, quit + +- Try to create our new resource: + ```bash + kubectl apply -f rng.yml + ``` + +] + +-- + +We all knew this couldn't be that easy, right! + +--- + +## Understanding the problem + +- The core of the error is: + ``` + error validating data: + [ValidationError(DaemonSet.spec): + unknown field "replicas" in io.k8s.api.extensions.v1beta1.DaemonSetSpec, + ... + ``` + +-- + +- *Obviously,* it doesn't make sense to specify a number of replicas for a daemon set + +-- + +- Workaround: fix the YAML + + - remove the `replicas` field + - remove the `strategy` field (which defines the rollout mechanism for a deployment) + - remove the `status: {}` line at the end + +-- + +- Or, we could also ... + +--- + +## Use the `--force`, Luke + +- We could also tell Kubernetes to ignore these errors and try anyway + +- The `--force` flag actual name is `--validate=false` + +.exercise[ + +- Try to load our YAML file and ignore errors: + ```bash + kubectl apply -f rng.yml --validate=false + ``` + +] + +-- + +Wait ... Now, can it be *that* easy? + +--- + +## Checking what we've done + +- Did we transform our `deployment` into a `daemonset`? + +.exercise[ + +- Look at the resources that we have now: + ```bash + kubectl get all + ``` + +] + +-- + +We have both `deploy/rng` and `ds/rng` now! + +-- + +And one too many pod ... + +--- + +## Explanation + +- You can have different resource types with the same name + + (i.e. a *deployment* and a *daemonset* both named `rng`) + +- We still have the old `rng` *deployment* + +- But now we have the new `rng` *daemonset* as well + +- If we look at the pods, we have: + + - *one pod* for the deployment + + - *one pod per node* for the daemonset + +--- + +## What are all these pods doing? + +- Let's check the logs of all these `rng` pods + +- All these pods have a `run=rng` label: + + - the first pod, because that's what `kubectl run` does + - the other ones (in the daemon set), because we + *copied the spec from the first one* + +- Therefore, we can query everybody's logs using that `run=rng` selector + +.exercise[ + +- Check the logs of all the pods having a label `run=rng`: + ```bash + kubectl logs -l run=rng --tail 1 + ``` + +] + +-- + +It appears that *all the pods* are serving requests at the moment. + +--- + +## The magic of selectors + +- The `rng` *service* is load balancing requests to a set of pod + +- This set of pod is defined as "pods having the label `run=rng`" + +.exercise[ + +- Check the *selector* in the `rng` service definition: + ```bash + kubectl describe service rng + ``` + +] + +When we created additional pods with this label, they were +automatically detected by `svc/rng` and added as *endpoints* +to the associated load balancer. + +--- + +## Removing the first pod from the load balancer + +- What would happen if we removed that pod, with `kubectl delete pod ...`? + +-- + + The `replicaset` would re-create it immediately. + +-- + +- What would happen if we removed the `run=rng` label from that pod? + +-- + + The `replicaset` would re-create it immediately. + +-- + + ... Because what matters to the `replicaset` is the number of pods *matching that selector.* + +-- + +- But but but ... Don't we have more than one pod with `run=rng` now? + +-- + + The answer lies in the exact selector used by the `replicaset` ... + +--- + +## Deep dive into selectors + +- Let's look at the selectors for the `rng` *deployment* and the associated *replica set* + +.exercise[ + +- Show detailed information about the `rng` deployment: + ```bash + kubectl describe deploy rng + ``` + +- Show detailed information about the `rng` replica: +
(The second command doesn't require you to get the exact name of the replica set) + ```bash + kubectl describe rs rng-yyyy + kubectl describe rs -l run=rng + ``` + +] + +-- + +The replica set selector also has a `pod-template-hash`, unlike the pods in our daemon set. + +--- + +# Updating a service through labels and selectors + +- What if we want to drop the `rng` deployment from the load balancer? + +- Option 1: + + - destroy it + +- Option 2: + + - add an extra *label* to the daemon set + + - update the service *selector* to refer to that *label* + +-- + +Of course, option 2 offers more learning opportunities. Right? + +--- + +## Add an extra label to the daemon set + +- We will update the daemon set "spec" + +- Option 1: + + - edit the `rng.yml` file that we used earlier + + - `kubectl apply -f rng.yml` to load the new definition + +- Option 2: + + - use `kubectl edit` + +.exercise[ + +- Use one of the two options! + +] + +--- + +## A few possible gotchas ... + +- There is a difference between: + + - the label(s) of a resource (in the `metadata` block in the beginning) + + - the selector of a resource (in the `spec` block) + + - the label(s) of the resource(s) created by the first resource (in the `template` block) + +- You want to update the selector and the template (at least) + +- The template must match the selector + + (i.e. the resource will refuse to create resources that it will not select) + +- In YAML, `yes` should be quoted; i.e. `isactive: "yes"` + +--- + +## Wrapping up + +.exercise[ + +- Update the replica set selector and template label + +- Update the service selector + +- Check the logs of all `run=rng` pods to check that only 4 of them are now active + +- Look at the pods that we have right now + +- Bonus exercise 1: clean up the pods of the "old" daemon set + +- Bonus exercise 2: how could we have done to avoid creating new pods? + +] diff --git a/docs/docker-compose.yml b/docs/docker-compose.yml deleted file mode 100644 index 4de8a255..00000000 --- a/docs/docker-compose.yml +++ /dev/null @@ -1,11 +0,0 @@ -version: "3" - -services: - - www: - image: nginx - volumes: - - ".:/usr/share/nginx/html" - ports: - - "8888:80" - diff --git a/docs/kube.yml b/docs/kube.yml index 5870376e..93b786d9 100644 --- a/docs/kube.yml +++ b/docs/kube.yml @@ -8,7 +8,7 @@ chapters: .small[ - Orchestrating microservices with Docker and Kubernetes + Deploying and scaling microservices
with Docker and Kubernetes .small[.small[ @@ -96,6 +96,7 @@ chapters: - kubectlexpose.md - ourapponkube.md - kubectlscale.md + - daemonset.md - | class: title diff --git a/docs/kubectlscale.md b/docs/kubectlscale.md index b8789ca3..6744789c 100644 --- a/docs/kubectlscale.md +++ b/docs/kubectlscale.md @@ -1,2 +1,22 @@ # Scaling a deployment +- We will start with an easy one: the `worker` deployment + +.exercise[ + +- Open two new terminals to check what's going on with pods and deployments: + ```bash + kubectl get pods -w + kubectl get deployments -w + ``` + +- Now, create more `worker` replicas: + ```bash + kubectl scale deploy/worker --replicas=10 + ``` + +] + +After a few seconds, the graph in the web UI should show up. +
+(And peak at 10 hashes/second, just like when we were running on a single one.) \ No newline at end of file diff --git a/docs/ourapponkube.md b/docs/ourapponkube.md index c30648af..d7c0df80 100644 --- a/docs/ourapponkube.md +++ b/docs/ourapponkube.md @@ -75,7 +75,7 @@ In this part, we will: - Create the registry service: ```bash - kubectl deploy registry --image=registry:2 + kubectl run registry --image=registry:2 ``` - Expose it on a NodePort: @@ -227,13 +227,13 @@ services: - Deploy `redis`: ```bash - kubectl deploy redis --image=redis + kubectl run redis --image=redis ``` - Deploy everything else: ```bash for SERVICE in hasher rng webui worker; do - kubectl deploy $SERVICE --image=$REGISTRY/$SERVICE + kubectl run $SERVICE --image=$REGISTRY/$SERVICE done ``` @@ -273,15 +273,15 @@ services: - `worker` doesn't need to be exposed -- `webui` will be dealth with later +- `webui` will be dealt with later .exercise[ -- Expose each service, specifying the right port: +- Expose each deployment, specifying the right port: ```bash - kubectl expose redis --port 6379 - kubectl expose rng --port 80 - kubectl expose hasher --port 80 + kubectl expose deployment redis --port 6379 + kubectl expose deployment rng --port 80 + kubectl expose deployment hasher --port 80 ``` ]