Fix overlapsing slide number and pics

Warning: do not merge this branch to your content, otherwise you
will get the ENIX logo in the top right of all your decks

.gitignore

@@ -7,4 +7,7 @@ prepare-vms/ips.pdf
 prepare-vms/settings.yaml
 prepare-vms/tags
 slides/*.yml.html
-autotest/nextstep
+slides/autopilot/state.yaml
+slides/index.html
+slides/past.html
+node_modules

CHECKLIST.md

@@ -0,0 +1,24 @@
+Checklist to use when delivering a workshop
+Authored by Jérôme; additions by Bridget
+
+- [ ] Create event-named branch (such as `conferenceYYYY`) in the [main repo](https://github.com/jpetazzo/container.training/)
+  - [ ] Create file `slides/_redirects` containing a link to the desired tutorial: `/ /kube-halfday.yml.html 200`
+  - [ ] Push local branch to GitHub and merge into main repo
+  - [ ] [Netlify setup](https://app.netlify.com/sites/container-training/settings/domain): create subdomain for event-named branch
+- [ ] Add link to event-named branch to [container.training front page](https://github.com/jpetazzo/container.training/blob/master/slides/index.html)
+- [ ] Update the slides that says which versions we are using for [kube](https://github.com/jpetazzo/container.training/blob/master/slides/kube/versions-k8s.md) or [swarm](https://github.com/jpetazzo/container.training/blob/master/slides/swarm/versions.md) workshops
+- [ ] Update the version of Compose and Machine in [settings](https://github.com/jpetazzo/container.training/tree/master/prepare-vms/settings)
+- [ ] (optional) Create chatroom
+- [ ] (optional) Set chatroom in YML ([kube half-day example](https://github.com/jpetazzo/container.training/blob/master/slides/kube-halfday.yml#L6-L8)) and deploy
+- [ ] (optional) Put chat link on [container.training front page](https://github.com/jpetazzo/container.training/blob/master/slides/index.html)
+- [ ] How many VMs do we need? Check with event organizers ahead of time
+- [ ] Provision VMs (slightly more than we think we'll need)
+- [ ] Change password on presenter's VMs (to forestall any hijinx)
+- [ ] Onsite: walk the room to count seats, check power supplies, lectern, A/V setup
+- [ ] Print cards
+- [ ] Cut cards
+- [ ] Last-minute merge from master
+- [ ] Check that all looks good
+- [ ] DELIVER!
+- [ ] Shut down VMs
+- [ ] Update index.html to remove chat link and move session to past things

LICENSE

@@ -1,13 +1,12 @@
-Copyright 2015 Jérôme Petazzoni
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
+The code in this repository is licensed under the Apache License
+Version 2.0. You may obtain a copy of this license at:
 
     http://www.apache.org/licenses/LICENSE-2.0
 
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
+The instructions and slides in this repository (e.g. the files
+with extension .md and .yml in the "slides" subdirectory) are
+under the Creative Commons Attribution 4.0 International Public
+License. You may obtain a copy of this license at:
+
+    https://creativecommons.org/licenses/by/4.0/legalcode
+

README.md

@@ -39,14 +39,16 @@ your own tutorials.
 All these materials have been gathered in a single repository
 because they have a few things in common:
 
-- a [build system](slides/) generating HTML slides from
-  Markdown source files;
 - some [common slides](slides/common/) that are re-used
   (and updated) identically between different decks;
+- a [build system](slides/) generating HTML slides from
+  Markdown source files;
+- a [semi-automated test harness](slides/autopilot/) to check
+  that the exercises and examples provided work properly;
+- a [PhantomJS script](slides/slidechecker.js) to check
+  that the slides look good and don't have formatting issues;
 - [deployment scripts](prepare-vms/) to start training
   VMs in bulk;
-- a [semi-automated test harness](autotest/) to check
-  that the exercises and examples provided work properly;
 - a fancy pipeline powered by
   [Netlify](https://www.netlify.com/) and continuously
   deploying `master` to http://container.training/.
@@ -74,9 +76,6 @@ a few other contributors. It is actively maintained.
 
 ## Repository structure
 
-- [autotest](autotest/)
-  - Semi-automated testing system to check that all the exercises
-    in the slides work properly.
 - [bin](bin/)
   - A few helper scripts that you can safely ignore for now.
 - [dockercoins](dockercoins/)
@@ -248,6 +247,17 @@ content but you also know to skip during presentation.
 - Last 15-30 minutes is for stateful services, DAB files, and questions.
 
 
+### Pre-built images
+
+There are pre-built images for the 4 components of the DockerCoins demo app: `dockercoins/hasher:v0.1`, `dockercoins/rng:v0.1`, `dockercoins/webui:v0.1`, and `dockercoins/worker:v0.1`. They correspond to the code in this repository.
+
+There are also three variants, for demo purposes:
+
+- `dockercoins/rng:v0.2` is broken (the server won't even start),
+- `dockercoins/webui:v0.2` has bigger font on the Y axis and a green graph (instead of blue),
+- `dockercoins/worker:v0.2` is 11x slower than `v0.1`.
+
+
 ## Past events
 
 Since its inception, this workshop has been delivered dozens of times,
@@ -282,15 +292,31 @@ If there is a bug and you can't even reproduce it:
 sorry. It is probably an Heisenbug. We can't act on it
 until it's reproducible, alas.
 
-If you have attended this workshop and have feedback,
-or if you want somebody  to deliver that workshop at your
-conference or for your company: you can contact one of us!
 
-- jerome at docker dot com
+# “Please teach us!”
+
+If you have attended one of these workshops, and want
+your team or organization to attend a similar one, you
+can look at the list of upcoming events on
+http://container.training/.
+
+You are also welcome to reuse these materials to run
+your own workshop, for your team or even at a meetup
+or conference. In that case, you might enjoy watching
+[Bridget Kromhout's talk at KubeCon 2018 Europe](
+https://www.youtube.com/watch?v=mYsp_cGY2O0), explaining
+precisely how to run such a workshop yourself.
+
+Finally, you can also contact the following persons,
+who are experienced speakers, are familiar with the
+material, and are available to deliver these workshops
+at your conference or for your company:
+
+- jerome dot petazzoni at gmail dot com
 - bret at bretfisher dot com
 
-If you are willing and able to deliver such workshops,
-feel free to submit a PR to add your name to that list!
+(If you are willing and able to deliver such workshops,
+feel free to submit a PR to add your name to that list!)
 
 **Thank you!**
 

autotest/autotest.py

@@ -1,229 +0,0 @@
-#!/usr/bin/env python
-
-import uuid
-import logging
-import os
-import re
-import subprocess
-import sys
-import time
-import uuid
-
-logging.basicConfig(level=logging.DEBUG)
-
-
-TIMEOUT = 60 # 1 minute
-
-
-def hrule():
-    return "="*int(subprocess.check_output(["tput", "cols"]))
-
-# A "snippet" is something that the user is supposed to do in the workshop.
-# Most of the "snippets" are shell commands.
-# Some of them can be key strokes or other actions.
-# In the markdown source, they are the code sections (identified by triple-
-# quotes) within .exercise[] sections.
-
-class Snippet(object):
-
-    def __init__(self, slide, content):
-        self.slide = slide
-        self.content = content
-        self.actions = []
-
-    def __str__(self):
-        return self.content
-
-
-class Slide(object):
-
-    current_slide = 0
-
-    def __init__(self, content):
-        Slide.current_slide += 1
-        self.number = Slide.current_slide
-
-        # Remove commented-out slides
-        # (remark.js considers ??? to be the separator for speaker notes)
-        content = re.split("\n\?\?\?\n", content)[0]
-        self.content = content
-
-        self.snippets = []
-        exercises = re.findall("\.exercise\[(.*)\]", content, re.DOTALL)
-        for exercise in exercises:
-            if "```" in exercise:
-                for snippet in exercise.split("```")[1::2]:
-                    self.snippets.append(Snippet(self, snippet))
-            else:
-                logging.warning("Exercise on slide {} does not have any ``` snippet."
-                                .format(self.number))
-                self.debug()
-
-    def __str__(self):
-        text = self.content
-        for snippet in self.snippets:
-            text = text.replace(snippet.content, ansi("7")(snippet.content))
-        return text
-
-    def debug(self):
-        logging.debug("\n{}\n{}\n{}".format(hrule(), self.content, hrule()))
-
-
-def ansi(code):
-    return lambda s: "\x1b[{}m{}\x1b[0m".format(code, s)
-
-
-def wait_for_string(s):
-    logging.debug("Waiting for string: {}".format(s))
-    deadline = time.time() + TIMEOUT
-    while time.time() < deadline:
-        output = capture_pane()
-        if s in output:
-            return
-        time.sleep(1)
-    raise Exception("Timed out while waiting for {}!".format(s))
-
-
-def wait_for_prompt():
-    logging.debug("Waiting for prompt.")
-    deadline = time.time() + TIMEOUT
-    while time.time() < deadline:
-        output = capture_pane()
-        # If we are not at the bottom of the screen, there will be a bunch of extra \n's
-        output = output.rstrip('\n')
-        if output[-2:] == "\n$":
-            return
-        time.sleep(1)
-    raise Exception("Timed out while waiting for prompt!")
-
-
-def check_exit_status():
-    token = uuid.uuid4().hex
-    data = "echo {} $?\n".format(token)
-    logging.debug("Sending {!r} to get exit status.".format(data))
-    send_keys(data)
-    time.sleep(0.5)
-    wait_for_prompt()
-    screen = capture_pane()
-    status = re.findall("\n{} ([0-9]+)\n".format(token), screen, re.MULTILINE)
-    logging.debug("Got exit status: {}.".format(status))
-    if len(status) == 0:
-        raise Exception("Couldn't retrieve status code {}. Timed out?".format(token))
-    if len(status) > 1:
-        raise Exception("More than one status code {}. I'm seeing double! Shoot them both.".format(token))
-    code = int(status[0])
-    if code != 0:
-        raise Exception("Non-zero exit status: {}.".format(code))
-    # Otherwise just return peacefully.
-
-
-slides = []
-content = open(sys.argv[1]).read()
-for slide in re.split("\n---?\n", content):
-    slides.append(Slide(slide))
-
-actions = []
-for slide in slides:
-    for snippet in slide.snippets:
-        content = snippet.content
-        # Extract the "method" (e.g. bash, keys, ...)
-        # On multi-line snippets, the method is alone on the first line
-        # On single-line snippets, the data follows the method immediately
-        if '\n' in content:
-            method, data = content.split('\n', 1)
-        else:
-            method, data = content.split(' ', 1)
-        actions.append((slide, snippet, method, data))
-
-
-def send_keys(data):
-    subprocess.check_call(["tmux", "send-keys", data])
-
-def capture_pane():
-    return subprocess.check_output(["tmux", "capture-pane", "-p"])
-
-
-try:
-    i = int(open("nextstep").read())
-    logging.info("Loaded next step ({}) from file.".format(i))
-except Exception as e:
-    logging.warning("Could not read nextstep file ({}), initializing to 0.".format(e))
-    i = 0
-
-interactive = True
-
-while i < len(actions):
-    with open("nextstep", "w") as f:
-        f.write(str(i))
-    slide, snippet, method, data = actions[i]
-
-    # Remove extra spaces (we don't want them in the terminal) and carriage returns
-    data = data.strip()
-
-    print(hrule())
-    print(slide.content.replace(snippet.content, ansi(7)(snippet.content)))
-    print(hrule())
-    if interactive:
-        print("[{}/{}] Shall we execute that snippet above?".format(i, len(actions)))
-        print("(ENTER to execute, 'c' to continue until next error, N to jump to step #N)")
-        command = raw_input("> ")
-    else:
-        command = ""
-
-    # For now, remove the `highlighted` sections
-    # (Make sure to use $() in shell snippets!)
-    if '`' in data:
-        logging.info("Stripping ` from snippet.")
-        data = data.replace('`', '')
-
-    if command == "c":
-        # continue until next timeout
-        interactive = False
-    elif command.isdigit():
-        i = int(command)
-    elif command == "":
-        logging.info("Running with method {}: {}".format(method, data))
-        if method == "keys":
-            send_keys(data)
-        elif method == "bash":
-            # Make sure that we're ready
-            wait_for_prompt()
-            # Strip leading spaces
-            data = re.sub("\n +", "\n", data)
-            # Add "RETURN" at the end of the command :)
-            data += "\n"
-            # Send command
-            send_keys(data)
-            # Force a short sleep to avoid race condition
-            time.sleep(0.5)
-            _, _, next_method, next_data = actions[i+1]
-            if next_method == "wait":
-                wait_for_string(next_data)
-            else:
-                wait_for_prompt()
-                # Verify return code FIXME should be optional
-                check_exit_status()
-        elif method == "copypaste":
-            screen = capture_pane()
-            matches = re.findall(data, screen, flags=re.DOTALL)
-            if len(matches) == 0:
-                raise Exception("Could not find regex {} in output.".format(data))
-            # Arbitrarily get the most recent match
-            match = matches[-1]
-            # Remove line breaks (like a screen copy paste would do)
-            match = match.replace('\n', '')
-            send_keys(match + '\n')
-            # FIXME: we should factor out the "bash" method
-            wait_for_prompt()
-            check_exit_status()
-        else:
-            logging.warning("Unknown method {}: {!r}".format(method, data))
-        i += 1
-
-    else:
-        i += 1
-        logging.warning("Unknown command {}, skipping to next step.".format(command))
-
-# Reset slide counter
-with open("nextstep", "w") as f:
-    f.write(str(0))

dockercoins/rng/rng.py

@@ -28,5 +28,5 @@ def rng(how_many_bytes):
 
 
 if __name__ == "__main__":
-    app.run(host="0.0.0.0", port=80)
+    app.run(host="0.0.0.0", port=80, threaded=False)
 

k8s/efk.yaml

@@ -0,0 +1,222 @@
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: fluentd
+
+---
+apiVersion: rbac.authorization.k8s.io/v1beta1
+kind: ClusterRole
+metadata:
+  name: fluentd
+rules:
+- apiGroups:
+  - ""
+  resources:
+  - pods
+  - namespaces
+  verbs:
+  - get
+  - list
+  - watch
+
+---
+kind: ClusterRoleBinding
+apiVersion: rbac.authorization.k8s.io/v1beta1
+metadata:
+  name: fluentd
+roleRef:
+  kind: ClusterRole
+  name: fluentd
+  apiGroup: rbac.authorization.k8s.io
+subjects:
+- kind: ServiceAccount
+  name: fluentd
+  namespace: default
+
+---
+apiVersion: extensions/v1beta1
+kind: DaemonSet
+metadata:
+  name: fluentd
+  labels:
+    k8s-app: fluentd-logging
+    version: v1
+    kubernetes.io/cluster-service: "true"
+spec:
+  template:
+    metadata:
+      labels:
+        k8s-app: fluentd-logging
+        version: v1
+        kubernetes.io/cluster-service: "true"
+    spec:
+      serviceAccount: fluentd
+      serviceAccountName: fluentd
+      tolerations:
+      - key: node-role.kubernetes.io/master
+        effect: NoSchedule
+      containers:
+      - name: fluentd
+        image: fluent/fluentd-kubernetes-daemonset:elasticsearch
+        env:
+          - name:  FLUENT_ELASTICSEARCH_HOST
+            value: "elasticsearch"
+          - name:  FLUENT_ELASTICSEARCH_PORT
+            value: "9200"
+          - name: FLUENT_ELASTICSEARCH_SCHEME
+            value: "http"
+          # X-Pack Authentication
+          # =====================
+          - name: FLUENT_ELASTICSEARCH_USER
+            value: "elastic"
+          - name: FLUENT_ELASTICSEARCH_PASSWORD
+            value: "changeme"
+        resources:
+          limits:
+            memory: 200Mi
+          requests:
+            cpu: 100m
+            memory: 200Mi
+        volumeMounts:
+        - name: varlog
+          mountPath: /var/log
+        - name: varlibdockercontainers
+          mountPath: /var/lib/docker/containers
+          readOnly: true
+      terminationGracePeriodSeconds: 30
+      volumes:
+      - name: varlog
+        hostPath:
+          path: /var/log
+      - name: varlibdockercontainers
+        hostPath:
+          path: /var/lib/docker/containers
+
+---
+apiVersion: extensions/v1beta1
+kind: Deployment
+metadata:
+  annotations:
+    deployment.kubernetes.io/revision: "1"
+  creationTimestamp: null
+  generation: 1
+  labels:
+    run: elasticsearch
+  name: elasticsearch
+  selfLink: /apis/extensions/v1beta1/namespaces/default/deployments/elasticsearch
+spec:
+  progressDeadlineSeconds: 600
+  replicas: 1
+  revisionHistoryLimit: 10
+  selector:
+    matchLabels:
+      run: elasticsearch
+  strategy:
+    rollingUpdate:
+      maxSurge: 1
+      maxUnavailable: 1
+    type: RollingUpdate
+  template:
+    metadata:
+      creationTimestamp: null
+      labels:
+        run: elasticsearch
+    spec:
+      containers:
+      - image: elasticsearch:5.6.8
+        imagePullPolicy: IfNotPresent
+        name: elasticsearch
+        resources: {}
+        terminationMessagePath: /dev/termination-log
+        terminationMessagePolicy: File
+      dnsPolicy: ClusterFirst
+      restartPolicy: Always
+      schedulerName: default-scheduler
+      securityContext: {}
+      terminationGracePeriodSeconds: 30
+
+---
+apiVersion: v1
+kind: Service
+metadata:
+  creationTimestamp: null
+  labels:
+    run: elasticsearch
+  name: elasticsearch
+  selfLink: /api/v1/namespaces/default/services/elasticsearch
+spec:
+  ports:
+  - port: 9200
+    protocol: TCP
+    targetPort: 9200
+  selector:
+    run: elasticsearch
+  sessionAffinity: None
+  type: ClusterIP
+
+---
+apiVersion: extensions/v1beta1
+kind: Deployment
+metadata:
+  annotations:
+    deployment.kubernetes.io/revision: "1"
+  creationTimestamp: null
+  generation: 1
+  labels:
+    run: kibana
+  name: kibana
+  selfLink: /apis/extensions/v1beta1/namespaces/default/deployments/kibana
+spec:
+  progressDeadlineSeconds: 600
+  replicas: 1
+  revisionHistoryLimit: 10
+  selector:
+    matchLabels:
+      run: kibana
+  strategy:
+    rollingUpdate:
+      maxSurge: 1
+      maxUnavailable: 1
+    type: RollingUpdate
+  template:
+    metadata:
+      creationTimestamp: null
+      labels:
+        run: kibana
+    spec:
+      containers:
+      - env:
+        - name: ELASTICSEARCH_URL
+          value: http://elasticsearch:9200/
+        image: kibana:5.6.8
+        imagePullPolicy: Always
+        name: kibana
+        resources: {}
+        terminationMessagePath: /dev/termination-log
+        terminationMessagePolicy: File
+      dnsPolicy: ClusterFirst
+      restartPolicy: Always
+      schedulerName: default-scheduler
+      securityContext: {}
+      terminationGracePeriodSeconds: 30
+
+---
+apiVersion: v1
+kind: Service
+metadata:
+  creationTimestamp: null
+  labels:
+    run: kibana
+  name: kibana
+  selfLink: /api/v1/namespaces/default/services/kibana
+spec:
+  externalTrafficPolicy: Cluster
+  ports:
+  - port: 5601
+    protocol: TCP
+    targetPort: 5601
+  selector:
+    run: kibana
+  sessionAffinity: None
+  type: NodePort

k8s/grant-admin-to-dashboard.yaml

@@ -0,0 +1,14 @@
+apiVersion: rbac.authorization.k8s.io/v1beta1
+kind: ClusterRoleBinding
+metadata:
+  name: kubernetes-dashboard
+  labels:
+    k8s-app: kubernetes-dashboard
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: cluster-admin
+subjects:
+- kind: ServiceAccount
+  name: kubernetes-dashboard
+  namespace: kube-system

k8s/kubernetes-dashboard.yaml

@@ -0,0 +1,167 @@
+# Copyright 2017 The Kubernetes Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Configuration to deploy release version of the Dashboard UI compatible with
+# Kubernetes 1.8.
+#
+# Example usage: kubectl create -f <this_file>
+
+# ------------------- Dashboard Secret ------------------- #
+
+apiVersion: v1
+kind: Secret
+metadata:
+  labels:
+    k8s-app: kubernetes-dashboard
+  name: kubernetes-dashboard-certs
+  namespace: kube-system
+type: Opaque
+
+---
+# ------------------- Dashboard Service Account ------------------- #
+
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  labels:
+    k8s-app: kubernetes-dashboard
+  name: kubernetes-dashboard
+  namespace: kube-system
+
+---
+# ------------------- Dashboard Role & Role Binding ------------------- #
+
+kind: Role
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: kubernetes-dashboard-minimal
+  namespace: kube-system
+rules:
+  # Allow Dashboard to create 'kubernetes-dashboard-key-holder' secret.
+- apiGroups: [""]
+  resources: ["secrets"]
+  verbs: ["create"]
+  # Allow Dashboard to create 'kubernetes-dashboard-settings' config map.
+- apiGroups: [""]
+  resources: ["configmaps"]
+  verbs: ["create"]
+  # Allow Dashboard to get, update and delete Dashboard exclusive secrets.
+- apiGroups: [""]
+  resources: ["secrets"]
+  resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs"]
+  verbs: ["get", "update", "delete"]
+  # Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map.
+- apiGroups: [""]
+  resources: ["configmaps"]
+  resourceNames: ["kubernetes-dashboard-settings"]
+  verbs: ["get", "update"]
+  # Allow Dashboard to get metrics from heapster.
+- apiGroups: [""]
+  resources: ["services"]
+  resourceNames: ["heapster"]
+  verbs: ["proxy"]
+- apiGroups: [""]
+  resources: ["services/proxy"]
+  resourceNames: ["heapster", "http:heapster:", "https:heapster:"]
+  verbs: ["get"]
+
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: kubernetes-dashboard-minimal
+  namespace: kube-system
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: Role
+  name: kubernetes-dashboard-minimal
+subjects:
+- kind: ServiceAccount
+  name: kubernetes-dashboard
+  namespace: kube-system
+
+---
+# ------------------- Dashboard Deployment ------------------- #
+
+kind: Deployment
+apiVersion: apps/v1beta2
+metadata:
+  labels:
+    k8s-app: kubernetes-dashboard
+  name: kubernetes-dashboard
+  namespace: kube-system
+spec:
+  replicas: 1
+  revisionHistoryLimit: 10
+  selector:
+    matchLabels:
+      k8s-app: kubernetes-dashboard
+  template:
+    metadata:
+      labels:
+        k8s-app: kubernetes-dashboard
+    spec:
+      containers:
+      - name: kubernetes-dashboard
+        image: k8s.gcr.io/kubernetes-dashboard-amd64:v1.8.3
+        ports:
+        - containerPort: 8443
+          protocol: TCP
+        args:
+          - --auto-generate-certificates
+          # Uncomment the following line to manually specify Kubernetes API server Host
+          # If not specified, Dashboard will attempt to auto discover the API server and connect
+          # to it. Uncomment only if the default does not work.
+          # - --apiserver-host=http://my-address:port
+        volumeMounts:
+        - name: kubernetes-dashboard-certs
+          mountPath: /certs
+          # Create on-disk volume to store exec logs
+        - mountPath: /tmp
+          name: tmp-volume
+        livenessProbe:
+          httpGet:
+            scheme: HTTPS
+            path: /
+            port: 8443
+          initialDelaySeconds: 30
+          timeoutSeconds: 30
+      volumes:
+      - name: kubernetes-dashboard-certs
+        secret:
+          secretName: kubernetes-dashboard-certs
+      - name: tmp-volume
+        emptyDir: {}
+      serviceAccountName: kubernetes-dashboard
+      # Comment the following tolerations if Dashboard must not be deployed on master
+      tolerations:
+      - key: node-role.kubernetes.io/master
+        effect: NoSchedule
+
+---
+# ------------------- Dashboard Service ------------------- #
+
+kind: Service
+apiVersion: v1
+metadata:
+  labels:
+    k8s-app: kubernetes-dashboard
+  name: kubernetes-dashboard
+  namespace: kube-system
+spec:
+  ports:
+    - port: 443
+      targetPort: 8443
+  selector:
+    k8s-app: kubernetes-dashboard

k8s/netpol-allow-testcurl-for-testweb.yaml

@@ -0,0 +1,14 @@
+kind: NetworkPolicy
+apiVersion: networking.k8s.io/v1
+metadata:
+  name: allow-testcurl-for-testweb
+spec:
+  podSelector:
+    matchLabels:
+      run: testweb
+  ingress:
+  - from:
+    - podSelector:
+        matchLabels:
+          run: testcurl
+

k8s/netpol-deny-all-for-testweb.yaml

@@ -0,0 +1,10 @@
+kind: NetworkPolicy
+apiVersion: networking.k8s.io/v1
+metadata:
+  name: deny-all-for-testweb
+spec:
+  podSelector:
+    matchLabels:
+      run: testweb
+  ingress: []
+

k8s/socat.yaml

@@ -0,0 +1,67 @@
+apiVersion: extensions/v1beta1
+kind: Deployment
+metadata:
+  annotations:
+    deployment.kubernetes.io/revision: "2"
+  creationTimestamp: null
+  generation: 1
+  labels:
+    run: socat
+  name: socat
+  namespace: kube-system
+  selfLink: /apis/extensions/v1beta1/namespaces/kube-system/deployments/socat
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      run: socat
+  strategy:
+    rollingUpdate:
+      maxSurge: 1
+      maxUnavailable: 1
+    type: RollingUpdate
+  template:
+    metadata:
+      creationTimestamp: null
+      labels:
+        run: socat
+    spec:
+      containers:
+      - args:
+        - sh
+        - -c
+        - apk add --no-cache socat && socat TCP-LISTEN:80,fork,reuseaddr OPENSSL:kubernetes-dashboard:443,verify=0
+        image: alpine
+        imagePullPolicy: Always
+        name: socat
+        resources: {}
+        terminationMessagePath: /dev/termination-log
+        terminationMessagePolicy: File
+      dnsPolicy: ClusterFirst
+      restartPolicy: Always
+      schedulerName: default-scheduler
+      securityContext: {}
+      terminationGracePeriodSeconds: 30
+status: {}
+---
+apiVersion: v1
+kind: Service
+metadata:
+  creationTimestamp: null
+  labels:
+    run: socat
+  name: socat
+  namespace: kube-system
+  selfLink: /api/v1/namespaces/kube-system/services/socat
+spec:
+  externalTrafficPolicy: Cluster
+  ports:
+  - port: 80
+    protocol: TCP
+    targetPort: 80
+  selector:
+    run: socat
+  sessionAffinity: None
+  type: NodePort
+status:
+  loadBalancer: {}

prepare-vms/README.md

@@ -1,15 +1,22 @@
-# Trainer tools to create and prepare VMs for Docker workshops on AWS
+# Trainer tools to create and prepare VMs for Docker workshops on AWS or Azure
 
 ## Prerequisites
 
 - [Docker](https://docs.docker.com/engine/installation/)
 - [Docker Compose](https://docs.docker.com/compose/install/)
+- [Parallel SSH](https://code.google.com/archive/p/parallel-ssh/) (on a Mac: `brew install pssh`) - the configuration scripts require this
+
+And if you want to generate printable cards:
+
+- [pyyaml](https://pypi.python.org/pypi/PyYAML) (on a Mac: `brew install pyyaml`)
+- [jinja2](https://pypi.python.org/pypi/Jinja2) (on a Mac: `brew install jinja2`)
 
 ## General Workflow
 
 - fork/clone repo
-- set required environment variables for AWS
+- set required environment variables
 - create your own setting file from `settings/example.yaml`
+- if necessary, increase allowed open files: `ulimit -Sn 10000`
 - run `./workshopctl` commands to create instances, install docker, setup each users environment in node1, other management tasks
 - run `./workshopctl cards` command to generate PDF for printing handouts of each users host IP's and login info
 
@@ -35,6 +42,16 @@ The Docker Compose file here is used to build a image with all the dependencies
 - `AWS_SECRET_ACCESS_KEY`
 - `AWS_DEFAULT_REGION`
 
+If you're not using AWS, set these to placeholder values:
+
+```
+export AWS_ACCESS_KEY_ID="foo"
+export AWS_SECRET_ACCESS_KEY="foo"
+export AWS_DEFAULT_REGION="foo"
+```
+
+If you don't have the `aws` CLI installed, you will get a warning that it's a missing dependency. If you're not using AWS you can ignore this.
+
 ### Update/copy `settings/example.yaml`
 
 Then pass `settings/YOUR_WORKSHOP_NAME-settings.yaml` as an argument to `./workshopctl deploy`, `./workshopctl cards`, etc.
@@ -48,6 +65,7 @@ workshopctl - the orchestration workshop swiss army knife
 Commands:
 ami          Show the AMI that will be used for deployment
 amis         List Ubuntu AMIs in the current region
+build        Build the Docker image to run this program in a container
 cards        Generate ready-to-print cards for a batch of VMs
 deploy       Install Docker on a bunch of running VMs
 ec2quotas    Check our EC2 quotas (max instances)
@@ -55,6 +73,7 @@ help         Show available commands
 ids          List the instance IDs belonging to a given tag or token
 ips          List the IP addresses of the VMs for a given tag or token
 kube         Setup kubernetes clusters with kubeadm (must be run AFTER deploy)
+kubetest     Check that all notes are reporting as Ready
 list         List available batches in the current region
 opensg       Open the default security group to ALL ingress traffic
 pull_images  Pre-pull a bunch of Docker images
@@ -63,6 +82,7 @@ start        Start a batch of VMs
 status       List instance status for a given batch
 stop         Stop (terminate, shutdown, kill, remove, destroy...) instances
 test         Run tests (pre-flight checks) on a batch of VMs
+wrap         Run this program in a container
 ```
 
 ### Summary of What `./workshopctl` Does For You
@@ -73,21 +93,82 @@ test         Run tests (pre-flight checks) on a batch of VMs
 - The `./workshopctl` script can be executed directly.
 - It will run locally if all its dependencies are fulfilled; otherwise it will run in the Docker container you created with `docker-compose build` (preparevms_prepare-vms).
 - During `start` it will add your default local SSH key to all instances under the `ubuntu` user.
-- During `deploy` it will create the `docker` user with password `training`, which is printing on the cards for students. For now, this is hard coded.
+- During `deploy` it will create the `docker` user with password `training`, which is printing on the cards for students. This can be configured with the `docker_user_password` property in the settings file.
 
-### Example Steps to Launch a Batch of Instances for a Workshop
+### Example Steps to Launch a Batch of AWS Instances for a Workshop
 
 - Run `./workshopctl start N` Creates `N` EC2 instances
   - Your local SSH key will be synced to instances under `ubuntu` user
   - AWS instances will be created and tagged based on date, and IP's stored in `prepare-vms/tags/`
-- Run `./workshopctl deploy TAG settings/somefile.yaml` to run `scripts/postprep.rc` via parallel-ssh
+- Run `./workshopctl deploy TAG settings/somefile.yaml` to run `lib/postprep.py` via parallel-ssh
   - If it errors or times out, you should be able to rerun
   - Requires good connection to run all the parallel SSH connections, up to 100 parallel (ProTip: create dedicated management instance in same AWS region where you run all these utils from)
-- Run `./workshopctl pull-images TAG` to pre-pull a bunch of Docker images to the instances
+- Run `./workshopctl pull_images TAG` to pre-pull a bunch of Docker images to the instances
 - Run `./workshopctl cards TAG settings/somefile.yaml` generates PDF/HTML files to print and cut and hand out to students
 - *Have a great workshop*
 - Run `./workshopctl stop TAG` to terminate instances.
 
+### Example Steps to Launch Azure Instances
+
+- Install the [Azure CLI](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest) and authenticate with a valid account
+- Customize `azuredeploy.parameters.json`
+  - Required:
+    - Provide the SSH public key you plan to use for instance configuration
+  - Optional:
+    - Choose a name for the workshop (default is "workshop")
+    - Choose the number of instances (default is 3)
+    - Customize the desired instance size (default is Standard_D1_v2)
+ - Launch instances with your chosen resource group name and your preferred region; the examples are "workshop" and "eastus":
+```
+az group create --name workshop --location eastus
+az group deployment create --resource-group workshop --template-file azuredeploy.json --parameters @azuredeploy.parameters.json
+```
+
+The `az group deployment create` command can take several minutes and will only say `- Running ..` until it completes, unless you increase the verbosity with `--verbose` or `--debug`.
+
+To display the IPs of the instances you've launched:
+
+```
+az vm list-ip-addresses --resource-group workshop --output table
+```
+
+If you want to put the IPs into `prepare-vms/tags/<tag>/ips.txt` for a tag of "myworkshop":
+
+1) If you haven't yet installed `jq` and/or created your event's tags directory in `prepare-vms`:
+
+```
+brew install jq
+mkdir -p tags/myworkshop
+```
+
+2) And then generate the IP list:
+
+```
+az vm list-ip-addresses --resource-group workshop --output json | jq -r '.[].virtualMachine.network.publicIpAddresses[].ipAddress' > tags/myworkshop/ips.txt
+```
+
+After the workshop is over, remove the instances:
+
+```
+az group delete --resource-group workshop
+```
+
+### Example Steps to Configure Instances from a non-AWS Source
+
+- Launch instances via your preferred method. You'll need to get the instance IPs and be able to ssh into them.
+- Set placeholder values for [AWS environment variable settings](#required-environment-variables).
+- Choose a tag. It could be an event name, datestamp, etc. Ensure you have created a directory for your tag: `prepare-vms/tags/<tag>/`
+- If you have not already generated a file with the IPs to be configured:
+  - The file should be named `prepare-vms/tags/<tag>/ips.txt`
+  - Format is one IP per line, no other info needed.
+- Ensure the settings file is as desired (especially the number of nodes): `prepare-vms/settings/kube101.yaml`
+- For a tag called `myworkshop`, configure instances: `workshopctl deploy myworkshop settings/kube101.yaml`
+- Optionally, configure Kubernetes clusters of the size in the settings: `workshopctl kube myworkshop`
+- Optionally, test your Kubernetes clusters. They may take a little time to become ready: `workshopctl kubetest myworkshop`
+- Generate cards to print and hand out: `workshopctl cards myworkshop settings/kube101.yaml`
+- Print the cards file: `prepare-vms/tags/myworkshop/ips.html`
+
+
 ## Other Tools
 
 ### Deploying your SSH key to all the machines
@@ -97,13 +178,6 @@ test         Run tests (pre-flight checks) on a batch of VMs
 - Run `pcopykey`.
 
 
-### Installing extra packages
-
-- Source `postprep.rc`.
-  (This will install a few extra packages, add entries to
-  /etc/hosts, generate SSH keys, and deploy them on all hosts.)
-
-
 ## Even More Details
 
 #### Sync of SSH keys
@@ -132,16 +206,20 @@ Instances can be deployed manually using the `deploy` command:
 
     $ ./workshopctl deploy TAG settings/somefile.yaml
 
-The `postprep.rc` file will be copied via parallel-ssh to all of the VMs and executed.
+The `postprep.py` file will be copied via parallel-ssh to all of the VMs and executed.
 
 #### Pre-pull images
 
-    $ ./workshopctl pull-images TAG
+    $ ./workshopctl pull_images TAG
 
 #### Generate cards
 
     $ ./workshopctl cards TAG settings/somefile.yaml
 
+If you want to generate both HTML and PDF cards, install [wkhtmltopdf](https://wkhtmltopdf.org/downloads.html); without that installed, only HTML cards will be generated.
+
+If you don't have `wkhtmltopdf` installed, you will get a warning that it is a missing dependency. If you plan to just print the HTML cards, you can ignore this.
+
 #### List tags
 
     $ ./workshopctl list

prepare-vms/azuredeploy.json

@@ -0,0 +1,250 @@
+{
+  "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
+  "contentVersion": "1.0.0.0",
+  "parameters": {
+    "workshopName": {
+      "type": "string",
+      "defaultValue": "workshop",
+      "metadata": {
+        "description": "Workshop name."
+      }
+    },
+    "vmPrefix": {
+      "type": "string",
+      "defaultValue": "node",
+      "metadata": {
+        "description": "Prefix for VM names."
+      }
+    },
+    "numberOfInstances": {
+      "type": "int",
+      "defaultValue": 3,
+      "metadata": {
+        "description": "Number of VMs to create."
+      }
+    },
+    "adminUsername": {
+      "type": "string",
+      "defaultValue": "ubuntu",
+      "metadata": {
+        "description": "Admin username for VMs."
+      }
+    },
+    "sshKeyData": {
+      "type": "string",
+      "defaultValue": "",
+      "metadata": {
+        "description": "SSH rsa public key file as a string."
+      }
+    },
+    "imagePublisher": {
+      "type": "string",
+      "defaultValue": "Canonical",
+      "metadata": {
+        "description": "OS image publisher; default Canonical."
+      }
+    },
+    "imageOffer": {
+      "type": "string",
+      "defaultValue": "UbuntuServer",
+      "metadata": {
+        "description": "The name of the image offer. The default is Ubuntu"
+      }
+    },
+    "imageSKU": {
+      "type": "string",
+      "defaultValue": "16.04-LTS",
+      "metadata": {
+        "description": "Version of the image. The default is 16.04-LTS"
+      }
+    },
+    "vmSize": {
+      "type": "string",
+      "defaultValue": "Standard_D1_v2",
+      "metadata": {
+        "description": "VM Size."
+      }
+    }
+  },
+  "variables": {
+    "vnetID": "[resourceId('Microsoft.Network/virtualNetworks',variables('virtualNetworkName'))]",
+    "subnet1Ref": "[concat(variables('vnetID'),'/subnets/',variables('subnet1Name'))]",
+    "vmName": "[parameters('vmPrefix')]",
+    "sshKeyPath": "[concat('/home/',parameters('adminUsername'),'/.ssh/authorized_keys')]",
+    "publicIPAddressName": "PublicIP",
+    "publicIPAddressType": "Dynamic",
+    "virtualNetworkName": "MyVNET",
+    "netSecurityGroup": "MyNSG",
+    "addressPrefix": "10.0.0.0/16",
+    "subnet1Name": "subnet-1",
+    "subnet1Prefix": "10.0.0.0/24",
+    "nicName": "myVMNic"
+  },
+  "resources": [
+    {
+      "apiVersion": "2017-11-01",
+      "type": "Microsoft.Network/publicIPAddresses",
+      "name": "[concat(variables('publicIPAddressName'),copyIndex(1))]",
+      "location": "[resourceGroup().location]",
+      "copy": {
+        "name": "publicIPLoop",
+        "count": "[parameters('numberOfInstances')]"
+      },
+      "properties": {
+        "publicIPAllocationMethod": "[variables('publicIPAddressType')]"
+      },
+      "tags": {
+        "workshop": "[parameters('workshopName')]"
+      }
+    },
+    {
+      "apiVersion": "2017-11-01",
+      "type": "Microsoft.Network/virtualNetworks",
+      "name": "[variables('virtualNetworkName')]",
+      "location": "[resourceGroup().location]",
+      "dependsOn": [
+        "[concat('Microsoft.Network/networkSecurityGroups/', variables('netSecurityGroup'))]"
+      ],
+      "properties": {
+        "addressSpace": {
+          "addressPrefixes": [
+            "[variables('addressPrefix')]"
+          ]
+        },
+        "subnets": [
+          {
+            "name": "[variables('subnet1Name')]",
+            "properties": {
+              "addressPrefix": "[variables('subnet1Prefix')]",
+              "networkSecurityGroup": {
+                "id": "[resourceId('Microsoft.Network/networkSecurityGroups', variables('netSecurityGroup'))]"
+              }
+            }
+          }
+        ]
+      },
+      "tags": {
+        "workshop": "[parameters('workshopName')]"
+      }
+    },
+    {
+      "apiVersion": "2017-11-01",
+      "type": "Microsoft.Network/networkInterfaces",
+      "name": "[concat(variables('nicName'),copyIndex(1))]",
+      "location": "[resourceGroup().location]",
+      "copy": {
+        "name": "nicLoop",
+        "count": "[parameters('numberOfInstances')]"
+      },
+      "dependsOn": [
+        "[concat('Microsoft.Network/publicIPAddresses/', variables('publicIPAddressName'),copyIndex(1))]",
+        "[concat('Microsoft.Network/virtualNetworks/', variables('virtualNetworkName'))]"
+      ],
+      "properties": {
+        "ipConfigurations": [
+          {
+            "name": "ipconfig1",
+            "properties": {
+              "privateIPAllocationMethod": "Dynamic",
+              "publicIPAddress": {
+                "id": "[resourceId('Microsoft.Network/publicIPAddresses', concat(variables('publicIPAddressName'), copyIndex(1)))]"
+              },
+              "subnet": {
+                "id": "[variables('subnet1Ref')]"
+              }
+            }
+          }
+        ]
+      },
+      "tags": {
+        "workshop": "[parameters('workshopName')]"
+      }
+    },
+    {
+      "apiVersion": "2017-12-01",
+      "type": "Microsoft.Compute/virtualMachines",
+      "name": "[concat(variables('vmName'),copyIndex(1))]",
+      "location": "[resourceGroup().location]",
+      "copy": {
+        "name": "vmLoop",
+        "count": "[parameters('numberOfInstances')]"
+      },
+      "dependsOn": [
+        "[concat('Microsoft.Network/networkInterfaces/', variables('nicName'), copyIndex(1))]"
+      ],
+      "properties": {
+        "hardwareProfile": {
+          "vmSize": "[parameters('vmSize')]"
+        },
+        "osProfile": {
+          "computerName": "[concat(variables('vmName'),copyIndex(1))]",
+          "adminUsername": "[parameters('adminUsername')]",
+          "linuxConfiguration": {
+            "disablePasswordAuthentication": true,
+            "ssh": {
+              "publicKeys": [
+                {
+                  "path": "[variables('sshKeyPath')]",
+                  "keyData": "[parameters('sshKeyData')]"
+                }
+              ]
+            }
+          }
+        },
+        "storageProfile": {
+          "osDisk": {
+            "createOption": "FromImage"
+          },
+          "imageReference": {
+            "publisher": "[parameters('imagePublisher')]",
+            "offer": "[parameters('imageOffer')]",
+            "sku": "[parameters('imageSKU')]",
+            "version": "latest"
+          }
+        },
+        "networkProfile": {
+          "networkInterfaces": [
+            {
+              "id": "[resourceId('Microsoft.Network/networkInterfaces', concat(variables('nicName'),copyIndex(1)))]"
+            }
+          ]
+        }
+      },
+      "tags": {
+        "workshop": "[parameters('workshopName')]"
+      }
+    },
+    {
+      "apiVersion": "2017-11-01",
+      "type": "Microsoft.Network/networkSecurityGroups",
+      "name": "[variables('netSecurityGroup')]",
+      "location": "[resourceGroup().location]",
+      "tags": {
+        "workshop": "[parameters('workshopName')]"
+      },
+      "properties": {
+        "securityRules": [
+          {
+            "name": "default-open-ports",
+            "properties": {
+              "protocol": "Tcp",
+              "sourcePortRange": "*",
+              "destinationPortRange": "*",
+              "sourceAddressPrefix": "*",
+              "destinationAddressPrefix": "*",
+              "access": "Allow",
+              "priority": 1000,
+              "direction": "Inbound"
+            }
+          }
+        ]
+      }
+    }
+  ],
+  "outputs": {
+    "resourceID": {
+      "type": "string",
+      "value": "[resourceId('Microsoft.Network/publicIPAddresses', concat(variables('publicIPAddressName'),'1'))]"
+    }
+  }
+}

prepare-vms/azuredeploy.parameters.json

@@ -0,0 +1,18 @@
+{
+  "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
+  "contentVersion": "1.0.0.0",
+  "parameters": {
+    "sshKeyData": {
+      "value": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXTIl/M9oeSlcsC5Rfe+nZr4Jc4sl200pSw2lpdxlZ3xzeP15NgSSMJnigUrKUXHfqRQ+2wiPxEf0Odz2GdvmXvR0xodayoOQsO24AoERjeSBXCwqITsfp1bGKzMb30/3ojRBo6LBR6r1+lzJYnNCGkT+IQwLzRIpm0LCNz1j08PUI2aZ04+mcDANvHuN/hwi/THbLLp6SNWN43m9r02RcC6xlCNEhJi4wk4VzMzVbSv9RlLGST2ocbUHwmQ2k9OUmpzoOx73aQi9XNnEaFh2w/eIdXM75VtkT3mRryyykg9y0/hH8/MVmIuRIdzxHQqlm++DLXVH5Ctw6a4kS+ki7 workshop"
+      },
+    "workshopName": {
+      "value": "workshop"
+    },
+    "numberOfInstances": {
+      "value": 3
+    },
+    "vmSize": {
+      "value": "Standard_D1_v2"
+    }
+  }
+}

prepare-vms/cards.html

@@ -12,7 +12,9 @@
     {%- set cluster_or_machine = "cluster" -%}
     {%- set this_or_each = "each" -%}
     {%- set machine_is_or_machines_are = "machines are" -%}
-    {%- set image_src = "https://cdn.wp.nginx.com/wp-content/uploads/2016/07/docker-swarm-hero2.png" -%}
+    {%- set image_src_swarm = "https://cdn.wp.nginx.com/wp-content/uploads/2016/07/docker-swarm-hero2.png" -%}
+    {%- set image_src_kube = "https://avatars1.githubusercontent.com/u/13629408" -%}
+    {%- set image_src = image_src_swarm -%}
 {%- endif -%}
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
@@ -83,7 +85,7 @@ img {
                 <tr><td>login:</td></tr>
                 <tr><td class="logpass">docker</td></tr>
                 <tr><td>password:</td></tr>
-                <tr><td class="logpass">training</td></tr>
+                <tr><td class="logpass">{{ docker_user_password }}</td></tr>
             </table>
 
         </p>

prepare-vms/clusters.csv

@@ -0,0 +1,5 @@
+Put your initials in the first column to "claim" a cluster.
+Initials{% for node in clusters[0] %}	node{{ loop.index }}{% endfor %}
+{% for cluster in clusters -%}
+  {%- for node in cluster %}	{{ node|trim }}{% endfor %}
+{% endfor %}

prepare-vms/cncsetup.sh

@@ -0,0 +1,21 @@
+#!/bin/sh
+if [ $(whoami) != ubuntu ]; then
+  echo "This script should be executed on a freshly deployed node,"
+  echo "with the 'ubuntu' user. Aborting."
+  exit 1
+fi
+if id docker; then
+  sudo userdel -r docker
+fi
+pip install --user awscli jinja2 pdfkit
+sudo apt-get install -y wkhtmltopdf xvfb
+tmux new-session \; send-keys "
+[ -f ~/.ssh/id_rsa ] || ssh-keygen
+
+eval \$(ssh-agent)
+ssh-add
+Xvfb :0 &
+export DISPLAY=:0
+mkdir -p ~/www
+sudo docker run -d -p 80:80 -v \$HOME/www:/usr/share/nginx/html nginx
+"

prepare-vms/docker-compose.yml

@@ -7,7 +7,6 @@ services:
         working_dir: /root/prepare-vms
         volumes:
             - $HOME/.aws/:/root/.aws/
-            - /etc/localtime:/etc/localtime:ro
             - $SSH_AUTH_SOCK:$SSH_AUTH_SOCK
             - $PWD/:/root/prepare-vms/
         environment:
@@ -15,5 +14,6 @@ services:
             AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
             AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
             AWS_DEFAULT_REGION: ${AWS_DEFAULT_REGION}
+            AWS_INSTANCE_TYPE: ${AWS_INSTANCE_TYPE}
             USER: ${USER}
         entrypoint: /root/prepare-vms/workshopctl

prepare-vms/lib/cli.sh

@@ -2,7 +2,7 @@
 _ERR() {
     error "Command $BASH_COMMAND failed (exit status: $?)"
 }
-set -e
+set -eE
 trap _ERR ERR
 
 die() {

prepare-vms/lib/commands.sh

@@ -39,13 +39,16 @@ _cmd_cards() {
     need_tag $TAG
     need_settings $SETTINGS
 
-    aws_get_instance_ips_by_tag $TAG >tags/$TAG/ips.txt
+    # If you're not using AWS, populate the ips.txt file manually
+    if [ ! -f tags/$TAG/ips.txt ]; then
+      aws_get_instance_ips_by_tag $TAG >tags/$TAG/ips.txt
+    fi
 
     # Remove symlinks to old cards
     rm -f ips.html ips.pdf
 
     # This will generate two files in the base dir: ips.pdf and ips.html
-    python lib/ips-txt-to-html.py $SETTINGS
+    lib/ips-txt-to-html.py $SETTINGS
 
     for f in ips.html ips.pdf; do
         # Remove old versions of cards if they exist
@@ -124,27 +127,21 @@ _cmd kube "Setup kubernetes clusters with kubeadm (must be run AFTER deploy)"
 _cmd_kube() {
 
     # Install packages
-    pssh "
+    pssh --timeout 200 "
     curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg |
     sudo apt-key add - &&
     echo deb http://apt.kubernetes.io/ kubernetes-xenial main |
     sudo tee /etc/apt/sources.list.d/kubernetes.list"
-    pssh "
+    pssh --timeout 200 "
     sudo apt-get update -q &&
     sudo apt-get install -qy kubelet kubeadm kubectl
     kubectl completion bash | sudo tee /etc/bash_completion.d/kubectl"
 
-    # Work around https://github.com/kubernetes/kubernetes/issues/53356
-    pssh "
-    if [ ! -f /etc/kubernetes/kubelet.conf ]; then
-        sudo systemctl stop kubelet
-        sudo rm -rf /var/lib/kubelet/pki
-    fi"
-
     # Initialize kube master
-    pssh "
+    pssh --timeout 200 "
     if grep -q node1 /tmp/node && [ ! -f /etc/kubernetes/admin.conf ]; then
-        sudo kubeadm init
+        kubeadm token generate > /tmp/token
+	sudo kubeadm init --token \$(cat /tmp/token)
     fi"
 
     # Put kubeconfig in ubuntu's and docker's accounts
@@ -157,15 +154,6 @@ _cmd_kube() {
         sudo chown -R docker /home/docker/.kube
     fi"
 
-    # Get bootstrap token
-    pssh "
-    if grep -q node1 /tmp/node; then
-        TOKEN_NAME=\$(kubectl -n kube-system get secret -o name | grep bootstrap-token)
-        TOKEN_ID=\$(kubectl -n kube-system get \$TOKEN_NAME -o go-template --template '{{ index .data \"token-id\" }}' | base64 -d)
-        TOKEN_SECRET=\$(kubectl -n kube-system get \$TOKEN_NAME -o go-template --template '{{ index .data \"token-secret\" }}' | base64 -d)
-        echo \$TOKEN_ID.\$TOKEN_SECRET >/tmp/token
-    fi"
-
     # Install weave as the pod network
     pssh "
     if grep -q node1 /tmp/node; then
@@ -174,15 +162,46 @@ _cmd_kube() {
     fi"
 
     # Join the other nodes to the cluster
-    pssh "
+    pssh --timeout 200 "
     if ! grep -q node1 /tmp/node && [ ! -f /etc/kubernetes/kubelet.conf ]; then
         TOKEN=\$(ssh -o StrictHostKeyChecking=no node1 cat /tmp/token)
-        sudo kubeadm join --token \$TOKEN node1:6443
+        sudo kubeadm join --discovery-token-unsafe-skip-ca-verification --token \$TOKEN node1:6443
     fi"
 
+    # Install stern
+    pssh "
+    if [ ! -x /usr/local/bin/stern ]; then
+        sudo curl -L -o /usr/local/bin/stern https://github.com/wercker/stern/releases/download/1.8.0/stern_linux_amd64
+        sudo chmod +x /usr/local/bin/stern
+        stern --completion bash | sudo tee /etc/bash_completion.d/stern
+    fi"
+
+    # Install helm
+    pssh "
+    if [ ! -x /usr/local/bin/helm ]; then
+        curl https://raw.githubusercontent.com/kubernetes/helm/master/scripts/get | sudo bash
+        helm completion bash | sudo tee /etc/bash_completion.d/helm
+    fi"
+
+
     sep "Done"
 }
 
+_cmd kubetest "Check that all notes are reporting as Ready"
+_cmd_kubetest() {
+    # There are way too many backslashes in the command below.
+    # Feel free to make that better ♥
+    pssh "
+    set -e
+    [ -f /tmp/node ]
+    if grep -q node1 /tmp/node; then
+      which kubectl
+      for NODE in \$(awk /\ node/\ {print\ \\\$2} /etc/hosts); do
+        echo \$NODE ; kubectl get nodes | grep -w \$NODE | grep -w Ready
+      done
+    fi"
+}
+
 _cmd ids "List the instance IDs belonging to a given tag or token"
 _cmd_ids() {
     TAG=$1
@@ -280,6 +299,9 @@ _cmd_start() {
     key_name=$(sync_keys)
 
     AMI=$(_cmd_ami)    # Retrieve the AWS image ID
+    if [ -z "$AMI" ]; then
+        die "I could not find which AMI to use in this region. Try another region?"
+    fi
     TOKEN=$(get_token) # generate a timestamp token for this batch of VMs
     AWS_KEY_NAME=$(make_key_name)
 
@@ -292,7 +314,7 @@ _cmd_start() {
     result=$(aws ec2 run-instances \
         --key-name $AWS_KEY_NAME \
         --count $COUNT \
-        --instance-type t2.medium \
+        --instance-type ${AWS_INSTANCE_TYPE-t2.medium} \
         --client-token $TOKEN \
         --image-id $AMI)
     reservation_id=$(echo "$result" | head -1 | awk '{print $2}')
@@ -387,9 +409,23 @@ pull_tag() {
             ubuntu:latest \
             fedora:latest \
             centos:latest \
+	    elasticsearch:2 \
             postgres \
             redis \
+	    alpine \
+	    registry \
+	    nicolaka/netshoot \
+	    jpetazzo/trainingwheels \
+	    golang \
             training/namer \
+	    dockercoins/hasher \
+	    dockercoins/rng \
+	    dockercoins/webui \
+	    dockercoins/worker \
+	    logstash \
+	    prom/node-exporter \
+	    google/cadvisor \
+	    dockersamples/visualizer \
             nathanleclaire/redisonrails; do
         sudo -u docker docker pull $I
     done'
@@ -430,6 +466,7 @@ tag_is_reachable() {
 }
 
 test_tag() {
+    TAG=$1
     ips_file=tags/$TAG/ips.txt
     info "Picking a random IP address in $ips_file to run tests."
     n=$((1 + $RANDOM % $(wc -l <$ips_file)))

prepare-vms/lib/postprep.py

@@ -13,6 +13,7 @@ COMPOSE_VERSION = config["compose_version"]
 MACHINE_VERSION = config["machine_version"]
 CLUSTER_SIZE = config["clustersize"]
 ENGINE_VERSION = config["engine_version"]
+DOCKER_USER_PASSWORD = config["docker_user_password"]
 
 #################################
 
@@ -45,7 +46,7 @@ def system(cmd):
 
 # On EC2, the ephemeral disk might be mounted on /mnt.
 # If /mnt is a mountpoint, place Docker workspace on it.
-system("if mountpoint -q /mnt; then sudo mkdir /mnt/docker && sudo ln -s /mnt/docker /var/lib/docker; fi")
+system("if mountpoint -q /mnt; then sudo mkdir -p /mnt/docker && sudo ln -sfn /mnt/docker /var/lib/docker; fi")
 
 # Put our public IP in /tmp/ipv4
 # ipv4_retrieval_endpoint = "http://169.254.169.254/latest/meta-data/public-ipv4"
@@ -54,9 +55,9 @@ system("curl --silent {} > /tmp/ipv4".format(ipv4_retrieval_endpoint))
 
 ipv4 = open("/tmp/ipv4").read()
 
-# Add a "docker" user with password "training"
+# Add a "docker" user with password coming from the settings
 system("id docker || sudo useradd -d /home/docker -m -s /bin/bash docker")
-system("echo docker:training | sudo chpasswd")
+system("echo docker:{} | sudo chpasswd".format(DOCKER_USER_PASSWORD))
 
 # Fancy prompt courtesy of @soulshake.
 system("""sudo -u docker tee -a /home/docker/.bashrc <<SQRL
@@ -108,7 +109,7 @@ system("sudo chmod +x /usr/local/bin/docker-machine")
 system("docker-machine version")
 
 system("sudo apt-get remove -y --purge dnsmasq-base")
-system("sudo apt-get -qy install python-setuptools pssh apache2-utils httping htop unzip mosh")
+system("sudo apt-get -qy install python-setuptools pssh apache2-utils httping htop unzip mosh tree")
 
 ### Wait for Docker to be up.
 ### (If we don't do this, Docker will not be responsive during the next step.)

prepare-vms/settings/csv.yaml

@@ -0,0 +1,5 @@
+# Number of VMs per cluster
+clustersize: 5
+
+# Jinja2 template to use to generate ready-to-cut cards
+cards_template: clusters.csv

prepare-vms/settings/example.yaml

@@ -0,0 +1,27 @@
+# customize your cluster size, your cards template, and the versions
+
+# Number of VMs per cluster
+clustersize: 5
+
+# Jinja2 template to use to generate ready-to-cut cards
+cards_template: cards.html
+
+# Use "Letter" in the US, and "A4" everywhere else
+paper_size: Letter
+
+# Feel free to reduce this if your printer can handle it
+paper_margin: 0.2in
+
+# Note: paper_size and paper_margin only apply to PDF generated with pdfkit.
+# If you print (or generate a PDF) using ips.html, they will be ignored.
+# (The equivalent parameters must be set from the browser's print dialog.)
+
+# This can be "test" or "stable"
+engine_version: test
+
+# These correspond to the version numbers visible on their respective GitHub release pages
+compose_version: 1.18.0
+machine_version: 0.13.0
+
+# Password used to connect with the "docker user"
+docker_user_password: training

prepare-vms/settings/fundamentals.yaml

@@ -17,8 +17,11 @@ paper_margin: 0.2in
 # (The equivalent parameters must be set from the browser's print dialog.)
 
 # This can be "test" or "stable"
-engine_version: test
+engine_version: stable
 
 # These correspond to the version numbers visible on their respective GitHub release pages
-compose_version: 1.16.1
-machine_version: 0.12.0
+compose_version: 1.21.1
+machine_version: 0.14.0
+
+# Password used to connect with the "docker user"
+docker_user_password: training

prepare-vms/settings/kube101.html

@@ -0,0 +1,106 @@
+{# Feel free to customize or override anything in there! #}
+{%- set url = "http://container.training/" -%}
+{%- set pagesize = 12 -%}
+{%- if clustersize == 1 -%}
+    {%- set workshop_name = "Docker workshop" -%}
+    {%- set cluster_or_machine = "machine" -%}
+    {%- set this_or_each = "this" -%}
+    {%- set machine_is_or_machines_are = "machine is" -%}
+    {%- set image_src = "https://s3-us-west-2.amazonaws.com/www.breadware.com/integrations/docker.png" -%}
+{%- else -%}
+    {%- set workshop_name = "Kubernetes workshop" -%}
+    {%- set cluster_or_machine = "cluster" -%}
+    {%- set this_or_each = "each" -%}
+    {%- set machine_is_or_machines_are = "machines are" -%}
+    {%- set image_src_swarm = "https://cdn.wp.nginx.com/wp-content/uploads/2016/07/docker-swarm-hero2.png" -%}
+    {%- set image_src_kube = "https://avatars1.githubusercontent.com/u/13629408" -%}
+    {%- set image_src = image_src_kube -%}
+{%- endif -%}
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
+<html>
+<head><style>
+body, table {
+    margin: 0;
+    padding: 0;
+    line-height: 1em;
+    font-size: 14px;
+}
+
+table {
+    border-spacing: 0;
+    margin-top: 0.4em;
+    margin-bottom: 0.4em;
+    border-left: 0.8em double grey;
+    padding-left: 0.4em;
+}
+
+div {
+    float: left;
+    border: 1px dotted black;
+    padding-top: 1%;
+    padding-bottom: 1%;
+    /* columns * (width+left+right) < 100% */
+    width: 21.5%;
+    padding-left: 1.5%;
+    padding-right: 1.5%;
+}
+
+p {
+    margin: 0.4em 0 0.4em 0;
+}
+
+img {
+    height: 4em;
+    float: right;
+    margin-right: -0.4em;
+}
+
+.logpass {
+    font-family: monospace;
+    font-weight: bold;
+}
+
+.pagebreak {
+    page-break-after: always;
+    clear: both;
+    display: block;
+    height: 8px;
+}
+</style></head>
+<body>
+{% for cluster in clusters %}
+    {% if loop.index0>0 and loop.index0%pagesize==0 %}
+        <span class="pagebreak"></span>
+    {% endif %}
+    <div>
+
+        <p>
+            Here is the connection information to your very own
+            {{ cluster_or_machine }} for this {{ workshop_name }}.
+            You can connect to {{ this_or_each }} VM with any SSH client.
+        </p>
+        <p>
+            <img src="{{ image_src }}" />
+            <table>
+                <tr><td>login:</td></tr>
+                <tr><td class="logpass">docker</td></tr>
+                <tr><td>password:</td></tr>
+                <tr><td class="logpass">{{ docker_user_password }}</td></tr>
+            </table>
+
+        </p>
+        <p>
+            Your {{ machine_is_or_machines_are }}:
+            <table>
+                {% for node in cluster %}
+                <tr><td>node{{ loop.index }}:</td><td>{{ node }}</td></tr>
+                {% endfor %}
+            </table>
+        </p>
+        <p>You can find the slides at:
+            <center>{{ url }}</center>
+        </p>
+    </div>
+{% endfor %}
+</body>
+</html>

prepare-vms/settings/kube101.yaml

@@ -0,0 +1,27 @@
+# 3 nodes for k8s 101 workshops
+
+# Number of VMs per cluster
+clustersize: 3
+
+# Jinja2 template to use to generate ready-to-cut cards
+cards_template: settings/kube101.html
+
+# Use "Letter" in the US, and "A4" everywhere else
+paper_size: Letter
+
+# Feel free to reduce this if your printer can handle it
+paper_margin: 0.2in
+
+# Note: paper_size and paper_margin only apply to PDF generated with pdfkit.
+# If you print (or generate a PDF) using ips.html, they will be ignored.
+# (The equivalent parameters must be set from the browser's print dialog.)
+
+# This can be "test" or "stable"
+engine_version: stable
+
+# These correspond to the version numbers visible on their respective GitHub release pages
+compose_version: 1.21.1
+machine_version: 0.14.0
+
+# Password used to connect with the "docker user"
+docker_user_password: training

prepare-vms/settings/swarm.yaml

@@ -1,7 +1,7 @@
 # This file is passed by trainer-cli to scripts/ips-txt-to-html.py
 
 # Number of VMs per cluster
-clustersize: 5
+clustersize: 3
 
 # Jinja2 template to use to generate ready-to-cut cards
 cards_template: cards.html
@@ -17,8 +17,11 @@ paper_margin: 0.2in
 # (The equivalent parameters must be set from the browser's print dialog.)
 
 # This can be "test" or "stable"
-engine_version: test
+engine_version: stable
 
 # These correspond to the version numbers visible on their respective GitHub release pages
-compose_version: 1.16.1
-machine_version: 0.12.0
+compose_version: 1.21.1
+machine_version: 0.14.0
+
+# Password used to connect with the "docker user"
+docker_user_password: training

prepare-vms/workshopctl

@@ -38,7 +38,7 @@ check_envvars() {
         if [ -z "${!envvar}" ]; then
             error "Environment variable $envvar is not set."
             if [ "$envvar" = "SSH_AUTH_SOCK" ]; then
-                error "Hint: run '\$(ssh-agent) ; ssh-add' and try again?"
+                error "Hint: run 'eval \$(ssh-agent) ; ssh-add' and try again?"
             fi
             status=1
         fi

slides/appendcheck.py

@@ -0,0 +1,17 @@
+#!/usr/bin/env python
+
+import logging
+import os
+import subprocess
+import sys
+
+logging.basicConfig(level=os.environ.get("LOG_LEVEL", "INFO"))
+
+filename = sys.argv[1]
+
+logging.info("Checking file {}...".format(filename))
+text = subprocess.check_output(["./slidechecker.js", filename])
+html = open(filename).read()
+html = html.replace("</textarea>", "\n---\n```\n{}\n```\n</textarea>".format(text))
+
+open(filename, "w").write(html)

slides/autopilot/autotest.py

@@ -0,0 +1,453 @@
+#!/usr/bin/env python
+# coding: utf-8
+
+import click
+import logging
+import os
+import random
+import re
+import select
+import subprocess
+import sys
+import time
+import uuid
+import yaml
+
+
+logging.basicConfig(level=os.environ.get("LOG_LEVEL", "INFO"))
+
+
+TIMEOUT = 60 # 1 minute
+
+# This one is not a constant. It's an ugly global.
+IPADDR = None
+
+
+class State(object):
+
+    def __init__(self):
+        self.interactive = True
+        self.verify_status = False
+        self.simulate_type = True
+        self.switch_desktop = False
+        self.sync_slides = False
+        self.open_links = False
+        self.run_hidden = True
+        self.slide = 1
+        self.snippet = 0
+
+    def load(self):
+        data = yaml.load(open("state.yaml"))
+        self.interactive = bool(data["interactive"])
+        self.verify_status = bool(data["verify_status"])
+        self.simulate_type = bool(data["simulate_type"])
+        self.switch_desktop = bool(data["switch_desktop"])
+        self.sync_slides = bool(data["sync_slides"])
+        self.open_links = bool(data["open_links"])
+        self.run_hidden = bool(data["run_hidden"])
+        self.slide = int(data["slide"])
+        self.snippet = int(data["snippet"])
+
+    def save(self):
+        with open("state.yaml", "w") as f:
+            yaml.dump(dict(
+                interactive=self.interactive,
+                verify_status=self.verify_status,
+                simulate_type=self.simulate_type,
+                switch_desktop=self.switch_desktop,
+                sync_slides=self.sync_slides,
+                open_links=self.open_links,
+                run_hidden=self.run_hidden,
+                slide=self.slide,
+                snippet=self.snippet,
+                ), f, default_flow_style=False)
+
+
+state = State()
+
+
+def hrule():
+    return "="*int(subprocess.check_output(["tput", "cols"]))
+
+# A "snippet" is something that the user is supposed to do in the workshop.
+# Most of the "snippets" are shell commands.
+# Some of them can be key strokes or other actions.
+# In the markdown source, they are the code sections (identified by triple-
+# quotes) within .exercise[] sections.
+
+class Snippet(object):
+
+    def __init__(self, slide, content):
+        self.slide = slide
+        self.content = content
+        # Extract the "method" (e.g. bash, keys, ...)
+        # On multi-line snippets, the method is alone on the first line
+        # On single-line snippets, the data follows the method immediately
+        if '\n' in content:
+            self.method, self.data = content.split('\n', 1)
+        else:
+            self.method, self.data = content.split(' ', 1)
+        self.data = self.data.strip()
+        self.next = None
+
+    def __str__(self):
+        return self.content
+
+
+class Slide(object):
+
+    current_slide = 0
+
+    def __init__(self, content):
+        self.number = Slide.current_slide
+        Slide.current_slide += 1
+
+        # Remove commented-out slides
+        # (remark.js considers ??? to be the separator for speaker notes)
+        content = re.split("\n\?\?\?\n", content)[0]
+        self.content = content
+
+        self.snippets = []
+        exercises = re.findall("\.exercise\[(.*)\]", content, re.DOTALL)
+        for exercise in exercises:
+            if "```" in exercise:
+                previous = None
+                for snippet_content in exercise.split("```")[1::2]:
+                    snippet = Snippet(self, snippet_content)
+                    if previous:
+                        previous.next = snippet
+                    previous = snippet
+                    self.snippets.append(snippet)
+            else:
+                logging.warning("Exercise on slide {} does not have any ``` snippet."
+                                .format(self.number))
+                self.debug()
+
+    def __str__(self):
+        text = self.content
+        for snippet in self.snippets:
+            text = text.replace(snippet.content, ansi("7")(snippet.content))
+        return text
+
+    def debug(self):
+        logging.debug("\n{}\n{}\n{}".format(hrule(), self.content, hrule()))
+
+
+def focus_slides():
+    if not state.switch_desktop:
+        return
+    subprocess.check_output(["i3-msg", "workspace", "3"])
+    subprocess.check_output(["i3-msg", "workspace", "1"])
+
+def focus_terminal():
+    if not state.switch_desktop:
+        return
+    subprocess.check_output(["i3-msg", "workspace", "2"])
+    subprocess.check_output(["i3-msg", "workspace", "1"])
+
+def focus_browser():
+    if not state.switch_desktop:
+        return
+    subprocess.check_output(["i3-msg", "workspace", "4"])
+    subprocess.check_output(["i3-msg", "workspace", "1"])
+
+
+def ansi(code):
+    return lambda s: "\x1b[{}m{}\x1b[0m".format(code, s)
+
+
+# Sleeps the indicated delay, but interruptible by pressing ENTER.
+# If interrupted, returns True.
+def interruptible_sleep(t):
+    rfds, _, _ = select.select([0], [], [], t)
+    return 0 in rfds
+
+
+def wait_for_string(s, timeout=TIMEOUT):
+    logging.debug("Waiting for string: {}".format(s))
+    deadline = time.time() + timeout
+    while time.time() < deadline:
+        output = capture_pane()
+        if s in output:
+            return
+        if interruptible_sleep(1): return
+    raise Exception("Timed out while waiting for {}!".format(s))
+
+
+def wait_for_prompt():
+    logging.debug("Waiting for prompt.")
+    deadline = time.time() + TIMEOUT
+    while time.time() < deadline:
+        output = capture_pane()
+        # If we are not at the bottom of the screen, there will be a bunch of extra \n's
+        output = output.rstrip('\n')
+        last_line = output.split('\n')[-1]
+        # Our custom prompt on the VMs has two lines; the 2nd line is just '$'
+        if last_line == "$":
+            # This is a perfect opportunity to grab the node's IP address
+            global IPADDR
+            IPADDR = re.findall("^\[(.*)\]", output, re.MULTILINE)[-1]
+            return
+        # When we are in an alpine container, the prompt will be "/ #"
+        if last_line == "/ #":
+            return
+        # We did not recognize a known prompt; wait a bit and check again
+        logging.debug("Could not find a known prompt on last line: {!r}"
+                      .format(last_line))
+        if interruptible_sleep(1): return
+    raise Exception("Timed out while waiting for prompt!")
+
+
+def check_exit_status():
+    if not state.verify_status:
+        return
+    token = uuid.uuid4().hex
+    data = "echo {} $?\n".format(token)
+    logging.debug("Sending {!r} to get exit status.".format(data))
+    send_keys(data)
+    time.sleep(0.5)
+    wait_for_prompt()
+    screen = capture_pane()
+    status = re.findall("\n{} ([0-9]+)\n".format(token), screen, re.MULTILINE)
+    logging.debug("Got exit status: {}.".format(status))
+    if len(status) == 0:
+        raise Exception("Couldn't retrieve status code {}. Timed out?".format(token))
+    if len(status) > 1:
+        raise Exception("More than one status code {}. I'm seeing double! Shoot them both.".format(token))
+    code = int(status[0])
+    if code != 0:
+        raise Exception("Non-zero exit status: {}.".format(code))
+    # Otherwise just return peacefully.
+
+
+def setup_tmux_and_ssh():
+    if subprocess.call(["tmux", "has-session"]):
+        logging.error("Couldn't connect to tmux. Please setup tmux first.")
+        ipaddr = open("../../prepare-vms/ips.txt").read().split("\n")[0]
+        uid = os.getuid()
+
+        raise Exception("""
+1. If you're running this directly from a node:
+
+tmux
+
+2. If you want to control a remote tmux:
+
+rm -f /tmp/tmux-{uid}/default && ssh -t -L /tmp/tmux-{uid}/default:/tmp/tmux-1001/default docker@{ipaddr} tmux new-session -As 0
+
+3. If you cannot control a remote tmux:
+
+tmux new-session ssh docker@{ipaddr}
+""".format(uid=uid, ipaddr=ipaddr))
+    else:
+        logging.info("Found tmux session. Trying to acquire shell prompt.")
+        wait_for_prompt()
+    logging.info("Successfully connected to test cluster in tmux session.")
+
+
+slides = [Slide("Dummy slide zero")]
+content = open(sys.argv[1]).read()
+
+# OK, this part is definitely hackish, and will break if the
+# excludedClasses parameter is not on a single line.
+excluded_classes = re.findall("excludedClasses: (\[.*\])", content)
+excluded_classes = set(eval(excluded_classes[0]))
+
+for slide in re.split("\n---?\n", content):
+    slide_classes = re.findall("class: (.*)", slide)
+    if slide_classes:
+        slide_classes = slide_classes[0].split(",")
+        slide_classes = [c.strip() for c in slide_classes]
+    if excluded_classes & set(slide_classes):
+        logging.info("Skipping excluded slide.")
+        continue
+    slides.append(Slide(slide))
+
+
+def send_keys(data):
+    if state.simulate_type and data[0] != '^':
+        for key in data:
+            if key == ";":
+                key = "\\;"
+            if key == "\n":
+                if interruptible_sleep(1): return
+            subprocess.check_call(["tmux", "send-keys", key])
+            if interruptible_sleep(0.15*random.random()): return
+            if key == "\n":
+                if interruptible_sleep(1): return
+    else:
+        subprocess.check_call(["tmux", "send-keys", data])
+
+
+def capture_pane():
+    return subprocess.check_output(["tmux", "capture-pane", "-p"]).decode('utf-8')
+
+
+setup_tmux_and_ssh()
+
+
+try:
+    state.load()
+    logging.info("Successfully loaded state from file.")
+    # Let's override the starting state, so that when an error occurs,
+    # we can restart the auto-tester and then single-step or debug.
+    # (Instead of running again through the same issue immediately.)
+    state.interactive = True
+except Exception as e:
+    logging.exception("Could not load state from file.")
+    logging.warning("Using default values.")
+
+def move_forward():
+    state.snippet += 1
+    if state.snippet > len(slides[state.slide].snippets):
+        state.slide += 1
+        state.snippet = 0
+    check_bounds()
+
+
+def move_backward():
+    state.snippet -= 1
+    if state.snippet < 0:
+        state.slide -= 1
+        state.snippet = 0
+    check_bounds()
+
+
+def check_bounds():
+    if state.slide < 1:
+        state.slide = 1
+    if state.slide >= len(slides):
+        state.slide = len(slides)-1
+
+
+while True:
+    state.save()
+    slide = slides[state.slide]
+    snippet = slide.snippets[state.snippet-1] if state.snippet else None
+    click.clear()
+    print("[Slide {}/{}] [Snippet {}/{}] [simulate_type:{}] [verify_status:{}] "
+          "[switch_desktop:{}] [sync_slides:{}] [open_links:{}] [run_hidden:{}]"
+          .format(state.slide, len(slides)-1,
+                  state.snippet, len(slide.snippets) if slide.snippets else 0,
+                  state.simulate_type, state.verify_status,
+                  state.switch_desktop, state.sync_slides,
+                  state.open_links, state.run_hidden))
+    print(hrule())
+    if snippet:
+        print(slide.content.replace(snippet.content, ansi(7)(snippet.content)))
+        focus_terminal()
+    else:
+        print(slide.content)
+        if state.sync_slides:
+            subprocess.check_output(["./gotoslide.js", str(slide.number)])
+        focus_slides()
+    print(hrule())
+    if state.interactive:
+        print("y/⎵/⏎   Execute snippet or advance to next snippet")
+        print("p/←     Previous")
+        print("n/→     Next")
+        print("s       Simulate keystrokes")
+        print("v       Validate exit status")
+        print("d       Switch desktop")
+        print("k       Sync slides")
+        print("o       Open links")
+        print("h       Run hidden commands")
+        print("g       Go to a specific slide")
+        print("q       Quit")
+        print("c       Continue non-interactively until next error")
+        command = click.getchar()
+    else:
+        command = "y"
+
+    if command in ("n", "\x1b[C"):
+        move_forward()
+    elif command in ("p", "\x1b[D"):
+        move_backward()
+    elif command == "s":
+        state.simulate_type = not state.simulate_type
+    elif command == "v":
+        state.verify_status = not state.verify_status
+    elif command == "d":
+        state.switch_desktop = not state.switch_desktop
+    elif command == "k":
+        state.sync_slides = not state.sync_slides
+    elif command == "o":
+        state.open_links = not state.open_links
+    elif command == "h":
+        state.run_hidden = not state.run_hidden
+    elif command == "g":
+        state.slide = click.prompt("Enter slide number", type=int)
+        state.snippet = 0
+        check_bounds()
+    elif command == "q":
+        break
+    elif command == "c":
+        # continue until next timeout
+        state.interactive = False
+    elif command in ("y", "\r", " "):
+        if not snippet:
+            # Advance to next snippet
+            # Advance until a slide that has snippets
+            while not slides[state.slide].snippets:
+                move_forward()
+                # But stop if we reach the last slide
+                if state.slide == len(slides)-1:
+                    break
+            # And then advance to the snippet
+            move_forward()
+            continue
+        method, data = snippet.method, snippet.data
+        logging.info("Running with method {}: {}".format(method, data))
+        if method == "keys":
+            send_keys(data)
+        elif method == "bash" or (method == "hide" and state.run_hidden):
+            # Make sure that we're ready
+            wait_for_prompt()
+            # Strip leading spaces
+            data = re.sub("\n +", "\n", data)
+            # Remove backticks (they are used to highlight sections)
+            data = data.replace('`', '')
+            # Add "RETURN" at the end of the command :)
+            data += "\n"
+            # Send command
+            send_keys(data)
+            # Force a short sleep to avoid race condition
+            time.sleep(0.5)
+            if snippet.next and snippet.next.method == "wait":
+                wait_for_string(snippet.next.data)
+            elif snippet.next and snippet.next.method == "longwait":
+                wait_for_string(snippet.next.data, 10*TIMEOUT)
+            else:
+                wait_for_prompt()
+                # Verify return code
+                check_exit_status()
+        elif method == "copypaste":
+            screen = capture_pane()
+            matches = re.findall(data, screen, flags=re.DOTALL)
+            if len(matches) == 0:
+                raise Exception("Could not find regex {} in output.".format(data))
+            # Arbitrarily get the most recent match
+            match = matches[-1]
+            # Remove line breaks (like a screen copy paste would do)
+            match = match.replace('\n', '')
+            send_keys(match + '\n')
+            # FIXME: we should factor out the "bash" method
+            wait_for_prompt()
+            check_exit_status()
+        elif method == "open":
+            # Cheap way to get node1's IP address
+            screen = capture_pane()
+            url = data.replace("/node1", "/{}".format(IPADDR))
+            # This should probably be adapted to run on different OS
+            if state.open_links:
+                subprocess.check_output(["xdg-open", url])
+                focus_browser()
+                if state.interactive:
+                    print("Press any key to continue to next step...")
+                    click.getchar()
+        else:
+            logging.warning("Unknown method {}: {!r}".format(method, data))
+        move_forward()
+
+    else:
+        logging.warning("Unknown command {}.".format(command))

slides/autopilot/gotoslide.js

@@ -0,0 +1,17 @@
+#!/usr/bin/env node
+
+/* Expects a slide number as first argument.
+ * Will connect to the local pub/sub server,
+ * and issue a "go to slide X" command, which
+ * will be sent to all connected browsers.
+ */
+
+var io = require('socket.io-client');
+var socket = io('http://localhost:3000');
+socket.on('connect_error', function(){ 
+  console.log('connection error');
+  socket.close();
+});
+socket.emit('slide change', process.argv[2], function(){
+  socket.close();
+});

slides/autopilot/package-lock.json

@@ -0,0 +1,603 @@
+{
+  "name": "container-training-pub-sub-server",
+  "version": "0.0.1",
+  "lockfileVersion": 1,
+  "requires": true,
+  "dependencies": {
+    "accepts": {
+      "version": "1.3.4",
+      "resolved": "https://registry.npmjs.org/accepts/-/accepts-1.3.4.tgz",
+      "integrity": "sha1-hiRnWMfdbSGmR0/whKR0DsBesh8=",
+      "requires": {
+        "mime-types": "2.1.17",
+        "negotiator": "0.6.1"
+      }
+    },
+    "after": {
+      "version": "0.8.2",
+      "resolved": "https://registry.npmjs.org/after/-/after-0.8.2.tgz",
+      "integrity": "sha1-/ts5T58OAqqXaOcCvaI7UF+ufh8="
+    },
+    "array-flatten": {
+      "version": "1.1.1",
+      "resolved": "https://registry.npmjs.org/array-flatten/-/array-flatten-1.1.1.tgz",
+      "integrity": "sha1-ml9pkFGx5wczKPKgCJaLZOopVdI="
+    },
+    "arraybuffer.slice": {
+      "version": "0.0.6",
+      "resolved": "https://registry.npmjs.org/arraybuffer.slice/-/arraybuffer.slice-0.0.6.tgz",
+      "integrity": "sha1-8zshWfBTKj8xB6JywMz70a0peco="
+    },
+    "async-limiter": {
+      "version": "1.0.0",
+      "resolved": "https://registry.npmjs.org/async-limiter/-/async-limiter-1.0.0.tgz",
+      "integrity": "sha512-jp/uFnooOiO+L211eZOoSyzpOITMXx1rBITauYykG3BRYPu8h0UcxsPNB04RR5vo4Tyz3+ay17tR6JVf9qzYWg=="
+    },
+    "backo2": {
+      "version": "1.0.2",
+      "resolved": "https://registry.npmjs.org/backo2/-/backo2-1.0.2.tgz",
+      "integrity": "sha1-MasayLEpNjRj41s+u2n038+6eUc="
+    },
+    "base64-arraybuffer": {
+      "version": "0.1.5",
+      "resolved": "https://registry.npmjs.org/base64-arraybuffer/-/base64-arraybuffer-0.1.5.tgz",
+      "integrity": "sha1-c5JncZI7Whl0etZmqlzUv5xunOg="
+    },
+    "base64id": {
+      "version": "1.0.0",
+      "resolved": "https://registry.npmjs.org/base64id/-/base64id-1.0.0.tgz",
+      "integrity": "sha1-R2iMuZu2gE8OBtPnY7HDLlfY5rY="
+    },
+    "better-assert": {
+      "version": "1.0.2",
+      "resolved": "https://registry.npmjs.org/better-assert/-/better-assert-1.0.2.tgz",
+      "integrity": "sha1-QIZrnhueC1W0gYlDEeaPr/rrxSI=",
+      "requires": {
+        "callsite": "1.0.0"
+      }
+    },
+    "blob": {
+      "version": "0.0.4",
+      "resolved": "https://registry.npmjs.org/blob/-/blob-0.0.4.tgz",
+      "integrity": "sha1-vPEwUspURj8w+fx+lbmkdjCpSSE="
+    },
+    "body-parser": {
+      "version": "1.18.2",
+      "resolved": "https://registry.npmjs.org/body-parser/-/body-parser-1.18.2.tgz",
+      "integrity": "sha1-h2eKGdhLR9hZuDGZvVm84iKxBFQ=",
+      "requires": {
+        "bytes": "3.0.0",
+        "content-type": "1.0.4",
+        "debug": "2.6.9",
+        "depd": "1.1.1",
+        "http-errors": "1.6.2",
+        "iconv-lite": "0.4.19",
+        "on-finished": "2.3.0",
+        "qs": "6.5.1",
+        "raw-body": "2.3.2",
+        "type-is": "1.6.15"
+      }
+    },
+    "bytes": {
+      "version": "3.0.0",
+      "resolved": "https://registry.npmjs.org/bytes/-/bytes-3.0.0.tgz",
+      "integrity": "sha1-0ygVQE1olpn4Wk6k+odV3ROpYEg="
+    },
+    "callsite": {
+      "version": "1.0.0",
+      "resolved": "https://registry.npmjs.org/callsite/-/callsite-1.0.0.tgz",
+      "integrity": "sha1-KAOY5dZkvXQDi28JBRU+borxvCA="
+    },
+    "component-bind": {
+      "version": "1.0.0",
+      "resolved": "https://registry.npmjs.org/component-bind/-/component-bind-1.0.0.tgz",
+      "integrity": "sha1-AMYIq33Nk4l8AAllGx06jh5zu9E="
+    },
+    "component-emitter": {
+      "version": "1.2.1",
+      "resolved": "https://registry.npmjs.org/component-emitter/-/component-emitter-1.2.1.tgz",
+      "integrity": "sha1-E3kY1teCg/ffemt8WmPhQOaUJeY="
+    },
+    "component-inherit": {
+      "version": "0.0.3",
+      "resolved": "https://registry.npmjs.org/component-inherit/-/component-inherit-0.0.3.tgz",
+      "integrity": "sha1-ZF/ErfWLcrZJ1crmUTVhnbJv8UM="
+    },
+    "content-disposition": {
+      "version": "0.5.2",
+      "resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-0.5.2.tgz",
+      "integrity": "sha1-DPaLud318r55YcOoUXjLhdunjLQ="
+    },
+    "content-type": {
+      "version": "1.0.4",
+      "resolved": "https://registry.npmjs.org/content-type/-/content-type-1.0.4.tgz",
+      "integrity": "sha512-hIP3EEPs8tB9AT1L+NUqtwOAps4mk2Zob89MWXMHjHWg9milF/j4osnnQLXBCBFBk/tvIG/tUc9mOUJiPBhPXA=="
+    },
+    "cookie": {
+      "version": "0.3.1",
+      "resolved": "https://registry.npmjs.org/cookie/-/cookie-0.3.1.tgz",
+      "integrity": "sha1-5+Ch+e9DtMi6klxcWpboBtFoc7s="
+    },
+    "cookie-signature": {
+      "version": "1.0.6",
+      "resolved": "https://registry.npmjs.org/cookie-signature/-/cookie-signature-1.0.6.tgz",
+      "integrity": "sha1-4wOogrNCzD7oylE6eZmXNNqzriw="
+    },
+    "debug": {
+      "version": "2.6.9",
+      "resolved": "https://registry.npmjs.org/debug/-/debug-2.6.9.tgz",
+      "integrity": "sha512-bC7ElrdJaJnPbAP+1EotYvqZsb3ecl5wi6Bfi6BJTUcNowp6cvspg0jXznRTKDjm/E7AdgFBVeAPVMNcKGsHMA==",
+      "requires": {
+        "ms": "2.0.0"
+      }
+    },
+    "depd": {
+      "version": "1.1.1",
+      "resolved": "https://registry.npmjs.org/depd/-/depd-1.1.1.tgz",
+      "integrity": "sha1-V4O04cRZ8G+lyif5kfPQbnoxA1k="
+    },
+    "destroy": {
+      "version": "1.0.4",
+      "resolved": "https://registry.npmjs.org/destroy/-/destroy-1.0.4.tgz",
+      "integrity": "sha1-l4hXRCxEdJ5CBmE+N5RiBYJqvYA="
+    },
+    "ee-first": {
+      "version": "1.1.1",
+      "resolved": "https://registry.npmjs.org/ee-first/-/ee-first-1.1.1.tgz",
+      "integrity": "sha1-WQxhFWsK4vTwJVcyoViyZrxWsh0="
+    },
+    "encodeurl": {
+      "version": "1.0.1",
+      "resolved": "https://registry.npmjs.org/encodeurl/-/encodeurl-1.0.1.tgz",
+      "integrity": "sha1-eePVhlU0aQn+bw9Fpd5oEDspTSA="
+    },
+    "engine.io": {
+      "version": "3.1.4",
+      "resolved": "https://registry.npmjs.org/engine.io/-/engine.io-3.1.4.tgz",
+      "integrity": "sha1-PQIRtwpVLOhB/8fahiezAamkFi4=",
+      "requires": {
+        "accepts": "1.3.3",
+        "base64id": "1.0.0",
+        "cookie": "0.3.1",
+        "debug": "2.6.9",
+        "engine.io-parser": "2.1.1",
+        "uws": "0.14.5",
+        "ws": "3.3.3"
+      },
+      "dependencies": {
+        "accepts": {
+          "version": "1.3.3",
+          "resolved": "https://registry.npmjs.org/accepts/-/accepts-1.3.3.tgz",
+          "integrity": "sha1-w8p0NJOGSMPg2cHjKN1otiLChMo=",
+          "requires": {
+            "mime-types": "2.1.17",
+            "negotiator": "0.6.1"
+          }
+        }
+      }
+    },
+    "engine.io-client": {
+      "version": "3.1.4",
+      "resolved": "https://registry.npmjs.org/engine.io-client/-/engine.io-client-3.1.4.tgz",
+      "integrity": "sha1-T88TcLRxY70s6b4nM5ckMDUNTqE=",
+      "requires": {
+        "component-emitter": "1.2.1",
+        "component-inherit": "0.0.3",
+        "debug": "2.6.9",
+        "engine.io-parser": "2.1.1",
+        "has-cors": "1.1.0",
+        "indexof": "0.0.1",
+        "parseqs": "0.0.5",
+        "parseuri": "0.0.5",
+        "ws": "3.3.3",
+        "xmlhttprequest-ssl": "1.5.4",
+        "yeast": "0.1.2"
+      }
+    },
+    "engine.io-parser": {
+      "version": "2.1.1",
+      "resolved": "https://registry.npmjs.org/engine.io-parser/-/engine.io-parser-2.1.1.tgz",
+      "integrity": "sha1-4Ps/DgRi9/WLt3waUun1p+JuRmg=",
+      "requires": {
+        "after": "0.8.2",
+        "arraybuffer.slice": "0.0.6",
+        "base64-arraybuffer": "0.1.5",
+        "blob": "0.0.4",
+        "has-binary2": "1.0.2"
+      }
+    },
+    "escape-html": {
+      "version": "1.0.3",
+      "resolved": "https://registry.npmjs.org/escape-html/-/escape-html-1.0.3.tgz",
+      "integrity": "sha1-Aljq5NPQwJdN4cFpGI7wBR0dGYg="
+    },
+    "etag": {
+      "version": "1.8.1",
+      "resolved": "https://registry.npmjs.org/etag/-/etag-1.8.1.tgz",
+      "integrity": "sha1-Qa4u62XvpiJorr/qg6x9eSmbCIc="
+    },
+    "express": {
+      "version": "4.16.2",
+      "resolved": "https://registry.npmjs.org/express/-/express-4.16.2.tgz",
+      "integrity": "sha1-41xt/i1kt9ygpc1PIXgb4ymeB2w=",
+      "requires": {
+        "accepts": "1.3.4",
+        "array-flatten": "1.1.1",
+        "body-parser": "1.18.2",
+        "content-disposition": "0.5.2",
+        "content-type": "1.0.4",
+        "cookie": "0.3.1",
+        "cookie-signature": "1.0.6",
+        "debug": "2.6.9",
+        "depd": "1.1.1",
+        "encodeurl": "1.0.1",
+        "escape-html": "1.0.3",
+        "etag": "1.8.1",
+        "finalhandler": "1.1.0",
+        "fresh": "0.5.2",
+        "merge-descriptors": "1.0.1",
+        "methods": "1.1.2",
+        "on-finished": "2.3.0",
+        "parseurl": "1.3.2",
+        "path-to-regexp": "0.1.7",
+        "proxy-addr": "2.0.2",
+        "qs": "6.5.1",
+        "range-parser": "1.2.0",
+        "safe-buffer": "5.1.1",
+        "send": "0.16.1",
+        "serve-static": "1.13.1",
+        "setprototypeof": "1.1.0",
+        "statuses": "1.3.1",
+        "type-is": "1.6.15",
+        "utils-merge": "1.0.1",
+        "vary": "1.1.2"
+      }
+    },
+    "finalhandler": {
+      "version": "1.1.0",
+      "resolved": "https://registry.npmjs.org/finalhandler/-/finalhandler-1.1.0.tgz",
+      "integrity": "sha1-zgtoVbRYU+eRsvzGgARtiCU91/U=",
+      "requires": {
+        "debug": "2.6.9",
+        "encodeurl": "1.0.1",
+        "escape-html": "1.0.3",
+        "on-finished": "2.3.0",
+        "parseurl": "1.3.2",
+        "statuses": "1.3.1",
+        "unpipe": "1.0.0"
+      }
+    },
+    "forwarded": {
+      "version": "0.1.2",
+      "resolved": "https://registry.npmjs.org/forwarded/-/forwarded-0.1.2.tgz",
+      "integrity": "sha1-mMI9qxF1ZXuMBXPozszZGw/xjIQ="
+    },
+    "fresh": {
+      "version": "0.5.2",
+      "resolved": "https://registry.npmjs.org/fresh/-/fresh-0.5.2.tgz",
+      "integrity": "sha1-PYyt2Q2XZWn6g1qx+OSyOhBWBac="
+    },
+    "has-binary2": {
+      "version": "1.0.2",
+      "resolved": "https://registry.npmjs.org/has-binary2/-/has-binary2-1.0.2.tgz",
+      "integrity": "sha1-6D26SfC5vk0CbSc2U1DZ8D9Uvpg=",
+      "requires": {
+        "isarray": "2.0.1"
+      }
+    },
+    "has-cors": {
+      "version": "1.1.0",
+      "resolved": "https://registry.npmjs.org/has-cors/-/has-cors-1.1.0.tgz",
+      "integrity": "sha1-XkdHk/fqmEPRu5nCPu9J/xJv/zk="
+    },
+    "http-errors": {
+      "version": "1.6.2",
+      "resolved": "https://registry.npmjs.org/http-errors/-/http-errors-1.6.2.tgz",
+      "integrity": "sha1-CgAsyFcHGSp+eUbO7cERVfYOxzY=",
+      "requires": {
+        "depd": "1.1.1",
+        "inherits": "2.0.3",
+        "setprototypeof": "1.0.3",
+        "statuses": "1.3.1"
+      },
+      "dependencies": {
+        "setprototypeof": {
+          "version": "1.0.3",
+          "resolved": "https://registry.npmjs.org/setprototypeof/-/setprototypeof-1.0.3.tgz",
+          "integrity": "sha1-ZlZ+NwQ+608E2RvWWMDL77VbjgQ="
+        }
+      }
+    },
+    "iconv-lite": {
+      "version": "0.4.19",
+      "resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.4.19.tgz",
+      "integrity": "sha512-oTZqweIP51xaGPI4uPa56/Pri/480R+mo7SeU+YETByQNhDG55ycFyNLIgta9vXhILrxXDmF7ZGhqZIcuN0gJQ=="
+    },
+    "indexof": {
+      "version": "0.0.1",
+      "resolved": "https://registry.npmjs.org/indexof/-/indexof-0.0.1.tgz",
+      "integrity": "sha1-gtwzbSMrkGIXnQWrMpOmYFn9Q10="
+    },
+    "inherits": {
+      "version": "2.0.3",
+      "resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.3.tgz",
+      "integrity": "sha1-Yzwsg+PaQqUC9SRmAiSA9CCCYd4="
+    },
+    "ipaddr.js": {
+      "version": "1.5.2",
+      "resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-1.5.2.tgz",
+      "integrity": "sha1-1LUFvemUaYfM8PxY2QEP+WB+P6A="
+    },
+    "isarray": {
+      "version": "2.0.1",
+      "resolved": "https://registry.npmjs.org/isarray/-/isarray-2.0.1.tgz",
+      "integrity": "sha1-o32U7ZzaLVmGXJ92/llu4fM4dB4="
+    },
+    "media-typer": {
+      "version": "0.3.0",
+      "resolved": "https://registry.npmjs.org/media-typer/-/media-typer-0.3.0.tgz",
+      "integrity": "sha1-hxDXrwqmJvj/+hzgAWhUUmMlV0g="
+    },
+    "merge-descriptors": {
+      "version": "1.0.1",
+      "resolved": "https://registry.npmjs.org/merge-descriptors/-/merge-descriptors-1.0.1.tgz",
+      "integrity": "sha1-sAqqVW3YtEVoFQ7J0blT8/kMu2E="
+    },
+    "methods": {
+      "version": "1.1.2",
+      "resolved": "https://registry.npmjs.org/methods/-/methods-1.1.2.tgz",
+      "integrity": "sha1-VSmk1nZUE07cxSZmVoNbD4Ua/O4="
+    },
+    "mime": {
+      "version": "1.4.1",
+      "resolved": "https://registry.npmjs.org/mime/-/mime-1.4.1.tgz",
+      "integrity": "sha512-KI1+qOZu5DcW6wayYHSzR/tXKCDC5Om4s1z2QJjDULzLcmf3DvzS7oluY4HCTrc+9FiKmWUgeNLg7W3uIQvxtQ=="
+    },
+    "mime-db": {
+      "version": "1.30.0",
+      "resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.30.0.tgz",
+      "integrity": "sha1-dMZD2i3Z1qRTmZY0ZbJtXKfXHwE="
+    },
+    "mime-types": {
+      "version": "2.1.17",
+      "resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.17.tgz",
+      "integrity": "sha1-Cdejk/A+mVp5+K+Fe3Cp4KsWVXo=",
+      "requires": {
+        "mime-db": "1.30.0"
+      }
+    },
+    "ms": {
+      "version": "2.0.0",
+      "resolved": "https://registry.npmjs.org/ms/-/ms-2.0.0.tgz",
+      "integrity": "sha1-VgiurfwAvmwpAd9fmGF4jeDVl8g="
+    },
+    "negotiator": {
+      "version": "0.6.1",
+      "resolved": "https://registry.npmjs.org/negotiator/-/negotiator-0.6.1.tgz",
+      "integrity": "sha1-KzJxhOiZIQEXeyhWP7XnECrNDKk="
+    },
+    "object-component": {
+      "version": "0.0.3",
+      "resolved": "https://registry.npmjs.org/object-component/-/object-component-0.0.3.tgz",
+      "integrity": "sha1-8MaapQ78lbhmwYb0AKM3acsvEpE="
+    },
+    "on-finished": {
+      "version": "2.3.0",
+      "resolved": "https://registry.npmjs.org/on-finished/-/on-finished-2.3.0.tgz",
+      "integrity": "sha1-IPEzZIGwg811M3mSoWlxqi2QaUc=",
+      "requires": {
+        "ee-first": "1.1.1"
+      }
+    },
+    "parseqs": {
+      "version": "0.0.5",
+      "resolved": "https://registry.npmjs.org/parseqs/-/parseqs-0.0.5.tgz",
+      "integrity": "sha1-1SCKNzjkZ2bikbouoXNoSSGouJ0=",
+      "requires": {
+        "better-assert": "1.0.2"
+      }
+    },
+    "parseuri": {
+      "version": "0.0.5",
+      "resolved": "https://registry.npmjs.org/parseuri/-/parseuri-0.0.5.tgz",
+      "integrity": "sha1-gCBKUNTbt3m/3G6+J3jZDkvOMgo=",
+      "requires": {
+        "better-assert": "1.0.2"
+      }
+    },
+    "parseurl": {
+      "version": "1.3.2",
+      "resolved": "https://registry.npmjs.org/parseurl/-/parseurl-1.3.2.tgz",
+      "integrity": "sha1-/CidTtiZMRlGDBViUyYs3I3mW/M="
+    },
+    "path-to-regexp": {
+      "version": "0.1.7",
+      "resolved": "https://registry.npmjs.org/path-to-regexp/-/path-to-regexp-0.1.7.tgz",
+      "integrity": "sha1-32BBeABfUi8V60SQ5yR6G/qmf4w="
+    },
+    "proxy-addr": {
+      "version": "2.0.2",
+      "resolved": "https://registry.npmjs.org/proxy-addr/-/proxy-addr-2.0.2.tgz",
+      "integrity": "sha1-ZXFQT0e7mI7IGAJT+F3X4UlSvew=",
+      "requires": {
+        "forwarded": "0.1.2",
+        "ipaddr.js": "1.5.2"
+      }
+    },
+    "qs": {
+      "version": "6.5.1",
+      "resolved": "https://registry.npmjs.org/qs/-/qs-6.5.1.tgz",
+      "integrity": "sha512-eRzhrN1WSINYCDCbrz796z37LOe3m5tmW7RQf6oBntukAG1nmovJvhnwHHRMAfeoItc1m2Hk02WER2aQ/iqs+A=="
+    },
+    "range-parser": {
+      "version": "1.2.0",
+      "resolved": "https://registry.npmjs.org/range-parser/-/range-parser-1.2.0.tgz",
+      "integrity": "sha1-9JvmtIeJTdxA3MlKMi9hEJLgDV4="
+    },
+    "raw-body": {
+      "version": "2.3.2",
+      "resolved": "https://registry.npmjs.org/raw-body/-/raw-body-2.3.2.tgz",
+      "integrity": "sha1-vNYMd9Prk83gBQKVw/N5OJvIj4k=",
+      "requires": {
+        "bytes": "3.0.0",
+        "http-errors": "1.6.2",
+        "iconv-lite": "0.4.19",
+        "unpipe": "1.0.0"
+      }
+    },
+    "safe-buffer": {
+      "version": "5.1.1",
+      "resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.1.1.tgz",
+      "integrity": "sha512-kKvNJn6Mm93gAczWVJg7wH+wGYWNrDHdWvpUmHyEsgCtIwwo3bqPtV4tR5tuPaUhTOo/kvhVwd8XwwOllGYkbg=="
+    },
+    "send": {
+      "version": "0.16.1",
+      "resolved": "https://registry.npmjs.org/send/-/send-0.16.1.tgz",
+      "integrity": "sha512-ElCLJdJIKPk6ux/Hocwhk7NFHpI3pVm/IZOYWqUmoxcgeyM+MpxHHKhb8QmlJDX1pU6WrgaHBkVNm73Sv7uc2A==",
+      "requires": {
+        "debug": "2.6.9",
+        "depd": "1.1.1",
+        "destroy": "1.0.4",
+        "encodeurl": "1.0.1",
+        "escape-html": "1.0.3",
+        "etag": "1.8.1",
+        "fresh": "0.5.2",
+        "http-errors": "1.6.2",
+        "mime": "1.4.1",
+        "ms": "2.0.0",
+        "on-finished": "2.3.0",
+        "range-parser": "1.2.0",
+        "statuses": "1.3.1"
+      }
+    },
+    "serve-static": {
+      "version": "1.13.1",
+      "resolved": "https://registry.npmjs.org/serve-static/-/serve-static-1.13.1.tgz",
+      "integrity": "sha512-hSMUZrsPa/I09VYFJwa627JJkNs0NrfL1Uzuup+GqHfToR2KcsXFymXSV90hoyw3M+msjFuQly+YzIH/q0MGlQ==",
+      "requires": {
+        "encodeurl": "1.0.1",
+        "escape-html": "1.0.3",
+        "parseurl": "1.3.2",
+        "send": "0.16.1"
+      }
+    },
+    "setprototypeof": {
+      "version": "1.1.0",
+      "resolved": "https://registry.npmjs.org/setprototypeof/-/setprototypeof-1.1.0.tgz",
+      "integrity": "sha512-BvE/TwpZX4FXExxOxZyRGQQv651MSwmWKZGqvmPcRIjDqWub67kTKuIMx43cZZrS/cBBzwBcNDWoFxt2XEFIpQ=="
+    },
+    "socket.io": {
+      "version": "2.0.4",
+      "resolved": "https://registry.npmjs.org/socket.io/-/socket.io-2.0.4.tgz",
+      "integrity": "sha1-waRZDO/4fs8TxyZS8Eb3FrKeYBQ=",
+      "requires": {
+        "debug": "2.6.9",
+        "engine.io": "3.1.4",
+        "socket.io-adapter": "1.1.1",
+        "socket.io-client": "2.0.4",
+        "socket.io-parser": "3.1.2"
+      }
+    },
+    "socket.io-adapter": {
+      "version": "1.1.1",
+      "resolved": "https://registry.npmjs.org/socket.io-adapter/-/socket.io-adapter-1.1.1.tgz",
+      "integrity": "sha1-KoBeihTWNyEk3ZFZrUUC+MsH8Gs="
+    },
+    "socket.io-client": {
+      "version": "2.0.4",
+      "resolved": "https://registry.npmjs.org/socket.io-client/-/socket.io-client-2.0.4.tgz",
+      "integrity": "sha1-CRilUkBtxeVAs4Dc2Xr8SmQzL44=",
+      "requires": {
+        "backo2": "1.0.2",
+        "base64-arraybuffer": "0.1.5",
+        "component-bind": "1.0.0",
+        "component-emitter": "1.2.1",
+        "debug": "2.6.9",
+        "engine.io-client": "3.1.4",
+        "has-cors": "1.1.0",
+        "indexof": "0.0.1",
+        "object-component": "0.0.3",
+        "parseqs": "0.0.5",
+        "parseuri": "0.0.5",
+        "socket.io-parser": "3.1.2",
+        "to-array": "0.1.4"
+      }
+    },
+    "socket.io-parser": {
+      "version": "3.1.2",
+      "resolved": "https://registry.npmjs.org/socket.io-parser/-/socket.io-parser-3.1.2.tgz",
+      "integrity": "sha1-28IoIVH8T6675Aru3Ady66YZ9/I=",
+      "requires": {
+        "component-emitter": "1.2.1",
+        "debug": "2.6.9",
+        "has-binary2": "1.0.2",
+        "isarray": "2.0.1"
+      }
+    },
+    "statuses": {
+      "version": "1.3.1",
+      "resolved": "https://registry.npmjs.org/statuses/-/statuses-1.3.1.tgz",
+      "integrity": "sha1-+vUbnrdKrvOzrPStX2Gr8ky3uT4="
+    },
+    "to-array": {
+      "version": "0.1.4",
+      "resolved": "https://registry.npmjs.org/to-array/-/to-array-0.1.4.tgz",
+      "integrity": "sha1-F+bBH3PdTz10zaek/zI46a2b+JA="
+    },
+    "type-is": {
+      "version": "1.6.15",
+      "resolved": "https://registry.npmjs.org/type-is/-/type-is-1.6.15.tgz",
+      "integrity": "sha1-yrEPtJCeRByChC6v4a1kbIGARBA=",
+      "requires": {
+        "media-typer": "0.3.0",
+        "mime-types": "2.1.17"
+      }
+    },
+    "ultron": {
+      "version": "1.1.1",
+      "resolved": "https://registry.npmjs.org/ultron/-/ultron-1.1.1.tgz",
+      "integrity": "sha512-UIEXBNeYmKptWH6z8ZnqTeS8fV74zG0/eRU9VGkpzz+LIJNs8W/zM/L+7ctCkRrgbNnnR0xxw4bKOr0cW0N0Og=="
+    },
+    "unpipe": {
+      "version": "1.0.0",
+      "resolved": "https://registry.npmjs.org/unpipe/-/unpipe-1.0.0.tgz",
+      "integrity": "sha1-sr9O6FFKrmFltIF4KdIbLvSZBOw="
+    },
+    "utils-merge": {
+      "version": "1.0.1",
+      "resolved": "https://registry.npmjs.org/utils-merge/-/utils-merge-1.0.1.tgz",
+      "integrity": "sha1-n5VxD1CiZ5R7LMwSR0HBAoQn5xM="
+    },
+    "uws": {
+      "version": "0.14.5",
+      "resolved": "https://registry.npmjs.org/uws/-/uws-0.14.5.tgz",
+      "integrity": "sha1-Z6rzPEaypYel9mZtAPdpEyjxSdw=",
+      "optional": true
+    },
+    "vary": {
+      "version": "1.1.2",
+      "resolved": "https://registry.npmjs.org/vary/-/vary-1.1.2.tgz",
+      "integrity": "sha1-IpnwLG3tMNSllhsLn3RSShj2NPw="
+    },
+    "ws": {
+      "version": "3.3.3",
+      "resolved": "https://registry.npmjs.org/ws/-/ws-3.3.3.tgz",
+      "integrity": "sha512-nnWLa/NwZSt4KQJu51MYlCcSQ5g7INpOrOMt4XV8j4dqTXdmlUmSHQ8/oLC069ckre0fRsgfvsKwbTdtKLCDkA==",
+      "requires": {
+        "async-limiter": "1.0.0",
+        "safe-buffer": "5.1.1",
+        "ultron": "1.1.1"
+      }
+    },
+    "xmlhttprequest-ssl": {
+      "version": "1.5.4",
+      "resolved": "https://registry.npmjs.org/xmlhttprequest-ssl/-/xmlhttprequest-ssl-1.5.4.tgz",
+      "integrity": "sha1-BPVgkVcks4kIhxXMDteBPpZ3v1c="
+    },
+    "yeast": {
+      "version": "0.1.2",
+      "resolved": "https://registry.npmjs.org/yeast/-/yeast-0.1.2.tgz",
+      "integrity": "sha1-AI4G2AlDIMNy28L47XagymyKxBk="
+    }
+  }
+}

slides/autopilot/package.json

@@ -0,0 +1,8 @@
+{
+  "name": "container-training-pub-sub-server",
+  "version": "0.0.1",
+  "dependencies": {
+    "express": "^4.16.2",
+    "socket.io": "^2.0.4"
+  }
+}

slides/autopilot/remote.js

@@ -0,0 +1,21 @@
+/* This snippet is loaded from the workshop HTML file.
+ * It sets up callbacks to synchronize the local slide
+ * number with the remote pub/sub server.
+ */
+
+var socket = io();
+var leader = true;
+
+slideshow.on('showSlide', function (slide) {
+  if (leader) {
+    var n = slide.getSlideIndex()+1;
+    socket.emit('slide change', n);
+  }
+});
+
+socket.on('slide change', function (n) {
+  leader = false;
+  slideshow.gotoSlide(n);
+  leader = true;
+});
+

slides/autopilot/requirements.txt

@@ -0,0 +1 @@
+click

slides/autopilot/server.js

@@ -0,0 +1,41 @@
+#!/usr/bin/env node
+
+/* This is a very simple pub/sub server, allowing to
+ * remote control browsers displaying the slides.
+ * The browsers connect to this pub/sub server using
+ * Socket.IO, and the server tells them which slides
+ * to display.
+ *
+ * The server can be controlled with a little CLI,
+ * or by one of the browsers.
+ */
+
+var express = require('express');
+var app = express();
+var http = require('http').Server(app);
+var io = require('socket.io')(http);
+
+app.get('/', function(req, res){
+  res.send('container.training autopilot pub/sub server');
+});
+
+/* Serve remote.js from the current directory */
+app.use(express.static('.'));
+
+/* Serve slides etc. from current and the parent directory */
+app.use(express.static('..'));
+
+io.on('connection', function(socket){
+  console.log('a client connected: ' + socket.handshake.address);
+  socket.on('slide change', function(n, ack){
+    console.log('slide change: ' + n);
+    socket.broadcast.emit('slide change', n);
+    if (typeof ack === 'function') {
+      ack();
+    }
+  });
+});
+
+http.listen(3000, function(){
+  console.log('listening on *:3000');
+});

slides/autopilot/tmux-style.sh

@@ -0,0 +1,7 @@
+#!/bin/sh
+# This removes the clock (and other extraneous stuff) from the
+# tmux status bar, and it gives it a non-default color.
+tmux set-option -g status-left ""
+tmux set-option -g status-right ""
+tmux set-option -g status-style bg=cyan
+

slides/build.sh

@@ -1,15 +1,29 @@
 #!/bin/sh
+set -e
 case "$1" in
 once)
+  ./index.py
   for YAML in *.yml; do
-    ./markmaker.py < $YAML > $YAML.html || { 
+    ./markmaker.py $YAML > $YAML.html || { 
       rm $YAML.html
       break
     }
   done
+  if [ -n "$SLIDECHECKER" ]; then
+    for YAML in *.yml; do
+      ./appendcheck.py $YAML.html
+    done
+  fi
   ;;
 
 forever)
+  set +e
+  # check if entr is installed
+  if ! command -v entr >/dev/null; then
+    echo >&2 "First install 'entr' with apt, brew, etc."
+    exit
+  fi
+  
   # There is a weird bug in entr, at least on MacOS,
   # where it doesn't restore the terminal to a clean
   # state when exitting. So let's try to work around

slides/common/sampleapp.md

@@ -1,477 +0,0 @@
-# Our sample application
-
-- Visit the GitHub repository with all the materials of this workshop:
-  <br/>https://github.com/jpetazzo/orchestration-workshop
-
-- The application is in the [dockercoins](
-  https://github.com/jpetazzo/orchestration-workshop/tree/master/dockercoins)
-  subdirectory
-
-- Let's look at the general layout of the source code:
-
-  there is a Compose file [docker-compose.yml](
-  https://github.com/jpetazzo/orchestration-workshop/blob/master/dockercoins/docker-compose.yml) ...
-
-  ... and 4 other services, each in its own directory:
-
-  - `rng` = web service generating random bytes
-  - `hasher` = web service computing hash of POSTed data
-  - `worker` = background process using `rng` and `hasher`
-  - `webui` = web interface to watch progress
-
----
-
-class: extra-details
-
-## Compose file format version
-
-*Particularly relevant if you have used Compose before...*
-
-- Compose 1.6 introduced support for a new Compose file format (aka "v2")
-
-- Services are no longer at the top level, but under a `services` section
-
-- There has to be a `version` key at the top level, with value `"2"` (as a string, not an integer)
-
-- Containers are placed on a dedicated network, making links unnecessary
-
-- There are other minor differences, but upgrade is easy and straightforward
-
----
-
-## Links, naming, and service discovery
-
-- Containers can have network aliases (resolvable through DNS)
-
-- Compose file version 2+ makes each container reachable through its service name
-
-- Compose file version 1 did require "links" sections
-
-- Our code can connect to services using their short name
-
-  (instead of e.g. IP address or FQDN)
-
-- Network aliases are automatically namespaced
-
-  (i.e. you can have multiple apps declaring and using a service named `database`)
-
----
-
-## Example in `worker/worker.py`
-
-![Service discovery](images/service-discovery.png)
-
----
-
-## What's this application?
-
----
-
-class: pic
-
-![DockerCoins logo](images/dockercoins.png)
-
-(DockerCoins 2016 logo courtesy of [@XtlCnslt](https://twitter.com/xtlcnslt) and [@ndeloof](https://twitter.com/ndeloof). Thanks!)
-
----
-
-## What's this application?
-
-- It is a DockerCoin miner! 💰🐳📦🚢
-
---
-
-- No, you can't buy coffee with DockerCoins
-
---
-
-- How DockerCoins works:
-
-  - `worker` asks to `rng` to generate a few random bytes
-
-  - `worker` feeds these bytes into `hasher`
-
-  - and repeat forever!
-
-  - every second, `worker` updates `redis` to indicate how many loops were done
-
-  - `webui` queries `redis`, and computes and exposes "hashing speed" in your browser
-
----
-
-## Getting the application source code
-
-- We will clone the GitHub repository
-
-- The repository also contains scripts and tools that we will use through the workshop
-
-.exercise[
-
-<!--
-```bash
-if [ -d orchestration-workshop ]; then
-  mv orchestration-workshop orchestration-workshop.$$
-fi
-```
--->
-
-- Clone the repository on `node1`:
-  ```bash
-  git clone git://github.com/jpetazzo/orchestration-workshop
-  ```
-
-]
-
-(You can also fork the repository on GitHub and clone your fork if you prefer that.)
-
----
-
-# Running the application
-
-Without further ado, let's start our application.
-
-.exercise[
-
-- Go to the `dockercoins` directory, in the cloned repo:
-  ```bash
-  cd ~/orchestration-workshop/dockercoins
-  ```
-
-- Use Compose to build and run all containers:
-  ```bash
-  docker-compose up
-  ```
-
-<!--
-```wait units of work done```
-```keys ^C```
--->
-
-]
-
-Compose tells Docker to build all container images (pulling
-the corresponding base images), then starts all containers,
-and displays aggregated logs.
-
----
-
-## Lots of logs
-
-- The application continuously generates logs
-
-- We can see the `worker` service making requests to `rng` and `hasher`
-
-- Let's put that in the background
-
-.exercise[
-
-- Stop the application by hitting `^C`
-
-]
-
-- `^C` stops all containers by sending them the `TERM` signal
-
-- Some containers exit immediately, others take longer
-  <br/>(because they don't handle `SIGTERM` and end up being killed after a 10s timeout)
-
----
-
-## Restarting in the background
-
-- Many flags and commands of Compose are modeled after those of `docker`
-
-.exercise[
-
-- Start the app in the background with the `-d` option:
-  ```bash
-  docker-compose up -d
-  ```
-
-- Check that our app is running with the `ps` command:
-  ```bash
-  docker-compose ps
-  ```
-
-]
-
-`docker-compose ps` also shows the ports exposed by the application.
-
----
-
-class: extra-details
-
-## Viewing logs
-
-- The `docker-compose logs` command works like `docker logs`
-
-.exercise[
-
-- View all logs since container creation and exit when done:
-  ```bash
-  docker-compose logs
-  ```
-
-- Stream container logs, starting at the last 10 lines for each container:
-  ```bash
-  docker-compose logs --tail 10 --follow
-  ```
-
-<!--
-```wait units of work done```
-```keys ^C```
--->
-
-]
-
-Tip: use `^S` and `^Q` to pause/resume log output.
-
----
-
-class: extra-details
-
-## Upgrading from Compose 1.6
-
-.warning[The `logs` command has changed between Compose 1.6 and 1.7!]
-
-- Up to 1.6
-
-  - `docker-compose logs` is the equivalent of `logs --follow`
-
-  - `docker-compose logs` must be restarted if containers are added
-
-- Since 1.7
-
-  - `--follow` must be specified explicitly
-
-  - new containers are automatically picked up by `docker-compose logs`
-
----
-
-## Connecting to the web UI
-
-- The `webui` container exposes a web dashboard; let's view it
-
-.exercise[
-
-- With a web browser, connect to `node1` on port 8000
-
-- Remember: the `nodeX` aliases are valid only on the nodes themselves
-
-- In your browser, you need to enter the IP address of your node
-
-<!-- ```open http://node1:8000``` -->
-
-]
-
-You should see a speed of approximately 4 hashes/second.
-
-More precisely: 4 hashes/second, with regular dips down to zero.
-<br/>This is because Jérôme is incapable of writing good frontend code.
-<br/>Don't ask. Seriously, don't ask. This is embarrassing.
-
----
-
-class: extra-details
-
-## Why does the speed seem irregular?
-
-- The app actually has a constant, steady speed: 3.33 hashes/second
-  <br/>
-  (which corresponds to 1 hash every 0.3 seconds, for *reasons*)
-
-- The worker doesn't update the counter after every loop, but up to once per second
-
-- The speed is computed by the browser, checking the counter about once per second
-
-- Between two consecutive updates, the counter will increase either by 4, or by 0
-
-- The perceived speed will therefore be 4 - 4 - 4 - 0 - 4 - 4 - etc.
-
-*We told you to not ask!!!*
-
----
-
-## Scaling up the application
-
-- Our goal is to make that performance graph go up (without changing a line of code!)
-
---
-
-- Before trying to scale the application, we'll figure out if we need more resources
-
-  (CPU, RAM...)
-
-- For that, we will use good old UNIX tools on our Docker node
-
----
-
-## Looking at resource usage
-
-- Let's look at CPU, memory, and I/O usage
-
-.exercise[
-
-- run `top` to see CPU and memory usage (you should see idle cycles)
-
-<!--
-```bash top```
-
-```wait Tasks```
-```keys ^C```
--->
-
-- run `vmstat 1` to see I/O usage (si/so/bi/bo)
-  <br/>(the 4 numbers should be almost zero, except `bo` for logging)
-
-<!--
-```bash vmstat 1```
-
-```wait memory```
-```keys ^C```
--->
-
-]
-
-We have available resources.
-
-- Why?
-- How can we use them?
-
----
-
-## Scaling workers on a single node
-
-- Docker Compose supports scaling
-- Let's scale `worker` and see what happens!
-
-.exercise[
-
-- Start one more `worker` container:
-  ```bash
-  docker-compose scale worker=2
-  ```
-
-- Look at the performance graph (it should show a x2 improvement)
-
-- Look at the aggregated logs of our containers (`worker_2` should show up)
-
-- Look at the impact on CPU load with e.g. top (it should be negligible)
-
-]
-
----
-
-## Adding more workers
-
-- Great, let's add more workers and call it a day, then!
-
-.exercise[
-
-- Start eight more `worker` containers:
-  ```bash
-  docker-compose scale worker=10
-  ```
-
-- Look at the performance graph: does it show a x10 improvement?
-
-- Look at the aggregated logs of our containers
-
-- Look at the impact on CPU load and memory usage
-
-]
-
----
-
-# Identifying bottlenecks
-
-- You should have seen a 3x speed bump (not 10x)
-
-- Adding workers didn't result in linear improvement
-
-- *Something else* is slowing us down
-
---
-
-- ... But what?
-
---
-
-- The code doesn't have instrumentation
-
-- Let's use state-of-the-art HTTP performance analysis!
-  <br/>(i.e. good old tools like `ab`, `httping`...)
-
----
-
-## Accessing internal services
-
-- `rng` and `hasher` are exposed on ports 8001 and 8002
-
-- This is declared in the Compose file:
-
-  ```yaml
-    ...
-    rng:
-      build: rng
-      ports:
-      - "8001:80"
-
-    hasher:
-      build: hasher
-      ports:
-      - "8002:80"
-    ...
-  ```
-
----
-
-## Measuring latency under load
-
-We will use `httping`.
-
-.exercise[
-
-- Check the latency of `rng`:
-  ```bash
-  httping -c 10 localhost:8001
-  ```
-
-- Check the latency of `hasher`:
-  ```bash
-  httping -c 10 localhost:8002
-  ```
-
-]
-
-`rng` has a much higher latency than `hasher`.
-
----
-
-## Let's draw hasty conclusions
-
-- The bottleneck seems to be `rng`
-
-- *What if* we don't have enough entropy and can't generate enough random numbers?
-
-- We need to scale out the `rng` service on multiple machines!
-
-Note: this is a fiction! We have enough entropy. But we need a pretext to scale out.
-
-(In fact, the code of `rng` uses `/dev/urandom`, which never runs out of entropy...
-<br/>
-...and is [just as good as `/dev/random`](http://www.slideshare.net/PacSecJP/filippo-plain-simple-reality-of-entropy).)
-
----
-
-## Clean up
-
-- Before moving on, let's remove those containers
-
-.exercise[
-
-- Tell Compose to remove everything:
-  ```bash
-  docker-compose down
-  ```
-
-]

slides/containers/Advanced_Dockerfiles.md

@@ -1,9 +1,6 @@
-
-class: title
-
 # Advanced Dockerfiles
 
-![construction](images/construction.jpg)
+![construction](images/title-advanced-dockerfiles.jpg)
 
 ---
 
@@ -37,18 +34,6 @@ In this section, we will see more Dockerfile commands.
 
 ---
 
-## The `MAINTAINER` instruction
-
-The `MAINTAINER` instruction tells you who wrote the `Dockerfile`.
-
-```dockerfile
-MAINTAINER Docker Education Team <education@docker.com>
-```
-
-It's optional but recommended.
-
----
-
 ## The `RUN` instruction
 
 The `RUN` instruction can be specified in two ways.
@@ -97,8 +82,6 @@ RUN apt-get update && apt-get install -y wget && apt-get clean
 
 It is also possible to break a command onto multiple lines:
 
-It is possible to execute multiple commands in a single step:
-
 ```dockerfile
 RUN apt-get update \
  && apt-get install -y wget \
@@ -372,7 +355,7 @@ class: extra-details
 
 ## Overriding the `ENTRYPOINT` instruction
 
-The entry point can be overriden as well.
+The entry point can be overridden as well.
 
 ```bash
 $ docker run -it training/ls
@@ -433,5 +416,4 @@ ONBUILD COPY . /src
 ```
 
 * You can't chain `ONBUILD` instructions with `ONBUILD`.
-* `ONBUILD` can't be used to trigger `FROM` and `MAINTAINER`
-  instructions.
+* `ONBUILD` can't be used to trigger `FROM` instructions.

slides/containers/Ambassadors.md

@@ -3,7 +3,7 @@ class: title
 
 # Ambassadors
 
-![](images/ambassador.jpg)
+![Two serious-looking persons shaking hands](images/title-ambassador.jpg)
 
 ---
 
@@ -40,7 +40,9 @@ ambassador containers.
 
 ---
 
-![ambassador](images/diagram.png)
+class: pic
+
+![ambassador](images/ambassador-diagram.png)
 
 ---
 

slides/containers/Application_Configuration.md

@@ -0,0 +1,201 @@
+# Application Configuration
+
+There are many ways to provide configuration to containerized applications.
+
+There is no "best way" — it depends on factors like:
+
+* configuration size,
+
+* mandatory and optional parameters,
+
+* scope of configuration (per container, per app, per customer, per site, etc),
+
+* frequency of changes in the configuration.
+
+---
+
+## Command-line parameters
+
+```bash
+docker run jpetazzo/hamba 80 www1:80 www2:80
+```
+
+* Configuration is provided through command-line parameters.
+
+* In the above example, the `ENTRYPOINT` is a script that will:
+
+  - parse the parameters,
+
+  - generate a configuration file,
+
+  - start the actual service.
+
+---
+
+## Command-line parameters pros and cons
+
+* Appropriate for mandatory parameters (without which the service cannot start).
+
+* Convenient for "toolbelt" services instanciated many times.
+
+  (Because there is no extra step: just run it!)
+
+* Not great for dynamic configurations or bigger configurations.
+
+  (These things are still possible, but more cumbersome.)
+
+---
+
+## Environment variables
+
+```bash
+docker run -e ELASTICSEARCH_URL=http://es42:9201/ kibana
+```
+
+* Configuration is provided through environment variables.
+
+* The environment variable can be used straight by the program,
+  <br/>or by a script generating a configuration file.
+
+---
+
+## Environment variables pros and cons
+
+* Appropriate for optional parameters (since the image can provide default values).
+
+* Also convenient for services instanciated many times.
+
+  (It's as easy as command-line parameters.)
+
+* Great for services with lots of parameters, but you only want to specify a few.
+
+  (And use default values for everything else.)
+
+* Ability to introspect possible parameters and their default values.
+
+* Not great for dynamic configurations.
+
+---
+
+## Baked-in configuration
+
+```
+FROM prometheus
+COPY prometheus.conf /etc
+```
+
+* The configuration is added to the image.
+
+* The image may have a default configuration; the new configuration can:
+
+  - replace the default configuration,
+
+  - extend it (if the code can read multiple configuration files).
+
+---
+
+## Baked-in configuration pros and cons
+
+* Allows arbitrary customization and complex configuration files.
+
+* Requires to write a configuration file. (Obviously!)
+
+* Requires to build an image to start the service.
+
+* Requires to rebuild the image to reconfigure the service.
+
+* Requires to rebuild the image to upgrade the service.
+
+* Configured images can be stored in registries.
+
+  (Which is great, but requires a registry.)
+
+---
+
+## Configuration volume
+
+```bash
+docker run -v appconfig:/etc/appconfig myapp
+```
+
+* The configuration is stored in a volume.
+
+* The volume is attached to the container.
+
+* The image may have a default configuration.
+
+  (But this results in a less "obvious" setup, that needs more documentation.)
+
+---
+
+## Configuration volume pros and cons
+
+* Allows arbitrary customization and complex configuration files.
+
+* Requires to create a volume for each different configuration.
+
+* Services with identical configurations can use the same volume.
+
+* Doesn't require to build / rebuild an image when upgrading / reconfiguring.
+
+* Configuration can be generated or edited through another container.
+
+---
+
+## Dynamic configuration volume
+
+* This is a powerful pattern for dynamic, complex configurations.
+
+* The configuration is stored in a volume.
+
+* The configuration is generated / updated by a special container.
+
+* The application container detects when the configuration is changed.
+
+  (And automatically reloads the configuration when necessary.)
+
+* The configuration can be shared between multiple services if needed.
+
+---
+
+## Dynamic configuration volume example
+
+In a first terminal, start a load balancer with an initial configuration:
+
+```bash
+$ docker run --name loadbalancer jpetazzo/hamba \
+  80 goo.gl:80
+```
+
+In another terminal, reconfigure that load balancer:
+
+```bash
+$ docker run --rm --volumes-from loadbalancer jpetazzo/hamba reconfigure \
+  80 google.com:80
+```
+
+The configuration could also be updated through e.g. a REST API.
+
+(The REST API being itself served from another container.)
+
+---
+
+## Keeping secrets
+
+.warning[Ideally, you should not put secrets (passwords, tokens...) in:]
+
+* command-line or environment variables (anyone with Docker API access can get them),
+
+* images, especially stored in a registry.
+
+Secrets management is better handled with an orchestrator (like Swarm or Kubernetes).
+
+Orchestrators will allow to pass secrets in a "one-way" manner.
+
+Managing secrets securely without an orchestrator can be contrived.
+
+E.g.:
+
+- read the secret on stdin when the service starts,
+
+- pass the secret using an API endpoint.

slides/containers/Background_Containers.md

@@ -1,9 +1,9 @@
 
 class: title
 
-# Background Containers
+# Background containers
 
-![Background Containers](images/background-containers.jpg)
+![Background containers](images/title-background-containers.jpg)
 
 ---
 
@@ -117,7 +117,7 @@ CONTAINER ID  IMAGE           ...  CREATED        STATUS        ...
 
 Many Docker commands will work on container IDs: `docker stop`, `docker rm`...
 
-If we want to list only the IDs of our containers (without the other colums
+If we want to list only the IDs of our containers (without the other columns
 or the header line),
 we can use the `-q` ("Quiet", "Quick") flag:
 

slides/containers/Building_Images_Interactively.md

@@ -1,4 +1,4 @@
-# Building Images Interactively
+# Building images interactively
 
 In this section, we will create our first container image.
 
@@ -16,27 +16,21 @@ We will:
 
 ---
 
-## Building Images Interactively
+## The plan
 
-As we have seen, the images on the Docker Hub are sometimes very basic.
+1. Create a container (with `docker run`) using our base distro of choice.
 
-How do we want to construct our own images?
+2. Run a bunch of commands to install and set up our software in the container.
 
-As an example, we will build an image that has `figlet`.
+3. (Optionally) review changes in the container with `docker diff`.
 
-First, we will do it manually with `docker commit`.
+4. Turn the container into a new image with `docker commit`.
 
-Then, in an upcoming chapter, we will use a `Dockerfile` and `docker build`.
+5. (Optionally) add tags to the image with `docker tag`.
 
 ---
 
-## Building from a base
-
-Our base will be the `ubuntu` image.
-
----
-
-## Create a new container and make some changes
+## Setting up our container
 
 Start an Ubuntu container:
 
@@ -107,7 +101,7 @@ As explained before:
 
 ---
 
-## Commit and run your image
+## Commit our changes into a new image
 
 The `docker commit` command will create a new layer with those changes,
 and a new image using this new layer.
@@ -119,7 +113,13 @@ $ docker commit <yourContainerId>
 
 The output of the `docker commit` command will be the ID for your newly created image.
 
-We can run this image:
+We can use it as an argument to `docker run`.
+
+---
+
+## Testing our new image
+
+Let's run this image:
 
 ```bash
 $ docker run -it <newImageId>
@@ -131,6 +131,8 @@ root@fcfb62f0bfde:/# figlet hello
 |_| |_|\___|_|_|\___/ 
 ```
 
+It works! .emoji[🎉]
+
 ---
 
 ## Tagging images

slides/containers/Building_Images_With_Dockerfiles.md

@@ -3,7 +3,7 @@ class: title
 
 # Building Docker images with a Dockerfile
 
-![construction](images/construction.jpg)
+![Construction site with containers](images/title-building-docker-images-with-a-dockerfile.jpg)
 
 ---
 
@@ -93,20 +93,22 @@ The output of `docker build` looks like this:
 
 .small[
 ```bash
-$ docker build -t figlet .
-Sending build context to Docker daemon 2.048 kB
-Sending build context to Docker daemon 
-Step 0 : FROM ubuntu
- ---> e54ca5efa2e9
-Step 1 : RUN apt-get update
- ---> Running in 840cb3533193
- ---> 7257c37726a1
-Removing intermediate container 840cb3533193
-Step 2 : RUN apt-get install figlet
- ---> Running in 2b44df762a2f
- ---> f9e8f1642759
-Removing intermediate container 2b44df762a2f
-Successfully built f9e8f1642759
+docker build -t figlet .
+Sending build context to Docker daemon  2.048kB
+Step 1/3 : FROM ubuntu
+ ---> f975c5035748
+Step 2/3 : RUN apt-get update
+ ---> Running in e01b294dbffd
+(...output of the RUN command...)
+Removing intermediate container e01b294dbffd
+ ---> eb8d9b561b37
+Step 3/3 : RUN apt-get install figlet
+ ---> Running in c29230d70f9b
+(...output of the RUN command...)
+Removing intermediate container c29230d70f9b
+ ---> 0dfd7a253f21
+Successfully built 0dfd7a253f21
+Successfully tagged figlet:latest
 ```
 ]
 
@@ -134,20 +136,20 @@ Sending build context to Docker daemon 2.048 kB
 ## Executing each step
 
 ```bash
-Step 1 : RUN apt-get update
- ---> Running in 840cb3533193
+Step 2/3 : RUN apt-get update
+ ---> Running in e01b294dbffd
 (...output of the RUN command...)
- ---> 7257c37726a1
-Removing intermediate container 840cb3533193
+Removing intermediate container e01b294dbffd
+ ---> eb8d9b561b37
 ```
 
-* A container (`840cb3533193`) is created from the base image.
+* A container (`e01b294dbffd`) is created from the base image.
 
 * The `RUN` command is executed in this container.
 
-* The container is committed into an image (`7257c37726a1`).
+* The container is committed into an image (`eb8d9b561b37`).
 
-* The build container (`840cb3533193`) is removed.
+* The build container (`e01b294dbffd`) is removed.
 
 * The output of this step will be the base image for the next one.
 
@@ -188,7 +190,7 @@ root@91f3c974c9a1:/# figlet hello
 ```
 
 
-Yay! 🎉
+Yay! .emoji[🎉]
 
 ---
 

slides/containers/Cmd_And_Entrypoint.md

@@ -1,7 +1,7 @@
 
 class: title
 
-# CMD and ENTRYPOINT
+# `CMD` and `ENTRYPOINT`
 
 ![Container entry doors](images/entrypoint.jpg)
 
@@ -64,6 +64,7 @@ Let's build it:
 $ docker build -t figlet .
 ...
 Successfully built 042dff3b4a8d
+Successfully tagged figlet:latest
 ```
 
 And run it:
@@ -141,7 +142,7 @@ Why did we use JSON syntax for our `ENTRYPOINT`?
 
 * When CMD or ENTRYPOINT use string syntax, they get wrapped in `sh -c`.
 
-* To avoid this wrapping, you must use JSON syntax.
+* To avoid this wrapping, we can use JSON syntax.
 
 What if we used `ENTRYPOINT` with string syntax?
 
@@ -165,6 +166,7 @@ Let's build it:
 $ docker build -t figlet .
 ...
 Successfully built 36f588918d73
+Successfully tagged figlet:latest
 ```
 
 And run it:
@@ -178,8 +180,6 @@ $ docker run figlet salut
  \/ \_/|_/|__/ \_/|_/|_/
 ```
 
-Great success!
-
 ---
 
 ## Using `CMD` and `ENTRYPOINT` together
@@ -225,11 +225,11 @@ Let's build it:
 $ docker build -t figlet .
 ...
 Successfully built 6e0b6a048a07
+Successfully tagged figlet:latest
 ```
 
-And run it:
+Run it without parameters:
 
-.small[
 ```bash
 $ docker run figlet
  _          _   _                             _        
@@ -237,7 +237,15 @@ $ docker run figlet
 | |     _  | | | |  __             __   ,_   | |  __|  
 |/ \   |/  |/  |/  /  \_  |  |  |_/  \_/  |  |/  /  |  
 |   |_/|__/|__/|__/\__/    \/ \/  \__/    |_/|__/\_/|_/
+```
 
+---
+
+## Overriding the image default parameters
+
+Now let's pass extra arguments to the image.
+
+```bash
 $ docker run figlet hola mundo
  _           _                                               
 | |         | |                                      |       
@@ -245,7 +253,8 @@ $ docker run figlet hola mundo
 |/ \   /  \_|/  /  |    / |/ |/ |  |   |  / |/ |  /  |  /  \_
 |   |_/\__/ |__/\_/|_/    |  |  |_/ \_/|_/  |  |_/\_/|_/\__/ 
 ```
-]
+
+We overrode `CMD` but still used `ENTRYPOINT`.
 
 ---
 

slides/containers/Compose_For_Dev_Stacks.md

@@ -1,5 +1,4 @@
-
-# Compose For Development Stacks
+# Compose for development stacks
 
 Dockerfiles are great to build container images.
 
@@ -50,7 +49,7 @@ Before diving in, let's see a small example of Compose in action.
 
 ---
 
-## Compose in action
+class: pic
 
 ![composeup](images/composeup.gif)
 
@@ -61,6 +60,10 @@ Before diving in, let's see a small example of Compose in action.
 If you are using the official training virtual machines, Compose has been
 pre-installed.
 
+If you are using Docker for Mac/Windows or the Docker Toolbox, Compose comes with them.
+
+If you are on Linux (desktop or server environment), you will need to install Compose from its [release page](https://github.com/docker/compose/releases) or with `pip install docker-compose`.
+
 You can always check that it is installed by running:
 
 ```bash
@@ -113,6 +116,7 @@ them.
 
 Here is the file used in the demo:
 
+.small[
 ```yaml
 version: "2"
 
@@ -131,25 +135,37 @@ services:
   redis:
     image: redis
 ```
+]
+
+---
+
+## Compose file structure
+
+A Compose file has multiple sections:
+
+* `version` is mandatory. (We should use `"2"` or later; version 1 is deprecated.)
+
+* `services` is mandatory. A service is one or more replicas of the same image running as containers.
+
+* `networks` is optional and indicates to which networks containers should be connected.
+  <br/>(By default, containers will be connected on a private, per-compose-file network.)
+
+* `volumes` is optional and can define volumes to be used and/or shared by the containers.
 
 ---
 
 ## Compose file versions
 
-Version 1 directly has the various containers (`www`, `redis`...) at the top level of the  file.
+* Version 1 is legacy and shouldn't be used.
 
-Version 2 has multiple sections:
+  (If you see a Compose file without `version` and `services`, it's a legacy v1 file.)
 
-* `version` is mandatory and should be `"2"`.
+* Version 2 added support for networks and volumes.
 
-* `services` is mandatory and corresponds to the content of the version 1 format.
+* Version 3 added support for deployment options (scaling, rolling updates, etc).
 
-* `networks` is optional and indicates to which networks containers should be connected.
-  <br/>(By default, containers will be connected on a private, per-app network.)
-
-* `volumes` is optional and can define volumes to be used and/or shared by the containers.
-
-Version 3 adds support for deployment options (scaling, rolling updates, etc.)
+The [Docker documentation](https://docs.docker.com/compose/compose-file/)
+has excellent information about the Compose file format if you need to know more about versions.
 
 ---
 
@@ -259,6 +275,8 @@ Removing trainingwheels_www_1 ... done
 Removing trainingwheels_redis_1 ... done
 ```
 
+Use `docker-compose down -v` to remove everything including volumes.
+
 ---
 
 ## Special handling of volumes

slides/containers/Connecting_Containers_With_Links.md

@@ -1,9 +1,9 @@
 
 class: title
 
-# Connecting Containers With Links
+# Connecting containers with links
 
-![graph](images/graph.gif)
+![graph](images/title-connecting-containers-with-links.gif)
 
 ---
 

slides/containers/Container_Engines.md

@@ -0,0 +1,177 @@
+# Docker Engine and other container engines
+
+* We are going to cover the architecture of the Docker Engine.
+
+* We will also present other container engines.
+
+---
+
+class: pic
+
+## Docker Engine external architecture
+
+![](images/docker-engine-architecture.svg)
+
+---
+
+## Docker Engine external architecture
+
+* The Engine is a daemon (service running in the background).
+
+* All interaction is done through a REST API exposed over a socket.
+
+* On Linux, the default socket is a UNIX socket: `/var/run/docker.sock`.
+
+* We can also use a TCP socket, with optional mutual TLS authentication.
+
+* The `docker` CLI communicates with the Engine over the socket.
+
+Note: strictly speaking, the Docker API is not fully REST.
+
+Some operations (e.g. dealing with interactive containers
+and log streaming) don't fit the REST model.
+ 
+---
+
+class: pic
+
+## Docker Engine internal architecture
+
+![](images/dockerd-and-containerd.png)
+
+---
+
+## Docker Engine internal architecture
+
+* Up to Docker 1.10: the Docker Engine is one single monolithic binary.
+
+* Starting with Docker 1.11, the Engine is split into multiple parts:
+
+  - `dockerd` (REST API, auth, networking, storage)
+
+  - `containerd` (container lifecycle, controlled over a gRPC API)
+
+  - `containerd-shim` (per-container; does almost nothing but allows to restart the Engine without restarting the containers)
+
+  - `runc` (per-container; does the actual heavy lifting to start the container)
+
+* Some features (like image and snapshot management) are progressively being pushed from `dockerd` to `containerd`.
+
+For more details, check [this short presentation by Phil Estes](https://www.slideshare.net/PhilEstes/diving-through-the-layers-investigating-runc-containerd-and-the-docker-engine-architecture).
+
+---
+
+## Other container engines
+
+The following list is not exhaustive.
+
+Furthermore, we limited the scope to Linux containers.
+
+Containers also exist (sometimes with other names) on Windows, macOS, Solaris, FreeBSD ...
+
+---
+
+## LXC
+
+* The venerable ancestor (first released in 2008).
+
+* Docker initially relied on it to execute containers.
+
+* No daemon; no central API.
+
+* Each container is managed by a `lxc-start` process.
+
+* Each `lxc-start` process exposes a custom API over a local UNIX socket, allowing to interact with the container.
+
+* No notion of image (container filesystems have to be managed manually).
+
+* Networking has to be setup manually.
+
+---
+
+## LXD
+
+* Re-uses LXC code (through liblxc).
+
+* Builds on top of LXC to offer a more modern experience.
+
+* Daemon exposing a REST API.
+
+* Can manage images, snapshots, migrations, networking, storage.
+
+* "offers a user experience similar to virtual machines but using Linux containers instead."
+
+---
+
+## rkt
+
+* Compares to `runc`.
+
+* No daemon or API.
+
+* Strong emphasis on security (through privilege separation).
+
+* Networking has to be setup separately (e.g. through CNI plugins).
+
+* Partial image management (pull, but no push).
+
+  (Image build is handled by separate tools.)
+
+---
+
+## CRI-O
+
+* Designed to be used with Kubernetes as a simple, basic runtime.
+
+* Compares to `containerd`.
+
+* Daemon exposing a gRPC interface.
+
+* Controlled using the CRI API (Container Runtime Interface defined by Kubernetes).
+
+* Needs an underlying OCI runtime (e.g. runc).
+
+* Handles storage, images, networking (through CNI plugins).
+
+We're not aware of anyone using it directly (i.e. outside of Kubernetes).
+
+---
+
+## systemd
+
+* "init" system (PID 1) in most modern Linux distributions.
+
+* Offers tools like `systemd-nspawn` and `machinectl` to manage containers.
+
+* `systemd-nspawn` is "In many ways it is similar to chroot(1), but more powerful".
+
+* `machinectl` can interact with VMs and containers managed by systemd.
+
+* Exposes a DBUS API.
+
+* Basic image support (tar archives and raw disk images).
+
+* Network has to be setup manually.
+
+---
+
+## Overall ...
+
+* The Docker Engine is very developer-centric:
+
+  - easy to install
+
+  - easy to use
+
+  - no manual setup
+
+  - first-class image build and transfer
+
+* As a result, it is a fantastic tool in development environments.
+
+* On servers:
+
+  - Docker is a good default choice
+
+  - If you use Kubernetes, the engine doesn't matter
+

slides/containers/Container_Network_Model.md

@@ -3,7 +3,7 @@ class: title
 
 # The Container Network Model
 
-![A denser graph network](images/complex-network.jpg)
+![A denser graph network](images/title-the-container-network-model.jpg)
 
 ---
 
@@ -65,9 +65,17 @@ eb0eeab782f4        host                host
 
 * A network is managed by a *driver*.
 
-* All the drivers that we have seen before are available.
+* The built-in drivers include:
 
-* A new multi-host driver, *overlay*, is available out of the box.
+  * `bridge` (default)
+
+  * `none`
+
+  * `host`
+
+  * `macvlan`
+
+* A multi-host driver, *overlay*, is available out of the box (for Swarm clusters).
 
 * More drivers can be provided by plugins (OVS, VLAN...)
 
@@ -75,6 +83,8 @@ eb0eeab782f4        host                host
 
 ---
 
+class: extra-details
+
 ## Differences with the CNI
 
 * CNI = Container Network Interface
@@ -87,6 +97,22 @@ eb0eeab782f4        host                host
 
 ---
 
+class: pic
+
+## Single container in a Docker network
+
+![bridge0](images/bridge1.png)
+
+---
+
+class: pic
+
+## Two containers on two Docker networks
+
+![bridge3](images/bridge2.png)
+
+---
+
 ## Creating a network
 
 Let's create a network called `dev`.
@@ -126,13 +152,16 @@ $ docker run -d --name es --net dev elasticsearch:2
 
 Now, create another container on this network.
 
+.small[
 ```bash
 $ docker run -ti --net dev alpine sh
 root@0ecccdfa45ef:/#
 ```
+]
 
 From this new container, we can resolve and ping the other one, using its assigned name:
 
+.small[
 ```bash
 / # ping es
 PING es (172.18.0.2) 56(84) bytes of data.
@@ -145,6 +174,7 @@ PING es (172.18.0.2) 56(84) bytes of data.
 rtt min/avg/max/mdev = 0.114/0.149/0.221/0.052 ms
 root@0ecccdfa45ef:/#
 ```
+]
 
 ---
 
@@ -155,6 +185,7 @@ class: extra-details
 In Docker Engine 1.9, name resolution is implemented with `/etc/hosts`, and
 updating it each time containers are added/removed.
 
+.small[
 ```bash
 [root@0ecccdfa45ef /]# cat /etc/hosts
 172.18.0.3  0ecccdfa45ef
@@ -167,6 +198,7 @@ ff02::2 ip6-allrouters
 172.18.0.2      es
 172.18.0.2      es.dev
 ```
+]
 
 In Docker Engine 1.10, this has been replaced by a dynamic resolver.
 
@@ -174,7 +206,7 @@ In Docker Engine 1.10, this has been replaced by a dynamic resolver.
 
 ---
 
-## Connecting multiple containers together
+# Service discovery with containers
 
 * Let's try to run an application that requires two containers.
 
@@ -210,9 +242,7 @@ $ docker ps -l
 
 * If we connect to the application now, we will see an error page:
 
-.small[
 ![Trainingwheels error](images/trainingwheels-error.png)
-]
 
 * This is because the Redis service is not running.
 * This container tries to resolve the name `redis`.
@@ -241,9 +271,7 @@ $ docker run --net dev --name redis -d redis
 
 * If we connect to the application now, we should see that the app is working correctly:
 
-.small[
 ![Trainingwheels OK](images/trainingwheels-ok.png)
-]
 
 * When the app tries to resolve `redis`, instead of getting a DNS error, it gets the IP address of our Redis container.
 
@@ -282,7 +310,7 @@ since we wiped out the old Redis container).
 
 ---
 
-class: x-extra-details
+class: extra-details
 
 ## Names are *local* to each network
 
@@ -322,7 +350,7 @@ class: extra-details
 Create the `prod` network.
 
 ```bash
-$ docker create network prod
+$ docker network create prod
 5a41562fecf2d8f115bedc16865f7336232a04268bdf2bd816aecca01b68d50c
 ```
 
@@ -362,6 +390,7 @@ Each ElasticSearch instance has a name (generated when it is started). This name
 
 Try the following command a few times:
 
+.small[
 ```bash
 $ docker run --rm --net dev centos curl -s es:9200
 {
@@ -369,9 +398,11 @@ $ docker run --rm --net dev centos curl -s es:9200
 ...
 }
 ```
+]
 
 Then try it a few times by replacing `--net dev` with `--net prod`:
 
+.small[
 ```bash
 $ docker run --rm --net prod centos curl -s es:9200
 {
@@ -379,6 +410,7 @@ $ docker run --rm --net prod centos curl -s es:9200
 ...
 }
 ```
+]
 
 ---
 
@@ -428,7 +460,7 @@ When creating a network, extra options can be provided.
 
 * `--internal` disables outbound traffic (the network won't have a default gateway).
 
-* `--gateway` indicates which address to use for the gateway (when utbound traffic is allowed).
+* `--gateway` indicates which address to use for the gateway (when outbound traffic is allowed).
 
 * `--subnet` (in CIDR notation) indicates the subnet to use.
 
@@ -466,11 +498,13 @@ b2887adeb5578a01fd9c55c435cad56bbbe802350711d2743691f95743680b09
 
 * If containers span multiple hosts, we need an *overlay* network to connect them together.
 
-* Docker ships with a default network plugin, `overlay`, implementing an overlay network leveraging VXLAN.
+* Docker ships with a default network plugin, `overlay`, implementing an overlay network leveraging
+  VXLAN, *enabled with Swarm Mode*.
 
 * Other plugins (Weave, Calico...) can provide overlay networks as well.
 
-* Once you have an overlay network, *all the features that we've used in this chapter work identically.*
+* Once you have an overlay network, *all the features that we've used in this chapter work identically
+  across multiple hosts.*
 
 ---
 
@@ -486,7 +520,7 @@ Very short instructions:
 - `docker network create mynet --driver overlay`
 - `docker service create --network mynet myimage`
 
-See http://jpetazzo.github.io/orchestration-workshop for all the deets about clustering!
+See http://jpetazzo.github.io/container.training for all the deets about clustering!
 
 ---
 
@@ -508,13 +542,174 @@ General idea:
 
 ---
 
-## Section summary
+## Connecting and disconnecting dynamically
 
-We've learned how to:
+* So far, we have specified which network to use when starting the container.
 
-* Create private networks for groups of containers.
+* The Docker Engine also allows to connect and disconnect while the container runs.
 
-* Assign IP addresses to containers.
+* This feature is exposed through the Docker API, and through two Docker CLI commands:
 
-* Use container naming to implement service discovery.
+  * `docker network connect <network> <container>`
 
+  * `docker network disconnect <network> <container>`
+
+---
+
+## Dynamically connecting to a network
+
+* We have a container named `es` connected to a network named `dev`.
+
+* Let's start a simple alpine container on the default network:
+
+  ```bash
+  $ docker run -ti alpine sh
+  / #
+  ```
+
+* In this container, try to ping the `es` container:
+
+  ```bash
+  / # ping es
+  ping: bad address 'es'
+  ```
+
+  This doesn't work, but we will change that by connecting the container.
+
+---
+
+## Finding the container ID and connecting it
+
+* Figure out the ID of our alpine container; here are two methods:
+
+  * looking at `/etc/hostname` in the container,
+
+  * running `docker ps -lq` on the host.
+
+* Run the following command on the host:
+
+  ```bash
+  $ docker network connect dev `<container_id>`
+  ```
+
+---
+
+## Checking what we did
+
+* Try again to `ping es` from the container.
+
+* It should now work correctly:
+
+  ```bash
+  / # ping es
+  PING es (172.20.0.3): 56 data bytes
+  64 bytes from 172.20.0.3: seq=0 ttl=64 time=0.376 ms
+  64 bytes from 172.20.0.3: seq=1 ttl=64 time=0.130 ms
+  ^C
+  ```
+
+* Interrupt it with Ctrl-C.
+
+---
+
+## Looking at the network setup in the container
+
+We can look at the list of network interfaces with `ifconfig`, `ip a`, or `ip l`:
+
+.small[
+```bash
+/ # ip a
+1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
+    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
+    inet 127.0.0.1/8 scope host lo
+       valid_lft forever preferred_lft forever
+18: eth0@if19: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP
+    link/ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:ff
+    inet 172.17.0.2/16 brd 172.17.255.255 scope global eth0
+       valid_lft forever preferred_lft forever
+20: eth1@if21: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP
+    link/ether 02:42:ac:14:00:04 brd ff:ff:ff:ff:ff:ff
+    inet 172.20.0.4/16 brd 172.20.255.255 scope global eth1
+       valid_lft forever preferred_lft forever
+/ #
+```
+]
+
+Each network connection is materialized with a virtual network interface.
+
+As we can see, we can be connected to multiple networks at the same time.
+
+---
+
+## Disconnecting from a network
+
+* Let's try the symmetrical command to disconnect the container:
+  ```bash
+  $ docker network disconnect dev <container_id>
+  ```
+
+* From now on, if we try to ping `es`, it will not resolve:
+  ```bash
+  / # ping es
+  ping: bad address 'es'
+  ```
+
+* Trying to ping the IP address directly won't work either:
+  ```bash
+  / # ping 172.20.0.3
+  ... (nothing happens until we interrupt it with Ctrl-C)
+  ```
+
+---
+
+class: extra-details
+
+## Network aliases are scoped per network
+
+* Each network has its own set of network aliases.
+
+* We saw this earlier: `es` resolves to different addresses in `dev` and `prod`.
+
+* If we are connected to multiple networks, the resolver looks up names in each of them
+  (as of Docker Engine 18.03, it is the connection order) and stops as soon as the name
+  is found.
+
+* Therefore, if we are connected to both `dev` and `prod`, resolving `es` will **not**
+  give us the addresses of all the `es` services; but only the ones in `dev` or `prod`.
+
+* However, we can lookup `es.dev` or `es.prod` if we need to.
+
+---
+
+class: extra-details
+
+## Finding out about our networks and names
+
+* We can do reverse DNS lookups on containers' IP addresses.
+
+* If the IP address belongs to a network (other than the default bridge), the result will be:
+
+  ```
+  name-or-first-alias-or-container-id.network-name
+  ```
+
+* Example:
+
+.small[
+```bash
+$ docker run -ti --net prod --net-alias hello alpine
+/ # apk add --no-cache drill
+...
+OK: 5 MiB in 13 packages
+/ # ifconfig
+eth0      Link encap:Ethernet  HWaddr 02:42:AC:15:00:03
+          inet addr:`172.21.0.3`  Bcast:172.21.255.255  Mask:255.255.0.0
+          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
+...
+/ # drill -t ptr `3.0.21.172`.in-addr.arpa
+...
+;; ANSWER SECTION:
+3.0.21.172.in-addr.arpa.	600	IN	PTR	`hello.prod`.
+...
+```
+]

slides/containers/Container_Networking_Basics.md

@@ -1,9 +1,9 @@
 
 class: title
 
-# Container Networking Basics
+# Container networking basics
 
-![A dense graph network](images/network.jpg)
+![A dense graph network](images/title-container-networking-basics.jpg)
 
 ---
 
@@ -49,14 +49,14 @@ We will use `docker ps`:
 
 ```bash
 $ docker ps
-CONTAINER ID  IMAGE  ...  PORTS                                          ...
-e40ffb406c9e  nginx  ...  0.0.0.0:32769->80/tcp, 0.0.0.0:32768->443/tcp  ...
+CONTAINER ID  IMAGE  ...  PORTS                  ...
+e40ffb406c9e  nginx  ...  0.0.0.0:32768->80/tcp  ...
 ```
 
 
-* The web server is running on ports 80 and 443 inside the container.
+* The web server is running on port 80 inside the container.
 
-* Those ports are mapped to ports 32769 and 32768 on our Docker host.
+* This port is mapped to port 32768 on our Docker host.
 
 We will explain the whys and hows of this port mapping.
 
@@ -69,7 +69,7 @@ But first, let's make sure that everything works properly.
 Point your browser to the IP address of your Docker host, on the port
 shown by `docker ps` for container port 80.
 
-![Screenshot](images/web.png)
+![Screenshot](images/welcome-to-nginx.png)
 
 ---
 
@@ -81,7 +81,7 @@ Make sure to use the right port number if it is different
 from the example below:
 
 ```bash
-$ curl localhost:32769
+$ curl localhost:32768
 <!DOCTYPE html>
 <html>
 <head>
@@ -91,6 +91,31 @@ $ curl localhost:32769
 
 ---
 
+## How does Docker know which port to map?
+
+* There is metadata in the image telling "this image has something on port 80".
+
+* We can see that metadata with `docker inspect`:
+
+```bash
+$ docker inspect --format '{{.Config.ExposedPorts}}' nginx
+map[80/tcp:{}]
+```
+
+* This metadata was set in the Dockerfile, with the `EXPOSE` keyword.
+
+* We can see that with `docker history`:
+
+```bash
+$ docker history nginx
+IMAGE               CREATED             CREATED BY
+7f70b30f2cc6        11 days ago         /bin/sh -c #(nop)  CMD ["nginx" "-g" "…
+<missing>           11 days ago         /bin/sh -c #(nop)  STOPSIGNAL [SIGTERM]
+<missing>           11 days ago         /bin/sh -c #(nop)  EXPOSE 80/tcp
+```
+
+---
+
 ## Why are we mapping ports?
 
 * We are out of IPv4 addresses.
@@ -113,7 +138,7 @@ There is a command to help us:
 
 ```bash
 $ docker port <containerID> 80
-32769
+32768
 ```
 
 ---
@@ -128,7 +153,7 @@ $ docker run -d -p 8000:80 nginx
 $ docker run -d -p 8080:80 -p 8888:80 nginx
 ```
 
-* We are running two NGINX web servers.
+* We are running three NGINX web servers.
 * The first one is exposed on port 80.
 * The second one is exposed on port 8000.
 * The third one is exposed on ports 8080 and 8888.
@@ -189,87 +214,6 @@ $ ping <ipAddress>
 
 ---
 
-## The different network drivers
-
-A container can use one of the following drivers:
-
-* `bridge` (default)
-* `none`
-* `host`
-* `container`
-
-The driver is selected with `docker run --net ...`.
-
-The different drivers are explained with more details on the following slides.
-
----
-
-## The default bridge
-
-* By default, the container gets a virtual `eth0` interface.
-  <br/>(In addition to its own private `lo` loopback interface.)
-
-* That interface is provided by a `veth` pair.
-
-* It is connected to the Docker bridge.
-  <br/>(Named `docker0` by default; configurable with `--bridge`.)
-
-* Addresses are allocated on a private, internal subnet.
-  <br/>(Docker uses 172.17.0.0/16 by default; configurable with `--bip`.)
-
-* Outbound traffic goes through an iptables MASQUERADE rule.
-
-* Inbound traffic goes through an iptables DNAT rule.
-
-* The container can have its own routes, iptables rules, etc.
-
----
-
-## The null driver
-
-* Container is started with `docker run --net none ...`
-
-* It only gets the `lo` loopback interface. No `eth0`.
-
-* It can't send or receive network traffic.
-
-* Useful for isolated/untrusted workloads.
-
----
-
-## The host driver
-
-* Container is started with `docker run --net host ...`
-
-* It sees (and can access) the network interfaces of the host.
-
-* It can bind any address, any port (for ill and for good).
-
-* Network traffic doesn't have to go through NAT, bridge, or veth.
-
-* Performance = native!
-
-Use cases:
-
-* Performance sensitive applications (VOIP, gaming, streaming...)
-
-* Peer discovery (e.g. Erlang port mapper, Raft, Serf...)
-
----
-
-## The container driver
-
-* Container is started with `docker run --net container:id ...`
-
-* It re-uses the network stack of another container.
-
-* It shares with this other container the same interfaces, IP address(es), routes, iptables rules, etc.
-
-* Those containers can communicate over their `lo` interface.
-  <br/>(i.e. one can bind to 127.0.0.1 and the others can connect to it.)
-
----
-
 ## Section summary
 
 We've learned how to:

slides/containers/Containers_From_Scratch.md

@@ -0,0 +1,3 @@
+# Building containers from scratch
+
+(This is a "bonus section" done if time permits.)

slides/containers/Copy_On_Write.md

@@ -0,0 +1,339 @@
+# Copy-on-write filesystems
+
+Container engines rely on copy-on-write to be able
+to start containers quickly, regardless of their size.
+
+We will explain how that works, and review some of
+the copy-on-write storage systems available on Linux.
+
+---
+
+## What is copy-on-write?
+
+- Copy-on-write is a mechanism allowing to share data.
+
+- The data appears to be a copy, but is only
+  a link (or reference) to the original data.
+
+- The actual copy happens only when someone
+  tries to change the shared data.
+
+- Whoever changes the shared data ends up
+  using their own copy instead of the shared data.
+
+---
+
+## A few metaphors
+
+--
+
+- First metaphor:
+  <br/>white board and tracing paper
+
+--
+
+- Second metaphor:
+  <br/>magic books with shadowy pages
+
+--
+
+- Third metaphor:
+  <br/>just-in-time house building
+
+---
+
+## Copy-on-write is *everywhere*
+
+- Process creation with `fork()`.
+
+- Consistent disk snapshots.
+
+- Efficient VM provisioning.
+
+- And, of course, containers.
+
+---
+
+## Copy-on-write and containers
+
+Copy-on-write is essential to give us "convenient" containers.
+
+- Creating a new container (from an existing image) is "free".
+
+  (Otherwise, we would have to copy the image first.)
+
+- Customizing a container (by tweaking a few files) is cheap.
+
+  (Adding a 1 KB configuration file to a 1 GB container takes 1 KB, not 1 GB.)
+
+- We can take snapshots, i.e. have "checkpoints" or "save points"
+  when building images.
+
+---
+
+## AUFS overview
+
+- The original (legacy) copy-on-write filesystem used by first versions of Docker.
+
+- Combine multiple *branches* in a specific order.
+
+- Each branch is just a normal directory.
+
+- You generally have:
+
+  - at least one read-only branch (at the bottom),
+
+  - exactly one read-write branch (at the top).
+
+  (But other fun combinations are possible too!)
+
+---
+
+## AUFS operations: opening a file
+
+- With `O_RDONLY` - read-only access:
+
+  - look it up in each branch, starting from the top
+
+  - open the first one we find
+
+- With `O_WRONLY` or `O_RDWR` - write access:
+
+  - if the file exists on the top branch: open it
+
+  - if the file exists on another branch: "copy up"
+    <br/>
+    (i.e. copy the file to the top branch and open the copy)
+
+  - if the file doesn't exist on any branch: create it on the top branch
+
+That "copy-up" operation can take a while if the file is big!
+
+---
+
+## AUFS operations: deleting a file
+
+- A *whiteout* file is created.
+
+- This is similar to the concept of "tombstones" used in some data systems.
+
+```
+ # docker run ubuntu rm /etc/shadow
+
+ # ls -la /var/lib/docker/aufs/diff/$(docker ps --no-trunc -lq)/etc
+ total 8
+ drwxr-xr-x 2 root root 4096 Jan 27 15:36 .
+ drwxr-xr-x 5 root root 4096 Jan 27 15:36 ..
+ -r--r--r-- 2 root root    0 Jan 27 15:36 .wh.shadow
+```
+
+---
+
+## AUFS performance
+
+- AUFS `mount()` is fast, so creation of containers is quick.
+
+- Read/write access has native speeds.
+
+- But initial `open()` is expensive in two scenarios:
+
+  - when writing big files (log files, databases ...),
+
+  - when searching many directories (PATH, classpath, etc.) over many layers.
+
+- Protip: when we built dotCloud, we ended up putting
+  all important data on *volumes*.
+
+- When starting the same container multiple times:
+
+  - the data is loaded only once from disk, and cached only once in memory;
+
+  - but `dentries` will be duplicated.
+
+---
+
+## Device Mapper
+
+Device Mapper is a rich subsystem with many features.
+
+It can be used for: RAID, encrypted devices, snapshots, and more.
+
+In the context of containers (and Docker in particular), "Device Mapper"
+means:
+
+"the Device Mapper system + its *thin provisioning target*"
+
+If you see the abbreviation "thinp" it stands for "thin provisioning".
+
+---
+
+## Device Mapper principles
+
+- Copy-on-write happens on the *block* level
+  (instead of the *file* level).
+
+- Each container and each image get their own block device.
+
+- At any given time, it is possible to take a snapshot:
+
+  - of an existing container (to create a frozen image),
+
+  - of an existing image (to create a container from it).
+
+- If a block has never been written to:
+
+  - it's assumed to be all zeros,
+
+  - it's not allocated on disk.
+
+(That last property is the reason for the name "thin" provisioning.)
+
+---
+
+## Device Mapper operational details
+
+- Two storage areas are needed:
+  one for *data*, another for *metadata*.
+
+- "data" is also called the "pool"; it's just a big pool of blocks.
+
+  (Docker uses the smallest possible block size, 64 KB.)
+
+- "metadata" contains the mappings between virtual offsets (in the
+  snapshots) and physical offsets (in the pool).
+
+- Each time a new block (or a copy-on-write block) is written,
+  a block is allocated from the pool.
+
+- When there are no more blocks in the pool, attempts to write
+  will stall until the pool is increased (or the write operation
+  aborted).
+
+- In other words: when running out of space, containers are
+  frozen, but operations will resume as soon as space is available.
+
+---
+
+## Device Mapper performance
+
+- By default, Docker puts data and metadata on a loop device
+  backed by a sparse file.
+
+- This is great from a usability point of view,
+  since zero configuration is needed.
+
+- But it is terrible from a performance point of view:
+
+  - each time a container writes to a new block,
+  - a block has to be allocated from the pool,
+  - and when it's written to,
+  - a block has to be allocated from the sparse file,
+  - and sparse file performance isn't great anyway.
+
+- If you use Device Mapper, make sure to put data (and metadata)
+  on devices!
+
+---
+
+## BTRFS principles
+
+- BTRFS is a filesystem (like EXT4, XFS, NTFS...) with built-in snapshots.
+
+- The "copy-on-write" happens at the filesystem level.
+
+- BTRFS integrates the snapshot and block pool management features
+  at the filesystem level.
+
+  (Instead of the block level for Device Mapper.)
+
+- In practice, we create a "subvolume" and 
+  later take a "snapshot" of that subvolume.
+
+  Imagine: `mkdir` with Super Powers and `cp -a` with Super Powers.
+
+- These operations can be executed with the `btrfs` CLI tool.
+
+---
+
+## BTRFS in practice with Docker
+
+- Docker can use BTRFS and its snapshotting features to store container images.
+
+- The only requirement is that `/var/lib/docker` is on a BTRFS filesystem.
+
+  (Or, the directory specified with the `--data-root` flag when starting the engine.)
+
+---
+
+class: extra-details
+
+## BTRFS quirks
+
+- BTRFS works by dividing its storage in *chunks*.
+
+- A chunk can contain data or metadata.
+
+- You can run out of chunks (and get `No space left on device`)
+  even though `df` shows space available.
+  
+  (Because chunks are only partially allocated.)
+
+- Quick fix:
+
+```
+ # btrfs filesys balance start -dusage=1 /var/lib/docker
+```
+
+---
+
+## Overlay2
+
+- Overlay2 is very similar to AUFS.
+
+- However, it has been merged in "upstream" kernel.
+
+- It is therefore available on all modern kernels.
+
+  (AUFS was available on Debian and Ubuntu, but required custom kernels on other distros.)
+
+- It is simpler than AUFS (it can only have two branches, called "layers").
+
+- The container engine abstracts this detail, so this is not a concern.
+
+- Overlay2 storage drivers generally use hard links between layers.
+
+- This improves `stat()` and `open()` performance, at the expense of inode usage.
+
+---
+
+## ZFS
+
+- ZFS is similar to BTRFS (at least from a container user's perspective).
+
+- Pros:
+
+  - high performance
+  - high reliability (with e.g. data checksums)
+  - optional data compression and deduplication
+
+- Cons:
+
+  - high memory usage
+  - not in upstream kernel
+
+- It is available as a kernel module or through FUSE.
+
+---
+
+## Which one is the best?
+
+- Eventually, overlay2 should be the best option.
+
+- It is available on all modern systems.
+
+- Its memory usage is better than Device Mapper, BTRFS, or ZFS.
+
+- The remarks about *write performance* shouldn't bother you:
+  <br/>
+  data should always be stored in volumes anyway!
+

slides/containers/Copying_Files_During_Build.md

@@ -3,7 +3,7 @@ class: title
 
 # Copying files during the build
 
-![Monks copying books](images/copy.jpg)
+![Monks copying books](images/title-copying-files-during-build.jpg)
 
 ---
 
@@ -64,7 +64,7 @@ Create this Dockerfile.
 
 ## Testing our C program
 
-* Create `hello.c` and `Dockerfile` in the same direcotry.
+* Create `hello.c` and `Dockerfile` in the same directory.
 
 * Run `docker build -t hello .` in this directory.
 
@@ -93,7 +93,7 @@ Success!
 * Older Dockerfiles also have the `ADD` instruction.
   <br/>It is similar but can automatically extract archives.
 
-* If we really wanted to compile C code in a compiler, we would:
+* If we really wanted to compile C code in a container, we would:
 
   * Place it in a different directory, with the `WORKDIR` instruction.
 

slides/containers/Docker_History.md

@@ -10,17 +10,19 @@
 
 * [Solaris Containers (2004)](https://en.wikipedia.org/wiki/Solaris_Containers)
 
-* [FreeBSD jails (1999)](https://www.freebsd.org/cgi/man.cgi?query=jail&sektion=8&manpath=FreeBSD+4.0-RELEASE)
+* [FreeBSD jails (1999-2000)](https://www.freebsd.org/cgi/man.cgi?query=jail&sektion=8&manpath=FreeBSD+4.0-RELEASE)
 
 Containers have been around for a *very long time* indeed.
 
+(See [this excellent blog post by Serge Hallyn](https://s3hh.wordpress.com/2018/03/22/history-of-containers/) for more historic details.)
+
 ---
 
 class: pic
 
 ## The VPS age (until 2007-2008)
 
-![lightcont](images/lightcont.png)
+![lightcont](images/containers-as-lightweight-vms.png)
 
 ---
 

slides/containers/Docker_Machine.md

@@ -0,0 +1,81 @@
+# Managing hosts with Docker Machine
+
+- Docker Machine is a tool to provision and manage Docker hosts.
+
+- It automates the creation of a virtual machine:
+
+  - locally, with a tool like VirtualBox or VMware;
+
+  - on a public cloud like AWS EC2, Azure, Digital Ocean, GCP, etc.;
+
+  - on a private cloud like OpenStack.
+
+- It can also configure existing machines through an SSH connection.
+
+- It can manage as many hosts as you want, with as many "drivers" as you want.
+
+---
+
+## Docker Machine workflow
+
+1) Prepare the environment: setup VirtualBox, obtain cloud credentials ...
+
+2) Create hosts with `docker-machine create -d drivername machinename`.
+
+3) Use a specific machine with `eval $(docker-machine env machinename)`.
+
+4) Profit!
+
+---
+
+## Environment variables
+
+- Most of the tools (CLI, libraries...) connecting to the Docker API can use environment variables.
+
+- These variables are:
+
+  - `DOCKER_HOST` (indicates address+port to connect to, or path of UNIX socket)
+
+  - `DOCKER_TLS_VERIFY` (indicates that TLS mutual auth should be used)
+
+  - `DOCKER_CERT_PATH` (path to the keypair and certificate to use for auth)
+
+- `docker-machine env ...` will generate the variables needed to connect to a host.
+
+- `$(eval docker-machine env ...)` sets these variables in the current shell.
+
+---
+
+## Host management features
+
+With `docker-machine`, we can:
+
+- upgrade a host to the latest version of the Docker Engine,
+
+- start/stop/restart hosts,
+
+- get a shell on a remote machine (with SSH),
+
+- copy files to/from remotes machines (with SCP),
+
+- mount a remote host's directory on the local machine (with SSHFS),
+
+- ...
+
+---
+
+## The `generic` driver
+
+When provisioning a new host, `docker-machine` executes these steps:
+
+1) Create the host using a cloud or hypervisor API.
+
+2) Connect to the host over SSH.
+
+3) Install and configure Docker on the host.
+
+With the `generic` driver, we provide the IP address of an existing host
+(instead of e.g. cloud credentials) and we omit the first step.
+
+This allows to provision physical machines, or VMs provided by a 3rd
+party, or use a cloud for which we don't have a provisioning API.

slides/containers/Docker_Overview.md

@@ -58,7 +58,7 @@ class: pic
 
 ## The deployment problem
 
-![problem](images/problem.png)
+![problem](images/shipping-software-problem.png)
 
 ---
 
@@ -66,15 +66,15 @@ class: pic
 
 ## The matrix from hell
 
-![matrix](images/matrix.png)
+![matrix](images/shipping-matrix-from-hell.png)
 
 ---
 
 class: pic
 
-## The parallel with the shipping indsutry
+## The parallel with the shipping industry
 
-![history](images/shippingindustry.png)
+![history](images/shipping-industry-problem.png)
 
 ---
 
@@ -82,7 +82,7 @@ class: pic
 
 ## Intermodal shipping containers
 
-![shipping](images/shipping.png)
+![shipping](images/shipping-industry-solution.png)
 
 ---
 
@@ -90,7 +90,7 @@ class: pic
 
 ## A new shipping ecosystem
 
-![shipeco](images/shipeco.png)
+![shipeco](images/shipping-indsutry-results.png)
 
 ---
 
@@ -98,7 +98,7 @@ class: pic
 
 ## A shipping container system for applications
 
-![shipapp](images/appcont.png)
+![shipapp](images/shipping-software-solution.png)
 
 ---
 
@@ -106,17 +106,29 @@ class: pic
 
 ## Eliminate the matrix from hell
 
-![elimatrix](images/elimatrix.png)
+![elimatrix](images/shipping-matrix-solved.png)
 
 ---
 
 ## Results
 
-* Dev-to-prod reduced from 9 months to 15 minutes (ING)
+* [Dev-to-prod reduced from 9 months to 15 minutes (ING)](
+  https://www.docker.com/sites/default/files/CS_ING_01.25.2015_1.pdf)
 
-* Continuous integration job time reduced by more than 60% (BBC)
+* [Continuous integration job time reduced by more than 60% (BBC)](
+  https://www.docker.com/sites/default/files/CS_BBCNews_01.25.2015_1.pdf)
 
-* Dev-to-prod reduced from weeks to minutes (GILT)
+* [Deploy 100 times a day instead of once a week (GILT)](
+  https://www.docker.com/sites/default/files/CS_Gilt%20Groupe_03.18.2015_0.pdf)
+
+* [70% infrastructure consolidation (MetLife)](
+  https://www.docker.com/customers/metlife-transforms-customer-experience-legacy-and-microservices-mashup)
+
+* [60% infrastructure consolidation (Intesa Sanpaolo)](
+  https://blog.docker.com/2017/11/intesa-sanpaolo-builds-resilient-foundation-banking-docker-enterprise-edition/)
+
+* [14x application density; 60% of legacy datacenter migrated in 4 months (GE Appliances)](
+  https://www.docker.com/customers/ge-uses-docker-enable-self-service-their-developers)
 
 * etc.
 

slides/containers/Dockerfile_Tips.md

@@ -0,0 +1,361 @@
+# Tips for efficient Dockerfiles
+
+We will see how to:
+
+* Reduce the number of layers.
+
+* Leverage the build cache so that builds can be faster.
+
+* Embed unit testing in the build process.
+
+---
+
+## Reducing the number of layers
+
+* Each line in a `Dockerfile` creates a new layer.
+
+* Build your `Dockerfile` to take advantage of Docker's caching system.
+
+* Combine commands by using `&&` to continue commands and `\` to wrap lines.
+
+Note: it is frequent to build a Dockerfile line by line:
+
+```dockerfile
+RUN apt-get install thisthing
+RUN apt-get install andthatthing andthatotherone
+RUN apt-get install somemorestuff
+```
+
+And then refactor it trivially before shipping:
+
+```dockerfile
+RUN apt-get install thisthing andthatthing andthatotherone somemorestuff
+```
+
+---
+
+## Avoid re-installing dependencies at each build
+
+* Classic Dockerfile problem:
+
+  "each time I change a line of code, all my dependencies are re-installed!"
+
+* Solution: `COPY` dependency lists (`package.json`, `requirements.txt`, etc.)
+  by themselves to avoid reinstalling unchanged dependencies every time.
+
+---
+
+## Example "bad" `Dockerfile`
+
+The dependencies are reinstalled every time, because the build system does not know if `requirements.txt` has been updated.
+
+```bash
+FROM python
+WORKDIR /src
+COPY . .
+RUN pip install -qr requirements.txt
+EXPOSE 5000
+CMD ["python", "app.py"]
+```
+
+---
+
+## Fixed `Dockerfile`
+
+Adding the dependencies as a separate step means that Docker can cache more efficiently and only install them when `requirements.txt` changes.
+
+```bash
+FROM python
+COPY requirements.txt /tmp/requirements.txt
+RUN pip install -qr /tmp/requirements.txt
+WORKDIR /src
+COPY . .
+EXPOSE 5000
+CMD ["python", "app.py"]
+```
+
+---
+
+## Embedding unit tests in the build process
+
+```dockerfile
+FROM <baseimage>
+RUN <install dependencies>
+COPY <code>
+RUN <build code>
+RUN <install test dependencies>
+COPY <test data sets and fixtures>
+RUN <unit tests>
+FROM <baseimage>
+RUN <install dependencies>
+COPY <code>
+RUN <build code>
+CMD, EXPOSE ...
+```
+
+* The build fails as soon as an instruction fails
+* If `RUN <unit tests>` fails, the build doesn't produce an image
+* If it succeeds, it produces a clean image (without test libraries and data)
+
+---
+
+# Dockerfile examples
+
+There are a number of tips, tricks, and techniques that we can use in Dockerfiles.
+
+But sometimes, we have to use different (and even opposed) practices depending on:
+
+- the complexity of our project,
+
+- the programming language or framework that we are using,
+
+- the stage of our project (early MVP vs. super-stable production),
+
+- whether we're building a final image or a base for further images,
+
+- etc.
+
+We are going to show a few examples using very different techniques.
+
+---
+
+## When to optimize an image
+
+When authoring official images, it is a good idea to reduce as much as possible:
+
+- the number of layers,
+
+- the size of the final image.
+
+This is often done at the expense of build time and convenience for the image maintainer;
+but when an image is downloaded millions of time, saving even a few seconds of pull time
+can be worth it.
+
+.small[
+```dockerfile
+RUN apt-get update && apt-get install -y libpng12-dev libjpeg-dev && rm -rf /var/lib/apt/lists/* \
+	&& docker-php-ext-configure gd --with-png-dir=/usr --with-jpeg-dir=/usr \
+	&& docker-php-ext-install gd
+...
+RUN curl -o wordpress.tar.gz -SL https://wordpress.org/wordpress-${WORDPRESS_UPSTREAM_VERSION}.tar.gz \
+	&& echo "$WORDPRESS_SHA1 *wordpress.tar.gz" | sha1sum -c - \
+	&& tar -xzf wordpress.tar.gz -C /usr/src/ \
+	&& rm wordpress.tar.gz \
+	&& chown -R www-data:www-data /usr/src/wordpress
+```
+]
+
+(Source: [Wordpress official image](https://github.com/docker-library/wordpress/blob/618490d4bdff6c5774b84b717979bfe3d6ba8ad1/apache/Dockerfile))
+
+---
+
+## When to *not* optimize an image
+
+Sometimes, it is better to prioritize *maintainer convenience*.
+
+In particular, if:
+
+- the image changes a lot,
+
+- the image has very few users (e.g. only 1, the maintainer!),
+
+- the image is built and run on the same machine,
+
+- the image is built and run on machines with a very fast link ...
+
+In these cases, just keep things simple!
+
+(Next slide: a Dockerfile that can be used to preview a Jekyll / github pages site.)
+
+---
+
+```dockerfile
+FROM debian:sid
+
+RUN apt-get update -q
+RUN apt-get install -yq build-essential make
+RUN apt-get install -yq zlib1g-dev
+RUN apt-get install -yq ruby ruby-dev
+RUN apt-get install -yq python-pygments
+RUN apt-get install -yq nodejs
+RUN apt-get install -yq cmake
+RUN gem install --no-rdoc --no-ri github-pages
+
+COPY . /blog
+WORKDIR /blog
+
+VOLUME /blog/_site
+
+EXPOSE 4000
+CMD ["jekyll", "serve", "--host", "0.0.0.0", "--incremental"]
+```
+
+---
+
+## Multi-dimensional versioning systems
+
+Images can have a tag, indicating the version of the image.
+
+But sometimes, there are multiple important components, and we need to indicate the versions
+for all of them.
+
+This can be done with environment variables:
+
+```dockerfile
+ENV PIP=9.0.3 \
+    ZC_BUILDOUT=2.11.2 \
+    SETUPTOOLS=38.7.0 \
+    PLONE_MAJOR=5.1 \
+    PLONE_VERSION=5.1.0 \
+    PLONE_MD5=76dc6cfc1c749d763c32fff3a9870d8d
+```
+
+(Source: [Plone official image](https://github.com/plone/plone.docker/blob/master/5.1/5.1.0/alpine/Dockerfile))
+
+---
+
+## Entrypoints and wrappers
+
+It is very common to define a custom entrypoint.
+
+That entrypoint will generally be a script, performing any combination of:
+
+- pre-flights checks (if a required dependency is not available, display
+  a nice error message early instead of an obscure one in a deep log file),
+
+- generation or validation of configuration files,
+
+- dropping privileges (with e.g. `su` or `gosu`, sometimes combined with `chown`),
+
+- and more.
+
+---
+
+## A typical entrypoint script
+
+```dockerfile
+ #!/bin/sh
+ set -e
+ 
+ # first arg is '-f' or '--some-option'
+ # or first arg is 'something.conf'
+ if [ "${1#-}" != "$1" ] || [ "${1%.conf}" != "$1" ]; then
+ 	set -- redis-server "$@"
+ fi
+ 
+ # allow the container to be started with '--user'
+ if [ "$1" = 'redis-server' -a "$(id -u)" = '0' ]; then
+ 	chown -R redis .
+ 	exec su-exec redis "$0" "$@"
+ fi
+ 
+ exec "$@"
+```
+
+(Source: [Redis official image](https://github.com/docker-library/redis/blob/d24f2be82673ccef6957210cc985e392ebdc65e4/4.0/alpine/docker-entrypoint.sh))
+
+---
+
+## Factoring information
+
+To facilitate maintenance (and avoid human errors), avoid to repeat information like:
+
+- version numbers,
+
+- remote asset URLs (e.g. source tarballs) ...
+
+Instead, use environment variables.
+
+.small[
+```dockerfile
+ENV NODE_VERSION 10.2.1
+...
+RUN ...
+    && curl -fsSLO --compressed "https://nodejs.org/dist/v$NODE_VERSION/node-v$NODE_VERSION.tar.xz" \
+    && curl -fsSLO --compressed "https://nodejs.org/dist/v$NODE_VERSION/SHASUMS256.txt.asc" \
+    && gpg --batch --decrypt --output SHASUMS256.txt SHASUMS256.txt.asc \
+    && grep " node-v$NODE_VERSION.tar.xz\$" SHASUMS256.txt | sha256sum -c - \
+    && tar -xf "node-v$NODE_VERSION.tar.xz" \
+    && cd "node-v$NODE_VERSION" \
+...
+```
+]
+
+(Source: [Nodejs official image](https://github.com/nodejs/docker-node/blob/master/10/alpine/Dockerfile))
+
+---
+
+## Overrides
+
+In theory, development and production images should be the same.
+
+In practice, we often need to enable specific behaviors in development (e.g. debug statements).
+
+One way to reconcile both needs is to use Compose to enable these behaviors.
+
+Let's look at the [trainingwheels](https://github.com/jpetazzo/trainingwheels) demo app for an example.
+
+---
+
+## Production image
+
+This Dockerfile builds an image leveraging gunicorn:
+
+```dockerfile
+FROM python
+RUN pip install flask
+RUN pip install gunicorn
+RUN pip install redis
+COPY . /src
+WORKDIR /src
+CMD gunicorn --bind 0.0.0.0:5000 --workers 10 counter:app
+EXPOSE 5000
+```
+
+(Source: [trainingwheels Dockerfile](https://github.com/jpetazzo/trainingwheels/blob/master/www/Dockerfile))
+
+---
+
+## Development Compose file
+
+This Compose file uses the same image, but with a few overrides for development:
+
+- the Flask development server is used (overriding `CMD`),
+
+- the `DEBUG` environment variable is set,
+
+- a volume is used to provide a faster local development workflow.
+
+.small[
+```yaml
+services:
+  www:
+    build: www
+    ports:
+      - 8000:5000
+    user: nobody
+    environment:
+      DEBUG: 1
+    command: python counter.py
+    volumes:
+      - ./www:/src
+```
+]
+
+(Source: [trainingwheels Compose file](https://github.com/jpetazzo/trainingwheels/blob/master/docker-compose.yml))
+
+---
+
+## How to know which best practices are better?
+
+- The main goal of containers is to make our lives easier.
+
+- In this chapter, we showed many ways to write Dockerfiles.
+
+- These Dockerfiles use sometimes diametrally opposed techniques.
+
+- Yet, they were the "right" ones *for a specific situation.*
+
+- It's OK (and even encouraged) to start simple and evolve as needed.
+
+- Feel free to review this chapter later (after writing a few Dockerfiles) for inspiration!

slides/containers/Ecosystem.md

@@ -0,0 +1,173 @@
+# The container ecosystem
+
+In this chapter, we will talk about a few actors of the container ecosystem.
+
+We have (arbitrarily) decided to focus on two groups:
+
+- the Docker ecosystem,
+
+- the Cloud Native Computing Foundation (CNCF) and its projects.
+
+---
+
+class: pic
+
+## The Docker ecosystem
+
+![The Docker ecosystem in 2015](images/docker-ecosystem-2015.png)
+
+---
+
+## Moby vs. Docker
+
+- Docker Inc. (the company) started Docker (the open source project).
+
+- At some point, it became necessary to differentiate between:
+
+  - the open source project (code base, contributors...),
+
+  - the product that we use to run containers (the engine),
+
+  - the platform that we use to manage containerized applications,
+
+  - the brand.
+
+---
+
+class: pic
+
+![Picture of a Tesla](images/tesla.jpg)
+
+---
+
+## Exercise in brand management
+
+Questions:
+
+--
+
+- What is the brand of the car on the previous slide?
+
+--
+
+- What kind of engine does it have?
+
+--
+
+- Would you say that it's a safe or unsafe car?
+
+--
+
+- Harder question: can you drive from the US West to East coasts with it?
+
+--
+
+The answers to these questions are part of the Tesla brand.
+
+---
+
+## What if ...
+
+- The blueprints for Tesla cars were available for free.
+
+- You could legally build your own Tesla.
+
+- You were allowed to customize it entirely.
+
+  (Put a combustion engine, drive it with a game pad ...)
+
+- You could even sell the customized versions.
+
+--
+
+- ... And call your customized version "Tesla".
+
+--
+
+Would we give the same answers to the questions on the previous slide?
+
+---
+
+## From Docker to Moby
+
+- Docker Inc. decided to split the brand.
+
+- Moby is the open source project.
+
+  (= Components and libraries that you can use, reuse, customize, sell ...)
+
+- Docker is the product.
+
+  (= Software that you can use, buy support contracts ...)
+
+- Docker is made with Moby.
+
+- When Docker Inc. improves the Docker products, it improves Moby.
+
+  (And vice versa.)
+
+
+---
+
+## Other examples
+
+- *Read the Docs* is an open source project to generate and host documentation.
+
+- You can host it yourself (on your own servers).
+
+- You can also get hosted on readthedocs.org.
+
+- The maintainers of the open source project often receive
+  support requests from users of the hosted product ...
+
+- ... And the maintainers of the hosted product often
+  receive support requests from users of self-hosted instances.
+
+- Another example:
+
+  *WordPress.com is a blogging platform that is owned and hosted online by
+  Automattic. It is run on WordPress, an open source piece of software used by
+  bloggers. (Wikipedia)*
+
+---
+
+## Docker CE vs Docker EE
+
+- Docker CE = Community Edition.
+
+- Available on most Linux distros, Mac, Windows.
+
+- Optimized for developers and ease of use.
+
+- Docker EE = Enterprise Edition.
+
+- Available only on a subset of Linux distros + Windows servers.
+
+  (Only available when there is a strong partnership to offer enterprise-class support.)
+
+- Optimized for production use.
+
+- Comes with additional components: security scanning, RBAC ...
+
+---
+
+## The CNCF
+
+- Non-profit, part of the Linux Foundation; founded in December 2015. 
+
+  *The Cloud Native Computing Foundation builds sustainable ecosystems and fosters
+  a community around a constellation of high-quality projects that orchestrate
+  containers as part of a microservices architecture.*
+
+  *CNCF is an open source software foundation dedicated to making cloud-native computing universal and sustainable.*
+
+- Home of Kubernetes (and many other projects now).
+
+- Funded by corporate memberships.
+
+---
+
+class: pic
+
+![Cloud Native Landscape](https://raw.githubusercontent.com/cncf/landscape/master/landscape/CloudNativeLandscape_latest.png)
+

slides/containers/First_Containers.md

@@ -1,9 +1,9 @@
 
 class: title
 
-# Our First Containers
+# Our first containers
 
-![Plastic Containers](images/firstcontainers.jpg)
+![Colorful plastic tubs](images/title-our-first-containers.jpg)
 
 ---
 
@@ -51,10 +51,13 @@ root@04c0bb0a6c07:/#
 ```
 
 * This is a brand new container.
+
 * It runs a bare-bones, no-frills `ubuntu` system.
+
 * `-it` is shorthand for `-i -t`.
 
   * `-i` tells Docker to connect us to the container's stdin.
+
   * `-t` tells Docker that we want a pseudo-terminal.
 
 ---
@@ -72,22 +75,6 @@ Alright, we need to install it.
 
 ---
 
-## An observation
-
-Let's check how many packages are installed here.
-
-```bash
-root@04c0bb0a6c07:/# dpkg -l | wc -l
-189
-```
-
-* `dpkg -l` lists the packages installed in our container
-* `wc -l` counts them
-* If you have a Debian or Ubuntu machine, you can run the same command 
-  and compare the results.
-
----
-
 ## Install a package in our container
 
 We want `figlet`, so let's install it:
@@ -104,6 +91,12 @@ Reading package lists... Done
 
 One minute later, `figlet` is installed!
 
+---
+
+## Try to run our freshly installed program
+
+The `figlet` program takes a message as parameter.
+
 ```bash
 root@04c0bb0a6c07:/# figlet hello
  _          _ _       
@@ -113,11 +106,34 @@ root@04c0bb0a6c07:/# figlet hello
 |_| |_|\___|_|_|\___/ 
 ```
 
+Beautiful! .emoji[😍]
+
 ---
 
-## Exiting our container
+class: in-person
 
-Just exit the shell, like you would usually do.
+## Counting packages in the container
+
+Let's check how many packages are installed there.
+
+```bash
+root@04c0bb0a6c07:/# dpkg -l | wc -l
+190
+```
+
+* `dpkg -l` lists the packages installed in our container
+
+* `wc -l` counts them
+
+How many packages do we have on our host?
+
+---
+
+class: in-person
+
+## Counting packages on the host
+
+Exit the container by logging out of the shell, like you would usually do.
 
 (E.g. with `^D` or `exit`)
 
@@ -125,10 +141,52 @@ Just exit the shell, like you would usually do.
 root@04c0bb0a6c07:/# exit
 ```
 
+Now, try to:
+
+* run `dpkg -l | wc -l`. How many packages are installed?
+
+* run `figlet`. Does that work?
+
+---
+
+class: self-paced
+
+## Comparing the container and the host
+
+Exit the container by logging out of the shell, with `^D` or `exit`.
+
+Now try to run `figlet`. Does that work?
+
+(It shouldn't; except if, by coincidence, you are running on a machine where figlet was installed before.)
+
+---
+
+## Host and containers are independent things
+
+* We ran an `ubuntu` container on an Linux/Windows/macOS host.
+
+* They have different, independent packages.
+
+* Installing something on the host doesn't expose it to the container.
+
+* And vice-versa.
+
+* Even if both the host and the container have the same Linux distro!
+
+* We can run *any container* on *any host*.
+
+  (One exception: Windows containers cannot run on Linux machines; at least not yet.)
+
+---
+
+## Where's our container?
+
 * Our container is now in a *stopped* state.
 
 * It still exists on disk, but all compute resources have been freed up.
 
+* We will see later how to get back to that container.
+
 ---
 
 ## Starting another container

slides/containers/Getting_Inside.md

@@ -0,0 +1,227 @@
+class: title
+
+# Getting inside a container
+
+![Person standing inside a container](images/getting-inside.png)
+
+---
+
+## Objectives
+
+On a traditional server or VM, we sometimes need to:
+
+* log into the machine (with SSH or on the console),
+
+* analyze the disks (by removing them or rebooting with a rescue system).
+
+In this chapter, we will see how to do that with containers.
+
+---
+
+## Getting a shell
+
+Every once in a while, we want to log into a machine.
+
+In an perfect world, this shouldn't be necessary.
+
+* You need to install or update packages (and their configuration)?
+
+  Use configuration management. (e.g. Ansible, Chef, Puppet, Salt...)
+
+* You need to view logs and metrics?
+
+  Collect and access them through a centralized platform.
+
+In the real world, though ... we often need shell access!
+
+---
+
+## Not getting a shell
+
+Even without a perfect deployment system, we can do many operations without getting a shell.
+
+* Installing packages can (and should) be done in the container image.
+
+* Configuration can be done at the image level, or when the container starts.
+
+* Dynamic configuration can be stored in a volume (shared with another container).
+
+* Logs written to stdout are automatically collected by the Docker Engine.
+
+* Other logs can be written to a shared volume.
+
+* Process information and metrics are visible from the host.
+
+_Let's save logging, volumes ... for later, but let's have a look at process information!_
+
+---
+
+## Viewing container processes from the host
+
+If you run Docker on Linux, container processes are visible on the host.
+
+```bash
+$ ps faux | less
+```
+
+* Scroll around the output of this command.
+
+* You should see the `jpetazzo/clock` container.
+
+* A containerized process is just like any other process on the host.
+
+* We can use tools like `lsof`, `strace`, `gdb` ... To analyze them.
+
+---
+
+class: extra-details
+
+## What's the difference between a container process and a host process?
+
+* Each process (containerized or not) belongs to *namespaces* and *cgroups*.
+
+* The namespaces and cgroups determine what a process can "see" and "do".
+
+* Analogy: each process (containerized or not) runs with a specific UID (user ID).
+
+* UID=0 is root, and has elevated privileges. Other UIDs are normal users.
+
+_We will give more details about namespaces and cgroups later._
+
+---
+
+## Getting a shell in a running container
+
+* Sometimes, we need to get a shell anyway.
+
+* We _could_ run some SSH server in the container ...
+
+* But it is easier to use `docker exec`.
+
+```bash
+$ docker exec -ti ticktock sh
+```
+
+* This creates a new process (running `sh`) _inside_ the container.
+
+* This can also be done "manually" with the tool `nsenter`.
+
+---
+
+## Caveats
+
+* The tool that you want to run needs to exist in the container.
+
+* Some tools (like `ip netns exec`) let you attach to _one_ namespace at a time.
+
+  (This lets you e.g. setup network interfaces, even if you don't have `ifconfig` or `ip` in the container.)
+
+* Most importantly: the container needs to be running.
+
+* What if the container is stopped or crashed?
+
+---
+
+## Getting a shell in a stopped container
+
+* A stopped container is only _storage_ (like a disk drive).
+
+* We cannot SSH into a disk drive or USB stick!
+
+* We need to connect the disk to a running machine.
+
+* How does that translate into the container world?
+
+---
+
+## Analyzing a stopped container
+
+As an exercise, we are going to try to find out what's wrong with `jpetazzo/crashtest`.
+
+```bash
+docker run jpetazzo/crashtest
+```
+
+The container starts, but then stops immediately, without any output.
+
+What would MacGyver™ do?
+
+First, let's check the status of that container.
+
+```bash
+docker ps -l
+```
+
+---
+
+## Viewing filesystem changes
+
+* We can use `docker diff` to see files that were added / changed / removed.
+
+```bash
+docker diff <container_id>
+```
+
+* The container ID was shown by `docker ps -l`.
+
+* We can also see it with `docker ps -lq`.
+
+* The output of `docker diff` shows some interesting log files!
+
+---
+
+## Accessing files
+
+* We can extract files with `docker cp`.
+
+```bash
+docker cp <container_id>:/var/log/nginx/error.log .
+```
+
+* Then we can look at that log file.
+
+```bash
+cat error.log
+```
+
+(The directory `/run/nginx` doesn't exist.)
+
+---
+
+## Exploring a crashed container
+
+* We can restart a container with `docker start` ...
+
+* ... But it will probably crash again immediately!
+
+* We cannot specify a different program to run with `docker start`
+
+* But we can create a new image from the crashed container
+
+```bash
+docker commit <container_id> debugimage
+```
+
+* Then we can run a new container from that image, with a custom entrypoint
+
+```bash
+docker run -ti --entrypoint sh debugimage
+```
+
+---
+
+class: extra-details
+
+## Obtaining a complete dump
+
+* We can also dump the entire filesystem of a container.
+
+* This is done with `docker export`.
+
+* It generates a tar archive.
+
+```bash
+docker export <container_id> | tar tv
+```
+
+This will give a detailed listing of the content of the container.

slides/containers/Initial_Images.md

@@ -1,9 +1,9 @@
 
 class: title
 
-# Understanding Docker Images
+# Understanding Docker images
 
-![image](images/image.png)
+![image](images/title-understanding-docker-images.png)
 
 ---
 
@@ -46,6 +46,8 @@ In this section, we will explain:
 
 ## Example for a Java webapp
 
+Each of the following items will correspond to one layer:
+
 * CentOS base layer
 * Packages and configuration files added by our local IT
 * JRE
@@ -56,6 +58,22 @@ In this section, we will explain:
 
 ---
 
+class: pic
+
+## The read-write layer
+
+![layers](images/container-layers.jpg)
+
+---
+
+class: pic
+
+## Multiple containers sharing the same image
+
+![layers](images/sharing-layers.jpg)
+
+---
+
 ## Differences between containers and images
 
 * An image is a read-only filesystem.
@@ -63,24 +81,14 @@ In this section, we will explain:
 * A container is an encapsulated set of processes running in a
   read-write copy of that filesystem.
 
-* To optimize container boot time, *copy-on-write* is used 
+* To optimize container boot time, *copy-on-write* is used
   instead of regular copy.
 
 * `docker run` starts a container from a given image.
 
-Let's give a couple of metaphors to illustrate those concepts.
-
 ---
 
-## Image as stencils
-
-Images are like templates or stencils that you can create containers from.
-
-![stencil](images/stenciling-wall.jpg)
-
----
-
-## Object-oriented programming
+## Comparison with object-oriented programming
 
 * Images are conceptually similar to *classes*.
 
@@ -99,7 +107,7 @@ If an image is read-only, how do we change it?
 * We create a new container from that image.
 
 * Then we make changes to that container.
- 
+
 * When we are satisfied with those changes, we transform them into a new layer.
 
 * A new image is created by stacking the new layer on top of the old image.
@@ -118,7 +126,7 @@ If an image is read-only, how do we change it?
 
 ## Creating the first images
 
-There is a special empty image called `scratch`. 
+There is a special empty image called `scratch`.
 
 * It allows to *build from scratch*.
 
@@ -138,7 +146,7 @@ Note: you will probably never have to do this yourself.
 * Saves all the changes made to a container into a new layer.
 * Creates a new image (effectively a copy of the container).
 
-`docker build`
+`docker build` **(used 99% of the time)**
 
 * Performs a repeatable build sequence.
 * This is the preferred method!
@@ -180,6 +188,8 @@ Those images include:
 
 * Ready-to-use components and services, like redis, postgresql...
 
+* Over 130 at this point!
+
 ---
 
 ## User namespace
@@ -299,9 +309,9 @@ There are two ways to download images.
 ```bash
 $ docker pull debian:jessie
 Pulling repository debian
-b164861940b8: Download complete 
-b164861940b8: Pulling image (jessie) from debian 
-d1881793a057: Download complete 
+b164861940b8: Download complete
+b164861940b8: Pulling image (jessie) from debian
+d1881793a057: Download complete
 ```
 
 * As seen previously, images are made up of layers.

slides/containers/Installing_Docker.md

@@ -1,10 +1,8 @@
-
 class: title
 
-# Install Docker
+# Installing Docker
 
-
-![install](images/install.jpg)
+![install](images/title-installing-docker.jpg)
 
 ---
 
@@ -31,7 +29,7 @@ We can arbitrarily distinguish:
 
 * Installing Docker on an existing Linux machine (physical or VM)
 
-* Installing Docker on MacOS or Windows
+* Installing Docker on macOS or Windows
 
 * Installing Docker on a fleet of cloud VMs
 
@@ -39,7 +37,9 @@ We can arbitrarily distinguish:
 
 ## Installing Docker on Linux
 
-* The recommended method is to install the packages supplied by Docker Inc.
+* The recommended method is to install the packages supplied by Docker Inc.:
+
+  https://store.docker.com
 
 * The general method is:
 
@@ -57,13 +57,35 @@ We can arbitrarily distinguish:
 
 ---
 
-## Installing Docker on MacOS and Windows
+class: extra-details
 
-* On MacOS, the recommended method is to use Docker4Mac:
+## Docker Inc. packages vs distribution packages
+
+* Docker Inc. releases new versions monthly (edge) and quarterly (stable)
+
+* Releases are immediately available on Docker Inc.'s package repositories
+
+* Linux distros don't always update to the latest Docker version
+
+  (Sometimes, updating would break their guidelines for major/minor upgrades)
+
+* Sometimes, some distros have carried packages with custom patches
+
+* Sometimes, these patches added critical security bugs ☹
+
+* Installing through Docker Inc.'s repositories is a bit of extra work …
+
+  … but it is generally worth it!
+
+---
+
+## Installing Docker on macOS and Windows
+
+* On macOS, the recommended method is to use Docker for Mac:
 
   https://docs.docker.com/docker-for-mac/install/
 
-* On Windows 10 Pro, Enterprise, and Eduction, you can use Docker4Windows:
+* On Windows 10 Pro, Enterprise, and Education, you can use Docker for Windows:
 
   https://docs.docker.com/docker-for-windows/install/
 
@@ -71,9 +93,36 @@ We can arbitrarily distinguish:
 
   https://docs.docker.com/toolbox/toolbox_install_windows/
 
+* On Windows Server 2016, you can also install the native engine:
+
+  https://docs.docker.com/install/windows/docker-ee/
+
 ---
 
-## Running Docker on MacOS and Windows
+## Docker for Mac and Docker for Windows
+
+* Special Docker Editions that integrate well with their respective host OS
+
+* Provide user-friendly GUI to edit Docker configuration and settings
+
+* Leverage the host OS virtualization subsystem (e.g. the [Hypervisor API](https://developer.apple.com/documentation/hypervisor) on macOS)
+
+* Installed like normal user applications on the host
+
+* Under the hood, they both run a tiny VM (transparent to our daily use)
+
+* Access network resources like normal applications
+  <br/>(and therefore, play better with enterprise VPNs and firewalls)
+
+* Support filesystem sharing through volumes (we'll talk about this later)
+
+* They only support running one Docker VM at a time ...
+  <br/>
+  ... but we can use `docker-machine`, the Docker Toolbox, VirtualBox, etc. to get a cluster.
+
+---
+
+## Running Docker on macOS and Windows
 
 When you execute `docker version` from the terminal:
 
@@ -90,25 +139,6 @@ This will also allow to use remote Engines exactly as if they were local.
 
 ---
 
-## Docker4Mac and Docker4Windows
-
-* They let you run Docker without VirtualBox
-
-* They are installed like normal applications (think QEMU, but faster)
-
-* They access network resources like normal applications
-  <br/>(and therefore, play well with enterprise VPNs and firewalls)
-
-* They support filesystem sharing through volumes (we'll talk about this later)
-
-* They only support running one Docker VM at a time ...
-
-  ... so if you want to run a full cluster locally, install e.g. the Docker Toolbox
-
-* They can co-exist with the Docker Toolbox
-
----
-
 ## Important PSA about security
 
 * If you have access to the Docker control socket, you can take over the machine

slides/containers/Labels.md

@@ -0,0 +1,82 @@
+# Labels
+
+* Labels allow to attach arbitrary metadata to containers.
+
+* Labels are key/value pairs.
+
+* They are specified at container creation.
+
+* You can query them with `docker inspect`.
+
+* They can also be used as filters with some commands (e.g. `docker ps`).
+
+---
+
+## Using labels
+
+Let's create a few containers with a label `owner`.
+
+```bash
+docker run -d -l owner=alice nginx
+docker run -d -l owner=bob nginx
+docker run -d -l owner nginx
+```
+
+We didn't specify a value for the `owner` label in the last example.
+
+This is equivalent to setting the value to be an empty string.
+
+---
+
+## Querying labels
+
+We can view the labels with `docker inspect`.
+
+```bash
+$ docker inspect $(docker ps -lq) | grep -A3 Labels
+            "Labels": {
+                "maintainer": "NGINX Docker Maintainers <docker-maint@nginx.com>",
+                "owner": ""
+            },
+```
+
+We can use the `--format` flag to list the value of a label.
+
+```bash
+$ docker inspect $(docker ps -q) --format 'OWNER={{.Config.Labels.owner}}'
+```
+
+---
+
+## Using labels to select containers
+
+We can list containers having a specific label.
+
+```bash
+$ docker ps --filter label=owner
+```
+
+Or we can list containers having a specific label with a specific value.
+
+```bash
+$ docker ps --filter label=owner=alice
+```
+
+---
+
+## Use-cases for labels
+
+
+* HTTP vhost of a web app or web service.
+
+  (The label is used to generate the configuration for NGINX, HAProxy, etc.)
+
+* Backup schedule for a stateful service.
+
+  (The label is used by a cron job to determine if/when to backup container data.)
+
+* Service ownership.
+
+  (To determine internal cross-billing, or who to page in case of outage.)
+
+* etc.

slides/containers/Local_Development_Workflow.md

@@ -0,0 +1,441 @@
+
+class: title
+
+# Local development workflow with Docker
+
+![Construction site](images/title-local-development-workflow-with-docker.jpg)
+
+---
+
+## Objectives
+
+At the end of this section, you will be able to:
+
+* Share code between container and host.
+
+* Use a simple local development workflow.
+
+---
+
+## Local development in a container
+
+We want to solve the following issues:
+
+- "Works on my machine"
+
+- "Not the same version"
+
+- "Missing dependency"
+
+By using Docker containers, we will get a consistent development environment.
+
+---
+
+## Working on the "namer" application
+
+* We have to work on some application whose code is at:
+
+  https://github.com/jpetazzo/namer.
+
+* What is it? We don't know yet!
+
+* Let's download the code.
+
+```bash
+$ git clone https://github.com/jpetazzo/namer
+```
+
+---
+
+## Looking at the code
+
+```bash
+$ cd namer
+$ ls -1
+company_name_generator.rb
+config.ru
+docker-compose.yml
+Dockerfile
+Gemfile
+```
+
+--
+
+Aha, a `Gemfile`! This is Ruby. Probably. We know this. Maybe?
+
+---
+
+## Looking at the `Dockerfile`
+
+```dockerfile
+FROM ruby
+
+COPY . /src
+WORKDIR /src
+RUN bundler install
+
+CMD ["rackup", "--host", "0.0.0.0"]
+EXPOSE 9292
+```
+
+* This application is using a base `ruby` image.
+* The code is copied in `/src`.
+* Dependencies are installed with `bundler`.
+* The application is started with `rackup`.
+* It is listening on port 9292.
+
+---
+
+## Building and running the "namer" application
+
+* Let's build the application with the `Dockerfile`!
+
+--
+
+```bash
+$ docker build -t namer .
+```
+
+--
+
+* Then run it. *We need to expose its ports.*
+
+--
+
+```bash
+$ docker run -dP namer
+```
+
+--
+
+* Check on which port the container is listening.
+
+--
+
+```bash
+$ docker ps -l
+```
+
+---
+
+## Connecting to our application
+
+* Point our browser to our Docker node, on the port allocated to the container.
+
+--
+
+* Hit "reload" a few times.
+
+--
+
+* This is an enterprise-class, carrier-grade, ISO-compliant company name generator!
+
+  (With 50% more bullshit than the average competition!)
+
+  (Wait, was that 50% more, or 50% less? *Anyway!*)
+
+  ![web application 1](images/webapp-in-blue.png)
+
+---
+
+## Making changes to the code
+
+Option 1:
+
+* Edit the code locally
+* Rebuild the image
+* Re-run the container
+
+Option 2:
+
+* Enter the container (with `docker exec`)
+* Install an editor
+* Make changes from within the container
+
+Option 3:
+
+* Use a *volume* to mount local files into the container
+* Make changes locally
+* Changes are reflected into the container
+
+---
+
+## Our first volume
+
+We will tell Docker to map the current directory to `/src` in the container.
+
+```bash
+$ docker run -d -v $(pwd):/src -P namer
+```
+
+* `-d`: the container should run in detached mode (in the background).
+
+* `-v`: the following host directory should be mounted inside the container.
+
+* `-P`: publish all the ports exposed by this image.
+
+* `namer` is the name of the image we will run.
+
+* We don't specify a command to run because it is already set in the Dockerfile.
+
+Note: on Windows, replace `$(pwd)` with `%cd%` (or `${pwd}` if you use PowerShell).
+
+---
+
+## Mounting volumes inside containers
+
+The `-v` flag mounts a directory from your host into your Docker container.
+
+The flag structure is:
+
+```bash
+[host-path]:[container-path]:[rw|ro]
+```
+
+* If `[host-path]` or `[container-path]` doesn't exist it is created.
+
+* You can control the write status of the volume with the `ro` and
+  `rw` options.
+
+* If you don't specify `rw` or `ro`, it will be `rw` by default.
+
+There will be a full chapter about volumes!
+
+---
+
+## Testing the development container
+
+* Check the port used by our new container.
+
+```bash
+$ docker ps -l
+CONTAINER ID  IMAGE  COMMAND  CREATED        STATUS  PORTS                   NAMES
+045885b68bc5  namer  rackup   3 seconds ago  Up ...  0.0.0.0:32770->9292/tcp ...
+```
+
+* Open the application in your web browser.
+
+---
+
+## Making a change to our application
+
+Our customer really doesn't like the color of our text. Let's change it.
+
+```bash
+$ vi company_name_generator.rb
+```
+
+And change
+
+```css
+color: royalblue;
+```
+
+To:
+
+```css
+color: red;
+```
+
+---
+
+## Viewing our changes
+
+* Reload the application in our browser.
+
+--
+
+* The color should have changed.
+
+  ![web application 2](images/webapp-in-red.png)
+
+---
+
+## Understanding volumes
+
+* Volumes are *not* copying or synchronizing files between the host and the container.
+
+* Volumes are *bind mounts*: a kernel mechanism associating a path to another.
+
+* Bind mounts are *kind of* similar to symbolic links, but at a very different level.
+
+* Changes made on the host or on the container will be visible on the other side.
+
+  (Since under the hood, it's the same file on both anyway.)
+
+---
+
+## Trash your servers and burn your code
+
+*(This is the title of a
+[2013 blog post](http://chadfowler.com/2013/06/23/immutable-deployments.html)
+by Chad Fowler, where he explains the concept of immutable infrastructure.)*
+
+--
+
+* Let's mess up majorly with our container.
+
+  (Remove files or whatever.)
+
+* Now, how can we fix this?
+
+--
+
+* Our old container (with the blue version of the code) is still running.
+
+* See on which port it is exposed:
+  ```bash
+  docker ps
+  ```
+
+* Point our browser to it to confirm that it still works fine.
+
+---
+
+## Immutable infrastructure in a nutshell
+
+* Instead of *updating* a server, we deploy a new one.
+
+* This might be challenging with classical servers, but it's trivial with containers.
+
+* In fact, with Docker, the most logical workflow is to build a new image and run it.
+
+* If something goes wrong with the new image, we can always restart the old one.
+
+* We can even keep both versions running side by side.
+
+If this pattern sounds interesting, you might want to read about *blue/green deployment*
+and *canary deployments*.
+
+---
+
+## Improving the workflow
+
+The workflow that we showed is nice, but it requires us to:
+
+* keep track of all the `docker run` flags required to run the container,
+
+* inspect the `Dockerfile` to know which path(s) to mount,
+
+* write scripts to hide that complexity.
+
+There has to be a better way!
+
+---
+
+## Docker Compose to the rescue
+
+* Docker Compose allows us to "encode" `docker run` parameters in a YAML file.
+
+* Here is the `docker-compose.yml` file that we can use for our "namer" app:
+
+  ```yaml
+  www:
+    build: .
+    volumes:
+      - .:/src
+    ports:
+      - 80:9292
+  ```
+
+* Try it:
+  ```bash
+  $ docker-compose up -d
+  ```
+
+---
+
+## Working with Docker Compose
+
+* When you see a `docker-compose.yml` file, you can use `docker-compose up`.
+
+* It can build images and run them with the required parameters.
+
+* Compose can also deal with complex, multi-container apps.
+
+  (More on this later!)
+
+---
+
+## Recap of the development workflow
+
+1. Write a Dockerfile to build an image containing our development environment.
+   <br/>
+   (Rails, Django, ... and all the dependencies for our app)
+
+2. Start a container from that image.
+   <br/>
+   Use the `-v` flag to mount our source code inside the container.
+
+3. Edit the source code outside the containers, using regular tools.
+   <br/>
+   (vim, emacs, textmate...)
+
+4. Test the application.
+   <br/>
+   (Some frameworks pick up changes automatically.
+   <br/>Others require you to Ctrl-C + restart after each modification.)
+
+5. Iterate and repeat steps 3 and 4 until satisfied.
+
+6. When done, commit+push source code changes.
+
+---
+
+class: extra-details
+
+## Debugging inside the container
+
+Docker has a command called `docker exec`.
+
+It allows users to run a new process in a container which is already running.
+
+If sometimes you find yourself wishing you could SSH into a container: you can use `docker exec` instead.
+
+You can get a shell prompt inside an existing container this way, or run an arbitrary process for automation.
+
+---
+
+class: extra-details
+
+## `docker exec` example
+
+```bash
+$ # You can run ruby commands in the area the app is running and more!
+$ docker exec -it <yourContainerId> bash
+root@5ca27cf74c2e:/opt/namer# irb
+irb(main):001:0> [0, 1, 2, 3, 4].map {|x| x ** 2}.compact
+=> [0, 1, 4, 9, 16]
+irb(main):002:0> exit
+```
+
+---
+
+class: extra-details
+
+## Stopping the container
+
+Now that we're done let's stop our container.
+
+```bash
+$ docker stop <yourContainerID>
+```
+
+And remove it.
+
+```bash
+$ docker rm <yourContainerID>
+```
+
+---
+
+## Section summary
+
+We've learned how to:
+
+* Share code between container and host.
+
+* Set our working directory.
+
+* Use a simple local development workflow.
+

slides/containers/Logging.md

@@ -0,0 +1,294 @@
+# Logging
+
+In this chapter, we will explain the different ways to send logs from containers.
+
+We will then show one particular method in action, using ELK and Docker's logging drivers.
+
+---
+
+## There are many ways to send logs
+
+- The simplest method is to write on the standard output and error.
+
+- Applications can write their logs to local files.
+
+  (The files are usually periodically rotated and compressed.)
+
+- It is also very common (on UNIX systems) to use syslog.
+
+  (The logs are collected by syslogd or an equivalent like journald.)
+
+- In large applications with many components, it is common to use a logging service.
+
+  (The code uses a library to send messages to the logging service.)
+
+*All these methods are available with containers.*
+
+---
+
+## Writing on stdout/stderr
+
+- The standard output and error of containers is managed by the container engine.
+
+- This means that each line written by the container is received by the engine.
+
+- The engine can then do "whatever" with these log lines.
+
+- With Docker, the default configuration is to write the logs to local files.
+
+- The files can then be queried with e.g. `docker logs` (and the equivalent API request).
+
+- This can be customized, as we will see later.
+
+---
+
+## Writing to local files
+
+- If we write to files, it is possible to access them but cumbersome.
+
+  (We have to use `docker exec` or `docker cp`.)
+
+- Furthermore, if the container is stopped, we cannot use `docker exec`.
+
+- If the container is deleted, the logs disappear.
+
+- What should we do for programs who can only log to local files?
+
+--
+
+- There are multiple solutions.
+
+---
+
+## Using a volume or bind mount
+
+- Instead of writing logs to a normal directory, we can place them on a volume.
+
+- The volume can be accessed by other containers.
+
+- We can run a program like `filebeat` in another container accessing the same volume.
+
+  (`filebeat` reads local log files continuously, like `tail -f`, and sends them
+  to a centralized system like ElasticSearch.)
+
+- We can also use a bind mount, e.g. `-v /var/log/containers/www:/var/log/tomcat`.
+
+- The container will write log files to a directory mapped to a host directory.
+
+- The log files will appear on the host and be consumable directly from the host.
+
+---
+
+## Using logging services
+
+- We can use logging frameworks (like log4j or the Python `logging` package).
+
+- These frameworks require some code and/or configuration in our application code.
+
+- These mechanisms can be used identically inside or outside of containers.
+
+- Sometimes, we can leverage containerized networking to simplify their setup.
+
+- For instance, our code can send log messages to a server named `log`.
+
+- The name `log` will resolve to different addresses in development, production, etc.
+
+---
+
+## Using syslog
+
+- What if our code (or the program we are running in containers) uses syslog?
+
+- One possibility is to run a syslog daemon in the container.
+
+- Then that daemon can be setup to write to local files or forward to the network.
+
+- Under the hood, syslog clients connect to a local UNIX socket, `/dev/log`.
+
+- We can expose a syslog socket to the container (by using a volume or bind-mount).
+
+- Then just create a symlink from `/dev/log` to the syslog socket.
+
+- Voilà!
+
+---
+
+## Using logging drivers
+
+- If we log to stdout and stderr, the container engine receives the log messages.
+
+- The Docker Engine has a modular logging system with many plugins, including:
+
+  - json-file (the default one)
+  - syslog
+  - journald
+  - gelf
+  - fluentd
+  - splunk
+  - etc.
+
+- Each plugin can process and forward the logs to another process or system.
+
+---
+
+## A word of warning about `json-file`
+
+- By default, log file size is unlimited.
+
+- This means that a very verbose container *will* use up all your disk space.
+
+  (Or a less verbose container, but running for a very long time.)
+
+- Log rotation can be enabled by setting a `max-size` option.
+
+- Older log files can be removed by setting a `max-file` option.
+
+- Just like other logging options, these can be set per container, or globally.
+
+Example:
+```bash
+$ docker run --log-opt max-size=10m --log-opt max-file=3 elasticsearch
+```
+
+---
+
+## Demo: sending logs to ELK
+
+- We are going to deploy an ELK stack.
+
+- It will accept logs over a GELF socket.
+
+- We will run a few containers with the `gelf` logging driver.
+
+- We will then see our logs in Kibana, the web interface provided by ELK.
+
+*Important foreword: this is not an "official" or "recommended"
+setup; it is just an example. We used ELK in this demo because
+it's a popular setup and we keep being asked about it; but you
+will have equal success with Fluent or other logging stacks!*
+
+---
+
+## What's in an ELK stack?
+
+- ELK is three components:
+
+  - ElasticSearch (to store and index log entries)
+
+  - Logstash (to receive log entries from various
+    sources, process them, and forward them to various
+    destinations)
+
+  - Kibana (to view/search log entries with a nice UI)
+
+- The only component that we will configure is Logstash
+
+- We will accept log entries using the GELF protocol
+
+- Log entries will be stored in ElasticSearch,
+  <br/>and displayed on Logstash's stdout for debugging
+
+---
+
+## Running ELK
+
+- We are going to use a Compose file describing the ELK stack.
+
+```bash
+$ cd ~/container.training/stacks
+$ docker-compose -f elk.yml up -d
+```
+
+- Let's have a look at the Compose file while it's deploying.
+
+---
+
+## Our basic ELK deployment
+
+- We are using images from the Docker Hub: `elasticsearch`, `logstash`, `kibana`.
+
+- We don't need to change the configuration of ElasticSearch.
+
+- We need to tell Kibana the address of ElasticSearch:
+
+  - it is set with the `ELASTICSEARCH_URL` environment variable,
+
+  - by default it is `localhost:9200`, we change it to `elasticsearch:9200`.
+
+- We need to configure Logstash:
+
+  - we pass the entire configuration file through command-line arguments,
+
+  - this is a hack so that we don't have to create an image just for the config.
+
+---
+
+## Sending logs to ELK
+
+- The ELK stack accepts log messages through a GELF socket.
+
+- The GELF socket listens on UDP port 12201.
+
+- To send a message, we need to change the logging driver used by Docker.
+
+- This can be done globally (by reconfiguring the Engine) or on a per-container basis.
+
+- Let's override the logging driver for a single container:
+
+```bash
+$ docker run --log-driver=gelf --log-opt=gelf-address=udp://localhost:12201 \
+  alpine echo hello world
+```
+
+---
+
+## Viewing the logs in ELK
+
+- Connect to the Kibana interface.
+
+- It is exposed on port 5601.
+
+- Browse http://X.X.X.X:5601.
+
+---
+
+## "Configuring" Kibana
+
+- Kibana should offer you to "Configure an index pattern":
+  <br/>in the "Time-field name" drop down, select "@timestamp", and hit the
+  "Create" button.
+
+- Then:
+
+  - click "Discover" (in the top-left corner),
+  - click "Last 15 minutes" (in the top-right corner),
+  - click "Last 1 hour" (in the list in the middle),
+  - click "Auto-refresh" (top-right corner),
+  - click "5 seconds" (top-left of the list).
+
+- You should see a series of green bars (with one new green bar every minute).
+
+- Our 'hello world' message should be visible there.
+
+---
+
+## Important afterword
+
+**This is not a "production-grade" setup.**
+
+It is just an educational example. Since we have only
+one node , we did set up a single
+ElasticSearch instance and a single Logstash instance.
+
+In a production setup, you need an ElasticSearch cluster
+(both for capacity and availability reasons). You also
+need multiple Logstash instances.
+
+And if you want to withstand
+bursts of logs, you need some kind of message queue:
+Redis if you're cheap, Kafka if you want to make sure
+that you don't drop messages on the floor. Good luck.
+
+If you want to learn more about the GELF driver,
+have a look at [this blog post](
+http://jpetazzo.github.io/2017/01/20/docker-logging-gelf/).

slides/containers/Multi_Stage_Builds.md

@@ -0,0 +1,295 @@
+# Reducing image size
+
+* In the previous example, our final image contained:
+
+  * our `hello` program
+
+  * its source code
+
+  * the compiler
+
+* Only the first one is strictly necessary.
+
+* We are going to see how to obtain an image without the superfluous components.
+
+---
+
+## Can't we remove superfluous files with `RUN`?
+
+What happens if we do one of the following commands?
+
+- `RUN rm -rf ...`
+
+- `RUN apt-get remove ...`
+
+- `RUN make clean ...`
+
+--
+
+This adds a layer which removes a bunch of files.
+
+But the previous layers (which added the files) still exist.
+
+---
+
+## Removing files with an extra layer
+
+When downloading an image, all the layers must be downloaded.
+
+| Dockerfile instruction | Layer size | Image size |
+| ---------------------- | ---------- | ---------- |
+| `FROM ubuntu` | Size of base image | Size of base image |
+| `...` | ... | Sum of this layer <br/>+ all previous ones |
+| `RUN apt-get install somepackage` | Size of files added <br/>(e.g. a few MB) | Sum of this layer <br/>+ all previous ones |
+| `...` | ... | Sum of this layer <br/>+ all previous ones |
+| `RUN apt-get remove somepackage` | Almost zero <br/>(just metadata) | Same as previous one |
+
+Therefore, `RUN rm` does not reduce the size of the image or free up disk space.
+
+---
+
+## Removing unnecessary files
+
+Various techniques are available to obtain smaller images:
+
+- collapsing layers,
+
+- adding binaries that are built outside of the Dockerfile,
+
+- squashing the final image,
+
+- multi-stage builds.
+
+Let's review them quickly.
+
+---
+
+## Collapsing layers
+
+You will frequently see Dockerfiles like this:
+
+```dockerfile
+FROM ubuntu
+RUN apt-get update && apt-get install xxx && ... && apt-get remove xxx && ...
+```
+
+Or the (more readable) variant:
+
+```dockerfile
+FROM ubuntu
+RUN apt-get update \
+ && apt-get install xxx \
+ && ... \
+ && apt-get remove xxx \
+ && ...
+```
+
+This `RUN` command gives us a single layer.
+
+The files that are added, then removed in the same layer, do not grow the layer size.
+
+---
+
+## Collapsing layers: pros and cons
+
+Pros:
+
+- works on all versions of Docker
+
+- doesn't require extra tools
+
+Cons:
+
+- not very readable
+
+- some unnecessary files might still remain if the cleanup is not thorough
+
+- that layer is expensive (slow to build)
+
+---
+
+## Building binaries outside of the Dockerfile
+
+This results in a Dockerfile looking like this:
+
+```dockerfile
+FROM ubuntu
+COPY xxx /usr/local/bin
+```
+
+Of course, this implies that the file `xxx` exists in the build context.
+
+That file has to exist before you can run `docker build`.
+
+For instance, it can:
+
+- exist in the code repository,
+- be created by another tool (script, Makefile...),
+- be created by another container image and extracted from the image.
+
+See for instance the [busybox official image](https://github.com/docker-library/busybox/blob/fe634680e32659aaf0ee0594805f74f332619a90/musl/Dockerfile) or this [older busybox image](https://github.com/jpetazzo/docker-busybox).
+
+---
+
+## Building binaries outside: pros and cons
+
+Pros:
+
+- final image can be very small
+
+Cons:
+
+- requires an extra build tool
+
+- we're back in dependency hell and "works on my machine"
+
+Cons, if binary is added to code repository:
+
+- breaks portability across different platforms
+
+- grows repository size a lot if the binary is updated frequently
+
+---
+
+## Squashing the final image
+
+The idea is to transform the final image into a single-layer image.
+
+This can be done in (at least) two ways.
+
+- Activate experimental features and squash the final image:
+  ```bash
+  docker image build --squash ...
+  ```
+
+- Export/import the final image.
+  ```bash
+  docker build -t temp-image .
+  docker run --entrypoint true --name temp-container temp-image
+  docker export temp-container | docker import - final-image
+  docker rm temp-container
+  docker rmi temp-image
+  ```
+
+---
+
+## Squashing the image: pros and cons
+
+Pros:
+
+- single-layer images are smaller and faster to download
+
+- removed files no longer take up storage and network resources
+
+Cons:
+
+- we still need to actively remove unnecessary files
+
+- squash operation can take a lot of time (on big images)
+
+- squash operation does not benefit from cache
+  <br/>
+  (even if we change just a tiny file, the whole image needs to be re-squashed)
+
+---
+
+## Multi-stage builds
+
+Multi-stage builds allow us to have multiple *stages*.
+
+Each stage is a separate image, and can copy files from previous stages.
+
+We're going to see how they work in more detail.
+
+---
+
+# Multi-stage builds
+
+* At any point in our `Dockerfile`, we can add a new `FROM` line.
+
+* This line starts a new stage of our build.
+
+* Each stage can access the files of the previous stages with `COPY --from=...`.
+
+* When a build is tagged (with `docker build -t ...`), the last stage is tagged.
+
+* Previous stages are not discarded: they will be used for caching, and can be referenced.
+
+---
+
+## Multi-stage builds in practice
+
+* Each stage is numbered, starting at `0`
+
+* We can copy a file from a previous stage by indicating its number, e.g.:
+
+  ```dockerfile
+  COPY --from=0 /file/from/first/stage /location/in/current/stage
+  ```
+
+* We can also name stages, and reference these names:
+
+  ```dockerfile
+  FROM golang AS builder
+  RUN ...
+  FROM alpine
+  COPY --from=builder /go/bin/mylittlebinary /usr/local/bin/
+  ```
+
+---
+
+## Multi-stage builds for our C program
+
+We will change our Dockerfile to:
+
+* give a nickname to the first stage: `compiler`
+
+* add a second stage using the same `ubuntu` base image
+
+* add the `hello` binary to the second stage
+
+* make sure that `CMD` is in the second stage 
+
+The resulting Dockerfile is on the next slide.
+
+---
+
+## Multi-stage build `Dockerfile`
+
+Here is the final Dockerfile:
+
+```dockerfile
+FROM ubuntu AS compiler
+RUN apt-get update
+RUN apt-get install -y build-essential
+COPY hello.c /
+RUN make hello
+FROM ubuntu
+COPY --from=compiler /hello /hello
+CMD /hello
+```
+
+Let's build it, and check that it works correctly:
+
+```bash
+docker build -t hellomultistage .
+docker run hellomultistage
+```
+
+---
+
+## Comparing single/multi-stage build image sizes
+
+List our images with `docker images`, and check the size of:
+
+- the `ubuntu` base image,
+
+- the single-stage `hello` image,
+
+- the multi-stage `hellomultistage` image.
+
+We can achieve even smaller images if we use smaller base images.
+
+However, if we use common base images (e.g. if we standardize on `ubuntu`),
+these common images will be pulled only once per node, so they are
+virtually "free."

slides/containers/Naming_And_Inspecting.md

@@ -3,7 +3,7 @@ class: title
 
 # Naming and inspecting containers
 
-![Markings on container door](images/containermarkings.jpg)
+![Markings on container door](images/title-naming-and-inspecting-containers.jpg)
 
 ---
 
@@ -85,16 +85,8 @@ The `docker inspect` command will output a very detailed JSON map.
 ```bash
 $ docker inspect <containerID>
 [{
-	"AppArmorProfile": "",
-	"Args": [],
-	"Config": {
-	    "AttachStderr": true,
-	    "AttachStdin": false,
-	    "AttachStdout": true,
-	    "Cmd": [
-		"bash"
-	    ],
-	    "CpuShares": 0,
+...
+(many pages of JSON here)
 ...
 ```
 

slides/containers/Network_Drivers.md

@@ -0,0 +1,84 @@
+# Container network drivers
+
+The Docker Engine supports many different network drivers.
+
+The built-in drivers include:
+
+* `bridge` (default)
+
+* `none`
+
+* `host`
+
+* `container`
+
+The driver is selected with `docker run --net ...`.
+
+The different drivers are explained with more details on the following slides.
+
+---
+
+## The default bridge
+
+* By default, the container gets a virtual `eth0` interface.
+  <br/>(In addition to its own private `lo` loopback interface.)
+
+* That interface is provided by a `veth` pair.
+
+* It is connected to the Docker bridge.
+  <br/>(Named `docker0` by default; configurable with `--bridge`.)
+
+* Addresses are allocated on a private, internal subnet.
+  <br/>(Docker uses 172.17.0.0/16 by default; configurable with `--bip`.)
+
+* Outbound traffic goes through an iptables MASQUERADE rule.
+
+* Inbound traffic goes through an iptables DNAT rule.
+
+* The container can have its own routes, iptables rules, etc.
+
+---
+
+## The null driver
+
+* Container is started with `docker run --net none ...`
+
+* It only gets the `lo` loopback interface. No `eth0`.
+
+* It can't send or receive network traffic.
+
+* Useful for isolated/untrusted workloads.
+
+---
+
+## The host driver
+
+* Container is started with `docker run --net host ...`
+
+* It sees (and can access) the network interfaces of the host.
+
+* It can bind any address, any port (for ill and for good).
+
+* Network traffic doesn't have to go through NAT, bridge, or veth.
+
+* Performance = native!
+
+Use cases:
+
+* Performance sensitive applications (VOIP, gaming, streaming...)
+
+* Peer discovery (e.g. Erlang port mapper, Raft, Serf...)
+
+---
+
+## The container driver
+
+* Container is started with `docker run --net container:id ...`
+
+* It re-uses the network stack of another container.
+
+* It shares with this other container the same interfaces, IP address(es), routes, iptables rules, etc.
+
+* Those containers can communicate over their `lo` interface.
+  <br/>(i.e. one can bind to 127.0.0.1 and the others can connect to it.)
+

slides/containers/Orchestration_Overview.md

@@ -0,0 +1,422 @@
+# Orchestration, an overview
+
+In this chapter, we will:
+
+* Explain what is orchestration and why we would need it.
+
+* Present (from a high-level perspective) some orchestrators.
+
+* Show one orchestrator (Kubernetes) in action.
+
+---
+
+class: pic
+
+## What's orchestration?
+
+![Joana Carneiro (orchestra conductor)](images/conductor.jpg)
+
+---
+
+## What's orchestration?
+
+According to Wikipedia:
+
+*Orchestration describes the __automated__ arrangement,
+coordination, and management of complex computer systems,
+middleware, and services.*
+
+--
+
+*[...] orchestration is often discussed in the context of 
+__service-oriented architecture__, __virtualization__, provisioning, 
+Converged Infrastructure and __dynamic datacenter__ topics.*
+
+--
+
+What does that really mean?
+
+---
+
+## Example 1: dynamic cloud instances
+
+--
+
+- Q: do we always use 100% of our servers?
+
+--
+
+- A: obviously not!
+
+.center[![Daily variations of traffic](images/traffic-graph.png)]
+
+---
+
+## Example 1: dynamic cloud instances
+
+- Every night, scale down
+  
+  (by shutting down extraneous replicated instances)
+
+- Every morning, scale up
+  
+  (by deploying new copies)
+
+- "Pay for what you use"
+  
+  (i.e. save big $$$ here)
+
+---
+
+## Example 1: dynamic cloud instances
+
+How do we implement this?
+
+- Crontab
+  
+- Autoscaling (save even bigger $$$)
+
+That's *relatively* easy.
+
+Now, how are things for our IAAS provider?
+
+---
+
+## Example 2: dynamic datacenter
+
+- Q: what's the #1 cost in a datacenter?
+
+--
+
+- A: electricity!
+
+--
+
+- Q: what uses electricity?
+
+--
+
+- A: servers, obviously
+
+- A: ... and associated cooling
+
+--
+
+- Q: do we always use 100% of our servers?
+
+--
+
+- A: obviously not!
+
+---
+
+## Example 2: dynamic datacenter
+
+- If only we could turn off unused servers during the night...
+
+- Problem: we can only turn off a server if it's totally empty!
+  
+  (i.e. all VMs on it are stopped/moved)
+
+- Solution: *migrate* VMs and shutdown empty servers
+  
+  (e.g. combine two hypervisors with 40% load into 80%+0%,
+  <br/>and shutdown the one at 0%)
+
+---
+
+## Example 2: dynamic datacenter
+
+How do we implement this?
+
+- Shutdown empty hosts (but keep some spare capacity)
+
+- Start hosts again when capacity gets low
+
+- Ability to "live migrate" VMs
+  
+  (Xen already did this 10+ years ago)
+
+- Rebalance VMs on a regular basis
+  
+  - what if a VM is stopped while we move it?
+  - should we allow provisioning on hosts involved in a migration?
+
+*Scheduling* becomes more complex.
+
+---
+
+## What is scheduling?
+
+According to Wikipedia (again):
+
+*In computing, scheduling is the method by which threads, 
+processes or data flows are given access to system resources.*
+
+The scheduler is concerned mainly with:
+
+- throughput (total amount or work done per time unit);
+- turnaround time (between submission and completion);
+- response time (between submission and start);
+- waiting time (between job readiness and execution);
+- fairness (appropriate times according to priorities).
+
+In practice, these goals often conflict.
+
+**"Scheduling" = decide which resources to use.**
+
+---
+
+## Exercise 1
+
+- You have:
+
+  - 5 hypervisors (physical machines)
+
+- Each server has:
+
+  - 16 GB RAM, 8 cores, 1 TB disk
+
+- Each week, your team asks:
+
+  - one VM with X RAM, Y CPU, Z disk
+
+Scheduling = deciding which hypervisor to use for each VM.
+
+Difficulty: easy!
+
+---
+
+<!-- Warning, two almost identical slides (for img effect) -->
+
+## Exercise 2
+
+- You have:
+
+  - 1000+ hypervisors (and counting!)
+
+- Each server has different resources:
+
+  - 8-500 GB of RAM, 4-64 cores, 1-100 TB disk
+
+- Multiple times a day, a different team asks for:
+
+  - up to 50 VMs with different characteristics
+
+Scheduling = deciding which hypervisor to use for each VM.
+
+Difficulty: ???
+
+---
+
+<!-- Warning, two almost identical slides (for img effect) -->
+
+## Exercise 2
+
+- You have:
+
+  - 1000+ hypervisors (and counting!)
+
+- Each server has different resources:
+
+  - 8-500 GB of RAM, 4-64 cores, 1-100 TB disk
+
+- Multiple times a day, a different team asks for:
+
+  - up to 50 VMs with different characteristics
+
+Scheduling = deciding which hypervisor to use for each VM.
+
+![Troll face](images/trollface.png)
+
+---
+
+## Exercise 3
+
+- You have machines (physical and/or virtual)
+
+- You have containers
+
+- You are trying to put the containers on the machines
+
+- Sounds familiar?
+
+---
+
+## Scheduling with one resource
+
+.center[![Not-so-good bin packing](images/binpacking-1d-1.gif)]
+
+Can we do better?
+
+---
+
+## Scheduling with one resource
+
+.center[![Better bin packing](images/binpacking-1d-2.gif)]
+
+Yup!
+
+---
+
+## Scheduling with two resources
+
+.center[![2D bin packing](images/binpacking-2d.gif)]
+
+---
+
+## Scheduling with three resources
+
+.center[![3D bin packing](images/binpacking-3d.gif)]
+
+---
+
+## You need to be good at this
+
+.center[![Tangram](images/tangram.gif)]
+
+---
+
+## But also, you must be quick!
+
+.center[![Tetris](images/tetris-1.png)]
+
+---
+
+## And be web scale!
+
+.center[![Big tetris](images/tetris-2.gif)]
+
+---
+
+## And think outside (?) of the box!
+
+.center[![3D tetris](images/tetris-3.png)]
+
+---
+
+## Good luck!
+
+.center[![FUUUUUU face](images/fu-face.jpg)]
+
+---
+
+## TL,DR
+
+* Scheduling with multiple resources (dimensions) is hard.
+
+* Don't expect to solve the problem with a Tiny Shell Script.
+
+* There are literally tons of research papers written on this.
+
+---
+
+## But our orchestrator also needs to manage ...
+
+* Network connectivity (or filtering) between containers.
+
+* Load balancing (external and internal).
+
+* Failure recovery (if a node or a whole datacenter fails).
+
+* Rolling out new versions of our applications.
+
+  (Canary deployments, blue/green deployments...)
+
+
+---
+
+## Some orchestrators
+
+We are going to present briefly a few orchestrators.
+
+There is no "absolute best" orchestrator.
+
+It depends on:
+
+- your applications,
+
+- your requirements,
+
+- your pre-existing skills...
+
+---
+
+## Nomad
+
+- Open Source project by Hashicorp.
+
+- Arbitrary scheduler (not just for containers).
+
+- Great if you want to schedule mixed workloads.
+
+  (VMs, containers, processes...)
+
+- Less integration with the rest of the container ecosystem.
+
+---
+
+## Mesos
+
+- Open Source project in the Apache Foundation.
+
+- Arbitrary scheduler (not just for containers).
+
+- Two-level scheduler.
+
+- Top-level scheduler acts as a resource broker.
+
+- Second-level schedulers (aka "frameworks") obtain resources from top-level.
+
+- Frameworks implement various strategies.
+
+  (Marathon = long running processes; Chronos = run at intervals; ...)
+
+- Commercial offering through DC/OS my Mesosphere.
+
+---
+
+## Rancher
+
+- Rancher 1 offered a simple interface for Docker hosts.
+
+- Rancher 2 is a complete management platform for Docker and Kubernetes.
+
+- Technically not an orchestrator, but it's a popular option.
+
+---
+
+## Swarm
+
+- Tightly integrated with the Docker Engine.
+
+- Extremely simple to deploy and setup, even in multi-manager (HA) mode.
+
+- Secure by default.
+
+- Strongly opinionated:
+
+  - smaller set of features,
+
+  - easier to operate.
+
+---
+
+## Kubernetes
+
+- Open Source project initiated by Google.
+
+- Contributions from many other actors.
+
+- *De facto* standard for container orchestration.
+
+- Many deployment options; some of them very complex.
+
+- Reputation: steep learning curve.
+
+- Reality:
+
+  - true, if we try to understand *everything*;
+
+  - false, if we focus on what matters.
+

slides/containers/Publishing_To_Docker_Hub.md

@@ -0,0 +1,102 @@
+# Publishing images to the Docker Hub
+
+We have built our first images.
+
+We can now publish it to the Docker Hub!
+
+*You don't have to do the exercises in this section,
+because they require an account on the Docker Hub, and we
+don't want to force anyone to create one.*
+
+*Note, however, that creating an account on the Docker Hub
+is free (and doesn't require a credit card), and hosting
+public images is free as well.*
+
+---
+
+## Logging into our Docker Hub account
+
+* This can be done from the Docker CLI:
+  ```bash
+  docker login
+  ```
+
+.warning[When running Docker for Mac/Windows, or
+Docker on a Linux workstation, it can (and will when
+possible) integrate with your system's keyring to
+store your credentials securely. However, on most Linux
+servers, it will store your credentials in `~/.docker/config`.]
+
+---
+
+## Image tags and registry addresses
+
+* Docker images tags are like Git tags and branches.
+
+* They are like *bookmarks* pointing at a specific image ID.
+
+* Tagging an image doesn't *rename* an image: it adds another tag.
+
+* When pushing an image to a registry, the registry address is in the tag.
+
+  Example: `registry.example.net:5000/image`
+
+* What about Docker Hub images?
+
+--
+
+* `jpetazzo/clock` is, in fact, `index.docker.io/jpetazzo/clock`
+
+* `ubuntu` is, in fact, `library/ubuntu`, i.e. `index.docker.io/library/ubuntu`
+
+---
+
+## Tagging an image to push it on the Hub
+
+* Let's tag our `figlet` image (or any other to our liking):
+  ```bash
+  docker tag figlet jpetazzo/figlet
+  ```
+
+* And push it to the Hub:
+  ```bash
+  docker push jpetazzo/figlet
+  ```
+
+* That's it!
+
+--
+
+* Anybody can now `docker run jpetazzo/figlet` anywhere.
+
+---
+
+## The goodness of automated builds
+
+* You can link a Docker Hub repository with a GitHub or BitBucket repository
+
+* Each push to GitHub or BitBucket will trigger a build on Docker Hub
+
+* If the build succeeds, the new image is available on Docker Hub
+
+* You can map tags and branches between source and container images
+
+* If you work with public repositories, this is free
+
+---
+
+class: extra-details
+
+## Setting up an automated build
+
+* We need a Dockerized repository!
+* Let's go to https://github.com/jpetazzo/trainingwheels and fork it.
+* Go to the Docker Hub (https://hub.docker.com/).
+* Select "Create" in the top-right bar, and select "Create Automated Build."
+* Connect your Docker Hub account to your GitHub account.
+* Select your user and the repository that we just forked.
+* Create.
+* Then go to "Build Settings."
+* Put `/www` in "Dockerfile Location" (or whichever directory the Dockerfile is in).
+* Click "Trigger" to build the repository immediately (without waiting for a git push).
+* Subsequent builds will happen automatically, thanks to GitHub hooks.

slides/containers/Resource_Limits.md

@@ -0,0 +1,229 @@
+# Limiting resources
+
+- So far, we have used containers as convenient units of deployment.
+
+- What happens when a container tries to use more resources than available?
+
+  (RAM, CPU, disk usage, disk and network I/O...)
+
+- What happens when multiple containers compete for the same resource?
+
+- Can we limit resources available to a container?
+
+  (Spoiler alert: yes!)
+
+---
+
+## Container processes are normal processes
+
+- Containers are closer to "fancy processes" than to "lightweight VMs".
+
+- A process running in a container is, in fact, a process running on the host.
+
+- Let's look at the output of `ps` on a container host running 3 containers :
+
+  ```
+       0  2662  0.2  0.3 /usr/bin/dockerd -H fd://
+       0  2766  0.1  0.1  \_ docker-containerd --config /var/run/docker/containe
+       0 23479  0.0  0.0      \_ docker-containerd-shim -namespace moby -workdir
+       0 23497  0.0  0.0      |   \_ `nginx`: master process nginx -g daemon off;
+     101 23543  0.0  0.0      |       \_ `nginx`: worker process
+       0 23565  0.0  0.0      \_ docker-containerd-shim -namespace moby -workdir
+     102 23584  9.4 11.3      |   \_ `/docker-java-home/jre/bin/java` -Xms2g -Xmx2
+       0 23707  0.0  0.0      \_ docker-containerd-shim -namespace moby -workdir
+       0 23725  0.0  0.0          \_ `/bin/sh`
+  ```
+
+- The highlighted processes are containerized processes.
+  <br/>
+  (That host is running nginx, elasticsearch, and alpine.)
+
+---
+
+## By default: nothing changes
+
+- What happens when a process uses too much memory on a Linux system?
+
+--
+
+- Simplified answer:
+
+  - swap is used (if available);
+
+  - if there is not enough swap space, eventually, the out-of-memory killer is invoked;
+
+  - the OOM killer uses heuristics to kill processes;
+
+  - sometimes, it kills an unrelated process.
+
+--
+
+- What happens when a container uses too much memory?
+
+- The same thing!
+
+  (i.e., a process eventually gets killed, possibly in another container.)
+
+---
+
+## Limiting container resources
+
+- The Linux kernel offers rich mechanisms to limit container resources.
+
+- For memory usage, the mechanism is part of the *cgroup* subsystem.
+
+- This subsystem allows to limit the memory for a process or a group of processes.
+
+- A container engine leverages these mechanisms to limit memory for a container.
+
+- The out-of-memory killer has a new behavior:
+
+  - it runs when a container exceeds its allowed memory usage,
+
+  - in that case, it only kills processes in that container.
+
+---
+
+## Limiting memory in practice
+
+- The Docker Engine offers multiple flags to limit memory usage.
+
+- The two most useful ones are `--memory` and `--memory-swap`.
+
+- `--memory` limits the amount of physical RAM used by a container.
+
+- `--memory-swap` limits the total amount (RAM+swap) used by a container.
+
+- The memory limit can be expressed in bytes, or with a unit suffix.
+
+  (e.g.: `--memory 100m` = 100 megabytes.)
+
+- We will see two strategies: limiting RAM usage, or limiting both
+
+---
+
+## Limiting RAM usage
+
+Example:
+
+```bash
+docker run -ti --memory 100m python
+```
+
+If the container tries to use more than 100 MB of RAM, *and* swap is available:
+
+- the container will not be killed,
+
+- memory above 100 MB will be swapped out,
+
+- in most cases, the app in the container will be slowed down (a lot).
+
+If we run out of swap, the global OOM killer still intervenes.
+
+---
+
+## Limiting both RAM and swap usage
+
+Example:
+
+```bash
+docker run -ti --memory 100m --memory-swap 100m python
+```
+
+If the container tries to use more than 100 MB of memory, it is killed.
+
+On the other hand, the application will never be slowed down because of swap.
+
+---
+
+## When to pick which strategy?
+
+- Stateful services (like databases) will lose or corrupt data when killed
+
+- Allow them to use swap space, but monitor swap usage
+
+- Stateless services can usually be killed with little impact
+
+- Limit their mem+swap usage, but monitor if they get killed
+
+- Ultimately, this is no different from "do I want swap, and how much?"
+
+---
+
+## Limiting CPU usage
+
+- There are no less than 3 ways to limit CPU usage:
+
+  - setting a relative priority with `--cpu-shares`,
+
+  - setting a CPU% limit with `--cpus`,
+
+  - pinning a container to specific CPUs with `--cpuset-cpus`.
+
+- They can be used separately or together.
+
+---
+
+## Setting relative priority
+
+- Each container has a relative priority used by the Linux scheduler.
+
+- By default, this priority is 1024.
+
+- As long as CPU usage is not maxed out, this has no effect.
+
+- When CPU usage is maxed out, each container receives CPU cycles in proportion of its relative priority.
+
+- In other words: a container with `--cpu-shares 2048` will receive twice as much than the default.
+
+---
+
+## Setting a CPU% limit
+
+- This setting will make sure that a container doesn't use more than a given % of CPU.
+
+- The value is expressed in CPUs; therefore:
+
+  `--cpus 0.1` means 10% of one CPU,
+
+  `--cpus 1.0` means 100% of one whole CPU,
+
+  `--cpus 10.0` means 10 entire CPUs.
+
+---
+
+## Pinning containers to CPUs
+
+- On multi-core machines, it is possible to restrict the execution on a set of CPUs.
+
+- Examples:
+
+  `--cpuset-cpus 0` forces the container to run on CPU 0;
+
+  `--cpuset-cpus 3,5,7` restricts the container to CPUs 3, 5, 7;
+
+  `--cpuset-cpus 0-3,8-11` restricts the container to CPUs 0, 1, 2, 3, 8, 9, 10, 11.
+
+- This will not reserve the corresponding CPUs!
+
+  (They might still be used by other containers, or uncontainerized processes.)
+
+---
+
+## Limiting disk usage
+
+- Most storage drivers do not support limiting the disk usage of containers.
+
+  (With the exception of devicemapper, but the limit cannot be set easily.)
+ 
+- This means that a single container could exhaust disk space for everyone.
+
+- In practice, however, this is not a concern, because:
+
+  - data files (for stateful services) should reside on volumes,
+
+  - assets (e.g. images, user-generated content...) should reside on object stores or on volume,
+
+  - logs are written on standard output and gathered by the container engine.
+
+- Container disk usage can be audited with `docker ps -s` and `docker diff`.

slides/containers/Training_Environment.md

@@ -0,0 +1,126 @@
+class: title
+
+# Our training environment
+
+![SSH terminal](images/title-our-training-environment.jpg)
+
+---
+
+## Our training environment
+
+- If you are attending a tutorial or workshop:
+
+  - a VM has been provisioned for each student
+
+- If you are doing or re-doing this course on your own, you can:
+
+  - install Docker locally (as explained in the chapter "Installing Docker")
+
+  - install Docker on e.g. a cloud VM
+
+  - use http://www.play-with-docker.com/ to instantly get a training environment
+
+---
+
+## Our Docker VM
+
+*This section assumes that you are following this course as part of
+a tutorial, training or workshop, where each student is given an
+individual Docker VM.*
+
+- The VM is created just before the training.
+
+- It will stay up during the whole training.
+
+- It will be destroyed shortly after the training.
+
+- It comes pre-loaded with Docker and some other useful tools.
+
+---
+
+## What *is* Docker?
+
+- "Installing Docker" really means "Installing the Docker Engine and CLI".
+
+- The Docker Engine is a daemon (a service running in the background).
+
+- This daemon manages containers, the same way that an hypervisor manages VMs.
+
+- We interact with the Docker Engine by using the Docker CLI.
+
+- The Docker CLI and the Docker Engine communicate through an API.
+
+- There are many other programs, and many client libraries, to use that API.
+
+---
+
+## Why don't we run Docker locally?
+
+- We are going to download container images and distribution packages.
+
+- This could put a bit of stress on the local WiFi and slow us down.
+
+- Instead, we use a remote VM that has a good connectivity
+
+- In some rare cases, installing Docker locally is challenging:
+
+  - no administrator/root access (computer managed by strict corp IT)
+
+  - 32-bit CPU or OS
+
+  - old OS version (e.g. CentOS 6, OSX pre-Yosemite, Windows 7)
+
+- It's better to spend time learning containers than fiddling with the installer!
+
+---
+
+## Connecting to your Virtual Machine
+
+You need an SSH client.
+
+* On OS X, Linux, and other UNIX systems, just use `ssh`:
+
+```bash
+$ ssh <login>@<ip-address>
+```
+
+* On Windows, if you don't have an SSH client, you can download:
+
+  * Putty (www.putty.org)
+
+  * Git BASH (https://git-for-windows.github.io/)
+
+  * MobaXterm (http://moabaxterm.mobatek.net)
+
+---
+
+## Checking your Virtual Machine
+
+Once logged in, make sure that you can run a basic Docker command:
+
+.small[
+```bash
+$ docker version
+Client:
+ Version:       18.03.0-ce
+ API version:   1.37
+ Go version:    go1.9.4
+ Git commit:    0520e24
+ Built:         Wed Mar 21 23:10:06 2018
+ OS/Arch:       linux/amd64
+ Experimental:  false
+ Orchestrator:  swarm
+
+Server:
+ Engine:
+  Version:      18.03.0-ce
+  API version:  1.37 (minimum version 1.12)
+  Go version:   go1.9.4
+  Git commit:   0520e24
+  Built:        Wed Mar 21 23:08:35 2018
+  OS/Arch:      linux/amd64
+  Experimental: false
+```
+]
+
+If this doesn't work, raise your hand so that an instructor can assist you!

slides/containers/Working_With_Volumes.md

@@ -1,9 +1,9 @@
 
 class: title
 
-# Working with Volumes
+# Working with volumes
 
-![volume](images/volume.jpg)
+![volume](images/title-working-with-volumes.jpg)
 
 ---
 
@@ -19,7 +19,7 @@ At the end of this section, you will be able to:
 
 ---
 
-## Working with Volumes
+## Working with volumes
 
 Docker volumes can be used to achieve many things, including:
 
@@ -33,6 +33,8 @@ Docker volumes can be used to achieve many things, including:
 
 * Sharing a *single file* between the host and a container.
 
+* Using remote storage and custom storage with "volume drivers".
+
 ---
 
 ## Volumes are special directories in a container
@@ -95,12 +97,12 @@ We will see an example in the following slides.
 
 class: extra-details
 
-## Sharing web application logs with another container
+## Sharing app server logs with another container
 
 Let's start a Tomcat container:
 
 ```bash
-$ docker run --name webapp -d -p 8080:8080 -v /usr/local/tomcat/logs
+$ docker run --name webapp -d -p 8080:8080 -v /usr/local/tomcat/logs tomcat
 ```
 
 Now, start an `alpine` container accessing the same volume:
@@ -118,7 +120,7 @@ $ curl localhost:8080
 
 ## Volumes exist independently of containers
 
-If a container is stopped, its volumes still exist and are available.
+If a container is stopped or removed, its volumes still exist and are available.
 
 Volumes can be listed and manipulated with `docker volume` subcommands:
 
@@ -201,7 +203,7 @@ Then run `curl localhost:1234` again to see your changes.
 
 ---
 
-## Managing volumes explicitly
+## Using custom "bind-mounts"
 
 In some cases, you want a specific directory on the host to be mapped
 inside the container:
@@ -244,6 +246,8 @@ of an existing container.
 
 * Newer containers can use `--volumes-from` too.
 
+* Doesn't work across servers, so not usable in clusters (Swarm, Kubernetes).
+
 ---
 
 class: extra-details
@@ -259,7 +263,7 @@ $ docker run -d --name redis28 redis:2.8
 Connect to the Redis container and set some data.
 
 ```bash
-$ docker run -ti --link redis28:redis alpine telnet redis 6379
+$ docker run -ti --link redis28:redis busybox telnet redis 6379
 ```
 
 Issue the following commands:
@@ -298,7 +302,7 @@ class: extra-details
 Connect to the Redis container and see our data.
 
 ```bash
-docker run -ti --link redis30:redis alpine telnet redis 6379
+docker run -ti --link redis30:redis busybox telnet redis 6379
 ```
 
 Issue a few commands.
@@ -311,9 +315,9 @@ QUIT
 
 ---
 
-## What happens when you remove containers with volumes?
+## Volumes lifecycle
 
-* Volumes are kept around.
+* When you remove a container, its volumes are kept around.
 
 * You can list them with `docker volume ls`.
 
@@ -371,9 +375,9 @@ $ docker inspect <yourContainerID>
 
 ---
 
-## Sharing a single file between the host and a container
+## Sharing a single file
 
-The same `-v` flag can be used to share a single file.
+The same `-v` flag can be used to share a single file (instead of a directory).
 
 One of the most interesting examples is to share the Docker control socket.
 
@@ -381,8 +385,11 @@ One of the most interesting examples is to share the Docker control socket.
 $ docker run -it -v /var/run/docker.sock:/var/run/docker.sock docker sh
 ```
 
-Warning: when using such mounts, the container gains root-like access to the host.
-It can potentially do bad things.
+From that container, you can now run `docker` commands communicating with
+the Docker Engine running on the host. Try `docker ps`!
+
+.warning[Since that container has access to the Docker socket, it
+has root-like access to the host.]
 
 ---
 
@@ -391,10 +398,56 @@ It can potentially do bad things.
 You can install plugins to manage volumes backed by particular storage systems,
 or providing extra features. For instance:
 
-* [dvol](https://github.com/ClusterHQ/dvol) - allows to commit/branch/rollback volumes;
-* [Flocker](https://clusterhq.com/flocker/introduction/), [REX-Ray](https://github.com/emccode/rexray) - create and manage volumes backed by an enterprise storage system (e.g. SAN or NAS), or by cloud block stores (e.g. EBS);
-* [Blockbridge](http://www.blockbridge.com/), [Portworx](http://portworx.com/) - provide distributed block store for containers;
-* and much more!
+* [REX-Ray](https://rexray.io/) - create and manage volumes backed by an enterprise storage system (e.g.
+  SAN or NAS), or by cloud block stores (e.g. EBS, EFS).
+
+* [Portworx](http://portworx.com/) - provides distributed block store for containers.
+
+* [Gluster](https://www.gluster.org/) - open source software-defined distributed storage that can scale
+  to several petabytes. It provides interfaces for object, block and file storage.
+
+* and much more at the [Docker Store](https://store.docker.com/search?category=volume&q=&type=plugin)!
+
+---
+
+## Volumes vs. Mounts
+
+* Since Docker 17.06, a new options is available: `--mount`.
+
+* It offers a new, richer syntax to manipulate data in containers.
+
+* It makes an explicit difference between:
+
+  - volumes (identified with a unique name, managed by a storage plugin),
+
+  - bind mounts (identified with a host path, not managed).
+
+* The former `-v` / `--volume` option is still usable.
+
+---
+
+## `--mount` syntax
+
+Binding a host path to a container path:
+
+```bash
+$ docker run \
+  --mount type=bind,source=/path/on/host,target=/path/in/container alpine
+```
+
+Mounting a volume to a container path:
+
+```bash
+$ docker run \
+  --mount source=myvolume,target=/path/in/container alpine
+```
+
+Mounting a tmpfs (in-memory, for temporary files):
+
+```bash
+$ docker run \
+  --mount type=tmpfs,destination=/path/in/container,tmpfs-size=1000000 alpine
+```
 
 ---
 

slides/containers/intro.md

@@ -0,0 +1,39 @@
+## A brief introduction
+
+- This was initially written to support in-person, instructor-led workshops and tutorials
+
+- These materials are maintained by [Jérôme Petazzoni](https://twitter.com/jpetazzo) and [multiple contributors](https://@@GITREPO@@/graphs/contributors)
+
+- You can also follow along on your own, at your own pace
+
+- We included as much information as possible in these slides
+
+- We recommend having a mentor to help you ...
+
+- ... Or be comfortable spending some time reading the Docker
+ [documentation](https://docs.docker.com/) ...
+
+- ... And looking for answers in the [Docker forums](forums.docker.com),
+  [StackOverflow](http://stackoverflow.com/questions/tagged/docker),
+  and other outlets
+
+---
+
+class: self-paced
+
+## Hands on, you shall practice
+
+- Nobody ever became a Jedi by spending their lives reading Wookiepedia
+
+- Likewise, it will take more than merely *reading* these slides
+  to make you an expert
+
+- These slides include *tons* of exercises and examples
+
+- They assume that you have acccess to a machine running Docker
+
+- If you are attending a workshop or tutorial:
+  <br/>you will be given specific instructions to access a cloud VM
+
+- If you are doing this on your own:
+  <br/>we will tell you how to install Docker or access a Docker environment

slides/containers/links.md

@@ -0,0 +1 @@
+../swarm/links.md

slides/count-slides.py

@@ -0,0 +1,57 @@
+#!/usr/bin/env python
+import re
+import sys
+
+PREFIX = "name: toc-"
+EXCLUDED = ["in-person"]
+
+class State(object):
+    def __init__(self):
+        self.current_slide = 1
+        self.section_title = None
+        self.section_start = 0
+        self.section_slides = 0
+        self.chapters = {}
+        self.sections = {}
+    def show(self):
+        if self.section_title.startswith("chapter-"):
+            return
+        print("{0.section_title}\t{0.section_start}\t{0.section_slides}".format(self))
+        self.sections[self.section_title] = self.section_slides
+
+state = State()
+
+title = None
+for line in open(sys.argv[1]):
+    line = line.rstrip()
+    if line.startswith(PREFIX):
+        if state.section_title is None:
+            print("{}\t{}\t{}".format("title", "index", "size"))
+        else:
+            state.show()
+        state.section_title = line[len(PREFIX):].strip()
+        state.section_start = state.current_slide
+        state.section_slides = 0
+    if line == "---":
+        state.current_slide += 1
+        state.section_slides += 1
+    if line == "--":
+        state.current_slide += 1
+    toc_links = re.findall("\(#toc-(.*)\)", line)
+    if toc_links and state.section_title.startswith("chapter-"):
+        if state.section_title not in state.chapters:
+            state.chapters[state.section_title] = []
+        state.chapters[state.section_title].append(toc_links[0])
+    # This is really hackish
+    if line.startswith("class:"):
+        for klass in EXCLUDED:
+            if klass in line:
+                state.section_slides -= 1
+                state.current_slide -= 1
+
+state.show()
+
+for chapter in sorted(state.chapters):
+    chapter_size = sum(state.sections[s] for s in state.chapters[chapter])
+    print("{}\t{}\t{}".format("total size for", chapter, chapter_size))
+

slides/dockercon.yml

@@ -1,182 +0,0 @@
-chat: "[Slack](https://dockercommunity.slack.com/messages/C7ET1GY4Q)"
-
-exclude:
-- self-paced
-- snap
-- auto-btp
-- benchmarking
-- elk-manual
-- prom-manual
-
-title: "Swarm: from Zero to Hero (DC17EU)"
-chapters:
-- |
-  class: title
-
-  .small[
-
-  Swarm: from Zero to Hero
-
-  .small[.small[
-
-  **Be kind to the WiFi!**
-
-  *Use the 5G network*
-  <br/>
-  *Don't use your hotspot*
-  <br/>
-  *Don't stream videos from YouTube, Netflix, etc.
-  <br/>(if you're bored, watch local content instead)*
-
-  Also: share the power outlets
-  <br/>
-  *(with limited power comes limited responsibility?)*
-  <br/>
-  *(or something?)*
-
-  Thank you!
-
-  ]
-  ]
-  ]
-
-  ---
-
-  ## Intros
- 
-  <!--
-  - Hello! We are
-    AJ ([@s0ulshake](https://twitter.com/s0ulshake))
-    &
-    Jérôme ([@jpetazzo](https://twitter.com/jpetazzo))
-  -->
-
-  - Hello! We are Jérôme, Lee, Nicholas, and Scott
-
-  <!--
-    I am
-    Jérôme ([@jpetazzo](https://twitter.com/jpetazzo))
-  -->
-
-  --
-
-  - This is our collective Docker knowledge:
-
-    ![Bell Curve](images/bell-curve.jpg)
-
-  ---
-
-  ## "From zero to hero"
-
-  --
-
-  - It rhymes, but it's a pretty bad title, to be honest
-
-  --
-  
-  - None of you is a "zero"
-
-  --
-  
-  - None of us is a "hero"
-
-  --
-  
-  - None of us should even try to be a hero
-
-  --
-  
-    *The hero syndrome is a phenomenon affecting people who seek heroism or recognition,
-    usually by creating a desperate situation which they can resolve.
-    This can include unlawful acts, such as arson.
-    The phenomenon has been noted to affect civil servants,
-    such as firefighters, nurses, police officers, and security guards.*
-
-    (Wikipedia page on [hero syndrome](https://en.wikipedia.org/wiki/Hero_syndrome))
-
-  ---
-
-  ## Agenda
-
-  .small[
-  - 09:00-09:10 Hello!
-  - 09:10-10:30 Part 1
-  - 10:30-11:00 coffee break
-  - 11:00-12:30 Part 2
-  - 12:30-13:30 lunch break
-  - 13:30-15:00 Part 3
-  - 15:00-15:30 coffee break
-  - 15:30-17:00 Part 4
-  - 17:00-18:00 Afterhours and Q&A
-  ]
-
-  <!--
-  - The tutorial will run from 9:00am to 12:20pm
-
-  - This will be fast-paced, but DON'T PANIC!
-
-  - There will be a coffee break at 10:30am
-    <br/>
-    (please remind me if I forget about it!)
-  -->
-
-  - All the content is publicly available (slides, code samples, scripts)
-
-    Upstream URL: https://github.com/jpetazzo/orchestration-workshop
-
-  - Feel free to interrupt for questions at any time
-
-  - Live feedback, questions, help on [Gitter](chat)
-
-    http://container.training/chat
-
-- swarm/intro.md
-- |
-  @@TOC@@
-- - swarm/prereqs.md
-  - swarm/versions.md
-  - |
-    class: title
-
-    All right!
-    <br/>
-    We're all set.
-    <br/>
-    Let's do this.
-  - common/sampleapp.md
-  - swarm/swarmkit.md
-  - swarm/creatingswarm.md
-  - swarm/morenodes.md
-- - swarm/firstservice.md
-  - swarm/ourapponswarm.md
-  - swarm/updatingservices.md
-  - swarm/healthchecks.md
-- - swarm/operatingswarm.md
-  - swarm/netshoot.md
-  - swarm/ipsec.md
-  - swarm/swarmtools.md
-  - swarm/security.md
-  - swarm/secrets.md
-  - swarm/encryptionatrest.md
-  - swarm/leastprivilege.md
-  - swarm/apiscope.md
-- - swarm/logging.md
-  - swarm/metrics.md
-  - swarm/stateful.md
-  - swarm/extratips.md
-  - swarm/end.md
-- |
-  class: title
-
-  That's all folks! <br/> Questions?
-
-  .small[.small[
-
-  Jérôme ([@jpetazzo](https://twitter.com/jpetazzo)) — [@docker](https://twitter.com/docker)
-
-  ]]
-
-  <!--
-  Tiffany ([@tiffanyfayj](https://twitter.com/tiffanyfayj))
-  AJ ([@s0ulshake](https://twitter.com/s0ulshake))
-  -->

slides/find-non-ascii.sh

@@ -0,0 +1,2 @@
+#!/bin/sh
+grep --color=auto -P -n "[^\x00-\x80]" */*.md

slides/images/docker-con-15-logo.svg

@@ -0,0 +1,213 @@
+<?xml version="1.0" encoding="utf-8"?>
+<!-- Generator: Adobe Illustrator 18.0.0, SVG Export Plug-In . SVG Version: 6.00 Build 0)  -->
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<svg version="1.1" id="Layer_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
+	 viewBox="0 0 445 390" enable-background="new 0 0 445 390" xml:space="preserve">
+<g>
+	<path fill="#3A4D54" d="M158.8,352.2h-25.9c3.2,0,5.8-2.6,5.8-5.9s-2.6-5.9-5.8-5.9h-19c3.2,0,5.8-2.6,5.8-5.9s-2.6-5.9-5.8-5.9
+		h25.3c3.2,0,5.8-2.6,5.8-5.9c0-3.2-2.6-5.9-5.8-5.9h-15.9c3.2,0,5.8-2.6,5.8-5.9s-2.6-5.9-5.8-5.9h6.8c3.2,0,5.8-2.6,5.8-5.9
+		c0-3.2-2.6-5.9-5.8-5.9H64.9c-0.1,0-0.3,0-0.4,0c3,0.2,5.4,2.7,5.4,5.9c0,3.1-2.4,5.7-5.4,5.9c0.1,0,0.3,0,0.4,0h-0.8h-6.1
+		c-3.2,0-5.8,2.6-5.8,5.9s2.6,5.9,5.8,5.9H74h3.7c3.2,0,5.8,2.6,5.8,5.9c0,3.2-2.6,5.9-5.8,5.9H74H47.9c-3.2,0-5.8,2.6-5.8,5.9
+		s2.6,5.9,5.8,5.9h44.8H93c0,0-0.1,0-0.1,0c3.1,0.1,5.6,2.7,5.6,5.9c0,3.2-2.5,5.8-5.6,5.9c0,0,0.1,0,0.1,0h-0.2
+		c-3.2,0-5.8,2.6-5.8,5.9c0,3.2,2.6,5.9,5.8,5.9h66c3.2,0,5.8-2.6,5.8-5.9C164.6,354.8,162,352.2,158.8,352.2z"/>
+	<circle fill="#FBBF45" stroke="#3A4D54" stroke-width="4" stroke-miterlimit="10" cx="214.6" cy="124.2" r="68.7"/>
+	<circle fill="#3A4D54" cx="367.5" cy="335.5" r="5.9"/>
+	<g>
+		<polygon fill="#E8593A" stroke="#3A4D54" stroke-width="4" stroke-miterlimit="10" points="116.1,199.1 116.1,214.6 302.9,214.5 
+			302.9,199.1 		"/>
+		<rect x="159.4" y="78.6" fill="#3A4D54" width="4.2" height="50.4"/>
+		<rect x="174.5" y="93.8" fill="#3A4D54" width="4.2" height="35.1"/>
+		<rect x="280.2" y="108.2" fill="#3A4D54" width="4.2" height="20.8"/>
+		<rect x="190.2" y="106.9" fill="#3A4D54" width="4.2" height="22"/>
+		<rect x="143.3" y="59.8" fill="#3A4D54" width="4.2" height="69.1"/>
+		<path fill="#3A4D54" d="M294.3,107.9c3.5-2.3,6.9-4.8,10.4-7.4V87.7c-5.2,4.3-10.6,8.2-15.9,11.6c-7.8,4.9-15.1,8.5-22.4,11
+			c-7.9,2.8-15.7,4.3-23.4,4.7c-7.6,0.3-15.3-0.5-22.8-2.6c-6.9-1.9-13.7-4.7-20.4-8.6C188.8,97.5,178.4,89,168,77.6
+			c-7.7-8.4-14.7-17.7-21.6-28.2c-5-7.8-9.6-15.8-13.6-23.9c-4-8.1-6.1-13.5-6.9-16c-0.7-1.8-1-3.1-1.2-3.8l0-0.1l0.1-2.7l-0.5,0
+			l0-0.1H123l-8.1-0.6l-3.1-0.1l-0.1,3.4l0,0.4c0,1.2,0.2,1.9,0.3,2.5l0,0.1c0.3,1.4,0.9,3.2,1.7,5.3c1.2,3.4,3.6,9.1,7.7,17.2
+			c4.3,8.4,9.2,16.8,14.6,25c7.3,11.1,14.9,20.8,23.2,29.6c11.4,12.1,22.9,21.3,35.1,28.1c7.6,4.2,15.4,7.4,23.2,9.4
+			c7,1.8,14.2,2.7,21.4,2.7c0,0,0,0,0,0c1.6,0,3.2,0,4.7-0.1c8.7-0.5,17.6-2.4,26.4-5.6 M141.1,52.8c-5.2-7.9-10-16.1-14.2-24.4
+			c-4-7.9-6.3-13.4-7.5-16.6c-0.5-1.3-0.8-2.4-1.1-3.3l1,0.1c0.3,0.9,0.6,1.9,1,2.9c1.6,4.5,4.2,10.4,7.2,16.6
+			c4.1,8.3,8.8,16.5,13.9,24.5c5.5,8.5,11.1,16.2,17.1,23.3C152.4,68.9,146.7,61.3,141.1,52.8z"/>
+		<path fill="#E8593A" stroke="#3A4D54" stroke-width="4" stroke-miterlimit="10" d="M340.9,53h-7.9h-4.3v8.2h-19.4V53h-4.3h-7.9
+			h-4.3v8.2v2.7v186.7c0,0.8,0.6,1.4,1.3,1.4h3h42.4h4.3c0.7,0,1.3-0.6,1.3-1.4V62v-0.8V53H340.9z M334.8,206.6h-31.5V152
+			c0-0.4,0.3-0.7,0.6-0.7h30.2c0.4,0,0.6,0.3,0.6,0.7V206.6z M334.8,142.1h-31.5V125c0-0.4,0.3-0.7,0.6-0.7h30.2
+			c0.4,0,0.6,0.3,0.6,0.7V142.1z M334.8,115.1h-31.5V97.9c0-0.4,0.3-0.7,0.6-0.7h30.2c0.4,0,0.6,0.3,0.6,0.7V115.1z M334.8,88h-31.5
+			V70.9c0-0.4,0.3-0.7,0.6-0.7h30.2c0.4,0,0.6,0.3,0.6,0.7V88z"/>
+		<polygon fill="#E8593A" points="272.2,203 286.7,201.1 297.2,201.1 297.2,214.6 271.7,214.6 		"/>
+		<path fill="#E8593A" d="M298.7,96.2c-2.7,2-5.5,3.9-8.3,5.7c-7.3,4.6-15,8.5-23,11.3c-7.9,2.8-16.1,4.5-24.3,4.8
+			c-8.1,0.4-16.1-0.6-23.7-2.7c-7.6-2-14.6-5.1-21.1-8.9c-13-7.5-23.7-17.1-32.6-26.8c-8.9-9.8-16-19.6-21.9-28.6
+			c-5.8-9-10.3-17.3-13.7-24.2c-3.4-6.9-5.7-12.5-7.1-16.3c-0.7-1.9-1.1-3.3-1.3-4.2c-0.1-0.4-0.1-0.7-0.1-0.4l0,0.1
+			c0,0,0-0.1,0-0.1c0-0.1,0-0.1,0-0.1c0-0.1,0-0.1,0-0.1l-7-0.5c0,0,0,0,0,0.1c0,0,0,0.1,0,0.1c0,0,0,0.1,0,0.1c0,0.1,0,0.2,0,0.3
+			c0,0.9,0.1,1.4,0.3,2.1c0.3,1.3,0.8,2.9,1.6,5c1.5,4.1,4,9.8,7.6,16.9c3.6,7.1,8.3,15.5,14.4,24.7c6.1,9.2,13.5,19.2,22.9,29.2
+			c9.3,9.9,20.5,19.8,34.3,27.5c6.9,3.8,14.4,7,22.5,9.1c8,2.1,16.6,3,25.2,2.5c8.6-0.5,17.3-2.4,25.5-5.4c8.3-3,16.2-7.2,23.7-12
+			c2-1.3,4.1-2.7,6-4.2V96.2z"/>
+		<path fill="#E8593A" stroke="#3A4D54" stroke-width="4" stroke-miterlimit="10" d="M122.9,4.2h-3.2h-6.6v11.7H66.1V4.2h-4.6h-6.2
+			h-6.6v11.7v3.8v265.1c0,1.1,0.9,2,2,2h4.6h65.7h6.6c1.1,0,2-0.9,2-2V17v-1.1V4.2H122.9z M113.5,204.2H64.7v-59.4c0-0.6,0.4-1,1-1
+			h46.7c0.6,0,1,0.4,1,1V204.2z M113.5,130.8H64.7v-24.3c0-0.6,0.4-1,1-1h46.7c0.6,0,1,0.4,1,1V130.8z M113.5,92.4H64.7V68.1
+			c0-0.6,0.4-1,1-1h46.7c0.6,0,1,0.4,1,1V92.4z M113.5,54H64.7V29.7c0-0.6,0.4-1,1-1h46.7c0.6,0,1,0.4,1,1V54z"/>
+		<g>
+			<g>
+				<path fill="#2BB8EB" stroke="#3A4D54" stroke-width="5" stroke-miterlimit="10" d="M435.8,132.9H364c-1.4,0-2.6,1.3-2.6,3v44.2
+					c0,1.7,1.2,3,2.6,3h71.8c2.5,0,3.6-3.7,1.5-5.4l-11.4-13.5c-3.2-3.3-3.2-9,0-12.3l11.4-13.5
+					C439.3,136.6,438.3,132.9,435.8,132.9z"/>
+				<path fill="#FFFFFF" stroke="#3A4D54" stroke-width="5" stroke-miterlimit="10" d="M9.8,183.1h129.7c1.4,0,2.6-1.3,2.6-3v-44.2
+					c0-1.7-1.2-3-2.6-3H9.8c-2.5,0-3.6,3.7-1.5,5.4l11.4,13.5c3.2,3.3,3.2,9,0,12.3L8.3,177.7C6.2,179.4,7.3,183.1,9.8,183.1z"/>
+				<path fill="#FFFFFF" stroke="#3A4E55" stroke-width="5" stroke-miterlimit="10" d="M402.5,190H42.1c-3.6,0-6.5-1.1-6.5-4.6
+					v-54.7c0-3.6,2.9-6.5,6.5-6.5h360.4c3.6,0,6.5,2.9,6.5,6.5v52.9C409,187.1,406.1,190,402.5,190z"/>
+				<path fill="#2BB8EB" d="M402.5,124.2h-46.3V190h46.3c3.6,0,6.5-2.9,6.5-6.5v-52.9C409,127.1,406.1,124.2,402.5,124.2z"/>
+				<g>
+					<path fill="#FFFFFF" d="M376.2,144.3v21.3c0,1.1-0.9,2-2,2c-1.1,0-2-0.9-2-2v-17.8l-1.4,0.8c-0.3,0.2-0.7,0.3-1,0.3
+						c-0.7,0-1.3-0.4-1.7-1c-0.6-0.9-0.3-2.2,0.7-2.7l4.4-2.6c0,0,0.1,0,0.1-0.1c0.1,0,0.1-0.1,0.2-0.1c0.1,0,0.1,0,0.2,0
+						c0,0,0.1,0,0.1,0c0.1,0,0.2,0,0.3,0c0,0,0.1,0,0.1,0h0c0.1,0,0.2,0,0.3,0c0,0,0.1,0,0.1,0c0.1,0,0.1,0,0.2,0.1c0,0,0.1,0,0.1,0
+						c0.1,0.1,0.1,0.1,0.2,0.1c0,0,0.1,0.1,0.1,0.1c0,0,0.1,0.1,0.1,0.1c0.1,0,0.1,0.1,0.1,0.1c0,0,0.1,0.1,0.1,0.1
+						c0,0,0.1,0.1,0.1,0.1l0,0.1c0,0,0,0.1,0,0.1c0,0.1,0.1,0.1,0.1,0.2c0,0.1,0,0.1,0.1,0.2c0,0.1,0,0.1,0,0.2c0,0.1,0,0.2,0.1,0.3
+						C376.2,144.3,376.2,144.3,376.2,144.3z"/>
+					<path fill="#FFFFFF" d="M393.4,152.3c1.8,1.7,2.6,4.1,2.6,6.4c0,2.3-0.9,4.6-2.6,6.3c-1.7,1.8-4.1,2.6-6.3,2.6
+						c-0.1,0-0.1,0-0.1,0c-2.2,0-4.6-0.9-6.3-2.6c-0.8-0.8-0.8-2.1,0-2.9c0.8-0.8,2.1-0.8,2.9,0c0.9,1,2.2,1.4,3.5,1.4
+						c1.2,0,2.5-0.5,3.4-1.4c0.9-0.9,1.4-2.2,1.4-3.4c0-1.3-0.5-2.5-1.4-3.5c-0.9-1-2.2-1.4-3.4-1.4c-1.2,0-2.5,0.4-3.5,1.4
+						c-0.8,0.8-2.1,0.8-2.9,0c-0.1-0.1-0.3-0.3-0.4-0.5c0-0.1,0-0.1,0-0.1c0-0.1,0-0.1-0.1-0.2c0-0.1,0-0.2,0-0.3c0,0,0,0,0-0.1
+						c0-0.2,0-0.4,0-0.6l1.1-9.4c0.1-0.6,0.4-1.1,0.9-1.4c0.1,0,0.1,0,0.1-0.1c0,0,0.1,0,0.1-0.1c0.3-0.1,0.6-0.2,0.9-0.2h9.2
+						c1.2,0,2.1,0.9,2.1,2.1c0,1.1-0.9,2-2.1,2h-7.4l-0.4,3.6c0.8-0.2,1.6-0.3,2.4-0.3C389.4,149.7,391.7,150.6,393.4,152.3z"/>
+				</g>
+				<g>
+					<path fill="#3A4D54" d="M157.8,142.1L157.8,142.1l-0.9,0c-0.7,0-2.6,2-3,2.5c-1.7,1.7-3.5,3.4-5.2,5.1v-13.7
+						c0-1.2-0.8-2.2-2-2.2h-0.3c-1.3,0-2,1-2,2.2v29.9c0,1.2,0.8,2.2,2,2.2h0.3c1.3,0,2-1,2-2.2v-5.3l3.4,3.3c1,1,2,2,3,3
+						c0.5,0.5,1.3,1.3,2.1,1.3h0.4c1.1,0,1.8-0.8,2-1.8l0-0.1v-0.5c0-0.4-0.1-0.7-0.3-1c-0.2-0.3-0.5-0.6-0.7-0.8
+						c-0.6-0.7-1.2-1.3-1.9-1.9c-2.3-2.3-4.6-4.6-6.9-6.9l5.3-5.4c1-1.1,2.1-2.1,3.1-3.2c0.5-0.5,1.3-1.4,1.3-2.1V144
+						C159.6,142.9,158.9,142.3,157.8,142.1z"/>
+					<path fill="#3A4D54" d="M138.9,143.9l-0.2-0.1c-1.9-1.3-4.1-2-6.5-2h-0.9c-2.2,0-4.3,0.6-6.2,1.7c-4.1,2.4-6.5,6.2-6.5,11v0.9
+						c0,1.1,0.1,2.2,0.5,3.3c1.9,6.3,6.8,9.9,13.4,9.5c1.9-0.1,6.8-0.7,6.8-3.4v-0.4c0-1.1-0.8-1.7-1.8-1.9l-0.1,0h-0.8l-0.2,0.1
+						c-1.1,0.5-2.7,1.2-3.9,1.2c-1.3,0-2.9-0.1-4.2-0.7c-3.4-1.6-5.4-4.3-5.4-8c0-1.2,0.2-2.4,0.8-3.6c1.6-3.3,4.2-5.3,7.9-5.2
+						c0.7,0,2,0.1,2.6,0.4c0.6,0.3,2.1,1,2.7,1h0.3l0.1,0c1-0.2,1.9-0.8,1.9-1.9v-0.4c0-0.4-0.2-0.8-0.4-1.2L138.9,143.9z"/>
+					<path fill="#3A4D54" d="M85.2,133.7h-0.4c-1.3,0-2,1-2,2.2v9.3c-2.3-2-5.1-3.3-8.3-3.3h-0.9c-2.2,0-4.3,0.6-6.2,1.7
+						c-4.1,2.4-6.5,6.2-6.5,11v0.9c0,2.2,0.6,4.3,1.7,6.2c2.4,4.1,6.2,6.5,11,6.5h0.9c2.2,0,4.3-0.6,6.2-1.7c4.1-2.4,6.5-6.2,6.5-11
+						v-19.6C87.2,134.6,86.5,133.7,85.2,133.7z M81.6,159.3c-1.7,2.9-4.2,4.5-7.6,4.5c-1.4,0-2.7-0.4-3.9-1c-3-1.7-4.7-4.3-4.7-7.7
+						c0-1.2,0.2-2.4,0.8-3.6c1.6-3.3,4.3-5.2,8-5.2c1.8,0,3.4,0.5,4.9,1.6c2.4,1.7,3.8,4.1,3.8,7.1C82.8,156.5,82.4,158,81.6,159.3z
+						"/>
+					<path fill="#3A4D54" d="M103.1,141.9h-0.6c-2.2,0-4.3,0.6-6.2,1.7c-4.1,2.4-6.5,6.2-6.5,11v0.9c0,2.2,0.6,4.3,1.7,6.2
+						c2.4,4.1,6.2,6.5,11,6.5h0.9c2.2,0,4.3-0.6,6.2-1.7c4.1-2.4,6.5-6.2,6.5-11v-0.9c0-2-0.5-4-1.5-5.8
+						C112.1,144.4,108.2,141.9,103.1,141.9z M110.5,159.3c-1.7,2.8-4.2,4.5-7.5,4.5c-1.6,0-3-0.4-4.3-1.2c-2.8-1.7-4.5-4.2-4.5-7.6
+						c0-1.2,0.2-2.4,0.8-3.6c1.6-3.3,4.3-5.2,8-5.2c1.7,0,3.3,0.5,4.7,1.4c2.6,1.7,4.1,4.1,4.1,7.2
+						C111.7,156.5,111.3,158,110.5,159.3z"/>
+					<path fill="#3A4D54" d="M186.4,148c-1.2-2.1-3-3.7-5.2-4.8c-4-2-8.3-2.2-12.2,0.1l-0.6,0.3c-1.6,0.9-3,2.1-4,3.6
+						c-3,4.4-3.4,9.3-0.7,14l0.3,0.5c1.1,2,2.7,3.6,4.6,4.6c4.2,2.3,8.6,2.6,12.8,0.2l0.4-0.2c1.1-0.7,1.4-1.8,0.8-3
+						c-0.2-0.5-0.7-0.8-1.2-1.1l-0.1-0.1l-0.1,0c-0.8-0.1-2.9,0.8-3.8,1.2c-1.6,0.3-3.5,0.4-5.1-0.2c2.9-2.5,5.8-5.1,8.8-7.6
+						c1.3-1.1,2.7-2.4,4.1-3.5c1.2-0.9,2.3-2.2,1.4-3.8L186.4,148z M178.4,152.1c-3.3,2.8-6.5,5.6-9.8,8.4c-0.3-0.4-0.6-0.8-0.9-1.2
+						c-0.7-1.2-1.1-2.5-1.1-3.9c-0.1-3.5,1.2-6.3,4.2-8.1c2.3-1.3,4.8-1.7,7.4-0.7c1.3,0.5,2.7,1.3,3.6,2.4
+						C180.7,150.2,179.5,151.2,178.4,152.1z"/>
+					<path fill="#3A4D54" d="M204.2,142.1h-0.4c-2.6,0-5,0.8-7.1,2.3c-3.5,2.5-5.6,6-5.6,10.4V166c0,1.2,0.8,2.2,2,2.2h0.3
+						c1.3,0,2-1,2-2.2v-10.7c0-2.4,0.7-4.5,2.4-6.2c1.4-1.3,3.3-2.5,5.2-2.5c1.5,0,3.3-0.5,3.3-2.3
+						C206.4,142.9,205.5,142.1,204.2,142.1z"/>
+				</g>
+				<g>
+					<path fill-rule="evenodd" clip-rule="evenodd" fill="#2BB8EB" d="M281.3,146.6c-0.7-0.3-1.9-0.4-2.6-0.4
+						c-3.7-0.1-6.4,1.9-7.9,5.2c-0.5,1.1-0.8,2.3-0.8,3.6c0,3.8,2,6.4,5.4,8c1.2,0.6,2.8,0.7,4.2,0.7c1.2,0,2.9-0.7,3.9-1.2l0.2-0.1
+						h0.8l0.1,0c1,0.2,1.8,0.8,1.8,1.9v0.4c0,2.7-4.9,3.3-6.8,3.4c-6.6,0.5-11.6-3.2-13.4-9.5c-0.3-1.1-0.5-2.2-0.5-3.3v-0.9
+						c0-4.8,2.4-8.6,6.5-11c1.9-1.1,4-1.7,6.2-1.7h0.9c2.4,0,4.5,0.7,6.5,2l0.2,0.1l0.1,0.2c0.2,0.3,0.4,0.7,0.4,1.2v0.4
+						c0,1.1-0.8,1.7-1.9,1.9l-0.1,0H284C283.4,147.6,281.9,146.9,281.3,146.6z"/>
+					<path fill-rule="evenodd" clip-rule="evenodd" fill="#2BB8EB" d="M301.3,141.9h0.6c5.1,0,9,2.5,11.5,6.9c1,1.8,1.5,3.7,1.5,5.8
+						v0.9c0,4.8-2.4,8.6-6.5,11c-1.9,1.1-4,1.7-6.2,1.7h-0.9c-4.8,0-8.6-2.4-11-6.5c-1.1-1.9-1.7-4-1.7-6.2v-0.9
+						c0-4.8,2.4-8.6,6.5-11C297,142.4,299.1,141.9,301.3,141.9z M293,155c0,3.4,1.6,5.8,4.5,7.6c1.3,0.8,2.8,1.2,4.3,1.2
+						c3.3,0,5.8-1.7,7.5-4.5c0.8-1.3,1.2-2.8,1.2-4.4c0-3.1-1.5-5.5-4.1-7.2c-1.4-0.9-3-1.4-4.7-1.4c-3.7,0-6.4,1.9-8,5.2
+						C293.3,152.6,293,153.8,293,155z"/>
+					<path fill="#2BB8EB" d="M344,148.8c-2.5-4.5-6.4-6.9-11.5-6.9h-0.6c-2.2,0-4.3,0.6-6.2,1.7c-4.1,2.4-6.5,6.2-6.5,11v0.3v11
+						c0,1.2,0.8,2.2,2,2.2h0.3c1.3,0,2-1,2-2.2v-11h0c0-1.2,0.3-2.4,0.8-3.5c1.6-3.3,4.3-5.2,8-5.2c1.7,0,3.3,0.5,4.7,1.4
+						c2.6,1.7,4.1,4.1,4.1,7.2v11c0,1.2,0.8,2.2,2,2.2h0.3c1.3,0,2-1,2-2.2v-11v-0.3C345.5,152.6,345,150.6,344,148.8z"/>
+				</g>
+			</g>
+			<path fill="none" stroke="#3A4D54" stroke-width="5" stroke-miterlimit="10" d="M402.5,190H42.1c-3.6,0-6.5-2.9-6.5-6.5v-52.9
+				c0-3.6,2.9-6.5,6.5-6.5h360.4c3.6,0,6.5,2.9,6.5,6.5v52.9C409,187.1,406.1,190,402.5,190z"/>
+		</g>
+		<polygon fill="#E8593A" points="147.8,203 133.3,201.1 122.8,201.1 122.8,214.6 148.3,214.6 		"/>
+		<rect x="353.6" y="124.2" fill="#3A4D54" width="5.1" height="55.2"/>
+	</g>
+	<g>
+		<path fill="#3A4D54" d="M91.8,293.4H20.2c-3.2,0-5.8-2.6-5.8-5.9s2.6-5.9,5.8-5.9h71.6c3.2,0,5.8,2.6,5.8,5.9S95,293.4,91.8,293.4
+			z"/>
+	</g>
+	<path fill="#3A4D54" d="M428.9,282.7h-83c-3.2,0-5.8,2.6-5.8,5.9c0,3.2,2.6,5.9,5.8,5.9h-54.7c-3.2,0-5.8,2.6-5.8,5.9
+		c0,3.2,2.6,5.9,5.8,5.9H308c-3.2,0-5.8,2.6-5.8,5.9c0,3.2,2.6,5.9,5.8,5.9h-28.9c-3.2,0-5.8,2.6-5.8,5.9c0,3.2,2.6,5.9,5.8,5.9H262
+		c-3.2,0-5.8,2.6-5.8,5.9s2.6,5.9,5.8,5.9h13.7c-3.2,0-5.8,2.6-5.8,5.9s2.6,5.9,5.8,5.9h-37.8c-3.2,0-5.8,2.6-5.8,5.9
+		c0,3,2.2,5.5,5.1,5.8h-48.8c-0.9-0.6-2-1-3.2-1h-47.1c3.2,0,5.8,2.6,5.8,5.9c0,3.2-2.6,5.9-5.8,5.9h-2.8c-3.2,0-5.8,2.9-5.8,6.4
+		c0,3.5,2.6,6.4,5.8,6.4h58.5h7.5H286c3.2,0,5.8-2.6,5.8-5.9c0-3.2-2.6-5.9-5.8-5.9H286h-2.7c-3.2,0-5.8-2.6-5.8-5.9
+		c0-3.2,2.6-5.9,5.8-5.9h66c0.2,0,0.4,0,0.6,0h6.7c3.2,0,5.8-2.6,5.8-5.9c0-3.2-2.6-5.9-5.8-5.9h-27.2c0,0,0,0,0,0h-0.7
+		c-3.2,0-5.8-2.6-5.8-5.9c0-3.2,2.6-5.9,5.8-5.9h0.7h14.1c3.2,0,5.8-2.6,5.8-5.9s-2.6-5.9-5.8-5.9h0.2c-3.2,0-5.8-2.6-5.8-5.9
+		c0-3.2,2.6-5.9,5.8-5.9h0.7h28.9c3.2,0,5.8-2.6,5.8-5.9c0-3.2-2.6-5.9-5.8-5.9h-16.1h-0.8c0.1,0,0.3,0,0.4,0
+		c-3-0.2-5.4-2.7-5.4-5.9c0-3.1,2.4-5.7,5.4-5.9c-0.1,0-0.3,0-0.4,0h0.8h65.2h6.5c3.2,0,5.8-2.6,5.8-5.9
+		C434.6,285.3,432.1,282.7,428.9,282.7z"/>
+	<g>
+		<path id="outline_3_" fill-rule="evenodd" clip-rule="evenodd" fill="#3A4D54" d="M258,210.8h37v37.8h18.7
+			c8.6,0,17.5-1.5,25.7-4.3c4-1.4,8.5-3.3,12.5-5.6c-5.2-6.8-7.9-15.4-8.7-23.9c-1.1-11.5,1.3-26.5,9.1-35.6l3.9-4.5l4.6,3.7
+			c11.7,9.4,21.5,22.5,23.2,37.4c14-4.1,30.5-3.2,42.9,4l5.1,2.9l-2.7,5.2c-10.5,20.4-32.3,26.7-53.7,25.6
+			C343.5,333.3,273.8,371,189.4,371c-43.6,0-83.7-16.3-106.5-55l-0.4-0.6l-3.3-6.8c-7.7-17-10.3-35.7-8.5-54.4l0.5-5.6h31.6v-37.8
+			h37v-37h73.9v-37H258V210.8z"/>
+		<g id="body_colors_3_">
+			<path fill="#08AADA" d="M377.8,224.8c2.5-19.3-11.9-34.4-20.9-41.6c-10.3,11.9-11.9,43.1,4.3,56.3c-9,8-28,15.3-47.5,15.3H76.8
+				c-1.9,20.3,1.7,39,9.8,55l2.7,4.9c1.7,2.9,3.6,5.7,5.6,8.4h0c9.7,0.6,18.7,0.8,26.9,0.7c0,0,0,0,0,0c16.1-0.4,29.3-2.3,39.3-5.7
+				c1.5-0.5,3.1,0.3,3.6,1.8c0.5,1.5-0.3,3.1-1.8,3.6c-1.3,0.5-2.7,0.9-4.1,1.3c0,0,0,0,0,0c-7.9,2.2-16.3,3.8-27.2,4.4
+				c0.6,0-0.7,0.1-0.7,0.1c-0.4,0-0.8,0.1-1.2,0.1c-4.3,0.2-8.9,0.3-13.6,0.3c-5.2,0-10.3-0.1-15.9-0.4l-0.1,0.1
+				c19.7,22.2,50.6,35.5,89.3,35.5c81.9,0,151.3-36.3,182.1-117.8c21.8,2.2,42.8-3.3,52.3-21.9C408.6,216.4,389,219.2,377.8,224.8z"
+				/>
+			<path fill="#2BB8EB" d="M377.8,224.8c2.5-19.3-11.9-34.4-20.9-41.6c-10.3,11.9-11.9,43.1,4.3,56.3c-9,8-28,15.3-47.5,15.3H90.8
+				c-1,31.1,10.6,54.7,31,69c0,0,0,0,0,0c16.1-0.4,29.3-2.3,39.3-5.7c1.5-0.5,3.1,0.3,3.6,1.8c0.5,1.5-0.3,3.1-1.8,3.6
+				c-1.3,0.5-2.7,0.9-4.1,1.3c0,0,0,0,0,0c-7.9,2.2-17,3.9-27.9,4.6c0,0-0.3-0.3-0.3-0.3c27.9,14.3,68.3,14.2,114.6-3.6
+				c51.9-20,100.3-58,134-101.5C378.8,224.3,378.3,224.6,377.8,224.8z"/>
+			<path fill="#088CB9" d="M76.6,279.5c1.5,10.9,4.7,21.1,9.4,30.4l2.7,4.9c1.7,2.9,3.6,5.7,5.6,8.4c9.7,0.6,18.7,0.8,26.9,0.7
+				c16.1-0.4,29.3-2.3,39.3-5.7c1.5-0.5,3.1,0.3,3.6,1.8c0.5,1.5-0.3,3.1-1.8,3.6c-1.3,0.5-2.7,0.9-4.1,1.3c0,0,0,0,0,0
+				c-7.9,2.2-17,3.9-27.8,4.5c-0.4,0-1,0-1.4,0c-4.3,0.2-8.9,0.4-13.6,0.4c-5.2,0-10.4-0.1-16.1-0.4c19.7,22.2,50.8,35.5,89.5,35.5
+				c70.1,0,131.1-26.6,166.5-85.4H76.6z"/>
+			<path fill="#069BC6" d="M92.9,279.5c4.2,19.1,14.3,34.1,28.9,44.3c16.1-0.4,29.3-2.3,39.3-5.7c1.5-0.5,3.1,0.3,3.6,1.8
+				c0.5,1.5-0.3,3.1-1.8,3.6c-1.3,0.5-2.7,0.9-4.1,1.3c0,0,0,0,0,0c-7.9,2.2-17.2,3.9-28,4.5c27.9,14.3,68.2,14.1,114.5-3.7
+				c28-10.8,55-26.8,79.2-46.1H92.9z"/>
+		</g>
+		<g id="Containers_3_">
+			<path fill-rule="evenodd" clip-rule="evenodd" fill="#00ACD2" d="M135.8,219.7h2.5v26.7h-2.5V219.7z M130.9,219.7h2.6v26.7h-2.6
+				V219.7z M126.1,219.7h2.6v26.7h-2.6V219.7z M121.2,219.7h2.6v26.7h-2.6V219.7z M116.3,219.7h2.6v26.7h-2.6V219.7z M111.6,219.7
+				h2.5v26.7h-2.5V219.7z M108.9,217h32v32h-32V217z"/>
+			<path fill-rule="evenodd" clip-rule="evenodd" fill="#00ACD2" d="M172.7,182.7h2.5v26.7h-2.5V182.7z M167.9,182.7h2.6v26.7h-2.6
+				V182.7z M163,182.7h2.6v26.7H163V182.7z M158.2,182.7h2.6v26.7h-2.6V182.7z M153.3,182.7h2.6v26.7h-2.6V182.7z M148.6,182.7h2.5
+				v26.7h-2.5V182.7z M145.9,180h32v32h-32V180z"/>
+			<path fill-rule="evenodd" clip-rule="evenodd" fill="#26C2EE" d="M172.7,219.7h2.5v26.7h-2.5V219.7z M167.9,219.7h2.6v26.7h-2.6
+				V219.7z M163,219.7h2.6v26.7H163V219.7z M158.2,219.7h2.6v26.7h-2.6V219.7z M153.3,219.7h2.6v26.7h-2.6V219.7z M148.6,219.7h2.5
+				v26.7h-2.5V219.7z M145.9,217h32v32h-32V217z"/>
+			<path fill-rule="evenodd" clip-rule="evenodd" fill="#00ACD2" d="M209.7,219.7h2.5v26.7h-2.5V219.7z M204.8,219.7h2.6v26.7h-2.6
+				V219.7z M200,219.7h2.6v26.7H200V219.7z M195.1,219.7h2.6v26.7h-2.6V219.7z M190.3,219.7h2.6v26.7h-2.6V219.7z M185.5,219.7h2.5
+				v26.7h-2.5V219.7z M182.9,217h32v32h-32V217z"/>
+			<path fill-rule="evenodd" clip-rule="evenodd" fill="#26C2EE" d="M209.7,182.7h2.5v26.7h-2.5V182.7z M204.8,182.7h2.6v26.7h-2.6
+				V182.7z M200,182.7h2.6v26.7H200V182.7z M195.1,182.7h2.6v26.7h-2.6V182.7z M190.3,182.7h2.6v26.7h-2.6V182.7z M185.5,182.7h2.5
+				v26.7h-2.5V182.7z M182.9,180h32v32h-32V180z"/>
+			<path fill-rule="evenodd" clip-rule="evenodd" fill="#26C2EE" d="M246.7,219.7h2.5v26.7h-2.5V219.7z M241.8,219.7h2.6v26.7h-2.6
+				V219.7z M237,219.7h2.6v26.7H237V219.7z M232.1,219.7h2.6v26.7h-2.6V219.7z M227.3,219.7h2.6v26.7h-2.6V219.7z M222.5,219.7h2.5
+				v26.7h-2.5V219.7z M219.8,217h32v32h-32V217z"/>
+			<path fill-rule="evenodd" clip-rule="evenodd" fill="#00ACD2" d="M246.7,182.7h2.5v26.7h-2.5V182.7z M241.8,182.7h2.6v26.7h-2.6
+				V182.7z M237,182.7h2.6v26.7H237V182.7z M232.1,182.7h2.6v26.7h-2.6V182.7z M227.3,182.7h2.6v26.7h-2.6V182.7z M222.5,182.7h2.5
+				v26.7h-2.5V182.7z M219.8,180h32v32h-32V180z"/>
+			<path fill-rule="evenodd" clip-rule="evenodd" fill="#26C2EE" d="M246.7,145.7h2.5v26.7h-2.5V145.7z M241.8,145.7h2.6v26.7h-2.6
+				V145.7z M237,145.7h2.6v26.7H237V145.7z M232.1,145.7h2.6v26.7h-2.6V145.7z M227.3,145.7h2.6v26.7h-2.6V145.7z M222.5,145.7h2.5
+				v26.7h-2.5V145.7z M219.8,143.1h32v32h-32V143.1z"/>
+			<path fill-rule="evenodd" clip-rule="evenodd" fill="#00ACD2" d="M283.6,219.7h2.5v26.7h-2.5V219.7z M278.8,219.7h2.6v26.7h-2.6
+				V219.7z M273.9,219.7h2.6v26.7h-2.6V219.7z M269.1,219.7h2.6v26.7h-2.6V219.7z M264.2,219.7h2.6v26.7h-2.6V219.7z M259.5,219.7
+				h2.5v26.7h-2.5V219.7z M256.8,217h32v32h-32V217z"/>
+		</g>
+		<path fill-rule="evenodd" clip-rule="evenodd" fill="#D4EDF1" d="M175.9,301c4.9,0,8.8,4,8.8,8.8s-4,8.8-8.8,8.8
+			c-4.9,0-8.8-4-8.8-8.8S171,301,175.9,301"/>
+		<path fill-rule="evenodd" clip-rule="evenodd" fill="#3A4D54" d="M175.9,303.5c0.8,0,1.6,0.2,2.3,0.4c-0.8,0.4-1.3,1.3-1.3,2.2
+			c0,1.4,1.2,2.6,2.6,2.6c1,0,1.8-0.5,2.3-1.3c0.3,0.7,0.5,1.6,0.5,2.4c0,3.5-2.8,6.3-6.3,6.3c-3.5,0-6.3-2.8-6.3-6.3
+			C169.6,306.3,172.4,303.5,175.9,303.5"/>
+		<path fill-rule="evenodd" clip-rule="evenodd" fill="#3A4D54" d="M19.6,282.7h193.6h23.9h190.5c0.4,0,1.6,0.1,1.2,0
+			c-9.2-2.2-24.9-6.2-23.5-15.8c0.1-0.7-0.2-0.8-0.6-0.3c-16.6,17.5-54.1,12.2-64.3,3.2c-0.2-0.1-0.4-0.1-0.5,0.1
+			c-11.5,15.4-73.3,9.7-79.3-2.3c-0.1-0.2-0.4-0.3-0.6-0.1c-14.1,15.7-55.7,15.7-69.8,0c-0.2-0.2-0.5-0.1-0.6,0.1
+			c-6,12-67.8,17.7-79.3,2.3c-0.1-0.2-0.3-0.2-0.5-0.1c-10.1,8.9-44.5,14.3-61.2-3c-0.3-0.3-0.8-0.1-0.8,0.4
+			C48.9,277.6,28.1,280.5,19.6,282.7"/>
+		<path fill="#C0DBE0" d="M199.4,364.7c-21.9-10.4-33.9-24.5-40.6-39.9c-8.1,2.3-17.9,3.8-29.3,4.4c-4.3,0.2-8.8,0.4-13.5,0.4
+			c-5.4,0-11.2-0.2-17.2-0.5c20.1,20.1,44.8,35.5,90.5,35.8C192.7,364.9,196.1,364.8,199.4,364.7z"/>
+		<path fill="#D4EDF1" d="M167,339c-3-4.1-6-9.3-8.1-14.2c-8.1,2.3-17.9,3.8-29.3,4.4C137.4,333.4,148.5,337.4,167,339z"/>
+	</g>
+	<circle fill="#3A4D54" cx="34.8" cy="311" r="5.9"/>
+	<path fill="#3A4D54" d="M346.8,297.2l-1-2.8c0,0,5.3-11.7-7.4-11.7c-12.7,0,3.5-4.7,3.5-4.7l21.8,2.8l9.6,6.8l-16.1,4.1
+		L346.8,297.2z"/>
+	<path fill="#3A4D54" d="M78.7,297.2l1-2.8c0,0-5.3-11.7,7.4-11.7s-3.5-4.7-3.5-4.7l-21.8,2.8l-9.6,6.8l16.1,4.1L78.7,297.2z"/>
+	<path fill="#3A4D54" d="M361.7,279.5v4.4l15.6,6.7l45.5-4.1l7.3-3.7c0,0-3.8-0.6-7.3-1.7c-3.6-1.1-15.2-1.6-15.2-1.6h-28.3
+		l-13.6,1.8L361.7,279.5z"/>
+</g>
+</svg>