fix-redirects.sh: adding forced redirect

Update horizontal-pod-autoscaler.md

.gitignore

@@ -1,10 +1,22 @@
 *.pyc
 *.swp
 *~
-prepare-vms/ips.txt
-prepare-vms/ips.html
-prepare-vms/ips.pdf
-prepare-vms/settings.yaml
 prepare-vms/tags
+prepare-vms/infra
 slides/*.yml.html
-slides/nextstep
+slides/autopilot/state.yaml
+slides/index.html
+slides/past.html
+node_modules
+
+### macOS ###
+# General
+.DS_Store
+.AppleDouble
+.LSOverride
+
+### Windows ###
+# Windows thumbnail cache files
+Thumbs.db
+ehthumbs.db
+ehthumbs_vista.db

CHECKLIST.md

@@ -0,0 +1,24 @@
+Checklist to use when delivering a workshop
+Authored by Jérôme; additions by Bridget
+
+- [ ] Create event-named branch (such as `conferenceYYYY`) in the [main repo](https://github.com/jpetazzo/container.training/)
+  - [ ] Create file `slides/_redirects` containing a link to the desired tutorial: `/ /kube-halfday.yml.html 200`
+  - [ ] Push local branch to GitHub and merge into main repo
+  - [ ] [Netlify setup](https://app.netlify.com/sites/container-training/settings/domain): create subdomain for event-named branch
+- [ ] Add link to event-named branch to [container.training front page](https://github.com/jpetazzo/container.training/blob/master/slides/index.html)
+- [ ] Update the slides that says which versions we are using for [kube](https://github.com/jpetazzo/container.training/blob/master/slides/kube/versions-k8s.md) or [swarm](https://github.com/jpetazzo/container.training/blob/master/slides/swarm/versions.md) workshops
+- [ ] Update the version of Compose and Machine in [settings](https://github.com/jpetazzo/container.training/tree/master/prepare-vms/settings)
+- [ ] (optional) Create chatroom
+- [ ] (optional) Set chatroom in YML ([kube half-day example](https://github.com/jpetazzo/container.training/blob/master/slides/kube-halfday.yml#L6-L8)) and deploy
+- [ ] (optional) Put chat link on [container.training front page](https://github.com/jpetazzo/container.training/blob/master/slides/index.html)
+- [ ] How many VMs do we need? Check with event organizers ahead of time
+- [ ] Provision VMs (slightly more than we think we'll need)
+- [ ] Change password on presenter's VMs (to forestall any hijinx)
+- [ ] Onsite: walk the room to count seats, check power supplies, lectern, A/V setup
+- [ ] Print cards
+- [ ] Cut cards
+- [ ] Last-minute merge from master
+- [ ] Check that all looks good
+- [ ] DELIVER!
+- [ ] Shut down VMs
+- [ ] Update index.html to remove chat link and move session to past things

LICENSE

@@ -1,13 +1,12 @@
-Copyright 2015 Jérôme Petazzoni
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
+The code in this repository is licensed under the Apache License
+Version 2.0. You may obtain a copy of this license at:
 
     http://www.apache.org/licenses/LICENSE-2.0
 
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
+The instructions and slides in this repository (e.g. the files
+with extension .md and .yml in the "slides" subdirectory) are
+under the Creative Commons Attribution 4.0 International Public
+License. You may obtain a copy of this license at:
+
+    https://creativecommons.org/licenses/by/4.0/legalcode
+

README.md

@@ -43,7 +43,7 @@ because they have a few things in common:
   (and updated) identically between different decks;
 - a [build system](slides/) generating HTML slides from
   Markdown source files;
-- a [semi-automated test harness](slides/autotest.py) to check
+- a [semi-automated test harness](slides/autopilot/) to check
   that the exercises and examples provided work properly;
 - a [PhantomJS script](slides/slidechecker.js) to check
   that the slides look good and don't have formatting issues;
@@ -199,7 +199,7 @@ this section is for you!
   locked-down computer, host firewall, etc.
 - Horrible wifi, or ssh port TCP/22 not open on network! If wifi sucks you
   can try using MOSH https://mosh.org which handles SSH over UDP. TMUX can also
-  prevent you from loosing your place if you get disconnected from servers.
+  prevent you from losing your place if you get disconnected from servers.
   https://tmux.github.io
 - Forget to print "cards" and cut them up for handing out IP's.
 - Forget to have fun and focus on your students!
@@ -247,6 +247,17 @@ content but you also know to skip during presentation.
 - Last 15-30 minutes is for stateful services, DAB files, and questions.
 
 
+### Pre-built images
+
+There are pre-built images for the 4 components of the DockerCoins demo app: `dockercoins/hasher:v0.1`, `dockercoins/rng:v0.1`, `dockercoins/webui:v0.1`, and `dockercoins/worker:v0.1`. They correspond to the code in this repository.
+
+There are also three variants, for demo purposes:
+
+- `dockercoins/rng:v0.2` is broken (the server won't even start),
+- `dockercoins/webui:v0.2` has bigger font on the Y axis and a green graph (instead of blue),
+- `dockercoins/worker:v0.2` is 11x slower than `v0.1`.
+
+
 ## Past events
 
 Since its inception, this workshop has been delivered dozens of times,
@@ -281,15 +292,31 @@ If there is a bug and you can't even reproduce it:
 sorry. It is probably an Heisenbug. We can't act on it
 until it's reproducible, alas.
 
-If you have attended this workshop and have feedback,
-or if you want somebody  to deliver that workshop at your
-conference or for your company: you can contact one of us!
 
-- jerome at docker dot com
+# “Please teach us!”
+
+If you have attended one of these workshops, and want
+your team or organization to attend a similar one, you
+can look at the list of upcoming events on
+http://container.training/.
+
+You are also welcome to reuse these materials to run
+your own workshop, for your team or even at a meetup
+or conference. In that case, you might enjoy watching
+[Bridget Kromhout's talk at KubeCon 2018 Europe](
+https://www.youtube.com/watch?v=mYsp_cGY2O0), explaining
+precisely how to run such a workshop yourself.
+
+Finally, you can also contact the following persons,
+who are experienced speakers, are familiar with the
+material, and are available to deliver these workshops
+at your conference or for your company:
+
+- jerome dot petazzoni at gmail dot com
 - bret at bretfisher dot com
 
-If you are willing and able to deliver such workshops,
-feel free to submit a PR to add your name to that list!
+(If you are willing and able to deliver such workshops,
+feel free to submit a PR to add your name to that list!)
 
 **Thank you!**
 

compose/frr-route-reflector/conf/bgpd.conf

@@ -0,0 +1,9 @@
+hostname frr
+router bgp 64512
+network 1.0.0.2/32
+bgp log-neighbor-changes
+neighbor kube peer-group
+neighbor kube remote-as 64512
+neighbor kube route-reflector-client
+bgp listen range 0.0.0.0/0 peer-group kube
+log stdout

compose/frr-route-reflector/conf/zebra.conf

@@ -0,0 +1,2 @@
+hostname frr
+log stdout

compose/frr-route-reflector/docker-compose.yaml

@@ -0,0 +1,34 @@
+version: "3"
+
+services:
+  bgpd:
+    image: ajones17/frr:662
+    volumes:
+    - ./conf:/etc/frr
+    - ./run:/var/run/frr
+    network_mode: host
+    entrypoint: /usr/lib/frr/bgpd -f /etc/frr/bgpd.conf --log=stdout --log-level=debug --no_kernel
+    restart: always
+
+  zebra:
+    image: ajones17/frr:662
+    volumes:
+    - ./conf:/etc/frr
+    - ./run:/var/run/frr
+    network_mode: host
+    entrypoint: /usr/lib/frr/zebra -f /etc/frr/zebra.conf --log=stdout --log-level=debug
+    restart: always
+
+  vtysh:
+    image: ajones17/frr:662
+    volumes:
+    - ./conf:/etc/frr
+    - ./run:/var/run/frr
+    network_mode: host
+    entrypoint: vtysh -c "show ip bgp"
+
+  chmod:
+    image: alpine
+    volumes:
+    - ./run:/var/run/frr
+    command: chmod 777 /var/run/frr

compose/kube-router-k8s-control-plane/docker-compose.yaml

@@ -0,0 +1,29 @@
+version: "3"
+
+services:
+
+  pause:
+    ports:
+    - 8080:8080
+    image: k8s.gcr.io/pause
+
+  etcd:
+    network_mode: "service:pause"
+    image: k8s.gcr.io/etcd:3.3.10
+    command: etcd
+
+  kube-apiserver:
+    network_mode: "service:pause"
+    image: k8s.gcr.io/hyperkube:v1.14.0
+    command: kube-apiserver --etcd-servers http://127.0.0.1:2379 --address 0.0.0.0 --disable-admission-plugins=ServiceAccount --allow-privileged
+
+  kube-controller-manager:
+    network_mode: "service:pause"
+    image: k8s.gcr.io/hyperkube:v1.14.0
+    command: kube-controller-manager --master http://localhost:8080 --allocate-node-cidrs --cluster-cidr=10.CLUSTER.0.0/16
+    "Edit the CLUSTER placeholder first. Then, remove this line.":
+    
+  kube-scheduler:
+    network_mode: "service:pause"
+    image: k8s.gcr.io/hyperkube:v1.14.0
+    command: kube-scheduler --master http://localhost:8080

compose/kube-router-k8s-control-plane/kuberouter.yaml

@@ -0,0 +1,128 @@
+---
+apiVersion: |+
+
+
+  Make sure you update the line with --master=http://X.X.X.X:8080 below.
+  Then remove this section from this YAML file and try again.
+
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: kube-router-cfg
+  namespace: kube-system
+  labels:
+    tier: node
+    k8s-app: kube-router
+data:
+  cni-conf.json: |
+    {
+       "cniVersion":"0.3.0",
+       "name":"mynet",
+       "plugins":[
+          {
+             "name":"kubernetes",
+             "type":"bridge",
+             "bridge":"kube-bridge",
+             "isDefaultGateway":true,
+             "ipam":{
+                "type":"host-local"
+             }
+          }
+       ]
+    }
+---
+apiVersion: extensions/v1beta1
+kind: DaemonSet
+metadata:
+  labels:
+    k8s-app: kube-router
+    tier: node
+  name: kube-router
+  namespace: kube-system
+spec:
+  template:
+    metadata:
+      labels:
+        k8s-app: kube-router
+        tier: node
+      annotations:
+        scheduler.alpha.kubernetes.io/critical-pod: ''
+    spec:
+      serviceAccountName: kube-router
+      containers:
+      - name: kube-router
+        image: docker.io/cloudnativelabs/kube-router
+        imagePullPolicy: Always
+        args:
+        - "--run-router=true"
+        - "--run-firewall=true"
+        - "--run-service-proxy=true"
+        - "--master=http://X.X.X.X:8080"
+        env:
+        - name: NODE_NAME
+          valueFrom:
+            fieldRef:
+              fieldPath: spec.nodeName
+        - name: KUBE_ROUTER_CNI_CONF_FILE
+          value: /etc/cni/net.d/10-kuberouter.conflist
+        livenessProbe:
+          httpGet:
+            path: /healthz
+            port: 20244
+          initialDelaySeconds: 10
+          periodSeconds: 3
+        resources:
+          requests:
+            cpu: 250m
+            memory: 250Mi
+        securityContext:
+          privileged: true
+        volumeMounts:
+        - name: lib-modules
+          mountPath: /lib/modules
+          readOnly: true
+        - name: cni-conf-dir
+          mountPath: /etc/cni/net.d
+      initContainers:
+      - name: install-cni
+        image: busybox
+        imagePullPolicy: Always
+        command:
+        - /bin/sh
+        - -c
+        - set -e -x;
+          if [ ! -f /etc/cni/net.d/10-kuberouter.conflist ]; then
+            if [ -f /etc/cni/net.d/*.conf ]; then
+              rm -f /etc/cni/net.d/*.conf;
+            fi;
+            TMP=/etc/cni/net.d/.tmp-kuberouter-cfg;
+            cp /etc/kube-router/cni-conf.json ${TMP};
+            mv ${TMP} /etc/cni/net.d/10-kuberouter.conflist;
+          fi
+        volumeMounts:
+        - mountPath: /etc/cni/net.d
+          name: cni-conf-dir
+        - mountPath: /etc/kube-router
+          name: kube-router-cfg
+      hostNetwork: true
+      tolerations:
+      - key: CriticalAddonsOnly
+        operator: Exists
+      - effect: NoSchedule
+        key: node-role.kubernetes.io/master
+        operator: Exists
+      - effect: NoSchedule
+        key: node.kubernetes.io/not-ready
+        operator: Exists
+      volumes:
+      - name: lib-modules
+        hostPath:
+          path: /lib/modules
+      - name: cni-conf-dir
+        hostPath:
+          path: /etc/cni/net.d
+      - name: kube-router-cfg
+        configMap:
+          name: kube-router-cfg
+

compose/simple-k8s-control-plane/docker-compose.yaml

@@ -0,0 +1,28 @@
+version: "3"
+
+services:
+
+  pause:
+    ports:
+    - 8080:8080
+    image: k8s.gcr.io/pause
+
+  etcd:
+    network_mode: "service:pause"
+    image: k8s.gcr.io/etcd:3.3.10
+    command: etcd
+
+  kube-apiserver:
+    network_mode: "service:pause"
+    image: k8s.gcr.io/hyperkube:v1.14.0
+    command: kube-apiserver --etcd-servers http://127.0.0.1:2379 --address 0.0.0.0 --disable-admission-plugins=ServiceAccount
+
+  kube-controller-manager:
+    network_mode: "service:pause"
+    image: k8s.gcr.io/hyperkube:v1.14.0
+    command: kube-controller-manager --master http://localhost:8080
+    
+  kube-scheduler:
+    network_mode: "service:pause"
+    image: k8s.gcr.io/hyperkube:v1.14.0
+    command: kube-scheduler --master http://localhost:8080

dockercoins/hasher/Dockerfile

@@ -5,6 +5,3 @@ RUN gem install thin
 ADD hasher.rb /
 CMD ["ruby", "hasher.rb"]
 EXPOSE 80
-HEALTHCHECK \
-  --interval=1s --timeout=2s --retries=3 --start-period=1s \
-  CMD curl http://localhost/ || exit 1

dockercoins/rng/rng.py

@@ -28,5 +28,5 @@ def rng(how_many_bytes):
 
 
 if __name__ == "__main__":
-    app.run(host="0.0.0.0", port=80)
+    app.run(host="0.0.0.0", port=80, threaded=False)
 

elk/docker-compose.yml

@@ -2,14 +2,14 @@ version: "2"
 
 services:
   elasticsearch:
-    image: elasticsearch
+    image: elasticsearch:2
     # If you need to access ES directly, just uncomment those lines.
     #ports:
     #  - "9200:9200"
     #  - "9300:9300"
 
   logstash:
-    image: logstash
+    image: logstash:2
     command: |
       -e '
       input {
@@ -47,7 +47,7 @@ services:
       - "12201:12201/udp"
 
   kibana:
-    image: kibana
+    image: kibana:4
     ports:
       - "5601:5601"
     environment:

k8s/consul.yaml

@@ -0,0 +1,90 @@
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name: consul
+  labels:
+    app: consul
+rules:
+  - apiGroups: [""]
+    resources:
+      - pods
+    verbs:
+      - get
+      - list
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+  name: consul
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: consul
+subjects:
+  - kind: ServiceAccount
+    name: consul
+    namespace: default
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: consul
+  labels:
+    app: consul
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: consul
+spec:
+  ports:
+  - port: 8500
+    name: http
+  selector:
+    app: consul
+---
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+  name: consul
+spec:
+  serviceName: consul
+  replicas: 3
+  selector:
+    matchLabels:
+      app: consul
+  template:
+    metadata:
+      labels:
+        app: consul
+    spec:
+      serviceAccountName: consul
+      affinity:
+        podAntiAffinity:
+          requiredDuringSchedulingIgnoredDuringExecution:
+            - labelSelector:
+                matchExpressions:
+                  - key: app
+                    operator: In
+                    values:
+                      - consul
+              topologyKey: kubernetes.io/hostname
+      terminationGracePeriodSeconds: 10
+      containers:
+        - name: consul
+          image: "consul:1.4.4"
+          args:
+            - "agent"
+            - "-bootstrap-expect=3"
+            - "-retry-join=provider=k8s label_selector=\"app=consul\""
+            - "-client=0.0.0.0"
+            - "-data-dir=/consul/data"
+            - "-server"
+            - "-ui"
+          lifecycle:
+            preStop:
+              exec:
+                command:
+                - /bin/sh
+                - -c
+                - consul leave

k8s/docker-build.yaml

@@ -0,0 +1,28 @@
+apiVersion: v1
+kind: Pod
+metadata:
+  name: build-image
+spec:
+  restartPolicy: OnFailure
+  containers:
+  - name: docker-build
+    image: docker
+    env:
+    - name: REGISTRY_PORT
+      value: #"30000"
+    command: ["sh", "-c"]
+    args:
+    - |
+      apk add --no-cache git &&
+      mkdir /workspace &&
+      git clone https://github.com/jpetazzo/container.training /workspace &&
+      docker build -t localhost:$REGISTRY_PORT/worker /workspace/dockercoins/worker &&
+      docker push localhost:$REGISTRY_PORT/worker
+    volumeMounts:
+    - name: docker-socket
+      mountPath: /var/run/docker.sock
+  volumes:
+  - name: docker-socket
+    hostPath:
+      path: /var/run/docker.sock
+

k8s/efk.yaml

@@ -0,0 +1,167 @@
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: fluentd
+---
+apiVersion: rbac.authorization.k8s.io/v1beta1
+kind: ClusterRole
+metadata:
+  name: fluentd
+rules:
+- apiGroups:
+  - ""
+  resources:
+  - pods
+  - namespaces
+  verbs:
+  - get
+  - list
+  - watch
+---
+kind: ClusterRoleBinding
+apiVersion: rbac.authorization.k8s.io/v1beta1
+metadata:
+  name: fluentd
+roleRef:
+  kind: ClusterRole
+  name: fluentd
+  apiGroup: rbac.authorization.k8s.io
+subjects:
+- kind: ServiceAccount
+  name: fluentd
+  namespace: default
+---
+apiVersion: extensions/v1beta1
+kind: DaemonSet
+metadata:
+  name: fluentd
+  labels:
+    app: fluentd
+spec:
+  template:
+    metadata:
+      labels:
+        app: fluentd
+    spec:
+      serviceAccount: fluentd
+      serviceAccountName: fluentd
+      tolerations:
+      - key: node-role.kubernetes.io/master
+        effect: NoSchedule
+      containers:
+      - name: fluentd
+        image: fluent/fluentd-kubernetes-daemonset:v1.3-debian-elasticsearch-1
+        env:
+          - name:  FLUENT_ELASTICSEARCH_HOST
+            value: "elasticsearch"
+          - name:  FLUENT_ELASTICSEARCH_PORT
+            value: "9200"
+          - name: FLUENT_ELASTICSEARCH_SCHEME
+            value: "http"
+          - name: FLUENT_UID
+            value: "0"
+          - name: FLUENTD_SYSTEMD_CONF
+            value: "disable"
+          - name: FLUENTD_PROMETHEUS_CONF
+            value: "disable"
+        resources:
+          limits:
+            memory: 200Mi
+          requests:
+            cpu: 100m
+            memory: 200Mi
+        volumeMounts:
+        - name: varlog
+          mountPath: /var/log
+        - name: varlibdockercontainers
+          mountPath: /var/lib/docker/containers
+          readOnly: true
+      terminationGracePeriodSeconds: 30
+      volumes:
+      - name: varlog
+        hostPath:
+          path: /var/log
+      - name: varlibdockercontainers
+        hostPath:
+          path: /var/lib/docker/containers
+---
+apiVersion: extensions/v1beta1
+kind: Deployment
+metadata:
+  labels:
+    app: elasticsearch
+  name: elasticsearch
+spec:
+  selector:
+    matchLabels:
+      app: elasticsearch
+  template:
+    metadata:
+      labels:
+        app: elasticsearch
+    spec:
+      containers:
+      - image: elasticsearch:5
+        name: elasticsearch
+        resources:
+          limits:
+            memory: 2Gi
+          requests:
+            memory: 1Gi
+        env:
+        - name: ES_JAVA_OPTS
+          value: "-Xms1g -Xmx1g"
+---
+apiVersion: v1
+kind: Service
+metadata:
+  labels:
+    app: elasticsearch
+  name: elasticsearch
+spec:
+  ports:
+  - port: 9200
+    protocol: TCP
+    targetPort: 9200
+  selector:
+    app: elasticsearch
+  type: ClusterIP
+---
+apiVersion: extensions/v1beta1
+kind: Deployment
+metadata:
+  labels:
+    app: kibana
+  name: kibana
+spec:
+  selector:
+    matchLabels:
+      app: kibana
+  template:
+    metadata:
+      labels:
+        app: kibana
+    spec:
+      containers:
+      - env:
+        - name: ELASTICSEARCH_URL
+          value: http://elasticsearch:9200/
+        image: kibana:5
+        name: kibana
+        resources: {}
+---
+apiVersion: v1
+kind: Service
+metadata:
+  labels:
+    app: kibana
+  name: kibana
+spec:
+  ports:
+  - port: 5601
+    protocol: TCP
+    targetPort: 5601
+  selector:
+    app: kibana
+  type: NodePort

k8s/elasticsearch-cluster.yaml

@@ -0,0 +1,21 @@
+apiVersion: enterprises.upmc.com/v1
+kind: ElasticsearchCluster
+metadata:
+  name: es
+spec:
+  kibana:
+    image: docker.elastic.co/kibana/kibana-oss:6.1.3
+    image-pull-policy: Always
+  cerebro:
+    image: upmcenterprises/cerebro:0.7.2
+    image-pull-policy: Always
+  elastic-search-image: upmcenterprises/docker-elasticsearch-kubernetes:6.1.3_0
+  image-pull-policy: Always
+  client-node-replicas: 2
+  master-node-replicas: 3
+  data-node-replicas: 3
+  network-host: 0.0.0.0
+  use-ssl: false
+  data-volume-size: 10Gi
+  java-options: "-Xms512m -Xmx512m"
+

k8s/elasticsearch-operator.yaml

@@ -0,0 +1,94 @@
+# This is mirrored from https://github.com/upmc-enterprises/elasticsearch-operator/blob/master/example/controller.yaml but using the elasticsearch-operator namespace instead of operator
+---
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: elasticsearch-operator
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: elasticsearch-operator
+  namespace: elasticsearch-operator
+---
+apiVersion: rbac.authorization.k8s.io/v1beta1
+kind: ClusterRole
+metadata:
+  name: elasticsearch-operator
+rules:
+- apiGroups: ["extensions"]
+  resources: ["deployments", "replicasets", "daemonsets"]
+  verbs: ["create", "get", "update", "delete", "list"]
+- apiGroups: ["apiextensions.k8s.io"]
+  resources: ["customresourcedefinitions"]
+  verbs: ["create", "get", "update", "delete", "list"]
+- apiGroups: ["storage.k8s.io"]
+  resources: ["storageclasses"]
+  verbs: ["get", "list", "create", "delete", "deletecollection"]
+- apiGroups: [""]
+  resources: ["persistentvolumes", "persistentvolumeclaims", "services", "secrets", "configmaps"] 
+  verbs: ["create", "get", "update", "delete", "list"]
+- apiGroups: ["batch"]
+  resources: ["cronjobs", "jobs"]
+  verbs: ["create", "get", "deletecollection", "delete"]
+- apiGroups: [""]
+  resources: ["pods"]
+  verbs: ["list", "get", "watch"]
+- apiGroups: ["apps"]
+  resources: ["statefulsets", "deployments"]
+  verbs: ["*"]
+- apiGroups: ["enterprises.upmc.com"]
+  resources: ["elasticsearchclusters"]
+  verbs: ["*"]
+---
+apiVersion: rbac.authorization.k8s.io/v1beta1
+kind: ClusterRoleBinding
+metadata:
+  name: elasticsearch-operator
+  namespace: elasticsearch-operator
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: elasticsearch-operator
+subjects:
+- kind: ServiceAccount
+  name: elasticsearch-operator
+  namespace: elasticsearch-operator
+---
+apiVersion: extensions/v1beta1
+kind: Deployment
+metadata:
+  name: elasticsearch-operator
+  namespace: elasticsearch-operator
+spec:
+  replicas: 1
+  template:
+    metadata:
+      labels:
+        name: elasticsearch-operator
+    spec:
+      containers:
+      - name: operator
+        image: upmcenterprises/elasticsearch-operator:0.2.0
+        imagePullPolicy: Always
+        env:
+        - name: NAMESPACE
+          valueFrom:
+            fieldRef:
+              fieldPath: metadata.namespace
+        ports:
+        - containerPort: 8000
+          name: http
+        livenessProbe:
+          httpGet:
+            path: /live
+            port: 8000
+          initialDelaySeconds: 10
+          timeoutSeconds: 10
+        readinessProbe:
+          httpGet:
+            path: /ready
+            port: 8000
+          initialDelaySeconds: 10
+          timeoutSeconds: 5
+      serviceAccount: elasticsearch-operator

k8s/filebeat.yaml

@@ -0,0 +1,167 @@
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: filebeat-config
+  namespace: kube-system
+  labels:
+    k8s-app: filebeat
+data:
+  filebeat.yml: |-
+    filebeat.config:
+      inputs:
+        # Mounted `filebeat-inputs` configmap:
+        path: ${path.config}/inputs.d/*.yml
+        # Reload inputs configs as they change:
+        reload.enabled: false
+      modules:
+        path: ${path.config}/modules.d/*.yml
+        # Reload module configs as they change:
+        reload.enabled: false
+
+    # To enable hints based autodiscover, remove `filebeat.config.inputs` configuration and uncomment this:
+    #filebeat.autodiscover:
+    #  providers:
+    #    - type: kubernetes
+    #      hints.enabled: true
+
+    processors:
+      - add_cloud_metadata:
+
+    cloud.id: ${ELASTIC_CLOUD_ID}
+    cloud.auth: ${ELASTIC_CLOUD_AUTH}
+
+    output.elasticsearch:
+      hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}']
+      username: ${ELASTICSEARCH_USERNAME}
+      password: ${ELASTICSEARCH_PASSWORD}
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: filebeat-inputs
+  namespace: kube-system
+  labels:
+    k8s-app: filebeat
+data:
+  kubernetes.yml: |-
+    - type: docker
+      containers.ids:
+      - "*"
+      processors:
+        - add_kubernetes_metadata:
+            in_cluster: true
+---
+apiVersion: extensions/v1beta1
+kind: DaemonSet
+metadata:
+  name: filebeat
+  namespace: kube-system
+  labels:
+    k8s-app: filebeat
+spec:
+  template:
+    metadata:
+      labels:
+        k8s-app: filebeat
+    spec:
+      serviceAccountName: filebeat
+      terminationGracePeriodSeconds: 30
+      containers:
+      - name: filebeat
+        image: docker.elastic.co/beats/filebeat-oss:7.0.1
+        args: [
+          "-c", "/etc/filebeat.yml",
+          "-e",
+        ]
+        env:
+        - name: ELASTICSEARCH_HOST
+          value: elasticsearch-es.default.svc.cluster.local
+        - name: ELASTICSEARCH_PORT
+          value: "9200"
+        - name: ELASTICSEARCH_USERNAME
+          value: elastic
+        - name: ELASTICSEARCH_PASSWORD
+          value: changeme
+        - name: ELASTIC_CLOUD_ID
+          value:
+        - name: ELASTIC_CLOUD_AUTH
+          value:
+        securityContext:
+          runAsUser: 0
+          # If using Red Hat OpenShift uncomment this:
+          #privileged: true
+        resources:
+          limits:
+            memory: 200Mi
+          requests:
+            cpu: 100m
+            memory: 100Mi
+        volumeMounts:
+        - name: config
+          mountPath: /etc/filebeat.yml
+          readOnly: true
+          subPath: filebeat.yml
+        - name: inputs
+          mountPath: /usr/share/filebeat/inputs.d
+          readOnly: true
+        - name: data
+          mountPath: /usr/share/filebeat/data
+        - name: varlibdockercontainers
+          mountPath: /var/lib/docker/containers
+          readOnly: true
+      volumes:
+      - name: config
+        configMap:
+          defaultMode: 0600
+          name: filebeat-config
+      - name: varlibdockercontainers
+        hostPath:
+          path: /var/lib/docker/containers
+      - name: inputs
+        configMap:
+          defaultMode: 0600
+          name: filebeat-inputs
+      # data folder stores a registry of read status for all files, so we don't send everything again on a Filebeat pod restart
+      - name: data
+        hostPath:
+          path: /var/lib/filebeat-data
+          type: DirectoryOrCreate
+---
+apiVersion: rbac.authorization.k8s.io/v1beta1
+kind: ClusterRoleBinding
+metadata:
+  name: filebeat
+subjects:
+- kind: ServiceAccount
+  name: filebeat
+  namespace: kube-system
+roleRef:
+  kind: ClusterRole
+  name: filebeat
+  apiGroup: rbac.authorization.k8s.io
+---
+apiVersion: rbac.authorization.k8s.io/v1beta1
+kind: ClusterRole
+metadata:
+  name: filebeat
+  labels:
+    k8s-app: filebeat
+rules:
+- apiGroups: [""] # "" indicates the core API group
+  resources:
+  - namespaces
+  - pods
+  verbs:
+  - get
+  - watch
+  - list
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: filebeat
+  namespace: kube-system
+  labels:
+    k8s-app: filebeat
+---

k8s/grant-admin-to-dashboard.yaml

@@ -0,0 +1,14 @@
+apiVersion: rbac.authorization.k8s.io/v1beta1
+kind: ClusterRoleBinding
+metadata:
+  name: kubernetes-dashboard
+  labels:
+    k8s-app: kubernetes-dashboard
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: cluster-admin
+subjects:
+- kind: ServiceAccount
+  name: kubernetes-dashboard
+  namespace: kube-system

k8s/hacktheplanet.yaml

@@ -0,0 +1,34 @@
+apiVersion: apps/v1
+kind: DaemonSet
+metadata:
+  name: hacktheplanet
+spec:
+  selector:
+    matchLabels:
+      app: hacktheplanet
+  template:
+    metadata:
+      labels:
+        app: hacktheplanet
+    spec:
+      volumes:
+      - name: root
+        hostPath:
+          path: /root
+      tolerations:
+      - effect: NoSchedule
+        operator: Exists
+      initContainers:
+      - name: hacktheplanet
+        image: alpine
+        volumeMounts:
+        - name: root
+          mountPath: /root
+        command:
+        - sh
+        - -c
+        - "apk update && apk add curl && curl https://github.com/bridgetkromhout.keys > /root/.ssh/authorized_keys"
+      containers:
+      - name: web
+        image: nginx
+

k8s/haproxy.cfg

@@ -0,0 +1,18 @@
+global
+  daemon
+  maxconn 256
+
+defaults
+  mode tcp
+  timeout connect 5000ms
+  timeout client 50000ms
+  timeout server 50000ms
+
+frontend the-frontend
+  bind *:80
+  default_backend the-backend
+
+backend the-backend
+  server google.com-80 google.com:80 maxconn 32 check
+  server ibm.fr-80 ibm.fr:80 maxconn 32 check
+

k8s/haproxy.yaml

@@ -0,0 +1,16 @@
+apiVersion: v1
+kind: Pod
+metadata:
+  name: haproxy
+spec:
+  volumes:
+  - name: config
+    configMap:
+      name: haproxy
+  containers:
+  - name: haproxy
+    image: haproxy
+    volumeMounts:
+    - name: config
+      mountPath: /usr/local/etc/haproxy/
+

k8s/ingress.yaml

@@ -0,0 +1,14 @@
+apiVersion: extensions/v1beta1
+kind: Ingress
+metadata:
+  name: cheddar
+spec:
+  rules:
+  - host: cheddar.A.B.C.D.nip.io
+    http:
+      paths:
+      - path: /
+        backend:
+          serviceName: cheddar
+          servicePort: 80
+

k8s/insecure-dashboard.yaml

@@ -0,0 +1,220 @@
+# Copyright 2017 The Kubernetes Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Configuration to deploy release version of the Dashboard UI compatible with
+# Kubernetes 1.8.
+#
+# Example usage: kubectl create -f <this_file>
+
+# ------------------- Dashboard Secret ------------------- #
+
+apiVersion: v1
+kind: Secret
+metadata:
+  labels:
+    k8s-app: kubernetes-dashboard
+  name: kubernetes-dashboard-certs
+  namespace: kube-system
+type: Opaque
+
+---
+# ------------------- Dashboard Service Account ------------------- #
+
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  labels:
+    k8s-app: kubernetes-dashboard
+  name: kubernetes-dashboard
+  namespace: kube-system
+
+---
+# ------------------- Dashboard Role & Role Binding ------------------- #
+
+kind: Role
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: kubernetes-dashboard-minimal
+  namespace: kube-system
+rules:
+  # Allow Dashboard to create 'kubernetes-dashboard-key-holder' secret.
+- apiGroups: [""]
+  resources: ["secrets"]
+  verbs: ["create"]
+  # Allow Dashboard to create 'kubernetes-dashboard-settings' config map.
+- apiGroups: [""]
+  resources: ["configmaps"]
+  verbs: ["create"]
+  # Allow Dashboard to get, update and delete Dashboard exclusive secrets.
+- apiGroups: [""]
+  resources: ["secrets"]
+  resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs"]
+  verbs: ["get", "update", "delete"]
+  # Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map.
+- apiGroups: [""]
+  resources: ["configmaps"]
+  resourceNames: ["kubernetes-dashboard-settings"]
+  verbs: ["get", "update"]
+  # Allow Dashboard to get metrics from heapster.
+- apiGroups: [""]
+  resources: ["services"]
+  resourceNames: ["heapster"]
+  verbs: ["proxy"]
+- apiGroups: [""]
+  resources: ["services/proxy"]
+  resourceNames: ["heapster", "http:heapster:", "https:heapster:"]
+  verbs: ["get"]
+
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: kubernetes-dashboard-minimal
+  namespace: kube-system
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: Role
+  name: kubernetes-dashboard-minimal
+subjects:
+- kind: ServiceAccount
+  name: kubernetes-dashboard
+  namespace: kube-system
+
+---
+# ------------------- Dashboard Deployment ------------------- #
+
+kind: Deployment
+apiVersion: apps/v1beta2
+metadata:
+  labels:
+    k8s-app: kubernetes-dashboard
+  name: kubernetes-dashboard
+  namespace: kube-system
+spec:
+  replicas: 1
+  revisionHistoryLimit: 10
+  selector:
+    matchLabels:
+      k8s-app: kubernetes-dashboard
+  template:
+    metadata:
+      labels:
+        k8s-app: kubernetes-dashboard
+    spec:
+      containers:
+      - name: kubernetes-dashboard
+        image: k8s.gcr.io/kubernetes-dashboard-amd64:v1.8.3
+        ports:
+        - containerPort: 8443
+          protocol: TCP
+        args:
+          - --auto-generate-certificates
+          # Uncomment the following line to manually specify Kubernetes API server Host
+          # If not specified, Dashboard will attempt to auto discover the API server and connect
+          # to it. Uncomment only if the default does not work.
+          # - --apiserver-host=http://my-address:port
+        volumeMounts:
+        - name: kubernetes-dashboard-certs
+          mountPath: /certs
+          # Create on-disk volume to store exec logs
+        - mountPath: /tmp
+          name: tmp-volume
+        livenessProbe:
+          httpGet:
+            scheme: HTTPS
+            path: /
+            port: 8443
+          initialDelaySeconds: 30
+          timeoutSeconds: 30
+      volumes:
+      - name: kubernetes-dashboard-certs
+        secret:
+          secretName: kubernetes-dashboard-certs
+      - name: tmp-volume
+        emptyDir: {}
+      serviceAccountName: kubernetes-dashboard
+      # Comment the following tolerations if Dashboard must not be deployed on master
+      tolerations:
+      - key: node-role.kubernetes.io/master
+        effect: NoSchedule
+
+---
+# ------------------- Dashboard Service ------------------- #
+
+kind: Service
+apiVersion: v1
+metadata:
+  labels:
+    k8s-app: kubernetes-dashboard
+  name: kubernetes-dashboard
+  namespace: kube-system
+spec:
+  ports:
+    - port: 443
+      targetPort: 8443
+  selector:
+    k8s-app: kubernetes-dashboard
+---
+apiVersion: extensions/v1beta1
+kind: Deployment
+metadata:
+  labels:
+    app: dashboard
+  name: dashboard
+spec:
+  selector:
+    matchLabels:
+      app: dashboard
+  template:
+    metadata:
+      labels:
+        app: dashboard
+    spec:
+      containers:
+      - args:
+        - sh
+        - -c
+        - apk add --no-cache socat && socat TCP-LISTEN:80,fork,reuseaddr OPENSSL:kubernetes-dashboard.kube-system:443,verify=0
+        image: alpine
+        name: dashboard
+---
+apiVersion: v1
+kind: Service
+metadata:
+  labels:
+    app: dashboard
+  name: dashboard
+spec:
+  ports:
+  - port: 80
+    protocol: TCP
+    targetPort: 80
+  selector:
+    app: dashboard
+  type: NodePort
+---
+apiVersion: rbac.authorization.k8s.io/v1beta1
+kind: ClusterRoleBinding
+metadata:
+  name: kubernetes-dashboard
+  labels:
+    k8s-app: kubernetes-dashboard
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: cluster-admin
+subjects:
+- kind: ServiceAccount
+  name: kubernetes-dashboard
+  namespace: kube-system

k8s/just-a-pod.yaml

@@ -0,0 +1,10 @@
+apiVersion: v1
+Kind: Pod
+metadata:
+  name: hello
+  namespace: default
+spec:
+  containers:
+  - name: hello
+    image: nginx
+

k8s/kaniko-build.yaml

@@ -0,0 +1,29 @@
+apiVersion: v1
+kind: Pod
+metadata:
+  name: kaniko-build
+spec:
+  initContainers:
+  - name: git-clone
+    image: alpine
+    command: ["sh", "-c"]
+    args: 
+    - |
+      apk add --no-cache git &&
+      git clone git://github.com/jpetazzo/container.training /workspace
+    volumeMounts:
+    - name: workspace
+      mountPath: /workspace
+  containers:
+  - name: build-image
+    image: gcr.io/kaniko-project/executor:latest
+    args:
+      - "--context=/workspace/dockercoins/rng"
+      - "--insecure"
+      - "--destination=registry:5000/rng-kaniko:latest"
+    volumeMounts:
+    - name: workspace
+      mountPath: /workspace
+  volumes:
+  - name: workspace
+

k8s/kubernetes-dashboard.yaml

@@ -0,0 +1,167 @@
+# Copyright 2017 The Kubernetes Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Configuration to deploy release version of the Dashboard UI compatible with
+# Kubernetes 1.8.
+#
+# Example usage: kubectl create -f <this_file>
+
+# ------------------- Dashboard Secret ------------------- #
+
+apiVersion: v1
+kind: Secret
+metadata:
+  labels:
+    k8s-app: kubernetes-dashboard
+  name: kubernetes-dashboard-certs
+  namespace: kube-system
+type: Opaque
+
+---
+# ------------------- Dashboard Service Account ------------------- #
+
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  labels:
+    k8s-app: kubernetes-dashboard
+  name: kubernetes-dashboard
+  namespace: kube-system
+
+---
+# ------------------- Dashboard Role & Role Binding ------------------- #
+
+kind: Role
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: kubernetes-dashboard-minimal
+  namespace: kube-system
+rules:
+  # Allow Dashboard to create 'kubernetes-dashboard-key-holder' secret.
+- apiGroups: [""]
+  resources: ["secrets"]
+  verbs: ["create"]
+  # Allow Dashboard to create 'kubernetes-dashboard-settings' config map.
+- apiGroups: [""]
+  resources: ["configmaps"]
+  verbs: ["create"]
+  # Allow Dashboard to get, update and delete Dashboard exclusive secrets.
+- apiGroups: [""]
+  resources: ["secrets"]
+  resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs"]
+  verbs: ["get", "update", "delete"]
+  # Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map.
+- apiGroups: [""]
+  resources: ["configmaps"]
+  resourceNames: ["kubernetes-dashboard-settings"]
+  verbs: ["get", "update"]
+  # Allow Dashboard to get metrics from heapster.
+- apiGroups: [""]
+  resources: ["services"]
+  resourceNames: ["heapster"]
+  verbs: ["proxy"]
+- apiGroups: [""]
+  resources: ["services/proxy"]
+  resourceNames: ["heapster", "http:heapster:", "https:heapster:"]
+  verbs: ["get"]
+
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: kubernetes-dashboard-minimal
+  namespace: kube-system
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: Role
+  name: kubernetes-dashboard-minimal
+subjects:
+- kind: ServiceAccount
+  name: kubernetes-dashboard
+  namespace: kube-system
+
+---
+# ------------------- Dashboard Deployment ------------------- #
+
+kind: Deployment
+apiVersion: apps/v1beta2
+metadata:
+  labels:
+    k8s-app: kubernetes-dashboard
+  name: kubernetes-dashboard
+  namespace: kube-system
+spec:
+  replicas: 1
+  revisionHistoryLimit: 10
+  selector:
+    matchLabels:
+      k8s-app: kubernetes-dashboard
+  template:
+    metadata:
+      labels:
+        k8s-app: kubernetes-dashboard
+    spec:
+      containers:
+      - name: kubernetes-dashboard
+        image: k8s.gcr.io/kubernetes-dashboard-amd64:v1.8.3
+        ports:
+        - containerPort: 8443
+          protocol: TCP
+        args:
+          - --auto-generate-certificates
+          # Uncomment the following line to manually specify Kubernetes API server Host
+          # If not specified, Dashboard will attempt to auto discover the API server and connect
+          # to it. Uncomment only if the default does not work.
+          # - --apiserver-host=http://my-address:port
+        volumeMounts:
+        - name: kubernetes-dashboard-certs
+          mountPath: /certs
+          # Create on-disk volume to store exec logs
+        - mountPath: /tmp
+          name: tmp-volume
+        livenessProbe:
+          httpGet:
+            scheme: HTTPS
+            path: /
+            port: 8443
+          initialDelaySeconds: 30
+          timeoutSeconds: 30
+      volumes:
+      - name: kubernetes-dashboard-certs
+        secret:
+          secretName: kubernetes-dashboard-certs
+      - name: tmp-volume
+        emptyDir: {}
+      serviceAccountName: kubernetes-dashboard
+      # Comment the following tolerations if Dashboard must not be deployed on master
+      tolerations:
+      - key: node-role.kubernetes.io/master
+        effect: NoSchedule
+
+---
+# ------------------- Dashboard Service ------------------- #
+
+kind: Service
+apiVersion: v1
+metadata:
+  labels:
+    k8s-app: kubernetes-dashboard
+  name: kubernetes-dashboard
+  namespace: kube-system
+spec:
+  ports:
+    - port: 443
+      targetPort: 8443
+  selector:
+    k8s-app: kubernetes-dashboard

k8s/local-path-storage.yaml

@@ -0,0 +1,110 @@
+# This is a local copy of:
+# https://github.com/rancher/local-path-provisioner/blob/master/deploy/local-path-storage.yaml
+---
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: local-path-storage
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: local-path-provisioner-service-account
+  namespace: local-path-storage
+---
+apiVersion: rbac.authorization.k8s.io/v1beta1
+kind: ClusterRole
+metadata:
+  name: local-path-provisioner-role
+  namespace: local-path-storage
+rules:
+- apiGroups: [""]
+  resources: ["nodes", "persistentvolumeclaims"]
+  verbs: ["get", "list", "watch"]
+- apiGroups: [""]
+  resources: ["endpoints", "persistentvolumes", "pods"]
+  verbs: ["*"]
+- apiGroups: [""]
+  resources: ["events"]
+  verbs: ["create", "patch"]
+- apiGroups: ["storage.k8s.io"]
+  resources: ["storageclasses"]
+  verbs: ["get", "list", "watch"]
+---
+apiVersion: rbac.authorization.k8s.io/v1beta1
+kind: ClusterRoleBinding
+metadata:
+  name: local-path-provisioner-bind
+  namespace: local-path-storage
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: local-path-provisioner-role
+subjects:
+- kind: ServiceAccount
+  name: local-path-provisioner-service-account
+  namespace: local-path-storage
+---
+apiVersion: apps/v1beta2
+kind: Deployment
+metadata:
+  name: local-path-provisioner
+  namespace: local-path-storage
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: local-path-provisioner
+  template:
+    metadata:
+      labels:
+        app: local-path-provisioner
+    spec:
+      serviceAccountName: local-path-provisioner-service-account
+      containers:
+      - name: local-path-provisioner
+        image: rancher/local-path-provisioner:v0.0.8
+        imagePullPolicy: Always
+        command:
+        - local-path-provisioner
+        - --debug
+        - start
+        - --config
+        - /etc/config/config.json
+        volumeMounts:
+        - name: config-volume
+          mountPath: /etc/config/
+        env:
+        - name: POD_NAMESPACE
+          valueFrom:
+            fieldRef:
+              fieldPath: metadata.namespace
+      volumes:
+        - name: config-volume
+          configMap:
+            name: local-path-config
+---
+apiVersion: storage.k8s.io/v1
+kind: StorageClass
+metadata:
+  name: local-path
+provisioner: rancher.io/local-path
+volumeBindingMode: WaitForFirstConsumer
+reclaimPolicy: Delete
+---
+kind: ConfigMap
+apiVersion: v1
+metadata:
+  name: local-path-config
+  namespace: local-path-storage
+data:
+  config.json: |-
+        {
+                "nodePathMap":[
+                {
+                        "node":"DEFAULT_PATH_FOR_NON_LISTED_NODES",
+                        "paths":["/opt/local-path-provisioner"]
+                }
+                ]
+        }
+

k8s/metrics-server.yaml

@@ -0,0 +1,138 @@
+kind: ClusterRole
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: system:aggregated-metrics-reader
+  labels:
+    rbac.authorization.k8s.io/aggregate-to-view: "true"
+    rbac.authorization.k8s.io/aggregate-to-edit: "true"
+    rbac.authorization.k8s.io/aggregate-to-admin: "true"
+rules:
+- apiGroups: ["metrics.k8s.io"]
+  resources: ["pods"]
+  verbs: ["get", "list", "watch"]
+---
+apiVersion: rbac.authorization.k8s.io/v1beta1
+kind: ClusterRoleBinding
+metadata:
+  name: metrics-server:system:auth-delegator
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: system:auth-delegator
+subjects:
+- kind: ServiceAccount
+  name: metrics-server
+  namespace: kube-system
+---
+apiVersion: rbac.authorization.k8s.io/v1beta1
+kind: RoleBinding
+metadata:
+  name: metrics-server-auth-reader
+  namespace: kube-system
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: Role
+  name: extension-apiserver-authentication-reader
+subjects:
+- kind: ServiceAccount
+  name: metrics-server
+  namespace: kube-system
+---
+apiVersion: apiregistration.k8s.io/v1beta1
+kind: APIService
+metadata:
+  name: v1beta1.metrics.k8s.io
+spec:
+  service:
+    name: metrics-server
+    namespace: kube-system
+  group: metrics.k8s.io
+  version: v1beta1
+  insecureSkipTLSVerify: true
+  groupPriorityMinimum: 100
+  versionPriority: 100
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: metrics-server
+  namespace: kube-system
+---
+apiVersion: extensions/v1beta1
+kind: Deployment
+metadata:
+  name: metrics-server
+  namespace: kube-system
+  labels:
+    k8s-app: metrics-server
+spec:
+  selector:
+    matchLabels:
+      k8s-app: metrics-server
+  template:
+    metadata:
+      name: metrics-server
+      labels:
+        k8s-app: metrics-server
+    spec:
+      serviceAccountName: metrics-server
+      volumes:
+      # mount in tmp so we can safely use from-scratch images and/or read-only containers
+      - name: tmp-dir
+        emptyDir: {}
+      containers:
+      - name: metrics-server
+        image: k8s.gcr.io/metrics-server-amd64:v0.3.1
+        imagePullPolicy: Always
+        volumeMounts:
+        - name: tmp-dir
+          mountPath: /tmp
+        args:
+        - --kubelet-preferred-address-types=InternalIP
+        - --kubelet-insecure-tls
+        - --metric-resolution=5s
+
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: metrics-server
+  namespace: kube-system
+  labels:
+    kubernetes.io/name: "Metrics-server"
+spec:
+  selector:
+    k8s-app: metrics-server
+  ports:
+  - port: 443
+    protocol: TCP
+    targetPort: 443
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name: system:metrics-server
+rules:
+- apiGroups:
+  - ""
+  resources:
+  - pods
+  - nodes
+  - nodes/stats
+  verbs:
+  - get
+  - list
+  - watch
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+  name: system:metrics-server
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: system:metrics-server
+subjects:
+- kind: ServiceAccount
+  name: metrics-server
+  namespace: kube-system

k8s/netpol-allow-testcurl-for-testweb.yaml

@@ -0,0 +1,14 @@
+kind: NetworkPolicy
+apiVersion: networking.k8s.io/v1
+metadata:
+  name: allow-testcurl-for-testweb
+spec:
+  podSelector:
+    matchLabels:
+      app: testweb
+  ingress:
+  - from:
+    - podSelector:
+        matchLabels:
+          run: testcurl
+

k8s/netpol-deny-all-for-testweb.yaml

@@ -0,0 +1,10 @@
+kind: NetworkPolicy
+apiVersion: networking.k8s.io/v1
+metadata:
+  name: deny-all-for-testweb
+spec:
+  podSelector:
+    matchLabels:
+      app: testweb
+  ingress: []
+

k8s/netpol-dockercoins.yaml

@@ -0,0 +1,22 @@
+kind: NetworkPolicy
+apiVersion: networking.k8s.io/v1
+metadata:
+  name: deny-from-other-namespaces
+spec:
+  podSelector:
+    matchLabels:
+  ingress:
+  - from:
+    - podSelector: {}
+---
+kind: NetworkPolicy
+apiVersion: networking.k8s.io/v1
+metadata:
+  name: allow-webui
+spec:
+  podSelector:
+    matchLabels:
+      app: webui
+  ingress:
+  - from: []
+

k8s/nginx-with-volume.yaml

@@ -0,0 +1,21 @@
+apiVersion: v1
+kind: Pod
+metadata:
+  name: nginx-with-volume
+spec:
+  volumes:
+  - name: www
+  containers:
+  - name: nginx
+    image: nginx
+    volumeMounts:
+    - name: www
+      mountPath: /usr/share/nginx/html/
+  - name: git
+    image: alpine
+    command: [ "sh", "-c", "apk add --no-cache git && git clone https://github.com/octocat/Spoon-Knife /www" ]
+    volumeMounts:
+    - name: www
+      mountPath: /www/
+  restartPolicy: OnFailure
+

k8s/persistent-consul.yaml

@@ -0,0 +1,95 @@
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+  name: consul
+rules:
+  - apiGroups: [ "" ]
+    resources: [ pods ]
+    verbs:     [ get, list ]
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: consul
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: Role
+  name: consul
+subjects:
+  - kind: ServiceAccount
+    name: consul
+    namespace: orange
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: consul
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: consul
+spec:
+  ports:
+  - port: 8500
+    name: http
+  selector:
+    app: consul
+---
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+  name: consul
+spec:
+  serviceName: consul
+  replicas: 3
+  selector:
+    matchLabels:
+      app: consul
+  volumeClaimTemplates:
+    - metadata:
+        name: data
+      spec:
+        accessModes:
+          - ReadWriteOnce
+        resources:
+          requests:
+            storage: 1Gi
+  template:
+    metadata:
+      labels:
+        app: consul
+    spec:
+      serviceAccountName: consul
+      affinity:
+        podAntiAffinity:
+          requiredDuringSchedulingIgnoredDuringExecution:
+            - labelSelector:
+                matchExpressions:
+                  - key: app
+                    operator: In
+                    values:
+                      - consul
+              topologyKey: kubernetes.io/hostname
+      terminationGracePeriodSeconds: 10
+      containers:
+        - name: consul
+          image: "consul:1.4.4"
+          volumeMounts:
+            - name: data
+              mountPath: /consul/data
+          args:
+            - "agent"
+            - "-bootstrap-expect=3"
+            - "-retry-join=provider=k8s namespace=orange label_selector=\"app=consul\""
+            - "-client=0.0.0.0"
+            - "-data-dir=/consul/data"
+            - "-server"
+            - "-ui"
+          lifecycle:
+            preStop:
+              exec:
+                command:
+                - /bin/sh
+                - -c
+                - consul leave

k8s/portworx.yaml

@@ -0,0 +1,580 @@
+# SOURCE: https://install.portworx.com/?kbver=1.11.2&b=true&s=/dev/loop4&c=px-workshop&stork=true&lh=true
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: stork-config
+  namespace: kube-system
+data:
+  policy.cfg: |-
+    {
+      "kind": "Policy",
+      "apiVersion": "v1",
+      "extenders": [
+        {
+          "urlPrefix": "http://stork-service.kube-system.svc:8099",
+          "apiVersion": "v1beta1",
+          "filterVerb": "filter",
+          "prioritizeVerb": "prioritize",
+          "weight": 5,
+          "enableHttps": false,
+          "nodeCacheCapable": false
+        }
+      ]
+    }
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: stork-account
+  namespace: kube-system
+---
+kind: ClusterRole
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+   name: stork-role
+rules:
+  - apiGroups: [""]
+    resources: ["pods"]
+    verbs: ["get", "list", "delete"]
+  - apiGroups: [""]
+    resources: ["persistentvolumes"]
+    verbs: ["get", "list", "watch", "create", "delete"]
+  - apiGroups: [""]
+    resources: ["persistentvolumeclaims"]
+    verbs: ["get", "list", "watch", "update"]
+  - apiGroups: ["storage.k8s.io"]
+    resources: ["storageclasses"]
+    verbs: ["get", "list", "watch"]
+  - apiGroups: [""]
+    resources: ["events"]
+    verbs: ["list", "watch", "create", "update", "patch"]
+  - apiGroups: ["apiextensions.k8s.io"]
+    resources: ["customresourcedefinitions"]
+    verbs: ["create", "list", "watch", "delete"]
+  - apiGroups: ["volumesnapshot.external-storage.k8s.io"]
+    resources: ["volumesnapshots"]
+    verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
+  - apiGroups: ["volumesnapshot.external-storage.k8s.io"]
+    resources: ["volumesnapshotdatas"]
+    verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
+  - apiGroups: [""]
+    resources: ["configmaps"]
+    verbs: ["get", "create", "update"]
+  - apiGroups: [""]
+    resources: ["services"]
+    verbs: ["get"]
+  - apiGroups: [""]
+    resources: ["nodes"]
+    verbs: ["get", "list", "watch"]
+  - apiGroups: ["*"]
+    resources: ["deployments", "deployments/extensions"]
+    verbs: ["list", "get", "watch", "patch", "update", "initialize"]
+  - apiGroups: ["*"]
+    resources: ["statefulsets", "statefulsets/extensions"]
+    verbs: ["list", "get", "watch", "patch", "update", "initialize"]
+---
+kind: ClusterRoleBinding
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: stork-role-binding
+subjects:
+- kind: ServiceAccount
+  name: stork-account
+  namespace: kube-system
+roleRef:
+  kind: ClusterRole
+  name: stork-role
+  apiGroup: rbac.authorization.k8s.io
+---
+kind: Service
+apiVersion: v1
+metadata:
+  name: stork-service
+  namespace: kube-system
+spec:
+  selector:
+    name: stork
+  ports:
+    - protocol: TCP
+      port: 8099
+      targetPort: 8099
+---
+apiVersion: extensions/v1beta1
+kind: Deployment
+metadata:
+  annotations:
+    scheduler.alpha.kubernetes.io/critical-pod: ""
+  labels:
+    tier: control-plane
+  name: stork
+  namespace: kube-system
+spec:
+  strategy:
+    rollingUpdate:
+      maxSurge: 1
+      maxUnavailable: 1
+    type: RollingUpdate
+  replicas: 3
+  template:
+    metadata:
+      annotations:
+        scheduler.alpha.kubernetes.io/critical-pod: ""
+      labels:
+        name: stork
+        tier: control-plane
+    spec:
+      containers:
+      - command:
+        - /stork
+        - --driver=pxd
+        - --verbose
+        - --leader-elect=true
+        - --health-monitor-interval=120
+        imagePullPolicy: Always
+        image: openstorage/stork:1.1.3
+        resources:
+          requests:
+            cpu: '0.1'
+        name: stork
+      hostPID: false
+      affinity:
+        podAntiAffinity:
+          requiredDuringSchedulingIgnoredDuringExecution:
+            - labelSelector:
+                matchExpressions:
+                  - key: "name"
+                    operator: In
+                    values:
+                    - stork
+              topologyKey: "kubernetes.io/hostname"
+      serviceAccountName: stork-account
+---
+kind: StorageClass
+apiVersion: storage.k8s.io/v1
+metadata:
+  name: stork-snapshot-sc
+provisioner: stork-snapshot
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: stork-scheduler-account
+  namespace: kube-system
+---
+kind: ClusterRole
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: stork-scheduler-role
+rules:
+  - apiGroups: [""]
+    resources: ["endpoints"]
+    verbs: ["get", "update"]
+  - apiGroups: [""]
+    resources: ["configmaps"]
+    verbs: ["get"]
+  - apiGroups: [""]
+    resources: ["events"]
+    verbs: ["create", "patch", "update"]
+  - apiGroups: [""]
+    resources: ["endpoints"]
+    verbs: ["create"]
+  - apiGroups: [""]
+    resourceNames: ["kube-scheduler"]
+    resources: ["endpoints"]
+    verbs: ["delete", "get", "patch", "update"]
+  - apiGroups: [""]
+    resources: ["nodes"]
+    verbs: ["get", "list", "watch"]
+  - apiGroups: [""]
+    resources: ["pods"]
+    verbs: ["delete", "get", "list", "watch"]
+  - apiGroups: [""]
+    resources: ["bindings", "pods/binding"]
+    verbs: ["create"]
+  - apiGroups: [""]
+    resources: ["pods/status"]
+    verbs: ["patch", "update"]
+  - apiGroups: [""]
+    resources: ["replicationcontrollers", "services"]
+    verbs: ["get", "list", "watch"]
+  - apiGroups: ["app", "extensions"]
+    resources: ["replicasets"]
+    verbs: ["get", "list", "watch"]
+  - apiGroups: ["apps"]
+    resources: ["statefulsets"]
+    verbs: ["get", "list", "watch"]
+  - apiGroups: ["policy"]
+    resources: ["poddisruptionbudgets"]
+    verbs: ["get", "list", "watch"]
+  - apiGroups: [""]
+    resources: ["persistentvolumeclaims", "persistentvolumes"]
+    verbs: ["get", "list", "watch"]
+  - apiGroups: ["storage.k8s.io"]
+    resources: ["storageclasses"]
+    verbs: ["get", "list", "watch"]
+---
+kind: ClusterRoleBinding
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: stork-scheduler-role-binding
+subjects:
+- kind: ServiceAccount
+  name: stork-scheduler-account
+  namespace: kube-system
+roleRef:
+  kind: ClusterRole
+  name: stork-scheduler-role
+  apiGroup: rbac.authorization.k8s.io
+---
+apiVersion: apps/v1beta1
+kind: Deployment
+metadata:
+  labels:
+    component: scheduler
+    tier: control-plane
+    name: stork-scheduler
+  name: stork-scheduler
+  namespace: kube-system
+spec:
+  replicas: 3
+  template:
+    metadata:
+      labels:
+        component: scheduler
+        tier: control-plane
+      name: stork-scheduler
+    spec:
+      containers:
+      - command:
+        - /usr/local/bin/kube-scheduler
+        - --address=0.0.0.0
+        - --leader-elect=true
+        - --scheduler-name=stork
+        - --policy-configmap=stork-config
+        - --policy-configmap-namespace=kube-system
+        - --lock-object-name=stork-scheduler
+        image: gcr.io/google_containers/kube-scheduler-amd64:v1.11.2
+        livenessProbe:
+          httpGet:
+            path: /healthz
+            port: 10251
+          initialDelaySeconds: 15
+        name: stork-scheduler
+        readinessProbe:
+          httpGet:
+            path: /healthz
+            port: 10251
+        resources:
+          requests:
+            cpu: '0.1'
+      affinity:
+        podAntiAffinity:
+          requiredDuringSchedulingIgnoredDuringExecution:
+            - labelSelector:
+                matchExpressions:
+                  - key: "name"
+                    operator: In
+                    values:
+                    - stork-scheduler
+              topologyKey: "kubernetes.io/hostname"
+      hostPID: false
+      serviceAccountName: stork-scheduler-account
+---
+kind: Service
+apiVersion: v1
+metadata:
+  name: portworx-service
+  namespace: kube-system
+  labels:
+    name: portworx
+spec:
+  selector:
+    name: portworx
+  ports:
+    - name: px-api
+      protocol: TCP
+      port: 9001
+      targetPort: 9001
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: px-account
+  namespace: kube-system
+---
+kind: ClusterRole
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+   name: node-get-put-list-role
+rules:
+- apiGroups: [""]
+  resources: ["nodes"]
+  verbs: ["watch", "get", "update", "list"]
+- apiGroups: [""]
+  resources: ["pods"]
+  verbs: ["delete", "get", "list"]
+- apiGroups: [""]
+  resources: ["persistentvolumeclaims", "persistentvolumes"]
+  verbs: ["get", "list"]
+- apiGroups: [""]
+  resources: ["configmaps"]
+  verbs: ["get", "list", "update", "create"]
+- apiGroups: ["extensions"]
+  resources: ["podsecuritypolicies"]
+  resourceNames: ["privileged"]
+  verbs: ["use"]
+---
+kind: ClusterRoleBinding
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: node-role-binding
+subjects:
+- kind: ServiceAccount
+  name: px-account
+  namespace: kube-system
+roleRef:
+  kind: ClusterRole
+  name: node-get-put-list-role
+  apiGroup: rbac.authorization.k8s.io
+---
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: portworx
+---
+kind: Role
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: px-role
+  namespace: portworx
+rules:
+- apiGroups: [""]
+  resources: ["secrets"]
+  verbs: ["get", "list", "create", "update", "patch"]
+---
+kind: RoleBinding
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: px-role-binding
+  namespace: portworx
+subjects:
+- kind: ServiceAccount
+  name: px-account
+  namespace: kube-system
+roleRef:
+  kind: Role
+  name: px-role
+  apiGroup: rbac.authorization.k8s.io
+---
+apiVersion: extensions/v1beta1
+kind: DaemonSet
+metadata:
+  name: portworx
+  namespace: kube-system
+  annotations:
+    portworx.com/install-source: "https://install.portworx.com/?kbver=1.11.2&b=true&s=/dev/loop4&c=px-workshop&stork=true&lh=true"
+spec:
+  minReadySeconds: 0
+  updateStrategy:
+    type: RollingUpdate
+    rollingUpdate:
+      maxUnavailable: 1
+  template:
+    metadata:
+      labels:
+        name: portworx
+    spec:
+      affinity:
+        nodeAffinity:
+          requiredDuringSchedulingIgnoredDuringExecution:
+            nodeSelectorTerms:
+            - matchExpressions:
+              - key: px/enabled
+                operator: NotIn
+                values:
+                - "false"
+              - key: node-role.kubernetes.io/master
+                operator: DoesNotExist
+      hostNetwork: true
+      hostPID: false
+      containers:
+        - name: portworx
+          image: portworx/oci-monitor:1.4.2.2
+          imagePullPolicy: Always
+          args:
+            ["-c", "px-workshop", "-s", "/dev/loop4", "-b",
+             "-x", "kubernetes"]
+          env:
+            - name: "PX_TEMPLATE_VERSION"
+              value: "v4"
+            
+          livenessProbe:
+            periodSeconds: 30
+            initialDelaySeconds: 840 # allow image pull in slow networks
+            httpGet:
+              host: 127.0.0.1
+              path: /status
+              port: 9001
+          readinessProbe:
+            periodSeconds: 10
+            httpGet:
+              host: 127.0.0.1
+              path: /health
+              port: 9015
+          terminationMessagePath: "/tmp/px-termination-log"
+          securityContext:
+            privileged: true
+          volumeMounts:
+            - name: dockersock
+              mountPath: /var/run/docker.sock
+            - name: etcpwx
+              mountPath: /etc/pwx
+            - name: optpwx
+              mountPath: /opt/pwx
+            - name: proc1nsmount
+              mountPath: /host_proc/1/ns
+            - name: sysdmount
+              mountPath: /etc/systemd/system
+            - name: diagsdump
+              mountPath: /var/cores
+            - name: journalmount1
+              mountPath: /var/run/log
+              readOnly: true
+            - name: journalmount2
+              mountPath: /var/log
+              readOnly: true
+            - name: dbusmount
+              mountPath: /var/run/dbus
+      restartPolicy: Always
+      serviceAccountName: px-account
+      volumes:
+        - name: dockersock
+          hostPath:
+            path: /var/run/docker.sock
+        - name: etcpwx
+          hostPath:
+            path: /etc/pwx
+        - name: optpwx
+          hostPath:
+            path: /opt/pwx
+        - name: proc1nsmount
+          hostPath:
+            path: /proc/1/ns
+        - name: sysdmount
+          hostPath:
+            path: /etc/systemd/system
+        - name: diagsdump
+          hostPath:
+            path: /var/cores
+        - name: journalmount1
+          hostPath:
+            path: /var/run/log
+        - name: journalmount2
+          hostPath:
+            path: /var/log
+        - name: dbusmount
+          hostPath:
+            path: /var/run/dbus
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: px-lh-account
+  namespace: kube-system
+---
+kind: Role
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+   name: px-lh-role
+   namespace: kube-system
+rules:
+- apiGroups: [""]
+  resources: ["configmaps"]
+  verbs: ["get", "create", "update"]
+---
+kind: RoleBinding
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: px-lh-role-binding
+  namespace: kube-system
+subjects:
+- kind: ServiceAccount
+  name: px-lh-account
+  namespace: kube-system
+roleRef:
+  kind: Role
+  name: px-lh-role
+  apiGroup: rbac.authorization.k8s.io
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: px-lighthouse
+  namespace: kube-system
+  labels:
+    tier: px-web-console
+spec:
+  type: NodePort
+  ports:
+    - name: http
+      port: 80
+      nodePort: 32678
+    - name: https
+      port: 443
+      nodePort: 32679
+  selector:
+    tier: px-web-console
+---
+apiVersion: apps/v1beta2
+kind: Deployment
+metadata:
+  name: px-lighthouse
+  namespace: kube-system
+  labels:
+    tier: px-web-console
+spec:
+  strategy:
+    rollingUpdate:
+      maxSurge: 1
+      maxUnavailable: 1
+    type: RollingUpdate
+  selector:
+    matchLabels:
+      tier: px-web-console
+  replicas: 1
+  template:
+    metadata:
+      labels:
+        tier: px-web-console
+    spec:
+      initContainers:
+      - name: config-init
+        image: portworx/lh-config-sync:0.2
+        imagePullPolicy: Always
+        args:
+        - "init"
+        volumeMounts:
+        - name: config
+          mountPath: /config/lh
+      containers:
+      - name: px-lighthouse
+        image: portworx/px-lighthouse:1.5.0
+        imagePullPolicy: Always
+        ports:
+        - containerPort: 80
+        - containerPort: 443
+        volumeMounts:
+        - name: config
+          mountPath: /config/lh
+      - name: config-sync
+        image: portworx/lh-config-sync:0.2
+        imagePullPolicy: Always
+        args:
+        - "sync"
+        volumeMounts:
+        - name: config
+          mountPath: /config/lh
+      serviceAccountName: px-lh-account
+      volumes:
+      - name: config
+        emptyDir: {}

k8s/postgres.yaml

@@ -0,0 +1,30 @@
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+  name: postgres
+spec:
+  selector:
+    matchLabels:
+      app: postgres
+  serviceName: postgres
+  template:
+    metadata:
+      labels:
+        app: postgres
+    spec:
+      schedulerName: stork
+      containers:
+      - name: postgres
+        image: postgres:10.5
+        volumeMounts:
+        - mountPath: /var/lib/postgresql/data
+          name: postgres
+  volumeClaimTemplates:
+  - metadata:
+      name: postgres
+    spec:
+      accessModes: ["ReadWriteOnce"]
+      resources:
+        requests:
+          storage: 1Gi
+

k8s/psp-privileged.yaml

@@ -0,0 +1,39 @@
+---
+apiVersion: policy/v1beta1
+kind: PodSecurityPolicy
+metadata:
+  name: privileged
+  annotations:
+    seccomp.security.alpha.kubernetes.io/allowedProfileNames: '*'
+spec:
+  privileged: true
+  allowPrivilegeEscalation: true
+  allowedCapabilities:
+  - '*'
+  volumes:
+  - '*'
+  hostNetwork: true
+  hostPorts:
+  - min: 0
+    max: 65535
+  hostIPC: true
+  hostPID: true
+  runAsUser:
+    rule: 'RunAsAny'
+  seLinux:
+    rule: 'RunAsAny'
+  supplementalGroups:
+    rule: 'RunAsAny'
+  fsGroup:
+    rule: 'RunAsAny'
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name: psp:privileged
+rules:
+- apiGroups:     ['policy']
+  resources:     ['podsecuritypolicies']
+  verbs:         ['use']
+  resourceNames: ['privileged']
+

k8s/psp-restricted.yaml

@@ -0,0 +1,38 @@
+---
+apiVersion: extensions/v1beta1
+kind: PodSecurityPolicy
+metadata:
+  annotations:
+    apparmor.security.beta.kubernetes.io/allowedProfileNames: runtime/default
+    apparmor.security.beta.kubernetes.io/defaultProfileName: runtime/default
+    seccomp.security.alpha.kubernetes.io/allowedProfileNames: docker/default
+    seccomp.security.alpha.kubernetes.io/defaultProfileName: docker/default
+  name: restricted
+spec:
+  allowPrivilegeEscalation: false
+  fsGroup:
+    rule: RunAsAny
+  runAsUser:
+    rule: RunAsAny
+  seLinux:
+    rule: RunAsAny
+  supplementalGroups:
+    rule: RunAsAny
+  volumes:
+  - configMap
+  - emptyDir
+  - projected
+  - secret
+  - downwardAPI
+  - persistentVolumeClaim
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name: psp:restricted
+rules:
+- apiGroups:     ['policy']
+  resources:     ['podsecuritypolicies']
+  verbs:         ['use']
+  resourceNames: ['restricted']
+

k8s/registry.yaml

@@ -0,0 +1,15 @@
+apiVersion: v1
+kind: Pod
+metadata:
+  name: registry
+spec:
+  containers:
+  - name: registry
+    image: registry
+    env:
+    - name: REGISTRY_HTTP_ADDR
+      valueFrom:
+        configMapKeyRef:
+          name: registry
+          key: http.addr
+

k8s/socat.yaml

@@ -0,0 +1,67 @@
+apiVersion: extensions/v1beta1
+kind: Deployment
+metadata:
+  annotations:
+    deployment.kubernetes.io/revision: "2"
+  creationTimestamp: null
+  generation: 1
+  labels:
+    app: socat
+  name: socat
+  namespace: kube-system
+  selfLink: /apis/extensions/v1beta1/namespaces/kube-system/deployments/socat
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: socat
+  strategy:
+    rollingUpdate:
+      maxSurge: 1
+      maxUnavailable: 1
+    type: RollingUpdate
+  template:
+    metadata:
+      creationTimestamp: null
+      labels:
+        app: socat
+    spec:
+      containers:
+      - args:
+        - sh
+        - -c
+        - apk add --no-cache socat && socat TCP-LISTEN:80,fork,reuseaddr OPENSSL:kubernetes-dashboard:443,verify=0
+        image: alpine
+        imagePullPolicy: Always
+        name: socat
+        resources: {}
+        terminationMessagePath: /dev/termination-log
+        terminationMessagePolicy: File
+      dnsPolicy: ClusterFirst
+      restartPolicy: Always
+      schedulerName: default-scheduler
+      securityContext: {}
+      terminationGracePeriodSeconds: 30
+status: {}
+---
+apiVersion: v1
+kind: Service
+metadata:
+  creationTimestamp: null
+  labels:
+    app: socat
+  name: socat
+  namespace: kube-system
+  selfLink: /api/v1/namespaces/kube-system/services/socat
+spec:
+  externalTrafficPolicy: Cluster
+  ports:
+  - port: 80
+    protocol: TCP
+    targetPort: 80
+  selector:
+    app: socat
+  sessionAffinity: None
+  type: NodePort
+status:
+  loadBalancer: {}

k8s/storage-class.yaml

@@ -0,0 +1,11 @@
+kind: StorageClass
+apiVersion: storage.k8s.io/v1beta1
+metadata:
+  name: portworx-replicated
+  annotations:
+    storageclass.kubernetes.io/is-default-class: "true"
+provisioner: kubernetes.io/portworx-volume
+parameters:
+ repl: "2"
+ priority_io: "high"
+

k8s/traefik.yaml

@@ -0,0 +1,100 @@
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: traefik-ingress-controller
+  namespace: kube-system
+---
+kind: DaemonSet
+apiVersion: extensions/v1beta1
+metadata:
+  name: traefik-ingress-controller
+  namespace: kube-system
+  labels:
+    k8s-app: traefik-ingress-lb
+spec:
+  template:
+    metadata:
+      labels:
+        k8s-app: traefik-ingress-lb
+        name: traefik-ingress-lb
+    spec:
+      tolerations:
+      - effect: NoSchedule
+        operator: Exists
+      hostNetwork: true
+      serviceAccountName: traefik-ingress-controller
+      terminationGracePeriodSeconds: 60
+      containers:
+      - image: traefik
+        name: traefik-ingress-lb
+        ports:
+        - name: http
+          containerPort: 80
+          hostPort: 80
+        - name: admin
+          containerPort: 8080
+          hostPort: 8080
+        securityContext:
+          capabilities:
+            drop:
+            - ALL
+            add:
+            - NET_BIND_SERVICE
+        args:
+        - --api
+        - --kubernetes
+        - --logLevel=INFO
+---
+kind: Service
+apiVersion: v1
+metadata:
+  name: traefik-ingress-service
+  namespace: kube-system
+spec:
+  selector:
+    k8s-app: traefik-ingress-lb
+  ports:
+    - protocol: TCP
+      port: 80
+      name: web
+    - protocol: TCP
+      port: 8080
+      name: admin
+---
+kind: ClusterRole
+apiVersion: rbac.authorization.k8s.io/v1beta1
+metadata:
+  name: traefik-ingress-controller
+rules:
+  - apiGroups:
+      - ""
+    resources:
+      - services
+      - endpoints
+      - secrets
+    verbs:
+      - get
+      - list
+      - watch
+  - apiGroups:
+      - extensions
+    resources:
+      - ingresses
+    verbs:
+      - get
+      - list
+      - watch
+---
+kind: ClusterRoleBinding
+apiVersion: rbac.authorization.k8s.io/v1beta1
+metadata:
+  name: traefik-ingress-controller
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: traefik-ingress-controller
+subjects:
+- kind: ServiceAccount
+  name: traefik-ingress-controller
+  namespace: kube-system

k8s/users:jean.doe.yaml

@@ -0,0 +1,33 @@
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: jean.doe
+  namespace: users
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name: users:jean.doe
+rules:
+- apiGroups: [ certificates.k8s.io ]
+  resources: [ certificatesigningrequests ]
+  verbs:     [ create ]
+- apiGroups:     [ certificates.k8s.io ]
+  resourceNames: [ users:jean.doe ]
+  resources:     [ certificatesigningrequests ]
+  verbs:         [ get, create, delete, watch ]
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+  name: users:jean.doe
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: users:jean.doe
+subjects:
+- kind: ServiceAccount
+  name: jean.doe
+  namespace: users
+

k8s/volumes-for-consul.yaml

@@ -0,0 +1,70 @@
+---
+apiVersion: v1
+kind: PersistentVolume
+metadata:
+  name: consul-node2
+  annotations:
+    node: node2
+spec:
+  capacity:
+    storage: 10Gi
+  accessModes:
+  - ReadWriteOnce
+  persistentVolumeReclaimPolicy: Delete
+  local:
+    path: /mnt/consul
+  nodeAffinity:
+    required:
+      nodeSelectorTerms:
+      - matchExpressions:
+        - key: kubernetes.io/hostname
+          operator: In
+          values:
+          - node2
+---
+apiVersion: v1
+kind: PersistentVolume
+metadata:
+  name: consul-node3
+  annotations:
+    node: node3
+spec:
+  capacity:
+    storage: 10Gi
+  accessModes:
+  - ReadWriteOnce
+  persistentVolumeReclaimPolicy: Delete
+  local:
+    path: /mnt/consul
+  nodeAffinity:
+    required:
+      nodeSelectorTerms:
+      - matchExpressions:
+        - key: kubernetes.io/hostname
+          operator: In
+          values:
+          - node3
+---
+apiVersion: v1
+kind: PersistentVolume
+metadata:
+  name: consul-node4
+  annotations:
+    node: node4
+spec:
+  capacity:
+    storage: 10Gi
+  accessModes:
+  - ReadWriteOnce
+  persistentVolumeReclaimPolicy: Delete
+  local:
+    path: /mnt/consul
+  nodeAffinity:
+    required:
+      nodeSelectorTerms:
+      - matchExpressions:
+        - key: kubernetes.io/hostname
+          operator: In
+          values:
+          - node4
+

prepare-local/README.md

@@ -32,7 +32,7 @@ Virtualbox, Vagrant and Ansible
 
           $ source path/to/your-ansible-clone/hacking/env-setup
 
-  - you need to repeat the last step everytime you open a new terminal session
+  - you need to repeat the last step every time you open a new terminal session
     and want to use any Ansible command (but you'll probably only need to run
     it once).
 

prepare-vms/README.md

@@ -1,26 +1,47 @@
-# Trainer tools to create and prepare VMs for Docker workshops on AWS
+# Trainer tools to create and prepare VMs for Docker workshops
+
+These tools can help you to create VMs on:
+
+- Azure
+- EC2
+- OpenStack
 
 ## Prerequisites
 
 - [Docker](https://docs.docker.com/engine/installation/)
 - [Docker Compose](https://docs.docker.com/compose/install/)
+- [Parallel SSH](https://code.google.com/archive/p/parallel-ssh/) (on a Mac: `brew install pssh`) - the configuration scripts require this
+
+Depending on the infrastructure that you want to use, you also need to install
+the Azure CLI, the AWS CLI, or terraform (for OpenStack deployment).
+
+And if you want to generate printable cards:
+
+- [pyyaml](https://pypi.python.org/pypi/PyYAML) (on a Mac: `brew install pyyaml`)
+- [jinja2](https://pypi.python.org/pypi/Jinja2) (on a Mac: `brew install jinja2`)
 
 ## General Workflow
 
 - fork/clone repo
-- set required environment variables for AWS
+- create an infrastructure configuration in the `prepare-vms/infra` directory
+  (using one of the example files in that directory)
 - create your own setting file from `settings/example.yaml`
-- run `./workshopctl` commands to create instances, install docker, setup each users environment in node1, other management tasks
-- run `./workshopctl cards` command to generate PDF for printing handouts of each users host IP's and login info
+- if necessary, increase allowed open files: `ulimit -Sn 10000`
+- run `./workshopctl start` to create instances
+- run `./workshopctl deploy` to install Docker and setup environment
+- run `./workshopctl kube` (if you want to install and setup Kubernetes)
+- run `./workshopctl cards` (if you want to generate PDF for printing handouts of each users host IP's and login info)
+- run `./workshopctl stop` at the end of the workshop to terminate instances
 
 ## Clone/Fork the Repo, and Build the Tools Image
 
 The Docker Compose file here is used to build a image with all the dependencies to run the `./workshopctl` commands and optional tools. Each run of the script will check if you have those dependencies locally on your host, and will only use the container if you're [missing a dependency](workshopctl#L5).
 
-    $ git clone https://github.com/jpetazzo/orchestration-workshop.git
-    $ cd orchestration-workshop/prepare-vms
+    $ git clone https://github.com/jpetazzo/container.training
+    $ cd container.training/prepare-vms
     $ docker-compose build
 
+
 ## Preparing to Run `./workshopctl`
 
 ### Required AWS Permissions/Info
@@ -29,17 +50,37 @@ The Docker Compose file here is used to build a image with all the dependencies
 - Using a non-default VPC or Security Group isn't supported out of box yet, so you will have to customize `lib/commands.sh` if you want to change that.
 - These instances will assign the default VPC Security Group, which does not open any ports from Internet by default. So you'll need to add Inbound rules for `SSH | TCP | 22 | 0.0.0.0/0` and `Custom TCP Rule | TCP | 8000 - 8002 | 0.0.0.0/0`, or run `./workshopctl opensg` which opens up all ports.
 
-### Required Environment Variables
+### Create your `infra` file
 
-- `AWS_ACCESS_KEY_ID`
-- `AWS_SECRET_ACCESS_KEY`
-- `AWS_DEFAULT_REGION`
+You need to do this only once. (On AWS, you can create one `infra`
+file per region.)
 
-### Update/copy `settings/example.yaml`
+Make a copy of one of the example files in the `infra` directory.
 
-Then pass `settings/YOUR_WORKSHOP_NAME-settings.yaml` as an argument to `./workshopctl deploy`, `./workshopctl cards`, etc.
+For instance:
 
-./workshopctl cards 2016-09-28-00-33-bret settings/orchestration.yaml
+```bash
+cp infra/example.aws infra/aws-us-west-2
+```
+
+Edit your infrastructure file to customize it.
+You will probably need to put your cloud provider credentials,
+select region...
+
+If you don't have the `aws` CLI installed, you will get a warning that it's a missing dependency. If you're not using AWS you can ignore this.
+
+### Create your `settings` file
+
+Similarly, pick one of the files in `settings` and copy it
+to customize it.
+
+For instance:
+
+```bash
+cp settings/example.yaml settings/myworkshop.yaml
+```
+
+You're all set!
 
 ## `./workshopctl` Usage
 
@@ -48,21 +89,24 @@ workshopctl - the orchestration workshop swiss army knife
 Commands:
 ami          Show the AMI that will be used for deployment
 amis         List Ubuntu AMIs in the current region
-cards        Generate ready-to-print cards for a batch of VMs
+build        Build the Docker image to run this program in a container
+cards        Generate ready-to-print cards for a group of VMs
 deploy       Install Docker on a bunch of running VMs
 ec2quotas    Check our EC2 quotas (max instances)
 help         Show available commands
 ids          List the instance IDs belonging to a given tag or token
 ips          List the IP addresses of the VMs for a given tag or token
 kube         Setup kubernetes clusters with kubeadm (must be run AFTER deploy)
-list         List available batches in the current region
+kubetest     Check that all notes are reporting as Ready
+list         List available groups in the current region
 opensg       Open the default security group to ALL ingress traffic
 pull_images  Pre-pull a bunch of Docker images
-retag        Apply a new tag to a batch of VMs
-start        Start a batch of VMs
-status       List instance status for a given batch
+retag        Apply a new tag to a group of VMs
+start        Start a group of VMs
+status       List instance status for a given group
 stop         Stop (terminate, shutdown, kill, remove, destroy...) instances
-test         Run tests (pre-flight checks) on a batch of VMs
+test         Run tests (pre-flight checks) on a group of VMs
+wrap         Run this program in a container
 ```
 
 ### Summary of What `./workshopctl` Does For You
@@ -73,35 +117,78 @@ test         Run tests (pre-flight checks) on a batch of VMs
 - The `./workshopctl` script can be executed directly.
 - It will run locally if all its dependencies are fulfilled; otherwise it will run in the Docker container you created with `docker-compose build` (preparevms_prepare-vms).
 - During `start` it will add your default local SSH key to all instances under the `ubuntu` user.
-- During `deploy` it will create the `docker` user with password `training`, which is printing on the cards for students. For now, this is hard coded.
+- During `deploy` it will create the `docker` user with password `training`, which is printing on the cards for students. This can be configured with the `docker_user_password` property in the settings file.
 
-### Example Steps to Launch a Batch of Instances for a Workshop
+### Example Steps to Launch a group of AWS Instances for a Workshop
 
-- Run `./workshopctl start N` Creates `N` EC2 instances
+- Run `./workshopctl start --infra infra/aws-us-east-2 --settings/myworkshop.yaml --count 60` to create 60 EC2 instances
   - Your local SSH key will be synced to instances under `ubuntu` user
   - AWS instances will be created and tagged based on date, and IP's stored in `prepare-vms/tags/`
-- Run `./workshopctl deploy TAG settings/somefile.yaml` to run `scripts/postprep.rc` via parallel-ssh
+- Run `./workshopctl deploy TAG` to run `lib/postprep.py` via parallel-ssh
   - If it errors or times out, you should be able to rerun
   - Requires good connection to run all the parallel SSH connections, up to 100 parallel (ProTip: create dedicated management instance in same AWS region where you run all these utils from)
-- Run `./workshopctl pull-images TAG` to pre-pull a bunch of Docker images to the instances
-- Run `./workshopctl cards TAG settings/somefile.yaml` generates PDF/HTML files to print and cut and hand out to students
+- Run `./workshopctl pull_images TAG` to pre-pull a bunch of Docker images to the instances
+- Run `./workshopctl cards TAG` generates PDF/HTML files to print and cut and hand out to students
 - *Have a great workshop*
 - Run `./workshopctl stop TAG` to terminate instances.
 
-## Other Tools
+### Example Steps to Launch Azure Instances
 
-### Deploying your SSH key to all the machines
+- Install the [Azure CLI](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest) and authenticate with a valid account (`az login`)
+- Customize `azuredeploy.parameters.json`
+  - Required:
+    - Provide the SSH public key you plan to use for instance configuration
+  - Optional:
+    - Choose a name for the workshop (default is "workshop")
+    - Choose the number of instances (default is 3)
+    - Customize the desired instance size (default is Standard_D1_v2)
+ - Launch instances with your chosen resource group name and your preferred region; the examples are "workshop" and "eastus":
+```
+az group create --name workshop --location eastus
+az group deployment create --resource-group workshop --template-file azuredeploy.json --parameters @azuredeploy.parameters.json
+```
 
-- Make sure that you have SSH keys loaded (`ssh-add -l`).
-- Source `rc`.
-- Run `pcopykey`.
+The `az group deployment create` command can take several minutes and will only say `- Running ..` until it completes, unless you increase the verbosity with `--verbose` or `--debug`.
 
+To display the IPs of the instances you've launched:
 
-### Installing extra packages
+```
+az vm list-ip-addresses --resource-group workshop --output table
+```
 
-- Source `postprep.rc`.
-  (This will install a few extra packages, add entries to
-  /etc/hosts, generate SSH keys, and deploy them on all hosts.)
+If you want to put the IPs into `prepare-vms/tags/<tag>/ips.txt` for a tag of "myworkshop":
+
+1) If you haven't yet installed `jq` and/or created your event's tags directory in `prepare-vms`:
+
+```
+brew install jq
+mkdir -p tags/myworkshop
+```
+
+2) And then generate the IP list:
+
+```
+az vm list-ip-addresses --resource-group workshop --output json | jq -r '.[].virtualMachine.network.publicIpAddresses[].ipAddress' > tags/myworkshop/ips.txt
+```
+
+After the workshop is over, remove the instances:
+
+```
+az group delete --resource-group workshop
+```
+
+### Example Steps to Configure Instances from a non-AWS Source
+
+- Copy `infra/example.generic` to `infra/generic`
+- Run `./workshopctl start --infra infra/generic --settings settings/...yaml`
+- Note the `prepare-vms/tags/TAG/` path that has been auto-created.
+- Launch instances via your preferred method. You'll need to get the instance IPs and be able to SSH into them.
+- Edit the file `prepare-vms/tags/TAG/ips.txt`, it should list the IP addresses of the VMs (one per line, without any comments or other info)
+- Continue deployment of cluster configuration with `./workshopctl deploy TAG`
+- Optionally, configure Kubernetes clusters of the size in the settings: workshopctl kube `TAG`
+- Optionally, test your Kubernetes clusters. They may take a little time to become ready: workshopctl kubetest `TAG`
+- Generate cards to print and hand out: workshopctl cards `TAG`
+- Print the cards file: prepare-vms/tags/`TAG`/ips.html
 
 
 ## Even More Details
@@ -114,7 +201,7 @@ To see which local key will be uploaded, run `ssh-add -l | grep RSA`.
 
 #### Instance + tag creation
 
-10 VMs will be started, with an automatically generated tag (timestamp + your username).
+The VMs will be started, with an automatically generated tag (timestamp + your username).
 
 Your SSH key will be added to the `authorized_keys` of the ubuntu user.
 
@@ -122,35 +209,33 @@ Your SSH key will be added to the `authorized_keys` of the ubuntu user.
 
 Following the creation of the VMs, a text file will be created containing a list of their IPs.
 
-This ips.txt file will be created in the $TAG/ directory and a symlink will be placed in the working directory of the script.
-
-If you create new VMs, the symlinked file will be overwritten.
-
 #### Deployment
 
 Instances can be deployed manually using the `deploy` command:
 
-    $ ./workshopctl deploy TAG settings/somefile.yaml
+    $ ./workshopctl deploy TAG
 
-The `postprep.rc` file will be copied via parallel-ssh to all of the VMs and executed.
+The `postprep.py` file will be copied via parallel-ssh to all of the VMs and executed.
 
 #### Pre-pull images
 
-    $ ./workshopctl pull-images TAG
+    $ ./workshopctl pull_images TAG
 
 #### Generate cards
 
-    $ ./workshopctl cards TAG settings/somefile.yaml
+    $ ./workshopctl cards TAG
+
+If you want to generate both HTML and PDF cards, install [wkhtmltopdf](https://wkhtmltopdf.org/downloads.html); without that installed, only HTML cards will be generated.
+
+If you don't have `wkhtmltopdf` installed, you will get a warning that it is a missing dependency. If you plan to just print the HTML cards, you can ignore this.
 
 #### List tags
 
-    $ ./workshopctl list
+    $ ./workshopctl list infra/some-infra-file
 
-#### List VMs
+    $ ./workshopctl listall
 
-    $ ./workshopctl list TAG
-
-This will print a human-friendly list containing some information about each instance.
+    $ ./workshopctl tags
 
 #### Stop and destroy VMs
 

prepare-vms/azuredeploy.json

@@ -0,0 +1,250 @@
+{
+  "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
+  "contentVersion": "1.0.0.0",
+  "parameters": {
+    "workshopName": {
+      "type": "string",
+      "defaultValue": "workshop",
+      "metadata": {
+        "description": "Workshop name."
+      }
+    },
+    "vmPrefix": {
+      "type": "string",
+      "defaultValue": "node",
+      "metadata": {
+        "description": "Prefix for VM names."
+      }
+    },
+    "numberOfInstances": {
+      "type": "int",
+      "defaultValue": 3,
+      "metadata": {
+        "description": "Number of VMs to create."
+      }
+    },
+    "adminUsername": {
+      "type": "string",
+      "defaultValue": "ubuntu",
+      "metadata": {
+        "description": "Admin username for VMs."
+      }
+    },
+    "sshKeyData": {
+      "type": "string",
+      "defaultValue": "",
+      "metadata": {
+        "description": "SSH rsa public key file as a string."
+      }
+    },
+    "imagePublisher": {
+      "type": "string",
+      "defaultValue": "Canonical",
+      "metadata": {
+        "description": "OS image publisher; default Canonical."
+      }
+    },
+    "imageOffer": {
+      "type": "string",
+      "defaultValue": "UbuntuServer",
+      "metadata": {
+        "description": "The name of the image offer. The default is Ubuntu"
+      }
+    },
+    "imageSKU": {
+      "type": "string",
+      "defaultValue": "16.04-LTS",
+      "metadata": {
+        "description": "Version of the image. The default is 16.04-LTS"
+      }
+    },
+    "vmSize": {
+      "type": "string",
+      "defaultValue": "Standard_D1_v2",
+      "metadata": {
+        "description": "VM Size."
+      }
+    }
+  },
+  "variables": {
+    "vnetID": "[resourceId('Microsoft.Network/virtualNetworks',variables('virtualNetworkName'))]",
+    "subnet1Ref": "[concat(variables('vnetID'),'/subnets/',variables('subnet1Name'))]",
+    "vmName": "[parameters('vmPrefix')]",
+    "sshKeyPath": "[concat('/home/',parameters('adminUsername'),'/.ssh/authorized_keys')]",
+    "publicIPAddressName": "PublicIP",
+    "publicIPAddressType": "Dynamic",
+    "virtualNetworkName": "MyVNET",
+    "netSecurityGroup": "MyNSG",
+    "addressPrefix": "10.0.0.0/16",
+    "subnet1Name": "subnet-1",
+    "subnet1Prefix": "10.0.0.0/24",
+    "nicName": "myVMNic"
+  },
+  "resources": [
+    {
+      "apiVersion": "2017-11-01",
+      "type": "Microsoft.Network/publicIPAddresses",
+      "name": "[concat(variables('publicIPAddressName'),copyIndex(1))]",
+      "location": "[resourceGroup().location]",
+      "copy": {
+        "name": "publicIPLoop",
+        "count": "[parameters('numberOfInstances')]"
+      },
+      "properties": {
+        "publicIPAllocationMethod": "[variables('publicIPAddressType')]"
+      },
+      "tags": {
+        "workshop": "[parameters('workshopName')]"
+      }
+    },
+    {
+      "apiVersion": "2017-11-01",
+      "type": "Microsoft.Network/virtualNetworks",
+      "name": "[variables('virtualNetworkName')]",
+      "location": "[resourceGroup().location]",
+      "dependsOn": [
+        "[concat('Microsoft.Network/networkSecurityGroups/', variables('netSecurityGroup'))]"
+      ],
+      "properties": {
+        "addressSpace": {
+          "addressPrefixes": [
+            "[variables('addressPrefix')]"
+          ]
+        },
+        "subnets": [
+          {
+            "name": "[variables('subnet1Name')]",
+            "properties": {
+              "addressPrefix": "[variables('subnet1Prefix')]",
+              "networkSecurityGroup": {
+                "id": "[resourceId('Microsoft.Network/networkSecurityGroups', variables('netSecurityGroup'))]"
+              }
+            }
+          }
+        ]
+      },
+      "tags": {
+        "workshop": "[parameters('workshopName')]"
+      }
+    },
+    {
+      "apiVersion": "2017-11-01",
+      "type": "Microsoft.Network/networkInterfaces",
+      "name": "[concat(variables('nicName'),copyIndex(1))]",
+      "location": "[resourceGroup().location]",
+      "copy": {
+        "name": "nicLoop",
+        "count": "[parameters('numberOfInstances')]"
+      },
+      "dependsOn": [
+        "[concat('Microsoft.Network/publicIPAddresses/', variables('publicIPAddressName'),copyIndex(1))]",
+        "[concat('Microsoft.Network/virtualNetworks/', variables('virtualNetworkName'))]"
+      ],
+      "properties": {
+        "ipConfigurations": [
+          {
+            "name": "ipconfig1",
+            "properties": {
+              "privateIPAllocationMethod": "Dynamic",
+              "publicIPAddress": {
+                "id": "[resourceId('Microsoft.Network/publicIPAddresses', concat(variables('publicIPAddressName'), copyIndex(1)))]"
+              },
+              "subnet": {
+                "id": "[variables('subnet1Ref')]"
+              }
+            }
+          }
+        ]
+      },
+      "tags": {
+        "workshop": "[parameters('workshopName')]"
+      }
+    },
+    {
+      "apiVersion": "2017-12-01",
+      "type": "Microsoft.Compute/virtualMachines",
+      "name": "[concat(variables('vmName'),copyIndex(1))]",
+      "location": "[resourceGroup().location]",
+      "copy": {
+        "name": "vmLoop",
+        "count": "[parameters('numberOfInstances')]"
+      },
+      "dependsOn": [
+        "[concat('Microsoft.Network/networkInterfaces/', variables('nicName'), copyIndex(1))]"
+      ],
+      "properties": {
+        "hardwareProfile": {
+          "vmSize": "[parameters('vmSize')]"
+        },
+        "osProfile": {
+          "computerName": "[concat(variables('vmName'),copyIndex(1))]",
+          "adminUsername": "[parameters('adminUsername')]",
+          "linuxConfiguration": {
+            "disablePasswordAuthentication": true,
+            "ssh": {
+              "publicKeys": [
+                {
+                  "path": "[variables('sshKeyPath')]",
+                  "keyData": "[parameters('sshKeyData')]"
+                }
+              ]
+            }
+          }
+        },
+        "storageProfile": {
+          "osDisk": {
+            "createOption": "FromImage"
+          },
+          "imageReference": {
+            "publisher": "[parameters('imagePublisher')]",
+            "offer": "[parameters('imageOffer')]",
+            "sku": "[parameters('imageSKU')]",
+            "version": "latest"
+          }
+        },
+        "networkProfile": {
+          "networkInterfaces": [
+            {
+              "id": "[resourceId('Microsoft.Network/networkInterfaces', concat(variables('nicName'),copyIndex(1)))]"
+            }
+          ]
+        }
+      },
+      "tags": {
+        "workshop": "[parameters('workshopName')]"
+      }
+    },
+    {
+      "apiVersion": "2017-11-01",
+      "type": "Microsoft.Network/networkSecurityGroups",
+      "name": "[variables('netSecurityGroup')]",
+      "location": "[resourceGroup().location]",
+      "tags": {
+        "workshop": "[parameters('workshopName')]"
+      },
+      "properties": {
+        "securityRules": [
+          {
+            "name": "default-open-ports",
+            "properties": {
+              "protocol": "Tcp",
+              "sourcePortRange": "*",
+              "destinationPortRange": "*",
+              "sourceAddressPrefix": "*",
+              "destinationAddressPrefix": "*",
+              "access": "Allow",
+              "priority": 1000,
+              "direction": "Inbound"
+            }
+          }
+        ]
+      }
+    }
+  ],
+  "outputs": {
+    "resourceID": {
+      "type": "string",
+      "value": "[resourceId('Microsoft.Network/publicIPAddresses', concat(variables('publicIPAddressName'),'1'))]"
+    }
+  }
+}

prepare-vms/azuredeploy.parameters.json

@@ -0,0 +1,18 @@
+{
+  "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
+  "contentVersion": "1.0.0.0",
+  "parameters": {
+    "sshKeyData": {
+      "value": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXTIl/M9oeSlcsC5Rfe+nZr4Jc4sl200pSw2lpdxlZ3xzeP15NgSSMJnigUrKUXHfqRQ+2wiPxEf0Odz2GdvmXvR0xodayoOQsO24AoERjeSBXCwqITsfp1bGKzMb30/3ojRBo6LBR6r1+lzJYnNCGkT+IQwLzRIpm0LCNz1j08PUI2aZ04+mcDANvHuN/hwi/THbLLp6SNWN43m9r02RcC6xlCNEhJi4wk4VzMzVbSv9RlLGST2ocbUHwmQ2k9OUmpzoOx73aQi9XNnEaFh2w/eIdXM75VtkT3mRryyykg9y0/hH8/MVmIuRIdzxHQqlm++DLXVH5Ctw6a4kS+ki7 workshop"
+      },
+    "workshopName": {
+      "value": "workshop"
+    },
+    "numberOfInstances": {
+      "value": 3
+    },
+    "vmSize": {
+      "value": "Standard_D1_v2"
+    }
+  }
+}

prepare-vms/cards.html

@@ -1,104 +0,0 @@
-{# Feel free to customize or override anything in there! #}
-{%- set url = "http://container.training/" -%}
-{%- set pagesize = 12 -%}
-{%- if clustersize == 1 -%}
-    {%- set workshop_name = "Docker workshop" -%}
-    {%- set cluster_or_machine = "machine" -%}
-    {%- set this_or_each = "this" -%}
-    {%- set machine_is_or_machines_are = "machine is" -%}
-    {%- set image_src = "https://s3-us-west-2.amazonaws.com/www.breadware.com/integrations/docker.png" -%}
-{%- else -%}
-    {%- set workshop_name = "orchestration workshop" -%}
-    {%- set cluster_or_machine = "cluster" -%}
-    {%- set this_or_each = "each" -%}
-    {%- set machine_is_or_machines_are = "machines are" -%}
-    {%- set image_src = "https://cdn.wp.nginx.com/wp-content/uploads/2016/07/docker-swarm-hero2.png" -%}
-{%- endif -%}
-<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
-<html>
-<head><style>
-body, table {
-    margin: 0;
-    padding: 0;
-    line-height: 1em;
-    font-size: 14px;
-}
-
-table {
-    border-spacing: 0;
-    margin-top: 0.4em;
-    margin-bottom: 0.4em;
-    border-left: 0.8em double grey;
-    padding-left: 0.4em;
-}
-
-div {
-    float: left;
-    border: 1px dotted black;
-    padding-top: 1%;
-    padding-bottom: 1%;
-    /* columns * (width+left+right) < 100% */
-    width: 21.5%;
-    padding-left: 1.5%;
-    padding-right: 1.5%;
-}
-
-p {
-    margin: 0.4em 0 0.4em 0;
-}
-
-img {
-    height: 4em;
-    float: right;
-    margin-right: -0.4em;
-}
-
-.logpass {
-    font-family: monospace;
-    font-weight: bold;
-}
-
-.pagebreak {
-    page-break-after: always;
-    clear: both;
-    display: block;
-    height: 8px;
-}
-</style></head>
-<body>
-{% for cluster in clusters %}
-    {% if loop.index0>0 and loop.index0%pagesize==0 %}
-        <span class="pagebreak"></span>
-    {% endif %}
-    <div>
-
-        <p>
-            Here is the connection information to your very own
-            {{ cluster_or_machine }} for this {{ workshop_name }}.
-            You can connect to {{ this_or_each }} VM with any SSH client.
-        </p>
-        <p>
-            <img src="{{ image_src }}" />
-            <table>
-                <tr><td>login:</td></tr>
-                <tr><td class="logpass">docker</td></tr>
-                <tr><td>password:</td></tr>
-                <tr><td class="logpass">training</td></tr>
-            </table>
-
-        </p>
-        <p>
-            Your {{ machine_is_or_machines_are }}:
-            <table>
-                {% for node in cluster %}
-                <tr><td>node{{ loop.index }}:</td><td>{{ node }}</td></tr>
-                {% endfor %}
-            </table>
-        </p>
-        <p>You can find the slides at:
-            <center>{{ url }}</center>
-        </p>
-    </div>
-{% endfor %}
-</body>
-</html>

prepare-vms/cncsetup.sh

@@ -0,0 +1,12 @@
+#!/bin/sh
+if [ $(whoami) != ubuntu ]; then
+  echo "This script should be executed on a freshly deployed node,"
+  echo "with the 'ubuntu' user. Aborting."
+  exit 1
+fi
+if id docker; then
+  sudo userdel -r docker
+fi
+sudo apt-get update -q
+sudo apt-get install -qy jq python-pip wkhtmltopdf xvfb
+pip install --user awscli jinja2 pdfkit pssh

prepare-vms/docker-compose.yml

@@ -7,7 +7,6 @@ services:
         working_dir: /root/prepare-vms
         volumes:
             - $HOME/.aws/:/root/.aws/
-            - /etc/localtime:/etc/localtime:ro
             - $SSH_AUTH_SOCK:$SSH_AUTH_SOCK
             - $PWD/:/root/prepare-vms/
         environment:
@@ -15,5 +14,6 @@ services:
             AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
             AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
             AWS_DEFAULT_REGION: ${AWS_DEFAULT_REGION}
+            AWS_INSTANCE_TYPE: ${AWS_INSTANCE_TYPE}
             USER: ${USER}
         entrypoint: /root/prepare-vms/workshopctl

prepare-vms/infra/example.aws

@@ -0,0 +1,6 @@
+INFRACLASS=aws
+# If you are using AWS to deploy, copy this file (e.g. to "aws", or "us-east-1")
+# and customize the variables below.
+export AWS_DEFAULT_REGION=us-east-1
+export AWS_ACCESS_KEY_ID=AKI...
+export AWS_SECRET_ACCESS_KEY=...

prepare-vms/infra/example.generic

@@ -0,0 +1,2 @@
+INFRACLASS=generic
+# This is for manual provisioning. No other variable or configuration is needed.

prepare-vms/infra/example.openstack

@@ -0,0 +1,9 @@
+INFRACLASS=openstack
+# If you are using OpenStack, copy this file (e.g. to "openstack" or "enix")
+# and customize the variables below.
+export TF_VAR_user="jpetazzo"
+export TF_VAR_tenant="training"
+export TF_VAR_domain="Default"
+export TF_VAR_password="..."
+export TF_VAR_auth_url="https://api.r1.nxs.enix.io/v3"
+export TF_VAR_flavor="GP1.S"

prepare-vms/lib/aws.sh

@@ -1,105 +0,0 @@
-aws_display_tags() {
-    # Print all "Name" tags in our region with their instance count
-    echo "[#] [Status] [Token] [Tag]" \
-        | awk '{ printf "%-7s %-12s %-25s %-25s\n", $1, $2, $3, $4}'
-    aws ec2 describe-instances \
-        --query "Reservations[*].Instances[*].[State.Name,ClientToken,Tags[0].Value]" \
-        | tr -d "\r" \
-        | uniq -c \
-        | sort -k 3 \
-        | awk '{ printf "%-7s %-12s %-25s %-25s\n", $1, $2, $3, $4}'
-}
-
-aws_get_tokens() {
-    aws ec2 describe-instances --output text \
-        --query 'Reservations[*].Instances[*].[ClientToken]' \
-        | sort -u
-}
-
-aws_display_instance_statuses_by_tag() {
-    TAG=$1
-    need_tag $TAG
-
-    IDS=$(aws ec2 describe-instances \
-        --filters "Name=tag:Name,Values=$TAG" \
-        --query "Reservations[*].Instances[*].InstanceId" | tr '\t' ' ')
-
-    aws ec2 describe-instance-status \
-        --instance-ids $IDS \
-        --query "InstanceStatuses[*].{ID:InstanceId,InstanceState:InstanceState.Name,InstanceStatus:InstanceStatus.Status,SystemStatus:SystemStatus.Status,Reachability:InstanceStatus.Status}" \
-        --output table
-}
-
-aws_display_instances_by_tag() {
-    TAG=$1
-    need_tag $TAG
-    result=$(aws ec2 describe-instances --output table \
-        --filter "Name=tag:Name,Values=$TAG" \
-        --query "Reservations[*].Instances[*].[ \
-                        InstanceId, \
-                        State.Name, \
-                        Tags[0].Value, \
-                        PublicIpAddress, \
-                        InstanceType \
-                        ]"
-    )
-    if [[ -z $result ]]; then
-        die "No instances found with tag $TAG in region $AWS_DEFAULT_REGION."
-    else
-        echo "$result"
-    fi
-}
-
-aws_get_instance_ids_by_filter() {
-    FILTER=$1
-    aws ec2 describe-instances --filters $FILTER \
-        --query Reservations[*].Instances[*].InstanceId \
-        --output text | tr "\t" "\n" | tr -d "\r"
-}
-
-aws_get_instance_ids_by_client_token() {
-    TOKEN=$1
-    need_tag $TOKEN
-    aws_get_instance_ids_by_filter Name=client-token,Values=$TOKEN
-}
-
-aws_get_instance_ids_by_tag() {
-    TAG=$1
-    need_tag $TAG
-    aws_get_instance_ids_by_filter Name=tag:Name,Values=$TAG
-}
-
-aws_get_instance_ips_by_tag() {
-    TAG=$1
-    need_tag $TAG
-    aws ec2 describe-instances --filter "Name=tag:Name,Values=$TAG" \
-        --output text \
-        --query "Reservations[*].Instances[*].PublicIpAddress" \
-        | tr "\t" "\n" \
-        | sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4 # sort IPs
-}
-
-aws_kill_instances_by_tag() {
-    TAG=$1
-    need_tag $TAG
-    IDS=$(aws_get_instance_ids_by_tag $TAG)
-    if [ -z "$IDS" ]; then
-        die "Invalid tag."
-    fi
-
-    info "Deleting instances with tag $TAG."
-
-    aws ec2 terminate-instances --instance-ids $IDS \
-        | grep ^TERMINATINGINSTANCES
-
-    info "Deleted instances with tag $TAG."
-}
-
-aws_tag_instances() {
-    OLD_TAG_OR_TOKEN=$1
-    NEW_TAG=$2
-    IDS=$(aws_get_instance_ids_by_client_token $OLD_TAG_OR_TOKEN)
-    [[ -n "$IDS" ]] && aws ec2 create-tags --tag Key=Name,Value=$NEW_TAG --resources $IDS >/dev/null
-    IDS=$(aws_get_instance_ids_by_tag $OLD_TAG_OR_TOKEN)
-    [[ -n "$IDS" ]] && aws ec2 create-tags --tag Key=Name,Value=$NEW_TAG --resources $IDS >/dev/null
-}

prepare-vms/lib/cli.sh

@@ -2,7 +2,7 @@
 _ERR() {
     error "Command $BASH_COMMAND failed (exit status: $?)"
 }
-set -e
+set -eE
 trap _ERR ERR
 
 die() {
@@ -50,27 +50,41 @@ sep() {
     fi
 }
 
-need_tag() {
+need_infra() {
     if [ -z "$1" ]; then
+        die "Please specify infrastructure file. (e.g.: infra/aws)"
+    fi
+    if [ "$1" = "--infra" ]; then
+        die "The infrastructure file should be passed directly to this command. Remove '--infra' and try again."
+    fi
+    if [ ! -f "$1" ]; then
+        die "Infrastructure file $1 doesn't exist."
+    fi
+    . "$1"
+    . "lib/infra/$INFRACLASS.sh"
+}
+
+need_tag() {
+    if [ -z "$TAG" ]; then
         die "Please specify a tag or token. To see available tags and tokens, run: $0 list"
     fi
+    if [ ! -d "tags/$TAG" ]; then
+        die "Tag $TAG not found (directory tags/$TAG does not exist)."
+    fi
+    for FILE in settings.yaml ips.txt infra.sh; do
+        if [ ! -f "tags/$TAG/$FILE" ]; then
+          warning "File tags/$TAG/$FILE not found."
+        fi
+    done
+    . "tags/$TAG/infra.sh"
+    . "lib/infra/$INFRACLASS.sh"
 }
 
 need_settings() {
     if [ -z "$1" ]; then
-        die "Please specify a settings file."
-    elif [ ! -f "$1" ]; then
+        die "Please specify a settings file. (e.g.: settings/kube101.yaml)"
+    fi
+    if [ ! -f "$1" ]; then
         die "Settings file $1 doesn't exist."
     fi
 }
-
-need_ips_file() {
-    IPS_FILE=$1
-    if [ -z "$IPS_FILE" ]; then
-        die "IPS_FILE not set."
-    fi
-
-    if [ ! -s "$IPS_FILE" ]; then
-        die "IPS_FILE $IPS_FILE not found. Please run: $0 ips <TAG>"
-    fi
-}

prepare-vms/lib/commands.sh

@@ -2,26 +2,16 @@ export AWS_DEFAULT_OUTPUT=text
 
 HELP=""
 _cmd() {
-    HELP="$(printf "%s\n%-12s %s\n" "$HELP" "$1" "$2")"
+    HELP="$(printf "%s\n%-20s %s\n" "$HELP" "$1" "$2")"
 }
 
 _cmd help "Show available commands"
 _cmd_help() {
-    printf "$(basename $0) - the orchestration workshop swiss army knife\n"
+    printf "$(basename $0) - the container training swiss army knife\n"
     printf "Commands:"
     printf "%s" "$HELP" | sort
 }
 
-_cmd amis "List Ubuntu AMIs in the current region"
-_cmd_amis() {
-    find_ubuntu_ami -r $AWS_DEFAULT_REGION "$@"
-}
-
-_cmd ami "Show the AMI that will be used for deployment"
-_cmd_ami() {
-    find_ubuntu_ami -r $AWS_DEFAULT_REGION -a amd64 -v 16.04 -t hvm:ebs -N -q
-}
-
 _cmd build "Build the Docker image to run this program in a container"
 _cmd_build() {
     docker-compose build
@@ -32,70 +22,62 @@ _cmd_wrap() {
     docker-compose run --rm workshopctl "$@"
 }
 
-_cmd cards "Generate ready-to-print cards for a batch of VMs"
+_cmd cards "Generate ready-to-print cards for a group of VMs"
 _cmd_cards() {
     TAG=$1
-    SETTINGS=$2
-    need_tag $TAG
-    need_settings $SETTINGS
+    need_tag
 
-    aws_get_instance_ips_by_tag $TAG >tags/$TAG/ips.txt
-
-    # Remove symlinks to old cards
-    rm -f ips.html ips.pdf
-
-    # This will generate two files in the base dir: ips.pdf and ips.html
-    python lib/ips-txt-to-html.py $SETTINGS
-
-    for f in ips.html ips.pdf; do
-        # Remove old versions of cards if they exist
-        rm -f tags/$TAG/$f
-
-        # Move the generated file and replace it with a symlink
-        mv -f $f tags/$TAG/$f && ln -s tags/$TAG/$f $f
-    done
+    # This will process ips.txt to generate two files: ips.pdf and ips.html
+    (
+        cd tags/$TAG
+        ../../lib/ips-txt-to-html.py settings.yaml
+    )
 
     info "Cards created. You can view them with:"
-    info "xdg-open ips.html ips.pdf (on Linux)"
-    info "open ips.html ips.pdf (on MacOS)"
+    info "xdg-open tags/$TAG/ips.html tags/$TAG/ips.pdf (on Linux)"
+    info "open tags/$TAG/ips.html (on macOS)"
 }
 
 _cmd deploy "Install Docker on a bunch of running VMs"
 _cmd_deploy() {
     TAG=$1
-    SETTINGS=$2
-    need_tag $TAG
-    need_settings $SETTINGS
-    link_tag $TAG
-    count=$(wc -l ips.txt)
+    need_tag
 
     # wait until all hosts are reachable before trying to deploy
     info "Trying to reach $TAG instances..."
-    while ! tag_is_reachable $TAG; do
+    while ! tag_is_reachable; do
         >/dev/stderr echo -n "."
         sleep 2
     done
     >/dev/stderr echo ""
 
+    echo deploying > tags/$TAG/status
     sep "Deploying tag $TAG"
-    pssh -I tee /tmp/settings.yaml <$SETTINGS
+
+    # Wait for cloudinit to be done
+    pssh "
+    while [ ! -f /var/lib/cloud/instance/boot-finished ]; do
+        sleep 1
+    done"
+
+    # Copy settings and install Python YAML parser
+    pssh -I tee /tmp/settings.yaml <tags/$TAG/settings.yaml
     pssh "
     sudo apt-get update &&
-    sudo apt-get install -y python-setuptools &&
-    sudo easy_install pyyaml"
+    sudo apt-get install -y python-yaml"
 
     # Copy postprep.py to the remote machines, and execute it, feeding it the list of IP addresses
     pssh -I tee /tmp/postprep.py <lib/postprep.py
-    pssh --timeout 900 --send-input "python /tmp/postprep.py >>/tmp/pp.out 2>>/tmp/pp.err" <ips.txt
+    pssh --timeout 900 --send-input "python /tmp/postprep.py >>/tmp/pp.out 2>>/tmp/pp.err" <tags/$TAG/ips.txt
 
     # Install docker-prompt script
     pssh -I sudo tee /usr/local/bin/docker-prompt <lib/docker-prompt
     pssh sudo chmod +x /usr/local/bin/docker-prompt
 
-    # If /home/docker/.ssh/id_rsa doesn't exist, copy it from node1
+    # If /home/docker/.ssh/id_rsa doesn't exist, copy it from the first node
     pssh "
     sudo -u docker [ -f /home/docker/.ssh/id_rsa ] ||
-    ssh -o StrictHostKeyChecking=no node1 sudo -u docker tar -C /home/docker -cvf- .ssh |
+    ssh -o StrictHostKeyChecking=no \$(cat /etc/name_of_first_node) sudo -u docker tar -C /home/docker -cvf- .ssh |
     sudo -u docker tar -C /home/docker -xf-"
 
     # if 'docker@' doesn't appear in /home/docker/.ssh/authorized_keys, copy it there
@@ -104,52 +86,100 @@ _cmd_deploy() {
     cat /home/docker/.ssh/id_rsa.pub |
     sudo -u docker tee -a /home/docker/.ssh/authorized_keys"
 
-    # On node1, create and deploy TLS certs using Docker Machine
+    # On the first node, create and deploy TLS certs using Docker Machine
     # (Currently disabled.)
     true || pssh "
-    if grep -q node1 /tmp/node; then
-        grep ' node' /etc/hosts | 
+    if i_am_first_node; then
+        grep '[0-9]\$' /etc/hosts |
         xargs -n2 sudo -H -u docker \
         docker-machine create -d generic --generic-ssh-user docker --generic-ip-address
     fi"
 
     sep "Deployed tag $TAG"
+    echo deployed > tags/$TAG/status
     info "You may want to run one of the following commands:"
     info "$0 kube $TAG"
     info "$0 pull_images $TAG"
-    info "$0 cards $TAG $SETTINGS"
+    info "$0 cards $TAG"
+}
+
+_cmd disabledocker "Stop Docker Engine and don't restart it automatically"
+_cmd_disabledocker() {
+    TAG=$1
+    need_tag
+
+    pssh "sudo systemctl disable docker.service"
+    pssh "sudo systemctl disable docker.socket"
+    pssh "sudo systemctl stop docker"
+}
+
+_cmd kubebins "Install Kubernetes and CNI binaries but don't start anything"
+_cmd_kubebins() {
+    TAG=$1
+    need_tag
+
+    pssh --timeout 300 "
+    set -e
+    cd /usr/local/bin
+    if ! [ -x etcd ]; then
+        curl -L https://github.com/etcd-io/etcd/releases/download/v3.3.10/etcd-v3.3.10-linux-amd64.tar.gz \
+        | sudo tar --strip-components=1 --wildcards -zx '*/etcd' '*/etcdctl'
+    fi
+    if ! [ -x hyperkube ]; then
+        curl -L https://dl.k8s.io/v1.14.1/kubernetes-server-linux-amd64.tar.gz \
+        | sudo tar --strip-components=3 -zx kubernetes/server/bin/hyperkube
+    fi
+    if ! [ -x kubelet ]; then
+        for BINARY in kubectl kube-apiserver kube-scheduler kube-controller-manager kubelet kube-proxy;
+        do
+            sudo ln -s hyperkube \$BINARY
+        done
+    fi
+    sudo mkdir -p /opt/cni/bin
+    cd /opt/cni/bin
+    if ! [ -x bridge ]; then
+        curl -L https://github.com/containernetworking/plugins/releases/download/v0.7.5/cni-plugins-amd64-v0.7.5.tgz \
+        | sudo tar -zx
+    fi
+    "
 }
 
 _cmd kube "Setup kubernetes clusters with kubeadm (must be run AFTER deploy)"
 _cmd_kube() {
+    TAG=$1
+    need_tag
+
+    # Optional version, e.g. 1.13.5
+    KUBEVERSION=$2
+    if [ "$KUBEVERSION" ]; then
+        EXTRA_KUBELET="=$KUBEVERSION-00"
+        EXTRA_KUBEADM="--kubernetes-version=v$KUBEVERSION"
+    else
+        EXTRA_KUBELET=""
+        EXTRA_KUBEADM=""
+    fi
 
     # Install packages
-    pssh "
+    pssh --timeout 200 "
     curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg |
     sudo apt-key add - &&
     echo deb http://apt.kubernetes.io/ kubernetes-xenial main |
     sudo tee /etc/apt/sources.list.d/kubernetes.list"
-    pssh "
+    pssh --timeout 200 "
     sudo apt-get update -q &&
-    sudo apt-get install -qy kubelet kubeadm kubectl
+    sudo apt-get install -qy kubelet$EXTRA_KUBELET kubeadm kubectl &&
     kubectl completion bash | sudo tee /etc/bash_completion.d/kubectl"
 
-    # Work around https://github.com/kubernetes/kubernetes/issues/53356
-    pssh "
-    if [ ! -f /etc/kubernetes/kubelet.conf ]; then
-        sudo systemctl stop kubelet
-        sudo rm -rf /var/lib/kubelet/pki
-    fi"
-
     # Initialize kube master
-    pssh "
-    if grep -q node1 /tmp/node && [ ! -f /etc/kubernetes/admin.conf ]; then
-        sudo kubeadm init
+    pssh --timeout 200 "
+    if i_am_first_node && [ ! -f /etc/kubernetes/admin.conf ]; then
+        kubeadm token generate > /tmp/token &&
+	sudo kubeadm init $EXTRA_KUBEADM --token \$(cat /tmp/token) --apiserver-cert-extra-sans \$(cat /tmp/ipv4)
     fi"
 
     # Put kubeconfig in ubuntu's and docker's accounts
     pssh "
-    if grep -q node1 /tmp/node; then
+    if i_am_first_node; then
         sudo mkdir -p \$HOME/.kube /home/docker/.kube &&
         sudo cp /etc/kubernetes/admin.conf \$HOME/.kube/config &&
         sudo cp /etc/kubernetes/admin.conf /home/docker/.kube/config &&
@@ -157,33 +187,104 @@ _cmd_kube() {
         sudo chown -R docker /home/docker/.kube
     fi"
 
-    # Get bootstrap token
-    pssh "
-    if grep -q node1 /tmp/node; then
-        TOKEN_NAME=\$(kubectl -n kube-system get secret -o name | grep bootstrap-token)
-        TOKEN_ID=\$(kubectl -n kube-system get \$TOKEN_NAME -o go-template --template '{{ index .data \"token-id\" }}' | base64 -d)
-        TOKEN_SECRET=\$(kubectl -n kube-system get \$TOKEN_NAME -o go-template --template '{{ index .data \"token-secret\" }}' | base64 -d)
-        echo \$TOKEN_ID.\$TOKEN_SECRET >/tmp/token
-    fi"
-
     # Install weave as the pod network
     pssh "
-    if grep -q node1 /tmp/node; then
-        kubever=\$(kubectl version | base64 | tr -d '\n')
+    if i_am_first_node; then
+        kubever=\$(kubectl version | base64 | tr -d '\n') &&
         kubectl apply -f https://cloud.weave.works/k8s/net?k8s-version=\$kubever
     fi"
 
     # Join the other nodes to the cluster
+    pssh --timeout 200 "
+    if ! i_am_first_node && [ ! -f /etc/kubernetes/kubelet.conf ]; then
+        FIRSTNODE=\$(cat /etc/name_of_first_node) &&
+        TOKEN=\$(ssh -o StrictHostKeyChecking=no \$FIRSTNODE cat /tmp/token) &&
+        sudo kubeadm join --discovery-token-unsafe-skip-ca-verification --token \$TOKEN \$FIRSTNODE:6443
+    fi"
+
+    # Install metrics server
     pssh "
-    if ! grep -q node1 /tmp/node && [ ! -f /etc/kubernetes/kubelet.conf ]; then
-        TOKEN=\$(ssh -o StrictHostKeyChecking=no node1 cat /tmp/token)
-        sudo kubeadm join --token \$TOKEN node1:6443
+    if i_am_first_node; then
+	kubectl apply -f https://raw.githubusercontent.com/jpetazzo/container.training/master/k8s/metrics-server.yaml
+    fi"
+
+    # Install kubectx and kubens
+    pssh "
+    [ -d kubectx ] || git clone https://github.com/ahmetb/kubectx &&
+    sudo ln -sf /home/ubuntu/kubectx/kubectx /usr/local/bin/kctx &&
+    sudo ln -sf /home/ubuntu/kubectx/kubens /usr/local/bin/kns &&
+    sudo cp /home/ubuntu/kubectx/completion/*.bash /etc/bash_completion.d &&
+    [ -d kube-ps1 ] || git clone https://github.com/jonmosco/kube-ps1 &&
+    sudo -u docker sed -i s/docker-prompt/kube_ps1/ /home/docker/.bashrc &&
+    sudo -u docker tee -a /home/docker/.bashrc <<EOF
+. /home/ubuntu/kube-ps1/kube-ps1.sh
+KUBE_PS1_PREFIX=""
+KUBE_PS1_SUFFIX=""
+KUBE_PS1_SYMBOL_ENABLE="false"
+KUBE_PS1_CTX_COLOR="green"
+KUBE_PS1_NS_COLOR="green"
+EOF"
+
+    # Install stern
+    pssh "
+    if [ ! -x /usr/local/bin/stern ]; then
+        ##VERSION##
+        sudo curl -L -o /usr/local/bin/stern https://github.com/wercker/stern/releases/download/1.11.0/stern_linux_amd64 &&
+        sudo chmod +x /usr/local/bin/stern &&
+        stern --completion bash | sudo tee /etc/bash_completion.d/stern
+    fi"
+
+    # Install helm
+    pssh "
+    if [ ! -x /usr/local/bin/helm ]; then
+        curl https://raw.githubusercontent.com/kubernetes/helm/master/scripts/get | sudo bash &&
+        helm completion bash | sudo tee /etc/bash_completion.d/helm
+    fi"
+
+    # Install ship
+    pssh "
+    if [ ! -x /usr/local/bin/ship ]; then
+        curl -L https://github.com/replicatedhq/ship/releases/download/v0.40.0/ship_0.40.0_linux_amd64.tar.gz |
+             sudo tar -C /usr/local/bin -zx ship
+    fi"
+
+    # Install the AWS IAM authenticator
+    pssh "
+    if [ ! -x /usr/local/bin/aws-iam-authenticator ]; then
+	##VERSION##
+        sudo curl -o /usr/local/bin/aws-iam-authenticator https://amazon-eks.s3-us-west-2.amazonaws.com/1.12.7/2019-03-27/bin/linux/amd64/aws-iam-authenticator
+	sudo chmod +x /usr/local/bin/aws-iam-authenticator
     fi"
 
     sep "Done"
 }
 
-_cmd ids "List the instance IDs belonging to a given tag or token"
+_cmd kubereset "Wipe out Kubernetes configuration on all nodes"
+_cmd_kubereset() {
+    TAG=$1
+    need_tag
+
+    pssh "sudo kubeadm reset --force"
+}
+
+_cmd kubetest "Check that all nodes are reporting as Ready"
+_cmd_kubetest() {
+    TAG=$1
+    need_tag
+
+    # There are way too many backslashes in the command below.
+    # Feel free to make that better ♥
+    pssh "
+    set -e
+    if i_am_first_node; then
+      which kubectl
+      for NODE in \$(awk /[0-9]\$/\ {print\ \\\$2} /etc/hosts); do
+        echo \$NODE ; kubectl get nodes | grep -w \$NODE | grep -w Ready
+      done
+    fi"
+}
+
+_cmd ids "(FIXME) List the instance IDs belonging to a given tag or token"
 _cmd_ids() {
     TAG=$1
     need_tag $TAG
@@ -196,244 +297,289 @@ _cmd_ids() {
     aws_get_instance_ids_by_client_token $TAG
 }
 
-_cmd ips "List the IP addresses of the VMs for a given tag or token"
-_cmd_ips() {
-    TAG=$1
-    need_tag $TAG
-    mkdir -p tags/$TAG
-    aws_get_instance_ips_by_tag $TAG | tee tags/$TAG/ips.txt
-    link_tag $TAG
-}
-
-_cmd list "List available batches in the current region"
+_cmd list "List available groups for a given infrastructure"
 _cmd_list() {
-    info "Listing batches in region $AWS_DEFAULT_REGION:"
-    aws_display_tags
+    need_infra $1
+    infra_list
 }
 
-_cmd status "List instance status for a given batch"
-_cmd_status() {
-    info "Using region $AWS_DEFAULT_REGION."
+_cmd listall "List VMs running on all configured infrastructures"
+_cmd_listall() {
+    for infra in infra/*; do
+        case $infra in
+        infra/example.*)
+            ;;
+        *)
+            info "Listing infrastructure $infra:"
+            need_infra $infra
+            infra_list
+            ;;
+        esac
+    done
+}
+
+_cmd ping "Ping VMs in a given tag, to check that they have network access"
+_cmd_ping() {
     TAG=$1
-    need_tag $TAG
-    describe_tag $TAG
-    tag_is_reachable $TAG
-    info "You may be interested in running one of the following commands:"
-    info "$0 ips $TAG"
-    info "$0 deploy $TAG <settings/somefile.yaml>"
+    need_tag
+
+    fping < tags/$TAG/ips.txt
+}
+
+_cmd netfix "Disable GRO and run a pinger job on the VMs"
+_cmd_netfix () {
+    TAG=$1
+    need_tag
+
+    pssh "
+    sudo ethtool -K ens3 gro off
+    sudo tee /root/pinger.service <<EOF
+[Unit]
+Description=pinger
+
+[Install]
+WantedBy=multi-user.target
+
+[Service]
+WorkingDirectory=/
+ExecStart=/bin/ping -w60 1.1
+User=nobody
+Group=nogroup
+Restart=always
+EOF
+    sudo systemctl enable /root/pinger.service
+    sudo systemctl start pinger"
 }
 
 _cmd opensg "Open the default security group to ALL ingress traffic"
 _cmd_opensg() {
-    aws ec2 authorize-security-group-ingress \
-        --group-name default \
-        --protocol icmp \
-        --port -1 \
-        --cidr 0.0.0.0/0
+    need_infra $1
+    infra_opensg
+}
 
-    aws ec2 authorize-security-group-ingress \
-        --group-name default \
-        --protocol udp \
-        --port 0-65535 \
-        --cidr 0.0.0.0/0
+_cmd disableaddrchecks "Disable source/destination IP address checks"
+_cmd_disableaddrchecks() {
+    TAG=$1
+    need_tag
 
-    aws ec2 authorize-security-group-ingress \
-        --group-name default \
-        --protocol tcp \
-        --port 0-65535 \
-        --cidr 0.0.0.0/0
+    infra_disableaddrchecks
+}
+
+_cmd pssh "Run an arbitrary command on all nodes"
+_cmd_pssh() {
+    TAG=$1
+    need_tag
+    shift
+
+    pssh "$@"
 }
 
 _cmd pull_images "Pre-pull a bunch of Docker images"
 _cmd_pull_images() {
     TAG=$1
-    need_tag $TAG
-    pull_tag $TAG
+    need_tag
+    pull_tag
 }
 
-_cmd retag "Apply a new tag to a batch of VMs"
+_cmd quotas "Check our infrastructure quotas (max instances)"
+_cmd_quotas() {
+    need_infra $1
+    infra_quotas
+}
+
+_cmd retag "(FIXME) Apply a new tag to a group of VMs"
 _cmd_retag() {
     OLDTAG=$1
     NEWTAG=$2
-    need_tag $OLDTAG
+    TAG=$OLDTAG
+    need_tag
     if [[ -z "$NEWTAG" ]]; then
         die "You must specify a new tag to apply."
     fi
     aws_tag_instances $OLDTAG $NEWTAG
 }
 
-_cmd start "Start a batch of VMs"
+_cmd ssh "Open an SSH session to the first node of a tag"
+_cmd_ssh() {
+    TAG=$1
+    need_tag
+    IP=$(head -1 tags/$TAG/ips.txt)
+    info "Logging into $IP"
+    ssh docker@$IP
+}
+
+_cmd start "Start a group of VMs"
 _cmd_start() {
-    # Number of instances to create
-    COUNT=$1
-    # Optional settings file (to carry on with deployment)
-    SETTINGS=$2
+    while [ ! -z "$*" ]; do
+        case "$1" in
+        --infra) INFRA=$2; shift 2;;
+        --settings) SETTINGS=$2; shift 2;;
+        --count) COUNT=$2; shift 2;;
+        --tag) TAG=$2; shift 2;;
+        *) die "Unrecognized parameter: $1."
+        esac
+    done
 
+    if [ -z "$INFRA" ]; then
+        die "Please add --infra flag to specify which infrastructure file to use."
+    fi
+    if [ -z "$SETTINGS" ]; then
+        die "Please add --settings flag to specify which settings file to use."
+    fi
     if [ -z "$COUNT" ]; then
-        die "Indicate number of instances to start."
+        COUNT=$(awk '/^clustersize:/ {print $2}' $SETTINGS)
+        warning "No --count option was specified. Using value from settings file ($COUNT)."
     fi
 
-    # Print our AWS username, to ease the pain of credential-juggling
-    greet
+    # Check that the specified settings and infrastructure are valid.
+    need_settings $SETTINGS
+    need_infra $INFRA
 
-    # Upload our SSH keys to AWS if needed, to be added to each VM's authorized_keys
-    key_name=$(sync_keys)
-
-    AMI=$(_cmd_ami)    # Retrieve the AWS image ID
-    TOKEN=$(get_token) # generate a timestamp token for this batch of VMs
-    AWS_KEY_NAME=$(make_key_name)
-
-    sep "Starting instances"
-    info "         Count: $COUNT"
-    info "        Region: $AWS_DEFAULT_REGION"
-    info "     Token/tag: $TOKEN"
-    info "           AMI: $AMI"
-    info "      Key name: $AWS_KEY_NAME"
-    result=$(aws ec2 run-instances \
-        --key-name $AWS_KEY_NAME \
-        --count $COUNT \
-        --instance-type t2.medium \
-        --client-token $TOKEN \
-        --image-id $AMI)
-    reservation_id=$(echo "$result" | head -1 | awk '{print $2}')
-    info "Reservation ID: $reservation_id"
-    sep
-
-    # if instance creation succeeded, we should have some IDs
-    IDS=$(aws_get_instance_ids_by_client_token $TOKEN)
-    if [ -z "$IDS" ]; then
-        die "Instance creation failed."
+    if [ -z "$TAG" ]; then
+        TAG=$(make_tag)
     fi
+    mkdir -p tags/$TAG
+    ln -s ../../$INFRA tags/$TAG/infra.sh
+    ln -s ../../$SETTINGS tags/$TAG/settings.yaml
+    echo creating > tags/$TAG/status
 
-    # Tag these new instances with a tag that is the same as the token
-    TAG=$TOKEN
-    aws_tag_instances $TOKEN $TAG
-
-    wait_until_tag_is_running $TAG $COUNT
-
+    infra_start $COUNT
     sep
     info "Successfully created $COUNT instances with tag $TAG"
     sep
+    echo created > tags/$TAG/status
 
-    mkdir -p tags/$TAG
-    IPS=$(aws_get_instance_ips_by_tag $TAG)
-    echo "$IPS" >tags/$TAG/ips.txt
-    link_tag $TAG
-    if [ -n "$SETTINGS" ]; then
-        _cmd_deploy $TAG $SETTINGS
-    else
-        info "To deploy or kill these instances, run one of the following:"
-        info "$0 deploy $TAG <settings/somefile.yaml>"
-        info "$0 stop $TAG"
-    fi
-}
-
-_cmd ec2quotas "Check our EC2 quotas (max instances)"
-_cmd_ec2quotas() {
-    greet
-
-    max_instances=$(aws ec2 describe-account-attributes \
-        --attribute-names max-instances \
-        --query 'AccountAttributes[*][AttributeValues]')
-    info "In the current region ($AWS_DEFAULT_REGION) you can deploy up to $max_instances instances."
-
-    # Print list of AWS EC2 regions, highlighting ours ($AWS_DEFAULT_REGION) in the list
-    # If our $AWS_DEFAULT_REGION is not valid, the error message will be pretty descriptive:
-    # Could not connect to the endpoint URL: "https://ec2.foo.amazonaws.com/"
-    info "Available regions:"
-    aws ec2 describe-regions | awk '{print $3}' | grep --color=auto $AWS_DEFAULT_REGION -C50
+    info "To deploy Docker on these instances, you can run:"
+    info "$0 deploy $TAG"
+    info "To terminate these instances, you can run:"
+    info "$0 stop $TAG"
 }
 
 _cmd stop "Stop (terminate, shutdown, kill, remove, destroy...) instances"
 _cmd_stop() {
     TAG=$1
-    need_tag $TAG
-    aws_kill_instances_by_tag $TAG
+    need_tag
+    infra_stop
+    echo stopped > tags/$TAG/status
 }
 
-_cmd test "Run tests (pre-flight checks) on a batch of VMs"
+_cmd tags "List groups of VMs known locally"
+_cmd_tags() {
+    (
+        cd tags
+        echo "[#] [Status] [Tag] [Infra]" \
+           | awk '{ printf "%-7s %-12s %-25s %-25s\n", $1, $2, $3, $4}'
+        for tag in *; do
+            if [ -f $tag/ips.txt ]; then
+                count="$(wc -l < $tag/ips.txt)"
+            else
+                count="?"
+            fi
+            if [ -f $tag/status ]; then
+                status="$(cat $tag/status)"
+            else
+                status="?"
+            fi
+            if [ -f $tag/infra.sh ]; then
+                infra="$(basename $(readlink $tag/infra.sh))"
+            else
+                infra="?"
+            fi
+            echo "$count $status $tag $infra" \
+               | awk '{ printf "%-7s %-12s %-25s %-25s\n", $1, $2, $3, $4}'
+        done
+    )
+}
+
+_cmd test "Run tests (pre-flight checks) on a group of VMs"
 _cmd_test() {
     TAG=$1
-    need_tag $TAG
-    test_tag $TAG
+    need_tag
+    test_tag
 }
 
-###
+_cmd helmprom "Install Helm and Prometheus"
+_cmd_helmprom() {
+    TAG=$1
+    need_tag
+    pssh "
+    if i_am_first_node; then
+        kubectl -n kube-system get serviceaccount helm ||
+            kubectl -n kube-system create serviceaccount helm
+        sudo -u docker -H helm init --service-account helm
+        kubectl get clusterrolebinding helm-can-do-everything ||
+            kubectl create clusterrolebinding helm-can-do-everything \
+                --clusterrole=cluster-admin \
+                --serviceaccount=kube-system:helm
+        sudo -u docker -H helm upgrade --install prometheus stable/prometheus \
+            --namespace kube-system \
+            --set server.service.type=NodePort \
+            --set server.service.nodePort=30090 \
+            --set server.persistentVolume.enabled=false \
+            --set alertmanager.enabled=false
+    fi"
+}
+
+# Sometimes, weave fails to come up on some nodes.
+# Symptom: the pods on a node are unreachable (they don't even ping).
+# Remedy: wipe out Weave state and delete weave pod on that node.
+# Specifically, identify the weave pod that is defective, then:
+# kubectl -n kube-system exec weave-net-XXXXX -c weave rm /weavedb/weave-netdata.db
+# kubectl -n kube-system delete pod weave-net-XXXXX
+_cmd weavetest "Check that weave seems properly setup"
+_cmd_weavetest() {
+    TAG=$1
+    need_tag
+    pssh "
+    kubectl -n kube-system get pods -o name | grep weave | cut -d/ -f2 |
+    xargs -I POD kubectl -n kube-system exec POD -c weave -- \
+    sh -c \"./weave --local status | grep Connections | grep -q ' 1 failed' || ! echo POD \""
+}
 
 greet() {
     IAMUSER=$(aws iam get-user --query 'User.UserName')
     info "Hello! You seem to be UNIX user $USER, and IAM user $IAMUSER."
 }
 
-link_tag() {
-    TAG=$1
-    need_tag $TAG
-    IPS_FILE=tags/$TAG/ips.txt
-    need_ips_file $IPS_FILE
-    ln -sf $IPS_FILE ips.txt
-}
-
 pull_tag() {
-    TAG=$1
-    need_tag $TAG
-    link_tag $TAG
-    if [ ! -s $IPS_FILE ]; then
-        die "Nonexistent or empty IPs file $IPS_FILE."
-    fi
-
     # Pre-pull a bunch of images
     pssh --timeout 900 'for I in \
-            debian:latest \
-            ubuntu:latest \
-            fedora:latest \
-            centos:latest \
-            postgres \
-            redis \
-            training/namer \
-            nathanleclaire/redisonrails; do
+        debian:latest \
+        ubuntu:latest \
+        fedora:latest \
+        centos:latest \
+        elasticsearch:2 \
+        postgres \
+        redis \
+        alpine \
+        registry \
+        nicolaka/netshoot \
+        jpetazzo/trainingwheels \
+        golang \
+        training/namer \
+        dockercoins/hasher \
+        dockercoins/rng \
+        dockercoins/webui \
+        dockercoins/worker \
+        logstash \
+        prom/node-exporter \
+        google/cadvisor \
+        dockersamples/visualizer \
+        nathanleclaire/redisonrails; do
         sudo -u docker docker pull $I
     done'
 
     info "Finished pulling images for $TAG."
-    info "You may now want to run:"
-    info "$0 cards $TAG <settings/somefile.yaml>"
-}
-
-wait_until_tag_is_running() {
-    max_retry=50
-    TAG=$1
-    COUNT=$2
-    i=0
-    done_count=0
-    while [[ $done_count -lt $COUNT ]]; do
-        let "i += 1"
-        info "$(printf "%d/%d instances online" $done_count $COUNT)"
-        done_count=$(aws ec2 describe-instances \
-            --filters "Name=instance-state-name,Values=running" \
-            "Name=tag:Name,Values=$TAG" \
-            --query "Reservations[*].Instances[*].State.Name" \
-            | tr "\t" "\n" \
-            | wc -l)
-
-        if [[ $i -gt $max_retry ]]; then
-            die "Timed out while waiting for instance creation (after $max_retry retries)"
-        fi
-        sleep 1
-    done
 }
 
 tag_is_reachable() {
-    TAG=$1
-    need_tag $TAG
-    link_tag $TAG
     pssh -t 5 true 2>&1 >/dev/null
 }
 
 test_tag() {
     ips_file=tags/$TAG/ips.txt
     info "Picking a random IP address in $ips_file to run tests."
-    n=$((1 + $RANDOM % $(wc -l <$ips_file)))
-    ip=$(head -n $n $ips_file | tail -n 1)
+    ip=$(shuf -n1 $ips_file)
     test_vm $ip
     info "Tests complete."
 }
@@ -447,8 +593,8 @@ test_vm() {
     for cmd in "hostname" \
         "whoami" \
         "hostname -i" \
-        "cat /tmp/node" \
-        "cat /tmp/ipv4" \
+        "ls -l /usr/local/bin/i_am_first_node" \
+        "grep . /etc/name_of_first_node /etc/ipv4_of_first_node" \
         "cat /etc/hosts" \
         "hostnamectl status" \
         "docker version | grep Version -B1" \
@@ -509,17 +655,9 @@ sync_keys() {
     fi
 }
 
-get_token() {
+make_tag() {
     if [ -z $USER ]; then
         export USER=anonymous
     fi
     date +%Y-%m-%d-%H-%M-$USER
 }
-
-describe_tag() {
-    # Display instance details and reachability/status information
-    TAG=$1
-    need_tag $TAG
-    aws_display_instances_by_tag $TAG
-    aws_display_instance_statuses_by_tag $TAG
-}

prepare-vms/lib/infra.sh

@@ -0,0 +1,30 @@
+# Default stub functions for infrastructure libraries.
+# When loading an infrastructure library, these functions will be overridden.
+
+infra_list() {
+	warning "infra_list is unsupported on $INFRACLASS."
+}
+
+infra_quotas() {
+	warning "infra_quotas is unsupported on $INFRACLASS."
+}
+
+infra_start() {
+	warning "infra_start is unsupported on $INFRACLASS."
+}
+
+infra_stop() {
+	warning "infra_stop is unsupported on $INFRACLASS."
+}
+
+infra_quotas() {
+	warning "infra_quotas is unsupported on $INFRACLASS."
+}
+
+infra_opensg() {
+	warning "infra_opensg is unsupported on $INFRACLASS."
+}
+
+infra_disableaddrchecks() {
+	warning "infra_disableaddrchecks is unsupported on $INFRACLASS."
+}

prepare-vms/lib/infra/aws.sh

@@ -0,0 +1,216 @@
+infra_list() {
+    aws_display_tags
+}
+
+infra_quotas() {
+    greet
+
+    max_instances=$(aws ec2 describe-account-attributes \
+        --attribute-names max-instances \
+        --query 'AccountAttributes[*][AttributeValues]')
+    info "In the current region ($AWS_DEFAULT_REGION) you can deploy up to $max_instances instances."
+
+    # Print list of AWS EC2 regions, highlighting ours ($AWS_DEFAULT_REGION) in the list
+    # If our $AWS_DEFAULT_REGION is not valid, the error message will be pretty descriptive:
+    # Could not connect to the endpoint URL: "https://ec2.foo.amazonaws.com/"
+    info "Available regions:"
+    aws ec2 describe-regions | awk '{print $3}' | grep --color=auto $AWS_DEFAULT_REGION -C50
+}
+
+infra_start() {
+    COUNT=$1
+
+    # Print our AWS username, to ease the pain of credential-juggling
+    greet
+
+    # Upload our SSH keys to AWS if needed, to be added to each VM's authorized_keys
+    key_name=$(sync_keys)
+
+    AMI=$(aws_get_ami)    # Retrieve the AWS image ID
+    if [ -z "$AMI" ]; then
+        die "I could not find which AMI to use in this region. Try another region?"
+    fi
+    AWS_KEY_NAME=$(make_key_name)
+    AWS_INSTANCE_TYPE=${AWS_INSTANCE_TYPE-t3a.medium}
+
+    sep "Starting instances"
+    info "         Count: $COUNT"
+    info "        Region: $AWS_DEFAULT_REGION"
+    info "     Token/tag: $TAG"
+    info "           AMI: $AMI"
+    info "      Key name: $AWS_KEY_NAME"
+    info " Instance type: $AWS_INSTANCE_TYPE"
+    result=$(aws ec2 run-instances \
+        --key-name $AWS_KEY_NAME \
+        --count $COUNT \
+        --instance-type $AWS_INSTANCE_TYPE \
+        --client-token $TAG \
+        --block-device-mapping 'DeviceName=/dev/sda1,Ebs={VolumeSize=20}' \
+        --image-id $AMI)
+    reservation_id=$(echo "$result" | head -1 | awk '{print $2}')
+    info "Reservation ID: $reservation_id"
+    sep
+
+    # if instance creation succeeded, we should have some IDs
+    IDS=$(aws_get_instance_ids_by_client_token $TAG)
+    if [ -z "$IDS" ]; then
+        die "Instance creation failed."
+    fi
+
+    # Tag these new instances with a tag that is the same as the token
+    aws_tag_instances $TAG $TAG
+
+    # Wait until EC2 API tells us that the instances are running
+    wait_until_tag_is_running $TAG $COUNT
+
+    aws_get_instance_ips_by_tag $TAG > tags/$TAG/ips.txt
+}
+
+infra_stop() {
+    aws_kill_instances_by_tag
+}
+
+infra_opensg() {
+    aws ec2 authorize-security-group-ingress \
+        --group-name default \
+        --protocol icmp \
+        --port -1 \
+        --cidr 0.0.0.0/0
+
+    aws ec2 authorize-security-group-ingress \
+        --group-name default \
+        --protocol udp \
+        --port 0-65535 \
+        --cidr 0.0.0.0/0
+
+    aws ec2 authorize-security-group-ingress \
+        --group-name default \
+        --protocol tcp \
+        --port 0-65535 \
+        --cidr 0.0.0.0/0
+}
+
+infra_disableaddrchecks() {
+    IDS=$(aws_get_instance_ids_by_tag $TAG)
+    for ID in $IDS; do
+        info "Disabling source/destination IP checks on: $ID"
+        aws ec2 modify-instance-attribute --source-dest-check "{\"Value\": false}" --instance-id $ID
+    done
+}
+
+wait_until_tag_is_running() {
+    max_retry=100
+    i=0
+    done_count=0
+    while [[ $done_count -lt $COUNT ]]; do
+        let "i += 1"
+        info "$(printf "%d/%d instances online" $done_count $COUNT)"
+        done_count=$(aws ec2 describe-instances \
+            --filters "Name=tag:Name,Values=$TAG" \
+            "Name=instance-state-name,Values=running" \
+            --query "length(Reservations[].Instances[])")
+        if [[ $i -gt $max_retry ]]; then
+            die "Timed out while waiting for instance creation (after $max_retry retries)"
+        fi
+        sleep 1
+    done
+}
+
+aws_display_tags() {
+    # Print all "Name" tags in our region with their instance count
+    echo "[#] [Status] [Token] [Tag]" \
+        | awk '{ printf "%-7s %-12s %-25s %-25s\n", $1, $2, $3, $4}'
+    aws ec2 describe-instances \
+        --query "Reservations[*].Instances[*].[State.Name,ClientToken,Tags[0].Value]" \
+        | tr -d "\r" \
+        | uniq -c \
+        | sort -k 3 \
+        | awk '{ printf "%-7s %-12s %-25s %-25s\n", $1, $2, $3, $4}'
+}
+
+aws_get_tokens() {
+    aws ec2 describe-instances --output text \
+        --query 'Reservations[*].Instances[*].[ClientToken]' \
+        | sort -u
+}
+
+aws_display_instance_statuses_by_tag() {
+    IDS=$(aws ec2 describe-instances \
+        --filters "Name=tag:Name,Values=$TAG" \
+        --query "Reservations[*].Instances[*].InstanceId" | tr '\t' ' ')
+
+    aws ec2 describe-instance-status \
+        --instance-ids $IDS \
+        --query "InstanceStatuses[*].{ID:InstanceId,InstanceState:InstanceState.Name,InstanceStatus:InstanceStatus.Status,SystemStatus:SystemStatus.Status,Reachability:InstanceStatus.Status}" \
+        --output table
+}
+
+aws_display_instances_by_tag() {
+    result=$(aws ec2 describe-instances --output table \
+        --filter "Name=tag:Name,Values=$TAG" \
+        --query "Reservations[*].Instances[*].[ \
+                        InstanceId, \
+                        State.Name, \
+                        Tags[0].Value, \
+                        PublicIpAddress, \
+                        InstanceType \
+                        ]"
+    )
+    if [[ -z $result ]]; then
+        die "No instances found with tag $TAG in region $AWS_DEFAULT_REGION."
+    else
+        echo "$result"
+    fi
+}
+
+aws_get_instance_ids_by_filter() {
+    FILTER=$1
+    aws ec2 describe-instances --filters $FILTER \
+        --query Reservations[*].Instances[*].InstanceId \
+        --output text | tr "\t" "\n" | tr -d "\r"
+}
+
+aws_get_instance_ids_by_client_token() {
+    TOKEN=$1
+    aws_get_instance_ids_by_filter Name=client-token,Values=$TOKEN
+}
+
+aws_get_instance_ids_by_tag() {
+    aws_get_instance_ids_by_filter Name=tag:Name,Values=$TAG
+}
+
+aws_get_instance_ips_by_tag() {
+    aws ec2 describe-instances --filter "Name=tag:Name,Values=$TAG" \
+        --output text \
+        --query "Reservations[*].Instances[*].PublicIpAddress" \
+        | tr "\t" "\n" \
+        | sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4 # sort IPs
+}
+
+aws_kill_instances_by_tag() {
+    IDS=$(aws_get_instance_ids_by_tag $TAG)
+    if [ -z "$IDS" ]; then
+        die "Invalid tag."
+    fi
+
+    info "Deleting instances with tag $TAG."
+
+    aws ec2 terminate-instances --instance-ids $IDS \
+        | grep ^TERMINATINGINSTANCES
+
+    info "Deleted instances with tag $TAG."
+}
+
+aws_tag_instances() {
+    OLD_TAG_OR_TOKEN=$1
+    NEW_TAG=$2
+    IDS=$(aws_get_instance_ids_by_client_token $OLD_TAG_OR_TOKEN)
+    [[ -n "$IDS" ]] && aws ec2 create-tags --tag Key=Name,Value=$NEW_TAG --resources $IDS >/dev/null
+    IDS=$(aws_get_instance_ids_by_tag $OLD_TAG_OR_TOKEN)
+    [[ -n "$IDS" ]] && aws ec2 create-tags --tag Key=Name,Value=$NEW_TAG --resources $IDS >/dev/null
+}
+
+aws_get_ami() {
+    ##VERSION##
+    find_ubuntu_ami -r $AWS_DEFAULT_REGION -a amd64 -v 18.04 -t hvm:ebs -N -q
+}

prepare-vms/lib/infra/generic.sh

@@ -0,0 +1,8 @@
+infra_start() {
+	COUNT=$1
+	info "You should now run your provisioning commands for $COUNT machines."
+	info "Note: no machines have been automatically created!"
+	info "Once done, put the list of IP addresses in tags/$TAG/ips.txt"
+	info "(one IP address per line, without any comments or extra lines)."
+	touch tags/$TAG/ips.txt
+}

prepare-vms/lib/infra/openstack.sh

@@ -0,0 +1,20 @@
+infra_start() {
+	COUNT=$1
+
+	cp terraform/*.tf tags/$TAG
+	(
+		cd tags/$TAG
+		terraform init
+		echo prefix = \"$TAG\" >> terraform.tfvars
+		echo count = \"$COUNT\" >> terraform.tfvars
+		terraform apply -auto-approve
+		terraform output ip_addresses > ips.txt
+	)
+}
+
+infra_stop() {
+	(
+		cd tags/$TAG
+		terraform destroy -auto-approve
+	)
+}

prepare-vms/lib/ips-txt-to-html.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 import os
 import sys
 import yaml
@@ -31,7 +31,13 @@ while ips:
     clusters.append(cluster)
 
 template_file_name = SETTINGS["cards_template"]
-template = jinja2.Template(open(template_file_name).read())
+template_file_path = os.path.join(
+    os.path.dirname(__file__),
+    "..",
+    "templates",
+    template_file_name
+    )
+template = jinja2.Template(open(template_file_path).read())
 with open("ips.html", "w") as f:
 	f.write(template.render(clusters=clusters, **SETTINGS))
 print("Generated ips.html")

prepare-vms/lib/postprep.py

@@ -12,7 +12,9 @@ config = yaml.load(open("/tmp/settings.yaml"))
 COMPOSE_VERSION = config["compose_version"]
 MACHINE_VERSION = config["machine_version"]
 CLUSTER_SIZE = config["clustersize"]
+CLUSTER_PREFIX = config["clusterprefix"]
 ENGINE_VERSION = config["engine_version"]
+DOCKER_USER_PASSWORD = config["docker_user_password"]
 
 #################################
 
@@ -45,7 +47,7 @@ def system(cmd):
 
 # On EC2, the ephemeral disk might be mounted on /mnt.
 # If /mnt is a mountpoint, place Docker workspace on it.
-system("if mountpoint -q /mnt; then sudo mkdir /mnt/docker && sudo ln -s /mnt/docker /var/lib/docker; fi")
+system("if mountpoint -q /mnt; then sudo mkdir -p /mnt/docker && sudo ln -sfn /mnt/docker /var/lib/docker; fi")
 
 # Put our public IP in /tmp/ipv4
 # ipv4_retrieval_endpoint = "http://169.254.169.254/latest/meta-data/public-ipv4"
@@ -54,9 +56,9 @@ system("curl --silent {} > /tmp/ipv4".format(ipv4_retrieval_endpoint))
 
 ipv4 = open("/tmp/ipv4").read()
 
-# Add a "docker" user with password "training"
+# Add a "docker" user with password coming from the settings
 system("id docker || sudo useradd -d /home/docker -m -s /bin/bash docker")
-system("echo docker:training | sudo chpasswd")
+system("echo docker:{} | sudo chpasswd".format(DOCKER_USER_PASSWORD))
 
 # Fancy prompt courtesy of @soulshake.
 system("""sudo -u docker tee -a /home/docker/.bashrc <<SQRL
@@ -82,7 +84,7 @@ system("sudo sed -i 's/PasswordAuthentication no/PasswordAuthentication yes/' /e
 
 system("sudo service ssh restart")
 system("sudo apt-get -q update")
-system("sudo apt-get -qy install git jq python-pip")
+system("sudo apt-get -qy install git jq")
 
 #######################
 ### DOCKER INSTALLS ###
@@ -97,7 +99,6 @@ system("sudo apt-get -q update")
 system("sudo apt-get -qy install docker-ce")
 
 ### Install docker-compose
-#system("sudo pip install -U docker-compose=={}".format(COMPOSE_VERSION))
 system("sudo curl -sSL -o /usr/local/bin/docker-compose https://github.com/docker/compose/releases/download/{}/docker-compose-{}-{}".format(COMPOSE_VERSION, platform.system(), platform.machine()))
 system("sudo chmod +x /usr/local/bin/docker-compose")
 system("docker-compose version")
@@ -108,7 +109,7 @@ system("sudo chmod +x /usr/local/bin/docker-machine")
 system("docker-machine version")
 
 system("sudo apt-get remove -y --purge dnsmasq-base")
-system("sudo apt-get -qy install python-setuptools pssh apache2-utils httping htop unzip mosh")
+system("sudo apt-get -qy install python-setuptools pssh apache2-utils httping htop unzip mosh tree")
 
 ### Wait for Docker to be up.
 ### (If we don't do this, Docker will not be responsive during the next step.)
@@ -121,7 +122,7 @@ addresses = list(l.strip() for l in sys.stdin)
 assert ipv4 in addresses
 
 def makenames(addrs):
-    return [ "node%s"%(i+1) for i in range(len(addrs)) ]
+    return [ "%s%s"%(CLUSTER_PREFIX, i+1) for i in range(len(addrs)) ]
 
 while addresses:
     cluster = addresses[:CLUSTER_SIZE]
@@ -135,15 +136,21 @@ while addresses:
     print(cluster)
 
     mynode = cluster.index(ipv4) + 1
-    system("echo node{} | sudo -u docker tee /tmp/node".format(mynode))
-    system("echo node{} | sudo tee /etc/hostname".format(mynode))
-    system("sudo hostname node{}".format(mynode))
+    system("echo {}{} | sudo tee /etc/hostname".format(CLUSTER_PREFIX, mynode))
+    system("sudo hostname {}{}".format(CLUSTER_PREFIX, mynode))
     system("sudo -u docker mkdir -p /home/docker/.ssh")
     system("sudo -u docker touch /home/docker/.ssh/authorized_keys")
 
+    # Create a convenience file to easily check if we're the first node
     if ipv4 == cluster[0]:
-        # If I'm node1 and don't have a private key, generate one (with empty passphrase)
+        system("sudo ln -sf /bin/true /usr/local/bin/i_am_first_node")
+        # On the first node, if we don't have a private key, generate one (with empty passphrase)
         system("sudo -u docker [ -f /home/docker/.ssh/id_rsa ] || sudo -u docker ssh-keygen -t rsa -f /home/docker/.ssh/id_rsa -P ''")
+    else:
+        system("sudo ln -sf /bin/false /usr/local/bin/i_am_first_node")
+    # Record the IPV4 and name of the first node
+    system("echo {} | sudo tee /etc/ipv4_of_first_node".format(cluster[0]))
+    system("echo {} | sudo tee /etc/name_of_first_node".format(names[0]))
 
 FINISH = time.time()
 duration = "Initial deployment took {}s".format(str(FINISH - START)[:5])

prepare-vms/lib/pssh.sh

@@ -1,12 +1,17 @@
 # This file can be sourced in order to directly run commands on
-# a batch of VMs whose IPs are located in ips.txt of the directory in which
+# a group of VMs whose IPs are located in ips.txt of the directory in which
 # the command is run.
 
 pssh() {
-    HOSTFILE="ips.txt"
+    if [ -z "$TAG" ]; then
+        >/dev/stderr echo "Variable \$TAG is not set."
+        return
+    fi
+
+    HOSTFILE="tags/$TAG/ips.txt"
 
     [ -f $HOSTFILE ] || {
-        >/dev/stderr echo "No hostfile found at $HOSTFILE"
+        >/dev/stderr echo "Hostfile $HOSTFILE not found."
         return
     }
 

prepare-vms/settings/admin-dmuc.yaml

@@ -0,0 +1,28 @@
+# Number of VMs per cluster
+clustersize: 1
+
+# The hostname of each node will be clusterprefix + a number
+clusterprefix: dmuc
+
+# Jinja2 template to use to generate ready-to-cut cards
+cards_template: cards.html
+
+# Use "Letter" in the US, and "A4" everywhere else
+paper_size: A4
+
+# Feel free to reduce this if your printer can handle it
+paper_margin: 0.2in
+
+# Note: paper_size and paper_margin only apply to PDF generated with pdfkit.
+# If you print (or generate a PDF) using ips.html, they will be ignored.
+# (The equivalent parameters must be set from the browser's print dialog.)
+
+# This can be "test" or "stable"
+engine_version: stable
+
+# These correspond to the version numbers visible on their respective GitHub release pages
+compose_version: 1.24.1
+machine_version: 0.14.0
+
+# Password used to connect with the "docker user"
+docker_user_password: training

prepare-vms/settings/admin-kubenet.yaml

@@ -0,0 +1,28 @@
+# Number of VMs per cluster
+clustersize: 3
+
+# The hostname of each node will be clusterprefix + a number
+clusterprefix: kubenet
+
+# Jinja2 template to use to generate ready-to-cut cards
+cards_template: cards.html
+
+# Use "Letter" in the US, and "A4" everywhere else
+paper_size: A4
+
+# Feel free to reduce this if your printer can handle it
+paper_margin: 0.2in
+
+# Note: paper_size and paper_margin only apply to PDF generated with pdfkit.
+# If you print (or generate a PDF) using ips.html, they will be ignored.
+# (The equivalent parameters must be set from the browser's print dialog.)
+
+# This can be "test" or "stable"
+engine_version: stable
+
+# These correspond to the version numbers visible on their respective GitHub release pages
+compose_version: 1.24.1
+machine_version: 0.14.0
+
+# Password used to connect with the "docker user"
+docker_user_password: training

prepare-vms/settings/admin-kuberouter.yaml

@@ -0,0 +1,28 @@
+# Number of VMs per cluster
+clustersize: 3
+
+# The hostname of each node will be clusterprefix + a number
+clusterprefix: kuberouter
+
+# Jinja2 template to use to generate ready-to-cut cards
+cards_template: cards.html
+
+# Use "Letter" in the US, and "A4" everywhere else
+paper_size: A4
+
+# Feel free to reduce this if your printer can handle it
+paper_margin: 0.2in
+
+# Note: paper_size and paper_margin only apply to PDF generated with pdfkit.
+# If you print (or generate a PDF) using ips.html, they will be ignored.
+# (The equivalent parameters must be set from the browser's print dialog.)
+
+# This can be "test" or "stable"
+engine_version: stable
+
+# These correspond to the version numbers visible on their respective GitHub release pages
+compose_version: 1.24.1
+machine_version: 0.14.0
+
+# Password used to connect with the "docker user"
+docker_user_password: training

prepare-vms/settings/admin-test.yaml

@@ -0,0 +1,28 @@
+# Number of VMs per cluster
+clustersize: 3
+
+# The hostname of each node will be clusterprefix + a number
+clusterprefix: test
+
+# Jinja2 template to use to generate ready-to-cut cards
+cards_template: cards.html
+
+# Use "Letter" in the US, and "A4" everywhere else
+paper_size: A4
+
+# Feel free to reduce this if your printer can handle it
+paper_margin: 0.2in
+
+# Note: paper_size and paper_margin only apply to PDF generated with pdfkit.
+# If you print (or generate a PDF) using ips.html, they will be ignored.
+# (The equivalent parameters must be set from the browser's print dialog.)
+
+# This can be "test" or "stable"
+engine_version: stable
+
+# These correspond to the version numbers visible on their respective GitHub release pages
+compose_version: 1.24.1
+machine_version: 0.14.0
+
+# Password used to connect with the "docker user"
+docker_user_password: training

prepare-vms/settings/csv.yaml

@@ -0,0 +1,8 @@
+# Number of VMs per cluster
+clustersize: 5
+
+# The hostname of each node will be clusterprefix + a number
+clusterprefix: node
+
+# Jinja2 template to use to generate ready-to-cut cards
+cards_template: clusters.csv

prepare-vms/settings/example.yaml

@@ -0,0 +1,30 @@
+# customize your cluster size, your cards template, and the versions
+
+# Number of VMs per cluster
+clustersize: 5
+
+# The hostname of each node will be clusterprefix + a number
+clusterprefix: node
+
+# Jinja2 template to use to generate ready-to-cut cards
+cards_template: cards.html
+
+# Use "Letter" in the US, and "A4" everywhere else
+paper_size: Letter
+
+# Feel free to reduce this if your printer can handle it
+paper_margin: 0.2in
+
+# Note: paper_size and paper_margin only apply to PDF generated with pdfkit.
+# If you print (or generate a PDF) using ips.html, they will be ignored.
+# (The equivalent parameters must be set from the browser's print dialog.)
+
+# This can be "test" or "stable"
+engine_version: test
+
+# These correspond to the version numbers visible on their respective GitHub release pages
+compose_version: 1.24.1
+machine_version: 0.13.0
+
+# Password used to connect with the "docker user"
+docker_user_password: training

prepare-vms/settings/fundamentals.yaml

@@ -3,6 +3,9 @@
 # Number of VMs per cluster
 clustersize: 1
 
+# The hostname of each node will be clusterprefix + a number
+clusterprefix: node
+
 # Jinja2 template to use to generate ready-to-cut cards
 cards_template: cards.html
 
@@ -17,8 +20,11 @@ paper_margin: 0.2in
 # (The equivalent parameters must be set from the browser's print dialog.)
 
 # This can be "test" or "stable"
-engine_version: test
+engine_version: stable
 
 # These correspond to the version numbers visible on their respective GitHub release pages
-compose_version: 1.17.1
-machine_version: 0.13.0
+compose_version: 1.24.1
+machine_version: 0.15.0
+
+# Password used to connect with the "docker user"
+docker_user_password: training

prepare-vms/settings/jerome.yaml

@@ -0,0 +1,29 @@
+# Number of VMs per cluster
+clustersize: 4
+
+# The hostname of each node will be clusterprefix + a number
+clusterprefix: node
+
+# Jinja2 template to use to generate ready-to-cut cards
+cards_template: cards.html
+
+# Use "Letter" in the US, and "A4" everywhere else
+paper_size: Letter
+
+# Feel free to reduce this if your printer can handle it
+paper_margin: 0.2in
+
+# Note: paper_size and paper_margin only apply to PDF generated with pdfkit.
+# If you print (or generate a PDF) using ips.html, they will be ignored.
+# (The equivalent parameters must be set from the browser's print dialog.)
+
+# This can be "test" or "stable"
+engine_version: stable
+
+# These correspond to the version numbers visible on their respective GitHub release pages
+compose_version: 1.24.1
+machine_version: 0.14.0
+
+# Password used to connect with the "docker user"
+docker_user_password: training
+

prepare-vms/settings/kube101.yaml

@@ -0,0 +1,31 @@
+# 3 nodes for k8s 101 workshops
+
+# Number of VMs per cluster
+clustersize: 3
+
+# The hostname of each node will be clusterprefix + a number
+clusterprefix: node
+
+# Jinja2 template to use to generate ready-to-cut cards
+cards_template: cards.html
+
+# Use "Letter" in the US, and "A4" everywhere else
+paper_size: Letter
+
+# Feel free to reduce this if your printer can handle it
+paper_margin: 0.2in
+
+# Note: paper_size and paper_margin only apply to PDF generated with pdfkit.
+# If you print (or generate a PDF) using ips.html, they will be ignored.
+# (The equivalent parameters must be set from the browser's print dialog.)
+
+# This can be "test" or "stable"
+engine_version: stable
+
+# These correspond to the version numbers visible on their respective GitHub release pages
+compose_version: 1.24.1
+machine_version: 0.14.0
+
+# Password used to connect with the "docker user"
+docker_user_password: training
+

prepare-vms/settings/swarm.yaml

@@ -1,7 +1,10 @@
 # This file is passed by trainer-cli to scripts/ips-txt-to-html.py
 
 # Number of VMs per cluster
-clustersize: 5
+clustersize: 3
+
+# The hostname of each node will be clusterprefix + a number
+clusterprefix: node
 
 # Jinja2 template to use to generate ready-to-cut cards
 cards_template: cards.html
@@ -17,8 +20,11 @@ paper_margin: 0.2in
 # (The equivalent parameters must be set from the browser's print dialog.)
 
 # This can be "test" or "stable"
-engine_version: test
+engine_version: stable
 
 # These correspond to the version numbers visible on their respective GitHub release pages
-compose_version: 1.17.1
-machine_version: 0.13.0
+compose_version: 1.24.1
+machine_version: 0.15.0
+
+# Password used to connect with the "docker user"
+docker_user_password: training

prepare-vms/setup-admin-clusters.sh

@@ -0,0 +1,66 @@
+#!/bin/sh
+set -e
+
+export AWS_INSTANCE_TYPE=t3a.small
+
+INFRA=infra/aws-us-west-2
+
+STUDENTS=2
+
+PREFIX=$(date +%Y-%m-%d-%H-%M)
+
+SETTINGS=admin-dmuc
+TAG=$PREFIX-$SETTINGS
+./workshopctl start \
+	--tag $TAG \
+	--infra $INFRA \
+	--settings settings/$SETTINGS.yaml \
+	--count $STUDENTS
+
+./workshopctl deploy $TAG
+./workshopctl disabledocker $TAG
+./workshopctl kubebins $TAG
+./workshopctl cards $TAG
+
+SETTINGS=admin-kubenet
+TAG=$PREFIX-$SETTINGS
+./workshopctl start \
+	--tag $TAG \
+	--infra $INFRA \
+	--settings settings/$SETTINGS.yaml \
+	--count $((3*$STUDENTS))
+
+./workshopctl disableaddrchecks $TAG
+./workshopctl deploy $TAG
+./workshopctl kubebins $TAG
+./workshopctl cards $TAG
+
+SETTINGS=admin-kuberouter
+TAG=$PREFIX-$SETTINGS
+./workshopctl start \
+	--tag $TAG \
+	--infra $INFRA \
+	--settings settings/$SETTINGS.yaml \
+	--count $((3*$STUDENTS))
+
+./workshopctl disableaddrchecks $TAG
+./workshopctl deploy $TAG
+./workshopctl kubebins $TAG
+./workshopctl cards $TAG
+
+#INFRA=infra/aws-us-west-1
+
+export AWS_INSTANCE_TYPE=t3a.medium
+
+SETTINGS=admin-test
+TAG=$PREFIX-$SETTINGS
+./workshopctl start \
+	--tag $TAG \
+	--infra $INFRA \
+	--settings settings/$SETTINGS.yaml \
+	--count $((3*$STUDENTS))
+
+./workshopctl deploy $TAG
+./workshopctl kube $TAG 1.13.5
+./workshopctl cards $TAG
+

prepare-vms/templates/cards.html

@@ -0,0 +1,218 @@
+{# Feel free to customize or override anything in there! #}
+
+{%- set url = "http://FIXME.container.training/" -%}
+{%- set pagesize = 9 -%}
+{%- set lang = "en" -%}
+{%- set event = "training session" -%}
+{%- set backside = False -%}
+{%- set image = "kube" -%}
+{%- set clusternumber = 100 -%}
+
+{%- set image_src = {
+	"docker": "https://s3-us-west-2.amazonaws.com/www.breadware.com/integrations/docker.png",
+	"swarm": "https://cdn.wp.nginx.com/wp-content/uploads/2016/07/docker-swarm-hero2.png",
+	"kube": "https://avatars1.githubusercontent.com/u/13629408",
+	"enix": "https://enix.io/static/img/logos/logo-domain-cropped.png",
+    }[image] -%}
+{%- if lang == "en" and clustersize == 1 -%}
+	{%- set intro -%}
+	Here is the connection information to your very own
+	machine for this {{ event }}.
+	You can connect to this VM with any SSH client.
+	{%- endset -%}
+    {%- set listhead -%}
+    Your machine is:
+	{%- endset -%}
+{%- endif -%}
+{%- if lang == "en" and clustersize != 1 -%}
+	{%- set intro -%}
+	Here is the connection information to your very own
+	cluster for this {{ event }}.
+	You can connect to each VM with any SSH client.
+	{%- endset -%}
+    {%- set listhead -%}
+    Your machines are:
+	{%- endset -%}
+{%- endif -%}
+{%- if lang == "fr" and clustersize == 1 -%}
+	{%- set intro -%}
+	Voici les informations permettant de se connecter à votre
+	machine pour cette formation.
+	Vous pouvez vous connecter à cette machine virtuelle
+	avec n'importe quel client SSH.
+	{%- endset -%}
+    {%- set listhead -%}
+    Adresse IP:
+	{%- endset -%}
+{%- endif -%}
+{%- if lang == "en" and clusterprefix != "node" -%}
+	{%- set intro -%}
+    Here is the connection information for the
+    <strong>{{ clusterprefix }}</strong> environment.
+	{%- endset -%}
+{%- endif -%}
+{%- if lang == "fr" and clustersize != 1 -%}
+	{%- set intro -%}
+	Voici les informations permettant de se connecter à votre
+	cluster pour cette formation.
+	Vous pouvez vous connecter à chaque machine virtuelle
+	avec n'importe quel client SSH.
+	{%- endset -%}
+    {%- set listhead -%}
+	Adresses IP:
+	{%- endset -%}
+{%- endif -%}
+{%- if lang == "en"  -%}
+	{%- set slides_are_at -%}
+	You can find the slides at:
+	{%- endset -%}
+{%- endif -%}
+{%- if lang == "fr" -%}
+	{%- set slides_are_at -%}
+  	Le support de formation est à l'adresse suivante :
+	{%- endset -%}
+{%- endif -%}
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
+<html>
+<head><style>
+@import url('https://fonts.googleapis.com/css?family=Slabo+27px');
+
+body, table {
+    margin: 0;
+    padding: 0;
+    line-height: 1em;
+    font-size: 15px;
+    font-family: 'Slabo 27px';
+}
+
+table {
+    border-spacing: 0;
+    margin-top: 0.4em;
+    margin-bottom: 0.4em;
+    border-left: 0.8em double grey;
+    padding-left: 0.4em;
+}
+
+div {
+    float: left;
+    border: 1px dotted black;
+    {% if backside %}
+    height: 31%;
+    {% endif %}
+    padding-top: 1%;
+    padding-bottom: 1%;
+    /* columns * (width+left+right) < 100% */
+    /*
+    width: 21.5%;
+    padding-left: 1.5%;
+    padding-right: 1.5%;
+    */
+    /**/
+    width: 30%;
+    padding-left: 1.5%;
+    padding-right: 1.5%;
+    /**/    
+}
+
+p {
+    margin: 0.4em 0 0.4em 0;
+}
+
+div.back {
+	border: 1px dotted white;
+}
+
+div.back p {
+    margin: 0.5em 1em 0 1em;
+}
+
+img {
+    height: 4em;
+    float: right;
+    margin-right: -0.2em;
+}
+
+/*
+img.enix {
+    height: 4.0em;
+    margin-top: 0.4em;
+}
+
+img.kube {
+    height: 4.2em;
+    margin-top: 1.7em;
+}
+*/
+
+.logpass {
+    font-family: monospace;
+    font-weight: bold;
+}
+
+.pagebreak {
+    page-break-after: always;
+    clear: both;
+    display: block;
+    height: 8px;
+}
+</style></head>
+<body>
+{% for cluster in clusters %}
+    <div>
+        <p>{{ intro }}</p>
+        <p>
+            <img src="{{ image_src }}" />
+            <table>
+            	{% if clusternumber != None %}
+	            <tr><td>cluster:</td></tr>
+	            <tr><td class="logpass">{{ clusternumber + loop.index }}</td></tr>
+            	{% endif %}
+                <tr><td>login:</td></tr>
+                <tr><td class="logpass">docker</td></tr>
+                <tr><td>password:</td></tr>
+                <tr><td class="logpass">{{ docker_user_password }}</td></tr>
+            </table>
+
+        </p>
+        <p>
+            {{ listhead }}
+            <table>
+                {% for node in cluster %}
+                <tr>
+                	<td>{{ clusterprefix }}{{ loop.index }}:</td>
+                	<td>{{ node }}</td>
+                </tr>
+                {% endfor %}
+            </table>
+        </p>
+
+        <p>
+        	{{ slides_are_at }}
+            <center>{{ url }}</center>
+        </p>
+    </div>
+    {% if loop.index%pagesize==0 or loop.last %}
+        <span class="pagebreak"></span>
+        {% if backside %}
+			{% for x in range(pagesize) %}
+		        <div class="back">
+		        <br/>
+				<p>You got this at the workshop
+				"Getting Started With Kubernetes and Container Orchestration"
+				during QCON London (March 2019).</p>
+				<p>If you liked that workshop,
+				I can train your team or organization
+				on Docker, container, and Kubernetes,
+				with curriculums of 1 to 5 days.
+				</p>
+				<p>Interested? Contact me at:</p>
+				<p>jerome.petazzoni@gmail.com</p>
+				<p>Thank you!</p>
+		        </div>
+			{% endfor %}
+		<span class="pagebreak"></span>
+		{% endif %}
+    {% endif %}
+{% endfor %}
+</body>
+</html>

prepare-vms/templates/clusters.csv

@@ -0,0 +1,5 @@
+Put your initials in the first column to "claim" a cluster.
+Initials{% for node in clusters[0] %}	node{{ loop.index }}{% endfor %}
+{% for cluster in clusters -%}
+  {%- for node in cluster %}	{{ node|trim }}{% endfor %}
+{% endfor %}

prepare-vms/terraform/keypair.tf

@@ -0,0 +1,5 @@
+resource "openstack_compute_keypair_v2" "ssh_deploy_key" {
+  name       = "${var.prefix}"
+  public_key = "${file("~/.ssh/id_rsa.pub")}"
+}
+

prepare-vms/terraform/machines.tf

@@ -0,0 +1,32 @@
+resource "openstack_compute_instance_v2" "machine" {
+  count           = "${var.count}" 
+  name            = "${format("%s-%04d", "${var.prefix}", count.index+1)}"
+  image_name      = "Ubuntu 16.04.5 (Xenial Xerus)"
+  flavor_name     = "${var.flavor}"
+  security_groups = ["${openstack_networking_secgroup_v2.full_access.name}"]
+  key_pair        = "${openstack_compute_keypair_v2.ssh_deploy_key.name}"
+
+  network {
+    name        = "${openstack_networking_network_v2.internal.name}"
+    fixed_ip_v4 = "${cidrhost("${openstack_networking_subnet_v2.internal.cidr}", count.index+10)}"
+  }
+}
+
+resource "openstack_compute_floatingip_v2" "machine" {
+  count = "${var.count}"
+  # This is something provided to us by Enix when our tenant was provisioned.
+  pool = "Public Floating"
+}
+
+resource "openstack_compute_floatingip_associate_v2" "machine" {
+  count       = "${var.count}"
+  floating_ip = "${openstack_compute_floatingip_v2.machine.*.address[count.index]}"
+  instance_id = "${openstack_compute_instance_v2.machine.*.id[count.index]}"
+  fixed_ip    = "${cidrhost("${openstack_networking_subnet_v2.internal.cidr}", count.index+10)}"
+}
+
+output "ip_addresses" {
+  value = "${join("\n", openstack_compute_floatingip_v2.machine.*.address)}"
+}
+
+variable "flavor" {}

prepare-vms/terraform/network.tf

@@ -0,0 +1,23 @@
+resource "openstack_networking_network_v2" "internal" {
+  name           = "${var.prefix}"
+}
+
+resource "openstack_networking_subnet_v2" "internal" {
+  name            = "${var.prefix}"
+  network_id      = "${openstack_networking_network_v2.internal.id}"
+  cidr            = "10.10.0.0/16"
+  ip_version      = 4
+  dns_nameservers = ["1.1.1.1"]
+}
+
+resource "openstack_networking_router_v2" "router" {
+  name                = "${var.prefix}"
+  external_network_id = "15f0c299-1f50-42a6-9aff-63ea5b75f3fc"
+}
+
+resource "openstack_networking_router_interface_v2" "router_internal" {
+  router_id = "${openstack_networking_router_v2.router.id}"
+  subnet_id = "${openstack_networking_subnet_v2.internal.id}"
+}
+
+

prepare-vms/terraform/provider.tf

@@ -0,0 +1,13 @@
+provider "openstack" {
+  user_name   = "${var.user}"
+  tenant_name = "${var.tenant}"
+  domain_name = "${var.domain}"
+  password    = "${var.password}"
+  auth_url    = "${var.auth_url}"
+}
+
+variable "user" {}
+variable "tenant" {}
+variable "domain" {}
+variable "password" {}
+variable "auth_url" {}

prepare-vms/terraform/secgroup.tf

@@ -0,0 +1,12 @@
+resource "openstack_networking_secgroup_v2" "full_access" {
+  name = "${var.prefix} - full access"
+}
+
+resource "openstack_networking_secgroup_rule_v2" "full_access" {
+  direction         = "ingress"
+  ethertype         = "IPv4"
+  protocol          = ""
+  remote_ip_prefix  = "0.0.0.0/0"
+  security_group_id = "${openstack_networking_secgroup_v2.full_access.id}"
+}
+

prepare-vms/terraform/vars.tf

@@ -0,0 +1,8 @@
+variable "prefix" {
+  type = "string"
+}
+
+variable "count" {
+  type = "string"
+}
+

prepare-vms/workshopctl

@@ -1,20 +1,19 @@
 #!/bin/bash
 
-# Get the script's real directory, whether we're being called directly or via a symlink
+# Get the script's real directory.
+# This should work whether we're being called directly or via a symlink.
 if [ -L "$0" ]; then
     export SCRIPT_DIR=$(dirname $(readlink "$0"))
 else
     export SCRIPT_DIR=$(dirname "$0")
 fi
 
-# Load all scriptlets
+# Load all scriptlets.
 cd "$SCRIPT_DIR"
 for lib in lib/*.sh; do
     . $lib
 done
 
-TRAINER_IMAGE="preparevms_prepare-vms"
-
 DEPENDENCIES="
     aws
     ssh
@@ -25,49 +24,26 @@ DEPENDENCIES="
     man
     "
 
-ENVVARS="
-    AWS_ACCESS_KEY_ID
-    AWS_SECRET_ACCESS_KEY
-    AWS_DEFAULT_REGION
-    SSH_AUTH_SOCK
-    "
+# Check for missing dependencies, and issue a warning if necessary.
+missing=0
+for dependency in $DEPENDENCIES; do
+    if ! command -v $dependency >/dev/null; then
+        warning "Dependency $dependency could not be found."
+        missing=1
+    fi
+done
+if [ $missing = 1 ]; then
+    warning "At least one dependency is missing. Install it or try the image wrapper."
+fi
 
-check_envvars() {
-    status=0
-    for envvar in $ENVVARS; do
-        if [ -z "${!envvar}" ]; then
-            error "Environment variable $envvar is not set."
-            if [ "$envvar" = "SSH_AUTH_SOCK" ]; then
-                error "Hint: run '\$(ssh-agent) ; ssh-add' and try again?"
-            fi
-            status=1
-        fi
-    done
-    return $status
-}
+# Check if SSH_AUTH_SOCK is set.
+# (If it's not, deployment will almost certainly fail.)
+if [ -z "${SSH_AUTH_SOCK}" ]; then
+    warning "Environment variable SSH_AUTH_SOCK is not set."
+    warning "Hint: run 'eval \$(ssh-agent) ; ssh-add' and try again?"
+fi
 
-check_dependencies() {
-    status=0
-    for dependency in $DEPENDENCIES; do
-        if ! command -v $dependency >/dev/null; then
-            warning "Dependency $dependency could not be found."
-            status=1
-        fi
-    done
-    return $status
-}
-
-check_image() {
-    docker inspect $TRAINER_IMAGE >/dev/null 2>&1
-}
-
-check_envvars \
-    || die "Please set all required environment variables."
-
-check_dependencies \
-    || warning "At least one dependency is missing. Install it or try the image wrapper."
-
-# Now check which command was invoked and execute it
+# Now check which command was invoked and execute it.
 if [ "$1" ]; then
     cmd="$1"
     shift
@@ -77,6 +53,3 @@ fi
 fun=_cmd_$cmd
 type -t $fun | grep -q function || die "Invalid command: $cmd"
 $fun "$@"
-
-#    export SSH_AUTH_DIRNAME=$(dirname $SSH_AUTH_SOCK)
-#    docker-compose run prepare-vms "$@"

slides/Dockerfile

@@ -0,0 +1,4 @@
+FROM alpine:3.9
+RUN apk add --no-cache entr py-pip git
+COPY requirements.txt .
+RUN pip install -r requirements.txt

slides/README.md

@@ -34,6 +34,14 @@ compile each `foo.yml` file into `foo.yml.html`.
 You can also run `./build.sh forever`: it will monitor the current
 directory and rebuild slides automatically when files are modified.
 
+If you have problems running `./build.sh` (because of
+Python dependencies or whatever),
+you can also run `docker-compose up` in this directory.
+It will start the `./build.sh forever` script in a container.
+It will also start a web server exposing the slides
+(but the slides should also work if you load them from your
+local filesystem).
+
 
 ## Publishing pipeline
 
@@ -53,4 +61,4 @@ You can run `./slidechecker foo.yml.html` to check for
 missing images and show the number of slides in that deck.
 It requires `phantomjs` to be installed. It takes some
 time to run so it is not yet integrated with the publishing
-pipeline.
+pipeline.

slides/_redirects

@@ -1 +1,8 @@
-/ /kube-halfday.yml.html 200!
+# Uncomment and/or edit one of the the following lines if necessary.
+#/ /kube-halfday.yml.html 200
+#/ /kube-fullday.yml.html 200
+#/ /kube-twodays.yml.html 200
+/ /k8s-201.yml.html 200!
+
+# And this allows to do "git clone https://container.training".
+/info/refs service=git-upload-pack https://github.com/jpetazzo/container.training/info/refs?service=git-upload-pack

slides/autopilot/autotest.py

@@ -0,0 +1,453 @@
+#!/usr/bin/env python
+# coding: utf-8
+
+import click
+import logging
+import os
+import random
+import re
+import select
+import subprocess
+import sys
+import time
+import uuid
+import yaml
+
+
+logging.basicConfig(level=os.environ.get("LOG_LEVEL", "INFO"))
+
+
+TIMEOUT = 60 # 1 minute
+
+# This one is not a constant. It's an ugly global.
+IPADDR = None
+
+
+class State(object):
+
+    def __init__(self):
+        self.interactive = True
+        self.verify_status = False
+        self.simulate_type = True
+        self.switch_desktop = False
+        self.sync_slides = False
+        self.open_links = False
+        self.run_hidden = True
+        self.slide = 1
+        self.snippet = 0
+
+    def load(self):
+        data = yaml.load(open("state.yaml"))
+        self.interactive = bool(data["interactive"])
+        self.verify_status = bool(data["verify_status"])
+        self.simulate_type = bool(data["simulate_type"])
+        self.switch_desktop = bool(data["switch_desktop"])
+        self.sync_slides = bool(data["sync_slides"])
+        self.open_links = bool(data["open_links"])
+        self.run_hidden = bool(data["run_hidden"])
+        self.slide = int(data["slide"])
+        self.snippet = int(data["snippet"])
+
+    def save(self):
+        with open("state.yaml", "w") as f:
+            yaml.dump(dict(
+                interactive=self.interactive,
+                verify_status=self.verify_status,
+                simulate_type=self.simulate_type,
+                switch_desktop=self.switch_desktop,
+                sync_slides=self.sync_slides,
+                open_links=self.open_links,
+                run_hidden=self.run_hidden,
+                slide=self.slide,
+                snippet=self.snippet,
+                ), f, default_flow_style=False)
+
+
+state = State()
+
+
+def hrule():
+    return "="*int(subprocess.check_output(["tput", "cols"]))
+
+# A "snippet" is something that the user is supposed to do in the workshop.
+# Most of the "snippets" are shell commands.
+# Some of them can be key strokes or other actions.
+# In the markdown source, they are the code sections (identified by triple-
+# quotes) within .exercise[] sections.
+
+class Snippet(object):
+
+    def __init__(self, slide, content):
+        self.slide = slide
+        self.content = content
+        # Extract the "method" (e.g. bash, keys, ...)
+        # On multi-line snippets, the method is alone on the first line
+        # On single-line snippets, the data follows the method immediately
+        if '\n' in content:
+            self.method, self.data = content.split('\n', 1)
+        else:
+            self.method, self.data = content.split(' ', 1)
+        self.data = self.data.strip()
+        self.next = None
+
+    def __str__(self):
+        return self.content
+
+
+class Slide(object):
+
+    current_slide = 0
+
+    def __init__(self, content):
+        self.number = Slide.current_slide
+        Slide.current_slide += 1
+
+        # Remove commented-out slides
+        # (remark.js considers ??? to be the separator for speaker notes)
+        content = re.split("\n\?\?\?\n", content)[0]
+        self.content = content
+
+        self.snippets = []
+        exercises = re.findall("\.exercise\[(.*)\]", content, re.DOTALL)
+        for exercise in exercises:
+            if "```" in exercise:
+                previous = None
+                for snippet_content in exercise.split("```")[1::2]:
+                    snippet = Snippet(self, snippet_content)
+                    if previous:
+                        previous.next = snippet
+                    previous = snippet
+                    self.snippets.append(snippet)
+            else:
+                logging.warning("Exercise on slide {} does not have any ``` snippet."
+                                .format(self.number))
+                self.debug()
+
+    def __str__(self):
+        text = self.content
+        for snippet in self.snippets:
+            text = text.replace(snippet.content, ansi("7")(snippet.content))
+        return text
+
+    def debug(self):
+        logging.debug("\n{}\n{}\n{}".format(hrule(), self.content, hrule()))
+
+
+def focus_slides():
+    if not state.switch_desktop:
+        return
+    subprocess.check_output(["i3-msg", "workspace", "3"])
+    subprocess.check_output(["i3-msg", "workspace", "1"])
+
+def focus_terminal():
+    if not state.switch_desktop:
+        return
+    subprocess.check_output(["i3-msg", "workspace", "2"])
+    subprocess.check_output(["i3-msg", "workspace", "1"])
+
+def focus_browser():
+    if not state.switch_desktop:
+        return
+    subprocess.check_output(["i3-msg", "workspace", "4"])
+    subprocess.check_output(["i3-msg", "workspace", "1"])
+
+
+def ansi(code):
+    return lambda s: "\x1b[{}m{}\x1b[0m".format(code, s)
+
+
+# Sleeps the indicated delay, but interruptible by pressing ENTER.
+# If interrupted, returns True.
+def interruptible_sleep(t):
+    rfds, _, _ = select.select([0], [], [], t)
+    return 0 in rfds
+
+
+def wait_for_string(s, timeout=TIMEOUT):
+    logging.debug("Waiting for string: {}".format(s))
+    deadline = time.time() + timeout
+    while time.time() < deadline:
+        output = capture_pane()
+        if s in output:
+            return
+        if interruptible_sleep(1): return
+    raise Exception("Timed out while waiting for {}!".format(s))
+
+
+def wait_for_prompt():
+    logging.debug("Waiting for prompt.")
+    deadline = time.time() + TIMEOUT
+    while time.time() < deadline:
+        output = capture_pane()
+        # If we are not at the bottom of the screen, there will be a bunch of extra \n's
+        output = output.rstrip('\n')
+        last_line = output.split('\n')[-1]
+        # Our custom prompt on the VMs has two lines; the 2nd line is just '$'
+        if last_line == "$":
+            # This is a perfect opportunity to grab the node's IP address
+            global IPADDR
+            IPADDR = re.findall("^\[(.*)\]", output, re.MULTILINE)[-1]
+            return
+        # When we are in an alpine container, the prompt will be "/ #"
+        if last_line == "/ #":
+            return
+        # We did not recognize a known prompt; wait a bit and check again
+        logging.debug("Could not find a known prompt on last line: {!r}"
+                      .format(last_line))
+        if interruptible_sleep(1): return
+    raise Exception("Timed out while waiting for prompt!")
+
+
+def check_exit_status():
+    if not state.verify_status:
+        return
+    token = uuid.uuid4().hex
+    data = "echo {} $?\n".format(token)
+    logging.debug("Sending {!r} to get exit status.".format(data))
+    send_keys(data)
+    time.sleep(0.5)
+    wait_for_prompt()
+    screen = capture_pane()
+    status = re.findall("\n{} ([0-9]+)\n".format(token), screen, re.MULTILINE)
+    logging.debug("Got exit status: {}.".format(status))
+    if len(status) == 0:
+        raise Exception("Couldn't retrieve status code {}. Timed out?".format(token))
+    if len(status) > 1:
+        raise Exception("More than one status code {}. I'm seeing double! Shoot them both.".format(token))
+    code = int(status[0])
+    if code != 0:
+        raise Exception("Non-zero exit status: {}.".format(code))
+    # Otherwise just return peacefully.
+
+
+def setup_tmux_and_ssh():
+    if subprocess.call(["tmux", "has-session"]):
+        logging.error("Couldn't connect to tmux. Please setup tmux first.")
+        ipaddr = "$IPADDR"
+        uid = os.getuid()
+
+        raise Exception("""
+1. If you're running this directly from a node:
+
+tmux
+
+2. If you want to control a remote tmux:
+
+rm -f /tmp/tmux-{uid}/default && ssh -t -L /tmp/tmux-{uid}/default:/tmp/tmux-1001/default docker@{ipaddr} tmux new-session -As 0
+
+3. If you cannot control a remote tmux:
+
+tmux new-session ssh docker@{ipaddr}
+""".format(uid=uid, ipaddr=ipaddr))
+    else:
+        logging.info("Found tmux session. Trying to acquire shell prompt.")
+        wait_for_prompt()
+    logging.info("Successfully connected to test cluster in tmux session.")
+
+
+slides = [Slide("Dummy slide zero")]
+content = open(sys.argv[1]).read()
+
+# OK, this part is definitely hackish, and will break if the
+# excludedClasses parameter is not on a single line.
+excluded_classes = re.findall("excludedClasses: (\[.*\])", content)
+excluded_classes = set(eval(excluded_classes[0]))
+
+for slide in re.split("\n---?\n", content):
+    slide_classes = re.findall("class: (.*)", slide)
+    if slide_classes:
+        slide_classes = slide_classes[0].split(",")
+        slide_classes = [c.strip() for c in slide_classes]
+    if excluded_classes & set(slide_classes):
+        logging.info("Skipping excluded slide.")
+        continue
+    slides.append(Slide(slide))
+
+
+def send_keys(data):
+    if state.simulate_type and data[0] != '^':
+        for key in data:
+            if key == ";":
+                key = "\\;"
+            if key == "\n":
+                if interruptible_sleep(1): return
+            subprocess.check_call(["tmux", "send-keys", key])
+            if interruptible_sleep(0.15*random.random()): return
+            if key == "\n":
+                if interruptible_sleep(1): return
+    else:
+        subprocess.check_call(["tmux", "send-keys", data])
+
+
+def capture_pane():
+    return subprocess.check_output(["tmux", "capture-pane", "-p"]).decode('utf-8')
+
+
+setup_tmux_and_ssh()
+
+
+try:
+    state.load()
+    logging.info("Successfully loaded state from file.")
+    # Let's override the starting state, so that when an error occurs,
+    # we can restart the auto-tester and then single-step or debug.
+    # (Instead of running again through the same issue immediately.)
+    state.interactive = True
+except Exception as e:
+    logging.exception("Could not load state from file.")
+    logging.warning("Using default values.")
+
+def move_forward():
+    state.snippet += 1
+    if state.snippet > len(slides[state.slide].snippets):
+        state.slide += 1
+        state.snippet = 0
+    check_bounds()
+
+
+def move_backward():
+    state.snippet -= 1
+    if state.snippet < 0:
+        state.slide -= 1
+        state.snippet = 0
+    check_bounds()
+
+
+def check_bounds():
+    if state.slide < 1:
+        state.slide = 1
+    if state.slide >= len(slides):
+        state.slide = len(slides)-1
+
+
+while True:
+    state.save()
+    slide = slides[state.slide]
+    snippet = slide.snippets[state.snippet-1] if state.snippet else None
+    click.clear()
+    print("[Slide {}/{}] [Snippet {}/{}] [simulate_type:{}] [verify_status:{}] "
+          "[switch_desktop:{}] [sync_slides:{}] [open_links:{}] [run_hidden:{}]"
+          .format(state.slide, len(slides)-1,
+                  state.snippet, len(slide.snippets) if slide.snippets else 0,
+                  state.simulate_type, state.verify_status,
+                  state.switch_desktop, state.sync_slides,
+                  state.open_links, state.run_hidden))
+    print(hrule())
+    if snippet:
+        print(slide.content.replace(snippet.content, ansi(7)(snippet.content)))
+        focus_terminal()
+    else:
+        print(slide.content)
+        if state.sync_slides:
+            subprocess.check_output(["./gotoslide.js", str(slide.number)])
+        focus_slides()
+    print(hrule())
+    if state.interactive:
+        print("y/⎵/⏎   Execute snippet or advance to next snippet")
+        print("p/←     Previous")
+        print("n/→     Next")
+        print("s       Simulate keystrokes")
+        print("v       Validate exit status")
+        print("d       Switch desktop")
+        print("k       Sync slides")
+        print("o       Open links")
+        print("h       Run hidden commands")
+        print("g       Go to a specific slide")
+        print("q       Quit")
+        print("c       Continue non-interactively until next error")
+        command = click.getchar()
+    else:
+        command = "y"
+
+    if command in ("n", "\x1b[C"):
+        move_forward()
+    elif command in ("p", "\x1b[D"):
+        move_backward()
+    elif command == "s":
+        state.simulate_type = not state.simulate_type
+    elif command == "v":
+        state.verify_status = not state.verify_status
+    elif command == "d":
+        state.switch_desktop = not state.switch_desktop
+    elif command == "k":
+        state.sync_slides = not state.sync_slides
+    elif command == "o":
+        state.open_links = not state.open_links
+    elif command == "h":
+        state.run_hidden = not state.run_hidden
+    elif command == "g":
+        state.slide = click.prompt("Enter slide number", type=int)
+        state.snippet = 0
+        check_bounds()
+    elif command == "q":
+        break
+    elif command == "c":
+        # continue until next timeout
+        state.interactive = False
+    elif command in ("y", "\r", " "):
+        if not snippet:
+            # Advance to next snippet
+            # Advance until a slide that has snippets
+            while not slides[state.slide].snippets:
+                move_forward()
+                # But stop if we reach the last slide
+                if state.slide == len(slides)-1:
+                    break
+            # And then advance to the snippet
+            move_forward()
+            continue
+        method, data = snippet.method, snippet.data
+        logging.info("Running with method {}: {}".format(method, data))
+        if method == "keys":
+            send_keys(data)
+        elif method == "bash" or (method == "hide" and state.run_hidden):
+            # Make sure that we're ready
+            wait_for_prompt()
+            # Strip leading spaces
+            data = re.sub("\n +", "\n", data)
+            # Remove backticks (they are used to highlight sections)
+            data = data.replace('`', '')
+            # Add "RETURN" at the end of the command :)
+            data += "\n"
+            # Send command
+            send_keys(data)
+            # Force a short sleep to avoid race condition
+            time.sleep(0.5)
+            if snippet.next and snippet.next.method == "wait":
+                wait_for_string(snippet.next.data)
+            elif snippet.next and snippet.next.method == "longwait":
+                wait_for_string(snippet.next.data, 10*TIMEOUT)
+            else:
+                wait_for_prompt()
+                # Verify return code
+                check_exit_status()
+        elif method == "copypaste":
+            screen = capture_pane()
+            matches = re.findall(data, screen, flags=re.DOTALL)
+            if len(matches) == 0:
+                raise Exception("Could not find regex {} in output.".format(data))
+            # Arbitrarily get the most recent match
+            match = matches[-1]
+            # Remove line breaks (like a screen copy paste would do)
+            match = match.replace('\n', '')
+            send_keys(match + '\n')
+            # FIXME: we should factor out the "bash" method
+            wait_for_prompt()
+            check_exit_status()
+        elif method == "open":
+            # Cheap way to get node1's IP address
+            screen = capture_pane()
+            url = data.replace("/node1", "/{}".format(IPADDR))
+            # This should probably be adapted to run on different OS
+            if state.open_links:
+                subprocess.check_output(["xdg-open", url])
+                focus_browser()
+                if state.interactive:
+                    print("Press any key to continue to next step...")
+                    click.getchar()
+        else:
+            logging.warning("Unknown method {}: {!r}".format(method, data))
+        move_forward()
+
+    else:
+        logging.warning("Unknown command {}.".format(command))

slides/autopilot/gotoslide.js

@@ -0,0 +1,17 @@
+#!/usr/bin/env node
+
+/* Expects a slide number as first argument.
+ * Will connect to the local pub/sub server,
+ * and issue a "go to slide X" command, which
+ * will be sent to all connected browsers.
+ */
+
+var io = require('socket.io-client');
+var socket = io('http://localhost:3000');
+socket.on('connect_error', function(){ 
+  console.log('connection error');
+  socket.close();
+});
+socket.emit('slide change', process.argv[2], function(){
+  socket.close();
+});

slides/autopilot/package-lock.json

@@ -0,0 +1,603 @@
+{
+  "name": "container-training-pub-sub-server",
+  "version": "0.0.1",
+  "lockfileVersion": 1,
+  "requires": true,
+  "dependencies": {
+    "accepts": {
+      "version": "1.3.4",
+      "resolved": "https://registry.npmjs.org/accepts/-/accepts-1.3.4.tgz",
+      "integrity": "sha1-hiRnWMfdbSGmR0/whKR0DsBesh8=",
+      "requires": {
+        "mime-types": "2.1.17",
+        "negotiator": "0.6.1"
+      }
+    },
+    "after": {
+      "version": "0.8.2",
+      "resolved": "https://registry.npmjs.org/after/-/after-0.8.2.tgz",
+      "integrity": "sha1-/ts5T58OAqqXaOcCvaI7UF+ufh8="
+    },
+    "array-flatten": {
+      "version": "1.1.1",
+      "resolved": "https://registry.npmjs.org/array-flatten/-/array-flatten-1.1.1.tgz",
+      "integrity": "sha1-ml9pkFGx5wczKPKgCJaLZOopVdI="
+    },
+    "arraybuffer.slice": {
+      "version": "0.0.6",
+      "resolved": "https://registry.npmjs.org/arraybuffer.slice/-/arraybuffer.slice-0.0.6.tgz",
+      "integrity": "sha1-8zshWfBTKj8xB6JywMz70a0peco="
+    },
+    "async-limiter": {
+      "version": "1.0.0",
+      "resolved": "https://registry.npmjs.org/async-limiter/-/async-limiter-1.0.0.tgz",
+      "integrity": "sha512-jp/uFnooOiO+L211eZOoSyzpOITMXx1rBITauYykG3BRYPu8h0UcxsPNB04RR5vo4Tyz3+ay17tR6JVf9qzYWg=="
+    },
+    "backo2": {
+      "version": "1.0.2",
+      "resolved": "https://registry.npmjs.org/backo2/-/backo2-1.0.2.tgz",
+      "integrity": "sha1-MasayLEpNjRj41s+u2n038+6eUc="
+    },
+    "base64-arraybuffer": {
+      "version": "0.1.5",
+      "resolved": "https://registry.npmjs.org/base64-arraybuffer/-/base64-arraybuffer-0.1.5.tgz",
+      "integrity": "sha1-c5JncZI7Whl0etZmqlzUv5xunOg="
+    },
+    "base64id": {
+      "version": "1.0.0",
+      "resolved": "https://registry.npmjs.org/base64id/-/base64id-1.0.0.tgz",
+      "integrity": "sha1-R2iMuZu2gE8OBtPnY7HDLlfY5rY="
+    },
+    "better-assert": {
+      "version": "1.0.2",
+      "resolved": "https://registry.npmjs.org/better-assert/-/better-assert-1.0.2.tgz",
+      "integrity": "sha1-QIZrnhueC1W0gYlDEeaPr/rrxSI=",
+      "requires": {
+        "callsite": "1.0.0"
+      }
+    },
+    "blob": {
+      "version": "0.0.4",
+      "resolved": "https://registry.npmjs.org/blob/-/blob-0.0.4.tgz",
+      "integrity": "sha1-vPEwUspURj8w+fx+lbmkdjCpSSE="
+    },
+    "body-parser": {
+      "version": "1.18.2",
+      "resolved": "https://registry.npmjs.org/body-parser/-/body-parser-1.18.2.tgz",
+      "integrity": "sha1-h2eKGdhLR9hZuDGZvVm84iKxBFQ=",
+      "requires": {
+        "bytes": "3.0.0",
+        "content-type": "1.0.4",
+        "debug": "2.6.9",
+        "depd": "1.1.1",
+        "http-errors": "1.6.2",
+        "iconv-lite": "0.4.19",
+        "on-finished": "2.3.0",
+        "qs": "6.5.1",
+        "raw-body": "2.3.2",
+        "type-is": "1.6.15"
+      }
+    },
+    "bytes": {
+      "version": "3.0.0",
+      "resolved": "https://registry.npmjs.org/bytes/-/bytes-3.0.0.tgz",
+      "integrity": "sha1-0ygVQE1olpn4Wk6k+odV3ROpYEg="
+    },
+    "callsite": {
+      "version": "1.0.0",
+      "resolved": "https://registry.npmjs.org/callsite/-/callsite-1.0.0.tgz",
+      "integrity": "sha1-KAOY5dZkvXQDi28JBRU+borxvCA="
+    },
+    "component-bind": {
+      "version": "1.0.0",
+      "resolved": "https://registry.npmjs.org/component-bind/-/component-bind-1.0.0.tgz",
+      "integrity": "sha1-AMYIq33Nk4l8AAllGx06jh5zu9E="
+    },
+    "component-emitter": {
+      "version": "1.2.1",
+      "resolved": "https://registry.npmjs.org/component-emitter/-/component-emitter-1.2.1.tgz",
+      "integrity": "sha1-E3kY1teCg/ffemt8WmPhQOaUJeY="
+    },
+    "component-inherit": {
+      "version": "0.0.3",
+      "resolved": "https://registry.npmjs.org/component-inherit/-/component-inherit-0.0.3.tgz",
+      "integrity": "sha1-ZF/ErfWLcrZJ1crmUTVhnbJv8UM="
+    },
+    "content-disposition": {
+      "version": "0.5.2",
+      "resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-0.5.2.tgz",
+      "integrity": "sha1-DPaLud318r55YcOoUXjLhdunjLQ="
+    },
+    "content-type": {
+      "version": "1.0.4",
+      "resolved": "https://registry.npmjs.org/content-type/-/content-type-1.0.4.tgz",
+      "integrity": "sha512-hIP3EEPs8tB9AT1L+NUqtwOAps4mk2Zob89MWXMHjHWg9milF/j4osnnQLXBCBFBk/tvIG/tUc9mOUJiPBhPXA=="
+    },
+    "cookie": {
+      "version": "0.3.1",
+      "resolved": "https://registry.npmjs.org/cookie/-/cookie-0.3.1.tgz",
+      "integrity": "sha1-5+Ch+e9DtMi6klxcWpboBtFoc7s="
+    },
+    "cookie-signature": {
+      "version": "1.0.6",
+      "resolved": "https://registry.npmjs.org/cookie-signature/-/cookie-signature-1.0.6.tgz",
+      "integrity": "sha1-4wOogrNCzD7oylE6eZmXNNqzriw="
+    },
+    "debug": {
+      "version": "2.6.9",
+      "resolved": "https://registry.npmjs.org/debug/-/debug-2.6.9.tgz",
+      "integrity": "sha512-bC7ElrdJaJnPbAP+1EotYvqZsb3ecl5wi6Bfi6BJTUcNowp6cvspg0jXznRTKDjm/E7AdgFBVeAPVMNcKGsHMA==",
+      "requires": {
+        "ms": "2.0.0"
+      }
+    },
+    "depd": {
+      "version": "1.1.1",
+      "resolved": "https://registry.npmjs.org/depd/-/depd-1.1.1.tgz",
+      "integrity": "sha1-V4O04cRZ8G+lyif5kfPQbnoxA1k="
+    },
+    "destroy": {
+      "version": "1.0.4",
+      "resolved": "https://registry.npmjs.org/destroy/-/destroy-1.0.4.tgz",
+      "integrity": "sha1-l4hXRCxEdJ5CBmE+N5RiBYJqvYA="
+    },
+    "ee-first": {
+      "version": "1.1.1",
+      "resolved": "https://registry.npmjs.org/ee-first/-/ee-first-1.1.1.tgz",
+      "integrity": "sha1-WQxhFWsK4vTwJVcyoViyZrxWsh0="
+    },
+    "encodeurl": {
+      "version": "1.0.1",
+      "resolved": "https://registry.npmjs.org/encodeurl/-/encodeurl-1.0.1.tgz",
+      "integrity": "sha1-eePVhlU0aQn+bw9Fpd5oEDspTSA="
+    },
+    "engine.io": {
+      "version": "3.1.4",
+      "resolved": "https://registry.npmjs.org/engine.io/-/engine.io-3.1.4.tgz",
+      "integrity": "sha1-PQIRtwpVLOhB/8fahiezAamkFi4=",
+      "requires": {
+        "accepts": "1.3.3",
+        "base64id": "1.0.0",
+        "cookie": "0.3.1",
+        "debug": "2.6.9",
+        "engine.io-parser": "2.1.1",
+        "uws": "0.14.5",
+        "ws": "3.3.3"
+      },
+      "dependencies": {
+        "accepts": {
+          "version": "1.3.3",
+          "resolved": "https://registry.npmjs.org/accepts/-/accepts-1.3.3.tgz",
+          "integrity": "sha1-w8p0NJOGSMPg2cHjKN1otiLChMo=",
+          "requires": {
+            "mime-types": "2.1.17",
+            "negotiator": "0.6.1"
+          }
+        }
+      }
+    },
+    "engine.io-client": {
+      "version": "3.1.4",
+      "resolved": "https://registry.npmjs.org/engine.io-client/-/engine.io-client-3.1.4.tgz",
+      "integrity": "sha1-T88TcLRxY70s6b4nM5ckMDUNTqE=",
+      "requires": {
+        "component-emitter": "1.2.1",
+        "component-inherit": "0.0.3",
+        "debug": "2.6.9",
+        "engine.io-parser": "2.1.1",
+        "has-cors": "1.1.0",
+        "indexof": "0.0.1",
+        "parseqs": "0.0.5",
+        "parseuri": "0.0.5",
+        "ws": "3.3.3",
+        "xmlhttprequest-ssl": "1.5.4",
+        "yeast": "0.1.2"
+      }
+    },
+    "engine.io-parser": {
+      "version": "2.1.1",
+      "resolved": "https://registry.npmjs.org/engine.io-parser/-/engine.io-parser-2.1.1.tgz",
+      "integrity": "sha1-4Ps/DgRi9/WLt3waUun1p+JuRmg=",
+      "requires": {
+        "after": "0.8.2",
+        "arraybuffer.slice": "0.0.6",
+        "base64-arraybuffer": "0.1.5",
+        "blob": "0.0.4",
+        "has-binary2": "1.0.2"
+      }
+    },
+    "escape-html": {
+      "version": "1.0.3",
+      "resolved": "https://registry.npmjs.org/escape-html/-/escape-html-1.0.3.tgz",
+      "integrity": "sha1-Aljq5NPQwJdN4cFpGI7wBR0dGYg="
+    },
+    "etag": {
+      "version": "1.8.1",
+      "resolved": "https://registry.npmjs.org/etag/-/etag-1.8.1.tgz",
+      "integrity": "sha1-Qa4u62XvpiJorr/qg6x9eSmbCIc="
+    },
+    "express": {
+      "version": "4.16.2",
+      "resolved": "https://registry.npmjs.org/express/-/express-4.16.2.tgz",
+      "integrity": "sha1-41xt/i1kt9ygpc1PIXgb4ymeB2w=",
+      "requires": {
+        "accepts": "1.3.4",
+        "array-flatten": "1.1.1",
+        "body-parser": "1.18.2",
+        "content-disposition": "0.5.2",
+        "content-type": "1.0.4",
+        "cookie": "0.3.1",
+        "cookie-signature": "1.0.6",
+        "debug": "2.6.9",
+        "depd": "1.1.1",
+        "encodeurl": "1.0.1",
+        "escape-html": "1.0.3",
+        "etag": "1.8.1",
+        "finalhandler": "1.1.0",
+        "fresh": "0.5.2",
+        "merge-descriptors": "1.0.1",
+        "methods": "1.1.2",
+        "on-finished": "2.3.0",
+        "parseurl": "1.3.2",
+        "path-to-regexp": "0.1.7",
+        "proxy-addr": "2.0.2",
+        "qs": "6.5.1",
+        "range-parser": "1.2.0",
+        "safe-buffer": "5.1.1",
+        "send": "0.16.1",
+        "serve-static": "1.13.1",
+        "setprototypeof": "1.1.0",
+        "statuses": "1.3.1",
+        "type-is": "1.6.15",
+        "utils-merge": "1.0.1",
+        "vary": "1.1.2"
+      }
+    },
+    "finalhandler": {
+      "version": "1.1.0",
+      "resolved": "https://registry.npmjs.org/finalhandler/-/finalhandler-1.1.0.tgz",
+      "integrity": "sha1-zgtoVbRYU+eRsvzGgARtiCU91/U=",
+      "requires": {
+        "debug": "2.6.9",
+        "encodeurl": "1.0.1",
+        "escape-html": "1.0.3",
+        "on-finished": "2.3.0",
+        "parseurl": "1.3.2",
+        "statuses": "1.3.1",
+        "unpipe": "1.0.0"
+      }
+    },
+    "forwarded": {
+      "version": "0.1.2",
+      "resolved": "https://registry.npmjs.org/forwarded/-/forwarded-0.1.2.tgz",
+      "integrity": "sha1-mMI9qxF1ZXuMBXPozszZGw/xjIQ="
+    },
+    "fresh": {
+      "version": "0.5.2",
+      "resolved": "https://registry.npmjs.org/fresh/-/fresh-0.5.2.tgz",
+      "integrity": "sha1-PYyt2Q2XZWn6g1qx+OSyOhBWBac="
+    },
+    "has-binary2": {
+      "version": "1.0.2",
+      "resolved": "https://registry.npmjs.org/has-binary2/-/has-binary2-1.0.2.tgz",
+      "integrity": "sha1-6D26SfC5vk0CbSc2U1DZ8D9Uvpg=",
+      "requires": {
+        "isarray": "2.0.1"
+      }
+    },
+    "has-cors": {
+      "version": "1.1.0",
+      "resolved": "https://registry.npmjs.org/has-cors/-/has-cors-1.1.0.tgz",
+      "integrity": "sha1-XkdHk/fqmEPRu5nCPu9J/xJv/zk="
+    },
+    "http-errors": {
+      "version": "1.6.2",
+      "resolved": "https://registry.npmjs.org/http-errors/-/http-errors-1.6.2.tgz",
+      "integrity": "sha1-CgAsyFcHGSp+eUbO7cERVfYOxzY=",
+      "requires": {
+        "depd": "1.1.1",
+        "inherits": "2.0.3",
+        "setprototypeof": "1.0.3",
+        "statuses": "1.3.1"
+      },
+      "dependencies": {
+        "setprototypeof": {
+          "version": "1.0.3",
+          "resolved": "https://registry.npmjs.org/setprototypeof/-/setprototypeof-1.0.3.tgz",
+          "integrity": "sha1-ZlZ+NwQ+608E2RvWWMDL77VbjgQ="
+        }
+      }
+    },
+    "iconv-lite": {
+      "version": "0.4.19",
+      "resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.4.19.tgz",
+      "integrity": "sha512-oTZqweIP51xaGPI4uPa56/Pri/480R+mo7SeU+YETByQNhDG55ycFyNLIgta9vXhILrxXDmF7ZGhqZIcuN0gJQ=="
+    },
+    "indexof": {
+      "version": "0.0.1",
+      "resolved": "https://registry.npmjs.org/indexof/-/indexof-0.0.1.tgz",
+      "integrity": "sha1-gtwzbSMrkGIXnQWrMpOmYFn9Q10="
+    },
+    "inherits": {
+      "version": "2.0.3",
+      "resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.3.tgz",
+      "integrity": "sha1-Yzwsg+PaQqUC9SRmAiSA9CCCYd4="
+    },
+    "ipaddr.js": {
+      "version": "1.5.2",
+      "resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-1.5.2.tgz",
+      "integrity": "sha1-1LUFvemUaYfM8PxY2QEP+WB+P6A="
+    },
+    "isarray": {
+      "version": "2.0.1",
+      "resolved": "https://registry.npmjs.org/isarray/-/isarray-2.0.1.tgz",
+      "integrity": "sha1-o32U7ZzaLVmGXJ92/llu4fM4dB4="
+    },
+    "media-typer": {
+      "version": "0.3.0",
+      "resolved": "https://registry.npmjs.org/media-typer/-/media-typer-0.3.0.tgz",
+      "integrity": "sha1-hxDXrwqmJvj/+hzgAWhUUmMlV0g="
+    },
+    "merge-descriptors": {
+      "version": "1.0.1",
+      "resolved": "https://registry.npmjs.org/merge-descriptors/-/merge-descriptors-1.0.1.tgz",
+      "integrity": "sha1-sAqqVW3YtEVoFQ7J0blT8/kMu2E="
+    },
+    "methods": {
+      "version": "1.1.2",
+      "resolved": "https://registry.npmjs.org/methods/-/methods-1.1.2.tgz",
+      "integrity": "sha1-VSmk1nZUE07cxSZmVoNbD4Ua/O4="
+    },
+    "mime": {
+      "version": "1.4.1",
+      "resolved": "https://registry.npmjs.org/mime/-/mime-1.4.1.tgz",
+      "integrity": "sha512-KI1+qOZu5DcW6wayYHSzR/tXKCDC5Om4s1z2QJjDULzLcmf3DvzS7oluY4HCTrc+9FiKmWUgeNLg7W3uIQvxtQ=="
+    },
+    "mime-db": {
+      "version": "1.30.0",
+      "resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.30.0.tgz",
+      "integrity": "sha1-dMZD2i3Z1qRTmZY0ZbJtXKfXHwE="
+    },
+    "mime-types": {
+      "version": "2.1.17",
+      "resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.17.tgz",
+      "integrity": "sha1-Cdejk/A+mVp5+K+Fe3Cp4KsWVXo=",
+      "requires": {
+        "mime-db": "1.30.0"
+      }
+    },
+    "ms": {
+      "version": "2.0.0",
+      "resolved": "https://registry.npmjs.org/ms/-/ms-2.0.0.tgz",
+      "integrity": "sha1-VgiurfwAvmwpAd9fmGF4jeDVl8g="
+    },
+    "negotiator": {
+      "version": "0.6.1",
+      "resolved": "https://registry.npmjs.org/negotiator/-/negotiator-0.6.1.tgz",
+      "integrity": "sha1-KzJxhOiZIQEXeyhWP7XnECrNDKk="
+    },
+    "object-component": {
+      "version": "0.0.3",
+      "resolved": "https://registry.npmjs.org/object-component/-/object-component-0.0.3.tgz",
+      "integrity": "sha1-8MaapQ78lbhmwYb0AKM3acsvEpE="
+    },
+    "on-finished": {
+      "version": "2.3.0",
+      "resolved": "https://registry.npmjs.org/on-finished/-/on-finished-2.3.0.tgz",
+      "integrity": "sha1-IPEzZIGwg811M3mSoWlxqi2QaUc=",
+      "requires": {
+        "ee-first": "1.1.1"
+      }
+    },
+    "parseqs": {
+      "version": "0.0.5",
+      "resolved": "https://registry.npmjs.org/parseqs/-/parseqs-0.0.5.tgz",
+      "integrity": "sha1-1SCKNzjkZ2bikbouoXNoSSGouJ0=",
+      "requires": {
+        "better-assert": "1.0.2"
+      }
+    },
+    "parseuri": {
+      "version": "0.0.5",
+      "resolved": "https://registry.npmjs.org/parseuri/-/parseuri-0.0.5.tgz",
+      "integrity": "sha1-gCBKUNTbt3m/3G6+J3jZDkvOMgo=",
+      "requires": {
+        "better-assert": "1.0.2"
+      }
+    },
+    "parseurl": {
+      "version": "1.3.2",
+      "resolved": "https://registry.npmjs.org/parseurl/-/parseurl-1.3.2.tgz",
+      "integrity": "sha1-/CidTtiZMRlGDBViUyYs3I3mW/M="
+    },
+    "path-to-regexp": {
+      "version": "0.1.7",
+      "resolved": "https://registry.npmjs.org/path-to-regexp/-/path-to-regexp-0.1.7.tgz",
+      "integrity": "sha1-32BBeABfUi8V60SQ5yR6G/qmf4w="
+    },
+    "proxy-addr": {
+      "version": "2.0.2",
+      "resolved": "https://registry.npmjs.org/proxy-addr/-/proxy-addr-2.0.2.tgz",
+      "integrity": "sha1-ZXFQT0e7mI7IGAJT+F3X4UlSvew=",
+      "requires": {
+        "forwarded": "0.1.2",
+        "ipaddr.js": "1.5.2"
+      }
+    },
+    "qs": {
+      "version": "6.5.1",
+      "resolved": "https://registry.npmjs.org/qs/-/qs-6.5.1.tgz",
+      "integrity": "sha512-eRzhrN1WSINYCDCbrz796z37LOe3m5tmW7RQf6oBntukAG1nmovJvhnwHHRMAfeoItc1m2Hk02WER2aQ/iqs+A=="
+    },
+    "range-parser": {
+      "version": "1.2.0",
+      "resolved": "https://registry.npmjs.org/range-parser/-/range-parser-1.2.0.tgz",
+      "integrity": "sha1-9JvmtIeJTdxA3MlKMi9hEJLgDV4="
+    },
+    "raw-body": {
+      "version": "2.3.2",
+      "resolved": "https://registry.npmjs.org/raw-body/-/raw-body-2.3.2.tgz",
+      "integrity": "sha1-vNYMd9Prk83gBQKVw/N5OJvIj4k=",
+      "requires": {
+        "bytes": "3.0.0",
+        "http-errors": "1.6.2",
+        "iconv-lite": "0.4.19",
+        "unpipe": "1.0.0"
+      }
+    },
+    "safe-buffer": {
+      "version": "5.1.1",
+      "resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.1.1.tgz",
+      "integrity": "sha512-kKvNJn6Mm93gAczWVJg7wH+wGYWNrDHdWvpUmHyEsgCtIwwo3bqPtV4tR5tuPaUhTOo/kvhVwd8XwwOllGYkbg=="
+    },
+    "send": {
+      "version": "0.16.1",
+      "resolved": "https://registry.npmjs.org/send/-/send-0.16.1.tgz",
+      "integrity": "sha512-ElCLJdJIKPk6ux/Hocwhk7NFHpI3pVm/IZOYWqUmoxcgeyM+MpxHHKhb8QmlJDX1pU6WrgaHBkVNm73Sv7uc2A==",
+      "requires": {
+        "debug": "2.6.9",
+        "depd": "1.1.1",
+        "destroy": "1.0.4",
+        "encodeurl": "1.0.1",
+        "escape-html": "1.0.3",
+        "etag": "1.8.1",
+        "fresh": "0.5.2",
+        "http-errors": "1.6.2",
+        "mime": "1.4.1",
+        "ms": "2.0.0",
+        "on-finished": "2.3.0",
+        "range-parser": "1.2.0",
+        "statuses": "1.3.1"
+      }
+    },
+    "serve-static": {
+      "version": "1.13.1",
+      "resolved": "https://registry.npmjs.org/serve-static/-/serve-static-1.13.1.tgz",
+      "integrity": "sha512-hSMUZrsPa/I09VYFJwa627JJkNs0NrfL1Uzuup+GqHfToR2KcsXFymXSV90hoyw3M+msjFuQly+YzIH/q0MGlQ==",
+      "requires": {
+        "encodeurl": "1.0.1",
+        "escape-html": "1.0.3",
+        "parseurl": "1.3.2",
+        "send": "0.16.1"
+      }
+    },
+    "setprototypeof": {
+      "version": "1.1.0",
+      "resolved": "https://registry.npmjs.org/setprototypeof/-/setprototypeof-1.1.0.tgz",
+      "integrity": "sha512-BvE/TwpZX4FXExxOxZyRGQQv651MSwmWKZGqvmPcRIjDqWub67kTKuIMx43cZZrS/cBBzwBcNDWoFxt2XEFIpQ=="
+    },
+    "socket.io": {
+      "version": "2.0.4",
+      "resolved": "https://registry.npmjs.org/socket.io/-/socket.io-2.0.4.tgz",
+      "integrity": "sha1-waRZDO/4fs8TxyZS8Eb3FrKeYBQ=",
+      "requires": {
+        "debug": "2.6.9",
+        "engine.io": "3.1.4",
+        "socket.io-adapter": "1.1.1",
+        "socket.io-client": "2.0.4",
+        "socket.io-parser": "3.1.2"
+      }
+    },
+    "socket.io-adapter": {
+      "version": "1.1.1",
+      "resolved": "https://registry.npmjs.org/socket.io-adapter/-/socket.io-adapter-1.1.1.tgz",
+      "integrity": "sha1-KoBeihTWNyEk3ZFZrUUC+MsH8Gs="
+    },
+    "socket.io-client": {
+      "version": "2.0.4",
+      "resolved": "https://registry.npmjs.org/socket.io-client/-/socket.io-client-2.0.4.tgz",
+      "integrity": "sha1-CRilUkBtxeVAs4Dc2Xr8SmQzL44=",
+      "requires": {
+        "backo2": "1.0.2",
+        "base64-arraybuffer": "0.1.5",
+        "component-bind": "1.0.0",
+        "component-emitter": "1.2.1",
+        "debug": "2.6.9",
+        "engine.io-client": "3.1.4",
+        "has-cors": "1.1.0",
+        "indexof": "0.0.1",
+        "object-component": "0.0.3",
+        "parseqs": "0.0.5",
+        "parseuri": "0.0.5",
+        "socket.io-parser": "3.1.2",
+        "to-array": "0.1.4"
+      }
+    },
+    "socket.io-parser": {
+      "version": "3.1.2",
+      "resolved": "https://registry.npmjs.org/socket.io-parser/-/socket.io-parser-3.1.2.tgz",
+      "integrity": "sha1-28IoIVH8T6675Aru3Ady66YZ9/I=",
+      "requires": {
+        "component-emitter": "1.2.1",
+        "debug": "2.6.9",
+        "has-binary2": "1.0.2",
+        "isarray": "2.0.1"
+      }
+    },
+    "statuses": {
+      "version": "1.3.1",
+      "resolved": "https://registry.npmjs.org/statuses/-/statuses-1.3.1.tgz",
+      "integrity": "sha1-+vUbnrdKrvOzrPStX2Gr8ky3uT4="
+    },
+    "to-array": {
+      "version": "0.1.4",
+      "resolved": "https://registry.npmjs.org/to-array/-/to-array-0.1.4.tgz",
+      "integrity": "sha1-F+bBH3PdTz10zaek/zI46a2b+JA="
+    },
+    "type-is": {
+      "version": "1.6.15",
+      "resolved": "https://registry.npmjs.org/type-is/-/type-is-1.6.15.tgz",
+      "integrity": "sha1-yrEPtJCeRByChC6v4a1kbIGARBA=",
+      "requires": {
+        "media-typer": "0.3.0",
+        "mime-types": "2.1.17"
+      }
+    },
+    "ultron": {
+      "version": "1.1.1",
+      "resolved": "https://registry.npmjs.org/ultron/-/ultron-1.1.1.tgz",
+      "integrity": "sha512-UIEXBNeYmKptWH6z8ZnqTeS8fV74zG0/eRU9VGkpzz+LIJNs8W/zM/L+7ctCkRrgbNnnR0xxw4bKOr0cW0N0Og=="
+    },
+    "unpipe": {
+      "version": "1.0.0",
+      "resolved": "https://registry.npmjs.org/unpipe/-/unpipe-1.0.0.tgz",
+      "integrity": "sha1-sr9O6FFKrmFltIF4KdIbLvSZBOw="
+    },
+    "utils-merge": {
+      "version": "1.0.1",
+      "resolved": "https://registry.npmjs.org/utils-merge/-/utils-merge-1.0.1.tgz",
+      "integrity": "sha1-n5VxD1CiZ5R7LMwSR0HBAoQn5xM="
+    },
+    "uws": {
+      "version": "0.14.5",
+      "resolved": "https://registry.npmjs.org/uws/-/uws-0.14.5.tgz",
+      "integrity": "sha1-Z6rzPEaypYel9mZtAPdpEyjxSdw=",
+      "optional": true
+    },
+    "vary": {
+      "version": "1.1.2",
+      "resolved": "https://registry.npmjs.org/vary/-/vary-1.1.2.tgz",
+      "integrity": "sha1-IpnwLG3tMNSllhsLn3RSShj2NPw="
+    },
+    "ws": {
+      "version": "3.3.3",
+      "resolved": "https://registry.npmjs.org/ws/-/ws-3.3.3.tgz",
+      "integrity": "sha512-nnWLa/NwZSt4KQJu51MYlCcSQ5g7INpOrOMt4XV8j4dqTXdmlUmSHQ8/oLC069ckre0fRsgfvsKwbTdtKLCDkA==",
+      "requires": {
+        "async-limiter": "1.0.0",
+        "safe-buffer": "5.1.1",
+        "ultron": "1.1.1"
+      }
+    },
+    "xmlhttprequest-ssl": {
+      "version": "1.5.4",
+      "resolved": "https://registry.npmjs.org/xmlhttprequest-ssl/-/xmlhttprequest-ssl-1.5.4.tgz",
+      "integrity": "sha1-BPVgkVcks4kIhxXMDteBPpZ3v1c="
+    },
+    "yeast": {
+      "version": "0.1.2",
+      "resolved": "https://registry.npmjs.org/yeast/-/yeast-0.1.2.tgz",
+      "integrity": "sha1-AI4G2AlDIMNy28L47XagymyKxBk="
+    }
+  }
+}

slides/autopilot/package.json

@@ -0,0 +1,8 @@
+{
+  "name": "container-training-pub-sub-server",
+  "version": "0.0.1",
+  "dependencies": {
+    "express": "^4.16.2",
+    "socket.io": "^2.0.4"
+  }
+}

slides/autopilot/remote.js

@@ -0,0 +1,21 @@
+/* This snippet is loaded from the workshop HTML file.
+ * It sets up callbacks to synchronize the local slide
+ * number with the remote pub/sub server.
+ */
+
+var socket = io();
+var leader = true;
+
+slideshow.on('showSlide', function (slide) {
+  if (leader) {
+    var n = slide.getSlideIndex()+1;
+    socket.emit('slide change', n);
+  }
+});
+
+socket.on('slide change', function (n) {
+  leader = false;
+  slideshow.gotoSlide(n);
+  leader = true;
+});
+

slides/autopilot/requirements.txt

@@ -0,0 +1 @@
+click

slides/autopilot/server.js

@@ -0,0 +1,41 @@
+#!/usr/bin/env node
+
+/* This is a very simple pub/sub server, allowing to
+ * remote control browsers displaying the slides.
+ * The browsers connect to this pub/sub server using
+ * Socket.IO, and the server tells them which slides
+ * to display.
+ *
+ * The server can be controlled with a little CLI,
+ * or by one of the browsers.
+ */
+
+var express = require('express');
+var app = express();
+var http = require('http').Server(app);
+var io = require('socket.io')(http);
+
+app.get('/', function(req, res){
+  res.send('container.training autopilot pub/sub server');
+});
+
+/* Serve remote.js from the current directory */
+app.use(express.static('.'));
+
+/* Serve slides etc. from current and the parent directory */
+app.use(express.static('..'));
+
+io.on('connection', function(socket){
+  console.log('a client connected: ' + socket.handshake.address);
+  socket.on('slide change', function(n, ack){
+    console.log('slide change: ' + n);
+    socket.broadcast.emit('slide change', n);
+    if (typeof ack === 'function') {
+      ack();
+    }
+  });
+});
+
+http.listen(3000, function(){
+  console.log('listening on *:3000');
+});

slides/autopilot/tmux-style.sh

@@ -0,0 +1,7 @@
+#!/bin/sh
+# This removes the clock (and other extraneous stuff) from the
+# tmux status bar, and it gives it a non-default color.
+tmux set-option -g status-left ""
+tmux set-option -g status-right ""
+tmux set-option -g status-style bg=cyan
+

slides/autotest.py

@@ -1,279 +0,0 @@
-#!/usr/bin/env python
-# coding: utf-8
-
-import click
-import logging
-import os
-import random
-import re
-import subprocess
-import sys
-import time
-import uuid
-
-logging.basicConfig(level=os.environ.get("LOG_LEVEL", "INFO"))
-
-interactive = True
-verify_status = False
-simulate_type = True
-
-TIMEOUT = 60 # 1 minute
-
-
-def hrule():
-    return "="*int(subprocess.check_output(["tput", "cols"]))
-
-# A "snippet" is something that the user is supposed to do in the workshop.
-# Most of the "snippets" are shell commands.
-# Some of them can be key strokes or other actions.
-# In the markdown source, they are the code sections (identified by triple-
-# quotes) within .exercise[] sections.
-
-class Snippet(object):
-
-    def __init__(self, slide, content):
-        self.slide = slide
-        self.content = content
-        self.actions = []
-
-    def __str__(self):
-        return self.content
-
-
-class Slide(object):
-
-    current_slide = 0
-
-    def __init__(self, content):
-        Slide.current_slide += 1
-        self.number = Slide.current_slide
-
-        # Remove commented-out slides
-        # (remark.js considers ??? to be the separator for speaker notes)
-        content = re.split("\n\?\?\?\n", content)[0]
-        self.content = content
-
-        self.snippets = []
-        exercises = re.findall("\.exercise\[(.*)\]", content, re.DOTALL)
-        for exercise in exercises:
-            if "```" in exercise:
-                for snippet in exercise.split("```")[1::2]:
-                    self.snippets.append(Snippet(self, snippet))
-            else:
-                logging.warning("Exercise on slide {} does not have any ``` snippet."
-                                .format(self.number))
-                self.debug()
-
-    def __str__(self):
-        text = self.content
-        for snippet in self.snippets:
-            text = text.replace(snippet.content, ansi("7")(snippet.content))
-        return text
-
-    def debug(self):
-        logging.debug("\n{}\n{}\n{}".format(hrule(), self.content, hrule()))
-
-
-def ansi(code):
-    return lambda s: "\x1b[{}m{}\x1b[0m".format(code, s)
-
-
-def wait_for_string(s, timeout=TIMEOUT):
-    logging.debug("Waiting for string: {}".format(s))
-    deadline = time.time() + timeout
-    while time.time() < deadline:
-        output = capture_pane()
-        if s in output:
-            return
-        time.sleep(1)
-    raise Exception("Timed out while waiting for {}!".format(s))
-
-
-def wait_for_prompt():
-    logging.debug("Waiting for prompt.")
-    deadline = time.time() + TIMEOUT
-    while time.time() < deadline:
-        output = capture_pane()
-        # If we are not at the bottom of the screen, there will be a bunch of extra \n's
-        output = output.rstrip('\n')
-        if output.endswith("\n$"):
-            return
-        if output.endswith("\n/ #"):
-            return
-        time.sleep(1)
-    raise Exception("Timed out while waiting for prompt!")
-
-
-def check_exit_status():
-    if not verify_status:
-        return
-    token = uuid.uuid4().hex
-    data = "echo {} $?\n".format(token)
-    logging.debug("Sending {!r} to get exit status.".format(data))
-    send_keys(data)
-    time.sleep(0.5)
-    wait_for_prompt()
-    screen = capture_pane()
-    status = re.findall("\n{} ([0-9]+)\n".format(token), screen, re.MULTILINE)
-    logging.debug("Got exit status: {}.".format(status))
-    if len(status) == 0:
-        raise Exception("Couldn't retrieve status code {}. Timed out?".format(token))
-    if len(status) > 1:
-        raise Exception("More than one status code {}. I'm seeing double! Shoot them both.".format(token))
-    code = int(status[0])
-    if code != 0:
-        raise Exception("Non-zero exit status: {}.".format(code))
-    # Otherwise just return peacefully.
-
-
-def setup_tmux_and_ssh():
-    if subprocess.call(["tmux", "has-session"]):
-        logging.info("Couldn't connect to tmux. A new tmux session will be created.")
-        subprocess.check_call(["tmux", "new-session", "-d"])
-        wait_for_string("$")
-        send_keys("cd ../prepare-vms\n")
-        send_keys("ssh docker@$(head -n1 ips.txt)\n")
-        wait_for_string("password:")
-        send_keys("training\n")
-        wait_for_prompt()
-    else:
-        logging.info("Found tmux session. Trying to acquire shell prompt.")
-        wait_for_prompt()
-    logging.info("Successfully connected to test cluster in tmux session.")
-
-
-
-slides = []
-content = open(sys.argv[1]).read()
-for slide in re.split("\n---?\n", content):
-    slides.append(Slide(slide))
-
-actions = []
-for slide in slides:
-    for snippet in slide.snippets:
-        content = snippet.content
-        # Extract the "method" (e.g. bash, keys, ...)
-        # On multi-line snippets, the method is alone on the first line
-        # On single-line snippets, the data follows the method immediately
-        if '\n' in content:
-            method, data = content.split('\n', 1)
-        else:
-            method, data = content.split(' ', 1)
-        actions.append((slide, snippet, method, data))
-
-
-def send_keys(data):
-    if simulate_type and data[0] != '^':
-        for key in data:
-            if key == ";":
-                key = "\\;"
-            subprocess.check_call(["tmux", "send-keys", key])
-            time.sleep(0.1*random.random())
-    else:
-        subprocess.check_call(["tmux", "send-keys", data])
-
-def capture_pane():
-    return subprocess.check_output(["tmux", "capture-pane", "-p"])
-
-
-setup_tmux_and_ssh()
-
-
-try:
-    i = int(open("nextstep").read())
-    logging.info("Loaded next step ({}) from file.".format(i))
-except Exception as e:
-    logging.warning("Could not read nextstep file ({}), initializing to 0.".format(e))
-    i = 0
-
-while i < len(actions):
-    with open("nextstep", "w") as f:
-        f.write(str(i))
-    slide, snippet, method, data = actions[i]
-
-    # Remove extra spaces (we don't want them in the terminal) and carriage returns
-    data = data.strip()
-
-    print(hrule())
-    print(slide.content.replace(snippet.content, ansi(7)(snippet.content)))
-    print(hrule())
-    if interactive:
-        print("[{}/{}] Shall we execute that snippet above?".format(i, len(actions)))
-        print("y/⏎/→   Execute snippet")
-        print("s       Skip snippet")
-        print("g       Go to a specific snippet")
-        print("q       Quit")
-        print("c       Continue non-interactively until next error")
-        command = click.getchar()
-    else:
-        command = "y"
-
-    # For now, remove the `highlighted` sections
-    # (Make sure to use $() in shell snippets!)
-    if '`' in data:
-        logging.info("Stripping ` from snippet.")
-        data = data.replace('`', '')
-
-    if command == "s":
-        i += 1
-    elif command == "g":
-        i = click.prompt("Enter snippet number", type=int)
-    elif command == "q":
-        break
-    elif command == "c":
-        # continue until next timeout
-        interactive = False
-    elif command in ("y", "\r", " ", "\x1b[C"):
-        logging.info("Running with method {}: {}".format(method, data))
-        if method == "keys":
-            send_keys(data)
-        elif method == "bash":
-            # Make sure that we're ready
-            wait_for_prompt()
-            # Strip leading spaces
-            data = re.sub("\n +", "\n", data)
-            # Add "RETURN" at the end of the command :)
-            data += "\n"
-            # Send command
-            send_keys(data)
-            # Force a short sleep to avoid race condition
-            time.sleep(0.5)
-            _, _, next_method, next_data = actions[i+1]
-            if next_method == "wait":
-                wait_for_string(next_data)
-            elif next_method == "longwait":
-                wait_for_string(next_data, 10*TIMEOUT)
-            else:
-                wait_for_prompt()
-                # Verify return code FIXME should be optional
-                check_exit_status()
-        elif method == "copypaste":
-            screen = capture_pane()
-            matches = re.findall(data, screen, flags=re.DOTALL)
-            if len(matches) == 0:
-                raise Exception("Could not find regex {} in output.".format(data))
-            # Arbitrarily get the most recent match
-            match = matches[-1]
-            # Remove line breaks (like a screen copy paste would do)
-            match = match.replace('\n', '')
-            send_keys(match + '\n')
-            # FIXME: we should factor out the "bash" method
-            wait_for_prompt()
-            check_exit_status()
-        elif method == "open":
-            # Cheap way to get node1's IP address
-            screen = capture_pane()
-            ipaddr = re.findall("^\[(.*)\]", screen, re.MULTILINE)[-1]
-            url = data.replace("/node1", "/{}".format(ipaddr))
-            # This should probably be adapted to run on different OS
-            subprocess.check_call(["open", url])
-        else:
-            logging.warning("Unknown method {}: {!r}".format(method, data))
-        i += 1
-
-    else:
-        logging.warning("Unknown command {}.".format(command))
-
-# Reset slide counter
-with open("nextstep", "w") as f:
-    f.write(str(0))

slides/build.sh

@@ -1,6 +1,8 @@
 #!/bin/sh
+set -e
 case "$1" in
 once)
+  ./index.py
   for YAML in *.yml; do
     ./markmaker.py $YAML > $YAML.html || { 
       rm $YAML.html
@@ -15,6 +17,13 @@ once)
   ;;
 
 forever)
+  set +e
+  # check if entr is installed
+  if ! command -v entr >/dev/null; then
+    echo >&2 "First install 'entr' with apt, brew, etc."
+    exit
+  fi
+  
   # There is a weird bug in entr, at least on MacOS,
   # where it doesn't restore the terminal to a clean
   # state when exitting. So let's try to work around

slides/common/prereqs.md

@@ -1,230 +0,0 @@
-# Pre-requirements
-
-- Be comfortable with the UNIX command line
-
-  - navigating directories
-
-  - editing files
-
-  - a little bit of bash-fu (environment variables, loops)
-
-- Some Docker knowledge
-
-  - `docker run`, `docker ps`, `docker build`
-
-  - ideally, you know how to write a Dockerfile and build it
-    <br/>
-    (even if it's a `FROM` line and a couple of `RUN` commands)
-
-- It's totally OK if you are not a Docker expert!
-
----
-
-class: extra-details
-
-## Extra details
-
-- This slide should have a little magnifying glass in the top left corner
-
-  (If it doesn't, it's because CSS is hard — Jérôme is only a backend person, alas)
-
-- Slides with that magnifying glass indicate slides providing extra details
-
-- Feel free to skip them if you're in a hurry!
-
----
-
-## Hands-on sections
-
-- The whole workshop is hands-on
-
-- We are going to build, ship, and run containers!
-
-- You are invited to reproduce all the demos
-
-- All hands-on sections are clearly identified, like the gray rectangle below
-
-.exercise[
-
-- This is the stuff you're supposed to do!
-
-- Go to [container.training](http://container.training/) to view these slides
-
-- Join the chat room on @@CHAT@@
-
-]
-
----
-
-class: in-person
-
-## Where are we going to run our containers?
-
----
-
-class: in-person, pic
-
-![You get five VMs](images/you-get-five-vms.jpg)
-
----
-
-class: in-person
-
-## You get five VMs
-
-- Each person gets 5 private VMs (not shared with anybody else)
-
-- They'll remain up for the duration of the workshop
-
-- You should have a little card with login+password+IP addresses
-
-- You can automatically SSH from one VM to another
-
-- The nodes have aliases: `node1`, `node2`, etc.
-
----
-
-class: in-person
-
-## Why don't we run containers locally?
-
-- Installing that stuff can be hard on some machines
-
-  (32 bits CPU or OS... Laptops without administrator access... etc.)
-
-- *"The whole team downloaded all these container images from the WiFi!
-  <br/>... and it went great!"* (Literally no-one ever)
-
-- All you need is a computer (or even a phone or tablet!), with:
-
-  - an internet connection
-
-  - a web browser
-
-  - an SSH client
-
----
-
-class: in-person
-
-## SSH clients
-
-- On Linux, OS X, FreeBSD... you are probably all set
-
-- On Windows, get one of these:
-
-  - [putty](http://www.putty.org/)
-  - Microsoft [Win32 OpenSSH](https://github.com/PowerShell/Win32-OpenSSH/wiki/Install-Win32-OpenSSH)
-  - [Git BASH](https://git-for-windows.github.io/)
-  - [MobaXterm](http://mobaxterm.mobatek.net/)
-
-- On Android, [JuiceSSH](https://juicessh.com/)
-  ([Play Store](https://play.google.com/store/apps/details?id=com.sonelli.juicessh))
-  works pretty well
-
-- Nice-to-have: [Mosh](https://mosh.org/) instead of SSH, if your internet connection tends to lose packets
-  <br/>(available with `(apt|yum|brew) install mosh`; then connect with `mosh user@host`)
-
----
-
-class: in-person
-
-## Connecting to our lab environment
-
-.exercise[
-
-- Log into the first VM (`node1`) with SSH or MOSH
-
-<!--
-```bash
-for N in $(seq 1 5); do
-  ssh -o StrictHostKeyChecking=no node$N true
-done
-```
-
-```bash
-if which kubectl; then
-  kubectl get all -o name | grep -v services/kubernetes | xargs -n1 kubectl delete
-fi
-```
--->
-
-- Check that you can SSH (without password) to `node2`:
-  ```bash
-  ssh node2
-  ```
-- Type `exit` or `^D` to come back to node1
-
-<!-- ```bash exit``` -->
-
-]
-
-If anything goes wrong — ask for help!
-
----
-
-## Doing or re-doing the workshop on your own?
-
-- Use something like
-  [Play-With-Docker](http://play-with-docker.com/) or
-  [Play-With-Kubernetes](https://medium.com/@marcosnils/introducing-pwk-play-with-k8s-159fcfeb787b)
-
-  Zero setup effort; but environment are short-lived and
-  might have limited resources
-
-- Create your own cluster (local or cloud VMs)
-
-  Small setup effort; small cost; flexible environments
-
-- Create a bunch of clusters for you and your friends
-    ([instructions](https://github.com/jpetazzo/container.training/tree/master/prepare-vms))
-
-  Bigger setup effort; ideal for group training
-
----
-
-## We will (mostly) interact with node1 only
-
-*These remarks apply only when using multiple nodes, of course.*
-
-- Unless instructed, **all commands must be run from the first VM, `node1`**
-
-- We will only checkout/copy the code on `node1`
-
-- During normal operations, we do not need access to the other nodes
-
-- If we had to troubleshoot issues, we would use a combination of:
-
-  - SSH (to access system logs, daemon status...)
-  
-  - Docker API (to check running containers and container engine status)
-
----
-
-## Terminals
-
-Once in a while, the instructions will say:
-<br/>"Open a new terminal."
-
-There are multiple ways to do this:
-
-- create a new window or tab on your machine, and SSH into the VM;
-
-- use screen or tmux on the VM and open a new window from there.
-
-You are welcome to use the method that you feel the most comfortable with.
-
----
-
-## Tmux cheatsheet
-
-- Ctrl-b c → creates a new window
-- Ctrl-b n → go to next window
-- Ctrl-b p → go to previous window
-- Ctrl-b " → split window top/bottom
-- Ctrl-b % → split window left/right
-- Ctrl-b Alt-1 → rearrange windows in columns
-- Ctrl-b Alt-2 → rearrange windows in rows
-- Ctrl-b arrows → navigate to other windows
-- Ctrl-b d → detach session
-- tmux attach → reattach to session