Compare commits

...

75 Commits

Author SHA1 Message Date
Jerome Petazzoni
0900d605ef fix-redirects.sh: adding forced redirect 2020-04-07 16:48:55 -05:00
Jerome Petazzoni
f67cfa8693 Add Lucas' amazing diagram 2018-02-21 18:17:57 -08:00
Jerome Petazzoni
cb8690f4a3 Add Lucas' amazing diagram 2018-02-21 18:16:59 -08:00
Jérôme Petazzoni
7a6d488d60 Merge pull request #112 from bridgetkromhout/indexconf2018
Clarification
2018-02-21 18:12:45 -08:00
Bridget Kromhout
b1d8b5eec8 Merge branch 'master' of https://github.com/jpetazzo/container.training into indexconf2018 2018-02-21 18:10:04 -08:00
Bridget Kromhout
1983d6cb4f Detail fix 2018-02-21 18:06:12 -08:00
Jérôme Petazzoni
e565da49ca Merge pull request #111 from bridgetkromhout/indexconf2018
Indexconf2018
2018-02-20 15:07:49 -08:00
Bridget Kromhout
ee33799a8f Only 2 worker nodes 2018-02-20 15:34:05 -06:00
Bridget Kromhout
b61426a044 emoji fix 2018-02-19 19:41:39 -06:00
Bridget Kromhout
fd057d8a1e Slight edits 2018-02-19 19:40:16 -06:00
Jerome Petazzoni
aab519177d Add indexconf2018 to index 2018-02-19 15:55:22 -08:00
Jerome Petazzoni
4b76fbcc4b Add _redirects 2018-02-19 15:50:22 -08:00
Bridget Kromhout
b25d40f48e indexconf2017 2018-02-18 20:21:55 -06:00
Jerome Petazzoni
5116ad7c44 Use kubeadm token generate to simplify things a bit.
Thanks @rmb938 for the suggestion!

Closes #110.
2018-01-18 21:14:40 +01:00
Jerome Petazzoni
7305e911e5 Update for k8s 1.9 2018-01-10 17:12:49 +01:00
Jerome Petazzoni
b2f670acf6 Add error checking for AMI finder script 2018-01-10 16:48:04 +01:00
Jerome Petazzoni
dc040aa693 Make sleep interruptible; fix slide count 2017-12-23 19:38:32 +01:00
Jerome Petazzoni
9b7a8494b0 Fix logic to advance to next snippet 2017-12-23 18:49:42 +01:00
Jerome Petazzoni
ae6c1bb8eb Major UI refactor
Navigation now includes all slides and all snippets.
ENTER skips to the next snippet, or executes the
selected snippet.

More improvements to come: allow SPACE to navigate
step by step through slides and snippets, executing
the snippets.
2017-12-23 09:31:01 +01:00
Jerome Petazzoni
a9a4f0ea07 Create only one remote session 2017-12-23 05:06:30 +01:00
Jerome Petazzoni
68af5940e3 Script node3 setup as well 2017-12-23 05:00:22 +01:00
Jerome Petazzoni
9df5313da4 Remove spurious output from desktop integration 2017-12-22 23:15:21 +01:00
Jerome Petazzoni
ba3f00e64e Clear screen before showing UI 2017-12-22 23:11:11 +01:00
Jerome Petazzoni
4d7a6d5c70 Stupid typo 2017-12-22 23:03:48 +01:00
Jerome Petazzoni
aef833c3f5 Add pause before switching away from browser 2017-12-22 23:02:47 +01:00
Jerome Petazzoni
6f58fee29b Automatically open links in intro section 2017-12-22 22:54:08 +01:00
Jerome Petazzoni
dda09ddbcb slightly edit tmux commands 2017-12-22 22:49:53 +01:00
Jerome Petazzoni
8b13fe6eb4 fix formatting in PWD reference 2017-12-22 22:48:42 +01:00
Jerome Petazzoni
21f345a96a Improve open command 2017-12-22 22:42:41 +01:00
Jerome Petazzoni
eaa4dc63bf Instruct to use PWD in self-paced mode 2017-12-22 22:40:23 +01:00
Jerome Petazzoni
af5ea2188b More typos 2017-12-22 22:26:08 +01:00
Jerome Petazzoni
7f23a4c964 Fix minor typos 2017-12-22 22:24:37 +01:00
Jerome Petazzoni
345e04c956 Improve tmux detection logic and add instructions 2017-12-21 22:24:27 +01:00
Jerome Petazzoni
2a138102fc Add client address in pub/sub server 2017-12-21 05:53:24 +01:00
Jerome Petazzoni
ef5e8f00f8 Add script to remember myself of how to customize tmux status bar 2017-12-21 05:46:57 +01:00
Jerome Petazzoni
badb73a413 Slower pace for virtual typist 2017-12-21 05:45:38 +01:00
Jerome Petazzoni
2aced95c86 Improve UX for remote control 2017-12-21 05:40:35 +01:00
Jerome Petazzoni
720989e829 Add remote control of slide deck 2017-12-21 05:29:59 +01:00
Jerome Petazzoni
718031565e Exit gracefully if server is not running instead of waiting forever 2017-12-21 05:29:45 +01:00
Jerome Petazzoni
ec7b46b779 Add remote.js to workshop template and pub/sub server 2017-12-21 04:51:49 +01:00
Jerome Petazzoni
270c36b29a Add pub/sub server and CLI remote 2017-12-21 04:43:42 +01:00
Jerome Petazzoni
bc2eb53bb2 Python 3 compatibility 2017-12-21 04:33:35 +01:00
Jerome Petazzoni
afe7b8523c Move autotest to autopilot/ directory 2017-12-21 04:32:16 +01:00
Jérôme Petazzoni
a7743a4314 Update Engine version 2017-12-20 18:05:52 -06:00
Jérôme Petazzoni
ba74fdc841 Round of update for video content 2017-12-20 00:17:49 -06:00
Jérôme Petazzoni
41c047e12a Always start in interactive mode 2017-12-20 00:17:40 -06:00
Jérôme Petazzoni
f4fc055405 Add manifest for video content 2017-12-20 00:17:25 -06:00
Jérôme Petazzoni
2eb6fcfbf5 Add command to backtrack 1 slide 2017-12-18 18:46:24 -06:00
Jérôme Petazzoni
c665e1a2d6 httping only 3 requests is enough 2017-12-18 18:45:40 -06:00
Jérôme Petazzoni
bb7cdafe47 Comment out machine chapter 2017-12-18 18:39:33 -06:00
Jérôme Petazzoni
95fcfadb17 state.yml -> state.yaml to avoid collision with manifests 2017-12-18 18:39:17 -06:00
Jérôme Petazzoni
1ef47531c8 autotest: save all parameters in state.yml 2017-12-18 18:36:46 -06:00
Jérôme Petazzoni
9589b641b6 Another fix in CNC script 2017-12-18 18:35:57 -06:00
Jérôme Petazzoni
63463bda64 Merge branch 'master' of github.com:jpetazzo/container.training 2017-12-18 17:46:11 -06:00
Jérôme Petazzoni
b642412639 Clarify listen-addr and advertise-addr
Closes #108
2017-12-18 17:46:06 -06:00
Jérôme Petazzoni
21f9b73cb4 Update CNC deployment script + workshopctl deps 2017-12-18 17:45:29 -06:00
Jérôme Petazzoni
b73e5432f3 Merge pull request #109 from juliogomez/patch-6
curl is not installed in this step
2017-12-18 17:20:52 -06:00
Jérôme Petazzoni
de5cc9b0bf Merge branch 'master' of github.com:jpetazzo/container.training 2017-12-18 15:25:48 -06:00
Jérôme Petazzoni
08b38127d3 Clarify license for slides since they're not code 2017-12-18 15:25:41 -06:00
Julio
383804b7f1 curl is not installed in that step
curl was actually installed in a previous step, not here
2017-12-16 23:05:29 +01:00
Jérôme Petazzoni
20bf80910e Merge pull request #107 from juliogomez/patch-3
Fixing number of replicas per node
2017-12-16 09:26:18 -06:00
Jérôme Petazzoni
29a2014745 Merge pull request #106 from juliogomez/patch-2
Typo correction in detach mode?
2017-12-16 09:24:36 -06:00
Julio
40f6ee236f Fixing number of replicas per node
If 3 copies per node are 15, 4 copies per node should be 20.
2017-12-15 20:46:25 +01:00
Julio
5551cbd11f Typo correction in detach mode?
While you wrote:
`--detach=false` does not complete *faster*. It just *doesn't wait* for completion.
I think you actually meant:
`--detach=TRUE` does not complete *faster*. It just *doesn't wait* for completion.
2017-12-15 20:25:29 +01:00
Jérôme Petazzoni
9e84a05325 Merge pull request #105 from juliogomez/patch-1
Fixed missing image name (tomcat) in 'docker run' command
2017-12-14 15:33:24 -06:00
Julio
558e990907 Fixed missing image name in command 2017-12-14 22:15:29 +01:00
Jérôme Petazzoni
c2e88bb343 add qcon video 2017-12-07 14:28:46 -06:00
Jérôme Petazzoni
b7582397fe Add quote by not-Benjamin-Franklin 2017-12-04 18:19:04 -06:00
Jérôme Petazzoni
3e7b8615ab Move kube workshop to archives section 2017-12-04 13:57:17 -06:00
Jérôme Petazzoni
6f5d8c5372 Merge pull request #104 from gurayyildirim/patch-1
Kubernetes.io link fixed.
2017-11-24 11:31:07 -06:00
gurayyildirim
c116d75408 Kubernetes.io link fixed.
Kubernetes.io link had a wrong ']' mark which was causing a 404 from Kubernetes.io blog.
2017-11-24 02:45:49 +03:00
Jérôme Petazzoni
bb4ee4e77d Add helper script to setup CNC node 2017-11-20 17:04:38 -08:00
Jérôme Petazzoni
fc0e46988c Fix hint for ssh agent 2017-11-20 16:37:17 -08:00
Jérôme Petazzoni
c71b93c3a7 Add files to generate a CSV file with nodes 2017-11-20 16:35:25 -08:00
Jérôme Petazzoni
2c6b79c17d Add kube image to cards.html 2017-11-20 16:33:08 -08:00
52 changed files with 1450 additions and 790 deletions

3
.gitignore vendored
View File

@@ -7,4 +7,5 @@ prepare-vms/ips.pdf
prepare-vms/settings.yaml
prepare-vms/tags
slides/*.yml.html
slides/nextstep
slides/autopilot/state.yaml
node_modules

19
LICENSE
View File

@@ -1,13 +1,12 @@
Copyright 2015 Jérôme Petazzoni
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
The code in this repository is licensed under the Apache License
Version 2.0. You may obtain a copy of this license at:
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
The instructions and slides in this repository (e.g. the files
with extension .md and .yml in the "slides" subdirectory) are
under the Creative Commons Attribution 4.0 International Public
License. You may obtain a copy of this license at:
https://creativecommons.org/licenses/by/4.0/legalcode

View File

@@ -43,7 +43,7 @@ because they have a few things in common:
(and updated) identically between different decks;
- a [build system](slides/) generating HTML slides from
Markdown source files;
- a [semi-automated test harness](slides/autotest.py) to check
- a [semi-automated test harness](slides/autopilot/) to check
that the exercises and examples provided work properly;
- a [PhantomJS script](slides/slidechecker.js) to check
that the slides look good and don't have formatting issues;

View File

@@ -12,7 +12,9 @@
{%- set cluster_or_machine = "cluster" -%}
{%- set this_or_each = "each" -%}
{%- set machine_is_or_machines_are = "machines are" -%}
{%- set image_src = "https://cdn.wp.nginx.com/wp-content/uploads/2016/07/docker-swarm-hero2.png" -%}
{%- set image_src_swarm = "https://cdn.wp.nginx.com/wp-content/uploads/2016/07/docker-swarm-hero2.png" -%}
{%- set image_src_kube = "https://avatars1.githubusercontent.com/u/13629408" -%}
{%- set image_src = image_src_swarm -%}
{%- endif -%}
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>

5
prepare-vms/clusters.csv Normal file
View File

@@ -0,0 +1,5 @@
Put your initials in the first column to "claim" a cluster.
Initials{% for node in clusters[0] %} node{{ loop.index }}{% endfor %}
{% for cluster in clusters -%}
{%- for node in cluster %} {{ node|trim }}{% endfor %}
{% endfor %}
Can't render this file because it contains an unexpected character in line 1 and column 42.

21
prepare-vms/cncsetup.sh Normal file
View File

@@ -0,0 +1,21 @@
#!/bin/sh
if [ $(whoami) != ubuntu ]; then
echo "This script should be executed on a freshly deployed node,"
echo "with the 'ubuntu' user. Aborting."
exit 1
fi
if id docker; then
sudo userdel -r docker
fi
pip install --user awscli jinja2 pdfkit
sudo apt-get install -y wkhtmltopdf xvfb
tmux new-session \; send-keys "
[ -f ~/.ssh/id_rsa ] || ssh-keygen
eval \$(ssh-agent)
ssh-add
Xvfb :0 &
export DISPLAY=:0
mkdir -p ~/www
sudo docker run -d -p 80:80 -v \$HOME/www:/usr/share/nginx/html nginx
"

View File

@@ -134,17 +134,11 @@ _cmd_kube() {
sudo apt-get install -qy kubelet kubeadm kubectl
kubectl completion bash | sudo tee /etc/bash_completion.d/kubectl"
# Work around https://github.com/kubernetes/kubernetes/issues/53356
pssh "
if [ ! -f /etc/kubernetes/kubelet.conf ]; then
sudo systemctl stop kubelet
sudo rm -rf /var/lib/kubelet/pki
fi"
# Initialize kube master
pssh "
if grep -q node1 /tmp/node && [ ! -f /etc/kubernetes/admin.conf ]; then
sudo kubeadm init
kubeadm token generate > /tmp/token
sudo kubeadm init --token \$(cat /tmp/token)
fi"
# Put kubeconfig in ubuntu's and docker's accounts
@@ -157,15 +151,6 @@ _cmd_kube() {
sudo chown -R docker /home/docker/.kube
fi"
# Get bootstrap token
pssh "
if grep -q node1 /tmp/node; then
TOKEN_NAME=\$(kubectl -n kube-system get secret -o name | grep bootstrap-token)
TOKEN_ID=\$(kubectl -n kube-system get \$TOKEN_NAME -o go-template --template '{{ index .data \"token-id\" }}' | base64 -d)
TOKEN_SECRET=\$(kubectl -n kube-system get \$TOKEN_NAME -o go-template --template '{{ index .data \"token-secret\" }}' | base64 -d)
echo \$TOKEN_ID.\$TOKEN_SECRET >/tmp/token
fi"
# Install weave as the pod network
pssh "
if grep -q node1 /tmp/node; then
@@ -177,7 +162,7 @@ _cmd_kube() {
pssh "
if ! grep -q node1 /tmp/node && [ ! -f /etc/kubernetes/kubelet.conf ]; then
TOKEN=\$(ssh -o StrictHostKeyChecking=no node1 cat /tmp/token)
sudo kubeadm join --token \$TOKEN node1:6443
sudo kubeadm join --discovery-token-unsafe-skip-ca-verification --token \$TOKEN node1:6443
fi"
sep "Done"
@@ -280,6 +265,9 @@ _cmd_start() {
key_name=$(sync_keys)
AMI=$(_cmd_ami) # Retrieve the AWS image ID
if [ -z "$AMI" ]; then
die "I could not find which AMI to use in this region. Try another region?"
fi
TOKEN=$(get_token) # generate a timestamp token for this batch of VMs
AWS_KEY_NAME=$(make_key_name)

View File

@@ -0,0 +1,5 @@
# Number of VMs per cluster
clustersize: 5
# Jinja2 template to use to generate ready-to-cut cards
cards_template: clusters.csv

View File

@@ -20,7 +20,7 @@ DEPENDENCIES="
ssh
curl
jq
pssh
parallel-ssh
wkhtmltopdf
man
"
@@ -38,7 +38,7 @@ check_envvars() {
if [ -z "${!envvar}" ]; then
error "Environment variable $envvar is not set."
if [ "$envvar" = "SSH_AUTH_SOCK" ]; then
error "Hint: run '\$(ssh-agent) ; ssh-add' and try again?"
error "Hint: run 'eval \$(ssh-agent) ; ssh-add' and try again?"
fi
status=1
fi

2
slides/_redirects Normal file
View File

@@ -0,0 +1,2 @@
/ /kube-halfday.yml.html 200!

413
slides/autopilot/autotest.py Executable file
View File

@@ -0,0 +1,413 @@
#!/usr/bin/env python
# coding: utf-8
import click
import logging
import os
import random
import re
import select
import subprocess
import sys
import time
import uuid
import yaml
logging.basicConfig(level=os.environ.get("LOG_LEVEL", "INFO"))
TIMEOUT = 60 # 1 minute
class State(object):
def __init__(self):
self.interactive = True
self.verify_status = False
self.simulate_type = True
self.slide = 1
self.snippet = 0
def load(self):
data = yaml.load(open("state.yaml"))
self.interactive = bool(data["interactive"])
self.verify_status = bool(data["verify_status"])
self.simulate_type = bool(data["simulate_type"])
self.slide = int(data["slide"])
self.snippet = int(data["snippet"])
def save(self):
with open("state.yaml", "w") as f:
yaml.dump(dict(
interactive=self.interactive,
verify_status=self.verify_status,
simulate_type=self.simulate_type,
slide=self.slide,
snippet=self.snippet,
), f, default_flow_style=False)
state = State()
def hrule():
return "="*int(subprocess.check_output(["tput", "cols"]))
# A "snippet" is something that the user is supposed to do in the workshop.
# Most of the "snippets" are shell commands.
# Some of them can be key strokes or other actions.
# In the markdown source, they are the code sections (identified by triple-
# quotes) within .exercise[] sections.
class Snippet(object):
def __init__(self, slide, content):
self.slide = slide
self.content = content
# Extract the "method" (e.g. bash, keys, ...)
# On multi-line snippets, the method is alone on the first line
# On single-line snippets, the data follows the method immediately
if '\n' in content:
self.method, self.data = content.split('\n', 1)
else:
self.method, self.data = content.split(' ', 1)
self.data = self.data.strip()
self.next = None
def __str__(self):
return self.content
class Slide(object):
current_slide = 0
def __init__(self, content):
self.number = Slide.current_slide
Slide.current_slide += 1
# Remove commented-out slides
# (remark.js considers ??? to be the separator for speaker notes)
content = re.split("\n\?\?\?\n", content)[0]
self.content = content
self.snippets = []
exercises = re.findall("\.exercise\[(.*)\]", content, re.DOTALL)
for exercise in exercises:
if "```" in exercise:
previous = None
for snippet_content in exercise.split("```")[1::2]:
snippet = Snippet(self, snippet_content)
if previous:
previous.next = snippet
previous = snippet
self.snippets.append(snippet)
else:
logging.warning("Exercise on slide {} does not have any ``` snippet."
.format(self.number))
self.debug()
def __str__(self):
text = self.content
for snippet in self.snippets:
text = text.replace(snippet.content, ansi("7")(snippet.content))
return text
def debug(self):
logging.debug("\n{}\n{}\n{}".format(hrule(), self.content, hrule()))
def focus_slides():
subprocess.check_output(["i3-msg", "workspace", "3"])
subprocess.check_output(["i3-msg", "workspace", "1"])
def focus_terminal():
subprocess.check_output(["i3-msg", "workspace", "2"])
subprocess.check_output(["i3-msg", "workspace", "1"])
def focus_browser():
subprocess.check_output(["i3-msg", "workspace", "4"])
subprocess.check_output(["i3-msg", "workspace", "1"])
def ansi(code):
return lambda s: "\x1b[{}m{}\x1b[0m".format(code, s)
# Sleeps the indicated delay, but interruptible by pressing ENTER.
# If interrupted, returns True.
def interruptible_sleep(t):
rfds, _, _ = select.select([0], [], [], t)
return 0 in rfds
def wait_for_string(s, timeout=TIMEOUT):
logging.debug("Waiting for string: {}".format(s))
deadline = time.time() + timeout
while time.time() < deadline:
output = capture_pane()
if s in output:
return
if interruptible_sleep(1): return
raise Exception("Timed out while waiting for {}!".format(s))
def wait_for_prompt():
logging.debug("Waiting for prompt.")
deadline = time.time() + TIMEOUT
while time.time() < deadline:
output = capture_pane()
# If we are not at the bottom of the screen, there will be a bunch of extra \n's
output = output.rstrip('\n')
last_line = output.split('\n')[-1]
# Our custom prompt on the VMs has two lines; the 2nd line is just '$'
if last_line == "$":
return
# When we are in an alpine container, the prompt will be "/ #"
if last_line == "/ #":
return
# We did not recognize a known prompt; wait a bit and check again
logging.debug("Could not find a known prompt on last line: {!r}"
.format(last_line))
if interruptible_sleep(1): return
raise Exception("Timed out while waiting for prompt!")
def check_exit_status():
if not state.verify_status:
return
token = uuid.uuid4().hex
data = "echo {} $?\n".format(token)
logging.debug("Sending {!r} to get exit status.".format(data))
send_keys(data)
time.sleep(0.5)
wait_for_prompt()
screen = capture_pane()
status = re.findall("\n{} ([0-9]+)\n".format(token), screen, re.MULTILINE)
logging.debug("Got exit status: {}.".format(status))
if len(status) == 0:
raise Exception("Couldn't retrieve status code {}. Timed out?".format(token))
if len(status) > 1:
raise Exception("More than one status code {}. I'm seeing double! Shoot them both.".format(token))
code = int(status[0])
if code != 0:
raise Exception("Non-zero exit status: {}.".format(code))
# Otherwise just return peacefully.
def setup_tmux_and_ssh():
if subprocess.call(["tmux", "has-session"]):
logging.error("Couldn't connect to tmux. Please setup tmux first.")
ipaddr = open("../../prepare-vms/ips.txt").read().split("\n")[0]
uid = os.getuid()
raise Exception("""
1. If you're running this directly from a node:
tmux
2. If you want to control a remote tmux:
rm -f /tmp/tmux-{uid}/default && ssh -t -L /tmp/tmux-{uid}/default:/tmp/tmux-1001/default docker@{ipaddr} tmux new-session -As 0
3. If you cannot control a remote tmux:
tmux new-session ssh docker@{ipaddr}
""".format(uid=uid, ipaddr=ipaddr))
else:
logging.info("Found tmux session. Trying to acquire shell prompt.")
wait_for_prompt()
logging.info("Successfully connected to test cluster in tmux session.")
slides = [Slide("Dummy slide zero")]
content = open(sys.argv[1]).read()
# OK, this part is definitely hackish, and will break if the
# excludedClasses parameter is not on a single line.
excluded_classes = re.findall("excludedClasses: (\[.*\])", content)
excluded_classes = set(eval(excluded_classes[0]))
for slide in re.split("\n---?\n", content):
slide_classes = re.findall("class: (.*)", slide)
if slide_classes:
slide_classes = slide_classes[0].split(",")
slide_classes = [c.strip() for c in slide_classes]
if excluded_classes & set(slide_classes):
logging.info("Skipping excluded slide.")
continue
slides.append(Slide(slide))
def send_keys(data):
if state.simulate_type and data[0] != '^':
for key in data:
if key == ";":
key = "\\;"
if key == "\n":
if interruptible_sleep(1): return
subprocess.check_call(["tmux", "send-keys", key])
if interruptible_sleep(0.15*random.random()): return
if key == "\n":
if interruptible_sleep(1): return
else:
subprocess.check_call(["tmux", "send-keys", data])
def capture_pane():
return subprocess.check_output(["tmux", "capture-pane", "-p"]).decode('utf-8')
setup_tmux_and_ssh()
try:
state.load()
logging.info("Successfully loaded state from file.")
# Let's override the starting state, so that when an error occurs,
# we can restart the auto-tester and then single-step or debug.
# (Instead of running again through the same issue immediately.)
state.interactive = True
except Exception as e:
logging.exception("Could not load state from file.")
logging.warning("Using default values.")
def move_forward():
state.snippet += 1
if state.snippet > len(slides[state.slide].snippets):
state.slide += 1
state.snippet = 0
check_bounds()
def move_backward():
state.snippet -= 1
if state.snippet < 0:
state.slide -= 1
state.snippet = 0
check_bounds()
def check_bounds():
if state.slide < 1:
state.slide = 1
if state.slide >= len(slides):
state.slide = len(slides)-1
while True:
state.save()
slide = slides[state.slide]
snippet = slide.snippets[state.snippet-1] if state.snippet else None
click.clear()
print("[Slide {}/{}] [Snippet {}/{}] [simulate_type:{}] [verify_status:{}]"
.format(state.slide, len(slides)-1,
state.snippet, len(slide.snippets) if slide.snippets else 0,
state.simulate_type, state.verify_status))
print(hrule())
if snippet:
print(slide.content.replace(snippet.content, ansi(7)(snippet.content)))
focus_terminal()
else:
print(slide.content)
subprocess.check_output(["./gotoslide.js", str(slide.number)])
focus_slides()
print(hrule())
if state.interactive:
print("y/⎵/⏎ Execute snippet or advance to next snippet")
print("p/← Previous")
print("n/→ Next")
print("s Simulate keystrokes")
print("v Validate exit status")
print("g Go to a specific slide")
print("q Quit")
print("c Continue non-interactively until next error")
command = click.getchar()
else:
command = "y"
if command in ("n", "\x1b[C"):
move_forward()
elif command in ("p", "\x1b[D"):
move_backward()
elif command == "s":
state.simulate_type = not state.simulate_type
elif command == "v":
state.verify_status = not state.verify_status
elif command == "g":
state.slide = click.prompt("Enter slide number", type=int)
state.snippet = 0
check_bounds()
elif command == "q":
break
elif command == "c":
# continue until next timeout
state.interactive = False
elif command in ("y", "\r", " "):
if not snippet:
# Advance to next snippet
# Advance until a slide that has snippets
while not slides[state.slide].snippets:
move_forward()
# But stop if we reach the last slide
if state.slide == len(slides)-1:
break
# And then advance to the snippet
move_forward()
continue
method, data = snippet.method, snippet.data
logging.info("Running with method {}: {}".format(method, data))
if method == "keys":
send_keys(data)
elif method == "bash":
# Make sure that we're ready
wait_for_prompt()
# Strip leading spaces
data = re.sub("\n +", "\n", data)
# Remove backticks (they are used to highlight sections)
data = data.replace('`', '')
# Add "RETURN" at the end of the command :)
data += "\n"
# Send command
send_keys(data)
# Force a short sleep to avoid race condition
time.sleep(0.5)
if snippet.next and snippet.next.method == "wait":
wait_for_string(snippet.next.data)
elif snippet.next and snippet.next.method == "longwait":
wait_for_string(snippet.next.data, 10*TIMEOUT)
else:
wait_for_prompt()
# Verify return code
check_exit_status()
elif method == "copypaste":
screen = capture_pane()
matches = re.findall(data, screen, flags=re.DOTALL)
if len(matches) == 0:
raise Exception("Could not find regex {} in output.".format(data))
# Arbitrarily get the most recent match
match = matches[-1]
# Remove line breaks (like a screen copy paste would do)
match = match.replace('\n', '')
send_keys(match + '\n')
# FIXME: we should factor out the "bash" method
wait_for_prompt()
check_exit_status()
elif method == "open":
# Cheap way to get node1's IP address
screen = capture_pane()
ipaddr = re.findall("^\[(.*)\]", screen, re.MULTILINE)[-1]
url = data.replace("/node1", "/{}".format(ipaddr))
# This should probably be adapted to run on different OS
subprocess.check_output(["xdg-open", url])
focus_browser()
if state.interactive:
print("Press any key to continue to next step...")
click.getchar()
else:
logging.warning("Unknown method {}: {!r}".format(method, data))
move_forward()
else:
logging.warning("Unknown command {}.".format(command))

17
slides/autopilot/gotoslide.js Executable file
View File

@@ -0,0 +1,17 @@
#!/usr/bin/env node
/* Expects a slide number as first argument.
* Will connect to the local pub/sub server,
* and issue a "go to slide X" command, which
* will be sent to all connected browsers.
*/
var io = require('socket.io-client');
var socket = io('http://localhost:3000');
socket.on('connect_error', function(){
console.log('connection error');
socket.close();
});
socket.emit('slide change', process.argv[2], function(){
socket.close();
});

603
slides/autopilot/package-lock.json generated Normal file
View File

@@ -0,0 +1,603 @@
{
"name": "container-training-pub-sub-server",
"version": "0.0.1",
"lockfileVersion": 1,
"requires": true,
"dependencies": {
"accepts": {
"version": "1.3.4",
"resolved": "https://registry.npmjs.org/accepts/-/accepts-1.3.4.tgz",
"integrity": "sha1-hiRnWMfdbSGmR0/whKR0DsBesh8=",
"requires": {
"mime-types": "2.1.17",
"negotiator": "0.6.1"
}
},
"after": {
"version": "0.8.2",
"resolved": "https://registry.npmjs.org/after/-/after-0.8.2.tgz",
"integrity": "sha1-/ts5T58OAqqXaOcCvaI7UF+ufh8="
},
"array-flatten": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/array-flatten/-/array-flatten-1.1.1.tgz",
"integrity": "sha1-ml9pkFGx5wczKPKgCJaLZOopVdI="
},
"arraybuffer.slice": {
"version": "0.0.6",
"resolved": "https://registry.npmjs.org/arraybuffer.slice/-/arraybuffer.slice-0.0.6.tgz",
"integrity": "sha1-8zshWfBTKj8xB6JywMz70a0peco="
},
"async-limiter": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/async-limiter/-/async-limiter-1.0.0.tgz",
"integrity": "sha512-jp/uFnooOiO+L211eZOoSyzpOITMXx1rBITauYykG3BRYPu8h0UcxsPNB04RR5vo4Tyz3+ay17tR6JVf9qzYWg=="
},
"backo2": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/backo2/-/backo2-1.0.2.tgz",
"integrity": "sha1-MasayLEpNjRj41s+u2n038+6eUc="
},
"base64-arraybuffer": {
"version": "0.1.5",
"resolved": "https://registry.npmjs.org/base64-arraybuffer/-/base64-arraybuffer-0.1.5.tgz",
"integrity": "sha1-c5JncZI7Whl0etZmqlzUv5xunOg="
},
"base64id": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/base64id/-/base64id-1.0.0.tgz",
"integrity": "sha1-R2iMuZu2gE8OBtPnY7HDLlfY5rY="
},
"better-assert": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/better-assert/-/better-assert-1.0.2.tgz",
"integrity": "sha1-QIZrnhueC1W0gYlDEeaPr/rrxSI=",
"requires": {
"callsite": "1.0.0"
}
},
"blob": {
"version": "0.0.4",
"resolved": "https://registry.npmjs.org/blob/-/blob-0.0.4.tgz",
"integrity": "sha1-vPEwUspURj8w+fx+lbmkdjCpSSE="
},
"body-parser": {
"version": "1.18.2",
"resolved": "https://registry.npmjs.org/body-parser/-/body-parser-1.18.2.tgz",
"integrity": "sha1-h2eKGdhLR9hZuDGZvVm84iKxBFQ=",
"requires": {
"bytes": "3.0.0",
"content-type": "1.0.4",
"debug": "2.6.9",
"depd": "1.1.1",
"http-errors": "1.6.2",
"iconv-lite": "0.4.19",
"on-finished": "2.3.0",
"qs": "6.5.1",
"raw-body": "2.3.2",
"type-is": "1.6.15"
}
},
"bytes": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/bytes/-/bytes-3.0.0.tgz",
"integrity": "sha1-0ygVQE1olpn4Wk6k+odV3ROpYEg="
},
"callsite": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/callsite/-/callsite-1.0.0.tgz",
"integrity": "sha1-KAOY5dZkvXQDi28JBRU+borxvCA="
},
"component-bind": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/component-bind/-/component-bind-1.0.0.tgz",
"integrity": "sha1-AMYIq33Nk4l8AAllGx06jh5zu9E="
},
"component-emitter": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/component-emitter/-/component-emitter-1.2.1.tgz",
"integrity": "sha1-E3kY1teCg/ffemt8WmPhQOaUJeY="
},
"component-inherit": {
"version": "0.0.3",
"resolved": "https://registry.npmjs.org/component-inherit/-/component-inherit-0.0.3.tgz",
"integrity": "sha1-ZF/ErfWLcrZJ1crmUTVhnbJv8UM="
},
"content-disposition": {
"version": "0.5.2",
"resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-0.5.2.tgz",
"integrity": "sha1-DPaLud318r55YcOoUXjLhdunjLQ="
},
"content-type": {
"version": "1.0.4",
"resolved": "https://registry.npmjs.org/content-type/-/content-type-1.0.4.tgz",
"integrity": "sha512-hIP3EEPs8tB9AT1L+NUqtwOAps4mk2Zob89MWXMHjHWg9milF/j4osnnQLXBCBFBk/tvIG/tUc9mOUJiPBhPXA=="
},
"cookie": {
"version": "0.3.1",
"resolved": "https://registry.npmjs.org/cookie/-/cookie-0.3.1.tgz",
"integrity": "sha1-5+Ch+e9DtMi6klxcWpboBtFoc7s="
},
"cookie-signature": {
"version": "1.0.6",
"resolved": "https://registry.npmjs.org/cookie-signature/-/cookie-signature-1.0.6.tgz",
"integrity": "sha1-4wOogrNCzD7oylE6eZmXNNqzriw="
},
"debug": {
"version": "2.6.9",
"resolved": "https://registry.npmjs.org/debug/-/debug-2.6.9.tgz",
"integrity": "sha512-bC7ElrdJaJnPbAP+1EotYvqZsb3ecl5wi6Bfi6BJTUcNowp6cvspg0jXznRTKDjm/E7AdgFBVeAPVMNcKGsHMA==",
"requires": {
"ms": "2.0.0"
}
},
"depd": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/depd/-/depd-1.1.1.tgz",
"integrity": "sha1-V4O04cRZ8G+lyif5kfPQbnoxA1k="
},
"destroy": {
"version": "1.0.4",
"resolved": "https://registry.npmjs.org/destroy/-/destroy-1.0.4.tgz",
"integrity": "sha1-l4hXRCxEdJ5CBmE+N5RiBYJqvYA="
},
"ee-first": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/ee-first/-/ee-first-1.1.1.tgz",
"integrity": "sha1-WQxhFWsK4vTwJVcyoViyZrxWsh0="
},
"encodeurl": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/encodeurl/-/encodeurl-1.0.1.tgz",
"integrity": "sha1-eePVhlU0aQn+bw9Fpd5oEDspTSA="
},
"engine.io": {
"version": "3.1.4",
"resolved": "https://registry.npmjs.org/engine.io/-/engine.io-3.1.4.tgz",
"integrity": "sha1-PQIRtwpVLOhB/8fahiezAamkFi4=",
"requires": {
"accepts": "1.3.3",
"base64id": "1.0.0",
"cookie": "0.3.1",
"debug": "2.6.9",
"engine.io-parser": "2.1.1",
"uws": "0.14.5",
"ws": "3.3.3"
},
"dependencies": {
"accepts": {
"version": "1.3.3",
"resolved": "https://registry.npmjs.org/accepts/-/accepts-1.3.3.tgz",
"integrity": "sha1-w8p0NJOGSMPg2cHjKN1otiLChMo=",
"requires": {
"mime-types": "2.1.17",
"negotiator": "0.6.1"
}
}
}
},
"engine.io-client": {
"version": "3.1.4",
"resolved": "https://registry.npmjs.org/engine.io-client/-/engine.io-client-3.1.4.tgz",
"integrity": "sha1-T88TcLRxY70s6b4nM5ckMDUNTqE=",
"requires": {
"component-emitter": "1.2.1",
"component-inherit": "0.0.3",
"debug": "2.6.9",
"engine.io-parser": "2.1.1",
"has-cors": "1.1.0",
"indexof": "0.0.1",
"parseqs": "0.0.5",
"parseuri": "0.0.5",
"ws": "3.3.3",
"xmlhttprequest-ssl": "1.5.4",
"yeast": "0.1.2"
}
},
"engine.io-parser": {
"version": "2.1.1",
"resolved": "https://registry.npmjs.org/engine.io-parser/-/engine.io-parser-2.1.1.tgz",
"integrity": "sha1-4Ps/DgRi9/WLt3waUun1p+JuRmg=",
"requires": {
"after": "0.8.2",
"arraybuffer.slice": "0.0.6",
"base64-arraybuffer": "0.1.5",
"blob": "0.0.4",
"has-binary2": "1.0.2"
}
},
"escape-html": {
"version": "1.0.3",
"resolved": "https://registry.npmjs.org/escape-html/-/escape-html-1.0.3.tgz",
"integrity": "sha1-Aljq5NPQwJdN4cFpGI7wBR0dGYg="
},
"etag": {
"version": "1.8.1",
"resolved": "https://registry.npmjs.org/etag/-/etag-1.8.1.tgz",
"integrity": "sha1-Qa4u62XvpiJorr/qg6x9eSmbCIc="
},
"express": {
"version": "4.16.2",
"resolved": "https://registry.npmjs.org/express/-/express-4.16.2.tgz",
"integrity": "sha1-41xt/i1kt9ygpc1PIXgb4ymeB2w=",
"requires": {
"accepts": "1.3.4",
"array-flatten": "1.1.1",
"body-parser": "1.18.2",
"content-disposition": "0.5.2",
"content-type": "1.0.4",
"cookie": "0.3.1",
"cookie-signature": "1.0.6",
"debug": "2.6.9",
"depd": "1.1.1",
"encodeurl": "1.0.1",
"escape-html": "1.0.3",
"etag": "1.8.1",
"finalhandler": "1.1.0",
"fresh": "0.5.2",
"merge-descriptors": "1.0.1",
"methods": "1.1.2",
"on-finished": "2.3.0",
"parseurl": "1.3.2",
"path-to-regexp": "0.1.7",
"proxy-addr": "2.0.2",
"qs": "6.5.1",
"range-parser": "1.2.0",
"safe-buffer": "5.1.1",
"send": "0.16.1",
"serve-static": "1.13.1",
"setprototypeof": "1.1.0",
"statuses": "1.3.1",
"type-is": "1.6.15",
"utils-merge": "1.0.1",
"vary": "1.1.2"
}
},
"finalhandler": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/finalhandler/-/finalhandler-1.1.0.tgz",
"integrity": "sha1-zgtoVbRYU+eRsvzGgARtiCU91/U=",
"requires": {
"debug": "2.6.9",
"encodeurl": "1.0.1",
"escape-html": "1.0.3",
"on-finished": "2.3.0",
"parseurl": "1.3.2",
"statuses": "1.3.1",
"unpipe": "1.0.0"
}
},
"forwarded": {
"version": "0.1.2",
"resolved": "https://registry.npmjs.org/forwarded/-/forwarded-0.1.2.tgz",
"integrity": "sha1-mMI9qxF1ZXuMBXPozszZGw/xjIQ="
},
"fresh": {
"version": "0.5.2",
"resolved": "https://registry.npmjs.org/fresh/-/fresh-0.5.2.tgz",
"integrity": "sha1-PYyt2Q2XZWn6g1qx+OSyOhBWBac="
},
"has-binary2": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/has-binary2/-/has-binary2-1.0.2.tgz",
"integrity": "sha1-6D26SfC5vk0CbSc2U1DZ8D9Uvpg=",
"requires": {
"isarray": "2.0.1"
}
},
"has-cors": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/has-cors/-/has-cors-1.1.0.tgz",
"integrity": "sha1-XkdHk/fqmEPRu5nCPu9J/xJv/zk="
},
"http-errors": {
"version": "1.6.2",
"resolved": "https://registry.npmjs.org/http-errors/-/http-errors-1.6.2.tgz",
"integrity": "sha1-CgAsyFcHGSp+eUbO7cERVfYOxzY=",
"requires": {
"depd": "1.1.1",
"inherits": "2.0.3",
"setprototypeof": "1.0.3",
"statuses": "1.3.1"
},
"dependencies": {
"setprototypeof": {
"version": "1.0.3",
"resolved": "https://registry.npmjs.org/setprototypeof/-/setprototypeof-1.0.3.tgz",
"integrity": "sha1-ZlZ+NwQ+608E2RvWWMDL77VbjgQ="
}
}
},
"iconv-lite": {
"version": "0.4.19",
"resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.4.19.tgz",
"integrity": "sha512-oTZqweIP51xaGPI4uPa56/Pri/480R+mo7SeU+YETByQNhDG55ycFyNLIgta9vXhILrxXDmF7ZGhqZIcuN0gJQ=="
},
"indexof": {
"version": "0.0.1",
"resolved": "https://registry.npmjs.org/indexof/-/indexof-0.0.1.tgz",
"integrity": "sha1-gtwzbSMrkGIXnQWrMpOmYFn9Q10="
},
"inherits": {
"version": "2.0.3",
"resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.3.tgz",
"integrity": "sha1-Yzwsg+PaQqUC9SRmAiSA9CCCYd4="
},
"ipaddr.js": {
"version": "1.5.2",
"resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-1.5.2.tgz",
"integrity": "sha1-1LUFvemUaYfM8PxY2QEP+WB+P6A="
},
"isarray": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/isarray/-/isarray-2.0.1.tgz",
"integrity": "sha1-o32U7ZzaLVmGXJ92/llu4fM4dB4="
},
"media-typer": {
"version": "0.3.0",
"resolved": "https://registry.npmjs.org/media-typer/-/media-typer-0.3.0.tgz",
"integrity": "sha1-hxDXrwqmJvj/+hzgAWhUUmMlV0g="
},
"merge-descriptors": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/merge-descriptors/-/merge-descriptors-1.0.1.tgz",
"integrity": "sha1-sAqqVW3YtEVoFQ7J0blT8/kMu2E="
},
"methods": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/methods/-/methods-1.1.2.tgz",
"integrity": "sha1-VSmk1nZUE07cxSZmVoNbD4Ua/O4="
},
"mime": {
"version": "1.4.1",
"resolved": "https://registry.npmjs.org/mime/-/mime-1.4.1.tgz",
"integrity": "sha512-KI1+qOZu5DcW6wayYHSzR/tXKCDC5Om4s1z2QJjDULzLcmf3DvzS7oluY4HCTrc+9FiKmWUgeNLg7W3uIQvxtQ=="
},
"mime-db": {
"version": "1.30.0",
"resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.30.0.tgz",
"integrity": "sha1-dMZD2i3Z1qRTmZY0ZbJtXKfXHwE="
},
"mime-types": {
"version": "2.1.17",
"resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.17.tgz",
"integrity": "sha1-Cdejk/A+mVp5+K+Fe3Cp4KsWVXo=",
"requires": {
"mime-db": "1.30.0"
}
},
"ms": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/ms/-/ms-2.0.0.tgz",
"integrity": "sha1-VgiurfwAvmwpAd9fmGF4jeDVl8g="
},
"negotiator": {
"version": "0.6.1",
"resolved": "https://registry.npmjs.org/negotiator/-/negotiator-0.6.1.tgz",
"integrity": "sha1-KzJxhOiZIQEXeyhWP7XnECrNDKk="
},
"object-component": {
"version": "0.0.3",
"resolved": "https://registry.npmjs.org/object-component/-/object-component-0.0.3.tgz",
"integrity": "sha1-8MaapQ78lbhmwYb0AKM3acsvEpE="
},
"on-finished": {
"version": "2.3.0",
"resolved": "https://registry.npmjs.org/on-finished/-/on-finished-2.3.0.tgz",
"integrity": "sha1-IPEzZIGwg811M3mSoWlxqi2QaUc=",
"requires": {
"ee-first": "1.1.1"
}
},
"parseqs": {
"version": "0.0.5",
"resolved": "https://registry.npmjs.org/parseqs/-/parseqs-0.0.5.tgz",
"integrity": "sha1-1SCKNzjkZ2bikbouoXNoSSGouJ0=",
"requires": {
"better-assert": "1.0.2"
}
},
"parseuri": {
"version": "0.0.5",
"resolved": "https://registry.npmjs.org/parseuri/-/parseuri-0.0.5.tgz",
"integrity": "sha1-gCBKUNTbt3m/3G6+J3jZDkvOMgo=",
"requires": {
"better-assert": "1.0.2"
}
},
"parseurl": {
"version": "1.3.2",
"resolved": "https://registry.npmjs.org/parseurl/-/parseurl-1.3.2.tgz",
"integrity": "sha1-/CidTtiZMRlGDBViUyYs3I3mW/M="
},
"path-to-regexp": {
"version": "0.1.7",
"resolved": "https://registry.npmjs.org/path-to-regexp/-/path-to-regexp-0.1.7.tgz",
"integrity": "sha1-32BBeABfUi8V60SQ5yR6G/qmf4w="
},
"proxy-addr": {
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/proxy-addr/-/proxy-addr-2.0.2.tgz",
"integrity": "sha1-ZXFQT0e7mI7IGAJT+F3X4UlSvew=",
"requires": {
"forwarded": "0.1.2",
"ipaddr.js": "1.5.2"
}
},
"qs": {
"version": "6.5.1",
"resolved": "https://registry.npmjs.org/qs/-/qs-6.5.1.tgz",
"integrity": "sha512-eRzhrN1WSINYCDCbrz796z37LOe3m5tmW7RQf6oBntukAG1nmovJvhnwHHRMAfeoItc1m2Hk02WER2aQ/iqs+A=="
},
"range-parser": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/range-parser/-/range-parser-1.2.0.tgz",
"integrity": "sha1-9JvmtIeJTdxA3MlKMi9hEJLgDV4="
},
"raw-body": {
"version": "2.3.2",
"resolved": "https://registry.npmjs.org/raw-body/-/raw-body-2.3.2.tgz",
"integrity": "sha1-vNYMd9Prk83gBQKVw/N5OJvIj4k=",
"requires": {
"bytes": "3.0.0",
"http-errors": "1.6.2",
"iconv-lite": "0.4.19",
"unpipe": "1.0.0"
}
},
"safe-buffer": {
"version": "5.1.1",
"resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.1.1.tgz",
"integrity": "sha512-kKvNJn6Mm93gAczWVJg7wH+wGYWNrDHdWvpUmHyEsgCtIwwo3bqPtV4tR5tuPaUhTOo/kvhVwd8XwwOllGYkbg=="
},
"send": {
"version": "0.16.1",
"resolved": "https://registry.npmjs.org/send/-/send-0.16.1.tgz",
"integrity": "sha512-ElCLJdJIKPk6ux/Hocwhk7NFHpI3pVm/IZOYWqUmoxcgeyM+MpxHHKhb8QmlJDX1pU6WrgaHBkVNm73Sv7uc2A==",
"requires": {
"debug": "2.6.9",
"depd": "1.1.1",
"destroy": "1.0.4",
"encodeurl": "1.0.1",
"escape-html": "1.0.3",
"etag": "1.8.1",
"fresh": "0.5.2",
"http-errors": "1.6.2",
"mime": "1.4.1",
"ms": "2.0.0",
"on-finished": "2.3.0",
"range-parser": "1.2.0",
"statuses": "1.3.1"
}
},
"serve-static": {
"version": "1.13.1",
"resolved": "https://registry.npmjs.org/serve-static/-/serve-static-1.13.1.tgz",
"integrity": "sha512-hSMUZrsPa/I09VYFJwa627JJkNs0NrfL1Uzuup+GqHfToR2KcsXFymXSV90hoyw3M+msjFuQly+YzIH/q0MGlQ==",
"requires": {
"encodeurl": "1.0.1",
"escape-html": "1.0.3",
"parseurl": "1.3.2",
"send": "0.16.1"
}
},
"setprototypeof": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/setprototypeof/-/setprototypeof-1.1.0.tgz",
"integrity": "sha512-BvE/TwpZX4FXExxOxZyRGQQv651MSwmWKZGqvmPcRIjDqWub67kTKuIMx43cZZrS/cBBzwBcNDWoFxt2XEFIpQ=="
},
"socket.io": {
"version": "2.0.4",
"resolved": "https://registry.npmjs.org/socket.io/-/socket.io-2.0.4.tgz",
"integrity": "sha1-waRZDO/4fs8TxyZS8Eb3FrKeYBQ=",
"requires": {
"debug": "2.6.9",
"engine.io": "3.1.4",
"socket.io-adapter": "1.1.1",
"socket.io-client": "2.0.4",
"socket.io-parser": "3.1.2"
}
},
"socket.io-adapter": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/socket.io-adapter/-/socket.io-adapter-1.1.1.tgz",
"integrity": "sha1-KoBeihTWNyEk3ZFZrUUC+MsH8Gs="
},
"socket.io-client": {
"version": "2.0.4",
"resolved": "https://registry.npmjs.org/socket.io-client/-/socket.io-client-2.0.4.tgz",
"integrity": "sha1-CRilUkBtxeVAs4Dc2Xr8SmQzL44=",
"requires": {
"backo2": "1.0.2",
"base64-arraybuffer": "0.1.5",
"component-bind": "1.0.0",
"component-emitter": "1.2.1",
"debug": "2.6.9",
"engine.io-client": "3.1.4",
"has-cors": "1.1.0",
"indexof": "0.0.1",
"object-component": "0.0.3",
"parseqs": "0.0.5",
"parseuri": "0.0.5",
"socket.io-parser": "3.1.2",
"to-array": "0.1.4"
}
},
"socket.io-parser": {
"version": "3.1.2",
"resolved": "https://registry.npmjs.org/socket.io-parser/-/socket.io-parser-3.1.2.tgz",
"integrity": "sha1-28IoIVH8T6675Aru3Ady66YZ9/I=",
"requires": {
"component-emitter": "1.2.1",
"debug": "2.6.9",
"has-binary2": "1.0.2",
"isarray": "2.0.1"
}
},
"statuses": {
"version": "1.3.1",
"resolved": "https://registry.npmjs.org/statuses/-/statuses-1.3.1.tgz",
"integrity": "sha1-+vUbnrdKrvOzrPStX2Gr8ky3uT4="
},
"to-array": {
"version": "0.1.4",
"resolved": "https://registry.npmjs.org/to-array/-/to-array-0.1.4.tgz",
"integrity": "sha1-F+bBH3PdTz10zaek/zI46a2b+JA="
},
"type-is": {
"version": "1.6.15",
"resolved": "https://registry.npmjs.org/type-is/-/type-is-1.6.15.tgz",
"integrity": "sha1-yrEPtJCeRByChC6v4a1kbIGARBA=",
"requires": {
"media-typer": "0.3.0",
"mime-types": "2.1.17"
}
},
"ultron": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/ultron/-/ultron-1.1.1.tgz",
"integrity": "sha512-UIEXBNeYmKptWH6z8ZnqTeS8fV74zG0/eRU9VGkpzz+LIJNs8W/zM/L+7ctCkRrgbNnnR0xxw4bKOr0cW0N0Og=="
},
"unpipe": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/unpipe/-/unpipe-1.0.0.tgz",
"integrity": "sha1-sr9O6FFKrmFltIF4KdIbLvSZBOw="
},
"utils-merge": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/utils-merge/-/utils-merge-1.0.1.tgz",
"integrity": "sha1-n5VxD1CiZ5R7LMwSR0HBAoQn5xM="
},
"uws": {
"version": "0.14.5",
"resolved": "https://registry.npmjs.org/uws/-/uws-0.14.5.tgz",
"integrity": "sha1-Z6rzPEaypYel9mZtAPdpEyjxSdw=",
"optional": true
},
"vary": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/vary/-/vary-1.1.2.tgz",
"integrity": "sha1-IpnwLG3tMNSllhsLn3RSShj2NPw="
},
"ws": {
"version": "3.3.3",
"resolved": "https://registry.npmjs.org/ws/-/ws-3.3.3.tgz",
"integrity": "sha512-nnWLa/NwZSt4KQJu51MYlCcSQ5g7INpOrOMt4XV8j4dqTXdmlUmSHQ8/oLC069ckre0fRsgfvsKwbTdtKLCDkA==",
"requires": {
"async-limiter": "1.0.0",
"safe-buffer": "5.1.1",
"ultron": "1.1.1"
}
},
"xmlhttprequest-ssl": {
"version": "1.5.4",
"resolved": "https://registry.npmjs.org/xmlhttprequest-ssl/-/xmlhttprequest-ssl-1.5.4.tgz",
"integrity": "sha1-BPVgkVcks4kIhxXMDteBPpZ3v1c="
},
"yeast": {
"version": "0.1.2",
"resolved": "https://registry.npmjs.org/yeast/-/yeast-0.1.2.tgz",
"integrity": "sha1-AI4G2AlDIMNy28L47XagymyKxBk="
}
}
}

View File

@@ -0,0 +1,8 @@
{
"name": "container-training-pub-sub-server",
"version": "0.0.1",
"dependencies": {
"express": "^4.16.2",
"socket.io": "^2.0.4"
}
}

View File

@@ -0,0 +1,21 @@
/* This snippet is loaded from the workshop HTML file.
* It sets up callbacks to synchronize the local slide
* number with the remote pub/sub server.
*/
var socket = io();
var leader = true;
slideshow.on('showSlide', function (slide) {
if (leader) {
var n = slide.getSlideIndex()+1;
socket.emit('slide change', n);
}
});
socket.on('slide change', function (n) {
leader = false;
slideshow.gotoSlide(n);
leader = true;
});

41
slides/autopilot/server.js Executable file
View File

@@ -0,0 +1,41 @@
#!/usr/bin/env node
/* This is a very simple pub/sub server, allowing to
* remote control browsers displaying the slides.
* The browsers connect to this pub/sub server using
* Socket.IO, and the server tells them which slides
* to display.
*
* The server can be controlled with a little CLI,
* or by one of the browsers.
*/
var express = require('express');
var app = express();
var http = require('http').Server(app);
var io = require('socket.io')(http);
app.get('/', function(req, res){
res.send('container.training autopilot pub/sub server');
});
/* Serve remote.js from the current directory */
app.use(express.static('.'));
/* Serve slides etc. from current and the parent directory */
app.use(express.static('..'));
io.on('connection', function(socket){
console.log('a client connected: ' + socket.handshake.address);
socket.on('slide change', function(n, ack){
console.log('slide change: ' + n);
socket.broadcast.emit('slide change', n);
if (typeof ack === 'function') {
ack();
}
});
});
http.listen(3000, function(){
console.log('listening on *:3000');
});

7
slides/autopilot/tmux-style.sh Executable file
View File

@@ -0,0 +1,7 @@
#!/bin/sh
# This removes the clock (and other extraneous stuff) from the
# tmux status bar, and it gives it a non-default color.
tmux set-option -g status-left ""
tmux set-option -g status-right ""
tmux set-option -g status-style bg=cyan

View File

@@ -1,279 +0,0 @@
#!/usr/bin/env python
# coding: utf-8
import click
import logging
import os
import random
import re
import subprocess
import sys
import time
import uuid
logging.basicConfig(level=os.environ.get("LOG_LEVEL", "INFO"))
interactive = True
verify_status = False
simulate_type = True
TIMEOUT = 60 # 1 minute
def hrule():
return "="*int(subprocess.check_output(["tput", "cols"]))
# A "snippet" is something that the user is supposed to do in the workshop.
# Most of the "snippets" are shell commands.
# Some of them can be key strokes or other actions.
# In the markdown source, they are the code sections (identified by triple-
# quotes) within .exercise[] sections.
class Snippet(object):
def __init__(self, slide, content):
self.slide = slide
self.content = content
self.actions = []
def __str__(self):
return self.content
class Slide(object):
current_slide = 0
def __init__(self, content):
Slide.current_slide += 1
self.number = Slide.current_slide
# Remove commented-out slides
# (remark.js considers ??? to be the separator for speaker notes)
content = re.split("\n\?\?\?\n", content)[0]
self.content = content
self.snippets = []
exercises = re.findall("\.exercise\[(.*)\]", content, re.DOTALL)
for exercise in exercises:
if "```" in exercise:
for snippet in exercise.split("```")[1::2]:
self.snippets.append(Snippet(self, snippet))
else:
logging.warning("Exercise on slide {} does not have any ``` snippet."
.format(self.number))
self.debug()
def __str__(self):
text = self.content
for snippet in self.snippets:
text = text.replace(snippet.content, ansi("7")(snippet.content))
return text
def debug(self):
logging.debug("\n{}\n{}\n{}".format(hrule(), self.content, hrule()))
def ansi(code):
return lambda s: "\x1b[{}m{}\x1b[0m".format(code, s)
def wait_for_string(s, timeout=TIMEOUT):
logging.debug("Waiting for string: {}".format(s))
deadline = time.time() + timeout
while time.time() < deadline:
output = capture_pane()
if s in output:
return
time.sleep(1)
raise Exception("Timed out while waiting for {}!".format(s))
def wait_for_prompt():
logging.debug("Waiting for prompt.")
deadline = time.time() + TIMEOUT
while time.time() < deadline:
output = capture_pane()
# If we are not at the bottom of the screen, there will be a bunch of extra \n's
output = output.rstrip('\n')
if output.endswith("\n$"):
return
if output.endswith("\n/ #"):
return
time.sleep(1)
raise Exception("Timed out while waiting for prompt!")
def check_exit_status():
if not verify_status:
return
token = uuid.uuid4().hex
data = "echo {} $?\n".format(token)
logging.debug("Sending {!r} to get exit status.".format(data))
send_keys(data)
time.sleep(0.5)
wait_for_prompt()
screen = capture_pane()
status = re.findall("\n{} ([0-9]+)\n".format(token), screen, re.MULTILINE)
logging.debug("Got exit status: {}.".format(status))
if len(status) == 0:
raise Exception("Couldn't retrieve status code {}. Timed out?".format(token))
if len(status) > 1:
raise Exception("More than one status code {}. I'm seeing double! Shoot them both.".format(token))
code = int(status[0])
if code != 0:
raise Exception("Non-zero exit status: {}.".format(code))
# Otherwise just return peacefully.
def setup_tmux_and_ssh():
if subprocess.call(["tmux", "has-session"]):
logging.info("Couldn't connect to tmux. A new tmux session will be created.")
subprocess.check_call(["tmux", "new-session", "-d"])
wait_for_string("$")
send_keys("cd ../prepare-vms\n")
send_keys("ssh docker@$(head -n1 ips.txt)\n")
wait_for_string("password:")
send_keys("training\n")
wait_for_prompt()
else:
logging.info("Found tmux session. Trying to acquire shell prompt.")
wait_for_prompt()
logging.info("Successfully connected to test cluster in tmux session.")
slides = []
content = open(sys.argv[1]).read()
for slide in re.split("\n---?\n", content):
slides.append(Slide(slide))
actions = []
for slide in slides:
for snippet in slide.snippets:
content = snippet.content
# Extract the "method" (e.g. bash, keys, ...)
# On multi-line snippets, the method is alone on the first line
# On single-line snippets, the data follows the method immediately
if '\n' in content:
method, data = content.split('\n', 1)
else:
method, data = content.split(' ', 1)
actions.append((slide, snippet, method, data))
def send_keys(data):
if simulate_type and data[0] != '^':
for key in data:
if key == ";":
key = "\\;"
subprocess.check_call(["tmux", "send-keys", key])
time.sleep(0.1*random.random())
else:
subprocess.check_call(["tmux", "send-keys", data])
def capture_pane():
return subprocess.check_output(["tmux", "capture-pane", "-p"])
setup_tmux_and_ssh()
try:
i = int(open("nextstep").read())
logging.info("Loaded next step ({}) from file.".format(i))
except Exception as e:
logging.warning("Could not read nextstep file ({}), initializing to 0.".format(e))
i = 0
while i < len(actions):
with open("nextstep", "w") as f:
f.write(str(i))
slide, snippet, method, data = actions[i]
# Remove extra spaces (we don't want them in the terminal) and carriage returns
data = data.strip()
print(hrule())
print(slide.content.replace(snippet.content, ansi(7)(snippet.content)))
print(hrule())
if interactive:
print("[{}/{}] Shall we execute that snippet above?".format(i, len(actions)))
print("y/⏎/→ Execute snippet")
print("s Skip snippet")
print("g Go to a specific snippet")
print("q Quit")
print("c Continue non-interactively until next error")
command = click.getchar()
else:
command = "y"
# For now, remove the `highlighted` sections
# (Make sure to use $() in shell snippets!)
if '`' in data:
logging.info("Stripping ` from snippet.")
data = data.replace('`', '')
if command == "s":
i += 1
elif command == "g":
i = click.prompt("Enter snippet number", type=int)
elif command == "q":
break
elif command == "c":
# continue until next timeout
interactive = False
elif command in ("y", "\r", " ", "\x1b[C"):
logging.info("Running with method {}: {}".format(method, data))
if method == "keys":
send_keys(data)
elif method == "bash":
# Make sure that we're ready
wait_for_prompt()
# Strip leading spaces
data = re.sub("\n +", "\n", data)
# Add "RETURN" at the end of the command :)
data += "\n"
# Send command
send_keys(data)
# Force a short sleep to avoid race condition
time.sleep(0.5)
_, _, next_method, next_data = actions[i+1]
if next_method == "wait":
wait_for_string(next_data)
elif next_method == "longwait":
wait_for_string(next_data, 10*TIMEOUT)
else:
wait_for_prompt()
# Verify return code FIXME should be optional
check_exit_status()
elif method == "copypaste":
screen = capture_pane()
matches = re.findall(data, screen, flags=re.DOTALL)
if len(matches) == 0:
raise Exception("Could not find regex {} in output.".format(data))
# Arbitrarily get the most recent match
match = matches[-1]
# Remove line breaks (like a screen copy paste would do)
match = match.replace('\n', '')
send_keys(match + '\n')
# FIXME: we should factor out the "bash" method
wait_for_prompt()
check_exit_status()
elif method == "open":
# Cheap way to get node1's IP address
screen = capture_pane()
ipaddr = re.findall("^\[(.*)\]", screen, re.MULTILINE)[-1]
url = data.replace("/node1", "/{}".format(ipaddr))
# This should probably be adapted to run on different OS
subprocess.check_call(["open", url])
else:
logging.warning("Unknown method {}: {!r}".format(method, data))
i += 1
else:
logging.warning("Unknown command {}.".format(command))
# Reset slide counter
with open("nextstep", "w") as f:
f.write(str(0))

View File

@@ -9,12 +9,10 @@
- We recommend having a mentor to help you ...
- ... Or be comfortable spending some time reading the Docker
[documentation](https://docs.docker.com/) ...
- ... Or be comfortable spending some time reading the Kubernetes
[documentation](https://kubernetes.io/docs/) ...
- ... And looking for answers in the [Docker forums](forums.docker.com),
[StackOverflow](http://stackoverflow.com/questions/tagged/docker),
and other outlets
- ... And looking for answers on [StackOverflow](http://stackoverflow.com/questions/tagged/kubernetes) and other outlets
---
@@ -35,9 +33,7 @@ class: self-paced
<br/>you will be given specific instructions to access your cluster
- If you are doing this on your own:
<br/>the first chapter will give you various options like
[Play-With-Docker](http://www.play-with-docker.com/)
to get your own cluster
<br/>the first chapter will give you various options to get your own cluster
---
@@ -51,8 +47,21 @@ class: self-paced
http://container.training/
- Typos? Mistakes? Questions? Feel free to hover over the bottom of the slide ...
<!--
.exercise[
```open https://github.com/jpetazzo/container.training```
```open http://container.training/```
]
-->
--
- Typos? Mistakes? Questions? Feel free to hover over the bottom of the slide ...
.footnote[.emoji[👇] Try it! The source file will be shown and you can view it on GitHub and fork and edit it.]
<!--
.exercise[
```open https://github.com/jpetazzo/container.training/tree/master/slides/common/intro.md```
]
-->

View File

@@ -26,7 +26,7 @@ class: extra-details
- This slide should have a little magnifying glass in the top left corner
(If it doesn't, it's because CSS is hard — Jérôme is only a backend person, alas)
(If it doesn't, it's because CSS is hard — we're only backend people, alas!)
- Slides with that magnifying glass indicate slides providing extra details
@@ -34,6 +34,20 @@ class: extra-details
---
class: title
*Tell me and I forget.*
<br/>
*Teach me and I remember.*
<br/>
*Involve me and I learn.*
Misattributed to Benjamin Franklin
[(Probably inspired by Chinese Confucian philosopher Xunzi)](https://www.barrypopik.com/index.php/new_york_city/entry/tell_me_and_i_forget_teach_me_and_i_may_remember_involve_me_and_i_will_lear/)
---
## Hands-on sections
- The whole workshop is hands-on
@@ -48,10 +62,12 @@ class: extra-details
- This is the stuff you're supposed to do!
- Go to [container.training](http://container.training/) to view these slides
- Go to [indexconf2018.container.training](http://indexconf2018.container.training/) to view these slides
- Join the chat room on @@CHAT@@
<!-- ```open http://container.training/``` -->
]
---
@@ -62,17 +78,11 @@ class: in-person
---
class: in-person, pic
![You get five VMs](images/you-get-five-vms.jpg)
---
class: in-person
## You get five VMs
## You get three VMs
- Each person gets 5 private VMs (not shared with anybody else)
- Each person gets 3 private VMs (not shared with anybody else)
- They'll remain up for the duration of the workshop
@@ -80,7 +90,7 @@ class: in-person
- You can automatically SSH from one VM to another
- The nodes have aliases: `node1`, `node2`, etc.
- The nodes have aliases: `node1`, `node2`, `node3`.
---
@@ -137,7 +147,7 @@ class: in-person
<!--
```bash
for N in $(seq 1 5); do
for N in $(seq 1 3); do
ssh -o StrictHostKeyChecking=no node$N true
done
```
@@ -183,6 +193,32 @@ If anything goes wrong — ask for help!
---
class: self-paced
## Get your own Docker nodes
- If you already have some Docker nodes: great!
- If not: let's get some thanks to Play-With-Docker
.exercise[
- Go to http://www.play-with-docker.com/
- Log in
- Create your first node
<!-- ```open http://www.play-with-docker.com/``` -->
]
You will need a Docker ID to use Play-With-Docker.
(Creating a Docker ID is free.)
---
## We will (mostly) interact with node1 only
*These remarks apply only when using multiple nodes, of course.*

View File

@@ -21,65 +21,6 @@
---
class: extra-details
## Compose file format version
*Particularly relevant if you have used Compose before...*
- Compose 1.6 introduced support for a new Compose file format (aka "v2")
- Services are no longer at the top level, but under a `services` section
- There has to be a `version` key at the top level, with value `"2"` (as a string, not an integer)
- Containers are placed on a dedicated network, making links unnecessary
- There are other minor differences, but upgrade is easy and straightforward
---
## Links, naming, and service discovery
- Containers can have network aliases (resolvable through DNS)
- Compose file version 2+ makes each container reachable through its service name
- Compose file version 1 did require "links" sections
- Our code can connect to services using their short name
(instead of e.g. IP address or FQDN)
- Network aliases are automatically namespaced
(i.e. you can have multiple apps declaring and using a service named `database`)
---
## Example in `worker/worker.py`
```python
redis = Redis("`redis`")
def get_random_bytes():
r = requests.get("http://`rng`/32")
return r.content
def hash_bytes(data):
r = requests.post("http://`hasher`/",
data=data,
headers={"Content-Type": "application/octet-stream"})
```
(Full source code available [here](
https://github.com/jpetazzo/container.training/blob/8279a3bce9398f7c1a53bdd95187c53eda4e6435/dockercoins/worker/worker.py#L17
))
---
## What's this application?
--
@@ -124,7 +65,7 @@ fi
- Clone the repository on `node1`:
```bash
git clone git://github.com/jpetazzo/container.training
git clone https://github.com/jpetazzo/container.training/
```
]
@@ -183,77 +124,6 @@ and displays aggregated logs.
---
## Restarting in the background
- Many flags and commands of Compose are modeled after those of `docker`
.exercise[
- Start the app in the background with the `-d` option:
```bash
docker-compose up -d
```
- Check that our app is running with the `ps` command:
```bash
docker-compose ps
```
]
`docker-compose ps` also shows the ports exposed by the application.
---
class: extra-details
## Viewing logs
- The `docker-compose logs` command works like `docker logs`
.exercise[
- View all logs since container creation and exit when done:
```bash
docker-compose logs
```
- Stream container logs, starting at the last 10 lines for each container:
```bash
docker-compose logs --tail 10 --follow
```
<!--
```wait units of work done```
```keys ^C```
-->
]
Tip: use `^S` and `^Q` to pause/resume log output.
---
class: extra-details
## Upgrading from Compose 1.6
.warning[The `logs` command has changed between Compose 1.6 and 1.7!]
- Up to 1.6
- `docker-compose logs` is the equivalent of `logs --follow`
- `docker-compose logs` must be restarted if containers are added
- Since 1.7
- `--follow` must be specified explicitly
- new containers are automatically picked up by `docker-compose logs`
---
## Connecting to the web UI
- The `webui` container exposes a web dashboard; let's view it
@@ -275,245 +145,6 @@ graph will appear.
---
class: self-paced, extra-details
## If the graph doesn't load
If you just see a `Page not found` error, it might be because your
Docker Engine is running on a different machine. This can be the case if:
- you are using the Docker Toolbox
- you are using a VM (local or remote) created with Docker Machine
- you are controlling a remote Docker Engine
When you run DockerCoins in development mode, the web UI static files
are mapped to the container using a volume. Alas, volumes can only
work on a local environment, or when using Docker4Mac or Docker4Windows.
How to fix this?
Edit `dockercoins.yml` and comment out the `volumes` section, and try again.
---
class: extra-details
## Why does the speed seem irregular?
- It *looks like* the speed is approximately 4 hashes/second
- Or more precisely: 4 hashes/second, with regular dips down to zero
- Why?
--
class: extra-details
- The app actually has a constant, steady speed: 3.33 hashes/second
<br/>
(which corresponds to 1 hash every 0.3 seconds, for *reasons*)
- Yes, and?
---
class: extra-details
## The reason why this graph is *not awesome*
- The worker doesn't update the counter after every loop, but up to once per second
- The speed is computed by the browser, checking the counter about once per second
- Between two consecutive updates, the counter will increase either by 4, or by 0
- The perceived speed will therefore be 4 - 4 - 4 - 0 - 4 - 4 - 0 etc.
- What can we conclude from this?
--
class: extra-details
- Jérôme is clearly incapable of writing good frontend code
---
## Scaling up the application
- Our goal is to make that performance graph go up (without changing a line of code!)
--
- Before trying to scale the application, we'll figure out if we need more resources
(CPU, RAM...)
- For that, we will use good old UNIX tools on our Docker node
---
## Looking at resource usage
- Let's look at CPU, memory, and I/O usage
.exercise[
- run `top` to see CPU and memory usage (you should see idle cycles)
<!--
```bash top```
```wait Tasks```
```keys ^C```
-->
- run `vmstat 1` to see I/O usage (si/so/bi/bo)
<br/>(the 4 numbers should be almost zero, except `bo` for logging)
<!--
```bash vmstat 1```
```wait memory```
```keys ^C```
-->
]
We have available resources.
- Why?
- How can we use them?
---
## Scaling workers on a single node
- Docker Compose supports scaling
- Let's scale `worker` and see what happens!
.exercise[
- Start one more `worker` container:
```bash
docker-compose scale worker=2
```
- Look at the performance graph (it should show a x2 improvement)
- Look at the aggregated logs of our containers (`worker_2` should show up)
- Look at the impact on CPU load with e.g. top (it should be negligible)
]
---
## Adding more workers
- Great, let's add more workers and call it a day, then!
.exercise[
- Start eight more `worker` containers:
```bash
docker-compose scale worker=10
```
- Look at the performance graph: does it show a x10 improvement?
- Look at the aggregated logs of our containers
- Look at the impact on CPU load and memory usage
]
---
# Identifying bottlenecks
- You should have seen a 3x speed bump (not 10x)
- Adding workers didn't result in linear improvement
- *Something else* is slowing us down
--
- ... But what?
--
- The code doesn't have instrumentation
- Let's use state-of-the-art HTTP performance analysis!
<br/>(i.e. good old tools like `ab`, `httping`...)
---
## Accessing internal services
- `rng` and `hasher` are exposed on ports 8001 and 8002
- This is declared in the Compose file:
```yaml
...
rng:
build: rng
ports:
- "8001:80"
hasher:
build: hasher
ports:
- "8002:80"
...
```
---
## Measuring latency under load
We will use `httping`.
.exercise[
- Check the latency of `rng`:
```bash
httping -c 10 localhost:8001
```
- Check the latency of `hasher`:
```bash
httping -c 10 localhost:8002
```
]
`rng` has a much higher latency than `hasher`.
---
## Let's draw hasty conclusions
- The bottleneck seems to be `rng`
- *What if* we don't have enough entropy and can't generate enough random numbers?
- We need to scale out the `rng` service on multiple machines!
Note: this is a fiction! We have enough entropy. But we need a pretext to scale out.
(In fact, the code of `rng` uses `/dev/urandom`, which never runs out of entropy...
<br/>
...and is [just as good as `/dev/random`](http://www.slideshare.net/PacSecJP/filippo-plain-simple-reality-of-entropy).)
---
## Clean up
- Before moving on, let's remove those containers

View File

@@ -6,7 +6,7 @@ Thank you!
class: title, in-person
That's all folks! <br/> Questions?
That's all, folks! <br/> Questions?
![end](images/end.jpg)
@@ -14,13 +14,9 @@ That's all folks! <br/> Questions?
# Links and resources
- [Docker Community Slack](https://community.docker.com/registrations/groups/4316)
- [Docker Community Forums](https://forums.docker.com/)
- [Docker Hub](https://hub.docker.com)
- [Docker Blog](http://blog.docker.com/)
- [Docker documentation](http://docs.docker.com/)
- [Docker on StackOverflow](https://stackoverflow.com/questions/tagged/docker)
- [Docker on Twitter](http://twitter.com/docker)
- [Play With Docker Hands-On Labs](http://training.play-with-docker.com/)
- [Kubernetes Community](https://kubernetes.io/community/) - Slack, Google Groups, meetups
- [Play With Kubernetes Hands-On Labs](https://medium.com/@marcosnils/introducing-pwk-play-with-k8s-159fcfeb787b)
- [Local meetups](https://www.meetup.com/)
- [Microsoft Cloud Developer Advocates](https://developer.microsoft.com/en-us/advocates/)
.footnote[These slides (and future updates) are on → http://container.training/]

View File

@@ -17,5 +17,5 @@ class: title, in-person
*Don't stream videos or download big files during the workshop.*<br/>
*Thank you!*
**Slides: http://container.training/**
]
**Slides: http://indexconf2018.container.training/**
]

Binary file not shown.

After

Width:  |  Height:  |  Size: 144 KiB

View File

@@ -66,24 +66,22 @@
<tr><td class="title" colspan="4">Coming soon at a conference near you</td></tr>
<tr>
<!--
<td>Nothing for now (stay tuned...)</td>
-->
<td>February 22, 2018: IndexConf — Kubernetes 101</td>
<td><a class="slides" href="http://indexconf2018.container.training/" /></td>
<td><a class="attend" href="https://developer.ibm.com/indexconf/sessions/#!?id=5474" />
</tr>
<tr><td class="title" colspan="4">Past workshops</td></tr>
<tr>
<td>Kubernetes enablement at Docker</td>
<td><a class="slides" href="http://kube.container.training/" /></td>
<td><a class="chat" href="https://docker.slack.com/messages/C83M572J2" /></td>
</tr>
<!--
<td><a class="attend" href="https://qconsf.com/sf2017/workshop/orchestrating-microservices-docker-swarm" /></td>
-->
<!--
<tr>
<td>Nothing for now (stay tuned...)</td>
</tr>
-->
<tr><td class="title" colspan="4">Past workshops</td></tr>
<tr>
<td>QCON SF: Orchestrating Microservices with Docker Swarm</td>
<td><a class="slides" href="http://qconsf2017swarm.container.training/" /></td>
@@ -92,6 +90,7 @@
<tr>
<td>QCON SF: Introduction to Docker and Containers</td>
<td><a class="slides" href="http://qconsf2017intro.container.training/" /></td>
<td><a class="video" href="https://www.youtube.com/playlist?list=PLBAFXs0YjviLgqTum8MkspG_8VzGl6C07" /></td>
</tr>
<tr>
@@ -159,4 +158,4 @@
</table>
</div>
</body>
</html>
</html>

View File

@@ -100,7 +100,7 @@ class: extra-details
Let's start a Tomcat container:
```bash
$ docker run --name webapp -d -p 8080:8080 -v /usr/local/tomcat/logs
$ docker run --name webapp -d -p 8080:8080 -v /usr/local/tomcat/logs tomcat
```
Now, start an `alpine` container accessing the same volume:

View File

@@ -1,9 +1,7 @@
title: |
Deploying and Scaling Microservices
with Docker and Kubernetes
Kubernetes 101
chat: "[Slack](https://dockercommunity.slack.com/messages/C7GKACWDV)"
#chat: "[Gitter](https://gitter.im/jpetazzo/workshop-yyyymmdd-city)"
chat: "[Gitter](https://gitter.im/jpetazzo/workshop-20180222-sf)"
exclude:
- self-paced

View File

@@ -184,7 +184,7 @@ Yes!
*Probably not (in the future)*
.footnote[More information about CRI [on the Kubernetes blog](http://blog.kubernetes.io/2016/12/]container-runtime-interface-cri-in-kubernetes.html).
.footnote[More information about CRI [on the Kubernetes blog](http://blog.kubernetes.io/2016/12/container-runtime-interface-cri-in-kubernetes.html)]
---
@@ -211,3 +211,11 @@ class: pic
![Node, pod, container](images/k8s-arch3-thanks-weave.png)
(Diagram courtesy of Weave Works, used with permission.)
---
class: pic
![One of the best Kubernetes architecture diagrams available](images/k8s-arch4-thanks-luxas.png)
(Diagram courtesy of Lucas Käldström, in [this presentation](https://speakerdeck.com/luxas/kubeadm-cluster-creation-internals-from-self-hosting-to-upgradability-and-ha).)

View File

@@ -1,10 +1,8 @@
# Daemon sets
- Remember: we did all that cluster orchestration business for `rng`
- What if we want one (and exactly one) instance of `rng` per node?
- We want one (and exactly one) instance of `rng` per node
- If we just scale `deploy/rng` to 4, nothing guarantees that they spread
- If we just scale `deploy/rng` to 2, nothing guarantees that they spread
- Instead of a `deployment`, we will use a `daemonset`
@@ -22,7 +20,7 @@
## Creating a daemon set
- Unfortunately, as of Kubernetes 1.8, the CLI cannot create daemon sets
- Unfortunately, as of Kubernetes 1.9, the CLI cannot create daemon sets
--
@@ -382,7 +380,7 @@ Of course, option 2 offers more learning opportunities. Right?
.exercise[
- Check the logs of all `run=rng` pods to confirm that only 4 of them are now active:
- Check the logs of all `run=rng` pods to confirm that only 2 of them are now active:
```bash
kubectl logs -l run=rng
```
@@ -406,4 +404,4 @@ The timestamps should give us a hint about how many pods are currently receiving
- Bonus exercise 1: clean up the pods of the "old" daemon set
- Bonus exercise 2: how could we have done to avoid creating new pods?
- Bonus exercise 2: how could we have done this to avoid creating new pods?

View File

@@ -89,7 +89,13 @@ The goo.gl URL expands to:
- Connect to https://oneofournodes:3xxxx/
(You will have to work around the TLS certificate validation warning)
- Yes, https. If you use http it will say:
This page isnt working
<oneofournodes> sent an invalid response.
ERR_INVALID_HTTP_RESPONSE
- You will have to work around the TLS certificate validation warning
<!-- ```open https://node1:3xxxx/``` -->
@@ -109,7 +115,7 @@ The goo.gl URL expands to:
## Granting more rights to the dashboard
- The dashboard documentation [explains how to do](https://github.com/kubernetes/dashboard/wiki/Access-control#admin-privileges)
- The dashboard documentation [explains how to do this](https://github.com/kubernetes/dashboard/wiki/Access-control#admin-privileges)
- We just need to load another YAML file!

View File

@@ -48,7 +48,7 @@
.exercise[
- Give us more info about them nodes:
- Give us more info about the nodes:
```bash
kubectl get nodes -o wide
```
@@ -136,7 +136,7 @@ There is already one service on our cluster: the Kubernetes API itself.
```
- `-k` is used to skip certificate verification
- Make sure to replace 10.96.0.1 with the CLUSTER-IP shown earlier
- Make sure to replace 10.96.0.1 with the CLUSTER-IP shown by `$ kubectl get svc`
]
@@ -173,7 +173,7 @@ The error that we see is expected: the Kubernetes API requires authentication.
## Namespaces
- Namespaces allow to segregate resources
- Namespaces allow us to segregate resources
.exercise[

View File

@@ -245,4 +245,4 @@ at the Google NOC ...
<br/>
.small[are we getting 1000 packets per second]
<br/>
.small[of ICMP ECHO traffic from EC2 ?!?”]
.small[of ICMP ECHO traffic from Azure ?!?”]

View File

@@ -40,7 +40,7 @@ In this part, we will:
- We could use the Docker Hub
- Or a service offered by our cloud provider (GCR, ECR...)
- Or a service offered by our cloud provider (ACR, GCR, ECR...)
- Or we could just self-host that registry

View File

@@ -149,7 +149,7 @@ Our rollout is stuck. However, the app is not dead (just 10% slower).
- We want to:
- revert to `v0.1`
- revert to `v0.1` (which we now realize we didn't tag - yikes!)
- be conservative on availability (always have desired number of available workers)
- be aggressive on rollout speed (update more than one pod at a time)
- give some time to our workers to "warm up" before starting more
@@ -163,7 +163,7 @@ spec:
spec:
containers:
- name: worker
image: $REGISTRY/worker:v0.1
image: $REGISTRY/worker:latest
strategy:
rollingUpdate:
maxUnavailable: 0
@@ -192,7 +192,7 @@ spec:
spec:
containers:
- name: worker
image: $REGISTRY/worker:v0.1
image: $REGISTRY/worker:latest
strategy:
rollingUpdate:
maxUnavailable: 0

View File

@@ -4,7 +4,7 @@
--
- We used `kubeadm` on "fresh" EC2 instances with Ubuntu 16.04 LTS
- We used `kubeadm` on Azure instances with Ubuntu 16.04 LTS
1. Install Docker
@@ -36,26 +36,25 @@
--
- It's still twice as many steps as setting up a Swarm cluster 😕
- "It's still twice as many steps as setting up a Swarm cluster 😕 " -- Jérôme
---
## Other deployment options
- If you are on Google Cloud:
[GKE](https://cloud.google.com/container-engine/)
- If you are on Azure:
[AKS](https://azure.microsoft.com/services/container-service/)
Empirically the best Kubernetes deployment out there
- If you are on Google Cloud:
[GKE](https://cloud.google.com/kubernetes-engine/)
- If you are on AWS:
[kops](https://github.com/kubernetes/kops)
... But with AWS re:invent just around the corner, expect some changes
[EKS](https://aws.amazon.com/eks/)
- On a local machine:
[minikube](https://kubernetes.io/docs/getting-started-guides/minikube/),
[kubespawn](https://github.com/kinvolk/kube-spawn),
[Docker4Mac (coming soon)](https://beta.docker.com/)
[Docker4Mac](https://docs.docker.com/docker-for-mac/kubernetes/)
- If you want something customizable:
[kubicorn](https://github.com/kris-nova/kubicorn)

View File

@@ -1,8 +1,8 @@
## Brand new versions!
## Versions Installed
- Kubernetes 1.8
- Docker Engine 17.11
- Docker Compose 1.17
- Kubernetes 1.9.3
- Docker Engine 18.02.0-ce
- Docker Compose 1.18.0
.exercise[

View File

@@ -131,6 +131,7 @@ And *then* it is time to look at orchestration!
- shell scripts invoking `kubectl`
- YAML resources descriptions committed to a repo
- [Brigade](https://brigade.sh/) (event-driven scripting; no YAML)
- [Helm](https://github.com/kubernetes/helm) (~package manager)
- [Spinnaker](https://www.spinnaker.io/) (Netflix' CD platform)
@@ -160,7 +161,7 @@ Sorry Star Trek fans, this is not the federation you're looking for!
- Raft recommends low latency between nodes
- What if our cluster spreads multiple regions?
- What if our cluster spreads to multiple regions?
--

View File

@@ -2,18 +2,19 @@
- Hello! We are:
- .emoji[👷🏻‍♀️] AJ ([@s0ulshake](https://twitter.com/s0ulshake), Travis CI)
- .emoji[✨] Bridget ([@bridgetkromhout](https://twitter.com/bridgetkromhout))
- .emoji[🐳] Jérôme ([@jpetazzo](https://twitter.com/jpetazzo), Docker Inc.)
- .emoji[🌟] Jessica ([@jldeen](https://twitter.com/jldeen))
- The workshop will run from 9am to 4pm
- .emoji[🐳] Jérôme ([@jpetazzo](https://twitter.com/jpetazzo))
- There will be a lunch break at noon
- This workshop will run from 10:30am-12:45pm.
(And coffee breaks!)
- Lunchtime is after the workshop!
(And we will take a 15min break at 11:30am!)
- Feel free to interrupt for questions at any time
- *Especially when you see full screen container pictures!*
- Live feedback, questions, help on @@CHAT@@

View File

@@ -26,7 +26,7 @@ chapters:
- common/declarative.md
- swarm/swarmmode.md
- swarm/creatingswarm.md
- swarm/machine.md
#- swarm/machine.md
- swarm/morenodes.md
- - swarm/firstservice.md
- swarm/ourapponswarm.md

62
slides/swarm-video.yml Normal file
View File

@@ -0,0 +1,62 @@
title: |
Container Orchestration
with Docker and Swarm
chat: "[Slack](https://dockercommunity.slack.com/messages/C7GKACWDV)"
exclude:
- in-person
- btp-auto
chapters:
- common/title.md
#- common/logistics.md
- common/intro.md
- common/toc.md
- - common/prereqs.md
- swarm/versions.md
- |
name: part-1
class: title, self-paced
Part 1
- common/sampleapp.md
- swarm/swarmkit.md
- common/declarative.md
- swarm/swarmmode.md
- swarm/creatingswarm.md
#- swarm/machine.md
- swarm/morenodes.md
- - swarm/firstservice.md
- swarm/ourapponswarm.md
- swarm/hostingregistry.md
- swarm/testingregistry.md
- swarm/btp-manual.md
- swarm/swarmready.md
- swarm/compose2swarm.md
- |
name: part-2
class: title, self-paced
Part 2
- - swarm/operatingswarm.md
#- swarm/netshoot.md
#- swarm/swarmnbt.md
- swarm/ipsec.md
- swarm/updatingservices.md
- swarm/rollingupdates.md
#- swarm/healthchecks.md
- swarm/nodeinfo.md
- swarm/swarmtools.md
- - swarm/security.md
- swarm/secrets.md
- swarm/encryptionatrest.md
- swarm/leastprivilege.md
- swarm/apiscope.md
#- swarm/logging.md
#- swarm/metrics.md
- swarm/stateful.md
- swarm/extratips.md
- common/thankyou.md

View File

@@ -33,17 +33,43 @@ class: advertise-addr
<br/>
(i.e. it tells them *"you can contact me on 10.1.2.3:2377"*)
- If the node has only one IP address (other than 127.0.0.1), it is used automatically
- If the node has only one IP address, it is used automatically
<br/>
(The addresses of the loopback interface and the Docker bridge are ignored)
- If the node has multiple IP addresses, you **must** specify which one to use
<br/>
(Docker refuses to pick one randomly)
- You can specify an IP address or an interface name
<br/>(in the latter case, Docker will read the IP address of the interface and use it)
<br/>
(in the latter case, Docker will read the IP address of the interface and use it)
- You can also specify a port number
<br/>(otherwise, the default port 2377 will be used)
<br/>
(otherwise, the default port 2377 will be used)
---
class: advertise-addr
## Using a non-default port number
- Changing the *advertised* port does not change the *listening* port
- If you only pass `--advertise-addr eth0:7777`, Swarm will still listen on port 2377
- You will probably need to pass `--listen-addr eth0:7777` as well
- This is to accommodate scenarios where these ports *must* be different
<br/>
(port mapping, load balancers...)
Example to run Swarm on a different port:
```bash
docker swarm init --advertise-addr eth0:7777 --listen-addr eth0:7777
```
---
@@ -70,8 +96,8 @@ class: advertise-addr
Examples:
```bash
docker swarm init --advertise-addr 10.0.9.2
docker swarm init --advertise-addr eth0:7777
docker swarm init --advertise-addr 172.24.0.2
docker swarm init --advertise-addr eth0
```
---
@@ -82,7 +108,7 @@ class: extra-details
- You can use different interfaces (or IP addresses) for control and data
- You set the _control plane path_ with `--advertise-addr`
- You set the _control plane path_ with `--advertise-addr` and `--listen-addr`
(This will be used for SwarmKit manager/worker communication, leader election, etc.)

View File

@@ -68,6 +68,8 @@ that you [provisioned yourself](https://github.com/jpetazzo/container.training/t
docker node ls
```
<!-- ```wait Swarm is encrypted``` -->
]
(The last command should fail, and it will tell you how to unlock this node.)

View File

@@ -89,7 +89,7 @@ class: extra-details
- If you use Play-With-Docker, switch to that node's tab, or set `DOCKER_HOST`
- Otherwise, `ssh` into tht node or use `$(eval docker-machine env node...)`
- Otherwise, `ssh` into that node or use `$(eval docker-machine env node...)`
]
@@ -174,7 +174,7 @@ class: extra-details
- great in headless scripts (where nobody's watching anyway)
.warning[`--detach=false` does not complete *faster*. It just *doesn't wait* for completion.]
.warning[`--detach=true` does not complete *faster*. It just *doesn't wait* for completion.]
---
@@ -203,7 +203,7 @@ class: extra-details
- And then to 4 copies per node:
```bash
docker service update pingpong --replicas 15 --detach=true
docker service update pingpong --replicas 20 --detach=true
```
]
@@ -281,6 +281,8 @@ class: extra-details
.exercise[
<!-- Give it a few seconds to be ready ```bash sleep 5``` -->
- Try the following command:
```bash
curl localhost:9200

View File

@@ -71,6 +71,9 @@
docker run --net host nicolaka/netshoot ngrep -tpd eth0 HTTP
```
<!-- ```wait User-Agent``` -->
<!-- ```keys ^C``` -->
]
--

View File

@@ -22,7 +22,7 @@ With Play-With-Docker:
```bash
TOKEN=$(docker swarm join-token -q manager)
for N in $(seq 4 5); do
for N in $(seq 3 5); do
export DOCKER_HOST=tcp://node$N:2375
docker swarm join --token $TOKEN node1:2377
done

View File

@@ -257,7 +257,7 @@ First, we need to put the POST payload in a temporary file.
.exercise[
- Install curl in the container, and generate 10 bytes of random data:
- Generate 10 bytes of random data:
```bash
curl http://rng/10 >/tmp/random
```
@@ -284,6 +284,8 @@ Once again, we will send 50 requests, with different levels of concurrency.
ab -c 50 -n 50 -T application/octet-stream -p /tmp/random http://hasher/
```
<!-- ```bash exit``` -->
]
---
@@ -382,6 +384,6 @@ class: extra-details
## More about overlay networks
.blackbelt[[Deep Dive in Docker Overlay Networks](https://www.youtube.com/watch?v=b3XDl0YsVsg&index=1&list=PLkA60AVN3hh-biQ6SCtBJ-WVTyBmmYho8) by Laurent Bernaille (DC17US)]
.blackbelt[DC17US: Deep Dive in Docker Overlay Networks ([video](https://www.youtube.com/watch?v=b3XDl0YsVsg&index=1&list=PLkA60AVN3hh-biQ6SCtBJ-WVTyBmmYho8))]
.blackbelt[Deeper Dive in Docker Overlay Networks by Laurent Bernaille (Wednesday 13:30)]
.blackbelt[DC17EU: Deeper Dive in Docker Overlay Networks ([video](https://dockercon.docker.com/watch/XkRRA7Etsznv7uAk1UKsri))]

View File

@@ -29,11 +29,17 @@
watch docker service ps dockercoins_worker
```
<!-- ```wait dockercoins_worker.1``` -->
<!-- ```keys ^C``` -->
- Hide the tasks that are shutdown:
```bash
watch -n1 "docker service ps dockercoins_worker | grep -v Shutdown.*Shutdown"
```
<!-- ```wait dockercoins_worker.1``` -->
<!-- ```keys ^C``` -->
]
If you had stopped the workers earlier, this will automatically restart them.

View File

@@ -68,13 +68,16 @@ class: secrets
```bash
docker service create \
--secret hackme --secret arewesecureyet \
--name dummyservice --mode global \
--name dummyservice \
--constraint node.hostname==$HOSTNAME \
alpine sleep 1000000000
```
]
We use a global service to make sure that there will be an instance on the local node.
We constrain the container to be on the local node for convenience.
<br/>
(We are going to use `docker exec` in just a moment!)
---
@@ -98,6 +101,9 @@ class: secrets
- Check the files in `/run/secrets`
<!-- ```bash grep . /run/secrets/*``` -->
<!-- ```bash exit``` -->
]
---

View File

@@ -46,7 +46,7 @@ class: extra-details
- SwarmKit implements the Raft algorithm directly
<br/>
(Nomad is similar; thanks [@cbednarski](https://twitter.com/@cbednarski),
[@diptanu](https://twitter.com/diptanu) and others for point it out!)
[@diptanu](https://twitter.com/diptanu) and others for pointing it out!)
- Analogy courtesy of [@aluzzardi](https://twitter.com/aluzzardi):

View File

@@ -152,6 +152,8 @@ It *cannot* work on live files, so you must stop Docker or make a copy first.
cp -r /graph/swarm /swarmdata
```
<!-- ```wait cp: cannot stat``` -->
- Otherwise, it is in the default `/var/lib/docker`:
```bash
sudo cp -r /var/lib/docker/swarm /swarmdata
@@ -174,6 +176,8 @@ It *cannot* work on live files, so you must stop Docker or make a copy first.
/lib/ld-musl-x86_64.so.1 /usr/local/bin/swarm-rafttool -d /swarmdata/ dump-wal
```
<!-- ```wait -bash:``` -->
- Otherwise, you don't need the musl linker but you need to get root:
```bash
sudo swarm-rafttool -d /swarmdata/ dump-wal

View File

@@ -1,6 +1,6 @@
## Brand new versions!
- Engine 17.11
- Engine 17.12
- Compose 1.17
- Machine 0.13
@@ -17,7 +17,7 @@
---
## Wait, what, 17.11 ?!?
## Wait, what, 17.12 ?!?
--

View File

@@ -31,5 +31,16 @@
excludedClasses: [@@EXCLUDE@@]
});
</script>
<!--
These two scripts will be available only when loading the
content using the pub/sub server. Otherwise, they'll just
404 and that's OK.
-->
<script src="/socket.io/socket.io.js">
</script>
<script src="/remote.js">
</script>
</body>
</html>