Compare commits

...

21 Commits

Author SHA1 Message Date
Jerome Petazzoni
0900d605ef fix-redirects.sh: adding forced redirect 2020-04-07 16:48:55 -05:00
Jerome Petazzoni
f67cfa8693 Add Lucas' amazing diagram 2018-02-21 18:17:57 -08:00
Jerome Petazzoni
cb8690f4a3 Add Lucas' amazing diagram 2018-02-21 18:16:59 -08:00
Jérôme Petazzoni
7a6d488d60 Merge pull request #112 from bridgetkromhout/indexconf2018
Clarification
2018-02-21 18:12:45 -08:00
Bridget Kromhout
b1d8b5eec8 Merge branch 'master' of https://github.com/jpetazzo/container.training into indexconf2018 2018-02-21 18:10:04 -08:00
Bridget Kromhout
1983d6cb4f Detail fix 2018-02-21 18:06:12 -08:00
Jérôme Petazzoni
e565da49ca Merge pull request #111 from bridgetkromhout/indexconf2018
Indexconf2018
2018-02-20 15:07:49 -08:00
Bridget Kromhout
ee33799a8f Only 2 worker nodes 2018-02-20 15:34:05 -06:00
Bridget Kromhout
b61426a044 emoji fix 2018-02-19 19:41:39 -06:00
Bridget Kromhout
fd057d8a1e Slight edits 2018-02-19 19:40:16 -06:00
Jerome Petazzoni
aab519177d Add indexconf2018 to index 2018-02-19 15:55:22 -08:00
Jerome Petazzoni
4b76fbcc4b Add _redirects 2018-02-19 15:50:22 -08:00
Bridget Kromhout
b25d40f48e indexconf2017 2018-02-18 20:21:55 -06:00
Jerome Petazzoni
5116ad7c44 Use kubeadm token generate to simplify things a bit.
Thanks @rmb938 for the suggestion!

Closes #110.
2018-01-18 21:14:40 +01:00
Jerome Petazzoni
7305e911e5 Update for k8s 1.9 2018-01-10 17:12:49 +01:00
Jerome Petazzoni
b2f670acf6 Add error checking for AMI finder script 2018-01-10 16:48:04 +01:00
Jerome Petazzoni
dc040aa693 Make sleep interruptible; fix slide count 2017-12-23 19:38:32 +01:00
Jerome Petazzoni
9b7a8494b0 Fix logic to advance to next snippet 2017-12-23 18:49:42 +01:00
Jerome Petazzoni
ae6c1bb8eb Major UI refactor
Navigation now includes all slides and all snippets.
ENTER skips to the next snippet, or executes the
selected snippet.

More improvements to come: allow SPACE to navigate
step by step through slides and snippets, executing
the snippets.
2017-12-23 09:31:01 +01:00
Jerome Petazzoni
a9a4f0ea07 Create only one remote session 2017-12-23 05:06:30 +01:00
Jerome Petazzoni
68af5940e3 Script node3 setup as well 2017-12-23 05:00:22 +01:00
23 changed files with 201 additions and 572 deletions

View File

@@ -134,17 +134,11 @@ _cmd_kube() {
sudo apt-get install -qy kubelet kubeadm kubectl
kubectl completion bash | sudo tee /etc/bash_completion.d/kubectl"
# Work around https://github.com/kubernetes/kubernetes/issues/53356
pssh "
if [ ! -f /etc/kubernetes/kubelet.conf ]; then
sudo systemctl stop kubelet
sudo rm -rf /var/lib/kubelet/pki
fi"
# Initialize kube master
pssh "
if grep -q node1 /tmp/node && [ ! -f /etc/kubernetes/admin.conf ]; then
sudo kubeadm init
kubeadm token generate > /tmp/token
sudo kubeadm init --token \$(cat /tmp/token)
fi"
# Put kubeconfig in ubuntu's and docker's accounts
@@ -157,15 +151,6 @@ _cmd_kube() {
sudo chown -R docker /home/docker/.kube
fi"
# Get bootstrap token
pssh "
if grep -q node1 /tmp/node; then
TOKEN_NAME=\$(kubectl -n kube-system get secret -o name | grep bootstrap-token)
TOKEN_ID=\$(kubectl -n kube-system get \$TOKEN_NAME -o go-template --template '{{ index .data \"token-id\" }}' | base64 -d)
TOKEN_SECRET=\$(kubectl -n kube-system get \$TOKEN_NAME -o go-template --template '{{ index .data \"token-secret\" }}' | base64 -d)
echo \$TOKEN_ID.\$TOKEN_SECRET >/tmp/token
fi"
# Install weave as the pod network
pssh "
if grep -q node1 /tmp/node; then
@@ -177,7 +162,7 @@ _cmd_kube() {
pssh "
if ! grep -q node1 /tmp/node && [ ! -f /etc/kubernetes/kubelet.conf ]; then
TOKEN=\$(ssh -o StrictHostKeyChecking=no node1 cat /tmp/token)
sudo kubeadm join --token \$TOKEN node1:6443
sudo kubeadm join --discovery-token-unsafe-skip-ca-verification --token \$TOKEN node1:6443
fi"
sep "Done"
@@ -280,6 +265,9 @@ _cmd_start() {
key_name=$(sync_keys)
AMI=$(_cmd_ami) # Retrieve the AWS image ID
if [ -z "$AMI" ]; then
die "I could not find which AMI to use in this region. Try another region?"
fi
TOKEN=$(get_token) # generate a timestamp token for this batch of VMs
AWS_KEY_NAME=$(make_key_name)

2
slides/_redirects Normal file
View File

@@ -0,0 +1,2 @@
/ /kube-halfday.yml.html 200!

View File

@@ -6,6 +6,7 @@ import logging
import os
import random
import re
import select
import subprocess
import sys
import time
@@ -25,14 +26,16 @@ class State(object):
self.interactive = True
self.verify_status = False
self.simulate_type = True
self.next_step = 0
self.slide = 1
self.snippet = 0
def load(self):
data = yaml.load(open("state.yaml"))
self.interactive = bool(data["interactive"])
self.verify_status = bool(data["verify_status"])
self.simulate_type = bool(data["simulate_type"])
self.next_step = int(data["next_step"])
self.slide = int(data["slide"])
self.snippet = int(data["snippet"])
def save(self):
with open("state.yaml", "w") as f:
@@ -40,7 +43,12 @@ class State(object):
interactive=self.interactive,
verify_status=self.verify_status,
simulate_type=self.simulate_type,
next_step=self.next_step), f, default_flow_style=False)
slide=self.slide,
snippet=self.snippet,
), f, default_flow_style=False)
state = State()
def hrule():
@@ -57,7 +65,15 @@ class Snippet(object):
def __init__(self, slide, content):
self.slide = slide
self.content = content
self.actions = []
# Extract the "method" (e.g. bash, keys, ...)
# On multi-line snippets, the method is alone on the first line
# On single-line snippets, the data follows the method immediately
if '\n' in content:
self.method, self.data = content.split('\n', 1)
else:
self.method, self.data = content.split(' ', 1)
self.data = self.data.strip()
self.next = None
def __str__(self):
return self.content
@@ -68,8 +84,8 @@ class Slide(object):
current_slide = 0
def __init__(self, content):
Slide.current_slide += 1
self.number = Slide.current_slide
Slide.current_slide += 1
# Remove commented-out slides
# (remark.js considers ??? to be the separator for speaker notes)
@@ -80,8 +96,13 @@ class Slide(object):
exercises = re.findall("\.exercise\[(.*)\]", content, re.DOTALL)
for exercise in exercises:
if "```" in exercise:
for snippet in exercise.split("```")[1::2]:
self.snippets.append(Snippet(self, snippet))
previous = None
for snippet_content in exercise.split("```")[1::2]:
snippet = Snippet(self, snippet_content)
if previous:
previous.next = snippet
previous = snippet
self.snippets.append(snippet)
else:
logging.warning("Exercise on slide {} does not have any ``` snippet."
.format(self.number))
@@ -96,43 +117,6 @@ class Slide(object):
def debug(self):
logging.debug("\n{}\n{}\n{}".format(hrule(), self.content, hrule()))
# Synchronize slides in a remote browser
class Remote(object):
def __init__(self):
self.slide_on_screen = 0
# Directly go to a specific slide
def goto(self, slide_number):
subprocess.check_call(["./gotoslide.js", str(slide_number)])
self.slide_on_screen = slide_number
focus_slides()
# Offer the opportunity to go step by step to the given slide
def catchup(self, slide_number):
if self.slide_on_screen > slide_number:
return self.goto(slide_number)
while self.slide_on_screen < slide_number:
if state.interactive:
click.clear()
print("Catching up on slide: {} -> {}"
.format(self.slide_on_screen, slide_number))
print("z/⏎ Zoom to target slide")
print("n/→/⎵ Next slide")
print("p/← Previous slide")
print("q Abort remote control")
command = click.getchar()
else:
command = "z"
if command in ("z", "\r"):
self.goto(slide_number)
elif command in ("n", "\x1b[C", " "):
self.goto(self.slide_on_screen+1)
elif command in ("p", "\x1b[D"):
self.goto(self.slide_on_screen-1)
elif command == "q":
return
def focus_slides():
subprocess.check_output(["i3-msg", "workspace", "3"])
@@ -147,14 +131,17 @@ def focus_browser():
subprocess.check_output(["i3-msg", "workspace", "1"])
remote = Remote()
state = State()
def ansi(code):
return lambda s: "\x1b[{}m{}\x1b[0m".format(code, s)
# Sleeps the indicated delay, but interruptible by pressing ENTER.
# If interrupted, returns True.
def interruptible_sleep(t):
rfds, _, _ = select.select([0], [], [], t)
return 0 in rfds
def wait_for_string(s, timeout=TIMEOUT):
logging.debug("Waiting for string: {}".format(s))
deadline = time.time() + timeout
@@ -162,7 +149,7 @@ def wait_for_string(s, timeout=TIMEOUT):
output = capture_pane()
if s in output:
return
time.sleep(1)
if interruptible_sleep(1): return
raise Exception("Timed out while waiting for {}!".format(s))
@@ -173,11 +160,17 @@ def wait_for_prompt():
output = capture_pane()
# If we are not at the bottom of the screen, there will be a bunch of extra \n's
output = output.rstrip('\n')
if output.endswith("\n$"):
last_line = output.split('\n')[-1]
# Our custom prompt on the VMs has two lines; the 2nd line is just '$'
if last_line == "$":
return
if output.endswith("\n/ #"):
# When we are in an alpine container, the prompt will be "/ #"
if last_line == "/ #":
return
time.sleep(1)
# We did not recognize a known prompt; wait a bit and check again
logging.debug("Could not find a known prompt on last line: {!r}"
.format(last_line))
if interruptible_sleep(1): return
raise Exception("Timed out while waiting for prompt!")
@@ -216,7 +209,7 @@ tmux
2. If you want to control a remote tmux:
rm -f /tmp/tmux-{uid}/default && ssh -t -L /tmp/tmux-{uid}/default:/tmp/tmux-1001/default docker@{ipaddr} tmux
rm -f /tmp/tmux-{uid}/default && ssh -t -L /tmp/tmux-{uid}/default:/tmp/tmux-1001/default docker@{ipaddr} tmux new-session -As 0
3. If you cannot control a remote tmux:
@@ -228,8 +221,7 @@ tmux new-session ssh docker@{ipaddr}
logging.info("Successfully connected to test cluster in tmux session.")
slides = []
slides = [Slide("Dummy slide zero")]
content = open(sys.argv[1]).read()
# OK, this part is definitely hackish, and will break if the
@@ -247,19 +239,6 @@ for slide in re.split("\n---?\n", content):
continue
slides.append(Slide(slide))
actions = []
for slide in slides:
for snippet in slide.snippets:
content = snippet.content
# Extract the "method" (e.g. bash, keys, ...)
# On multi-line snippets, the method is alone on the first line
# On single-line snippets, the data follows the method immediately
if '\n' in content:
method, data = content.split('\n', 1)
else:
method, data = content.split(' ', 1)
actions.append((slide, snippet, method, data))
def send_keys(data):
if state.simulate_type and data[0] != '^':
@@ -267,14 +246,15 @@ def send_keys(data):
if key == ";":
key = "\\;"
if key == "\n":
time.sleep(1)
if interruptible_sleep(1): return
subprocess.check_call(["tmux", "send-keys", key])
time.sleep(0.2*random.random())
if interruptible_sleep(0.15*random.random()): return
if key == "\n":
time.sleep(1)
if interruptible_sleep(1): return
else:
subprocess.check_call(["tmux", "send-keys", data])
def capture_pane():
return subprocess.check_output(["tmux", "capture-pane", "-p"]).decode('utf-8')
@@ -293,63 +273,90 @@ except Exception as e:
logging.exception("Could not load state from file.")
logging.warning("Using default values.")
def move_forward():
state.snippet += 1
if state.snippet > len(slides[state.slide].snippets):
state.slide += 1
state.snippet = 0
check_bounds()
while state.next_step < len(actions):
def move_backward():
state.snippet -= 1
if state.snippet < 0:
state.slide -= 1
state.snippet = 0
check_bounds()
def check_bounds():
if state.slide < 1:
state.slide = 1
if state.slide >= len(slides):
state.slide = len(slides)-1
while True:
state.save()
slide, snippet, method, data = actions[state.next_step]
# Remove extra spaces (we don't want them in the terminal) and carriage returns
data = data.strip()
# Synchronize the remote slides
remote.catchup(slide.number)
slide = slides[state.slide]
snippet = slide.snippets[state.snippet-1] if state.snippet else None
click.clear()
print("[Slide {}/{}] [Snippet {}/{}] [simulate_type:{}] [verify_status:{}]"
.format(state.slide, len(slides)-1,
state.snippet, len(slide.snippets) if slide.snippets else 0,
state.simulate_type, state.verify_status))
print(hrule())
print(slide.content.replace(snippet.content, ansi(7)(snippet.content)))
if snippet:
print(slide.content.replace(snippet.content, ansi(7)(snippet.content)))
focus_terminal()
else:
print(slide.content)
subprocess.check_output(["./gotoslide.js", str(slide.number)])
focus_slides()
print(hrule())
if state.interactive:
print("simulate_type:{} verify_status:{}".format(state.simulate_type, state.verify_status))
print("[{}/{}] Shall we execute that snippet above?".format(state.next_step, len(actions)))
print("y/⎵/⏎ Execute snippet")
print("p/← Previous snippet")
print("n/→ Next snippet")
print("y/⎵/⏎ Execute snippet or advance to next snippet")
print("p/← Previous")
print("n/→ Next")
print("s Simulate keystrokes")
print("v Validate exit status")
print("g Go to a specific snippet")
print("g Go to a specific slide")
print("q Quit")
print("c Continue non-interactively until next error")
command = click.getchar()
else:
command = "y"
# For now, remove the `highlighted` sections
# (Make sure to use $() in shell snippets!)
if '`' in data:
logging.info("Stripping ` from snippet.")
data = data.replace('`', '')
if command in ("n", "\x1b[C"):
state.next_step += 1
move_forward()
elif command in ("p", "\x1b[D"):
state.next_step -= 1
move_backward()
elif command == "s":
state.simulate_type = not state.simulate_type
elif command == "v":
state.verify_status = not state.verify_status
elif command == "g":
state.next_step = click.prompt("Enter snippet number", type=int)
# Special case: if we go to snippet 0, also reset the slides deck
if state.next_step == 0:
remote.goto(1)
state.slide = click.prompt("Enter slide number", type=int)
state.snippet = 0
check_bounds()
elif command == "q":
break
elif command == "c":
# continue until next timeout
state.interactive = False
elif command in ("y", "\r", " "):
focus_terminal()
if not snippet:
# Advance to next snippet
# Advance until a slide that has snippets
while not slides[state.slide].snippets:
move_forward()
# But stop if we reach the last slide
if state.slide == len(slides)-1:
break
# And then advance to the snippet
move_forward()
continue
method, data = snippet.method, snippet.data
logging.info("Running with method {}: {}".format(method, data))
if method == "keys":
send_keys(data)
@@ -358,20 +365,21 @@ while state.next_step < len(actions):
wait_for_prompt()
# Strip leading spaces
data = re.sub("\n +", "\n", data)
# Remove backticks (they are used to highlight sections)
data = data.replace('`', '')
# Add "RETURN" at the end of the command :)
data += "\n"
# Send command
send_keys(data)
# Force a short sleep to avoid race condition
time.sleep(0.5)
_, _, next_method, next_data = actions[state.next_step+1]
if next_method == "wait":
wait_for_string(next_data)
elif next_method == "longwait":
wait_for_string(next_data, 10*TIMEOUT)
if snippet.next and snippet.next.method == "wait":
wait_for_string(snippet.next.data)
elif snippet.next and snippet.next.method == "longwait":
wait_for_string(snippet.next.data, 10*TIMEOUT)
else:
wait_for_prompt()
# Verify return code FIXME should be optional
# Verify return code
check_exit_status()
elif method == "copypaste":
screen = capture_pane()
@@ -399,7 +407,7 @@ while state.next_step < len(actions):
click.getchar()
else:
logging.warning("Unknown method {}: {!r}".format(method, data))
state.next_step += 1
move_forward()
else:
logging.warning("Unknown command {}.".format(command))

View File

@@ -9,12 +9,10 @@
- We recommend having a mentor to help you ...
- ... Or be comfortable spending some time reading the Docker
[documentation](https://docs.docker.com/) ...
- ... Or be comfortable spending some time reading the Kubernetes
[documentation](https://kubernetes.io/docs/) ...
- ... And looking for answers in the [Docker forums](forums.docker.com),
[StackOverflow](http://stackoverflow.com/questions/tagged/docker),
and other outlets
- ... And looking for answers on [StackOverflow](http://stackoverflow.com/questions/tagged/kubernetes) and other outlets
---

View File

@@ -26,7 +26,7 @@ class: extra-details
- This slide should have a little magnifying glass in the top left corner
(If it doesn't, it's because CSS is hard — Jérôme is only a backend person, alas)
(If it doesn't, it's because CSS is hard — we're only backend people, alas!)
- Slides with that magnifying glass indicate slides providing extra details
@@ -62,7 +62,7 @@ Misattributed to Benjamin Franklin
- This is the stuff you're supposed to do!
- Go to [container.training](http://container.training/) to view these slides
- Go to [indexconf2018.container.training](http://indexconf2018.container.training/) to view these slides
- Join the chat room on @@CHAT@@
@@ -78,17 +78,11 @@ class: in-person
---
class: in-person, pic
![You get five VMs](images/you-get-five-vms.jpg)
---
class: in-person
## You get five VMs
## You get three VMs
- Each person gets 5 private VMs (not shared with anybody else)
- Each person gets 3 private VMs (not shared with anybody else)
- They'll remain up for the duration of the workshop
@@ -96,7 +90,7 @@ class: in-person
- You can automatically SSH from one VM to another
- The nodes have aliases: `node1`, `node2`, etc.
- The nodes have aliases: `node1`, `node2`, `node3`.
---
@@ -153,7 +147,7 @@ class: in-person
<!--
```bash
for N in $(seq 1 5); do
for N in $(seq 1 3); do
ssh -o StrictHostKeyChecking=no node$N true
done
```

View File

@@ -21,65 +21,6 @@
---
class: extra-details
## Compose file format version
*Particularly relevant if you have used Compose before...*
- Compose 1.6 introduced support for a new Compose file format (aka "v2")
- Services are no longer at the top level, but under a `services` section
- There has to be a `version` key at the top level, with value `"2"` (as a string, not an integer)
- Containers are placed on a dedicated network, making links unnecessary
- There are other minor differences, but upgrade is easy and straightforward
---
## Links, naming, and service discovery
- Containers can have network aliases (resolvable through DNS)
- Compose file version 2+ makes each container reachable through its service name
- Compose file version 1 did require "links" sections
- Our code can connect to services using their short name
(instead of e.g. IP address or FQDN)
- Network aliases are automatically namespaced
(i.e. you can have multiple apps declaring and using a service named `database`)
---
## Example in `worker/worker.py`
```python
redis = Redis("`redis`")
def get_random_bytes():
r = requests.get("http://`rng`/32")
return r.content
def hash_bytes(data):
r = requests.post("http://`hasher`/",
data=data,
headers={"Content-Type": "application/octet-stream"})
```
(Full source code available [here](
https://github.com/jpetazzo/container.training/blob/8279a3bce9398f7c1a53bdd95187c53eda4e6435/dockercoins/worker/worker.py#L17
))
---
## What's this application?
--
@@ -124,7 +65,7 @@ fi
- Clone the repository on `node1`:
```bash
git clone git://github.com/jpetazzo/container.training
git clone https://github.com/jpetazzo/container.training/
```
]
@@ -183,77 +124,6 @@ and displays aggregated logs.
---
## Restarting in the background
- Many flags and commands of Compose are modeled after those of `docker`
.exercise[
- Start the app in the background with the `-d` option:
```bash
docker-compose up -d
```
- Check that our app is running with the `ps` command:
```bash
docker-compose ps
```
]
`docker-compose ps` also shows the ports exposed by the application.
---
class: extra-details
## Viewing logs
- The `docker-compose logs` command works like `docker logs`
.exercise[
- View all logs since container creation and exit when done:
```bash
docker-compose logs
```
- Stream container logs, starting at the last 10 lines for each container:
```bash
docker-compose logs --tail 10 --follow
```
<!--
```wait units of work done```
```keys ^C```
-->
]
Tip: use `^S` and `^Q` to pause/resume log output.
---
class: extra-details
## Upgrading from Compose 1.6
.warning[The `logs` command has changed between Compose 1.6 and 1.7!]
- Up to 1.6
- `docker-compose logs` is the equivalent of `logs --follow`
- `docker-compose logs` must be restarted if containers are added
- Since 1.7
- `--follow` must be specified explicitly
- new containers are automatically picked up by `docker-compose logs`
---
## Connecting to the web UI
- The `webui` container exposes a web dashboard; let's view it
@@ -275,245 +145,6 @@ graph will appear.
---
class: self-paced, extra-details
## If the graph doesn't load
If you just see a `Page not found` error, it might be because your
Docker Engine is running on a different machine. This can be the case if:
- you are using the Docker Toolbox
- you are using a VM (local or remote) created with Docker Machine
- you are controlling a remote Docker Engine
When you run DockerCoins in development mode, the web UI static files
are mapped to the container using a volume. Alas, volumes can only
work on a local environment, or when using Docker4Mac or Docker4Windows.
How to fix this?
Edit `dockercoins.yml` and comment out the `volumes` section, and try again.
---
class: extra-details
## Why does the speed seem irregular?
- It *looks like* the speed is approximately 4 hashes/second
- Or more precisely: 4 hashes/second, with regular dips down to zero
- Why?
--
class: extra-details
- The app actually has a constant, steady speed: 3.33 hashes/second
<br/>
(which corresponds to 1 hash every 0.3 seconds, for *reasons*)
- Yes, and?
---
class: extra-details
## The reason why this graph is *not awesome*
- The worker doesn't update the counter after every loop, but up to once per second
- The speed is computed by the browser, checking the counter about once per second
- Between two consecutive updates, the counter will increase either by 4, or by 0
- The perceived speed will therefore be 4 - 4 - 4 - 0 - 4 - 4 - 0 etc.
- What can we conclude from this?
--
class: extra-details
- Jérôme is clearly incapable of writing good frontend code
---
## Scaling up the application
- Our goal is to make that performance graph go up (without changing a line of code!)
--
- Before trying to scale the application, we'll figure out if we need more resources
(CPU, RAM...)
- For that, we will use good old UNIX tools on our Docker node
---
## Looking at resource usage
- Let's look at CPU, memory, and I/O usage
.exercise[
- run `top` to see CPU and memory usage (you should see idle cycles)
<!--
```bash top```
```wait Tasks```
```keys ^C```
-->
- run `vmstat 1` to see I/O usage (si/so/bi/bo)
<br/>(the 4 numbers should be almost zero, except `bo` for logging)
<!--
```bash vmstat 1```
```wait memory```
```keys ^C```
-->
]
We have available resources.
- Why?
- How can we use them?
---
## Scaling workers on a single node
- Docker Compose supports scaling
- Let's scale `worker` and see what happens!
.exercise[
- Start one more `worker` container:
```bash
docker-compose scale worker=2
```
- Look at the performance graph (it should show a x2 improvement)
- Look at the aggregated logs of our containers (`worker_2` should show up)
- Look at the impact on CPU load with e.g. top (it should be negligible)
]
---
## Adding more workers
- Great, let's add more workers and call it a day, then!
.exercise[
- Start eight more `worker` containers:
```bash
docker-compose scale worker=10
```
- Look at the performance graph: does it show a x10 improvement?
- Look at the aggregated logs of our containers
- Look at the impact on CPU load and memory usage
]
---
# Identifying bottlenecks
- You should have seen a 3x speed bump (not 10x)
- Adding workers didn't result in linear improvement
- *Something else* is slowing us down
--
- ... But what?
--
- The code doesn't have instrumentation
- Let's use state-of-the-art HTTP performance analysis!
<br/>(i.e. good old tools like `ab`, `httping`...)
---
## Accessing internal services
- `rng` and `hasher` are exposed on ports 8001 and 8002
- This is declared in the Compose file:
```yaml
...
rng:
build: rng
ports:
- "8001:80"
hasher:
build: hasher
ports:
- "8002:80"
...
```
---
## Measuring latency under load
We will use `httping`.
.exercise[
- Check the latency of `rng`:
```bash
httping -c 3 localhost:8001
```
- Check the latency of `hasher`:
```bash
httping -c 3 localhost:8002
```
]
`rng` has a much higher latency than `hasher`.
---
## Let's draw hasty conclusions
- The bottleneck seems to be `rng`
- *What if* we don't have enough entropy and can't generate enough random numbers?
- We need to scale out the `rng` service on multiple machines!
Note: this is a fiction! We have enough entropy. But we need a pretext to scale out.
(In fact, the code of `rng` uses `/dev/urandom`, which never runs out of entropy...
<br/>
...and is [just as good as `/dev/random`](http://www.slideshare.net/PacSecJP/filippo-plain-simple-reality-of-entropy).)
---
## Clean up
- Before moving on, let's remove those containers

View File

@@ -6,7 +6,7 @@ Thank you!
class: title, in-person
That's all folks! <br/> Questions?
That's all, folks! <br/> Questions?
![end](images/end.jpg)
@@ -14,13 +14,9 @@ That's all folks! <br/> Questions?
# Links and resources
- [Docker Community Slack](https://community.docker.com/registrations/groups/4316)
- [Docker Community Forums](https://forums.docker.com/)
- [Docker Hub](https://hub.docker.com)
- [Docker Blog](http://blog.docker.com/)
- [Docker documentation](http://docs.docker.com/)
- [Docker on StackOverflow](https://stackoverflow.com/questions/tagged/docker)
- [Docker on Twitter](http://twitter.com/docker)
- [Play With Docker Hands-On Labs](http://training.play-with-docker.com/)
- [Kubernetes Community](https://kubernetes.io/community/) - Slack, Google Groups, meetups
- [Play With Kubernetes Hands-On Labs](https://medium.com/@marcosnils/introducing-pwk-play-with-k8s-159fcfeb787b)
- [Local meetups](https://www.meetup.com/)
- [Microsoft Cloud Developer Advocates](https://developer.microsoft.com/en-us/advocates/)
.footnote[These slides (and future updates) are on → http://container.training/]

View File

@@ -17,5 +17,5 @@ class: title, in-person
*Don't stream videos or download big files during the workshop.*<br/>
*Thank you!*
**Slides: http://container.training/**
]
**Slides: http://indexconf2018.container.training/**
]

Binary file not shown.

After

Width:  |  Height:  |  Size: 144 KiB

View File

@@ -66,14 +66,15 @@
<tr><td class="title" colspan="4">Coming soon at a conference near you</td></tr>
<!--
<td><a class="attend" href="https://qconsf.com/sf2017/workshop/orchestrating-microservices-docker-swarm" /></td>
-->
<tr>
<!--
<td>Nothing for now (stay tuned...)</td>
-->
<td>February 22, 2018: IndexConf — Kubernetes 101</td>
<td><a class="slides" href="http://indexconf2018.container.training/" /></td>
<td><a class="attend" href="https://developer.ibm.com/indexconf/sessions/#!?id=5474" />
</tr>
<tr><td class="title" colspan="4">Past workshops</td></tr>
<tr>
@@ -157,4 +158,4 @@
</table>
</div>
</body>
</html>
</html>

View File

@@ -1,9 +1,7 @@
title: |
Deploying and Scaling Microservices
with Docker and Kubernetes
Kubernetes 101
chat: "[Slack](https://dockercommunity.slack.com/messages/C7GKACWDV)"
#chat: "[Gitter](https://gitter.im/jpetazzo/workshop-yyyymmdd-city)"
chat: "[Gitter](https://gitter.im/jpetazzo/workshop-20180222-sf)"
exclude:
- self-paced

View File

@@ -211,3 +211,11 @@ class: pic
![Node, pod, container](images/k8s-arch3-thanks-weave.png)
(Diagram courtesy of Weave Works, used with permission.)
---
class: pic
![One of the best Kubernetes architecture diagrams available](images/k8s-arch4-thanks-luxas.png)
(Diagram courtesy of Lucas Käldström, in [this presentation](https://speakerdeck.com/luxas/kubeadm-cluster-creation-internals-from-self-hosting-to-upgradability-and-ha).)

View File

@@ -1,10 +1,8 @@
# Daemon sets
- Remember: we did all that cluster orchestration business for `rng`
- What if we want one (and exactly one) instance of `rng` per node?
- We want one (and exactly one) instance of `rng` per node
- If we just scale `deploy/rng` to 4, nothing guarantees that they spread
- If we just scale `deploy/rng` to 2, nothing guarantees that they spread
- Instead of a `deployment`, we will use a `daemonset`
@@ -22,7 +20,7 @@
## Creating a daemon set
- Unfortunately, as of Kubernetes 1.8, the CLI cannot create daemon sets
- Unfortunately, as of Kubernetes 1.9, the CLI cannot create daemon sets
--
@@ -382,7 +380,7 @@ Of course, option 2 offers more learning opportunities. Right?
.exercise[
- Check the logs of all `run=rng` pods to confirm that only 4 of them are now active:
- Check the logs of all `run=rng` pods to confirm that only 2 of them are now active:
```bash
kubectl logs -l run=rng
```
@@ -406,4 +404,4 @@ The timestamps should give us a hint about how many pods are currently receiving
- Bonus exercise 1: clean up the pods of the "old" daemon set
- Bonus exercise 2: how could we have done to avoid creating new pods?
- Bonus exercise 2: how could we have done this to avoid creating new pods?

View File

@@ -89,7 +89,13 @@ The goo.gl URL expands to:
- Connect to https://oneofournodes:3xxxx/
(You will have to work around the TLS certificate validation warning)
- Yes, https. If you use http it will say:
This page isnt working
<oneofournodes> sent an invalid response.
ERR_INVALID_HTTP_RESPONSE
- You will have to work around the TLS certificate validation warning
<!-- ```open https://node1:3xxxx/``` -->
@@ -109,7 +115,7 @@ The goo.gl URL expands to:
## Granting more rights to the dashboard
- The dashboard documentation [explains how to do](https://github.com/kubernetes/dashboard/wiki/Access-control#admin-privileges)
- The dashboard documentation [explains how to do this](https://github.com/kubernetes/dashboard/wiki/Access-control#admin-privileges)
- We just need to load another YAML file!

View File

@@ -48,7 +48,7 @@
.exercise[
- Give us more info about them nodes:
- Give us more info about the nodes:
```bash
kubectl get nodes -o wide
```
@@ -136,7 +136,7 @@ There is already one service on our cluster: the Kubernetes API itself.
```
- `-k` is used to skip certificate verification
- Make sure to replace 10.96.0.1 with the CLUSTER-IP shown earlier
- Make sure to replace 10.96.0.1 with the CLUSTER-IP shown by `$ kubectl get svc`
]
@@ -173,7 +173,7 @@ The error that we see is expected: the Kubernetes API requires authentication.
## Namespaces
- Namespaces allow to segregate resources
- Namespaces allow us to segregate resources
.exercise[

View File

@@ -245,4 +245,4 @@ at the Google NOC ...
<br/>
.small[are we getting 1000 packets per second]
<br/>
.small[of ICMP ECHO traffic from EC2 ?!?”]
.small[of ICMP ECHO traffic from Azure ?!?”]

View File

@@ -40,7 +40,7 @@ In this part, we will:
- We could use the Docker Hub
- Or a service offered by our cloud provider (GCR, ECR...)
- Or a service offered by our cloud provider (ACR, GCR, ECR...)
- Or we could just self-host that registry

View File

@@ -149,7 +149,7 @@ Our rollout is stuck. However, the app is not dead (just 10% slower).
- We want to:
- revert to `v0.1`
- revert to `v0.1` (which we now realize we didn't tag - yikes!)
- be conservative on availability (always have desired number of available workers)
- be aggressive on rollout speed (update more than one pod at a time)
- give some time to our workers to "warm up" before starting more
@@ -163,7 +163,7 @@ spec:
spec:
containers:
- name: worker
image: $REGISTRY/worker:v0.1
image: $REGISTRY/worker:latest
strategy:
rollingUpdate:
maxUnavailable: 0
@@ -192,7 +192,7 @@ spec:
spec:
containers:
- name: worker
image: $REGISTRY/worker:v0.1
image: $REGISTRY/worker:latest
strategy:
rollingUpdate:
maxUnavailable: 0

View File

@@ -4,7 +4,7 @@
--
- We used `kubeadm` on "fresh" EC2 instances with Ubuntu 16.04 LTS
- We used `kubeadm` on Azure instances with Ubuntu 16.04 LTS
1. Install Docker
@@ -36,26 +36,25 @@
--
- It's still twice as many steps as setting up a Swarm cluster 😕
- "It's still twice as many steps as setting up a Swarm cluster 😕 " -- Jérôme
---
## Other deployment options
- If you are on Google Cloud:
[GKE](https://cloud.google.com/container-engine/)
- If you are on Azure:
[AKS](https://azure.microsoft.com/services/container-service/)
Empirically the best Kubernetes deployment out there
- If you are on Google Cloud:
[GKE](https://cloud.google.com/kubernetes-engine/)
- If you are on AWS:
[kops](https://github.com/kubernetes/kops)
... But with AWS re:invent just around the corner, expect some changes
[EKS](https://aws.amazon.com/eks/)
- On a local machine:
[minikube](https://kubernetes.io/docs/getting-started-guides/minikube/),
[kubespawn](https://github.com/kinvolk/kube-spawn),
[Docker4Mac (coming soon)](https://beta.docker.com/)
[Docker4Mac](https://docs.docker.com/docker-for-mac/kubernetes/)
- If you want something customizable:
[kubicorn](https://github.com/kris-nova/kubicorn)

View File

@@ -1,8 +1,8 @@
## Brand new versions!
## Versions Installed
- Kubernetes 1.8
- Docker Engine 17.11
- Docker Compose 1.17
- Kubernetes 1.9.3
- Docker Engine 18.02.0-ce
- Docker Compose 1.18.0
.exercise[

View File

@@ -131,6 +131,7 @@ And *then* it is time to look at orchestration!
- shell scripts invoking `kubectl`
- YAML resources descriptions committed to a repo
- [Brigade](https://brigade.sh/) (event-driven scripting; no YAML)
- [Helm](https://github.com/kubernetes/helm) (~package manager)
- [Spinnaker](https://www.spinnaker.io/) (Netflix' CD platform)
@@ -160,7 +161,7 @@ Sorry Star Trek fans, this is not the federation you're looking for!
- Raft recommends low latency between nodes
- What if our cluster spreads multiple regions?
- What if our cluster spreads to multiple regions?
--

View File

@@ -2,18 +2,19 @@
- Hello! We are:
- .emoji[👷🏻‍♀️] AJ ([@s0ulshake](https://twitter.com/s0ulshake), Travis CI)
- .emoji[✨] Bridget ([@bridgetkromhout](https://twitter.com/bridgetkromhout))
- .emoji[🐳] Jérôme ([@jpetazzo](https://twitter.com/jpetazzo), Docker Inc.)
- .emoji[🌟] Jessica ([@jldeen](https://twitter.com/jldeen))
- The workshop will run from 9am to 4pm
- .emoji[🐳] Jérôme ([@jpetazzo](https://twitter.com/jpetazzo))
- There will be a lunch break at noon
- This workshop will run from 10:30am-12:45pm.
(And coffee breaks!)
- Lunchtime is after the workshop!
(And we will take a 15min break at 11:30am!)
- Feel free to interrupt for questions at any time
- *Especially when you see full screen container pictures!*
- Live feedback, questions, help on @@CHAT@@

View File

@@ -22,7 +22,7 @@ With Play-With-Docker:
```bash
TOKEN=$(docker swarm join-token -q manager)
for N in $(seq 4 5); do
for N in $(seq 3 5); do
export DOCKER_HOST=tcp://node$N:2375
docker swarm join --token $TOKEN node1:2377
done