Compare commits

...

219 Commits

Author SHA1 Message Date
Jerome Petazzoni
c97d7faa40 paris.container.training 2018-03-07 12:51:03 -08:00
Jerome Petazzoni
aca51901a1 Tag images properly
This tags the first build with v0.1, allowing for a smoother, more
logical rollback. Also adds a slide explaining why to stay away
from latest. @kelseyhightower would be proud :-)
2018-03-05 16:13:30 -08:00
Jerome Petazzoni
c778fc84ed Add a dump of the checklist I use when delivering 2018-03-05 14:30:39 -08:00
Jérôme Petazzoni
1981ac0b93 Merge pull request #135 from bridgetkromhout/bridget-specific
Adding Bridget-specific files
2018-03-05 13:36:06 -08:00
Jérôme Petazzoni
a8f2fb4586 Merge pull request #134 from bridgetkromhout/dedup-thanks
De-dup thanks; add comma
2018-03-05 13:35:45 -08:00
Jérôme Petazzoni
a69d3d0828 Merge pull request #133 from bridgetkromhout/no-chatroom
Makes more sense for "in person" chat
2018-03-05 13:32:51 -08:00
Jérôme Petazzoni
40760f9e98 Merge pull request #131 from bridgetkromhout/change-instance-type
Changing Azure instance type
2018-03-05 13:25:49 -08:00
Bridget Kromhout
b64b16dd67 Adding Bridget-specific files 2018-03-05 14:54:28 -06:00
Bridget Kromhout
8c2c9bc5df De-dup thanks; add comma 2018-03-05 14:51:26 -06:00
Bridget Kromhout
3a21cbc72b Makes more sense for "in person" chat 2018-03-05 14:37:10 -06:00
Bridget Kromhout
a09521ceb1 Changing Azure instance type 2018-03-05 13:44:02 -06:00
Jérôme Petazzoni
0d6501a926 Merge pull request #130 from atsaloli/patch-1
Two small fixes
2018-03-05 10:10:25 -08:00
Aleksey Tsalolikhin
c25f7a119b Fix very small typo -- remove extra "v" in "code" 2018-03-04 19:58:27 -08:00
Aleksey Tsalolikhin
1958c85a96 Fix noun plural tense (change "instructions" -> "instruction")
"An" means one. So "an instruction" rather than "an instructions".  (Small grammar fix.)
2018-03-04 19:56:03 -08:00
Jérôme Petazzoni
a7ba4418c6 Merge pull request #129 from bridgetkromhout/improve-directions
Improve directions
2018-03-03 19:52:15 -06:00
Bridget Kromhout
d6fcbb85e8 Improve directions 2018-03-03 18:44:56 -06:00
Jérôme Petazzoni
278fbf285a Merge pull request #128 from bridgetkromhout/cleanup
Cleanup
2018-03-03 14:39:56 -06:00
Bridget Kromhout
ca828343e4 Remove azure instances post-workshop. 2018-03-03 08:51:54 -06:00
Bridget Kromhout
5c663f9e09 Updating help output 2018-03-03 08:48:02 -06:00
Bridget Kromhout
9debd76816 Document kubetest 2018-03-03 08:44:58 -06:00
Bridget Kromhout
848679829d Removed -i and trailing space 2018-03-02 18:18:04 -06:00
Bridget Kromhout
6727007754 Missing variable 2018-03-02 18:11:32 -06:00
Jerome Petazzoni
03a563c172 Merge branch 'master' of github.com:jpetazzo/container.training 2018-03-02 14:17:54 -06:00
Jerome Petazzoni
cfbd54bebf Add hacky-backslashy kubetest command 2018-03-02 14:17:37 -06:00
Jérôme Petazzoni
7f1e9db0fa Missing curly brace 2018-03-02 13:08:48 -06:00
Jérôme Petazzoni
1367a30a11 Merge pull request #126 from bridgetkromhout/add-azure
Adding Azure examples
2018-03-02 12:46:02 -06:00
Bridget Kromhout
31b234ee3a Adding Azure examples 2018-03-02 12:42:55 -06:00
Jérôme Petazzoni
57dd5e295e Merge pull request #125 from bridgetkromhout/increase-timeouts
Increase timeouts
2018-03-01 17:43:29 -06:00
Bridget Kromhout
c188923f1a Increase timeouts 2018-03-01 17:39:51 -06:00
Jérôme Petazzoni
7a8716d38b Merge pull request #124 from bridgetkromhout/postprep
Postprep is now python
2018-03-01 17:17:04 -06:00
Bridget Kromhout
2e77c13297 Postprep is now python 2018-03-01 17:15:01 -06:00
Jerome Petazzoni
d5279d881d Add info about pre-built images 2018-03-01 15:13:39 -06:00
Jerome Petazzoni
34e9cc1944 Don't assume 5 nodes 2018-03-01 14:55:02 -06:00
Jerome Petazzoni
2a7498e30e A bit of rewording, and a couple of links about dashboard security 2018-03-01 14:51:00 -06:00
Jerome Petazzoni
4689d09e1f One typo and two minor tweaks 2018-03-01 14:18:48 -06:00
Jerome Petazzoni
b818a38307 Correctly report errors happening in functions
`trap ... ERR` does not automatically propagate to functions. Therefore,
Our fancy error-reporting mechanism did not catch errors happening in
functions; and we do most of the actual work in functions. The solution
is to `set -E` or `set -o errtrace`.
2018-03-01 13:56:08 -06:00
Jérôme Petazzoni
7e5d869472 Merge pull request #123 from bridgetkromhout/kube101
Kube101 & non-AWS
2018-03-01 13:23:04 -06:00
Jérôme Petazzoni
3eaf31fd48 Merge pull request #122 from bridgetkromhout/pssh-clarity
Pssh clarity
2018-03-01 13:21:05 -06:00
Bridget Kromhout
fe5e22f5ae How to set up non-AWS workshops 2018-02-28 21:45:36 -06:00
Bridget Kromhout
61da583080 Don't overwrite ip file if exists 2018-02-28 21:44:58 -06:00
Bridget Kromhout
94dfe1a0cd Adding sample file mentioned in README 2018-02-28 21:44:29 -06:00
Bridget Kromhout
412dbadafd Adding settings for kube101 2018-02-28 21:43:41 -06:00
Bridget Kromhout
8c5e4e0b09 Require pssh 2018-02-28 21:28:20 -06:00
Bridget Kromhout
2ac6072d80 Invoke as pssh 2018-02-28 21:26:17 -06:00
Jerome Petazzoni
ef4591c4fc Allow to override instance type (closes #39) 2018-02-28 13:45:08 -06:00
Jerome Petazzoni
22dfbab09b Minor formatting 2018-02-28 13:41:22 -06:00
Jérôme Petazzoni
37f595c480 Merge pull request #120 from bridgetkromhout/clarify-kube-public
Clarify kube-public; define kube-system
2018-02-27 17:42:11 -06:00
Bridget Kromhout
1fc951037d Slight clarification per request 2018-02-27 17:39:52 -06:00
Jérôme Petazzoni
affd46dd88 Merge pull request #121 from bridgetkromhout/obviate-https
Remove need for https in the workshop dashboard
2018-02-27 17:34:27 -06:00
Bridget Kromhout
cfaff3df04 Remove need for https in the workshop dashboard 2018-02-27 17:31:14 -06:00
Jérôme Petazzoni
ce2451971d Merge pull request #118 from bridgetkromhout/twice-the-steps
Proper attribution
2018-02-27 16:57:52 -06:00
Jérôme Petazzoni
8cf5d0efbd Merge pull request #119 from bridgetkromhout/naming-things
Naming things is hard; considering scope here
2018-02-27 16:40:40 -06:00
Bridget Kromhout
f61d61223d Clarify kube-public; define kube-system 2018-02-27 16:31:36 -06:00
Bridget Kromhout
6b6eb50f9a Naming things is hard; considering scope here 2018-02-27 15:26:43 -06:00
Jerome Petazzoni
89ab66335f ... and trim down kube half-day 2018-02-27 14:49:39 -06:00
Jerome Petazzoni
5bc4e95515 Clarify service discovery 2018-02-27 14:45:08 -06:00
Jerome Petazzoni
893f05e401 Move docker-compose logs to the composescale.md chapter 2018-02-27 14:38:41 -06:00
Bridget Kromhout
4abc8ce34c Proper attribution 2018-02-27 14:38:32 -06:00
Jérôme Petazzoni
34d2c610bf Merge pull request #117 from bridgetkromhout/self-deprecating-humor
Attributing humor so it doesn't sound negative
2018-02-27 14:06:58 -06:00
Jerome Petazzoni
1492a8a0bc Rephrase daemon set intro to fit even without the entropy spiel 2018-02-27 13:53:34 -06:00
Bridget Kromhout
388d616048 Attributing humor so it doesn't sound negative 2018-02-27 13:46:19 -06:00
Jerome Petazzoni
28589f5a83 Remove cluster-size specific reference 2018-02-27 13:40:52 -06:00
Jerome Petazzoni
e7a80f7bfb Merge branch 'master' of github.com:jpetazzo/container.training 2018-02-27 13:39:55 -06:00
Jerome Petazzoni
ea47e0ac05 Add link to brigade 2018-02-27 13:39:50 -06:00
Jérôme Petazzoni
09d204038f Merge pull request #116 from bridgetkromhout/versions-installed
Clarify that these are the installed versions
2018-02-27 13:36:40 -06:00
Jérôme Petazzoni
47cb0afac2 Merge pull request #115 from bridgetkromhout/any-cloud
More cloud-provider generic
2018-02-27 13:36:10 -06:00
Jerome Petazzoni
8e2e7f44d3 Break out 'scale things on a single node' section 2018-02-27 13:35:03 -06:00
Bridget Kromhout
8c7702deda Clarify that these are the installed versions
* "Brand new" is a moving target
2018-02-27 13:29:40 -06:00
Bridget Kromhout
bdc1ca01cd More cloud-provider generic 2018-02-27 13:27:11 -06:00
Jerome Petazzoni
dca58d6663 Merge Lucas awesome diagram 2018-02-27 12:22:02 -06:00
Jerome Petazzoni
a0cf4b97c0 Add Lucas' amazing diagram 2018-02-27 12:17:10 -06:00
Jerome Petazzoni
a1c239260f Add Lucas' amazing diagram 2018-02-27 12:17:02 -06:00
Jerome Petazzoni
a8a2cf54a5 Factor out links in separate files 2018-02-27 12:01:53 -06:00
Jerome Petazzoni
d5ba80da55 Replace 'five VMs' with 'a cluster of VMs' 2018-02-27 11:53:01 -06:00
Jerome Petazzoni
3f2da04763 CSS is hard but it's not an excuse 2018-02-27 09:44:32 -06:00
Jerome Petazzoni
e092f50645 Branch out intro/intro.md into per-workshop variants 2018-02-27 09:40:54 -06:00
Jérôme Petazzoni
7f698bd690 Merge pull request #114 from bridgetkromhout/master
Adding upcoming events
2018-02-27 09:28:27 -06:00
Bridget Kromhout
7fe04b9944 Adding upcoming events 2018-02-27 09:26:03 -06:00
Jerome Petazzoni
2671714df3 Move indexconf2018 to past workshops section 2018-02-27 09:11:09 -06:00
Jerome Petazzoni
630e275d99 Merge branch 'bridgetkromhout-master-updates' 2018-02-26 17:52:14 -06:00
Jerome Petazzoni
614f10432e Mostly reformatting so that slides are nice and tidy 2018-02-26 17:52:06 -06:00
Bridget Kromhout
223b5e152b Version updates 2018-02-26 16:56:45 -06:00
Bridget Kromhout
ec55cd2465 Including ACR as one of the cloud k8s offerings 2018-02-26 16:55:56 -06:00
Bridget Kromhout
c59510f921 Updates & clarifications 2018-02-26 16:54:41 -06:00
Bridget Kromhout
0f5f481213 Typo fix 2018-02-26 16:52:23 -06:00
Bridget Kromhout
b40fa45fd3 Clarifications 2018-02-26 16:50:31 -06:00
Bridget Kromhout
8faaf35da0 Clarify we didn't tag the v1 release 2018-02-26 16:48:52 -06:00
Bridget Kromhout
ce0f79af16 Updates & links for all cloud-provided k8s 2018-02-26 16:46:49 -06:00
Bridget Kromhout
faa420f9fd Clarify language and explain https use 2018-02-26 16:41:21 -06:00
Jerome Petazzoni
aab519177d Add indexconf2018 to index 2018-02-19 15:55:22 -08:00
Jerome Petazzoni
5116ad7c44 Use kubeadm token generate to simplify things a bit.
Thanks @rmb938 for the suggestion!

Closes #110.
2018-01-18 21:14:40 +01:00
Jerome Petazzoni
7305e911e5 Update for k8s 1.9 2018-01-10 17:12:49 +01:00
Jerome Petazzoni
b2f670acf6 Add error checking for AMI finder script 2018-01-10 16:48:04 +01:00
Jerome Petazzoni
dc040aa693 Make sleep interruptible; fix slide count 2017-12-23 19:38:32 +01:00
Jerome Petazzoni
9b7a8494b0 Fix logic to advance to next snippet 2017-12-23 18:49:42 +01:00
Jerome Petazzoni
ae6c1bb8eb Major UI refactor
Navigation now includes all slides and all snippets.
ENTER skips to the next snippet, or executes the
selected snippet.

More improvements to come: allow SPACE to navigate
step by step through slides and snippets, executing
the snippets.
2017-12-23 09:31:01 +01:00
Jerome Petazzoni
a9a4f0ea07 Create only one remote session 2017-12-23 05:06:30 +01:00
Jerome Petazzoni
68af5940e3 Script node3 setup as well 2017-12-23 05:00:22 +01:00
Jerome Petazzoni
9df5313da4 Remove spurious output from desktop integration 2017-12-22 23:15:21 +01:00
Jerome Petazzoni
ba3f00e64e Clear screen before showing UI 2017-12-22 23:11:11 +01:00
Jerome Petazzoni
4d7a6d5c70 Stupid typo 2017-12-22 23:03:48 +01:00
Jerome Petazzoni
aef833c3f5 Add pause before switching away from browser 2017-12-22 23:02:47 +01:00
Jerome Petazzoni
6f58fee29b Automatically open links in intro section 2017-12-22 22:54:08 +01:00
Jerome Petazzoni
dda09ddbcb slightly edit tmux commands 2017-12-22 22:49:53 +01:00
Jerome Petazzoni
8b13fe6eb4 fix formatting in PWD reference 2017-12-22 22:48:42 +01:00
Jerome Petazzoni
21f345a96a Improve open command 2017-12-22 22:42:41 +01:00
Jerome Petazzoni
eaa4dc63bf Instruct to use PWD in self-paced mode 2017-12-22 22:40:23 +01:00
Jerome Petazzoni
af5ea2188b More typos 2017-12-22 22:26:08 +01:00
Jerome Petazzoni
7f23a4c964 Fix minor typos 2017-12-22 22:24:37 +01:00
Jerome Petazzoni
345e04c956 Improve tmux detection logic and add instructions 2017-12-21 22:24:27 +01:00
Jerome Petazzoni
2a138102fc Add client address in pub/sub server 2017-12-21 05:53:24 +01:00
Jerome Petazzoni
ef5e8f00f8 Add script to remember myself of how to customize tmux status bar 2017-12-21 05:46:57 +01:00
Jerome Petazzoni
badb73a413 Slower pace for virtual typist 2017-12-21 05:45:38 +01:00
Jerome Petazzoni
2aced95c86 Improve UX for remote control 2017-12-21 05:40:35 +01:00
Jerome Petazzoni
720989e829 Add remote control of slide deck 2017-12-21 05:29:59 +01:00
Jerome Petazzoni
718031565e Exit gracefully if server is not running instead of waiting forever 2017-12-21 05:29:45 +01:00
Jerome Petazzoni
ec7b46b779 Add remote.js to workshop template and pub/sub server 2017-12-21 04:51:49 +01:00
Jerome Petazzoni
270c36b29a Add pub/sub server and CLI remote 2017-12-21 04:43:42 +01:00
Jerome Petazzoni
bc2eb53bb2 Python 3 compatibility 2017-12-21 04:33:35 +01:00
Jerome Petazzoni
afe7b8523c Move autotest to autopilot/ directory 2017-12-21 04:32:16 +01:00
Jérôme Petazzoni
a7743a4314 Update Engine version 2017-12-20 18:05:52 -06:00
Jérôme Petazzoni
ba74fdc841 Round of update for video content 2017-12-20 00:17:49 -06:00
Jérôme Petazzoni
41c047e12a Always start in interactive mode 2017-12-20 00:17:40 -06:00
Jérôme Petazzoni
f4fc055405 Add manifest for video content 2017-12-20 00:17:25 -06:00
Jérôme Petazzoni
2eb6fcfbf5 Add command to backtrack 1 slide 2017-12-18 18:46:24 -06:00
Jérôme Petazzoni
c665e1a2d6 httping only 3 requests is enough 2017-12-18 18:45:40 -06:00
Jérôme Petazzoni
bb7cdafe47 Comment out machine chapter 2017-12-18 18:39:33 -06:00
Jérôme Petazzoni
95fcfadb17 state.yml -> state.yaml to avoid collision with manifests 2017-12-18 18:39:17 -06:00
Jérôme Petazzoni
1ef47531c8 autotest: save all parameters in state.yml 2017-12-18 18:36:46 -06:00
Jérôme Petazzoni
9589b641b6 Another fix in CNC script 2017-12-18 18:35:57 -06:00
Jérôme Petazzoni
63463bda64 Merge branch 'master' of github.com:jpetazzo/container.training 2017-12-18 17:46:11 -06:00
Jérôme Petazzoni
b642412639 Clarify listen-addr and advertise-addr
Closes #108
2017-12-18 17:46:06 -06:00
Jérôme Petazzoni
21f9b73cb4 Update CNC deployment script + workshopctl deps 2017-12-18 17:45:29 -06:00
Jérôme Petazzoni
b73e5432f3 Merge pull request #109 from juliogomez/patch-6
curl is not installed in this step
2017-12-18 17:20:52 -06:00
Jérôme Petazzoni
de5cc9b0bf Merge branch 'master' of github.com:jpetazzo/container.training 2017-12-18 15:25:48 -06:00
Jérôme Petazzoni
08b38127d3 Clarify license for slides since they're not code 2017-12-18 15:25:41 -06:00
Julio
383804b7f1 curl is not installed in that step
curl was actually installed in a previous step, not here
2017-12-16 23:05:29 +01:00
Jérôme Petazzoni
20bf80910e Merge pull request #107 from juliogomez/patch-3
Fixing number of replicas per node
2017-12-16 09:26:18 -06:00
Jérôme Petazzoni
29a2014745 Merge pull request #106 from juliogomez/patch-2
Typo correction in detach mode?
2017-12-16 09:24:36 -06:00
Julio
40f6ee236f Fixing number of replicas per node
If 3 copies per node are 15, 4 copies per node should be 20.
2017-12-15 20:46:25 +01:00
Julio
5551cbd11f Typo correction in detach mode?
While you wrote:
`--detach=false` does not complete *faster*. It just *doesn't wait* for completion.
I think you actually meant:
`--detach=TRUE` does not complete *faster*. It just *doesn't wait* for completion.
2017-12-15 20:25:29 +01:00
Jérôme Petazzoni
9e84a05325 Merge pull request #105 from juliogomez/patch-1
Fixed missing image name (tomcat) in 'docker run' command
2017-12-14 15:33:24 -06:00
Julio
558e990907 Fixed missing image name in command 2017-12-14 22:15:29 +01:00
Jérôme Petazzoni
c2e88bb343 add qcon video 2017-12-07 14:28:46 -06:00
Jérôme Petazzoni
b7582397fe Add quote by not-Benjamin-Franklin 2017-12-04 18:19:04 -06:00
Jérôme Petazzoni
3e7b8615ab Move kube workshop to archives section 2017-12-04 13:57:17 -06:00
Jérôme Petazzoni
6f5d8c5372 Merge pull request #104 from gurayyildirim/patch-1
Kubernetes.io link fixed.
2017-11-24 11:31:07 -06:00
gurayyildirim
c116d75408 Kubernetes.io link fixed.
Kubernetes.io link had a wrong ']' mark which was causing a 404 from Kubernetes.io blog.
2017-11-24 02:45:49 +03:00
Jérôme Petazzoni
bb4ee4e77d Add helper script to setup CNC node 2017-11-20 17:04:38 -08:00
Jérôme Petazzoni
fc0e46988c Fix hint for ssh agent 2017-11-20 16:37:17 -08:00
Jérôme Petazzoni
c71b93c3a7 Add files to generate a CSV file with nodes 2017-11-20 16:35:25 -08:00
Jérôme Petazzoni
2c6b79c17d Add kube image to cards.html 2017-11-20 16:33:08 -08:00
Jérôme Petazzoni
f264838ec5 Update versions 2017-11-19 23:34:27 -08:00
Jérôme Petazzoni
54e7d10226 Add internal training 2017-11-19 17:45:24 -08:00
Jérôme Petazzoni
f8d3246005 Move QCON to 'done' section 2017-11-18 11:15:21 -08:00
Jérôme Petazzoni
7856e8c5f2 Put chat room in index 2017-11-16 23:32:12 -08:00
Jérôme Petazzoni
69bafe332f Update interstitials with self-hosted version 2017-11-16 23:29:37 -08:00
Jérôme Petazzoni
e2b24a20d2 Remove pixabay for now: 2017-11-16 08:46:47 -08:00
Jerome Petazzoni
d2d1771fd3 Better emoji support 2017-11-15 23:41:14 +01:00
Jérôme Petazzoni
6c5e3eb3f3 Update versions 2017-11-13 22:39:21 -08:00
Jérôme Petazzoni
d99dbd5878 Add first support to auto-open URLs 2017-11-12 22:47:07 -08:00
Jérôme Petazzoni
6d4894458a Add note about interstitials 2017-11-12 18:06:43 -08:00
Jérôme Petazzoni
bc367a1297 Simplify reference to PWD in intro slides 2017-11-12 17:45:15 -08:00
Jérôme Petazzoni
31d1074ee0 autotest: process single keypresses 2017-11-12 17:38:01 -08:00
Jérôme Petazzoni
cd18a87b8c Add self-paced kube manifest + small fixes in kube content 2017-11-12 17:37:40 -08:00
Jérôme Petazzoni
8279a3bce9 Add QCON slides 2017-11-10 23:01:17 -08:00
Jérôme Petazzoni
93524898cc Fix build script exit status 2017-11-10 22:57:06 -08:00
Jérôme Petazzoni
e9319060f6 Add pause images between chapters 2017-11-10 22:48:20 -08:00
Jérôme Petazzoni
4322478a4a A bunch of changes in there
- added self-paced manifest for intro workshop
- refactored intro content to work for all workshops
- fixed footnote CSS to work in all contexts
- intro+logistics content is good to go
2017-11-10 19:24:38 -08:00
Jérôme Petazzoni
00262c767e Move autotest.py to the slides/ directory 2017-11-10 13:31:00 -08:00
Jérôme Petazzoni
c8030e1500 Move autotest.py to the slides/ directory 2017-11-10 13:30:43 -08:00
Jérôme Petazzoni
3717b90444 Fix one long title 2017-11-10 13:08:19 -08:00
Jérôme Petazzoni
d003bdb765 Update --detach explanations 2017-11-10 13:06:52 -08:00
Jérôme Petazzoni
8b246ac334 Fix overflowing titles and slides in kube material 2017-11-10 12:55:13 -08:00
Jérôme Petazzoni
9df543a6bb Fix long titles and long slides in Swarm content 2017-11-09 19:58:55 -08:00
Jérôme Petazzoni
77befc1092 Improve appearance of code snippets 2017-11-09 19:58:35 -08:00
Jérôme Petazzoni
4417771315 Allow to automatically check slides + append check result at the end 2017-11-09 19:58:22 -08:00
Jérôme Petazzoni
04fa6ec1d8 Fix overflow in intro material 2017-11-09 16:43:56 -08:00
Jérôme Petazzoni
5e8bdcb1f6 Improve intro section 2017-11-09 14:27:30 -08:00
Jérôme Petazzoni
07d99763d3 Fix back-to-toc links 2017-11-09 13:41:29 -08:00
Jérôme Petazzoni
87a2051b24 Move one stray image 2017-11-09 13:36:21 -08:00
Jérôme Petazzoni
0eddc876f1 Add typist simulator 2017-11-08 23:24:43 -08:00
Jérôme Petazzoni
1a67a1397b Misc autotest improvements 2017-11-08 23:10:34 -08:00
Jérôme Petazzoni
dc6bf9d0cf Improve autotest by setting up tmux automatically 2017-11-08 23:10:25 -08:00
Jérôme Petazzoni
84f8bf007e Add spacer row 2017-11-08 21:28:22 -08:00
Jérôme Petazzoni
6a2a66c165 Improve local devel workflow; add notes about immutable infra 2017-11-08 18:49:40 -08:00
Jérôme Petazzoni
f491d559e3 Setting up an automated build 2017-11-08 14:32:04 -08:00
Jérôme Petazzoni
5d9bdc303c Fix title capitalization 2017-11-08 13:39:34 -08:00
Jérôme Petazzoni
f7f0ecddd4 Pushing to the hub 2017-11-08 13:31:22 -08:00
Jérôme Petazzoni
f0f03e2440 Rename other images 2017-11-08 11:04:51 -08:00
Jérôme Petazzoni
1f78264e9f Add helper script to rename assets 2017-11-08 10:41:56 -08:00
Jérôme Petazzoni
2aad5319f9 Rename title images for consistency 2017-11-08 10:41:46 -08:00
Jérôme Petazzoni
479b858398 Update use-cases and testimonials 2017-11-08 09:57:00 -08:00
Jérôme Petazzoni
34b8bfd1b2 Improve reporting for slide layout checker 2017-11-08 09:56:38 -08:00
Jérôme Petazzoni
408036b09c Implement checker for slides too high/too wide 2017-11-08 00:08:00 -08:00
Jérôme Petazzoni
a45ec1cb84 Reorg intro material a bit 2017-11-07 22:41:32 -08:00
Jérôme Petazzoni
8decf1852f Refactor build-tag-push
We now have two clean classes: btp-manual and btp-auto

Exclude one or the other depending on whether you want to
show the full process of building and pushing images, or
if you want to shortcut and use stack files directly.
2017-11-07 19:43:42 -08:00
Jérôme Petazzoni
350fac21f6 Add PyCon intro to docker 2017-11-07 18:41:48 -08:00
Jérôme Petazzoni
cedd386eee Add LISA16 recording 2017-11-07 18:29:08 -08:00
Jérôme Petazzoni
5751765d66 MOAR materials; smaller footer 2017-11-07 18:09:38 -08:00
Jérôme Petazzoni
7293c071bd CSS is hard, yo 2017-11-07 17:46:56 -08:00
Jérôme Petazzoni
98cf8f4f04 Rename manifests 2017-11-07 14:48:20 -08:00
Jérôme Petazzoni
c61ccd1c27 Uniformize selfpaced/halfday/fullday 2017-11-07 11:30:36 -08:00
Jérôme Petazzoni
6b3d0efa56 split out the 'declarative/imperative' explanation 2017-11-07 11:23:53 -08:00
Jérôme Petazzoni
164578f1c8 Images should be good now 2017-11-06 23:26:39 -08:00
Jérôme Petazzoni
938fe956cf Better closing links 2017-11-06 22:48:21 -08:00
Jérôme Petazzoni
b44d402c1d Tweak warning image 2017-11-06 22:43:50 -08:00
Jérôme Petazzoni
fbaa511813 add colophon 2017-11-06 22:17:54 -08:00
Jérôme Petazzoni
f000594c62 Refactored CSS 2017-11-06 22:15:12 -08:00
Jérôme Petazzoni
71ba94063e Add explanations when using Docker Machine or Toolbox
Thanks @jgarrouste for reminding me of this issue :-)
2017-11-06 17:05:34 -08:00
Jérôme Petazzoni
b025a8c966 add application metrics 2017-11-06 16:47:32 -08:00
Jérôme Petazzoni
c36aab132b Add prev/next navigation links + fix TOC backlinks 2017-11-06 16:39:02 -08:00
Jérôme Petazzoni
1e7a47ed37 Factor conclusion 2017-11-06 11:40:30 -08:00
Jérôme Petazzoni
87f544774e Factor pre-requirements 2017-11-06 11:26:16 -08:00
Jérôme Petazzoni
db9ee5f03a Factor title, toc, logistics, thankyou slides 2017-11-05 12:41:30 -08:00
Jérôme Petazzoni
d0f5d69157 Add some extra debug info in first slide 2017-11-05 11:52:27 -08:00
Jérôme Petazzoni
742c7a78bc Debug bar now shows manifest file 2017-11-05 10:22:44 -08:00
Jérôme Petazzoni
918d3a6c23 orchestration-workshop -> container.training (in slides) 2017-11-05 10:16:01 -08:00
Jérôme Petazzoni
1f9b304eec Update homepage 2017-11-05 10:08:19 -08:00
175 changed files with 5439 additions and 3212 deletions

3
.gitignore vendored
View File

@@ -7,4 +7,5 @@ prepare-vms/ips.pdf
prepare-vms/settings.yaml
prepare-vms/tags
slides/*.yml.html
autotest/nextstep
slides/autopilot/state.yaml
node_modules

19
CHECKLIST.md Normal file
View File

@@ -0,0 +1,19 @@
This is the checklist that I (Jérôme) use when delivering a workshop.
- [ ] Create branch + `_redirects` + push to GitHub + Netlify setup
- [ ] Add branch to index.html
- [ ] Update the slides that says which versions we are using
- [ ] Update the version of Compose and Machine in settings
- [ ] Create chatroom
- [ ] Set chatroom in YML and deploy
- [ ] Put chat room in index.html
- [ ] Walk the room to count seats, check power supplies, lectern, A/V setup
- [ ] How many VMs do we need?
- [ ] Provision VMs
- [ ] Print cards
- [ ] Cut cards
- [ ] Last minute merge from master
- [ ] Check that all looks good
- [ ] DELIVER!
- [ ] Shutdown VMs
- [ ] Update index.html to remove chat link and move session to past things

19
LICENSE
View File

@@ -1,13 +1,12 @@
Copyright 2015 Jérôme Petazzoni
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
The code in this repository is licensed under the Apache License
Version 2.0. You may obtain a copy of this license at:
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
The instructions and slides in this repository (e.g. the files
with extension .md and .yml in the "slides" subdirectory) are
under the Creative Commons Attribution 4.0 International Public
License. You may obtain a copy of this license at:
https://creativecommons.org/licenses/by/4.0/legalcode

View File

@@ -39,14 +39,16 @@ your own tutorials.
All these materials have been gathered in a single repository
because they have a few things in common:
- a [build system](slides/) generating HTML slides from
Markdown source files;
- some [common slides](slides/common/) that are re-used
(and updated) identically between different decks;
- a [build system](slides/) generating HTML slides from
Markdown source files;
- a [semi-automated test harness](slides/autopilot/) to check
that the exercises and examples provided work properly;
- a [PhantomJS script](slides/slidechecker.js) to check
that the slides look good and don't have formatting issues;
- [deployment scripts](prepare-vms/) to start training
VMs in bulk;
- a [semi-automated test harness](autotest/) to check
that the exercises and examples provided work properly;
- a fancy pipeline powered by
[Netlify](https://www.netlify.com/) and continuously
deploying `master` to http://container.training/.
@@ -74,9 +76,6 @@ a few other contributors. It is actively maintained.
## Repository structure
- [autotest](autotest/)
- Semi-automated testing system to check that all the exercises
in the slides work properly.
- [bin](bin/)
- A few helper scripts that you can safely ignore for now.
- [dockercoins](dockercoins/)
@@ -248,6 +247,17 @@ content but you also know to skip during presentation.
- Last 15-30 minutes is for stateful services, DAB files, and questions.
### Pre-built images
There are pre-built images for the 4 components of the DockerCoins demo app: `dockercoins/hasher:v0.1`, `dockercoins/rng:v0.1`, `dockercoins/webui:v0.1`, and `dockercoins/worker:v0.1`. They correspond to the code in this repository.
There are also three variants, for demo purposes:
- `dockercoins/rng:v0.2` is broken (the server won't even start),
- `dockercoins/webui:v0.2` has bigger font on the Y axis and a green graph (instead of blue),
- `dockercoins/worker:v0.2` is 11x slower than `v0.1`.
## Past events
Since its inception, this workshop has been delivered dozens of times,

View File

@@ -1,229 +0,0 @@
#!/usr/bin/env python
import uuid
import logging
import os
import re
import subprocess
import sys
import time
import uuid
logging.basicConfig(level=logging.DEBUG)
TIMEOUT = 60 # 1 minute
def hrule():
return "="*int(subprocess.check_output(["tput", "cols"]))
# A "snippet" is something that the user is supposed to do in the workshop.
# Most of the "snippets" are shell commands.
# Some of them can be key strokes or other actions.
# In the markdown source, they are the code sections (identified by triple-
# quotes) within .exercise[] sections.
class Snippet(object):
def __init__(self, slide, content):
self.slide = slide
self.content = content
self.actions = []
def __str__(self):
return self.content
class Slide(object):
current_slide = 0
def __init__(self, content):
Slide.current_slide += 1
self.number = Slide.current_slide
# Remove commented-out slides
# (remark.js considers ??? to be the separator for speaker notes)
content = re.split("\n\?\?\?\n", content)[0]
self.content = content
self.snippets = []
exercises = re.findall("\.exercise\[(.*)\]", content, re.DOTALL)
for exercise in exercises:
if "```" in exercise:
for snippet in exercise.split("```")[1::2]:
self.snippets.append(Snippet(self, snippet))
else:
logging.warning("Exercise on slide {} does not have any ``` snippet."
.format(self.number))
self.debug()
def __str__(self):
text = self.content
for snippet in self.snippets:
text = text.replace(snippet.content, ansi("7")(snippet.content))
return text
def debug(self):
logging.debug("\n{}\n{}\n{}".format(hrule(), self.content, hrule()))
def ansi(code):
return lambda s: "\x1b[{}m{}\x1b[0m".format(code, s)
def wait_for_string(s):
logging.debug("Waiting for string: {}".format(s))
deadline = time.time() + TIMEOUT
while time.time() < deadline:
output = capture_pane()
if s in output:
return
time.sleep(1)
raise Exception("Timed out while waiting for {}!".format(s))
def wait_for_prompt():
logging.debug("Waiting for prompt.")
deadline = time.time() + TIMEOUT
while time.time() < deadline:
output = capture_pane()
# If we are not at the bottom of the screen, there will be a bunch of extra \n's
output = output.rstrip('\n')
if output[-2:] == "\n$":
return
time.sleep(1)
raise Exception("Timed out while waiting for prompt!")
def check_exit_status():
token = uuid.uuid4().hex
data = "echo {} $?\n".format(token)
logging.debug("Sending {!r} to get exit status.".format(data))
send_keys(data)
time.sleep(0.5)
wait_for_prompt()
screen = capture_pane()
status = re.findall("\n{} ([0-9]+)\n".format(token), screen, re.MULTILINE)
logging.debug("Got exit status: {}.".format(status))
if len(status) == 0:
raise Exception("Couldn't retrieve status code {}. Timed out?".format(token))
if len(status) > 1:
raise Exception("More than one status code {}. I'm seeing double! Shoot them both.".format(token))
code = int(status[0])
if code != 0:
raise Exception("Non-zero exit status: {}.".format(code))
# Otherwise just return peacefully.
slides = []
content = open(sys.argv[1]).read()
for slide in re.split("\n---?\n", content):
slides.append(Slide(slide))
actions = []
for slide in slides:
for snippet in slide.snippets:
content = snippet.content
# Extract the "method" (e.g. bash, keys, ...)
# On multi-line snippets, the method is alone on the first line
# On single-line snippets, the data follows the method immediately
if '\n' in content:
method, data = content.split('\n', 1)
else:
method, data = content.split(' ', 1)
actions.append((slide, snippet, method, data))
def send_keys(data):
subprocess.check_call(["tmux", "send-keys", data])
def capture_pane():
return subprocess.check_output(["tmux", "capture-pane", "-p"])
try:
i = int(open("nextstep").read())
logging.info("Loaded next step ({}) from file.".format(i))
except Exception as e:
logging.warning("Could not read nextstep file ({}), initializing to 0.".format(e))
i = 0
interactive = True
while i < len(actions):
with open("nextstep", "w") as f:
f.write(str(i))
slide, snippet, method, data = actions[i]
# Remove extra spaces (we don't want them in the terminal) and carriage returns
data = data.strip()
print(hrule())
print(slide.content.replace(snippet.content, ansi(7)(snippet.content)))
print(hrule())
if interactive:
print("[{}/{}] Shall we execute that snippet above?".format(i, len(actions)))
print("(ENTER to execute, 'c' to continue until next error, N to jump to step #N)")
command = raw_input("> ")
else:
command = ""
# For now, remove the `highlighted` sections
# (Make sure to use $() in shell snippets!)
if '`' in data:
logging.info("Stripping ` from snippet.")
data = data.replace('`', '')
if command == "c":
# continue until next timeout
interactive = False
elif command.isdigit():
i = int(command)
elif command == "":
logging.info("Running with method {}: {}".format(method, data))
if method == "keys":
send_keys(data)
elif method == "bash":
# Make sure that we're ready
wait_for_prompt()
# Strip leading spaces
data = re.sub("\n +", "\n", data)
# Add "RETURN" at the end of the command :)
data += "\n"
# Send command
send_keys(data)
# Force a short sleep to avoid race condition
time.sleep(0.5)
_, _, next_method, next_data = actions[i+1]
if next_method == "wait":
wait_for_string(next_data)
else:
wait_for_prompt()
# Verify return code FIXME should be optional
check_exit_status()
elif method == "copypaste":
screen = capture_pane()
matches = re.findall(data, screen, flags=re.DOTALL)
if len(matches) == 0:
raise Exception("Could not find regex {} in output.".format(data))
# Arbitrarily get the most recent match
match = matches[-1]
# Remove line breaks (like a screen copy paste would do)
match = match.replace('\n', '')
send_keys(match + '\n')
# FIXME: we should factor out the "bash" method
wait_for_prompt()
check_exit_status()
else:
logging.warning("Unknown method {}: {!r}".format(method, data))
i += 1
else:
i += 1
logging.warning("Unknown command {}, skipping to next step.".format(command))
# Reset slide counter
with open("nextstep", "w") as f:
f.write(str(0))

View File

@@ -4,6 +4,12 @@
- [Docker](https://docs.docker.com/engine/installation/)
- [Docker Compose](https://docs.docker.com/compose/install/)
- [Parallel SSH](https://code.google.com/archive/p/parallel-ssh/) (on a Mac: `brew install pssh`) - the configuration scripts require this
And if you want to generate printable cards:
- [pyyaml](https://pypi.python.org/pypi/PyYAML) (on a Mac: `brew install pyyaml`)
- [jinja2](https://pypi.python.org/pypi/Jinja2) (on a Mac: `brew install jinja2`)
## General Workflow
@@ -35,6 +41,16 @@ The Docker Compose file here is used to build a image with all the dependencies
- `AWS_SECRET_ACCESS_KEY`
- `AWS_DEFAULT_REGION`
If you're not using AWS, set these to placeholder values:
```
export AWS_ACCESS_KEY_ID="foo"
export AWS_SECRET_ACCESS_KEY="foo"
export AWS_DEFAULT_REGION="foo"
```
If you don't have the `aws` CLI installed, you will get a warning that it's a missing dependency. If you're not using AWS you can ignore this.
### Update/copy `settings/example.yaml`
Then pass `settings/YOUR_WORKSHOP_NAME-settings.yaml` as an argument to `./workshopctl deploy`, `./workshopctl cards`, etc.
@@ -48,6 +64,7 @@ workshopctl - the orchestration workshop swiss army knife
Commands:
ami Show the AMI that will be used for deployment
amis List Ubuntu AMIs in the current region
build Build the Docker image to run this program in a container
cards Generate ready-to-print cards for a batch of VMs
deploy Install Docker on a bunch of running VMs
ec2quotas Check our EC2 quotas (max instances)
@@ -55,6 +72,7 @@ help Show available commands
ids List the instance IDs belonging to a given tag or token
ips List the IP addresses of the VMs for a given tag or token
kube Setup kubernetes clusters with kubeadm (must be run AFTER deploy)
kubetest Check that all notes are reporting as Ready
list List available batches in the current region
opensg Open the default security group to ALL ingress traffic
pull_images Pre-pull a bunch of Docker images
@@ -63,6 +81,7 @@ start Start a batch of VMs
status List instance status for a given batch
stop Stop (terminate, shutdown, kill, remove, destroy...) instances
test Run tests (pre-flight checks) on a batch of VMs
wrap Run this program in a container
```
### Summary of What `./workshopctl` Does For You
@@ -75,12 +94,12 @@ test Run tests (pre-flight checks) on a batch of VMs
- During `start` it will add your default local SSH key to all instances under the `ubuntu` user.
- During `deploy` it will create the `docker` user with password `training`, which is printing on the cards for students. For now, this is hard coded.
### Example Steps to Launch a Batch of Instances for a Workshop
### Example Steps to Launch a Batch of AWS Instances for a Workshop
- Run `./workshopctl start N` Creates `N` EC2 instances
- Your local SSH key will be synced to instances under `ubuntu` user
- AWS instances will be created and tagged based on date, and IP's stored in `prepare-vms/tags/`
- Run `./workshopctl deploy TAG settings/somefile.yaml` to run `scripts/postprep.rc` via parallel-ssh
- Run `./workshopctl deploy TAG settings/somefile.yaml` to run `lib/postprep.py` via parallel-ssh
- If it errors or times out, you should be able to rerun
- Requires good connection to run all the parallel SSH connections, up to 100 parallel (ProTip: create dedicated management instance in same AWS region where you run all these utils from)
- Run `./workshopctl pull-images TAG` to pre-pull a bunch of Docker images to the instances
@@ -88,6 +107,67 @@ test Run tests (pre-flight checks) on a batch of VMs
- *Have a great workshop*
- Run `./workshopctl stop TAG` to terminate instances.
### Example Steps to Launch Azure Instances
- Install the [Azure CLI](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest) and authenticate with a valid account
- Customize `azuredeploy.parameters.json`
- Required:
- Provide the SSH public key you plan to use for instance configuration
- Optional:
- Choose a name for the workshop (default is "workshop")
- Choose the number of instances (default is 3)
- Customize the desired instance size (default is Standard_D1_v2)
- Launch instances with your chosen resource group name and your preferred region; the examples are "workshop" and "eastus":
```
az group create --name workshop --location eastus
az group deployment create --resource-group workshop --template-file azuredeploy.json --parameters @azuredeploy.parameters.json
```
The `az group deployment create` command can take several minutes and will only say `- Running ..` until it completes, unless you increase the verbosity with `--verbose` or `--debug`.
To display the IPs of the instances you've launched:
```
az vm list-ip-addresses --resource-group workshop --output table
```
If you want to put the IPs into `prepare-vms/tags/<tag>/ips.txt` for a tag of "myworkshop":
1) If you haven't yet installed `jq` and/or created your event's tags directory in `prepare-vms`:
```
brew install jq
mkdir -p tags/myworkshop
```
2) And then generate the IP list:
```
az vm list-ip-addresses --resource-group workshop --output json | jq -r '.[].virtualMachine.network.publicIpAddresses[].ipAddress' > tags/myworkshop/ips.txt
```
After the workshop is over, remove the instances:
```
az group delete --resource-group workshop
```
### Example Steps to Configure Instances from a non-AWS Source
- Launch instances via your preferred method. You'll need to get the instance IPs and be able to ssh into them.
- Set placeholder values for [AWS environment variable settings](#required-environment-variables).
- Choose a tag. It could be an event name, datestamp, etc. Ensure you have created a directory for your tag: `prepare-vms/tags/<tag>/`
- If you have not already generated a file with the IPs to be configured:
- The file should be named `prepare-vms/tags/<tag>/ips.txt`
- Format is one IP per line, no other info needed.
- Ensure the settings file is as desired (especially the number of nodes): `prepare-vms/settings/kube101.yaml`
- For a tag called `myworkshop`, configure instances: `workshopctl deploy myworkshop settings/kube101.yaml`
- Optionally, configure Kubernetes clusters of the size in the settings: `workshopctl kube myworkshop`
- Optionally, test your Kubernetes clusters. They may take a little time to become ready: `workshopctl kubetest myworkshop`
- Generate cards to print and hand out: `workshopctl cards myworkshop settings/kube101.yaml`
- Print the cards file: `prepare-vms/tags/myworkshop/ips.html`
## Other Tools
### Deploying your SSH key to all the machines
@@ -97,13 +177,6 @@ test Run tests (pre-flight checks) on a batch of VMs
- Run `pcopykey`.
### Installing extra packages
- Source `postprep.rc`.
(This will install a few extra packages, add entries to
/etc/hosts, generate SSH keys, and deploy them on all hosts.)
## Even More Details
#### Sync of SSH keys
@@ -132,7 +205,7 @@ Instances can be deployed manually using the `deploy` command:
$ ./workshopctl deploy TAG settings/somefile.yaml
The `postprep.rc` file will be copied via parallel-ssh to all of the VMs and executed.
The `postprep.py` file will be copied via parallel-ssh to all of the VMs and executed.
#### Pre-pull images
@@ -142,6 +215,10 @@ The `postprep.rc` file will be copied via parallel-ssh to all of the VMs and exe
$ ./workshopctl cards TAG settings/somefile.yaml
If you want to generate both HTML and PDF cards, install [wkhtmltopdf](https://wkhtmltopdf.org/downloads.html); without that installed, only HTML cards will be generated.
If you don't have `wkhtmltopdf` installed, you will get a warning that it is a missing dependency. If you plan to just print the HTML cards, you can ignore this.
#### List tags
$ ./workshopctl list

View File

@@ -0,0 +1,250 @@
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"workshopName": {
"type": "string",
"defaultValue": "workshop",
"metadata": {
"description": "Workshop name."
}
},
"vmPrefix": {
"type": "string",
"defaultValue": "node",
"metadata": {
"description": "Prefix for VM names."
}
},
"numberOfInstances": {
"type": "int",
"defaultValue": 3,
"metadata": {
"description": "Number of VMs to create."
}
},
"adminUsername": {
"type": "string",
"defaultValue": "ubuntu",
"metadata": {
"description": "Admin username for VMs."
}
},
"sshKeyData": {
"type": "string",
"defaultValue": "",
"metadata": {
"description": "SSH rsa public key file as a string."
}
},
"imagePublisher": {
"type": "string",
"defaultValue": "Canonical",
"metadata": {
"description": "OS image publisher; default Canonical."
}
},
"imageOffer": {
"type": "string",
"defaultValue": "UbuntuServer",
"metadata": {
"description": "The name of the image offer. The default is Ubuntu"
}
},
"imageSKU": {
"type": "string",
"defaultValue": "16.04-LTS",
"metadata": {
"description": "Version of the image. The default is 16.04-LTS"
}
},
"vmSize": {
"type": "string",
"defaultValue": "Standard_D1_v2",
"metadata": {
"description": "VM Size."
}
}
},
"variables": {
"vnetID": "[resourceId('Microsoft.Network/virtualNetworks',variables('virtualNetworkName'))]",
"subnet1Ref": "[concat(variables('vnetID'),'/subnets/',variables('subnet1Name'))]",
"vmName": "[parameters('vmPrefix')]",
"sshKeyPath": "[concat('/home/',parameters('adminUsername'),'/.ssh/authorized_keys')]",
"publicIPAddressName": "PublicIP",
"publicIPAddressType": "Dynamic",
"virtualNetworkName": "MyVNET",
"netSecurityGroup": "MyNSG",
"addressPrefix": "10.0.0.0/16",
"subnet1Name": "subnet-1",
"subnet1Prefix": "10.0.0.0/24",
"nicName": "myVMNic"
},
"resources": [
{
"apiVersion": "2017-11-01",
"type": "Microsoft.Network/publicIPAddresses",
"name": "[concat(variables('publicIPAddressName'),copyIndex(1))]",
"location": "[resourceGroup().location]",
"copy": {
"name": "publicIPLoop",
"count": "[parameters('numberOfInstances')]"
},
"properties": {
"publicIPAllocationMethod": "[variables('publicIPAddressType')]"
},
"tags": {
"workshop": "[parameters('workshopName')]"
}
},
{
"apiVersion": "2017-11-01",
"type": "Microsoft.Network/virtualNetworks",
"name": "[variables('virtualNetworkName')]",
"location": "[resourceGroup().location]",
"dependsOn": [
"[concat('Microsoft.Network/networkSecurityGroups/', variables('netSecurityGroup'))]"
],
"properties": {
"addressSpace": {
"addressPrefixes": [
"[variables('addressPrefix')]"
]
},
"subnets": [
{
"name": "[variables('subnet1Name')]",
"properties": {
"addressPrefix": "[variables('subnet1Prefix')]",
"networkSecurityGroup": {
"id": "[resourceId('Microsoft.Network/networkSecurityGroups', variables('netSecurityGroup'))]"
}
}
}
]
},
"tags": {
"workshop": "[parameters('workshopName')]"
}
},
{
"apiVersion": "2017-11-01",
"type": "Microsoft.Network/networkInterfaces",
"name": "[concat(variables('nicName'),copyIndex(1))]",
"location": "[resourceGroup().location]",
"copy": {
"name": "nicLoop",
"count": "[parameters('numberOfInstances')]"
},
"dependsOn": [
"[concat('Microsoft.Network/publicIPAddresses/', variables('publicIPAddressName'),copyIndex(1))]",
"[concat('Microsoft.Network/virtualNetworks/', variables('virtualNetworkName'))]"
],
"properties": {
"ipConfigurations": [
{
"name": "ipconfig1",
"properties": {
"privateIPAllocationMethod": "Dynamic",
"publicIPAddress": {
"id": "[resourceId('Microsoft.Network/publicIPAddresses', concat(variables('publicIPAddressName'), copyIndex(1)))]"
},
"subnet": {
"id": "[variables('subnet1Ref')]"
}
}
}
]
},
"tags": {
"workshop": "[parameters('workshopName')]"
}
},
{
"apiVersion": "2017-12-01",
"type": "Microsoft.Compute/virtualMachines",
"name": "[concat(variables('vmName'),copyIndex(1))]",
"location": "[resourceGroup().location]",
"copy": {
"name": "vmLoop",
"count": "[parameters('numberOfInstances')]"
},
"dependsOn": [
"[concat('Microsoft.Network/networkInterfaces/', variables('nicName'), copyIndex(1))]"
],
"properties": {
"hardwareProfile": {
"vmSize": "[parameters('vmSize')]"
},
"osProfile": {
"computerName": "[concat(variables('vmName'),copyIndex(1))]",
"adminUsername": "[parameters('adminUsername')]",
"linuxConfiguration": {
"disablePasswordAuthentication": true,
"ssh": {
"publicKeys": [
{
"path": "[variables('sshKeyPath')]",
"keyData": "[parameters('sshKeyData')]"
}
]
}
}
},
"storageProfile": {
"osDisk": {
"createOption": "FromImage"
},
"imageReference": {
"publisher": "[parameters('imagePublisher')]",
"offer": "[parameters('imageOffer')]",
"sku": "[parameters('imageSKU')]",
"version": "latest"
}
},
"networkProfile": {
"networkInterfaces": [
{
"id": "[resourceId('Microsoft.Network/networkInterfaces', concat(variables('nicName'),copyIndex(1)))]"
}
]
}
},
"tags": {
"workshop": "[parameters('workshopName')]"
}
},
{
"apiVersion": "2017-11-01",
"type": "Microsoft.Network/networkSecurityGroups",
"name": "[variables('netSecurityGroup')]",
"location": "[resourceGroup().location]",
"tags": {
"workshop": "[parameters('workshopName')]"
},
"properties": {
"securityRules": [
{
"name": "default-open-ports",
"properties": {
"protocol": "Tcp",
"sourcePortRange": "*",
"destinationPortRange": "*",
"sourceAddressPrefix": "*",
"destinationAddressPrefix": "*",
"access": "Allow",
"priority": 1000,
"direction": "Inbound"
}
}
]
}
}
],
"outputs": {
"resourceID": {
"type": "string",
"value": "[resourceId('Microsoft.Network/publicIPAddresses', concat(variables('publicIPAddressName'),'1'))]"
}
}
}

View File

@@ -0,0 +1,18 @@
{
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"sshKeyData": {
"value": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXTIl/M9oeSlcsC5Rfe+nZr4Jc4sl200pSw2lpdxlZ3xzeP15NgSSMJnigUrKUXHfqRQ+2wiPxEf0Odz2GdvmXvR0xodayoOQsO24AoERjeSBXCwqITsfp1bGKzMb30/3ojRBo6LBR6r1+lzJYnNCGkT+IQwLzRIpm0LCNz1j08PUI2aZ04+mcDANvHuN/hwi/THbLLp6SNWN43m9r02RcC6xlCNEhJi4wk4VzMzVbSv9RlLGST2ocbUHwmQ2k9OUmpzoOx73aQi9XNnEaFh2w/eIdXM75VtkT3mRryyykg9y0/hH8/MVmIuRIdzxHQqlm++DLXVH5Ctw6a4kS+ki7 workshop"
},
"workshopName": {
"value": "workshop"
},
"numberOfInstances": {
"value": 3
},
"vmSize": {
"value": "Standard_D1_v2"
}
}
}

View File

@@ -12,7 +12,9 @@
{%- set cluster_or_machine = "cluster" -%}
{%- set this_or_each = "each" -%}
{%- set machine_is_or_machines_are = "machines are" -%}
{%- set image_src = "https://cdn.wp.nginx.com/wp-content/uploads/2016/07/docker-swarm-hero2.png" -%}
{%- set image_src_swarm = "https://cdn.wp.nginx.com/wp-content/uploads/2016/07/docker-swarm-hero2.png" -%}
{%- set image_src_kube = "https://avatars1.githubusercontent.com/u/13629408" -%}
{%- set image_src = image_src_swarm -%}
{%- endif -%}
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>

5
prepare-vms/clusters.csv Normal file
View File

@@ -0,0 +1,5 @@
Put your initials in the first column to "claim" a cluster.
Initials{% for node in clusters[0] %} node{{ loop.index }}{% endfor %}
{% for cluster in clusters -%}
{%- for node in cluster %} {{ node|trim }}{% endfor %}
{% endfor %}
Can't render this file because it contains an unexpected character in line 1 and column 42.

21
prepare-vms/cncsetup.sh Normal file
View File

@@ -0,0 +1,21 @@
#!/bin/sh
if [ $(whoami) != ubuntu ]; then
echo "This script should be executed on a freshly deployed node,"
echo "with the 'ubuntu' user. Aborting."
exit 1
fi
if id docker; then
sudo userdel -r docker
fi
pip install --user awscli jinja2 pdfkit
sudo apt-get install -y wkhtmltopdf xvfb
tmux new-session \; send-keys "
[ -f ~/.ssh/id_rsa ] || ssh-keygen
eval \$(ssh-agent)
ssh-add
Xvfb :0 &
export DISPLAY=:0
mkdir -p ~/www
sudo docker run -d -p 80:80 -v \$HOME/www:/usr/share/nginx/html nginx
"

View File

@@ -15,5 +15,6 @@ services:
AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
AWS_DEFAULT_REGION: ${AWS_DEFAULT_REGION}
AWS_INSTANCE_TYPE: ${AWS_INSTANCE_TYPE}
USER: ${USER}
entrypoint: /root/prepare-vms/workshopctl

View File

@@ -2,7 +2,7 @@
_ERR() {
error "Command $BASH_COMMAND failed (exit status: $?)"
}
set -e
set -eE
trap _ERR ERR
die() {

View File

@@ -39,7 +39,10 @@ _cmd_cards() {
need_tag $TAG
need_settings $SETTINGS
aws_get_instance_ips_by_tag $TAG >tags/$TAG/ips.txt
# If you're not using AWS, populate the ips.txt file manually
if [ ! -f tags/$TAG/ips.txt ]; then
aws_get_instance_ips_by_tag $TAG >tags/$TAG/ips.txt
fi
# Remove symlinks to old cards
rm -f ips.html ips.pdf
@@ -124,7 +127,7 @@ _cmd kube "Setup kubernetes clusters with kubeadm (must be run AFTER deploy)"
_cmd_kube() {
# Install packages
pssh "
pssh --timeout 200 "
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg |
sudo apt-key add - &&
echo deb http://apt.kubernetes.io/ kubernetes-xenial main |
@@ -134,17 +137,11 @@ _cmd_kube() {
sudo apt-get install -qy kubelet kubeadm kubectl
kubectl completion bash | sudo tee /etc/bash_completion.d/kubectl"
# Work around https://github.com/kubernetes/kubernetes/issues/53356
pssh "
if [ ! -f /etc/kubernetes/kubelet.conf ]; then
sudo systemctl stop kubelet
sudo rm -rf /var/lib/kubelet/pki
fi"
# Initialize kube master
pssh "
pssh --timeout 200 "
if grep -q node1 /tmp/node && [ ! -f /etc/kubernetes/admin.conf ]; then
sudo kubeadm init
kubeadm token generate > /tmp/token
sudo kubeadm init --token \$(cat /tmp/token)
fi"
# Put kubeconfig in ubuntu's and docker's accounts
@@ -157,15 +154,6 @@ _cmd_kube() {
sudo chown -R docker /home/docker/.kube
fi"
# Get bootstrap token
pssh "
if grep -q node1 /tmp/node; then
TOKEN_NAME=\$(kubectl -n kube-system get secret -o name | grep bootstrap-token)
TOKEN_ID=\$(kubectl -n kube-system get \$TOKEN_NAME -o go-template --template '{{ index .data \"token-id\" }}' | base64 -d)
TOKEN_SECRET=\$(kubectl -n kube-system get \$TOKEN_NAME -o go-template --template '{{ index .data \"token-secret\" }}' | base64 -d)
echo \$TOKEN_ID.\$TOKEN_SECRET >/tmp/token
fi"
# Install weave as the pod network
pssh "
if grep -q node1 /tmp/node; then
@@ -174,15 +162,28 @@ _cmd_kube() {
fi"
# Join the other nodes to the cluster
pssh "
pssh --timeout 200 "
if ! grep -q node1 /tmp/node && [ ! -f /etc/kubernetes/kubelet.conf ]; then
TOKEN=\$(ssh -o StrictHostKeyChecking=no node1 cat /tmp/token)
sudo kubeadm join --token \$TOKEN node1:6443
sudo kubeadm join --discovery-token-unsafe-skip-ca-verification --token \$TOKEN node1:6443
fi"
sep "Done"
}
_cmd kubetest "Check that all notes are reporting as Ready"
_cmd_kubetest() {
# There are way too many backslashes in the command below.
# Feel free to make that better ♥
pssh "
set -e
if grep -q node1 /tmp/node; then
for NODE in \$(awk /\ node/\ {print\ \\\$2} /etc/hosts); do
echo \$NODE ; kubectl get nodes | grep -w \$NODE | grep -w Ready
done
fi"
}
_cmd ids "List the instance IDs belonging to a given tag or token"
_cmd_ids() {
TAG=$1
@@ -280,6 +281,9 @@ _cmd_start() {
key_name=$(sync_keys)
AMI=$(_cmd_ami) # Retrieve the AWS image ID
if [ -z "$AMI" ]; then
die "I could not find which AMI to use in this region. Try another region?"
fi
TOKEN=$(get_token) # generate a timestamp token for this batch of VMs
AWS_KEY_NAME=$(make_key_name)
@@ -292,7 +296,7 @@ _cmd_start() {
result=$(aws ec2 run-instances \
--key-name $AWS_KEY_NAME \
--count $COUNT \
--instance-type t2.medium \
--instance-type ${AWS_INSTANCE_TYPE-t2.medium} \
--client-token $TOKEN \
--image-id $AMI)
reservation_id=$(echo "$result" | head -1 | awk '{print $2}')
@@ -430,6 +434,7 @@ tag_is_reachable() {
}
test_tag() {
TAG=$1
ips_file=tags/$TAG/ips.txt
info "Picking a random IP address in $ips_file to run tests."
n=$((1 + $RANDOM % $(wc -l <$ips_file)))

View File

@@ -0,0 +1,5 @@
# Number of VMs per cluster
clustersize: 5
# Jinja2 template to use to generate ready-to-cut cards
cards_template: clusters.csv

View File

@@ -0,0 +1,24 @@
# customize your cluster size, your cards template, and the versions
# Number of VMs per cluster
clustersize: 5
# Jinja2 template to use to generate ready-to-cut cards
cards_template: cards.html
# Use "Letter" in the US, and "A4" everywhere else
paper_size: Letter
# Feel free to reduce this if your printer can handle it
paper_margin: 0.2in
# Note: paper_size and paper_margin only apply to PDF generated with pdfkit.
# If you print (or generate a PDF) using ips.html, they will be ignored.
# (The equivalent parameters must be set from the browser's print dialog.)
# This can be "test" or "stable"
engine_version: test
# These correspond to the version numbers visible on their respective GitHub release pages
compose_version: 1.18.0
machine_version: 0.13.0

View File

@@ -20,5 +20,5 @@ paper_margin: 0.2in
engine_version: test
# These correspond to the version numbers visible on their respective GitHub release pages
compose_version: 1.16.1
machine_version: 0.12.0
compose_version: 1.17.1
machine_version: 0.13.0

View File

@@ -0,0 +1,106 @@
{# Feel free to customize or override anything in there! #}
{%- set url = "http://container.training/" -%}
{%- set pagesize = 12 -%}
{%- if clustersize == 1 -%}
{%- set workshop_name = "Docker workshop" -%}
{%- set cluster_or_machine = "machine" -%}
{%- set this_or_each = "this" -%}
{%- set machine_is_or_machines_are = "machine is" -%}
{%- set image_src = "https://s3-us-west-2.amazonaws.com/www.breadware.com/integrations/docker.png" -%}
{%- else -%}
{%- set workshop_name = "Kubernetes workshop" -%}
{%- set cluster_or_machine = "cluster" -%}
{%- set this_or_each = "each" -%}
{%- set machine_is_or_machines_are = "machines are" -%}
{%- set image_src_swarm = "https://cdn.wp.nginx.com/wp-content/uploads/2016/07/docker-swarm-hero2.png" -%}
{%- set image_src_kube = "https://avatars1.githubusercontent.com/u/13629408" -%}
{%- set image_src = image_src_kube -%}
{%- endif -%}
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head><style>
body, table {
margin: 0;
padding: 0;
line-height: 1em;
font-size: 14px;
}
table {
border-spacing: 0;
margin-top: 0.4em;
margin-bottom: 0.4em;
border-left: 0.8em double grey;
padding-left: 0.4em;
}
div {
float: left;
border: 1px dotted black;
padding-top: 1%;
padding-bottom: 1%;
/* columns * (width+left+right) < 100% */
width: 21.5%;
padding-left: 1.5%;
padding-right: 1.5%;
}
p {
margin: 0.4em 0 0.4em 0;
}
img {
height: 4em;
float: right;
margin-right: -0.4em;
}
.logpass {
font-family: monospace;
font-weight: bold;
}
.pagebreak {
page-break-after: always;
clear: both;
display: block;
height: 8px;
}
</style></head>
<body>
{% for cluster in clusters %}
{% if loop.index0>0 and loop.index0%pagesize==0 %}
<span class="pagebreak"></span>
{% endif %}
<div>
<p>
Here is the connection information to your very own
{{ cluster_or_machine }} for this {{ workshop_name }}.
You can connect to {{ this_or_each }} VM with any SSH client.
</p>
<p>
<img src="{{ image_src }}" />
<table>
<tr><td>login:</td></tr>
<tr><td class="logpass">docker</td></tr>
<tr><td>password:</td></tr>
<tr><td class="logpass">training</td></tr>
</table>
</p>
<p>
Your {{ machine_is_or_machines_are }}:
<table>
{% for node in cluster %}
<tr><td>node{{ loop.index }}:</td><td>{{ node }}</td></tr>
{% endfor %}
</table>
</p>
<p>You can find the slides at:
<center>{{ url }}</center>
</p>
</div>
{% endfor %}
</body>
</html>

View File

@@ -0,0 +1,24 @@
# 3 nodes for k8s 101 workshops
# Number of VMs per cluster
clustersize: 3
# Jinja2 template to use to generate ready-to-cut cards
cards_template: settings/kube101.html
# Use "Letter" in the US, and "A4" everywhere else
paper_size: Letter
# Feel free to reduce this if your printer can handle it
paper_margin: 0.2in
# Note: paper_size and paper_margin only apply to PDF generated with pdfkit.
# If you print (or generate a PDF) using ips.html, they will be ignored.
# (The equivalent parameters must be set from the browser's print dialog.)
# This can be "test" or "stable"
engine_version: test
# These correspond to the version numbers visible on their respective GitHub release pages
compose_version: 1.18.0
machine_version: 0.13.0

View File

@@ -20,5 +20,5 @@ paper_margin: 0.2in
engine_version: test
# These correspond to the version numbers visible on their respective GitHub release pages
compose_version: 1.16.1
machine_version: 0.12.0
compose_version: 1.17.1
machine_version: 0.13.0

View File

@@ -38,7 +38,7 @@ check_envvars() {
if [ -z "${!envvar}" ]; then
error "Environment variable $envvar is not set."
if [ "$envvar" = "SSH_AUTH_SOCK" ]; then
error "Hint: run '\$(ssh-agent) ; ssh-add' and try again?"
error "Hint: run 'eval \$(ssh-agent) ; ssh-add' and try again?"
fi
status=1
fi

1
slides/_redirects Normal file
View File

@@ -0,0 +1 @@
/* http://paris-container-training.netlify.com/:splat 200!

17
slides/appendcheck.py Executable file
View File

@@ -0,0 +1,17 @@
#!/usr/bin/env python
import logging
import os
import subprocess
import sys
logging.basicConfig(level=os.environ.get("LOG_LEVEL", "INFO"))
filename = sys.argv[1]
logging.info("Checking file {}...".format(filename))
text = subprocess.check_output(["./slidechecker.js", filename])
html = open(filename).read()
html = html.replace("</textarea>", "\n---\n```\n{}\n```\n</textarea>".format(text))
open(filename, "w").write(html)

413
slides/autopilot/autotest.py Executable file
View File

@@ -0,0 +1,413 @@
#!/usr/bin/env python
# coding: utf-8
import click
import logging
import os
import random
import re
import select
import subprocess
import sys
import time
import uuid
import yaml
logging.basicConfig(level=os.environ.get("LOG_LEVEL", "INFO"))
TIMEOUT = 60 # 1 minute
class State(object):
def __init__(self):
self.interactive = True
self.verify_status = False
self.simulate_type = True
self.slide = 1
self.snippet = 0
def load(self):
data = yaml.load(open("state.yaml"))
self.interactive = bool(data["interactive"])
self.verify_status = bool(data["verify_status"])
self.simulate_type = bool(data["simulate_type"])
self.slide = int(data["slide"])
self.snippet = int(data["snippet"])
def save(self):
with open("state.yaml", "w") as f:
yaml.dump(dict(
interactive=self.interactive,
verify_status=self.verify_status,
simulate_type=self.simulate_type,
slide=self.slide,
snippet=self.snippet,
), f, default_flow_style=False)
state = State()
def hrule():
return "="*int(subprocess.check_output(["tput", "cols"]))
# A "snippet" is something that the user is supposed to do in the workshop.
# Most of the "snippets" are shell commands.
# Some of them can be key strokes or other actions.
# In the markdown source, they are the code sections (identified by triple-
# quotes) within .exercise[] sections.
class Snippet(object):
def __init__(self, slide, content):
self.slide = slide
self.content = content
# Extract the "method" (e.g. bash, keys, ...)
# On multi-line snippets, the method is alone on the first line
# On single-line snippets, the data follows the method immediately
if '\n' in content:
self.method, self.data = content.split('\n', 1)
else:
self.method, self.data = content.split(' ', 1)
self.data = self.data.strip()
self.next = None
def __str__(self):
return self.content
class Slide(object):
current_slide = 0
def __init__(self, content):
self.number = Slide.current_slide
Slide.current_slide += 1
# Remove commented-out slides
# (remark.js considers ??? to be the separator for speaker notes)
content = re.split("\n\?\?\?\n", content)[0]
self.content = content
self.snippets = []
exercises = re.findall("\.exercise\[(.*)\]", content, re.DOTALL)
for exercise in exercises:
if "```" in exercise:
previous = None
for snippet_content in exercise.split("```")[1::2]:
snippet = Snippet(self, snippet_content)
if previous:
previous.next = snippet
previous = snippet
self.snippets.append(snippet)
else:
logging.warning("Exercise on slide {} does not have any ``` snippet."
.format(self.number))
self.debug()
def __str__(self):
text = self.content
for snippet in self.snippets:
text = text.replace(snippet.content, ansi("7")(snippet.content))
return text
def debug(self):
logging.debug("\n{}\n{}\n{}".format(hrule(), self.content, hrule()))
def focus_slides():
subprocess.check_output(["i3-msg", "workspace", "3"])
subprocess.check_output(["i3-msg", "workspace", "1"])
def focus_terminal():
subprocess.check_output(["i3-msg", "workspace", "2"])
subprocess.check_output(["i3-msg", "workspace", "1"])
def focus_browser():
subprocess.check_output(["i3-msg", "workspace", "4"])
subprocess.check_output(["i3-msg", "workspace", "1"])
def ansi(code):
return lambda s: "\x1b[{}m{}\x1b[0m".format(code, s)
# Sleeps the indicated delay, but interruptible by pressing ENTER.
# If interrupted, returns True.
def interruptible_sleep(t):
rfds, _, _ = select.select([0], [], [], t)
return 0 in rfds
def wait_for_string(s, timeout=TIMEOUT):
logging.debug("Waiting for string: {}".format(s))
deadline = time.time() + timeout
while time.time() < deadline:
output = capture_pane()
if s in output:
return
if interruptible_sleep(1): return
raise Exception("Timed out while waiting for {}!".format(s))
def wait_for_prompt():
logging.debug("Waiting for prompt.")
deadline = time.time() + TIMEOUT
while time.time() < deadline:
output = capture_pane()
# If we are not at the bottom of the screen, there will be a bunch of extra \n's
output = output.rstrip('\n')
last_line = output.split('\n')[-1]
# Our custom prompt on the VMs has two lines; the 2nd line is just '$'
if last_line == "$":
return
# When we are in an alpine container, the prompt will be "/ #"
if last_line == "/ #":
return
# We did not recognize a known prompt; wait a bit and check again
logging.debug("Could not find a known prompt on last line: {!r}"
.format(last_line))
if interruptible_sleep(1): return
raise Exception("Timed out while waiting for prompt!")
def check_exit_status():
if not state.verify_status:
return
token = uuid.uuid4().hex
data = "echo {} $?\n".format(token)
logging.debug("Sending {!r} to get exit status.".format(data))
send_keys(data)
time.sleep(0.5)
wait_for_prompt()
screen = capture_pane()
status = re.findall("\n{} ([0-9]+)\n".format(token), screen, re.MULTILINE)
logging.debug("Got exit status: {}.".format(status))
if len(status) == 0:
raise Exception("Couldn't retrieve status code {}. Timed out?".format(token))
if len(status) > 1:
raise Exception("More than one status code {}. I'm seeing double! Shoot them both.".format(token))
code = int(status[0])
if code != 0:
raise Exception("Non-zero exit status: {}.".format(code))
# Otherwise just return peacefully.
def setup_tmux_and_ssh():
if subprocess.call(["tmux", "has-session"]):
logging.error("Couldn't connect to tmux. Please setup tmux first.")
ipaddr = open("../../prepare-vms/ips.txt").read().split("\n")[0]
uid = os.getuid()
raise Exception("""
1. If you're running this directly from a node:
tmux
2. If you want to control a remote tmux:
rm -f /tmp/tmux-{uid}/default && ssh -t -L /tmp/tmux-{uid}/default:/tmp/tmux-1001/default docker@{ipaddr} tmux new-session -As 0
3. If you cannot control a remote tmux:
tmux new-session ssh docker@{ipaddr}
""".format(uid=uid, ipaddr=ipaddr))
else:
logging.info("Found tmux session. Trying to acquire shell prompt.")
wait_for_prompt()
logging.info("Successfully connected to test cluster in tmux session.")
slides = [Slide("Dummy slide zero")]
content = open(sys.argv[1]).read()
# OK, this part is definitely hackish, and will break if the
# excludedClasses parameter is not on a single line.
excluded_classes = re.findall("excludedClasses: (\[.*\])", content)
excluded_classes = set(eval(excluded_classes[0]))
for slide in re.split("\n---?\n", content):
slide_classes = re.findall("class: (.*)", slide)
if slide_classes:
slide_classes = slide_classes[0].split(",")
slide_classes = [c.strip() for c in slide_classes]
if excluded_classes & set(slide_classes):
logging.info("Skipping excluded slide.")
continue
slides.append(Slide(slide))
def send_keys(data):
if state.simulate_type and data[0] != '^':
for key in data:
if key == ";":
key = "\\;"
if key == "\n":
if interruptible_sleep(1): return
subprocess.check_call(["tmux", "send-keys", key])
if interruptible_sleep(0.15*random.random()): return
if key == "\n":
if interruptible_sleep(1): return
else:
subprocess.check_call(["tmux", "send-keys", data])
def capture_pane():
return subprocess.check_output(["tmux", "capture-pane", "-p"]).decode('utf-8')
setup_tmux_and_ssh()
try:
state.load()
logging.info("Successfully loaded state from file.")
# Let's override the starting state, so that when an error occurs,
# we can restart the auto-tester and then single-step or debug.
# (Instead of running again through the same issue immediately.)
state.interactive = True
except Exception as e:
logging.exception("Could not load state from file.")
logging.warning("Using default values.")
def move_forward():
state.snippet += 1
if state.snippet > len(slides[state.slide].snippets):
state.slide += 1
state.snippet = 0
check_bounds()
def move_backward():
state.snippet -= 1
if state.snippet < 0:
state.slide -= 1
state.snippet = 0
check_bounds()
def check_bounds():
if state.slide < 1:
state.slide = 1
if state.slide >= len(slides):
state.slide = len(slides)-1
while True:
state.save()
slide = slides[state.slide]
snippet = slide.snippets[state.snippet-1] if state.snippet else None
click.clear()
print("[Slide {}/{}] [Snippet {}/{}] [simulate_type:{}] [verify_status:{}]"
.format(state.slide, len(slides)-1,
state.snippet, len(slide.snippets) if slide.snippets else 0,
state.simulate_type, state.verify_status))
print(hrule())
if snippet:
print(slide.content.replace(snippet.content, ansi(7)(snippet.content)))
focus_terminal()
else:
print(slide.content)
subprocess.check_output(["./gotoslide.js", str(slide.number)])
focus_slides()
print(hrule())
if state.interactive:
print("y/⎵/⏎ Execute snippet or advance to next snippet")
print("p/← Previous")
print("n/→ Next")
print("s Simulate keystrokes")
print("v Validate exit status")
print("g Go to a specific slide")
print("q Quit")
print("c Continue non-interactively until next error")
command = click.getchar()
else:
command = "y"
if command in ("n", "\x1b[C"):
move_forward()
elif command in ("p", "\x1b[D"):
move_backward()
elif command == "s":
state.simulate_type = not state.simulate_type
elif command == "v":
state.verify_status = not state.verify_status
elif command == "g":
state.slide = click.prompt("Enter slide number", type=int)
state.snippet = 0
check_bounds()
elif command == "q":
break
elif command == "c":
# continue until next timeout
state.interactive = False
elif command in ("y", "\r", " "):
if not snippet:
# Advance to next snippet
# Advance until a slide that has snippets
while not slides[state.slide].snippets:
move_forward()
# But stop if we reach the last slide
if state.slide == len(slides)-1:
break
# And then advance to the snippet
move_forward()
continue
method, data = snippet.method, snippet.data
logging.info("Running with method {}: {}".format(method, data))
if method == "keys":
send_keys(data)
elif method == "bash":
# Make sure that we're ready
wait_for_prompt()
# Strip leading spaces
data = re.sub("\n +", "\n", data)
# Remove backticks (they are used to highlight sections)
data = data.replace('`', '')
# Add "RETURN" at the end of the command :)
data += "\n"
# Send command
send_keys(data)
# Force a short sleep to avoid race condition
time.sleep(0.5)
if snippet.next and snippet.next.method == "wait":
wait_for_string(snippet.next.data)
elif snippet.next and snippet.next.method == "longwait":
wait_for_string(snippet.next.data, 10*TIMEOUT)
else:
wait_for_prompt()
# Verify return code
check_exit_status()
elif method == "copypaste":
screen = capture_pane()
matches = re.findall(data, screen, flags=re.DOTALL)
if len(matches) == 0:
raise Exception("Could not find regex {} in output.".format(data))
# Arbitrarily get the most recent match
match = matches[-1]
# Remove line breaks (like a screen copy paste would do)
match = match.replace('\n', '')
send_keys(match + '\n')
# FIXME: we should factor out the "bash" method
wait_for_prompt()
check_exit_status()
elif method == "open":
# Cheap way to get node1's IP address
screen = capture_pane()
ipaddr = re.findall("^\[(.*)\]", screen, re.MULTILINE)[-1]
url = data.replace("/node1", "/{}".format(ipaddr))
# This should probably be adapted to run on different OS
subprocess.check_output(["xdg-open", url])
focus_browser()
if state.interactive:
print("Press any key to continue to next step...")
click.getchar()
else:
logging.warning("Unknown method {}: {!r}".format(method, data))
move_forward()
else:
logging.warning("Unknown command {}.".format(command))

17
slides/autopilot/gotoslide.js Executable file
View File

@@ -0,0 +1,17 @@
#!/usr/bin/env node
/* Expects a slide number as first argument.
* Will connect to the local pub/sub server,
* and issue a "go to slide X" command, which
* will be sent to all connected browsers.
*/
var io = require('socket.io-client');
var socket = io('http://localhost:3000');
socket.on('connect_error', function(){
console.log('connection error');
socket.close();
});
socket.emit('slide change', process.argv[2], function(){
socket.close();
});

603
slides/autopilot/package-lock.json generated Normal file
View File

@@ -0,0 +1,603 @@
{
"name": "container-training-pub-sub-server",
"version": "0.0.1",
"lockfileVersion": 1,
"requires": true,
"dependencies": {
"accepts": {
"version": "1.3.4",
"resolved": "https://registry.npmjs.org/accepts/-/accepts-1.3.4.tgz",
"integrity": "sha1-hiRnWMfdbSGmR0/whKR0DsBesh8=",
"requires": {
"mime-types": "2.1.17",
"negotiator": "0.6.1"
}
},
"after": {
"version": "0.8.2",
"resolved": "https://registry.npmjs.org/after/-/after-0.8.2.tgz",
"integrity": "sha1-/ts5T58OAqqXaOcCvaI7UF+ufh8="
},
"array-flatten": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/array-flatten/-/array-flatten-1.1.1.tgz",
"integrity": "sha1-ml9pkFGx5wczKPKgCJaLZOopVdI="
},
"arraybuffer.slice": {
"version": "0.0.6",
"resolved": "https://registry.npmjs.org/arraybuffer.slice/-/arraybuffer.slice-0.0.6.tgz",
"integrity": "sha1-8zshWfBTKj8xB6JywMz70a0peco="
},
"async-limiter": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/async-limiter/-/async-limiter-1.0.0.tgz",
"integrity": "sha512-jp/uFnooOiO+L211eZOoSyzpOITMXx1rBITauYykG3BRYPu8h0UcxsPNB04RR5vo4Tyz3+ay17tR6JVf9qzYWg=="
},
"backo2": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/backo2/-/backo2-1.0.2.tgz",
"integrity": "sha1-MasayLEpNjRj41s+u2n038+6eUc="
},
"base64-arraybuffer": {
"version": "0.1.5",
"resolved": "https://registry.npmjs.org/base64-arraybuffer/-/base64-arraybuffer-0.1.5.tgz",
"integrity": "sha1-c5JncZI7Whl0etZmqlzUv5xunOg="
},
"base64id": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/base64id/-/base64id-1.0.0.tgz",
"integrity": "sha1-R2iMuZu2gE8OBtPnY7HDLlfY5rY="
},
"better-assert": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/better-assert/-/better-assert-1.0.2.tgz",
"integrity": "sha1-QIZrnhueC1W0gYlDEeaPr/rrxSI=",
"requires": {
"callsite": "1.0.0"
}
},
"blob": {
"version": "0.0.4",
"resolved": "https://registry.npmjs.org/blob/-/blob-0.0.4.tgz",
"integrity": "sha1-vPEwUspURj8w+fx+lbmkdjCpSSE="
},
"body-parser": {
"version": "1.18.2",
"resolved": "https://registry.npmjs.org/body-parser/-/body-parser-1.18.2.tgz",
"integrity": "sha1-h2eKGdhLR9hZuDGZvVm84iKxBFQ=",
"requires": {
"bytes": "3.0.0",
"content-type": "1.0.4",
"debug": "2.6.9",
"depd": "1.1.1",
"http-errors": "1.6.2",
"iconv-lite": "0.4.19",
"on-finished": "2.3.0",
"qs": "6.5.1",
"raw-body": "2.3.2",
"type-is": "1.6.15"
}
},
"bytes": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/bytes/-/bytes-3.0.0.tgz",
"integrity": "sha1-0ygVQE1olpn4Wk6k+odV3ROpYEg="
},
"callsite": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/callsite/-/callsite-1.0.0.tgz",
"integrity": "sha1-KAOY5dZkvXQDi28JBRU+borxvCA="
},
"component-bind": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/component-bind/-/component-bind-1.0.0.tgz",
"integrity": "sha1-AMYIq33Nk4l8AAllGx06jh5zu9E="
},
"component-emitter": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/component-emitter/-/component-emitter-1.2.1.tgz",
"integrity": "sha1-E3kY1teCg/ffemt8WmPhQOaUJeY="
},
"component-inherit": {
"version": "0.0.3",
"resolved": "https://registry.npmjs.org/component-inherit/-/component-inherit-0.0.3.tgz",
"integrity": "sha1-ZF/ErfWLcrZJ1crmUTVhnbJv8UM="
},
"content-disposition": {
"version": "0.5.2",
"resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-0.5.2.tgz",
"integrity": "sha1-DPaLud318r55YcOoUXjLhdunjLQ="
},
"content-type": {
"version": "1.0.4",
"resolved": "https://registry.npmjs.org/content-type/-/content-type-1.0.4.tgz",
"integrity": "sha512-hIP3EEPs8tB9AT1L+NUqtwOAps4mk2Zob89MWXMHjHWg9milF/j4osnnQLXBCBFBk/tvIG/tUc9mOUJiPBhPXA=="
},
"cookie": {
"version": "0.3.1",
"resolved": "https://registry.npmjs.org/cookie/-/cookie-0.3.1.tgz",
"integrity": "sha1-5+Ch+e9DtMi6klxcWpboBtFoc7s="
},
"cookie-signature": {
"version": "1.0.6",
"resolved": "https://registry.npmjs.org/cookie-signature/-/cookie-signature-1.0.6.tgz",
"integrity": "sha1-4wOogrNCzD7oylE6eZmXNNqzriw="
},
"debug": {
"version": "2.6.9",
"resolved": "https://registry.npmjs.org/debug/-/debug-2.6.9.tgz",
"integrity": "sha512-bC7ElrdJaJnPbAP+1EotYvqZsb3ecl5wi6Bfi6BJTUcNowp6cvspg0jXznRTKDjm/E7AdgFBVeAPVMNcKGsHMA==",
"requires": {
"ms": "2.0.0"
}
},
"depd": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/depd/-/depd-1.1.1.tgz",
"integrity": "sha1-V4O04cRZ8G+lyif5kfPQbnoxA1k="
},
"destroy": {
"version": "1.0.4",
"resolved": "https://registry.npmjs.org/destroy/-/destroy-1.0.4.tgz",
"integrity": "sha1-l4hXRCxEdJ5CBmE+N5RiBYJqvYA="
},
"ee-first": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/ee-first/-/ee-first-1.1.1.tgz",
"integrity": "sha1-WQxhFWsK4vTwJVcyoViyZrxWsh0="
},
"encodeurl": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/encodeurl/-/encodeurl-1.0.1.tgz",
"integrity": "sha1-eePVhlU0aQn+bw9Fpd5oEDspTSA="
},
"engine.io": {
"version": "3.1.4",
"resolved": "https://registry.npmjs.org/engine.io/-/engine.io-3.1.4.tgz",
"integrity": "sha1-PQIRtwpVLOhB/8fahiezAamkFi4=",
"requires": {
"accepts": "1.3.3",
"base64id": "1.0.0",
"cookie": "0.3.1",
"debug": "2.6.9",
"engine.io-parser": "2.1.1",
"uws": "0.14.5",
"ws": "3.3.3"
},
"dependencies": {
"accepts": {
"version": "1.3.3",
"resolved": "https://registry.npmjs.org/accepts/-/accepts-1.3.3.tgz",
"integrity": "sha1-w8p0NJOGSMPg2cHjKN1otiLChMo=",
"requires": {
"mime-types": "2.1.17",
"negotiator": "0.6.1"
}
}
}
},
"engine.io-client": {
"version": "3.1.4",
"resolved": "https://registry.npmjs.org/engine.io-client/-/engine.io-client-3.1.4.tgz",
"integrity": "sha1-T88TcLRxY70s6b4nM5ckMDUNTqE=",
"requires": {
"component-emitter": "1.2.1",
"component-inherit": "0.0.3",
"debug": "2.6.9",
"engine.io-parser": "2.1.1",
"has-cors": "1.1.0",
"indexof": "0.0.1",
"parseqs": "0.0.5",
"parseuri": "0.0.5",
"ws": "3.3.3",
"xmlhttprequest-ssl": "1.5.4",
"yeast": "0.1.2"
}
},
"engine.io-parser": {
"version": "2.1.1",
"resolved": "https://registry.npmjs.org/engine.io-parser/-/engine.io-parser-2.1.1.tgz",
"integrity": "sha1-4Ps/DgRi9/WLt3waUun1p+JuRmg=",
"requires": {
"after": "0.8.2",
"arraybuffer.slice": "0.0.6",
"base64-arraybuffer": "0.1.5",
"blob": "0.0.4",
"has-binary2": "1.0.2"
}
},
"escape-html": {
"version": "1.0.3",
"resolved": "https://registry.npmjs.org/escape-html/-/escape-html-1.0.3.tgz",
"integrity": "sha1-Aljq5NPQwJdN4cFpGI7wBR0dGYg="
},
"etag": {
"version": "1.8.1",
"resolved": "https://registry.npmjs.org/etag/-/etag-1.8.1.tgz",
"integrity": "sha1-Qa4u62XvpiJorr/qg6x9eSmbCIc="
},
"express": {
"version": "4.16.2",
"resolved": "https://registry.npmjs.org/express/-/express-4.16.2.tgz",
"integrity": "sha1-41xt/i1kt9ygpc1PIXgb4ymeB2w=",
"requires": {
"accepts": "1.3.4",
"array-flatten": "1.1.1",
"body-parser": "1.18.2",
"content-disposition": "0.5.2",
"content-type": "1.0.4",
"cookie": "0.3.1",
"cookie-signature": "1.0.6",
"debug": "2.6.9",
"depd": "1.1.1",
"encodeurl": "1.0.1",
"escape-html": "1.0.3",
"etag": "1.8.1",
"finalhandler": "1.1.0",
"fresh": "0.5.2",
"merge-descriptors": "1.0.1",
"methods": "1.1.2",
"on-finished": "2.3.0",
"parseurl": "1.3.2",
"path-to-regexp": "0.1.7",
"proxy-addr": "2.0.2",
"qs": "6.5.1",
"range-parser": "1.2.0",
"safe-buffer": "5.1.1",
"send": "0.16.1",
"serve-static": "1.13.1",
"setprototypeof": "1.1.0",
"statuses": "1.3.1",
"type-is": "1.6.15",
"utils-merge": "1.0.1",
"vary": "1.1.2"
}
},
"finalhandler": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/finalhandler/-/finalhandler-1.1.0.tgz",
"integrity": "sha1-zgtoVbRYU+eRsvzGgARtiCU91/U=",
"requires": {
"debug": "2.6.9",
"encodeurl": "1.0.1",
"escape-html": "1.0.3",
"on-finished": "2.3.0",
"parseurl": "1.3.2",
"statuses": "1.3.1",
"unpipe": "1.0.0"
}
},
"forwarded": {
"version": "0.1.2",
"resolved": "https://registry.npmjs.org/forwarded/-/forwarded-0.1.2.tgz",
"integrity": "sha1-mMI9qxF1ZXuMBXPozszZGw/xjIQ="
},
"fresh": {
"version": "0.5.2",
"resolved": "https://registry.npmjs.org/fresh/-/fresh-0.5.2.tgz",
"integrity": "sha1-PYyt2Q2XZWn6g1qx+OSyOhBWBac="
},
"has-binary2": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/has-binary2/-/has-binary2-1.0.2.tgz",
"integrity": "sha1-6D26SfC5vk0CbSc2U1DZ8D9Uvpg=",
"requires": {
"isarray": "2.0.1"
}
},
"has-cors": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/has-cors/-/has-cors-1.1.0.tgz",
"integrity": "sha1-XkdHk/fqmEPRu5nCPu9J/xJv/zk="
},
"http-errors": {
"version": "1.6.2",
"resolved": "https://registry.npmjs.org/http-errors/-/http-errors-1.6.2.tgz",
"integrity": "sha1-CgAsyFcHGSp+eUbO7cERVfYOxzY=",
"requires": {
"depd": "1.1.1",
"inherits": "2.0.3",
"setprototypeof": "1.0.3",
"statuses": "1.3.1"
},
"dependencies": {
"setprototypeof": {
"version": "1.0.3",
"resolved": "https://registry.npmjs.org/setprototypeof/-/setprototypeof-1.0.3.tgz",
"integrity": "sha1-ZlZ+NwQ+608E2RvWWMDL77VbjgQ="
}
}
},
"iconv-lite": {
"version": "0.4.19",
"resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.4.19.tgz",
"integrity": "sha512-oTZqweIP51xaGPI4uPa56/Pri/480R+mo7SeU+YETByQNhDG55ycFyNLIgta9vXhILrxXDmF7ZGhqZIcuN0gJQ=="
},
"indexof": {
"version": "0.0.1",
"resolved": "https://registry.npmjs.org/indexof/-/indexof-0.0.1.tgz",
"integrity": "sha1-gtwzbSMrkGIXnQWrMpOmYFn9Q10="
},
"inherits": {
"version": "2.0.3",
"resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.3.tgz",
"integrity": "sha1-Yzwsg+PaQqUC9SRmAiSA9CCCYd4="
},
"ipaddr.js": {
"version": "1.5.2",
"resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-1.5.2.tgz",
"integrity": "sha1-1LUFvemUaYfM8PxY2QEP+WB+P6A="
},
"isarray": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/isarray/-/isarray-2.0.1.tgz",
"integrity": "sha1-o32U7ZzaLVmGXJ92/llu4fM4dB4="
},
"media-typer": {
"version": "0.3.0",
"resolved": "https://registry.npmjs.org/media-typer/-/media-typer-0.3.0.tgz",
"integrity": "sha1-hxDXrwqmJvj/+hzgAWhUUmMlV0g="
},
"merge-descriptors": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/merge-descriptors/-/merge-descriptors-1.0.1.tgz",
"integrity": "sha1-sAqqVW3YtEVoFQ7J0blT8/kMu2E="
},
"methods": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/methods/-/methods-1.1.2.tgz",
"integrity": "sha1-VSmk1nZUE07cxSZmVoNbD4Ua/O4="
},
"mime": {
"version": "1.4.1",
"resolved": "https://registry.npmjs.org/mime/-/mime-1.4.1.tgz",
"integrity": "sha512-KI1+qOZu5DcW6wayYHSzR/tXKCDC5Om4s1z2QJjDULzLcmf3DvzS7oluY4HCTrc+9FiKmWUgeNLg7W3uIQvxtQ=="
},
"mime-db": {
"version": "1.30.0",
"resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.30.0.tgz",
"integrity": "sha1-dMZD2i3Z1qRTmZY0ZbJtXKfXHwE="
},
"mime-types": {
"version": "2.1.17",
"resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.17.tgz",
"integrity": "sha1-Cdejk/A+mVp5+K+Fe3Cp4KsWVXo=",
"requires": {
"mime-db": "1.30.0"
}
},
"ms": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/ms/-/ms-2.0.0.tgz",
"integrity": "sha1-VgiurfwAvmwpAd9fmGF4jeDVl8g="
},
"negotiator": {
"version": "0.6.1",
"resolved": "https://registry.npmjs.org/negotiator/-/negotiator-0.6.1.tgz",
"integrity": "sha1-KzJxhOiZIQEXeyhWP7XnECrNDKk="
},
"object-component": {
"version": "0.0.3",
"resolved": "https://registry.npmjs.org/object-component/-/object-component-0.0.3.tgz",
"integrity": "sha1-8MaapQ78lbhmwYb0AKM3acsvEpE="
},
"on-finished": {
"version": "2.3.0",
"resolved": "https://registry.npmjs.org/on-finished/-/on-finished-2.3.0.tgz",
"integrity": "sha1-IPEzZIGwg811M3mSoWlxqi2QaUc=",
"requires": {
"ee-first": "1.1.1"
}
},
"parseqs": {
"version": "0.0.5",
"resolved": "https://registry.npmjs.org/parseqs/-/parseqs-0.0.5.tgz",
"integrity": "sha1-1SCKNzjkZ2bikbouoXNoSSGouJ0=",
"requires": {
"better-assert": "1.0.2"
}
},
"parseuri": {
"version": "0.0.5",
"resolved": "https://registry.npmjs.org/parseuri/-/parseuri-0.0.5.tgz",
"integrity": "sha1-gCBKUNTbt3m/3G6+J3jZDkvOMgo=",
"requires": {
"better-assert": "1.0.2"
}
},
"parseurl": {
"version": "1.3.2",
"resolved": "https://registry.npmjs.org/parseurl/-/parseurl-1.3.2.tgz",
"integrity": "sha1-/CidTtiZMRlGDBViUyYs3I3mW/M="
},
"path-to-regexp": {
"version": "0.1.7",
"resolved": "https://registry.npmjs.org/path-to-regexp/-/path-to-regexp-0.1.7.tgz",
"integrity": "sha1-32BBeABfUi8V60SQ5yR6G/qmf4w="
},
"proxy-addr": {
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/proxy-addr/-/proxy-addr-2.0.2.tgz",
"integrity": "sha1-ZXFQT0e7mI7IGAJT+F3X4UlSvew=",
"requires": {
"forwarded": "0.1.2",
"ipaddr.js": "1.5.2"
}
},
"qs": {
"version": "6.5.1",
"resolved": "https://registry.npmjs.org/qs/-/qs-6.5.1.tgz",
"integrity": "sha512-eRzhrN1WSINYCDCbrz796z37LOe3m5tmW7RQf6oBntukAG1nmovJvhnwHHRMAfeoItc1m2Hk02WER2aQ/iqs+A=="
},
"range-parser": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/range-parser/-/range-parser-1.2.0.tgz",
"integrity": "sha1-9JvmtIeJTdxA3MlKMi9hEJLgDV4="
},
"raw-body": {
"version": "2.3.2",
"resolved": "https://registry.npmjs.org/raw-body/-/raw-body-2.3.2.tgz",
"integrity": "sha1-vNYMd9Prk83gBQKVw/N5OJvIj4k=",
"requires": {
"bytes": "3.0.0",
"http-errors": "1.6.2",
"iconv-lite": "0.4.19",
"unpipe": "1.0.0"
}
},
"safe-buffer": {
"version": "5.1.1",
"resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.1.1.tgz",
"integrity": "sha512-kKvNJn6Mm93gAczWVJg7wH+wGYWNrDHdWvpUmHyEsgCtIwwo3bqPtV4tR5tuPaUhTOo/kvhVwd8XwwOllGYkbg=="
},
"send": {
"version": "0.16.1",
"resolved": "https://registry.npmjs.org/send/-/send-0.16.1.tgz",
"integrity": "sha512-ElCLJdJIKPk6ux/Hocwhk7NFHpI3pVm/IZOYWqUmoxcgeyM+MpxHHKhb8QmlJDX1pU6WrgaHBkVNm73Sv7uc2A==",
"requires": {
"debug": "2.6.9",
"depd": "1.1.1",
"destroy": "1.0.4",
"encodeurl": "1.0.1",
"escape-html": "1.0.3",
"etag": "1.8.1",
"fresh": "0.5.2",
"http-errors": "1.6.2",
"mime": "1.4.1",
"ms": "2.0.0",
"on-finished": "2.3.0",
"range-parser": "1.2.0",
"statuses": "1.3.1"
}
},
"serve-static": {
"version": "1.13.1",
"resolved": "https://registry.npmjs.org/serve-static/-/serve-static-1.13.1.tgz",
"integrity": "sha512-hSMUZrsPa/I09VYFJwa627JJkNs0NrfL1Uzuup+GqHfToR2KcsXFymXSV90hoyw3M+msjFuQly+YzIH/q0MGlQ==",
"requires": {
"encodeurl": "1.0.1",
"escape-html": "1.0.3",
"parseurl": "1.3.2",
"send": "0.16.1"
}
},
"setprototypeof": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/setprototypeof/-/setprototypeof-1.1.0.tgz",
"integrity": "sha512-BvE/TwpZX4FXExxOxZyRGQQv651MSwmWKZGqvmPcRIjDqWub67kTKuIMx43cZZrS/cBBzwBcNDWoFxt2XEFIpQ=="
},
"socket.io": {
"version": "2.0.4",
"resolved": "https://registry.npmjs.org/socket.io/-/socket.io-2.0.4.tgz",
"integrity": "sha1-waRZDO/4fs8TxyZS8Eb3FrKeYBQ=",
"requires": {
"debug": "2.6.9",
"engine.io": "3.1.4",
"socket.io-adapter": "1.1.1",
"socket.io-client": "2.0.4",
"socket.io-parser": "3.1.2"
}
},
"socket.io-adapter": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/socket.io-adapter/-/socket.io-adapter-1.1.1.tgz",
"integrity": "sha1-KoBeihTWNyEk3ZFZrUUC+MsH8Gs="
},
"socket.io-client": {
"version": "2.0.4",
"resolved": "https://registry.npmjs.org/socket.io-client/-/socket.io-client-2.0.4.tgz",
"integrity": "sha1-CRilUkBtxeVAs4Dc2Xr8SmQzL44=",
"requires": {
"backo2": "1.0.2",
"base64-arraybuffer": "0.1.5",
"component-bind": "1.0.0",
"component-emitter": "1.2.1",
"debug": "2.6.9",
"engine.io-client": "3.1.4",
"has-cors": "1.1.0",
"indexof": "0.0.1",
"object-component": "0.0.3",
"parseqs": "0.0.5",
"parseuri": "0.0.5",
"socket.io-parser": "3.1.2",
"to-array": "0.1.4"
}
},
"socket.io-parser": {
"version": "3.1.2",
"resolved": "https://registry.npmjs.org/socket.io-parser/-/socket.io-parser-3.1.2.tgz",
"integrity": "sha1-28IoIVH8T6675Aru3Ady66YZ9/I=",
"requires": {
"component-emitter": "1.2.1",
"debug": "2.6.9",
"has-binary2": "1.0.2",
"isarray": "2.0.1"
}
},
"statuses": {
"version": "1.3.1",
"resolved": "https://registry.npmjs.org/statuses/-/statuses-1.3.1.tgz",
"integrity": "sha1-+vUbnrdKrvOzrPStX2Gr8ky3uT4="
},
"to-array": {
"version": "0.1.4",
"resolved": "https://registry.npmjs.org/to-array/-/to-array-0.1.4.tgz",
"integrity": "sha1-F+bBH3PdTz10zaek/zI46a2b+JA="
},
"type-is": {
"version": "1.6.15",
"resolved": "https://registry.npmjs.org/type-is/-/type-is-1.6.15.tgz",
"integrity": "sha1-yrEPtJCeRByChC6v4a1kbIGARBA=",
"requires": {
"media-typer": "0.3.0",
"mime-types": "2.1.17"
}
},
"ultron": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/ultron/-/ultron-1.1.1.tgz",
"integrity": "sha512-UIEXBNeYmKptWH6z8ZnqTeS8fV74zG0/eRU9VGkpzz+LIJNs8W/zM/L+7ctCkRrgbNnnR0xxw4bKOr0cW0N0Og=="
},
"unpipe": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/unpipe/-/unpipe-1.0.0.tgz",
"integrity": "sha1-sr9O6FFKrmFltIF4KdIbLvSZBOw="
},
"utils-merge": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/utils-merge/-/utils-merge-1.0.1.tgz",
"integrity": "sha1-n5VxD1CiZ5R7LMwSR0HBAoQn5xM="
},
"uws": {
"version": "0.14.5",
"resolved": "https://registry.npmjs.org/uws/-/uws-0.14.5.tgz",
"integrity": "sha1-Z6rzPEaypYel9mZtAPdpEyjxSdw=",
"optional": true
},
"vary": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/vary/-/vary-1.1.2.tgz",
"integrity": "sha1-IpnwLG3tMNSllhsLn3RSShj2NPw="
},
"ws": {
"version": "3.3.3",
"resolved": "https://registry.npmjs.org/ws/-/ws-3.3.3.tgz",
"integrity": "sha512-nnWLa/NwZSt4KQJu51MYlCcSQ5g7INpOrOMt4XV8j4dqTXdmlUmSHQ8/oLC069ckre0fRsgfvsKwbTdtKLCDkA==",
"requires": {
"async-limiter": "1.0.0",
"safe-buffer": "5.1.1",
"ultron": "1.1.1"
}
},
"xmlhttprequest-ssl": {
"version": "1.5.4",
"resolved": "https://registry.npmjs.org/xmlhttprequest-ssl/-/xmlhttprequest-ssl-1.5.4.tgz",
"integrity": "sha1-BPVgkVcks4kIhxXMDteBPpZ3v1c="
},
"yeast": {
"version": "0.1.2",
"resolved": "https://registry.npmjs.org/yeast/-/yeast-0.1.2.tgz",
"integrity": "sha1-AI4G2AlDIMNy28L47XagymyKxBk="
}
}
}

View File

@@ -0,0 +1,8 @@
{
"name": "container-training-pub-sub-server",
"version": "0.0.1",
"dependencies": {
"express": "^4.16.2",
"socket.io": "^2.0.4"
}
}

View File

@@ -0,0 +1,21 @@
/* This snippet is loaded from the workshop HTML file.
* It sets up callbacks to synchronize the local slide
* number with the remote pub/sub server.
*/
var socket = io();
var leader = true;
slideshow.on('showSlide', function (slide) {
if (leader) {
var n = slide.getSlideIndex()+1;
socket.emit('slide change', n);
}
});
socket.on('slide change', function (n) {
leader = false;
slideshow.gotoSlide(n);
leader = true;
});

41
slides/autopilot/server.js Executable file
View File

@@ -0,0 +1,41 @@
#!/usr/bin/env node
/* This is a very simple pub/sub server, allowing to
* remote control browsers displaying the slides.
* The browsers connect to this pub/sub server using
* Socket.IO, and the server tells them which slides
* to display.
*
* The server can be controlled with a little CLI,
* or by one of the browsers.
*/
var express = require('express');
var app = express();
var http = require('http').Server(app);
var io = require('socket.io')(http);
app.get('/', function(req, res){
res.send('container.training autopilot pub/sub server');
});
/* Serve remote.js from the current directory */
app.use(express.static('.'));
/* Serve slides etc. from current and the parent directory */
app.use(express.static('..'));
io.on('connection', function(socket){
console.log('a client connected: ' + socket.handshake.address);
socket.on('slide change', function(n, ack){
console.log('slide change: ' + n);
socket.broadcast.emit('slide change', n);
if (typeof ack === 'function') {
ack();
}
});
});
http.listen(3000, function(){
console.log('listening on *:3000');
});

7
slides/autopilot/tmux-style.sh Executable file
View File

@@ -0,0 +1,7 @@
#!/bin/sh
# This removes the clock (and other extraneous stuff) from the
# tmux status bar, and it gives it a non-default color.
tmux set-option -g status-left ""
tmux set-option -g status-right ""
tmux set-option -g status-style bg=cyan

View File

@@ -2,11 +2,16 @@
case "$1" in
once)
for YAML in *.yml; do
./markmaker.py < $YAML > $YAML.html || {
./markmaker.py $YAML > $YAML.html || {
rm $YAML.html
break
}
done
if [ -n "$SLIDECHECKER" ]; then
for YAML in *.yml; do
./appendcheck.py $YAML.html
done
fi
;;
forever)

View File

@@ -0,0 +1,28 @@
## About these slides
- All the content is available in a public GitHub repository:
https://github.com/jpetazzo/container.training
- You can get updated "builds" of the slides there:
http://container.training/
<!--
.exercise[
```open https://github.com/jpetazzo/container.training```
```open http://container.training/```
]
-->
--
- Typos? Mistakes? Questions? Feel free to hover over the bottom of the slide ...
.footnote[.emoji[👇] Try it! The source file will be shown and you can view it on GitHub and fork and edit it.]
<!--
.exercise[
```open https://github.com/jpetazzo/container.training/tree/master/slides/common/about-slides.md```
]
-->

View File

@@ -0,0 +1,12 @@
## Clean up
- Before moving on, let's remove those containers
.exercise[
- Tell Compose to remove everything:
```bash
docker-compose down
```
]

View File

@@ -0,0 +1,240 @@
## Restarting in the background
- Many flags and commands of Compose are modeled after those of `docker`
.exercise[
- Start the app in the background with the `-d` option:
```bash
docker-compose up -d
```
- Check that our app is running with the `ps` command:
```bash
docker-compose ps
```
]
`docker-compose ps` also shows the ports exposed by the application.
---
class: extra-details
## Viewing logs
- The `docker-compose logs` command works like `docker logs`
.exercise[
- View all logs since container creation and exit when done:
```bash
docker-compose logs
```
- Stream container logs, starting at the last 10 lines for each container:
```bash
docker-compose logs --tail 10 --follow
```
<!--
```wait units of work done```
```keys ^C```
-->
]
Tip: use `^S` and `^Q` to pause/resume log output.
---
class: extra-details
## Upgrading from Compose 1.6
.warning[The `logs` command has changed between Compose 1.6 and 1.7!]
- Up to 1.6
- `docker-compose logs` is the equivalent of `logs --follow`
- `docker-compose logs` must be restarted if containers are added
- Since 1.7
- `--follow` must be specified explicitly
- new containers are automatically picked up by `docker-compose logs`
---
## Scaling up the application
- Our goal is to make that performance graph go up (without changing a line of code!)
--
- Before trying to scale the application, we'll figure out if we need more resources
(CPU, RAM...)
- For that, we will use good old UNIX tools on our Docker node
---
## Looking at resource usage
- Let's look at CPU, memory, and I/O usage
.exercise[
- run `top` to see CPU and memory usage (you should see idle cycles)
<!--
```bash top```
```wait Tasks```
```keys ^C```
-->
- run `vmstat 1` to see I/O usage (si/so/bi/bo)
<br/>(the 4 numbers should be almost zero, except `bo` for logging)
<!--
```bash vmstat 1```
```wait memory```
```keys ^C```
-->
]
We have available resources.
- Why?
- How can we use them?
---
## Scaling workers on a single node
- Docker Compose supports scaling
- Let's scale `worker` and see what happens!
.exercise[
- Start one more `worker` container:
```bash
docker-compose scale worker=2
```
- Look at the performance graph (it should show a x2 improvement)
- Look at the aggregated logs of our containers (`worker_2` should show up)
- Look at the impact on CPU load with e.g. top (it should be negligible)
]
---
## Adding more workers
- Great, let's add more workers and call it a day, then!
.exercise[
- Start eight more `worker` containers:
```bash
docker-compose scale worker=10
```
- Look at the performance graph: does it show a x10 improvement?
- Look at the aggregated logs of our containers
- Look at the impact on CPU load and memory usage
]
---
# Identifying bottlenecks
- You should have seen a 3x speed bump (not 10x)
- Adding workers didn't result in linear improvement
- *Something else* is slowing us down
--
- ... But what?
--
- The code doesn't have instrumentation
- Let's use state-of-the-art HTTP performance analysis!
<br/>(i.e. good old tools like `ab`, `httping`...)
---
## Accessing internal services
- `rng` and `hasher` are exposed on ports 8001 and 8002
- This is declared in the Compose file:
```yaml
...
rng:
build: rng
ports:
- "8001:80"
hasher:
build: hasher
ports:
- "8002:80"
...
```
---
## Measuring latency under load
We will use `httping`.
.exercise[
- Check the latency of `rng`:
```bash
httping -c 3 localhost:8001
```
- Check the latency of `hasher`:
```bash
httping -c 3 localhost:8002
```
]
`rng` has a much higher latency than `hasher`.
---
## Let's draw hasty conclusions
- The bottleneck seems to be `rng`
- *What if* we don't have enough entropy and can't generate enough random numbers?
- We need to scale out the `rng` service on multiple machines!
Note: this is a fiction! We have enough entropy. But we need a pretext to scale out.
(In fact, the code of `rng` uses `/dev/urandom`, which never runs out of entropy...
<br/>
...and is [just as good as `/dev/random`](http://www.slideshare.net/PacSecJP/filippo-plain-simple-reality-of-entropy).)

View File

@@ -0,0 +1,63 @@
# Declarative vs imperative
- Our container orchestrator puts a very strong emphasis on being *declarative*
- Declarative:
*I would like a cup of tea.*
- Imperative:
*Boil some water. Pour it in a teapot. Add tea leaves. Steep for a while. Serve in cup.*
--
- Declarative seems simpler at first ...
--
- ... As long as you know how to brew tea
---
## Declarative vs imperative
- What declarative would really be:
*I want a cup of tea, obtained by pouring an infusion¹ of tea leaves in a cup.*
--
*¹An infusion is obtained by letting the object steep a few minutes in hot² water.*
--
*²Hot liquid is obtained by pouring it in an appropriate container³ and setting it on a stove.*
--
*³Ah, finally, containers! Something we know about. Let's get to work, shall we?*
--
.footnote[Did you know there was an [ISO standard](https://en.wikipedia.org/wiki/ISO_3103)
specifying how to brew tea?]
---
## Declarative vs imperative
- Imperative systems:
- simpler
- if a task is interrupted, we have to restart from scratch
- Declarative systems:
- if a task is interrupted (or if we show up to the party half-way through),
we can figure out what's missing and do only what's necessary
- we need to be able to *observe* the system
- ... and compute a "diff" between *what we have* and *what we want*

278
slides/common/prereqs.md Normal file
View File

@@ -0,0 +1,278 @@
# Pre-requirements
- Be comfortable with the UNIX command line
- navigating directories
- editing files
- a little bit of bash-fu (environment variables, loops)
- Some Docker knowledge
- `docker run`, `docker ps`, `docker build`
- ideally, you know how to write a Dockerfile and build it
<br/>
(even if it's a `FROM` line and a couple of `RUN` commands)
- It's totally OK if you are not a Docker expert!
---
class: extra-details
## Extra details
- This slide has a little magnifying glass in the top left corner
- This magnifiying glass indicates slides that provide extra details
- Feel free to skip them if:
- you are in a hurry
- you are new to this and want to avoid cognitive overload
- you want only the most essential information
- You can review these slides another time if you want, they'll be waiting for you ☺
---
class: title
*Tell me and I forget.*
<br/>
*Teach me and I remember.*
<br/>
*Involve me and I learn.*
Misattributed to Benjamin Franklin
[(Probably inspired by Chinese Confucian philosopher Xunzi)](https://www.barrypopik.com/index.php/new_york_city/entry/tell_me_and_i_forget_teach_me_and_i_may_remember_involve_me_and_i_will_lear/)
---
## Hands-on sections
- The whole workshop is hands-on
- We are going to build, ship, and run containers!
- You are invited to reproduce all the demos
- All hands-on sections are clearly identified, like the gray rectangle below
.exercise[
- This is the stuff you're supposed to do!
- Go to [container.training](http://container.training/) to view these slides
- Join the chat room: @@CHAT@@
<!-- ```open http://container.training/``` -->
]
---
class: in-person
## Where are we going to run our containers?
---
class: in-person, pic
![You get a cluster](images/you-get-a-cluster.jpg)
---
class: in-person
## You get a cluster of cloud VMs
- Each person gets a private cluster of cloud VMs (not shared with anybody else)
- They'll remain up for the duration of the workshop
- You should have a little card with login+password+IP addresses
- You can automatically SSH from one VM to another
- The nodes have aliases: `node1`, `node2`, etc.
---
class: in-person
## Why don't we run containers locally?
- Installing that stuff can be hard on some machines
(32 bits CPU or OS... Laptops without administrator access... etc.)
- *"The whole team downloaded all these container images from the WiFi!
<br/>... and it went great!"* (Literally no-one ever)
- All you need is a computer (or even a phone or tablet!), with:
- an internet connection
- a web browser
- an SSH client
---
class: in-person
## SSH clients
- On Linux, OS X, FreeBSD... you are probably all set
- On Windows, get one of these:
- [putty](http://www.putty.org/)
- Microsoft [Win32 OpenSSH](https://github.com/PowerShell/Win32-OpenSSH/wiki/Install-Win32-OpenSSH)
- [Git BASH](https://git-for-windows.github.io/)
- [MobaXterm](http://mobaxterm.mobatek.net/)
- On Android, [JuiceSSH](https://juicessh.com/)
([Play Store](https://play.google.com/store/apps/details?id=com.sonelli.juicessh))
works pretty well
- Nice-to-have: [Mosh](https://mosh.org/) instead of SSH, if your internet connection tends to lose packets
<br/>(available with `(apt|yum|brew) install mosh`; then connect with `mosh user@host`)
---
class: in-person
## Connecting to our lab environment
.exercise[
- Log into the first VM (`node1`) with SSH or MOSH
<!--
```bash
for N in $(awk '/node/{print $2}' /etc/hosts); do
ssh -o StrictHostKeyChecking=no node$N true
done
```
```bash
if which kubectl; then
kubectl get all -o name | grep -v services/kubernetes | xargs -n1 kubectl delete
fi
```
-->
- Check that you can SSH (without password) to `node2`:
```bash
ssh node2
```
- Type `exit` or `^D` to come back to `node1`
<!-- ```bash exit``` -->
]
If anything goes wrong — ask for help!
---
## Doing or re-doing the workshop on your own?
- Use something like
[Play-With-Docker](http://play-with-docker.com/) or
[Play-With-Kubernetes](https://medium.com/@marcosnils/introducing-pwk-play-with-k8s-159fcfeb787b)
Zero setup effort; but environment are short-lived and
might have limited resources
- Create your own cluster (local or cloud VMs)
Small setup effort; small cost; flexible environments
- Create a bunch of clusters for you and your friends
([instructions](https://github.com/jpetazzo/container.training/tree/master/prepare-vms))
Bigger setup effort; ideal for group training
---
class: self-paced
## Get your own Docker nodes
- If you already have some Docker nodes: great!
- If not: let's get some thanks to Play-With-Docker
.exercise[
- Go to http://www.play-with-docker.com/
- Log in
- Create your first node
<!-- ```open http://www.play-with-docker.com/``` -->
]
You will need a Docker ID to use Play-With-Docker.
(Creating a Docker ID is free.)
---
## We will (mostly) interact with node1 only
*These remarks apply only when using multiple nodes, of course.*
- Unless instructed, **all commands must be run from the first VM, `node1`**
- We will only checkout/copy the code on `node1`
- During normal operations, we do not need access to the other nodes
- If we had to troubleshoot issues, we would use a combination of:
- SSH (to access system logs, daemon status...)
- Docker API (to check running containers and container engine status)
---
## Terminals
Once in a while, the instructions will say:
<br/>"Open a new terminal."
There are multiple ways to do this:
- create a new window or tab on your machine, and SSH into the VM;
- use screen or tmux on the VM and open a new window from there.
You are welcome to use the method that you feel the most comfortable with.
---
## Tmux cheatsheet
- Ctrl-b c → creates a new window
- Ctrl-b n → go to next window
- Ctrl-b p → go to previous window
- Ctrl-b " → split window top/bottom
- Ctrl-b % → split window left/right
- Ctrl-b Alt-1 → rearrange windows in columns
- Ctrl-b Alt-2 → rearrange windows in rows
- Ctrl-b arrows → navigate to other windows
- Ctrl-b d → detach session
- tmux attach → reattach to session

19
slides/common/pwd.md Normal file
View File

@@ -0,0 +1,19 @@
## Using Play-With-Docker
- Open a new browser tab to [www.play-with-docker.com](http://www.play-with-docker.com/)
- Confirm that you're not a robot
- Click on "ADD NEW INSTANCE": congratulations, you have your first Docker node!
- When you will need more nodes, just click on "ADD NEW INSTANCE" again
- Note the countdown in the corner; when it expires, your instances are destroyed
- If you give your URL to somebody else, they can access your nodes too
<br/>
(You can use that for pair programming, or to get help from a mentor)
- Loving it? Not loving it? Tell it to the wonderful authors,
[@marcosnils](https://twitter.com/marcosnils) &
[@xetorthio](https://twitter.com/xetorthio)!

View File

@@ -1,16 +1,16 @@
# Our sample application
- Visit the GitHub repository with all the materials of this workshop:
<br/>https://github.com/jpetazzo/orchestration-workshop
<br/>https://github.com/jpetazzo/container.training
- The application is in the [dockercoins](
https://github.com/jpetazzo/orchestration-workshop/tree/master/dockercoins)
https://github.com/jpetazzo/container.training/tree/master/dockercoins)
subdirectory
- Let's look at the general layout of the source code:
there is a Compose file [docker-compose.yml](
https://github.com/jpetazzo/orchestration-workshop/blob/master/dockercoins/docker-compose.yml) ...
https://github.com/jpetazzo/container.training/blob/master/dockercoins/docker-compose.yml) ...
... and 4 other services, each in its own directory:
@@ -39,6 +39,43 @@ class: extra-details
---
## Service discovery in container-land
- We do not hard-code IP addresses in the code
- We do not hard-code FQDN in the code, either
- We just connect to a service name, and container-magic does the rest
(And by container-magic, we mean "a crafty, dynamic, embedded DNS server")
---
## Example in `worker/worker.py`
```python
redis = Redis("`redis`")
def get_random_bytes():
r = requests.get("http://`rng`/32")
return r.content
def hash_bytes(data):
r = requests.post("http://`hasher`/",
data=data,
headers={"Content-Type": "application/octet-stream"})
```
(Full source code available [here](
https://github.com/jpetazzo/container.training/blob/8279a3bce9398f7c1a53bdd95187c53eda4e6435/dockercoins/worker/worker.py#L17
))
---
class: extra-details
## Links, naming, and service discovery
- Containers can have network aliases (resolvable through DNS)
@@ -47,37 +84,21 @@ class: extra-details
- Compose file version 1 did require "links" sections
- Our code can connect to services using their short name
(instead of e.g. IP address or FQDN)
- Network aliases are automatically namespaced
(i.e. you can have multiple apps declaring and using a service named `database`)
- you can have multiple apps declaring and using a service named `database`
---
- containers in the blue app will resolve `database` to the IP of the blue database
## Example in `worker/worker.py`
![Service discovery](images/service-discovery.png)
- containers in the green app will resolve `database` to the IP of the green database
---
## What's this application?
---
--
class: pic
![DockerCoins logo](images/dockercoins.png)
(DockerCoins 2016 logo courtesy of [@XtlCnslt](https://twitter.com/xtlcnslt) and [@ndeloof](https://twitter.com/ndeloof). Thanks!)
---
## What's this application?
- It is a DockerCoin miner! 💰🐳📦🚢
- It is a DockerCoin miner! .emoji[💰🐳📦🚢]
--
@@ -109,15 +130,15 @@ class: pic
<!--
```bash
if [ -d orchestration-workshop ]; then
mv orchestration-workshop orchestration-workshop.$$
if [ -d container.training ]; then
mv container.training container.training.$$
fi
```
-->
- Clone the repository on `node1`:
```bash
git clone git://github.com/jpetazzo/orchestration-workshop
git clone git://github.com/jpetazzo/container.training
```
]
@@ -134,7 +155,7 @@ Without further ado, let's start our application.
- Go to the `dockercoins` directory, in the cloned repo:
```bash
cd ~/orchestration-workshop/dockercoins
cd ~/container.training/dockercoins
```
- Use Compose to build and run all containers:
@@ -143,8 +164,7 @@ Without further ado, let's start our application.
```
<!--
```wait units of work done```
```keys ^C```
```longwait units of work done```
-->
]
@@ -155,100 +175,22 @@ and displays aggregated logs.
---
## Lots of logs
## Our application at work
- The application continuously generates logs
- On the left-hand side, the "rainbow strip" shows the container names
- On the right-hand side, we see the output of our containers
- We can see the `worker` service making requests to `rng` and `hasher`
- Let's put that in the background
.exercise[
- Stop the application by hitting `^C`
]
- `^C` stops all containers by sending them the `TERM` signal
- Some containers exit immediately, others take longer
<br/>(because they don't handle `SIGTERM` and end up being killed after a 10s timeout)
---
## Restarting in the background
- Many flags and commands of Compose are modeled after those of `docker`
.exercise[
- Start the app in the background with the `-d` option:
```bash
docker-compose up -d
```
- Check that our app is running with the `ps` command:
```bash
docker-compose ps
```
]
`docker-compose ps` also shows the ports exposed by the application.
---
class: extra-details
## Viewing logs
- The `docker-compose logs` command works like `docker logs`
.exercise[
- View all logs since container creation and exit when done:
```bash
docker-compose logs
```
- Stream container logs, starting at the last 10 lines for each container:
```bash
docker-compose logs --tail 10 --follow
```
<!--
```wait units of work done```
```keys ^C```
-->
]
Tip: use `^S` and `^Q` to pause/resume log output.
---
class: extra-details
## Upgrading from Compose 1.6
.warning[The `logs` command has changed between Compose 1.6 and 1.7!]
- Up to 1.6
- `docker-compose logs` is the equivalent of `logs --follow`
- `docker-compose logs` must be restarted if containers are added
- Since 1.7
- `--follow` must be specified explicitly
- new containers are automatically picked up by `docker-compose logs`
- For `rng` and `hasher`, we see HTTP access logs
---
## Connecting to the web UI
- "Logs are exciting and fun!" (No-one, ever)
- The `webui` container exposes a web dashboard; let's view it
.exercise[
@@ -263,11 +205,31 @@ class: extra-details
]
You should see a speed of approximately 4 hashes/second.
A drawing area should show up, and after a few seconds, a blue
graph will appear.
More precisely: 4 hashes/second, with regular dips down to zero.
<br/>This is because Jérôme is incapable of writing good frontend code.
<br/>Don't ask. Seriously, don't ask. This is embarrassing.
---
class: self-paced, extra-details
## If the graph doesn't load
If you just see a `Page not found` error, it might be because your
Docker Engine is running on a different machine. This can be the case if:
- you are using the Docker Toolbox
- you are using a VM (local or remote) created with Docker Machine
- you are controlling a remote Docker Engine
When you run DockerCoins in development mode, the web UI static files
are mapped to the container using a volume. Alas, volumes can only
work on a local environment, or when using Docker4Mac or Docker4Windows.
How to fix this?
Stop the app with `^C`, edit `dockercoins.yml`, comment out the `volumes` section, and try again.
---
@@ -275,203 +237,67 @@ class: extra-details
## Why does the speed seem irregular?
- It *looks like* the speed is approximately 4 hashes/second
- Or more precisely: 4 hashes/second, with regular dips down to zero
- Why?
--
class: extra-details
- The app actually has a constant, steady speed: 3.33 hashes/second
<br/>
(which corresponds to 1 hash every 0.3 seconds, for *reasons*)
- Yes, and?
---
class: extra-details
## The reason why this graph is *not awesome*
- The worker doesn't update the counter after every loop, but up to once per second
- The speed is computed by the browser, checking the counter about once per second
- Between two consecutive updates, the counter will increase either by 4, or by 0
- The perceived speed will therefore be 4 - 4 - 4 - 0 - 4 - 4 - etc.
- The perceived speed will therefore be 4 - 4 - 4 - 0 - 4 - 4 - 0 etc.
*We told you to not ask!!!*
---
## Scaling up the application
- Our goal is to make that performance graph go up (without changing a line of code!)
- What can we conclude from this?
--
- Before trying to scale the application, we'll figure out if we need more resources
class: extra-details
(CPU, RAM...)
- For that, we will use good old UNIX tools on our Docker node
- "I'm clearly incapable of writing good frontend code!" 😀 — Jérôme
---
## Looking at resource usage
## Stopping the application
- Let's look at CPU, memory, and I/O usage
- If we interrupt Compose (with `^C`), it will politely ask the Docker Engine to stop the app
- The Docker Engine will send a `TERM` signal to the containers
- If the containers do not exit in a timely manner, the Engine sends a `KILL` signal
.exercise[
- run `top` to see CPU and memory usage (you should see idle cycles)
- Stop the application by hitting `^C`
<!--
```bash top```
```wait Tasks```
```keys ^C```
-->
- run `vmstat 1` to see I/O usage (si/so/bi/bo)
<br/>(the 4 numbers should be almost zero, except `bo` for logging)
<!--
```bash vmstat 1```
```wait memory```
```keys ^C```
-->
]
We have available resources.
- Why?
- How can we use them?
---
## Scaling workers on a single node
- Docker Compose supports scaling
- Let's scale `worker` and see what happens!
.exercise[
- Start one more `worker` container:
```bash
docker-compose scale worker=2
```
- Look at the performance graph (it should show a x2 improvement)
- Look at the aggregated logs of our containers (`worker_2` should show up)
- Look at the impact on CPU load with e.g. top (it should be negligible)
]
---
## Adding more workers
- Great, let's add more workers and call it a day, then!
.exercise[
- Start eight more `worker` containers:
```bash
docker-compose scale worker=10
```
- Look at the performance graph: does it show a x10 improvement?
- Look at the aggregated logs of our containers
- Look at the impact on CPU load and memory usage
]
---
# Identifying bottlenecks
- You should have seen a 3x speed bump (not 10x)
- Adding workers didn't result in linear improvement
- *Something else* is slowing us down
--
- ... But what?
Some containers exit immediately, others take longer.
--
The containers that do not handle `SIGTERM` end up being killed after a 10s timeout.
- The code doesn't have instrumentation
- Let's use state-of-the-art HTTP performance analysis!
<br/>(i.e. good old tools like `ab`, `httping`...)
---
## Accessing internal services
- `rng` and `hasher` are exposed on ports 8001 and 8002
- This is declared in the Compose file:
```yaml
...
rng:
build: rng
ports:
- "8001:80"
hasher:
build: hasher
ports:
- "8002:80"
...
```
---
## Measuring latency under load
We will use `httping`.
.exercise[
- Check the latency of `rng`:
```bash
httping -c 10 localhost:8001
```
- Check the latency of `hasher`:
```bash
httping -c 10 localhost:8002
```
]
`rng` has a much higher latency than `hasher`.
---
## Let's draw hasty conclusions
- The bottleneck seems to be `rng`
- *What if* we don't have enough entropy and can't generate enough random numbers?
- We need to scale out the `rng` service on multiple machines!
Note: this is a fiction! We have enough entropy. But we need a pretext to scale out.
(In fact, the code of `rng` uses `/dev/urandom`, which never runs out of entropy...
<br/>
...and is [just as good as `/dev/random`](http://www.slideshare.net/PacSecJP/filippo-plain-simple-reality-of-entropy).)
---
## Clean up
- Before moving on, let's remove those containers
.exercise[
- Tell Compose to remove everything:
```bash
docker-compose down
```
]

11
slides/common/thankyou.md Normal file
View File

@@ -0,0 +1,11 @@
class: title, self-paced
Thank you!
---
class: title, in-person
That's all, folks! <br/> Questions?
![end](images/end.jpg)

21
slides/common/title.md Normal file
View File

@@ -0,0 +1,21 @@
class: title, self-paced
@@TITLE@@
.nav[*Self-paced version*]
---
class: title, in-person
@@TITLE@@<br/></br>
.footnote[
**Be kind to the WiFi!**<br/>
<!-- *Use the 5G network.* -->
*Don't use your hotspot.*<br/>
*Don't stream videos or download big files during the workshop.*<br/>
*Thank you!*
**Slides: http://container.training/**
]

4
slides/common/toc.md Normal file
View File

@@ -0,0 +1,4 @@
@@TOC@@

View File

@@ -1,182 +0,0 @@
chat: "[Slack](https://dockercommunity.slack.com/messages/C7ET1GY4Q)"
exclude:
- self-paced
- snap
- auto-btp
- benchmarking
- elk-manual
- prom-manual
title: "Swarm: from Zero to Hero (DC17EU)"
chapters:
- |
class: title
.small[
Swarm: from Zero to Hero
.small[.small[
**Be kind to the WiFi!**
*Use the 5G network*
<br/>
*Don't use your hotspot*
<br/>
*Don't stream videos from YouTube, Netflix, etc.
<br/>(if you're bored, watch local content instead)*
Also: share the power outlets
<br/>
*(with limited power comes limited responsibility?)*
<br/>
*(or something?)*
Thank you!
]
]
]
---
## Intros
<!--
- Hello! We are
AJ ([@s0ulshake](https://twitter.com/s0ulshake))
&
Jérôme ([@jpetazzo](https://twitter.com/jpetazzo))
-->
- Hello! We are Jérôme, Lee, Nicholas, and Scott
<!--
I am
Jérôme ([@jpetazzo](https://twitter.com/jpetazzo))
-->
--
- This is our collective Docker knowledge:
![Bell Curve](images/bell-curve.jpg)
---
## "From zero to hero"
--
- It rhymes, but it's a pretty bad title, to be honest
--
- None of you is a "zero"
--
- None of us is a "hero"
--
- None of us should even try to be a hero
--
*The hero syndrome is a phenomenon affecting people who seek heroism or recognition,
usually by creating a desperate situation which they can resolve.
This can include unlawful acts, such as arson.
The phenomenon has been noted to affect civil servants,
such as firefighters, nurses, police officers, and security guards.*
(Wikipedia page on [hero syndrome](https://en.wikipedia.org/wiki/Hero_syndrome))
---
## Agenda
.small[
- 09:00-09:10 Hello!
- 09:10-10:30 Part 1
- 10:30-11:00 coffee break
- 11:00-12:30 Part 2
- 12:30-13:30 lunch break
- 13:30-15:00 Part 3
- 15:00-15:30 coffee break
- 15:30-17:00 Part 4
- 17:00-18:00 Afterhours and Q&A
]
<!--
- The tutorial will run from 9:00am to 12:20pm
- This will be fast-paced, but DON'T PANIC!
- There will be a coffee break at 10:30am
<br/>
(please remind me if I forget about it!)
-->
- All the content is publicly available (slides, code samples, scripts)
Upstream URL: https://github.com/jpetazzo/orchestration-workshop
- Feel free to interrupt for questions at any time
- Live feedback, questions, help on [Gitter](chat)
http://container.training/chat
- swarm/intro.md
- |
@@TOC@@
- - swarm/prereqs.md
- swarm/versions.md
- |
class: title
All right!
<br/>
We're all set.
<br/>
Let's do this.
- common/sampleapp.md
- swarm/swarmkit.md
- swarm/creatingswarm.md
- swarm/morenodes.md
- - swarm/firstservice.md
- swarm/ourapponswarm.md
- swarm/updatingservices.md
- swarm/healthchecks.md
- - swarm/operatingswarm.md
- swarm/netshoot.md
- swarm/ipsec.md
- swarm/swarmtools.md
- swarm/security.md
- swarm/secrets.md
- swarm/encryptionatrest.md
- swarm/leastprivilege.md
- swarm/apiscope.md
- - swarm/logging.md
- swarm/metrics.md
- swarm/stateful.md
- swarm/extratips.md
- swarm/end.md
- |
class: title
That's all folks! <br/> Questions?
.small[.small[
Jérôme ([@jpetazzo](https://twitter.com/jpetazzo)) — [@docker](https://twitter.com/docker)
]]
<!--
Tiffany ([@tiffanyfayj](https://twitter.com/tiffanyfayj))
AJ ([@s0ulshake](https://twitter.com/s0ulshake))
-->

2
slides/find-non-ascii.sh Executable file
View File

@@ -0,0 +1,2 @@
#!/bin/sh
grep --color=auto -P -n "[^\x00-\x80]" */*.md

View File

Before

Width:  |  Height:  |  Size: 84 KiB

After

Width:  |  Height:  |  Size: 84 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 64 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 187 KiB

View File

Before

Width:  |  Height:  |  Size: 68 KiB

After

Width:  |  Height:  |  Size: 68 KiB

View File

Before

Width:  |  Height:  |  Size: 41 KiB

After

Width:  |  Height:  |  Size: 41 KiB

View File

Before

Width:  |  Height:  |  Size: 174 KiB

After

Width:  |  Height:  |  Size: 174 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 144 KiB

View File

Before

Width:  |  Height:  |  Size: 350 KiB

After

Width:  |  Height:  |  Size: 350 KiB

View File

Before

Width:  |  Height:  |  Size: 155 KiB

After

Width:  |  Height:  |  Size: 155 KiB

View File

Before

Width:  |  Height:  |  Size: 230 KiB

After

Width:  |  Height:  |  Size: 230 KiB

View File

Before

Width:  |  Height:  |  Size: 62 KiB

After

Width:  |  Height:  |  Size: 62 KiB

View File

Before

Width:  |  Height:  |  Size: 203 KiB

After

Width:  |  Height:  |  Size: 203 KiB

View File

Before

Width:  |  Height:  |  Size: 122 KiB

After

Width:  |  Height:  |  Size: 122 KiB

View File

Before

Width:  |  Height:  |  Size: 174 KiB

After

Width:  |  Height:  |  Size: 174 KiB

View File

Before

Width:  |  Height:  |  Size: 64 KiB

After

Width:  |  Height:  |  Size: 64 KiB

View File

Before

Width:  |  Height:  |  Size: 97 KiB

After

Width:  |  Height:  |  Size: 97 KiB

View File

Before

Width:  |  Height:  |  Size: 927 KiB

After

Width:  |  Height:  |  Size: 927 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 72 KiB

View File

Before

Width:  |  Height:  |  Size: 9.8 KiB

After

Width:  |  Height:  |  Size: 9.8 KiB

View File

Before

Width:  |  Height:  |  Size: 190 KiB

After

Width:  |  Height:  |  Size: 190 KiB

View File

Before

Width:  |  Height:  |  Size: 49 KiB

After

Width:  |  Height:  |  Size: 49 KiB

View File

Before

Width:  |  Height:  |  Size: 25 KiB

After

Width:  |  Height:  |  Size: 25 KiB

View File

Before

Width:  |  Height:  |  Size: 80 KiB

After

Width:  |  Height:  |  Size: 80 KiB

View File

Before

Width:  |  Height:  |  Size: 101 KiB

After

Width:  |  Height:  |  Size: 101 KiB

View File

Before

Width:  |  Height:  |  Size: 30 KiB

After

Width:  |  Height:  |  Size: 30 KiB

View File

Before

Width:  |  Height:  |  Size: 49 KiB

After

Width:  |  Height:  |  Size: 49 KiB

View File

Before

Width:  |  Height:  |  Size: 595 KiB

After

Width:  |  Height:  |  Size: 595 KiB

View File

Before

Width:  |  Height:  |  Size: 43 KiB

After

Width:  |  Height:  |  Size: 43 KiB

View File

Before

Width:  |  Height:  |  Size: 85 KiB

After

Width:  |  Height:  |  Size: 85 KiB

View File

Before

Width:  |  Height:  |  Size: 28 KiB

After

Width:  |  Height:  |  Size: 28 KiB

View File

Before

Width:  |  Height:  |  Size: 25 KiB

After

Width:  |  Height:  |  Size: 25 KiB

View File

Before

Width:  |  Height:  |  Size: 42 KiB

After

Width:  |  Height:  |  Size: 42 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 75 KiB

View File

@@ -1,29 +1,178 @@
<ul style="font-size:2em;">
<li>
LISA17 M7: Getting Started with Docker and Containers
<a href="intro.yml.html">(slides)</a>
<a href="https://gitter.im/jpetazzo/workshop-20171030-sanfrancisco">(chat)</a>
</li>
<li>
LISA17 T9: Build, Ship, and Run Microservices on a Docker Swarm Cluster
<a href="lisa.yml.html">(slides)</a>
<a href="https://gitter.im/jpetazzo/workshop-20171031-sanfrancisco">(chat)</a>
</li>
<li>
Deploying and scaling microservices with Docker and Kubernetes
<a href="kube.yml.html">(slides)</a>
<a href="https://gitter.im/jpetazzo/workshop-20171026-prague">(chat)</a>
</li>
<li>
DockerCon Workshop: from Zero to Hero (full day, B3 M1-2)
<a href="dockercon.yml.html">(slides)</a>
</li>
<li>
DockerCon Workshop: Orchestration for Advanced Users (afternoon, B4 M5-6)
<a href="https://www.bretfisher.com/dockercon17eu/">(slides)</a>
</li>
<li>
Self-paced Workshop: Orchestrating Microservices with Docker and Swarm
<a href="selfpaced.yml.html">(slides)</a>
</li>
</ul>
<html>
<head>
<title>Container Training</title>
<style type="text/css">
body {
background-image: url("images/container-background.jpg");
max-width: 1024px;
margin: 0 auto;
}
table {
font-size: 20px;
font-family: sans-serif;
background: white;
width: 100%;
height: 100%;
padding: 20px;
}
.header {
font-size: 300%;
font-weight: bold;
}
.title {
font-size: 150%;
font-weight: bold;
}
td {
padding: 1px;
height: 1em;
}
td.spacer {
height: unset;
}
td.footer {
padding-top: 80px;
height: 100px;
}
td.title {
border-bottom: thick solid black;
padding-bottom: 2px;
padding-top: 20px;
}
a {
text-decoration: none;
}
a:hover {
background: yellow;
}
a.attend:after {
content: "📅 attend";
}
a.slides:after {
content: "📚 slides";
}
a.chat:after {
content: "💬 chat";
}
a.video:after {
content: "📺 video";
}
</style>
</head>
<body>
<div class="main">
<table>
<tr><td class="header" colspan="4">Container Training</td></tr>
<tr><td class="title" colspan="4">Coming soon at a conference near you</td></tr>
<tr>
<!--
<td>Nothing for now (stay tuned...)</td>
thing for now (stay tuned...)</td>
-->
<td>March 14, 2018: Boosterconf — Kubernetes 101</td>
<td>&nbsp;</td>
<td><a class="attend" href="https://2018.boosterconf.no/talks/1179" />
</tr>
<tr>
<td>March 27, 2018: SREcon Americas — Kubernetes 101</td>
<td>&nbsp;</td>
<td><a class="attend" href="https://www.usenix.org/conference/srecon18americas/presentation/kromhout" />
</tr>
<tr><td class="title" colspan="4">Past workshops</td></tr>
<tr>
<!-- February 22, 2018 -->
<td>IndexConf: Kubernetes 101</td>
<td><a class="slides" href="http://indexconf2018.container.training/" /></td>
<!--
<td><a class="attend" href="https://developer.ibm.com/indexconf/sessions/#!?id=5474" />
-->
</tr>
<tr>
<td>Kubernetes enablement at Docker</td>
<td><a class="slides" href="http://kube.container.training/" /></td>
</tr>
<tr>
<td>QCON SF: Orchestrating Microservices with Docker Swarm</td>
<td><a class="slides" href="http://qconsf2017swarm.container.training/" /></td>
</tr>
<tr>
<td>QCON SF: Introduction to Docker and Containers</td>
<td><a class="slides" href="http://qconsf2017intro.container.training/" /></td>
<td><a class="video" href="https://www.youtube.com/playlist?list=PLBAFXs0YjviLgqTum8MkspG_8VzGl6C07" /></td>
</tr>
<tr>
<td>LISA17 M7: Getting Started with Docker and Containers</td>
<td><a class="slides" href="http://lisa17m7.container.training/" /></td>
</tr>
<tr>
<td>LISA17 T9: Build, Ship, and Run Microservices on a Docker Swarm Cluster</td>
<td><a class="slides" href="http://lisa17t9.container.training/" /></td>
</tr>
<tr>
<td>Deploying and scaling microservices with Docker and Kubernetes</td>
<td><a class="slides" href="http://osseu17.container.training/" /></td>
<td><a class="video" href="https://www.youtube.com/playlist?list=PLBAFXs0YjviLrsyydCzxWrIP_1-wkcSHS" /></td>
</tr>
<tr>
<td>DockerCon Workshop: from Zero to Hero (full day, B3 M1-2)</td>
<td><a class="slides" href="http://dc17eu.container.training/" /></td>
</tr>
<tr>
<td>DockerCon Workshop: Orchestration for Advanced Users (afternoon, B4 M5-6)</td>
<td><a class="slides" href="https://www.bretfisher.com/dockercon17eu/" /></td>
</tr>
<tr>
<td>LISA16 T1: Deploying and Scaling Applications with Docker Swarm</td>
<td><a class="slides" href="http://lisa16t1.container.training/" /></td>
<td><a class="video" href="https://www.youtube.com/playlist?list=PLBAFXs0YjviIDDhr8vIwCN1wkyNGXjbbc" /></td>
</tr>
<tr>
<td>PyCon2016: Introduction to Docker and containers</td>
<td><a class="slides" href="https://us.pycon.org/2016/site_media/media/tutorial_handouts/DockerSlides.pdf" /></td>
<td><a class="video" href="https://www.youtube.com/watch?v=ZVaRK10HBjo" /></td>
</tr>
<tr><td class="title" colspan="4">Self-paced tutorials</td></tr>
<tr>
<td>Introduction to Docker and Containers</td>
<td><a class="slides" href="intro-fullday.yml.html" /></td>
</tr>
<tr>
<td>Container Orchestration with Docker and Swarm</td>
<td><a class="slides" href="swarm-selfpaced.yml.html" /></td>
</tr>
<tr>
<td>Deploying and Scaling Microservices with Docker and Kubernetes</td>
<td><a class="slides" href="kube-halfday.yml.html" /></td>
</tr>
<tr><td class="spacer"></td></tr>
<tr>
<td class="footer">
Maintained by Jérôme Petazzoni (<a href="https://twitter.com/jpetazzo">@jpetazzo</a>)
</td>
</tr>
</table>
</div>
</body>
</html>

15
slides/interstitials.txt Normal file
View File

@@ -0,0 +1,15 @@
https://gallant-turing-d0d520.netlify.com/containers/Container-Ship-Freighter-Navigation-Elbe-Romance-1782991.jpg
https://gallant-turing-d0d520.netlify.com/containers/ShippingContainerSFBay.jpg
https://gallant-turing-d0d520.netlify.com/containers/aerial-view-of-containers.jpg
https://gallant-turing-d0d520.netlify.com/containers/blue-containers.jpg
https://gallant-turing-d0d520.netlify.com/containers/chinook-helicopter-container.jpg
https://gallant-turing-d0d520.netlify.com/containers/container-cranes.jpg
https://gallant-turing-d0d520.netlify.com/containers/container-housing.jpg
https://gallant-turing-d0d520.netlify.com/containers/containers-by-the-water.jpg
https://gallant-turing-d0d520.netlify.com/containers/distillery-containers.jpg
https://gallant-turing-d0d520.netlify.com/containers/lots-of-containers.jpg
https://gallant-turing-d0d520.netlify.com/containers/plastic-containers.JPG
https://gallant-turing-d0d520.netlify.com/containers/train-of-containers-1.jpg
https://gallant-turing-d0d520.netlify.com/containers/train-of-containers-2.jpg
https://gallant-turing-d0d520.netlify.com/containers/two-containers-on-a-truck.jpg
https://gallant-turing-d0d520.netlify.com/containers/wall-of-containers.jpeg

44
slides/intro-fullday.yml Normal file
View File

@@ -0,0 +1,44 @@
title: |
Introduction
to Docker and
Containers
chat: "[Slack](https://dockercommunity.slack.com/messages/C7GKACWDV)"
#chat: "[Gitter](https://gitter.im/jpetazzo/workshop-yyyymmdd-city)"
exclude:
- self-paced
chapters:
- common/title.md
- logistics.md
- intro/intro.md
- common/about-slides.md
- common/toc.md
- - intro/Docker_Overview.md
#- intro/Docker_History.md
- intro/Training_Environment.md
- intro/Installing_Docker.md
- intro/First_Containers.md
- intro/Background_Containers.md
- intro/Start_And_Attach.md
- - intro/Initial_Images.md
- intro/Building_Images_Interactively.md
- intro/Building_Images_With_Dockerfiles.md
- intro/Cmd_And_Entrypoint.md
- intro/Copying_Files_During_Build.md
- intro/Multi_Stage_Builds.md
- intro/Publishing_To_Docker_Hub.md
- intro/Dockerfile_Tips.md
- - intro/Naming_And_Inspecting.md
- intro/Container_Networking_Basics.md
- intro/Network_Drivers.md
- intro/Container_Network_Model.md
#- intro/Connecting_Containers_With_Links.md
- intro/Ambassadors.md
- - intro/Local_Development_Workflow.md
- intro/Working_With_Volumes.md
- intro/Compose_For_Dev_Stacks.md
- intro/Advanced_Dockerfiles.md
- common/thankyou.md
- intro/links.md

View File

@@ -0,0 +1,44 @@
title: |
Introduction
to Docker and
Containers
chat: "[Slack](https://dockercommunity.slack.com/messages/C7GKACWDV)"
#chat: "[Gitter](https://gitter.im/jpetazzo/workshop-yyyymmdd-city)"
exclude:
- in-person
chapters:
- common/title.md
# - common/logistics.md
- intro/intro.md
- common/about-slides.md
- common/toc.md
- - intro/Docker_Overview.md
#- intro/Docker_History.md
- intro/Training_Environment.md
- intro/Installing_Docker.md
- intro/First_Containers.md
- intro/Background_Containers.md
- intro/Start_And_Attach.md
- - intro/Initial_Images.md
- intro/Building_Images_Interactively.md
- intro/Building_Images_With_Dockerfiles.md
- intro/Cmd_And_Entrypoint.md
- intro/Copying_Files_During_Build.md
- intro/Multi_Stage_Builds.md
- intro/Publishing_To_Docker_Hub.md
- intro/Dockerfile_Tips.md
- - intro/Naming_And_Inspecting.md
- intro/Container_Networking_Basics.md
- intro/Network_Drivers.md
- intro/Container_Network_Model.md
#- intro/Connecting_Containers_With_Links.md
- intro/Ambassadors.md
- - intro/Local_Development_Workflow.md
- intro/Working_With_Volumes.md
- intro/Compose_For_Dev_Stacks.md
- intro/Advanced_Dockerfiles.md
- common/thankyou.md
- intro/links.md

View File

@@ -1,91 +0,0 @@
title: "LISA17 M7: Getting Started with Docker and Containers"
chat: "[Slack](https://usenix-lisa.slack.com/messages/C0E6N1NJW)"
chapters:
- |
class: title
.small[
LISA17 M7
Getting Started <br/> with Docker and Containers
.small[.small[
**Be kind to the WiFi!**
*Use the 5G network*
<br/>
*Don't use your hotspot*
<br/>
*Don't stream videos from YouTube, Netflix, etc.
<br/>(if you're bored, watch local content instead)*
<!--
Also: share the power outlets
<br/>
*(with limited power comes limited responsibility?)*
<br/>
*(or something?)*
-->
Thank you!
]
]
]
---
## Logistics
- Hello! We are
Jérôme ([@jpetazzo](https://twitter.com/jpetazzo), Docker Inc.)
&
AJ ([@s0ulshake](https://twitter.com/s0ulshake), Travis CI)
- The tutorial will run from 1:30pm to 5:00pm
- This will be fast-paced, but DON'T PANIC!
- There will be a coffee break at 3:00pm
<br/>
(please remind me if I forget about it!)
- All the content is publicly available
One URL to remember: http://container.training
- Feel free to interrupt for questions at any time
- Live feedback, questions, help on @@CHAT@@
- |
@@TOC@@
- - intro/Docker_Overview.md
#- intro/Docker_History.md
- intro/Training_Environment.md
- intro/Install_Docker.md
- intro/First_Containers.md
- intro/Background_Containers.md
- intro/Start_And_Attach.md
- - intro/Initial_Images.md
- intro/Building_Images_Interactively.md
- intro/Building_Images_With_Dockerfiles.md
- intro/Cmd_And_Entrypoint.md
- intro/Copying_Files_During_Build.md
- intro/Multi_Stage_Builds.md
- intro/Dockerfile_Tips.md
#- intro/Advanced_Dockerfiles.md
- intro/Docker_Hub_Tease.md
- - intro/Naming_And_Inspecting.md
- intro/Container_Networking_Basics.md
- intro/Container_Network_Model.md
#- intro/Connecting_Containers_With_Links.md
- intro/Ambassadors.md
- - intro/Local_Development_Workflow.md
- intro/Working_With_Volumes.md
- intro/Compose_For_Dev_Stacks.md
- intro/Course_Conclusion.md

View File

@@ -1,9 +1,6 @@
class: title
# Advanced Dockerfiles
![construction](images/construction.jpg)
![construction](images/title-advanced-dockerfiles.jpg)
---

View File

@@ -3,7 +3,7 @@ class: title
# Ambassadors
![](images/ambassador.jpg)
![Two serious-looking persons shaking hands](images/title-ambassador.jpg)
---
@@ -40,7 +40,7 @@ ambassador containers.
---
![ambassador](images/diagram.png)
![ambassador](images/ambassador-diagram.png)
---

View File

@@ -1,9 +1,9 @@
class: title
# Background Containers
# Background containers
![Background Containers](images/background-containers.jpg)
![Background containers](images/title-background-containers.jpg)
---

View File

@@ -1,4 +1,4 @@
# Building Images Interactively
# Building images interactively
In this section, we will create our first container image.
@@ -16,27 +16,21 @@ We will:
---
## Building Images Interactively
## The plan
As we have seen, the images on the Docker Hub are sometimes very basic.
1. Create a container (with `docker run`) using our base distro of choice.
How do we want to construct our own images?
2. Run a bunch of commands to install and set up our software in the container.
As an example, we will build an image that has `figlet`.
3. (Optionally) review changes in the container with `docker diff`.
First, we will do it manually with `docker commit`.
4. Turn the container into a new image with `docker commit`.
Then, in an upcoming chapter, we will use a `Dockerfile` and `docker build`.
5. (Optionally) add tags to the image with `docker tag`.
---
## Building from a base
Our base will be the `ubuntu` image.
---
## Create a new container and make some changes
## Setting up our container
Start an Ubuntu container:
@@ -107,7 +101,7 @@ As explained before:
---
## Commit and run your image
## Commit our changes into a new image
The `docker commit` command will create a new layer with those changes,
and a new image using this new layer.
@@ -119,7 +113,13 @@ $ docker commit <yourContainerId>
The output of the `docker commit` command will be the ID for your newly created image.
We can run this image:
We can use it as an argument to `docker run`.
---
## Testing our new image
Let's run this image:
```bash
$ docker run -it <newImageId>
@@ -131,6 +131,8 @@ root@fcfb62f0bfde:/# figlet hello
|_| |_|\___|_|_|\___/
```
It works! .emoji[🎉]
---
## Tagging images

View File

@@ -3,7 +3,7 @@ class: title
# Building Docker images with a Dockerfile
![construction](images/construction.jpg)
![Construction site with containers](images/title-building-docker-images-with-a-dockerfile.jpg)
---
@@ -188,7 +188,7 @@ root@91f3c974c9a1:/# figlet hello
```
Yay! 🎉
Yay! .emoji[🎉]
---

View File

@@ -1,7 +1,7 @@
class: title
# CMD and ENTRYPOINT
# `CMD` and `ENTRYPOINT`
![Container entry doors](images/entrypoint.jpg)
@@ -141,7 +141,7 @@ Why did we use JSON syntax for our `ENTRYPOINT`?
* When CMD or ENTRYPOINT use string syntax, they get wrapped in `sh -c`.
* To avoid this wrapping, you must use JSON syntax.
* To avoid this wrapping, we can use JSON syntax.
What if we used `ENTRYPOINT` with string syntax?
@@ -178,8 +178,6 @@ $ docker run figlet salut
\/ \_/|_/|__/ \_/|_/|_/
```
Great success!
---
## Using `CMD` and `ENTRYPOINT` together
@@ -227,9 +225,8 @@ $ docker build -t figlet .
Successfully built 6e0b6a048a07
```
And run it:
Run it without parameters:
.small[
```bash
$ docker run figlet
_ _ _ _
@@ -237,7 +234,15 @@ $ docker run figlet
| | _ | | | | __ __ ,_ | | __|
|/ \ |/ |/ |/ / \_ | | |_/ \_/ | |/ / |
| |_/|__/|__/|__/\__/ \/ \/ \__/ |_/|__/\_/|_/
```
---
## Overriding the image default parameters
Now let's pass extra arguments to the image.
```bash
$ docker run figlet hola mundo
_ _
| | | | |
@@ -245,7 +250,8 @@ $ docker run figlet hola mundo
|/ \ / \_|/ / | / |/ |/ | | | / |/ | / | / \_
| |_/\__/ |__/\_/|_/ | | |_/ \_/|_/ | |_/\_/|_/\__/
```
]
We overrode `CMD` but still used `ENTRYPOINT`.
---

View File

@@ -1,5 +1,4 @@
# Compose For Development Stacks
# Compose for development stacks
Dockerfiles are great to build container images.
@@ -113,6 +112,7 @@ them.
Here is the file used in the demo:
.small[
```yaml
version: "2"
@@ -131,6 +131,7 @@ services:
redis:
image: redis
```
]
---

View File

@@ -1,9 +1,9 @@
class: title
# Connecting Containers With Links
# Connecting containers with links
![graph](images/graph.gif)
![graph](images/title-connecting-containers-with-links.gif)
---

View File

@@ -3,7 +3,7 @@ class: title
# The Container Network Model
![A denser graph network](images/complex-network.jpg)
![A denser graph network](images/title-the-container-network-model.jpg)
---
@@ -126,13 +126,16 @@ $ docker run -d --name es --net dev elasticsearch:2
Now, create another container on this network.
.small[
```bash
$ docker run -ti --net dev alpine sh
root@0ecccdfa45ef:/#
```
]
From this new container, we can resolve and ping the other one, using its assigned name:
.small[
```bash
/ # ping es
PING es (172.18.0.2) 56(84) bytes of data.
@@ -145,6 +148,7 @@ PING es (172.18.0.2) 56(84) bytes of data.
rtt min/avg/max/mdev = 0.114/0.149/0.221/0.052 ms
root@0ecccdfa45ef:/#
```
]
---
@@ -155,6 +159,7 @@ class: extra-details
In Docker Engine 1.9, name resolution is implemented with `/etc/hosts`, and
updating it each time containers are added/removed.
.small[
```bash
[root@0ecccdfa45ef /]# cat /etc/hosts
172.18.0.3 0ecccdfa45ef
@@ -167,6 +172,7 @@ ff02::2 ip6-allrouters
172.18.0.2 es
172.18.0.2 es.dev
```
]
In Docker Engine 1.10, this has been replaced by a dynamic resolver.
@@ -174,7 +180,7 @@ In Docker Engine 1.10, this has been replaced by a dynamic resolver.
---
## Connecting multiple containers together
# Service discovery with containers
* Let's try to run an application that requires two containers.
@@ -210,9 +216,7 @@ $ docker ps -l
* If we connect to the application now, we will see an error page:
.small[
![Trainingwheels error](images/trainingwheels-error.png)
]
* This is because the Redis service is not running.
* This container tries to resolve the name `redis`.
@@ -241,9 +245,7 @@ $ docker run --net dev --name redis -d redis
* If we connect to the application now, we should see that the app is working correctly:
.small[
![Trainingwheels OK](images/trainingwheels-ok.png)
]
* When the app tries to resolve `redis`, instead of getting a DNS error, it gets the IP address of our Redis container.
@@ -362,6 +364,7 @@ Each ElasticSearch instance has a name (generated when it is started). This name
Try the following command a few times:
.small[
```bash
$ docker run --rm --net dev centos curl -s es:9200
{
@@ -369,9 +372,11 @@ $ docker run --rm --net dev centos curl -s es:9200
...
}
```
]
Then try it a few times by replacing `--net dev` with `--net prod`:
.small[
```bash
$ docker run --rm --net prod centos curl -s es:9200
{
@@ -379,6 +384,7 @@ $ docker run --rm --net prod centos curl -s es:9200
...
}
```
]
---
@@ -486,7 +492,7 @@ Very short instructions:
- `docker network create mynet --driver overlay`
- `docker service create --network mynet myimage`
See http://jpetazzo.github.io/orchestration-workshop for all the deets about clustering!
See http://jpetazzo.github.io/container.training for all the deets about clustering!
---

View File

@@ -1,9 +1,9 @@
class: title
# Container Networking Basics
# Container networking basics
![A dense graph network](images/network.jpg)
![A dense graph network](images/title-container-networking-basics.jpg)
---
@@ -69,7 +69,7 @@ But first, let's make sure that everything works properly.
Point your browser to the IP address of your Docker host, on the port
shown by `docker ps` for container port 80.
![Screenshot](images/web.png)
![Screenshot](images/welcome-to-nginx.png)
---
@@ -189,87 +189,6 @@ $ ping <ipAddress>
---
## The different network drivers
A container can use one of the following drivers:
* `bridge` (default)
* `none`
* `host`
* `container`
The driver is selected with `docker run --net ...`.
The different drivers are explained with more details on the following slides.
---
## The default bridge
* By default, the container gets a virtual `eth0` interface.
<br/>(In addition to its own private `lo` loopback interface.)
* That interface is provided by a `veth` pair.
* It is connected to the Docker bridge.
<br/>(Named `docker0` by default; configurable with `--bridge`.)
* Addresses are allocated on a private, internal subnet.
<br/>(Docker uses 172.17.0.0/16 by default; configurable with `--bip`.)
* Outbound traffic goes through an iptables MASQUERADE rule.
* Inbound traffic goes through an iptables DNAT rule.
* The container can have its own routes, iptables rules, etc.
---
## The null driver
* Container is started with `docker run --net none ...`
* It only gets the `lo` loopback interface. No `eth0`.
* It can't send or receive network traffic.
* Useful for isolated/untrusted workloads.
---
## The host driver
* Container is started with `docker run --net host ...`
* It sees (and can access) the network interfaces of the host.
* It can bind any address, any port (for ill and for good).
* Network traffic doesn't have to go through NAT, bridge, or veth.
* Performance = native!
Use cases:
* Performance sensitive applications (VOIP, gaming, streaming...)
* Peer discovery (e.g. Erlang port mapper, Raft, Serf...)
---
## The container driver
* Container is started with `docker run --net container:id ...`
* It re-uses the network stack of another container.
* It shares with this other container the same interfaces, IP address(es), routes, iptables rules, etc.
* Those containers can communicate over their `lo` interface.
<br/>(i.e. one can bind to 127.0.0.1 and the others can connect to it.)
---
## Section summary
We've learned how to:

View File

@@ -3,7 +3,7 @@ class: title
# Copying files during the build
![Monks copying books](images/copy.jpg)
![Monks copying books](images/title-copying-files-during-build.jpg)
---

View File

@@ -1,32 +0,0 @@
class: title
# Course Conclusion
![end](images/end.jpg)
---
## Questions & Next Steps
A bunch of useful links:
* Docker homepage - http://www.docker.com/
* Docker Hub - https://hub.docker.com
* Docker blog - http://blog.docker.com/
* Docker documentation - http://docs.docker.com/
* Docker code on GitHub - https://github.com/docker/docker
* Docker mailing list - [https://groups.google.com/forum/#!forum/docker-user
* Docker on IRC: irc.freenode.net and channels `#docker` and `#docker-dev`
* Docker on Twitter - http://twitter.com/docker
* Get Docker help on Stack Overflow - http://stackoverflow.com/search?q=docker
* Play With Docker Hands-On Labs - http://training.play-with-docker.com/
These slides are at: http://container.training/
---
class: title
Thank You!
.small[http://container.training/]

View File

@@ -20,7 +20,7 @@ class: pic
## The VPS age (until 2007-2008)
![lightcont](images/lightcont.png)
![lightcont](images/containers-as-lightweight-vms.png)
---

View File

@@ -1,35 +0,0 @@
# Publishing images to the Docker Hub
We have built our first images.
If we were so inclined, we could share those images through the Docker Hub.
We won't do it since we don't want to force everyone to create a Docker Hub account (although it's free, yay!) but the steps would be:
* have an account on the Docker Hub
* tag our image accordingly (i.e. `username/imagename`)
* `docker push username/imagename`
Anybody can now `docker run username/imagename` from any Docker host.
Images can be set to be private as well.
---
## The goodness of automated builds
* You can link a Docker Hub repository with a GitHub or BitBucket repository
* Each push to GitHub or BitBucket will trigger a build on Docker Hub
* If the build succeeds, the new image is available on Docker Hub
* You can map tags and branches between source and container images
* If you work with public repositories, this is free
* Corollary: this gives you a very simple way to get free, basic CI
(With the technique presented earlier)

View File

@@ -58,7 +58,7 @@ class: pic
## The deployment problem
![problem](images/problem.png)
![problem](images/shipping-software-problem.png)
---
@@ -66,7 +66,7 @@ class: pic
## The matrix from hell
![matrix](images/matrix.png)
![matrix](images/shipping-matrix-from-hell.png)
---
@@ -74,7 +74,7 @@ class: pic
## The parallel with the shipping indsutry
![history](images/shippingindustry.png)
![history](images/shipping-industry-problem.png)
---
@@ -82,7 +82,7 @@ class: pic
## Intermodal shipping containers
![shipping](images/shipping.png)
![shipping](images/shipping-industry-solution.png)
---
@@ -90,7 +90,7 @@ class: pic
## A new shipping ecosystem
![shipeco](images/shipeco.png)
![shipeco](images/shipping-indsutry-results.png)
---
@@ -98,7 +98,7 @@ class: pic
## A shipping container system for applications
![shipapp](images/appcont.png)
![shipapp](images/shipping-software-solution.png)
---
@@ -106,17 +106,29 @@ class: pic
## Eliminate the matrix from hell
![elimatrix](images/elimatrix.png)
![elimatrix](images/shipping-matrix-solved.png)
---
## Results
* Dev-to-prod reduced from 9 months to 15 minutes (ING)
* [Dev-to-prod reduced from 9 months to 15 minutes (ING)](
https://www.docker.com/sites/default/files/CS_ING_01.25.2015_1.pdf)
* Continuous integration job time reduced by more than 60% (BBC)
* [Continuous integration job time reduced by more than 60% (BBC)](
https://www.docker.com/sites/default/files/CS_BBCNews_01.25.2015_1.pdf)
* Dev-to-prod reduced from weeks to minutes (GILT)
* [Deploy 100 times a day instead of once a week (GILT)](
https://www.docker.com/sites/default/files/CS_Gilt%20Groupe_03.18.2015_0.pdf)
* [70% infrastructure consolidation (MetLife)](
https://www.docker.com/customers/metlife-transforms-customer-experience-legacy-and-microservices-mashup)
* [60% infrastructure consolidation (Intesa Sanpaolo)](
https://blog.docker.com/2017/11/intesa-sanpaolo-builds-resilient-foundation-banking-docker-enterprise-edition/)
* [14x application density; 60% of legacy datacenter migrated in 4 months (GE Appliances)](
https://www.docker.com/customers/ge-uses-docker-enable-self-service-their-developers)
* etc.

View File

@@ -90,11 +90,11 @@ COPY <test data sets and fixtures>
RUN <unit tests>
FROM <baseimage>
RUN <install dependencies>
COPY <vcode>
COPY <code>
RUN <build code>
CMD, EXPOSE ...
```
* The build fails as soon as an instructions fails
* The build fails as soon as an instruction fails
* If `RUN <unit tests>` fails, the build doesn't produce an image
* If it succeeds, it produces a clean image (without test libraries and data)

View File

@@ -1,9 +1,9 @@
class: title
# Our First Containers
# Our first containers
![Plastic Containers](images/firstcontainers.jpg)
![Colorful plastic tubs](images/title-our-first-containers.jpg)
---
@@ -51,10 +51,13 @@ root@04c0bb0a6c07:/#
```
* This is a brand new container.
* It runs a bare-bones, no-frills `ubuntu` system.
* `-it` is shorthand for `-i -t`.
* `-i` tells Docker to connect us to the container's stdin.
* `-t` tells Docker that we want a pseudo-terminal.
---
@@ -72,22 +75,6 @@ Alright, we need to install it.
---
## An observation
Let's check how many packages are installed here.
```bash
root@04c0bb0a6c07:/# dpkg -l | wc -l
189
```
* `dpkg -l` lists the packages installed in our container
* `wc -l` counts them
* If you have a Debian or Ubuntu machine, you can run the same command
and compare the results.
---
## Install a package in our container
We want `figlet`, so let's install it:
@@ -104,6 +91,12 @@ Reading package lists... Done
One minute later, `figlet` is installed!
---
## Try to run our freshly installed program
The `figlet` program takes a message as parameter.
```bash
root@04c0bb0a6c07:/# figlet hello
_ _ _
@@ -113,11 +106,30 @@ root@04c0bb0a6c07:/# figlet hello
|_| |_|\___|_|_|\___/
```
Beautiful! .emoji[😍]
---
## Exiting our container
## Counting packages in the container
Just exit the shell, like you would usually do.
Let's check how many packages are installed there.
```bash
root@04c0bb0a6c07:/# dpkg -l | wc -l
190
```
* `dpkg -l` lists the packages installed in our container
* `wc -l` counts them
How many packages do we have on our host?
---
## Counting packages on the host
Exit the container by logging out of the shell, like you would usually do.
(E.g. with `^D` or `exit`)
@@ -125,10 +137,36 @@ Just exit the shell, like you would usually do.
root@04c0bb0a6c07:/# exit
```
Now, try to:
* run `dpkg -l | wc -l`. How many packages are installed?
* run `figlet`. Does that work?
---
## Host and containers are independent things
* We ran an `ubuntu` container on an `ubuntu` host.
* But they have different, independent packages.
* Installing something on the host doesn't expose it to the container.
* And vice-versa.
* We can run *any container* on *any host*.
---
## Where's our container?
* Our container is now in a *stopped* state.
* It still exists on disk, but all compute resources have been freed up.
* We will see later how to get back to that container.
---
## Starting another container

View File

@@ -1,9 +1,9 @@
class: title
# Understanding Docker Images
# Understanding Docker images
![image](images/image.png)
![image](images/title-understanding-docker-images.png)
---

Some files were not shown because too many files have changed in this diff Show More