diff --git a/.gitignore b/.gitignore index cc8fd624..21ae9d25 100644 --- a/.gitignore +++ b/.gitignore @@ -8,3 +8,15 @@ slides/autopilot/state.yaml slides/index.html slides/past.html node_modules + +### macOS ### +# General +.DS_Store +.AppleDouble +.LSOverride + +### Windows ### +# Windows thumbnail cache files +Thumbs.db +ehthumbs.db +ehthumbs_vista.db diff --git a/prepare-vms/terraform/machines.tf b/prepare-vms/terraform/machines.tf index 44b5b8c2..41ff96e7 100644 --- a/prepare-vms/terraform/machines.tf +++ b/prepare-vms/terraform/machines.tf @@ -1,7 +1,7 @@ resource "openstack_compute_instance_v2" "machine" { count = "${var.count}" name = "${format("%s-%04d", "${var.prefix}", count.index+1)}" - image_name = "Ubuntu 16.04 (Xenial Xerus)" + image_name = "Ubuntu 16.04.5 (Xenial Xerus)" flavor_name = "${var.flavor}" security_groups = ["${openstack_networking_secgroup_v2.full_access.name}"] key_pair = "${openstack_compute_keypair_v2.ssh_deploy_key.name}" diff --git a/slides/containers/Advanced_Dockerfiles.md b/slides/containers/Advanced_Dockerfiles.md index f581af8c..58cb1713 100644 --- a/slides/containers/Advanced_Dockerfiles.md +++ b/slides/containers/Advanced_Dockerfiles.md @@ -1,3 +1,6 @@ + +class: title + # Advanced Dockerfiles ![construction](images/title-advanced-dockerfiles.jpg) diff --git a/slides/containers/Getting_Inside.md b/slides/containers/Getting_Inside.md index 9f232f92..9167dfdc 100644 --- a/slides/containers/Getting_Inside.md +++ b/slides/containers/Getting_Inside.md @@ -1,3 +1,4 @@ + class: title # Getting inside a container diff --git a/slides/containers/Installing_Docker.md b/slides/containers/Installing_Docker.md index 41cfa90b..ced137d4 100644 --- a/slides/containers/Installing_Docker.md +++ b/slides/containers/Installing_Docker.md @@ -1,3 +1,4 @@ + class: title # Installing Docker diff --git a/slides/containers/Training_Environment.md b/slides/containers/Training_Environment.md index 2176cb59..45302a17 100644 --- a/slides/containers/Training_Environment.md +++ b/slides/containers/Training_Environment.md @@ -1,3 +1,4 @@ + class: title # Our training environment diff --git a/slides/containers/Windows_Containers.md b/slides/containers/Windows_Containers.md new file mode 100644 index 00000000..6b090882 --- /dev/null +++ b/slides/containers/Windows_Containers.md @@ -0,0 +1,164 @@ +class: title + +# Windows Containers + +![Container with Windows](images/windows-containers.jpg) + +--- + +## Objectives + +At the end of this section, you will be able to: + +* Understand Windows Container vs. Linux Container. + +* Know about the features of Docker for Windows for choosing architecture. + +* Run other container architectures via QEMU emulation. + +--- + +## Are containers *just* for Linux? + +Remember that a container must run on the kernel of the OS it's on. + +- This is both a benefit and a limitation. + + (It makes containers lightweight, but limits them to a specific kernel.) + +- At its launch in 2013, Docker did only support Linux, and only on amd64 CPUs. + +- Since then, many platforms and OS have been added. + + (Windows, ARM, i386, IBM mainframes ... But no macOS or iOS yet!) + +-- + +- Docker Desktop (macOS and Windows) can run containers for other architectures + + (Check the docs to see how to [run a Raspberry Pi (ARM) or PPC container](https://docs.docker.com/docker-for-mac/multi-arch/)!) + +--- + +## History of Windows containers + +- Early 2016, Windows 10 gained support for running Windows binaries in containers. + + - These are known as "Windows Containers" + + - Win 10 expects Docker for Windows to be installed for full features + + - These must run in Hyper-V mini-VM's with a Windows Server x64 kernel + + - No "scratch" containers, so use "Core" and "Nano" Server OS base layers + + - Since Hyper-V is required, Windows 10 Home won't work (yet...) + +-- + +- Late 2016, Windows Server 2016 ships with native Docker support + + - Installed via PowerShell, doesn't need Docker for Windows + + - Can run native (without VM), or with [Hyper-V Isolation](https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-containers/hyperv-container) + +--- + +## LCOW (Linux Containers On Windows) + +While Docker on Windows is largely playing catch up with Docker on Linux, +it's moving fast; and this is one thing that you *cannot* do on Linux! + +- LCOW came with the [2017 Fall Creators Update](https://blog.docker.com/2018/02/docker-for-windows-18-02-with-windows-10-fall-creators-update/). + +- It can run Linux and Windows containers side-by-side on Win 10. + +- It is no longer necessary to switch the Engine to "Linux Containers". + + (In fact, if you want to run both Linux and Windows containers at the same time, + make sure that your Engine is set to "Windows Containers" mode!) + +-- + +If you are a Docker for Windows user, start your engine and try this: + +```bash +docker pull microsoft/nanoserver:1803 +``` + +(Make sure to switch to "Windows Containers mode" if necessary.) + +--- + +## Run Both Windows and Linux containers + +- Run a Windows Nano Server (minimal CLI-only server) + + ```bash + docker run --rm -it microsoft/nanoserver:1803 powershell + Get-Process + exit + ``` + +- Run busybox on Linux in LCOW + + ```bash + docker run --rm --platform linux busybox echo hello + ``` + +(Although you will not be able to see them, this will create hidden +Nano and LinuxKit VMs in Hyper-V!) + +--- + +## Did We Say Things Move Fast + +- Things keep improving. + +- Now `--platform` defaults to `windows`, some images support both: + + - golang, mongo, python, redis, hello-world ... and more being added + + - you should still use `--plaform` with multi-os images to be certain + +- Windows Containers now support `localhost` accessable containers (July 2018) + +- Microsoft (April 2018) added Hyper-V support to Windows 10 Home ... + + ... so stay tuned for Docker support, maybe?!? + +--- + +## Other Windows container options + +Most "official" Docker images don't run on Windows yet. + +Places to Look: + + - Hub Official: https://hub.docker.com/u/winamd64/ + + - Microsoft: https://hub.docker.com/r/microsoft/ + +--- + +## SQL Server? Choice of Linux or Windows + +- Microsoft [SQL Server for Linux 2017](https://hub.docker.com/r/microsoft/mssql-server-linux/) (amd64/linux) + +- Microsoft [SQL Server Express 2017](https://hub.docker.com/r/microsoft/mssql-server-windows-express/) (amd64/windows) + +--- + +## Windows Tools and Tips + +- PowerShell [Tab Completion: DockerCompletion](https://github.com/matt9ucci/DockerCompletion) + +- Best Shell GUI: [Cmder.net](http://cmder.net/) + +- Good Windows Container Blogs and How-To's + + - Dockers DevRel [Elton Stoneman, Microsoft MVP](https://blog.sixeyed.com/) + + - Docker Captian [Nicholas Dille](https://dille.name/blog/) + + - Docker Captain [Stefan Scherer](https://stefanscherer.github.io/) diff --git a/slides/images/ci-cd-with-docker.png b/slides/images/ci-cd-with-docker.png new file mode 100644 index 00000000..333d7ecc Binary files /dev/null and b/slides/images/ci-cd-with-docker.png differ diff --git a/slides/images/dockercoins-diagram.svg b/slides/images/dockercoins-diagram.svg new file mode 100644 index 00000000..ac05fe94 --- /dev/null +++ b/slides/images/dockercoins-diagram.svg @@ -0,0 +1,2 @@ + +
worker
worker
rng
rng
hasher
hasher
redis
redis
webui
webui
GET
GET
POST
POST
TCP
TCP
TCP
TCP
user
user
GET
GET
DockerCoins application
(five containers)
DockerCoins application<br>(five containers)<br> \ No newline at end of file diff --git a/slides/images/dockercoins-diagram.xml b/slides/images/dockercoins-diagram.xml new file mode 100644 index 00000000..c2c3893e --- /dev/null +++ b/slides/images/dockercoins-diagram.xml @@ -0,0 +1 @@ +7Vnfb9sgEP5rLG0vlQ3+kTyuabs9bFq1Vtr2iO2LjUqMhfGS9q8fxNgxJtXSqmmmrVEeuAMOuO874LCHF6vNR0Hq8gvPgXnIzzcevvAQCkKEPP338/tOk0RxpygEzU2jneKGPoBR+kbb0hwaq6HknEla28qMVxVk0tIRIfjabrbkzB61JgU4ipuMMFf7neay7LSzyN/pPwEtyn7kwDc1KcnuCsHbyoznIbzc/rrqFeltmfZNSXK+HqnwpYcXgnPZlVabBTDt295tXb+rR2qHeQuo5EEdDC6/CGuhn/J2YvK+d8Z2OaA7+B4+X5dUwk1NMl27VvArXSlXTEmBKi4pYwvOuNj2xTmB2TJT+kYKfgejmjibQbpUNQUjTWOMZ3xFM1MeXKOFJa/kFVlRpgn1jadccj0uF/RB1ZBhdCUYNqFQyWZxICRsHvVQMPhd8Rn4CqS4V01Mhx4pw+RgZuT1jhdJrytHnMC9khguFoPpHR6qYCDZDw920FlzcQfCwUg5q9bFrE3hzyClHaKf00Ex0PZrKxmtYOTPZ7h9QgJFf5TtJUEep7HaGvqaPtbwS0FnY4cSF7sA7cHuJaALHehEVbzhdghusQ1bGL4ibJEDW0ma8i3iDow4dELo3KNMQE6bN+QOQW44rk6BXOIec5C29A25g3ZLfELk+gv7CLqPl7cOcFDlH/S1XGOnr3v6kmfdGp+MQDBz/KkxYSQFdj7gPALh6mqhfoPLIXcygInD1QJ4KzKwbmKSiALk6IR3YRm5Pdrj9V4ngBFJf9mT2AeFGeGaUzW9AfVgutPO57aJbvKm1zgDmBpKpvSZGOqW7BjaMmNY9mFkCRyyXH+9+U/YEp2SLWge2SDHk9g/lC04nBgKJoZekC3IYcvt4vr/IEt8UrIk4VniR7MZwhGahz0OBnH8XOqgWXSmzQVJHG7N20SKjkckN4v+N4mU/GVEwoF9tDybOuEkkT8mWdy8vW32pH+qF60b6N6puksp421+wAPZyV6ykokX4+g1L4puYn3SiyJsqPyhyv5ZL/3cSoGRrkFQtUjQkekfM2h7wo2jNjll1E5eX5LnBm0wif7kiFcFN4G84NkdiIUy3ugh65rRTHmFVx6KmdTJoIrpuNCld0vtrO3HBElUViia924jh6kqDqXNTTvvq7jOL60k0agIo0WlRAZLbUHHtJob+2DUkteP7CL2Q7z1Pv7YI/oRtpFJuhnM3V0kjvfQM8BP30aUuPsW0hFj98EJX/4G \ No newline at end of file diff --git a/slides/images/windows-containers.jpg b/slides/images/windows-containers.jpg new file mode 100644 index 00000000..44d85c1a Binary files /dev/null and b/slides/images/windows-containers.jpg differ diff --git a/slides/index.yaml b/slides/index.yaml index 9e361704..7f9801c3 100644 --- a/slides/index.yaml +++ b/slides/index.yaml @@ -1,11 +1,3 @@ -- date: 2018-11-23 - city: Copenhagen - country: dk - event: GOTO - title: Build Container Orchestration with Docker Swarm - speaker: bretfisher - attend: https://gotocph.com/2018/workshops/121 - - date: 2018-11-08 city: San Francisco, CA country: us @@ -54,8 +46,9 @@ title: Kubernetes 101 speaker: bridgetkromhout attend: https://conferences.oreilly.com/velocity/vl-ny/public/schedule/detail/70102 + slides: https://velny-k8s101-2018.container.training -- date: 2018-09-30 +- date: 2018-10-01 city: New York, NY country: us event: Velocity @@ -64,7 +57,7 @@ attend: https://conferences.oreilly.com/velocity/vl-ny/public/schedule/detail/69875 slides: https://k8s2d.container.training -- date: 2018-09-30 +- date: 2018-10-01 city: New York, NY country: us event: Velocity diff --git a/slides/intro-fullday.yml b/slides/intro-fullday.yml index c7aa4107..7dee3550 100644 --- a/slides/intro-fullday.yml +++ b/slides/intro-fullday.yml @@ -42,6 +42,7 @@ chapters: #- containers/Connecting_Containers_With_Links.md - containers/Ambassadors.md - - containers/Local_Development_Workflow.md + - containers/Windows_Containers.md - containers/Working_With_Volumes.md - containers/Compose_For_Dev_Stacks.md - containers/Docker_Machine.md diff --git a/slides/intro-selfpaced.yml b/slides/intro-selfpaced.yml index 31aa0815..6020ed78 100644 --- a/slides/intro-selfpaced.yml +++ b/slides/intro-selfpaced.yml @@ -42,6 +42,7 @@ chapters: #- containers/Connecting_Containers_With_Links.md - containers/Ambassadors.md - - containers/Local_Development_Workflow.md + - containers/Windows_Containers.md - containers/Working_With_Volumes.md - containers/Compose_For_Dev_Stacks.md - containers/Docker_Machine.md diff --git a/slides/k8s/portworx.md b/slides/k8s/portworx.md index dfdb7b86..f6810a4e 100644 --- a/slides/k8s/portworx.md +++ b/slides/k8s/portworx.md @@ -139,7 +139,7 @@ - To install Portworx, we need to go to https://install.portworx.com/ -- This website will ask us a bunch of questoins about our cluster +- This website will ask us a bunch of questions about our cluster - Then, it will generate a YAML file that we should apply to our cluster diff --git a/slides/shared/sampleapp.md b/slides/shared/sampleapp.md index fd4f14ff..2d0425a6 100644 --- a/slides/shared/sampleapp.md +++ b/slides/shared/sampleapp.md @@ -103,6 +103,12 @@ and displays aggregated logs. --- +class: pic + +![Diagram showing the 5 containers of the applications](images/dockercoins-diagram.svg) + +--- + ## Our application at work - On the left-hand side, the "rainbow strip" shows the container names diff --git a/slides/swarm-fullday.yml b/slides/swarm-fullday.yml index 9a558279..356ee521 100644 --- a/slides/swarm-fullday.yml +++ b/slides/swarm-fullday.yml @@ -41,6 +41,7 @@ chapters: - swarm/btp-manual.md - swarm/swarmready.md - swarm/compose2swarm.md + - swarm/cicd.md - swarm/updatingservices.md #- swarm/rollingupdates.md - swarm/healthchecks.md diff --git a/slides/swarm-halfday.yml b/slides/swarm-halfday.yml index 7de7776a..da596410 100644 --- a/slides/swarm-halfday.yml +++ b/slides/swarm-halfday.yml @@ -41,6 +41,7 @@ chapters: #- swarm/btp-manual.md #- swarm/swarmready.md - swarm/compose2swarm.md + - swarm/cicd.md - swarm/updatingservices.md #- swarm/rollingupdates.md #- swarm/healthchecks.md diff --git a/slides/swarm-selfpaced.yml b/slides/swarm-selfpaced.yml index d38a6cb7..2a34a2fd 100644 --- a/slides/swarm-selfpaced.yml +++ b/slides/swarm-selfpaced.yml @@ -42,6 +42,7 @@ chapters: - swarm/btp-manual.md - swarm/swarmready.md - swarm/compose2swarm.md + - swarm/cicd.md - | name: part-2 diff --git a/slides/swarm/cicd.md b/slides/swarm/cicd.md new file mode 100644 index 00000000..2199a7a4 --- /dev/null +++ b/slides/swarm/cicd.md @@ -0,0 +1,37 @@ +name: cicd + +# CI/CD for Docker and orchestration + +A quick note about continuous integration and deployment + +- This lab won't have you building out CI/CD pipelines + +- We're cheating a bit by building images on server hosts and not in CI tool + +- Docker and orchestration works with all the CI and deployment tools + +--- + +## CI/CD general process + +- Have your CI build your images, run tests *in them*, then push to registry + +- If you security scan, do it then on your images after tests but before push + +- Optionally, have CI do continuous deployment if build/test/push is successful + +- CD tool would SSH into nodes, or use docker cli against remote engine + +- If supported, it could use docker engine TCP API (swarm API is built-in) + +- Docker KBase [Development Pipeline Best Practices](https://success.docker.com/article/dev-pipeline) + +- Docker KBase [Continuous Integration with Docker Hub](https://success.docker.com/article/continuous-integration-with-docker-hub) + +- Docker KBase [Building a Docker Secure Supply Chain](https://success.docker.com/article/secure-supply-chain) + +--- + +class: pic + +![CI-CD with Docker](images/ci-cd-with-docker.png) diff --git a/slides/swarm/morenodes.md b/slides/swarm/morenodes.md index 8de5deed..7c780f4f 100644 --- a/slides/swarm/morenodes.md +++ b/slides/swarm/morenodes.md @@ -131,42 +131,50 @@ class: self-paced - 5 managers = 2 failures (or 1 failure during 1 maintenance) -- 7 managers and more = now you might be overdoing it a little bit +- 7 managers and more = now you might be overdoing it for most designs + +.footnote[ + + see [Docker's admin guide](https://docs.docker.com/engine/swarm/admin_guide/#add-manager-nodes-for-fault-tolerance) + on node failure and datacenter redundancy + +] --- ## Why not have *all* nodes be managers? -- Intuitively, it's harder to reach consensus in larger groups - - With Raft, writes have to go to (and be acknowledged by) all nodes -- More nodes = more network traffic +- Thus, it's harder to reach consensus in larger groups -- Bigger network = more latency +- Only one manager is Leader (writable), so more managers ≠ more capacity + +- Managers should be < 10ms latency from each other + +- These design parameters lead us to recommended designs --- ## What would McGyver do? -- If some of your machines are more than 10ms away from each other, -
- try to break them down in multiple clusters - (keeping internal latency low) +- Keep managers in one region (multi-zone/datacenter/rack) -- Groups of up to 9 nodes: all of them are managers +- Groups of 3 or 5 nodes: all are managers. Beyond 5, seperate out managers and workers -- Groups of 10 nodes and up: pick 5 "stable" nodes to be managers -
- (Cloud pro-tip: use separate auto-scaling groups for managers and workers) +- Groups of 10-100 nodes: pick 5 "stable" nodes to be managers - Groups of more than 100 nodes: watch your managers' CPU and RAM -- Groups of more than 1000 nodes: + - 16GB memory or more, 4 CPU's or more, SSD's for Raft I/O + - otherwise, break down your nodes in multiple smaller clusters - - if you can afford to have fast, stable managers, add more of them - - otherwise, break down your nodes in multiple clusters +.footnote[ + Cloud pro-tip: use separate auto-scaling groups for managers and workers + + See docker's "[Running Docker at scale](http://success.docker.com/article/running-docker-ee-at-scale)" document +] --- ## What's the upper limit? @@ -181,11 +189,11 @@ class: self-paced - Testing by the community: [4700 heterogeneous nodes all over the 'net](https://sematext.com/blog/2016/11/14/docker-swarm-lessons-from-swarm3k/) - - it just works + - it just works, assuming they have the resources - - more nodes require more CPU; more containers require more RAM + - more nodes require manager CPU and networking; more containers require RAM - - scheduling of large jobs (70000 containers) is slow, though (working on it!) + - scheduling of large jobs (70,000 containers) is slow, though ([getting better](https://github.com/moby/moby/pull/37372)!) ---