github/container.training
1
0
Fork 0
mirror of https://github.com/jpetazzo/container.training.git synced 2026-07-28 01:01:12 +00:00
Code Issues Packages Projects Releases Wiki Activity

short version for DoD MSP

Browse Source
This commit is contained in:
Jérôme Petazzoni
2017-07-24 15:34:06 -05:00
parent 0d7ee1dda0
commit 84c88ed4c2
2 changed files with 215 additions and 42 deletions
Download Patch File Download Diff File Expand all files Collapse all files

251
docs/index.html
View File

@@ -109,14 +109,12 @@ class: title, in-person
.small[
Deploy and scale containers with Docker native, open source orchestration
Deploying And Scaling Microservices <br/> In Containers With Docker Swarm
.small[.small[
**Be kind to the WiFi!**
*Use the 5G network*
<br/>
*Don't use your hotspot*
<br/>
*Don't stream videos from YouTube, Netflix, etc.
@@ -128,7 +126,7 @@ Thank you!
]
]
---
???
class: in-person
@@ -139,7 +137,7 @@ class: in-person
&
Jérôme ([@jpetazzo](https://twitter.com/jpetazzo))
--
???
class: in-person
@@ -159,12 +157,12 @@ at e.g. 9am, and start at 9:30.
class: in-person
## Agenda
<!--
- Agenda:
## Agenda
-->
## Intros
<!--
.small[
@@ -180,20 +178,28 @@ class: in-person
-->
- The tutorial will run from 9:00am to 12:20pm
- I'm
Jérôme ([@jpetazzo](https://twitter.com/jpetazzo))
- The tutorial will run from 3:30pm to 5pm
- This will be fast-paced, but DON'T PANIC!
- All the content is publicly available (slides, code samples, scripts)
<!--
- There will be a coffee break at 10:30am
<br/>
(please remind me if I forget about it!)
-->
- Feel free to interrupt for questions at any time
- Live feedback, questions, help on [Gitter](chat)
- There is an extended version of this workshop if you want to know more
???
class: in-person
@@ -268,6 +274,23 @@ grep '^# ' index.html | grep -v '<br' | tr '#' '-'
class: in-person
## Table of contents
- Pre-requirements and VM environment
- Running our sample application on a single node
- When a single node is not enough: scaling out
- Introducing SwarmKit concepts
- Assembling our Swarm cluster
- The meaning of the Docker motto: "Build, Ship, and Run"
- Setting up a container registry
- Deploying on our cluster
- Rolling upgrades
- A few words on "least privilege model"
???
class: in-person
## Chapter 1: getting started
- Pre-requirements
@@ -282,7 +305,7 @@ class: in-person
- Introducing SwarmKit
---
???
class: in-person
@@ -304,7 +327,7 @@ class: in-person
- Integration with Compose
---
???
class: in-person
@@ -322,7 +345,7 @@ class: in-person
- Metrics collection
---
???
class: in-person
@@ -489,13 +512,13 @@ wait
---
class: in-person
class: in-person, redo
## If doing or re-doing the workshop on your own ...
---
class: self-paced
class: self-paced, redo
## How to get your own Docker nodes?
@@ -503,6 +526,8 @@ class: self-paced
--
class: redo
- Main differences:
- you don't need to SSH to the machines
@@ -598,9 +623,9 @@ You are welcome to use the method that you feel the most comfortable with.
## Brand new versions!
- Engine 17.05
- Engine 17.06
- Compose 1.12
- Machine 0.11
- Machine 0.12
.exercise[
@@ -615,15 +640,21 @@ You are welcome to use the method that you feel the most comfortable with.
---
## Wait, what, 17.05 ?!?
class: extra-details
## From Docker 1.X to 17.Y ...
--
- Docker inc. [recently announced](https://blog.docker.com/2017/03/docker-enterprise-edition/)
Docker Enterprise Edition
class: extra-details
- Docker [now has two editions](https://blog.docker.com/2017/03/docker-enterprise-edition/):
Community and Enterprise
--
class: extra-details
- Docker 1.13 = Docker 17.03 (year.month, like Ubuntu)
- Every month, there is a new "edge" release (with new features)
@@ -674,7 +705,7 @@ class: extra-details
---
class: pic
class: pic, extra-details
![Docker CE/EE release cycle](lifecycle.png)
@@ -751,6 +782,10 @@ class: extra-details
(instead of e.g. IP address or FQDN)
- Network aliases are automatically namespaced
(i.e. you can have multiple apps declaring and using a service named `database`)
---
## Example in `worker/worker.py`
@@ -1173,18 +1208,6 @@ Scaling out
- It is a plumbing part of the Docker ecosystem
<!--
- SwarmKit comes with two examples:
- `swarmctl` (a CLI tool to "speak" the SwarmKit API)
- `swarmd` (an agent that can federate existing Docker Engines into a Swarm)
-->
- SwarmKit/swarmd/swarmctl → libcontainer/containerd/container-ctr
---
## SwarmKit features
@@ -1807,6 +1830,8 @@ class: docker-machine
---
class: under-the-hood
## Under the hood: docker swarm init
When we do `docker swarm init`:
@@ -1821,6 +1846,8 @@ When we do `docker swarm init`:
---
class: under-the-hood
## Under the hood: join tokens
There is one token to *join as a worker*, and another to *join as a manager*.
@@ -1838,6 +1865,8 @@ docker swarm join-token --rotate <worker|manager>
---
class: under-the-hood
## Under the hood: docker swarm join
When a node joins the Swarm:
@@ -1856,6 +1885,8 @@ When a node joins the Swarm:
---
class: under-the-hood
## Under the hood: cluster communication
- The *control plane* is encrypted with AES-GCM; keys are rotated every 12 hours
@@ -1871,6 +1902,8 @@ When a node joins the Swarm:
---
class: under-the-hood
## Under the hood: I want to know more!
Revisit SwarmKit concepts:
@@ -2108,6 +2141,8 @@ class: self-paced
---
class: extra-details
## What's the upper limit?
- We don't know!
@@ -2154,7 +2189,7 @@ class: self-paced
(New in Docker Engine 17.05)
If you are running Docker 17.05, you will see the following message:
If you are running Docker 17.05+, you will see the following message:
```
Since --detach=false was not specified, tasks will be created in the background.
@@ -2188,6 +2223,8 @@ Note: by default, when a container is destroyed (e.g. when scaling down), its lo
---
class: extra-details
## Before Docker Engine 17.05
- Docker 1.13/17.03/17.04 have `docker service logs` as an experimental feature
@@ -2325,6 +2362,8 @@ The latest version of the ElasticSearch image won't start without mandatory conf
---
class: extra-details
## Tasks lifecycle
- During the deployment, you will be able to see multiple states:
@@ -2436,6 +2475,8 @@ There are many ways to deal with inbound traffic on a Swarm cluster.
---
class: btw-labels
## Managing HTTP traffic
- The TCP routing mesh doesn't parse HTTP headers
@@ -2453,6 +2494,8 @@ There are many ways to deal with inbound traffic on a Swarm cluster.
---
class: btw-labels
## You should use labels
- Labels are a great way to attach arbitrary information to services
@@ -2620,6 +2663,8 @@ We just have to adapt this to our application, which has 4 services!
---
class: extra-details
## Which registry do we want to use?
.small[
@@ -2696,6 +2741,8 @@ class: extra-details
---
class: extra-details
## Using the open source registry
- We need to run a `registry:2` container
@@ -2717,6 +2764,8 @@ class: extra-details
---
class: manual-btp
# Deploying a local registry
- We will create a single-instance service, publishing its port
@@ -2740,6 +2789,8 @@ class: extra-details
---
class: manual-btp
## Testing our local registry
- We can retag a small image, and push it to the registry
@@ -2761,6 +2812,8 @@ class: extra-details
---
class: manual-btp
## Checking what's on our local registry
- The registry API has endpoints to query what's there
@@ -2994,6 +3047,8 @@ It has been replaced by the new version, with port 80 accessible from outside.
---
class: manual-btp
## Connect to the web UI
- The web UI is now available on port 8000, *on all the nodes of the cluster*
@@ -3008,6 +3063,8 @@ It has been replaced by the new version, with port 80 accessible from outside.
---
class: manual-btp
## Scaling the application
- We can change scaling parameters with `docker update` as well
@@ -3029,6 +3086,8 @@ You should see the performance peaking at 10 hashes/s (like before).
---
class: manual-btp
# Global scheduling
- We want to utilize as best as we can the entropy generators
@@ -3044,6 +3103,8 @@ You should see the performance peaking at 10 hashes/s (like before).
---
class: manual-btp
## Scaling the `rng` service
.exercise[
@@ -3065,7 +3126,7 @@ You should see the performance peaking at 10 hashes/s (like before).
---
class: extra-details
class: manual-btp, extra-details
## Why do we have to re-create the service to enable global scheduling?
@@ -3083,6 +3144,8 @@ class: extra-details
---
class: swarm-ready
## How did we make our app "Swarm-ready"?
This app was written in June 2015. (One year before Swarm mode was released.)
@@ -3091,6 +3154,8 @@ What did we change to make it compatible with Swarm mode?
--
class: swarm-ready
.exercise[
- Go to the app directory:
@@ -3107,6 +3172,8 @@ What did we change to make it compatible with Swarm mode?
---
class: swarm-ready
## What did we change in our app since its inception?
- Compose files
@@ -3119,10 +3186,14 @@ What did we change to make it compatible with Swarm mode?
--
class: swarm-ready
*We didn't change a single line of code in this app since it was written.*
--
class: swarm-ready
*The images that were [built in June 2015](
https://hub.docker.com/r/jpetazzo/dockercoins_worker/tags/)
(when the app was written) can still run today ...
@@ -3131,6 +3202,8 @@ https://hub.docker.com/r/jpetazzo/dockercoins_worker/tags/)
---
class: swarm-ready
## How did we design our app in the first place?
- [Twelve-Factor App](https://12factor.net/) principles
@@ -3147,6 +3220,8 @@ https://hub.docker.com/r/jpetazzo/dockercoins_worker/tags/)
---
class: manual-btp
# Integration with Compose
- The previous section showed us how to streamline image build and push
@@ -3177,6 +3252,8 @@ https://hub.docker.com/r/jpetazzo/dockercoins_worker/tags/)
---
class: manual-btp
## Removing everything
- Before deploying using "stacks," let's get a clean slate
@@ -3196,7 +3273,7 @@ https://hub.docker.com/r/jpetazzo/dockercoins_worker/tags/)
We need a registry to move images around.
Before, we deployed it with the following command:
Without a stack file, it would be deployed with the following command:
```bash
docker service create --publish 5000:5000 registry:2
@@ -3277,6 +3354,8 @@ services:
---
class: manual-btp
## Specifics of stack deployment
Our registry is not *exactly* identical to the one deployed with `docker service create`!
@@ -3433,7 +3512,7 @@ class: extra-details
- Awesome job, team!
---
???
class: title, in-person
@@ -4274,6 +4353,8 @@ The current upgrade will continue at a faster pace.
---
class: extra-details
## Rolling back
- At any time (e.g. before the upgrade is complete), we can rollback:
@@ -4295,6 +4376,8 @@ What happens with the web UI graph?
---
class: extra-details
## The fine print with rollback
- Rollback reverts to the previous service definition
@@ -4748,7 +4831,7 @@ class: secrets
---
class: secrets
class: secrets, extra-details
## Secrets in practice
@@ -4772,6 +4855,8 @@ class: secrets
---
class: api-scope
## A reminder about *scope*
- Out of the box, Docker API access is "all or nothing"
@@ -4788,6 +4873,8 @@ class: secrets
---
class: api-scope
## Fine-grained API access control
A few solutions, by increasing order of flexibility:
@@ -4798,10 +4885,14 @@ A few solutions, by increasing order of flexibility:
--
class: api-scope
- Add an extra layer of abstraction (sudo scripts, hooks, or full-blown PAAS)
--
class: api-scope
- Enable [authorization plugins]
- each API request is vetted by your plugin(s)
@@ -4990,6 +5081,8 @@ For more information about locking, you can check the [upcoming documentation](h
---
class: elk
name: logging
# Centralized logging
@@ -5000,6 +5093,8 @@ name: logging
--
class: elk
- We are going to deploy an ELK stack
- It will accept logs over a GELF socket
@@ -5008,6 +5103,8 @@ name: logging
---
class: elk
# Setting up ELK to store container logs
*Important foreword: this is not an "official" or "recommended"
@@ -5027,6 +5124,8 @@ What we will do:
---
class: elk
## What's in an ELK stack?
- ELK is three components:
@@ -5224,6 +5323,8 @@ You should see the heartbeat messages:
---
class: elk
## Testing the GELF receiver
- In a new window, we will generate a logging message
@@ -5243,6 +5344,8 @@ The test message should show up in the logstash container logs.
---
class: elk
## Sending logs from a service
- We were sending from a "classic" container so far; let's send logs from a service instead
@@ -5265,10 +5368,14 @@ The test message should show up as well in the logstash container logs.
--
class: elk
In fact, *multiple messages will show up, and continue to show up every few seconds!*
---
class: elk
## Restart conditions
- By default, if a container exits (or is killed with `docker kill`, or runs out of memory ...),
@@ -5290,6 +5397,8 @@ You can also set `--restart-delay`, `--restart-max-attempts`, and `--restart-win
---
class: elk
## Connect to Kibana
- The Kibana web UI is exposed on cluster port 5601
@@ -5306,6 +5415,8 @@ You can also set `--restart-delay`, `--restart-max-attempts`, and `--restart-win
---
class: elk
## "Configuring" Kibana
- If you see a status page with a yellow item, wait a minute and reload
@@ -5327,6 +5438,8 @@ You can also set `--restart-delay`, `--restart-max-attempts`, and `--restart-win
---
class: elk
## Updating our services to use GELF
- We will now inform our Swarm to add GELF logging to all our services
@@ -5361,6 +5474,8 @@ After ~15 seconds, you should see the log messages in Kibana.
---
class: elk
## Viewing container logs
- Go back to Kibana
@@ -5388,6 +5503,8 @@ After ~15 seconds, you should see the log messages in Kibana.
---
class: elk
## .warning[Don't update stateful services!]
- What would have happened if we had updated the Redis service?
@@ -5402,6 +5519,8 @@ After ~15 seconds, you should see the log messages in Kibana.
---
class: elk
## Important afterword
**This is not a "production-grade" setup.**
@@ -5424,6 +5543,8 @@ http://jpetazzo.github.io/2017/01/20/docker-logging-gelf/).
---
class: metrics
# Metrics collection
- We want to gather metrics in a central place
@@ -5434,6 +5555,8 @@ http://jpetazzo.github.io/2017/01/20/docker-logging-gelf/).
---
class: metrics
## Node metrics
- CPU, RAM, disk usage on the whole node
@@ -5450,6 +5573,8 @@ http://jpetazzo.github.io/2017/01/20/docker-logging-gelf/).
---
class: metrics
## Container metrics
- Similar to node metrics, but not totally identical
@@ -6870,6 +6995,8 @@ class: prom, snap
---
class: metrics
## More resources on container metrics
- [Prometheus, a Whirlwind Tour](https://speakerdeck.com/copyconstructor/prometheus-a-whirlwind-tour),
@@ -6886,6 +7013,8 @@ class: prom, snap
---
class: stateful
# Dealing with stateful services
- First of all, you need to make sure that the data files are on a *volume*
@@ -6902,6 +7031,8 @@ class: prom, snap
---
class: stateful
## Building a stateful service experiment
- We will use Redis for this example
@@ -6924,6 +7055,8 @@ class: prom, snap
---
class: stateful
## Accessing our Redis service easily
- Typing that whole command is going to be tedious
@@ -6944,6 +7077,8 @@ class: prom, snap
---
class: stateful
## Basic Redis commands
.exercise[
@@ -6967,6 +7102,8 @@ class: prom, snap
---
class: stateful
## Local volumes vs. global volumes
- Global volumes exist in a single namespace
@@ -6985,6 +7122,8 @@ class: prom, snap
---
class: stateful
## Local volumes
- We will use the default volume driver, `local`
@@ -7006,6 +7145,8 @@ class: prom, snap
---
class: stateful
## Where is our data?
- If we look for our `foo` key, it's gone!
@@ -7030,6 +7171,8 @@ This ensures consistent behavior.
---
class: stateful
## Setting the key again
- Since our database was wiped out, let's populate it again
@@ -7050,6 +7193,8 @@ This ensures consistent behavior.
---
class: stateful
## Service updates cause containers to be replaced
- Let's try to make a trivial update to the service and see what happens
@@ -7072,6 +7217,8 @@ The key is blank again!
---
class: stateful
## Service volumes are ephemeral by default
- Let's highlight what's going on with volumes!
@@ -7095,6 +7242,8 @@ even when it is not strictly technically necessary.
---
class: stateful
## The data is gone again
- What happened to our data?
@@ -7112,6 +7261,8 @@ even when it is not strictly technically necessary.
---
class: stateful
## Assigning a persistent volume to the container
- Let's add an explicit volume mount to our service, referencing a named volume
@@ -7135,6 +7286,8 @@ Note: the `local` volume driver automatically creates volumes.
---
class: stateful
## Checking that persistence actually works across service updates
.exercise[
@@ -7158,6 +7311,8 @@ Note: the `local` volume driver automatically creates volumes.
---
class: stateful
## Recap
- The service must commit its state to disk when being shutdown.red[*]
@@ -7180,6 +7335,8 @@ It's easy to make that mistake — __Trust me!__]
---
class: stateful
## Cleaning up
.exercise[
@@ -7200,33 +7357,47 @@ Note: we could keep the volume around if we wanted.
---
class: stateful
## Should I run stateful services in containers?
--
class: stateful
Depending whom you ask, they'll tell you:
--
class: stateful
- certainly not, heathen!
--
class: stateful
- we've been running a few thousands PostgreSQL instances in containers ...
<br/>for a few years now ... in production ... is that bad?
--
class: stateful
- what's a container?
--
class: stateful
Perhaps a better question would be:
*"Should I run stateful services?"*
--
class: stateful
- is it critical for my business?
- is it my value-add?
- or should I find somebody else to run them for me?
@@ -7521,9 +7692,11 @@ That's all folks! <br/> Questions?
Jérôme ([@jpetazzo](https://twitter.com/jpetazzo)) — [@docker](https://twitter.com/docker)
<!--
AJ ([@s0ulshake](https://twitter.com/s0ulshake)) — *For hire!*
<br/>
`curl cv.soulshake.net`
-->
]]
@@ -7538,8 +7711,8 @@ AJ ([@s0ulshake](https://twitter.com/s0ulshake)) — *For hire!*
var slideshow = remark.create({
ratio: '16:9',
highlightSpans: true,
excludedClasses: ["in-person", "elk-auto", "prom-auto"]
//excludedClasses: ["self-paced", "extra-details", "advertise-addr", "docker-machine", "netshoot", "sbt", "ipsec", "node-info", "swarmtools", "secrets", "encryption-at-rest", "elk-manual", "snap", "prom-manual"]
//excludedClasses: ["in-person", "elk-auto", "prom-auto"]
excludedClasses: ["self-paced", "extra-details", "advertise-addr", "docker-machine", "netshoot", "sbt", "ipsec", "node-info", "swarmtools", "_secrets", "encryption-at-rest", "elk-manual", "snap", "prom", "prom-manual", "prom-auto", "redo", "under-the-hood", "btw-labels", "manual-btp", "swarm-ready", "api-scope", "elk", "elk-auto", "metrics", "stateful"]
});
</script>
</body>

6
prepare-vms/settings/orchestration.yaml
View File

@@ -27,7 +27,7 @@ footer: >
url: http://container.training/
engine_version: test
compose_version: 1.12
machine_version: 0.12.0
engine_version: stable
compose_version: 1.12.0
machine_version: 0.12.2
swarm_version: latest