diff --git a/slides/images/ambassador.jpg b/slides/images/ambassador.jpg new file mode 100644 index 00000000..4be767b0 Binary files /dev/null and b/slides/images/ambassador.jpg differ diff --git a/slides/images/appcont.png b/slides/images/appcont.png new file mode 100644 index 00000000..8ffff1ce Binary files /dev/null and b/slides/images/appcont.png differ diff --git a/slides/images/background-containers.jpg b/slides/images/background-containers.jpg new file mode 100644 index 00000000..828a265d Binary files /dev/null and b/slides/images/background-containers.jpg differ diff --git a/slides/images/complex-network.jpg b/slides/images/complex-network.jpg new file mode 100644 index 00000000..b3d81e6a Binary files /dev/null and b/slides/images/complex-network.jpg differ diff --git a/slides/images/composeapp.png b/slides/images/composeapp.png new file mode 100644 index 00000000..907731bf Binary files /dev/null and b/slides/images/composeapp.png differ diff --git a/slides/images/composeup.gif b/slides/images/composeup.gif new file mode 100644 index 00000000..b84db1c9 Binary files /dev/null and b/slides/images/composeup.gif differ diff --git a/slides/images/construction.jpg b/slides/images/construction.jpg new file mode 100644 index 00000000..0fc7b480 Binary files /dev/null and b/slides/images/construction.jpg differ diff --git a/slides/images/construction1.jpg b/slides/images/construction1.jpg new file mode 100644 index 00000000..94736b88 Binary files /dev/null and b/slides/images/construction1.jpg differ diff --git a/slides/images/construction2.jpg b/slides/images/construction2.jpg new file mode 100644 index 00000000..94736b88 Binary files /dev/null and b/slides/images/construction2.jpg differ diff --git a/slides/images/containermarkings.jpg b/slides/images/containermarkings.jpg new file mode 100644 index 00000000..65d5f6c4 Binary files /dev/null and b/slides/images/containermarkings.jpg differ diff --git a/slides/images/copy.jpg b/slides/images/copy.jpg new file mode 100644 index 00000000..eb14228c Binary files /dev/null and b/slides/images/copy.jpg differ diff --git a/slides/images/diagram.odg b/slides/images/diagram.odg new file mode 100644 index 00000000..a35d409f Binary files /dev/null and b/slides/images/diagram.odg differ diff --git a/slides/images/diagram.png b/slides/images/diagram.png new file mode 100644 index 00000000..5acfcb51 Binary files /dev/null and b/slides/images/diagram.png differ diff --git a/slides/images/elimatrix.png b/slides/images/elimatrix.png new file mode 100644 index 00000000..8a2cdf66 Binary files /dev/null and b/slides/images/elimatrix.png differ diff --git a/slides/images/end.jpg b/slides/images/end.jpg new file mode 100644 index 00000000..9e4e424d Binary files /dev/null and b/slides/images/end.jpg differ diff --git a/slides/images/entrypoint.jpg b/slides/images/entrypoint.jpg new file mode 100644 index 00000000..0d10360f Binary files /dev/null and b/slides/images/entrypoint.jpg differ diff --git a/slides/images/firstcontainers.jpg b/slides/images/firstcontainers.jpg new file mode 100644 index 00000000..58d76703 Binary files /dev/null and b/slides/images/firstcontainers.jpg differ diff --git a/slides/images/graph.gif b/slides/images/graph.gif new file mode 100644 index 00000000..0ed3f4c0 Binary files /dev/null and b/slides/images/graph.gif differ diff --git a/slides/images/heroku-first-homepage.png b/slides/images/heroku-first-homepage.png new file mode 100644 index 00000000..519bf2df Binary files /dev/null and b/slides/images/heroku-first-homepage.png differ diff --git a/slides/images/image.png b/slides/images/image.png new file mode 100644 index 00000000..e4e20872 Binary files /dev/null and b/slides/images/image.png differ diff --git a/slides/images/install.jpg b/slides/images/install.jpg new file mode 100644 index 00000000..bb7f31d2 Binary files /dev/null and b/slides/images/install.jpg differ diff --git a/slides/images/lightcont.png b/slides/images/lightcont.png new file mode 100644 index 00000000..74b91687 Binary files /dev/null and b/slides/images/lightcont.png differ diff --git a/slides/images/matrix.png b/slides/images/matrix.png new file mode 100644 index 00000000..12358c96 Binary files /dev/null and b/slides/images/matrix.png differ diff --git a/slides/images/network.jpg b/slides/images/network.jpg new file mode 100644 index 00000000..68f6b961 Binary files /dev/null and b/slides/images/network.jpg differ diff --git a/slides/images/problem.png b/slides/images/problem.png new file mode 100644 index 00000000..f9e1cafd Binary files /dev/null and b/slides/images/problem.png differ diff --git a/slides/images/shipeco.png b/slides/images/shipeco.png new file mode 100644 index 00000000..977d81e4 Binary files /dev/null and b/slides/images/shipeco.png differ diff --git a/slides/images/shipping.png b/slides/images/shipping.png new file mode 100644 index 00000000..7513b09a Binary files /dev/null and b/slides/images/shipping.png differ diff --git a/slides/images/shippingindustry.png b/slides/images/shippingindustry.png new file mode 100644 index 00000000..f4794eba Binary files /dev/null and b/slides/images/shippingindustry.png differ diff --git a/slides/images/ssh.jpg b/slides/images/ssh.jpg new file mode 100644 index 00000000..512c2076 Binary files /dev/null and b/slides/images/ssh.jpg differ diff --git a/slides/images/stenciling-wall.jpg b/slides/images/stenciling-wall.jpg new file mode 100644 index 00000000..e0f1ddb7 Binary files /dev/null and b/slides/images/stenciling-wall.jpg differ diff --git a/slides/images/trainingwheels-error.png b/slides/images/trainingwheels-error.png new file mode 100644 index 00000000..c4b28cce Binary files /dev/null and b/slides/images/trainingwheels-error.png differ diff --git a/slides/images/trainingwheels-ok.png b/slides/images/trainingwheels-ok.png new file mode 100644 index 00000000..8ed1f4e0 Binary files /dev/null and b/slides/images/trainingwheels-ok.png differ diff --git a/slides/images/volume.jpg b/slides/images/volume.jpg new file mode 100644 index 00000000..0c725329 Binary files /dev/null and b/slides/images/volume.jpg differ diff --git a/slides/images/web.png b/slides/images/web.png new file mode 100644 index 00000000..5ce935fa Binary files /dev/null and b/slides/images/web.png differ diff --git a/slides/images/webapp1.png b/slides/images/webapp1.png new file mode 100644 index 00000000..41cd9a4f Binary files /dev/null and b/slides/images/webapp1.png differ diff --git a/slides/images/webapp2.png b/slides/images/webapp2.png new file mode 100644 index 00000000..711a0c3e Binary files /dev/null and b/slides/images/webapp2.png differ diff --git a/slides/intro.yml b/slides/intro.yml new file mode 100644 index 00000000..6a3d842e --- /dev/null +++ b/slides/intro.yml @@ -0,0 +1,91 @@ +title: "LISA17 M7: Getting Started with Docker and Containers" + +chat: "[Slack](https://usenix-lisa.slack.com/messages/C0E6N1NJW)" + +chapters: +- | + class: title + + .small[ + + LISA17 M7 + + Getting Started
with Docker and Containers + + .small[.small[ + + **Be kind to the WiFi!** + + *Use the 5G network* +
+ *Don't use your hotspot* +
+ *Don't stream videos from YouTube, Netflix, etc. +
(if you're bored, watch local content instead)* + + + + Thank you! + + ] + ] + ] + + --- + + ## Logistics + + - Hello! We are + Jérôme ([@jpetazzo](https://twitter.com/jpetazzo), Docker Inc.) + & + AJ ([@s0ulshake](https://twitter.com/s0ulshake), Travis CI) + + - The tutorial will run from 1:30pm to 5:00pm + + - This will be fast-paced, but DON'T PANIC! + + - There will be a coffee break at 3:00pm +
+ (please remind me if I forget about it!) + + - All the content is publicly available + + One URL to remember: http://container.training + + - Feel free to interrupt for questions at any time + + - Live feedback, questions, help on @@CHAT@@ + +- | + @@TOC@@ +- - intro/Docker_Overview.md + #- intro/Docker_History.md + - intro/Training_Environment.md + - intro/Install_Docker.md + - intro/First_Containers.md + - intro/Background_Containers.md + - intro/Start_And_Attach.md +- - intro/Initial_Images.md + - intro/Building_Images_Interactively.md + - intro/Building_Images_With_Dockerfiles.md + - intro/Cmd_And_Entrypoint.md + - intro/Copying_Files_During_Build.md + - intro/Multi_Stage_Builds.md + - intro/Dockerfile_Tips.md + #- intro/Advanced_Dockerfiles.md + - intro/Docker_Hub_Tease.md +- - intro/Naming_And_Inspecting.md + - intro/Container_Networking_Basics.md + - intro/Container_Network_Model.md + #- intro/Connecting_Containers_With_Links.md + - intro/Ambassadors.md +- - intro/Local_Development_Workflow.md + - intro/Working_With_Volumes.md + - intro/Compose_For_Dev_Stacks.md + - intro/Course_Conclusion.md diff --git a/slides/intro/Advanced_Dockerfiles.md b/slides/intro/Advanced_Dockerfiles.md new file mode 100644 index 00000000..4c489634 --- /dev/null +++ b/slides/intro/Advanced_Dockerfiles.md @@ -0,0 +1,437 @@ + +class: title + +# Advanced Dockerfiles + +![construction](Dockerfile_Reference/construction.jpg) + +--- + +## Objectives + +We have seen simple Dockerfiles to illustrate how Docker build +container images. + +In this section, we will see more Dockerfile commands. + +--- + +## `Dockerfile` usage summary + +* `Dockerfile` instructions are executed in order. + +* Each instruction creates a new layer in the image. + +* Docker maintains a cache with the layers of previous builds. + +* When there are no changes in the instructions and files making a layer, + the builder re-uses the cached layer, without executing the instruction for that layer. + +* The `FROM` instruction MUST be the first non-comment instruction. + +* Lines starting with `#` are treated as comments. + +* Some instructions (like `CMD` or `ENTRYPOINT`) update a piece of metadata. + + (As a result, each call to these instructions makes the previous one useless.) + +--- + +## The `MAINTAINER` instruction + +The `MAINTAINER` instruction tells you who wrote the `Dockerfile`. + +```dockerfile +MAINTAINER Docker Education Team +``` + +It's optional but recommended. + +--- + +## The `RUN` instruction + +The `RUN` instruction can be specified in two ways. + +With shell wrapping, which runs the specified command inside a shell, +with `/bin/sh -c`: + +```dockerfile +RUN apt-get update +``` + +Or using the `exec` method, which avoids shell string expansion, and +allows execution in images that don't have `/bin/sh`: + +```dockerfile +RUN [ "apt-get", "update" ] +``` + +--- + +## More about the `RUN` instruction + +`RUN` will do the following: + +* Execute a command. +* Record changes made to the filesystem. +* Work great to install libraries, packages, and various files. + +`RUN` will NOT do the following: + +* Record state of *processes*. +* Automatically start daemons. + +If you want to start something automatically when the container runs, +you should use `CMD` and/or `ENTRYPOINT`. + +--- + +## Collapsing layers + +It is possible to execute multiple commands in a single step: + +```dockerfile +RUN apt-get update && apt-get install -y wget && apt-get clean +``` + +It is also possible to break a command onto multiple lines: + +It is possible to execute multiple commands in a single step: + +```dockerfile +RUN apt-get update \ + && apt-get install -y wget \ + && apt-get clean +``` + +--- + +## The `EXPOSE` instruction + +The `EXPOSE` instruction tells Docker what ports are to be published +in this image. + +```dockerfile +EXPOSE 8080 +EXPOSE 80 443 +EXPOSE 53/tcp 53/udp +``` + +* All ports are private by default. + +* Declaring a port with `EXPOSE` is not enough to make it public. + +* The `Dockerfile` doesn't control on which port a service gets exposed. + +--- + +## Exposing ports + +* When you `docker run -p ...`, that port becomes public. + + (Even if it was not declared with `EXPOSE`.) + +* When you `docker run -P ...` (without port number), all ports + declared with `EXPOSE` become public. + +A *public port* is reachable from other containers and from outside the host. + +A *private port* is not reachable from outside. + +--- + +## The `COPY` instruction + +The `COPY` instruction adds files and content from your host into the +image. + +```dockerfile +COPY . /src +``` + +This will add the contents of the *build context* (the directory +passed as an argument to `docker build`) to the directory `/src` +in the container. + +--- + +## Build context isolation + +Note: you can only reference files and directories *inside* the +build context. Absolute paths are taken as being anchored to +the build context, so the two following lines are equivalent: + +```dockerfile +COPY . /src +COPY / /src +``` + +Attempts to use `..` to get out of the build context will be +detected and blocked with Docker, and the build will fail. + +Otherwise, a `Dockerfile` could succeed on host A, but fail on host B. + +--- + +## `ADD` + +`ADD` works almost like `COPY`, but has a few extra features. + +`ADD` can get remote files: + +```dockerfile +ADD http://www.example.com/webapp.jar /opt/ +``` + +This would download the `webapp.jar` file and place it in the `/opt` +directory. + +`ADD` will automatically unpack zip files and tar archives: + +```dockerfile +ADD ./assets.zip /var/www/htdocs/assets/ +``` + +This would unpack `assets.zip` into `/var/www/htdocs/assets`. + +*However,* `ADD` will not automatically unpack remote archives. + +--- + +## `ADD`, `COPY`, and the build cache + +* Before creating a new layer, Docker checks its build cache. + +* For most Dockerfile instructions, Docker only looks at the + `Dockerfile` content to do the cache lookup. + +* For `ADD` and `COPY` instructions, Docker also checks if the files + to be added to the container have been changed. + +* `ADD` always needs to download the remote file before + it can check if it has been changed. + + (It cannot use, + e.g., ETags or If-Modified-Since headers.) + +--- + +## `VOLUME` + +The `VOLUME` instruction tells Docker that a specific directory +should be a *volume*. + +```dockerfile +VOLUME /var/lib/mysql +``` + +Filesystem access in volumes bypasses the copy-on-write layer, +offering native performance to I/O done in those directories. + +Volumes can be attached to multiple containers, allowing to +"port" data over from a container to another, e.g. to +upgrade a database to a newer version. + +It is possible to start a container in "read-only" mode. +The container filesystem will be made read-only, but volumes +can still have read/write access if necessary. + +--- + +## The `WORKDIR` instruction + +The `WORKDIR` instruction sets the working directory for subsequent +instructions. + +It also affects `CMD` and `ENTRYPOINT`, since it sets the working +directory used when starting the container. + +```dockerfile +WORKDIR /src +``` + +You can specify `WORKDIR` again to change the working directory for +further operations. + +--- + +## The `ENV` instruction + +The `ENV` instruction specifies environment variables that should be +set in any container launched from the image. + +```dockerfile +ENV WEBAPP_PORT 8080 +``` + +This will result in an environment variable being created in any +containers created from this image of + +```bash +WEBAPP_PORT=8080 +``` + +You can also specify environment variables when you use `docker run`. + +```bash +$ docker run -e WEBAPP_PORT=8000 -e WEBAPP_HOST=www.example.com ... +``` + +--- + +## The `USER` instruction + +The `USER` instruction sets the user name or UID to use when running +the image. + +It can be used multiple times to change back to root or to another user. + +--- + +## The `CMD` instruction + +The `CMD` instruction is a default command run when a container is +launched from the image. + +```dockerfile +CMD [ "nginx", "-g", "daemon off;" ] +``` + +Means we don't need to specify `nginx -g "daemon off;"` when running the +container. + +Instead of: + +```bash +$ docker run /web_image nginx -g "daemon off;" +``` + +We can just do: + +```bash +$ docker run /web_image +``` + +--- + +## More about the `CMD` instruction + +Just like `RUN`, the `CMD` instruction comes in two forms. +The first executes in a shell: + +```dockerfile +CMD nginx -g "daemon off;" +``` + +The second executes directly, without shell processing: + +```dockerfile +CMD [ "nginx", "-g", "daemon off;" ] +``` + +--- + +class: extra-details + +## Overriding the `CMD` instruction + +The `CMD` can be overridden when you run a container. + +```bash +$ docker run -it /web_image bash +``` + +Will run `bash` instead of `nginx -g "daemon off;"`. + +--- + +## The `ENTRYPOINT` instruction + +The `ENTRYPOINT` instruction is like the `CMD` instruction, +but arguments given on the command line are *appended* to the +entry point. + +Note: you have to use the "exec" syntax (`[ "..." ]`). + +```dockerfile +ENTRYPOINT [ "/bin/ls" ] +``` + +If we were to run: + +```bash +$ docker run training/ls -l +``` + +Instead of trying to run `-l`, the container will run `/bin/ls -l`. + +--- + +class: extra-details + +## Overriding the `ENTRYPOINT` instruction + +The entry point can be overriden as well. + +```bash +$ docker run -it training/ls +bin dev home lib64 mnt proc run srv tmp var +boot etc lib media opt root sbin sys usr +$ docker run -it --entrypoint bash training/ls +root@d902fb7b1fc7:/# +``` + +--- + +## How `CMD` and `ENTRYPOINT` interact + +The `CMD` and `ENTRYPOINT` instructions work best when used +together. + +```dockerfile +ENTRYPOINT [ "nginx" ] +CMD [ "-g", "daemon off;" ] +``` + +The `ENTRYPOINT` specifies the command to be run and the `CMD` +specifies its options. On the command line we can then potentially +override the options when needed. + +```bash +$ docker run -d /web_image -t +``` + +This will override the options `CMD` provided with new flags. + +--- + +## Advanced Dockerfile instructions + +* `ONBUILD` lets you stash instructions that will be executed + when this image is used as a base for another one. +* `LABEL` adds arbitrary metadata to the image. +* `ARG` defines build-time variables (optional or mandatory). +* `STOPSIGNAL` sets the signal for `docker stop` (`TERM` by default). +* `HEALTHCHECK` defines a command assessing the status of the container. +* `SHELL` sets the default program to use for string-syntax RUN, CMD, etc. + +--- + +class: extra-details + +## The `ONBUILD` instruction + +The `ONBUILD` instruction is a trigger. It sets instructions that will +be executed when another image is built from the image being build. + +This is useful for building images which will be used as a base +to build other images. + +```dockerfile +ONBUILD COPY . /src +``` + +* You can't chain `ONBUILD` instructions with `ONBUILD`. +* `ONBUILD` can't be used to trigger `FROM` and `MAINTAINER` + instructions. diff --git a/slides/intro/Ambassadors.md b/slides/intro/Ambassadors.md new file mode 100644 index 00000000..a3c122dc --- /dev/null +++ b/slides/intro/Ambassadors.md @@ -0,0 +1,168 @@ + +class: title + +# Ambassadors + +![](Ambassadors/ambassador.jpg) + +--- + +## The ambassador pattern + +Ambassadors are containers that "masquerade" or "proxy" for another service. + +They abstract the connection details for this services, and can help with: + +* discovery (where is my service actually running?) + +* migration (what if my service has to be moved while I use it?) + +* fail over (how do I know to which instance of a replicated service I should connect?) + +* load balancing (how to I spread my requests across multiple instances of a service?) + +* authentication (what if my service requires credentials, certificates, or otherwise?) + +--- + +## Introduction to Ambassadors + +The ambassador pattern: + +* Takes advantage of Docker's per-container naming system and abstracts + connections between services. + +* Allows you to manage services without hard-coding connection + information inside applications. + +To do this, instead of directly connecting containers you insert +ambassador containers. + +--- + +![ambassador](Ambassadors/diagram.png) + +--- + +## Interacting with ambassadors + +* The web container uses normal Docker networking to connect + to the ambassador. + +* The database container also talks with an ambassador. + +* For both containers, the ambassador is totally transparent. +
+ (There is no difference between normal + operation and operation with an ambassador.) + +* If the database container is moved (or a failover happens), its new location will + be tracked by the ambassador containers, and the web application + container will still be able to connect, without reconfiguration. + +--- + +## Ambassadors for simple service discovery + +Use case: + +* my application code connects to `redis` on the default port (6379), +* my Redis service runs on another machine, on a non-default port (e.g. 12345), +* I want to use an ambassador to let my application connect without modification. + +The ambassador will be: + +* a container running right next to my application, +* using the name `redis` (or linked as `redis`), +* listening on port 6379, +* forwarding connections to the actual Redis service. + +--- + +## Ambassadors for service migration + +Use case: + +* my application code still connects to `redis`, +* my Redis service runs somewhere else, +* my Redis service is moved to a different host+port, +* the location of the Redis service is given to me via e.g. DNS SRV records, +* I want to use an ambassador to automatically connect to the new location, with as little disruption as possible. + +The ambassador will be: + +* the same kind of container as before, +* running an additional routine to monitor DNS SRV records, +* updating the forwarding destination when the DNS SRV records are updated. + +--- + +## Ambassadors for credentials injection + +Use case: + +* my application code still connects to `redis`, +* my application code doesn't provide Redis credentials, +* my production Redis service requires credentials, +* my staging Redis service requires different credentials, +* I want to use an ambassador to abstract those credentials. + +The ambassador will be: + +* a container using the name `redis` (or a link), +* passed the credentials to use, +* running a custom proxy that accepts connections on Redis default port, +* performing authentication with the target Redis service before forwarding traffic. + +--- + +## Ambassadors for load balancing + +Use case: + +* my application code connects to a web service called `api`, +* I want to run multiple instances of the `api` backend, +* those instances will be on different machines and ports, +* I want to use an ambassador to abstract those details. + +The ambassador will be: + +* a container using the name `api` (or a link), +* passed the list of backends to use (statically or dynamically), +* running a load balancer (e.g. HAProxy or NGINX), +* dispatching requests across all backends transparently. + +--- + +## "Ambassador" is a *pattern* + +There are many ways to implement the pattern. + +Different deployments will use different underlying technologies. + +* On-premise deployments with a trusted network can track + container locations in e.g. Zookeeper, and generate HAproxy + configurations each time a location key changes. +* Public cloud deployments or deployments across unsafe + networks can add TLS encryption. +* Ad-hoc deployments can use a master-less discovery protocol + like avahi to register and discover services. +* It is also possible to do one-shot reconfiguration of the + ambassadors. It is slightly less dynamic but has much less + requirements. +* Ambassadors can be used in addition to, or instead of, overlay networks. + +--- + +## Section summary + +We've learned how to: + +* Understand the ambassador pattern and what it is used for (service portability). + +For more information about the ambassador pattern, including demos on Swarm and ECS: + +* AWS re:invent 2015 [DVO317](https://www.youtube.com/watch?v=7CZFpHUPqXw) + +* [SwarmWeek video about Swarm+Compose](https://youtube.com/watch?v=qbIvUvwa6As) + diff --git a/slides/intro/Background_Containers.md b/slides/intro/Background_Containers.md new file mode 100644 index 00000000..605ad81e --- /dev/null +++ b/slides/intro/Background_Containers.md @@ -0,0 +1,278 @@ + +class: title + +# Background Containers + +![Background Containers](Background_Containers/background-containers.jpg) + +--- + +## Objectives + +Our first containers were *interactive*. + +We will now see how to: + +* Run a non-interactive container. +* Run a container in the background. +* List running containers. +* Check the logs of a container. +* Stop a container. +* List stopped containers. + +--- + +## A non-interactive container + +We will run a small custom container. + +This container just displays the time every second. + +```bash +$ docker run jpetazzo/clock +Fri Feb 20 00:28:53 UTC 2015 +Fri Feb 20 00:28:54 UTC 2015 +Fri Feb 20 00:28:55 UTC 2015 +... +``` + +* This container will run forever. +* To stop it, press `^C`. +* Docker has automatically downloaded the image `jpetazzo/clock`. +* This image is a user image, created by `jpetazzo`. +* We will hear more about user images (and other types of images) later. + +--- + +## Run a container in the background + +Containers can be started in the background, with the `-d` flag (daemon mode): + +```bash +$ docker run -d jpetazzo/clock +47d677dcfba4277c6cc68fcaa51f932b544cab1a187c853b7d0caf4e8debe5ad +``` + +* We don't see the output of the container. +* But don't worry: Docker collects that output and logs it! +* Docker gives us the ID of the container. + +--- + +## List running containers + +How can we check that our container is still running? + +With `docker ps`, just like the UNIX `ps` command, lists running processes. + +```bash +$ docker ps +CONTAINER ID IMAGE ... CREATED STATUS ... +47d677dcfba4 jpetazzo/clock ... 2 minutes ago Up 2 minutes ... +``` + +Docker tells us: + +* The (truncated) ID of our container. +* The image used to start the container. +* That our container has been running (`Up`) for a couple of minutes. +* Other information (COMMAND, PORTS, NAMES) that we will explain later. + +--- + +## Starting more containers + +Let's start two more containers. + +```bash +$ docker run -d jpetazzo/clock +57ad9bdfc06bb4407c47220cf59ce21585dce9a1298d7a67488359aeaea8ae2a +``` + +```bash +$ docker run -d jpetazzo/clock +068cc994ffd0190bbe025ba74e4c0771a5d8f14734af772ddee8dc1aaf20567d +``` + +Check that `docker ps` correctly reports all 3 containers. + +--- + +## Viewing only the last container started + +When many containers are already running, it can be useful to +see only the last container that was started. + +This can be achieved with the `-l` ("Last") flag: + +```bash +$ docker ps -l +CONTAINER ID IMAGE ... CREATED STATUS ... +068cc994ffd0 jpetazzo/clock ... 2 minutes ago Up 2 minutes ... +``` + +--- + +## View only the IDs of the containers + +Many Docker commands will work on container IDs: `docker stop`, `docker rm`... + +If we want to list only the IDs of our containers (without the other colums +or the header line), +we can use the `-q` ("Quiet", "Quick") flag: + +```bash +$ docker ps -q +068cc994ffd0 +57ad9bdfc06b +47d677dcfba4 +``` + +--- + +## Combining flags + +We can combine `-l` and `-q` to see only the ID of the last container started: + +```bash +$ docker ps -lq +068cc994ffd0 +``` + +At a first glance, it looks like this would be particularly useful in scripts. + +However, if we want to start a container and get its ID in a reliable way, +it is better to use `docker run -d`, which we will cover in a bit. + +--- + +## View the logs of a container + +We told you that Docker was logging the container output. + +Let's see that now. + +```bash +$ docker logs 068 +Fri Feb 20 00:39:52 UTC 2015 +Fri Feb 20 00:39:53 UTC 2015 +... +``` + +* We specified a *prefix* of the full container ID. +* You can, of course, specify the full ID. +* The `logs` command will output the *entire* logs of the container. +
(Sometimes, that will be too much. Let's see how to address that.) + +--- + +## View only the tail of the logs + +To avoid being spammed with eleventy pages of output, +we can use the `--tail` option: + +```bash +$ docker logs --tail 3 068 +Fri Feb 20 00:55:35 UTC 2015 +Fri Feb 20 00:55:36 UTC 2015 +Fri Feb 20 00:55:37 UTC 2015 +``` + +* The parameter is the number of lines that we want to see. + +--- + +## Follow the logs in real time + +Just like with the standard UNIX command `tail -f`, we can +follow the logs of our container: + +```bash +$ docker logs --tail 1 --follow 068 +Fri Feb 20 00:57:12 UTC 2015 +Fri Feb 20 00:57:13 UTC 2015 +^C +``` + +* This will display the last line in the log file. +* Then, it will continue to display the logs in real time. +* Use `^C` to exit. + +--- + +## Stop our container + +There are two ways we can terminate our detached container. + +* Killing it using the `docker kill` command. +* Stopping it using the `docker stop` command. + +The first one stops the container immediately, by using the +`KILL` signal. + +The second one is more graceful. It sends a `TERM` signal, +and after 10 seconds, if the container has not stopped, it +sends `KILL.` + +Reminder: the `KILL` signal cannot be intercepted, and will +forcibly terminate the container. + +--- + +## Stopping our containers + +Let's stop one of those containers: + +```bash +$ docker stop 47d6 +47d6 +``` + +This will take 10 seconds: + +* Docker sends the TERM signal; +* the container doesn't react to this signal + (it's a simple Shell script with no special + signal handling); +* 10 seconds later, since the container is still + running, Docker sends the KILL signal; +* this terminates the container. + +--- + +## Killing the remaining containers + +Let's be less patient with the two other containers: + +```bash +$ docker kill 068 57ad +068 +57ad +``` + +The `stop` and `kill` commands can take multiple container IDs. + +Those containers will be terminated immediately (without +the 10 seconds delay). + +Let's check that our containers don't show up anymore: + +```bash +$ docker ps +``` + +--- + +## List stopped containers + +We can also see stopped containers, with the `-a` (`--all`) option. + +```bash +$ docker ps -a +CONTAINER ID IMAGE ... CREATED STATUS +068cc994ffd0 jpetazzo/clock ... 21 min. ago Exited (137) 3 min. ago +57ad9bdfc06b jpetazzo/clock ... 21 min. ago Exited (137) 3 min. ago +47d677dcfba4 jpetazzo/clock ... 23 min. ago Exited (137) 3 min. ago +5c1dfd4d81f1 jpetazzo/clock ... 40 min. ago Exited (0) 40 min. ago +b13c164401fb ubuntu ... 55 min. ago Exited (130) 53 min. ago +``` diff --git a/slides/intro/Building_Images_Interactively.md b/slides/intro/Building_Images_Interactively.md new file mode 100644 index 00000000..19a9c414 --- /dev/null +++ b/slides/intro/Building_Images_Interactively.md @@ -0,0 +1,167 @@ +# Building Images Interactively + +In this section, we will create our first container image. + +It will be a basic distribution image, but we will pre-install +the package `figlet`. + +We will: + +* Create a container from a base image. + +* Install software manually in the container, and turn it + into a new image. + +* Learn about new commands: `docker commit`, `docker tag`, and `docker diff`. + +--- + +## Building Images Interactively + +As we have seen, the images on the Docker Hub are sometimes very basic. + +How do we want to construct our own images? + +As an example, we will build an image that has `figlet`. + +First, we will do it manually with `docker commit`. + +Then, in an upcoming chapter, we will use a `Dockerfile` and `docker build`. + +--- + +## Building from a base + +Our base will be the `ubuntu` image. + +--- + +## Create a new container and make some changes + +Start an Ubuntu container: + +```bash +$ docker run -it ubuntu +root@:#/ +``` + +Run the command `apt-get update` to refresh the list of packages available to install. + +Then run the command `apt-get install figlet` to install the program we are interested in. + +```bash +root@:#/ apt-get update && apt-get install figlet +.... OUTPUT OF APT-GET COMMANDS .... +``` + +--- + +## Inspect the changes + +Type `exit` at the container prompt to leave the interactive session. + +Now let's run `docker diff` to see the difference between the base image +and our container. + +```bash +$ docker diff +C /root +A /root/.bash_history +C /tmp +C /usr +C /usr/bin +A /usr/bin/figlet +... +``` + +--- + +class: x-extra-details + +## Docker tracks filesystem changes + +As explained before: + +* An image is read-only. + +* When we make changes, they happen in a copy of the image. + +* Docker can show the difference between the image, and its copy. + +* For performance, Docker uses copy-on-write systems. +
(i.e. starting a container based on a big image + doesn't incur a huge copy.) + +--- + +## Copy-on-write security benefits + +* `docker diff` gives us an easy way to audit changes + + (à la Tripwire) + +* Containers can also be started in read-only mode + + (their root filesystem will be read-only, but they can still have read-write data volumes) + + +--- + +## Commit and run your image + +The `docker commit` command will create a new layer with those changes, +and a new image using this new layer. + +```bash +$ docker commit + +``` + +The output of the `docker commit` command will be the ID for your newly created image. + +We can run this image: + +```bash +$ docker run -it +root@fcfb62f0bfde:/# figlet hello + _ _ _ +| |__ ___| | | ___ +| '_ \ / _ \ | |/ _ \ +| | | | __/ | | (_) | +|_| |_|\___|_|_|\___/ +``` + +--- + +## Tagging images + +Referring to an image by its ID is not convenient. Let's tag it instead. + +We can use the `tag` command: + +```bash +$ docker tag figlet +``` + +But we can also specify the tag as an extra argument to `commit`: + +```bash +$ docker commit figlet +``` + +And then run it using its tag: + +```bash +$ docker run -it figlet +``` + +--- + +## What's next? + +Manual process = bad. + +Automated process = good. + +In the next chapter, we will learn how to automate the build +process by writing a `Dockerfile`. diff --git a/slides/intro/Building_Images_With_Dockerfiles.md b/slides/intro/Building_Images_With_Dockerfiles.md new file mode 100644 index 00000000..24c7d071 --- /dev/null +++ b/slides/intro/Building_Images_With_Dockerfiles.md @@ -0,0 +1,284 @@ + +class: title + +# Building Docker images with a Dockerfile + +![construction](Building_Images_With_Dockerfiles/construction.jpg) + +--- + +## Objectives + +We will build a container image automatically, with a `Dockerfile`. + +At the end of this lesson, you will be able to: + +* Write a `Dockerfile`. + +* Build an image from a `Dockerfile`. + +--- + +## `Dockerfile` overview + +* A `Dockerfile` is a build recipe for a Docker image. + +* It contains a series of instructions telling Docker how an image is constructed. + +* The `docker build` command builds an image from a `Dockerfile`. + +--- + +## Writing our first `Dockerfile` + +Our Dockerfile must be in a **new, empty directory**. + +1. Create a directory to hold our `Dockerfile`. + +```bash +$ mkdir myimage +``` + +2. Create a `Dockerfile` inside this directory. + +```bash +$ cd myimage +$ vim Dockerfile +``` + +Of course, you can use any other editor of your choice. + +--- + +## Type this into our Dockerfile... + +```dockerfile +FROM ubuntu +RUN apt-get update +RUN apt-get install figlet +``` + +* `FROM` indicates the base image for our build. + +* Each `RUN` line will be executed by Docker during the build. + +* Our `RUN` commands **must be non-interactive.** +
(No input can be provided to Docker during the build.) + +* In many cases, we will add the `-y` flag to `apt-get`. + +--- + +## Build it! + +Save our file, then execute: + +```bash +$ docker build -t figlet . +``` + +* `-t` indicates the tag to apply to the image. + +* `.` indicates the location of the *build context*. + +We will talk more about the build context later. + +To keep things simple for now: this is the directory where our Dockerfile is located. + +--- + +## What happens when we build the image? + +The output of `docker build` looks like this: + +.small[ +```bash +$ docker build -t figlet . +Sending build context to Docker daemon 2.048 kB +Sending build context to Docker daemon +Step 0 : FROM ubuntu + ---> e54ca5efa2e9 +Step 1 : RUN apt-get update + ---> Running in 840cb3533193 + ---> 7257c37726a1 +Removing intermediate container 840cb3533193 +Step 2 : RUN apt-get install figlet + ---> Running in 2b44df762a2f + ---> f9e8f1642759 +Removing intermediate container 2b44df762a2f +Successfully built f9e8f1642759 +``` +] + +* The output of the `RUN` commands has been omitted. +* Let's explain what this output means. + +--- + +## Sending the build context to Docker + +```bash +Sending build context to Docker daemon 2.048 kB +``` + +* The build context is the `.` directory given to `docker build`. + +* It is sent (as an archive) by the Docker client to the Docker daemon. + +* This allows to use a remote machine to build using local files. + +* Be careful (or patient) if that directory is big and your link is slow. + +--- + +## Executing each step + +```bash +Step 1 : RUN apt-get update + ---> Running in 840cb3533193 +(...output of the RUN command...) + ---> 7257c37726a1 +Removing intermediate container 840cb3533193 +``` + +* A container (`840cb3533193`) is created from the base image. + +* The `RUN` command is executed in this container. + +* The container is committed into an image (`7257c37726a1`). + +* The build container (`840cb3533193`) is removed. + +* The output of this step will be the base image for the next one. + +--- + +## The caching system + +If you run the same build again, it will be instantaneous. Why? + +* After each build step, Docker takes a snapshot of the resulting image. + +* Before executing a step, Docker checks if it has already built the same sequence. + +* Docker uses the exact strings defined in your Dockerfile, so: + + * `RUN apt-get install figlet cowsay ` +
is different from +
`RUN apt-get install cowsay figlet` + + * `RUN apt-get update` is not re-executed when the mirrors are updated + +You can force a rebuild with `docker build --no-cache ...`. + +--- + +## Running the image + +The resulting image is not different from the one produced manually. + +```bash +$ docker run -ti figlet +root@91f3c974c9a1:/# figlet hello + _ _ _ +| |__ ___| | | ___ +| '_ \ / _ \ | |/ _ \ +| | | | __/ | | (_) | +|_| |_|\___|_|_|\___/ +``` + + +Yay! 🎉 + +--- + +## Using image and viewing history + +The `history` command lists all the layers composing an image. + +For each layer, it shows its creation time, size, and creation command. + +When an image was built with a Dockerfile, each layer corresponds to +a line of the Dockerfile. + +```bash +$ docker history figlet +IMAGE CREATED CREATED BY SIZE +f9e8f1642759 About an hour ago /bin/sh -c apt-get install fi 1.627 MB +7257c37726a1 About an hour ago /bin/sh -c apt-get update 21.58 MB +07c86167cdc4 4 days ago /bin/sh -c #(nop) CMD ["/bin 0 B + 4 days ago /bin/sh -c sed -i 's/^#\s*\( 1.895 kB + 4 days ago /bin/sh -c echo '#!/bin/sh' 194.5 kB + 4 days ago /bin/sh -c #(nop) ADD file:b 187.8 MB +``` + +--- + +## Introducing JSON syntax + +Most Dockerfile arguments can be passed in two forms: + +* plain string: +
`RUN apt-get install figlet` + +* JSON list: +
`RUN ["apt-get", "install", "figlet"]` + +We are going to change our Dockerfile to see how it affects the resulting image. + +--- + +## Using JSON syntax in our Dockerfile + +Let's change our Dockerfile as follows! + +```dockerfile +FROM ubuntu +RUN apt-get update +RUN ["apt-get", "install", "figlet"] +``` + +Then build the new Dockerfile. + +```bash +$ docker build -t figlet . +``` + +--- + +## JSON syntax vs string syntax + +Compare the new history: + +```bash +$ docker history figlet +IMAGE CREATED CREATED BY SIZE +27954bb5faaf 10 seconds ago apt-get install figlet 1.627 MB +7257c37726a1 About an hour ago /bin/sh -c apt-get update 21.58 MB +07c86167cdc4 4 days ago /bin/sh -c #(nop) CMD ["/bin 0 B + 4 days ago /bin/sh -c sed -i 's/^#\s*\( 1.895 kB + 4 days ago /bin/sh -c echo '#!/bin/sh' 194.5 kB + 4 days ago /bin/sh -c #(nop) ADD file:b 187.8 MB +``` + +* JSON syntax specifies an *exact* command to execute. + +* String syntax specifies a command to be wrapped within `/bin/sh -c "..."`. + +--- + +## When to use JSON syntax and string syntax + +* String syntax: + + * is easier to write + * interpolates environment variables and other shell expressions + * creates an extra process (`/bin/sh -c ...`) to parse the string + * requires `/bin/sh` to exist in the container + +* JSON syntax: + + * is harder to write (and read!) + * passes all arguments without extra processing + * doesn't create an extra process + * doesn't require `/bin/sh` to exist in the container diff --git a/slides/intro/Cmd_And_Entrypoint.md b/slides/intro/Cmd_And_Entrypoint.md new file mode 100644 index 00000000..1a3c646f --- /dev/null +++ b/slides/intro/Cmd_And_Entrypoint.md @@ -0,0 +1,265 @@ + +class: title + +# CMD and ENTRYPOINT + +![Container entry doors](Cmd_And_Entrypoint/entrypoint.jpg) + +--- + +## Objectives + +In this lesson, we will learn about two important +Dockerfile commands: + +`CMD` and `ENTRYPOINT`. + +These commands allow us to set the default command +to run in a container. + +--- + +## Defining a default command + +When people run our container, we want to greet them with a nice hello message, and using a custom font. + +For that, we will execute: + +```bash +figlet -f script hello +``` + +* `-f script` tells figlet to use a fancy font. + +* `hello` is the message that we want it to display. + +--- + +## Adding `CMD` to our Dockerfile + +Our new Dockerfile will look like this: + +```dockerfile +FROM ubuntu +RUN apt-get update +RUN ["apt-get", "install", "figlet"] +CMD figlet -f script hello +``` + +* `CMD` defines a default command to run when none is given. + +* It can appear at any point in the file. + +* Each `CMD` will replace and override the previous one. + +* As a result, while you can have multiple `CMD` lines, it is useless. + +--- + +## Build and test our image + +Let's build it: + +```bash +$ docker build -t figlet . +... +Successfully built 042dff3b4a8d +``` + +And run it: + +```bash +$ docker run figlet + _ _ _ +| | | | | | +| | _ | | | | __ +|/ \ |/ |/ |/ / \_ +| |_/|__/|__/|__/\__/ +``` + +--- + +## Overriding `CMD` + +If we want to get a shell into our container (instead of running +`figlet`), we just have to specify a different program to run: + +```bash +$ docker run -it figlet bash +root@7ac86a641116:/# +``` + +* We specified `bash`. + +* It replaced the value of `CMD`. + +--- + +## Using `ENTRYPOINT` + +We want to be able to specify a different message on the command line, +while retaining `figlet` and some default parameters. + +In other words, we would like to be able to do this: + +```bash +$ docker run figlet salut + _ + | | + , __, | | _|_ +/ \_/ | |/ | | | + \/ \_/|_/|__/ \_/|_/|_/ +``` + + +We will use the `ENTRYPOINT` verb in Dockerfile. + +--- + +## Adding `ENTRYPOINT` to our Dockerfile + +Our new Dockerfile will look like this: + +```dockerfile +FROM ubuntu +RUN apt-get update +RUN ["apt-get", "install", "figlet"] +ENTRYPOINT ["figlet", "-f", "script"] +``` + +* `ENTRYPOINT` defines a base command (and its parameters) for the container. + +* The command line arguments are appended to those parameters. + +* Like `CMD`, `ENTRYPOINT` can appear anywhere, and replaces the previous value. + +Why did we use JSON syntax for our `ENTRYPOINT`? + +--- + +## Implications of JSON vs string syntax + +* When CMD or ENTRYPOINT use string syntax, they get wrapped in `sh -c`. + +* To avoid this wrapping, you must use JSON syntax. + +What if we used `ENTRYPOINT` with string syntax? + +```bash +$ docker run figlet salut +``` + +This would run the following command in the `figlet` image: + +```bash +sh -c "figlet -f script" salut +``` + +--- + +## Build and test our image + +Let's build it: + +```bash +$ docker build -t figlet . +... +Successfully built 36f588918d73 +``` + +And run it: + +```bash +$ docker run figlet salut + _ + | | + , __, | | _|_ +/ \_/ | |/ | | | + \/ \_/|_/|__/ \_/|_/|_/ +``` + +Great success! + +--- + +## Using `CMD` and `ENTRYPOINT` together + +What if we want to define a default message for our container? + +Then we will use `ENTRYPOINT` and `CMD` together. + +* `ENTRYPOINT` will define the base command for our container. + +* `CMD` will define the default parameter(s) for this command. + +* They *both* have to use JSON syntax. + +--- + +## `CMD` and `ENTRYPOINT` together + +Our new Dockerfile will look like this: + +```dockerfile +FROM ubuntu +RUN apt-get update +RUN ["apt-get", "install", "figlet"] +ENTRYPOINT ["figlet", "-f", "script"] +CMD ["hello world"] +``` + +* `ENTRYPOINT` defines a base command (and its parameters) for the container. + +* If we don't specify extra command-line arguments when starting the container, + the value of `CMD` is appended. + +* Otherwise, our extra command-line arguments are used instead of `CMD`. + +--- + +## Build and test our image + +Let's build it: + +```bash +$ docker build -t figlet . +... +Successfully built 6e0b6a048a07 +``` + +And run it: + +.small[ +```bash +$ docker run figlet + _ _ _ _ +| | | | | | | | | +| | _ | | | | __ __ ,_ | | __| +|/ \ |/ |/ |/ / \_ | | |_/ \_/ | |/ / | +| |_/|__/|__/|__/\__/ \/ \/ \__/ |_/|__/\_/|_/ + +$ docker run figlet hola mundo + _ _ +| | | | | +| | __ | | __, _ _ _ _ _ __| __ +|/ \ / \_|/ / | / |/ |/ | | | / |/ | / | / \_ +| |_/\__/ |__/\_/|_/ | | |_/ \_/|_/ | |_/\_/|_/\__/ +``` +] + +--- + +## Overriding `ENTRYPOINT` + +What if we want to run a shell in our container? + +We cannot just do `docker run figlet bash` because +that would just tell figlet to display the word "bash." + +We use the `--entrypoint` parameter: + +```bash +$ docker run -it --entrypoint bash figlet +root@6027e44e2955:/# +``` + diff --git a/slides/intro/Compose_For_Dev_Stacks.md b/slides/intro/Compose_For_Dev_Stacks.md new file mode 100644 index 00000000..05a63869 --- /dev/null +++ b/slides/intro/Compose_For_Dev_Stacks.md @@ -0,0 +1,306 @@ + +# Compose For Development Stacks + +Dockerfiles are great to build container images. + +But what if we work with a complex stack made of multiple containers? + +Eventually, we will want to write some custom scripts and automation to build, run, and connect +our containers together. + +There is a better way: using Docker Compose. + +In this section, you will use Compose to bootstrap a development environment. + +--- + +## What is Docker Compose? + +Docker Compose (formerly known as `fig`) is an external tool. + +Unlike the Docker Engine, it is written in Python. It's open source as well. + +The general idea of Compose is to enable a very simple, powerful onboarding workflow: + +1. Checkout your code. + +2. Run `docker-compose up`. + +3. Your app is up and running! + +--- + +## Compose overview + +This is how you work with Compose: + +* You describe a set (or stack) of containers in a YAML file called `docker-compose.yml`. + +* You run `docker-compose up`. + +* Compose automatically pulls images, builds containers, and starts them. + +* Compose can set up links, volumes, and other Docker options for you. + +* Compose can run the containers in the background, or in the foreground. + +* When containers are running in the foreground, their aggregated output is shown. + +Before diving in, let's see a small example of Compose in action. + +--- + +## Compose in action + +![composeup](Compose_For_Dev_Stacks/composeup.gif) + +--- + +## Checking if Compose is installed + +If you are using the official training virtual machines, Compose has been +pre-installed. + +You can always check that it is installed by running: + +```bash +$ docker-compose --version +``` + +--- + +## Launching Our First Stack with Compose + +First step: clone the source code for the app we will be working on. + +```bash +$ cd +$ git clone git://github.com/jpetazzo/trainingwheels +... +$ cd trainingwheels +``` + + +Second step: start your app. + +```bash +$ docker-compose up +``` + +Watch Compose build and run your app with the correct parameters, +including linking the relevant containers together. + +--- + +## Launching Our First Stack with Compose + +Verify that the app is running at `http://:8000`. + +![composeapp](Compose_For_Dev_Stacks/composeapp.png) + +--- + +## Stopping the app + +When you hit `^C`, Compose tries to gracefully terminate all of the containers. + +After ten seconds (or if you press `^C` again) it will forcibly kill +them. + +--- + +## The `docker-compose.yml` file + +Here is the file used in the demo: + +```yaml +version: "2" + +services: + www: + build: www + ports: + - 8000:5000 + user: nobody + environment: + DEBUG: 1 + command: python counter.py + volumes: + - ./www:/src + + redis: + image: redis +``` + +--- + +## Compose file versions + +Version 1 directly has the various containers (`www`, `redis`...) at the top level of the file. + +Version 2 has multiple sections: + +* `version` is mandatory and should be `"2"`. + +* `services` is mandatory and corresponds to the content of the version 1 format. + +* `networks` is optional and indicates to which networks containers should be connected. +
(By default, containers will be connected on a private, per-app network.) + +* `volumes` is optional and can define volumes to be used and/or shared by the containers. + +Version 3 adds support for deployment options (scaling, rolling updates, etc.) + +--- + +## Containers in `docker-compose.yml` + +Each service in the YAML file must contain either `build`, or `image`. + +* `build` indicates a path containing a Dockerfile. + +* `image` indicates an image name (local, or on a registry). + +* If both are specified, an image will be built from the `build` directory and named `image`. + +The other parameters are optional. + +They encode the parameters that you would typically add to `docker run`. + +Sometimes they have several minor improvements. + +--- + +## Container parameters + +* `command` indicates what to run (like `CMD` in a Dockerfile). + +* `ports` translates to one (or multiple) `-p` options to map ports. +
You can specify local ports (i.e. `x:y` to expose public port `x`). + +* `volumes` translates to one (or multiple) `-v` options. +
You can use relative paths here. + +For the full list, check: https://docs.docker.com/compose/compose-file/ + +--- + +## Compose commands + +We already saw `docker-compose up`, but another one is `docker-compose build`. + +It will execute `docker build` for all containers mentioning a `build` path. + +It can also be invoked automatically when starting the application: + +```bash +docker-compose up --build +``` + +Another common option is to start containers in the background: + +```bash +docker-compose up -d +``` + +--- + +## Check container status + +It can be tedious to check the status of your containers with `docker ps`, +especially when running multiple apps at the same time. + +Compose makes it easier; with `docker-compose ps` you will see only the status of the +containers of the current stack: + + +```bash +$ docker-compose ps +Name Command State Ports +---------------------------------------------------------------------------- +trainingwheels_redis_1 /entrypoint.sh red Up 6379/tcp +trainingwheels_www_1 python counter.py Up 0.0.0.0:8000->5000/tcp +``` + +--- + +## Cleaning up (1) + +If you have started your application in the background with Compose and +want to stop it easily, you can use the `kill` command: + +```bash +$ docker-compose kill +``` + +Likewise, `docker-compose rm` will let you remove containers (after confirmation): + +```bash +$ docker-compose rm +Going to remove trainingwheels_redis_1, trainingwheels_www_1 +Are you sure? [yN] y +Removing trainingwheels_redis_1... +Removing trainingwheels_www_1... +``` + +--- + +## Cleaning up (2) + +Alternatively, `docker-compose down` will stop and remove containers. + +It will also remove other resources, like networks that were created for the application. + +```bash +$ docker-compose down +Stopping trainingwheels_www_1 ... done +Stopping trainingwheels_redis_1 ... done +Removing trainingwheels_www_1 ... done +Removing trainingwheels_redis_1 ... done +``` + +--- + +## Special handling of volumes + +Compose is smart. If your container uses volumes, when you restart your +application, Compose will create a new container, but carefully re-use +the volumes it was using previously. + +This makes it easy to upgrade a stateful service, by pulling its +new image and just restarting your stack with Compose. + +--- + +## Compose project name + +* When you run a Compose command, Compose infers the "project name" of your app. + +* By default, the "project name" is the name of the current directory. + +* For instance, if you are in `/home/zelda/src/ocarina`, the project name is `ocarina`. + +* All resources created by Compose are tagged with this project name. + +* The project name also appears as a prefix of the names of the resources. + + E.g. in the previous example, service `www` will create a container `ocarina_www_1`. + +* The project name can be overridden with `docker-compose -p`. + +--- + +## Running two copies of the same app + +If you want to run two copies of the same app simultaneously, all you have to do is to +make sure that each copy has a different project name. + +You can: + +* copy your code in a directory with a different name + +* start each copy with `docker-compose -p myprojname up` + +Each copy will run in a different network, totally isolated from the other. + +This is ideal to debug regressions, do side-by-side comparisons, etc. diff --git a/slides/intro/Connecting_Containers_With_Links.md b/slides/intro/Connecting_Containers_With_Links.md new file mode 100644 index 00000000..40cd4fc7 --- /dev/null +++ b/slides/intro/Connecting_Containers_With_Links.md @@ -0,0 +1,223 @@ + +class: title + +# Connecting Containers With Links + +![graph](Connecting_Containers_With_Links/graph.gif) + +--- + +## Objectives + +Links were the "legacy" way of connecting containers (before the implementation of the CNM). + +They are still useful in some scenarios. + +--- + +## How *links* work + +* Links are created *between two containers* +* Links are created *from the client to the server* +* Links associate an arbitrary name to an existing container +* Links exist *only in the context of the client* + +--- + +class: extra-details + +## The plan + +* We will create the `redis` container first. +* Then, we will create the `www` container, *with a link to the previous container.* +* We don't need to use a custom network for this to work. + +--- + +class: extra-details + +## Create the `redis` container + +Let's launch a container from the `redis` image. + +```bash +$ docker run -d --name datastore redis + +``` + +Let's check the container is running: + +```bash +$ docker ps -l +CONTAINER ID IMAGE COMMAND ... PORTS NAMES +9efd72a4f320 redis:latest redis-server ... 6379/tcp datastore +``` + + +* Our container is launched and running an instance of Redis. +* We used the `--name` flag to reference our container easily later. +* We could have used *any name we wanted.* + +--- + +class: extra-details + +## Create the `www` container + +If we create the web container without any extra option, it will not be able to connect to redis. + +```bash +$ docker run -dP jpetazzo/trainingwheels +``` + +Check the port number with `docker ps`, and connect to it. + +We get the same red error page as before. + +--- + +class: extra-details + +## How our app connects to Redis + +Remember, in the code, we connect to the name `redis`: + +```python +redis = redis.Redis("redis") +``` + +* This means "try to connect to 'redis'". +* Not 192.168.123.234. +* Not redis.prod.mycompany.net. + +*Obviously* it doesn't work. + +--- + +class: extra-details + +## Creating a linked container + +Docker allows to specify *links*. + +Links indicate an intent: "this container will connect to this other container." + +Here is how to create our first link: + +```bash +$ docker run -ti --link datastore:redis alpine sh +``` + +In this container, we can communicate with `datastore` using +the `redis` DNS alias. + +--- + +class: extra-details + +## DNS + +Docker has created a DNS entry for the container, resolving to its internal IP address. + +```bash +$ docker run -it --link datastore:redis alpine ping redis +PING redis (172.17.0.29): 56 data bytes +64 bytes from 172.17.0.29: icmp_seq=0 ttl=64 time=0.164 ms +64 bytes from 172.17.0.29: icmp_seq=1 ttl=64 time=0.122 ms +64 bytes from 172.17.0.29: icmp_seq=2 ttl=64 time=0.086 ms +^C--- redis ping statistics --- +3 packets transmitted, 3 packets received, 0% packet loss +round-trip min/avg/max/stddev = 0.086/0.124/0.164/0.032 ms +``` + + +* The `--link` flag connects one container to another. +* We specify the name of the container to link to, `datastore`, and an + alias for the link, `redis`, in the format `name:alias`. + +--- + +class: extra-details + +## Starting our application + +Now that we've poked around a bit let's start the application itself in +a fresh container: + +```bash +$ docker run -d -P --link datastore:redis jpetazzo/trainingwheels +``` + +Now let's check the port number associated to the container. + +```bash +$ docker ps -l +``` + +--- + +class: extra-details + +## Confirming that our application works properly + +Finally, let's browse to our application and confirm it's working. + +```bash +http://: +``` + +--- + +## Links and environment variables + +In addition to the DNS information, Docker will automatically set environment variables in our container, giving extra details about the linked container. + +```bash +$ docker run --link datastore:redis alpine env +PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin +HOSTNAME=0738e57b771e +REDIS_PORT=tcp://172.17.0.120:6379 +REDIS_PORT_6379_TCP=tcp://172.17.0.120:6379 +REDIS_PORT_6379_TCP_ADDR=172.17.0.120 +REDIS_PORT_6379_TCP_PORT=6379 +REDIS_PORT_6379_TCP_PROTO=tcp +REDIS_NAME=/dreamy_wilson/redis +REDIS_ENV_REDIS_VERSION=2.8.13 +REDIS_ENV_REDIS_DOWNLOAD_URL=http://download.redis.io/releases/redis-2.8.13.tar.gz +REDIS_ENV_REDIS_DOWNLOAD_SHA1=a72925a35849eb2d38a1ea076a3db82072d4ee43 +HOME=/ +RUBY_MAJOR=2.1 +RUBY_VERSION=2.1.2 +``` + + +* Each variables is prefixed with the link alias: `redis`. +* Includes connection information PLUS any environment variables set in + the `datastore` container via `ENV` instructions. + +--- + +## Differences between network aliases and links + +* With network aliases, you can start containers in *any order.* +* With links, you have to start the server (in our example: Redis) first. +* With network aliases, you cannot change the name of the server once it is running. If you want to add a name, you have to create a new container. +* With links, you can give new names to an existing container. +* Network aliases require the use of a custom network. +* Links can be used on the default bridge network. +* Network aliases work across multi-host networking. +* Links (as of Engine 1.11) only work with local containers (but this might be changed in the future). +* Network aliases don't populate environment variables. +* Links give access to the environment of the target container. + +--- + +class: extra-details + +## Section summary + +We've learned how to: + +* Create links between containers. +* Use names and links to communicate across containers. + diff --git a/slides/intro/Container_Network_Model.md b/slides/intro/Container_Network_Model.md new file mode 100644 index 00000000..798b0d0b --- /dev/null +++ b/slides/intro/Container_Network_Model.md @@ -0,0 +1,520 @@ + +class: title + +# The Container Network Model + +![A denser graph network](Container_Network_Model/complex-network.jpg) + +--- + +## Objectives + +We will learn about the CNM (Container Network Model). + +At the end of this lesson, you will be able to: + +* Create a private network for a group of containers. + +* Use container naming to connect services together. + +* Dynamically connect and disconnect containers to networks. + +* Set the IP address of a container. + +We will also explain the principle of overlay networks and network plugins. + +--- + +## The Container Network Model + +The CNM was introduced in Engine 1.9.0 (November 2015). + +The CNM adds the notion of a *network*, and a new top-level command to manipulate and see those networks: `docker network`. + +```bash +$ docker network ls +NETWORK ID NAME DRIVER +6bde79dfcf70 bridge bridge +8d9c78725538 none null +eb0eeab782f4 host host +4c1ff84d6d3f blog-dev overlay +228a4355d548 blog-prod overlay +``` + +--- + +## What's in a network? + +* Conceptually, a network is a virtual switch. + +* It can be local (to a single Engine) or global (spanning multiple hosts). + +* A network has an IP subnet associated to it. + +* Docker will allocate IP addresses to the containers connected to a network. + +* Containers can be connected to multiple networks. + +* Containers can be given per-network names and aliases. + +* The names and aliases can be resolved via an embedded DNS server. + +--- + +## Network implementation details + +* A network is managed by a *driver*. + +* All the drivers that we have seen before are available. + +* A new multi-host driver, *overlay*, is available out of the box. + +* More drivers can be provided by plugins (OVS, VLAN...) + +* A network can have a custom IPAM (IP allocator). + +--- + +## Differences with the CNI + +* CNI = Container Network Interface + +* CNI is used notably by Kubernetes + +* With CNI, all the nodes and containers are on a single IP network + +* Both CNI and CNM offer the same functionality, but with very different methods + +--- + +## Creating a network + +Let's create a network called `dev`. + +```bash +$ docker network create dev +4c1ff84d6d3f1733d3e233ee039cac276f425a9d5228a4355d54878293a889ba +``` + +The network is now visible with the `network ls` command: + +```bash +$ docker network ls +NETWORK ID NAME DRIVER +6bde79dfcf70 bridge bridge +8d9c78725538 none null +eb0eeab782f4 host host +4c1ff84d6d3f dev bridge +``` + +--- + +## Placing containers on a network + +We will create a *named* container on this network. + +It will be reachable with its name, `es`. + +```bash +$ docker run -d --name es --net dev elasticsearch:2 +8abb80e229ce8926c7223beb69699f5f34d6f1d438bfc5682db893e798046863 +``` + +--- + +## Communication between containers + +Now, create another container on this network. + +```bash +$ docker run -ti --net dev alpine sh +root@0ecccdfa45ef:/# +``` + +From this new container, we can resolve and ping the other one, using its assigned name: + +```bash +/ # ping es +PING es (172.18.0.2) 56(84) bytes of data. +64 bytes from es.dev (172.18.0.2): icmp_seq=1 ttl=64 time=0.221 ms +64 bytes from es.dev (172.18.0.2): icmp_seq=2 ttl=64 time=0.114 ms +64 bytes from es.dev (172.18.0.2): icmp_seq=3 ttl=64 time=0.114 ms +^C +--- es ping statistics --- +3 packets transmitted, 3 received, 0% packet loss, time 2000ms +rtt min/avg/max/mdev = 0.114/0.149/0.221/0.052 ms +root@0ecccdfa45ef:/# +``` + +--- + +class: extra-details + +## Resolving container addresses + +In Docker Engine 1.9, name resolution is implemented with `/etc/hosts`, and +updating it each time containers are added/removed. + +```bash +[root@0ecccdfa45ef /]# cat /etc/hosts +172.18.0.3 0ecccdfa45ef +127.0.0.1 localhost +::1 localhost ip6-localhost ip6-loopback +fe00::0 ip6-localnet +ff00::0 ip6-mcastprefix +ff02::1 ip6-allnodes +ff02::2 ip6-allrouters +172.18.0.2 es +172.18.0.2 es.dev +``` + +In Docker Engine 1.10, this has been replaced by a dynamic resolver. + +(This avoids race conditions when updating `/etc/hosts`.) + +--- + +## Connecting multiple containers together + +* Let's try to run an application that requires two containers. + +* The first container is a web server. + +* The other one is a redis data store. + +* We will place them both on the `dev` network created before. + +--- + +## Running the web server + +* The application is provided by the container image `jpetazzo/trainingwheels`. + +* We don't know much about it so we will try to run it and see what happens! + +Start the container, exposing all its ports: + +```bash +$ docker run --net dev -d -P jpetazzo/trainingwheels +``` + +Check the port that has been allocated to it: + +```bash +$ docker ps -l +``` + +--- + +## Test the web server + +* If we connect to the application now, we will see an error page: + +.small[ +![Trainingwheels error](Container_Network_Model/trainingwheels-error.png) +] + +* This is because the Redis service is not running. +* This container tries to resolve the name `redis`. + +Note: we're not using a FQDN or an IP address here; just `redis`. + +--- + +## Start the data store + +* We need to start a Redis container. + +* That container must be on the same network as the web server. + +* It must have the right name (`redis`) so the application can find it. + +Start the container: + +```bash +$ docker run --net dev --name redis -d redis +``` + +--- + +## Test the web server again + +* If we connect to the application now, we should see that the app is working correctly: + +.small[ +![Trainingwheels OK](Container_Network_Model/trainingwheels-ok.png) +] + +* When the app tries to resolve `redis`, instead of getting a DNS error, it gets the IP address of our Redis container. + +--- + +## A few words on *scope* + +* What if we want to run multiple copies of our application? + +* Since names are unique, there can be only one container named `redis` at a time. + +* However, we can specify the network name of our container with `--net-alias`. + +* `--net-alias` is scoped per network, and independent from the container name. + +--- + +class: extra-details + +## Using a network alias instead of a name + +Let's remove the `redis` container: + +```bash +$ docker rm -f redis +``` + +And create one that doesn't block the `redis` name: + +```bash +$ docker run --net dev --net-alias redis -d redis +``` + +Check that the app still works (but the counter is back to 1, +since we wiped out the old Redis container). + +--- + +class: x-extra-details + +## Names are *local* to each network + +Let's try to ping our `es` container from another container, when that other container is *not* on the `dev` network. + +```bash +$ docker run --rm alpine ping es +ping: bad address 'es' +``` + +Names can be resolved only when containers are on the same network. + +Containers can contact each other only when they are on the same network (you can try to ping using the IP address to verify). + +--- + +class: extra-details + +## Network aliases + +We would like to have another network, `prod`, with its own `es` container. But there can be only one container named `es`! + +We will use *network aliases*. + +A container can have multiple network aliases. + +Network aliases are *local* to a given network (only exist in this network). + +Multiple containers can have the same network alias (even on the same network). In Docker Engine 1.11, resolving a network alias yields the IP addresses of all containers holding this alias. + +--- + +class: extra-details + +## Creating containers on another network + +Create the `prod` network. + +```bash +$ docker create network prod +5a41562fecf2d8f115bedc16865f7336232a04268bdf2bd816aecca01b68d50c +``` + +We can now create multiple containers with the `es` alias on the new `prod` network. + +```bash +$ docker run -d --name prod-es-1 --net-alias es --net prod elasticsearch:2 +38079d21caf0c5533a391700d9e9e920724e89200083df73211081c8a356d771 +$ docker run -d --name prod-es-2 --net-alias es --net prod elasticsearch:2 +1820087a9c600f43159688050dcc164c298183e1d2e62d5694fd46b10ac3bc3d +``` + +--- + +class: extra-details + +## Resolving network aliases + +Let's try DNS resolution first, using the `nslookup` tool that ships with the `alpine` image. + +```bash +$ docker run --net prod --rm alpine nslookup es +Name: es +Address 1: 172.23.0.3 prod-es-2.prod +Address 2: 172.23.0.2 prod-es-1.prod +``` + +(You can ignore the `can't resolve '(null)'` errors.) + +--- + +class: extra-details + +## Connecting to aliased containers + +Each ElasticSearch instance has a name (generated when it is started). This name can be seen when we issue a simple HTTP request on the ElasticSearch API endpoint. + +Try the following command a few times: + +```bash +$ docker run --rm --net dev centos curl -s es:9200 +{ + "name" : "Tarot", +... +} +``` + +Then try it a few times by replacing `--net dev` with `--net prod`: + +```bash +$ docker run --rm --net prod centos curl -s es:9200 +{ + "name" : "The Symbiote", +... +} +``` + +--- + +## Good to know ... + +* Docker will not create network names and aliases on the default `bridge` network. + +* Therefore, if you want to use those features, you have to create a custom network first. + +* Network aliases are *not* unique on a given network. + +* i.e., multiple containers can have the same alias on the same network. + +* In that scenario, the Docker DNS server will return multiple records. +
+ (i.e. you will get DNS round robin out of the box.) + +* Enabling *Swarm Mode* gives access to clustering and load balancing with IPVS. + +* Creation of networks and network aliases is generally automated with tools like Compose. + +--- + +class: extra-details + +## A few words about round robin DNS + +Don't rely exclusively on round robin DNS to achieve load balancing. + +Many factors can affect DNS resolution, and you might see: + +- all traffic going to a single instance; +- traffic being split (unevenly) between some instances; +- different behavior depending on your application language; +- different behavior depending on your base distro; +- different behavior depending on other factors (sic). + +It's OK to use DNS to discover available endpoints, but remember that you have to re-resolve every now and then to discover new endpoints. + +--- + +class: extra-details + +## Custom networks + +When creating a network, extra options can be provided. + +* `--internal` disables outbound traffic (the network won't have a default gateway). + +* `--gateway` indicates which address to use for the gateway (when utbound traffic is allowed). + +* `--subnet` (in CIDR notation) indicates the subnet to use. + +* `--ip-range` (in CIDR notation) indicates the subnet to allocate from. + +* `--aux-address` allows to specify a list of reserved addresses (which won't be allocated to containers). + +--- + +class: extra-details + +## Setting containers' IP address + +* It is possible to set a container's address with `--ip`. +* The IP address has to be within the subnet used for the container. + +A full example would look like this. + +```bash +$ docker network create --subnet 10.66.0.0/16 pubnet +42fb16ec412383db6289a3e39c3c0224f395d7f85bcb1859b279e7a564d4e135 +$ docker run --net pubnet --ip 10.66.66.66 -d nginx +b2887adeb5578a01fd9c55c435cad56bbbe802350711d2743691f95743680b09 +``` + +*Note: don't hard code container IP addresses in your code!* + +*I repeat: don't hard code container IP addresses in your code!* + +--- + +## Overlay networks + +* The features we've seen so far only work when all containers are on a single host. + +* If containers span multiple hosts, we need an *overlay* network to connect them together. + +* Docker ships with a default network plugin, `overlay`, implementing an overlay network leveraging VXLAN. + +* Other plugins (Weave, Calico...) can provide overlay networks as well. + +* Once you have an overlay network, *all the features that we've used in this chapter work identically.* + +--- + +class: extra-details + +## Multi-host networking (overlay) + +Out of the scope for this intro-level workshop! + +Very short instructions: + +- enable Swarm Mode (`docker swarm init` then `docker swarm join` on other nodes) +- `docker network create mynet --driver overlay` +- `docker service create --network mynet myimage` + +See http://jpetazzo.github.io/orchestration-workshop for all the deets about clustering! + +--- + +class: extra-details + +## Multi-host networking (plugins) + +Out of the scope for this intro-level workshop! + +General idea: + +- install the plugin (they often ship within containers) + +- run the plugin (if it's in a container, it will often require extra parameters; don't just `docker run` it blindly!) + +- some plugins require configuration or activation (creating a special file that tells Docker "use the plugin whose control socket is at the following location") + +- you can then `docker network create --driver pluginname` + +--- + +## Section summary + +We've learned how to: + +* Create private networks for groups of containers. + +* Assign IP addresses to containers. + +* Use container naming to implement service discovery. + diff --git a/slides/intro/Container_Networking_Basics.md b/slides/intro/Container_Networking_Basics.md new file mode 100644 index 00000000..b9cfadc7 --- /dev/null +++ b/slides/intro/Container_Networking_Basics.md @@ -0,0 +1,284 @@ + +class: title + +# Container Networking Basics + +![A dense graph network](Container_Networking_Basics/network.jpg) + +--- + +## Objectives + +We will now run network services (accepting requests) in containers. + +At the end of this section, you will be able to: + +* Run a network service in a container. + +* Manipulate container networking basics. + +* Find a container's IP address. + +We will also explain the different network models used by Docker. + +--- + +## A simple, static web server + +Run the Docker Hub image `nginx`, which contains a basic web server: + +```bash +$ docker run -d -P nginx +66b1ce719198711292c8f34f84a7b68c3876cf9f67015e752b94e189d35a204e +``` + +* Docker will download the image from the Docker Hub. + +* `-d` tells Docker to run the image in the background. + +* `-P` tells Docker to make this service reachable from other computers. +
(`-P` is the short version of `--publish-all`.) + +But, how do we connect to our web server now? + +--- + +## Finding our web server port + +We will use `docker ps`: + +```bash +$ docker ps +CONTAINER ID IMAGE ... PORTS ... +e40ffb406c9e nginx ... 0.0.0.0:32769->80/tcp, 0.0.0.0:32768->443/tcp ... +``` + + +* The web server is running on ports 80 and 443 inside the container. + +* Those ports are mapped to ports 32769 and 32768 on our Docker host. + +We will explain the whys and hows of this port mapping. + +But first, let's make sure that everything works properly. + +--- + +## Connecting to our web server (GUI) + +Point your browser to the IP address of your Docker host, on the port +shown by `docker ps` for container port 80. + +![Screenshot](Container_Networking_Basics/web.png) + +--- + +## Connecting to our web server (CLI) + +You can also use `curl` directly from the Docker host. + +Make sure to use the right port number if it is different +from the example below: + +```bash +$ curl localhost:32769 + + + +Welcome to nginx! +... +``` + +--- + +## Why are we mapping ports? + +* We are out of IPv4 addresses. + +* Containers cannot have public IPv4 addresses. + +* They have private addresses. + +* Services have to be exposed port by port. + +* Ports have to be mapped to avoid conflicts. + +--- + +## Finding the web server port in a script + +Parsing the output of `docker ps` would be painful. + +There is a command to help us: + +```bash +$ docker port 80 +32769 +``` + +--- + +## Manual allocation of port numbers + +If you want to set port numbers yourself, no problem: + +```bash +$ docker run -d -p 80:80 nginx +$ docker run -d -p 8000:80 nginx +$ docker run -d -p 8080:80 -p 8888:80 nginx +``` + +* We are running two NGINX web servers. +* The first one is exposed on port 80. +* The second one is exposed on port 8000. +* The third one is exposed on ports 8080 and 8888. + +Note: the convention is `port-on-host:port-on-container`. + +--- + +## Plumbing containers into your infrastructure + +There are many ways to integrate containers in your network. + +* Start the container, letting Docker allocate a public port for it. +
Then retrieve that port number and feed it to your configuration. + +* Pick a fixed port number in advance, when you generate your configuration. +
Then start your container by setting the port numbers manually. + +* Use a network plugin, connecting your containers with e.g. VLANs, tunnels... + +* Enable *Swarm Mode* to deploy across a cluster. +
The container will then be reachable through any node of the cluster. + +When using Docker through an extra management layer like Mesos or Kubernetes, +these will usually provide their own mechanism to expose containers. + +--- + +## Finding the container's IP address + +We can use the `docker inspect` command to find the IP address of the +container. + +```bash +$ docker inspect --format '{{ .NetworkSettings.IPAddress }}' +172.17.0.3 +``` + +* `docker inspect` is an advanced command, that can retrieve a ton + of information about our containers. + +* Here, we provide it with a format string to extract exactly the + private IP address of the container. + +--- + +## Pinging our container + +We can test connectivity to the container using the IP address we've +just discovered. Let's see this now by using the `ping` tool. + +```bash +$ ping +64 bytes from : icmp_req=1 ttl=64 time=0.085 ms +64 bytes from : icmp_req=2 ttl=64 time=0.085 ms +64 bytes from : icmp_req=3 ttl=64 time=0.085 ms +``` + +--- + +## The different network drivers + +A container can use one of the following drivers: + +* `bridge` (default) +* `none` +* `host` +* `container` + +The driver is selected with `docker run --net ...`. + +The different drivers are explained with more details on the following slides. + +--- + +## The default bridge + +* By default, the container gets a virtual `eth0` interface. +
(In addition to its own private `lo` loopback interface.) + +* That interface is provided by a `veth` pair. + +* It is connected to the Docker bridge. +
(Named `docker0` by default; configurable with `--bridge`.) + +* Addresses are allocated on a private, internal subnet. +
(Docker uses 172.17.0.0/16 by default; configurable with `--bip`.) + +* Outbound traffic goes through an iptables MASQUERADE rule. + +* Inbound traffic goes through an iptables DNAT rule. + +* The container can have its own routes, iptables rules, etc. + +--- + +## The null driver + +* Container is started with `docker run --net none ...` + +* It only gets the `lo` loopback interface. No `eth0`. + +* It can't send or receive network traffic. + +* Useful for isolated/untrusted workloads. + +--- + +## The host driver + +* Container is started with `docker run --net host ...` + +* It sees (and can access) the network interfaces of the host. + +* It can bind any address, any port (for ill and for good). + +* Network traffic doesn't have to go through NAT, bridge, or veth. + +* Performance = native! + +Use cases: + +* Performance sensitive applications (VOIP, gaming, streaming...) + +* Peer discovery (e.g. Erlang port mapper, Raft, Serf...) + +--- + +## The container driver + +* Container is started with `docker run --net container:id ...` + +* It re-uses the network stack of another container. + +* It shares with this other container the same interfaces, IP address(es), routes, iptables rules, etc. + +* Those containers can communicate over their `lo` interface. +
(i.e. one can bind to 127.0.0.1 and the others can connect to it.) + +--- + +## Section summary + +We've learned how to: + +* Expose a network port. + +* Manipulate container networking basics. + +* Find a container's IP address. + +In the next chapter, we will see how to connect +containers together without exposing their ports. diff --git a/slides/intro/Copying_Files_During_Build.md b/slides/intro/Copying_Files_During_Build.md new file mode 100644 index 00000000..7df1b42f --- /dev/null +++ b/slides/intro/Copying_Files_During_Build.md @@ -0,0 +1,100 @@ + +class: title + +# Copying files during the build + +![Monks copying books](Copying_Files_During_Build/copy.jpg) + +--- + +## Objectives + +So far, we have installed things in our container images +by downloading packages. + +We can also copy files from the *build context* to the +container that we are building. + +Remember: the *build context* is the directory containing +the Dockerfile. + +In this chapter, we will learn a new Dockerfile keyword: `COPY`. + +--- + +## Build some C code + +We want to build a container that compiles a basic "Hello world" program in C. + +Here is the program, `hello.c`: + +```bash +int main () { + puts("Hello, world!"); + return 0; +} +``` + +Let's create a new directory, and put this file in there. + +Then we will write the Dockerfile. + +--- + +## The Dockerfile + +On Debian and Ubuntu, the package `build-essential` will get us a compiler. + +When installing it, don't forget to specify the `-y` flag, otherwise the build will fail (since the build cannot be interactive). + +Then we will use `COPY` to place the source file into the container. + +```bash +FROM ubuntu +RUN apt-get update +RUN apt-get install -y build-essential +COPY hello.c / +RUN make hello +CMD /hello +``` + +Create this Dockerfile. + +--- + +## Testing our C program + +* Create `hello.c` and `Dockerfile` in the same direcotry. + +* Run `docker build -t hello .` in this directory. + +* Run `docker run hello`, you should see `Hello, world!`. + +Success! + +--- + +## `COPY` and the build cache + +* Run the build again. + +* Now, modify `hello.c` and run the build again. + +* Docker can cache steps involving `COPY`. + +* Those steps will not be executed again if the files haven't been changed. + +--- + +## Details + +* You can `COPY` whole directories recursively. + +* Older Dockerfiles also have the `ADD` instruction. +
It is similar but can automatically extract archives. + +* If we really wanted to compile C code in a compiler, we would: + + * Place it in a different directory, with the `WORKDIR` instruction. + + * Even better, use the `gcc` official image. diff --git a/slides/intro/Course_Conclusion.md b/slides/intro/Course_Conclusion.md new file mode 100644 index 00000000..493bcacb --- /dev/null +++ b/slides/intro/Course_Conclusion.md @@ -0,0 +1,32 @@ +class: title + +# Course Conclusion + +![end](Course_Conclusion/end.jpg) + +--- + +## Questions & Next Steps + +A bunch of useful links: + +* Docker homepage - http://www.docker.com/ +* Docker Hub - https://hub.docker.com +* Docker blog - http://blog.docker.com/ +* Docker documentation - http://docs.docker.com/ +* Docker code on GitHub - https://github.com/docker/docker +* Docker mailing list - [https://groups.google.com/forum/#!forum/docker-user +* Docker on IRC: irc.freenode.net and channels `#docker` and `#docker-dev` +* Docker on Twitter - http://twitter.com/docker +* Get Docker help on Stack Overflow - http://stackoverflow.com/search?q=docker +* Play With Docker Hands-On Labs - http://training.play-with-docker.com/ + +These slides are at: http://container.training/ + +--- + +class: title + +Thank You! + +.small[http://container.training/] \ No newline at end of file diff --git a/slides/intro/Docker_History.md b/slides/intro/Docker_History.md new file mode 100644 index 00000000..bb26fdee --- /dev/null +++ b/slides/intro/Docker_History.md @@ -0,0 +1,141 @@ +# History of containers ... and Docker + +--- + +## First experimentations + +* [IBM VM/370 (1972)](https://en.wikipedia.org/wiki/VM_%28operating_system%29) + +* [Linux VServers (2001)](http://www.solucorp.qc.ca/changes.hc?projet=vserver) + +* [Solaris Containers (2004)](https://en.wikipedia.org/wiki/Solaris_Containers) + +* [FreeBSD jails (1999)](https://www.freebsd.org/cgi/man.cgi?query=jail&sektion=8&manpath=FreeBSD+4.0-RELEASE) + +Containers have been around for a *very long time* indeed. + +--- + +class: pic + +## The VPS age (until 2007-2008) + +![lightcont](About_Docker/lightcont.png) + +--- + +## Containers = cheaper than VMs + +* Users: hosting providers. + +* Highly specialized audience with strong ops culture. + +--- + +class: pic + +## The PAAS period (2008-2013) + +![heroku 2007](About_Docker/heroku-first-homepage.png) + +--- + +## Containers = easier than VMs + +* I can't speak for Heroku, but containers were (one of) dotCloud's secret weapon + +* dotCloud was operating a PaaS, using a custom container engine. + +* This engine was based on OpenVZ (and later, LXC) and AUFS. + +* It started (circa 2008) as a single Python script. + +* By 2012, the engine had multiple (~10) Python components. +
(and ~100 other micro-services!) + +* End of 2012, dotCloud refactors this container engine. + +* The codename for this project is "Docker." + +--- + +## First public release of Docker + +* March 2013, PyCon, Santa Clara: +
"Docker" is shown to a public audience for the first time. + +* It is released with an open source license. + +* Very positive reactions and feedback! + +* The dotCloud team progressively shifts to Docker development. + +* The same year, dotCloud changes name to Docker. + +* In 2014, the PaaS activity is sold. + +--- + +## Docker early days (2013-2014) + +--- + +## First users of Docker + +* PAAS builders (Flynn, Dokku, Tsuru, Deis...) + +* PAAS users (those big enough to justify building their own) + +* CI platforms + +* developers, developers, developers, developers + +--- + +## Positive feedback loop + +* In 2013, the technology under containers (cgroups, namespaces, copy-on-write storage...) + had many blind spots. + +* The growing popularity of Docker and containers exposed many bugs. + +* As a result, those bugs were fixed, resulting in better stability for containers. + +* Any decent hosting/cloud provider can run containers today. + +* Containers become a great tool to deploy/move workloads to/from on-prem/cloud. + +--- + +## Maturity (2015-2016) + +--- + +## Docker becomes an industry standard + +* Docker reaches the symbolic 1.0 milestone. + +* Existing systems like Mesos and Cloud Foundry add Docker support. + +* Standardization around the OCI (Open Containers Initiative). + +* Other container engines are developed. + +* Creation of the CNCF (Cloud Native Computing Foundation). + +--- + +## Docker becomes a platform + +* The initial container engine is now known as "Docker Engine." + +* Other tools are added: + * Docker Compose (formerly "Fig") + * Docker Machine + * Docker Swarm + * Kitematic + * Docker Cloud (formerly "Tutum") + * Docker Datacenter + * etc. + +* Docker Inc. launches commercial offers. diff --git a/slides/intro/Docker_Hub_Tease.md b/slides/intro/Docker_Hub_Tease.md new file mode 100644 index 00000000..1414cc59 --- /dev/null +++ b/slides/intro/Docker_Hub_Tease.md @@ -0,0 +1,35 @@ +# Publishing images to the Docker Hub + +We have built our first images. + +If we were so inclined, we could share those images through the Docker Hub. + +We won't do it since we don't want to force everyone to create a Docker Hub account (although it's free, yay!) but the steps would be: + +* have an account on the Docker Hub + +* tag our image accordingly (i.e. `username/imagename`) + +* `docker push username/imagename` + +Anybody can now `docker run username/imagename` from any Docker host. + +Images can be set to be private as well. + +--- + +## The goodness of automated builds + +* You can link a Docker Hub repository with a GitHub or BitBucket repository + +* Each push to GitHub or BitBucket will trigger a build on Docker Hub + +* If the build succeeds, the new image is available on Docker Hub + +* You can map tags and branches between source and container images + +* If you work with public repositories, this is free + +* Corollary: this gives you a very simple way to get free, basic CI + + (With the technique presented earlier) diff --git a/slides/intro/Docker_Overview.md b/slides/intro/Docker_Overview.md new file mode 100644 index 00000000..a4904e97 --- /dev/null +++ b/slides/intro/Docker_Overview.md @@ -0,0 +1,344 @@ +# Docker 30,000ft overview + +In this lesson, we will learn about: + +* Why containers (non-technical elevator pitch) + +* Why containers (technical elevator pitch) + +* How Docker helps us to build, ship, and run + +* The history of containers + +We won't actually run Docker or containers in this chapter (yet!). + +Don't worry, we will get to that fast enough! + +--- + +## Elevator pitch + +### (for your manager, your boss...) + +--- + +## OK... Why the buzz around containers? + +* The software industry has changed + +* Before: + * monolithic applications + * long development cycles + * single environment + * slowly scaling up + +* Now: + * decoupled services + * fast, iterative improvements + * multiple environments + * quickly scaling out + +--- + +## Deployment becomes very complex + +* Many different stacks: + * languages + * frameworks + * databases + +* Many different targets: + * individual development environments + * pre-production, QA, staging... + * production: on prem, cloud, hybrid + +--- + +class: pic + +## The deployment problem + +![problem](About_Docker/problem.png) + +--- + +class: pic + +## The matrix from hell + +![matrix](About_Docker/matrix.png) + +--- + +class: pic + +## The parallel with the shipping indsutry + +![history](About_Docker/shippingindustry.png) + +--- + +class: pic + +## Intermodal shipping containers + +![shipping](About_Docker/shipping.png) + +--- + +class: pic + +## A new shipping ecosystem + +![shipeco](About_Docker/shipeco.png) + +--- + +class: pic + +## A shipping container system for applications + +![shipapp](About_Docker/appcont.png) + +--- + +class: pic + +## Eliminate the matrix from hell + +![elimatrix](About_Docker/elimatrix.png) + +--- + +## Results + +* Dev-to-prod reduced from 9 months to 15 minutes (ING) + +* Continuous integration job time reduced by more than 60% (BBC) + +* Dev-to-prod reduced from weeks to minutes (GILT) + +* etc. + +--- + +## Elevator pitch + +### (for your fellow devs and ops) + +--- + +## Escape dependency hell + +1. Write installation instructions into an `INSTALL.txt` file + +2. Using this file, write an `install.sh` script that works *for you* + +3. Turn this file into a `Dockerfile`, test it on your machine + +4. If the Dockerfile builds on your machine, it will build *anywhere* + +5. Rejoice as you escape dependency hell and "works on my machine" + +Never again "worked in dev - ops problem now!" + +--- + +## On-board developers and contributors rapidly + +1. Write Dockerfiles for your application components + +2. Use pre-made images from the Docker Hub (mysql, redis...) + +3. Describe your stack with a Compose file + +4. On-board somebody with two commands: + +```bash +git clone ... +docker-compose up +``` + +With this, you can create development, integration, QA environments in minutes! + +--- + +class: extra-details + +## Implement reliable CI easily + +1. Build test environment with a Dockerfile or Compose file + +2. For each test run, stage up a new container or stack + +3. Each run is now in a clean environment + +4. No pollution from previous tests + +Way faster and cheaper than creating VMs each time! + +--- + +class: extra-details + +## Use container images as build artefacts + +1. Build your app from Dockerfiles + +2. Store the resulting images in a registry + +3. Keep them forever (or as long as necessary) + +4. Test those images in QA, CI, integration... + +5. Run the same images in production + +6. Something goes wrong? Rollback to previous image + +7. Investigating old regression? Old image has your back! + +Images contain all the libraries, dependencies, etc. needed to run the app. + +--- + +class: extra-details + +## Decouple "plumbing" from application logic + +1. Write your code to connect to named services ("db", "api"...) + +2. Use Compose to start your stack + +3. Docker will setup per-container DNS resolver for those names + +4. You can now scale, add load balancers, replication ... without changing your code + +Note: this is not covered in this intro level workshop! + +--- + +class: extra-details + +## What did Docker bring to the table? + +### Docker before/after + +--- + +class: extra-details + +## Formats and APIs, before Docker + +* No standardized exchange format. +
(No, a rootfs tarball is *not* a format!) + +* Containers are hard to use for developers. +
(Where's the equivalent of `docker run debian`?) + +* As a result, they are *hidden* from the end users. + +* No re-usable components, APIs, tools. +
(At best: VM abstractions, e.g. libvirt.) + +Analogy: + +* Shipping containers are not just steel boxes. +* They are steel boxes that are a standard size, with the same hooks and holes. + +--- + +class: extra-details + +## Formats and APIs, after Docker + +* Standardize the container format, because containers were not portable. + +* Make containers easy to use for developers. + +* Emphasis on re-usable components, APIs, ecosystem of standard tools. + +* Improvement over ad-hoc, in-house, specific tools. + +--- + +class: extra-details + +## Shipping, before Docker + +* Ship packages: deb, rpm, gem, jar, homebrew... + +* Dependency hell. + +* "Works on my machine." + +* Base deployment often done from scratch (debootstrap...) and unreliable. + +--- + +class: extra-details + +## Shipping, after Docker + +* Ship container images with all their dependencies. + +* Images are bigger, but they are broken down into layers. + +* Only ship layers that have changed. + +* Save disk, network, memory usage. + +--- + +class: extra-details + +## Example + +Layers: + +* CentOS +* JRE +* Tomcat +* Dependencies +* Application JAR +* Configuration + +--- + +class: extra-details + +## Devs vs Ops, before Docker + +* Drop a tarball (or a commit hash) with instructions. + +* Dev environment very different from production. + +* Ops don't always have a dev environment themselves ... + +* ... and when they do, it can differ from the devs'. + +* Ops have to sort out differences and make it work ... + +* ... or bounce it back to devs. + +* Shipping code causes frictions and delays. + +--- + +class: extra-details + +## Devs vs Ops, after Docker + +* Drop a container image or a Compose file. + +* Ops can always run that container image. + +* Ops can always run that Compose file. + +* Ops still have to adapt to prod environment, + but at least they have a reference point. + +* Ops have tools allowing to use the same image + in dev and prod. + +* Devs can be empowered to make releases themselves + more easily. diff --git a/slides/intro/Dockerfile_Tips.md b/slides/intro/Dockerfile_Tips.md new file mode 100644 index 00000000..556108ff --- /dev/null +++ b/slides/intro/Dockerfile_Tips.md @@ -0,0 +1,100 @@ +# Tips for efficient Dockerfiles + +We will see how to: + +* Reduce the number of layers. + +* Leverage the build cache so that builds can be faster. + +* Embed unit testing in the build process. + +--- + +## Reducing the number of layers + +* Each line in a `Dockerfile` creates a new layer. + +* Build your `Dockerfile` to take advantage of Docker's caching system. + +* Combine commands by using `&&` to continue commands and `\` to wrap lines. + +Note: it is frequent to build a Dockerfile line by line: + +```dockerfile +RUN apt-get install thisthing +RUN apt-get install andthatthing andthatotherone +RUN apt-get install somemorestuff +``` + +And then refactor it trivially before shipping: + +```dockerfile +RUN apt-get install thisthing andthatthing andthatotherone somemorestuff +``` + +--- + +## Avoid re-installing dependencies at each build + +* Classic Dockerfile problem: + + "each time I change a line of code, all my dependencies are re-installed!" + +* Solution: `COPY` dependency lists (`package.json`, `requirements.txt`, etc.) + by themselves to avoid reinstalling unchanged dependencies every time. + +--- + +## Example "bad" `Dockerfile` + +The dependencies are reinstalled every time, because the build system does not know if `requirements.txt` has been updated. + +```bash +FROM python +MAINTAINER Docker Education Team +COPY . /src/ +WORKDIR /src +RUN pip install -qr requirements.txt +EXPOSE 5000 +CMD ["python", "app.py"] +``` + +--- + +## Fixed `Dockerfile` + +Adding the dependencies as a separate step means that Docker can cache more efficiently and only install them when `requirements.txt` changes. + +```bash +FROM python +MAINTAINER Docker Education Team +COPY ./requirements.txt /tmp/requirements.txt +RUN pip install -qr /tmp/requirements.txt +COPY . /src/ +WORKDIR /src +EXPOSE 5000 +CMD ["python", "app.py"] +``` + +--- + +## Embedding unit tests in the build process + +```dockerfile +FROM +RUN +COPY +RUN +RUN +COPY +RUN +FROM +RUN +COPY +RUN +CMD, EXPOSE ... +``` + +* The build fails as soon as an instructions fails +* If `RUN ` fails, the build doesn't produce an image +* If it succeeds, it produces a clean image (without test libraries and data) diff --git a/slides/intro/First_Containers.md b/slides/intro/First_Containers.md new file mode 100644 index 00000000..e1285cde --- /dev/null +++ b/slides/intro/First_Containers.md @@ -0,0 +1,148 @@ + +class: title + +# Our First Containers + +![Plastic Containers](First_Containers/firstcontainers.jpg) + +--- + +## Objectives + +At the end of this lesson, you will have: + +* Seen Docker in action. + +* Started your first containers. + +--- + +## Hello World + +In your Docker environment, just run the following command: + +```bash +$ docker run busybox echo hello world +hello world +``` + +(If your Docker install is brand new, you will also see a few extra lines, +corresponding to the download of the `busybox` image.) + +--- + +## That was our first container! + +* We used one of the smallest, simplest images available: `busybox`. + +* `busybox` is typically used in embedded systems (phones, routers...) + +* We ran a single process and echo'ed `hello world`. + +--- + +## A more useful container + +Let's run a more exciting container: + +```bash +$ docker run -it ubuntu +root@04c0bb0a6c07:/# +``` + +* This is a brand new container. +* It runs a bare-bones, no-frills `ubuntu` system. +* `-it` is shorthand for `-i -t`. + + * `-i` tells Docker to connect us to the container's stdin. + * `-t` tells Docker that we want a pseudo-terminal. + +--- + +## Do something in our container + +Try to run `figlet` in our container. + +```bash +root@04c0bb0a6c07:/# figlet hello +bash: figlet: command not found +``` + +Alright, we need to install it. + +--- + +## An observation + +Let's check how many packages are installed here. + +```bash +root@04c0bb0a6c07:/# dpkg -l | wc -l +189 +``` + +* `dpkg -l` lists the packages installed in our container +* `wc -l` counts them +* If you have a Debian or Ubuntu machine, you can run the same command + and compare the results. + +--- + +## Install a package in our container + +We want `figlet`, so let's install it: + +```bash +root@04c0bb0a6c07:/# apt-get update +... +Fetched 1514 kB in 14s (103 kB/s) +Reading package lists... Done +root@04c0bb0a6c07:/# apt-get install figlet +Reading package lists... Done +... +``` + +One minute later, `figlet` is installed! + +```bash +root@04c0bb0a6c07:/# figlet hello + _ _ _ +| |__ ___| | | ___ +| '_ \ / _ \ | |/ _ \ +| | | | __/ | | (_) | +|_| |_|\___|_|_|\___/ +``` + +--- + +## Exiting our container + +Just exit the shell, like you would usually do. + +(E.g. with `^D` or `exit`) + +```bash +root@04c0bb0a6c07:/# exit +``` + +* Our container is now in a *stopped* state. + +* It still exists on disk, but all compute resources have been freed up. + +--- + +## Starting another container + +What if we start a new container, and try to run `figlet` again? + +```bash +$ docker run -it ubuntu +root@b13c164401fb:/# figlet +bash: figlet: command not found +``` + +* We started a *brand new container*. + +* The basic Ubuntu image was used, and `figlet` is not here. + +* We will see in the next chapters how to bake a custom image with `figlet`. diff --git a/slides/intro/Initial_Images.md b/slides/intro/Initial_Images.md new file mode 100644 index 00000000..76508929 --- /dev/null +++ b/slides/intro/Initial_Images.md @@ -0,0 +1,354 @@ + +class: title + +# Understanding Docker Images + +![image](Initial_Images/image.png) + +--- + +## Objectives + +In this section, we will explain: + +* What is an image. + +* What is a layer. + +* The various image namespaces. + +* How to search and download images. + +* Image tags and when to use them. + +--- + +## What is an image? + +* Image = files + metadata + +* These files form the root filesystem of our container. + +* The metadata can indicate a number of things, e.g.: + + * the author of the image + * the command to execute in the container when starting it + * environment variables to be set + * etc. + +* Images are made of *layers*, conceptually stacked on top of each other. + +* Each layer can add, change, and remove files and/or metadata. + +* Images can share layers to optimize disk usage, transfer times, and memory use. + +--- + +## Example for a Java webapp + +* CentOS base layer +* Packages and configuration files added by our local IT +* JRE +* Tomcat +* Our application's dependencies +* Our application code and assets +* Our application configuration + +--- + +## Differences between containers and images + +* An image is a read-only filesystem. + +* A container is an encapsulated set of processes running in a + read-write copy of that filesystem. + +* To optimize container boot time, *copy-on-write* is used + instead of regular copy. + +* `docker run` starts a container from a given image. + +Let's give a couple of metaphors to illustrate those concepts. + +--- + +## Image as stencils + +Images are like templates or stencils that you can create containers from. + +![stencil](Initial_Images/stenciling-wall.jpg) + +--- + +## Object-oriented programming + +* Images are conceptually similar to *classes*. + +* Layers are conceptually similar to *inheritance*. + +* Containers are conceptually similar to *instances*. + +--- + +## Wait a minute... + +If an image is read-only, how do we change it? + +* We don't. + +* We create a new container from that image. + +* Then we make changes to that container. + +* When we are satisfied with those changes, we transform them into a new layer. + +* A new image is created by stacking the new layer on top of the old image. + +--- + +## A chicken-and-egg problem + +* The only way to create an image is by "freezing" a container. + +* The only way to create a container is by instanciating an image. + +* Help! + +--- + +## Creating the first images + +There is a special empty image called `scratch`. + +* It allows to *build from scratch*. + +The `docker import` command loads a tarball into Docker. + +* The imported tarball becomes a standalone image. +* That new image has a single layer. + +Note: you will probably never have to do this yourself. + +--- + +## Creating other images + +`docker commit` + +* Saves all the changes made to a container into a new layer. +* Creates a new image (effectively a copy of the container). + +`docker build` + +* Performs a repeatable build sequence. +* This is the preferred method! + +We will explain both methods in a moment. + +--- + +## Images namespaces + +There are three namespaces: + +* Official images + + e.g. `ubuntu`, `busybox` ... + +* User (and organizations) images + + e.g. `jpetazzo/clock` + +* Self-hosted images + + e.g. `registry.example.com:5000/my-private/image` + +Let's explain each of them. + +--- + +## Root namespace + +The root namespace is for official images. They are put there by Docker Inc., +but they are generally authored and maintained by third parties. + +Those images include: + +* Small, "swiss-army-knife" images like busybox. + +* Distro images to be used as bases for your builds, like ubuntu, fedora... + +* Ready-to-use components and services, like redis, postgresql... + +--- + +## User namespace + +The user namespace holds images for Docker Hub users and organizations. + +For example: + +```bash +jpetazzo/clock +``` + +The Docker Hub user is: + +```bash +jpetazzo +``` + +The image name is: + +```bash +clock +``` + +--- + +## Self-Hosted namespace + +This namespace holds images which are not hosted on Docker Hub, but on third +party registries. + +They contain the hostname (or IP address), and optionally the port, of the +registry server. + +For example: + +```bash +localhost:5000/wordpress +``` + +* `localhost:5000` is the host and port of the registry +* `wordpress` is the name of the image + +--- + +## How do you store and manage images? + +Images can be stored: + +* On your Docker host. +* In a Docker registry. + +You can use the Docker client to download (pull) or upload (push) images. + +To be more accurate: you can use the Docker client to tell a Docker Engine +to push and pull images to and from a registry. + +--- + +## Showing current images + +Let's look at what images are on our host now. + +```bash +$ docker images +REPOSITORY TAG IMAGE ID CREATED SIZE +fedora latest ddd5c9c1d0f2 3 days ago 204.7 MB +centos latest d0e7f81ca65c 3 days ago 196.6 MB +ubuntu latest 07c86167cdc4 4 days ago 188 MB +redis latest 4f5f397d4b7c 5 days ago 177.6 MB +postgres latest afe2b5e1859b 5 days ago 264.5 MB +alpine latest 70c557e50ed6 5 days ago 4.798 MB +debian latest f50f9524513f 6 days ago 125.1 MB +busybox latest 3240943c9ea3 2 weeks ago 1.114 MB +training/namer latest 902673acc741 9 months ago 289.3 MB +jpetazzo/clock latest 12068b93616f 12 months ago 2.433 MB +``` + +--- + +## Searching for images + +We cannot list *all* images on a remote registry, but +we can search for a specific keyword: + +```bash +$ docker search marathon +NAME DESCRIPTION STARS OFFICIAL AUTOMATED +mesosphere/marathon A cluster-wide init and co... 105 [OK] +mesoscloud/marathon Marathon 31 [OK] +mesosphere/marathon-lb Script to update haproxy b... 22 [OK] +tobilg/mongodb-marathon A Docker image to start a ... 4 [OK] +``` + + +* "Stars" indicate the popularity of the image. + +* "Official" images are those in the root namespace. + +* "Automated" images are built automatically by the Docker Hub. +
(This means that their build recipe is always available.) + +--- + +## Downloading images + +There are two ways to download images. + +* Explicitly, with `docker pull`. + +* Implicitly, when executing `docker run` and the image is not found locally. + +--- + +## Pulling an image + +```bash +$ docker pull debian:jessie +Pulling repository debian +b164861940b8: Download complete +b164861940b8: Pulling image (jessie) from debian +d1881793a057: Download complete +``` + +* As seen previously, images are made up of layers. + +* Docker has downloaded all the necessary layers. + +* In this example, `:jessie` indicates which exact version of Debian + we would like. + + It is a *version tag*. + +--- + +## Image and tags + +* Images can have tags. + +* Tags define image versions or variants. + +* `docker pull ubuntu` will refer to `ubuntu:latest`. + +* The `:latest` tag is generally updated often. + +--- + +## When to (not) use tags + +Don't specify tags: + +* When doing rapid testing and prototyping. +* When experimenting. +* When you want the latest version. + +Do specify tags: + +* When recording a procedure into a script. +* When going to production. +* To ensure that the same version will be used everywhere. +* To ensure repeatability later. + +--- + +## Section summary + +We've learned how to: + +* Understand images and layers. +* Understand Docker image namespacing. +* Search and download images. + diff --git a/slides/intro/Install_Docker.md b/slides/intro/Install_Docker.md new file mode 100644 index 00000000..b8cccb61 --- /dev/null +++ b/slides/intro/Install_Docker.md @@ -0,0 +1,130 @@ + +class: title + +# Install Docker + + +![install](Install_Docker/install.jpg) + +--- + +## Objectives + +At the end of this lesson, you will know: + +* How to install Docker. + +* When to use `sudo` when running Docker commands. + +*Note:* if you were provided with a training VM for a hands-on +tutorial, you can skip this chapter, since that VM already +has Docker installed, and Docker has already been setup to run +without `sudo`. + +--- + +## Installing Docker + +There are many ways to install Docker. + +We can arbitrarily distinguish: + +* Installing Docker on an existing Linux machine (physical or VM) + +* Installing Docker on MacOS or Windows + +* Installing Docker on a fleet of cloud VMs + +--- + +## Installing Docker on Linux + +* The recommended method is to install the packages supplied by Docker Inc. + +* The general method is: + + - add Docker Inc.'s package repositories to your system configuration + + - install the Docker Engine + +* Detailed installation instructions (distro by distro) are available on: + + https://docs.docker.com/engine/installation/ + +* You can also install from binaries (if your distro is not supported): + + https://docs.docker.com/engine/installation/linux/docker-ce/binaries/ + +--- + +## Installing Docker on MacOS and Windows + +* On MacOS, the recommended method is to use Docker4Mac: + + https://docs.docker.com/docker-for-mac/install/ + +* On Windows 10 Pro, Enterprise, and Eduction, you can use Docker4Windows: + + https://docs.docker.com/docker-for-windows/install/ + +* On older versions of Windows, you can use the Docker Toolbox: + + https://docs.docker.com/toolbox/toolbox_install_windows/ + +--- + +## Running Docker on MacOS and Windows + +When you execute `docker version` from the terminal: + +* the CLI connects to the Docker Engine over a standard socket, +* the Docker Engine is, in fact, running in a VM, +* ... but the CLI doesn't know or care about that, +* the CLI sends a request using the REST API, +* the Docker Engine in the VM processes the request, +* the CLI gets the response and displays it to you. + +All communication with the Docker Engine happens over the API. + +This will also allow to use remote Engines exactly as if they were local. + +--- + +## Docker4Mac and Docker4Windows + +* They let you run Docker without VirtualBox + +* They are installed like normal applications (think QEMU, but faster) + +* They access network resources like normal applications +
(and therefore, play well with enterprise VPNs and firewalls) + +* They support filesystem sharing through volumes (we'll talk about this later) + +* They only support running one Docker VM at a time ... + + ... so if you want to run a full cluster locally, install e.g. the Docker Toolbox + +* They can co-exist with the Docker Toolbox + +--- + +## Important PSA about security + +* If you have access to the Docker control socket, you can take over the machine + + (Because you can run containers that will access the machine's resources) + +* Therefore, on Linux machines, the `docker` user is equivalent to `root` + +* You should restrict access to it like you would protect `root` + +* By default, the Docker control socket belongs to the `docker` group + +* You can add trusted users to the `docker` group + +* Otherwise, you will have to prefix every `docker` command with `sudo`, e.g.: + + ```bash + sudo docker version + ``` diff --git a/slides/intro/Local_Development_Workflow.md b/slides/intro/Local_Development_Workflow.md new file mode 100644 index 00000000..0f12b17f --- /dev/null +++ b/slides/intro/Local_Development_Workflow.md @@ -0,0 +1,308 @@ + +class: title + +# Local Development Workflow with Docker + +![construction](Local_Development_Workflow/construction.jpg) + +--- + +## Objectives + +At the end of this section, you will be able to: + +* Share code between container and host. + +* Use a simple local development workflow. + +--- + +## Using a Docker container for local development + +We want to solve the following issues: + +- "Works on my machine" + +- "Not the same version" + +- "Missing dependency" + +By using Docker containers, we will get a consistent development environment. + +--- + +## Our "namer" application + +* The code is available on https://github.com/jpetazzo/namer. + +* The image jpetazzo/namer is automatically built by the Docker Hub. + +Let's run it with: + +```bash +$ docker run -dP jpetazzo/namer +``` + +Check the port number with `docker ps` and open the application. + +--- + +## Let's look at the code + +Let's download our application's source code. + +```bash +$ git clone https://github.com/jpetazzo/namer +$ cd namer +$ ls -1 +company_name_generator.rb +config.ru +docker-compose.yml +Dockerfile +Gemfile +``` + +--- + +## Where's my code? + +According to the Dockerfile, the code is copied into `/src` : + +```dockerfile +FROM ruby +MAINTAINER Education Team at Docker + +COPY . /src +WORKDIR /src +RUN bundler install + +CMD ["rackup", "--host", "0.0.0.0"] +EXPOSE 9292 +``` + +We want to make changes *inside the container* without rebuilding it each time. + +For that, we will use a *volume*. + +--- + +## Our first volume + +We will tell Docker to map the current directory to `/src` in the container. + +```bash +$ docker run -d -v $(pwd):/src -p 80:9292 jpetazzo/namer +``` + +* `-d`: the container should run in detached mode (in the background). + +* `-v`: the following host directory should be mounted inside the container. + +* `-p`: connections to port 80 on the host should be routed to port 9292 in the container. + +* `jpetazzo/namer` is the name of the image we will run. + +* We don't specify a command to run because is is already set in the Dockerfile. + +--- + +## Mounting volumes inside containers + +The `-v` flag mounts a directory from your host into your Docker +container. The flag structure is: + +```bash +[host-path]:[container-path]:[rw|ro] +``` + +* If [host-path] or [container-path] doesn't exist it is created. + +* You can control the write status of the volume with the `ro` and + `rw` options. + +* If you don't specify `rw` or `ro`, it will be `rw` by default. + +There will be a full chapter about volumes! + +--- + +## Testing the development container + +Now let us see if our new container is running. + +```bash +$ docker ps +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +045885b68bc5 trai... rackup 3 seconds ago Up ... 0.0.0.0:80->9292/tcp ... +``` + +--- + +## Viewing our application + +Now let's browse to our web application on: + +```bash +http://:80 +``` + +We can see our company naming application. + +![web application 1](Local_Development_Workflow/webapp1.png) + +--- + +## Making a change to our application + +Our customer really doesn't like the color of our text. Let's change it. + +```bash +$ vi company_name_generator.rb +``` + +And change + +```css +color: royalblue; +``` + +To: + +```css +color: red; +``` + +--- + +## Refreshing our application + +Now let's refresh our browser: + +```bash +http://:80 +``` + +We can see the updated color of our company naming application. + +![web application 2](Local_Development_Workflow/webapp2.png) + +--- + +## Improving the workflow with Compose + +* You can also start the container with the following command: + +```bash +$ docker-compose up -d +``` + +* This works thanks to the Compose file, `docker-compose.yml`: + +```yaml +www: + build: . + volumes: + - .:/src + ports: + - 80:9292 +``` + +--- + +## Why Compose? + +* Specifying all those "docker run" parameters is tedious. + +* And error-prone. + +* We can "encode" those parameters in a "Compose file." + +* When you see a `docker-compose.yml` file, you know that you can use `docker-compose up`. + +* Compose can also deal with complex, multi-container apps. +
(More on this later.) + +--- + +## Recap of the development workflow + +1. Write a Dockerfile to build an image containing our development environment. +
+ (Rails, Django, ... and all the dependencies for our app) + +2. Start a container from that image. +
+ Use the `-v` flag to mount our source code inside the container. + +3. Edit the source code outside the containers, using regular tools. +
+ (vim, emacs, textmate...) + +4. Test the application. +
+ (Some frameworks pick up changes automatically. +
Others require you to Ctrl-C + restart after each modification.) + +5. Iterate and repeat steps 3 and 4 until satisfied. + +6. When done, commit+push source code changes. + +--- + +class: extra-details + +## Debugging inside the container + +Docker has a command called `docker exec`. + +It allows users to run a new process in a container which is already running. + +If sometimes you find yourself wishing you could SSH into a container: you can use `docker exec` instead. + +You can get a shell prompt inside an existing container this way, or run an arbitrary process for automation. + +--- + +class: extra-details + +## `docker exec` example + +```bash +$ # You can run ruby commands in the area the app is running and more! +$ docker exec -it bash +root@5ca27cf74c2e:/opt/namer# irb +irb(main):001:0> [0, 1, 2, 3, 4].map {|x| x ** 2}.compact +=> [0, 1, 4, 9, 16] +irb(main):002:0> exit +``` + +--- + +class: extra-details + +## Stopping the container + +Now that we're done let's stop our container. + +```bash +$ docker stop +``` + +And remove it. + +```bash +$ docker rm +``` + +--- + +## Section summary + +We've learned how to: + +* Share code between container and host. + +* Set our working directory. + +* Use a simple local development workflow. + diff --git a/slides/intro/Multi_Stage_Builds.md b/slides/intro/Multi_Stage_Builds.md new file mode 100644 index 00000000..999913db --- /dev/null +++ b/slides/intro/Multi_Stage_Builds.md @@ -0,0 +1,106 @@ +# Multi-stage builds + +* In the previous example, our final image contain: + + * our `hello` program + + * its source code + + * the compiler + +* Only the first one is strictly necessary. + +* We are going to see how to obtain an image without the superfluous components. + +--- + +## Multi-stage builds principles + +* At any point in our `Dockerfile`, we can add a new `FROM` line. + +* This line starts a new stage of our build. + +* Each stage can access the files of the previous stages with `COPY --from=...`. + +* When a build is tagged (with `docker build -t ...`), the last stage is tagged. + +* Previous stages are not discarded: they will be used for caching, and can be referenced. + +--- + +## Multi-stage builds in practice + +* Each stage is numbered, starting at `0` + +* We can copy a file from a previous stage by indicating its number, e.g.: + + ```dockerfile + COPY --from=0 /file/from/first/stage /location/in/current/stage + ``` + +* We can also name stages, and reference these names: + + ```dockerfile + FROM golang AS builder + RUN ... + FROM alpine + COPY --from=builder /go/bin/mylittlebinary /usr/local/bin/ + ``` + +--- + +## Implementing multi-stage builds for our C program + +We will change our Dockerfile to: + +* give a nickname to the first stage: `compiler` + +* add a second stage using the same `ubuntu` base image + +* add the `hello` binary to the second stage + +* make sure that `CMD` is in the second stage + +The resulting Dockerfile is on the next slide. + +--- + +## Revised Dockerfile implementing multi-stage build + +Here is the final Dockerfile: + +```dockerfile +FROM ubuntu AS compiler +RUN apt-get update +RUN apt-get install -y build-essential +COPY hello.c / +RUN make hello +FROM ubuntu +COPY --from=compiler /hello /hello +CMD /hello +``` + +Let's build it, and check that it works correctly: + +```bash +docker build -t hellomultistage . +docker run hellomultistage +``` + +--- + +## Comparing single-stage and multi-stage image sizes + +List our images with `docker images`, and check the size of: + +- the `ubuntu` base image, + +- the single-stage `hello` image, + +- the multi-stage `hellomultistage` image. + +We can achieve even smaller images if we use smaller base images. + +However, if we use common base images (e.g. if we standardize on `ubuntu`), +these common images will be pulled only once per node, so they are +virtually "free." diff --git a/slides/intro/Naming_And_Inspecting.md b/slides/intro/Naming_And_Inspecting.md new file mode 100644 index 00000000..b191f5e4 --- /dev/null +++ b/slides/intro/Naming_And_Inspecting.md @@ -0,0 +1,140 @@ + +class: title + +# Naming and inspecting containers + +![Markings on container door](Naming_And_Inspecting/containermarkings.jpg) + +--- + +## Objectives + +In this lesson, we will learn about an important +Docker concept: container *naming*. + +Naming allows us to: + +* Reference easily a container. + +* Ensure unicity of a specific container. + +We will also see the `inspect` command, which gives a lot of details about a container. + +--- + +## Naming our containers + +So far, we have referenced containers with their ID. + +We have copy-pasted the ID, or used a shortened prefix. + +But each container can also be referenced by its name. + +If a container is named `thumbnail-worker`, I can do: + +```bash +$ docker logs thumbnail-worker +$ docker stop thumbnail-worker +etc. +``` + +--- + +## Default names + +When we create a container, if we don't give a specific +name, Docker will pick one for us. + +It will be the concatenation of: + +* A mood (furious, goofy, suspicious, boring...) + +* The name of a famous inventor (tesla, darwin, wozniak...) + +Examples: `happy_curie`, `clever_hopper`, `jovial_lovelace` ... + +--- + +## Specifying a name + +You can set the name of the container when you create it. + +```bash +$ docker run --name ticktock jpetazzo/clock +``` + +If you specify a name that already exists, Docker will refuse +to create the container. + +This lets us enforce unicity of a given resource. + +--- + +## Renaming containers + +* You can rename containers with `docker rename`. + +* This allows you to "free up" a name without destroying the associated container. + +--- + +## Inspecting a container + +The `docker inspect` command will output a very detailed JSON map. + +```bash +$ docker inspect +[{ + "AppArmorProfile": "", + "Args": [], + "Config": { + "AttachStderr": true, + "AttachStdin": false, + "AttachStdout": true, + "Cmd": [ + "bash" + ], + "CpuShares": 0, +... +``` + +There are multiple ways to consume that information. + +--- + +## Parsing JSON with the Shell + +* You *could* grep and cut or awk the output of `docker inspect`. + +* Please, don't. + +* It's painful. + +* If you really must parse JSON from the Shell, use JQ! (It's great.) + +```bash +$ docker inspect | jq . +``` + +* We will see a better solution which doesn't require extra tools. + +--- + +## Using `--format` + +You can specify a format string, which will be parsed by +Go's text/template package. + +```bash +$ docker inspect --format '{{ json .Created }}' +"2015-02-24T07:21:11.712240394Z" +``` + +* The generic syntax is to wrap the expression with double curly braces. + +* The expression starts with a dot representing the JSON object. + +* Then each field or member can be accessed in dotted notation syntax. + +* The optional `json` keyword asks for valid JSON output. +
(e.g. here it adds the surrounding double-quotes.) diff --git a/slides/intro/Start_And_Attach.md b/slides/intro/Start_And_Attach.md new file mode 100644 index 00000000..9bef5328 --- /dev/null +++ b/slides/intro/Start_And_Attach.md @@ -0,0 +1,163 @@ +# Restarting and attaching to containers + +We have started containers in the foreground, and in the background. + +In this chapter, we will see how to: + +* Put a container in the background. +* Attach to a background container to bring it to the foreground. +* Restart a stopped container. + +--- + +## Background and foreground + +The distinction between foreground and background containers is arbitrary. + +From Docker's point of view, all containers are the same. + +All containers run the same way, whether there is a client attached to them or not. + +It is always possible to detach from a container, and to reattach to a container. + +Analogy: attaching to a container is like plugging a keyboard and screen to a physical server. + +--- + +## Detaching from a container + +* If you have started an *interactive* container (with option `-it`), you can detach from it. + +* The "detach" sequence is `^P^Q`. + +* Otherwise you can detach by killing the Docker client. + + (But not by hitting `^C`, as this would deliver `SIGINT` to the container.) + +What does `-it` stand for? + +* `-t` means "allocate a terminal." +* `-i` means "connect stdin to the terminal." + +--- + +class: extra-details + +## Specifying a custom detach sequence + +* You don't like `^P^Q`? No problem! +* You can change the sequence with `docker run --detach-keys`. +* This can also be passed as a global option to the engine. + +Start a container with a custom detach command: + +```bash +$ docker run -ti --detach-keys ctrl-x,x jpetazzo/clock +``` + +Detach by hitting `^X x`. (This is ctrl-x then x, not ctrl-x twice!) + +Check that our container is still running: + +```bash +$ docker ps -l +``` + +--- + +class: extra-details + +## Attaching to a container + +You can attach to a container: + +```bash +$ docker attach +``` + +* The container must be running. +* There *can* be multiple clients attached to the same container. +* If you don't specify `--detach-keys` when attaching, it defaults back to `^P^Q`. + +Try it on our previous container: + +```bash +$ docker attach $(docker ps -lq) +``` + +Check that `^X x` doesn't work, but `^P ^Q` does. + +--- + +## Detaching from non-interactive containers + +* **Warning:** if the container was started without `-it`... + + * You won't be able to detach with `^P^Q`. + * If you hit `^C`, the signal will be proxied to the container. + +* Remember: you can always detach by killing the Docker client. + +--- + +## Checking container output + +* Use `docker attach` if you intend to send input to the container. + +* If you just want to see the output of a container, use `docker logs`. + +```bash +$ docker logs --tail 1 --follow +``` + +--- + +## Restarting a container + +When a container has exited, it is in stopped state. + +It can then be restarted with the `start` command. + +```bash +$ docker start +``` + +The container will be restarted using the same options you launched it +with. + +You can re-attach to it if you want to interact with it: + +```bash +$ docker attach +``` + +Use `docker ps -a` to identify the container ID of a previous `jpetazzo/clock` container, +and try those commands. + +--- + +## Attaching to a REPL + +* REPL = Read Eval Print Loop + +* Shells, interpreters, TUI ... + +* Symptom: you `docker attach`, and see nothing + +* The REPL doesn't know that you just attached, and doesn't print anything + +* Try hitting `^L` or `Enter` + +--- + +class: extra-details + +## SIGWINCH + +* When you `docker attach`, the Docker Engine sends SIGWINCH signals to the container. + +* SIGWINCH = WINdow CHange; indicates a change in window size. + +* This will cause some CLI and TUI programs to redraw the screen. + +* But not all of them. diff --git a/slides/intro/Training_Environment.md b/slides/intro/Training_Environment.md new file mode 100644 index 00000000..9faf67d4 --- /dev/null +++ b/slides/intro/Training_Environment.md @@ -0,0 +1,87 @@ +class: title + +# Our training environment + +![SSH terminal](Use_Training_VM/ssh.jpg) + +--- + +## Our training environment + +- If you are attending a tutorial or workshop: + + - a VM has been provisioned for each student + +- If you are doing or re-doing this course on your own, you can: + + - install Docker locally (as explained in the chapter "Installing Docker") + + - install Docker on e.g. a cloud VM + + - use http://www.play-with-docker.com/ to instantly get a training environment + +--- + +## Our Docker VM + +*This section assumes that you are following this course as part of +a tutorial, training or workshop, where each student is given an +individual Docker VM.* + +- The VM is created just before the training. + +- It will stay up during the whole training. + +- It will be destroyed shortly after the training. + +- It comes pre-loaded with Docker and some other useful tools. + +--- + +## Connecting to your Virtual Machine + +You need an SSH client. + +* On OS X, Linux, and other UNIX systems, just use `ssh`: + +```bash +$ ssh @ +``` + +* On Windows, if you don't have an SSH client, you can download: + + * Putty (www.putty.org) + + * Git BASH (https://git-for-windows.github.io/) + + * MobaXterm (http://moabaxterm.mobatek.net) + +--- + +## Checking your Virtual Machine + +Once logged in, make sure that you can run a basic Docker command: + +.small[ +```bash +$ docker version +Client: + Version: 17.09.0-ce + API version: 1.32 + Go version: go1.8.3 + Git commit: afdb6d4 + Built: Tue Sep 26 22:40:09 2017 + OS/Arch: darwin/amd64 + +Server: + Version: 17.09.0-ce + API version: 1.32 (minimum version 1.12) + Go version: go1.8.3 + Git commit: afdb6d4 + Built: Tue Sep 26 22:45:38 2017 + OS/Arch: linux/amd64 + Experimental: true +``` +] + +If this doesn't work, raise your hand so that an instructor can assist you! diff --git a/slides/intro/Working_With_Volumes.md b/slides/intro/Working_With_Volumes.md new file mode 100644 index 00000000..4b2fe29d --- /dev/null +++ b/slides/intro/Working_With_Volumes.md @@ -0,0 +1,409 @@ + +class: title + +# Working with Volumes + +![volume](Working_With_Volumes/volume.jpg) + +--- + +## Objectives + +At the end of this section, you will be able to: + +* Create containers holding volumes. + +* Share volumes across containers. + +* Share a host directory with one or many containers. + +--- + +## Working with Volumes + +Docker volumes can be used to achieve many things, including: + +* Bypassing the copy-on-write system to obtain native disk I/O performance. + +* Bypassing copy-on-write to leave some files out of `docker commit`. + +* Sharing a directory between multiple containers. + +* Sharing a directory between the host and a container. + +* Sharing a *single file* between the host and a container. + +--- + +## Volumes are special directories in a container + +Volumes can be declared in two different ways. + +* Within a `Dockerfile`, with a `VOLUME` instruction. + +```dockerfile +VOLUME /uploads +``` + +* On the command-line, with the `-v` flag for `docker run`. + +```bash +$ docker run -d -v /uploads myapp +``` + +In both cases, `/uploads` (inside the container) will be a volume. + +--- + +class: extra-details + +## Volumes bypass the copy-on-write system + +Volumes act as passthroughs to the host filesystem. + +* The I/O performance on a volume is exactly the same as I/O performance + on the Docker host. + +* When you `docker commit`, the content of volumes is not brought into + the resulting image. + +* If a `RUN` instruction in a `Dockerfile` changes the content of a + volume, those changes are not recorded neither. + +* If a container is started with the `--read-only` flag, the volume + will still be writable (unless the volume is a read-only volume). + +--- + +class: extra-details + +## Volumes can be shared across containers + +You can start a container with *exactly the same volumes* as another one. + +The new container will have the same volumes, in the same directories. + +They will contain exactly the same thing, and remain in sync. + +Under the hood, they are actually the same directories on the host anyway. + +This is done using the `--volumes-from` flag for `docker run`. + +We will see an example in the following slides. + +--- + +class: extra-details + +## Sharing web application logs with another container + +Let's start a Tomcat container: + +```bash +$ docker run --name webapp -d -p 8080:8080 -v /usr/local/tomcat/logs +``` + +Now, start an `alpine` container accessing the same volume: + +```bash +$ docker run --volumes-from webapp alpine sh -c "tail -f /usr/local/tomcat/logs/*" +``` + +Then, from another window, send requests to our Tomcat container: +```bash +$ curl localhost:8080 +``` + +--- + +## Volumes exist independently of containers + +If a container is stopped, its volumes still exist and are available. + +Volumes can be listed and manipulated with `docker volume` subcommands: + +```bash +$ docker volume ls +DRIVER VOLUME NAME +local 5b0b65e4316da67c2d471086640e6005ca2264f3... +local pgdata-prod +local pgdata-dev +local 13b59c9936d78d109d094693446e174e5480d973... +``` + +Some of those volume names were explicit (pgdata-prod, pgdata-dev). + +The others (the hex IDs) were generated automatically by Docker. + +--- + +## Naming volumes + +* Volumes can be created without a container, then used in multiple containers. + +Let's create a couple of volumes directly. + +```bash +$ docker volume create webapps +webapps +``` + +```bash +$ docker volume create logs +logs +``` + +Volumes are not anchored to a specific path. + +--- + +## Using our named volumes + +* Volumes are used with the `-v` option. + +* When a host path does not contain a /, it is considered to be a volume name. + +Let's start a web server using the two previous volumes. + +```bash +$ docker run -d -p 1234:8080 \ + -v logs:/usr/local/tomcat/logs \ + -v webapps:/usr/local/tomcat/webapps \ + tomcat +``` + +Check that it's running correctly: + +```bash +$ curl localhost:1234 +... (Tomcat tells us how happy it is to be up and running) ... +``` + +--- + +## Using a volume in another container + +* We will make changes to the volume from another container. + +* In this example, we will run a text editor in the other container. + + (But this could be a FTP server, a WebDAV server, a Git receiver...) + +Let's start another container using the `webapps` volume. + +```bash +$ docker run -v webapps:/webapps -w /webapps -ti alpine vi ROOT/index.jsp +``` + +Vandalize the page, save, exit. + +Then run `curl localhost:1234` again to see your changes. + +--- + +## Managing volumes explicitly + +In some cases, you want a specific directory on the host to be mapped +inside the container: + +* You want to manage storage and snapshots yourself. + + (With LVM, or a SAN, or ZFS, or anything else!) + +* You have a separate disk with better performance (SSD) or resiliency (EBS) + than the system disk, and you want to put important data on that disk. + +* You want to share your source directory between your host (where the + source gets edited) and the container (where it is compiled or executed). + +Wait, we already met the last use-case in our example development workflow! +Nice. + +```bash +$ docker run -d -v /path/on/the/host:/path/in/container image ... +``` + +--- + +class: extra-details + +## Migrating data with `--volumes-from` + +The `--volumes-from` option tells Docker to re-use all the volumes +of an existing container. + +* Scenario: migrating from Redis 2.8 to Redis 3.0. + +* We have a container (`myredis`) running Redis 2.8. + +* Stop the `myredis` container. + +* Start a new container, using the Redis 3.0 image, and the `--volumes-from` option. + +* The new container will inherit the data of the old one. + +* Newer containers can use `--volumes-from` too. + +--- + +class: extra-details + +## Data migration in practice + +Let's create a Redis container. + +```bash +$ docker run -d --name redis28 redis:2.8 +``` + +Connect to the Redis container and set some data. + +```bash +$ docker run -ti --link redis28:redis alpine telnet redis 6379 +``` + +Issue the following commands: + +```bash +SET counter 42 +INFO server +SAVE +QUIT +``` + +--- + +class: extra-details + +## Upgrading Redis + +Stop the Redis container. + +```bash +$ docker stop redis28 +``` + +Start the new Redis container. + +```bash +$ docker run -d --name redis30 --volumes-from redis28 redis:3.0 +``` + +--- + +class: extra-details + +## Testing the new Redis + +Connect to the Redis container and see our data. + +```bash +docker run -ti --link redis30:redis alpine telnet redis 6379 +``` + +Issue a few commands. + +```bash +GET counter +INFO server +QUIT +``` + +--- + +## What happens when you remove containers with volumes? + +* Volumes are kept around. + +* You can list them with `docker volume ls`. + +* You can access them by creating a container with `docker run -v`. + +* You can remove them with `docker volume rm` or `docker system prune`. + +Ultimately, _you_ are the one responsible for logging, +monitoring, and backup of your volumes. + +--- + +class: extra-details + +## Checking volumes defined by an image + +Wondering if an image has volumes? Just use `docker inspect`: + +```bash +$ # docker inspect training/datavol +[{ + "config": { + . . . + "Volumes": { + "/var/webapp": {} + }, + . . . +}] +``` + +--- + +class: extra-details + +## Checking volumes used by a container + +To look which paths are actually volumes, and to what they are bound, +use `docker inspect` (again): + +```bash +$ docker inspect +[{ + "ID": "", +. . . + "Volumes": { + "/var/webapp": "/var/lib/docker/vfs/dir/f4280c5b6207ed531efd4cc673ff620cef2a7980f747dbbcca001db61de04468" + }, + "VolumesRW": { + "/var/webapp": true + }, +}] +``` + +* We can see that our volume is present on the file system of the Docker host. + +--- + +## Sharing a single file between the host and a container + +The same `-v` flag can be used to share a single file. + +One of the most interesting examples is to share the Docker control socket. + +```bash +$ docker run -it -v /var/run/docker.sock:/var/run/docker.sock docker sh +``` + +Warning: when using such mounts, the container gains root-like access to the host. +It can potentially do bad things. + +--- + +## Volume plugins + +You can install plugins to manage volumes backed by particular storage systems, +or providing extra features. For instance: + +* [dvol](https://github.com/ClusterHQ/dvol) - allows to commit/branch/rollback volumes; +* [Flocker](https://clusterhq.com/flocker/introduction/), [REX-Ray](https://github.com/emccode/rexray) - create and manage volumes backed by an enterprise storage system (e.g. SAN or NAS), or by cloud block stores (e.g. EBS); +* [Blockbridge](http://www.blockbridge.com/), [Portworx](http://portworx.com/) - provide distributed block store for containers; +* and much more! + +--- + +## Section summary + +We've learned how to: + +* Create and manage volumes. + +* Share volumes across containers. + +* Share a host directory with one or many containers.