container.training/slides/containers/Getting_Inside.md

class: title

Getting inside a container

---

Objectives

On a traditional server or VM, we sometimes need to:

• log into the machine (with SSH or on the console),

• analyze the disks (by removing them or rebooting with a rescue system).

In this chapter, we will see how to do that with containers.

---

Getting a shell

Every once in a while, we want to log into a machine.

In an perfect world, this shouldn't be necessary.

• You need to install or update packages (and their configuration)?

  Use configuration management. (e.g. Ansible, Chef, Puppet, Salt...)

• You need to view logs and metrics?

  Collect and access them through a centralized platform.

In the real world, though ... we often need shell access!

---

Not getting a shell

Even without a perfect deployment system, we can do many operations without getting a shell.

• Installing packages can (and should) be done in the container image.

• Configuration can be done at the image level, or when the container starts.

• Dynamic configuration can be stored in a volume (shared with another container).

• Logs written to stdout are automatically collected by the Docker Engine.

• Other logs can be written to a shared volume.

• Process information and metrics are visible from the host.

Let's save logging, volumes ... for later, but let's have a look at process information!

---

Viewing container processes from the host

If you run Docker on Linux, container processes are visible on the host.

$ ps faux | less

• Scroll around the output of this command.

• You should see the jpetazzo/clock container.

• A containerized process is just like any other process on the host.

• We can use tools like lsof, strace, gdb ... To analyze them.

---

class: extra-details

What's the difference between a container process and a host process?

• Each process (containerized or not) belongs to namespaces and cgroups.

• The namespaces and cgroups determine what a process can "see" and "do".

• Analogy: each process (containerized or not) runs with a specific UID (user ID).

• UID=0 is root, and has elevated privileges. Other UIDs are normal users.

We will give more details about namespaces and cgroups later.

---

Getting a shell in a running container

• Sometimes, we need to get a shell anyway.

• We could run some SSH server in the container ...

• But it is easier to use docker exec.

$ docker exec -ti ticktock sh

• This creates a new process (running sh) inside the container.

• This can also be done "manually" with the tool nsenter.

---

Caveats

• The tool that you want to run needs to exist in the container.

• Some tools (like ip netns exec) let you attach to one namespace at a time.

  (This lets you e.g. setup network interfaces, even if you don't have ifconfig or ip in the container.)

• Most importantly: the container needs to be running.

• What if the container is stopped or crashed?

---

Getting a shell in a stopped container

• A stopped container is only storage (like a disk drive).

• We cannot SSH into a disk drive or USB stick!

• We need to connect the disk to a running machine.

• How does that translate into the container world?

---

Analyzing a stopped container

As an exercise, we are going to try to find out what's wrong with jpetazzo/crashtest.

docker run jpetazzo/crashtest

The container starts, but then stops immediately, without any output.

What would MacGyver™ do?

First, let's check the status of that container.

docker ps -l

---

Viewing filesystem changes

• We can use docker diff to see files that were added / changed / removed.

docker diff <container_id>

• The container ID was shown by docker ps -l.

• We can also see it with docker ps -lq.

• The output of docker diff shows some interesting log files!

---

Accessing files

• We can extract files with docker cp.

docker cp <container_id>:/var/log/nginx/error.log .

• Then we can look at that log file.

cat error.log

(The directory /run/nginx doesn't exist.)

---

Exploring a crashed container

• We can restart a container with docker start ...

• ... But it will probably crash again immediately!

• We cannot specify a different program to run with docker start

• But we can create a new image from the crashed container

docker commit <container_id> debugimage

• Then we can run a new container from that image, with a custom entrypoint

docker run -ti --entrypoint sh debugimage

---

class: extra-details

Obtaining a complete dump

• We can also dump the entire filesystem of a container.

• This is done with docker export.

• It generates a tar archive.

docker export <container_id> | tar tv

This will give a detailed listing of the content of the container.