Merge master

This commit is contained in:
Jerome Petazzoni
2020-01-13 16:38:16 -06:00
16 changed files with 327 additions and 36 deletions

1
.gitignore vendored
View File

@@ -8,6 +8,7 @@ slides/*.yml.html
slides/autopilot/state.yaml
slides/index.html
slides/past.html
slides/slides.zip
node_modules
### macOS ###

View File

@@ -1,4 +1,4 @@
FROM alpine:3.9
RUN apk add --no-cache entr py-pip git
FROM alpine:3.11
RUN apk add --no-cache entr py3-pip git zip
COPY requirements.txt .
RUN pip install -r requirements.txt
RUN pip3 install -r requirements.txt

View File

@@ -5,3 +5,6 @@
# And this allows to do "git clone https://container.training".
/info/refs service=git-upload-pack https://github.com/jpetazzo/container.training/info/refs?service=git-upload-pack
/dockermastery https://www.udemy.com/course/docker-mastery/?referralCode=1410924A733D33635CCB
/kubernetesmastery https://www.udemy.com/course/kubernetesmastery/?referralCode=7E09090AF9B79E6C283F

View File

@@ -14,6 +14,7 @@ once)
./appendcheck.py $YAML.html
done
fi
zip -qr slides.zip . && echo "Created slides.zip archive."
;;
forever)

View File

@@ -0,0 +1,137 @@
# Init systems and PID 1
In this chapter, we will consider:
- the role of PID 1 in the world of Docker,
- how to avoid some common pitfalls due to the misuse of init systems.
---
## What's an init system?
- On UNIX, the "init system" (or "init" in short) is PID 1.
- It is the first process started by the kernel when the system starts.
- It has multiple responsibilities:
- start every other process on the machine,
- reap orphaned zombie processes.
---
class: extra-details
## Orphaned zombie processes ?!?
- When a process exits (or "dies"), it becomes a "zombie".
(Zombie processes show up in `ps` or `top` with the status code `Z`.)
- Its parent process must *reap* the zombie process.
(This is done by calling `waitpid()` to retrieve the process' exit status.)
- When a process exits, if it has child processes, these processes are "orphaned."
- They are then re-parented to PID 1, init.
- Init therefore needs to take care of these orphaned processes when they exit.
---
## Don't use init systems in containers
- It's often tempting to use an init system or a process manager.
(Examples: *systemd*, *supervisord*...)
- Our containers are then called "system containers".
(By contrast with "application containers".)
- "System containers" are similar to lightweight virtual machines.
- They have multiple downsides:
- when starting multiple processes, their logs get mixed on stdout,
- if the application process dies, the container engine doesn't see it.
- Overall, they make it harder to operate troubleshoot containerized apps.
---
## Exceptions and workarounds
- Sometimes, it's convenient to run a real init system like *systemd*.
(Example: a CI system whose goal is precisely to test an init script or unit file.)
- If we need to run multiple processes: can we use multiple containers?
(Example: [this Compose file](https://github.com/jpetazzo/container.training/blob/master/compose/simple-k8s-control-plane/docker-compose.yaml) runs multiple processes together.)
- When deploying with Kubernetes:
- a container belong to a pod,
- a pod can have multiple containers.
---
## What about these zombie processes?
- Our application runs as PID 1 in the container.
- Our application may or may not be designed to reap zombie processes.
- If our application uses subprocesses and doesn't reap them ...
... this can lead to PID exhaustion!
(Or, more realistically, to a confusing herd of zombie processes.)
- How can we solve this?
---
## Tini to the rescue
- Docker can automatically provide a minimal `init` process.
- This is enabled with `docker run --init ...`
- It uses a small init system ([tini](https://github.com/krallin/tini)) as PID 1:
- it reaps zombies,
- it forwards signals,
- it exits when the child exits.
- It is totally transparent to our application.
- We should use it if our application creates subprocess but doesn't reap them.
---
class: extra-details
## What about Kubernetes?
- Kubernetes does not expose that `--init` option.
- However, we can achieve the same result with [Process Namespace Sharing](https://kubernetes.io/docs/tasks/configure-pod-container/share-process-namespace/).
- When Process Namespace Sharing is enabled, PID 1 will be `pause`.
- That `pause` process takes care of reaping zombies.
- Process Namespace Sharing is available since Kubernetes 1.16.
- If you're using an older version of Kubernetes ...
... you might have to add `tini` explicitly to your Docker image.

View File

@@ -0,0 +1,47 @@
# Container Super-structure
- Multiple orchestration platforms support some kind of container super-structure.
(i.e., a construct or abstraction bigger than a single container.)
- For instance, on Kubernetes, this super-structure is called a *pod*.
- A pod is a group of containers (it could be a single container, too).
- These containers run together, on the same host.
(A pod cannot straddle multiple hosts.)
- All the containers in a pod have the same IP address.
- How does that map to the Docker world?
---
class: pic
## Anatomy of a Pod
![Pods](images/kubernetes_pods.svg)
---
## Pods in Docker
- The containers inside a pod share the same network namespace.
(Just like when using `docker run --net=container:<container_id>` with the CLI.)
- As a result, they can communicate together over `localhost`.
- In addition to "our" containers, the pod has a special container, the *sandbox*.
- That container uses a special image: `k8s.gcr.io/pause`.
(This is visible when listing containers running on a Kubernetes node.)
- Containers within a pod have independent filesystems.
- They can share directories by using a mechanism called *volumes.*
(Which is similar to the concept of volumes in Docker.)

View File

@@ -0,0 +1 @@
<mxfile host="www.draw.io" modified="2019-12-06T15:04:22.728Z" agent="Mozilla/5.0 (X11; Linux x86_64; rv:71.0) Gecko/20100101 Firefox/71.0" etag="zsQLtxL9GRXJF3jcROIq" version="12.3.7" type="device" pages="1"><diagram id="hOpsmMj0j3CSse8MyRSQ" name="Page-1">3VhLU9swEP41nmkPzcR2EpIjCaHtUEo6HCi9dBRb2BoUy8hyHvz6rmzJD9mBQBJgmoMjrVcr6dtvVytb7mSx/spRHF4yH1PL6fpryz2zHMcZ9Rz4k5JNLrFtd5RLAk58JSsF1+QRK2FXSVPi46SmKBijgsR1oceiCHuiJkOcs1Vd7Y7R+qwxCnBDcO0h2pTeEF+EuXTonJTyb5gEoZ7ZHqj9LZBWVjtJQuSzVUXkTi13whkTeWuxnmAq0dO45OPOt7wtFsZxJHYZ8PfXAwsvwsvLP5duOpn2bx4ufnyx1WqXiKZqx5YzoGBw7JMlNAPZDFkiQOkTGF8iDk9K5vC8T+eYYnhz3ul0Putxc66HaQkoVIwpNMRGQ8xZGvlYrrILr1chEfg6Rp58uwJWyfnFgkLPhiaiJIigzXPMlbUl5gKvtwJjF3ADUTFbYME3oKIGDJWDFEXdruqvSn/3ekoWVn2tPYsUx4LCdOkGaChPvMQrDafMmL8fbiHj5JFFAmmBhIwAz08VoILFR4GztyOaheLB0XQaaMYoTXCeNAQiEeb7YXsA0AoubeogVlBz3RbUjgaa2wAtCki0/nBA2S38elukei0Z1AAJR/6pPIug51GUJMSr4wJ755vf1c6tBLTT192ztQI47210b01EMQzalVHQKwfJTjHGh/NNLQ3TOVtNS4FykFR52j2wO5ZyDz9PIIF4gMVz0dl0d8Wd/RZvahnHFAmyrC+3zcVqhhkjkaiwqWvk/oHBknybalT1cDUN9Q1DtmEox6FhCGiBNhW1WCok2xfcM7Kr7dYOfWjkFks6F5i+nuHNGiHm0miI00TSZR0ziOiPl0SdlpP8bXOD3TzJd0sOCfBFaHHEIvxBE0a2znMiUcmUd00g7xXwPSNOHbOG2zXgTUNFJjl2wA/eIODtYQttG7eCn1isGL+3JIQDtJDxnD9B8n02yeU7XgkaxiO0wEmWLLKbEydRsON1AvKHaL8zeMBBSFPN2ndBfD+jM8cJeUTzzJSks/IO2O2Prf6ZnM4dUwTXnjHy7oMswU0YZTyb2r3LftIOE8BSJm2PyrBoSW7q2qqmtAo6VgPmicyyNRV2O1Bl92rM0XXwvkfm0AigugF2d5dgYVD0MKRslqQN3wNTYpxlTIGfP3LmhQ+vUkGJTLKZ3Ef8/gpGEZHlwE5XJsgk/zThHOmscp3mWTVoyYPDox1VB6hjP3r2t/XnKBP0F5d7hiF7aITBlux/sFgY/E+x4JhV+LvHwsn+saBLLV1P3VZrK7lxe1QWXtX6bIY5gW3Ig+pFJdUOd7KcNu8VfeaHoZNXBp9jlvlm+f7q4INu+T02Vy8/a7vTfw==</diagram></mxfile>

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 11 KiB

View File

@@ -1,5 +1,14 @@
#!/usr/bin/env python2
#!/usr/bin/env python3
# coding: utf-8
FLAGS=dict(
cz=u"🇨🇿",
de=u"🇩🇪",
fr=u"🇫🇷",
uk=u"🇬🇧",
us=u"🇺🇸",
)
TEMPLATE="""<html>
<head>
<title>{{ title }}</title>
@@ -34,7 +43,7 @@ TEMPLATE="""<html>
{% for item in coming_soon %}
<tr>
<td>{{ item.title }}</td>
<td>{{ item.flag }} {{ item.title }}</td>
<td>{% if item.slides %}<a class="slides" href="{{ item.slides }}" />{% endif %}</td>
<td>{% if item.attend %}<a class="attend" href="{{ item.attend }}" />
{% else %}
@@ -123,13 +132,13 @@ TEMPLATE="""<html>
</table>
</div>
</body>
</html>""".decode("utf-8")
</html>"""
import datetime
import jinja2
import yaml
items = yaml.load(open("index.yaml"))
items = yaml.safe_load(open("index.yaml"))
# Items with a date correspond to scheduled sessions.
# Items without a date correspond to self-paced content.
@@ -160,6 +169,7 @@ for item in items:
item["prettydate"] = date_begin.strftime("%B %d{}, %Y").format(suffix)
item["begin"] = date_begin
item["end"] = date_end
item["flag"] = FLAGS.get(item.get("country"),"")
today = datetime.date.today()
coming_soon = [i for i in items if i.get("date") and i["end"] >= today]
@@ -177,10 +187,10 @@ with open("index.html", "w") as f:
past_workshops=past_workshops,
self_paced=self_paced,
recorded_workshops=recorded_workshops
).encode("utf-8"))
))
with open("past.html", "w") as f:
f.write(template.render(
title="Container Training",
all_past_workshops=past_workshops
).encode("utf-8"))
))

View File

@@ -1,3 +1,66 @@
- date: 2020-03-06
country: uk
city: London
event: QCON
speaker: jpetazzo
title: Kubernetes Intensive Course
attend: https://qconlondon.com/london2020/workshop/kubernetes-intro
#slides: https://qconuk2019.container.training/
- date: 2020-03-05
country: uk
city: London
event: QCON
speaker: jpetazzo
title: Docker Intensive Course
attend: https://qconlondon.com/london2020/workshop/docker-intensive-course
#slides: https://qconuk2019.container.training/
- date: 2020-02-03
country: fr
city: Paris
event: ENIX SAS
speaker: jpetazzo
title: Fondamentaux Conteneurs et Docker (in French)
lang: fr
attend: https://enix.io/fr/services/formation/
- date: 2020-02-04
country: fr
city: Paris
event: ENIX SAS
speaker: jpetazzo
title: Fondamentaux Orchestration et Kubernetes (in French)
lang: fr
attend: https://enix.io/fr/services/formation/
- date: 2020-02-05
country: fr
city: Paris
event: ENIX SAS
speaker: jpetazzo
title: Kubernetes et Méthodologies DevOps (in French)
lang: fr
attend: https://enix.io/fr/services/formation/
- date: 2020-02-06
country: fr
city: Paris
event: ENIX SAS
speaker: jpetazzo
title: Kubernetes Avancé (in French)
lang: fr
attend: https://enix.io/fr/services/formation/
- date: 2020-02-07
country: fr
city: Paris
event: ENIX SAS
speaker: jpetazzo
title: Opérer Kubernetes (in French)
lang: fr
attend: https://enix.io/fr/services/formation/
- date: [2019-11-04, 2019-11-05]
country: de
city: Berlin

View File

@@ -66,6 +66,7 @@ chapters:
#- k8s/kustomize.md
#- k8s/helm.md
#- k8s/create-chart.md
#- k8s/create-more-charts.md
#- k8s/netpol.md
#- k8s/authn-authz.md
#- k8s/csr-api.md

View File

@@ -61,6 +61,7 @@ chapters:
- k8s/namespaces.md
- k8s/helm.md
- k8s/create-chart.md
#- k8s/create-more-charts.md
#- k8s/kustomize.md
#- k8s/netpol.md
- k8s/whatsnext.md

View File

@@ -31,21 +31,22 @@ chapters:
#- shared/hastyconclusions.md
- shared/composedown.md
- k8s/concepts-k8s.md
- k8s/kubectlget.md
-
- k8s/kubectlget.md
- k8s/kubectlrun.md
- k8s/logs-cli.md
- shared/declarative.md
- k8s/declarative.md
- k8s/deploymentslideshow.md
-
- k8s/kubenet.md
- k8s/kubectlexpose.md
- k8s/shippingimages.md
- k8s/buildshiprun-selfhosted.md
- k8s/buildshiprun-dockerhub.md
- k8s/ourapponkube.md
-
- k8s/yamldeploy.md
-
- k8s/setup-k8s.md
- k8s/dashboard.md
#- k8s/kubectlscale.md
@@ -53,9 +54,6 @@ chapters:
- shared/hastyconclusions.md
- k8s/daemonset.md
- k8s/dryrun.md
- k8s/kubectlproxy.md
- k8s/localkubeconfig.md
- k8s/accessinternal.md
-
- k8s/rollout.md
- k8s/healthchecks.md
@@ -63,35 +61,53 @@ chapters:
- k8s/record.md
-
- k8s/namespaces.md
- k8s/kubectlproxy.md
- k8s/localkubeconfig.md
- k8s/accessinternal.md
-
- k8s/ingress.md
- k8s/kustomize.md
- k8s/helm.md
- k8s/create-chart.md
- k8s/create-more-charts.md
-
- k8s/netpol.md
- k8s/authn-authz.md
-
- k8s/podsecuritypolicy.md
- k8s/csr-api.md
- k8s/openid-connect.md
- k8s/podsecuritypolicy.md
- k8s/control-plane-auth.md
-
- k8s/volumes.md
- k8s/build-with-docker.md
- k8s/build-with-kaniko.md
-
- k8s/configuration.md
-
- k8s/logs-centralized.md
- k8s/prometheus.md
-
- k8s/statefulsets.md
- k8s/local-persistent-volumes.md
- k8s/portworx.md
-
- k8s/logs-centralized.md
- k8s/prometheus.md
- k8s/resource-limits.md
- k8s/metrics-server.md
- k8s/cluster-sizing.md
- k8s/horizontal-pod-autoscaler.md
-
- k8s/extending-api.md
- k8s/operators.md
- k8s/operators-design.md
- k8s/staticpods.md
- k8s/owners-and-dependents.md
-
- k8s/dmuc.md
- k8s/multinode.md
- k8s/cni.md
- k8s/apilb.md
- k8s/staticpods.md
-
- k8s/cluster-upgrade.md
- k8s/cluster-backup.md
- k8s/cloud-controller-manager.md
- k8s/gitworkflows.md
-
- k8s/whatsnext.md

View File

@@ -67,6 +67,7 @@ chapters:
- k8s/kustomize.md
- k8s/helm.md
- k8s/create-chart.md
#- k8s/create-more-charts.md
-
- k8s/netpol.md
- k8s/authn-authz.md

View File

@@ -1,4 +1,4 @@
#!/usr/bin/env python2
#!/usr/bin/env python3
# transforms a YAML manifest into a HTML workshop file
import glob
@@ -20,12 +20,19 @@ def anchor(title):
return "toc-" + title
def interstitials_generator():
images = [url.strip() for url in open("interstitials.txt") if url.strip()]
while True:
for image in images:
yield image
interstitials = interstitials_generator()
class Interstitials(object):
def __init__(self):
self.index = 0
self.images = [url.strip() for url in open("interstitials.txt") if url.strip()]
def next(self):
index = self.index % len(self.images)
index += 1
return self.images[index]
interstitials = Interstitials()
def insertslide(markdown, title):
@@ -54,7 +61,7 @@ class: pic
name: {anchor}
class: title
{title}
{title}
.nav[
[Previous section](#{previouslink})
@@ -154,8 +161,6 @@ def gentoc(tree, path=()):
# Returns: (epxandedmarkdown,[list of titles])
# The list of titles can be nested.
def processchapter(chapter, filename):
if isinstance(chapter, unicode):
return processchapter(chapter.encode("utf-8"), filename)
if isinstance(chapter, str):
if "\n" in chapter:
titles = re.findall("^# (.*)", chapter, re.MULTILINE)
@@ -179,14 +184,14 @@ try:
if "REPOSITORY_URL" in os.environ:
repo = os.environ["REPOSITORY_URL"]
else:
repo = subprocess.check_output(["git", "config", "remote.origin.url"])
repo = subprocess.check_output(["git", "config", "remote.origin.url"]).decode("ascii")
repo = repo.strip().replace("git@github.com:", "https://github.com/")
if "BRANCH" in os.environ:
branch = os.environ["BRANCH"]
else:
branch = subprocess.check_output(["git", "rev-parse", "--abbrev-ref", "HEAD"])
branch = subprocess.check_output(["git", "rev-parse", "--abbrev-ref", "HEAD"]).decode("ascii")
branch = branch.strip()
base = subprocess.check_output(["git", "rev-parse", "--show-prefix"])
base = subprocess.check_output(["git", "rev-parse", "--show-prefix"]).decode("ascii")
base = base.strip().strip("/")
urltemplate = ("{repo}/tree/{branch}/{base}/{filename}"
.format(repo=repo, branch=branch, base=base, filename="{}"))
@@ -194,12 +199,12 @@ except:
logging.exception("Could not generate repository URL; generating local URLs instead.")
urltemplate = "file://{pwd}/{filename}".format(pwd=os.environ["PWD"], filename="{}")
try:
commit = subprocess.check_output(["git", "rev-parse", "--short", "HEAD"])
commit = subprocess.check_output(["git", "rev-parse", "--short", "HEAD"]).decode("ascii")
except:
logging.exception("Could not figure out HEAD commit.")
commit = "??????"
try:
dirtyfiles = subprocess.check_output(["git", "status", "--porcelain"])
dirtyfiles = subprocess.check_output(["git", "status", "--porcelain"]).decode("ascii")
except:
logging.exception("Could not figure out repository cleanliness.")
dirtyfiles = "?? git status --porcelain failed"

1
slides/runtime.txt Normal file
View File

@@ -0,0 +1 @@
3.7