Compare commits

..

33 Commits

Author SHA1 Message Date
Jerome Petazzoni
5f0380af79 fix-redirects.sh: adding forced redirect 2020-04-07 16:49:31 -05:00
Jerome Petazzoni
c825b97058 update final words 2018-09-30 20:38:57 -05:00
Jerome Petazzoni
7af11d3120 update final words 2018-09-30 20:38:16 -05:00
Jerome Petazzoni
454e8b8f23 Merge branch 'master' into k8s2d 2018-09-30 19:15:29 -05:00
Jerome Petazzoni
d1609845a9 Merge branch 'kube-ps1' into k8s2d 2018-09-30 17:43:48 -05:00
Jerome Petazzoni
ec3476173a Merge branch 'master' into k8s2d 2018-09-30 17:31:27 -05:00
Jerome Petazzoni
4c1169c785 add wifi info 2018-09-30 15:03:18 +02:00
Jerome Petazzoni
8b9ee3ecbb Merge branch 'cleanup-after-netpol' into k8s2d 2018-09-29 20:40:31 -05:00
Jerome Petazzoni
724753b8b9 For this workshop, deploy portworx to 4 nodes, using loop4 2018-09-29 18:56:05 -05:00
Jerome Petazzoni
5c4267afda Merge branch 'enixlogo' into k8s2d 2018-09-29 18:54:15 -05:00
Jerome Petazzoni
f01bc2a7a9 Fix overlapsing slide number and pics 2018-09-29 18:54:00 -05:00
Jerome Petazzoni
8f89953d25 Fix typo in logistics 2018-09-29 18:53:33 -05:00
Jerome Petazzoni
b07720c703 Merge branch 'master' into k8s2d 2018-09-29 18:44:26 -05:00
Jerome Petazzoni
e3b71d35a6 Merge branch 'master' into k8s2d 2018-09-29 10:06:37 -05:00
Jerome Petazzoni
a32d4132f9 Merge branch 'master' into k8s2d 2018-09-28 19:36:01 -05:00
Jerome Petazzoni
3e1fac323c Timing and location of breaks 2018-09-28 19:29:23 -05:00
Jerome Petazzoni
5bb8c2a6c2 Merge branch 'master' into k8s2d 2018-09-28 19:14:17 -05:00
Jerome Petazzoni
739c20fa4c Merge branch 'master' into k8s2d 2018-09-28 19:05:12 -05:00
Jerome Petazzoni
3200de3bea Merge branch 'master' into k8s2d 2018-09-28 18:49:13 -05:00
Jerome Petazzoni
4e0ec722bc Merge branch 'master' into k8s2d 2018-09-28 18:36:42 -05:00
Jerome Petazzoni
10e6891dca Merge branch 'master' into k8s2d 2018-09-28 18:31:50 -05:00
Jerome Petazzoni
8dd3bccf7a Update cards design; add settings for velocity nyc 2018-09-28 18:25:07 -05:00
Jerome Petazzoni
970cbdd8fd Merge branch 'new-prepare-scripts' into k8s2d 2018-09-28 18:24:36 -05:00
Jerome Petazzoni
67ed422aaf Bump up EBS size to 20G for Portworx 2018-09-28 18:24:14 -05:00
Jerome Petazzoni
720f2fc86e Actually modify the prompt 2018-09-28 18:18:05 -05:00
Jerome Petazzoni
a9e99e3bae merge master 2018-09-28 16:19:23 -05:00
Jerome Petazzoni
ee9b775f01 Merge branch 'enixlogo' into k8s2d 2018-09-28 16:14:32 -05:00
Bridget Kromhout
a94271c14c Slight modifications to current docs/scripts 2018-09-28 16:09:10 -05:00
Jerome Petazzoni
c04dafd698 Assemble deck for Velocity NYC 2018 2018-09-28 16:08:18 -05:00
Jerome Petazzoni
96be5c16bf Merge branch 'new-prepare-scripts' into nyc2018 2018-09-28 15:26:29 -05:00
Jerome Petazzoni
e4e0386efd Add kubectl, kubens, kube_ps1
kubectl and kubens are added as kctl and kns (to avoid clashing with
completion for kubectl). Their completion is added too (so you can
do 'kns kube-sy[TAB]' to switch to kube-system).

kube_ps1 is added and enabled. The default prompt for the docker
user now shows the current context and namespace.
2018-09-28 14:24:20 -05:00
Jerome Petazzoni
9b54ea95ac Massive refactoring of workshopctl
This allows to manage groups of VMs across multiple infrastructure
providers. It also adds support to create groups of VMs on OpenStack.

WARNING: the syntax of workshopctl has changed slightly. Check READMEs
for details.
2018-09-28 13:18:54 -05:00
Jerome Petazzoni
3eaa844c55 Add ENIX logo
Warning: do not merge this branch to your content, otherwise you
will get the ENIX logo in the top right of all your decks
2018-09-08 07:49:38 -05:00
163 changed files with 1503 additions and 9587 deletions

12
.gitignore vendored
View File

@@ -8,15 +8,3 @@ slides/autopilot/state.yaml
slides/index.html
slides/past.html
node_modules
### macOS ###
# General
.DS_Store
.AppleDouble
.LSOverride
### Windows ###
# Windows thumbnail cache files
Thumbs.db
ehthumbs.db
ehthumbs_vista.db

View File

@@ -199,7 +199,7 @@ this section is for you!
locked-down computer, host firewall, etc.
- Horrible wifi, or ssh port TCP/22 not open on network! If wifi sucks you
can try using MOSH https://mosh.org which handles SSH over UDP. TMUX can also
prevent you from losing your place if you get disconnected from servers.
prevent you from loosing your place if you get disconnected from servers.
https://tmux.github.io
- Forget to print "cards" and cut them up for handing out IP's.
- Forget to have fun and focus on your students!

View File

@@ -1,9 +0,0 @@
hostname frr
router bgp 64512
network 1.0.0.2/32
bgp log-neighbor-changes
neighbor kube peer-group
neighbor kube remote-as 64512
neighbor kube route-reflector-client
bgp listen range 0.0.0.0/0 peer-group kube
log stdout

View File

@@ -1,2 +0,0 @@
hostname frr
log stdout

View File

@@ -1,34 +0,0 @@
version: "3"
services:
bgpd:
image: ajones17/frr:662
volumes:
- ./conf:/etc/frr
- ./run:/var/run/frr
network_mode: host
entrypoint: /usr/lib/frr/bgpd -f /etc/frr/bgpd.conf --log=stdout --log-level=debug --no_kernel
restart: always
zebra:
image: ajones17/frr:662
volumes:
- ./conf:/etc/frr
- ./run:/var/run/frr
network_mode: host
entrypoint: /usr/lib/frr/zebra -f /etc/frr/zebra.conf --log=stdout --log-level=debug
restart: always
vtysh:
image: ajones17/frr:662
volumes:
- ./conf:/etc/frr
- ./run:/var/run/frr
network_mode: host
entrypoint: vtysh -c "show ip bgp"
chmod:
image: alpine
volumes:
- ./run:/var/run/frr
command: chmod 777 /var/run/frr

View File

@@ -1,29 +0,0 @@
version: "3"
services:
pause:
ports:
- 8080:8080
image: k8s.gcr.io/pause
etcd:
network_mode: "service:pause"
image: k8s.gcr.io/etcd:3.3.10
command: etcd
kube-apiserver:
network_mode: "service:pause"
image: k8s.gcr.io/hyperkube:v1.14.0
command: kube-apiserver --etcd-servers http://127.0.0.1:2379 --address 0.0.0.0 --disable-admission-plugins=ServiceAccount --allow-privileged
kube-controller-manager:
network_mode: "service:pause"
image: k8s.gcr.io/hyperkube:v1.14.0
command: kube-controller-manager --master http://localhost:8080 --allocate-node-cidrs --cluster-cidr=10.CLUSTER.0.0/16
"Edit the CLUSTER placeholder first. Then, remove this line.":
kube-scheduler:
network_mode: "service:pause"
image: k8s.gcr.io/hyperkube:v1.14.0
command: kube-scheduler --master http://localhost:8080

View File

@@ -1,128 +0,0 @@
---
apiVersion: |+
Make sure you update the line with --master=http://X.X.X.X:8080 below.
Then remove this section from this YAML file and try again.
---
apiVersion: v1
kind: ConfigMap
metadata:
name: kube-router-cfg
namespace: kube-system
labels:
tier: node
k8s-app: kube-router
data:
cni-conf.json: |
{
"cniVersion":"0.3.0",
"name":"mynet",
"plugins":[
{
"name":"kubernetes",
"type":"bridge",
"bridge":"kube-bridge",
"isDefaultGateway":true,
"ipam":{
"type":"host-local"
}
}
]
}
---
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
labels:
k8s-app: kube-router
tier: node
name: kube-router
namespace: kube-system
spec:
template:
metadata:
labels:
k8s-app: kube-router
tier: node
annotations:
scheduler.alpha.kubernetes.io/critical-pod: ''
spec:
serviceAccountName: kube-router
containers:
- name: kube-router
image: docker.io/cloudnativelabs/kube-router
imagePullPolicy: Always
args:
- "--run-router=true"
- "--run-firewall=true"
- "--run-service-proxy=true"
- "--master=http://X.X.X.X:8080"
env:
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: KUBE_ROUTER_CNI_CONF_FILE
value: /etc/cni/net.d/10-kuberouter.conflist
livenessProbe:
httpGet:
path: /healthz
port: 20244
initialDelaySeconds: 10
periodSeconds: 3
resources:
requests:
cpu: 250m
memory: 250Mi
securityContext:
privileged: true
volumeMounts:
- name: lib-modules
mountPath: /lib/modules
readOnly: true
- name: cni-conf-dir
mountPath: /etc/cni/net.d
initContainers:
- name: install-cni
image: busybox
imagePullPolicy: Always
command:
- /bin/sh
- -c
- set -e -x;
if [ ! -f /etc/cni/net.d/10-kuberouter.conflist ]; then
if [ -f /etc/cni/net.d/*.conf ]; then
rm -f /etc/cni/net.d/*.conf;
fi;
TMP=/etc/cni/net.d/.tmp-kuberouter-cfg;
cp /etc/kube-router/cni-conf.json ${TMP};
mv ${TMP} /etc/cni/net.d/10-kuberouter.conflist;
fi
volumeMounts:
- mountPath: /etc/cni/net.d
name: cni-conf-dir
- mountPath: /etc/kube-router
name: kube-router-cfg
hostNetwork: true
tolerations:
- key: CriticalAddonsOnly
operator: Exists
- effect: NoSchedule
key: node-role.kubernetes.io/master
operator: Exists
- effect: NoSchedule
key: node.kubernetes.io/not-ready
operator: Exists
volumes:
- name: lib-modules
hostPath:
path: /lib/modules
- name: cni-conf-dir
hostPath:
path: /etc/cni/net.d
- name: kube-router-cfg
configMap:
name: kube-router-cfg

View File

@@ -1,28 +0,0 @@
version: "3"
services:
pause:
ports:
- 8080:8080
image: k8s.gcr.io/pause
etcd:
network_mode: "service:pause"
image: k8s.gcr.io/etcd:3.3.10
command: etcd
kube-apiserver:
network_mode: "service:pause"
image: k8s.gcr.io/hyperkube:v1.14.0
command: kube-apiserver --etcd-servers http://127.0.0.1:2379 --address 0.0.0.0 --disable-admission-plugins=ServiceAccount
kube-controller-manager:
network_mode: "service:pause"
image: k8s.gcr.io/hyperkube:v1.14.0
command: kube-controller-manager --master http://localhost:8080
kube-scheduler:
network_mode: "service:pause"
image: k8s.gcr.io/hyperkube:v1.14.0
command: kube-scheduler --master http://localhost:8080

View File

@@ -5,3 +5,6 @@ RUN gem install thin
ADD hasher.rb /
CMD ["ruby", "hasher.rb"]
EXPOSE 80
HEALTHCHECK \
--interval=1s --timeout=2s --retries=3 --start-period=1s \
CMD curl http://localhost/ || exit 1

View File

@@ -2,14 +2,14 @@ version: "2"
services:
elasticsearch:
image: elasticsearch:2
image: elasticsearch
# If you need to access ES directly, just uncomment those lines.
#ports:
# - "9200:9200"
# - "9300:9300"
logstash:
image: logstash:2
image: logstash
command: |
-e '
input {
@@ -47,7 +47,7 @@ services:
- "12201:12201/udp"
kibana:
image: kibana:4
image: kibana
ports:
- "5601:5601"
environment:

View File

@@ -1,37 +1,3 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: consul
labels:
app: consul
rules:
- apiGroups: [""]
resources:
- pods
verbs:
- get
- list
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: consul
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: consul
subjects:
- kind: ServiceAccount
name: consul
namespace: default
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: consul
labels:
app: consul
---
apiVersion: v1
kind: Service
metadata:
@@ -58,7 +24,6 @@ spec:
labels:
app: consul
spec:
serviceAccountName: consul
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
@@ -72,11 +37,18 @@ spec:
terminationGracePeriodSeconds: 10
containers:
- name: consul
image: "consul:1.4.4"
image: "consul:1.2.2"
env:
- name: NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
args:
- "agent"
- "-bootstrap-expect=3"
- "-retry-join=provider=k8s label_selector=\"app=consul\""
- "-retry-join=consul-0.consul.$(NAMESPACE).svc.cluster.local"
- "-retry-join=consul-1.consul.$(NAMESPACE).svc.cluster.local"
- "-retry-join=consul-2.consul.$(NAMESPACE).svc.cluster.local"
- "-client=0.0.0.0"
- "-data-dir=/consul/data"
- "-server"

View File

@@ -3,6 +3,7 @@ apiVersion: v1
kind: ServiceAccount
metadata:
name: fluentd
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
@@ -18,6 +19,7 @@ rules:
- get
- list
- watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
@@ -31,18 +33,23 @@ subjects:
- kind: ServiceAccount
name: fluentd
namespace: default
---
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: fluentd
labels:
app: fluentd
k8s-app: fluentd-logging
version: v1
kubernetes.io/cluster-service: "true"
spec:
template:
metadata:
labels:
app: fluentd
k8s-app: fluentd-logging
version: v1
kubernetes.io/cluster-service: "true"
spec:
serviceAccount: fluentd
serviceAccountName: fluentd
@@ -51,7 +58,7 @@ spec:
effect: NoSchedule
containers:
- name: fluentd
image: fluent/fluentd-kubernetes-daemonset:v1.3-debian-elasticsearch-1
image: fluent/fluentd-kubernetes-daemonset:elasticsearch
env:
- name: FLUENT_ELASTICSEARCH_HOST
value: "elasticsearch"
@@ -59,12 +66,14 @@ spec:
value: "9200"
- name: FLUENT_ELASTICSEARCH_SCHEME
value: "http"
# X-Pack Authentication
# =====================
- name: FLUENT_ELASTICSEARCH_USER
value: "elastic"
- name: FLUENT_ELASTICSEARCH_PASSWORD
value: "changeme"
- name: FLUENT_UID
value: "0"
- name: FLUENTD_SYSTEMD_CONF
value: "disable"
- name: FLUENTD_PROMETHEUS_CONF
value: "disable"
resources:
limits:
memory: 200Mi
@@ -85,83 +94,131 @@ spec:
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "1"
creationTimestamp: null
generation: 1
labels:
app: elasticsearch
run: elasticsearch
name: elasticsearch
selfLink: /apis/extensions/v1beta1/namespaces/default/deployments/elasticsearch
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app: elasticsearch
run: elasticsearch
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
creationTimestamp: null
labels:
app: elasticsearch
run: elasticsearch
spec:
containers:
- image: elasticsearch:5
- image: elasticsearch:5.6.8
imagePullPolicy: IfNotPresent
name: elasticsearch
resources:
limits:
memory: 2Gi
requests:
memory: 1Gi
env:
- name: ES_JAVA_OPTS
value: "-Xms1g -Xmx1g"
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 30
---
apiVersion: v1
kind: Service
metadata:
creationTimestamp: null
labels:
app: elasticsearch
run: elasticsearch
name: elasticsearch
selfLink: /api/v1/namespaces/default/services/elasticsearch
spec:
ports:
- port: 9200
protocol: TCP
targetPort: 9200
selector:
app: elasticsearch
run: elasticsearch
sessionAffinity: None
type: ClusterIP
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "1"
creationTimestamp: null
generation: 1
labels:
app: kibana
run: kibana
name: kibana
selfLink: /apis/extensions/v1beta1/namespaces/default/deployments/kibana
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app: kibana
run: kibana
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
creationTimestamp: null
labels:
app: kibana
run: kibana
spec:
containers:
- env:
- name: ELASTICSEARCH_URL
value: http://elasticsearch:9200/
image: kibana:5
image: kibana:5.6.8
imagePullPolicy: Always
name: kibana
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 30
---
apiVersion: v1
kind: Service
metadata:
creationTimestamp: null
labels:
app: kibana
run: kibana
name: kibana
selfLink: /api/v1/namespaces/default/services/kibana
spec:
externalTrafficPolicy: Cluster
ports:
- port: 5601
protocol: TCP
targetPort: 5601
selector:
app: kibana
run: kibana
sessionAffinity: None
type: NodePort

View File

@@ -1,220 +0,0 @@
# Copyright 2017 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Configuration to deploy release version of the Dashboard UI compatible with
# Kubernetes 1.8.
#
# Example usage: kubectl create -f <this_file>
# ------------------- Dashboard Secret ------------------- #
apiVersion: v1
kind: Secret
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard-certs
namespace: kube-system
type: Opaque
---
# ------------------- Dashboard Service Account ------------------- #
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kube-system
---
# ------------------- Dashboard Role & Role Binding ------------------- #
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: kubernetes-dashboard-minimal
namespace: kube-system
rules:
# Allow Dashboard to create 'kubernetes-dashboard-key-holder' secret.
- apiGroups: [""]
resources: ["secrets"]
verbs: ["create"]
# Allow Dashboard to create 'kubernetes-dashboard-settings' config map.
- apiGroups: [""]
resources: ["configmaps"]
verbs: ["create"]
# Allow Dashboard to get, update and delete Dashboard exclusive secrets.
- apiGroups: [""]
resources: ["secrets"]
resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs"]
verbs: ["get", "update", "delete"]
# Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map.
- apiGroups: [""]
resources: ["configmaps"]
resourceNames: ["kubernetes-dashboard-settings"]
verbs: ["get", "update"]
# Allow Dashboard to get metrics from heapster.
- apiGroups: [""]
resources: ["services"]
resourceNames: ["heapster"]
verbs: ["proxy"]
- apiGroups: [""]
resources: ["services/proxy"]
resourceNames: ["heapster", "http:heapster:", "https:heapster:"]
verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: kubernetes-dashboard-minimal
namespace: kube-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: kubernetes-dashboard-minimal
subjects:
- kind: ServiceAccount
name: kubernetes-dashboard
namespace: kube-system
---
# ------------------- Dashboard Deployment ------------------- #
kind: Deployment
apiVersion: apps/v1beta2
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kube-system
spec:
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
k8s-app: kubernetes-dashboard
template:
metadata:
labels:
k8s-app: kubernetes-dashboard
spec:
containers:
- name: kubernetes-dashboard
image: k8s.gcr.io/kubernetes-dashboard-amd64:v1.8.3
ports:
- containerPort: 8443
protocol: TCP
args:
- --auto-generate-certificates
# Uncomment the following line to manually specify Kubernetes API server Host
# If not specified, Dashboard will attempt to auto discover the API server and connect
# to it. Uncomment only if the default does not work.
# - --apiserver-host=http://my-address:port
volumeMounts:
- name: kubernetes-dashboard-certs
mountPath: /certs
# Create on-disk volume to store exec logs
- mountPath: /tmp
name: tmp-volume
livenessProbe:
httpGet:
scheme: HTTPS
path: /
port: 8443
initialDelaySeconds: 30
timeoutSeconds: 30
volumes:
- name: kubernetes-dashboard-certs
secret:
secretName: kubernetes-dashboard-certs
- name: tmp-volume
emptyDir: {}
serviceAccountName: kubernetes-dashboard
# Comment the following tolerations if Dashboard must not be deployed on master
tolerations:
- key: node-role.kubernetes.io/master
effect: NoSchedule
---
# ------------------- Dashboard Service ------------------- #
kind: Service
apiVersion: v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kube-system
spec:
ports:
- port: 443
targetPort: 8443
selector:
k8s-app: kubernetes-dashboard
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
labels:
app: dashboard
name: dashboard
spec:
selector:
matchLabels:
app: dashboard
template:
metadata:
labels:
app: dashboard
spec:
containers:
- args:
- sh
- -c
- apk add --no-cache socat && socat TCP-LISTEN:80,fork,reuseaddr OPENSSL:kubernetes-dashboard.kube-system:443,verify=0
image: alpine
name: dashboard
---
apiVersion: v1
kind: Service
metadata:
labels:
app: dashboard
name: dashboard
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: dashboard
type: NodePort
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: kubernetes-dashboard
labels:
k8s-app: kubernetes-dashboard
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: kubernetes-dashboard
namespace: kube-system

View File

@@ -1,10 +0,0 @@
apiVersion: v1
Kind: Pod
metadata:
name: hello
namespace: default
spec:
containers:
- name: hello
image: nginx

View File

@@ -1,138 +0,0 @@
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: system:aggregated-metrics-reader
labels:
rbac.authorization.k8s.io/aggregate-to-view: "true"
rbac.authorization.k8s.io/aggregate-to-edit: "true"
rbac.authorization.k8s.io/aggregate-to-admin: "true"
rules:
- apiGroups: ["metrics.k8s.io"]
resources: ["pods"]
verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: metrics-server:system:auth-delegator
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:auth-delegator
subjects:
- kind: ServiceAccount
name: metrics-server
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
name: metrics-server-auth-reader
namespace: kube-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: extension-apiserver-authentication-reader
subjects:
- kind: ServiceAccount
name: metrics-server
namespace: kube-system
---
apiVersion: apiregistration.k8s.io/v1beta1
kind: APIService
metadata:
name: v1beta1.metrics.k8s.io
spec:
service:
name: metrics-server
namespace: kube-system
group: metrics.k8s.io
version: v1beta1
insecureSkipTLSVerify: true
groupPriorityMinimum: 100
versionPriority: 100
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: metrics-server
namespace: kube-system
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: metrics-server
namespace: kube-system
labels:
k8s-app: metrics-server
spec:
selector:
matchLabels:
k8s-app: metrics-server
template:
metadata:
name: metrics-server
labels:
k8s-app: metrics-server
spec:
serviceAccountName: metrics-server
volumes:
# mount in tmp so we can safely use from-scratch images and/or read-only containers
- name: tmp-dir
emptyDir: {}
containers:
- name: metrics-server
image: k8s.gcr.io/metrics-server-amd64:v0.3.1
imagePullPolicy: Always
volumeMounts:
- name: tmp-dir
mountPath: /tmp
args:
- --kubelet-preferred-address-types=InternalIP
- --kubelet-insecure-tls
- --metric-resolution=5s
---
apiVersion: v1
kind: Service
metadata:
name: metrics-server
namespace: kube-system
labels:
kubernetes.io/name: "Metrics-server"
spec:
selector:
k8s-app: metrics-server
ports:
- port: 443
protocol: TCP
targetPort: 443
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: system:metrics-server
rules:
- apiGroups:
- ""
resources:
- pods
- nodes
- nodes/stats
verbs:
- get
- list
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: system:metrics-server
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:metrics-server
subjects:
- kind: ServiceAccount
name: metrics-server
namespace: kube-system

View File

@@ -5,7 +5,7 @@ metadata:
spec:
podSelector:
matchLabels:
app: testweb
run: testweb
ingress:
- from:
- podSelector:

View File

@@ -5,6 +5,6 @@ metadata:
spec:
podSelector:
matchLabels:
app: testweb
run: testweb
ingress: []

View File

@@ -16,7 +16,7 @@ metadata:
spec:
podSelector:
matchLabels:
app: webui
run: webui
ingress:
- from: []

View File

@@ -6,7 +6,7 @@ metadata:
creationTimestamp: null
generation: 1
labels:
app: socat
run: socat
name: socat
namespace: kube-system
selfLink: /apis/extensions/v1beta1/namespaces/kube-system/deployments/socat
@@ -14,7 +14,7 @@ spec:
replicas: 1
selector:
matchLabels:
app: socat
run: socat
strategy:
rollingUpdate:
maxSurge: 1
@@ -24,7 +24,7 @@ spec:
metadata:
creationTimestamp: null
labels:
app: socat
run: socat
spec:
containers:
- args:
@@ -49,7 +49,7 @@ kind: Service
metadata:
creationTimestamp: null
labels:
app: socat
run: socat
name: socat
namespace: kube-system
selfLink: /api/v1/namespaces/kube-system/services/socat
@@ -60,7 +60,7 @@ spec:
protocol: TCP
targetPort: 80
selector:
app: socat
run: socat
sessionAffinity: None
type: NodePort
status:

View File

@@ -32,7 +32,7 @@ Virtualbox, Vagrant and Ansible
$ source path/to/your-ansible-clone/hacking/env-setup
- you need to repeat the last step every time you open a new terminal session
- you need to repeat the last step everytime you open a new terminal session
and want to use any Ansible command (but you'll probably only need to run
it once).

View File

@@ -54,9 +54,6 @@ need_infra() {
if [ -z "$1" ]; then
die "Please specify infrastructure file. (e.g.: infra/aws)"
fi
if [ "$1" = "--infra" ]; then
die "The infrastructure file should be passed directly to this command. Remove '--infra' and try again."
fi
if [ ! -f "$1" ]; then
die "Infrastructure file $1 doesn't exist."
fi

View File

@@ -2,7 +2,7 @@ export AWS_DEFAULT_OUTPUT=text
HELP=""
_cmd() {
HELP="$(printf "%s\n%-20s %s\n" "$HELP" "$1" "$2")"
HELP="$(printf "%s\n%-12s %s\n" "$HELP" "$1" "$2")"
}
_cmd help "Show available commands"
@@ -74,10 +74,10 @@ _cmd_deploy() {
pssh -I sudo tee /usr/local/bin/docker-prompt <lib/docker-prompt
pssh sudo chmod +x /usr/local/bin/docker-prompt
# If /home/docker/.ssh/id_rsa doesn't exist, copy it from the first node
# If /home/docker/.ssh/id_rsa doesn't exist, copy it from node1
pssh "
sudo -u docker [ -f /home/docker/.ssh/id_rsa ] ||
ssh -o StrictHostKeyChecking=no \$(cat /etc/name_of_first_node) sudo -u docker tar -C /home/docker -cvf- .ssh |
ssh -o StrictHostKeyChecking=no node1 sudo -u docker tar -C /home/docker -cvf- .ssh |
sudo -u docker tar -C /home/docker -xf-"
# if 'docker@' doesn't appear in /home/docker/.ssh/authorized_keys, copy it there
@@ -86,11 +86,11 @@ _cmd_deploy() {
cat /home/docker/.ssh/id_rsa.pub |
sudo -u docker tee -a /home/docker/.ssh/authorized_keys"
# On the first node, create and deploy TLS certs using Docker Machine
# On node1, create and deploy TLS certs using Docker Machine
# (Currently disabled.)
true || pssh "
if i_am_first_node; then
grep '[0-9]\$' /etc/hosts |
if grep -q node1 /tmp/node; then
grep ' node' /etc/hosts |
xargs -n2 sudo -H -u docker \
docker-machine create -d generic --generic-ssh-user docker --generic-ip-address
fi"
@@ -103,62 +103,11 @@ _cmd_deploy() {
info "$0 cards $TAG"
}
_cmd disabledocker "Stop Docker Engine and don't restart it automatically"
_cmd_disabledocker() {
TAG=$1
need_tag
pssh "sudo systemctl disable docker.service"
pssh "sudo systemctl disable docker.socket"
pssh "sudo systemctl stop docker"
}
_cmd kubebins "Install Kubernetes and CNI binaries but don't start anything"
_cmd_kubebins() {
TAG=$1
need_tag
pssh --timeout 300 "
set -e
cd /usr/local/bin
if ! [ -x etcd ]; then
curl -L https://github.com/etcd-io/etcd/releases/download/v3.3.10/etcd-v3.3.10-linux-amd64.tar.gz \
| sudo tar --strip-components=1 --wildcards -zx '*/etcd' '*/etcdctl'
fi
if ! [ -x hyperkube ]; then
curl -L https://dl.k8s.io/v1.14.1/kubernetes-server-linux-amd64.tar.gz \
| sudo tar --strip-components=3 -zx kubernetes/server/bin/hyperkube
fi
if ! [ -x kubelet ]; then
for BINARY in kubectl kube-apiserver kube-scheduler kube-controller-manager kubelet kube-proxy;
do
sudo ln -s hyperkube \$BINARY
done
fi
sudo mkdir -p /opt/cni/bin
cd /opt/cni/bin
if ! [ -x bridge ]; then
curl -L https://github.com/containernetworking/plugins/releases/download/v0.7.5/cni-plugins-amd64-v0.7.5.tgz \
| sudo tar -zx
fi
"
}
_cmd kube "Setup kubernetes clusters with kubeadm (must be run AFTER deploy)"
_cmd_kube() {
TAG=$1
need_tag
# Optional version, e.g. 1.13.5
KUBEVERSION=$2
if [ "$KUBEVERSION" ]; then
EXTRA_KUBELET="=$KUBEVERSION-00"
EXTRA_KUBEADM="--kubernetes-version=v$KUBEVERSION"
else
EXTRA_KUBELET=""
EXTRA_KUBEADM=""
fi
# Install packages
pssh --timeout 200 "
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg |
@@ -167,19 +116,19 @@ _cmd_kube() {
sudo tee /etc/apt/sources.list.d/kubernetes.list"
pssh --timeout 200 "
sudo apt-get update -q &&
sudo apt-get install -qy kubelet$EXTRA_KUBELET kubeadm kubectl &&
sudo apt-get install -qy kubelet kubeadm kubectl &&
kubectl completion bash | sudo tee /etc/bash_completion.d/kubectl"
# Initialize kube master
pssh --timeout 200 "
if i_am_first_node && [ ! -f /etc/kubernetes/admin.conf ]; then
if grep -q node1 /tmp/node && [ ! -f /etc/kubernetes/admin.conf ]; then
kubeadm token generate > /tmp/token &&
sudo kubeadm init $EXTRA_KUBEADM --token \$(cat /tmp/token) --apiserver-cert-extra-sans \$(cat /tmp/ipv4)
sudo kubeadm init --token \$(cat /tmp/token)
fi"
# Put kubeconfig in ubuntu's and docker's accounts
pssh "
if i_am_first_node; then
if grep -q node1 /tmp/node; then
sudo mkdir -p \$HOME/.kube /home/docker/.kube &&
sudo cp /etc/kubernetes/admin.conf \$HOME/.kube/config &&
sudo cp /etc/kubernetes/admin.conf /home/docker/.kube/config &&
@@ -189,23 +138,16 @@ _cmd_kube() {
# Install weave as the pod network
pssh "
if i_am_first_node; then
if grep -q node1 /tmp/node; then
kubever=\$(kubectl version | base64 | tr -d '\n') &&
kubectl apply -f https://cloud.weave.works/k8s/net?k8s-version=\$kubever
fi"
# Join the other nodes to the cluster
pssh --timeout 200 "
if ! i_am_first_node && [ ! -f /etc/kubernetes/kubelet.conf ]; then
FIRSTNODE=\$(cat /etc/name_of_first_node) &&
TOKEN=\$(ssh -o StrictHostKeyChecking=no \$FIRSTNODE cat /tmp/token) &&
sudo kubeadm join --discovery-token-unsafe-skip-ca-verification --token \$TOKEN \$FIRSTNODE:6443
fi"
# Install metrics server
pssh "
if i_am_first_node; then
kubectl apply -f https://raw.githubusercontent.com/jpetazzo/container.training/master/k8s/metrics-server.yaml
if ! grep -q node1 /tmp/node && [ ! -f /etc/kubernetes/kubelet.conf ]; then
TOKEN=\$(ssh -o StrictHostKeyChecking=no node1 cat /tmp/token) &&
sudo kubeadm join --discovery-token-unsafe-skip-ca-verification --token \$TOKEN node1:6443
fi"
# Install kubectx and kubens
@@ -228,8 +170,7 @@ EOF"
# Install stern
pssh "
if [ ! -x /usr/local/bin/stern ]; then
##VERSION##
sudo curl -L -o /usr/local/bin/stern https://github.com/wercker/stern/releases/download/1.10.0/stern_linux_amd64 &&
sudo curl -L -o /usr/local/bin/stern https://github.com/wercker/stern/releases/download/1.8.0/stern_linux_amd64 &&
sudo chmod +x /usr/local/bin/stern &&
stern --completion bash | sudo tee /etc/bash_completion.d/stern
fi"
@@ -241,13 +182,6 @@ EOF"
helm completion bash | sudo tee /etc/bash_completion.d/helm
fi"
# Install ship
pssh "
if [ ! -x /usr/local/bin/ship ]; then
curl -L https://github.com/replicatedhq/ship/releases/download/v0.40.0/ship_0.40.0_linux_amd64.tar.gz |
sudo tar -C /usr/local/bin -zx ship
fi"
sep "Done"
}
@@ -268,9 +202,10 @@ _cmd_kubetest() {
# Feel free to make that better ♥
pssh "
set -e
if i_am_first_node; then
[ -f /tmp/node ]
if grep -q node1 /tmp/node; then
which kubectl
for NODE in \$(awk /[0-9]\$/\ {print\ \\\$2} /etc/hosts); do
for NODE in \$(awk /\ node/\ {print\ \\\$2} /etc/hosts); do
echo \$NODE ; kubectl get nodes | grep -w \$NODE | grep -w Ready
done
fi"
@@ -341,14 +276,6 @@ _cmd_opensg() {
infra_opensg
}
_cmd disableaddrchecks "Disable source/destination IP address checks"
_cmd_disableaddrchecks() {
TAG=$1
need_tag
infra_disableaddrchecks
}
_cmd pssh "Run an arbitrary command on all nodes"
_cmd_pssh() {
TAG=$1
@@ -394,7 +321,7 @@ _cmd_start() {
*) die "Unrecognized parameter: $1."
esac
done
if [ -z "$INFRA" ]; then
die "Please add --infra flag to specify which infrastructure file to use."
fi
@@ -405,8 +332,8 @@ _cmd_start() {
COUNT=$(awk '/^clustersize:/ {print $2}' $SETTINGS)
warning "No --count option was specified. Using value from settings file ($COUNT)."
fi
# Check that the specified settings and infrastructure are valid.
# Check that the specified settings and infrastructure are valid.
need_settings $SETTINGS
need_infra $INFRA
@@ -473,28 +400,6 @@ _cmd_test() {
test_tag
}
_cmd helmprom "Install Helm and Prometheus"
_cmd_helmprom() {
TAG=$1
need_tag
pssh "
if i_am_first_node; then
kubectl -n kube-system get serviceaccount helm ||
kubectl -n kube-system create serviceaccount helm
helm init --service-account helm
kubectl get clusterrolebinding helm-can-do-everything ||
kubectl create clusterrolebinding helm-can-do-everything \
--clusterrole=cluster-admin \
--serviceaccount=kube-system:helm
helm upgrade --install prometheus stable/prometheus \
--namespace kube-system \
--set server.service.type=NodePort \
--set server.service.nodePort=30090 \
--set server.persistentVolume.enabled=false \
--set alertmanager.enabled=false
fi"
}
# Sometimes, weave fails to come up on some nodes.
# Symptom: the pods on a node are unreachable (they don't even ping).
# Remedy: wipe out Weave state and delete weave pod on that node.
@@ -568,8 +473,8 @@ test_vm() {
for cmd in "hostname" \
"whoami" \
"hostname -i" \
"ls -l /usr/local/bin/i_am_first_node" \
"grep . /etc/name_of_first_node /etc/ipv4_of_first_node" \
"cat /tmp/node" \
"cat /tmp/ipv4" \
"cat /etc/hosts" \
"hostnamectl status" \
"docker version | grep Version -B1" \

View File

@@ -24,7 +24,3 @@ infra_quotas() {
infra_opensg() {
warning "infra_opensg is unsupported on $INFRACLASS."
}
infra_disableaddrchecks() {
warning "infra_disableaddrchecks is unsupported on $INFRACLASS."
}

View File

@@ -88,14 +88,6 @@ infra_opensg() {
--cidr 0.0.0.0/0
}
infra_disableaddrchecks() {
IDS=$(aws_get_instance_ids_by_tag $TAG)
for ID in $IDS; do
info "Disabling source/destination IP checks on: $ID"
aws ec2 modify-instance-attribute --source-dest-check "{\"Value\": false}" --instance-id $ID
done
}
wait_until_tag_is_running() {
max_retry=50
i=0
@@ -209,6 +201,5 @@ aws_tag_instances() {
}
aws_get_ami() {
##VERSION##
find_ubuntu_ami -r $AWS_DEFAULT_REGION -a amd64 -v 18.04 -t hvm:ebs -N -q
find_ubuntu_ami -r $AWS_DEFAULT_REGION -a amd64 -v 16.04 -t hvm:ebs -N -q
}

View File

@@ -12,7 +12,6 @@ config = yaml.load(open("/tmp/settings.yaml"))
COMPOSE_VERSION = config["compose_version"]
MACHINE_VERSION = config["machine_version"]
CLUSTER_SIZE = config["clustersize"]
CLUSTER_PREFIX = config["clusterprefix"]
ENGINE_VERSION = config["engine_version"]
DOCKER_USER_PASSWORD = config["docker_user_password"]
@@ -122,7 +121,7 @@ addresses = list(l.strip() for l in sys.stdin)
assert ipv4 in addresses
def makenames(addrs):
return [ "%s%s"%(CLUSTER_PREFIX, i+1) for i in range(len(addrs)) ]
return [ "node%s"%(i+1) for i in range(len(addrs)) ]
while addresses:
cluster = addresses[:CLUSTER_SIZE]
@@ -136,21 +135,15 @@ while addresses:
print(cluster)
mynode = cluster.index(ipv4) + 1
system("echo {}{} | sudo tee /etc/hostname".format(CLUSTER_PREFIX, mynode))
system("sudo hostname {}{}".format(CLUSTER_PREFIX, mynode))
system("echo node{} | sudo -u docker tee /tmp/node".format(mynode))
system("echo node{} | sudo tee /etc/hostname".format(mynode))
system("sudo hostname node{}".format(mynode))
system("sudo -u docker mkdir -p /home/docker/.ssh")
system("sudo -u docker touch /home/docker/.ssh/authorized_keys")
# Create a convenience file to easily check if we're the first node
if ipv4 == cluster[0]:
system("sudo ln -sf /bin/true /usr/local/bin/i_am_first_node")
# On the first node, if we don't have a private key, generate one (with empty passphrase)
# If I'm node1 and don't have a private key, generate one (with empty passphrase)
system("sudo -u docker [ -f /home/docker/.ssh/id_rsa ] || sudo -u docker ssh-keygen -t rsa -f /home/docker/.ssh/id_rsa -P ''")
else:
system("sudo ln -sf /bin/false /usr/local/bin/i_am_first_node")
# Record the IPV4 and name of the first node
system("echo {} | sudo tee /etc/ipv4_of_first_node".format(cluster[0]))
system("echo {} | sudo tee /etc/name_of_first_node".format(names[0]))
FINISH = time.time()
duration = "Initial deployment took {}s".format(str(FINISH - START)[:5])

View File

@@ -1,28 +0,0 @@
# Number of VMs per cluster
clustersize: 1
# The hostname of each node will be clusterprefix + a number
clusterprefix: dmuc
# Jinja2 template to use to generate ready-to-cut cards
cards_template: admin.html
# Use "Letter" in the US, and "A4" everywhere else
paper_size: A4
# Feel free to reduce this if your printer can handle it
paper_margin: 0.2in
# Note: paper_size and paper_margin only apply to PDF generated with pdfkit.
# If you print (or generate a PDF) using ips.html, they will be ignored.
# (The equivalent parameters must be set from the browser's print dialog.)
# This can be "test" or "stable"
engine_version: stable
# These correspond to the version numbers visible on their respective GitHub release pages
compose_version: 1.21.1
machine_version: 0.14.0
# Password used to connect with the "docker user"
docker_user_password: training

View File

@@ -1,28 +0,0 @@
# Number of VMs per cluster
clustersize: 3
# The hostname of each node will be clusterprefix + a number
clusterprefix: kubenet
# Jinja2 template to use to generate ready-to-cut cards
cards_template: admin.html
# Use "Letter" in the US, and "A4" everywhere else
paper_size: A4
# Feel free to reduce this if your printer can handle it
paper_margin: 0.2in
# Note: paper_size and paper_margin only apply to PDF generated with pdfkit.
# If you print (or generate a PDF) using ips.html, they will be ignored.
# (The equivalent parameters must be set from the browser's print dialog.)
# This can be "test" or "stable"
engine_version: stable
# These correspond to the version numbers visible on their respective GitHub release pages
compose_version: 1.21.1
machine_version: 0.14.0
# Password used to connect with the "docker user"
docker_user_password: training

View File

@@ -1,28 +0,0 @@
# Number of VMs per cluster
clustersize: 3
# The hostname of each node will be clusterprefix + a number
clusterprefix: kuberouter
# Jinja2 template to use to generate ready-to-cut cards
cards_template: admin.html
# Use "Letter" in the US, and "A4" everywhere else
paper_size: A4
# Feel free to reduce this if your printer can handle it
paper_margin: 0.2in
# Note: paper_size and paper_margin only apply to PDF generated with pdfkit.
# If you print (or generate a PDF) using ips.html, they will be ignored.
# (The equivalent parameters must be set from the browser's print dialog.)
# This can be "test" or "stable"
engine_version: stable
# These correspond to the version numbers visible on their respective GitHub release pages
compose_version: 1.21.1
machine_version: 0.14.0
# Password used to connect with the "docker user"
docker_user_password: training

View File

@@ -1,28 +0,0 @@
# Number of VMs per cluster
clustersize: 3
# The hostname of each node will be clusterprefix + a number
clusterprefix: test
# Jinja2 template to use to generate ready-to-cut cards
cards_template: admin.html
# Use "Letter" in the US, and "A4" everywhere else
paper_size: A4
# Feel free to reduce this if your printer can handle it
paper_margin: 0.2in
# Note: paper_size and paper_margin only apply to PDF generated with pdfkit.
# If you print (or generate a PDF) using ips.html, they will be ignored.
# (The equivalent parameters must be set from the browser's print dialog.)
# This can be "test" or "stable"
engine_version: stable
# These correspond to the version numbers visible on their respective GitHub release pages
compose_version: 1.21.1
machine_version: 0.14.0
# Password used to connect with the "docker user"
docker_user_password: training

View File

@@ -1,8 +1,5 @@
# Number of VMs per cluster
clustersize: 5
# The hostname of each node will be clusterprefix + a number
clusterprefix: node
# Jinja2 template to use to generate ready-to-cut cards
cards_template: clusters.csv

View File

@@ -1,8 +1,5 @@
# Number of VMs per cluster
clustersize: 1
# The hostname of each node will be clusterprefix + a number
clusterprefix: node
clustersize: 5
# Jinja2 template to use to generate ready-to-cut cards
cards_template: enix.html
@@ -21,9 +18,8 @@ paper_margin: 0.2in
engine_version: stable
# These correspond to the version numbers visible on their respective GitHub release pages
compose_version: 1.21.1
compose_version: 1.22.0
machine_version: 0.14.0
# Password used to connect with the "docker user"
docker_user_password: training

View File

@@ -3,9 +3,6 @@
# Number of VMs per cluster
clustersize: 5
# The hostname of each node will be clusterprefix + a number
clusterprefix: node
# Jinja2 template to use to generate ready-to-cut cards
cards_template: cards.html

View File

@@ -3,9 +3,6 @@
# Number of VMs per cluster
clustersize: 1
# The hostname of each node will be clusterprefix + a number
clusterprefix: node
# Jinja2 template to use to generate ready-to-cut cards
cards_template: cards.html

View File

@@ -1,11 +1,8 @@
# Number of VMs per cluster
clustersize: 4
# The hostname of each node will be clusterprefix + a number
clusterprefix: node
clustersize: 5
# Jinja2 template to use to generate ready-to-cut cards
cards_template: jerome.html
cards_template: cards.html
# Use "Letter" in the US, and "A4" everywhere else
paper_size: Letter
@@ -21,9 +18,8 @@ paper_margin: 0.2in
engine_version: stable
# These correspond to the version numbers visible on their respective GitHub release pages
compose_version: 1.21.1
compose_version: 1.21.2
machine_version: 0.14.0
# Password used to connect with the "docker user"
docker_user_password: training

View File

@@ -3,9 +3,6 @@
# Number of VMs per cluster
clustersize: 3
# The hostname of each node will be clusterprefix + a number
clusterprefix: node
# Jinja2 template to use to generate ready-to-cut cards
cards_template: kube101.html
@@ -27,5 +24,4 @@ compose_version: 1.21.1
machine_version: 0.14.0
# Password used to connect with the "docker user"
docker_user_password: training
docker_user_password: training

View File

@@ -3,9 +3,6 @@
# Number of VMs per cluster
clustersize: 3
# The hostname of each node will be clusterprefix + a number
clusterprefix: node
# Jinja2 template to use to generate ready-to-cut cards
cards_template: cards.html

View File

@@ -1,53 +0,0 @@
#!/bin/sh
set -e
INFRA=infra/aws-eu-west-3
STUDENTS=2
TAG=admin-dmuc
./workshopctl start \
--tag $TAG \
--infra $INFRA \
--settings settings/$TAG.yaml \
--count $STUDENTS
./workshopctl deploy $TAG
./workshopctl disabledocker $TAG
./workshopctl kubebins $TAG
./workshopctl cards $TAG
TAG=admin-kubenet
./workshopctl start \
--tag $TAG \
--infra $INFRA \
--settings settings/$TAG.yaml \
--count $((3*$STUDENTS))
./workshopctl deploy $TAG
./workshopctl kubebins $TAG
./workshopctl disableaddrchecks $TAG
./workshopctl cards $TAG
TAG=admin-kuberouter
./workshopctl start \
--tag $TAG \
--infra $INFRA \
--settings settings/$TAG.yaml \
--count $((3*$STUDENTS))
./workshopctl deploy $TAG
./workshopctl kubebins $TAG
./workshopctl disableaddrchecks $TAG
./workshopctl cards $TAG
TAG=admin-test
./workshopctl start \
--tag $TAG \
--infra $INFRA \
--settings settings/$TAG.yaml \
--count $((3*$STUDENTS))
./workshopctl deploy $TAG
./workshopctl kube $TAG 1.13.5
./workshopctl cards $TAG

View File

@@ -1,124 +0,0 @@
{# Feel free to customize or override anything in there! #}
{%- set url = "http://FIXME.container.training" -%}
{%- set pagesize = 9 -%}
{%- if clustersize == 1 -%}
{%- set workshop_name = "Docker workshop" -%}
{%- set cluster_or_machine = "machine virtuelle" -%}
{%- set this_or_each = "cette" -%}
{%- set plural = "" -%}
{%- set image_src = "https://s3-us-west-2.amazonaws.com/www.breadware.com/integrations/docker.png" -%}
{%- else -%}
{%- set workshop_name = "Kubernetes workshop" -%}
{%- set cluster_or_machine = "cluster" -%}
{%- set this_or_each = "chaque" -%}
{%- set plural = "s" -%}
{%- set image_src_swarm = "https://cdn.wp.nginx.com/wp-content/uploads/2016/07/docker-swarm-hero2.png" -%}
{%- set image_src_kube = "https://avatars1.githubusercontent.com/u/13629408" -%}
{%- set image_src = image_src_kube -%}
{%- endif -%}
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head><style>
@import url('https://fonts.googleapis.com/css?family=Slabo+27px');
body, table {
margin: 0;
padding: 0;
line-height: 1em;
font-size: 15px;
font-family: 'Slabo 27px';
}
table {
border-spacing: 0;
margin-top: 0.4em;
margin-bottom: 0.4em;
border-left: 0.8em double grey;
padding-left: 0.4em;
}
div {
float: left;
border: 1px dotted black;
padding-top: 1%;
padding-bottom: 1%;
/* columns * (width+left+right) < 100% */
width: 30%;
padding-left: 1.5%;
padding-right: 1.5%;
}
p {
margin: 0.4em 0 0.4em 0;
}
img {
height: 4em;
float: right;
margin-right: -0.3em;
}
img.enix {
height: 4.0em;
margin-top: 0.4em;
}
img.kube {
height: 4.2em;
margin-top: 1.7em;
}
.logpass {
font-family: monospace;
font-weight: bold;
}
.pagebreak {
page-break-after: always;
clear: both;
display: block;
height: 8px;
}
</style></head>
<body>
{% for cluster in clusters %}
{% if loop.index0>0 and loop.index0%pagesize==0 %}
<span class="pagebreak"></span>
{% endif %}
<div>
<p>
Voici les informations permettant de se connecter à un
des environnements utilisés pour cette formation.
Vous pouvez vous connecter à {{ this_or_each }} machine
virtuelle avec n'importe quel client SSH.
</p>
<p>
<img class="enix" src="https://enix.io/static/img/logos/logo-domain-cropped.png" />
<table>
<tr><td>cluster:</td></tr>
<tr><td class="logpass">{{ clusterprefix }}</td></tr>
<tr><td>identifiant:</td></tr>
<tr><td class="logpass">docker</td></tr>
<tr><td>mot de passe:</td></tr>
<tr><td class="logpass">{{ docker_user_password }}</td></tr>
</table>
</p>
<p>
Adresse{{ plural }} IP :
<!--<img class="kube" src="{{ image_src }}" />-->
<table>
{% for node in cluster %}
<tr><td>{{ clusterprefix }}{{ loop.index }}:</td><td>{{ node }}</td></tr>
{% endfor %}
</table>
</p>
<p>Le support de formation est à l'adresse suivante :
<center>{{ url }}</center>
</p>
</div>
{% endfor %}
</body>
</html>

View File

@@ -1,5 +1,5 @@
{# Feel free to customize or override anything in there! #}
{%- set url = "http://container.training/" -%}
{%- set url = "http://k8s2d.container.training/" -%}
{%- set pagesize = 12 -%}
{%- if clustersize == 1 -%}
{%- set workshop_name = "Docker workshop" -%}

View File

@@ -1,17 +1,17 @@
{# Feel free to customize or override anything in there! #}
{%- set url = "http://FIXME.container.training" -%}
{%- set url = "http://septembre2018.container.training" -%}
{%- set pagesize = 9 -%}
{%- if clustersize == 1 -%}
{%- set workshop_name = "Docker workshop" -%}
{%- set cluster_or_machine = "machine virtuelle" -%}
{%- set this_or_each = "cette" -%}
{%- set plural = "" -%}
{%- set cluster_or_machine = "machine" -%}
{%- set this_or_each = "this" -%}
{%- set machine_is_or_machines_are = "machine is" -%}
{%- set image_src = "https://s3-us-west-2.amazonaws.com/www.breadware.com/integrations/docker.png" -%}
{%- else -%}
{%- set workshop_name = "Kubernetes workshop" -%}
{%- set cluster_or_machine = "cluster" -%}
{%- set this_or_each = "chaque" -%}
{%- set plural = "s" -%}
{%- set this_or_each = "each" -%}
{%- set machine_is_or_machines_are = "machines are" -%}
{%- set image_src_swarm = "https://cdn.wp.nginx.com/wp-content/uploads/2016/07/docker-swarm-hero2.png" -%}
{%- set image_src_kube = "https://avatars1.githubusercontent.com/u/13629408" -%}
{%- set image_src = image_src_kube -%}
@@ -19,14 +19,11 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head><style>
@import url('https://fonts.googleapis.com/css?family=Slabo+27px');
body, table {
margin: 0;
padding: 0;
line-height: 1em;
font-size: 15px;
font-family: 'Slabo 27px';
font-size: 14px;
}
table {
@@ -59,8 +56,8 @@ img {
}
img.enix {
height: 4.0em;
margin-top: 0.4em;
height: 4.5em;
margin-top: 0.2em;
}
img.kube {
@@ -89,9 +86,8 @@ img.kube {
<p>
Voici les informations permettant de se connecter à votre
{{ cluster_or_machine }} pour cette formation.
Vous pouvez vous connecter à {{ this_or_each }} machine virtuelle
avec n'importe quel client SSH.
cluster pour cette formation. Vous pouvez vous connecter
à ces machines virtuelles avec n'importe quel client SSH.
</p>
<p>
<img class="enix" src="https://enix.io/static/img/logos/logo-domain-cropped.png" />
@@ -104,8 +100,8 @@ img.kube {
</p>
<p>
Adresse{{ plural }} IP :
<!--<img class="kube" src="{{ image_src }}" />-->
Vos serveurs sont :
<img class="kube" src="{{ image_src }}" />
<table>
{% for node in cluster %}
<tr><td>node{{ loop.index }}:</td><td>{{ node }}</td></tr>

View File

@@ -1,134 +0,0 @@
{# Feel free to customize or override anything in there! #}
{%- set url = "http://qconuk2019.container.training/" -%}
{%- set pagesize = 9 -%}
{%- if clustersize == 1 -%}
{%- set workshop_name = "Docker workshop" -%}
{%- set cluster_or_machine = "machine" -%}
{%- set this_or_each = "this" -%}
{%- set machine_is_or_machines_are = "machine is" -%}
{%- set image_src = "https://s3-us-west-2.amazonaws.com/www.breadware.com/integrations/docker.png" -%}
{%- else -%}
{%- set workshop_name = "Kubernetes workshop" -%}
{%- set cluster_or_machine = "cluster" -%}
{%- set this_or_each = "each" -%}
{%- set machine_is_or_machines_are = "machines are" -%}
{%- set image_src_swarm = "https://cdn.wp.nginx.com/wp-content/uploads/2016/07/docker-swarm-hero2.png" -%}
{%- set image_src_kube = "https://avatars1.githubusercontent.com/u/13629408" -%}
{%- set image_src = image_src_kube -%}
{%- endif -%}
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head><style>
@import url('https://fonts.googleapis.com/css?family=Slabo+27px');
body, table {
margin: 0;
padding: 0;
line-height: 1.0em;
font-size: 15px;
font-family: 'Slabo 27px';
}
table {
border-spacing: 0;
margin-top: 0.4em;
margin-bottom: 0.4em;
border-left: 0.8em double grey;
padding-left: 0.4em;
}
div {
float: left;
border: 1px dotted black;
height: 31%;
padding-top: 1%;
padding-bottom: 1%;
/* columns * (width+left+right) < 100% */
width: 30%;
padding-left: 1.5%;
padding-right: 1.5%;
}
div.back {
border: 1px dotted white;
}
div.back p {
margin: 0.5em 1em 0 1em;
}
p {
margin: 0.4em 0 0.8em 0;
}
img {
height: 5em;
float: right;
margin-right: 1em;
}
.logpass {
font-family: monospace;
font-weight: bold;
}
.pagebreak {
page-break-after: always;
clear: both;
display: block;
height: 8px;
}
</style></head>
<body>
{% for cluster in clusters %}
<div>
<p>
Here is the connection information to your very own
{{ cluster_or_machine }} for this {{ workshop_name }}.
You can connect to {{ this_or_each }} VM with any SSH client.
</p>
<p>
<img src="{{ image_src }}" />
<table>
<tr><td>login:</td></tr>
<tr><td class="logpass">docker</td></tr>
<tr><td>password:</td></tr>
<tr><td class="logpass">{{ docker_user_password }}</td></tr>
</table>
</p>
<p>
Your {{ machine_is_or_machines_are }}:
<table>
{% for node in cluster %}
<tr><td>node{{ loop.index }}:</td><td>{{ node }}</td></tr>
{% endfor %}
</table>
</p>
<p>You can find the slides at:
<center>{{ url }}</center>
</p>
</div>
{% if loop.index%pagesize==0 or loop.last %}
<span class="pagebreak"></span>
{% for x in range(pagesize) %}
<div class="back">
<br/>
<p>You got this at the workshop
"Getting Started With Kubernetes and Container Orchestration"
during QCON London (March 2019).</p>
<p>If you liked that workshop,
I can train your team or organization
on Docker, container, and Kubernetes,
with curriculums of 1 to 5 days.
</p>
<p>Interested? Contact me at:</p>
<p>jerome.petazzoni@gmail.com</p>
<p>Thank you!</p>
</div>
{% endfor %}
<span class="pagebreak"></span>
{% endif %}
{% endfor %}
</body>
</html>

View File

@@ -1,7 +1,7 @@
resource "openstack_compute_instance_v2" "machine" {
count = "${var.count}"
name = "${format("%s-%04d", "${var.prefix}", count.index+1)}"
image_name = "Ubuntu 16.04.5 (Xenial Xerus)"
image_name = "Ubuntu 16.04 (Xenial Xerus)"
flavor_name = "${var.flavor}"
security_groups = ["${openstack_networking_secgroup_v2.full_access.name}"]
key_pair = "${openstack_compute_keypair_v2.ssh_deploy_key.name}"

View File

@@ -1,7 +0,0 @@
FROM alpine
RUN apk update
RUN apk add entr
RUN apk add py-pip
RUN apk add git
COPY requirements.txt .
RUN pip install -r requirements.txt

View File

@@ -34,14 +34,6 @@ compile each `foo.yml` file into `foo.yml.html`.
You can also run `./build.sh forever`: it will monitor the current
directory and rebuild slides automatically when files are modified.
If you have problems running `./build.sh` (because of
Python dependencies or whatever),
you can also run `docker-compose up` in this directory.
It will start the `./build.sh forever` script in a container.
It will also start a web server exposing the slides
(but the slides should also work if you load them from your
local filesystem).
## Publishing pipeline
@@ -61,4 +53,4 @@ You can run `./slidechecker foo.yml.html` to check for
missing images and show the number of slides in that deck.
It requires `phantomjs` to be installed. It takes some
time to run so it is not yet integrated with the publishing
pipeline.
pipeline.

View File

@@ -1,4 +1,2 @@
# Uncomment and/or edit one of the the following lines if necessary.
#/ /kube-halfday.yml.html 200
/ /kube-fullday.yml.html 200!
#/ /kube-twodays.yml.html 200

View File

@@ -1,6 +1,3 @@
class: title
# Advanced Dockerfiles
![construction](images/title-advanced-dockerfiles.jpg)

View File

@@ -156,36 +156,6 @@ Different deployments will use different underlying technologies.
---
## Service meshes
* A service mesh is a configurable network layer.
* It can provide service discovery, high availability, load balancing, observability...
* Service meshes are particularly useful for microservices applications.
* Service meshes are often implemented as proxies.
* Applications connect to the service mesh, which relays the connection where needed.
*Does that sound familiar?*
---
## Ambassadors and service meshes
* When using a service mesh, a "sidecar container" is often used as a proxy
* Our services connect (transparently) to that sidecar container
* That sidecar container figures out where to forward the traffic
... Does that sound familiar?
(It should, because service meshes are essentially app-wide or cluster-wide ambassadors!)
---
## Section summary
We've learned how to:
@@ -198,10 +168,3 @@ For more information about the ambassador pattern, including demos on Swarm and
* [SwarmWeek video about Swarm+Compose](https://youtube.com/watch?v=qbIvUvwa6As)
Some services meshes and related projects:
* [Istio](https://istio.io/)
* [Linkerd](https://linkerd.io/)
* [Gloo](https://gloo.solo.io/)

View File

@@ -36,7 +36,7 @@ docker run jpetazzo/hamba 80 www1:80 www2:80
* Appropriate for mandatory parameters (without which the service cannot start).
* Convenient for "toolbelt" services instantiated many times.
* Convenient for "toolbelt" services instanciated many times.
(Because there is no extra step: just run it!)
@@ -63,7 +63,7 @@ docker run -e ELASTICSEARCH_URL=http://es42:9201/ kibana
* Appropriate for optional parameters (since the image can provide default values).
* Also convenient for services instantiated many times.
* Also convenient for services instanciated many times.
(It's as easy as command-line parameters.)

View File

@@ -144,10 +144,6 @@ At a first glance, it looks like this would be particularly useful in scripts.
However, if we want to start a container and get its ID in a reliable way,
it is better to use `docker run -d`, which we will cover in a bit.
(Using `docker ps -lq` is prone to race conditions: what happens if someone
else, or another program or script, starts another container just before
we run `docker ps -lq`?)
---
## View the logs of a container

View File

@@ -131,12 +131,6 @@ Sending build context to Docker daemon 2.048 kB
* Be careful (or patient) if that directory is big and your link is slow.
* You can speed up the process with a [`.dockerignore`](https://docs.docker.com/engine/reference/builder/#dockerignore-file) file
* It tells docker to ignore specific files in the directory
* Only ignore files that you won't need in the build context!
---
## Executing each step

View File

@@ -78,7 +78,7 @@ First step: clone the source code for the app we will be working on.
```bash
$ cd
$ git clone https://github.com/jpetazzo/trainingwheels
$ git clone git://github.com/jpetazzo/trainingwheels
...
$ cd trainingwheels
```

View File

@@ -67,8 +67,7 @@ The following list is not exhaustive.
Furthermore, we limited the scope to Linux containers.
We can also find containers (or things that look like containers) on other platforms
like Windows, macOS, Solaris, FreeBSD ...
Containers also exist (sometimes with other names) on Windows, macOS, Solaris, FreeBSD ...
---
@@ -156,36 +155,6 @@ We're not aware of anyone using it directly (i.e. outside of Kubernetes).
---
## Kata containers
* OCI-compliant runtime.
* Fusion of two projects: Intel Clear Containers and Hyper runV.
* Run each container in a lightweight virtual machine.
* Requires to run on bare metal *or* with nested virtualization.
---
## gVisor
* OCI-compliant runtime.
* Implements a subset of the Linux kernel system calls.
* Written in go, uses a smaller subset of system calls.
* Can be heavily sandboxed.
* Can run in two modes:
* KVM (requires bare metal or nested virtualization),
* ptrace (no requirement, but slower).
---
## Overall ...
* The Docker Engine is very developer-centric:
@@ -205,3 +174,4 @@ We're not aware of anyone using it directly (i.e. outside of Kubernetes).
- Docker is a good default choice
- If you use Kubernetes, the engine doesn't matter

View File

@@ -528,7 +528,7 @@ Very short instructions:
- `docker network create mynet --driver overlay`
- `docker service create --network mynet myimage`
See https://jpetazzo.github.io/container.training for all the deets about clustering!
See http://jpetazzo.github.io/container.training for all the deets about clustering!
---
@@ -721,20 +721,3 @@ eth0 Link encap:Ethernet HWaddr 02:42:AC:15:00:03
...
```
]
---
class: extra-details
## Building with a custom network
* We can build a Dockerfile with a custom network with `docker build --network NAME`.
* This can be used to check that a build doesn't access the network.
(But keep in mind that most Dockerfiles will fail,
<br/>because they need to install remote packages and dependencies!)
* This may be used to access an internal package repository.
(But try to use a multi-stage build instead, if possible!)

View File

@@ -169,5 +169,5 @@ Would we give the same answers to the questions on the previous slide?
class: pic
![Cloud Native Landscape](https://landscape.cncf.io/images/landscape.png)
![Cloud Native Landscape](https://raw.githubusercontent.com/cncf/landscape/master/landscape/CloudNativeLandscape_latest.png)

View File

@@ -1,4 +1,3 @@
class: title
# Getting inside a container

View File

@@ -66,6 +66,14 @@ class: pic
---
class: pic
## Multiple containers sharing the same image
![layers](images/sharing-layers.jpg)
---
## Differences between containers and images
* An image is a read-only filesystem.
@@ -80,14 +88,6 @@ class: pic
---
class: pic
## Multiple containers sharing the same image
![layers](images/sharing-layers.jpg)
---
## Comparison with object-oriented programming
* Images are conceptually similar to *classes*.
@@ -118,7 +118,7 @@ If an image is read-only, how do we change it?
* The only way to create an image is by "freezing" a container.
* The only way to create a container is by instantiating an image.
* The only way to create a container is by instanciating an image.
* Help!
@@ -216,7 +216,7 @@ clock
---
## Self-hosted namespace
## Self-Hosted namespace
This namespace holds images which are not hosted on Docker Hub, but on third
party registries.
@@ -233,13 +233,6 @@ localhost:5000/wordpress
* `localhost:5000` is the host and port of the registry
* `wordpress` is the name of the image
Other examples:
```bash
quay.io/coreos/etcd
gcr.io/google-containers/hugo
```
---
## How do you store and manage images?
@@ -359,8 +352,6 @@ Do specify tags:
* To ensure that the same version will be used everywhere.
* To ensure repeatability later.
This is similar to what we would do with `pip install`, `npm install`, etc.
---
## Section summary

View File

@@ -1,4 +1,3 @@
class: title
# Installing Docker
@@ -38,7 +37,11 @@ We can arbitrarily distinguish:
## Installing Docker on Linux
* The recommended method is to install the packages supplied by Docker Inc :
* The recommended method is to install the packages supplied by Docker Inc.:
https://store.docker.com
* The general method is:
- add Docker Inc.'s package repositories to your system configuration
@@ -52,12 +55,6 @@ We can arbitrarily distinguish:
https://docs.docker.com/engine/installation/linux/docker-ce/binaries/
* To quickly setup a dev environment, Docker provides a convenience install script:
```bash
curl -fsSL get.docker.com | sh
```
---
class: extra-details
@@ -84,11 +81,11 @@ class: extra-details
## Installing Docker on macOS and Windows
* On macOS, the recommended method is to use Docker Desktop for Mac:
* On macOS, the recommended method is to use Docker for Mac:
https://docs.docker.com/docker-for-mac/install/
* On Windows 10 Pro, Enterprise, and Education, you can use Docker Desktop for Windows:
* On Windows 10 Pro, Enterprise, and Education, you can use Docker for Windows:
https://docs.docker.com/docker-for-windows/install/
@@ -102,7 +99,7 @@ class: extra-details
---
## Docker Desktop for Mac and Docker Desktop for Windows
## Docker for Mac and Docker for Windows
* Special Docker Editions that integrate well with their respective host OS

View File

@@ -194,13 +194,9 @@ will have equal success with Fluent or other logging stacks!*
- We are going to use a Compose file describing the ELK stack.
- The Compose file is in the container.training repository on GitHub.
```bash
$ git clone https://github.com/jpetazzo/container.training
$ cd container.training
$ cd elk
$ docker-compose up
$ cd ~/container.training/stacks
$ docker-compose -f elk.yml up -d
```
- Let's have a look at the Compose file while it's deploying.
@@ -295,4 +291,4 @@ that you don't drop messages on the floor. Good luck.
If you want to learn more about the GELF driver,
have a look at [this blog post](
https://jpetazzo.github.io/2017/01/20/docker-logging-gelf/).
http://jpetazzo.github.io/2017/01/20/docker-logging-gelf/).

View File

@@ -293,23 +293,3 @@ We can achieve even smaller images if we use smaller base images.
However, if we use common base images (e.g. if we standardize on `ubuntu`),
these common images will be pulled only once per node, so they are
virtually "free."
---
## Build targets
* We can also tag an intermediary stage with `docker build --target STAGE --tag NAME`
* This will create an image (named `NAME`) corresponding to stage `STAGE`
* This can be used to easily access an intermediary stage for inspection
(Instead of parsing the output of `docker build` to find out the image ID)
* This can also be used to describe multiple images from a single Dockerfile
(Instead of using multiple Dockerfiles, which could go out of sync)
* Sometimes, we want to inspect a specific intermediary build stage.
* Or, we want to describe multiple images using a single Dockerfile.

View File

@@ -155,7 +155,7 @@ processes or data flows are given access to system resources.*
The scheduler is concerned mainly with:
- throughput (total amount of work done per time unit);
- throughput (total amount or work done per time unit);
- turnaround time (between submission and completion);
- response time (between submission and start);
- waiting time (between job readiness and execution);
@@ -243,76 +243,58 @@ Scheduling = deciding which hypervisor to use for each VM.
---
class: pic
## Scheduling with one resource
.center[![Not-so-good bin packing](images/binpacking-1d-1.gif)]
## We can't fit a job of size 6 :(
Can we do better?
---
class: pic
## Scheduling with one resource
.center[![Better bin packing](images/binpacking-1d-2.gif)]
## ... Now we can!
Yup!
---
class: pic
## Scheduling with two resources
.center[![2D bin packing](images/binpacking-2d.gif)]
---
class: pic
## Scheduling with three resources
.center[![3D bin packing](images/binpacking-3d.gif)]
---
class: pic
## You need to be good at this
.center[![Tangram](images/tangram.gif)]
---
class: pic
## But also, you must be quick!
.center[![Tetris](images/tetris-1.png)]
---
class: pic
## And be web scale!
.center[![Big tetris](images/tetris-2.gif)]
---
class: pic
## And think outside (?) of the box!
.center[![3D tetris](images/tetris-3.png)]
---
class: pic
## Good luck!
.center[![FUUUUUU face](images/fu-face.jpg)]
@@ -390,7 +372,7 @@ It depends on:
(Marathon = long running processes; Chronos = run at intervals; ...)
- Commercial offering through DC/OS by Mesosphere.
- Commercial offering through DC/OS my Mesosphere.
---

View File

@@ -91,12 +91,12 @@ class: extra-details
* We need a Dockerized repository!
* Let's go to https://github.com/jpetazzo/trainingwheels and fork it.
* Go to the Docker Hub (https://hub.docker.com/) and sign-in. Select "Repositories" in the blue navigation menu.
* Select "Create" in the top-right bar, and select "Create Repository+".
* Go to the Docker Hub (https://hub.docker.com/).
* Select "Create" in the top-right bar, and select "Create Automated Build."
* Connect your Docker Hub account to your GitHub account.
* Click "Create" button.
* Then go to "Builds" folder.
* Click on Github icon and select your user and the repository that we just forked.
* In "Build rules" block near page bottom, put `/www` in "Build Context" column (or whichever directory the Dockerfile is in).
* Click "Save and Build" to build the repository immediately (without waiting for a git push).
* Select your user and the repository that we just forked.
* Create.
* Then go to "Build Settings."
* Put `/www` in "Dockerfile Location" (or whichever directory the Dockerfile is in).
* Click "Trigger" to build the repository immediately (without waiting for a git push).
* Subsequent builds will happen automatically, thanks to GitHub hooks.

View File

@@ -1,4 +1,3 @@
class: title
# Our training environment
@@ -19,7 +18,7 @@ class: title
- install Docker on e.g. a cloud VM
- use https://www.play-with-docker.com/ to instantly get a training environment
- use http://www.play-with-docker.com/ to instantly get a training environment
---
@@ -91,7 +90,7 @@ $ ssh <login>@<ip-address>
* Git BASH (https://git-for-windows.github.io/)
* MobaXterm (https://mobaxterm.mobatek.net/)
* MobaXterm (http://moabaxterm.mobatek.net)
---

View File

@@ -1,164 +0,0 @@
class: title
# Windows Containers
![Container with Windows](images/windows-containers.jpg)
---
## Objectives
At the end of this section, you will be able to:
* Understand Windows Container vs. Linux Container.
* Know about the features of Docker for Windows for choosing architecture.
* Run other container architectures via QEMU emulation.
---
## Are containers *just* for Linux?
Remember that a container must run on the kernel of the OS it's on.
- This is both a benefit and a limitation.
(It makes containers lightweight, but limits them to a specific kernel.)
- At its launch in 2013, Docker did only support Linux, and only on amd64 CPUs.
- Since then, many platforms and OS have been added.
(Windows, ARM, i386, IBM mainframes ... But no macOS or iOS yet!)
--
- Docker Desktop (macOS and Windows) can run containers for other architectures
(Check the docs to see how to [run a Raspberry Pi (ARM) or PPC container](https://docs.docker.com/docker-for-mac/multi-arch/)!)
---
## History of Windows containers
- Early 2016, Windows 10 gained support for running Windows binaries in containers.
- These are known as "Windows Containers"
- Win 10 expects Docker for Windows to be installed for full features
- These must run in Hyper-V mini-VM's with a Windows Server x64 kernel
- No "scratch" containers, so use "Core" and "Nano" Server OS base layers
- Since Hyper-V is required, Windows 10 Home won't work (yet...)
--
- Late 2016, Windows Server 2016 ships with native Docker support
- Installed via PowerShell, doesn't need Docker for Windows
- Can run native (without VM), or with [Hyper-V Isolation](https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-containers/hyperv-container)
---
## LCOW (Linux Containers On Windows)
While Docker on Windows is largely playing catch up with Docker on Linux,
it's moving fast; and this is one thing that you *cannot* do on Linux!
- LCOW came with the [2017 Fall Creators Update](https://blog.docker.com/2018/02/docker-for-windows-18-02-with-windows-10-fall-creators-update/).
- It can run Linux and Windows containers side-by-side on Win 10.
- It is no longer necessary to switch the Engine to "Linux Containers".
(In fact, if you want to run both Linux and Windows containers at the same time,
make sure that your Engine is set to "Windows Containers" mode!)
--
If you are a Docker for Windows user, start your engine and try this:
```bash
docker pull microsoft/nanoserver:1803
```
(Make sure to switch to "Windows Containers mode" if necessary.)
---
## Run Both Windows and Linux containers
- Run a Windows Nano Server (minimal CLI-only server)
```bash
docker run --rm -it microsoft/nanoserver:1803 powershell
Get-Process
exit
```
- Run busybox on Linux in LCOW
```bash
docker run --rm --platform linux busybox echo hello
```
(Although you will not be able to see them, this will create hidden
Nano and LinuxKit VMs in Hyper-V!)
---
## Did We Say Things Move Fast
- Things keep improving.
- Now `--platform` defaults to `windows`, some images support both:
- golang, mongo, python, redis, hello-world ... and more being added
- you should still use `--plaform` with multi-os images to be certain
- Windows Containers now support `localhost` accessible containers (July 2018)
- Microsoft (April 2018) added Hyper-V support to Windows 10 Home ...
... so stay tuned for Docker support, maybe?!?
---
## Other Windows container options
Most "official" Docker images don't run on Windows yet.
Places to Look:
- Hub Official: https://hub.docker.com/u/winamd64/
- Microsoft: https://hub.docker.com/r/microsoft/
---
## SQL Server? Choice of Linux or Windows
- Microsoft [SQL Server for Linux 2017](https://hub.docker.com/r/microsoft/mssql-server-linux/) (amd64/linux)
- Microsoft [SQL Server Express 2017](https://hub.docker.com/r/microsoft/mssql-server-windows-express/) (amd64/windows)
---
## Windows Tools and Tips
- PowerShell [Tab Completion: DockerCompletion](https://github.com/matt9ucci/DockerCompletion)
- Best Shell GUI: [Cmder.net](https://cmder.net/)
- Good Windows Container Blogs and How-To's
- Docker DevRel [Elton Stoneman, Microsoft MVP](https://blog.sixeyed.com/)
- Docker Captain [Nicholas Dille](https://dille.name/blog/)
- Docker Captain [Stefan Scherer](https://stefanscherer.github.io/)

View File

@@ -401,7 +401,7 @@ or providing extra features. For instance:
* [REX-Ray](https://rexray.io/) - create and manage volumes backed by an enterprise storage system (e.g.
SAN or NAS), or by cloud block stores (e.g. EBS, EFS).
* [Portworx](https://portworx.com/) - provides distributed block store for containers.
* [Portworx](http://portworx.com/) - provides distributed block store for containers.
* [Gluster](https://www.gluster.org/) - open source software-defined distributed storage that can scale
to several petabytes. It provides interfaces for object, block and file storage.

View File

@@ -30,7 +30,7 @@ class: self-paced
- These slides include *tons* of exercises and examples
- They assume that you have access to a machine running Docker
- They assume that you have acccess to a machine running Docker
- If you are attending a workshop or tutorial:
<br/>you will be given specific instructions to access a cloud VM

View File

@@ -1,16 +0,0 @@
version: "2"
services:
www:
image: nginx
volumes:
- .:/usr/share/nginx/html
ports:
- 80
builder:
build: .
volumes:
- ..:/repo
working_dir: /repo/slides
command: ./build.sh forever

Binary file not shown.

Before

Width:  |  Height:  |  Size: 203 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 85 KiB

File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 14 KiB

View File

@@ -1 +0,0 @@
<mxfile userAgent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36" version="9.3.0" editor="www.draw.io" type="device"><diagram id="cb13f823-9e55-f92e-d17e-d0d789fca2e0" name="Page-1">7Vnfb9sgEP5rLG0vlQ3+kTyuabs9bFq1Vtr2iO2LjUqMhfGS9q8fxNgxJtXSqmmmrVEeuAMOuO874LCHF6vNR0Hq8gvPgXnIzzcevvAQCkKEPP338/tOk0RxpygEzU2jneKGPoBR+kbb0hwaq6HknEla28qMVxVk0tIRIfjabrbkzB61JgU4ipuMMFf7neay7LSzyN/pPwEtyn7kwDc1KcnuCsHbyoznIbzc/rrqFeltmfZNSXK+HqnwpYcXgnPZlVabBTDt295tXb+rR2qHeQuo5EEdDC6/CGuhn/J2YvK+d8Z2OaA7+B4+X5dUwk1NMl27VvArXSlXTEmBKi4pYwvOuNj2xTmB2TJT+kYKfgejmjibQbpUNQUjTWOMZ3xFM1MeXKOFJa/kFVlRpgn1jadccj0uF/RB1ZBhdCUYNqFQyWZxICRsHvVQMPhd8Rn4CqS4V01Mhx4pw+RgZuT1jhdJrytHnMC9khguFoPpHR6qYCDZDw920FlzcQfCwUg5q9bFrE3hzyClHaKf00Ex0PZrKxmtYOTPZ7h9QgJFf5TtJUEep7HaGvqaPtbwS0FnY4cSF7sA7cHuJaALHehEVbzhdghusQ1bGL4ibJEDW0ma8i3iDow4dELo3KNMQE6bN+QOQW44rk6BXOIec5C29A25g3ZLfELk+gv7CLqPl7cOcFDlH/S1XGOnr3v6kmfdGp+MQDBz/KkxYSQFdj7gPALh6mqhfoPLIXcygInD1QJ4KzKwbmKSiALk6IR3YRm5Pdrj9V4ngBFJf9mT2AeFGeGaUzW9AfVgutPO57aJbvKm1zgDmBpKpvSZGOqW7BjaMmNY9mFkCRyyXH+9+U/YEp2SLWge2SDHk9g/lC04nBgKJoZekC3IYcvt4vr/IEt8UrIk4VniR7MZwhGahz0OBnH8XOqgWXSmzQVJHG7N20SKjkckN4v+N4mU/GVEwoF9tDybOuEkkT8mWdy8vW32pH+qF60b6N6puksp421+wAPZyV6ykokX4+g1L4puYn3SiyJsqPyhyv5ZL/3cSoGRrkFQtUjQkekfM2h7wo2jNjll1E5eX5LnBm0wif7kiFcFN4G84NkdiIUy3ugh65rRTHmFVx6KmdTJoIrpuNCld0vtrO3HBElUViia924jh6kqDqXNTTvvq7jOL60k0agIo0WlRAZLbUHHtJob+2DUkteP7CL2Q7z1Pv7YI/oRtpFJuhnM3V0kjvfQM8BP30aUuPsW0hFj98EJX/4G</diagram></mxfile>

Binary file not shown.

Before

Width:  |  Height:  |  Size: 426 KiB

View File

@@ -4,30 +4,11 @@ TEMPLATE="""<html>
<head>
<title>{{ title }}</title>
<link rel="stylesheet" href="index.css">
<meta charset="UTF-8">
</head>
<body>
<div class="main">
<table>
<tr><td class="header" colspan="3">{{ title }}</td></tr>
<tr><td class="details" colspan="3">Note: while some workshops are delivered in French, slides are always in English.</td></tr>
<tr><td class="title" colspan="3">Free video of our latest workshop</td></tr>
<tr>
<td>Getting Started With Kubernetes and Container Orchestration</td>
<td><a class="slides" href="https://qconuk2019.container.training" /></td>
<td><a class="video" href="https://www.youtube.com/playlist?list=PLBAFXs0YjviJwCoxSUkUPhsSxDJzpZbJd" /></td>
</tr>
<tr>
<td class="details">This is a live recording of a 1-day workshop that took place at QCON London in March 2019.</td>
</tr>
<tr>
<td class="details">If you're interested, we can deliver that workshop (or longer courses) to your team or organization.</td>
</tr>
<tr>
<td class="details">Contact <a href="mailto:jerome.petazzoni@gmail.com">Jérôme Petazzoni</a> to make that happen!</a></td>
</tr>
{% if coming_soon %}
<tr><td class="title" colspan="3">Coming soon near you</td></tr>
@@ -36,10 +17,7 @@ TEMPLATE="""<html>
<tr>
<td>{{ item.title }}</td>
<td>{% if item.slides %}<a class="slides" href="{{ item.slides }}" />{% endif %}</td>
<td>{% if item.attend %}<a class="attend" href="{{ item.attend }}" />
{% else %}
<p class="details">{{ item.status }}</p>
{% endif %}</td>
<td><a class="attend" href="{{ item.attend }}" /></td>
</tr>
<tr>
<td class="details">Scheduled {{ item.prettydate }} at {{ item.event }} in {{item.city }}.</td>
@@ -53,10 +31,7 @@ TEMPLATE="""<html>
{% for item in past_workshops[:5] %}
<tr>
<td>{{ item.title }}</td>
<td>{% if item.slides %}<a class="slides" href="{{ item.slides }}" />
{% else %}
<p class="details">{{ item.status }}</p>
{% endif %}</td>
<td><a class="slides" href="{{ item.slides }}" /></td>
<td>{% if item.video %}<a class="video" href="{{ item.video }}" />{% endif %}</td>
</tr>
<tr>
@@ -131,41 +106,26 @@ import yaml
items = yaml.load(open("index.yaml"))
# Items with a date correspond to scheduled sessions.
# Items without a date correspond to self-paced content.
# The date should be specified as a string (e.g. 2018-11-26).
# It can also be a list of two elements (e.g. [2018-11-26, 2018-11-28]).
# The latter indicates an event spanning multiple dates.
# The first date will be used in the generated page, but the event
# will be considered "current" (and therefore, shown in the list of
# upcoming events) until the second date.
for item in items:
if "date" in item:
date = item["date"]
if type(date) == list:
date_begin, date_end = date
else:
date_begin, date_end = date, date
suffix = {
1: "st", 2: "nd", 3: "rd",
21: "st", 22: "nd", 23: "rd",
31: "st"}.get(date_begin.day, "th")
31: "st"}.get(date.day, "th")
# %e is a non-standard extension (it displays the day, but without a
# leading zero). If strftime fails with ValueError, try to fall back
# on %d (which displays the day but with a leading zero when needed).
try:
item["prettydate"] = date_begin.strftime("%B %e{}, %Y").format(suffix)
item["prettydate"] = date.strftime("%B %e{}, %Y").format(suffix)
except ValueError:
item["prettydate"] = date_begin.strftime("%B %d{}, %Y").format(suffix)
item["begin"] = date_begin
item["end"] = date_end
item["prettydate"] = date.strftime("%B %d{}, %Y").format(suffix)
today = datetime.date.today()
coming_soon = [i for i in items if i.get("date") and i["end"] >= today]
coming_soon.sort(key=lambda i: i["begin"])
past_workshops = [i for i in items if i.get("date") and i["end"] < today]
past_workshops.sort(key=lambda i: i["begin"], reverse=True)
coming_soon = [i for i in items if i.get("date") and i["date"] >= today]
coming_soon.sort(key=lambda i: i["date"])
past_workshops = [i for i in items if i.get("date") and i["date"] < today]
past_workshops.sort(key=lambda i: i["date"], reverse=True)
self_paced = [i for i in items if not i.get("date")]
recorded_workshops = [i for i in items if i.get("video")]

View File

@@ -1,94 +1,18 @@
- date: 2019-05-01
country: us
city: Cleveland, OH
event: PyCon
speaker: jpetazzo, s0ulshake
title: Getting started with Kubernetes and container orchestration
attend: https://us.pycon.org/2019/schedule/presentation/74/
- date: 2019-04-28
country: us
city: Chicago, IL
- date: 2018-11-23
city: Copenhagen
country: dk
event: GOTO
speaker: jpetazzo
title: Getting Started With Kubernetes and Container Orchestration
attend: https://gotochgo.com/2019/workshops/148
slides: https://gotochgo2019.container.training/
- date: 2019-04-26
country: fr
city: Paris
event: ENIX SAS
speaker: jpetazzo
title: Opérer et administrer Kubernetes
attend: https://enix.io/fr/services/formation/operer-et-administrer-kubernetes/
- date: [2019-04-23, 2019-04-24]
country: fr
city: Paris
event: ENIX SAS
speaker: jpetazzo
title: Déployer ses applications avec Kubernetes (in French)
lang: fr
attend: https://enix.io/fr/services/formation/deployer-ses-applications-avec-kubernetes/
slides: https://kube-2019-04.container.training
- date: [2019-04-15, 2019-04-16]
country: fr
city: Paris
event: ENIX SAS
speaker: "jpetazzo, alexbuisine"
title: Bien démarrer avec les conteneurs (in French)
lang: fr
attend: https://enix.io/fr/services/formation/bien-demarrer-avec-les-conteneurs/
slides: http://intro-2019-04.container.training/
- date: 2019-03-08
country: uk
city: London
event: QCON
speaker: jpetazzo
title: Getting Started With Kubernetes and Container Orchestration
attend: https://qconlondon.com/london2019/workshop/getting-started-kubernetes-and-container-orchestration
slides: https://qconuk2019.container.training/
video: https://www.youtube.com/playlist?list=PLBAFXs0YjviJwCoxSUkUPhsSxDJzpZbJd
- date: [2019-01-07, 2019-01-08]
country: fr
city: Paris
event: ENIX SAS
speaker: "jpetazzo, alexbuisine"
title: Bien démarrer avec les conteneurs (in French)
lang: fr
attend: https://enix.io/fr/services/formation/bien-demarrer-avec-les-conteneurs/
slides: https://intro-2019-01.container.training
- date: [2018-12-17, 2018-12-18]
country: fr
city: Paris
event: ENIX SAS
speaker: "jpetazzo, rdegez"
title: Déployer ses applications avec Kubernetes
lang: fr
attend: https://enix.io/fr/services/formation/deployer-ses-applications-avec-kubernetes/
slides: http://decembre2018.container.training
title: Build Container Orchestration with Docker Swarm
speaker: bretfisher
attend: https://gotocph.com/2018/workshops/121
- date: 2018-11-08
city: San Francisco, CA
country: us
event: QCON
title: Introduction to Docker and Containers
speaker: zeroasterisk
attend: https://qconsf.com/sf2018/workshop/introduction-docker-and-containers
- date: 2018-11-08
city: San Francisco, CA
country: us
event: QCON
title: Getting Started With Kubernetes and Container Orchestration
speaker: jpetazzo
attend: https://qconsf.com/sf2018/workshop/getting-started-kubernetes-and-container-orchestration-thursday-section
slides: http://qconsf2018.container.training/
attend: https://qconsf.com/sf2018/workshop/introduction-docker-and-containers
- date: 2018-11-09
city: San Francisco, CA
@@ -96,8 +20,7 @@
event: QCON
title: Getting Started With Kubernetes and Container Orchestration
speaker: jpetazzo
attend: https://qconsf.com/sf2018/workshop/getting-started-kubernetes-and-container-orchestration-friday-section
slides: http://qconsf2018.container.training/
attend: https://qconsf.com/sf2018/workshop/getting-started-kubernetes-and-container-orchestration
- date: 2018-10-31
city: London, UK
@@ -105,7 +28,6 @@
event: Velocity EU
title: Kubernetes 101
speaker: bridgetkromhout
slides: https://velocityeu2018.container.training
attend: https://conferences.oreilly.com/velocity/vl-eu/public/schedule/detail/71149
- date: 2018-10-30
@@ -132,9 +54,8 @@
title: Kubernetes 101
speaker: bridgetkromhout
attend: https://conferences.oreilly.com/velocity/vl-ny/public/schedule/detail/70102
slides: https://velny-k8s101-2018.container.training
- date: 2018-10-01
- date: 2018-09-30
city: New York, NY
country: us
event: Velocity
@@ -143,7 +64,7 @@
attend: https://conferences.oreilly.com/velocity/vl-ny/public/schedule/detail/69875
slides: https://k8s2d.container.training
- date: 2018-10-01
- date: 2018-09-30
city: New York, NY
country: us
event: Velocity
@@ -156,7 +77,7 @@
city: Paris
event: ENIX SAS
speaker: jpetazzo
title: Déployer ses applications avec Kubernetes
title: Déployer ses applications avec Kubernetes (in French)
lang: fr
attend: https://enix.io/fr/services/formation/deployer-ses-applications-avec-kubernetes/
slides: https://septembre2018.container.training

View File

@@ -19,7 +19,7 @@ chapters:
- shared/about-slides.md
- shared/toc.md
- - containers/Docker_Overview.md
#- containers/Docker_History.md
- containers/Docker_History.md
- containers/Training_Environment.md
- containers/Installing_Docker.md
- containers/First_Containers.md
@@ -29,57 +29,31 @@ chapters:
- containers/Building_Images_Interactively.md
- containers/Building_Images_With_Dockerfiles.md
- containers/Cmd_And_Entrypoint.md
- - containers/Copying_Files_During_Build.md
- |
# Exercise — writing Dockerfiles
Let's write Dockerfiles for an existing application!
The code is at: https://github.com/jpetazzo/wordsmith
- containers/Multi_Stage_Builds.md
- containers/Copying_Files_During_Build.md
- - containers/Multi_Stage_Builds.md
- containers/Publishing_To_Docker_Hub.md
- containers/Dockerfile_Tips.md
- |
# Exercise — writing better Dockerfiles
Let's update our Dockerfiles to leverage multi-stage builds!
The code is at: https://github.com/jpetazzo/wordsmith
Use a different tag for these images, so that we can compare their sizes.
What's the size difference between single-stage and multi-stage builds?
- - containers/Naming_And_Inspecting.md
- containers/Labels.md
- containers/Getting_Inside.md
- containers/Resource_Limits.md
- - containers/Container_Networking_Basics.md
- containers/Network_Drivers.md
- containers/Container_Network_Model.md
#- containers/Connecting_Containers_With_Links.md
- containers/Ambassadors.md
- - containers/Local_Development_Workflow.md
- containers/Windows_Containers.md
- containers/Working_With_Volumes.md
- containers/Compose_For_Dev_Stacks.md
- |
# Exercise — writing a Compose file
Let's write a Compose file for the wordsmith app!
The code is at: https://github.com/jpetazzo/wordsmith
- - containers/Docker_Machine.md
- containers/Advanced_Dockerfiles.md
- containers/Docker_Machine.md
- - containers/Advanced_Dockerfiles.md
- containers/Application_Configuration.md
- containers/Logging.md
- containers/Resource_Limits.md
- - containers/Namespaces_Cgroups.md
- containers/Copy_On_Write.md
#- containers/Containers_From_Scratch.md
- - containers/Container_Engines.md
#- containers/Ecosystem.md
- containers/Ecosystem.md
- containers/Orchestration_Overview.md
- shared/thankyou.md
- containers/links.md

View File

@@ -42,7 +42,6 @@ chapters:
#- containers/Connecting_Containers_With_Links.md
- containers/Ambassadors.md
- - containers/Local_Development_Workflow.md
- containers/Windows_Containers.md
- containers/Working_With_Volumes.md
- containers/Compose_For_Dev_Stacks.md
- containers/Docker_Machine.md

View File

@@ -1,89 +0,0 @@
# API server availability
- When we set up a node, we need the address of the API server:
- for kubelet
- for kube-proxy
- sometimes for the pod network system (like kube-router)
- How do we ensure the availability of that endpoint?
(what if the node running the API server goes down?)
---
## Option 1: external load balancer
- Set up an external load balancer
- Point kubelet (and other components) to that load balancer
- Put the node(s) running the API server behind that load balancer
- Update the load balancer if/when an API server node needs to be replaced
- On cloud infrastructures, some mechanisms provide automation for this
(e.g. on AWS, an Elastic Load Balancer + Auto Scaling Group)
- [Example in Kubernetes The Hard Way](https://github.com/kelseyhightower/kubernetes-the-hard-way/blob/master/docs/08-bootstrapping-kubernetes-controllers.md#the-kubernetes-frontend-load-balancer)
---
## Option 2: local load balancer
- Set up a load balancer (like NGINX, HAProxy...) on *each* node
- Configure that load balancer to send traffic to the API server node(s)
- Point kubelet (and other components) to `localhost`
- Update the load balancer configuration when API server nodes are updated
---
## Updating the local load balancer config
- Distribute the updated configuration (push)
- Or regularly check for updates (pull)
- The latter requires an external, highly available store
(it could be an object store, an HTTP server, or even DNS...)
- Updates can be facilitated by a DaemonSet
(but remember that it can't be used when installing a new node!)
---
## Option 3: DNS records
- Put all the API server nodes behind a round-robin DNS
- Point kubelet (and other components) to that name
- Update the records when needed
- Note: this option is not officially supported
(but since kubelet supports reconnection anyway, it *should* work)
---
## Option 4: ....................
- Many managed clusters expose a high-availability API endpoint
(and you don't have to worry about it)
- You can also use HA mechanisms that you're familiar with
(e.g. virtual IPs)
- Tunnels are also fine
(e.g. [k3s](https://k3s.io/) uses a tunnel to allow each node to contact the API server)

View File

@@ -1,383 +0,0 @@
# Kubernetes architecture
We can arbitrarily split Kubernetes in two parts:
- the *nodes*, a set of machines that run our containerized workloads;
- the *control plane*, a set of processes implementing the Kubernetes APIs.
Kubernetes also relies on underlying infrastructure:
- servers, network connectivity (obviously!),
- optional components like storage systems, load balancers ...
---
## Control plane location
The control plane can run:
- in containers, on the same nodes that run other application workloads
(example: Minikube; 1 node runs everything)
- on a dedicated node
(example: a cluster installed with kubeadm)
- on a dedicated set of nodes
(example: Kubernetes The Hard Way; kops)
- outside of the cluster
(example: most managed clusters like AKS, EKS, GKE)
---
class: pic
![Kubernetes architecture diagram: control plane and nodes](images/k8s-arch2.png)
---
## What runs on a node
- Our containerized workloads
- A container engine like Docker, CRI-O, containerd...
(in theory, the choice doesn't matter, as the engine is abstracted by Kubernetes)
- kubelet: an agent connecting the node to the cluster
(it connects to the API server, registers the node, receives instructions)
- kube-proxy: a component used for internal cluster communication
(note that this is *not* an overlay network or a CNI plugin!)
---
## What's in the control plane
- Everything is stored in etcd
(it's the only stateful component)
- Everyone communicates exclusively through the API server:
- we (users) interact with the cluster through the API server
- the nodes register and get their instructions through the API server
- the other control plane components also register with the API server
- API server is the only component that reads/writes from/to etcd
---
## Communication protocols: API server
- The API server exposes a REST API
(except for some calls, e.g. to attach interactively to a container)
- Almost all requests and responses are JSON following a strict format
- For performance, the requests and responses can also be done over protobuf
(see this [design proposal](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/protobuf.md) for details)
- In practice, protobuf is used for all internal communication
(between control plane components, and with kubelet)
---
## Communication protocols: on the nodes
The kubelet agent uses a number of special-purpose protocols and interfaces, including:
- CRI (Container Runtime Interface)
- used for communication with the container engine
- abstracts the differences between container engines
- based on gRPC+protobuf
- [CNI (Container Network Interface)](https://github.com/containernetworking/cni/blob/master/SPEC.md)
- used for communication with network plugins
- network plugins are implemented as executable programs invoked by kubelet
- network plugins provide IPAM
- network plugins set up network interfaces in pods
---
class: pic
![Kubernetes architecture diagram: communication between components](images/k8s-arch4-thanks-luxas.png)
---
# The Kubernetes API
[
*The Kubernetes API server is a "dumb server" which offers storage, versioning, validation, update, and watch semantics on API resources.*
](
https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/protobuf.md#proposal-and-motivation
)
([Clayton Coleman](https://twitter.com/smarterclayton), Kubernetes Architect and Maintainer)
What does that mean?
---
## The Kubernetes API is declarative
- We cannot tell the API, "run a pod"
- We can tell the API, "here is the definition for pod X"
- The API server will store that definition (in etcd)
- *Controllers* will then wake up and create a pod matching the definition
---
## The core features of the Kubernetes API
- We can create, read, update, and delete objects
- We can also *watch* objects
(be notified when an object changes, or when an object of a given type is created)
- Objects are strongly typed
- Types are *validated* and *versioned*
- Storage and watch operations are provided by etcd
(note: the [k3s](https://k3s.io/) project allows us to use sqlite instead of etcd)
---
## Let's experiment a bit!
- For the exercises in this section, connect to the first node of the `test` cluster
.exercise[
- SSH to the first node of the test cluster
- Check that the cluster is operational:
```bash
kubectl get nodes
```
- All nodes should be `Ready`
]
---
## Create
- Let's create a simple object
.exercise[
- Create a namespace with the following command:
```bash
kubectl create -f- <<EOF
apiVersion: v1
kind: Namespace
metadata:
name: hello
EOF
```
]
This is equivalent to `kubectl create namespace hello`.
---
## Read
- Let's retrieve the object we just created
.exercise[
- Read back our object:
```bash
kubectl get namespace hello -o yaml
```
]
We see a lot of data that wasn't here when we created the object.
Some data was automatically added to the object (like `spec.finalizers`).
Some data is dynamic (typically, the content of `status`.)
---
## API requests and responses
- Almost every Kubernetes API payload (requests and responses) has the same format:
```yaml
apiVersion: xxx
kind: yyy
metadata:
name: zzz
(more metadata fields here)
(more fields here)
```
- The fields shown above are mandatory, except for some special cases
(e.g.: in lists of resources, the list itself doesn't have a `metadata.name`)
- We show YAML for convenience, but the API uses JSON
(with optional protobuf encoding)
---
class: extra-details
## API versions
- The `apiVersion` field corresponds to an *API group*
- It can be either `v1` (aka "core" group or "legacy group"), or `group/versions`; e.g.:
- `apps/v1`
- `rbac.authorization.k8s.io/v1`
- `extensions/v1beta1`
- It does not indicate which version of Kubernetes we're talking about
- It *indirectly* indicates the version of the `kind`
(which fields exist, their format, which ones are mandatory...)
- A single resource type (`kind`) is rarely versioned alone
(e.g.: the `batch` API group contains `jobs` and `cronjobs`)
---
## Update
- Let's update our namespace object
- There are many ways to do that, including:
- `kubectl apply` (and provide an updated YAML file)
- `kubectl edit`
- `kubectl patch`
- many helpers, like `kubectl label`, or `kubectl set`
- In each case, `kubectl` will:
- get the current definition of the object
- compute changes
- submit the changes (with `PATCH` requests)
---
## Adding a label
- For demonstration purposes, let's add a label to the namespace
- The easiest way is to use `kubectl label`
.exercise[
- In one terminal, watch namespaces:
```bash
kubectl get namespaces --show-labels -w
```
- In the other, update our namespace:
```bash
kubectl label namespaces hello color=purple
```
]
We demonstrated *update* and *watch* semantics.
---
## What's special about *watch*?
- The API server itself doesn't do anything: it's just a fancy object store
- All the actual logic in Kubernetes is implemented with *controllers*
- A *controller* watches a set of resources, and takes action when they change
- Examples:
- when a Pod object is created, it gets scheduled and started
- when a Pod belonging to a ReplicaSet terminates, it gets replaced
- when a Deployment object is updated, it can trigger a rolling update
---
# Other control plane components
- API server ✔️
- etcd ✔️
- Controller manager
- Scheduler
---
## Controller manager
- This is a collection of loops watching all kinds of objects
- That's where the actual logic of Kubernetes lives
- When we create a Deployment (e.g. with `kubectl run web --image=nginx`),
- we create a Deployment object
- the Deployment controller notices it, creates a ReplicaSet
- the ReplicaSet controller notices it, creates a Pod
---
## Scheduler
- When a pod is created, it is in `Pending` state
- The scheduler (or rather: *a scheduler*) must bind it to a node
- Kubernetes comes with an efficient scheduler with many features
- if we have special requirements, we can add another scheduler
<br/>
(example: this [demo scheduler](https://github.com/kelseyhightower/scheduler) uses the cost of nodes, stored in node annotations)
- A pod might stay in `Pending` state for a long time:
- if the cluster is full
- if the pod has special constraints that can't be met
- if the scheduler is not running (!)

View File

@@ -64,7 +64,7 @@
(`401 Unauthorized` HTTP code)
- If a request is neither rejected nor accepted by anyone, it's anonymous
- If a request is neither accepted nor accepted by anyone, it's anonymous
- the user name is `system:anonymous`
@@ -108,7 +108,7 @@ class: extra-details
--raw \
-o json \
| jq -r .users[0].user[\"client-certificate-data\"] \
| openssl base64 -d -A \
| base64 -d \
| openssl x509 -text \
| grep Subject:
```
@@ -127,14 +127,12 @@ class: extra-details
- `--raw` includes certificate information (which shows as REDACTED otherwise)
- `-o json` outputs the information in JSON format
- `| jq ...` extracts the field with the user certificate (in base64)
- `| openssl base64 -d -A` decodes the base64 format (now we have a PEM file)
- `| base64 -d` decodes the base64 format (now we have a PEM file)
- `| openssl x509 -text` parses the certificate and outputs it as plain text
- `| grep Subject:` shows us the line that interests us
→ We are user `kubernetes-admin`, in group `system:masters`.
(We will see later how and why this gives us the permissions that we have.)
---
## User certificates in practice
@@ -260,7 +258,7 @@ class: extra-details
- Extract the token and decode it:
```bash
TOKEN=$(kubectl get secret $SECRET -o json \
| jq -r .data.token | openssl base64 -d -A)
| jq -r .data.token | base64 -d)
```
]
@@ -540,7 +538,7 @@ It's important to note a couple of details in these flags ...
- But that we can't create things:
```
./kubectl create deployment testrbac --image=nginx
./kubectl run tryme --image=nginx
```
- Exit the container with `exit` or `^D`
@@ -569,68 +567,3 @@ It's important to note a couple of details in these flags ...
kubectl auth can-i list nodes \
--as system:serviceaccount:<namespace>:<name-of-service-account>
```
---
class: extra-details
## Where do our permissions come from?
- When interacting with the Kubernetes API, we are using a client certificate
- We saw previously that this client certificate contained:
`CN=kubernetes-admin` and `O=system:masters`
- Let's look for these in existing ClusterRoleBindings:
```bash
kubectl get clusterrolebindings -o yaml |
grep -e kubernetes-admin -e system:masters
```
(`system:masters` should show up, but not `kubernetes-admin`.)
- Where does this match come from?
---
class: extra-details
## The `system:masters` group
- If we eyeball the output of `kubectl get clusterrolebindings -o yaml`, we'll find out!
- It is in the `cluster-admin` binding:
```bash
kubectl describe clusterrolebinding cluster-admin
```
- This binding associates `system:masters` to the cluster role `cluster-admin`
- And the `cluster-admin` is, basically, `root`:
```bash
kubectl describe clusterrole cluster-admin
```
---
class: extra-details
## Figuring out who can do what
- For auditing purposes, sometimes we want to know who can perform an action
- Here is a proof-of-concept tool by Aqua Security, doing exactly that:
https://github.com/aquasecurity/kubectl-who-can
- This is one way to install it:
```bash
docker run --rm -v /usr/local/bin:/go/bin golang \
go get -v github.com/aquasecurity/kubectl-who-can
```
- This is one way to use it:
```bash
kubectl-who-can create pods
```

View File

@@ -1,259 +0,0 @@
# TLS bootstrap
- kubelet needs TLS keys and certificates to communicate with the control plane
- How do we generate this information?
- How do we make it available to kubelet?
---
## Option 1: push
- When we want to provision a node:
- generate its keys, certificate, and sign centrally
- push the files to the node
- OK for "traditional", on-premises deployments
- Not OK for cloud deployments with auto-scaling
---
## Option 2: poll + push
- Discover nodes when they are created
(e.g. with cloud API)
- When we detect a new node, push TLS material to the node
(like in option 1)
- It works, but:
- discovery code is specific to each provider
- relies heavily on the cloud provider API
- doesn't work on-premises
- doesn't scale
---
## Option 3: bootstrap tokens + CSR API
- Since Kubernetes 1.4, the Kubernetes API supports CSR
(Certificate Signing Requests)
- This is similar to the protocol used to obtain e.g. HTTPS certificates:
- subject (here, kubelet) generates TLS keys and CSR
- subject submits CSR to CA
- CA validates (or not) the CSR
- CA sends back signed certificate to subject
- This is combined with *bootstrap tokens*
---
## Bootstrap tokens
- A [bootstrap token](https://kubernetes.io/docs/reference/access-authn-authz/bootstrap-tokens/) is an API access token
- it is a Secret with type `bootstrap.kubernetes.io/token`
- it is 6 public characters (ID) + 16 secret characters
<br/>(example: `whd3pq.d1ushuf6ccisjacu`)
- it gives access to groups `system:bootstrap:<ID>` and `system:bootstrappers`
- additional groups can be specified in the Secret
---
## Bootstrap tokens with kubeadm
- kubeadm automatically creates a bootstrap token
(it is shown at the end of `kubeadm init`)
- That token adds the group `system:bootstrappers:kubeadm:default-node-token`
- kubeadm also creates a ClusterRoleBinding `kubeadm:kubelet-bootstrap`
<br/>binding `...:default-node-token` to ClusterRole `system:node-bootstrapper`
- That ClusterRole gives create/get/list/watch permissions on the CSR API
---
## Bootstrap tokens in practice
- Let's list our bootstrap tokens on a cluster created with kubeadm
.exercise[
- Log into node `test1`
- View bootstrap tokens:
```bash
sudo kubeadm token list
```
]
- Tokens are short-lived
- We can create new tokens with `kubeadm` if necessary
---
class: extra-details
## Retrieving bootstrap tokens with kubectl
- Bootstrap tokens are Secrets with type `bootstrap.kubernetes.io/token`
- Token ID and secret are in data fields `token-id` and `token-secret`
- In Secrets, data fields are encoded with Base64
- This "very simple" command will show us the tokens:
```
kubectl -n kube-system get secrets -o json |
jq -r '.items[]
| select(.type=="bootstrap.kubernetes.io/token")
| ( .data["token-id"] + "Lg==" + .data["token-secret"] + "Cg==")
' | base64 -d
```
(On recent versions of `jq`, you can simplify by using filter `@base64d`.)
---
class: extra-details
## Using a bootstrap token
- The token we need to use has the form `abcdef.1234567890abcdef`
.exercise[
- Check that it is accepted by the API server:
```bash
curl -k -H "Authorization: Bearer abcdef.1234567890abcdef"
```
- We should see that we are *authenticated* but not *authorized*:
```
User \"system:bootstrap:abcdef\" cannot get path \"/\""
```
- Check that we can access the CSR API:
```bash
curl -k -H "Authorization: Bearer abcdef.1234567890abcdef" \
https://10.96.0.1/apis/certificates.k8s.io/v1beta1/certificatesigningrequests
```
]
---
## The cluster-info ConfigMap
- Before we can talk to the API, we need:
- the API server address (obviously!)
- the cluster CA certificate
- That information is stored in a public ConfigMap
.exercise[
- Retrieve that ConfigMap:
```bash
curl -k https://10.96.0.1/api/v1/namespaces/kube-public/configmaps/cluster-info
```
]
*Extracting the kubeconfig file is left as an exercise for the reader.*
---
class: extra-details
## Signature of the config-map
- You might have noticed a few `jws-kubeconfig-...` fields
- These are config-map signatures
(so that the client can protect against MITM attacks)
- These are JWS signatures using HMAC-SHA256
(see [here](https://kubernetes.io/docs/reference/access-authn-authz/bootstrap-tokens/#configmap-signing) for more details)
---
## Putting it all together
This is the TLS bootstrap mechanism, step by step.
- The node uses the cluster-info ConfigMap to get the cluster CA certificate
- The node generates its keys and CSR
- Using the bootstrap token, the node creates a CertificateSigningRequest object
- The node watches the CSR object
- The CSR object is accepted (automatically or by an admin)
- The node gets notified, and retrieves the certificate
- The node can now join the cluster
---
## Bottom line
- If you paid attention, we still need a way to:
- either safely get the bootstrap token to the nodes
- or disable auto-approval and manually approve the nodes when they join
- The goal of the TLS bootstrap mechanism is *not* to solve this
(in terms of information knowledge, it's fundamentally impossible!)
- But it reduces the differences between environments, infrastructures, providers ...
- It gives a mechanism that is easier to use, and flexible enough, for most scenarios
---
## More information
- As always, the Kubernetes documentation has extra details:
- [TLS management](https://kubernetes.io/docs/tasks/tls/managing-tls-in-a-cluster/)
- [Authenticating with bootstrap tokens](https://kubernetes.io/docs/reference/access-authn-authz/bootstrap-tokens/)
- [TLS bootstrapping](https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping/)
- [kubeadm token](https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-token/) command
- [kubeadm join](https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-join/) command (has details about [the join workflow](https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-join/#join-workflow))

View File

@@ -1,40 +0,0 @@
## Using images from the Docker Hub
- For everyone's convenience, we took care of building DockerCoins images
- We pushed these images to the DockerHub, under the [dockercoins](https://hub.docker.com/u/dockercoins) user
- These images are *tagged* with a version number, `v0.1`
- The full image names are therefore:
- `dockercoins/hasher:v0.1`
- `dockercoins/rng:v0.1`
- `dockercoins/webui:v0.1`
- `dockercoins/worker:v0.1`
---
## Setting `$REGISTRY` and `$TAG`
- In the upcoming exercises and labs, we use a couple of environment variables:
- `$REGISTRY` as a prefix to all image names
- `$TAG` as the image version tag
- For example, the worker image is `$REGISTRY/worker:$TAG`
- If you copy-paste the commands in these exercises:
**make sure that you set `$REGISTRY` and `$TAG` first!**
- For example:
```
export REGISTRY=dockercoins TAG=v0.1
```
(this will expand `$REGISTRY/worker:$TAG` to `dockercoins/worker:v0.1`)

View File

@@ -1,235 +0,0 @@
## Self-hosting our registry
*Note: this section shows how to run the Docker
open source registry and use it to ship images
on our cluster. While this method works fine,
we recommend that you consider using one of the
hosted, free automated build services instead.
It will be much easier!*
*If you need to run a registry on premises,
this section gives you a starting point, but
you will need to make a lot of changes so that
the registry is secured, highly available, and
so that your build pipeline is automated.*
---
## Using the open source registry
- We need to run a `registry` container
- It will store images and layers to the local filesystem
<br/>(but you can add a config file to use S3, Swift, etc.)
- Docker *requires* TLS when communicating with the registry
- unless for registries on `127.0.0.0/8` (i.e. `localhost`)
- or with the Engine flag `--insecure-registry`
- Our strategy: publish the registry container on a NodePort,
<br/>so that it's available through `127.0.0.1:xxxxx` on each node
---
## Deploying a self-hosted registry
- We will deploy a registry container, and expose it with a NodePort
.exercise[
- Create the registry service:
```bash
kubectl create deployment registry --image=registry
```
- Expose it on a NodePort:
```bash
kubectl expose deploy/registry --port=5000 --type=NodePort
```
]
---
## Connecting to our registry
- We need to find out which port has been allocated
.exercise[
- View the service details:
```bash
kubectl describe svc/registry
```
- Get the port number programmatically:
```bash
NODEPORT=$(kubectl get svc/registry -o json | jq .spec.ports[0].nodePort)
REGISTRY=127.0.0.1:$NODEPORT
```
]
---
## Testing our registry
- A convenient Docker registry API route to remember is `/v2/_catalog`
.exercise[
<!-- ```hide kubectl wait deploy/registry --for condition=available```-->
- View the repositories currently held in our registry:
```bash
curl $REGISTRY/v2/_catalog
```
]
--
We should see:
```json
{"repositories":[]}
```
---
## Testing our local registry
- We can retag a small image, and push it to the registry
.exercise[
- Make sure we have the busybox image, and retag it:
```bash
docker pull busybox
docker tag busybox $REGISTRY/busybox
```
- Push it:
```bash
docker push $REGISTRY/busybox
```
]
---
## Checking again what's on our local registry
- Let's use the same endpoint as before
.exercise[
- Ensure that our busybox image is now in the local registry:
```bash
curl $REGISTRY/v2/_catalog
```
]
The curl command should now output:
```json
{"repositories":["busybox"]}
```
---
## Building and pushing our images
- We are going to use a convenient feature of Docker Compose
.exercise[
- Go to the `stacks` directory:
```bash
cd ~/container.training/stacks
```
- Build and push the images:
```bash
export REGISTRY
export TAG=v0.1
docker-compose -f dockercoins.yml build
docker-compose -f dockercoins.yml push
```
]
Let's have a look at the `dockercoins.yml` file while this is building and pushing.
---
```yaml
version: "3"
services:
rng:
build: dockercoins/rng
image: ${REGISTRY-127.0.0.1:5000}/rng:${TAG-latest}
deploy:
mode: global
...
redis:
image: redis
...
worker:
build: dockercoins/worker
image: ${REGISTRY-127.0.0.1:5000}/worker:${TAG-latest}
...
deploy:
replicas: 10
```
.warning[Just in case you were wondering ... Docker "services" are not Kubernetes "services".]
---
class: extra-details
## Avoiding the `latest` tag
.warning[Make sure that you've set the `TAG` variable properly!]
- If you don't, the tag will default to `latest`
- The problem with `latest`: nobody knows what it points to!
- the latest commit in the repo?
- the latest commit in some branch? (Which one?)
- the latest tag?
- some random version pushed by a random team member?
- If you keep pushing the `latest` tag, how do you roll back?
- Image tags should be meaningful, i.e. correspond to code branches, tags, or hashes
---
## Checking the content of the registry
- All our images should now be in the registry
.exercise[
- Re-run the same `curl` command as earlier:
```bash
curl $REGISTRY/v2/_catalog
```
]
*In these slides, all the commands to deploy
DockerCoins will use a $REGISTRY environment
variable, so that we can quickly switch from
the self-hosted registry to pre-built images
hosted on the Docker Hub. So make sure that
this $REGISTRY variable is set correctly when
running the exercises!*

View File

@@ -1,144 +0,0 @@
# The Cloud Controller Manager
- Kubernetes has many features that are cloud-specific
(e.g. providing cloud load balancers when a Service of type LoadBalancer is created)
- These features were initially implemented in API server and controller manager
- Since Kubernetes 1.6, these features are available through a separate process:
the *Cloud Controller Manager*
- The CCM is optional, but if we run in a cloud, we probably want it!
---
## Cloud Controller Manager duties
- Creating and updating cloud load balancers
- Configuring routing tables in the cloud network (specific to GCE)
- Updating node labels to indicate region, zone, instance type ...
- Obtain node name, internal and external addresses from cloud metadata service
- Deleting nodes from Kubernetes when they're deleted in the cloud
- Managing *some* volumes (e.g. ELBs, AzureDisks ...)
(Eventually, volumes will be managed by the CSI)
---
## In-tree vs. out-of-tree
- A number of cloud providers are supported "in-tree"
(in the main kubernetes/kubernetes repository on GitHub)
- More cloud providers are supported "out-of-tree"
(with code in different repositories)
- There is an [ongoing effort](https://github.com/kubernetes/kubernetes/tree/master/pkg/cloudprovider) to move everything to out-of-tree providers
---
## In-tree providers
The following providers are actively maintained:
- Amazon Web Services
- Azure
- Google Compute Engine
- IBM Cloud
- OpenStack
- VMware vSphere
These ones are less actively maintained:
- Apache CloudStack
- oVirt
- VMware Photon
---
## Out-of-tree providers
The list includes the following providers:
- DigitalOcean
- keepalived (not exactly a cloud; provides VIPs for load balancers)
- Linode
- Oracle Cloud Infrastructure
(And possibly others; there is no central registry for these.)
---
## Audience questions
- What kind of clouds are you using / planning to use?
- What kind of details would you like to see in this section?
- Would you appreciate details on clouds that you don't / won't use?
---
## Cloud Controller Manager in practice
- Write a configuration file
(typically `/etc/kubernetes/cloud.conf`)
- Run the CCM process
(on self-hosted clusters, this can be a DaemonSet selecting the control plane nodes)
- Start kubelet with `--cloud-provider=external`
- When using managed clusters, this is done automatically
- There is very little documentation to write the configuration file
(except for OpenStack)
---
## Bootstrapping challenges
- When a node joins the cluster, it needs to obtain a signed TLS certificate
- That certificate must contain the node's addresses
- These addresses are provided by the Cloud Controller Manager
(at least the external address)
- To get these addresses, the node needs to communicate with the control plane
- ... Which means joining the cluster
(The problem didn't occur when cloud-specific code was running in kubelet: kubelet could obtain the required information directly from the cloud provider's metadata service.)
---
## More information about CCM
- CCM configuration and operation is highly specific to each cloud provider
(which is why this section remains very generic)
- The Kubernetes documentation has *some* information:
- [architecture and diagrams](https://kubernetes.io/docs/concepts/architecture/cloud-controller/)
- [configuration](https://kubernetes.io/docs/concepts/cluster-administration/cloud-providers/) (mainly for OpenStack)
- [deployment](https://kubernetes.io/docs/tasks/administer-cluster/running-cloud-controller/)

View File

@@ -1,362 +0,0 @@
# Backing up clusters
- Backups can have multiple purposes:
- disaster recovery (servers or storage are destroyed or unreachable)
- error recovery (human or process has altered or corrupted data)
- cloning environments (for testing, validation ...)
- Let's see the strategies and tools available with Kubernetes!
---
## Important
- Kubernetes helps us with disaster recovery
(it gives us replication primitives)
- Kubernetes helps us to clone / replicate environments
(all resources can be described with manifests)
- Kubernetes *does not* help us with error recovery
- We still need to backup / snapshot our data:
- with database backups (mysqldump, pgdump, etc.)
- and/or snapshots at the storage layer
- and/or traditional full disk backups
---
## In a perfect world ...
- The deployment of our Kubernetes clusters is automated
(recreating a cluster takes less than a minute of human time)
- All the resources (Deployments, Services...) on our clusters are under version control
(never use `kubectl run`; always apply YAML files coming from a repository)
- Stateful components are either:
- stored on systems with regular snapshots
- backed up regularly to an external, durable storage
- outside of Kubernetes
---
## Kubernetes cluster deployment
- If our deployment system isn't fully automated, it should at least be documented
- Litmus test: how long does it take to deploy a cluster ...
- for a senior engineer?
- for a new hire?
- Does it require external intervention?
(e.g. provisioning servers, signing TLS certs ...)
---
## Plan B
- Full machine backups of the control plane can help
- If the control plane is in pods (or containers), pay attention to storage drivers
(if the backup mechanism is not container-aware, the backups can take way more resources than they should, or even be unusable!)
- If the previous sentence worries you:
**automate the deployment of your clusters!**
---
## Managing our Kubernetes resources
- Ideal scenario:
- never create a resource directly on a cluster
- push to a code repository
- a special branch (`production` or even `master`) gets automatically deployed
- Some folks call this "GitOps"
(it's the logical evolution of configuration management and infrastructure as code)
---
## GitOps in theory
- What do we keep in version control?
- For very simple scenarios: source code, Dockerfiles, scripts
- For real applications: add resources (as YAML files)
- For applications deployed multiple times: Helm, Kustomize ...
(staging and production count as "multiple times")
---
## GitOps tooling
- Various tools exist (Weave Flux, GitKube...)
- These tools are still very young
- You still need to write YAML for all your resources
- There is no tool to:
- list *all* resources in a namespace
- get resource YAML in a canonical form
- diff YAML descriptions with current state
---
## GitOps in practice
- Start describing your resources with YAML
- Leverage a tool like Kustomize or Helm
- Make sure that you can easily deploy to a new namespace
(or even better: to a new cluster)
- When tooling matures, you will be ready
---
## Plan B
- What if we can't describe everything with YAML?
- What if we manually create resources and forget to commit them to source control?
- What about global resources, that don't live in a namespace?
- How can we be sure that we saved *everything*?
---
## Backing up etcd
- All objects are saved in etcd
- etcd data should be relatively small
(and therefore, quick and easy to back up)
- Two options to back up etcd:
- snapshot the data directory
- use `etcdctl snapshot`
---
## Making an etcd snapshot
- The basic command is simple:
```bash
etcdctl snapshot save <filename>
```
- But we also need to specify:
- an environment variable to specify that we want etcdctl v3
- the address of the server to back up
- the path to the key, certificate, and CA certificate
<br/>(if our etcd uses TLS certificates)
---
## Snapshotting etcd on kubeadm
- The following command will work on clusters deployed with kubeadm
(and maybe others)
- It should be executed on a master node
```bash
docker run --rm --net host -v $PWD:/vol \
-v /etc/kubernetes/pki/etcd:/etc/kubernetes/pki/etcd:ro \
-e ETCDCTL_API=3 k8s.gcr.io/etcd:3.3.10 \
etcdctl --endpoints=https://[127.0.0.1]:2379 \
--cacert=/etc/kubernetes/pki/etcd/ca.crt \
--cert=/etc/kubernetes/pki/etcd/healthcheck-client.crt \
--key=/etc/kubernetes/pki/etcd/healthcheck-client.key \
snapshot save /vol/snapshot
```
- It will create a file named `snapshot` in the current directory
---
## How can we remember all these flags?
- Look at the static pod manifest for etcd
(in `/etc/kubernetes/manifests`)
- The healthcheck probe is calling `etcdctl` with all the right flags
😉👍✌️
- Exercise: write the YAML for a batch job to perform the backup
---
## Restoring an etcd snapshot
- ~~Execute exactly the same command, but replacing `save` with `restore`~~
(Believe it or not, doing that will *not* do anything useful!)
- The `restore` command does *not* load a snapshot into a running etcd server
- The `restore` command creates a new data directory from the snapshot
(it's an offline operation; it doesn't interact with an etcd server)
- It will create a new data directory in a temporary container
(leaving the running etcd node untouched)
---
## When using kubeadm
1. Create a new data directory from the snapshot:
```bash
sudo rm -rf /var/lib/etcd
docker run --rm -v /var/lib:/var/lib -v $PWD:/vol \
-e ETCDCTL_API=3 k8s.gcr.io/etcd:3.3.10 \
etcdctl snapshot restore /vol/snapshot --data-dir=/var/lib/etcd
```
2. Provision the control plane, using that data directory:
```bash
sudo kubeadm init \
--ignore-preflight-errors=DirAvailable--var-lib-etcd
```
3. Rejoin the other nodes
---
## The fine print
- This only saves etcd state
- It **does not** save persistent volumes and local node data
- Some critical components (like the pod network) might need to be reset
- As a result, our pods might have to be recreated, too
- If we have proper liveness checks, this should happen automatically
---
## More information about etcd backups
- [Kubernetes documentation](https://kubernetes.io/docs/tasks/administer-cluster/configure-upgrade-etcd/#built-in-snapshot) about etcd backups
- [etcd documentation](https://coreos.com/etcd/docs/latest/op-guide/recovery.html#snapshotting-the-keyspace) about snapshots and restore
- [A good blog post by elastisys](https://elastisys.com/2018/12/10/backup-kubernetes-how-and-why/) explaining how to restore a snapshot
- [Another good blog post by consol labs](https://labs.consol.de/kubernetes/2018/05/25/kubeadm-backup.html) on the same topic
---
## Don't forget ...
- Also back up the TLS information
(at the very least: CA key and cert; API server key and cert)
- With clusters provisioned by kubeadm, this is in `/etc/kubernetes/pki`
- If you don't:
- you will still be able to restore etcd state and bring everything back up
- you will need to redistribute user certificates
.warning[**TLS information is highly sensitive!
<br/>Anyone who has it has full access to your cluster!**]
---
## Stateful services
- It's totally fine to keep your production databases outside of Kubernetes
*Especially if you have only one database server!*
- Feel free to put development and staging databases on Kubernetes
(as long as they don't hold important data)
- Using Kubernetes for stateful services makes sense if you have *many*
(because then you can leverage Kubernetes automation)
---
## Snapshotting persistent volumes
- Option 1: snapshot volumes out of band
(with the API/CLI/GUI of our SAN/cloud/...)
- Option 2: storage system integration
(e.g. [Portworx](https://docs.portworx.com/portworx-install-with-kubernetes/storage-operations/create-snapshots/) can [create snapshots through annotations](https://docs.portworx.com/portworx-install-with-kubernetes/storage-operations/create-snapshots/snaps-annotations/#taking-periodic-snapshots-on-a-running-pod))
- Option 3: [snapshots through Kubernetes API](https://kubernetes.io/blog/2018/10/09/introducing-volume-snapshot-alpha-for-kubernetes/)
(now in alpha for a few storage providers: GCE, OpenSDS, Ceph, Portworx)
---
## More backup tools
- [Stash](https://appscode.com/products/stash/)
back up Kubernetes persistent volumes
- [ReShifter](https://github.com/mhausenblas/reshifter)
cluster state management
- ~~Heptio Ark~~ [Velero](https://github.com/heptio/velero)
full cluster backup
- [kube-backup](https://github.com/pieterlange/kube-backup)
simple scripts to save resource YAML to a git repository

View File

@@ -1,167 +0,0 @@
# Cluster sizing
- What happens when the cluster gets full?
- How can we scale up the cluster?
- Can we do it automatically?
- What are other methods to address capacity planning?
---
## When are we out of resources?
- kubelet monitors node resources:
- memory
- node disk usage (typically the root filesystem of the node)
- image disk usage (where container images and RW layers are stored)
- For each resource, we can provide two thresholds:
- a hard threshold (if it's met, it provokes immediate action)
- a soft threshold (provokes action only after a grace period)
- Resource thresholds and grace periods are configurable
(by passing kubelet command-line flags)
---
## What happens then?
- If disk usage is too high:
- kubelet will try to remove terminated pods
- then, it will try to *evict* pods
- If memory usage is too high:
- it will try to evict pods
- The node is marked as "under pressure"
- This temporarily prevents new pods from being scheduled on the node
---
## Which pods get evicted?
- kubelet looks at the pods' QoS and PriorityClass
- First, pods with BestEffort QoS are considered
- Then, pods with Burstable QoS exceeding their *requests*
(but only if the exceeding resource is the one that is low on the node)
- Finally, pods with Guaranteed QoS, and Burstable pods within their requests
- Within each group, pods are sorted by PriorityClass
- If there are pods with the same PriorityClass, they are sorted by usage excess
(i.e. the pods whose usage exceeds their requests the most are evicted first)
---
class: extra-details
## Eviction of Guaranteed pods
- *Normally*, pods with Guaranteed QoS should not be evicted
- A chunk of resources is reserved for node processes (like kubelet)
- It is expected that these processes won't use more than this reservation
- If they do use more resources anyway, all bets are off!
- If this happens, kubelet must evict Guaranteed pods to preserve node stability
(or Burstable pods that are still within their requested usage)
---
## What happens to evicted pods?
- The pod is terminated
- It is marked as `Failed` at the API level
- If the pod was created by a controller, the controller will recreate it
- The pod will be recreated on another node, *if there are resources available!*
- For more details about the eviction process, see:
- [this documentation page](https://kubernetes.io/docs/tasks/administer-cluster/out-of-resource/) about resource pressure and pod eviction,
- [this other documentation page](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/) about pod priority and preemption.
---
## What if there are no resources available?
- Sometimes, a pod cannot be scheduled anywhere:
- all the nodes are under pressure,
- or the pod requests more resources than are available
- The pod then remains in `Pending` state until the situation improves
---
## Cluster scaling
- One way to improve the situation is to add new nodes
- This can be done automatically with the [Cluster Autoscaler](https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler)
- The autoscaler will automatically scale up:
- if there are pods that failed to be scheduled
- The autoscaler will automatically scale down:
- if nodes have a low utilization for an extended period of time
---
## Restrictions, gotchas ...
- The Cluster Autoscaler only supports a few cloud infrastructures
(see [here](https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler/cloudprovider) for a list)
- The Cluster Autoscaler cannot scale down nodes that have pods using:
- local storage
- affinity/anti-affinity rules preventing them from being rescheduled
- a restrictive PodDisruptionBudget
---
## Other way to do capacity planning
- "Running Kubernetes without nodes"
- Systems like [Virtual Kubelet](https://virtual-kubelet.io/) or Kiyot can run pods using on-demand resources
- Virtual Kubelet can leverage e.g. ACI or Fargate to run pods
- Kiyot runs pods in ad-hoc EC2 instances (1 instance per pod)
- Economic advantage (no wasted capacity)
- Security advantage (stronger isolation between pods)
Check [this blog post](http://jpetazzo.github.io/2019/02/13/running-kubernetes-without-nodes-with-kiyot/) for more details.

View File

@@ -1,309 +0,0 @@
# Upgrading clusters
- It's *recommended* to run consistent versions across a cluster
(mostly to have feature parity and latest security updates)
- It's not *mandatory*
(otherwise, cluster upgrades would be a nightmare!)
- Components can be upgraded one at a time without problems
---
## Checking what we're running
- It's easy to check the version for the API server
.exercise[
- Log into node `test1`
- Check the version of kubectl and of the API server:
```bash
kubectl version
```
]
- In a HA setup with multiple API servers, they can have different versions
- Running the command above multiple times can return different values
---
## Node versions
- It's also easy to check the version of kubelet
.exercise[
- Check node versions (includes kubelet, kernel, container engine):
```bash
kubectl get nodes -o wide
```
]
- Different nodes can run different kubelet versions
- Different nodes can run different kernel versions
- Different nodes can run different container engines
---
## Control plane versions
- If the control plane is self-hosted (running in pods), we can check it
.exercise[
- Show image versions for all pods in `kube-system` namespace:
```bash
kubectl --namespace=kube-system get pods -o json \
| jq -r '
.items[]
| [.spec.nodeName, .metadata.name]
+
(.spec.containers[].image | split(":"))
| @tsv
' \
| column -t
```
]
---
## What version are we running anyway?
- When I say, "I'm running Kubernetes 1.11", is that the version of:
- kubectl
- API server
- kubelet
- controller manager
- something else?
---
## Other versions that are important
- etcd
- kube-dns or CoreDNS
- CNI plugin(s)
- Network controller, network policy controller
- Container engine
- Linux kernel
---
## General guidelines
- To update a component, use whatever was used to install it
- If it's a distro package, update that distro package
- If it's a container or pod, update that container or pod
- If you used configuration management, update with that
---
## Know where your binaries come from
- Sometimes, we need to upgrade *quickly*
(when a vulnerability is announced and patched)
- If we are using an installer, we should:
- make sure it's using upstream packages
- or make sure that whatever packages it uses are current
- make sure we can tell it to pin specific component versions
---
## In practice
- We are going to update a few cluster components
- We will change the kubelet version on one node
- We will change the version of the API server
- We will work with cluster `test` (nodes `test1`, `test2`, `test3`)
---
## Updating kubelet
- These nodes have been installed using the official Kubernetes packages
- We can therefore use `apt` or `apt-get`
.exercise[
- Log into node `test3`
- View available versions for package `kubelet`:
```bash
apt show kubelet -a | grep ^Version
```
- Upgrade kubelet:
```bash
apt install kubelet=1.14.1-00
```
]
---
## Checking what we've done
.exercise[
- Log into node `test1`
- Check node versions:
```bash
kubectl get nodes -o wide
```
- Create a deployment and scale it to make sure that the node still works
]
---
## Updating the API server
- This cluster has been deployed with kubeadm
- The control plane runs in *static pods*
- These pods are started automatically by kubelet
(even when kubelet can't contact the API server)
- They are defined in YAML files in `/etc/kubernetes/manifests`
(this path is set by a kubelet command-line flag)
- kubelet automatically updates the pods when the files are changed
---
## Changing the API server version
- We will edit the YAML file to use a different image version
.exercise[
- Log into node `test1`
- Check API server version:
```bash
kubectl version
```
- Edit the API server pod manifest:
```bash
sudo vim /etc/kubernetes/manifests/kube-apiserver.yaml
```
- Look for the `image:` line, and update it to e.g. `v1.14.0`
]
---
## Checking what we've done
- The API server will be briefly unavailable while kubelet restarts it
.exercise[
- Check the API server version:
```bash
kubectl version
```
]
---
## Updating the whole control plane
- As an example, we'll use kubeadm to upgrade the entire control plane
(note: this is possible only because the cluster was installed with kubeadm)
.exercise[
- Check what will be upgraded:
```bash
sudo kubeadm upgrade plan
```
(Note: kubeadm is confused by our manual upgrade of the API server.
<br/>It thinks the cluster is running 1.14.0!)
<!-- ##VERSION## -->
- Perform the upgrade:
```bash
sudo kubeadm upgrade apply v1.14.1
```
]
---
## Updating kubelets
- After updating the control plane, we need to update each kubelet
- This requires to run a special command on each node, to download the config
(this config is generated by kubeadm)
.exercise[
- Download the configuration on each node, and upgrade kubelet:
```bash
for N in 1 2 3; do
ssh node$N sudo kubeadm upgrade node config --kubelet-version v1.14.1
ssh node $N sudo apt install kubelet=1.14.1-00
done
```
]
---
## Checking what we've done
- All our nodes should now be updated to version 1.14.1
.exercise[
- Check nodes versions:
```bash
kubectl get nodes -o wide
```
]

View File

@@ -1,684 +0,0 @@
# The Container Network Interface
- Allows us to decouple network configuration from Kubernetes
- Implemented by *plugins*
- Plugins are executables that will be invoked by kubelet
- Plugins are responsible for:
- allocating IP addresses for containers
- configuring the network for containers
- Plugins can be combined and chained when it makes sense
---
## Combining plugins
- Interface could be created by e.g. `vlan` or `bridge` plugin
- IP address could be allocated by e.g. `dhcp` or `host-local` plugin
- Interface parameters (MTU, sysctls) could be tweaked by the `tuning` plugin
The reference plugins are available [here].
Look into each plugin's directory for its documentation.
[here]: https://github.com/containernetworking/plugins/tree/master/plugins
---
## How does kubelet know which plugins to use?
- The plugin (or list of plugins) is set in the CNI configuration
- The CNI configuration is a *single file* in `/etc/cni/net.d`
- If there are multiple files in that directory, the first one is used
(in lexicographic order)
- That path can be changed with the `--cni-conf-dir` flag of kubelet
---
## CNI configuration in practice
- When we set up the "pod network" (like Calico, Weave...) it ships a CNI configuration
(and sometimes, custom CNI plugins)
- Very often, that configuration (and plugins) is installed automatically
(by a DaemonSet featuring an initContainer with hostPath volumes)
- Examples:
- Calico [CNI config](https://github.com/projectcalico/calico/blob/1372b56e3bfebe2b9c9cbf8105d6a14764f44159/v2.6/getting-started/kubernetes/installation/hosted/calico.yaml#L25)
and [volume](https://github.com/projectcalico/calico/blob/1372b56e3bfebe2b9c9cbf8105d6a14764f44159/v2.6/getting-started/kubernetes/installation/hosted/calico.yaml#L219)
- kube-router [CNI config](https://github.com/cloudnativelabs/kube-router/blob/c2f893f64fd60cf6d2b6d3fee7191266c0fc0fe5/daemonset/generic-kuberouter.yaml#L10)
and [volume](https://github.com/cloudnativelabs/kube-router/blob/c2f893f64fd60cf6d2b6d3fee7191266c0fc0fe5/daemonset/generic-kuberouter.yaml#L73)
---
## Conf vs conflist
- There are two slightly different configuration formats
- Basic configuration format:
- holds configuration for a single plugin
- typically has a `.conf` name suffix
- has a `type` string field in the top-most structure
- [examples](https://github.com/containernetworking/cni/blob/master/SPEC.md#example-configurations)
- Configuration list format:
- can hold configuration for multiple (chained) plugins
- typically has a `.conflist` name suffix
- has a `plugins` list field in the top-most structure
- [examples](https://github.com/containernetworking/cni/blob/master/SPEC.md#network-configuration-lists)
---
class: extra-details
## How plugins are invoked
- Parameters are given through environment variables, including:
- CNI_COMMAND: desired operation (ADD, DEL, CHECK, or VERSION)
- CNI_CONTAINERID: container ID
- CNI_NETNS: path to network namespace file
- CNI_IFNAME: how the network interface should be named
- The network configuration must be provided to the plugin on stdin
(this avoids race conditions that could happen by passing a file path)
---
## In practice: kube-router
- We are going to set up a new cluster
- For this new cluster, we will use kube-router
- kube-router will provide the "pod network"
(connectivity with pods)
- kube-router will also provide internal service connectivity
(replacing kube-proxy)
---
## How kube-router works
- Very simple architecture
- Does not introduce new CNI plugins
(uses the `bridge` plugin, with `host-local` for IPAM)
- Pod traffic is routed between nodes
(no tunnel, no new protocol)
- Internal service connectivity is implemented with IPVS
- Can provide pod network and/or internal service connectivity
- kube-router daemon runs on every node
---
## What kube-router does
- Connect to the API server
- Obtain the local node's `podCIDR`
- Inject it into the CNI configuration file
(we'll use `/etc/cni/net.d/10-kuberouter.conflist`)
- Obtain the addresses of all nodes
- Establish a *full mesh* BGP peering with the other nodes
- Exchange routes over BGP
---
## What's BGP?
- BGP (Border Gateway Protocol) is the protocol used between internet routers
- It [scales](https://www.cidr-report.org/as2.0/)
pretty [well](https://www.cidr-report.org/cgi-bin/plota?file=%2fvar%2fdata%2fbgp%2fas2.0%2fbgp-active%2etxt&descr=Active%20BGP%20entries%20%28FIB%29&ylabel=Active%20BGP%20entries%20%28FIB%29&with=step)
(it is used to announce the 700k CIDR prefixes of the internet)
- It is spoken by many hardware routers from many vendors
- It also has many software implementations (Quagga, Bird, FRR...)
- Experienced network folks generally know it (and appreciate it)
- It also used by Calico (another popular network system for Kubernetes)
- Using BGP allows us to interconnect our "pod network" with other systems
---
## The plan
- We'll work in a new cluster (named `kuberouter`)
- We will run a simple control plane (like before)
- ... But this time, the controller manager will allocate `podCIDR` subnets
- We will start kube-router with a DaemonSet
- This DaemonSet will start one instance of kube-router on each node
---
## Logging into the new cluster
.exercise[
- Log into node `kuberouter1`
- Clone the workshop repository:
```bash
git clone https://@@GITREPO@@
```
- Move to this directory:
```bash
cd container.training/compose/kube-router-k8s-control-plane
```
]
---
## Our control plane
- We will use a Compose file to start the control plane
- It is similar to the one we used with the `kubenet` cluster
- The API server is started with `--allow-privileged`
(because we will start kube-router in privileged pods)
- The controller manager is started with extra flags too:
`--allocate-node-cidrs` and `--cluster-cidr`
- We need to edit the Compose file to set the Cluster CIDR
---
## Starting the control plane
- Our cluster CIDR will be `10.C.0.0/16`
(where `C` is our cluster number)
.exercise[
- Edit the Compose file to set the Cluster CIDR:
```bash
vim docker-compose.yaml
```
- Start the control plane:
```bash
docker-compose up
```
]
---
## The kube-router DaemonSet
- In the same directory, there is a `kuberouter.yaml` file
- It contains the definition for a DaemonSet and a ConfigMap
- Before we load it, we also need to edit it
- We need to indicate the address of the API server
(because kube-router needs to connect to it to retrieve node information)
---
## Creating the DaemonSet
- The address of the API server will be `http://A.B.C.D:8080`
(where `A.B.C.D` is the address of `kuberouter1`, running the control plane)
.exercise[
- Edit the YAML file to set the API server address:
```bash
vim kuberouter.yaml
```
- Create the DaemonSet:
```bash
kubectl create -f kuberouter.yaml
```
]
Note: the DaemonSet won't create any pods (yet) since there are no nodes (yet).
---
## Generating the kubeconfig for kubelet
- This is similar to what we did for the `kubenet` cluster
.exercise[
- Generate the kubeconfig file (replacing `X.X.X.X` with the address of `kuberouter1`):
```bash
kubectl --kubeconfig ~/kubeconfig config \
set-cluster kubenet --server http://`X.X.X.X`:8080
kubectl --kubeconfig ~/kubeconfig config \
set-context kubenet --cluster kubenet
kubectl --kubeconfig ~/kubeconfig config\
use-context kubenet
```
]
---
## Distributing kubeconfig
- We need to copy that kubeconfig file to the other nodes
.exercise[
- Copy `kubeconfig` to the other nodes:
```bash
for N in 2 3; do
scp ~/kubeconfig kuberouter$N:
done
```
]
---
## Starting kubelet
- We don't need the `--pod-cidr` option anymore
(the controller manager will allocate these automatically)
- We need to pass `--network-plugin=cni`
.exercise[
- Join the first node:
```bash
sudo kubelet --kubeconfig ~/kubeconfig --network-plugin=cni
```
- Open more terminals and join the other nodes:
```bash
ssh kuberouter2 sudo kubelet --kubeconfig ~/kubeconfig --network-plugin=cni
ssh kuberouter3 sudo kubelet --kubeconfig ~/kubeconfig --network-plugin=cni
```
]
---
## Setting up a test
- Let's create a Deployment and expose it with a Service
.exercise[
- Create a Deployment running a web server:
```bash
kubectl create deployment web --image=jpetazzo/httpenv
```
- Scale it so that it spans multiple nodes:
```bash
kubectl scale deployment web --replicas=5
```
- Expose it with a Service:
```bash
kubectl expose deployment web --port=8888
```
]
---
## Checking that everything works
.exercise[
- Get the ClusterIP address for the service:
```bash
kubectl get svc web
```
- Send a few requests there:
```bash
curl `X.X.X.X`:8888
```
]
Note that if you send multiple requests, they are load-balanced in a round robin manner.
This shows that we are using IPVS (vs. iptables, which picked random endpoints).
---
## Troubleshooting
- What if we need to check that everything is working properly?
.exercise[
- Check the IP addresses of our pods:
```bash
kubectl get pods -o wide
```
- Check our routing table:
```bash
route -n
ip route
```
]
We should see the local pod CIDR connected to `kube-bridge`, and the other nodes' pod CIDRs having individual routes, with each node being the gateway.
---
## More troubleshooting
- We can also look at the output of the kube-router pods
(with `kubectl logs`)
- kube-router also comes with a special shell that gives lots of useful info
(we can access it with `kubectl exec`)
- But with the current setup of the cluster, these options may not work!
- Why?
---
## Trying `kubectl logs` / `kubectl exec`
.exercise[
- Try to show the logs of a kube-router pod:
```bash
kubectl -n kube-system logs ds/kube-router
```
- Or try to exec into one of the kube-router pods:
```bash
kubectl -n kube-system exec kuber-router-xxxxx bash
```
]
These commands will give an error message that includes:
```
dial tcp: lookup kuberouterX on 127.0.0.11:53: no such host
```
What does that mean?
---
## Internal name resolution
- To execute these commands, the API server needs to connect to kubelet
- By default, it creates a connection using the kubelet's name
(e.g. `http://kuberouter1:...`)
- This requires our nodes names to be in DNS
- We can change that by setting a flag on the API server:
`--kubelet-preferred-address-types=InternalIP`
---
## Another way to check the logs
- We can also ask the logs directly to the container engine
- First, get the container ID, with `docker ps` or like this:
```bash
CID=$(docker ps
--filter label=io.kubernetes.pod.namespace=kube-system
--filter label=io.kubernetes.container.name=kube-router)
```
- Then view the logs:
```bash
docker logs $CID
```
---
class: extra-details
## Other ways to distribute routing tables
- We don't need kube-router and BGP to distribute routes
- The list of nodes (and associated `podCIDR` subnets) is available through the API
- This shell snippet generates the commands to add all required routes on a node:
```bash
NODES=$(kubectl get nodes -o name | cut -d/ -f2)
for DESTNODE in $NODES; do
if [ "$DESTNODE" != "$HOSTNAME" ]; then
echo $(kubectl get node $DESTNODE -o go-template="
route add -net {{.spec.podCIDR}} gw {{(index .status.addresses 0).address}}")
fi
done
```
- This could be useful for embedded platforms with very limited resources
(or lab environments for learning purposes)
---
# Interconnecting clusters
- We assigned different Cluster CIDRs to each cluster
- This allows us to connect our clusters together
- We will leverage kube-router BGP abilities for that
- We will *peer* each kube-router instance with a *route reflector*
- As a result, we will be able to ping each other's pods
---
## Disclaimers
- There are many methods to interconnect clusters
- Depending on your network implementation, you will use different methods
- The method shown here only works for nodes with direct layer 2 connection
- We will often need to use tunnels or other network techniques
---
## The plan
- Someone will start the *route reflector*
(typically, that will be the person presenting these slides!)
- We will update our kube-router configuration
- We will add a *peering* with the route reflector
(instructing kube-router to connect to it and exchange route information)
- We should see the routes to other clusters on our nodes
(in the output of e.g. `route -n` or `ip route show`)
- We should be able to ping pods of other nodes
---
## Starting the route reflector
- Only do this if you are doing this on your own
- There is a Compose file in the `compose/frr-route-reflector` directory
- Before continuing, make sure that you have the IP address of the route reflector
---
## Configuring kube-router
- This can be done in two ways:
- with command-line flags to the `kube-router` process
- with annotations to Node objects
- We will use the command-line flags
(because it will automatically propagate to all nodes)
.footnote[Note: with Calico, this is achieved by creating a BGPPeer CRD.]
---
## Updating kube-router configuration
- We need to add two command-line flags to the kube-router process
.exercise[
- Edit the `kuberouter.yaml` file
- Add the following flags to the kube-router arguments,:
```
- "--peer-router-ips=`X.X.X.X`"
- "--peer-router-asns=64512"
```
(Replace `X.X.X.X` with the route reflector address)
- Update the DaemonSet definition:
```bash
kubectl apply -f kuberouter.yaml
```
]
---
## Restarting kube-router
- The DaemonSet will not update the pods automatically
(it is using the default `updateStrategy`, which is `OnDelete`)
- We will therefore delete the pods
(they will be recreated with the updated definition)
.exercise[
- Delete all the kube-router pods:
```bash
kubectl delete pods -n kube-system -l k8s-app=kube-router
```
]
Note: the other `updateStrategy` for a DaemonSet is RollingUpdate.
<br/>
For critical services, we might want to precisely control the update process.
---
## Checking peering status
- We can see informative messages in the output of kube-router:
```
time="2019-04-07T15:53:56Z" level=info msg="Peer Up"
Key=X.X.X.X State=BGP_FSM_OPENCONFIRM Topic=Peer
```
- We should see the routes of the other clusters show up
- For debugging purposes, the reflector also exports a route to 1.0.0.2/32
- That route will show up like this:
```
1.0.0.2 172.31.X.Y 255.255.255.255 UGH 0 0 0 eth0
```
- We should be able to ping the pods of other clusters!
---
## If we wanted to do more ...
- kube-router can also export ClusterIP addresses
(by adding the flag `--advertise-cluster-ip`)
- They are exported individually (as /32)
- This would allow us to easily access other clusters' services
(without having to resolve the individual addresses of pods)
- Even better if it's combined with DNS integration
(to facilitate name → ClusterIP resolution)

View File

@@ -130,14 +130,6 @@ class: pic
---
class: pic
![One of the best Kubernetes architecture diagrams available](images/k8s-arch4-thanks-luxas.png)
---
class: extra-details
## Running the control plane on special nodes
- It is common to reserve a dedicated node for the control plane
@@ -160,8 +152,6 @@ class: extra-details
---
class: extra-details
## Running the control plane outside containers
- The services of the control plane can run in or out of containers
@@ -181,8 +171,6 @@ class: extra-details
---
class: extra-details
## Do we need to run Docker at all?
No!
@@ -199,8 +187,6 @@ No!
---
class: extra-details
## Do we need to run Docker at all?
Yes!
@@ -223,8 +209,6 @@ Yes!
---
class: extra-details
## Do we need to run Docker at all?
- On our development environments, CI pipelines ... :
@@ -241,21 +225,25 @@ class: extra-details
---
## Interacting with Kubernetes
## Kubernetes resources
- We will interact with our Kubernetes cluster through the Kubernetes API
- The Kubernetes API defines a lot of objects called *resources*
- The Kubernetes API is (mostly) RESTful
- It allows us to create, read, update, delete *resources*
- These resources are organized by type, or `Kind` (in the API)
- A few common resource types are:
- node (a machine — physical or virtual — in our cluster)
- pod (group of containers running together on a node)
- service (stable network endpoint to connect to one or multiple containers)
- namespace (more-or-less isolated group of things)
- secret (bundle of sensitive data to be passed to a container)
And much more!
- We can see the full list by running `kubectl api-resources`
(In Kubernetes 1.10 and prior, the command to list API resources was `kubectl get`)
---
@@ -265,16 +253,22 @@ class: pic
---
class: pic
![One of the best Kubernetes architecture diagrams available](images/k8s-arch4-thanks-luxas.png)
---
## Credits
- The first diagram is courtesy of Lucas Käldström, in [this presentation](https://speakerdeck.com/luxas/kubeadm-cluster-creation-internals-from-self-hosting-to-upgradability-and-ha)
- it's one of the best Kubernetes architecture diagrams available!
- The second diagram is courtesy of Weave Works
- The first diagram is courtesy of Weave Works
- a *pod* can have multiple containers working together
- IP addresses are associated with *pods*, not with individual containers
- The second diagram is courtesy of Lucas Käldström, in [this presentation](https://speakerdeck.com/luxas/kubeadm-cluster-creation-internals-from-self-hosting-to-upgradability-and-ha)
- it's one of the best Kubernetes architecture diagrams available!
Both diagrams used with permission.

View File

@@ -36,9 +36,7 @@
## Creating a daemon set
<!-- ##VERSION## -->
- Unfortunately, as of Kubernetes 1.14, the CLI cannot create daemon sets
- Unfortunately, as of Kubernetes 1.12, the CLI cannot create daemon sets
--
@@ -73,13 +71,18 @@
- Dump the `rng` resource in YAML:
```bash
kubectl get deploy/rng -o yaml >rng.yml
kubectl get deploy/rng -o yaml --export >rng.yml
```
- Edit `rng.yml`
]
Note: `--export` will remove "cluster-specific" information, i.e.:
- namespace (so that the resource is not tied to a specific namespace)
- status and creation timestamp (useless when creating a new resource)
- resourceVersion and uid (these would cause... *interesting* problems)
---
## "Casting" a resource to another
@@ -249,29 +252,38 @@ The master node has [taints](https://kubernetes.io/docs/concepts/configuration/t
---
## Is this working?
## What are all these pods doing?
- Look at the web UI
- Let's check the logs of all these `rng` pods
- All these pods have a `run=rng` label:
- the first pod, because that's what `kubectl run` does
- the other ones (in the daemon set), because we
*copied the spec from the first one*
- Therefore, we can query everybody's logs using that `run=rng` selector
.exercise[
- Check the logs of all the pods having a label `run=rng`:
```bash
kubectl logs -l run=rng --tail 1
```
]
--
- The graph should now go above 10 hashes per second!
--
- It looks like the newly created pods are serving traffic correctly
- How and why did this happen?
(We didn't do anything special to add them to the `rng` service load balancer!)
It appears that *all the pods* are serving requests at the moment.
---
# Labels and selectors
## The magic of selectors
- The `rng` *service* is load balancing requests to a set of pods
- That set of pods is defined by the *selector* of the `rng` service
- This set of pods is defined as "pods having the label `run=rng`"
.exercise[
@@ -282,333 +294,110 @@ The master node has [taints](https://kubernetes.io/docs/concepts/configuration/t
]
- The selector is `app=rng`
- It means "all the pods having the label `app=rng`"
(They can have additional labels as well, that's OK!)
When we created additional pods with this label, they were
automatically detected by `svc/rng` and added as *endpoints*
to the associated load balancer.
---
## Selector evaluation
- We can use selectors with many `kubectl` commands
- For instance, with `kubectl get`, `kubectl logs`, `kubectl delete` ... and more
.exercise[
- Get the list of pods matching selector `app=rng`:
```bash
kubectl get pods -l app=rng
kubectl get pods --selector app=rng
```
]
But ... why do these pods (in particular, the *new* ones) have this `app=rng` label?
---
## Where do labels come from?
- When we create a deployment with `kubectl create deployment rng`,
<br/>this deployment gets the label `app=rng`
- The replica sets created by this deployment also get the label `app=rng`
- The pods created by these replica sets also get the label `app=rng`
- When we created the daemon set from the deployment, we re-used the same spec
- Therefore, the pods created by the daemon set get the same labels
.footnote[Note: when we use `kubectl run stuff`, the label is `run=stuff` instead.]
---
## Updating load balancer configuration
- We would like to remove a pod from the load balancer
## Removing the first pod from the load balancer
- What would happen if we removed that pod, with `kubectl delete pod ...`?
--
It would be re-created immediately (by the replica set or the daemon set)
The `replicaset` would re-create it immediately.
--
- What would happen if we removed the `app=rng` label from that pod?
- What would happen if we removed the `run=rng` label from that pod?
--
It would *also* be re-created immediately
The `replicaset` would re-create it immediately.
--
Why?!?
---
## Selectors for replica sets and daemon sets
- The "mission" of a replica set is:
"Make sure that there is the right number of pods matching this spec!"
- The "mission" of a daemon set is:
"Make sure that there is a pod matching this spec on each node!"
... Because what matters to the `replicaset` is the number of pods *matching that selector.*
--
- *In fact,* replica sets and daemon sets do not check pod specifications
- They merely have a *selector*, and they look for pods matching that selector
- Yes, we can fool them by manually creating pods with the "right" labels
- Bottom line: if we remove our `app=rng` label ...
... The pod "diseappears" for its parent, which re-creates another pod to replace it
---
class: extra-details
## Isolation of replica sets and daemon sets
- Since both the `rng` daemon set and the `rng` replica set use `app=rng` ...
... Why don't they "find" each other's pods?
- But but but ... Don't we have more than one pod with `run=rng` now?
--
- *Replica sets* have a more specific selector, visible with `kubectl describe`
(It looks like `app=rng,pod-template-hash=abcd1234`)
- *Daemon sets* also have a more specific selector, but it's invisible
(It looks like `app=rng,controller-revision-hash=abcd1234`)
- As a result, each controller only "sees" the pods it manages
The answer lies in the exact selector used by the `replicaset` ...
---
## Removing a pod from the load balancer
## Deep dive into selectors
- Currently, the `rng` service is defined by the `app=rng` selector
- The only way to remove a pod is to remove or change the `app` label
- ... But that will cause another pod to be created instead!
- What's the solution?
--
- We need to change the selector of the `rng` service!
- Let's add another label to that selector (e.g. `enabled=yes`)
---
## Complex selectors
- If a selector specifies multiple labels, they are understood as a logical *AND*
(In other words: the pods must match all the labels)
- Kubernetes has support for advanced, set-based selectors
(But these cannot be used with services, at least not yet!)
---
## The plan
1. Add the label `enabled=yes` to all our `rng` pods
2. Update the selector for the `rng` service to also include `enabled=yes`
3. Toggle traffic to a pod by manually adding/removing the `enabled` label
4. Profit!
*Note: if we swap steps 1 and 2, it will cause a short
service disruption, because there will be a period of time
during which the service selector won't match any pod.
During that time, requests to the service will time out.
By doing things in the order above, we guarantee that there won't
be any interruption.*
---
## Adding labels to pods
- We want to add the label `enabled=yes` to all pods that have `app=rng`
- We could edit each pod one by one with `kubectl edit` ...
- ... Or we could use `kubectl label` to label them all
- `kubectl label` can use selectors itself
- Let's look at the selectors for the `rng` *deployment* and the associated *replica set*
.exercise[
- Add `enabled=yes` to all pods that have `app=rng`:
- Show detailed information about the `rng` deployment:
```bash
kubectl label pods -l app=rng enabled=yes
kubectl describe deploy rng
```
]
---
## Updating the service selector
- We need to edit the service specification
- Reminder: in the service definition, we will see `app: rng` in two places
- the label of the service itself (we don't need to touch that one)
- the selector of the service (that's the one we want to change)
.exercise[
- Update the service to add `enabled: yes` to its selector:
- Show detailed information about the `rng` replica:
<br/>(The second command doesn't require you to get the exact name of the replica set)
```bash
kubectl edit service rng
kubectl describe rs rng-yyyyyyyy
kubectl describe rs -l run=rng
```
<!--
```wait Please edit the object below```
```keys /app: rng```
```keys ^J```
```keys noenabled: yes```
```keys ^[``` ]
```keys :wq```
```keys ^J```
-->
]
--
... And then we get *the weirdest error ever.* Why?
The replica set selector also has a `pod-template-hash`, unlike the pods in our daemon set.
---
## When the YAML parser is being too smart
# Updating a service through labels and selectors
- YAML parsers try to help us:
- What if we want to drop the `rng` deployment from the load balancer?
- `xyz` is the string `"xyz"`
- Option 1:
- `42` is the integer `42`
- destroy it
- `yes` is the boolean value `true`
- Option 2:
- If we want the string `"42"` or the string `"yes"`, we have to quote them
- add an extra *label* to the daemon set
- So we have to use `enabled: "yes"`
- update the service *selector* to refer to that *label*
.footnote[For a good laugh: if we had used "ja", "oui", "si" ... as the value, it would have worked!]
--
Of course, option 2 offers more learning opportunities. Right?
---
## Updating the service selector, take 2
## Add an extra label to the daemon set
.exercise[
- We will update the daemon set "spec"
- Update the service to add `enabled: "yes"` to its selector:
```bash
kubectl edit service rng
```
- Option 1:
<!--
```wait Please edit the object below```
```keys /app: rng```
```keys ^J```
```keys noenabled: "yes"```
```keys ^[``` ]
```keys :wq```
```keys ^J```
-->
- edit the `rng.yml` file that we used earlier
]
- load the new definition with `kubectl apply`
This time it should work!
- Option 2:
If we did everything correctly, the web UI shouldn't show any change.
- use `kubectl edit`
--
*If you feel like you got this💕🌈, feel free to try directly.*
*We've included a few hints on the next slides for your convenience!*
---
## Updating labels
- We want to disable the pod that was created by the deployment
- All we have to do, is remove the `enabled` label from that pod
- To identify that pod, we can use its name
- ... Or rely on the fact that it's the only one with a `pod-template-hash` label
- Good to know:
- `kubectl label ... foo=` doesn't remove a label (it sets it to an empty string)
- to remove label `foo`, use `kubectl label ... foo-`
- to change an existing label, we would need to add `--overwrite`
---
## Removing a pod from the load balancer
.exercise[
- In one window, check the logs of that pod:
```bash
POD=$(kubectl get pod -l app=rng,pod-template-hash -o name)
kubectl logs --tail 1 --follow $POD
```
(We should see a steady stream of HTTP logs)
- In another window, remove the label from the pod:
```bash
kubectl label pod -l app=rng,pod-template-hash enabled-
```
(The stream of HTTP logs should stop immediately)
]
There might be a slight change in the web UI (since we removed a bit
of capacity from the `rng` service). If we remove more pods,
the effect should be more visible.
---
class: extra-details
## Updating the daemon set
- If we scale up our cluster by adding new nodes, the daemon set will create more pods
- These pods won't have the `enabled=yes` label
- If we want these pods to have that label, we need to edit the daemon set spec
- We can do that with e.g. `kubectl edit daemonset rng`
---
class: extra-details
## We've put resources in your resources
- Reminder: a daemon set is a resource that creates more resources!
@@ -621,9 +410,7 @@ class: extra-details
- the label(s) of the resource(s) created by the first resource (in the `template` block)
- We would need to update the selector and the template
(metadata labels are not mandatory)
- You need to update the selector and the template (metadata labels are not mandatory)
- The template must match the selector
@@ -631,6 +418,175 @@ class: extra-details
---
## Adding our label
- Let's add a label `isactive: yes`
- In YAML, `yes` should be quoted; i.e. `isactive: "yes"`
.exercise[
- Update the daemon set to add `isactive: "yes"` to the selector and template label:
```bash
kubectl edit daemonset rng
```
<!--
```wait Please edit the object below```
```keys /run: rng```
```keys ^J```
```keys noisactive: "yes"```
```keys ^[``` ]
```keys /run: rng```
```keys ^J```
```keys oisactive: "yes"```
```keys ^[``` ]
```keys :wq```
```keys ^J```
-->
- Update the service to add `isactive: "yes"` to its selector:
```bash
kubectl edit service rng
```
<!--
```wait Please edit the object below```
```keys /run: rng```
```keys ^J```
```keys noisactive: "yes"```
```keys ^[``` ]
```keys :wq```
```keys ^J```
-->
]
---
## Checking what we've done
.exercise[
- Check the most recent log line of all `run=rng` pods to confirm that exactly one per node is now active:
```bash
kubectl logs -l run=rng --tail 1
```
]
The timestamps should give us a hint about how many pods are currently receiving traffic.
.exercise[
- Look at the pods that we have right now:
```bash
kubectl get pods
```
]
---
## Cleaning up
- The pods of the deployment and the "old" daemon set are still running
- We are going to identify them programmatically
.exercise[
- List the pods with `run=rng` but without `isactive=yes`:
```bash
kubectl get pods -l run=rng,isactive!=yes
```
- Remove these pods:
```bash
kubectl delete pods -l run=rng,isactive!=yes
```
]
---
## Cleaning up stale pods
```
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
rng-54f57d4d49-7pt82 1/1 Terminating 0 51m
rng-54f57d4d49-vgz9h 1/1 Running 0 22s
rng-b85tm 1/1 Terminating 0 39m
rng-hfbrr 1/1 Terminating 0 39m
rng-vplmj 1/1 Running 0 7m
rng-xbpvg 1/1 Running 0 7m
[...]
```
- The extra pods (noted `Terminating` above) are going away
- ... But a new one (`rng-54f57d4d49-vgz9h` above) was restarted immediately!
--
- Remember, the *deployment* still exists, and makes sure that one pod is up and running
- If we delete the pod associated to the deployment, it is recreated automatically
---
## Deleting a deployment
.exercise[
- Remove the `rng` deployment:
```bash
kubectl delete deployment rng
```
]
--
- The pod that was created by the deployment is now being terminated:
```
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
rng-54f57d4d49-vgz9h 1/1 Terminating 0 4m
rng-vplmj 1/1 Running 0 11m
rng-xbpvg 1/1 Running 0 11m
[...]
```
Ding, dong, the deployment is dead! And the daemon set lives on.
---
## Avoiding extra pods
- When we changed the definition of the daemon set, it immediately created new pods. We had to remove the old ones manually.
- How could we have avoided this?
--
- By adding the `isactive: "yes"` label to the pods before changing the daemon set!
- This can be done programmatically with `kubectl patch`:
```bash
PATCH='
metadata:
labels:
isactive: "yes"
'
kubectl get pods -l run=rng -l controller-revision-hash -o name |
xargs kubectl patch -p "$PATCH"
```
---
## Labels and debugging
- When a pod is misbehaving, we can delete it: another one will be recreated

View File

@@ -2,60 +2,88 @@
- Kubernetes resources can also be viewed with a web dashboard
- That dashboard is usually exposed over HTTPS
- We are going to deploy that dashboard with *three commands:*
(this requires obtaining a proper TLS certificate)
1) actually *run* the dashboard
- Dashboard users need to authenticate
2) bypass SSL for the dashboard
- We are going to take a *dangerous* shortcut
3) bypass authentication for the dashboard
---
--
## The insecure method
There is an additional step to make the dashboard available from outside (we'll get to that)
- We could (and should) use [Let's Encrypt](https://letsencrypt.org/) ...
- ... but we don't want to deal with TLS certificates
- We could (and should) learn how authentication and authorization work ...
- ... but we will use a guest account with admin access instead
--
.footnote[.warning[Yes, this will open our cluster to all kinds of shenanigans. Don't do this at home.]]
---
## Running a very insecure dashboard
## 1) Running the dashboard
- We are going to deploy that dashboard with *one single command*
- We need to create a *deployment* and a *service* for the dashboard
- This command will create all the necessary resources
- But also a *secret*, a *service account*, a *role* and a *role binding*
(the dashboard itself, the HTTP wrapper, the admin/guest account)
- All these resources are defined in a YAML file
- All we have to do is load that YAML file with with `kubectl apply -f`
- All these things can be defined in a YAML file and created with `kubectl apply -f`
.exercise[
- Create all the dashboard resources, with the following command:
```bash
kubectl apply -f ~/container.training/k8s/insecure-dashboard.yaml
kubectl apply -f ~/container.training/k8s/kubernetes-dashboard.yaml
```
]
---
## 2) Bypassing SSL for the dashboard
- The Kubernetes dashboard uses HTTPS, but we don't have a certificate
- Recent versions of Chrome (63 and later) and Edge will refuse to connect
(You won't even get the option to ignore a security warning!)
- We could (and should!) get a certificate, e.g. with [Let's Encrypt](https://letsencrypt.org/)
- ... But for convenience, for this workshop, we'll forward HTTP to HTTPS
.warning[Do not do this at home, or even worse, at work!]
---
## Running the SSL unwrapper
- We are going to run [`socat`](http://www.dest-unreach.org/socat/doc/socat.html), telling it to accept TCP connections and relay them over SSL
- Then we will expose that `socat` instance with a `NodePort` service
- For convenience, these steps are neatly encapsulated into another YAML file
.exercise[
- Apply the convenient YAML file, and defeat SSL protection:
```bash
kubectl apply -f ~/container.training/k8s/socat.yaml
```
]
.warning[All our dashboard traffic is now clear-text, including passwords!]
---
## Connecting to the dashboard
.exercise[
- Check which port the dashboard is on:
```bash
kubectl get svc dashboard
kubectl -n kube-system get svc socat
```
]
@@ -85,7 +113,26 @@ The dashboard will then ask you which authentication you want to use.
- "skip" (use the dashboard "service account")
- Let's use "skip": we're logged in!
- Let's use "skip": we get a bunch of warnings and don't see much
---
## 3) Bypass authentication for the dashboard
- The dashboard documentation [explains how to do this](https://github.com/kubernetes/dashboard/wiki/Access-control#admin-privileges)
- We just need to load another YAML file!
.exercise[
- Grant admin privileges to the dashboard so we can see our resources:
```bash
kubectl apply -f ~/container.training/k8s/grant-admin-to-dashboard.yaml
```
- Reload the dashboard and enjoy!
]
--
@@ -93,11 +140,73 @@ The dashboard will then ask you which authentication you want to use.
---
## Exposing the dashboard over HTTPS
- We took a shortcut by forwarding HTTP to HTTPS inside the cluster
- Let's expose the dashboard over HTTPS!
- The dashboard is exposed through a `ClusterIP` service (internal traffic only)
- We will change that into a `NodePort` service (accepting outside traffic)
.exercise[
- Edit the service:
```
kubectl edit service kubernetes-dashboard
```
]
--
`NotFound`?!? Y U NO WORK?!?
---
## Editing the `kubernetes-dashboard` service
- If we look at the [YAML](https://github.com/jpetazzo/container.training/blob/master/k8s/kubernetes-dashboard.yaml) that we loaded before, we'll get a hint
--
- The dashboard was created in the `kube-system` namespace
--
.exercise[
- Edit the service:
```bash
kubectl -n kube-system edit service kubernetes-dashboard
```
- Change type `type:` from `ClusterIP` to `NodePort`, save, and exit
<!--
```wait Please edit the object below```
```keys /ClusterIP```
```keys ^J```
```keys cwNodePort```
```keys ^[ ``` ]
```keys :wq```
```keys ^J```
-->
- Check the port that was assigned with `kubectl -n kube-system get services`
- Connect to https://oneofournodes:3xxxx/ (yes, https)
]
---
## Running the Kubernetes dashboard securely
- The steps that we just showed you are *for educational purposes only!*
- If you do that on your production cluster, people [can and will abuse it](https://redlock.io/blog/cryptojacking-tesla)
- If you do that on your production cluster, people [can and will abuse it](https://blog.redlock.io/cryptojacking-tesla)
- For an in-depth discussion about securing the dashboard,
<br/>

View File

@@ -1,20 +1,6 @@
## Declarative vs imperative in Kubernetes
- With Kubernetes, we cannot say: "run this container"
- All we can do is write a *spec* and push it to the API server
(by creating a resource like e.g. a Pod or a Deployment)
- The API server will validate that spec (and reject it if it's invalid)
- Then it will store it in etcd
- A *controller* will "notice" that spec and act upon it
---
## Reconciling state
- Virtually everything we create in Kubernetes is created from a *spec*
- Watch for the `spec` fields in the YAML files later!

View File

@@ -1,837 +0,0 @@
# Building our own cluster
- Let's build our own cluster!
*Perfection is attained not when there is nothing left to add, but when there is nothing left to take away. (Antoine de Saint-Exupery)*
- Our goal is to build a minimal cluster allowing us to:
- create a Deployment (with `kubectl run` or `kubectl create deployment`)
- expose it with a Service
- connect to that service
- "Minimal" here means:
- smaller number of components
- smaller number of command-line flags
- smaller number of configuration files
---
## Non-goals
- For now, we don't care about security
- For now, we don't care about scalability
- For now, we don't care about high availability
- All we care about is *simplicity*
---
## Our environment
- We will use the machine indicated as `dmuc1`
(this stands for "Dessine Moi Un Cluster" or "Draw Me A Sheep",
<br/>in homage to Saint-Exupery's "The Little Prince")
- This machine:
- runs Ubuntu LTS
- has Kubernetes, Docker, and etcd binaries installed
- but nothing is running
---
## Checking our environment
- Let's make sure we have everything we need first
.exercise[
- Log into the `dmuc1` machine
- Get root:
```bash
sudo -i
```
- Check available versions:
```bash
etcd -version
kube-apiserver --version
dockerd --version
```
]
---
## The plan
1. Start API server
2. Interact with it (create Deployment and Service)
3. See what's broken
4. Fix it and go back to step 2 until it works!
---
## Dealing with multiple processes
- We are going to start many processes
- Depending on what you're comfortable with, you can:
- open multiple windows and multiple SSH connections
- use a terminal multiplexer like screen or tmux
- put processes in the background with `&`
<br/>(warning: log output might get confusing to read!)
---
## Starting API server
.exercise[
- Try to start the API server:
```bash
kube-apiserver
# It will fail with "--etcd-servers must be specified"
```
]
Since the API server stores everything in etcd,
it cannot start without it.
---
## Starting etcd
.exercise[
- Try to start etcd:
```bash
etcd
```
]
Success!
Note the last line of output:
```
serving insecure client requests on 127.0.0.1:2379, this is strongly discouraged!
```
*Sure, that's discouraged. But thanks for telling us the address!*
---
## Starting API server (for real)
- Try again, passing the `--etcd-servers` argument
- That argument should be a comma-separated list of URLs
.exercise[
- Start API server:
```bash
kube-apiserver --etcd-servers http://127.0.0.1:2379
```
]
Success!
---
## Interacting with API server
- Let's try a few "classic" commands
.exercise[
- List nodes:
```bash
kubectl get nodes
```
- List services:
```bash
kubectl get services
```
]
So far, so good.
Note: the API server automatically created the `kubernetes` service entry.
---
class: extra-details
## What about `kubeconfig`?
- We didn't need to create a `kubeconfig` file
- By default, the API server is listening on `localhost:8080`
(without requiring authentication)
- By default, `kubectl` connects to `localhost:8080`
(without providing authentication)
---
## Creating a Deployment
- Let's run a web server!
.exercise[
- Create a Deployment with NGINX:
```bash
kubectl create deployment web --image=nginx
```
]
Success?
---
## Checking our Deployment status
.exercise[
- Look at pods, deployments, etc.:
```bash
kubectl get all
```
]
Our Deployment is in a bad shape:
```
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/web 0/1 0 0 2m26s
```
And, there is no ReplicaSet, and no Pod.
---
## What's going on?
- We stored the definition of our Deployment in etcd
(through the API server)
- But there is no *controller* to do the rest of the work
- We need to start the *controller manager*
---
## Starting the controller manager
.exercise[
- Try to start the controller manager:
```bash
kube-controller-manager
```
]
The final error message is:
```
invalid configuration: no configuration has been provided
```
But the logs include another useful piece of information:
```
Neither --kubeconfig nor --master was specified.
Using the inClusterConfig. This might not work.
```
---
## Reminder: everyone talks to API server
- The controller manager needs to connect to the API server
- It *does not* have a convenient `localhost:8080` default
- We can pass the connection information in two ways:
- `--master` and a host:port combination (easy)
- `--kubeconfig` and a `kubeconfig` file
- For simplicity, we'll use the first option
---
## Starting the controller manager (for real)
.exercise[
- Start the controller manager:
```bash
kube-controller-manager --master http://localhost:8080
```
]
Success!
---
## Checking our Deployment status
.exercise[
- Check all our resources again:
```bash
kubectl get all
```
]
We now have a ReplicaSet.
But we still don't have a Pod.
---
## What's going on?
In the controller manager logs, we should see something like this:
```
E0404 15:46:25.753376 22847 replica_set.go:450] Sync "default/web-5bc9bd5b8d"
failed with `No API token found for service account "default"`, retry after the
token is automatically created and added to the service account
```
- The service account `default` was automatically added to our Deployment
(and to its pods)
- The service account `default` exists
- But it doesn't have an associated token
(the token is a secret; creating it requires signature; therefore a CA)
---
## Solving the missing token issue
There are many ways to solve that issue.
We are going to list a few (to get an idea of what's happening behind the scenes).
Of course, we don't need to perform *all* the solutions mentioned here.
---
## Option 1: disable service accounts
- Restart the API server with
`--disable-admission-plugins=ServiceAccount`
- The API server will no longer add a service account automatically
- Our pods will be created without a service account
---
## Option 2: do not mount the (missing) token
- Add `automountServiceAccountToken: false` to the Deployment spec
*or*
- Add `automountServiceAccountToken: false` to the default ServiceAccount
- The ReplicaSet controller will no longer create pods referencing the (missing) token
.exercise[
- Programmatically change the `default` ServiceAccount:
```bash
kubectl patch sa default -p "automountServiceAccountToken: false"
```
]
---
## Option 3: set up service accounts properly
- This is the most complex option!
- Generate a key pair
- Pass the private key to the controller manager
(to generate and sign tokens)
- Pass the public key to the API server
(to verify these tokens)
---
## Continuing without service account token
- Once we patch the default service account, the ReplicaSet can create a Pod
.exercise[
- Check that we now have a pod:
```bash
kubectl get all
```
]
Note: we might have to wait a bit for the ReplicaSet controller to retry.
If we're impatient, we can restart the controller manager.
---
## What's next?
- Our pod exists, but it is in `Pending` state
- Remember, we don't have a node so far
(`kubectl get nodes` shows an empty list)
- We need to:
- start a container engine
- start kubelet
---
## Starting a container engine
- We're going to use Docker (because it's the default option)
.exercise[
- Start the Docker Engine:
```bash
dockerd
```
]
Success!
Feel free to check that it actually works with e.g.:
```bash
docker run alpine echo hello world
```
---
## Starting kubelet
- If we start kubelet without arguments, it *will* start
- But it will not join the cluster!
- It will start in *standalone* mode
- Just like with the controller manager, we need to tell kubelet where the API server is
- Alas, kubelet doesn't have a simple `--master` option
- We have to use `--kubeconfig`
- We need to write a `kubeconfig` file for kubelet
---
## Writing a kubeconfig file
- We can copy/paste a bunch of YAML
- Or we can generate the file with `kubectl`
.exercise[
- Create the file `kubeconfig.kubelet` with `kubectl`:
```bash
kubectl --kubeconfig kubeconfig.kubelet config \
set-cluster localhost --server http://localhost:8080
kubectl --kubeconfig kubeconfig.kubelet config \
set-context localhost --cluster localhost
kubectl --kubeconfig kubeconfig.kubelet config \
use-context localhost
```
]
---
## All Kubernetes clients can use `kubeconfig`
- The `kubeconfig.kubelet` file has the same format as e.g. `~/.kubeconfig`
- All Kubernetes clients can use a similar file
- The `kubectl config` commands can be used to manipulate these files
- This highlights that kubelet is a "normal" client of the API server
---
## Our `kubeconfig.kubelet` file
The file that we generated looks like the one below.
That one has been slightly simplified (removing extraneous fields), but it is still valid.
```yaml
apiVersion: v1
kind: Config
current-context: localhost
contexts:
- name: localhost
context:
cluster: localhost
clusters:
- name: localhost
cluster:
server: http://localhost:8080
```
---
## Starting kubelet
.exercise[
- Start kubelet with that `kubeconfig.kubelet` file:
```bash
kubelet --kubeconfig kubeconfig.kubelet
```
]
Success!
---
## Looking at our 1-node cluster
- Let's check that our node registered correctly
.exercise[
- List the nodes in our cluster:
```bash
kubectl get nodes
```
]
Our node should show up.
Its name will be its hostname (it should be `dmuc1`).
---
## Are we there yet?
- Let's check if our pod is running
.exercise[
- List all resources:
```bash
kubectl get all
```
]
--
Our pod is still `Pending`. 🤔
--
Which is normal: it needs to be *scheduled*.
(i.e., something needs to decide on which node it should go.)
---
## Scheduling our pod
- Why do we need a scheduling decision, since we have only one node?
- The node might be full, unavailable; the pod might have constraints ...
- The easiest way to schedule our pod is to start the scheduler
(we could also schedule it manually)
---
## Starting the scheduler
- The scheduler also needs to know how to connect to the API server
- Just like for controller manager, we can use `--kubeconfig` or `--master`
.exercise[
- Start the scheduler:
```bash
kube-scheduler --master http://localhost:8080
```
]
- Our pod should now start correctly
---
## Checking the status of our pod
- Our pod will go through a short `ContainerCreating` phase
- Then it will be `Running`
.exercise[
- Check pod status:
```bash
kubectl get pods
```
]
Success!
---
class: extra-details
## Scheduling a pod manually
- We can schedule a pod in `Pending` state by creating a Binding, e.g.:
```bash
kubectl create -f- <<EOF
apiVersion: v1
kind: Binding
metadata:
name: name-of-the-pod
target:
apiVersion: v1
kind: Node
name: name-of-the-node
EOF
```
- This is actually how the scheduler works!
- It watches pods, takes scheduling decisions, creates Binding objects
---
## Connecting to our pod
- Let's check that our pod correctly runs NGINX
.exercise[
- Check our pod's IP address:
```bash
kubectl get pods -o wide
```
- Send some HTTP request to the pod:
```bash
curl `X.X.X.X`
```
]
We should see the `Welcome to nginx!` page.
---
## Exposing our Deployment
- We can now create a Service associated to this Deployment
.exercise[
- Expose the Deployment's port 80:
```bash
kubectl expose deployment web --port=80
```
- Check the Service's ClusterIP, and try connecting:
```bash
kubectl get service web
curl http://`X.X.X.X`
```
]
--
This won't work. We need kube-proxy to enable internal communication.
---
## Starting kube-proxy
- kube-proxy also needs to connect to API server
- It can work with the `--master` flag
(even though that will be deprecated in the future)
.exercise[
- Start kube-proxy:
```bash
kube-proxy --master http://localhost:8080
```
]
---
## Connecting to our Service
- Now that kube-proxy is running, we should be able to connect
.exercise[
- Check the Service's ClusterIP again, and retry connecting:
```bash
kubectl get service web
curl http://`X.X.X.X`
```
]
Success!
---
class: extra-details
## How kube-proxy works
- kube-proxy watches Service resources
- When a Service is created or updated, kube-proxy creates iptables rules
.exercise[
- Check out the `OUTPUT` chain in the `nat` table:
```bash
iptables -t nat -L OUTPUT
```
- Traffic is sent to `KUBE-SERVICES`; check that too:
```bash
iptables -t nat -L KUBE-SERVICES
```
]
For each Service, there is an entry in that chain.
---
class: extra-details
## Diving into iptables
- The last command showed a chain named `KUBE-SVC-...` corresponding to our service
.exercise[
- Check that `KUBE-SVC-...` chain:
```bash
iptables -t nat -L `KUBE-SVC-...`
```
- It should show a jump to a `KUBE-SEP-...` chains; check it out too:
```bash
iptables -t nat -L `KUBE-SEP-...`
```
]
This is a `DNAT` rule to rewrite the destination address of the connection to our pod.
This is how kube-proxy works!
---
class: extra-details
## kube-router, IPVS
- With recent versions of Kubernetes, it is possible to tell kube-proxy to use IPVS
- IPVS is a more powerful load balancing framework
(remember: iptables was primarily designed for firewalling, not load balancing!)
- It is also possible to replace kube-proxy with kube-router
- kube-router uses IPVS by default
- kube-router can also perform other functions
(e.g., we can use it as a CNI plugin to provide pod connectivity)
---
class: extra-details
## What about the `kubernetes` service?
- If we try to connect, it won't work
(by default, it should be `10.0.0.1`)
- If we look at the Endpoints for this service, we will see one endpoint:
`host-address:6443`
- By default, the API server expects to be running directly on the nodes
(it could be as a bare process, or in a container/pod using host network)
- ... And it expects to be listening on port 6443 with TLS

View File

@@ -1,216 +0,0 @@
# Extending the Kubernetes API
There are multiple ways to extend the Kubernetes API.
We are going to cover:
- Custom Resource Definitions (CRDs)
- Admission Webhooks
---
## Revisiting the API server
- The Kubernetes API server is a central point of the control plane
(everything connects to it: controller manager, scheduler, kubelets)
- Almost everything in Kubernetes is materialized by a resource
- Resources have a type (or "kind")
(similar to strongly typed languages)
- We can see existing types with `kubectl api-resources`
- We can list resources of a given type with `kubectl get <type>`
---
## Creating new types
- We can create new types with Custom Resource Definitions (CRDs)
- CRDs are created dynamically
(without recompiling or restarting the API server)
- CRDs themselves are resources:
- we can create a new type with `kubectl create` and some YAML
- we can see all our custom types with `kubectl get crds`
- After we create a CRD, the new type works just like built-in types
---
## What can we do with CRDs?
There are many possibilities!
- *Operators* encapsulate complex sets of resources
(e.g.: a PostgreSQL replicated cluster; an etcd cluster...
<br/>
see [awesome operators](https://github.com/operator-framework/awesome-operators) and
[OperatorHub](https://operatorhub.io/) to find more)
- Custom use-cases like [gitkube](https://gitkube.sh/)
- creates a new custom type, `Remote`, exposing a git+ssh server
- deploy by pushing YAML or Helm Charts to that remote
- Replacing built-in types with CRDs
(see [this lightning talk by Tim Hockin](https://www.youtube.com/watch?v=ji0FWzFwNhA&index=2&list=PLj6h78yzYM2PZf9eA7bhWnIh_mK1vyOfU))
---
## Little details
- By default, CRDs are not *validated*
(we can put anything we want in the `spec`)
- When creating a CRD, we can pass an OpenAPI v3 schema (BETA!)
(which will then be used to validate resources)
- Generally, when creating a CRD, we also want to run a *controller*
(otherwise nothing will happen when we create resources of that type)
- The controller will typically *watch* our custom resources
(and take action when they are created/updated)
*Example: [YAML to install the gitkube CRD](https://storage.googleapis.com/gitkube/gitkube-setup-stable.yaml)*
---
## Service catalog
- *Service catalog* is another extension mechanism
- It's not extending the Kubernetes API strictly speaking
(but it still provides new features!)
- It doesn't create new types; it uses:
- ClusterServiceBroker
- ClusterServiceClass
- ClusterServicePlan
- ServiceInstance
- ServiceBinding
- It uses the Open service broker API
---
## Admission controllers
- When a Pod is created, it is associated to a ServiceAccount
(even if we did not specify one explicitly)
- That ServiceAccount was added on the fly by an *admission controller*
(specifically, a *mutating admission controller*)
- Admission controllers sit on the API request path
(see the cool diagram on next slide, courtesy of Banzai Cloud)
---
class: pic
![API request lifecycle](images/api-request-lifecycle.png)
---
## Admission controllers
- *Validating* admission controllers can accept/reject the API call
- *Mutating* admission controllers can modify the API request payload
- Both types can also trigger additional actions
(e.g. automatically create a Namespace if it doesn't exist)
- There are a number of built-in admission controllers
(see [documentation](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#what-does-each-admission-controller-do) for a list)
- But we can also define our own!
---
## Admission Webhooks
- We can setup *admission webhooks* to extend the behavior of the API server
- The API server will submit incoming API requests to these webhooks
- These webhooks can be *validating* or *mutating*
- Webhooks can be setup dynamically (without restarting the API server)
- To setup a dynamic admission webhook, we create a special resource:
a `ValidatingWebhookConfiguration` or a `MutatingWebhookConfiguration`
- These resources are created and managed like other resources
(i.e. `kubectl create`, `kubectl get` ...)
---
## Webhook Configuration
- A ValidatingWebhookConfiguration or MutatingWebhookConfiguration contains:
- the address of the webhook
- the authentication information to use with the webhook
- a list of rules
- The rules indicate for which objects and actions the webhook is triggered
(to avoid e.g. triggering webhooks when setting up webhooks)
---
## (Ab)using the API server
- If we need to store something "safely" (as in: in etcd), we can use CRDs
- This gives us primitives to read/write/list objects (and optionally validate them)
- The Kubernetes API server can run on its own
(without the scheduler, controller manager, and kubelets)
- By loading CRDs, we can have it manage totally different objects
(unrelated to containers, clusters, etc.)
---
## Documentation
- [Custom Resource Definitions: when to use them](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/)
- [Custom Resources Definitions: how to use them](https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/)
- [Service Catalog](https://kubernetes.io/docs/concepts/extend-kubernetes/service-catalog/)
- [Built-in Admission Controllers](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/)
- [Dynamic Admission Controllers](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/)

View File

@@ -111,7 +111,7 @@
- Display that key:
```
kubectl logs deployment/flux | grep identity
kubectl logs deployment flux | grep identity
```
- Then add that key to the repository, giving it **write** access

View File

@@ -164,21 +164,6 @@ The chart's metadata includes an URL to the project's home page.
---
## Viewing installed charts
- Helm keeps track of what we've installed
.exercise[
- List installed Helm charts:
```bash
helm list
```
]
---
## Creating a chart
- We are going to show a way to create a *very simplified* chart

View File

@@ -176,7 +176,7 @@
- We are going to use a Daemon Set so that each node can accept connections
- We will do two minor changes to the [YAML provided by Traefik](https://github.com/containous/traefik/blob/v1.7/examples/k8s/traefik-ds.yaml):
- We will do two minor changes to the [YAML provided by Traefik](https://github.com/containous/traefik/blob/master/examples/k8s/traefik-ds.yaml):
- enable `hostNetwork`
@@ -194,7 +194,7 @@
- When deploying with `kubeadm`:
- a taint is placed on the node dedicated to the control plane
- a taint is placed on the node dedicated the control plane
- the pods running the control plane have a matching toleration
@@ -306,9 +306,9 @@ This one is a special case that means "ignore all taints and run anyway."
- We provide a YAML file (`k8s/traefik.yaml`) which is essentially the sum of:
- [Traefik's Daemon Set resources](https://github.com/containous/traefik/blob/v1.7/examples/k8s/traefik-ds.yaml) (patched with `hostNetwork` and tolerations)
- [Traefik's Daemon Set resources](https://github.com/containous/traefik/blob/master/examples/k8s/traefik-ds.yaml) (patched with `hostNetwork` and tolerations)
- [Traefik's RBAC rules](https://github.com/containous/traefik/blob/v1.7/examples/k8s/traefik-rbac.yaml) allowing it to watch necessary API objects
- [Traefik's RBAC rules](https://github.com/containous/traefik/blob/master/examples/k8s/traefik-rbac.yaml) allowing it to watch necessary API objects
.exercise[
@@ -364,8 +364,6 @@ This is normal: we haven't provided any ingress rule yet.
- Go to `http://node1:8080` (replacing `node1` with its IP address)
<!-- ```open http://node1:8080``` -->
]
---
@@ -394,9 +392,9 @@ This is normal: we haven't provided any ingress rule yet.
- Run all three deployments:
```bash
kubectl create deployment cheddar --image=errm/cheese:cheddar
kubectl create deployment stilton --image=errm/cheese:stilton
kubectl create deployment wensleydale --image=errm/cheese:wensleydale
kubectl run cheddar --image=errm/cheese:cheddar
kubectl run stilton --image=errm/cheese:stilton
kubectl run wensleydale --image=errm/cheese:wensleydale
```
- Create a service for each of them:

View File

@@ -57,49 +57,31 @@ Under the hood: `kube-proxy` is using a userland proxy and a bunch of `iptables`
- Since `ping` doesn't have anything to connect to, we'll have to run something else
- We could use the `nginx` official image, but ...
... we wouldn't be able to tell the backends from each other!
- We are going to use `jpetazzo/httpenv`, a tiny HTTP server written in Go
- `jpetazzo/httpenv` listens on port 8888
- It serves its environment variables in JSON format
- The environment variables will include `HOSTNAME`, which will be the pod name
(and therefore, will be different on each backend)
---
## Creating a deployment for our HTTP server
- We *could* do `kubectl run httpenv --image=jpetazzo/httpenv` ...
- But since `kubectl run` is being deprecated, let's see how to use `kubectl create` instead
.exercise[
- In another window, watch the pods (to see when they will be created):
- Start a bunch of HTTP servers:
```bash
kubectl run httpenv --image=jpetazzo/httpenv --replicas=10
```
- Watch them being started:
```bash
kubectl get pods -w
```
<!-- ```keys ^C``` -->
- Create a deployment for this very lightweight HTTP server:
```bash
kubectl create deployment httpenv --image=jpetazzo/httpenv
```
- Scale it to 10 replicas:
```bash
kubectl scale deployment httpenv --replicas=10
```
<!--
```wait httpenv-```
```keys ^C```
-->
]
The `jpetazzo/httpenv` image runs an HTTP server on port 8888.
<br/>
It serves its environment variables in JSON format.
The `-w` option "watches" events happening on the specified resources.
---
## Exposing our deployment
@@ -110,12 +92,12 @@ Under the hood: `kube-proxy` is using a userland proxy and a bunch of `iptables`
- Expose the HTTP port of our server:
```bash
kubectl expose deployment httpenv --port 8888
kubectl expose deploy/httpenv --port 8888
```
- Look up which IP address was allocated:
```bash
kubectl get service
kubectl get svc
```
]
@@ -255,7 +237,7 @@ class: extra-details
- These IP addresses should match the addresses of the corresponding pods:
```bash
kubectl get pods -l app=httpenv -o wide
kubectl get pods -l run=httpenv -o wide
```
---

View File

@@ -79,106 +79,26 @@
---
class: extra-details
## What's available?
## Exploring types and definitions
- `kubectl` has pretty good introspection facilities
- We can list all available resource types by running `kubectl api-resources`
<br/>
(In Kubernetes 1.10 and prior, this command used to be `kubectl get`)
- We can view details about a resource with:
```bash
kubectl describe type/name
kubectl describe type name
```
- We can view the definition for a resource type with:
```bash
kubectl explain type
```
- We can view the definition of a field in a resource, for instance:
```bash
kubectl explain node.spec
```
- Or get the full definition of all fields and sub-fields:
```bash
kubectl explain node --recursive
```
---
class: extra-details
## Introspection vs. documentation
- We can access the same information by reading the [API documentation](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.14/)
- The API documentation is usually easier to read, but:
- it won't show custom types (like Custom Resource Definitions)
- we need to make sure that we look at the correct version
- `kubectl api-resources` and `kubectl explain` perform *introspection*
(they communicate with the API server and obtain the exact type definitions)
---
## Type names
- The most common resource names have three forms:
- singular (e.g. `node`, `service`, `deployment`)
- plural (e.g. `nodes`, `services`, `deployments`)
- short (e.g. `no`, `svc`, `deploy`)
- Some resources do not have a short names
- `Endpoints` only have a plural form
(because even a single `Endpoints` resource is actually a list of endpoints)
---
## Viewing details
- We can use `kubectl get -o yaml` to see all available details
- However, YAML output is often simultaneously too much and not enough
- For instance, `kubectl get node node1 -o yaml` is:
- too much information (e.g.: list of images available on this node)
- not enough information (e.g.: doesn't show pods running on this node)
- difficult to read for a human operator
- For a comprehensive overview, we can use `kubectl describe` instead
---
## `kubectl describe`
- `kubectl describe` needs a resource type and (optionally) a resource name
- It is possible to provide a resource name *prefix*
(all matching objects will be displayed)
- `kubectl describe` will retrieve some extra information about the resource
.exercise[
- Look at the information available for `node1` with one of the following commands:
```bash
kubectl describe node/node1
kubectl describe node node1
```
]
(We should notice a bunch of control plane pods.)
Each time, `type` can be singular, plural, or abbreviated type name.
---
@@ -250,7 +170,7 @@ The error that we see is expected: the Kubernetes API requires authentication.
--
*Where are the pods that we saw just a moment earlier?!?*
*These are not the pods you're looking for.* But where are they?!?
---
@@ -273,33 +193,28 @@ The error that we see is expected: the Kubernetes API requires authentication.
*You know what ... This `kube-system` thing looks suspicious.*
*In fact, I'm pretty sure it showed up earlier, when we did:*
`kubectl describe node node1`
---
## Accessing namespaces
- By default, `kubectl` uses the `default` namespace
- We can see resources in all namespaces with `--all-namespaces`
- We can switch to a different namespace with the `-n` option
.exercise[
- List the pods in all namespaces:
- List the pods in the `kube-system` namespace:
```bash
kubectl get pods --all-namespaces
```
- Since Kubernetes 1.14, we can also use `-A` as a shorter version:
```bash
kubectl get pods -A
kubectl -n kube-system get pods
```
]
*Here are our system pods!*
--
*Ding ding ding ding ding!*
The `kube-system` namespace is used for the control plane.
---
@@ -309,7 +224,7 @@ The error that we see is expected: the Kubernetes API requires authentication.
- `kube-apiserver` is the API server
- `kube-controller-manager` and `kube-scheduler` are other control plane components
- `kube-controller-manager` and `kube-scheduler` are other master components
- `coredns` provides DNS-based service discovery ([replacing kube-dns as of 1.11](https://kubernetes.io/blog/2018/07/10/coredns-ga-for-kubernetes-cluster-dns/))
@@ -319,46 +234,12 @@ The error that we see is expected: the Kubernetes API requires authentication.
- the `READY` column indicates the number of containers in each pod
(1 for most pods, but `weave` has 2, for instance)
- the pods with a name ending with `-node1` are the master components
<br/>
(they have been specifically "pinned" to the master node)
---
## Scoping another namespace
- We can also look at a different namespace (other than `default`)
.exercise[
- List only the pods in the `kube-system` namespace:
```bash
kubectl get pods --namespace=kube-system
kubectl get pods -n kube-system
```
]
---
## Namespaces and other `kubectl` commands
- We can use `-n`/`--namespace` with almost every `kubectl` command
- Example:
- `kubectl create --namespace=X` to create something in namespace X
- We can use `-A`/`--all-namespaces` with most commands that manipulate multiple objects
- Examples:
- `kubectl delete` can delete resources across multiple namespaces
- `kubectl label` can add/remove/update labels across multiple namespaces
---
class: extra-details
## What about `kube-public`?
.exercise[
@@ -370,100 +251,20 @@ class: extra-details
]
Nothing!
--
`kube-public` is created by kubeadm & [used for security bootstrapping](https://kubernetes.io/blog/2017/01/stronger-foundation-for-creating-and-managing-kubernetes-clusters).
- Maybe it doesn't have pods, but what secrets is `kube-public` keeping?
---
class: extra-details
## Exploring `kube-public`
- The only interesting object in `kube-public` is a ConfigMap named `cluster-info`
--
.exercise[
- List ConfigMap objects:
- List the secrets in the `kube-public` namespace:
```bash
kubectl -n kube-public get configmaps
```
- Inspect `cluster-info`:
```bash
kubectl -n kube-public get configmap cluster-info -o yaml
kubectl -n kube-public get secrets
```
]
--
Note the `selfLink` URI: `/api/v1/namespaces/kube-public/configmaps/cluster-info`
We can use that!
---
class: extra-details
## Accessing `cluster-info`
- Earlier, when trying to access the API server, we got a `Forbidden` message
- But `cluster-info` is readable by everyone (even without authentication)
.exercise[
- Retrieve `cluster-info`:
```bash
curl -k https://10.96.0.1/api/v1/namespaces/kube-public/configmaps/cluster-info
```
]
- We were able to access `cluster-info` (without auth)
- It contains a `kubeconfig` file
---
class: extra-details
## Retrieving `kubeconfig`
- We can easily extract the `kubeconfig` file from this ConfigMap
.exercise[
- Display the content of `kubeconfig`:
```bash
curl -sk https://10.96.0.1/api/v1/namespaces/kube-public/configmaps/cluster-info \
| jq -r .data.kubeconfig
```
]
- This file holds the canonical address of the API server, and the public key of the CA
- This file *does not* hold client keys or tokens
- This is not sensitive information, but allows us to establish trust
---
class: extra-details
## What about `kube-node-lease`?
- Starting with Kubernetes 1.14, there is a `kube-node-lease` namespace
(or in Kubernetes 1.13 if the NodeLease feature gate is enabled)
- That namespace contains one Lease object per node
- *Node leases* are a new way to implement node heartbeats
(i.e. node regularly pinging the control plane to say "I'm alive!")
- For more details, see [KEP-0009] or the [node controller documentation]
[KEP-0009]: https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/0009-node-heartbeat.md
[node controller documentation]: https://kubernetes.io/docs/concepts/architecture/nodes/#node-controller
- `kube-public` is created by kubeadm & [used for security bootstrapping](https://kubernetes.io/blog/2017/01/stronger-foundation-for-creating-and-managing-kubernetes-clusters)

View File

@@ -170,12 +170,7 @@ pod/pingpong-7c8bbcd9bc-6c9qz 1/1 Running 0 10m
- Scale our `pingpong` deployment:
```bash
kubectl scale deploy/pingpong --replicas 3
```
- Note that this command does exactly the same thing:
```bash
kubectl scale deployment pingpong --replicas 3
kubectl scale deploy/pingpong --replicas 8
```
]
@@ -246,9 +241,6 @@ We could! But the *deployment* would notice it right away, and scale back to the
- `kubectl create job` to create a job
- `kubectl create cronjob` to run a job periodically
<br/>(since Kubernetes 1.14)
- Eventually, `kubectl run` will be used only to start one-shot pods
(see https://github.com/kubernetes/kubernetes/pull/68132)
@@ -265,7 +257,7 @@ We could! But the *deployment* would notice it right away, and scale back to the
- `kubectl create <resource>`
- explicit, but lacks some features
- can't create a CronJob before Kubernetes 1.14
- can't create a CronJob
- can't pass command-line arguments to deployments
- `kubectl create -f foo.yaml` or `kubectl apply -f foo.yaml`
@@ -294,115 +286,7 @@ We could! But the *deployment* would notice it right away, and scale back to the
]
---
### Streaming logs of multiple pods
- Can we stream the logs of all our `pingpong` pods?
.exercise[
- Combine `-l` and `-f` flags:
```bash
kubectl logs -l run=pingpong --tail 1 -f
```
<!--
```wait seq=```
```keys ^C```
-->
]
*Note: combining `-l` and `-f` is only possible since Kubernetes 1.14!*
*Let's try to understand why ...*
---
class: extra-details
### Streaming logs of many pods
- Let's see what happens if we try to stream the logs for more than 5 pods
.exercise[
- Scale up our deployment:
```bash
kubectl scale deployment pingpong --replicas=8
```
- Stream the logs:
```bash
kubectl logs -l run=pingpong --tail 1 -f
```
]
We see a message like the following one:
```
error: you are attempting to follow 8 log streams,
but maximum allowed concurency is 5,
use --max-log-requests to increase the limit
```
---
class: extra-details
## Why can't we stream the logs of many pods?
- `kubectl` opens one connection to the API server per pod
- For each pod, the API server opens one extra connection to the corresponding kubelet
- If there are 1000 pods in our deployment, that's 1000 inbound + 1000 outbound connections on the API server
- This could easily put a lot of stress on the API server
- Prior Kubernetes 1.14, it was decided to *not* allow multiple connections
- From Kubernetes 1.14, it is allowed, but limited to 5 connections
(this can be changed with `--max-log-requests`)
- For more details about the rationale, see
[PR #67573](https://github.com/kubernetes/kubernetes/pull/67573)
---
## Shortcomings of `kubectl logs`
- We don't see which pod sent which log line
- If pods are restarted / replaced, the log stream stops
- If new pods are added, we don't see their logs
- To stream the logs of multiple pods, we need to write a selector
- There are external tools to address these shortcomings
(e.g.: [Stern](https://github.com/wercker/stern))
---
class: extra-details
## `kubectl logs -l ... --tail N`
- If we run this with Kubernetes 1.12, the last command shows multiple lines
- This is a regression when `--tail` is used together with `-l`/`--selector`
- It always shows the last 10 lines of output for each container
(instead of the number of lines specified on the command line)
- The problem was fixed in Kubernetes 1.13
*See [#70554](https://github.com/kubernetes/kubernetes/issues/70554) for details.*
Unfortunately, `--follow` cannot (yet) be used to stream the logs from multiple containers.
---

View File

@@ -16,8 +16,6 @@
- each pod is aware of its IP address (no NAT)
- pod IP addresses are assigned by the network implementation
- Kubernetes doesn't mandate any particular implementation
---
@@ -32,7 +30,7 @@
- No new protocol
- The network implementation can decide how to allocate addresses
- Pods cannot move from a node to another and keep their IP address
- IP addresses don't have to be "portable" from a node to another
@@ -84,17 +82,13 @@
---
class: extra-details
## The Container Network Interface (CNI)
- Most Kubernetes clusters use CNI "plugins" to implement networking
- The CNI has a well-defined [specification](https://github.com/containernetworking/cni/blob/master/SPEC.md#network-configuration) for network plugins
- When a pod is created, Kubernetes delegates the network setup to these plugins
- When a pod is created, Kubernetes delegates the network setup to CNI plugins
(in can be a single plugin, or a combination of plugins, each doing one task)
- Typically, CNI plugins will:
- Typically, a CNI plugin will:
- allocate an IP address (by calling an IPAM plugin)
@@ -102,46 +96,8 @@ class: extra-details
- configure the interface as well as required routes etc.
---
- Using multiple plugins can be done with "meta-plugins" like CNI-Genie or Multus
class: extra-details
- Not all CNI plugins are equal
## Multiple moving parts
- The "pod-to-pod network" or "pod network":
- provides communication between pods and nodes
- is generally implemented with CNI plugins
- The "pod-to-service network":
- provides internal communication and load balancing
- is generally implemented with kube-proxy (or e.g. kube-router)
- Network policies:
- provide firewalling and isolation
- can be bundled with the "pod network" or provided by another component
---
class: extra-details
## Even more moving parts
- Inbound traffic can be handled by multiple components:
- something like kube-proxy or kube-router (for NodePort services)
- load balancers (ideally, connected to the pod network)
- It is possible to use multiple pod networks in parallel
(with "meta-plugins" like CNI-Genie or Multus)
- Some products can fill multiple roles
(e.g. kube-router can be set up to provide the pod network and/or network policies and/or replace kube-proxy)
(e.g. they don't all implement network policies, which are required to isolate pods)

Some files were not shown because too many files have changed in this diff Show More