Merge pull request #507 from bridgetkromhout/oscon2019

oscon updates

slides/k8s-201.yml

@@ -28,7 +28,7 @@ chapters:
   - k8s/authn-authz-k8s201.md
 - - k8s/resource-limits-k8s201.md
   - k8s/metrics-server.md
-- - k8s/cluster-sizing.md
+- - k8s/cluster-sizing-k8s201.md
   - k8s/horizontal-pod-autoscaler.md
   - k8s/extending-api.md
   - k8s/operators.md

slides/k8s/cluster-sizing-k8s201.md

@@ -0,0 +1,167 @@
+# Cluster sizing
+
+- What happens when the cluster gets full?
+
+- How can we scale up the cluster?
+
+- Can we do it automatically?
+
+- What are other methods to address capacity planning?
+
+---
+
+## When are we out of resources?
+
+- kubelet monitors node resources:
+
+  - memory
+
+  - node disk usage (typically the root filesystem of the node)
+
+  - image disk usage (where container images and RW layers are stored)
+
+- For each resource, we can provide two thresholds:
+
+  - a hard threshold (if it's met, it provokes immediate action)
+
+  - a soft threshold (provokes action only after a grace period)
+
+- Resource thresholds and grace periods are configurable
+
+  (by passing kubelet command-line flags)
+
+---
+
+## What happens then?
+
+- If disk usage is too high:
+
+  - kubelet will try to remove terminated pods
+
+  - then, it will try to *evict* pods
+
+- If memory usage is too high:
+
+  - it will try to evict pods
+
+- The node is marked as "under pressure"
+
+- This temporarily prevents new pods from being scheduled on the node
+
+---
+
+## Which pods get evicted?
+
+- kubelet looks at the pods' QoS and PriorityClass
+
+- First, pods with BestEffort QoS are considered
+
+- Then, pods with Burstable QoS exceeding their *requests*
+
+  (but only if the exceeding resource is the one that is low on the node)
+
+- Finally, pods with Guaranteed QoS, and Burstable pods within their requests
+
+- Within each group, pods are sorted by PriorityClass
+
+- If there are pods with the same PriorityClass, they are sorted by usage excess
+
+  (i.e. the pods whose usage exceeds their requests the most are evicted first)
+
+---
+
+class: extra-details
+
+## Eviction of Guaranteed pods
+
+- *Normally*, pods with Guaranteed QoS should not be evicted
+
+- A chunk of resources is reserved for node processes (like kubelet)
+
+- It is expected that these processes won't use more than this reservation
+
+- If they do use more resources anyway, all bets are off!
+
+- If this happens, kubelet must evict Guaranteed pods to preserve node stability
+
+  (or Burstable pods that are still within their requested usage)
+
+---
+
+## What happens to evicted pods?
+
+- The pod is terminated
+
+- It is marked as `Failed` at the API level
+
+- If the pod was created by a controller, the controller will recreate it
+
+- The pod will be recreated on another node, *if there are resources available!*
+
+- For more details about the eviction process, see:
+
+  - [this documentation page](https://kubernetes.io/docs/tasks/administer-cluster/out-of-resource/) about resource pressure and pod eviction,
+
+  - [this other documentation page](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/) about pod priority and preemption.
+
+---
+
+## What if there are no resources available?
+
+- Sometimes, a pod cannot be scheduled anywhere:
+
+  - all the nodes are under pressure,
+
+  - or the pod requests more resources than are available
+
+- The pod then remains in `Pending` state until the situation improves
+
+---
+
+## Cluster scaling
+
+- One way to improve the situation is to add new nodes
+
+- This can be done automatically with the [Cluster Autoscaler](https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler)
+
+- The autoscaler will automatically scale up:
+
+  - if there are pods that failed to be scheduled
+
+- The autoscaler will automatically scale down:
+
+  - if nodes have a low utilization for an extended period of time
+
+---
+
+## Restrictions, gotchas ...
+
+- The Cluster Autoscaler only supports a few cloud infrastructures
+
+  (see [here](https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler/cloudprovider) for a list) - ([in preview for AKS](https://docs.microsoft.com/en-us/azure/aks/cluster-autoscaler))
+
+- The Cluster Autoscaler cannot scale down nodes that have pods using:
+
+  - local storage
+
+  - affinity/anti-affinity rules preventing them from being rescheduled
+
+  - a restrictive PodDisruptionBudget
+
+---
+
+## Other way to do capacity planning
+
+- "Running Kubernetes without nodes"
+
+- Systems like [Virtual Kubelet](https://virtual-kubelet.io/) or Kiyot can run pods using on-demand resources
+
+  - Virtual Kubelet can leverage e.g. ACI or Fargate to run pods
+
+  - Kiyot runs pods in ad-hoc EC2 instances (1 instance per pod)
+
+- Economic advantage (no wasted capacity)
+
+- Security advantage (stronger isolation between pods)
+
+Check [this blog post](http://jpetazzo.github.io/2019/02/13/running-kubernetes-without-nodes-with-kiyot/) for more details.

slides/k8s/horizontal-pod-autoscaler.md

@@ -88,9 +88,10 @@
   kubectl expose deployment busyhttp --port=80
   ```
 
-- Get the ClusterIP allocated to the service:
+- Port-forward to our service
   ```bash
-  kubectl get svc busyhttp
+  kubectl port-forward service/busyhttp 8080:80 &
+  curl -k localhost:8080
   ```
 
 ]

slides/k8s/kubercoins-k8s201.md

@@ -211,11 +211,11 @@ https://@@GITREPO@@/blob/8279a3bce9398f7c1a53bdd95187c53eda4e6435/dockercoins/wo
   kubectl proxy &
   ```
 
-- Open in a web browser: [our webui](http://localhost:8001/api/v1/namespaces/default/services/webui/proxy/index.html)
+- Open in a web browser: [http://localhost:8001/api/v1/namespaces/default/services/webui/proxy/index.html](http://localhost:8001/api/v1/namespaces/default/services/webui/proxy/index.html)
 
 ]
 
 A drawing area should show up, and after a few seconds, a blue
 graph will appear.
 
-- If using Cloud Shell, use the [Cloud Shell Web Preview](https://docs.microsoft.com/en-us/azure/cloud-shell/using-the-shell-window#web-preview)
+- If using Cloud Shell, use the [Cloud Shell Web Preview](https://docs.microsoft.com/en-us/azure/cloud-shell/using-the-shell-window#web-preview) and append `/api/v1/namespaces/default/services/webui/proxy/index.html` to the existing path

slides/k8s/localkubeconfig-k8s201.md

@@ -31,6 +31,28 @@
 
 - If you're going to use Cloud Shell, you can skip ahead
 
+---
+class: extra-details
+
+## Preserving the existing `~/.kube/config` (optional)
+
+- If you already have a `~/.kube/config` file, rename it
+
+  (we are going to overwrite it in the following slides!)
+
+- If you never used `kubectl` on your machine before: nothing to do!
+
+.exercise[
+
+- Make a copy of `~/.kube/config`; if you are using macOS or Linux, you can do:
+  ```bash
+  cp ~/.kube/config ~/.kube/config.before.training
+  ```
+
+- If you are using Windows, you will need to adapt this command
+
+]
+
 ---
 
 ## Connecting to your AKS cluster via local tools
@@ -107,27 +129,6 @@ class: extra-details
 Note: if you are following along with a different platform (e.g. Linux on an architecture different from amd64, or with a phone or tablet), installing `kubectl` might be more complicated (or even impossible) so check with us about cloud shell.
 
 
----
-class: extra-details
-
-## Preserving the existing `~/.kube/config`
-
-- If you already have a `~/.kube/config` file, rename it
-
-  (we are going to overwrite it in the following slides!)
-
-- If you never used `kubectl` on your machine before: nothing to do!
-
-.exercise[
-
-- Make a copy of `~/.kube/config`; if you are using macOS or Linux, you can do:
-  ```bash
-  cp ~/.kube/config ~/.kube/config.before.training
-  ```
-
-- If you are using Windows, you will need to adapt this command
-
-]
 
 ---
 
@@ -170,6 +171,7 @@ Platform:"darwin/amd64"}
   API_URL=$(kubectl config view -o json | jq -r ".clusters[]  \
             | select(.name == \"$AKS_NAME\") | .cluster.server")
   echo $API_URL
+  ```
 ]
 
 ---

slides/k8s/resource-limits-k8s201.md

@@ -432,10 +432,6 @@ When a ResourceQuota is created, we can see how much of it is used:
 
   ```
 
-]
-
-.exercise[
-
 - Remove quota:
   ```bash
     kubectl delete quota my-resource-quota