diff --git a/slides/k8s/aggregation-layer.md b/slides/k8s/aggregation-layer.md new file mode 100644 index 00000000..5efd7b1a --- /dev/null +++ b/slides/k8s/aggregation-layer.md @@ -0,0 +1,386 @@ +# The Aggregation Layer + +- The aggregation layer is a way to extend the Kubernetes API + +- It is similar to CRDs + + - it lets us define new resource types + + - these resources can then be used with `kubectl` and other clients + +- The implementation is very different + + - CRDs are handled within the API server + + - the aggregation layer offloads requests to another process + +- They are designed for very different use-cases + +--- + +## CRDs vs aggregation layer + +- The Kubernetes API is a REST-ish API with a hierarchical structure + +- It can be extended with Custom Resource Definifions (CRDs) + +- Custom resources are managed by the Kubernetes API server + + - we don't need to write code + + - the API server does all the heavy lifting + + - these resources are persisted in Kubernetes' "standard" database +
+ (for most installations, that's `etcd`) + +- We can also define resources that are *not* managed by the API server + + (the API server merely proxies the requests to another server) + +--- + +## Which one is best? + +- For things that "map" well to objects stored in a traditional database: + + *probably CRDs* + +- For things that "exist" only in Kubernetes and don't represent external resources: + + *probably CRDs* + +- For things that are read-only, at least from Kubernetes' perspective: + + *probably aggregation layer* + +- For things that can't be stored in etcd because of size or access patterns: + + *probably aggregation layer* + + +--- + +## How are resources organized? + +- Let's have a look at the Kubernetes API hierarchical structure + +- Useful: `.metadata.selfLink` contains the URI of a resource + +.exercise[ + +- Check the `apiVersion` and URI of a "core" resource, e.g. a Node: + ```bash + kubectl get nodes -o json | jq .items[0].apiVersion + kubectl get nodes -o json | jq .items[0].metadata.selfLink + ``` + +- Get the `apiVersion` and URI for a "non-core" resource, e.g. a ClusterRole: + ```bash + kubectl get clusterrole view -o json | jq .apiVersion + kubectl get clusterrole view -o json | jq .metadata.selfLink + ``` + +] + +--- + +## Core vs non-core + +- This is the structure of the URIs that we just checked: + + ``` + /api/v1/nodes/node1 + ↑ ↑ ↑ + `version` `kind` `name` + + /apis/rbac.authorization.k8s.io/v1/clusterroles/view + ↑ ↑ ↑ ↑ + `group` `version` `kind` `name` + ``` + +- There is no group for "core" resources + +- Or, we could say that the group, `core`, is implied + +--- + +## Group-Version-Kind + +- In the API server, the Group-Version-Kind triple maps to a Go type + + (look for all the "GVK" occurrences in the source code!) + +- In the API server URI router, the GVK is parsed "relatively early" + + (so that the server can know which resource we're talking about) + +- "Well, actually ..." Things are a bit more complicated, see next slides! + +--- + +class: extra-details + +## Namespaced resources + +- Here are what namespaced resources URIs look like: + + ``` + /api/v1/namespaces/default/services/kubernetes + ↑ ↑ ↑ ↑ + `version` `namespace` `kind` `name` + + /apis/apps/v1/namespaces/kube-system/daemonsets/kube-proxy + ↑ ↑ ↑ ↑ ↑ + `group` `version` `namespace` `kind` `name` + ``` + +--- + +class: extra-details + +## Subresources + +- Many resources have *subresources*, for instance: + + - `/status` (decouples status updates from other updates) + + - `/scale` (exposes a consistent interface for autoscalers) + + - `/proxy` (allows access to HTTP resources) + + - `/portforward` (used by `kubectl port-forward`) + + - `/logs` (access pod logs) + +- These are added at the end of the URI + +--- + +class: extra-details + +## Accessing a subresource + +.exercise[ + +- List `kube-proxy` pods: + ```bash + kubectl get pods --namespace=kube-system --selector=k8s-app=kube-proxy + PODNAME=$( + kubectl get pods --namespace=kube-system --selector=k8s-app=kube-proxy \ + -o json | jq .items[0].metadata.name) + ``` + +- Execute a command in a pod, showing the API requests: + ```bash + kubectl -v6 exec --namespace=kube-system $PODNAME -- echo hello world + ``` + +] + +-- + +The full request looks like: +``` +POST https://.../api/v1/namespaces/kube-system/pods/kube-proxy-c7rlw/exec? +command=echo&command=hello&command=world&container=kube-proxy&stderr=true&stdout=true +``` + +--- + +## Listing what's supported on the server + +- There are at least three useful commands to introspect the API server + +.exercise[ + +- List resources types, their group, kind, short names, and scope: + ```bash + kubectl api-resources + ``` + +- List API groups + versions: + ```bash + kubectl api-versions + ``` + +- List APIServices: + ```bash + kubectl get apiservices + ``` + +] + +-- + +🤔 What's the difference between the last two? + +--- + +## API registration + +- `kubectl api-versions` shows all API groups, including `apiregistration.k8s.io` + +- `kubectl get apiservices` shows the "routing table" for API requests + +- The latter doesn't show `apiregistration.k8s.io` + + (APIServices belong to `apiregistration.k8s.io`) + +- Most API groups are `Local` (handled internally by the API server) + +- If we're running the `metrics-server`, it should handle `metrics.k8s.io` + +- This is an API group handled *outside* of the API server + +- This is the *aggregation layer!* + +--- + +## Finding resources + +The following assumes that `metrics-server` is deployed on your cluster. + +.exercise[ + +- Check that the metrics.k8s.io is registered with `metrics-server`: + ```bash + kubectl get apiservices | grep metrics.k8s.io + ``` + +- Check the resource kinds registered in the metrics.k8s.io group: + ```bash + kubectl api-resources --api-group=metrics.k8s.io + ``` + +] + +(If the output of either command is empty, install `metrics-server` first.) + +--- + +## `nodes` vs `nodes` + +- We can have multiple resources with the same name + +.exercise[ + +- Look for resources named `node`: + ```bash + kubectl api-resources | grep -w nodes + ``` + +- Compare the output of both commands: + ```bash + kubectl get nodes + kubectl get nodes.metrics.k8s.io + ``` + +] + +-- + +🤔 What are the second kind of nodes? How can we see what's really in them? + +--- + +## Node vs NodeMetrics + +- `nodes.metrics.k8s.io` (aka NodeMetrics) don't have fancy *printer columns* + +- But we can look at the raw data (with `-o json` or `-o yaml`) + +.exercise[ + +- Look at NodeMetrics objects with one of these commands: + ```bash + kubectl get -o yaml nodes.metrics.k8s.io + kubectl get -o yaml NodeMetrics + ``` + +] + +-- + +💡 Alright, these are the live metrics (CPU, RAM) for our nodes. + +--- + +## An easier way to consume metrics + +- We might have seen these metrics before ... With an easier command! + +-- + +.exercise[ + +- Display node metrics: + ```bash + kubectl top nodes + ``` + +- Check which API requests happen behind the scenes: + ```bash + kubectl top nodes -v6 + ``` + +] + +--- + +## Aggregation layer in practice + +- We can write an API server to handle a subset of the Kubernetes API + +- Then we can register that server by creating an APIService resource + +.exercise[ + +- Check the definition used for the `metrics-server`: + ```bash + kubectl describe apiservices v1beta1.metrics.k8s.io + ``` +] + +- Group priority is used when multiple API groups provide similar kinds + + (e.g. `nodes` and `nodes.metrics.k8s.io` as seen earlier) + +--- + +## Authentication flow + +- We have two Kubernetes API servers: + + - "aggregator" (the main one; clients connect to it) + + - "aggregated" (the one providing the extra API; aggregator connects to it) + +- Aggregator deals with client authentication + +- Aggregator authenticates with aggregated using mutual TLS + +- Aggregator passes (/forwards/proxies/...) requests to aggregated + +- Aggregated performs authorization by calling back aggregator + + ("can subject X perform action Y on resource Z?") + +[This doc page](https://kubernetes.io/docs/tasks/extend-kubernetes/configure-aggregation-layer/#authentication-flow) has very nice swim lanes showing that flow. + +--- + +## Discussion + +- Aggregation layer is great for metrics + + (fast-changing, ephemeral data, that would be outrageously bad for etcd) + +- It *could* be a good fit to expose other REST APIs as a pass-thru + + (but it's more common to see CRDs instead) + +??? + +:EN:- The aggregation layer +:FR:- Étendre l'API avec le *aggregation layer* \ No newline at end of file diff --git a/slides/k8s/extending-api.md b/slides/k8s/extending-api.md index 771d5619..81fa0654 100644 --- a/slides/k8s/extending-api.md +++ b/slides/k8s/extending-api.md @@ -375,9 +375,5 @@ class: extra-details ??? -:EN:- Extending the Kubernetes API -:EN:- The aggregation layer -:EN:- Admission control and webhooks - +:EN:- Overview of Kubernetes API extensions :FR:- Comment étendre l'API Kubernetes -:FR:- Extension via *aggregation layer*, *admission control*, *webhooks* diff --git a/slides/kube-selfpaced.yml b/slides/kube-selfpaced.yml index fb14e214..1d11d404 100644 --- a/slides/kube-selfpaced.yml +++ b/slides/kube-selfpaced.yml @@ -115,6 +115,7 @@ content: - k8s/extending-api.md - k8s/apiserver-deepdive.md - k8s/crd.md + - k8s/aggregation-layer.md - k8s/admission.md - k8s/operators.md - k8s/operators-design.md