Extending the Kubernetes API

There are multiple ways to extend the Kubernetes API.

We are going to cover:

Revisiting the API server

The Kubernetes API server is a central point of the control plane

(everything connects to it: controller manager, scheduler, kubelets)
Almost everything in Kubernetes is materialized by a resource
Resources have a type (or "kind")

(similar to strongly typed languages)
We can see existing types with kubectl api-resources
We can list resources of a given type with kubectl get <type>

We can create new types with Custom Resource Definitions (CRDs)
CRDs are created dynamically

(without recompiling or restarting the API server)
CRDs themselves are resources:
- we can create a new type with kubectl create and some YAML
- we can see all our custom types with kubectl get crds
After we create a CRD, the new type works just like built-in types

There are many possibilities!

Operators encapsulate complex sets of resources

(e.g.: a PostgreSQL replicated cluster; an etcd cluster...
see awesome operators and OperatorHub to find more)
Custom use-cases like gitkube
- creates a new custom type, Remote, exposing a git+ssh server
- deploy by pushing YAML or Helm Charts to that remote
Replacing built-in types with CRDs

(see this lightning talk by Tim Hockin)

By default, CRDs are not validated

(we can put anything we want in the spec)
When creating a CRD, we can pass an OpenAPI v3 schema (BETA!)

(which will then be used to validate resources)
Generally, when creating a CRD, we also want to run a controller

(otherwise nothing will happen when we create resources of that type)
The controller will typically watch our custom resources

(and take action when they are created/updated)

Service catalog is another extension mechanism
It's not extending the Kubernetes API strictly speaking

(but it still provides new features!)
It doesn't create new types; it uses:
- ClusterServiceBroker
- ClusterServiceClass
- ClusterServicePlan
- ServiceInstance
- ServiceBinding
It uses the Open service broker API

When a Pod is created, it is associated to a ServiceAccount

(even if we did not specify one explicitly)
That ServiceAccount was added on the fly by an admission controller

(specifically, a mutating admission controller)
Admission controllers sit on the API request path

(see the cool diagram on next slide, courtesy of Banzai Cloud)

class: pic

Validating admission controllers can accept/reject the API call
Mutating admission controllers can modify the API request payload
Both types can also trigger additional actions

(e.g. automatically create a Namespace if it doesn't exist)
There are a number of built-in admission controllers

(see documentation for a list)
But we can also define our own!

We can setup admission webhooks to extend the behavior of the API server
The API server will submit incoming API requests to these webhooks
These webhooks can be validating or mutating
Webhooks can be setup dynamically (without restarting the API server)
To setup a dynamic admission webhook, we create a special resource:

a ValidatingWebhookConfiguration or a MutatingWebhookConfiguration
These resources are created and managed like other resources

(i.e. kubectl create, kubectl get ...)

A ValidatingWebhookConfiguration or MutatingWebhookConfiguration contains:
- the address of the webhook
- the authentication information to use with the webhook
- a list of rules
The rules indicate for which objects and actions the webhook is triggered

(to avoid e.g. triggering webhooks when setting up webhooks)

If we need to store something "safely" (as in: in etcd), we can use CRDs
This gives us primitives to read/write/list objects (and optionally validate them)
The Kubernetes API server can run on its own

(without the scheduler, controller manager, and kubelets)
By loading CRDs, we can have it manage totally different objects

(unrelated to containers, clusters, etc.)