From 10fbfa135a97ef47874d6fd9bfd995cd156f72b2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=A9r=C3=B4me=20Petazzoni?= Date: Fri, 6 Jun 2025 15:35:20 +0200 Subject: [PATCH] =?UTF-8?q?=F0=9F=93=83=20Update=20control=20plane=20auth?= =?UTF-8?q?=20section?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- slides/k8s/control-plane-auth.md | 192 ++++++++++++++++++++++++------- 1 file changed, 150 insertions(+), 42 deletions(-) diff --git a/slides/k8s/control-plane-auth.md b/slides/k8s/control-plane-auth.md index ed4ac5a8..99f81b74 100644 --- a/slides/k8s/control-plane-auth.md +++ b/slides/k8s/control-plane-auth.md @@ -22,11 +22,13 @@ ## Authentication and authorization -- Authentication (checking "who you are") is done with mutual TLS +- Authentication (checking "who you are") can be done in different ways: - (both the client and the server need to hold a valid certificate) + - with mutual TLS (both client and server need to hold a valid certificate) -- Authorization (checking "what you can do") is done in different ways + - with service account tokens (issued by the Kubernetes API server) + +- Authorization (checking "what you can do") can also be done in multiple ways: - the API server implements a sophisticated permission logic (with RBAC) @@ -34,6 +36,30 @@ - some services require a certificate signed by a particular CA / sub-CA + - there is also a special "Node Authorizer" (for kubelet API access) + +--- + +## Mutual TLS vs tokens + +- Service account tokens: + + - automatically generated by API server + + - can be exposed to pods through e.g. volume mounts + + - require the control plane to be up and running + + - can't be used by kubelets or by static pods + +- Mutual TLS: + + - requires manual generation (and renewal!) + + - doesn't require the control plane to be up and running + + - particularly relevant for kubelets and static pods + --- ## In practice @@ -114,22 +140,17 @@ --- +## API server authentication with TLS certificates -## API server clients +- Some control plane components will authenticate with TLS certificates -- The API server has a sophisticated authentication and authorization system - -- For connections coming from other components of the control plane: - - - authentication uses certificates (trusting the certificates' subject or CN) - - - authorization uses whatever mechanism is enabled (most oftentimes, RBAC) + (typically: scheduler, controller manager; also: kubelets!) - The relevant API server flags are: `--client-ca-file`, `--tls-cert-file`, `--tls-private-key-file` -- Each component connecting to the API server takes a `--kubeconfig` flag +- These clients will typically accept a `--kubeconfig` flag (to specify a kubeconfig file containing the CA cert, client key, and client cert) @@ -137,6 +158,84 @@ --- +## API server authentication with tokens + +- Some control plane components may authenticate with Service Account tokens + + (typically: controllers like CNI, CSI, Ingress...) + +- The relevant API server flags are: + + `--service-account-signing-key-file`, `--service-account-issuer`, `--service-account-key-file` + +- These clients will automatically detect that they should use "in cluster config" + +- That detection relies on the following things to exist: + + - environment variables `KUBERNETES_SERVICE_HOST` and `KUBERNETES_SERVICE_PORT` + + - token in file `/var/run/secrets/kubernetes.io/serviceaccount/token` + +--- + +## API server clients authorization + +- Most clients will rely on the `RBAC` authorizer + + - enabled with API server flag `--authorization-mode=RBAC` + + - that flag will automatically create a bunch of roles and bindings + + - clients should use standard names (e.g. `system:kube-scheduler`) + +- Kubelets will rely on the `Node` authorizer + + - enabled with API server flag `--authorization-mode=Node` + + - this authorizer makes sure that kubelets work on a "need-to-know" basis + + - kubelets should use standard names (`system:node:`) + +- Note: to enable both authorizers, use `--authorization-mode=RBAC,Node` + +--- + +class: extra-details + +## How are these permissions set up? + +- A bunch of roles and bindings are defined as constants in the API server code: + + [auth/authorizer/rbac/bootstrappolicy/policy.go](https://github.com/kubernetes/kubernetes/blob/release-1.19/plugin/pkg/auth/authorizer/rbac/bootstrappolicy/policy.go#L188) + +- They are created automatically when the API server starts: + + [registry/rbac/rest/storage_rbac.go](https://github.com/kubernetes/kubernetes/blob/release-1.19/pkg/registry/rbac/rest/storage_rbac.go#L140) + +- We must use the correct Common Names (`CN`) for the control plane certificates + + (since the bindings defined above refer to these common names) + +--- + +class: extra-details + +## The Node Authorizer + +- Question: when should node `X` be able to access secret `Y`? + +-- + +- Answer: if, and only if, node `X` runs a pod that uses secret `Y` + +- The Node Authorizer implements that kind of logic + +- It also allows kubelets to set labels and taints for themselves + + (but not for other nodes) + +--- + ## Kubelet and API server - Communication between kubelet and API server can be established both ways @@ -167,6 +266,12 @@ (it will authenticate like any other client) +- Authorization will typically require the Node Authorizer mentioned earlier + +⚠️ Kubelet certificates need to be renewed regularly! + +- This is typically done through the CSR API + --- ## API server → kubelet @@ -203,37 +308,31 @@ - Its certificate will have `CN=system:kube-controller-manager` +- To improve security posture, each controller can use an individual Service Account + +- This is enabled with flag `--use-service-account-credentials=true` + +--- + +## Controller manager keys + +- The controller can create Secrets holding Service Account tokens + + - this is enabled with flag `--service-account-private-key-file` + + - this was used in older versions of Kubernetes (before *bound tokens*) + + - in modern clusters, kubelet uses the `TokenRequest` API instead + - If we use the CSR API, the controller manager needs the CA cert and key - (passed with flags `--cluster-signing-cert-file` and `--cluster-signing-key-file`) + - the CSR API is used in many clusters to renew kubelet certificates -- We usually want the controller manager to generate tokens for service accounts - -- These tokens deserve some details (on the next slide!) + - it's enabled with `--cluster-signing-cert-file` and `--cluster-signing-key-file` --- -class: extra-details - -## How are these permissions set up? - -- A bunch of roles and bindings are defined as constants in the API server code: - - [auth/authorizer/rbac/bootstrappolicy/policy.go](https://github.com/kubernetes/kubernetes/blob/release-1.19/plugin/pkg/auth/authorizer/rbac/bootstrappolicy/policy.go#L188) - -- They are created automatically when the API server starts: - - [registry/rbac/rest/storage_rbac.go](https://github.com/kubernetes/kubernetes/blob/release-1.19/pkg/registry/rbac/rest/storage_rbac.go#L140) - -- We must use the correct Common Names (`CN`) for the control plane certificates - - (since the bindings defined above refer to these common names) - ---- - -## Service account tokens - -- Each time we create a service account, the controller manager generates a token +## Service account tokens recap - These tokens are JWT tokens, signed with a particular key @@ -241,13 +340,14 @@ class: extra-details (and therefore, the API server needs to be able to verify their integrity) -- This uses another keypair: +- That key is passed to the API server using a couple of flags: - - the private key (used for signature) is passed to the controller manager -
(using flags `--service-account-private-key-file` and `--root-ca-file`) + - `--service-account-private-key-file` (used to issue tokens) - - the public key (used for verification) is passed to the API server -
(using flag `--service-account-key-file`) + - `--service-account-key-file` (used to verify tokens) + +- The private key is also passed to the controller manager +
(using flag `--service-account-private-key-file`) --- @@ -261,8 +361,14 @@ class: extra-details - It will authenticate using the token of that Service Account +- It's also possible (but rare) to run it with e.g. static pods + + (it will then require TLS keys; possibly the same as kubelet's!) + --- +class: extra-details + ## Webhooks - We mentioned webhooks earlier; how does that really work? @@ -283,6 +389,8 @@ class: extra-details --- +class: extra-details + ## Subject Access Review Here is an example showing how to check if `jean.doe` can `get` some `pods` in `kube-system`: