mirror of
https://github.com/jpetazzo/container.training.git
synced 2026-02-14 17:49:59 +00:00
📃 Update control plane auth section
This commit is contained in:
Jérôme Petazzoni
@@ -22,11 +22,13 @@
|
||||
|
||||
## Authentication and authorization
|
||||
|
||||
- Authentication (checking "who you are") is done with mutual TLS
|
||||
- Authentication (checking "who you are") can be done in different ways:
|
||||
|
||||
(both the client and the server need to hold a valid certificate)
|
||||
- with mutual TLS (both client and server need to hold a valid certificate)
|
||||
|
||||
- Authorization (checking "what you can do") is done in different ways
|
||||
- with service account tokens (issued by the Kubernetes API server)
|
||||
|
||||
- Authorization (checking "what you can do") can also be done in multiple ways:
|
||||
|
||||
- the API server implements a sophisticated permission logic (with RBAC)
|
||||
|
||||
@@ -34,6 +36,30 @@
|
||||
|
||||
- some services require a certificate signed by a particular CA / sub-CA
|
||||
|
||||
- there is also a special "Node Authorizer" (for kubelet API access)
|
||||
|
||||
---
|
||||
|
||||
## Mutual TLS vs tokens
|
||||
|
||||
- Service account tokens:
|
||||
|
||||
- automatically generated by API server
|
||||
|
||||
- can be exposed to pods through e.g. volume mounts
|
||||
|
||||
- require the control plane to be up and running
|
||||
|
||||
- can't be used by kubelets or by static pods
|
||||
|
||||
- Mutual TLS:
|
||||
|
||||
- requires manual generation (and renewal!)
|
||||
|
||||
- doesn't require the control plane to be up and running
|
||||
|
||||
- particularly relevant for kubelets and static pods
|
||||
|
||||
---
|
||||
|
||||
## In practice
|
||||
@@ -114,22 +140,17 @@
|
||||
|
||||
---
|
||||
|
||||
## API server authentication with TLS certificates
|
||||
|
||||
## API server clients
|
||||
- Some control plane components will authenticate with TLS certificates
|
||||
|
||||
- The API server has a sophisticated authentication and authorization system
|
||||
|
||||
- For connections coming from other components of the control plane:
|
||||
|
||||
- authentication uses certificates (trusting the certificates' subject or CN)
|
||||
|
||||
- authorization uses whatever mechanism is enabled (most oftentimes, RBAC)
|
||||
(typically: scheduler, controller manager; also: kubelets!)
|
||||
|
||||
- The relevant API server flags are:
|
||||
|
||||
`--client-ca-file`, `--tls-cert-file`, `--tls-private-key-file`
|
||||
|
||||
- Each component connecting to the API server takes a `--kubeconfig` flag
|
||||
- These clients will typically accept a `--kubeconfig` flag
|
||||
|
||||
(to specify a kubeconfig file containing the CA cert, client key, and client cert)
|
||||
|
||||
@@ -137,6 +158,84 @@
|
||||
|
||||
---
|
||||
|
||||
## API server authentication with tokens
|
||||
|
||||
- Some control plane components may authenticate with Service Account tokens
|
||||
|
||||
(typically: controllers like CNI, CSI, Ingress...)
|
||||
|
||||
- The relevant API server flags are:
|
||||
|
||||
`--service-account-signing-key-file`, `--service-account-issuer`, `--service-account-key-file`
|
||||
|
||||
- These clients will automatically detect that they should use "in cluster config"
|
||||
|
||||
- That detection relies on the following things to exist:
|
||||
|
||||
- environment variables `KUBERNETES_SERVICE_HOST` and `KUBERNETES_SERVICE_PORT`
|
||||
|
||||
- token in file `/var/run/secrets/kubernetes.io/serviceaccount/token`
|
||||
|
||||
---
|
||||
|
||||
## API server clients authorization
|
||||
|
||||
- Most clients will rely on the `RBAC` authorizer
|
||||
|
||||
- enabled with API server flag `--authorization-mode=RBAC`
|
||||
|
||||
- that flag will automatically create a bunch of roles and bindings
|
||||
|
||||
- clients should use standard names (e.g. `system:kube-scheduler`)
|
||||
|
||||
- Kubelets will rely on the `Node` authorizer
|
||||
|
||||
- enabled with API server flag `--authorization-mode=Node`
|
||||
|
||||
- this authorizer makes sure that kubelets work on a "need-to-know" basis
|
||||
|
||||
- kubelets should use standard names (`system:node:<name-of-the-node>`)
|
||||
|
||||
- Note: to enable both authorizers, use `--authorization-mode=RBAC,Node`
|
||||
|
||||
---
|
||||
|
||||
class: extra-details
|
||||
|
||||
## How are these permissions set up?
|
||||
|
||||
- A bunch of roles and bindings are defined as constants in the API server code:
|
||||
|
||||
[auth/authorizer/rbac/bootstrappolicy/policy.go](https://github.com/kubernetes/kubernetes/blob/release-1.19/plugin/pkg/auth/authorizer/rbac/bootstrappolicy/policy.go#L188)
|
||||
|
||||
- They are created automatically when the API server starts:
|
||||
|
||||
[registry/rbac/rest/storage_rbac.go](https://github.com/kubernetes/kubernetes/blob/release-1.19/pkg/registry/rbac/rest/storage_rbac.go#L140)
|
||||
|
||||
- We must use the correct Common Names (`CN`) for the control plane certificates
|
||||
|
||||
(since the bindings defined above refer to these common names)
|
||||
|
||||
---
|
||||
|
||||
class: extra-details
|
||||
|
||||
## The Node Authorizer
|
||||
|
||||
- Question: when should node `X` be able to access secret `Y`?
|
||||
|
||||
--
|
||||
|
||||
- Answer: if, and only if, node `X` runs a pod that uses secret `Y`
|
||||
|
||||
- The Node Authorizer implements that kind of logic
|
||||
|
||||
- It also allows kubelets to set labels and taints for themselves
|
||||
|
||||
(but not for other nodes)
|
||||
|
||||
---
|
||||
|
||||
## Kubelet and API server
|
||||
|
||||
- Communication between kubelet and API server can be established both ways
|
||||
@@ -167,6 +266,12 @@
|
||||
|
||||
(it will authenticate like any other client)
|
||||
|
||||
- Authorization will typically require the Node Authorizer mentioned earlier
|
||||
|
||||
⚠️ Kubelet certificates need to be renewed regularly!
|
||||
|
||||
- This is typically done through the CSR API
|
||||
|
||||
---
|
||||
|
||||
## API server → kubelet
|
||||
@@ -203,37 +308,31 @@
|
||||
|
||||
- Its certificate will have `CN=system:kube-controller-manager`
|
||||
|
||||
- To improve security posture, each controller can use an individual Service Account
|
||||
|
||||
- This is enabled with flag `--use-service-account-credentials=true`
|
||||
|
||||
---
|
||||
|
||||
## Controller manager keys
|
||||
|
||||
- The controller can create Secrets holding Service Account tokens
|
||||
|
||||
- this is enabled with flag `--service-account-private-key-file`
|
||||
|
||||
- this was used in older versions of Kubernetes (before *bound tokens*)
|
||||
|
||||
- in modern clusters, kubelet uses the `TokenRequest` API instead
|
||||
|
||||
- If we use the CSR API, the controller manager needs the CA cert and key
|
||||
|
||||
(passed with flags `--cluster-signing-cert-file` and `--cluster-signing-key-file`)
|
||||
- the CSR API is used in many clusters to renew kubelet certificates
|
||||
|
||||
- We usually want the controller manager to generate tokens for service accounts
|
||||
|
||||
- These tokens deserve some details (on the next slide!)
|
||||
- it's enabled with `--cluster-signing-cert-file` and `--cluster-signing-key-file`
|
||||
|
||||
---
|
||||
|
||||
class: extra-details
|
||||
|
||||
## How are these permissions set up?
|
||||
|
||||
- A bunch of roles and bindings are defined as constants in the API server code:
|
||||
|
||||
[auth/authorizer/rbac/bootstrappolicy/policy.go](https://github.com/kubernetes/kubernetes/blob/release-1.19/plugin/pkg/auth/authorizer/rbac/bootstrappolicy/policy.go#L188)
|
||||
|
||||
- They are created automatically when the API server starts:
|
||||
|
||||
[registry/rbac/rest/storage_rbac.go](https://github.com/kubernetes/kubernetes/blob/release-1.19/pkg/registry/rbac/rest/storage_rbac.go#L140)
|
||||
|
||||
- We must use the correct Common Names (`CN`) for the control plane certificates
|
||||
|
||||
(since the bindings defined above refer to these common names)
|
||||
|
||||
---
|
||||
|
||||
## Service account tokens
|
||||
|
||||
- Each time we create a service account, the controller manager generates a token
|
||||
## Service account tokens recap
|
||||
|
||||
- These tokens are JWT tokens, signed with a particular key
|
||||
|
||||
@@ -241,13 +340,14 @@ class: extra-details
|
||||
|
||||
(and therefore, the API server needs to be able to verify their integrity)
|
||||
|
||||
- This uses another keypair:
|
||||
- That key is passed to the API server using a couple of flags:
|
||||
|
||||
- the private key (used for signature) is passed to the controller manager
|
||||
<br/>(using flags `--service-account-private-key-file` and `--root-ca-file`)
|
||||
- `--service-account-private-key-file` (used to issue tokens)
|
||||
|
||||
- the public key (used for verification) is passed to the API server
|
||||
<br/>(using flag `--service-account-key-file`)
|
||||
- `--service-account-key-file` (used to verify tokens)
|
||||
|
||||
- The private key is also passed to the controller manager
|
||||
<br/>(using flag `--service-account-private-key-file`)
|
||||
|
||||
---
|
||||
|
||||
@@ -261,8 +361,14 @@ class: extra-details
|
||||
|
||||
- It will authenticate using the token of that Service Account
|
||||
|
||||
- It's also possible (but rare) to run it with e.g. static pods
|
||||
|
||||
(it will then require TLS keys; possibly the same as kubelet's!)
|
||||
|
||||
---
|
||||
|
||||
class: extra-details
|
||||
|
||||
## Webhooks
|
||||
|
||||
- We mentioned webhooks earlier; how does that really work?
|
||||
@@ -283,6 +389,8 @@ class: extra-details
|
||||
|
||||
---
|
||||
|
||||
class: extra-details
|
||||
|
||||
## Subject Access Review
|
||||
|
||||
Here is an example showing how to check if `jean.doe` can `get` some `pods` in `kube-system`:
|
||||
|
||||
Reference in New Issue
Block a user