mirror of
https://github.com/jpetazzo/container.training.git
synced 2026-07-18 12:29:18 +00:00
Add control plane auth info
This commit is contained in:
257
slides/k8s/control-plane-auth.md
Normal file
257
slides/k8s/control-plane-auth.md
Normal file
@@ -0,0 +1,257 @@
|
||||
# Securing the control plane
|
||||
|
||||
- Many components accept connections (and requests) from others:
|
||||
|
||||
- API server
|
||||
|
||||
- etcd
|
||||
|
||||
- kubelet
|
||||
|
||||
- We must secure these connections:
|
||||
|
||||
- to deny unauthorized requests
|
||||
|
||||
- to prevent eavesdropping secrets, tokens, and other sensitive information
|
||||
|
||||
- Disabling authentication and/or authorization is **strongly discouraged**
|
||||
|
||||
(but it's possible to do it, e.g. for learning / troubleshooting purposes)
|
||||
|
||||
---
|
||||
|
||||
## Authentication and authorization
|
||||
|
||||
- Authentication (checking "who you are") is done with mutual TLS
|
||||
|
||||
(both the client and the server need to hold a valid certificate)
|
||||
|
||||
- Authorization (checking "what you can do") is done in different ways
|
||||
|
||||
- the API server implements a sophisticated permission logic (with RBAC)
|
||||
|
||||
- some services will defer authorization to the API server (through webhooks)
|
||||
|
||||
- some services require a certificate signed by a particular CA / sub-CA
|
||||
|
||||
---
|
||||
|
||||
## In practice
|
||||
|
||||
- We will review the various communication channels in the control plane
|
||||
|
||||
- We will indicate how they are secured (client- and server-side)
|
||||
|
||||
- For TLS
|
||||
|
||||
---
|
||||
|
||||
## etcd peers
|
||||
|
||||
- Replication and coordination of etcd happens on a dedicated port
|
||||
|
||||
(typically port 2380; the default port for normal client connections is 2379)
|
||||
|
||||
- Authentication uses TLS certificates with a separate sub-CA
|
||||
|
||||
(otherwise, anyone with a Kubernetes client certificate could access etcd!)
|
||||
|
||||
- The etcd command line flags involved are:
|
||||
|
||||
`--peer-client-cert-auth=true` to activate it
|
||||
|
||||
`--peer-cert-file`, `--peer-key-file`, `--peer-trusted-ca-file`
|
||||
|
||||
---
|
||||
|
||||
## etcd clients
|
||||
|
||||
- The only¹ thing that connects to etcd is the API server
|
||||
|
||||
- Authentication uses TLS certificates with a separate sub-CA
|
||||
|
||||
(for the same reasons as for etcd inter-peer authentication)
|
||||
|
||||
- The etcd command line flags involved are:
|
||||
|
||||
`--client-cert-auth=true` to activate it
|
||||
|
||||
`--trusted-ca-file`, `--cert-file`, `--key-file`
|
||||
|
||||
- The API server command line flags involved are:
|
||||
|
||||
`--etcd-cafile`, `--etcd-certfile`, `--etcd-keyfile`
|
||||
|
||||
.footnote[¹Technically, there is also the etcd healthcheck. Let's ignore it for now.]
|
||||
|
||||
---
|
||||
|
||||
## API server clients
|
||||
|
||||
- The API server has a sophisticated authentication and authorization system
|
||||
|
||||
- For connections coming from other components of the control plane:
|
||||
|
||||
- authentication uses certificates (trusting the certificates' subject or CN)
|
||||
|
||||
- authorization uses whatever mechanism is enabled (most oftentimes, RBAC)
|
||||
|
||||
- The relevant API server flags are:
|
||||
|
||||
`--client-ca-file`, `--tls-cert-file`, `--tls-private-key-file`
|
||||
|
||||
- Each component connecting to the API server takes a `--kubeconfig` flag
|
||||
|
||||
(to specify a kubeconfig file containing the CA cert, client key, and client cert)
|
||||
|
||||
- Yes, that kubeconfig file follows the same format as our `~/.kube/config` file!
|
||||
|
||||
---
|
||||
|
||||
## Kubelet and API server
|
||||
|
||||
- Communication between kubelet and API server can be established both ways
|
||||
|
||||
- Kubelet → API server:
|
||||
|
||||
- kubelet registers itself ("hi, I'm node42, do you have work for me?")
|
||||
|
||||
- connection is kept open and re-established if it breaks
|
||||
|
||||
- that's how the kubelet knows which pods to start/stop
|
||||
|
||||
- API server → kubelet:
|
||||
|
||||
- used to retrieve logs, exec, attach to containers
|
||||
|
||||
---
|
||||
|
||||
## Kubelet → API server
|
||||
|
||||
- Kubelet is started with `--kubeconfig` with API server information
|
||||
|
||||
- The client certificate of the kubelet will typically have:
|
||||
|
||||
`CN=system:node:<nodename>` and groups `O=system:nodes`
|
||||
|
||||
- Nothing special on the API server side
|
||||
|
||||
(it will authenticate like any other client)
|
||||
|
||||
---
|
||||
|
||||
## API server → kubelet
|
||||
|
||||
- Kubelet is started with the flag `--client-ca-file`
|
||||
|
||||
(typically using the same CA as the API server)
|
||||
|
||||
- API server is started with flags `--kubelet-client-certificate` and `--kubelet-client-key`
|
||||
|
||||
- Authorization uses webhooks
|
||||
|
||||
(enabled with `--authorization-mode=Webhook` on kubelet)
|
||||
|
||||
- The webhook server is the API server itself
|
||||
|
||||
(the kubelet sends back a request to the API server to ask, "can this person do that?")
|
||||
|
||||
---
|
||||
|
||||
## Scheduler
|
||||
|
||||
- The scheduler connects to the API server like an ordinary client
|
||||
|
||||
- The certificate of the scheduler will have `CN=system:kube-scheduler`
|
||||
|
||||
---
|
||||
|
||||
## Controller manager
|
||||
|
||||
- The controller manager is also a normal client to the API server
|
||||
|
||||
- Its certificate will have `CN=system:kube-controller-manager`
|
||||
|
||||
- If we use the CSR API, the controller manager needs the CA cert and key
|
||||
|
||||
(passed with flags `--cluster-signing-cert-file` and `--cluster-signing-key-file`)
|
||||
|
||||
- We usually want the controller manager to generate tokens for service accounts
|
||||
|
||||
- These tokens deserve some details (on the next slide!)
|
||||
|
||||
---
|
||||
|
||||
## Service account tokens
|
||||
|
||||
- Each time we create a service account, the controller manager generates a token
|
||||
|
||||
- These tokens are JWT tokens, signed with a particular key
|
||||
|
||||
- These tokens are used for authentication with the API server
|
||||
|
||||
(and therefore, the API server needs to be able to verify their integrity)
|
||||
|
||||
- This uses another keypair:
|
||||
|
||||
- the private key (used for signature) is passed to the controller manager
|
||||
<br/>(using flags `--service-account-private-key-file` and `--root-ca-file`)
|
||||
|
||||
- the public key (used for verification) is passed to the API server
|
||||
<br/>(using flag `--service-account-key-file`)
|
||||
|
||||
---
|
||||
|
||||
## kube-proxy
|
||||
|
||||
- kube-proxy is "yet another API server client"
|
||||
|
||||
- In many clusters, it runs as a Daemon Set
|
||||
|
||||
- In that case, it will have its own Service Account and associated permissions
|
||||
|
||||
- It will authenticate using the token of that Service Account
|
||||
|
||||
---
|
||||
|
||||
## Webhooks
|
||||
|
||||
- We mentioned webhooks earlier; how does that really work?
|
||||
|
||||
- The Kubernetes API has special resource types to check permissions
|
||||
|
||||
- One of them is SubjectAccessReview
|
||||
|
||||
- To check if a particular user can do a particular action on a particular resource:
|
||||
|
||||
- we prepare a SubjectAccessReview object
|
||||
|
||||
- we send that object to the API server
|
||||
|
||||
- the API server responds with allow/deny (and optional explanations)
|
||||
|
||||
- Using webhooks for authorization = sending SAR to authorize each request
|
||||
|
||||
---
|
||||
|
||||
## Subject Access Review
|
||||
|
||||
Here is an example showing how to check if `jean.doe` can `get` some `pods` in `kube-system`:
|
||||
|
||||
```bash
|
||||
kubectl -v9 create -f- <<EOF
|
||||
apiVersion: authorization.k8s.io/v1beta1
|
||||
kind: SubjectAccessReview
|
||||
spec:
|
||||
user: jean.doe
|
||||
group:
|
||||
- foo
|
||||
- bar
|
||||
resourceAttributes:
|
||||
#group: blah.k8s.io
|
||||
namespace: kube-system
|
||||
resource: pods
|
||||
verb: get
|
||||
#name: web-xyz1234567-pqr89
|
||||
EOF
|
||||
```
|
||||
@@ -26,8 +26,7 @@ chapters:
|
||||
- - k8s/multinode.md
|
||||
- k8s/cni.md
|
||||
- k8s/apilb.md
|
||||
#FIXME: check le talk de Laurent Corbes pour voir s'il y a d'autres choses utiles à mentionner
|
||||
#BONUS: intégration CoreDNS pour résoudre les noms des clusters des voisins
|
||||
- k8s/control-plane-auth.md
|
||||
- - k8s/setup-managed.md
|
||||
- k8s/setup-selfhosted.md
|
||||
- k8s/cluster-upgrade.md
|
||||
|
||||
Reference in New Issue
Block a user