mirror of
https://github.com/projectcapsule/capsule.git
synced 2026-08-25 16:07:24 +00:00
docs: fix minor issues
This commit is contained in:
committed by
Dario Tranchitella
parent
d0530bbbe3
commit
bdafbcf90a
@@ -1,5 +1,5 @@
|
||||
# Capsule extension for Lens
|
||||
With Capsule extension for [Lens](https://github.com/lensapp/lens), a cluster administrator can easly manage from a single pane of glass all resources of a Kubernetes cluster, including all the Tenants created through the Capsule Operator.
|
||||
With Capsule extension for [Lens](https://github.com/lensapp/lens), a cluster administrator can easily manage from a single pane of glass all resources of a Kubernetes cluster, including all the Tenants created through the Capsule Operator.
|
||||
|
||||
## Features
|
||||
Capsule extension for Lens provides these capabilities:
|
||||
@@ -8,4 +8,4 @@ Capsule extension for Lens provides these capabilities:
|
||||
- See tenant details and change through the embedded Lens editor
|
||||
- Check Resources Quota and Budget at both the tenant and namespace level
|
||||
|
||||
Please, see the [README](https://raw.githubusercontent.com/clastix/capsule-lens-extension/master/README.md) for details about installation of the Capsule Lens Extension.
|
||||
Please, see the [README](https://github.com/clastix/capsule-lens-extension) for details about the installation of the Capsule Lens Extension.
|
||||
|
||||
@@ -27,7 +27,7 @@ the CRDs manifests, as well the deep copy functions, require _Operator SDK_:
|
||||
the binary has to be installed into your `PATH`.
|
||||
|
||||
### Installing Kubebuilder
|
||||
With the latest release of OperatorSDK there's a more tightly integration with
|
||||
With the latest release of OperatorSDK there's a more tighten integration with
|
||||
Kubebuilder and its opinionated testing suite: ensure to download the latest
|
||||
binaries available from the _Releases_ GitHub page and place them into the
|
||||
`/usr/local/kubebuilder/bin` folder, ensuring this is also in your `PATH`.
|
||||
@@ -97,7 +97,7 @@ You can check if Capsule is running tailing the logs:
|
||||
```
|
||||
|
||||
Since Capsule is built using _OperatorSDK_, logging is handled by the zap
|
||||
module: log verbosity of the Capsule controller can be increased by passing
|
||||
module: log verbosity of the Capsule controller can be increased passing
|
||||
the `--zap-log-level` option with a value from `1` to `10` or the
|
||||
[basic keywords](https://godoc.org/go.uber.org/zap/zapcore#Level) although
|
||||
it is suggested to use the `--zap-devel` flag to get also stack traces.
|
||||
@@ -124,7 +124,7 @@ deployment.apps/capsule-controller-manager scaled
|
||||
> This is mandatory since Capsule uses Leader Election
|
||||
|
||||
#### Providing TLS certificate for webhooks
|
||||
Next step is to replicate the same environment Capsule is expecting in the Pod,
|
||||
The next step is to replicate the same environment Capsule is expecting in the Pod,
|
||||
it means creating a fake certificate to handle HTTP requests.
|
||||
|
||||
``` bash
|
||||
@@ -133,8 +133,8 @@ kubectl -n capsule-system get secret capsule-tls -o jsonpath='{.data.tls\.crt}'
|
||||
kubectl -n capsule-system get secret capsule-tls -o jsonpath='{.data.tls\.key}' | base64 -d > /tmp/k8s-webhook-server/serving-certs/tls.key
|
||||
```
|
||||
|
||||
> We're using the certificates generate upon first installation of Capsule:
|
||||
> it means the Secret will be populated at first start-up.
|
||||
> We're using the certificates generate upon the first installation of Capsule:
|
||||
> it means the Secret will be populated at the first start-up.
|
||||
> If you plan to run it locally since the beginning, it means you will require
|
||||
> to provide a self-signed certificate in the said directory.
|
||||
|
||||
@@ -242,10 +242,10 @@ A commit description is welcomed to explain more the changes: just ensure
|
||||
to put a blank line and an arbitrary number of maximum 72 characters long
|
||||
lines, at most one blank line between them.
|
||||
|
||||
Please, split changes into several and documented small commits: this will help us to perform a better review. Commits must follow the the Conventional Commits Specification, a lightweight convention on top of commit messages. It provides an easy set of rules for creating an explicit commit history; which makes it easier to write automated tools on top of. This convention dovetails with Semantic Versioning, by describing the features, fixes, and breaking changes made in commit messages. See [Conventional Commits Specification](https://www.conventionalcommits.org) to learn about Conventional Commits.
|
||||
Please, split changes into several and documented small commits: this will help us to perform a better review. Commits must follow the Conventional Commits Specification, a lightweight convention on top of commit messages. It provides an easy set of rules for creating an explicit commit history; which makes it easier to write automated tools on top of. This convention dovetails with Semantic Versioning, by describing the features, fixes, and breaking changes made in commit messages. See [Conventional Commits Specification](https://www.conventionalcommits.org) to learn about Conventional Commits.
|
||||
|
||||
> In case of errors or need of changes to previous commits,
|
||||
> fix them squashing to make changes atomic.
|
||||
|
||||
### Miscellanea
|
||||
Please, add a new single line at end of any file as current coding style.
|
||||
Please, add a new single line at end of any file as the current coding style.
|
||||
|
||||
@@ -19,10 +19,10 @@ $ kubectl apply -f https://raw.githubusercontent.com/clastix/capsule/master/conf
|
||||
It will install the Capsule controller in a dedicated namespace `capsule-system`.
|
||||
|
||||
### Install with Helm Chart
|
||||
Please, refer to the instructions reported into the Capsule Helm Chart [README](https://github.com/clastix/capsule/blob/master/charts/capsule/README.md).
|
||||
Please, refer to the instructions reported in the Capsule Helm Chart [README](https://github.com/clastix/capsule/blob/master/charts/capsule/README.md).
|
||||
|
||||
# Create your first Tenant
|
||||
In Capsule, a _Tenant_ is an abstraction to group togheter multiple namespaces in a single entity within a set of bundaries defined by the Cluster Administrator. The tenant is then assigned to a user or group of users who is called _Tenant Owner_.
|
||||
In Capsule, a _Tenant_ is an abstraction to group multiple namespaces in a single entity within a set of boundaries defined by the Cluster Administrator. The tenant is then assigned to a user or group of users who is called _Tenant Owner_.
|
||||
|
||||
Capsule defines a Tenant as Custom Resource with cluster scope.
|
||||
|
||||
@@ -99,7 +99,7 @@ $ kubectl -n oil-development run nginx --image=docker.io/nginx
|
||||
$ kubectl -n oil-development get pods
|
||||
```
|
||||
|
||||
but limited to only your own namespaces:
|
||||
but limited to only your namespaces:
|
||||
|
||||
```
|
||||
$ kubectl -n kube-system get pods
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# Capsule on AWS EKS
|
||||
This is an example how to install AWS EKS cluster and one user
|
||||
This is an example of how to install AWS EKS cluster and one user
|
||||
manged by Capsule.
|
||||
|
||||
It is based on [Using IAM Groups to manage Kubernetes access](https://www.eksworkshop.com/beginner/091_iam-groups/intro/)
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# Capsule over Managed Kubernetes
|
||||
Capsule Operator can be easly installed on a Managed Kubernetes Service. Since in these services, you do not have access to the Kubernetes APIs Server, you should check with your service provider following pre-requisites:
|
||||
Capsule Operator can be easily installed on a Managed Kubernetes Service. Since in these services, you do not have access to the Kubernetes APIs Server, you should check with your service provider following pre-requisites:
|
||||
|
||||
- the default `cluster-admin` ClusterRole is accessible
|
||||
- the following Admission Webhooks are enabled on the APIs Server:
|
||||
@@ -8,8 +8,6 @@ Capsule Operator can be easly installed on a Managed Kubernetes Service. Since i
|
||||
- ResourceQuota
|
||||
- MutatingAdmissionWebhook
|
||||
- ValidatingAdmissionWebhook
|
||||
- PodSecurityPolicy (optional)
|
||||
- a CNI plugin with support for Network Policies can be used.
|
||||
|
||||
* [AWS EKS](./aws-eks.md)
|
||||
* CoAKS - Capsule over Azure Kubernetes Service
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Monitoring Capsule
|
||||
|
||||
The Capsule dashboard allows you to track the health and performance of Capsule manager and tenants, with particular attention to resources saturation, server responses and latencies.
|
||||
The Capsule dashboard allows you to track the health and performance of Capsule manager and tenants, with particular attention to resources saturation, server responses, and latencies.
|
||||
|
||||
## Requirements
|
||||
|
||||
@@ -76,7 +76,7 @@ making sure to select the correct Prometheus data source:
|
||||
|
||||
##### Description
|
||||
|
||||
This section provides information about the medium time delay between manager client input, side effects and new state determination (reconciliation).
|
||||
This section provides information about the medium time delay between manager client input, side effects, and new state determination (reconciliation).
|
||||
|
||||
##### Dependant variables and available values
|
||||
|
||||
@@ -117,7 +117,7 @@ This section provides information about webhook requests response, mainly focusi
|
||||
|
||||
##### Description
|
||||
|
||||
This section provides information about the medium time delay between webhook trigger, side effects and data written on etcd.
|
||||
This section provides information about the medium time delay between webhook trigger, side effects, and data written on etcd.
|
||||
|
||||
##### Dependant variables and available values
|
||||
|
||||
@@ -139,7 +139,7 @@ This section provides information about the medium time delay between webhook tr
|
||||
##### Description
|
||||
|
||||
This section provides information about the medium time delay between all the calls done by the controller and the API server.
|
||||
Data display may depends on the REST client verb considered and on available REST client URLs.
|
||||
Data display may depend on the REST client verb considered and on available REST client URLs.
|
||||
|
||||
YMMV
|
||||
|
||||
@@ -175,7 +175,7 @@ This section provides information about resources, giving a detailed picture of
|
||||
|
||||
##### Description
|
||||
|
||||
This section provides information about "actions" in queue, particularly:
|
||||
This section provides information about "actions" in the queue, particularly:
|
||||
- Workqueue latency: time to complete a series of actions in the queue ;
|
||||
- Workqueue rate: number of actions per unit time ;
|
||||
- Workqueue depth: number of pending actions waiting in the queue.
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
|
||||
**Category:** Self-Service Operations
|
||||
|
||||
**Description:** Tenants should be able to perform self-service operations by creating own network policies in their namespaces.
|
||||
**Description:** Tenants should be able to perform self-service operations by creating their own network policies in their namespaces.
|
||||
|
||||
**Rationale:** Enables self-service management of network-policies.
|
||||
|
||||
@@ -56,7 +56,7 @@ NAME POD-SELECTOR AGE
|
||||
capsule-oil-0 <none> 7m5s
|
||||
```
|
||||
|
||||
As tenant owner check for permissions to manage networkpolicy for each verb
|
||||
As a tenant, checks for permissions to manage networkpolicy for each verb
|
||||
|
||||
```bash
|
||||
kubectl --kubeconfig alice auth can-i get networkpolicies
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
|
||||
**Category:** Self-Service Operations
|
||||
|
||||
**Description:** Tenants should be able to perform self-service operations by creating own rolebindings in their namespaces.
|
||||
**Description:** Tenants should be able to perform self-service operations by creating their rolebindings in their namespaces.
|
||||
|
||||
**Rationale:** Enables self-service management of roles.
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
|
||||
**Category:** Self-Service Operations
|
||||
|
||||
**Description:** Tenants should be able to perform self-service operations by creating own roles in their namespaces.
|
||||
**Description:** Tenants should be able to perform self-service operations by creating their own roles in their namespaces.
|
||||
|
||||
**Rationale:** Enables self-service management of roles.
|
||||
|
||||
@@ -37,7 +37,7 @@ kubectl --kubeconfig alice create ns oil-production
|
||||
kubectl --kubeconfig alice config set-context --current --namespace oil-production
|
||||
```
|
||||
|
||||
As tenant owner check for permissions to manage roles for each verb
|
||||
As tenant owner, check for permissions to manage roles for each verb
|
||||
|
||||
```bash
|
||||
kubectl --kubeconfig alice auth can-i get roles
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
|
||||
**Category:** Control Plane Isolation
|
||||
|
||||
**Description:** Tenants should not be able to view, edit, create, or delete cluster (non-namespaced) resources such Node, ClusterRole, ClusterRoleBinding, etc.
|
||||
**Description:** Tenants should not be able to view, edit, create or delete cluster (non-namespaced) resources such Node, ClusterRole, ClusterRoleBinding, etc.
|
||||
|
||||
**Rationale:** Access controls should be configured for tenants so that a tenant cannot list, create, modify or delete cluster resources
|
||||
|
||||
@@ -53,7 +53,7 @@ kubectl --kubeconfig alice auth can-i create namespaces
|
||||
yes
|
||||
```
|
||||
|
||||
Any kubernetes user can create `SelfSubjectAccessReview` and `SelfSubjectRulesReviews` to checks whether he/she can perform an action. First two exceptions are not an issue.
|
||||
Any kubernetes user can create `SelfSubjectAccessReview` and `SelfSubjectRulesReviews` to checks whether he/she can act. First, two exceptions are not an issue.
|
||||
|
||||
```bash
|
||||
kubectl --anyuser auth can-i --list
|
||||
@@ -78,7 +78,7 @@ selfsubjectrulesreviews.authorization.k8s.io [] []
|
||||
[/version] [] [get]
|
||||
```
|
||||
|
||||
In order to enable namespace self-service provisioning, Capsule intentionally gives permissions to create namespaces to all users belonging to the Capsule group:
|
||||
To enable namespace self-service provisioning, Capsule intentionally gives permissions to create namespaces to all users belonging to the Capsule group:
|
||||
|
||||
```bash
|
||||
kubectl describe clusterrolebindings capsule-namespace-provisioner
|
||||
|
||||
@@ -6,9 +6,9 @@
|
||||
|
||||
**Category:** Tenant Isolation
|
||||
|
||||
**Description:** Each tenant namespace may contain resources setup by the cluster administrator for multi-tenancy, such as role bindings, and network policies. Tenants should not be allowed to modify the namespaced resources created by the cluster administrator for multi-tenancy. However, for some resources such as network policies, tenants can configure additional instances of the resource for their workloads.
|
||||
**Description:** Each tenant namespace may contain resources set up by the cluster administrator for multi-tenancy, such as role bindings, and network policies. Tenants should not be allowed to modify the namespaced resources created by the cluster administrator for multi-tenancy. However, for some resources such as network policies, tenants can configure additional instances of the resource for their workloads.
|
||||
|
||||
**Rationale:** Tenants can escalate priviliges and impact other tenants if they are able to delete or modify required multi-tenancy resources such as namespace resource quotas or default network policy.
|
||||
**Rationale:** Tenants can escalate privileges and impact other tenants if they can delete or modify required multi-tenancy resources such as namespace resource quotas or default network policy.
|
||||
|
||||
**Audit:**
|
||||
|
||||
@@ -99,7 +99,7 @@ spec:
|
||||
EOF
|
||||
```
|
||||
|
||||
However, due the additive nature of networkpolicies, the `DENY ALL` policy set by the cluster admin, prevents the hijacking.
|
||||
However, due to the additive nature of networkpolicies, the `DENY ALL` policy set by the cluster admin, prevents hijacking.
|
||||
|
||||
As tenant owner list RBAC permissions set by Capsule
|
||||
|
||||
@@ -110,13 +110,13 @@ namespace-deleter ClusterRole/capsule-namespace-deleter 11h
|
||||
namespace:admin ClusterRole/admin 11h
|
||||
```
|
||||
|
||||
As tenant owner, try to change/delete the rolebindings in order to escalate permissions
|
||||
As tenant owner, try to change/delete the rolebinding to escalate permissions
|
||||
|
||||
```bash
|
||||
kubectl --kubeconfig alice edit/delete rolebinding namespace:admin
|
||||
```
|
||||
|
||||
The rolebindings is immediately recreated by Capsule:
|
||||
The rolebinding is immediately recreated by Capsule:
|
||||
|
||||
```
|
||||
kubectl --kubeconfig alice get rolebindings
|
||||
@@ -125,7 +125,7 @@ namespace-deleter ClusterRole/capsule-namespace-deleter 11h
|
||||
namespace:admin ClusterRole/admin 2s
|
||||
```
|
||||
|
||||
However, the tenant owner can create and assign permissions inside namespace she owns
|
||||
However, the tenant owner can create and assign permissions inside the namespace she owns
|
||||
|
||||
```yaml
|
||||
kubectl create -f - << EOF
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
|
||||
**Category:** Tenant Isolation
|
||||
|
||||
**Description:** Each tenant has its own set of resources, such as namespaces, service accounts, secrets, pods, services, etc. Tenants should not be allowed to access eachother's resources.
|
||||
**Description:** Each tenant has its own set of resources, such as namespaces, service accounts, secrets, pods, services, etc. Tenants should not be allowed to access each other's resources.
|
||||
|
||||
**Rationale:** Tenant's resources must be not accessible by other tenants.
|
||||
|
||||
@@ -69,10 +69,10 @@ As `oil` tenant owner, try to retrieve the resources in the `gas` tenant namespa
|
||||
kubectl --kubeconfig alice get serviceaccounts --namespace gas-production
|
||||
```
|
||||
|
||||
You must receive an eror message:
|
||||
You must receive an error message:
|
||||
|
||||
```
|
||||
Error from server (Forbidden): serviceaccounts is forbidden:
|
||||
Error from server (Forbidden): serviceaccount is forbidden:
|
||||
User "oil" cannot list resource "serviceaccounts" in API group "" in the namespace "gas-production"
|
||||
```
|
||||
|
||||
@@ -82,10 +82,10 @@ As `gas` tenant owner, try to retrieve the resources in the `oil` tenant namespa
|
||||
kubectl --kubeconfig joe get serviceaccounts --namespace oil-production
|
||||
```
|
||||
|
||||
You must receive an eror message:
|
||||
You must receive an error message:
|
||||
|
||||
```
|
||||
Error from server (Forbidden): serviceaccounts is forbidden:
|
||||
Error from server (Forbidden): serviceaccount is forbidden:
|
||||
User "joe" cannot list resource "serviceaccounts" in API group "" in the namespace "oil-production"
|
||||
```
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
|
||||
**Description:** Control container permissions.
|
||||
|
||||
**Rationale:** The security `allowPrivilegeEscalation` setting allows a process to gain more privileges from its parent process. Processes in tenant containers should not be allowed to gain additional priviliges.
|
||||
**Rationale:** The security `allowPrivilegeEscalation` setting allows a process to gain more privileges from its parent process. Processes in tenant containers should not be allowed to gain additional privileges.
|
||||
|
||||
**Audit:**
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
|
||||
**Description:** Avoid a tenant to mount existing volumes`.
|
||||
|
||||
**Rationale:** Tenants have to be assured that their Persisten Volumes cannot be reclaimed by other tenants.
|
||||
**Rationale:** Tenants have to be assured that their Persistent Volumes cannot be reclaimed by other tenants.
|
||||
|
||||
**Audit:**
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
|
||||
**Description:** Tenants should not be able to mount host volumes and directories.
|
||||
|
||||
**Rationale:** The use of host volumes and directories can be used to access shared data or escalate priviliges and also creates a tight coupling between a tenant workload and a host.
|
||||
**Rationale:** The use of host volumes and directories can be used to access shared data or escalate privileges and also creates a tight coupling between a tenant workload and a host.
|
||||
|
||||
**Audit:**
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
|
||||
**Description:** Force a tenant to use a Storage Class with `reclaimPolicy=Delete`.
|
||||
|
||||
**Rationale:** Tenants have to be assured that their Persisten Volumes cannot be reclaimed by other tenants.
|
||||
**Rationale:** Tenants have to be assured that their Persistent Volumes cannot be reclaimed by other tenants.
|
||||
|
||||
**Audit:**
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
|
||||
**Description:** Control container permissions.
|
||||
|
||||
**Rationale:** Processes in containers run as the root user (uid 0), by default. To prevent potential compromise of container hosts, specify a least privileged user ID when building the container image and require that application containers run as non root users.
|
||||
**Rationale:** Processes in containers run as the root user (uid 0), by default. To prevent potential compromise of container hosts, specify a least-privileged user ID when building the container image and require that application containers run as non-root users.
|
||||
|
||||
**Audit:**
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# Meet the multi-tenancy benchmark MTB
|
||||
Actually, there's no yet a real standard for the multi-tenancy model in Kubernetes, although the [SIG multi-tenancy group](https://github.com/kubernetes-sigs/multi-tenancy) is working on that. SIG multi-tenancy drafted a generic validation schema appliable to generic multi-tenancy projects. Multi-Tenancy Benchmarks [MTB](https://github.com/kubernetes-sigs/multi-tenancy/tree/master/benchmarks) are guidelines for multi-tenant configuration of Kubernetes clusters. Capsule is an open source multi-tenacy operator and we decided to meet the requirements of MTB.
|
||||
Actually, there's no yet a real standard for the multi-tenancy model in Kubernetes, although the [SIG multi-tenancy group](https://github.com/kubernetes-sigs/multi-tenancy) is working on that. SIG multi-tenancy drafted a generic validation schema appliable to generic multi-tenancy projects. Multi-Tenancy Benchmarks [MTB](https://github.com/kubernetes-sigs/multi-tenancy/tree/master/benchmarks) are guidelines for multi-tenant configuration of Kubernetes clusters. Capsule is an open source multi-tenancy operator and we decided to meet the requirements of MTB.
|
||||
|
||||
> N.B. At time of writing, the MTB are in development and not ready for usage. Strictly speaking, we do not claim an official conformance to MTB, but just to adhere to the multi-tenancy requirements and best practices promoted by MTB.
|
||||
> N.B. At the time of writing, the MTB is in development and not ready for usage. Strictly speaking, we do not claim official conformance to MTB, but just to adhere to the multi-tenancy requirements and best practices promoted by MTB.
|
||||
|
||||
|MTB Benchmark |MTB Profile|Capsule Version|Conformance|Notes |
|
||||
|--------------|-----------|---------------|-----------|-------|
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
* [Getting Started](./getting-started.md)
|
||||
* [Use Cases](./use-cases/overview.md)
|
||||
* [SIG Multi-tenancy benchmark](./mtb/sig-multitenancy-bench.md)
|
||||
* [Run on Managed Kubernetes Services](./managed-kubernetes/getting-started-amazon-eks.md)
|
||||
* [Run on Managed Kubernetes Services](./managed-kubernetes/overview.md)
|
||||
* [Monitoring Capsule](./monitoring.md)
|
||||
* [References](./references.md)
|
||||
* [Contributing](./contributing.md)
|
||||
|
||||
@@ -9,7 +9,7 @@
|
||||
|
||||
## Custom Resource Definition
|
||||
|
||||
Capsule operator uses a Custom Resources Definition (CRD) for _Tenants_. In Caspule, Tenants are cluster wide resources. You need for cluster level permissions to work with tenants.
|
||||
Capsule operator uses a Custom Resources Definition (CRD) for _Tenants_. In Capsule, Tenants are cluster wide resources. You need cluster level permissions to work with tenants.
|
||||
|
||||
You can learn about tenant CRD by the `kubectl explain` command:
|
||||
|
||||
@@ -24,15 +24,11 @@ DESCRIPTION:
|
||||
|
||||
FIELDS:
|
||||
apiVersion <string>
|
||||
APIVersion defines the versioned schema of this representation of an
|
||||
object. Servers should convert recognized schemas to the latest internal
|
||||
value, and may reject unrecognized values. More info:
|
||||
APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info:
|
||||
https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
|
||||
|
||||
kind <string>
|
||||
Kind is a string value representing the REST resource this object
|
||||
represents. Servers may infer this from the endpoint the client submits
|
||||
requests to. Cannot be updated. In CamelCase. More info:
|
||||
Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info:
|
||||
https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
|
||||
|
||||
metadata <Object>
|
||||
@@ -205,7 +201,7 @@ capsule-mutating-webhook-configuration 1 2h
|
||||
|
||||
## Command Options
|
||||
|
||||
The Capsule operator provides following command options:
|
||||
The Capsule operator provides the following command options:
|
||||
|
||||
Option | Description | Default
|
||||
--- | --- | ---
|
||||
@@ -213,33 +209,8 @@ Option | Description | Default
|
||||
`--enable-leader-election` | Start a leader election client and gain leadership before executing the main loop. | `true`
|
||||
`--zap-log-level` | The log verbosity with a value from 1 to 10 or the basic keywords. | `4`
|
||||
`--zap-devel` | The flag to get the stack traces for deep debugging. | `null`
|
||||
`--configuration-name` | The Capsule Configuration CRD name, a default is installed automatically | `capsule-default`
|
||||
`--configuration-name` | The Capsule Configuration CRD name, default is installed automatically | `capsule-default`
|
||||
|
||||
## Capsule Configuration
|
||||
|
||||
The Capsule configuration can be piloted by a Custom Resource definition named `CapsuleConfiguration`.
|
||||
|
||||
```yaml
|
||||
apiVersion: capsule.clastix.io/v1alpha1
|
||||
kind: CapsuleConfiguration
|
||||
metadata:
|
||||
name: default
|
||||
spec:
|
||||
userGroups: ["capsule.clastix.io"]
|
||||
forceTenantPrefix: false
|
||||
protectedNamespaceRegex: ""
|
||||
```
|
||||
|
||||
Option | Description | Default
|
||||
--- | --- | ---
|
||||
`.spec.forceTenantPrefix` | Force the tenant name as prefix for namespaces: `<tenant_name>-<namespace>`. | `false`
|
||||
`.spec.userGroups` | Array of Capsule groups to which all tenant owners must belong. | `[capsule.clastix.io]`
|
||||
`.spec.protectedNamespaceRegex` | Disallows creation of namespaces matching the passed regexp. | `null`
|
||||
|
||||
Upon installation using Kustomize or Helm, a `default` resource will be created.
|
||||
The reference to this configuration is managed by the CLI flag `--configuration-name`.
|
||||
|
||||
>>>>>>> 0f3f6da... docs: updates structure for documentation
|
||||
|
||||
## Created Resources
|
||||
Once installed, the Capsule operator creates the following resources in your cluster:
|
||||
|
||||
@@ -3,13 +3,13 @@
|
||||
Bill needs to cordon a Tenant and its Namespaces for several reasons:
|
||||
|
||||
- Avoid accidental resource modification(s) including deletion during a Production Freeze Window
|
||||
- During Kubernetes upgrade, to prevent any workload updates
|
||||
- During the Kubernetes upgrade, to prevent any workload updates
|
||||
- During incidents or outages
|
||||
- During planned maintenance of a dedicated nodes pool in a BYOD scenario
|
||||
|
||||
With this said, the Tenant Owner and the related Service Account living into managed Namespaces, cannot proceed to any update, create or delete action.
|
||||
|
||||
This is possible just labelling the Tenant as follows:
|
||||
This is possible just labeling the Tenant as follows:
|
||||
|
||||
```shell
|
||||
kubectl label tenant oil capsule.clastix.io/cordon=enabled
|
||||
@@ -20,13 +20,13 @@ Any operation performed by Alice, the Tenant Owner, will be rejected.
|
||||
|
||||
```shell
|
||||
$ kubectl --as alice --as-group capsule.clastix.io -n oil-dev create deployment nginx --image nginx
|
||||
error: failed to create deployment: admission webhook "cordoning.tenant.capsule.clastix.io" denied the request: tenant oil is freezed: please, reach out to the system administrator
|
||||
error: failed to create deployment: admission webhook "cordoning.tenant.capsule.clastix.io" denied the request: tenant oil is frozen: please, reach out to the system administrator
|
||||
|
||||
$ kubectl --as alice --as-group capsule.clastix.io -n oil-dev delete ingress,deployment,serviceaccount --all
|
||||
error: failed to create deployment: admission webhook "cordoning.tenant.capsule.clastix.io" denied the request: tenant oil is freezed: please, reach out to the system administrator
|
||||
error: failed to create deployment: admission webhook "cordoning.tenant.capsule.clastix.io" denied the request: tenant oil is frozen: please, reach out to the system administrator
|
||||
```
|
||||
|
||||
Uncordoning can be done removing the said label:
|
||||
Uncordoning can be done by removing the said label:
|
||||
|
||||
```shell
|
||||
$ kubectl label tenant oil capsule.clastix.io/cordon-
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
# Create namespaces
|
||||
Alice, once logged with her own credentials, can create a new namespace in her tenant, as simply issuing:
|
||||
Alice, once logged with her credentials, can create a new namespace in her tenant, as simply issuing:
|
||||
|
||||
```
|
||||
kubectl create ns oil-production
|
||||
```
|
||||
|
||||
Alice started the name of the namespace prepended by the name of the tenant: this is not a strict requirement but it is highly suggested because it is likely that many different tenants would like to call their namespaces as `production`, `test`, or `demo`, etc.
|
||||
Alice started the name of the namespace prepended by the name of the tenant: this is not a strict requirement but it is highly suggested because it is likely that many different tenants would like to call their namespaces `production`, `test`, or `demo`, etc.
|
||||
|
||||
The enforcement of this naming convention is optional and can be controlled by the cluster administrator with the `--force-tenant-prefix` option as an argument of the Capsule controller.
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# Create Custom Resources
|
||||
Capsule grants admin permissions to the tenant owners but only limited to their namespaces. To achieve that, it assigns the ClusterRole [admin](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) to the tenant owner. This ClusterRole does not permit the installation of custom resources in the namespaces.
|
||||
Capsule grants admin permissions to the tenant owners but is only limited to their namespaces. To achieve that, it assigns the ClusterRole [admin](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) to the tenant owner. This ClusterRole does not permit the installation of custom resources in the namespaces.
|
||||
|
||||
In order to leave the tenant owner to create Custom Resources in their namespaces, the cluster admin defines a proper Cluster Role. For example:
|
||||
|
||||
@@ -75,4 +75,4 @@ With the above example, Capsule is leaving the tenant owner to create namespaced
|
||||
> Take Note: a tenant owner having the admin scope on its namespaces only, does not have the permission to create Custom Resources Definitions (CRDs) because this requires a cluster admin permission level. Only Bill, the cluster admin, can create CRDs. This is a known limitation of any multi-tenancy environment based on a single Kubernetes cluster.
|
||||
|
||||
# What’s next
|
||||
See how Bill, the cluster admin, can set taints on the Alice's namespaces. [Taint namespaces](./taint-namespaces.md).
|
||||
See how Bill, the cluster admin, can set taints on Alice's namespaces. [Taint namespaces](./taint-namespaces.md).
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# Control hostname collision in ingresses
|
||||
In a multi-tenant environment, as more and more ingresses are defined, there is a chance of collision on the hostname leading to an unpredictable behaviour of the Ingress Controller. Bill, the cluster admin, can enforce hostname collision detection at different scope level:
|
||||
In a multi-tenant environment, as more and more ingresses are defined, there is a chance of collision on the hostname leading to unpredictable behavior of the Ingress Controller. Bill, the cluster admin, can enforce hostname collision detection at different scope levels:
|
||||
|
||||
1. Cluster
|
||||
2. Tenant
|
||||
@@ -25,7 +25,7 @@ EOF
|
||||
|
||||
When a tenant owner creates an Ingress resource, Capsule will check the collision of hostname in the current ingress with all the hostnames already used, depending on the defined scope.
|
||||
|
||||
For example, Alice, one of the tenant owner, creates an Ingress
|
||||
For example, Alice, one of the tenant owners, creates an Ingress
|
||||
|
||||
|
||||
```yaml
|
||||
|
||||
@@ -4,7 +4,7 @@ Bill is a cluster admin providing a Container as a Service platform using shared
|
||||
|
||||
Alice, a Tenant Owner, can start container images using private images: according to the Kubernetes architecture, the `kubelet` will download the layers on its cache.
|
||||
|
||||
Bob, an attacker, could try to schedule a Pod on the same node where Alice is running her Pods backed by private images: they could start new Pods using `ImagePullPolicy=IfNotPresent` and able to start them, even without required authentication since the image is cached on the node.
|
||||
Bob, an attacker, could try to schedule a Pod on the same node where Alice is running her Pods backed by private images: they could start new Pods using `ImagePullPolicy=IfNotPresent` and be able to start them, even without required authentication since the image is cached on the node.
|
||||
|
||||
To avoid this kind of attack, Bill, the cluster admin, can force Alice, the tenant owner, to start her Pods using only the allowed values for `ImagePullPolicy`, enforcing the `kubelet` to check the authorization first.
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# Assign Trusted Images Registries
|
||||
Bill, the cluster admin, can set a strict policy on the applications running into Alice's tenant: he'd like to allow running just images hosted on a list of specific container registries.
|
||||
|
||||
The spec `containerRegistries` addresses this task and can provide combination with hard enforcement using a list of allowed values.
|
||||
The spec `containerRegistries` addresses this task and can provide a combination with hard enforcement using a list of allowed values.
|
||||
|
||||
|
||||
```yaml
|
||||
@@ -21,14 +21,14 @@ spec:
|
||||
allowedRegex: 'internal.registry.\\w.tld'
|
||||
```
|
||||
|
||||
> In case of `non FQDI` (non fully qualified Docker image) and official images hosted on Docker Hub,
|
||||
> In case of `non-FQDI` (non fully qualified Docker image) and official images hosted on Docker Hub,
|
||||
> Capsule is going to retrieve the registry even if it's not explicit: a `busybox:latest` Pod
|
||||
> running on a Tenant allowing `docker.io` will not be blocked, even if the image
|
||||
> field is not explicit as `docker.io/busybox:latest`.
|
||||
|
||||
A Pod running `internal.registry.foo.tld` as registry will be allowed, as well `internal.registry.bar.tld` since these are matching the regular expression.
|
||||
|
||||
> A catch-all regex entry as `.*` allows every kind of registry, that would be the same result of unsetting `containerRegistries` at all.
|
||||
> A catch-all regex entry as `.*` allows every kind of registry, which would be the same result of unsetting `containerRegistries` at all.
|
||||
|
||||
Any attempt of Alice to use a not allowed `containerRegistries` value is denied by the Validation Webhook enforcing it.
|
||||
|
||||
|
||||
@@ -46,7 +46,7 @@ spec:
|
||||
EOF
|
||||
```
|
||||
|
||||
Any attempt of Alice to use a non valid Ingress Class, or missing it, is denied by the Validation Webhook enforcing it.
|
||||
Any attempt of Alice to use a non-valid Ingress Class, or missing it, is denied by the Validation Webhook enforcing it.
|
||||
|
||||
# What’s next
|
||||
See how Bill, the cluster admin, can assign a set of dedicated ingress hostnames to Alice's tenant. [Assign Ingress Hostnames](./ingress-hostnames.md).
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# Assign Ingress Hostnames
|
||||
Bill can control ingress hostnames to the `oil` tenant in order to force the applications to be published only using the given hostname or set of hostnames:
|
||||
Bill can control ingress hostnames to the `oil` tenant to force the applications to be published only using the given hostname or set of hostnames:
|
||||
|
||||
```yaml
|
||||
kubectl apply -f - << EOF
|
||||
@@ -47,7 +47,7 @@ spec:
|
||||
EOF
|
||||
```
|
||||
|
||||
Any attempt of Alice to use a non valid hostname is denied by the Validation Webhook enforcing it.
|
||||
Any attempt of Alice to use a non-valid hostname is denied by the Validation Webhook enforcing it.
|
||||
|
||||
# What’s next
|
||||
See how Bill, the cluster admin, can control the hostname collision in Ingresses. [Control hostname collision in ingresses](./hostname-collision.md).
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# Assign multiple tenants to an owner
|
||||
In some scenarios, it's likely that a single team is responsible for multiple lines of business. For example, in our sample organization Acme Corp., Alice is responsible for both the Oil and Gas lines of business. It's more likely that Alice requires two different tenants, for example `oil` and `gas` to keep things isolated.
|
||||
In some scenarios, a single team is likely responsible for multiple lines of business. For example, in our sample organization Acme Corp., Alice is responsible for both the Oil and Gas lines of business. It's more likely that Alice requires two different tenants, for example, `oil` and `gas` to keep things isolated.
|
||||
|
||||
By design, the Capsule operator does not permit hierarchy of tenants, since all tenants are at the same levels. However, we can assign the ownership of multiple tenants to the same user or group of users.
|
||||
By design, the Capsule operator does not permit a hierarchy of tenants, since all tenants are at the same levels. However, we can assign the ownership of multiple tenants to the same user or group of users.
|
||||
|
||||
Bill, the cluster admin, creates multiple tenants having `alice` as owner:
|
||||
|
||||
@@ -63,7 +63,7 @@ spec:
|
||||
EOF
|
||||
```
|
||||
|
||||
The two tenants still remain isolated each other in terms of resources assignments, e.g. _ResourceQuota_, _Nodes Pool_, _Storage Calsses_ and _Ingress Classes_, and in terms of governance, e.g. _NetworkPolicies_, _PodSecurityPolicies_, _Trusted Registries_, etc.
|
||||
The two tenants remain isolated from each other in terms of resources assignments, e.g. _ResourceQuota_, _Nodes Pool_, _Storage Calsses_ and _Ingress Classes_, and in terms of governance, e.g. _NetworkPolicies_, _PodSecurityPolicies_, _Trusted Registries_, etc.
|
||||
|
||||
|
||||
When Alice logs in, she has access to all namespaces belonging to both the `oil` and `gas` tenants.
|
||||
@@ -73,7 +73,7 @@ kubectl create ns oil-production
|
||||
kubectl create ns gas-production
|
||||
```
|
||||
|
||||
When the enforcement of the naming convention with the `--force-tenant-prefix` option, is enabled, the namespaces are automatically assigned to the right tenant by Capsule because the operator does a lookups on the tenant names. If the `--force-tenant-prefix` option, is not set, Alice needs to specify the tenant name as a label `capsule.clastix.io/tenant=<desired_tenant>` in the namespace manifest:
|
||||
When the enforcement of the naming convention with the `--force-tenant-prefix` option, is enabled, the namespaces are automatically assigned to the right tenant by Capsule because the operator does a lookup on the tenant names. If the `--force-tenant-prefix` option, is not set, Alice needs to specify the tenant name as a label `capsule.clastix.io/tenant=<desired_tenant>` in the namespace manifest:
|
||||
|
||||
```yaml
|
||||
kubectl apply -f - << EOF
|
||||
|
||||
@@ -3,7 +3,7 @@ Kubernetes network policies control network traffic between namespaces and betwe
|
||||
|
||||
To meet this requirement, Bill needs to define network policies that deny pods belonging to Alice's namespaces to access pods in namespaces belonging to other tenants, e.g. Bob's tenant `water`, or in system namespaces, e.g. `kube-system`.
|
||||
|
||||
Also, Bill can make sure pods belonging to a tenant namespace cannot access other network infrastructure like cluster nodes, load balancers, and virtual machines running other services.
|
||||
Also, Bill can make sure pods belonging to a tenant namespace cannot access other network infrastructures like cluster nodes, load balancers, and virtual machines running other services.
|
||||
|
||||
Bill can set network policies in the tenant manifest, according to the requirements:
|
||||
|
||||
@@ -96,7 +96,7 @@ And delete the namespace network policies
|
||||
kubectl -n oil-production delete networkpolicy production-network-policy
|
||||
```
|
||||
|
||||
Any attempt of Alice to delete the tenant network policy defined in the tenant manifest, is denied by the Validation Webhook enforcing it.
|
||||
Any attempt of Alice to delete the tenant network policy defined in the tenant manifest is denied by the Validation Webhook enforcing it.
|
||||
|
||||
# What’s next
|
||||
See how Bill can enforce the Pod containers image pull policy to `Always` to avoid leaking of private images when running on shared nodes. [Enforcing Pod containers image PullPolicy](./images-pullpolicy.md)
|
||||
|
||||
@@ -51,9 +51,9 @@ spec:
|
||||
hardware: gpu
|
||||
```
|
||||
|
||||
Any attempt of Alice to change the selector on the pods will result an error from the `PodNodeSelector` Admission Controller plugin.
|
||||
Any attempt of Alice to change the selector on the pods will result in an error from the `PodNodeSelector` Admission Controller plugin.
|
||||
|
||||
Also RBAC prevents Alice to change the annotation on the namespace:
|
||||
Also, RBAC prevents Alice to change the annotation on the namespace:
|
||||
|
||||
```
|
||||
kubectl auth can-i edit ns -n oil-production
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
# Onboard a new tenant
|
||||
Bill, the cluster admin, receives a new request from Acme Corp.'s CTO asking for a new tenant to be onboarded and Alice user will be the tenant owner. Bill then assigns Alice's identity of `alice` in Acme Corp. identity management system. Since Alice is a tenant owner, Bill needs to assign `alice` the Capsule group defined by `--capsule-user-group` option, which defaults to `capsule.clastix.io`.
|
||||
Bill, the cluster admin, receives a new request from Acme Corp.'s CTO asking for a new tenant to be onboarded and Alice user will be the tenant owner. Bill then assigns Alice's identity of `alice` in the Acme Corp. identity management system. Since Alice is a tenant owner, Bill needs to assign `alice` the Capsule group defined by `--capsule-user-group` option, which defaults to `capsule.clastix.io`.
|
||||
|
||||
To keep the things simple, we assume that Bill just creates a client certificate for authentication using X.509 Certificate Signing Request, so Alice's certificate has `"/CN=alice/O=capsule.clastix.io"`.
|
||||
To keep things simple, we assume that Bill just creates a client certificate for authentication using X.509 Certificate Signing Request, so Alice's certificate has `"/CN=alice/O=capsule.clastix.io"`.
|
||||
|
||||
Bill creates a new tenant `oil` in the CaaS manangement portal according to the tenant's profile:
|
||||
Bill creates a new tenant `oil` in the CaaS management portal according to the tenant's profile:
|
||||
|
||||
```yaml
|
||||
kubectl create -f - << EOF
|
||||
@@ -32,7 +32,7 @@ oil Active 0 33m
|
||||
|
||||
Once the new tenant `oil` is in place, Bill sends the login credentials to Alice.
|
||||
|
||||
Alice can log in using her own credentials and check if she can create a namespace
|
||||
Alice can log in using her credentials and check if she can create a namespace
|
||||
|
||||
```
|
||||
kubectl auth can-i create namespaces
|
||||
@@ -84,7 +84,7 @@ spec:
|
||||
EOF
|
||||
```
|
||||
|
||||
However, it's more likely thet Bill assigns the ownership of the `oil` tenant to a group of users instead of a single one. Bill creates a new group account `oil-users` in the Acme Corp. identity management system and then he assigns Alice and Bob identities to the `oil-users` group.
|
||||
However, it's more likely that Bill assigns the ownership of the `oil` tenant to a group of users instead of a single one. Bill creates a new group account `oil-users` in the Acme Corp. identity management system and then he assigns Alice and Bob identities to the `oil-users` group.
|
||||
|
||||
The tenant manifest is modified as in the following:
|
||||
|
||||
@@ -101,7 +101,7 @@ spec:
|
||||
EOF
|
||||
```
|
||||
|
||||
With the configuration above, any user belonging to the `oil-users` group will be owner of the `oil` tenant with the same permissions of Alice. For example, Bob can login with his own credentials and issue
|
||||
With the configuration above, any user belonging to the `oil-users` group will be the owner of the `oil` tenant with the same permissions of Alice. For example, Bob can log in with his credentials and issue
|
||||
|
||||
```
|
||||
kubectl auth can-i create namespaces
|
||||
@@ -109,7 +109,7 @@ yes
|
||||
```
|
||||
|
||||
## Assign a robot account as tenant owner
|
||||
As GitOps metodology is gaining more and more adoption everywhere, it's more likely that an application (Service Account) should act as Tenant Owner. In Capsule, a Tenant can also be owned by a Kubernetes _ServiceAccount_ identity.
|
||||
As GitOps methodology is gaining more and more adoption everywhere, it's more likely that an application (Service Account) should act as Tenant Owner. In Capsule, a Tenant can also be owned by a Kubernetes _ServiceAccount_ identity.
|
||||
|
||||
The tenant manifest is modified as in the following:
|
||||
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# Use cases for Capsule
|
||||
Using Capsule, a cluster admin can implement complex multi-tenant scenarios for both public and private deployments. Here a list of common scenarios addressed by Capsule.
|
||||
Using Capsule, a cluster admin can implement complex multi-tenant scenarios for both public and private deployments. Here is a list of common scenarios addressed by Capsule.
|
||||
|
||||
# Container as a Service (CaaS)
|
||||
***Acme Corp***, our sample organization, built a Container as a Service platform (CaaS), based on Kubernetes to serve multiple lines of business. Each line of business, has its own team of engineers that are responsible for development, deployment, and operating their digital products.
|
||||
***Acme Corp***, our sample organization, built a Container as a Service platform (CaaS), based on Kubernetes to serve multiple lines of business. Each line of business has its team of engineers that are responsible for the development, deployment, and operating of their digital products.
|
||||
|
||||
To simplify the usage of Capsule in this scenario, we'll work with the following actors:
|
||||
|
||||
@@ -10,13 +10,13 @@ To simplify the usage of Capsule in this scenario, we'll work with the following
|
||||
he is the cluster administrator from the operations department of Acme Corp. and he is in charge of administration and maintains the CaaS platform.
|
||||
|
||||
* ***Alice***:
|
||||
she works as the IT Project Leader in the Oil & Gas Business Units. These are two new lines of business at Acme Corp. Alice is responsible for all the strategic IT projects in the two LOB's. She also is responsible for a team made of different job responsibilities (developers, administrators, SRE engineers, etc.) working in separate departments.
|
||||
she works as the IT Project Leader in the Oil & Gas Business Units. These are two new lines of business at Acme Corp. Alice is responsible for all the strategic IT projects in the two LOBs. She also is responsible for a team made of different job responsibilities (developers, administrators, SRE engineers, etc.) working in separate departments.
|
||||
|
||||
* ***Joe***:
|
||||
he works at Acme Corp, as a lead developer of a distributed team in Alice's organization. Joe is responsible for developing a mission-critical project in the Oil market.
|
||||
|
||||
* ***Bob***:
|
||||
he is the head of Engineering for the Water Business Unit, the main and historical line of business at Acme Corp. He is responsible for development, deployment, and operating multiple digital products in production for a large set of customers.
|
||||
he is the head of Engineering for the Water Business Unit, the main and historical line of business at Acme Corp. He is responsible for the development, deployment, and operation of multiple digital products in production for a large set of customers.
|
||||
|
||||
Use Capsule to address any of the following scenarios:
|
||||
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
# Assign permissions
|
||||
Alice acts as the tenant admin. Other users can operate inside the tenant with different levels of permissions and authorizations. Alice is responsible for creating additional roles and assigning these roles to other users to work in the same tenant.
|
||||
|
||||
One of the key design principles of the Capsule is the self-provisioning management from the tenant owner's perspective. Alice, the tenant owner, does not need to interact with Bill, the cluster admin, to complete her day-by-day duties. On the other side, Bill does not have to deal with multiple requests coming from multiple tenant owners that probably will overwhelm him.
|
||||
One of the key design principles of the Capsule is self-provisioning management from the tenant owner's perspective. Alice, the tenant owner, does not need to interact with Bill, the cluster admin, to complete her day-by-day duties. On the other side, Bill does not have to deal with multiple requests coming from multiple tenant owners that probably will overwhelm him.
|
||||
|
||||
Capsule leaves Alice, and the other tenant owners, the freedom to create RBAC roles at the namespace level, or using the pre-defined cluster roles already available in Kubernetes. Since roles and rolebindings are limited to a namespace scope, Alice can assign the roles to the other users accessing the same tenant only after the namespace is created. This gives Alice the power to administer the tenant without the intervention of the cluster admin.
|
||||
|
||||
From the cluster admin perspective, the only required action for Bill is to provision the other identities, eg. `joe` in the Identity Management system. This task can be done once, when onboarding the tenant and the number of users accessing the tenant can be part of the tenant business profile.
|
||||
From the cluster admin perspective, the only required action for Bill is to provide the other identities, eg. `joe` in the Identity Management system. This task can be done once when onboarding the tenant and the number of users accessing the tenant can be part of the tenant business profile.
|
||||
|
||||
Alice can create Roles and RoleBindings only in the namespaces she owns
|
||||
|
||||
@@ -51,4 +51,4 @@ no
|
||||
> Please, note the user `joe`, in the example above, is not acting as tenant owner. He can just operate in `oil-development` namespace as admin.
|
||||
|
||||
# What’s next
|
||||
See how Bill, the cluster admin, set resources quota and limits for Alice's tenant. [Enforce resources quota and limits](./resources-quota-limits.md).
|
||||
See how Bill, the cluster admin, sets resources quota and limits for Alice's tenant. [Enforce resources quota and limits](./resources-quota-limits.md).
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# Assign Pod Security Policies
|
||||
Bill, the cluster admin, can assign a dedicated Pod Security Policy (PSP) to the Alice's tenant. This is likely to be a requirement in a multi-tenancy environment.
|
||||
Bill, the cluster admin, can assign a dedicated Pod Security Policy (PSP) to Alice's tenant. This is likely to be a requirement in a multi-tenancy environment.
|
||||
|
||||
The cluster admin creates a PSP:
|
||||
|
||||
@@ -80,4 +80,4 @@ roleRef:
|
||||
With the above example, Capsule is forbidding any authenticated user in `oil-production` namespace to run privileged pods and to perform privilege escalation as declared by the Cluster Role `psp:privileged`.
|
||||
|
||||
# What’s next
|
||||
See how Bill, the cluster admin, can assign to Alice the permissions to create custom resources in her tenant. [Create Custom Resources](./custom-resources.md).
|
||||
See how Bill, the cluster admin, can assign to Alice the permissions to create custom resources in her tenant. [Create Custom Resources](./custom-resources.md).
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# Enforce resources quota and limits
|
||||
With help of Capsule, Bill, the cluster admin, can set and enforce resources quota and limits for the Alice's tenant.
|
||||
With help of Capsule, Bill, the cluster admin, can set and enforce resources quota and limits for Alice's tenant.
|
||||
|
||||
## Resources quota
|
||||
Set resources quota for each namespace in the Alice's tenant by defining them in the tenant spec:
|
||||
@@ -39,7 +39,7 @@ spec:
|
||||
EOF
|
||||
```
|
||||
|
||||
The resources quotas above will be inherited by all the namespaces created by Alice. In our case, when Alice creates the namespace `oil-production`, Capsule creates following resource quotas:
|
||||
The resource quotas above will be inherited by all the namespaces created by Alice. In our case, when Alice creates the namespace `oil-production`, Capsule creates the following resource quotas:
|
||||
|
||||
```yaml
|
||||
kind: ResourceQuota
|
||||
@@ -93,7 +93,7 @@ At tenant level, the behaviour is controlled by the `spec.resourceQuotas.scope`
|
||||
* Namespace
|
||||
|
||||
### Enforcement at tenant level
|
||||
By setting enforcement at tenant level, i.e. `spec.resourceQuotas.scope=Tenant`, Capsule aggregates resources usage for all namespaces in the tenant and adjusts all the `ResourceQuota` usage as aggregate. In such case, Alice can check the used resources at tenant level by inspecting the `annotations` in ResourceQuota object of any namespace in the tenant:
|
||||
By setting enforcement at tenant level, i.e. `spec.resourceQuotas.scope=Tenant`, Capsule aggregates resources usage for all namespaces in the tenant and adjusts all the `ResourceQuota` usage as aggregate. In such case, Alice can check the used resources at the tenant level by inspecting the `annotations` in ResourceQuota object of any namespace in the tenant:
|
||||
|
||||
```yaml
|
||||
kubectl -n oil-production get resourcequotas capsule-oil-1 -o yaml
|
||||
@@ -119,7 +119,7 @@ metadata:
|
||||
...
|
||||
```
|
||||
|
||||
When the aggregate usage for all namespaces crosses the hard quota, then the native `ResourceQuota` Admission Controller in Kubernetes denies the Alice's request to create resources exceeding the quota:
|
||||
When the aggregate usage for all namespaces crosses the hard quota, then the native `ResourceQuota` Admission Controller in Kubernetes denies Alice's request to create resources exceeding the quota:
|
||||
|
||||
```
|
||||
kubectl -n oil-development create deployment nginx --image nginx:latest --replicas 10
|
||||
@@ -151,11 +151,11 @@ nginx-55649fd747-tkv7m 1/1 Running 0 22m
|
||||
|
||||
### Enforcement at namespace level
|
||||
|
||||
By setting enforcement at namespace level, i.e. `spec.resourceQuotas.scope=Namespace`, Capsule does not aggregate the resources usage and all enforcement is done at namespace level.
|
||||
By setting enforcement at the namespace level, i.e. `spec.resourceQuotas.scope=Namespace`, Capsule does not aggregate the resources usage and all enforcement is done at the namespace level.
|
||||
|
||||
## Pods and containers limits
|
||||
|
||||
Bill, the cluster admin, can also set Limit Ranges for each namespace in the Alice's tenant by defining limits for pods and containers in the tenant spec:
|
||||
Bill, the cluster admin, can also set Limit Ranges for each namespace in Alice's tenant by defining limits for pods and containers in the tenant spec:
|
||||
|
||||
```yaml
|
||||
apiVersion: capsule.clastix.io/v1alpha1
|
||||
@@ -234,7 +234,7 @@ spec:
|
||||
|
||||
> Note: being the limit range specific of single resources, there is no aggregate to count.
|
||||
|
||||
Alice doesn't have permissions to change or delete the resources according to the assigned RBAC profile.
|
||||
Alice doesn't have permission to change or delete the resources according to the assigned RBAC profile.
|
||||
|
||||
```
|
||||
kubectl -n oil-production auth can-i patch resourcequota
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# Disable Service Types
|
||||
Bill, the cluster admin, can prevent creation of services with specific service types.
|
||||
Bill, the cluster admin, can prevent the creation of services with specific service types.
|
||||
|
||||
## NodePort
|
||||
When dealing with a _shared multi-tenant_ scenario, multiple _NodePort_ services can start becoming cumbersome to manage. Reason behind this could be related to the overlapping needs by the Tenant owners, since a _NodePort_ is going to be open on all nodes and, when using `hostNetwork=true`, accessible to any _Pod_ although any specific `NetworkPolicy`.
|
||||
When dealing with a _shared multi-tenant_ scenario, multiple _NodePort_ services can start becoming cumbersome to manage. The reason behind this could be related to the overlapping needs by the Tenant owners, since a _NodePort_ is going to be open on all nodes and, when using `hostNetwork=true`, accessible to any _Pod_ although any specific `NetworkPolicy`.
|
||||
|
||||
Bill, the cluster admin, can block the creation of services with `NodePort` service type for a given tenant
|
||||
|
||||
@@ -25,7 +25,7 @@ EOF
|
||||
With the above configuration, any attempt of Alice to create a Service of type `NodePort` is denied by the Validation Webhook enforcing it. Default value is `true`.
|
||||
|
||||
## ExternalName
|
||||
Service with type of `ExternalName` has been found subject to many security issue. To prevent tenant owners to create services with type of `ExternalName`, the cluster admin can prevent a tenant to create them:
|
||||
Service with the type of `ExternalName` has been found subject to many security issues. To prevent tenant owners to create services with the type of `ExternalName`, the cluster admin can prevent a tenant to create them:
|
||||
|
||||
```yaml
|
||||
kubectl apply -f - << EOF
|
||||
@@ -47,7 +47,7 @@ With the above configuration, any attempt of Alice to create a Service of type `
|
||||
|
||||
## LoadBalancer
|
||||
|
||||
Same as previously, Service of type of `LoadBalancer` could be blocked for various reasons. To prevent tenant owners to create these kinds of services, the cluster admin can prevent a tenant to create them:
|
||||
Same as previously, the Service of type of `LoadBalancer` could be blocked for various reasons. To prevent tenant owners to create these kinds of services, the cluster admin can prevent a tenant to create them:
|
||||
|
||||
```yaml
|
||||
kubectl apply -f - << EOF
|
||||
|
||||
@@ -38,7 +38,7 @@ spec:
|
||||
EOF
|
||||
```
|
||||
|
||||
Any attempt of Alice to use a non valid Storage Class, or missing it, is denied by the Validation Webhook enforcing it.
|
||||
Any attempt of Alice to use a non-valid Storage Class, or missing it, is denied by the Validation Webhook enforcing it.
|
||||
|
||||
# What’s next
|
||||
See how Bill, the cluster admin, can assign Network Policies to Alice's tenant. [Assign Network Policies](./network-policies.md).
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# Taint namespaces
|
||||
With Capsule, Bill can _"taint"_ the namespaces created by Alice with an additional labels and/or annotations. There is no specific semantic assigned to these labels and annotations: they just will be assigned to the namespaces in the tenant as they are created by Alice. This can help the cluster admin to implement specific use cases. As for example, it can be used to implement backup as a service for namespaces in the tenant.
|
||||
With Capsule, Bill can _"taint"_ the namespaces created by Alice with additional labels and/or annotations. There is no specific semantic assigned to these labels and annotations: they just will be assigned to the namespaces in the tenant as they are created by Alice. This can help the cluster admin to implement specific use cases. As it can be used to implement backup as a service for namespaces in the tenant.
|
||||
|
||||
Bill assigns additional labels and annotations to all namespaces created in the `oil` tenant:
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# Taint services
|
||||
With Capsule, Bill can _"taint"_ the services created by Alice with an additional labels and/or annotations. There is no specific semantic assigned to these labels and annotations: they just will be assigned to the services in the tenant as they are created by Alice. This can help the cluster admin to implement specific use cases.
|
||||
With Capsule, Bill can _"taint"_ the services created by Alice with additional labels and/or annotations. There is no specific semantic assigned to these labels and annotations: they just will be assigned to the services in the tenant as they are created by Alice. This can help the cluster admin to implement specific use cases.
|
||||
|
||||
Bill assigns additional labels and annotations to all services created in the `oil` tenant:
|
||||
|
||||
@@ -25,6 +25,6 @@ EOF
|
||||
When Alice creates a service in a namespace, this will inherit the given label and/or annotation.
|
||||
|
||||
# What’s next
|
||||
This end our tour in Capsule use cases. As we improve Capsule, more use cases about multi-tenancy, policy admission control, and cluster governance will be covered in the future.
|
||||
This ends our tour in Capsule use cases. As we improve Capsule, more use cases about multi-tenancy, policy admission control, and cluster governance will be covered in the future.
|
||||
|
||||
Stay tuned!
|
||||
@@ -1,8 +1,8 @@
|
||||
# Velero Backup Restoration
|
||||
|
||||
Velero is a backup system that perform disaster recovery, and migrate Kubernetes cluster resources and persistent volumes.
|
||||
Velero is a backup system that performs disaster recovery and migrates Kubernetes cluster resources and persistent volumes.
|
||||
|
||||
Using this in a Kubernetes cluster where Capsule is installed can lead to an incomplete restore of the cluster's Tenants. This is due to the fact that Velero omits the `ownerReferences` section from the tenant's namespace manifests when backup them.
|
||||
Using this in a Kubernetes cluster where Capsule is installed can lead to an incomplete restore of the cluster's Tenants. This is because Velero omits the `ownerReferences` section from the tenant's namespace manifests when backup them.
|
||||
|
||||
To avoid this problem you can use the script `velero-restore.sh` under the `hack/` folder.
|
||||
|
||||
@@ -14,7 +14,7 @@ In case of a data loss, the right thing to do is to restore the cluster with **V
|
||||
|
||||
Running this command, we are going to patch the tenant's namespaces manifests that are actually `ownerReferences`-less. Once the command has finished its run, you got the cluster back.
|
||||
|
||||
Additionally you can also specify a selected range of tenants to be restored:
|
||||
Additionally, you can also specify a selected range of tenants to be restored:
|
||||
|
||||
```bash
|
||||
./velero-restore.sh --tenant "gas oil" restore
|
||||
|
||||
@@ -8,11 +8,11 @@ You should setup your development environment as following:
|
||||
|
||||
> Please, refer to the general coding style rules for Capsule.
|
||||
|
||||
### Run locally for test and debug
|
||||
## Run locally for test and debug
|
||||
|
||||
This guide helps new contributors to locally debug in _out or cluster_ mode the project.
|
||||
|
||||
1. You need to run a Kind cluster and find the endpoint port of `kind-control-plane` using `docker ps`:
|
||||
1. You need to run a kind cluster and find the endpoint port of `kind-control-plane` using `docker ps`:
|
||||
|
||||
```bash
|
||||
❯ docker ps
|
||||
@@ -32,7 +32,11 @@ localhost-key.pem localhost.pem
|
||||
3. Run the proxy with the following options
|
||||
|
||||
```bash
|
||||
go run main.go --ssl-cert-path=/tmp/localhost.pem --ssl-key-path=/tmp/localhost-key.pem --enable-ssl=true --kubeconfig=<YOUR KUBERNETES CONFIGURATION FILE>
|
||||
go run main.go \
|
||||
--ssl-cert-path=/tmp/localhost.pem \
|
||||
--ssl-key-path=/tmp/localhost-key.pem \
|
||||
--enable-ssl=true \
|
||||
--kubeconfig=<YOUR KUBERNETES CONFIGURATION FILE>
|
||||
```
|
||||
|
||||
5. Edit the `KUBECONFIG` file (you should make a copy and work on it) as follows:
|
||||
@@ -42,7 +46,7 @@ go run main.go --ssl-cert-path=/tmp/localhost.pem --ssl-key-path=/tmp/localhost-
|
||||
|
||||
6. Now you should be able to run kubectl using the proxy!
|
||||
|
||||
### Debug in a remote Kubernetes cluster
|
||||
## Debug in a remote Kubernetes cluster
|
||||
|
||||
In some cases, you would need to debug the in-cluster mode and [`delve`](https://github.com/go-delve/delve) plays a big role here.
|
||||
|
||||
|
||||
@@ -0,0 +1,154 @@
|
||||
# OIDC Authentication
|
||||
The `capsule-proxy` works with `kubectl` users with a token-based authentication, e.g. OIDC or Bearer Token. In the following example, we'll use Keycloak as OIDC server capable to provides JWT tokens.
|
||||
|
||||
### Configuring Keycloak
|
||||
Configure Keycloak as OIDC server:
|
||||
|
||||
- Add a realm called `caas`, or use any existing realm instead
|
||||
- Add a group `capsule.clastix.io`
|
||||
- Add a user `alice` assigned to group `capsule.clastix.io`
|
||||
- Add an OIDC client called `kubernetes`
|
||||
- For the `kubernetes` client, create protocol mappers called `groups` and `audience`
|
||||
|
||||
If everything is done correctly, now you should be able to authenticate in Keycloak and see user groups in JWT tokens. Use the following snippet to authenticate in Keycloak as `alice` user:
|
||||
|
||||
```
|
||||
$ KEYCLOAK=sso.clastix.io
|
||||
$ REALM=caas
|
||||
$ OIDC_ISSUER=${KEYCLOAK}/auth/realms/${REALM}
|
||||
|
||||
$ curl -k -s https://${OIDC_ISSUER}/protocol/openid-connect/token \
|
||||
-d grant_type=password \
|
||||
-d response_type=id_token \
|
||||
-d scope=openid \
|
||||
-d client_id=${OIDC_CLIENT_ID} \
|
||||
-d client_secret=${OIDC_CLIENT_SECRET} \
|
||||
-d username=${USERNAME} \
|
||||
-d password=${PASSWORD} | jq
|
||||
```
|
||||
|
||||
The result will include an `ACCESS_TOKEN`, a `REFRESH_TOKEN`, and an `ID_TOKEN`. The access-token can generally be disregarded for Kubernetes. It would be used if the identity provider was managing roles and permissions for the users but that is done in Kubernetes itself with RBAC. The id-token is short lived while the refresh-token has longer expiration. The refresh-token is used to fetch a new id-token when the id-token expires.
|
||||
|
||||
```json
|
||||
{
|
||||
"access_token":"ACCESS_TOKEN",
|
||||
"refresh_token":"REFRESH_TOKEN",
|
||||
"id_token": "ID_TOKEN",
|
||||
"token_type":"bearer",
|
||||
"scope": "openid groups profile email"
|
||||
}
|
||||
```
|
||||
|
||||
To introspect the `ID_TOKEN` token run:
|
||||
```
|
||||
$ curl -k -s https://${OIDC_ISSUER}/protocol/openid-connect/introspect \
|
||||
-d token=${ID_TOKEN} \
|
||||
--user ${OIDC_CLIENT_ID}:${OIDC_CLIENT_SECRET} | jq
|
||||
```
|
||||
|
||||
The result will be like the following:
|
||||
|
||||
```json
|
||||
{
|
||||
"exp": 1601323086,
|
||||
"iat": 1601322186,
|
||||
"aud": "kubernetes",
|
||||
"typ": "ID",
|
||||
"azp": "kubernetes",
|
||||
"preferred_username": "alice",
|
||||
"email_verified": false,
|
||||
"acr": "1",
|
||||
"groups": [
|
||||
"capsule.clastix.io"
|
||||
],
|
||||
"client_id": "kubernetes",
|
||||
"username": "alice",
|
||||
"active": true
|
||||
}
|
||||
```
|
||||
|
||||
### Configuring Kubernetes API Server
|
||||
Configuring Kubernetes for OIDC Authentication requires adding several parameters to the API Server. Please, refer to the [documentation](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#openid-connect-tokens) for details and examples. Most likely, your `kube-apiserver.yaml` manifest will looks like the following:
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
containers:
|
||||
- command:
|
||||
- kube-apiserver
|
||||
...
|
||||
- --oidc-issuer-url=https://${OIDC_ISSUER}
|
||||
- --oidc-ca-file=/etc/kubernetes/oidc/ca.crt
|
||||
- --oidc-client-id=${OIDC_CLIENT_SECRET}
|
||||
- --oidc-username-claim=preferred_username
|
||||
- --oidc-groups-claim=groups
|
||||
- --oidc-username-prefix=-
|
||||
```
|
||||
|
||||
### Configuring kubectl
|
||||
There are two options to use `kubectl` with OIDC:
|
||||
|
||||
- OIDC Authenticator
|
||||
- Use the `--token` option
|
||||
|
||||
To use the OIDC Authenticator, add an `oidc` user entry to your `kubeconfig` file:
|
||||
```
|
||||
$ kubectl config set-credentials oidc \
|
||||
--auth-provider=oidc \
|
||||
--auth-provider-arg=idp-issuer-url=https://${OIDC_ISSUER} \
|
||||
--auth-provider-arg=idp-certificate-authority=/path/to/ca.crt \
|
||||
--auth-provider-arg=client-id=${OIDC_CLIENT_ID} \
|
||||
--auth-provider-arg=client-secret=${OIDC_CLIENT_SECRET} \
|
||||
--auth-provider-arg=refresh-token=${REFRESH_TOKEN} \
|
||||
--auth-provider-arg=id-token=${ID_TOKEN} \
|
||||
--auth-provider-arg=extra-scopes=groups
|
||||
```
|
||||
|
||||
To use the --token option:
|
||||
```
|
||||
$ kubectl config set-credentials oidc --token=${ID_TOKEN}
|
||||
```
|
||||
|
||||
Point the kubectl to the URL where the `capsule-proxy` service is reachable:
|
||||
```
|
||||
$ kubectl config set-cluster mycluster \
|
||||
--server=https://kube.clastix.io \
|
||||
--certificate-authority=~/.kube/ca.crt
|
||||
```
|
||||
|
||||
Create a new context for the OIDC authenticated users:
|
||||
```
|
||||
$ kubectl config set-context alice-oidc@mycluster \
|
||||
--cluster=mycluster \
|
||||
--user=oidc
|
||||
```
|
||||
|
||||
As user `alice`, you should be able to use `kubectl` to create some namespaces:
|
||||
```
|
||||
$ kubectl --context alice-oidc@mycluster create namespace oil-production
|
||||
$ kubectl --context alice-oidc@mycluster create namespace oil-development
|
||||
$ kubectl --context alice-oidc@mycluster create namespace gas-marketing
|
||||
```
|
||||
|
||||
and list only those namespaces:
|
||||
```
|
||||
$ kubectl --context alice-oidc@mycluster get namespaces
|
||||
NAME STATUS AGE
|
||||
gas-marketing Active 2m
|
||||
oil-development Active 2m
|
||||
oil-production Active 2m
|
||||
```
|
||||
|
||||
When logged as cluster-admin power user you should be able to see all namespaces:
|
||||
```
|
||||
$ kubectl get namespaces
|
||||
NAME STATUS AGE
|
||||
default Active 78d
|
||||
kube-node-lease Active 78d
|
||||
kube-public Active 78d
|
||||
kube-system Active 78d
|
||||
gas-marketing Active 2m
|
||||
oil-development Active 2m
|
||||
oil-production Active 2m
|
||||
```
|
||||
|
||||
_Nota Bene_: once your `ID_TOKEN` expires, the `kubectl` OIDC Authenticator will attempt to refresh automatically your `ID_TOKEN` using the `REFRESH_TOKEN`, the `OIDC_CLIENT_ID` and the `OIDC_CLIENT_SECRET` storing the new values for the `REFRESH_TOKEN` and `ID_TOKEN` in your `kubeconfig` file. In case the OIDC uses a self signed CA certificate, make sure to specify it with the `idp-certificate-authority` option in your `kubeconfig` file, otherwise you'll not able to refresh the tokens. Once the `REFRESH_TOKEN` is expired, you will need to refresh tokens manually.
|
||||
@@ -107,8 +107,7 @@ oil-production Active 2m
|
||||
|
||||
### Nodes
|
||||
|
||||
When a Tenant defines a `.spec.nodeSelector`, the nodes matching that labels can be easily retrieved.
|
||||
The annotation `capsule.clastix.io/enable-node-listing` allows the ability for the owners to retrieve the node list (useful in shared HW scenarios).
|
||||
The Capsule Proxy gives the owners the ability to access the nodes matching the `.spec.nodeSelector` in the Tenant manifest:
|
||||
|
||||
```yaml
|
||||
apiVersion: capsule.clastix.io/v1beta1
|
||||
@@ -133,6 +132,8 @@ NAME STATUS ROLES AGE VERSION
|
||||
capsule-gold-qwerty Ready <none> 43h v1.19.1
|
||||
```
|
||||
|
||||
> Warning: when no `nodeSelector` is specified, the tenant owners has access to all the nodes, according to the permissions listed in the `proxySettings` specs.
|
||||
|
||||
### Storage Classes
|
||||
|
||||
A Tenant may be limited to use a set of allowed Storage Class resources, as follows.
|
||||
@@ -327,9 +328,10 @@ oil-development Active 2m
|
||||
oil-production Active 2m
|
||||
```
|
||||
|
||||
|
||||
# What’s next
|
||||
Have a fun with `capsule-proxy`:
|
||||
|
||||
* [Standalone Installation](./standalone.md)
|
||||
* [Sidecar Installation](./sidecar.md)
|
||||
* [Sidecar Installation](./sidecar.md)
|
||||
* [OIDC Authentication](./oidc-auth.md)
|
||||
* [Contributing](./contributing.md)
|
||||
|
||||
@@ -55,164 +55,8 @@ $ helm install capsule-proxy clastix/capsule-proxy \
|
||||
|
||||
The `capsule-proxy` should be exposed with an Ingress in SSL passthrough mode and reachable at `https://kube.clastix.io`.
|
||||
|
||||
## TLS Client Authentication
|
||||
Users using a TLS client based authentication with certificate and key are able to talks with `capsule-proxy` since the current implementation of the reverse proxy is able to forward client certificates to the Kubernetes APIs server.
|
||||
|
||||
## OIDC Authentication
|
||||
The `capsule-proxy` works with `kubectl` users with a token-based authentication, e.g. OIDC or Bearer Token. In the following example, we'll use Keycloak as OIDC server capable to provides JWT tokens.
|
||||
|
||||
### Configuring Keycloak
|
||||
Configure Keycloak as OIDC server:
|
||||
|
||||
- Add a realm called `caas`, or use any existing realm instead
|
||||
- Add a group `capsule.clastix.io`
|
||||
- Add a user `alice` assigned to group `capsule.clastix.io`
|
||||
- Add an OIDC client called `kubernetes`
|
||||
- For the `kubernetes` client, create protocol mappers called `groups` and `audience`
|
||||
|
||||
If everything is done correctly, now you should be able to authenticate in Keycloak and see user groups in JWT tokens. Use the following snippet to authenticate in Keycloak as `alice` user:
|
||||
|
||||
```
|
||||
$ KEYCLOAK=sso.clastix.io
|
||||
$ REALM=caas
|
||||
$ OIDC_ISSUER=${KEYCLOAK}/auth/realms/${REALM}
|
||||
|
||||
$ curl -k -s https://${OIDC_ISSUER}/protocol/openid-connect/token \
|
||||
-d grant_type=password \
|
||||
-d response_type=id_token \
|
||||
-d scope=openid \
|
||||
-d client_id=${OIDC_CLIENT_ID} \
|
||||
-d client_secret=${OIDC_CLIENT_SECRET} \
|
||||
-d username=${USERNAME} \
|
||||
-d password=${PASSWORD} | jq
|
||||
```
|
||||
|
||||
The result will include an `ACCESS_TOKEN`, a `REFRESH_TOKEN`, and an `ID_TOKEN`. The access-token can generally be disregarded for Kubernetes. It would be used if the identity provider was managing roles and permissions for the users but that is done in Kubernetes itself with RBAC. The id-token is short lived while the refresh-token has longer expiration. The refresh-token is used to fetch a new id-token when the id-token expires.
|
||||
|
||||
```json
|
||||
{
|
||||
"access_token":"ACCESS_TOKEN",
|
||||
"refresh_token":"REFRESH_TOKEN",
|
||||
"id_token": "ID_TOKEN",
|
||||
"token_type":"bearer",
|
||||
"scope": "openid groups profile email"
|
||||
}
|
||||
```
|
||||
|
||||
To introspect the `ID_TOKEN` token run:
|
||||
```
|
||||
$ curl -k -s https://${OIDC_ISSUER}/protocol/openid-connect/introspect \
|
||||
-d token=${ID_TOKEN} \
|
||||
--user ${OIDC_CLIENT_ID}:${OIDC_CLIENT_SECRET} | jq
|
||||
```
|
||||
|
||||
The result will be like the following:
|
||||
|
||||
```json
|
||||
{
|
||||
"exp": 1601323086,
|
||||
"iat": 1601322186,
|
||||
"aud": "kubernetes",
|
||||
"typ": "ID",
|
||||
"azp": "kubernetes",
|
||||
"preferred_username": "alice",
|
||||
"email_verified": false,
|
||||
"acr": "1",
|
||||
"groups": [
|
||||
"capsule.clastix.io"
|
||||
],
|
||||
"client_id": "kubernetes",
|
||||
"username": "alice",
|
||||
"active": true
|
||||
}
|
||||
```
|
||||
|
||||
### Configuring Kubernetes API Server
|
||||
Configuring Kubernetes for OIDC Authentication requires adding several parameters to the API Server. Please, refer to the [documentation](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#openid-connect-tokens) for details and examples. Most likely, your `kube-apiserver.yaml` manifest will looks like the following:
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
containers:
|
||||
- command:
|
||||
- kube-apiserver
|
||||
...
|
||||
- --oidc-issuer-url=https://${OIDC_ISSUER}
|
||||
- --oidc-ca-file=/etc/kubernetes/oidc/ca.crt
|
||||
- --oidc-client-id=${OIDC_CLIENT_SECRET}
|
||||
- --oidc-username-claim=preferred_username
|
||||
- --oidc-groups-claim=groups
|
||||
- --oidc-username-prefix=-
|
||||
```
|
||||
|
||||
### Configuring kubectl
|
||||
There are two options to use `kubectl` with OIDC:
|
||||
|
||||
- OIDC Authenticator
|
||||
- Use the `--token` option
|
||||
|
||||
To use the OIDC Authenticator, add an `oidc` user entry to your `kubeconfig` file:
|
||||
```
|
||||
$ kubectl config set-credentials oidc \
|
||||
--auth-provider=oidc \
|
||||
--auth-provider-arg=idp-issuer-url=https://${OIDC_ISSUER} \
|
||||
--auth-provider-arg=idp-certificate-authority=/path/to/ca.crt \
|
||||
--auth-provider-arg=client-id=${OIDC_CLIENT_ID} \
|
||||
--auth-provider-arg=client-secret=${OIDC_CLIENT_SECRET} \
|
||||
--auth-provider-arg=refresh-token=${REFRESH_TOKEN} \
|
||||
--auth-provider-arg=id-token=${ID_TOKEN} \
|
||||
--auth-provider-arg=extra-scopes=groups
|
||||
```
|
||||
|
||||
To use the --token option:
|
||||
```
|
||||
$ kubectl config set-credentials oidc --token=${ID_TOKEN}
|
||||
```
|
||||
|
||||
Point the kubectl to the URL where the `capsule-proxy` service is reachable:
|
||||
```
|
||||
$ kubectl config set-cluster mycluster \
|
||||
--server=https://kube.clastix.io \
|
||||
--certificate-authority=~/.kube/ca.crt
|
||||
```
|
||||
|
||||
Create a new context for the OIDC authenticated users:
|
||||
```
|
||||
$ kubectl config set-context alice-oidc@mycluster \
|
||||
--cluster=mycluster \
|
||||
--user=oidc
|
||||
```
|
||||
|
||||
As user `alice`, you should be able to use `kubectl` to create some namespaces:
|
||||
```
|
||||
$ kubectl --context alice-oidc@mycluster create namespace oil-production
|
||||
$ kubectl --context alice-oidc@mycluster create namespace oil-development
|
||||
$ kubectl --context alice-oidc@mycluster create namespace gas-marketing
|
||||
```
|
||||
|
||||
and list only those namespaces:
|
||||
```
|
||||
$ kubectl --context alice-oidc@mycluster get namespaces
|
||||
NAME STATUS AGE
|
||||
gas-marketing Active 2m
|
||||
oil-development Active 2m
|
||||
oil-production Active 2m
|
||||
```
|
||||
|
||||
When logged as cluster-admin power user you should be able to see all namespaces:
|
||||
```
|
||||
$ kubectl get namespaces
|
||||
NAME STATUS AGE
|
||||
default Active 78d
|
||||
kube-node-lease Active 78d
|
||||
kube-public Active 78d
|
||||
kube-system Active 78d
|
||||
gas-marketing Active 2m
|
||||
oil-development Active 2m
|
||||
oil-production Active 2m
|
||||
```
|
||||
|
||||
_Nota Bene_: once your `ID_TOKEN` expires, the `kubectl` OIDC Authenticator will attempt to refresh automatically your `ID_TOKEN` using the `REFRESH_TOKEN`, the `OIDC_CLIENT_ID` and the `OIDC_CLIENT_SECRET` storing the new values for the `REFRESH_TOKEN` and `ID_TOKEN` in your `kubeconfig` file. In case the OIDC uses a self signed CA certificate, make sure to specify it with the `idp-certificate-authority` option in your `kubeconfig` file, otherwise you'll not able to refresh the tokens. Once the `REFRESH_TOKEN` is expired, you will need to refresh tokens manually.
|
||||
|
||||
## RBAC Considerations
|
||||
Currently, the service account used for `capsule-proxy` needs to have `cluster-admin` permissions.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user