This makes it possible to include extra variants of the service-name that aren't captured by the {{ include "capsule.fullname" }} macro
Co-authored-by: Travis Holton <heytrav@proton.me>
Co-authored-by: Dario Tranchitella <dario@tranchitella.eu>
This commit adds a quickstart section to setup and how to use
the integration both as a platform administrator and a tenant owner.
Signed-off-by: Massimiliano Giovagnoli <me@maxgio.it>
Allowed template values:
- `{{ tenant.name }}` for the Tenant name managing the Namespace
- `{{ namespace }}` for the Namespace where the resource is replicated
This is going to solve the issue when upgrading Capsule <v0.1.0 to
>=v0.1.0: due to a resource reflector many warning were polluting the
reconciliation loop and causing unmarshaling errors.
Additionally, just the CA secret was checked before starting the
Operator, when also the TLS is requested for the webhooks, along with
the `/convert` one that is used for the CR version conversion.
[Review the Major Changes section first before upgrading to a new version](https://artifacthub.io/packages/helm/projectcapsule/capsule/{{ .Version }}#major-changes)
Changes are published with their type and scope for each release in the release description. Changes are assigned based on their commit description. Read more on how commits should be formatted in the [Contributing](CONTRIBUTING.md#commits) guide.
See the [Releases](https://github.com/projectcapsule/capsule/releases)
## Helm Chart
For the helm chart, a dedicated changelog is created based on the chart's annotations ([See](./DEVELOPMENT.md#helm-changelog)).
Capsule follows the [CNCF Code of Conduct](https://github.com/cncf/foundation/blob/master/code-of-conduct.md).
## Our Pledge
We as members, contributors, and leaders pledge to make participation in our
community a harassment-free experience for everyone, regardless of age, body
size, visible or invisible disability, ethnicity, sex characteristics, gender
identity and expression, level of experience, education, socio-economic status,
nationality, personal appearance, race, religion, or sexual identity
and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming,
diverse, inclusive, and healthy community.
## Our Standards
Examples of behavior that contributes to a positive environment for our
community include:
* Demonstrating empathy and kindness toward other people
* Being respectful of differing opinions, viewpoints, and experiences
* Giving and gracefully accepting constructive feedback
* Accepting responsibility and apologizing to those affected by our mistakes,
and learning from the experience
* Focusing on what is best not just for us as individuals, but for the
overall community
Examples of unacceptable behavior include:
* The use of sexualized language or imagery, and sexual attention or
advances of any kind
* Trolling, insulting or derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or email
address, without their explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
## Enforcement Responsibilities
Community leaders are responsible for clarifying and enforcing our standards of
acceptable behavior and will take appropriate and fair corrective action in
response to any behavior that they deem inappropriate, threatening, offensive,
or harmful.
Community leaders have the right and responsibility to remove, edit, or reject
comments, commits, code, wiki edits, issues, and other contributions that are
not aligned to this Code of Conduct, and will communicate reasons for moderation
decisions when appropriate.
## Scope
This Code of Conduct applies within all community spaces, and also applies when
an individual is officially representing the community in public spaces.
Examples of representing our community include using an official e-mail address,
posting via an official social media account, or acting as an appointed
representative at an online or offline event.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement by contacting
one of the [maintainers](https://raw.githubusercontent.com/clastix/capsule/master/.github/maintainers.yaml).
All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the
reporter of any incident.
## Enforcement Guidelines
Community leaders will follow these Community Impact Guidelines in determining
the consequences for any action they deem in violation of this Code of Conduct:
### 1. Correction
**Community Impact**: Use of inappropriate language or other behavior deemed
unprofessional or unwelcome in the community.
**Consequence**: A private, written warning from community leaders, providing
clarity around the nature of the violation and an explanation of why the
behavior was inappropriate. A public apology may be requested.
### 2. Warning
**Community Impact**: A violation through a single incident or series
of actions.
**Consequence**: A warning with consequences for continued behavior. No
interaction with the people involved, including unsolicited interaction with
those enforcing the Code of Conduct, for a specified period of time. This
includes avoiding interactions in community spaces as well as external channels
like social media. Violating these terms may lead to a temporary or
permanent ban.
### 3. Temporary Ban
**Community Impact**: A serious violation of community standards, including
sustained inappropriate behavior.
**Consequence**: A temporary ban from any sort of interaction or public
communication with the community for a specified period of time. No public or
private interaction with the people involved, including unsolicited interaction
with those enforcing the Code of Conduct, is allowed during this period.
Violating these terms may lead to a permanent ban.
### 4. Permanent Ban
**Community Impact**: Demonstrating a pattern of violation of community
standards, including sustained inappropriate behavior, harassment of an
individual, or aggression toward or disparagement of classes of individuals.
**Consequence**: A permanent ban from any sort of public interaction within
the community.
## Attribution
This Code of Conduct follows the [CNCF Code of Conduct](https://github.com/cncf/foundation/blob/master/code-of-conduct.md) and is adapted from the [Contributor Covenant][homepage],
All contributions are welcome! If you find a bug or have a feature request, please open an issue or submit a pull request.
## Ways to contribute
### 1. Report Issues
Issues to Capsule help improve the project in multiple ways including the following:
* Report potential bugs
* Request a feature
* Request a sample policy
### 2. Engagement
Engage with the community on [Slack](https://kubernetes.slack.com/archives/C03GETTJQRL) and help new users with questions or issues they may have.
### 3. Submit changes
Submit technical changes via pull requests. New contributors may easily view all open issues labeled as [good first issues](https://github.com/projectcapsule/capsule/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) allowing you to get started in an approachable manner.
Once you wish to get started contributing to the code base, please refer to our [development guide](DEVELOPMENT.md) for a how-to. **[We accept pull requests from forks only](#create-a-pull-request)**.
Before creating a pull request, please ensure that your changes are tested and that the documentation is updated accordingly.
When creating a pull request, please visit:
* [commits](#commits)
## Guidelines
The following guidelines outline the semantics and processes which apply to technical contributions to the project.
## Supported Versions
Versions follow [Semantic Versioning](https://semver.org/) terminology and are expressed as `x.y.z`:
- where x is the major version
- y is the minor version
- and z is the patch version
Security fixes, may be backported to the three most recent minor releases, depending on severity and feasibility.
Prereleases are marked as `-rc.x` (release candidate) and may refere to any type of version bump.
## Pull Requests
The pull request title is checked according to the described [semantics](#semantics) (pull requests don't require a scope). However pull requests are currently not used to generate the changelog. Check if your pull requests body meets the following criteria:
- reference a previously opened issue: https://docs.github.com/en/github/writing-on-github/autolinked-references-and-urls#issues-and-pull-requests
- splitting changes into several and documented small commits
- limit the git subject to 50 characters and write as the continuation of the
sentence "If applied, this commit will ..."
- explain what and why in the body, if more than a trivial change, wrapping at
72 characters
If your pull request in a draft state and not ready yet for review, you can prefix the title with `[WIP]`. This will indicate that work is still ongoing:
[WIP] feat(controller): new cool feature
### Create a Pull Request
Head over to the project repository on GitHub and click the **"Fork"** button. With the forked copy, you can try new ideas and implement changes to the project.
1.**Clone the repository to your device:**
Get the link of your forked repository, paste it in your device terminal and clone it using the command.
Create a new branch and navigate to it using this command.
```sh
git checkout -b <new-branch>
```
3.**Stage, Commit, and Push changes:**
Now that we have implemented the required changes, use the command below to stage the changes and commit them.
```sh
git add .
```
```sh
git commit -s -m "Commit message"
```
Go ahead and push your changes to GitHub using this command.
```sh
git push
```
## Commits
The commit message is checked according to the described [semantics](#semantics). Commits are used to generate the changelog and their author will be referenced in the changelog.
### Reorganising commits
To reorganise your commits, do the following (or use your way of doing it):
For contributors to certify that they wrote or otherwise have the right to submit the code they are contributing to the project, we are requiring everyone to acknowledge this by signing their work which indicates you agree to the DCO found here.
To sign your work, just add a line like this at the end of your commit message:
Signed-off-by: Random J Developer <random@developer.example.org>
This can easily be done with the -s command line option to append this automatically to your commit message.
git commit -s -m 'This is my commit message'
## Semantics
The semantics should indicate the change and it's impact. The general format for commit messages and pull requests is the following:
feat(ui): Add `Button` component
^ ^ ^
| | |__ Subject
| |_______ Scope
|____________ Type
The commits are checked on pull-request. If the commit message does not follow the format, the workflow will fail. See the [Types](#types) for the supported types. The scope is not required but helps to provide more context for your changes. Try to use a scope if possible.
### Types
The following types are allowed for commits and pull requests:
*`chore`: housekeeping changes, no production code change
Our Makefile helps you with the development of new changes or fixes. [You may have a look at it](./Makefile), since not all targets are documented.
To execute your changes locally, you can run the binary locally. This will run just the capsule controller. We recommend [to setup a development environment](#development-environment) for a better development experience:
```bash
make run
```
## Building
You can build the docker image locally, Ko will be installed via go, so you don't need to install it manually.
```bash
make ko-build-all
```
This will push the build to your local docker images.
## Test
Execute unit testing:
```bash
make test
```
## E2E Test
**New changes always require dedcated E2E tests. E2E help us to ensure the quality of the code and it's functionality.**
For E2E test we use the [ginkgo](https://github.com/onsi/ginkgo) framework. Ou can see all the test under [e2e](./e2e/).
With the following command a new KinD cluster is created with the Kubernetes version `v1.20.7` (This can be done with any available Kubernetes version). A docker image is created and pushed and loaded into the KinD cluster. Then the E2E tests are executed against the KinD cluster.
```bash
make e2e/v1.20.7
```
You can also just run the e2e tests without the creation of a new kind cluster:
```
make e2e-exec
```
The E2E tests are also executed via the [github workflow](./.github/workflows/e2e.yaml) on every PR and push to the main branch.
# Development Environment
During development, we prefer that the code is running within our IDE locally, instead of running as the normal Pod(s) within the Kubernetes cluster.
To achieve that, there are some necessary steps we need to walk through, which have been made as a make target within our Makefile.
So the TL;DR answer is:
**Make sure a *KinD* cluster is running on your laptop, and then run `make dev-setup` to setup the dev environment.**. This is not done in the `make dev-setup` setup.
```bash
# If you haven't installed or run `make deploy` before, do it first
# Note: please retry if you saw errors
$ make deploy
# To retrieve your laptop's IP and execute `make dev-setup` to setup dev env
# For example: LAPTOP_HOST_IP=192.168.10.101 make dev-setup
$ LAPTOP_HOST_IP="<YOUR_LAPTOP_IP>" make dev-setup
```
### Explenation
We recommend to setup the development environment with the make `dev-setup` target. However here is a step by step guide to setup the development environment for understanding.
1. Scaling down the deployed Pod(s) to 0
We need to scale the existing replicas of capsule-controller-manager to 0 to avoid reconciliation competition between the Pod(s) and the code running outside of the cluster, in our preferred IDE for example.
By default, the webhooks will be registered with the services, which will route to the Pods, inside the cluster. We need to delegate the controllers' and webhook's services to the code running in our IDE by patching the `MutatingWebhookConfiguration` and `ValidatingWebhookConfiguration`.
```bash
# Export your laptop's IP with the 9443 port exposed by controllers/webhooks' services
To verify that, we can open a new console and create a new Tenant in a new shell:
```bash
$ kubectl apply -f - <<EOF
apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
name: gas
spec:
owners:
- name: alice
kind: User
EOF
```
We should see output and logs in the make run console.
Now it's time to work through our familiar inner loop for development in our preferred IDE. For example, if you're using [Visual Studio Code](https://code.visualstudio.com/), this launch.json file can be a good start.
## Helm Chart
You can test your changes made to the helm chart locally. They are almost identical to the checks executed in the github workflows.
Run chart linting (ct lint):
```bash
make helm-lint
```
Run chart tests (ct install). This creates a KinD cluster, builds the current image and loads it into the cluster and installs the helm chart:
```bash
make helm-test
```
### Documentation
Documentation of the chart is done with [helm-docs](https://github.com/norwoodj/helm-docs). Therefor all documentation relevant changes for the chart must be done in the [README.md.gotmpl](./charts/capsule/README.md.gotmpl) file. You can run this locally with this command (requires running docker daemon):
time="2023-10-23T13:45:08Z"level=info msg="Generating README Documentation for chart /helm-docs/charts/capsule"
```
This will update the documentation for the chart in the `README.md` file.
### Helm Changelog
The `version` of the chart does not require a bump, since it's driven by our release process. The `appVersion` of the chart is the version of the Capsule project. This is the version that should be bumped when a new Capsule version is released. This will be done by the maintainers.
To create the proper changelog for the helm chart, all changes which affect the helm chart must be documented as chart annotation. See all the available [chart annotations](https://artifacthub.io/docs/topics/annotations/helm/).
This annotation can be provided using two different formats: using a plain list of strings with the description of the change or using a list of objects with some extra structured information (see example below). Please feel free to use the one that better suits your needs. The UI experience will be slightly different depending on the choice. When using the list of objects option the valid supported kinds are `added`, `changed`, `deprecated`, `removed`, `fixed` and `security`.
The **Capsule** project is dedicated to creating a multi-tenancy and policy-based framework for Kubernetes. This governance explains how the project is run.
The Capsule and its leadership embrace the following values:
* Openness: Communication and decision-making happens in the open and is discoverable for future
reference. As much as possible, all discussions and work take place in public
Slack channels and open repositories.
* Fairness: All stakeholders have the opportunity to provide feedback and submit
contributions, which will be considered on their merits.
* Community over Product or Company: Sustaining and growing our community takes
priority over shipping code or sponsors' organizational goals. Each
contributor participates in the project as an individual.
* Community Before Individual Demand: As a community-driven open source project, we emphasize
the importance of collaboration and contribution. Maintainers and contributors work together towards the project's growth, not to serve unilateral user demands. Users pretending features or enhancements for their sole benefit without contributing to the effort are not aligned with our community values.
* Inclusivity: We innovate through different perspectives and skill sets, which
can only be accomplished in a welcoming and respectful environment.
* Participation: Responsibilities within the project are earned through
participation, and there is a clear path up the contributor ladder into leadership
positions.
## Maintainers
Capsule Maintainers have write access to the [project GitHub repository](https://github.com/orgs/projectcapsule). They can merge their own patches or patches from others. The current maintainers
can be found in [MAINTAINERS.md](./MAINTAINERS.md). Maintainers collectively manage the project's
resources and contributors.
This privilege is granted with some expectation of responsibility: maintainers
are people who care about the Capsule project and want to help it grow and
improve. A maintainer is not just someone who can make changes, but someone who
has demonstrated their ability to collaborate with the team, get the most
knowledgeable people to review code and docs.
A maintainer is a contributor to the project's success and a citizen helping
the project succeed. The collective team of all Maintainers is known as the Maintainer Council, which
is the governing body for the project.
### Becoming a Maintainer
To become a Maintainer you need to demonstrate the following:
* commitment to the project:
* participate in discussions, contributions, code and documentation reviews,
* perform reviews for non-trivial pull requests,
* contribute non-trivial pull requests and have them merged,
* ability to write quality code and/or documentation,
* ability to collaborate with the team,
* understanding of how the team works (policies, processes for testing and code review, etc),
* understanding of the project's purpose, code base and coding and documentation style.
A new Maintainer must be proposed by an existing maintainer by sending a message to all the other existing Maintainers. A simple majority vote of existing Maintainers
approves the application. Maintainers nominations will be evaluated without prejudice
to employer or demographics.
Maintainers who are selected will be granted the necessary GitHub rights.
### Removing a Maintainer
Maintainers may resign at any time if they feel that they will not be able to
continue fulfilling their project duties.
Maintainers may also be removed after being inactive, failure to fulfill their
Maintainer responsibilities, violating the Code of Conduct, or other reasons.
A Maintainer may be removed at any time by a 2/3 vote of the remaining maintainers.
Depending on the reason for removal, a Maintainer may be converted to Emeritus
status. Emeritus Maintainers will still be consulted on some project matters,
and can be rapidly returned to Maintainer status if their availability changes.
## Meetings
Time zones permitting, Maintainers are expected to participate in the public
developer meeting and/or public discussions.
Maintainers will also have closed meetings in order to discuss security reports
or Code of Conduct violations. Such meetings should be scheduled by any
Maintainer on receipt of a security issue or CoC report. All current Maintainers
must be invited to such closed meetings, except for any Maintainer who is
accused of a CoC violation.
## CNCF Resources
Any Maintainer may suggest a request for CNCF resources. A simple majority of Maintainers
approves the request. The Maintainers may also choose to delegate working with the CNCF to non-Maintainer community members, who will then be added to the [CNCF's Maintainer List](https://github.com/cncf/foundation/blob/main/project-maintainers.csv) for that purpose.
## Code of Conduct
[Code of Conduct](./CODE_OF_CONDUCT.md)
violations by community members will be discussed and resolved in private Maintainer meetings. If a Maintainer is directly involved in the report, the Maintainers will instead designate two Maintainers to work with the CNCF Code of Conduct Committee in resolving it.
## Security Response Team
The Maintainers will appoint a Security Response Team to handle security reports.
This committee may simply consist of the Maintainer Council themselves. If this
responsibility is delegated, the Maintainers will appoint a team of at least two
contributors to handle it. The Maintainers will review who is assigned to this
at least once a year.
The Security Response Team is responsible for handling all reports of security
holes and breaches according to the [security policy](TODO:Link to security.md).
## Voting
While most business in Capsule Project is conducted by "[lazy consensus](https://community.apache.org/committers/lazyConsensus.html)",
periodically the Maintainers may need to vote on specific actions or changes.
Any Maintainer may demand a vote be taken.
Most votes require a simple majority of all Maintainers to succeed, except where
otherwise noted. Two-thirds majority votes mean at least two-thirds of all
existing maintainers.
## Modifying this Charter
Changes to this Governance and its supporting documents may be approved by
**Join the community** on the [#capsule](https://kubernetes.slack.com/archives/C03GETTJQRL) channel in the [Kubernetes Slack](https://slack.k8s.io/).
# Kubernetes multi-tenancy made easy
**Capsule** helps to implement a multi-tenancy and policy-based environment in your Kubernetes cluster. It is not intended to be yet another _PaaS_, instead, it has been designed as a micro-services-based ecosystem with the minimalist approach, leveraging only on upstream Kubernetes.
**Capsule** implements a multi-tenant and policy-based environment in your Kubernetes cluster. It is designed as a micro-services-based ecosystem with the minimalist approach, leveraging only on upstream Kubernetes.
# What's the problem with the current status?
Kubernetes introduces the _Namespace_ object type to create logical partitions of the cluster as isolated *slices*. However, implementing advanced multi-tenancy scenarios, it soon becomes complicated because of the flat structure of Kubernetes namespaces and the impossibility to share resources among namespaces belonging to the same tenant. To overcome this, cluster admins tend to provision a dedicated cluster for each groups of users, teams, or departments. As an organization grows, the number of clusters to manage and keep aligned becomes an operational nightmare, described as the well know phenomena of the _clusters sprawl_.
Kubernetes introduces the _Namespace_ object type to create logical partitions of the cluster as isolated *slices*. However, implementing advanced multi-tenancy scenarios, it soon becomes complicated because of the flat structure of Kubernetes namespaces and the impossibility to share resources among namespaces belonging to the same tenant. To overcome this, cluster admins tend to provision a dedicated cluster for each groups of users, teams, or departments. As an organization grows, the number of clusters to manage and keep aligned becomes an operational nightmare, described as the well known phenomena of the _clusters sprawl_.
# Entering Capsule
Capsule takes a different approach. In a single cluster, the Capsule Controller aggregates multiple namespaces in a lightweight abstraction called _Tenant_, basically a grouping of Kubernetes Namespaces. Within each tenant, users are free to create their namespaces and share all the assigned resources while the Capsule Policy Engine keeps the different tenants isolated from each other.
The _Network and Security Policies_, _Resource Quota_, _Limit Ranges_, _RBAC_, and other policies defined at the tenant level are automatically inherited by all the namespaces in the tenant. Then users are free to operate their tenants in autonomy, without the intervention of the cluster administrator. Take a look at following diagram:
Capsule takes a different approach. In a single cluster, the Capsule Controller aggregates multiple namespaces in a lightweight abstraction called _Tenant_, basically a grouping of Kubernetes Namespaces. Within each tenant, users are free to create their namespaces and share all the assigned resources.
<p align="center" style="padding: 60px 20px">
<img src="assets/capsule-operator.svg" />
</p>
On the other side, the Capsule Policy Engine keeps the different tenants isolated from each other. _Network and Security Policies_, _Resource Quota_, _Limit Ranges_, _RBAC_, and other policies defined at the tenant level are automatically inherited by all the namespaces in the tenant. Then users are free to operate their tenants in autonomy, without the intervention of the cluster administrator.
# Features
## Self-Service
Leave to developers the freedom to self-provision their cluster resources according to the assigned boundaries.
Leave developers the freedom to self-provision their cluster resources according to the assigned boundaries.
## Preventing Clusters Sprawl
Share a single cluster with multiple teams, groups of users, or departments by saving operational and management efforts.
## Governance
Leverage Kubernetes Admission Controllers to enforce the industry security best practices and meet legal requirements.
Leverage Kubernetes Admission Controllers to enforce the industry security best practices and meet policy requirements.
## Resources Control
Take control of the resources consumed by users while preventing them to overtake.
## Native Experience
Provide multi-tenancy with a native Kubernetes experience without introducing additional management layers, plugins, or customized binaries.
## GitOps ready
Capsule is completely declarative and GitOps ready.
## Bring your own device (BYOD)
Assign to tenants a dedicated set of compute, storage, and network resources and avoid the noisy neighbors' effect.
# Common use cases for Capsule
Please, refer to the corresponding [section](./docs/operator/use-cases/overview.md) in the project documentation for a detailed list of common use cases that Capsule can address.
# Installation
Make sure you have access to a Kubernetes cluster as administrator.
There are two ways to install Capsule:
* Use the Helm Chart available [here](./charts/capsule/README.md)
* Use the [single YAML file installer](./config/install.yaml)
## Install with the single YAML file installer
Ensure you have `kubectl` installed in your `PATH`.
Clone this repository and move to the repo folder:
NAME STATE NAMESPACE QUOTA NAMESPACE COUNT NODE SELECTOR AGE
gas Active 3 0 {"kubernetes.io/os":"linux"} 25s
```
## Tenant owners
Each tenant comes with a delegated user or group of users acting as the tenant admin. In the Capsule jargon, this is called the _Tenant Owner_. Other users can operate inside a tenant with different levels of permissions and authorizations assigned directly by the Tenant Owner.
Capsule does not care about the authentication strategy used in the cluster and all the Kubernetes methods of [authentication](https://kubernetes.io/docs/reference/access-authn-authz/authentication/) are supported. The only requirement to use Capsule is to assign tenant users to the the group defined by `--capsule-user-group` option, which defaults to `capsule.clastix.io`.
Assignment to a group depends on the authentication strategy in your cluster.
For example, if you are using `capsule.clastix.io`, users authenticated through a _X.509_ certificate must have `capsule.clastix.io` as _Organization_: `-subj "/CN=${USER}/O=capsule.clastix.io"`
Users authenticated through an _OIDC token_ must have in their token:
```json
...
"users_groups":[
"capsule.clastix.io",
"other_group"
]
```
The [hack/create-user.sh](hack/create-user.sh) can help you set up a dummy `kubeconfig` for the `bob` user acting as owner of a tenant called `gas`
```bash
./hack/create-user.sh bob gas
...
certificatesigningrequest.certificates.k8s.io/bob-gas created
to use it as bob exportKUBECONFIG=bob-gas.kubeconfig
```
## Working with Tenants
Log in to the Kubernetes cluster as `bob` tenant owner
```
$ export KUBECONFIG=bob-gas.kubeconfig
```
and create a couple of new namespaces
```
$ kubectl create namespace gas-production
$ kubectl create namespace gas-development
```
As user `bob` you can operate with fully admin permissions:
```
$ kubectl -n gas-development run nginx --image=docker.io/nginx
$ kubectl -n gas-development get pods
```
but limited to only your own namespaces:
```
$ kubectl -n kube-system get pods
Error from server (Forbidden): pods is forbidden:
User "bob" cannot list resource "pods" in API group "" in the namespace "kube-system"
```
# Removal
Similar to `deploy`, you can get rid of Capsule using the `remove` target.
```
$ make remove
```
# Documentation
Please, check the project [documentation](./docs/index.md) for more cool things you can do with Capsule.
# Contribution
Please, check the project [documentation](https://projectcapsule.dev) for the cool things you can do with Capsule.
# Contributions
Capsule is Open Source with Apache 2 license and any contribution is welcome.
Please refer to the corresponding docs:
- [contributing.md](./docs/contributing.md) for the general guide; and
- [dev-guide.md](./docs/dev-guide.md) for how to set up the development env to get started.
## Chart Development
### Chart Linting
The chart is linted with [ct](https://github.com/helm/chart-testing). You can run the linter locally with this command:
```
make helm-lint
```
### Chart Documentation
The documentation for each chart is done with [helm-docs](https://github.com/norwoodj/helm-docs). This way we can ensure that values are consistent with the chart documentation. Run this anytime you make changes to a `values.yaml` file:
```
make helm-docs
```
## Community meeting
Join the community, share and learn from it. You can find all the resources to how to contribute code and docs, connect with people in the [community repository](https://github.com/projectcapsule/capsule-community).
Please read the [code of conduct](CODE_OF_CONDUCT.md).
## Adopters
See the [ADOPTERS.md](ADOPTERS.md) file for a list of companies that are using Capsule.
# Project Governance
You can find how the Capsule project is governed [here](https://projectcapsule.dev/project/governance/).
## Maintainers
Please, refer to the maintainers file available [here](.github/maintainers.yaml).
### Changelog
Read how we log changes [here](CHANGELOG.md)
### Software Bill of Materials
All OCI release artifacts include a Software Bill of Materials (SBOM) in CycloneDX JSON format. More information on this is available [here](SECURITY.md#software-bill-of-materials-sbom)
# FAQ
- Q. How to pronounce Capsule?
A. It should be pronounced as `/ˈkæpsjuːl/`.
- Q. Is it production grade?
A. Although under frequent development and improvements, Capsule is ready to be used in production environments as currently, people are using it in public and private deployments. Check out the [release](https://github.com/clastix/capsule/releases) page for a detailed list of available versions.
A. Although under frequent development and improvements, Capsule is ready to be used in production environments as currently, people are using it in public and private deployments. Check out the [release](https://github.com/projectcapsule/capsule/releases) page for a detailed list of available versions.
- Q. Does it work with my Kubernetes XYZ distribution?
future features and fixes are planned with [release milestones on GitHub](https://github.com/projectcapsule/capsule/milestones?direction=asc&sort=due_date&state=open). You can influence the roadmap by opening issues or joining our community meetings.
The Capsule community has adopted this security disclosures and response policy to ensure we responsibly handle critical issues.
## Bulletins
For information regarding the security of this project please join our [slack channel](https://kubernetes.slack.com/archives/C03GETTJQRL).
## Covered Repositories and Issues
When we say "a security vulnerability in capsule" we mean a security issue
in any repository under the [projectcapsule GitHub organization](https://github.com/projectcapsule/).
This reporting process is intended only for security issues in the capsule
project itself, and doesn't apply to applications _using_ capsule or to
issues which do not affect security.
Don't use this process if:
* You have issues with your capsule installation or configuration
* Your issue is not security related
### Explicitly Not Covered: Vulnerability Scanner Reports
We do not accept reports which amount to copy and pasted output from a vulnerability
scanning tool **unless** work has specifically been done to confirm that a vulnerability
reported by the tool _actually exists_ in capsule.
## Reporting a Vulnerability
To report a security issue or vulnerability, [submit a private vulnerability report via GitHub](https://github.com/projectcapsule/capsule/security/advisories/new) to the repository maintainers with a description of the issue, the steps you took to create the issue, affected versions, and, if known, mitigations for the issue.
Describe the issue in English, ideally with some example configuration or code which allows the issue to be reproduced. Explain why you believe this to be a security issue in capsule, if that's not obvious. should contain the following:
* description of the problem
* precise and detailed steps (include screenshots)
* the affected version(s). This may also include environment relevant versions.
* any possible mitigations
If the issue is confirmed as a vulnerability, we will open a Security Advisory and acknowledge your contributions as part of it.
## Reponse
Response times could be affected by weekends, holidays, breaks or time zone differences. That said, the security response team will endeavour to reply as soon as possible, ideally within 5 working days.
## Security Contacts
[Maintainers](./github/maintainers.yaml) of this project are responsible for the security of the project as outlined in this policy.
# Release Artifacts
[See all the available artifacts](https://github.com/orgs/projectcapsule/packages?repo_name=capsule)
## Verifing
To verify artifacts you need to have [cosign installed](https://github.com/sigstore/cosign#installation). This guide assumes you are using v2.x of cosign. All of the signatures are created using [keyless signing](https://docs.sigstore.dev/verifying/verify/#keyless-verification-using-openid-connect). You can set the environment variable `COSIGN_REPOSITORY` to point to this repository. For example:
To verify the signature of the docker image, run the following command. Replace `<release_tag>` with an [available release tag](https://github.com/projectcapsule/capsule/pkgs/container/capsule):
To verify the signature of the helm image, run the following command. Replace `<release_tag>` with an [available release tag](https://github.com/projectcapsule/capsule/pkgs/container/charts%2Fcapsule):
Capsule creates and attests to the provenance of its builds using the [SLSA standard](https://slsa.dev/spec/v0.2/provenance) and meets the [SLSA Level 3](https://slsa.dev/spec/v0.1/levels) specification. The attested provenance may be verified using the cosign tool.
Verify the provenance of the docker image. Replace `<release_tag>` with an [available release tag](https://github.com/projectcapsule/capsule/pkgs/container/capsule)
Verify the provenance of the helm image. Replace `<release_tag>` with an [available release tag](https://github.com/projectcapsule/capsule/pkgs/container/charts%2Fcapsule)
An SBOM (Software Bill of Materials) in CycloneDX JSON format is published for each release, including pre-releases. You can set the environment variable `COSIGN_REPOSITORY` to point to this repository. For example:
To inspect the SBOM of the docker image, run the following command. Replace `<release_tag>` with an [available release tag](https://github.com/projectcapsule/capsule/pkgs/container/capsule):
To inspect the SBOM of the helm image, run the following command. Replace `<release_tag>` with an [available release tag](https://github.com/projectcapsule/capsule/pkgs/container/charts%2Fcapsule):
Our Security Policy and Workflows are based on the work of the [Kyverno](https://github.com/kyverno) and [Cert-Manager](https://github.com/cert-manager) community.
Capsule implements a multi-tenant and policy-based environment in your Kubernetes cluster.
It is designed as a micro-services-based ecosystem with a minimalist approach, leveraging only upstream Kubernetes.
### Background
Capsule takes a different approach.
In a single cluster, the Capsule Controller aggregates multiple namespaces in a lightweight abstraction called Tenant, basically a grouping of Kubernetes Namespaces.
Within each tenant, users are free to create their namespaces and share all the assigned resources.
On the other side, the Capsule Policy Engine keeps the different tenants isolated from each other.
Network and Security Policies, Resource Quota, Limit Ranges, RBAC, and other policies defined at the tenant level are automatically inherited by all the namespaces in the tenant.
Then users are free to operate their tenants in autonomy, without the intervention of the cluster administrator.
Capsule was accepted as a CNCF sandbox project in December 2022.
## Actors
### Capsule Operator
It's the Operator which provides all the multi-tenant capabilities offered by Capsule.
It's made of two internal components, such as the webhooks server (known as _policy engine_), and the _tenant controller_.
**Capsule Tenant Controller**
The controller is responsible for managing the tenants by reconciling the required objects at the Namespace level, such as _Network Policy_, _LimitRange_, _ResourceQuota_, _Role Binding_, as well as labelling the Namespace objects belonging to a Tenant according to their desired metadata.
It is responsible for binding Namespaces to the selected Tenant, and managing their lifecycle.
Furthermore, the manager can replicate objects thanks to the **Tenant Resource** API, which offers two levels of interactions: a cluster-scoped one thanks to the `GlobalTenantResource` API, and a namespace-scoped one named `TenantResource`.
The replicated resources are dynamically created, and replicated by Capsule itself, as well as preserving the deletion of these objects by the Tenant owner.
**Capsule Tenant Controller (Policy Engine)**
Policies are defined on a Tenant basis: therefore the policy engine is enforcing these policies on the tenants's Namespaces and their children's resources.
The Policy Engine is currently not a dedicated component, but a part of the Capsule Tenant Controller.
The webhook server, also known as the policy engine, interpolates the Tenant rules and takes full advantage of the dynamic admission controllers offered by Kubernetes itself (such as `ValidatingWebhookConfiguration` and `MutatingWebhookConfiguration`).
Thanks to the _policy engine_ the cluster administrators can enforce specific rules such as preventing _Pod_ objects from untrusted registries to run or preventing the creation of _PersistentVolumeClaim_ resources using a non-allowed _StorageClass_, etc.
It also acts as a defaulter webhook, offloading the need to specify some classes (IngressClass, StorageClass, RuntimeClass, etc.).
### Capsule Proxy
The `capsule-proxy` is an addon which is offering a Kubernetes API Server shim aware of the multi-tenancy levels implemented by Capsule.
It's essentially a reverse proxy that decorates the incoming requests with the required `labelSelector` query string parameters to filter out some objects, such as:
- Namespaces
- IngressClass
- StorageClass
- PriorityClass
- RuntimeClass
- PersistentVolumes
Permissions on those resources are not enforced through the classic Kubernetes RBAC but rather at the Tenant Owner level, allowing fine-grained control over specific resources.
`capsule-proxy` is not serving itself Kubernetes API server responses, but rather, it's acting as a middle proxy server offering dynamic filtering of requests: this means the resulting responses from the upstream are not mangled, and don't require any additional plugins, or third-party binaries, and integrating with any external components, such as the Kubernetes dashboard, the `kubectl` binary, etc.
## Actions
Tenants are created by cluster administrators, who have the right to create Tenant custom resource instances.
End users should not manage tenants.
Therefore users without any cluster administration rights can't list tenants or create tenants.
## Creating namespaces in a tenant
When creating a tenant, the Capsule controller inspects the user's supplied groups and matches, if the groups were defined in the `capsuleConfiguration`. If at least one matching group is found, the user request is considered by Capsule. If not, Capsule ignores the request and does not perform any action. This also applies to modifications (`UPDATE` request).
To create namespaces within a tenant a User, Group or ServiceAccount must be configured as owner of the tenant.
A namespace is assigned to a tenant based on its label, its owner (if the owner only has one tenant) or the prefix of the namespace (which matches the tenant name).
If the request is considered, the namespace is created with all the configuration on the tenant, which is relevant. The additional resources (NetworkPolicy, ResourceQuota, LimitRange) are created in the new namespace.
### Applying Workload and configs to namespaces within a tenant
Whenever a tenant user applies new workloads or configs to a namespace, the capsule controller inspects the namespace and checks if it belongs to a tenant. If so, the capsule controller applies policies from the tenant configuration to the given workloads and configs.
If there are defaults defined on the tenant, they are applied to the workloads as well.
This is a further abstraction from having cluster defaults (eg. default `StorageClass`) to having tenant defaults (eg. default `StorageClass` for a tenant).
### Goals
**General**
* **Multitenancy**: Capsule should be able to support multiple tenants in a single Kubernetes cluster without introducing overhead or cognitive load, and barely relying on Namespace objects.
* **Kubernetes agnostic**: Capsule should integrate with Kubernetes primitives, such as _RBAC_, _NetworkPolicy_, _LimitRange_, and _ResourceQuota_.
* **Policy-based**: Capsule should be able to enforce policies on tenants, which are defined on a tenant basis.
* **Native User Experience**: Capsule shouldn't increase the cognitive load of developers, such as introducing `kubectl` plugins, or forcing the tenant owners to operate their tenant objects using Custom Resource Definitions.
### Non-Goals
**General**
* **Control Plane**: Capsule can't mimic for each tenant a feeling of a dedicated control plane.
* **Custom Resource Definitions**: Capsule doesn't want to provide virtual cluster capabilities and it's sticking to the native Kubernetes user experience and design; rather, its focus is to provide a governance solution by focusing on resource optimization and security lockdown.
## Self-assessment use
This self-assessment is created by the Capsule team to perform an internal analysis of the project's security.
It is not intended to provide a security audit of Capsule, or function as an independent assessment or attestation of Capsule’s security health.
This document serves to provide Capsule users with an initial understanding of Capsule's security, where to find existing security documentation, Capsule plans for security, and a general overview of Capsule security practices, both for the development of Capsule as well as security of Capsule.
This document provides the CNCF TAG-Security with an initial understanding of Capsule to assist in a joint review, necessary for projects under incubation. Taken together, this document and the joint review serve as a cornerstone for if and when Capsule seeks graduation.
## Security functions and features
See [Actors](#actors) and [Actions](#actions) for a more detailed description of the critical actors, actions, and potential threats.
## Project compliance
As of now, not applicable.
## Secure development practices
The Capsule project follows established CNCF and OSS best practices for code development and delivery.
Capsule follows [OpenSSF Best Practices](https://www.bestpractices.dev/en/projects/5601).
Although not perfect yet, we are constantly trying to improve and score optimal scores.
We will assess the issues during our community meetings and try to plan them for future releases.
### Development Pipeline
Changes must be reviewed and merged by the project maintainers.
Before changes are merged, all the changes must pass static checks, license checks, verifications on `gofmt`, `go lint`, `go vet`, and pass all unit tests and e2e tests.
Changes are scanned by trivy for the docker images.
We run E2E tests for different Kubernetes versions on Pull Requests.
Code changes are submitted via Pull Requests (PRs) and must be signed and verified.
Commits to the main branch directly are not allowed.
## Security issue resolution
Capsule project vulnerability handling related processes are recorded in the [Capsule Security Doc](https://github.com/projectcapsule/capsule/blob/main/SECURITY.md).
Related security vulnerabilities can be reported and communicated via email to [cncf-capsule-maintainers@lists.cncf.io](mailto:cncf-capsule-maintainers@lists.cncf.i).
## Appendix
All Capsule security-related issues (both fixes and enhancements) are labelled with "security" and can be queried using[ https://github.com/projectcapsule/capsule/labels/security](https://github.com/projectcapsule/capsule/labels/security).
The code review process requires maintainers to consider security while reviewing designs and pull requests.
// +kubebuilder:printcolumn:name="Namespace quota",type="integer",JSONPath=".spec.namespaceQuota",description="The max amount of Namespaces can be created"
// +kubebuilder:printcolumn:name="Namespace count",type="integer",JSONPath=".status.size",description="The total amount of Namespaces in use"
// Specifies the allowed IngressClasses assigned to the Tenant. Capsule assures that all Ingress resources created in the Tenant can use only one of the allowed IngressClasses. Optional.
// Specifies the allowed hostnames in Ingresses for the given Tenant. Capsule assures that all Ingress resources created in the Tenant can use only one of the allowed hostnames. Optional.
// Specifies the maximum number of namespaces allowed for that Tenant. Once the namespace quota assigned to the Tenant has been reached, the Tenant owner cannot create further namespaces. Optional.
Quota*int32`json:"quota,omitempty"`
// Specifies additional labels and annotations the Capsule operator places on any Namespace resource in the Tenant. Optional.
// TenantSpec defines the desired state of Tenant.
typeTenantSpecstruct{
// Specifies the owners of the Tenant. Mandatory.
OwnersOwnerListSpec`json:"owners"`
// Specifies options for the Namespaces, such as additional metadata or maximum number of namespaces allowed for that Tenant. Once the namespace quota assigned to the Tenant has been reached, the Tenant owner cannot create further namespaces. Optional.
// Specifies the allowed StorageClasses assigned to the Tenant. Capsule assures that all PersistentVolumeClaim resources created in the Tenant can use only one of the allowed StorageClasses. Optional.
// Specifies the trusted Image Registries assigned to the Tenant. Capsule assures that all Pods resources created in the Tenant can use only one of the allowed trusted registries. Optional.
// Specifies the label to control the placement of pods on a given pool of worker nodes. All namesapces created within the Tenant will have the node selector annotation. This annotation tells the Kubernetes scheduler to place pods on the nodes having the selector label. Optional.
// Specifies the label to control the placement of pods on a given pool of worker nodes. All namespaces created within the Tenant will have the node selector annotation. This annotation tells the Kubernetes scheduler to place pods on the nodes having the selector label. Optional.
// Specifies the resource min/max usage restrictions to the Tenant. The assigned values are inherited by any namespace created in the Tenant. Optional.
// Specifies a list of ResourceQuota resources assigned to the Tenant. The assigned values are inherited by any namespace created in the Tenant. The Capsule operator aggregates ResourceQuota at Tenant level, so that the hard quota is never crossed for the given Tenant. This permits the Tenant owner to consume resources in the Tenant regardless of the namespace. Optional.
// Specifies additional RoleBindings assigned to the Tenant. Capsule will ensure that all namespaces in the Tenant always contain the RoleBinding for the given ClusterRole. Optional.
// Specify the allowed values for the imagePullPolicies option in Pod resources. Capsule assures that all Pod resources created in the Tenant can use only one of the allowed policy. Optional.
// Specifies the allowed priorityClasses assigned to the Tenant. Capsule assures that all Pods resources created in the Tenant can use only one of the allowed PriorityClasses. Optional.
// +kubebuilder:printcolumn:name="State",type="string",JSONPath=".status.state",description="The actual state of the Tenant"
// +kubebuilder:printcolumn:name="Namespace quota",type="integer",JSONPath=".spec.namespaceOptions.quota",description="The max amount of Namespaces can be created"
@@ -47,7 +48,7 @@ type TenantSpec struct {
// +kubebuilder:printcolumn:name="Node selector",type="string",JSONPath=".spec.nodeSelector",description="Node Selector applied to Pods"
// Specifies the allowed hostnames in Ingresses for the given Tenant. Capsule assures that all Ingress resources created in the Tenant can use only one of the allowed hostnames. Optional.
// Specifies the maximum number of namespaces allowed for that Tenant. Once the namespace quota assigned to the Tenant has been reached, the Tenant owner cannot create further namespaces. Optional.
Quota*int32`json:"quota,omitempty"`
// Specifies additional labels and annotations the Capsule operator places on any Namespace resource in the Tenant. Optional.
// TenantSpec defines the desired state of Tenant.
typeTenantSpecstruct{
// Specifies the owners of the Tenant. Mandatory.
OwnersOwnerListSpec`json:"owners"`
// Specifies options for the Namespaces, such as additional metadata or maximum number of namespaces allowed for that Tenant. Once the namespace quota assigned to the Tenant has been reached, the Tenant owner cannot create further namespaces. Optional.
// Specifies the trusted Image Registries assigned to the Tenant. Capsule assures that all Pods resources created in the Tenant can use only one of the allowed trusted registries. Optional.
// Specifies the label to control the placement of pods on a given pool of worker nodes. All namespaces created within the Tenant will have the node selector annotation. This annotation tells the Kubernetes scheduler to place pods on the nodes having the selector label. Optional.
// Specifies the resource min/max usage restrictions to the Tenant. The assigned values are inherited by any namespace created in the Tenant. Optional.
// Specifies a list of ResourceQuota resources assigned to the Tenant. The assigned values are inherited by any namespace created in the Tenant. The Capsule operator aggregates ResourceQuota at Tenant level, so that the hard quota is never crossed for the given Tenant. This permits the Tenant owner to consume resources in the Tenant regardless of the namespace. Optional.
// Specifies additional RoleBindings assigned to the Tenant. Capsule will ensure that all namespaces in the Tenant always contain the RoleBinding for the given ClusterRole. Optional.
// Specify the allowed values for the imagePullPolicies option in Pod resources. Capsule assures that all Pod resources created in the Tenant can use only one of the allowed policy. Optional.
// +kubebuilder:printcolumn:name="State",type="string",JSONPath=".status.state",description="The actual state of the Tenant"
// +kubebuilder:printcolumn:name="Namespace quota",type="integer",JSONPath=".spec.namespaceOptions.quota",description="The max amount of Namespaces can be created"
// +kubebuilder:printcolumn:name="Namespace count",type="integer",JSONPath=".status.size",description="The total amount of Namespaces in use"
// +kubebuilder:printcolumn:name="Node selector",type="string",JSONPath=".spec.nodeSelector",description="Node Selector applied to Pods"
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
