Compare commits

..

88 Commits

Author SHA1 Message Date
Massimiliano Giovagnoli
857c338c53 wip
Signed-off-by: Massimiliano Giovagnoli <me@maxgio.it>
2022-08-15 20:51:45 +02:00
Massimiliano Giovagnoli
5a9c25b125 refactor(api/v1beta1/owner_role.go): split cluster role that need to be cluster bound
Signed-off-by: Massimiliano Giovagnoli <me@maxgio.it>
2022-08-13 18:56:35 +02:00
Massimiliano Giovagnoli
3cd7bfe6d4 chore(controllers/tenant): rename tenant clusterrole controller
Signed-off-by: Massimiliano Giovagnoli <me@maxgio.it>
2022-08-13 18:29:09 +02:00
Massimiliano Giovagnoli
ff53cc2f38 feat(controllers/tenant): ensure per-tenant owners roles
add gitops ready cluster roles per tenant owners.

Signed-off-by: Massimiliano Giovagnoli <me@maxgio.it>
2022-08-13 16:10:48 +02:00
Massimiliano Giovagnoli
852ab16323 feat(api/v1beta1/owner_role): bind gitops roles to owners
Signed-off-by: Massimiliano Giovagnoli <me@maxgio.it>
2022-08-13 16:00:04 +02:00
Massimiliano Giovagnoli
9c18471879 feat(tenant/tenant/spec): add initial knob to enable the gitops-ready rbac
Signed-off-by: Massimiliano Giovagnoli <me@maxgio.it>
2022-08-13 15:50:25 +02:00
Dario Tranchitella
3435f5464b chore: adding maintainers list 2022-08-05 11:48:31 +02:00
Dario Tranchitella
f216d0bd8d docs: adding adopters list section 2022-08-02 10:18:04 +02:00
alegrey91
f9e7256746 ci: add gosec pipeline 2022-08-01 09:37:12 +02:00
Adriano Pezzuto
5b46e8eb81 docs(tutorial): tenant backup and restore with velero (#626) 2022-07-31 11:00:29 +02:00
Adriano Pezzuto
dd5ed4575e Clarify tenant owner permissions in documentation (#625)
* docs(tutorial): clarify tenant owner permissions and minor improvements
2022-07-31 09:37:12 +02:00
Adriano Pezzuto
f9554d4cae Document how to implement Pod Security Standard (#624)
* docs(guides): add pod security guide and other minor enhancements
2022-07-30 21:30:14 +02:00
Dario Tranchitella
a36c7545db chore(helm): bumping up chart 2022-07-26 20:41:33 +02:00
Dario Tranchitella
f612ecea0c chore: bumping up to v0.1.2 release 2022-07-26 20:11:03 +02:00
Dario Tranchitella
098a74b565 refactor(capsuleconfiguration): allowing to skip tls reconciler 2022-07-26 17:48:58 +02:00
Dario Tranchitella
5a8a8ae77a feat(helm): support for cert-manager and externally managed tls secret 2022-07-26 17:48:58 +02:00
Dario Tranchitella
a8430f2e72 fix(helm): missing blank space in the notes 2022-07-26 17:48:58 +02:00
Dario Tranchitella
3afc470534 chore(e2e): triggering e2e also for pkg files 2022-07-22 19:29:27 +00:00
Dario Tranchitella
d84f0be76b fix: tenant owners cannot replace protected namesapce labels or annotations 2022-07-22 19:29:27 +00:00
dependabot[bot]
3a174bf755 build(deps): bump moment from 2.29.2 to 2.29.4 in /docs
Bumps [moment](https://github.com/moment/moment) from 2.29.2 to 2.29.4.
- [Release notes](https://github.com/moment/moment/releases)
- [Changelog](https://github.com/moment/moment/blob/develop/CHANGELOG.md)
- [Commits](https://github.com/moment/moment/compare/2.29.2...2.29.4)

---
updated-dependencies:
- dependency-name: moment
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-07-22 18:05:18 +00:00
Massimiliano Giovagnoli
90a2e9c742 docs(guides/flux2-capsule-gitops-multitenancy): add missing picture
Signed-off-by: Massimiliano Giovagnoli <me@maxgio.it>
2022-07-22 09:44:20 +00:00
Massimiliano Giovagnoli
a091331070 docs(guides/flux2-capsule-gitops-multitenancy): strip down introductory content
Signed-off-by: Massimiliano Giovagnoli <me@maxgio.it>
2022-07-22 09:44:20 +00:00
Massimiliano Giovagnoli
cb3439bd3d docs(guides/flux2-capsule-gitops-multitenancy): initial commit
Signed-off-by: Massimiliano Giovagnoli <me@maxgio.it>
2022-07-22 09:44:20 +00:00
dependabot[bot]
1fd390b91e build(deps): bump terser from 4.8.0 to 4.8.1 in /docs
Bumps [terser](https://github.com/terser/terser) from 4.8.0 to 4.8.1.
- [Release notes](https://github.com/terser/terser/releases)
- [Changelog](https://github.com/terser/terser/blob/master/CHANGELOG.md)
- [Commits](https://github.com/terser/terser/commits)

---
updated-dependencies:
- dependency-name: terser
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-07-22 09:44:08 +00:00
Dario Tranchitella
80c83689f5 docs: documenting capsule-proxy metrics 2022-07-20 16:08:11 +00:00
Dario Tranchitella
da3d42801b chore(helm): releasing new helm chart (#605) 2022-07-18 08:49:33 +02:00
Adriano Pezzuto
9643885574 feat(config): move Grafana dashboard as Config Map 2022-07-18 08:42:32 +02:00
bsctl
ac3f2bbdd7 feat(helm): update manifests 2022-07-14 07:08:29 +00:00
bsctl
adb214f7f9 feat(helm): change values description 2022-07-14 07:08:29 +00:00
bsctl
ef26d0e6db feat(helm): remove scale down before uninstall 2022-07-14 07:08:29 +00:00
bsctl
3d6f29fa43 feat(helm): add DaemonSet deploy option 2022-07-14 07:08:29 +00:00
Dario Tranchitella
261876b59b docs: documenting new support for dynamic tenant owners clusterrole 2022-06-29 10:53:35 +00:00
Dario Tranchitella
ab750141c6 refactor: support for rfc 1123 for tenant owners cluster roles overrides 2022-06-29 10:53:35 +00:00
Oliver Bähler
e237249815 feat: improve chart documentation
Signed-off-by: Oliver Bähler <oliverbaehler@hotmail.com>
2022-06-29 08:35:43 +00:00
Dario Tranchitella
e15191c2a0 refactor: sentinel error for running in out of cluster mode 2022-06-29 08:31:21 +00:00
Dario Tranchitella
741db523e5 chore(gh): adding 1.24 to the e2e test matrix 2022-06-14 14:39:05 +00:00
Dario Tranchitella
7b3f850035 chore(gh): disabling fail fast for e2e 2022-06-13 09:52:58 +00:00
jandres - moscardo
72733415f0 fix(docs): helm example was wrong when customizing value 2022-06-10 13:48:49 +02:00
Oliver Bähler
cac2920827 feat: grant global patch privileges and add patch handler 2022-06-09 18:32:39 +00:00
Dario Tranchitella
e0b339d68a fix(tests): cleaning up protected tenant upon test success 2022-06-09 18:30:52 +00:00
Dario Tranchitella
4f55dd8db8 refactor: removing unrequired verb for clusterrole namespace deleter 2022-06-09 18:30:52 +00:00
Massimiliano Giovagnoli
fd738341ed docs: fix typos
Signed-off-by: Massimiliano Giovagnoli <me@maxgio.it>
2022-06-09 12:29:10 +00:00
Maksim Fedotov
fce1658827 chore: remove unused CASecretNameAnnotation constant 2022-06-08 11:12:35 +00:00
Maksim Fedotov
93547c128f build(helm): revert bumping chart version 2022-06-08 11:12:35 +00:00
Maksim Fedotov
f1dc028649 feat: generate TLS certificates before starting controllers 2022-06-08 11:12:35 +00:00
Maksim Fedotov
37381184d2 build(helm): refactor capsule TLS certificates management 2022-06-08 11:12:35 +00:00
Maksim Fedotov
82b58d7d53 feat: refactor capsule TLS certificates management 2022-06-08 11:12:35 +00:00
tony
60e826dc83 docs: update tenant owner default cluster documentation 2022-06-06 06:48:18 +02:00
dependabot[bot]
6e8ddd102f build(deps): bump eventsource from 1.1.0 to 1.1.1 in /docs
Bumps [eventsource](https://github.com/EventSource/eventsource) from 1.1.0 to 1.1.1.
- [Release notes](https://github.com/EventSource/eventsource/releases)
- [Changelog](https://github.com/EventSource/eventsource/blob/master/HISTORY.md)
- [Commits](https://github.com/EventSource/eventsource/compare/v1.1.0...v1.1.1)

---
updated-dependencies:
- dependency-name: eventsource
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-06-03 13:10:21 +00:00
Dario Tranchitella
b64aaebc89 docs: referring to docker hub image 2022-05-31 12:38:38 +00:00
Dario Tranchitella
9a85631bb8 chore(yaml): using docker hub image 2022-05-31 12:38:38 +00:00
Dario Tranchitella
51ed42981f chore(helm): using docker hub image 2022-05-31 12:38:38 +00:00
Dario Tranchitella
cf313d415b chore(make): using docker hub image 2022-05-31 12:38:38 +00:00
Adriano Pezzuto
526a6053a5 docs: documenting charmed operator (#572)
* docs: documenting charmed operator
2022-05-27 21:20:35 +02:00
Dario Tranchitella
0dd13a96fc chore(yaml): aligning to v0.1.2-rc0 image 2022-05-24 15:50:48 +00:00
Dario Tranchitella
1c8a5d8f5a docs(proxy): documenting retrieval of a single namespace 2022-05-24 15:32:04 +00:00
song
b9fc50861b style: removing unused struct field 2022-05-24 15:31:24 +00:00
ptx96
29d29ccd4b feat(ci): added docker.io repository 2022-05-24 13:26:57 +00:00
Massimiliano Giovagnoli
f207546af0 docs(readme.md): add slack link
Signed-off-by: Massimiliano Giovagnoli <me@maxgio.it>
2022-05-24 09:02:10 +00:00
Maksim Fedotov
deb0858fae build(helm): support cert-manager for generating tls and ca 2022-05-23 07:17:20 +00:00
Maksim Fedotov
1af56b736b feat: support cert-manager for generating tls and ca 2022-05-23 07:17:20 +00:00
Maksim Fedotov
3c9228d1aa fix: protectedHandler OnDelete get tenant using client 2022-05-18 18:06:10 +02:00
Maksim Fedotov
bf6760fbd0 docs: documenting protected tenants annotation 2022-05-18 18:06:10 +02:00
Maksim Fedotov
23564f8e40 feat: protected tenant annotation 2022-05-18 18:06:10 +02:00
Dario Tranchitella
a8b84c8cb3 fix: using sentinel error for non limited custom resource 2022-05-16 15:51:07 +00:00
Abhinandan Baheti
8c0c8c653d docs: documenting proxysetting crd use cases in capsule-proxy 2022-05-16 14:21:17 +00:00
Massimiliano Giovagnoli
ec89f5dd26 docs(readme.md): add links to community repo and governance doc
Signed-off-by: Massimiliano Giovagnoli <me@maxgio.it>
2022-05-13 09:34:58 +00:00
Dario Tranchitella
68956a075a chore(ci): pinning golangci-lint version 2022-05-10 12:48:32 +00:00
Massimiliano Giovagnoli
c036feeefc docs(general/proxy): remove duplicated doc about nodes
Signed-off-by: Massimiliano Giovagnoli <me@maxgio.it>
2022-05-09 12:52:23 +00:00
Dario Tranchitella
9f6883d309 fix: formatting error message for service-related objects 2022-05-05 13:33:39 +00:00
Dario Tranchitella
e7227d24e9 build(helm): alignement with latest changes 2022-05-05 13:33:39 +00:00
Dario Tranchitella
f168137407 build(installer): alignement with latest changes 2022-05-05 13:33:39 +00:00
Dario Tranchitella
49e76f7f93 style: linters refactoring 2022-05-05 13:33:39 +00:00
Dario Tranchitella
9d69770888 style: fixing linters issues 2022-05-05 13:33:39 +00:00
Dario Tranchitella
f4ac85dfed refactor: using k8s client scheme 2022-05-05 13:33:39 +00:00
Dario Tranchitella
cb4289d45b refactor: using kubernetes tls secret key names 2022-05-05 13:33:39 +00:00
Dario Tranchitella
01197892a4 refactor: optimizing watchers predicates 2022-05-05 13:33:39 +00:00
Dario Tranchitella
345836630c refactor: avoiding using background context 2022-05-05 13:33:39 +00:00
dependabot[bot]
69a6394e59 build(deps): bump async from 2.6.3 to 2.6.4 in /docs
Bumps [async](https://github.com/caolan/async) from 2.6.3 to 2.6.4.
- [Release notes](https://github.com/caolan/async/releases)
- [Changelog](https://github.com/caolan/async/blob/v2.6.4/CHANGELOG.md)
- [Commits](https://github.com/caolan/async/compare/v2.6.3...v2.6.4)

---
updated-dependencies:
- dependency-name: async
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-05-02 09:51:18 +00:00
Dario Tranchitella
a3495cf614 chore: go 1.18 support 2022-04-14 15:21:49 +00:00
Dario Tranchitella
7662c3dc6a docs: aligning to dynamic tenant owner roles 2022-04-14 14:35:59 +00:00
Dario Tranchitella
137b0f083b test: aligning to new rolebindings sync policies 2022-04-14 14:35:59 +00:00
Dario Tranchitella
9fd18db5a5 feat: dynamic cluster roles for tenant owners 2022-04-14 14:35:59 +00:00
Dario Tranchitella
364adf7d9e style: using constant for rbac group 2022-04-14 14:35:59 +00:00
Dario Tranchitella
cb3ce372b9 fix: ensuring ca bundle replication upon helm upgrade 2022-04-14 14:10:32 +00:00
gernest
59d81c2002 chore(build): makefile for building local binary
This commit fixes `make manager` command which builds local capsure
binary to  bin/manager.
2022-04-12 10:12:33 +00:00
dependabot[bot]
85861ee5dc build(deps): bump moment from 2.29.1 to 2.29.2 in /docs
Bumps [moment](https://github.com/moment/moment) from 2.29.1 to 2.29.2.
- [Release notes](https://github.com/moment/moment/releases)
- [Changelog](https://github.com/moment/moment/blob/develop/CHANGELOG.md)
- [Commits](https://github.com/moment/moment/compare/2.29.1...2.29.2)

---
updated-dependencies:
- dependency-name: moment
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-04-09 12:29:34 +00:00
dependabot[bot]
ed88606031 build(deps): bump minimist from 1.2.5 to 1.2.6 in /docs
Bumps [minimist](https://github.com/substack/minimist) from 1.2.5 to 1.2.6.
- [Release notes](https://github.com/substack/minimist/releases)
- [Commits](https://github.com/substack/minimist/compare/1.2.5...1.2.6)

---
updated-dependencies:
- dependency-name: minimist
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-04-08 17:11:51 +00:00
216 changed files with 6347 additions and 3855 deletions

18
.github/maintainers.yaml vendored Normal file
View File

@@ -0,0 +1,18 @@
- name: Adriano Pezzuto
github: https://github.com/bsctl
company: Clastix
projects:
- https://github.com/clastix/capsule
- https://github.com/clastix/capsule-proxy
- name: Dario Tranchitella
github: https://github.com/prometherion
company: Clastix
projects:
- https://github.com/clastix/capsule
- https://github.com/clastix/capsule-proxy
- name: Maksim Fedotov
github: https://github.com/MaxFedotov
company: wargaming.net
projects:
- https://github.com/clastix/capsule
- https://github.com/clastix/capsule-proxy

View File

@@ -24,7 +24,7 @@ jobs:
- name: Run golangci-lint
uses: golangci/golangci-lint-action@v2.3.0
with:
version: latest
version: v1.45.2
only-new-issues: false
args: --timeout 2m --config .golangci.yml
diff:
@@ -36,7 +36,7 @@ jobs:
fetch-depth: 0
- uses: actions/setup-go@v2
with:
go-version: '1.16'
go-version: '1.18'
- run: make installer
- name: Checking if YAML installer file is not aligned
run: if [[ $(git diff | wc -l) -gt 0 ]]; then echo ">>> Untracked generated files have not been committed" && git --no-pager diff && exit 1; fi

View File

@@ -36,6 +36,7 @@ jobs:
with:
images: |
quay.io/${{ github.repository }}
docker.io/${{ github.repository }}
tags: |
type=semver,pattern={{raw}}
flavor: |
@@ -68,6 +69,13 @@ jobs:
username: ${{ github.repository_owner }}+github
password: ${{ secrets.BOT_QUAY_IO }}
- name: Login to docker.io Container Registry
uses: docker/login-action@v1
with:
registry: docker.io
username: ${{ secrets.USER_DOCKER_IO }}
password: ${{ secrets.BOT_DOCKER_IO }}
- name: Build and push
id: build-release
uses: docker/build-push-action@v2

View File

@@ -7,6 +7,7 @@ on:
- '.github/workflows/e2e.yml'
- 'api/**'
- 'controllers/**'
- 'pkg/**'
- 'e2e/*'
- 'Dockerfile'
- 'go.*'
@@ -18,6 +19,7 @@ on:
- '.github/workflows/e2e.yml'
- 'api/**'
- 'controllers/**'
- 'pkg/**'
- 'e2e/*'
- 'Dockerfile'
- 'go.*'
@@ -28,8 +30,9 @@ jobs:
kind:
name: Kubernetes
strategy:
fail-fast: false
matrix:
k8s-version: ['v1.16.15', 'v1.17.11', 'v1.18.8', 'v1.19.4', 'v1.20.7', 'v1.21.2', 'v1.22.4', 'v1.23.0']
k8s-version: ['v1.16.15', 'v1.17.11', 'v1.18.8', 'v1.19.4', 'v1.20.7', 'v1.21.2', 'v1.22.4', 'v1.23.6', 'v1.24.1']
runs-on: ubuntu-18.04
steps:
- uses: actions/checkout@v2
@@ -37,21 +40,16 @@ jobs:
fetch-depth: 0
- uses: actions/setup-go@v2
with:
go-version: '1.16'
go-version: '1.18'
- run: make manifests
- name: Checking if manifests are disaligned
run: test -z "$(git diff 2> /dev/null)"
- name: Checking if manifests generated untracked files
run: test -z "$(git ls-files --others --exclude-standard 2> /dev/null)"
- name: Installing Ginkgo
run: go get github.com/onsi/ginkgo/ginkgo
- uses: actions/setup-go@v2
with:
go-version: '1.16'
- uses: engineerd/setup-kind@v0.5.0
with:
skipClusterCreation: true
version: v0.11.1
version: v0.14.0
- uses: azure/setup-helm@v1
with:
version: 3.3.4

18
.github/workflows/gosec.yml vendored Normal file
View File

@@ -0,0 +1,18 @@
name: CI gosec
on:
push:
branches: [ "*" ]
pull_request:
branches: [ "*" ]
jobs:
tests:
runs-on: ubuntu-latest
env:
GO111MODULE: on
steps:
- name: Checkout Source
uses: actions/checkout@v2
- name: Run Gosec Security Scanner
uses: securego/gosec@master
with:
args: ./...

1
.gitignore vendored
View File

@@ -28,4 +28,5 @@ bin
**/*.crt
**/*.key
.DS_Store
*.tgz

View File

@@ -1,51 +1,39 @@
linters-settings:
govet:
check-shadowing: true
golint:
min-confidence: 0
maligned:
suggest-new: true
goimports:
local-prefixes: github.com/clastix/capsule
dupl:
threshold: 100
goconst:
min-len: 2
min-occurrences: 2
cyclop:
max-complexity: 27
gocognit:
min-complexity: 50
gci:
sections:
- standard
- default
- prefix(github.com/clastix/capsule)
linters:
disable-all: true
enable:
- bodyclose
- deadcode
- depguard
- dogsled
- dupl
- errcheck
- goconst
- gocritic
- gofmt
- goimports
- golint
- goprintffuncname
- gosec
- gosimple
- govet
- ineffassign
- interfacer
- misspell
- nolintlint
- rowserrcheck
- scopelint
- staticcheck
- structcheck
- stylecheck
- typecheck
- unconvert
- unparam
- unused
- varcheck
- whitespace
enable-all: true
disable:
- funlen
- gochecknoinits
- lll
- exhaustivestruct
- maligned
- interfacer
- scopelint
- golint
- gochecknoglobals
- goerr113
- gomnd
- paralleltest
- ireturn
- testpackage
- varnamelen
- wrapcheck
issues:
exclude:

5
ADOPTERS.md Normal file
View File

@@ -0,0 +1,5 @@
# Adopters
This is a list of companies that have adopted Capsule, feel free to open a Pull-Request to get yours listed.
## Adopters list (alphabetically)

View File

@@ -1,5 +1,5 @@
# Build the manager binary
FROM golang:1.16 as builder
FROM golang:1.18 as builder
ARG TARGETARCH
ARG GIT_HEAD_COMMIT

View File

@@ -2,7 +2,7 @@
VERSION ?= $$(git describe --abbrev=0 --tags --match "v*")
# Default bundle image tag
BUNDLE_IMG ?= quay.io/clastix/capsule:$(VERSION)-bundle
BUNDLE_IMG ?= clastix/capsule:$(VERSION)-bundle
# Options for 'bundle-build'
ifneq ($(origin CHANNELS), undefined)
BUNDLE_CHANNELS := --channels=$(CHANNELS)
@@ -13,7 +13,7 @@ endif
BUNDLE_METADATA_OPTS ?= $(BUNDLE_CHANNELS) $(BUNDLE_DEFAULT_CHANNEL)
# Image URL to use all building/pushing image targets
IMG ?= quay.io/clastix/capsule:$(VERSION)
IMG ?= clastix/capsule:$(VERSION)
# Produce CRDs that work back to Kubernetes 1.11 (no version conversion)
CRD_OPTIONS ?= "crd:preserveUnknownFields=false"
@@ -40,8 +40,8 @@ test: generate manifests
go test ./... -coverprofile cover.out
# Build manager binary
manager: generate fmt vet
go build -o bin/manager main.go
manager: generate golint
go build -o bin/manager
# Run against the configured Kubernetes cluster in ~/.kube/config
run: generate manifests
@@ -145,23 +145,33 @@ docker-push:
CONTROLLER_GEN = $(shell pwd)/bin/controller-gen
controller-gen: ## Download controller-gen locally if necessary.
$(call go-get-tool,$(CONTROLLER_GEN),sigs.k8s.io/controller-tools/cmd/controller-gen@v0.5.0)
$(call go-install-tool,$(CONTROLLER_GEN),sigs.k8s.io/controller-tools/cmd/controller-gen@v0.5.0)
GINKGO = $(shell pwd)/bin/ginkgo
ginkgo: ## Download ginkgo locally if necessary.
$(call go-install-tool,$(KUSTOMIZE),github.com/onsi/ginkgo/ginkgo@v1.16.5)
KUSTOMIZE = $(shell pwd)/bin/kustomize
kustomize: ## Download kustomize locally if necessary.
$(call go-get-tool,$(KUSTOMIZE),sigs.k8s.io/kustomize/kustomize/v3@v3.8.7)
$(call install-kustomize,$(KUSTOMIZE),3.8.7)
# go-get-tool will 'go get' any package $2 and install it to $1.
PROJECT_DIR := $(shell dirname $(abspath $(lastword $(MAKEFILE_LIST))))
define go-get-tool
define install-kustomize
@[ -f $(1) ] || { \
set -e ;\
TMP_DIR=$$(mktemp -d) ;\
cd $$TMP_DIR ;\
go mod init tmp ;\
echo "Downloading $(2)" ;\
GOBIN=$(PROJECT_DIR)/bin go get $(2) ;\
rm -rf $$TMP_DIR ;\
echo "Installing v$(2)" ;\
cd bin ;\
wget "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" ;\
bash ./install_kustomize.sh $(2) ;\
}
endef
# go-install-tool will 'go install' any package $2 and install it to $1.
PROJECT_DIR := $(shell dirname $(abspath $(lastword $(MAKEFILE_LIST))))
define go-install-tool
@[ -f $(1) ] || { \
set -e ;\
echo "Installing $(2)" ;\
GOBIN=$(PROJECT_DIR)/bin go install $(2) ;\
}
endef
@@ -187,7 +197,7 @@ golint:
# Running e2e tests in a KinD instance
.PHONY: e2e
e2e/%:
e2e/%: ginkgo
kind create cluster --name capsule --image=kindest/node:$*
make docker-build
kind load docker-image --nodes capsule-control-plane --name capsule $(IMG)
@@ -203,5 +213,5 @@ e2e/%:
--set 'manager.readinessProbe.failureThreshold=10' \
capsule \
./charts/capsule
ginkgo -v -tags e2e ./e2e
$(GINKGO) -v -tags e2e ./e2e
kind delete cluster --name capsule

View File

@@ -5,6 +5,9 @@
<a href="https://github.com/clastix/capsule/releases">
<img src="https://img.shields.io/github/v/release/clastix/capsule"/>
</a>
<a href="https://charmhub.io/capsule-k8s">
<img src="https://charmhub.io/capsule-k8s/badge.svg"/>
</a>
</p>
<p align="center">
@@ -13,13 +16,15 @@
---
**Join the community** on the [#capsule](https://kubernetes.slack.com/archives/C03GETTJQRL) channel in the [Kubernetes Slack](https://slack.k8s.io/).
# Kubernetes multi-tenancy made easy
**Capsule** implements a multi-tenant and policy-based environment in your Kubernetes cluster. It is designed as a micro-services-based ecosystem with the minimalist approach, leveraging only on upstream Kubernetes.
# What's the problem with the current status?
Kubernetes introduces the _Namespace_ object type to create logical partitions of the cluster as isolated *slices*. However, implementing advanced multi-tenancy scenarios, it soon becomes complicated because of the flat structure of Kubernetes namespaces and the impossibility to share resources among namespaces belonging to the same tenant. To overcome this, cluster admins tend to provision a dedicated cluster for each groups of users, teams, or departments. As an organization grows, the number of clusters to manage and keep aligned becomes an operational nightmare, described as the well know phenomena of the _clusters sprawl_.
Kubernetes introduces the _Namespace_ object type to create logical partitions of the cluster as isolated *slices*. However, implementing advanced multi-tenancy scenarios, it soon becomes complicated because of the flat structure of Kubernetes namespaces and the impossibility to share resources among namespaces belonging to the same tenant. To overcome this, cluster admins tend to provision a dedicated cluster for each groups of users, teams, or departments. As an organization grows, the number of clusters to manage and keep aligned becomes an operational nightmare, described as the well known phenomena of the _clusters sprawl_.
# Entering Capsule
@@ -65,6 +70,32 @@ Please, check the project [documentation](https://capsule.clastix.io) for the co
Capsule is Open Source with Apache 2 license and any contribution is welcome.
## Chart Development
The documentation for each chart is done with [helm-docs](https://github.com/norwoodj/helm-docs). This way we can ensure that values are consistent with the chart documentation.
We have a script on the repository which will execute the helm-docs docker container, so that you don't have to worry about downloading the binary etc. Simply execute the script (Bash compatible):
```
bash scripts/helm-docs.sh
```
## Community
Join the community, share and learn from it. You can find all the resources to how to contribute code and docs, connect with people in the [community repository](https://github.com/clastix/capsule-community).
## Adopters
See the [ADOPTERS.md](ADOPTERS.md) file for a list of companies that are using Capsule.
# Governance
You can find how the Capsule project is governed [here](https://capsule.clastix.io/docs/contributing/governance).
## Maintainers
Please, refer to the maintainers file available [here](.github/maintainers.yaml).
# FAQ
- Q. How to pronounce Capsule?

View File

@@ -19,9 +19,12 @@ func (in *AllowedListSpec) ExactMatch(value string) (ok bool) {
sort.SliceStable(in.Exact, func(i, j int) bool {
return strings.ToLower(in.Exact[i]) < strings.ToLower(in.Exact[j])
})
i := sort.SearchStrings(in.Exact, value)
ok = i < len(in.Exact) && in.Exact[i] == value
}
return
}
@@ -29,5 +32,6 @@ func (in AllowedListSpec) RegexMatch(value string) (ok bool) {
if len(in.Regex) > 0 {
ok = regexp.MustCompile(in.Regex).MatchString(value)
}
return
}

View File

@@ -15,6 +15,7 @@ func TestAllowedListSpec_ExactMatch(t *testing.T) {
True []string
False []string
}
for _, tc := range []tc{
{
[]string{"foo", "bar", "bizz", "buzz"},
@@ -35,9 +36,11 @@ func TestAllowedListSpec_ExactMatch(t *testing.T) {
a := AllowedListSpec{
Exact: tc.In,
}
for _, ok := range tc.True {
assert.True(t, a.ExactMatch(ok))
}
for _, ko := range tc.False {
assert.False(t, a.ExactMatch(ko))
}
@@ -50,6 +53,7 @@ func TestAllowedListSpec_RegexMatch(t *testing.T) {
True []string
False []string
}
for _, tc := range []tc{
{`first-\w+-pattern`, []string{"first-date-pattern", "first-year-pattern"}, []string{"broken", "first-year", "second-date-pattern"}},
{``, nil, []string{"any", "value"}},
@@ -57,9 +61,11 @@ func TestAllowedListSpec_RegexMatch(t *testing.T) {
a := AllowedListSpec{
Regex: tc.Regex,
}
for _, ok := range tc.True {
assert.True(t, a.RegexMatch(ok))
}
for _, ko := range tc.False {
assert.False(t, a.RegexMatch(ko))
}

View File

@@ -5,8 +5,8 @@ const (
ForbiddenNodeLabelsRegexpAnnotation = "capsule.clastix.io/forbidden-node-labels-regexp"
ForbiddenNodeAnnotationsAnnotation = "capsule.clastix.io/forbidden-node-annotations"
ForbiddenNodeAnnotationsRegexpAnnotation = "capsule.clastix.io/forbidden-node-annotations-regexp"
CASecretNameAnnotation = "capsule.clastix.io/ca-secret-name"
TLSSecretNameAnnotation = "capsule.clastix.io/tls-secret-name"
MutatingWebhookConfigurationName = "capsule.clastix.io/mutating-webhook-configuration-name"
ValidatingWebhookConfigurationName = "capsule.clastix.io/validating-webhook-configuration-name"
EnableTLSConfigurationAnnotationName = "capsule.clastix.io/enable-tls-configuration"
)

View File

@@ -7,7 +7,7 @@ import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
// CapsuleConfigurationSpec defines the Capsule configuration
// CapsuleConfigurationSpec defines the Capsule configuration.
type CapsuleConfigurationSpec struct {
// Names of the groups for Capsule users.
// +kubebuilder:default={capsule.clastix.io}
@@ -23,7 +23,7 @@ type CapsuleConfigurationSpec struct {
// +kubebuilder:object:root=true
// +kubebuilder:resource:scope=Cluster
// CapsuleConfiguration is the Schema for the Capsule configuration API
// CapsuleConfiguration is the Schema for the Capsule configuration API.
type CapsuleConfiguration struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty"`
@@ -33,7 +33,7 @@ type CapsuleConfiguration struct {
// +kubebuilder:object:root=true
// CapsuleConfigurationList contains a list of CapsuleConfiguration
// CapsuleConfigurationList contains a list of CapsuleConfiguration.
type CapsuleConfigurationList struct {
metav1.TypeMeta `json:",inline"`
metav1.ListMeta `json:"metadata,omitempty"`

View File

@@ -49,13 +49,13 @@ const (
)
func (t *Tenant) convertV1Alpha1OwnerToV1Beta1() capsulev1beta1.OwnerListSpec {
var serviceKindToAnnotationMap = map[capsulev1beta1.ProxyServiceKind][]string{
serviceKindToAnnotationMap := map[capsulev1beta1.ProxyServiceKind][]string{
capsulev1beta1.NodesProxy: {enableNodeListingAnnotation, enableNodeUpdateAnnotation, enableNodeDeletionAnnotation},
capsulev1beta1.StorageClassesProxy: {enableStorageClassListingAnnotation, enableStorageClassUpdateAnnotation, enableStorageClassDeletionAnnotation},
capsulev1beta1.IngressClassesProxy: {enableIngressClassListingAnnotation, enableIngressClassUpdateAnnotation, enableIngressClassDeletionAnnotation},
capsulev1beta1.PriorityClassesProxy: {enablePriorityClassListingAnnotation, enablePriorityClassUpdateAnnotation, enablePriorityClassDeletionAnnotation},
}
var annotationToOperationMap = map[string]capsulev1beta1.ProxyOperation{
annotationToOperationMap := map[string]capsulev1beta1.ProxyOperation{
enableNodeListingAnnotation: capsulev1beta1.ListOperation,
enableNodeUpdateAnnotation: capsulev1beta1.UpdateOperation,
enableNodeDeletionAnnotation: capsulev1beta1.DeleteOperation,
@@ -69,14 +69,15 @@ func (t *Tenant) convertV1Alpha1OwnerToV1Beta1() capsulev1beta1.OwnerListSpec {
enablePriorityClassUpdateAnnotation: capsulev1beta1.UpdateOperation,
enablePriorityClassDeletionAnnotation: capsulev1beta1.DeleteOperation,
}
var annotationToOwnerKindMap = map[string]capsulev1beta1.OwnerKind{
annotationToOwnerKindMap := map[string]capsulev1beta1.OwnerKind{
ownerUsersAnnotation: capsulev1beta1.UserOwner,
ownerGroupsAnnotation: capsulev1beta1.GroupOwner,
ownerServiceAccountAnnotation: capsulev1beta1.ServiceAccountOwner,
}
annotations := t.GetAnnotations()
var operations = make(map[string]map[capsulev1beta1.ProxyServiceKind][]capsulev1beta1.ProxyOperation)
operations := make(map[string]map[capsulev1beta1.ProxyServiceKind][]capsulev1beta1.ProxyOperation)
for serviceKind, operationAnnotations := range serviceKindToAnnotationMap {
for _, operationAnnotation := range operationAnnotations {
@@ -86,6 +87,7 @@ func (t *Tenant) convertV1Alpha1OwnerToV1Beta1() capsulev1beta1.OwnerListSpec {
if _, exists := operations[owner]; !exists {
operations[owner] = make(map[capsulev1beta1.ProxyServiceKind][]capsulev1beta1.ProxyOperation)
}
operations[owner][serviceKind] = append(operations[owner][serviceKind], annotationToOperationMap[operationAnnotation])
}
}
@@ -94,7 +96,7 @@ func (t *Tenant) convertV1Alpha1OwnerToV1Beta1() capsulev1beta1.OwnerListSpec {
var owners capsulev1beta1.OwnerListSpec
var getProxySettingsForOwner = func(ownerName string) (settings []capsulev1beta1.ProxySettings) {
getProxySettingsForOwner := func(ownerName string) (settings []capsulev1beta1.ProxySettings) {
ownerOperations, ok := operations[ownerName]
if ok {
for k, v := range ownerOperations {
@@ -104,6 +106,7 @@ func (t *Tenant) convertV1Alpha1OwnerToV1Beta1() capsulev1beta1.OwnerListSpec {
})
}
}
return
}
@@ -129,8 +132,13 @@ func (t *Tenant) convertV1Alpha1OwnerToV1Beta1() capsulev1beta1.OwnerListSpec {
return owners
}
// nolint:gocognit,gocyclo,cyclop,maintidx
func (t *Tenant) ConvertTo(dstRaw conversion.Hub) error {
dst := dstRaw.(*capsulev1beta1.Tenant)
dst, ok := dstRaw.(*capsulev1beta1.Tenant)
if !ok {
return fmt.Errorf("expected type *capsulev1beta1.Tenant, got %T", dst)
}
annotations := t.GetAnnotations()
// ObjectMeta
@@ -141,6 +149,7 @@ func (t *Tenant) ConvertTo(dstRaw conversion.Hub) error {
if dst.Spec.NamespaceOptions == nil {
dst.Spec.NamespaceOptions = &capsulev1beta1.NamespaceOptions{}
}
dst.Spec.NamespaceOptions.Quota = t.Spec.NamespaceQuota
}
@@ -152,11 +161,13 @@ func (t *Tenant) ConvertTo(dstRaw conversion.Hub) error {
if dst.Spec.NamespaceOptions == nil {
dst.Spec.NamespaceOptions = &capsulev1beta1.NamespaceOptions{}
}
dst.Spec.NamespaceOptions.AdditionalMetadata = &capsulev1beta1.AdditionalMetadataSpec{
Labels: t.Spec.NamespacesMetadata.AdditionalLabels,
Annotations: t.Spec.NamespacesMetadata.AdditionalAnnotations,
}
}
if t.Spec.ServicesMetadata != nil {
if dst.Spec.ServiceOptions == nil {
dst.Spec.ServiceOptions = &capsulev1beta1.ServiceOptions{
@@ -167,13 +178,15 @@ func (t *Tenant) ConvertTo(dstRaw conversion.Hub) error {
}
}
}
if t.Spec.StorageClasses != nil {
dst.Spec.StorageClasses = &capsulev1beta1.AllowedListSpec{
Exact: t.Spec.StorageClasses.Exact,
Regex: t.Spec.StorageClasses.Regex,
}
}
if v, ok := t.Annotations[ingressHostnameCollisionScope]; ok {
if v, annotationOk := t.Annotations[ingressHostnameCollisionScope]; annotationOk {
switch v {
case string(capsulev1beta1.HostnameCollisionScopeCluster), string(capsulev1beta1.HostnameCollisionScopeTenant), string(capsulev1beta1.HostnameCollisionScopeNamespace):
dst.Spec.IngressOptions.HostnameCollisionScope = capsulev1beta1.HostnameCollisionScope(v)
@@ -181,38 +194,44 @@ func (t *Tenant) ConvertTo(dstRaw conversion.Hub) error {
dst.Spec.IngressOptions.HostnameCollisionScope = capsulev1beta1.HostnameCollisionScopeDisabled
}
}
if t.Spec.IngressClasses != nil {
dst.Spec.IngressOptions.AllowedClasses = &capsulev1beta1.AllowedListSpec{
Exact: t.Spec.IngressClasses.Exact,
Regex: t.Spec.IngressClasses.Regex,
}
}
if t.Spec.IngressHostnames != nil {
dst.Spec.IngressOptions.AllowedHostnames = &capsulev1beta1.AllowedListSpec{
Exact: t.Spec.IngressHostnames.Exact,
Regex: t.Spec.IngressHostnames.Regex,
}
}
if t.Spec.ContainerRegistries != nil {
dst.Spec.ContainerRegistries = &capsulev1beta1.AllowedListSpec{
Exact: t.Spec.ContainerRegistries.Exact,
Regex: t.Spec.ContainerRegistries.Regex,
}
}
if len(t.Spec.NetworkPolicies) > 0 {
dst.Spec.NetworkPolicies = capsulev1beta1.NetworkPolicySpec{
Items: t.Spec.NetworkPolicies,
}
}
if len(t.Spec.LimitRanges) > 0 {
dst.Spec.LimitRanges = capsulev1beta1.LimitRangesSpec{
Items: t.Spec.LimitRanges,
}
}
if len(t.Spec.ResourceQuota) > 0 {
dst.Spec.ResourceQuota = capsulev1beta1.ResourceQuotaSpec{
Scope: func() capsulev1beta1.ResourceQuotaScope {
if v, ok := t.GetAnnotations()[resourceQuotaScopeAnnotation]; ok {
if v, annotationOk := t.GetAnnotations()[resourceQuotaScopeAnnotation]; annotationOk {
switch v {
case string(capsulev1beta1.ResourceQuotaScopeNamespace):
return capsulev1beta1.ResourceQuotaScopeNamespace
@@ -220,11 +239,13 @@ func (t *Tenant) ConvertTo(dstRaw conversion.Hub) error {
return capsulev1beta1.ResourceQuotaScopeTenant
}
}
return capsulev1beta1.ResourceQuotaScopeTenant
}(),
Items: t.Spec.ResourceQuota,
}
}
if len(t.Spec.AdditionalRoleBindings) > 0 {
for _, rb := range t.Spec.AdditionalRoleBindings {
dst.Spec.AdditionalRoleBindings = append(dst.Spec.AdditionalRoleBindings, capsulev1beta1.AdditionalRoleBindingsSpec{
@@ -233,10 +254,12 @@ func (t *Tenant) ConvertTo(dstRaw conversion.Hub) error {
})
}
}
if t.Spec.ExternalServiceIPs != nil {
if dst.Spec.ServiceOptions == nil {
dst.Spec.ServiceOptions = &capsulev1beta1.ServiceOptions{}
}
dst.Spec.ServiceOptions.ExternalServiceIPs = &capsulev1beta1.ExternalServiceIPsSpec{
Allowed: make([]capsulev1beta1.AllowedIP, len(t.Spec.ExternalServiceIPs.Allowed)),
}
@@ -256,10 +279,13 @@ func (t *Tenant) ConvertTo(dstRaw conversion.Hub) error {
priorityClasses := capsulev1beta1.AllowedListSpec{}
priorityClassAllowed, ok := annotations[podPriorityAllowedAnnotation]
if ok {
priorityClasses.Exact = strings.Split(priorityClassAllowed, ",")
}
priorityClassesRegexp, ok := annotations[podPriorityAllowedRegexAnnotation]
if ok {
priorityClasses.Regex = priorityClassesRegexp
}
@@ -274,12 +300,15 @@ func (t *Tenant) ConvertTo(dstRaw conversion.Hub) error {
if err != nil {
return errors.Wrap(err, fmt.Sprintf("unable to parse %s annotation on tenant %s", enableNodePortsAnnotation, t.GetName()))
}
if dst.Spec.ServiceOptions == nil {
dst.Spec.ServiceOptions = &capsulev1beta1.ServiceOptions{}
}
if dst.Spec.ServiceOptions.AllowedServices == nil {
dst.Spec.ServiceOptions.AllowedServices = &capsulev1beta1.AllowedServices{}
}
dst.Spec.ServiceOptions.AllowedServices.NodePort = pointer.BoolPtr(val)
}
@@ -289,12 +318,15 @@ func (t *Tenant) ConvertTo(dstRaw conversion.Hub) error {
if err != nil {
return errors.Wrap(err, fmt.Sprintf("unable to parse %s annotation on tenant %s", enableExternalNameAnnotation, t.GetName()))
}
if dst.Spec.ServiceOptions == nil {
dst.Spec.ServiceOptions = &capsulev1beta1.ServiceOptions{}
}
if dst.Spec.ServiceOptions.AllowedServices == nil {
dst.Spec.ServiceOptions.AllowedServices = &capsulev1beta1.AllowedServices{}
}
dst.Spec.ServiceOptions.AllowedServices.ExternalName = pointer.BoolPtr(val)
}
@@ -304,21 +336,22 @@ func (t *Tenant) ConvertTo(dstRaw conversion.Hub) error {
if err != nil {
return errors.Wrap(err, fmt.Sprintf("unable to parse %s annotation on tenant %s", enableLoadBalancerAnnotation, t.GetName()))
}
if dst.Spec.ServiceOptions == nil {
dst.Spec.ServiceOptions = &capsulev1beta1.ServiceOptions{}
}
if dst.Spec.ServiceOptions.AllowedServices == nil {
dst.Spec.ServiceOptions.AllowedServices = &capsulev1beta1.AllowedServices{}
}
dst.Spec.ServiceOptions.AllowedServices.LoadBalancer = pointer.BoolPtr(val)
}
// Status
dst.Status = capsulev1beta1.TenantStatus{
Size: t.Status.Size,
Namespaces: t.Status.Namespaces,
}
// Remove unneeded annotations
delete(dst.ObjectMeta.Annotations, podAllowedImagePullPolicyAnnotation)
delete(dst.ObjectMeta.Annotations, podPriorityAllowedAnnotation)
@@ -347,14 +380,15 @@ func (t *Tenant) ConvertTo(dstRaw conversion.Hub) error {
return nil
}
// nolint:gocognit,gocyclo,cyclop
func (t *Tenant) convertV1Beta1OwnerToV1Alpha1(src *capsulev1beta1.Tenant) {
var ownersAnnotations = map[string][]string{
ownersAnnotations := map[string][]string{
ownerGroupsAnnotation: nil,
ownerUsersAnnotation: nil,
ownerServiceAccountAnnotation: nil,
}
var proxyAnnotations = map[string][]string{
proxyAnnotations := map[string][]string{
enableNodeListingAnnotation: nil,
enableNodeUpdateAnnotation: nil,
enableNodeDeletionAnnotation: nil,
@@ -382,6 +416,7 @@ func (t *Tenant) convertV1Beta1OwnerToV1Alpha1(src *capsulev1beta1.Tenant) {
ownersAnnotations[ownerServiceAccountAnnotation] = append(ownersAnnotations[ownerServiceAccountAnnotation], owner.Name)
}
}
for _, setting := range owner.ProxyOperations {
switch setting.Kind {
case capsulev1beta1.NodesProxy:
@@ -437,6 +472,7 @@ func (t *Tenant) convertV1Beta1OwnerToV1Alpha1(src *capsulev1beta1.Tenant) {
t.Annotations[k] = strings.Join(v, ",")
}
}
for k, v := range proxyAnnotations {
if len(v) > 0 {
t.Annotations[k] = strings.Join(v, ",")
@@ -444,8 +480,12 @@ func (t *Tenant) convertV1Beta1OwnerToV1Alpha1(src *capsulev1beta1.Tenant) {
}
}
// nolint:gocyclo,cyclop
func (t *Tenant) ConvertFrom(srcRaw conversion.Hub) error {
src := srcRaw.(*capsulev1beta1.Tenant)
src, ok := srcRaw.(*capsulev1beta1.Tenant)
if !ok {
return fmt.Errorf("expected *capsulev1beta1.Tenant, got %T", srcRaw)
}
// ObjectMeta
t.ObjectMeta = src.ObjectMeta
@@ -469,47 +509,57 @@ func (t *Tenant) ConvertFrom(srcRaw conversion.Hub) error {
AdditionalAnnotations: src.Spec.NamespaceOptions.AdditionalMetadata.Annotations,
}
}
if src.Spec.ServiceOptions != nil && src.Spec.ServiceOptions.AdditionalMetadata != nil {
t.Spec.ServicesMetadata = &AdditionalMetadataSpec{
AdditionalLabels: src.Spec.ServiceOptions.AdditionalMetadata.Labels,
AdditionalAnnotations: src.Spec.ServiceOptions.AdditionalMetadata.Annotations,
}
}
if src.Spec.StorageClasses != nil {
t.Spec.StorageClasses = &AllowedListSpec{
Exact: src.Spec.StorageClasses.Exact,
Regex: src.Spec.StorageClasses.Regex,
}
}
t.Annotations[ingressHostnameCollisionScope] = string(src.Spec.IngressOptions.HostnameCollisionScope)
if src.Spec.IngressOptions.AllowedClasses != nil {
t.Spec.IngressClasses = &AllowedListSpec{
Exact: src.Spec.IngressOptions.AllowedClasses.Exact,
Regex: src.Spec.IngressOptions.AllowedClasses.Regex,
}
}
if src.Spec.IngressOptions.AllowedHostnames != nil {
t.Spec.IngressHostnames = &AllowedListSpec{
Exact: src.Spec.IngressOptions.AllowedHostnames.Exact,
Regex: src.Spec.IngressOptions.AllowedHostnames.Regex,
}
}
if src.Spec.ContainerRegistries != nil {
t.Spec.ContainerRegistries = &AllowedListSpec{
Exact: src.Spec.ContainerRegistries.Exact,
Regex: src.Spec.ContainerRegistries.Regex,
}
}
if len(src.Spec.NetworkPolicies.Items) > 0 {
t.Spec.NetworkPolicies = src.Spec.NetworkPolicies.Items
}
if len(src.Spec.LimitRanges.Items) > 0 {
t.Spec.LimitRanges = src.Spec.LimitRanges.Items
}
if len(src.Spec.ResourceQuota.Items) > 0 {
t.Annotations[resourceQuotaScopeAnnotation] = string(src.Spec.ResourceQuota.Scope)
t.Spec.ResourceQuota = src.Spec.ResourceQuota.Items
}
if len(src.Spec.AdditionalRoleBindings) > 0 {
for _, rb := range src.Spec.AdditionalRoleBindings {
t.Spec.AdditionalRoleBindings = append(t.Spec.AdditionalRoleBindings, AdditionalRoleBindingsSpec{
@@ -518,6 +568,7 @@ func (t *Tenant) ConvertFrom(srcRaw conversion.Hub) error {
})
}
}
if src.Spec.ServiceOptions != nil && src.Spec.ServiceOptions.ExternalServiceIPs != nil {
t.Spec.ExternalServiceIPs = &ExternalServiceIPsSpec{
Allowed: make([]AllowedIP, len(src.Spec.ServiceOptions.ExternalServiceIPs.Allowed)),
@@ -527,11 +578,14 @@ func (t *Tenant) ConvertFrom(srcRaw conversion.Hub) error {
t.Spec.ExternalServiceIPs.Allowed[i] = AllowedIP(IP)
}
}
if len(src.Spec.ImagePullPolicies) != 0 {
var pullPolicies []string
for _, policy := range src.Spec.ImagePullPolicies {
pullPolicies = append(pullPolicies, string(policy))
}
t.Annotations[podAllowedImagePullPolicyAnnotation] = strings.Join(pullPolicies, ",")
}
@@ -539,6 +593,7 @@ func (t *Tenant) ConvertFrom(srcRaw conversion.Hub) error {
if len(src.Spec.PriorityClasses.Exact) != 0 {
t.Annotations[podPriorityAllowedAnnotation] = strings.Join(src.Spec.PriorityClasses.Exact, ",")
}
if src.Spec.PriorityClasses.Regex != "" {
t.Annotations[podPriorityAllowedRegexAnnotation] = src.Spec.PriorityClasses.Regex
}
@@ -548,9 +603,11 @@ func (t *Tenant) ConvertFrom(srcRaw conversion.Hub) error {
if src.Spec.ServiceOptions.AllowedServices.NodePort != nil {
t.Annotations[enableNodePortsAnnotation] = strconv.FormatBool(*src.Spec.ServiceOptions.AllowedServices.NodePort)
}
if src.Spec.ServiceOptions.AllowedServices.ExternalName != nil {
t.Annotations[enableExternalNameAnnotation] = strconv.FormatBool(*src.Spec.ServiceOptions.AllowedServices.ExternalName)
}
if src.Spec.ServiceOptions.AllowedServices.LoadBalancer != nil {
t.Annotations[enableLoadBalancerAnnotation] = strconv.FormatBool(*src.Spec.ServiceOptions.AllowedServices.LoadBalancer)
}

View File

@@ -18,12 +18,14 @@ import (
capsulev1beta1 "github.com/clastix/capsule/api/v1beta1"
)
// nolint:maintidx
func generateTenantsSpecs() (Tenant, capsulev1beta1.Tenant) {
var namespaceQuota int32 = 5
var nodeSelector = map[string]string{
nodeSelector := map[string]string{
"foo": "bar",
}
var v1alpha1AdditionalMetadataSpec = &AdditionalMetadataSpec{
v1alpha1AdditionalMetadataSpec := &AdditionalMetadataSpec{
AdditionalLabels: map[string]string{
"foo": "bar",
},
@@ -31,11 +33,11 @@ func generateTenantsSpecs() (Tenant, capsulev1beta1.Tenant) {
"foo": "bar",
},
}
var v1alpha1AllowedListSpec = &AllowedListSpec{
v1alpha1AllowedListSpec := &AllowedListSpec{
Exact: []string{"foo", "bar"},
Regex: "^foo*",
}
var v1beta1AdditionalMetadataSpec = &capsulev1beta1.AdditionalMetadataSpec{
v1beta1AdditionalMetadataSpec := &capsulev1beta1.AdditionalMetadataSpec{
Labels: map[string]string{
"foo": "bar",
},
@@ -43,11 +45,11 @@ func generateTenantsSpecs() (Tenant, capsulev1beta1.Tenant) {
"foo": "bar",
},
}
var v1beta1NamespaceOptions = &capsulev1beta1.NamespaceOptions{
v1beta1NamespaceOptions := &capsulev1beta1.NamespaceOptions{
Quota: &namespaceQuota,
AdditionalMetadata: v1beta1AdditionalMetadataSpec,
}
var v1beta1ServiceOptions = &capsulev1beta1.ServiceOptions{
v1beta1ServiceOptions := &capsulev1beta1.ServiceOptions{
AdditionalMetadata: v1beta1AdditionalMetadataSpec,
AllowedServices: &capsulev1beta1.AllowedServices{
NodePort: pointer.BoolPtr(false),
@@ -58,11 +60,11 @@ func generateTenantsSpecs() (Tenant, capsulev1beta1.Tenant) {
Allowed: []capsulev1beta1.AllowedIP{"192.168.0.1"},
},
}
var v1beta1AllowedListSpec = &capsulev1beta1.AllowedListSpec{
v1beta1AllowedListSpec := &capsulev1beta1.AllowedListSpec{
Exact: []string{"foo", "bar"},
Regex: "^foo*",
}
var networkPolicies = []networkingv1.NetworkPolicySpec{
networkPolicies := []networkingv1.NetworkPolicySpec{
{
Ingress: []networkingv1.NetworkPolicyIngressRule{
{
@@ -87,7 +89,7 @@ func generateTenantsSpecs() (Tenant, capsulev1beta1.Tenant) {
},
},
}
var limitRanges = []corev1.LimitRangeSpec{
limitRanges := []corev1.LimitRangeSpec{
{
Limits: []corev1.LimitRangeItem{
{
@@ -104,7 +106,7 @@ func generateTenantsSpecs() (Tenant, capsulev1beta1.Tenant) {
},
},
}
var resourceQuotas = []corev1.ResourceQuotaSpec{
resourceQuotas := []corev1.ResourceQuotaSpec{
{
Hard: map[corev1.ResourceName]resource.Quantity{
corev1.ResourceLimitsCPU: resource.MustParse("8"),
@@ -118,7 +120,7 @@ func generateTenantsSpecs() (Tenant, capsulev1beta1.Tenant) {
},
}
var v1beta1Tnt = capsulev1beta1.Tenant{
v1beta1Tnt := capsulev1beta1.Tenant{
TypeMeta: metav1.TypeMeta{},
ObjectMeta: metav1.ObjectMeta{
Name: "alice",
@@ -256,7 +258,7 @@ func generateTenantsSpecs() (Tenant, capsulev1beta1.Tenant) {
Subjects: []rbacv1.Subject{
{
Kind: "Group",
APIGroup: "rbac.authorization.k8s.io",
APIGroup: rbacv1.GroupName,
Name: "system:authenticated",
},
},
@@ -274,7 +276,7 @@ func generateTenantsSpecs() (Tenant, capsulev1beta1.Tenant) {
},
}
var v1alpha1Tnt = Tenant{
v1alpha1Tnt := Tenant{
TypeMeta: metav1.TypeMeta{},
ObjectMeta: metav1.ObjectMeta{
Name: "alice",
@@ -327,7 +329,7 @@ func generateTenantsSpecs() (Tenant, capsulev1beta1.Tenant) {
Subjects: []rbacv1.Subject{
{
Kind: "Group",
APIGroup: "rbac.authorization.k8s.io",
APIGroup: rbacv1.GroupName,
Name: "system:authenticated",
},
},
@@ -347,10 +349,11 @@ func generateTenantsSpecs() (Tenant, capsulev1beta1.Tenant) {
}
func TestConversionHub_ConvertTo(t *testing.T) {
var v1beta1ConvertedTnt = capsulev1beta1.Tenant{}
v1beta1ConvertedTnt := capsulev1beta1.Tenant{}
v1alpha1Tnt, v1beta1tnt := generateTenantsSpecs()
err := v1alpha1Tnt.ConvertTo(&v1beta1ConvertedTnt)
if assert.NoError(t, err) {
sort.Slice(v1beta1tnt.Spec.Owners, func(i, j int) bool {
return v1beta1tnt.Spec.Owners[i].Name < v1beta1tnt.Spec.Owners[j].Name
@@ -364,17 +367,20 @@ func TestConversionHub_ConvertTo(t *testing.T) {
return owner.ProxyOperations[i].Kind < owner.ProxyOperations[j].Kind
})
}
for _, owner := range v1beta1ConvertedTnt.Spec.Owners {
sort.Slice(owner.ProxyOperations, func(i, j int) bool {
return owner.ProxyOperations[i].Kind < owner.ProxyOperations[j].Kind
})
}
assert.Equal(t, v1beta1tnt, v1beta1ConvertedTnt)
}
}
func TestConversionHub_ConvertFrom(t *testing.T) {
var v1alpha1ConvertedTnt = Tenant{}
v1alpha1ConvertedTnt := Tenant{}
v1alpha1Tnt, v1beta1tnt := generateTenantsSpecs()
err := v1alpha1ConvertedTnt.ConvertFrom(&v1beta1tnt)

View File

@@ -12,10 +12,10 @@ import (
)
var (
// GroupVersion is group version used to register these objects
// GroupVersion is group version used to register these objects.
GroupVersion = schema.GroupVersion{Group: "capsule.clastix.io", Version: "v1alpha1"}
// SchemeBuilder is used to add go types to the GroupVersionKind scheme
// SchemeBuilder is used to add go types to the GroupVersionKind scheme.
SchemeBuilder = &scheme.Builder{GroupVersion: GroupVersion}
// AddToScheme adds the types in this group-version to the given scheme.

View File

@@ -3,7 +3,7 @@
package v1alpha1
// OwnerSpec defines tenant owner name and kind
// OwnerSpec defines tenant owner name and kind.
type OwnerSpec struct {
Name string `json:"name"`
Kind Kind `json:"kind"`

View File

@@ -13,6 +13,7 @@ func (t *Tenant) IsCordoned() bool {
if v, ok := t.Labels["capsule.clastix.io/cordon"]; ok && v == "enabled" {
return true
}
return false
}
@@ -21,16 +22,19 @@ func (t *Tenant) IsFull() bool {
if t.Spec.NamespaceQuota == nil {
return false
}
return len(t.Status.Namespaces) >= int(*t.Spec.NamespaceQuota)
}
func (t *Tenant) AssignNamespaces(namespaces []corev1.Namespace) {
var l []string
for _, ns := range namespaces {
if ns.Status.Phase == corev1.NamespaceActive {
l = append(l, ns.GetName())
}
}
sort.Strings(l)
t.Status.Namespaces = l

View File

@@ -27,5 +27,6 @@ func GetTypeLabel(t runtime.Object) (label string, err error) {
default:
err = fmt.Errorf("type %T is not mapped as Capsule label recognized", v)
}
return
}

View File

@@ -9,7 +9,7 @@ import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
// TenantSpec defines the desired state of Tenant
// TenantSpec defines the desired state of Tenant.
type TenantSpec struct {
Owner OwnerSpec `json:"owner"`
@@ -29,7 +29,7 @@ type TenantSpec struct {
ExternalServiceIPs *ExternalServiceIPsSpec `json:"externalServiceIPs,omitempty"`
}
// TenantStatus defines the observed state of Tenant
// TenantStatus defines the observed state of Tenant.
type TenantStatus struct {
Size uint `json:"size"`
Namespaces []string `json:"namespaces,omitempty"`
@@ -45,7 +45,7 @@ type TenantStatus struct {
// +kubebuilder:printcolumn:name="Node selector",type="string",JSONPath=".spec.nodeSelector",description="Node Selector applied to Pods"
// +kubebuilder:printcolumn:name="Age",type="date",JSONPath=".metadata.creationTimestamp",description="Age"
// Tenant is the Schema for the tenants API
// Tenant is the Schema for the tenants API.
type Tenant struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty"`
@@ -56,7 +56,7 @@ type Tenant struct {
// +kubebuilder:object:root=true
// TenantList contains a list of Tenant
// TenantList contains a list of Tenant.
type TenantList struct {
metav1.TypeMeta `json:",inline"`
metav1.ListMeta `json:"metadata,omitempty"`

View File

@@ -19,9 +19,12 @@ func (in *AllowedListSpec) ExactMatch(value string) (ok bool) {
sort.SliceStable(in.Exact, func(i, j int) bool {
return strings.ToLower(in.Exact[i]) < strings.ToLower(in.Exact[j])
})
i := sort.SearchStrings(in.Exact, value)
ok = i < len(in.Exact) && in.Exact[i] == value
}
return
}
@@ -29,5 +32,6 @@ func (in AllowedListSpec) RegexMatch(value string) (ok bool) {
if len(in.Regex) > 0 {
ok = regexp.MustCompile(in.Regex).MatchString(value)
}
return
}

View File

@@ -15,6 +15,7 @@ func TestAllowedListSpec_ExactMatch(t *testing.T) {
True []string
False []string
}
for _, tc := range []tc{
{
[]string{"foo", "bar", "bizz", "buzz"},
@@ -35,9 +36,11 @@ func TestAllowedListSpec_ExactMatch(t *testing.T) {
a := AllowedListSpec{
Exact: tc.In,
}
for _, ok := range tc.True {
assert.True(t, a.ExactMatch(ok))
}
for _, ko := range tc.False {
assert.False(t, a.ExactMatch(ko))
}
@@ -50,6 +53,7 @@ func TestAllowedListSpec_RegexMatch(t *testing.T) {
True []string
False []string
}
for _, tc := range []tc{
{`first-\w+-pattern`, []string{"first-date-pattern", "first-year-pattern"}, []string{"broken", "first-year", "second-date-pattern"}},
{``, nil, []string{"any", "value"}},
@@ -57,9 +61,11 @@ func TestAllowedListSpec_RegexMatch(t *testing.T) {
a := AllowedListSpec{
Regex: tc.Regex,
}
for _, ok := range tc.True {
assert.True(t, a.RegexMatch(ok))
}
for _, ko := range tc.False {
assert.False(t, a.RegexMatch(ko))
}

View File

@@ -32,10 +32,22 @@ func GetUsedResourceFromTenant(tenant Tenant, kindGroup string) (int64, error) {
return used, nil
}
type NonLimitedResourceError struct {
kindGroup string
}
func NewNonLimitedResourceError(kindGroup string) *NonLimitedResourceError {
return &NonLimitedResourceError{kindGroup: kindGroup}
}
func (n NonLimitedResourceError) Error() string {
return fmt.Sprintf("resource %s is not limited for the current tenant", n.kindGroup)
}
func GetLimitResourceFromTenant(tenant Tenant, kindGroup string) (int64, error) {
limitStr, ok := tenant.GetAnnotations()[LimitAnnotationForResource(kindGroup)]
if !ok {
return 0, fmt.Errorf("resource %s is not limited for the current tenant", kindGroup)
return 0, NewNonLimitedResourceError(kindGroup)
}
limit, err := strconv.ParseInt(limitStr, 10, 10)

View File

@@ -11,5 +11,6 @@ func (t *Tenant) IsWildcardDenied() bool {
if v, ok := t.Annotations[denyWildcard]; ok && v == "true" {
return true
}
return false
}

View File

@@ -19,9 +19,12 @@ func (in *ForbiddenListSpec) ExactMatch(value string) (ok bool) {
sort.SliceStable(in.Exact, func(i, j int) bool {
return strings.ToLower(in.Exact[i]) < strings.ToLower(in.Exact[j])
})
i := sort.SearchStrings(in.Exact, value)
ok = i < len(in.Exact) && in.Exact[i] == value
}
return
}
@@ -29,5 +32,6 @@ func (in ForbiddenListSpec) RegexMatch(value string) (ok bool) {
if len(in.Regex) > 0 {
ok = regexp.MustCompile(in.Regex).MatchString(value)
}
return
}

View File

@@ -15,6 +15,7 @@ func TestForbiddenListSpec_ExactMatch(t *testing.T) {
True []string
False []string
}
for _, tc := range []tc{
{
[]string{"foo", "bar", "bizz", "buzz"},
@@ -35,9 +36,11 @@ func TestForbiddenListSpec_ExactMatch(t *testing.T) {
a := ForbiddenListSpec{
Exact: tc.In,
}
for _, ok := range tc.True {
assert.True(t, a.ExactMatch(ok))
}
for _, ko := range tc.False {
assert.False(t, a.ExactMatch(ko))
}
@@ -50,6 +53,7 @@ func TestForbiddenListSpec_RegexMatch(t *testing.T) {
True []string
False []string
}
for _, tc := range []tc{
{`first-\w+-pattern`, []string{"first-date-pattern", "first-year-pattern"}, []string{"broken", "first-year", "second-date-pattern"}},
{``, nil, []string{"any", "value"}},
@@ -57,9 +61,11 @@ func TestForbiddenListSpec_RegexMatch(t *testing.T) {
a := ForbiddenListSpec{
Regex: tc.Regex,
}
for _, ok := range tc.True {
assert.True(t, a.RegexMatch(ok))
}
for _, ko := range tc.False {
assert.False(t, a.RegexMatch(ko))
}

View File

@@ -12,10 +12,10 @@ import (
)
var (
// GroupVersion is group version used to register these objects
// GroupVersion is group version used to register these objects.
GroupVersion = schema.GroupVersion{Group: "capsule.clastix.io", Version: "v1beta1"}
// SchemeBuilder is used to add go types to the GroupVersionKind scheme
// SchemeBuilder is used to add go types to the GroupVersionKind scheme.
SchemeBuilder = &scheme.Builder{GroupVersion: GroupVersion}
// AddToScheme adds the types in this group-version to the given scheme.

View File

@@ -14,9 +14,11 @@ func (t *Tenant) hasForbiddenNamespaceLabelsAnnotations() bool {
if _, ok := t.Annotations[ForbiddenNamespaceLabelsAnnotation]; ok {
return true
}
if _, ok := t.Annotations[ForbiddenNamespaceLabelsRegexpAnnotation]; ok {
return true
}
return false
}
@@ -24,9 +26,11 @@ func (t *Tenant) hasForbiddenNamespaceAnnotationsAnnotations() bool {
if _, ok := t.Annotations[ForbiddenNamespaceAnnotationsAnnotation]; ok {
return true
}
if _, ok := t.Annotations[ForbiddenNamespaceAnnotationsRegexpAnnotation]; ok {
return true
}
return false
}
@@ -34,6 +38,7 @@ func (t *Tenant) ForbiddenUserNamespaceLabels() *ForbiddenListSpec {
if !t.hasForbiddenNamespaceLabelsAnnotations() {
return nil
}
return &ForbiddenListSpec{
Exact: strings.Split(t.Annotations[ForbiddenNamespaceLabelsAnnotation], ","),
Regex: t.Annotations[ForbiddenNamespaceLabelsRegexpAnnotation],
@@ -44,6 +49,7 @@ func (t *Tenant) ForbiddenUserNamespaceAnnotations() *ForbiddenListSpec {
if !t.hasForbiddenNamespaceAnnotationsAnnotations() {
return nil
}
return &ForbiddenListSpec{
Exact: strings.Split(t.Annotations[ForbiddenNamespaceAnnotationsAnnotation], ","),
Regex: t.Annotations[ForbiddenNamespaceAnnotationsRegexpAnnotation],

View File

@@ -15,6 +15,7 @@ func (o OwnerListSpec) FindOwner(name string, kind OwnerKind) (owner OwnerSpec)
if i < len(o) && o[i].Kind == kind && o[i].Name == name {
return o[i]
}
return
}
@@ -23,12 +24,15 @@ type ByKindAndName OwnerListSpec
func (b ByKindAndName) Len() int {
return len(b)
}
func (b ByKindAndName) Less(i, j int) bool {
if b[i].Kind.String() != b[j].Kind.String() {
return b[i].Kind.String() < b[j].Kind.String()
}
return b[i].Name < b[j].Name
}
func (b ByKindAndName) Swap(i, j int) {
b[i], b[j] = b[j], b[i]
}

View File

@@ -7,7 +7,7 @@ import (
)
func TestOwnerListSpec_FindOwner(t *testing.T) {
var bla = OwnerSpec{
bla := OwnerSpec{
Kind: UserOwner,
Name: "bla",
ProxyOperations: []ProxySettings{
@@ -17,7 +17,7 @@ func TestOwnerListSpec_FindOwner(t *testing.T) {
},
},
}
var bar = OwnerSpec{
bar := OwnerSpec{
Kind: GroupOwner,
Name: "bar",
ProxyOperations: []ProxySettings{
@@ -27,7 +27,7 @@ func TestOwnerListSpec_FindOwner(t *testing.T) {
},
},
}
var baz = OwnerSpec{
baz := OwnerSpec{
Kind: UserOwner,
Name: "baz",
ProxyOperations: []ProxySettings{
@@ -37,7 +37,7 @@ func TestOwnerListSpec_FindOwner(t *testing.T) {
},
},
}
var fim = OwnerSpec{
fim := OwnerSpec{
Kind: ServiceAccountOwner,
Name: "fim",
ProxyOperations: []ProxySettings{
@@ -47,7 +47,7 @@ func TestOwnerListSpec_FindOwner(t *testing.T) {
},
},
}
var bom = OwnerSpec{
bom := OwnerSpec{
Kind: GroupOwner,
Name: "bom",
ProxyOperations: []ProxySettings{
@@ -61,7 +61,7 @@ func TestOwnerListSpec_FindOwner(t *testing.T) {
},
},
}
var qip = OwnerSpec{
qip := OwnerSpec{
Kind: ServiceAccountOwner,
Name: "qip",
ProxyOperations: []ProxySettings{
@@ -71,7 +71,7 @@ func TestOwnerListSpec_FindOwner(t *testing.T) {
},
},
}
var owners = OwnerListSpec{bom, qip, bla, bar, baz, fim}
owners := OwnerListSpec{bom, qip, bla, bar, baz, fim}
assert.Equal(t, owners.FindOwner("bom", GroupOwner), bom)
assert.Equal(t, owners.FindOwner("qip", ServiceAccountOwner), qip)

74
api/v1beta1/owner_role.go Normal file
View File

@@ -0,0 +1,74 @@
// Copyright 2020-2021 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package v1beta1
import (
"fmt"
"strings"
)
const (
ClusterRoleNamesAnnotation = "clusterrolenames.capsule.clastix.io"
)
// GetRoles read the annotation available in the Tenant specification and if it matches the pattern
// clusterrolenames.capsule.clastix.io/${KIND}.${NAME} returns the associated roles.
// Kubernetes annotations and labels must respect RFC 1123 about DNS names and this could be cumbersome in two cases:
// 1. identifying users based on their email address
// 2. the overall length of the annotation key that is exceeding 63 characters
// For emails, the symbol @ can be replaced with the placeholder __AT__.
// For the latter one, the index of the owner can be used to force the retrieval.
func (in OwnerSpec) GetRoles(tenant Tenant, index int) []string {
for key, value := range tenant.GetAnnotations() {
if !strings.HasPrefix(key, fmt.Sprintf("%s/", ClusterRoleNamesAnnotation)) {
continue
}
for symbol, replace := range in.convertMap() {
key = strings.ReplaceAll(key, symbol, replace)
}
nameBased := key == fmt.Sprintf("%s/%s.%s", ClusterRoleNamesAnnotation, strings.ToLower(in.Kind.String()), strings.ToLower(in.Name))
indexBased := key == fmt.Sprintf("%s/%d", ClusterRoleNamesAnnotation, index)
if nameBased || indexBased {
return strings.Split(value, ",")
}
}
roles := []string{"admin", "capsule-namespace-deleter"}
if tenant.Spec.GitOpsReady {
roles = append(roles, in.getGitOpsRoles(tenant)...)
}
return roles
}
func (in OwnerSpec) GetClusterRoles(tenant Tenant) []string {
if tenant.Spec.GitOpsReady {
return in.getGitOpsClusterRoles(tenant)
}
return []string{}
}
func (in OwnerSpec) getGitOpsClusterRoles(tenant Tenant) []string {
return []string{
"capsule-tenant-impersonator-" + tenant.Name + "-" + in.Name,
}
}
func (in OwnerSpec) getGitOpsRoles(tenant Tenant) []string {
return []string{
"cluster-admin",
}
}
func (in OwnerSpec) convertMap() map[string]string {
return map[string]string{
"__AT__": "@",
}
}

View File

@@ -19,6 +19,7 @@ const (
ForbiddenNamespaceLabelsRegexpAnnotation = "capsule.clastix.io/forbidden-namespace-labels-regexp"
ForbiddenNamespaceAnnotationsAnnotation = "capsule.clastix.io/forbidden-namespace-annotations"
ForbiddenNamespaceAnnotationsRegexpAnnotation = "capsule.clastix.io/forbidden-namespace-annotations-regexp"
ProtectedTenantAnnotation = "capsule.clastix.io/protected"
)
func UsedQuotaFor(resource fmt.Stringer) string {

View File

@@ -13,6 +13,7 @@ func (t *Tenant) IsCordoned() bool {
if v, ok := t.Labels["capsule.clastix.io/cordon"]; ok && v == "enabled" {
return true
}
return false
}
@@ -21,16 +22,19 @@ func (t *Tenant) IsFull() bool {
if t.Spec.NamespaceOptions == nil || t.Spec.NamespaceOptions.Quota == nil {
return false
}
return len(t.Status.Namespaces) >= int(*t.Spec.NamespaceOptions.Quota)
}
func (t *Tenant) AssignNamespaces(namespaces []corev1.Namespace) {
var l []string
for _, ns := range namespaces {
if ns.Status.Phase == corev1.NamespaceActive {
l = append(l, ns.GetName())
}
}
sort.Strings(l)
t.Status.Namespaces = l

View File

@@ -24,8 +24,11 @@ func GetTypeLabel(t runtime.Object) (label string, err error) {
return "capsule.clastix.io/resource-quota", nil
case *rbacv1.RoleBinding:
return "capsule.clastix.io/role-binding", nil
case *rbacv1.ClusterRoleBinding:
return "capsule.clastix.io/cluster-role-binding", nil
default:
err = fmt.Errorf("type %T is not mapped as Capsule label recognized", v)
}
return
}

View File

@@ -11,7 +11,7 @@ const (
TenantStateCordoned tenantState = "Cordoned"
)
// Returns the observed state of the Tenant
// Returns the observed state of the Tenant.
type TenantStatus struct {
//+kubebuilder:default=Active
// The operational state of the Tenant. Possible values are "Active", "Cordoned".

View File

@@ -7,7 +7,7 @@ import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
// TenantSpec defines the desired state of Tenant
// TenantSpec defines the desired state of Tenant.
type TenantSpec struct {
// Specifies the owners of the Tenant. Mandatory.
Owners OwnerListSpec `json:"owners"`
@@ -35,6 +35,8 @@ type TenantSpec struct {
ImagePullPolicies []ImagePullPolicySpec `json:"imagePullPolicies,omitempty"`
// Specifies the allowed priorityClasses assigned to the Tenant. Capsule assures that all Pods resources created in the Tenant can use only one of the allowed PriorityClasses. Optional.
PriorityClasses *AllowedListSpec `json:"priorityClasses,omitempty"`
// Configured RBAC for machine owners tailored for GitOps controllers.
GitOpsReady bool `json:"gitOpsReady,omitempty"`
}
//+kubebuilder:object:root=true
@@ -47,7 +49,7 @@ type TenantSpec struct {
// +kubebuilder:printcolumn:name="Node selector",type="string",JSONPath=".spec.nodeSelector",description="Node Selector applied to Pods"
// +kubebuilder:printcolumn:name="Age",type="date",JSONPath=".metadata.creationTimestamp",description="Age"
// Tenant is the Schema for the tenants API
// Tenant is the Schema for the tenants API.
type Tenant struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty"`
@@ -60,7 +62,7 @@ func (t *Tenant) Hub() {}
//+kubebuilder:object:root=true
// TenantList contains a list of Tenant
// TenantList contains a list of Tenant.
type TenantList struct {
metav1.TypeMeta `json:",inline"`
metav1.ListMeta `json:"metadata,omitempty"`

View File

@@ -269,6 +269,21 @@ func (in *NetworkPolicySpec) DeepCopy() *NetworkPolicySpec {
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *NonLimitedResourceError) DeepCopyInto(out *NonLimitedResourceError) {
*out = *in
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new NonLimitedResourceError.
func (in *NonLimitedResourceError) DeepCopy() *NonLimitedResourceError {
if in == nil {
return nil
}
out := new(NonLimitedResourceError)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in OwnerListSpec) DeepCopyInto(out *OwnerListSpec) {
{

View File

@@ -21,3 +21,4 @@
.idea/
*.tmproj
.vscode/
README.md.gotmpl

View File

@@ -21,8 +21,8 @@ sources:
# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
version: 0.1.8
version: 0.1.11
# This is the version number of the application being deployed.
# This version number should be incremented each time you make changes to the application.
appVersion: 0.1.1
appVersion: 0.1.2

9
charts/capsule/Makefile Normal file
View File

@@ -0,0 +1,9 @@
docs: HELMDOCS_VERSION := v1.8.1
docs: docker
@docker run --rm -v "$$(pwd):/helm-docs" -u $$(id -u) jnorwood/helm-docs:$(HELMDOCS_VERSION)
docker:
@hash docker 2>/dev/null || {\
echo "You need docker" &&\
exit 1;\
}

View File

@@ -54,48 +54,101 @@ The values in your overrides file `myvalues.yaml` will override their counterpar
If you only need to make minor customizations, you can specify them on the command line by using the `--set` option. For example:
$ helm install capsule capsule-helm-chart --set force_tenant_prefix=false -n capsule-system
$ helm install capsule capsule-helm-chart --set manager.options.forceTenantPrefix=false -n capsule-system
Here the values you can override:
Parameter | Description | Default
--- | --- | ---
`manager.hostNetwork` | Specifies if the container should be started in `hostNetwork` mode. | `false`
`manager.options.logLevel` | Set the log verbosity of the controller with a value from 1 to 10.| `4`
`manager.options.forceTenantPrefix` | Boolean, enforces the Tenant owner, during Namespace creation, to name it using the selected Tenant name as prefix, separated by a dash | `false`
`manager.options.capsuleUserGroups` | Override the Capsule user groups | `[capsule.clastix.io]`
`manager.options.protectedNamespaceRegex` | If specified, disallows creation of namespaces matching the passed regexp | `null`
`manager.image.repository` | Set the image repository of the controller. | `quay.io/clastix/capsule`
`manager.image.tag` | Overrides the image tag whose default is the chart. `appVersion` | `null`
`manager.image.pullPolicy` | Set the image pull policy. | `IfNotPresent`
`manager.livenessProbe` | Configure the liveness probe using Deployment probe spec | `GET :10080/healthz`
`manager.readinessProbe` | Configure the readiness probe using Deployment probe spec | `GET :10080/readyz`
`manager.resources.requests/cpu` | Set the CPU requests assigned to the controller. | `200m`
`manager.resources.requests/memory` | Set the memory requests assigned to the controller. | `128Mi`
`manager.resources.limits/cpu` | Set the CPU limits assigned to the controller. | `200m`
`manager.resources.limits/cpu` | Set the memory limits assigned to the controller. | `128Mi`
`mutatingWebhooksTimeoutSeconds` | Timeout in seconds for mutating webhooks. | `30`
`validatingWebhooksTimeoutSeconds` | Timeout in seconds for validating webhooks. | `30`
`webhooks` | Additional configuration for capsule webhooks. |
`imagePullSecrets` | Configuration for `imagePullSecrets` so that you can use a private images registry. | `[]`
`serviceAccount.create` | Specifies whether a service account should be created. | `true`
`serviceAccount.annotations` | Annotations to add to the service account. | `{}`
`serviceAccount.name` | The name of the service account to use. If not set and `serviceAccount.create=true`, a name is generated using the fullname template | `capsule`
`podAnnotations` | Annotations to add to the Capsule pod. | `{}`
`priorityClassName` | Set the priority class name of the Capsule pod. | `null`
`nodeSelector` | Set the node selector for the Capsule pod. | `{}`
`tolerations` | Set list of tolerations for the Capsule pod. | `[]`
`replicaCount` | Set the replica count for Capsule pod. | `1`
`affinity` | Set affinity rules for the Capsule pod. | `{}`
`podSecurityPolicy.enabled` | Specify if a Pod Security Policy must be created. | `false`
`serviceMonitor.enabled` | Specifies if a service monitor must be created. | `false`
`serviceMonitor.labels` | Additional labels which will be added to service monitor. | `{}`
`serviceMonitor.annotations` | Additional annotations which will be added to service monitor. | `{}`
`serviceMonitor.matchLabels` | Additional matchLabels which will be added to service monitor. | `{}`
`serviceMonitor.serviceAccount.name` | Specifies service account name for metrics scrape. | `capsule`
`serviceMonitor.serviceAccount.namespace` | Specifies service account namespace for metrics scrape. | `capsule-system`
`customLabels` | Additional labels which will be added to all resources created by Capsule helm chart . | `{}`
`customAnnotations` | Additional annotations which will be added to all resources created by Capsule helm chart . | `{}`
### General Parameters
| Key | Type | Default | Description |
|-----|------|---------|-------------|
| affinity | object | `{}` | Set affinity rules for the Capsule pod |
| certManager.generateCertificates | bool | `false` | Specifies whether capsule webhooks certificates should be generated using cert-manager |
| customAnnotations | object | `{}` | Additional annotations which will be added to all resources created by Capsule helm chart |
| customLabels | object | `{}` | Additional labels which will be added to all resources created by Capsule helm chart |
| jobs.image.pullPolicy | string | `"IfNotPresent"` | Set the image pull policy of the helm chart job |
| jobs.image.repository | string | `"quay.io/clastix/kubectl"` | Set the image repository of the helm chart job |
| jobs.image.tag | string | `""` | Set the image tag of the helm chart job |
| mutatingWebhooksTimeoutSeconds | int | `30` | Timeout in seconds for mutating webhooks |
| nodeSelector | object | `{}` | Set the node selector for the Capsule pod |
| podAnnotations | object | `{}` | Annotations to add to the capsule pod. |
| podSecurityPolicy.enabled | bool | `false` | Specify if a Pod Security Policy must be created |
| priorityClassName | string | `""` | Set the priority class name of the Capsule pod |
| replicaCount | int | `1` | Set the replica count for capsule pod |
| serviceAccount.annotations | object | `{}` | Annotations to add to the service account. |
| serviceAccount.create | bool | `true` | Specifies whether a service account should be created. |
| serviceAccount.name | string | `"capsule"` | The name of the service account to use. If not set and `serviceAccount.create=true`, a name is generated using the fullname template |
| tls.create | bool | `true` | When cert-manager is disabled, Capsule will generate the TLS certificate for webhook and CRDs conversion. |
| tls.enableController | bool | `true` | Start the Capsule controller that injects the CA into mutating and validating webhooks, and CRD as well. |
| tls.name | string | `""` | Override name of the Capsule TLS Secret name when externally managed. |
| tolerations | list | `[]` | Set list of tolerations for the Capsule pod |
| validatingWebhooksTimeoutSeconds | int | `30` | Timeout in seconds for validating webhooks |
### Manager Parameters
| Key | Type | Default | Description |
|-----|------|---------|-------------|
| manager.hostNetwork | bool | `false` | Specifies if the container should be started in hostNetwork mode. Required for use in some managed kubernetes clusters (such as AWS EKS) with custom CNI (such as calico), because control-plane managed by AWS cannot communicate with pods' IP CIDR and admission webhooks are not working |
| manager.image.pullPolicy | string | `"IfNotPresent"` | Set the image pull policy. |
| manager.image.repository | string | `"clastix/capsule"` | Set the image repository of the capsule. |
| manager.image.tag | string | `""` | Overrides the image tag whose default is the chart appVersion. |
| manager.imagePullSecrets | list | `[]` | Configuration for `imagePullSecrets` so that you can use a private images registry. |
| manager.kind | string | `"Deployment"` | Set the controller deployment mode as `Deployment` or `DaemonSet`. |
| manager.livenessProbe | object | `{"httpGet":{"path":"/healthz","port":10080}}` | Configure the liveness probe using Deployment probe spec |
| manager.options.capsuleUserGroups | list | `["capsule.clastix.io"]` | Override the Capsule user groups |
| manager.options.forceTenantPrefix | bool | `false` | Boolean, enforces the Tenant owner, during Namespace creation, to name it using the selected Tenant name as prefix, separated by a dash |
| manager.options.generateCertificates | bool | `true` | Specifies whether capsule webhooks certificates should be generated by capsule operator |
| manager.options.logLevel | string | `"4"` | Set the log verbosity of the capsule with a value from 1 to 10 |
| manager.options.protectedNamespaceRegex | string | `""` | If specified, disallows creation of namespaces matching the passed regexp |
| manager.readinessProbe | object | `{"httpGet":{"path":"/readyz","port":10080}}` | Configure the readiness probe using Deployment probe spec |
| manager.resources.limits.cpu | string | `"200m"` | |
| manager.resources.limits.memory | string | `"128Mi"` | |
| manager.resources.requests.cpu | string | `"200m"` | |
| manager.resources.requests.memory | string | `"128Mi"` | |
### ServiceMonitor Parameters
| Key | Type | Default | Description |
|-----|------|---------|-------------|
| serviceMonitor.annotations | object | `{}` | Assign additional Annotations |
| serviceMonitor.enabled | bool | `false` | Enable ServiceMonitor |
| serviceMonitor.endpoint.interval | string | `"15s"` | Set the scrape interval for the endpoint of the serviceMonitor |
| serviceMonitor.endpoint.metricRelabelings | list | `[]` | Set metricRelabelings for the endpoint of the serviceMonitor |
| serviceMonitor.endpoint.relabelings | list | `[]` | Set relabelings for the endpoint of the serviceMonitor |
| serviceMonitor.endpoint.scrapeTimeout | string | `""` | Set the scrape timeout for the endpoint of the serviceMonitor |
| serviceMonitor.labels | object | `{}` | Assign additional labels according to Prometheus' serviceMonitorSelector matching labels |
| serviceMonitor.matchLabels | object | `{}` | Change matching labels |
| serviceMonitor.namespace | string | `""` | Install the ServiceMonitor into a different Namespace, as the monitoring stack one (default: the release one) |
| serviceMonitor.serviceAccount.name | string | `"capsule"` | ServiceAccount for Metrics RBAC |
| serviceMonitor.serviceAccount.namespace | string | `"capsule-system"` | ServiceAccount Namespace for Metrics RBAC |
| serviceMonitor.targetLabels | list | `[]` | Set targetLabels for the serviceMonitor |
### Webhook Parameters
| Key | Type | Default | Description |
|-----|------|---------|-------------|
| webhooks.cordoning.failurePolicy | string | `"Fail"` | |
| webhooks.cordoning.namespaceSelector.matchExpressions[0].key | string | `"capsule.clastix.io/tenant"` | |
| webhooks.cordoning.namespaceSelector.matchExpressions[0].operator | string | `"Exists"` | |
| webhooks.ingresses.failurePolicy | string | `"Fail"` | |
| webhooks.ingresses.namespaceSelector.matchExpressions[0].key | string | `"capsule.clastix.io/tenant"` | |
| webhooks.ingresses.namespaceSelector.matchExpressions[0].operator | string | `"Exists"` | |
| webhooks.namespaceOwnerReference.failurePolicy | string | `"Fail"` | |
| webhooks.namespaces.failurePolicy | string | `"Fail"` | |
| webhooks.networkpolicies.failurePolicy | string | `"Fail"` | |
| webhooks.networkpolicies.namespaceSelector.matchExpressions[0].key | string | `"capsule.clastix.io/tenant"` | |
| webhooks.networkpolicies.namespaceSelector.matchExpressions[0].operator | string | `"Exists"` | |
| webhooks.nodes.failurePolicy | string | `"Fail"` | |
| webhooks.persistentvolumeclaims.failurePolicy | string | `"Fail"` | |
| webhooks.persistentvolumeclaims.namespaceSelector.matchExpressions[0].key | string | `"capsule.clastix.io/tenant"` | |
| webhooks.persistentvolumeclaims.namespaceSelector.matchExpressions[0].operator | string | `"Exists"` | |
| webhooks.pods.failurePolicy | string | `"Fail"` | |
| webhooks.pods.namespaceSelector.matchExpressions[0].key | string | `"capsule.clastix.io/tenant"` | |
| webhooks.pods.namespaceSelector.matchExpressions[0].operator | string | `"Exists"` | |
| webhooks.services.failurePolicy | string | `"Fail"` | |
| webhooks.services.namespaceSelector.matchExpressions[0].key | string | `"capsule.clastix.io/tenant"` | |
| webhooks.services.namespaceSelector.matchExpressions[0].operator | string | `"Exists"` | |
| webhooks.tenants.failurePolicy | string | `"Fail"` | |
## Created resources
@@ -125,6 +178,34 @@ And optionally, depending on the values set:
Capsule, as many other add-ons, defines its own set of Custom Resource Definitions (CRDs). Helm3 removed the old CRDs installation method for a more simple methodology. In the Helm Chart, there is now a special directory called `crds` to hold the CRDs. These CRDs are not templated, but will be installed by default when running a `helm install` for the chart. If the CRDs already exist (for example, you already executed `helm install`), it will be skipped with a warning. When you wish to skip the CRDs installation, and do not see the warning, you can pass the `--skip-crds` flag to the `helm install` command.
## Cert-Manager integration
You can enable the generation of certificates using `cert-manager` as follows.
```
helm upgrade --install capsule clastix/capsule --namespace capsule-system --create-namespace \
--set "certManager.generateCertificates=true" \
--set "tls.create=false" \
--set "tls.enableController=false"
```
With the usage of `tls.enableController=false` value, you're delegating the injection of the Validating and Mutating Webhooks' CA to `cert-manager`.
Since Helm3 doesn't allow to template _CRDs_, you have to patch manually the Custom Resource Definition `tenants.capsule.clastix.io` adding the proper annotation (YMMV).
```yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.5.0
cert-manager.io/inject-ca-from: capsule-system/capsule-webhook-cert
creationTimestamp: "2022-07-22T08:32:51Z"
generation: 45
name: tenants.capsule.clastix.io
resourceVersion: "9832"
uid: 61e287df-319b-476d-88d5-bdb8dc14d4a6
```
## More
See Capsule [tutorial](https://github.com/clastix/capsule/blob/master/docs/content/general/tutorial.md) for more information about how to use Capsule.

View File

@@ -0,0 +1,160 @@
# Deploying the Capsule Operator
Use the Capsule Operator for easily implementing, managing, and maintaining multitenancy and access control in Kubernetes.
## Requirements
* [Helm 3](https://github.com/helm/helm/releases) is required when installing the Capsule Operator chart. Follow Helms official [steps](https://helm.sh/docs/intro/install/) for installing helm on your particular operating system.
* A Kubernetes cluster 1.16+ with following [Admission Controllers](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/) enabled:
* PodNodeSelector
* LimitRanger
* ResourceQuota
* MutatingAdmissionWebhook
* ValidatingAdmissionWebhook
* A [`kubeconfig`](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/) file accessing the Kubernetes cluster with cluster admin permissions.
## Quick Start
The Capsule Operator Chart can be used to instantly deploy the Capsule Operator on your Kubernetes cluster.
1. Add this repository:
$ helm repo add clastix https://clastix.github.io/charts
2. Install the Chart:
$ helm install capsule clastix/capsule -n capsule-system --create-namespace
3. Show the status:
$ helm status capsule -n capsule-system
4. Upgrade the Chart
$ helm upgrade capsule clastix/capsule -n capsule-system
5. Uninstall the Chart
$ helm uninstall capsule -n capsule-system
## Customize the installation
There are two methods for specifying overrides of values during chart installation: `--values` and `--set`.
The `--values` option is the preferred method because it allows you to keep your overrides in a YAML file, rather than specifying them all on the command line. Create a copy of the YAML file `values.yaml` and add your overrides to it.
Specify your overrides file when you install the chart:
$ helm install capsule capsule-helm-chart --values myvalues.yaml -n capsule-system
The values in your overrides file `myvalues.yaml` will override their counterparts in the charts values.yaml file. Any values in `values.yaml` that werent overridden will keep their defaults.
If you only need to make minor customizations, you can specify them on the command line by using the `--set` option. For example:
$ helm install capsule capsule-helm-chart --set manager.options.forceTenantPrefix=false -n capsule-system
Here the values you can override:
### General Parameters
| Key | Type | Default | Description |
|-----|------|---------|-------------|
{{- range .Values }}
{{- if not (or (hasPrefix "manager" .Key) (hasPrefix "serviceMonitor" .Key) (hasPrefix "webhook" .Key) (hasPrefix "capsule-proxy" .Key) ) }}
| {{ .Key }} | {{ .Type }} | {{ if .Default }}{{ .Default }}{{ else }}{{ .AutoDefault }}{{ end }} | {{ if .Description }}{{ .Description }}{{ else }}{{ .AutoDescription }}{{ end }} |
{{- end }}
{{- end }}
### Manager Parameters
| Key | Type | Default | Description |
|-----|------|---------|-------------|
{{- range .Values }}
{{- if hasPrefix "manager" .Key }}
| {{ .Key }} | {{ .Type }} | {{ if .Default }}{{ .Default }}{{ else }}{{ .AutoDefault }}{{ end }} | {{ if .Description }}{{ .Description }}{{ else }}{{ .AutoDescription }}{{ end }} |
{{- end }}
{{- end }}
### ServiceMonitor Parameters
| Key | Type | Default | Description |
|-----|------|---------|-------------|
{{- range .Values }}
{{- if hasPrefix "serviceMonitor" .Key }}
| {{ .Key }} | {{ .Type }} | {{ if .Default }}{{ .Default }}{{ else }}{{ .AutoDefault }}{{ end }} | {{ if .Description }}{{ .Description }}{{ else }}{{ .AutoDescription }}{{ end }} |
{{- end }}
{{- end }}
### Webhook Parameters
| Key | Type | Default | Description |
|-----|------|---------|-------------|
{{- range .Values }}
{{- if hasPrefix "webhook" .Key }}
| {{ .Key }} | {{ .Type }} | {{ if .Default }}{{ .Default }}{{ else }}{{ .AutoDefault }}{{ end }} | {{ if .Description }}{{ .Description }}{{ else }}{{ .AutoDescription }}{{ end }} |
{{- end }}
{{- end }}
## Created resources
This Helm Chart creates the following Kubernetes resources in the release namespace:
* Capsule Namespace
* Capsule Operator Deployment
* Capsule Service
* CA Secret
* Certificate Secret
* Tenant Custom Resource Definition
* CapsuleConfiguration Custom Resource Definition
* MutatingWebHookConfiguration
* ValidatingWebHookConfiguration
* RBAC Cluster Roles
* Metrics Service
And optionally, depending on the values set:
* Capsule ServiceAccount
* Capsule Service Monitor
* PodSecurityPolicy
* RBAC ClusterRole and RoleBinding for pod security policy
* RBAC Role and Rolebinding for metrics scrape
## Notes on installing Custom Resource Definitions with Helm3
Capsule, as many other add-ons, defines its own set of Custom Resource Definitions (CRDs). Helm3 removed the old CRDs installation method for a more simple methodology. In the Helm Chart, there is now a special directory called `crds` to hold the CRDs. These CRDs are not templated, but will be installed by default when running a `helm install` for the chart. If the CRDs already exist (for example, you already executed `helm install`), it will be skipped with a warning. When you wish to skip the CRDs installation, and do not see the warning, you can pass the `--skip-crds` flag to the `helm install` command.
## Cert-Manager integration
You can enable the generation of certificates using `cert-manager` as follows.
```
helm upgrade --install capsule clastix/capsule --namespace capsule-system --create-namespace \
--set "certManager.generateCertificates=true" \
--set "tls.create=false" \
--set "tls.enableController=false"
```
With the usage of `tls.enableController=false` value, you're delegating the injection of the Validating and Mutating Webhooks' CA to `cert-manager`.
Since Helm3 doesn't allow to template _CRDs_, you have to patch manually the Custom Resource Definition `tenants.capsule.clastix.io` adding the proper annotation (YMMV).
```yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.5.0
cert-manager.io/inject-ca-from: capsule-system/capsule-webhook-cert
creationTimestamp: "2022-07-22T08:32:51Z"
generation: 45
name: tenants.capsule.clastix.io
resourceVersion: "9832"
uid: 61e287df-319b-476d-88d5-bdb8dc14d4a6
```
## More
See Capsule [tutorial](https://github.com/clastix/capsule/blob/master/docs/content/general/tutorial.md) for more information about how to use Capsule.

View File

@@ -17,7 +17,7 @@ spec:
- name: v1alpha1
schema:
openAPIV3Schema:
description: CapsuleConfiguration is the Schema for the Capsule configuration API
description: CapsuleConfiguration is the Schema for the Capsule configuration API.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
@@ -28,7 +28,7 @@ spec:
metadata:
type: object
spec:
description: CapsuleConfigurationSpec defines the Capsule configuration
description: CapsuleConfigurationSpec defines the Capsule configuration.
properties:
forceTenantPrefix:
default: false

View File

@@ -56,7 +56,7 @@ spec:
name: v1alpha1
schema:
openAPIV3Schema:
description: Tenant is the Schema for the tenants API
description: Tenant is the Schema for the tenants API.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
@@ -67,7 +67,7 @@ spec:
metadata:
type: object
spec:
description: TenantSpec defines the desired state of Tenant
description: TenantSpec defines the desired state of Tenant.
properties:
additionalRoleBindings:
items:
@@ -485,7 +485,7 @@ spec:
type: string
type: object
owner:
description: OwnerSpec defines tenant owner name and kind
description: OwnerSpec defines tenant owner name and kind.
properties:
kind:
enum:
@@ -568,7 +568,7 @@ spec:
- owner
type: object
status:
description: TenantStatus defines the observed state of Tenant
description: TenantStatus defines the observed state of Tenant.
properties:
namespaces:
items:
@@ -608,7 +608,7 @@ spec:
name: v1beta1
schema:
openAPIV3Schema:
description: Tenant is the Schema for the tenants API
description: Tenant is the Schema for the tenants API.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
@@ -619,7 +619,7 @@ spec:
metadata:
type: object
spec:
description: TenantSpec defines the desired state of Tenant
description: TenantSpec defines the desired state of Tenant.
properties:
additionalRoleBindings:
description: Specifies additional RoleBindings assigned to the Tenant. Capsule will ensure that all namespaces in the Tenant always contain the RoleBinding for the given ClusterRole. Optional.
@@ -707,7 +707,7 @@ spec:
type: string
type: object
limitRanges:
description: Specifies the NetworkPolicies assigned to the Tenant. The assigned NetworkPolicies are inherited by any namespace created in the Tenant. Optional.
description: Specifies the resource min/max usage restrictions to the Tenant. The assigned values are inherited by any namespace created in the Tenant. Optional.
properties:
items:
items:
@@ -1065,7 +1065,7 @@ spec:
nodeSelector:
additionalProperties:
type: string
description: Specifies the label to control the placement of pods on a given pool of worker nodes. All namesapces created within the Tenant will have the node selector annotation. This annotation tells the Kubernetes scheduler to place pods on the nodes having the selector label. Optional.
description: Specifies the label to control the placement of pods on a given pool of worker nodes. All namespaces created within the Tenant will have the node selector annotation. This annotation tells the Kubernetes scheduler to place pods on the nodes having the selector label. Optional.
type: object
owners:
description: Specifies the owners of the Tenant. Mandatory.
@@ -1234,7 +1234,7 @@ spec:
- owners
type: object
status:
description: Returns the observed state of the Tenant
description: Returns the observed state of the Tenant.
properties:
namespaces:
description: List of namespaces assigned to the Tenant.

View File

@@ -5,7 +5,7 @@
# Check the capsule logs
$ kubectl logs -f deployment/{{ template "capsule.fullname" . }}-controller-manager -c manager -n{{ .Release.Namespace }}
$ kubectl logs -f deployment/{{ template "capsule.fullname" . }}-controller-manager -c manager -n {{ .Release.Namespace }}
- Manage this chart:

View File

@@ -65,7 +65,6 @@ ServiceAccount annotations
{{- end }}
{{- end }}
{{/*
Create the name of the service account to use
*/}}
@@ -114,22 +113,15 @@ Create the jobs fully-qualified Docker image to use
{{- end }}
{{/*
Create the Capsule Deployment name to use
Create the Capsule controller name to use
*/}}
{{- define "capsule.deploymentName" -}}
{{- define "capsule.controllerName" -}}
{{- printf "%s-controller-manager" (include "capsule.fullname" .) -}}
{{- end }}
{{/*
Create the Capsule CA Secret name to use
*/}}
{{- define "capsule.secretCaName" -}}
{{- printf "%s-ca" (include "capsule.fullname" .) -}}
{{- end }}
{{/*
Create the Capsule TLS Secret name to use
*/}}
{{- define "capsule.secretTlsName" -}}
{{- printf "%s-tls" (include "capsule.fullname" .) -}}
{{ default ( printf "%s-tls" ( include "capsule.fullname" . ) ) .Values.tls.name }}
{{- end }}

View File

@@ -1,10 +0,0 @@
apiVersion: v1
kind: Secret
metadata:
labels:
{{- include "capsule.labels" . | nindent 4 }}
{{- with .Values.customAnnotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
name: {{ include "capsule.secretCaName" . }}

View File

@@ -0,0 +1,36 @@
{{- if .Values.certManager.generateCertificates }}
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: {{ include "capsule.fullname" . }}-webhook-selfsigned
labels:
{{- include "capsule.labels" . | nindent 4 }}
{{- with .Values.customAnnotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
selfSigned: {}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: {{ include "capsule.fullname" . }}-webhook-cert
labels:
{{- include "capsule.labels" . | nindent 4 }}
{{- with .Values.customAnnotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
dnsNames:
- {{ include "capsule.fullname" . }}-webhook-service.{{ .Release.Namespace }}.svc
- {{ include "capsule.fullname" . }}-webhook-service.{{ .Release.Namespace }}.svc.cluster.local
issuerRef:
kind: Issuer
name: {{ include "capsule.fullname" . }}-webhook-selfsigned
secretName: {{ include "capsule.secretTlsName" . }}
subject:
organizations:
- clastix.io
{{- end }}

View File

@@ -1,3 +1,4 @@
{{- if or (not .Values.certManager.generateCertificates) (.Values.tls.create) }}
apiVersion: v1
kind: Secret
metadata:
@@ -8,3 +9,4 @@ metadata:
{{- toYaml . | nindent 4 }}
{{- end }}
name: {{ include "capsule.secretTlsName" . }}
{{- end }}

View File

@@ -5,10 +5,10 @@ metadata:
labels:
{{- include "capsule.labels" . | nindent 4 }}
annotations:
capsule.clastix.io/ca-secret-name: {{ include "capsule.secretCaName" . }}
capsule.clastix.io/mutating-webhook-configuration-name: {{ include "capsule.fullname" . }}-mutating-webhook-configuration
capsule.clastix.io/tls-secret-name: {{ include "capsule.secretTlsName" . }}
capsule.clastix.io/validating-webhook-configuration-name: {{ include "capsule.fullname" . }}-validating-webhook-configuration
capsule.clastix.io/enable-tls-configuration: "{{ .Values.tls.enableController }}"
{{- with .Values.customAnnotations }}
{{- toYaml . | nindent 4 }}
{{- end }}

View File

@@ -0,0 +1,88 @@
{{- if eq .Values.manager.kind "DaemonSet" }}
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: {{ include "capsule.controllerName" . }}
labels:
{{- include "capsule.labels" . | nindent 4 }}
{{- with .Values.customAnnotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
updateStrategy:
type: RollingUpdate
selector:
matchLabels:
{{- include "capsule.selectorLabels" . | nindent 6 }}
template:
metadata:
{{- with .Values.podAnnotations }}
annotations:
{{- toYaml . | nindent 8 }}
{{- end }}
labels:
{{- include "capsule.labels" . | nindent 8 }}
spec:
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
serviceAccountName: {{ include "capsule.serviceAccountName" . }}
{{- if .Values.manager.hostNetwork }}
hostNetwork: true
dnsPolicy: ClusterFirstWithHostNet
{{- end }}
priorityClassName: {{ .Values.priorityClassName }}
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
volumes:
- name: cert
secret:
defaultMode: 420
secretName: {{ include "capsule.secretTlsName" . }}
containers:
- name: manager
command:
- /manager
args:
- --enable-leader-election
- --zap-log-level={{ default 4 .Values.manager.options.logLevel }}
- --configuration-name=default
image: {{ include "capsule.managerFullyQualifiedDockerImage" . }}
imagePullPolicy: {{ .Values.manager.image.pullPolicy }}
env:
- name: NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
ports:
- name: webhook-server
containerPort: 9443
protocol: TCP
- name: metrics
containerPort: 8080
protocol: TCP
livenessProbe:
{{- toYaml .Values.manager.livenessProbe | nindent 12}}
readinessProbe:
{{- toYaml .Values.manager.readinessProbe | nindent 12}}
volumeMounts:
- mountPath: /tmp/k8s-webhook-server/serving-certs
name: cert
readOnly: true
resources:
{{- toYaml .Values.manager.resources | nindent 12 }}
securityContext:
allowPrivilegeEscalation: false
{{- end }}

View File

@@ -1,7 +1,8 @@
{{- if eq .Values.manager.kind "Deployment" }}
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "capsule.deploymentName" . }}
name: {{ include "capsule.controllerName" . }}
labels:
{{- include "capsule.labels" . | nindent 4 }}
{{- with .Values.customAnnotations }}
@@ -29,6 +30,7 @@ spec:
serviceAccountName: {{ include "capsule.serviceAccountName" . }}
{{- if .Values.manager.hostNetwork }}
hostNetwork: true
dnsPolicy: ClusterFirstWithHostNet
{{- end }}
priorityClassName: {{ .Values.priorityClassName }}
{{- with .Values.nodeSelector }}
@@ -47,7 +49,7 @@ spec:
- name: cert
secret:
defaultMode: 420
secretName: {{ include "capsule.fullname" . }}-tls
secretName: {{ include "capsule.secretTlsName" . }}
containers:
- name: manager
command:
@@ -82,3 +84,4 @@ spec:
{{- toYaml .Values.manager.resources | nindent 12 }}
securityContext:
allowPrivilegeEscalation: false
{{- end }}

View File

@@ -4,8 +4,11 @@ metadata:
name: {{ include "capsule.fullname" . }}-mutating-webhook-configuration
labels:
{{- include "capsule.labels" . | nindent 4 }}
{{- with .Values.customAnnotations }}
annotations:
{{- if .Values.certManager.generateCertificates }}
cert-manager.io/inject-ca-from: {{ .Release.Namespace }}/{{ include "capsule.fullname" . }}-webhook-cert
{{- end }}
{{- with .Values.customAnnotations }}
{{- toYaml . | nindent 4 }}
{{- end }}
webhooks:
@@ -13,7 +16,9 @@ webhooks:
- v1
- v1beta1
clientConfig:
{{- if not .Values.certManager.generateCertificates }}
caBundle: Cg==
{{- end }}
service:
name: {{ include "capsule.fullname" . }}-webhook-service
namespace: {{ .Release.Namespace }}

View File

@@ -1,4 +1,5 @@
{{- $cmd := printf "while [ -z $$(kubectl -n $NAMESPACE get secret %s -o jsonpath='{.data.tls\\\\.crt}') ];" (include "capsule.secretCaName" .) -}}
{{- if .Values.tls.create }}
{{- $cmd := printf "while [ -z $$(kubectl -n $NAMESPACE get secret %s -o jsonpath='{.data.tls\\\\.crt}') ];" (include "capsule.secretTlsName" .) -}}
{{- $cmd = printf "%s do echo 'waiting Capsule to be up and running...' && sleep 5;" $cmd -}}
{{- $cmd = printf "%s done" $cmd -}}
apiVersion: batch/v1
@@ -44,4 +45,5 @@ spec:
valueFrom:
fieldRef:
fieldPath: metadata.namespace
serviceAccountName: {{ include "capsule.serviceAccountName" . }}
serviceAccountName: {{ include "capsule.serviceAccountName" . }}
{{- end }}

View File

@@ -1,5 +1,7 @@
{{- $cmd := printf "kubectl scale deployment -n $NAMESPACE %s --replicas 0 &&" (include "capsule.deploymentName" .) -}}
{{- $cmd = printf "%s kubectl delete secret -n $NAMESPACE %s %s --ignore-not-found &&" $cmd (include "capsule.secretTlsName" .) (include "capsule.secretCaName" .) -}}
{{- $cmd := ""}}
{{- if or (.Values.tls.create) (.Values.certManager.generateCertificates) }}
{{- $cmd = printf "%s kubectl delete secret -n $NAMESPACE %s --ignore-not-found &&" $cmd (include "capsule.secretTlsName" .) -}}
{{- end }}
{{- $cmd = printf "%s kubectl delete clusterroles.rbac.authorization.k8s.io capsule-namespace-deleter capsule-namespace-provisioner --ignore-not-found &&" $cmd -}}
{{- $cmd = printf "%s kubectl delete clusterrolebindings.rbac.authorization.k8s.io capsule-namespace-deleter capsule-namespace-provisioner --ignore-not-found" $cmd -}}
apiVersion: batch/v1

View File

@@ -15,17 +15,33 @@ metadata:
{{- end }}
spec:
endpoints:
- interval: 15s
{{- with .Values.serviceMonitor.endpoint }}
- interval: {{ .interval }}
port: metrics
path: /metrics
{{- with .scrapeTimeout }}
scrapeTimeout: {{ . }}
{{- end }}
{{- with .metricRelabelings }}
metricRelabelings: {{- toYaml . | nindent 6 }}
{{- end }}
{{- with .relabelings }}
relabelings: {{- toYaml . | nindent 6 }}
{{- end }}
{{- end }}
jobLabel: app.kubernetes.io/name
{{- with .Values.serviceMonitor.targetLabels }}
targetLabels: {{- toYaml . | nindent 4 }}
{{- end }}
selector:
matchLabels:
{{- include "capsule.labels" . | nindent 6 }}
{{- with .Values.serviceMonitor.matchLabels }}
{{- toYaml . | nindent 6 }}
{{- if .Values.serviceMonitor.matchLabels }}
{{- toYaml .Values.serviceMonitor.matchLabels | nindent 6 }}
{{- else }}
{{- include "capsule.labels" . | nindent 6 }}
{{- end }}
namespaceSelector:
matchNames:
- {{ .Release.Namespace }}
{{- end }}

View File

@@ -4,8 +4,11 @@ metadata:
name: {{ include "capsule.fullname" . }}-validating-webhook-configuration
labels:
{{- include "capsule.labels" . | nindent 4 }}
{{- with .Values.customAnnotations }}
annotations:
{{- if .Values.certManager.generateCertificates }}
cert-manager.io/inject-ca-from: {{ .Release.Namespace }}/{{ include "capsule.fullname" . }}-webhook-cert
{{- end }}
{{- with .Values.customAnnotations }}
{{- toYaml . | nindent 4 }}
{{- end }}
webhooks:
@@ -13,7 +16,9 @@ webhooks:
- v1
- v1beta1
clientConfig:
{{- if not .Values.certManager.generateCertificates }}
caBundle: Cg==
{{- end }}
service:
name: {{ include "capsule.fullname" . }}-webhook-service
namespace: {{ .Release.Namespace }}
@@ -43,7 +48,9 @@ webhooks:
- v1
- v1beta1
clientConfig:
{{- if not .Values.certManager.generateCertificates }}
caBundle: Cg==
{{- end }}
service:
name: {{ include "capsule.fullname" . }}-webhook-service
namespace: {{ .Release.Namespace }}
@@ -74,7 +81,9 @@ webhooks:
- v1
- v1beta1
clientConfig:
{{- if not .Values.certManager.generateCertificates }}
caBundle: Cg==
{{- end }}
service:
name: {{ include "capsule.fullname" . }}-webhook-service
namespace: {{ .Release.Namespace }}
@@ -103,7 +112,9 @@ webhooks:
- v1
- v1beta1
clientConfig:
{{- if not .Values.certManager.generateCertificates }}
caBundle: Cg==
{{- end }}
service:
name: {{ include "capsule.fullname" . }}-webhook-service
namespace: {{ .Release.Namespace }}
@@ -132,7 +143,9 @@ webhooks:
- v1
- v1beta1
clientConfig:
{{- if not .Values.certManager.generateCertificates }}
caBundle: Cg==
{{- end }}
service:
name: {{ include "capsule.fullname" . }}-webhook-service
namespace: {{ .Release.Namespace }}
@@ -160,7 +173,9 @@ webhooks:
- v1
- v1beta1
clientConfig:
{{- if not .Values.certManager.generateCertificates }}
caBundle: Cg==
{{- end }}
service:
name: {{ include "capsule.fullname" . }}-webhook-service
namespace: {{ .Release.Namespace }}
@@ -186,7 +201,9 @@ webhooks:
- v1
- v1beta1
clientConfig:
{{- if not .Values.certManager.generateCertificates }}
caBundle: Cg==
{{- end }}
service:
name: {{ include "capsule.fullname" . }}-webhook-service
namespace: {{ .Release.Namespace }}
@@ -215,7 +232,9 @@ webhooks:
- v1
- v1beta1
clientConfig:
{{- if not .Values.certManager.generateCertificates }}
caBundle: Cg==
{{- end }}
service:
name: {{ include "capsule.fullname" . }}-webhook-service
namespace: {{ .Release.Namespace }}
@@ -244,7 +263,9 @@ webhooks:
- v1
- v1beta1
clientConfig:
{{- if not .Values.certManager.generateCertificates }}
caBundle: Cg==
{{- end }}
service:
name: {{ include "capsule.fullname" . }}-webhook-service
namespace: {{ .Release.Namespace }}

View File

@@ -2,29 +2,59 @@
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.
# Secret Options
tls:
# -- Start the Capsule controller that injects the CA into mutating and validating webhooks, and CRD as well.
enableController: true
# -- When cert-manager is disabled, Capsule will generate the TLS certificate for webhook and CRDs conversion.
create: true
# -- Override name of the Capsule TLS Secret name when externally managed.
name: ""
# Manager Options
manager:
# -- Set the controller deployment mode as `Deployment` or `DaemonSet`.
kind: Deployment
image:
repository: quay.io/clastix/capsule
# -- Set the image repository of the capsule.
repository: clastix/capsule
# -- Set the image pull policy.
pullPolicy: IfNotPresent
# -- Overrides the image tag whose default is the chart appVersion.
tag: ''
# Specifies if the container should be started in hostNetwork mode.
# -- Configuration for `imagePullSecrets` so that you can use a private images registry.
imagePullSecrets: []
# -- Specifies if the container should be started in hostNetwork mode.
#
# Required for use in some managed kubernetes clusters (such as AWS EKS) with custom
# CNI (such as calico), because control-plane managed by AWS cannot communicate
# with pods' IP CIDR and admission webhooks are not working
hostNetwork: false
# Additional Capsule options
# Additional Capsule Controller Options
options:
# -- Set the log verbosity of the capsule with a value from 1 to 10
logLevel: '4'
# -- Boolean, enforces the Tenant owner, during Namespace creation, to name it using the selected Tenant name as prefix, separated by a dash
forceTenantPrefix: false
# -- Override the Capsule user groups
capsuleUserGroups: ["capsule.clastix.io"]
# -- If specified, disallows creation of namespaces matching the passed regexp
protectedNamespaceRegex: ""
# -- Specifies whether capsule webhooks certificates should be generated by capsule operator
generateCertificates: true
# -- Configure the liveness probe using Deployment probe spec
livenessProbe:
httpGet:
path: /healthz
port: 10080
# -- Configure the readiness probe using Deployment probe spec
readinessProbe:
httpGet:
path: /readyz
@@ -37,46 +67,63 @@ manager:
requests:
cpu: 200m
memory: 128Mi
jobs:
image:
repository: quay.io/clastix/kubectl
pullPolicy: IfNotPresent
tag: ""
imagePullSecrets: []
serviceAccount:
create: true
annotations: {}
name: "capsule"
# -- Annotations to add to the capsule pod.
podAnnotations: {}
# The following annotations guarantee scheduling for critical add-on pods
# podAnnotations:
# scheduler.alpha.kubernetes.io/critical-pod: ''
# -- Set the priority class name of the Capsule pod
priorityClassName: '' #system-cluster-critical
# -- Set the node selector for the Capsule pod
nodeSelector: {}
# node-role.kubernetes.io/master: ""
# -- Set list of tolerations for the Capsule pod
tolerations: []
#- key: CriticalAddonsOnly
# operator: Exists
#- effect: NoSchedule
# key: node-role.kubernetes.io/master
# -- Set the replica count for capsule pod
replicaCount: 1
# -- Set affinity rules for the Capsule pod
affinity: {}
podSecurityPolicy:
# -- Specify if a Pod Security Policy must be created
enabled: false
serviceMonitor:
enabled: false
# Install the ServiceMonitor into a different Namespace, as the monitoring stack one (default: the release one)
namespace: ''
# Assign additional labels according to Prometheus' serviceMonitorSelector matching labels
labels: {}
jobs:
image:
# -- Set the image repository of the helm chart job
repository: quay.io/clastix/kubectl
# -- Set the image pull policy of the helm chart job
pullPolicy: IfNotPresent
# -- Set the image tag of the helm chart job
tag: ""
# ServiceAccount
serviceAccount:
# -- Specifies whether a service account should be created.
create: true
# -- Annotations to add to the service account.
annotations: {}
matchLabels: {}
serviceAccount:
name: capsule
namespace: capsule-system
# -- The name of the service account to use. If not set and `serviceAccount.create=true`, a name is generated using the fullname template
name: "capsule"
# Additional labels
certManager:
# -- Specifies whether capsule webhooks certificates should be generated using cert-manager
generateCertificates: false
# -- Additional labels which will be added to all resources created by Capsule helm chart
customLabels: {}
# Additional annotations
# -- Additional annotations which will be added to all resources created by Capsule helm chart
customAnnotations: {}
# Webhooks configurations
@@ -125,5 +172,37 @@ webhooks:
operator: Exists
nodes:
failurePolicy: Fail
# -- Timeout in seconds for mutating webhooks
mutatingWebhooksTimeoutSeconds: 30
# -- Timeout in seconds for validating webhooks
validatingWebhooksTimeoutSeconds: 30
# ServiceMonitor
serviceMonitor:
# -- Enable ServiceMonitor
enabled: false
# -- Install the ServiceMonitor into a different Namespace, as the monitoring stack one (default: the release one)
namespace: ''
# -- Assign additional labels according to Prometheus' serviceMonitorSelector matching labels
labels: {}
# -- Assign additional Annotations
annotations: {}
# -- Change matching labels
matchLabels: {}
# -- Set targetLabels for the serviceMonitor
targetLabels: []
serviceAccount:
# -- ServiceAccount for Metrics RBAC
name: capsule
# -- ServiceAccount Namespace for Metrics RBAC
namespace: capsule-system
endpoint:
# -- Set the scrape interval for the endpoint of the serviceMonitor
interval: "15s"
# -- Set the scrape timeout for the endpoint of the serviceMonitor
scrapeTimeout: ""
# -- Set metricRelabelings for the endpoint of the serviceMonitor
metricRelabelings: []
# -- Set relabelings for the endpoint of the serviceMonitor
relabelings: []

View File

@@ -19,7 +19,7 @@ spec:
- name: v1alpha1
schema:
openAPIV3Schema:
description: CapsuleConfiguration is the Schema for the Capsule configuration API
description: CapsuleConfiguration is the Schema for the Capsule configuration API.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
@@ -30,7 +30,7 @@ spec:
metadata:
type: object
spec:
description: CapsuleConfigurationSpec defines the Capsule configuration
description: CapsuleConfigurationSpec defines the Capsule configuration.
properties:
forceTenantPrefix:
default: false

View File

@@ -46,7 +46,7 @@ spec:
name: v1alpha1
schema:
openAPIV3Schema:
description: Tenant is the Schema for the tenants API
description: Tenant is the Schema for the tenants API.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
@@ -57,7 +57,7 @@ spec:
metadata:
type: object
spec:
description: TenantSpec defines the desired state of Tenant
description: TenantSpec defines the desired state of Tenant.
properties:
additionalRoleBindings:
items:
@@ -475,7 +475,7 @@ spec:
type: string
type: object
owner:
description: OwnerSpec defines tenant owner name and kind
description: OwnerSpec defines tenant owner name and kind.
properties:
kind:
enum:
@@ -558,7 +558,7 @@ spec:
- owner
type: object
status:
description: TenantStatus defines the observed state of Tenant
description: TenantStatus defines the observed state of Tenant.
properties:
namespaces:
items:
@@ -598,7 +598,7 @@ spec:
name: v1beta1
schema:
openAPIV3Schema:
description: Tenant is the Schema for the tenants API
description: Tenant is the Schema for the tenants API.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
@@ -609,7 +609,7 @@ spec:
metadata:
type: object
spec:
description: TenantSpec defines the desired state of Tenant
description: TenantSpec defines the desired state of Tenant.
properties:
additionalRoleBindings:
description: Specifies additional RoleBindings assigned to the Tenant. Capsule will ensure that all namespaces in the Tenant always contain the RoleBinding for the given ClusterRole. Optional.
@@ -654,6 +654,9 @@ spec:
allowedRegex:
type: string
type: object
gitOpsReady:
description: Configured RBAC for machine owners tailored for GitOps controllers.
type: boolean
imagePullPolicies:
description: Specify the allowed values for the imagePullPolicies option in Pod resources. Capsule assures that all Pod resources created in the Tenant can use only one of the allowed policy. Optional.
items:
@@ -1224,7 +1227,7 @@ spec:
- owners
type: object
status:
description: Returns the observed state of the Tenant
description: Returns the observed state of the Tenant.
properties:
namespaces:
description: List of namespaces assigned to the Tenant.

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,7 @@
apiVersion: v1
kind: ConfigMap
metadata:
labels:
# label selector used by Grafana to load the dashboards from Config Maps
grafana_dashboard: "1"
name: capsule-grafana-dashboard

View File

@@ -0,0 +1,8 @@
configMapGenerator:
- name: capsule-grafana-dashboard
files:
- dashboard.json
generatorOptions:
disableNameSuffixHash: true
patchesStrategicMerge:
- dashboard.yaml

View File

@@ -24,7 +24,7 @@ spec:
- name: v1alpha1
schema:
openAPIV3Schema:
description: CapsuleConfiguration is the Schema for the Capsule configuration API
description: CapsuleConfiguration is the Schema for the Capsule configuration API.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
@@ -35,7 +35,7 @@ spec:
metadata:
type: object
spec:
description: CapsuleConfigurationSpec defines the Capsule configuration
description: CapsuleConfigurationSpec defines the Capsule configuration.
properties:
forceTenantPrefix:
default: false
@@ -118,7 +118,7 @@ spec:
name: v1alpha1
schema:
openAPIV3Schema:
description: Tenant is the Schema for the tenants API
description: Tenant is the Schema for the tenants API.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
@@ -129,7 +129,7 @@ spec:
metadata:
type: object
spec:
description: TenantSpec defines the desired state of Tenant
description: TenantSpec defines the desired state of Tenant.
properties:
additionalRoleBindings:
items:
@@ -547,7 +547,7 @@ spec:
type: string
type: object
owner:
description: OwnerSpec defines tenant owner name and kind
description: OwnerSpec defines tenant owner name and kind.
properties:
kind:
enum:
@@ -630,7 +630,7 @@ spec:
- owner
type: object
status:
description: TenantStatus defines the observed state of Tenant
description: TenantStatus defines the observed state of Tenant.
properties:
namespaces:
items:
@@ -670,7 +670,7 @@ spec:
name: v1beta1
schema:
openAPIV3Schema:
description: Tenant is the Schema for the tenants API
description: Tenant is the Schema for the tenants API.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
@@ -681,7 +681,7 @@ spec:
metadata:
type: object
spec:
description: TenantSpec defines the desired state of Tenant
description: TenantSpec defines the desired state of Tenant.
properties:
additionalRoleBindings:
description: Specifies additional RoleBindings assigned to the Tenant. Capsule will ensure that all namespaces in the Tenant always contain the RoleBinding for the given ClusterRole. Optional.
@@ -1296,7 +1296,7 @@ spec:
- owners
type: object
status:
description: Returns the observed state of the Tenant
description: Returns the observed state of the Tenant.
properties:
namespaces:
description: List of namespaces assigned to the Tenant.
@@ -1411,7 +1411,7 @@ spec:
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: quay.io/clastix/capsule:v0.1.1
image: clastix/capsule:v0.1.2
imagePullPolicy: IfNotPresent
name: manager
ports:

View File

@@ -6,5 +6,5 @@ apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
images:
- name: controller
newName: quay.io/clastix/capsule
newTag: v0.1.1
newName: clastix/capsule
newTag: v0.1.2

View File

@@ -9,13 +9,11 @@ import (
"github.com/go-logr/logr"
"github.com/pkg/errors"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/builder"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/event"
"sigs.k8s.io/controller-runtime/pkg/predicate"
"sigs.k8s.io/controller-runtime/pkg/reconcile"
capsulev1alpha1 "github.com/clastix/capsule/api/v1alpha1"
"github.com/clastix/capsule/controllers/utils"
"github.com/clastix/capsule/pkg/configuration"
)
@@ -24,44 +22,23 @@ type Manager struct {
Client client.Client
}
// InjectClient injects the Client interface, required by the Runnable interface
// InjectClient injects the Client interface, required by the Runnable interface.
func (c *Manager) InjectClient(client client.Client) error {
c.Client = client
return nil
}
func filterByName(objName, desired string) bool {
return objName == desired
}
func forOptionPerInstanceName(instanceName string) builder.ForOption {
return builder.WithPredicates(predicate.Funcs{
CreateFunc: func(event event.CreateEvent) bool {
return filterByName(event.Object.GetName(), instanceName)
},
DeleteFunc: func(deleteEvent event.DeleteEvent) bool {
return filterByName(deleteEvent.Object.GetName(), instanceName)
},
UpdateFunc: func(updateEvent event.UpdateEvent) bool {
return filterByName(updateEvent.ObjectNew.GetName(), instanceName)
},
GenericFunc: func(genericEvent event.GenericEvent) bool {
return filterByName(genericEvent.Object.GetName(), instanceName)
},
})
}
func (c *Manager) SetupWithManager(mgr ctrl.Manager, configurationName string) error {
return ctrl.NewControllerManagedBy(mgr).
For(&capsulev1alpha1.CapsuleConfiguration{}, forOptionPerInstanceName(configurationName)).
For(&capsulev1alpha1.CapsuleConfiguration{}, utils.NamesMatchingPredicate(configurationName)).
Complete(c)
}
func (c *Manager) Reconcile(ctx context.Context, request reconcile.Request) (res reconcile.Result, err error) {
c.Log.Info("CapsuleConfiguration reconciliation started", "request.name", request.Name)
cfg := configuration.NewCapsuleConfiguration(c.Client, request.Name)
cfg := configuration.NewCapsuleConfiguration(ctx, c.Client, request.Name)
// Validating the Capsule Configuration options
if _, err = cfg.ProtectedNamespaceRegexp(); err != nil {
panic(errors.Wrap(err, "Invalid configuration for protected Namespace regex"))

View File

@@ -23,7 +23,7 @@ var (
{
APIGroups: []string{""},
Resources: []string{"namespaces"},
Verbs: []string{"create"},
Verbs: []string{"create", "patch"},
},
},
},
@@ -35,7 +35,7 @@ var (
{
APIGroups: []string{""},
Resources: []string{"namespaces"},
Verbs: []string{"delete", "patch"},
Verbs: []string{"delete"},
},
},
},
@@ -48,7 +48,7 @@ var (
RoleRef: rbacv1.RoleRef{
Kind: "ClusterRole",
Name: ProvisionerRoleName,
APIGroup: "rbac.authorization.k8s.io",
APIGroup: rbacv1.GroupName,
},
}
)

View File

@@ -10,20 +10,19 @@ import (
"github.com/go-logr/logr"
"github.com/hashicorp/go-multierror"
rbacv1 "k8s.io/api/rbac/v1"
"k8s.io/apimachinery/pkg/api/errors"
apierrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/util/workqueue"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/builder"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
"sigs.k8s.io/controller-runtime/pkg/event"
"sigs.k8s.io/controller-runtime/pkg/handler"
"sigs.k8s.io/controller-runtime/pkg/predicate"
"sigs.k8s.io/controller-runtime/pkg/reconcile"
"sigs.k8s.io/controller-runtime/pkg/source"
capsulev1alpha1 "github.com/clastix/capsule/api/v1alpha1"
"github.com/clastix/capsule/controllers/utils"
"github.com/clastix/capsule/pkg/configuration"
)
@@ -33,65 +32,40 @@ type Manager struct {
Configuration configuration.Configuration
}
// InjectClient injects the Client interface, required by the Runnable interface
// InjectClient injects the Client interface, required by the Runnable interface.
func (r *Manager) InjectClient(c client.Client) error {
r.Client = c
return nil
}
func (r *Manager) filterByNames(name string) bool {
return name == ProvisionerRoleName || name == DeleterRoleName
}
func (r *Manager) SetupWithManager(ctx context.Context, mgr ctrl.Manager, configurationName string) (err error) {
namesPredicate := utils.NamesMatchingPredicate(ProvisionerRoleName, DeleterRoleName)
//nolint:dupl
func (r *Manager) SetupWithManager(mgr ctrl.Manager, configurationName string) (err error) {
crErr := ctrl.NewControllerManagedBy(mgr).
For(&rbacv1.ClusterRole{}, builder.WithPredicates(predicate.Funcs{
CreateFunc: func(event event.CreateEvent) bool {
return r.filterByNames(event.Object.GetName())
},
DeleteFunc: func(deleteEvent event.DeleteEvent) bool {
return r.filterByNames(deleteEvent.Object.GetName())
},
UpdateFunc: func(updateEvent event.UpdateEvent) bool {
return r.filterByNames(updateEvent.ObjectNew.GetName())
},
GenericFunc: func(genericEvent event.GenericEvent) bool {
return r.filterByNames(genericEvent.Object.GetName())
},
})).
For(&rbacv1.ClusterRole{}, namesPredicate).
Complete(r)
if crErr != nil {
err = multierror.Append(err, crErr)
}
crbErr := ctrl.NewControllerManagedBy(mgr).
For(&rbacv1.ClusterRoleBinding{}, builder.WithPredicates(predicate.Funcs{
CreateFunc: func(event event.CreateEvent) bool {
return r.filterByNames(event.Object.GetName())
},
DeleteFunc: func(deleteEvent event.DeleteEvent) bool {
return r.filterByNames(deleteEvent.Object.GetName())
},
UpdateFunc: func(updateEvent event.UpdateEvent) bool {
return r.filterByNames(updateEvent.ObjectNew.GetName())
},
GenericFunc: func(genericEvent event.GenericEvent) bool {
return r.filterByNames(genericEvent.Object.GetName())
},
})).
For(&rbacv1.ClusterRoleBinding{}, namesPredicate).
Watches(source.NewKindWithCache(&capsulev1alpha1.CapsuleConfiguration{}, mgr.GetCache()), handler.Funcs{
UpdateFunc: func(updateEvent event.UpdateEvent, limitingInterface workqueue.RateLimitingInterface) {
if updateEvent.ObjectNew.GetName() == configurationName {
if crbErr := r.EnsureClusterRoleBindings(); crbErr != nil {
if crbErr := r.EnsureClusterRoleBindings(ctx); crbErr != nil {
r.Log.Error(err, "cannot update ClusterRoleBinding upon CapsuleConfiguration update")
}
}
},
}).
Complete(r)
if crbErr != nil {
err = multierror.Append(err, crbErr)
}
return
}
@@ -100,18 +74,19 @@ func (r *Manager) SetupWithManager(mgr ctrl.Manager, configurationName string) (
func (r *Manager) Reconcile(ctx context.Context, request reconcile.Request) (res reconcile.Result, err error) {
switch request.Name {
case ProvisionerRoleName:
if err = r.EnsureClusterRole(ProvisionerRoleName); err != nil {
if err = r.EnsureClusterRole(ctx, ProvisionerRoleName); err != nil {
r.Log.Error(err, "Reconciliation for ClusterRole failed", "ClusterRole", ProvisionerRoleName)
break
}
if err = r.EnsureClusterRoleBindings(); err != nil {
if err = r.EnsureClusterRoleBindings(ctx); err != nil {
r.Log.Error(err, "Reconciliation for ClusterRoleBindings failed")
break
}
case DeleterRoleName:
if err = r.EnsureClusterRole(DeleterRoleName); err != nil {
if err = r.EnsureClusterRole(ctx, DeleterRoleName); err != nil {
r.Log.Error(err, "Reconciliation for ClusterRole failed", "ClusterRole", DeleterRoleName)
}
}
@@ -119,14 +94,14 @@ func (r *Manager) Reconcile(ctx context.Context, request reconcile.Request) (res
return
}
func (r *Manager) EnsureClusterRoleBindings() (err error) {
func (r *Manager) EnsureClusterRoleBindings(ctx context.Context) (err error) {
crb := &rbacv1.ClusterRoleBinding{
ObjectMeta: metav1.ObjectMeta{
Name: ProvisionerRoleName,
},
}
_, err = controllerutil.CreateOrUpdate(context.TODO(), r.Client, crb, func() (err error) {
_, err = controllerutil.CreateOrUpdate(ctx, r.Client, crb, func() (err error) {
crb.RoleRef = provisionerClusterRoleBinding.RoleRef
crb.Subjects = []rbacv1.Subject{}
@@ -144,7 +119,7 @@ func (r *Manager) EnsureClusterRoleBindings() (err error) {
return
}
func (r *Manager) EnsureClusterRole(roleName string) (err error) {
func (r *Manager) EnsureClusterRole(ctx context.Context, roleName string) (err error) {
role, ok := clusterRoles[roleName]
if !ok {
return fmt.Errorf("clusterRole %s is not mapped", roleName)
@@ -156,8 +131,9 @@ func (r *Manager) EnsureClusterRole(roleName string) (err error) {
},
}
_, err = controllerutil.CreateOrUpdate(context.TODO(), r.Client, clusterRole, func() error {
_, err = controllerutil.CreateOrUpdate(ctx, r.Client, clusterRole, func() error {
clusterRole.Rules = role.Rules
return nil
})
@@ -170,8 +146,9 @@ func (r *Manager) EnsureClusterRole(roleName string) (err error) {
func (r *Manager) Start(ctx context.Context) error {
for roleName := range clusterRoles {
r.Log.Info("setting up ClusterRoles", "ClusterRole", roleName)
if err := r.EnsureClusterRole(roleName); err != nil {
if errors.IsAlreadyExists(err) {
if err := r.EnsureClusterRole(ctx, roleName); err != nil {
if apierrors.IsAlreadyExists(err) {
continue
}
@@ -180,8 +157,9 @@ func (r *Manager) Start(ctx context.Context) error {
}
r.Log.Info("setting up ClusterRoleBindings")
if err := r.EnsureClusterRoleBindings(); err != nil {
if errors.IsAlreadyExists(err) {
if err := r.EnsureClusterRoleBindings(ctx); err != nil {
if apierrors.IsAlreadyExists(err) {
return nil
}

View File

@@ -1,218 +0,0 @@
// Copyright 2020-2021 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package secret
import (
"bytes"
"context"
"errors"
"time"
"github.com/go-logr/logr"
"golang.org/x/sync/errgroup"
admissionregistrationv1 "k8s.io/api/admissionregistration/v1"
corev1 "k8s.io/api/core/v1"
apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/types"
"k8s.io/client-go/util/retry"
"k8s.io/utils/pointer"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
"sigs.k8s.io/controller-runtime/pkg/reconcile"
"github.com/clastix/capsule/pkg/cert"
"github.com/clastix/capsule/pkg/configuration"
)
type CAReconciler struct {
client.Client
Log logr.Logger
Scheme *runtime.Scheme
Namespace string
Configuration configuration.Configuration
}
func (r *CAReconciler) SetupWithManager(mgr ctrl.Manager) error {
return ctrl.NewControllerManagedBy(mgr).
For(&corev1.Secret{}).
Complete(r)
}
// By default helm doesn't allow to use templates in CRD (https://helm.sh/docs/chart_best_practices/custom_resource_definitions/#method-1-let-helm-do-it-for-you).
// In order to overcome this, we are setting conversion strategy in helm chart to None, and then update it with CA and namespace information.
func (r *CAReconciler) UpdateCustomResourceDefinition(caBundle []byte) error {
return retry.RetryOnConflict(retry.DefaultBackoff, func() (err error) {
crd := &apiextensionsv1.CustomResourceDefinition{}
err = r.Get(context.TODO(), types.NamespacedName{Name: "tenants.capsule.clastix.io"}, crd)
if err != nil {
r.Log.Error(err, "cannot retrieve CustomResourceDefinition")
return err
}
_, err = controllerutil.CreateOrUpdate(context.TODO(), r.Client, crd, func() error {
crd.Spec.Conversion = &apiextensionsv1.CustomResourceConversion{
Strategy: "Webhook",
Webhook: &apiextensionsv1.WebhookConversion{
ClientConfig: &apiextensionsv1.WebhookClientConfig{
Service: &apiextensionsv1.ServiceReference{
Namespace: r.Namespace,
Name: "capsule-webhook-service",
Path: pointer.StringPtr("/convert"),
Port: pointer.Int32Ptr(443),
},
CABundle: caBundle,
},
ConversionReviewVersions: []string{"v1alpha1", "v1beta1"},
},
}
return nil
})
return err
})
}
//nolint:dupl
func (r CAReconciler) UpdateValidatingWebhookConfiguration(caBundle []byte) error {
return retry.RetryOnConflict(retry.DefaultBackoff, func() (err error) {
vw := &admissionregistrationv1.ValidatingWebhookConfiguration{}
err = r.Get(context.TODO(), types.NamespacedName{Name: r.Configuration.ValidatingWebhookConfigurationName()}, vw)
if err != nil {
r.Log.Error(err, "cannot retrieve ValidatingWebhookConfiguration")
return err
}
for i, w := range vw.Webhooks {
// Updating CABundle only in case of an internal service reference
if w.ClientConfig.Service != nil {
vw.Webhooks[i].ClientConfig.CABundle = caBundle
}
}
return r.Update(context.TODO(), vw, &client.UpdateOptions{})
})
}
//nolint:dupl
func (r CAReconciler) UpdateMutatingWebhookConfiguration(caBundle []byte) error {
return retry.RetryOnConflict(retry.DefaultBackoff, func() (err error) {
mw := &admissionregistrationv1.MutatingWebhookConfiguration{}
err = r.Get(context.TODO(), types.NamespacedName{Name: r.Configuration.MutatingWebhookConfigurationName()}, mw)
if err != nil {
r.Log.Error(err, "cannot retrieve MutatingWebhookConfiguration")
return err
}
for i, w := range mw.Webhooks {
// Updating CABundle only in case of an internal service reference
if w.ClientConfig.Service != nil {
mw.Webhooks[i].ClientConfig.CABundle = caBundle
}
}
return r.Update(context.TODO(), mw, &client.UpdateOptions{})
})
}
func (r CAReconciler) Reconcile(ctx context.Context, request ctrl.Request) (ctrl.Result, error) {
var err error
if request.Name != r.Configuration.CASecretName() {
return ctrl.Result{}, nil
}
r.Log = r.Log.WithValues("Request.Namespace", request.Namespace, "Request.Name", request.Name)
r.Log.Info("Reconciling CA Secret")
// Fetch the CA instance
instance := &corev1.Secret{}
err = r.Client.Get(context.TODO(), request.NamespacedName, instance)
if err != nil {
// Error reading the object - requeue the request.
return reconcile.Result{}, err
}
var ca cert.CA
var rq time.Duration
ca, err = getCertificateAuthority(r.Client, r.Namespace, r.Configuration.CASecretName())
if err != nil && errors.Is(err, MissingCaError{}) {
ca, err = cert.GenerateCertificateAuthority()
if err != nil {
return reconcile.Result{}, err
}
} else if err != nil {
return reconcile.Result{}, err
}
r.Log.Info("Handling CA Secret")
rq, err = ca.ExpiresIn(time.Now())
if err != nil {
r.Log.Info("CA is expired, cleaning to obtain a new one")
instance.Data = map[string][]byte{}
} else {
r.Log.Info("Updating CA secret with new PEM and RSA")
var crt *bytes.Buffer
var key *bytes.Buffer
crt, _ = ca.CACertificatePem()
key, _ = ca.CAPrivateKeyPem()
instance.Data = map[string][]byte{
certSecretKey: crt.Bytes(),
privateKeySecretKey: key.Bytes(),
}
group := new(errgroup.Group)
group.Go(func() error {
return r.UpdateMutatingWebhookConfiguration(crt.Bytes())
})
group.Go(func() error {
return r.UpdateValidatingWebhookConfiguration(crt.Bytes())
})
group.Go(func() error {
return r.UpdateCustomResourceDefinition(crt.Bytes())
})
if err = group.Wait(); err != nil {
return reconcile.Result{}, err
}
}
var res controllerutil.OperationResult
t := &corev1.Secret{ObjectMeta: instance.ObjectMeta}
res, err = controllerutil.CreateOrUpdate(context.TODO(), r.Client, t, func() error {
t.Data = instance.Data
return nil
})
if err != nil {
r.Log.Error(err, "cannot update Capsule TLS")
return reconcile.Result{}, err
}
if res == controllerutil.OperationResultUpdated {
r.Log.Info("Capsule CA has been updated, we need to trigger TLS update too")
tls := &corev1.Secret{}
err = r.Get(ctx, types.NamespacedName{
Namespace: r.Namespace,
Name: r.Configuration.TLSSecretName(),
}, tls)
if err != nil {
r.Log.Error(err, "Capsule TLS Secret missing")
}
err = retry.RetryOnConflict(retry.DefaultBackoff, func() error {
_, err = controllerutil.CreateOrUpdate(ctx, r.Client, tls, func() error {
tls.Data = map[string][]byte{}
return nil
})
return err
})
if err != nil {
r.Log.Error(err, "Cannot clean Capsule TLS Secret due to CA update")
return reconcile.Result{}, err
}
}
r.Log.Info("Reconciliation completed, processing back in " + rq.String())
return reconcile.Result{Requeue: true, RequeueAfter: rq}, nil
}

View File

@@ -1,9 +0,0 @@
// Copyright 2020-2021 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package secret
const (
certSecretKey = "tls.crt"
privateKeySecretKey = "tls.key"
)

View File

@@ -1,11 +0,0 @@
// Copyright 2020-2021 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package secret
type MissingCaError struct {
}
func (MissingCaError) Error() string {
return "CA has not been created yet, please generate a new"
}

View File

@@ -1,38 +0,0 @@
// Copyright 2020-2021 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package secret
import (
"context"
"fmt"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/types"
"sigs.k8s.io/controller-runtime/pkg/client"
"github.com/clastix/capsule/pkg/cert"
)
func getCertificateAuthority(client client.Client, namespace, name string) (ca cert.CA, err error) {
instance := &corev1.Secret{}
err = client.Get(context.TODO(), types.NamespacedName{
Namespace: namespace,
Name: name,
}, instance)
if err != nil {
return nil, fmt.Errorf("missing secret %s, cannot reconcile", name)
}
if instance.Data == nil {
return nil, MissingCaError{}
}
ca, err = cert.NewCertificateAuthorityFromBytes(instance.Data[certSecretKey], instance.Data[privateKeySecretKey])
if err != nil {
return
}
return
}

View File

@@ -1,158 +0,0 @@
// Copyright 2020-2021 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package secret
import (
"bytes"
"context"
"crypto/x509"
"encoding/pem"
"fmt"
"os"
"time"
"github.com/go-logr/logr"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/types"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
"sigs.k8s.io/controller-runtime/pkg/reconcile"
"github.com/clastix/capsule/pkg/cert"
"github.com/clastix/capsule/pkg/configuration"
)
type TLSReconciler struct {
client.Client
Log logr.Logger
Scheme *runtime.Scheme
Namespace string
Configuration configuration.Configuration
}
func (r *TLSReconciler) SetupWithManager(mgr ctrl.Manager) error {
return ctrl.NewControllerManagedBy(mgr).
For(&corev1.Secret{}).
Complete(r)
}
func (r TLSReconciler) Reconcile(ctx context.Context, request ctrl.Request) (ctrl.Result, error) {
var err error
if request.Name != r.Configuration.TLSSecretName() {
return ctrl.Result{}, nil
}
r.Log = r.Log.WithValues("Request.Namespace", request.Namespace, "Request.Name", request.Name)
r.Log.Info("Reconciling TLS Secret")
// Fetch the Secret instance
instance := &corev1.Secret{}
err = r.Get(ctx, request.NamespacedName, instance)
if err != nil {
// Error reading the object - requeue the request.
return reconcile.Result{}, err
}
var ca cert.CA
var rq time.Duration
ca, err = getCertificateAuthority(r.Client, r.Namespace, r.Configuration.CASecretName())
if err != nil {
return reconcile.Result{}, err
}
var shouldCreate bool
for _, key := range []string{certSecretKey, privateKeySecretKey} {
if _, ok := instance.Data[key]; !ok {
shouldCreate = true
break
}
}
if shouldCreate {
r.Log.Info("Missing Capsule TLS certificate")
rq = 6 * 30 * 24 * time.Hour
opts := cert.NewCertOpts(time.Now().Add(rq), fmt.Sprintf("capsule-webhook-service.%s.svc", r.Namespace))
var crt, key *bytes.Buffer
crt, key, err = ca.GenerateCertificate(opts)
if err != nil {
r.Log.Error(err, "Cannot generate new TLS certificate")
return reconcile.Result{}, err
}
instance.Data = map[string][]byte{
certSecretKey: crt.Bytes(),
privateKeySecretKey: key.Bytes(),
}
} else {
var c *x509.Certificate
var b *pem.Block
b, _ = pem.Decode(instance.Data[certSecretKey])
c, err = x509.ParseCertificate(b.Bytes)
if err != nil {
r.Log.Error(err, "cannot parse Capsule TLS")
return reconcile.Result{}, err
}
rq = time.Until(c.NotAfter)
err = ca.ValidateCert(c)
if err != nil {
r.Log.Info("Capsule TLS is expired or invalid, cleaning to obtain a new one")
instance.Data = map[string][]byte{}
}
}
var res controllerutil.OperationResult
t := &corev1.Secret{ObjectMeta: instance.ObjectMeta}
res, err = controllerutil.CreateOrUpdate(ctx, r.Client, t, func() error {
t.Data = instance.Data
return nil
})
if err != nil {
r.Log.Error(err, "cannot update Capsule TLS")
return reconcile.Result{}, err
}
if instance.Name == r.Configuration.TLSSecretName() && res == controllerutil.OperationResultUpdated {
r.Log.Info("Capsule TLS certificates has been updated, Controller pods must be restarted to load new certificate")
hostname, _ := os.Hostname()
leaderPod := &corev1.Pod{}
if err = r.Client.Get(ctx, types.NamespacedName{Namespace: os.Getenv("NAMESPACE"), Name: hostname}, leaderPod); err != nil {
r.Log.Error(err, "cannot retrieve the leader Pod, probably running in out of the cluster mode")
return reconcile.Result{}, nil
}
podList := &corev1.PodList{}
if err = r.Client.List(ctx, podList, client.MatchingLabels(leaderPod.ObjectMeta.Labels)); err != nil {
r.Log.Error(err, "cannot retrieve list of Capsule pods requiring restart upon TLS update")
return reconcile.Result{}, nil
}
for _, p := range podList.Items {
nonLeaderPod := p
// Skipping this Pod, must be deleted at the end
if nonLeaderPod.GetName() == leaderPod.GetName() {
continue
}
if err = r.Client.Delete(ctx, &nonLeaderPod); err != nil {
r.Log.Error(err, "cannot delete the non-leader Pod due to TLS update")
}
}
if err = r.Client.Delete(ctx, leaderPod); err != nil {
r.Log.Error(err, "cannot delete the leader Pod due to TLS update")
}
}
r.Log.Info("Reconciliation completed, processing back in " + rq.String())
return reconcile.Result{Requeue: true, RequeueAfter: rq}, nil
}

View File

@@ -8,16 +8,15 @@ import (
"fmt"
"github.com/go-logr/logr"
"github.com/pkg/errors"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/errors"
apierr "k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/fields"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/types"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/builder"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
"sigs.k8s.io/controller-runtime/pkg/event"
"sigs.k8s.io/controller-runtime/pkg/predicate"
"sigs.k8s.io/controller-runtime/pkg/reconcile"
@@ -28,37 +27,39 @@ type abstractServiceLabelsReconciler struct {
obj client.Object
client client.Client
log logr.Logger
scheme *runtime.Scheme
}
func (r *abstractServiceLabelsReconciler) InjectClient(c client.Client) error {
r.client = c
return nil
}
func (r *abstractServiceLabelsReconciler) Reconcile(ctx context.Context, request ctrl.Request) (ctrl.Result, error) {
tenant, err := r.getTenant(ctx, request.NamespacedName, r.client)
if err != nil {
switch err.(type) {
case *NonTenantObject, *NoServicesMetadata:
if errors.As(err, &NonTenantObjectError{}) || errors.As(err, &NoServicesMetadataError{}) {
return reconcile.Result{}, nil
default:
r.log.Error(err, fmt.Sprintf("Cannot sync %t labels", r.obj))
return reconcile.Result{}, err
}
r.log.Error(err, fmt.Sprintf("Cannot sync %T %s/%s labels", r.obj, r.obj.GetNamespace(), r.obj.GetName()))
return reconcile.Result{}, err
}
err = r.client.Get(ctx, request.NamespacedName, r.obj)
if err != nil {
if errors.IsNotFound(err) {
if apierr.IsNotFound(err) {
return reconcile.Result{}, nil
}
return reconcile.Result{}, err
}
_, err = controllerutil.CreateOrUpdate(ctx, r.client, r.obj, func() (err error) {
r.obj.SetLabels(r.sync(r.obj.GetLabels(), tenant.Spec.ServiceOptions.AdditionalMetadata.Labels))
r.obj.SetAnnotations(r.sync(r.obj.GetAnnotations(), tenant.Spec.ServiceOptions.AdditionalMetadata.Annotations))
return nil
})
@@ -101,32 +102,23 @@ func (r *abstractServiceLabelsReconciler) sync(available map[string]string, tena
}
}
}
return available
}
func (r *abstractServiceLabelsReconciler) forOptionPerInstanceName() builder.ForOption {
return builder.WithPredicates(predicate.Funcs{
CreateFunc: func(event event.CreateEvent) bool {
return r.IsNamespaceInTenant(event.Object.GetNamespace())
},
DeleteFunc: func(deleteEvent event.DeleteEvent) bool {
return r.IsNamespaceInTenant(deleteEvent.Object.GetNamespace())
},
UpdateFunc: func(updateEvent event.UpdateEvent) bool {
return r.IsNamespaceInTenant(updateEvent.ObjectNew.GetNamespace())
},
GenericFunc: func(genericEvent event.GenericEvent) bool {
return r.IsNamespaceInTenant(genericEvent.Object.GetNamespace())
},
})
func (r *abstractServiceLabelsReconciler) forOptionPerInstanceName(ctx context.Context) builder.ForOption {
return builder.WithPredicates(predicate.NewPredicateFuncs(func(object client.Object) bool {
return r.IsNamespaceInTenant(ctx, object.GetNamespace())
}))
}
func (r *abstractServiceLabelsReconciler) IsNamespaceInTenant(namespace string) bool {
func (r *abstractServiceLabelsReconciler) IsNamespaceInTenant(ctx context.Context, namespace string) bool {
tl := &capsulev1beta1.TenantList{}
if err := r.client.List(context.Background(), tl, client.MatchingFieldsSelector{
if err := r.client.List(ctx, tl, client.MatchingFieldsSelector{
Selector: fields.OneTermEqualSelector(".status.namespaces", namespace),
}); err != nil {
return false
}
return len(tl.Items) > 0
}

View File

@@ -4,6 +4,8 @@
package servicelabels
import (
"context"
"github.com/go-logr/logr"
corev1 "k8s.io/api/core/v1"
ctrl "sigs.k8s.io/controller-runtime"
@@ -15,14 +17,13 @@ type EndpointsLabelsReconciler struct {
Log logr.Logger
}
func (r *EndpointsLabelsReconciler) SetupWithManager(mgr ctrl.Manager) error {
func (r *EndpointsLabelsReconciler) SetupWithManager(ctx context.Context, mgr ctrl.Manager) error {
r.abstractServiceLabelsReconciler = abstractServiceLabelsReconciler{
obj: &corev1.Endpoints{},
scheme: mgr.GetScheme(),
log: r.Log,
obj: &corev1.Endpoints{},
log: r.Log,
}
return ctrl.NewControllerManagedBy(mgr).
For(r.abstractServiceLabelsReconciler.obj, r.abstractServiceLabelsReconciler.forOptionPerInstanceName()).
For(r.abstractServiceLabelsReconciler.obj, r.abstractServiceLabelsReconciler.forOptionPerInstanceName(ctx)).
Complete(r)
}

View File

@@ -4,6 +4,8 @@
package servicelabels
import (
"context"
"github.com/go-logr/logr"
discoveryv1 "k8s.io/api/discovery/v1"
discoveryv1beta1 "k8s.io/api/discovery/v1beta1"
@@ -18,16 +20,15 @@ type EndpointSlicesLabelsReconciler struct {
VersionMajor uint
}
func (r *EndpointSlicesLabelsReconciler) SetupWithManager(mgr ctrl.Manager) error {
r.scheme = mgr.GetScheme()
func (r *EndpointSlicesLabelsReconciler) SetupWithManager(ctx context.Context, mgr ctrl.Manager) error {
r.abstractServiceLabelsReconciler = abstractServiceLabelsReconciler{
scheme: mgr.GetScheme(),
log: r.Log,
log: r.Log,
}
switch {
case r.VersionMajor == 1 && r.VersionMinor <= 16:
r.Log.Info("Skipping controller setup, as EndpointSlices are not supported on current kubernetes version", "VersionMajor", r.VersionMajor, "VersionMinor", r.VersionMinor)
return nil
case r.VersionMajor == 1 && r.VersionMinor >= 21:
r.abstractServiceLabelsReconciler.obj = &discoveryv1.EndpointSlice{}
@@ -36,6 +37,6 @@ func (r *EndpointSlicesLabelsReconciler) SetupWithManager(mgr ctrl.Manager) erro
}
return ctrl.NewControllerManagedBy(mgr).
For(r.obj, r.abstractServiceLabelsReconciler.forOptionPerInstanceName()).
For(r.obj, r.abstractServiceLabelsReconciler.forOptionPerInstanceName(ctx)).
Complete(r)
}

View File

@@ -5,26 +5,26 @@ package servicelabels
import "fmt"
type NonTenantObject struct {
type NonTenantObjectError struct {
objectName string
}
func NewNonTenantObject(objectName string) error {
return &NonTenantObject{objectName: objectName}
return &NonTenantObjectError{objectName: objectName}
}
func (n NonTenantObject) Error() string {
func (n NonTenantObjectError) Error() string {
return fmt.Sprintf("Skipping labels sync for %s as it doesn't belong to tenant", n.objectName)
}
type NoServicesMetadata struct {
type NoServicesMetadataError struct {
objectName string
}
func NewNoServicesMetadata(objectName string) error {
return &NoServicesMetadata{objectName: objectName}
return &NoServicesMetadataError{objectName: objectName}
}
func (n NoServicesMetadata) Error() string {
func (n NoServicesMetadataError) Error() string {
return fmt.Sprintf("Skipping labels sync for %s because no AdditionalLabels or AdditionalAnnotations presents in Tenant spec", n.objectName)
}

View File

@@ -4,6 +4,8 @@
package servicelabels
import (
"context"
"github.com/go-logr/logr"
corev1 "k8s.io/api/core/v1"
ctrl "sigs.k8s.io/controller-runtime"
@@ -15,13 +17,13 @@ type ServicesLabelsReconciler struct {
Log logr.Logger
}
func (r *ServicesLabelsReconciler) SetupWithManager(mgr ctrl.Manager) error {
func (r *ServicesLabelsReconciler) SetupWithManager(ctx context.Context, mgr ctrl.Manager) error {
r.abstractServiceLabelsReconciler = abstractServiceLabelsReconciler{
obj: &corev1.Service{},
scheme: mgr.GetScheme(),
log: r.Log,
obj: &corev1.Service{},
log: r.Log,
}
return ctrl.NewControllerManagedBy(mgr).
For(r.abstractServiceLabelsReconciler.obj, r.abstractServiceLabelsReconciler.forOptionPerInstanceName()).
For(r.abstractServiceLabelsReconciler.obj, r.abstractServiceLabelsReconciler.forOptionPerInstanceName(ctx)).
Complete(r)
}

View File

@@ -0,0 +1,119 @@
package tenant
import (
"context"
"fmt"
"hash/fnv"
"golang.org/x/sync/errgroup"
rbacv1 "k8s.io/api/rbac/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
capsulev1beta1 "github.com/clastix/capsule/api/v1beta1"
)
// Sync the dynamic Tenant Owner specific cluster-roles and additional ClusterRole Bindings, which can be used in many ways:
// applying Pod Security Policies or giving access to CRDs or specific API groups.
func (r *Manager) syncClusterRoleBindings(ctx context.Context, tenant *capsulev1beta1.Tenant) (err error) {
// hashing the ClusterRoleBinding name due to DNS RFC-1123 applied to Kubernetes labels
hashFn := func(binding capsulev1beta1.AdditionalRoleBindingsSpec) string {
h := fnv.New64a()
_, _ = h.Write([]byte(binding.ClusterRoleName))
for _, sub := range binding.Subjects {
_, _ = h.Write([]byte(sub.Kind + sub.Name))
}
return fmt.Sprintf("%x", h.Sum64())
}
// getting requested Role Binding keys
keys := make([]string, 0, len(tenant.Spec.Owners))
// Generating for dynamic tenant owners cluster roles
for _, owner := range tenant.Spec.Owners {
for _, clusterRoleName := range owner.GetClusterRoles(*tenant) {
cr := r.ownerClusterRoleBindings(owner, clusterRoleName)
keys = append(keys, hashFn(cr))
}
}
group := new(errgroup.Group)
group.Go(func() error {
return r.syncClusterRoleBinding(ctx, tenant, keys, hashFn)
})
return group.Wait()
}
func (r *Manager) syncClusterRoleBinding(ctx context.Context, tenant *capsulev1beta1.Tenant, keys []string, hashFn func(binding capsulev1beta1.AdditionalRoleBindingsSpec) string) (err error) {
var tenantLabel string
var clusterRoleBindingLabel string
if tenantLabel, err = capsulev1beta1.GetTypeLabel(&capsulev1beta1.Tenant{}); err != nil {
return
}
if clusterRoleBindingLabel, err = capsulev1beta1.GetTypeLabel(&rbacv1.ClusterRoleBinding{}); err != nil {
return
}
if err = r.pruningClusterResources(ctx, keys, &rbacv1.ClusterRoleBinding{}); err != nil {
return
}
var clusterRoleBindings []capsulev1beta1.AdditionalRoleBindingsSpec
for _, owner := range tenant.Spec.Owners {
for _, clusterRoleName := range owner.GetClusterRoles(*tenant) {
clusterRoleBindings = append(clusterRoleBindings, r.ownerClusterRoleBindings(owner, clusterRoleName))
}
}
for i, clusterRoleBinding := range clusterRoleBindings {
clusterRoleBindingHashLabel := hashFn(clusterRoleBinding)
target := &rbacv1.ClusterRoleBinding{
ObjectMeta: metav1.ObjectMeta{
Name: fmt.Sprintf("capsule-%s-%d-%s", tenant.Name, i, clusterRoleBinding.ClusterRoleName),
},
}
var res controllerutil.OperationResult
res, err = controllerutil.CreateOrUpdate(ctx, r.Client, target, func() error {
target.ObjectMeta.Labels = map[string]string{
tenantLabel: tenant.Name,
clusterRoleBindingLabel: clusterRoleBindingHashLabel,
}
target.RoleRef = rbacv1.RoleRef{
APIGroup: rbacv1.GroupName,
Kind: "ClusterRole",
Name: clusterRoleBinding.ClusterRoleName,
}
target.Subjects = clusterRoleBinding.Subjects
return controllerutil.SetControllerReference(tenant, target, r.Client.Scheme())
})
// TODO: find appropriate event Namespace.
r.emitEvent(tenant, target.GetNamespace(), res, fmt.Sprintf("Ensuring ClusterRoleBinding %s", target.GetName()), err)
if err != nil {
r.Log.Error(err, "Cannot sync ClusterRoleBinding")
}
r.Log.Info(fmt.Sprintf("ClusterRoleBinding sync result: %s", string(res)), "name", target.Name, "namespace", target.Namespace)
if err != nil {
return
}
}
return nil
}

View File

@@ -0,0 +1,66 @@
package tenant
import (
"context"
rbacv1 "k8s.io/api/rbac/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
capsulev1beta1 "github.com/clastix/capsule/api/v1beta1"
)
const (
ImpersonatorRoleName = "capsule-tenant-impersonator"
)
// Sync the Tenant Owner specific cluster-roles.
// When the Tenant is configured GitOpsReady additional (Cluster)Roles are created, then bound.
func (r *Manager) syncClusterRoles(ctx context.Context, tenant *capsulev1beta1.Tenant) (err error) {
// If the Tenant will be reconciled the GitOps-way,
// Tenant Owners might be machine GitOps reconciler identities.
if tenant.Spec.GitOpsReady {
for _, owner := range tenant.Spec.Owners {
if err = r.ensureOwnerClusterRole(ctx, tenant, &owner, ImpersonatorRoleName); err != nil {
r.Log.Error(err, "Reconciliation for ClusterRole failed", "ClusterRole", ImpersonatorRoleName)
return err
}
}
}
return
}
func (r *Manager) ensureOwnerClusterRole(ctx context.Context, tenant *capsulev1beta1.Tenant, owner *capsulev1beta1.OwnerSpec, roleName string) (err error) {
switch roleName {
case ImpersonatorRoleName:
clusterRole := &rbacv1.ClusterRole{
ObjectMeta: metav1.ObjectMeta{
Name: roleName + "-" + tenant.Name + "-" + owner.Name,
},
}
resource := "users"
if owner.Kind == capsulev1beta1.GroupOwner {
resource = "groups"
}
resourceName := owner.Name
_, err = controllerutil.CreateOrUpdate(ctx, r.Client, clusterRole, func() error {
clusterRole.Rules = []rbacv1.PolicyRule{
{
APIGroups: []string{""},
Resources: []string{resource},
Verbs: []string{"impersonate"},
ResourceNames: []string{resourceName},
},
}
return nil
})
}
return
}

View File

@@ -13,8 +13,9 @@ import (
capsulev1beta1 "github.com/clastix/capsule/api/v1beta1"
)
// nolint:dupl
// Ensuring all the LimitRange are applied to each Namespace handled by the Tenant.
func (r *Manager) syncLimitRanges(tenant *capsulev1beta1.Tenant) error {
func (r *Manager) syncLimitRanges(ctx context.Context, tenant *capsulev1beta1.Tenant) error {
// getting requested LimitRange keys
keys := make([]string, 0, len(tenant.Spec.LimitRanges.Items))
@@ -28,26 +29,27 @@ func (r *Manager) syncLimitRanges(tenant *capsulev1beta1.Tenant) error {
namespace := ns
group.Go(func() error {
return r.syncLimitRange(tenant, namespace, keys)
return r.syncLimitRange(ctx, tenant, namespace, keys)
})
}
return group.Wait()
}
func (r *Manager) syncLimitRange(tenant *capsulev1beta1.Tenant, namespace string, keys []string) (err error) {
func (r *Manager) syncLimitRange(ctx context.Context, tenant *capsulev1beta1.Tenant, namespace string, keys []string) (err error) {
// getting LimitRange labels for the mutateFn
var tenantLabel, limitRangeLabel string
if tenantLabel, err = capsulev1beta1.GetTypeLabel(&capsulev1beta1.Tenant{}); err != nil {
return
}
if limitRangeLabel, err = capsulev1beta1.GetTypeLabel(&corev1.LimitRange{}); err != nil {
return
return err
}
if err = r.pruningResources(namespace, keys, &corev1.LimitRange{}); err != nil {
return
if limitRangeLabel, err = capsulev1beta1.GetTypeLabel(&corev1.LimitRange{}); err != nil {
return err
}
if err = r.pruningResources(ctx, namespace, keys, &corev1.LimitRange{}); err != nil {
return err
}
for i, spec := range tenant.Spec.LimitRanges.Items {
@@ -59,22 +61,24 @@ func (r *Manager) syncLimitRange(tenant *capsulev1beta1.Tenant, namespace string
}
var res controllerutil.OperationResult
res, err = controllerutil.CreateOrUpdate(context.TODO(), r.Client, target, func() (err error) {
res, err = controllerutil.CreateOrUpdate(ctx, r.Client, target, func() (err error) {
target.ObjectMeta.Labels = map[string]string{
tenantLabel: tenant.Name,
limitRangeLabel: strconv.Itoa(i),
}
target.Spec = spec
return controllerutil.SetControllerReference(tenant, target, r.Scheme)
return controllerutil.SetControllerReference(tenant, target, r.Client.Scheme())
})
r.emitEvent(tenant, target.GetNamespace(), res, fmt.Sprintf("Ensuring LimitRange %s", target.GetName()), err)
r.Log.Info("LimitRange sync result: "+string(res), "name", target.Name, "namespace", target.Namespace)
if err != nil {
return
return err
}
}
return
return nil
}

View File

@@ -7,8 +7,7 @@ import (
corev1 "k8s.io/api/core/v1"
networkingv1 "k8s.io/api/networking/v1"
rbacv1 "k8s.io/api/rbac/v1"
"k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/runtime"
apierrors "k8s.io/apimachinery/pkg/api/errors"
"k8s.io/client-go/rest"
"k8s.io/client-go/tools/record"
"k8s.io/client-go/util/retry"
@@ -22,7 +21,6 @@ import (
type Manager struct {
client.Client
Log logr.Logger
Scheme *runtime.Scheme
Recorder record.EventRecorder
RESTConfig *rest.Config
}
@@ -40,83 +38,112 @@ func (r *Manager) SetupWithManager(mgr ctrl.Manager) error {
func (r Manager) Reconcile(ctx context.Context, request ctrl.Request) (result ctrl.Result, err error) {
r.Log = r.Log.WithValues("Request.Name", request.Name)
// Fetch the Tenant instance
instance := &capsulev1beta1.Tenant{}
if err = r.Get(ctx, request.NamespacedName, instance); err != nil {
if errors.IsNotFound(err) {
if apierrors.IsNotFound(err) {
r.Log.Info("Request object not found, could have been deleted after reconcile request")
return reconcile.Result{}, nil
}
r.Log.Error(err, "Error reading the object")
return
}
// Ensuring the Tenant Status
if err = r.updateTenantStatus(instance); err != nil {
if err = r.updateTenantStatus(ctx, instance); err != nil {
r.Log.Error(err, "Cannot update Tenant status")
return
}
// Ensuring ResourceQuota
r.Log.Info("Ensuring limit resources count is updated")
if err = r.syncCustomResourceQuotaUsages(ctx, instance); err != nil {
r.Log.Error(err, "Cannot count limited resources")
return
}
// Ensuring all namespaces are collected
r.Log.Info("Ensuring all Namespaces are collected")
if err = r.collectNamespaces(instance); err != nil {
if err = r.collectNamespaces(ctx, instance); err != nil {
r.Log.Error(err, "Cannot collect Namespace resources")
return
}
// Ensuring Namespace metadata
r.Log.Info("Starting processing of Namespaces", "items", len(instance.Status.Namespaces))
if err = r.syncNamespaces(instance); err != nil {
if err = r.syncNamespaces(ctx, instance); err != nil {
r.Log.Error(err, "Cannot sync Namespace items")
return
}
// Ensuring NetworkPolicy resources
r.Log.Info("Starting processing of Network Policies")
if err = r.syncNetworkPolicies(instance); err != nil {
if err = r.syncNetworkPolicies(ctx, instance); err != nil {
r.Log.Error(err, "Cannot sync NetworkPolicy items")
return
}
// Ensuring LimitRange resources
r.Log.Info("Starting processing of Limit Ranges", "items", len(instance.Spec.LimitRanges.Items))
if err = r.syncLimitRanges(instance); err != nil {
if err = r.syncLimitRanges(ctx, instance); err != nil {
r.Log.Error(err, "Cannot sync LimitRange items")
return
}
// Ensuring ResourceQuota resources
r.Log.Info("Starting processing of Resource Quotas", "items", len(instance.Spec.ResourceQuota.Items))
if err = r.syncResourceQuotas(instance); err != nil {
if err = r.syncResourceQuotas(ctx, instance); err != nil {
r.Log.Error(err, "Cannot sync ResourceQuota items")
return
}
// Ensuring ClusterRoles resources
r.Log.Info("Ensuring ClusterRoles for Owners and Tenant")
if err = r.syncClusterRoles(ctx, instance); err != nil {
r.Log.Error(err, "Cannot sync ClusterRoles items")
r.Log.Info("Ensuring additional RoleBindings for owner")
if err = r.syncAdditionalRoleBindings(instance); err != nil {
r.Log.Error(err, "Cannot sync additional RoleBindings items")
return
}
// Ensuring ClusterRoleBindings resources
r.Log.Info("Ensuring ClusterRoleBindings for Owners and Tenant")
if err = r.syncClusterRoleBindings(ctx, instance); err != nil {
r.Log.Error(err, "Cannot sync ClusterRoleBindings items")
r.Log.Info("Ensuring RoleBinding for owner")
if err = r.ownerRoleBinding(instance); err != nil {
r.Log.Error(err, "Cannot sync owner RoleBinding")
return
}
// Ensuring RoleBinding resources
r.Log.Info("Ensuring RoleBindings for Owners and Tenant")
if err = r.syncRoleBindings(ctx, instance); err != nil {
r.Log.Error(err, "Cannot sync RoleBindings items")
return
}
// Ensuring Namespace count
r.Log.Info("Ensuring Namespace count")
if err = r.ensureNamespaceCount(instance); err != nil {
if err = r.ensureNamespaceCount(ctx, instance); err != nil {
r.Log.Error(err, "Cannot sync Namespace count")
return
}
r.Log.Info("Tenant reconciling completed")
return ctrl.Result{}, err
}
func (r *Manager) updateTenantStatus(tnt *capsulev1beta1.Tenant) error {
func (r *Manager) updateTenantStatus(ctx context.Context, tnt *capsulev1beta1.Tenant) error {
return retry.RetryOnConflict(retry.DefaultBackoff, func() (err error) {
if tnt.IsCordoned() {
tnt.Status.State = capsulev1beta1.TenantStateCordoned
@@ -124,6 +151,6 @@ func (r *Manager) updateTenantStatus(tnt *capsulev1beta1.Tenant) error {
tnt.Status.State = capsulev1beta1.TenantStateActive
}
return r.Client.Status().Update(context.Background(), tnt)
return r.Client.Status().Update(ctx, tnt)
})
}

View File

@@ -20,37 +20,39 @@ import (
)
// Ensuring all annotations are applied to each Namespace handled by the Tenant.
func (r *Manager) syncNamespaces(tenant *capsulev1beta1.Tenant) (err error) {
func (r *Manager) syncNamespaces(ctx context.Context, tenant *capsulev1beta1.Tenant) (err error) {
group := new(errgroup.Group)
for _, item := range tenant.Status.Namespaces {
namespace := item
group.Go(func() error {
return r.syncNamespaceMetadata(namespace, tenant)
return r.syncNamespaceMetadata(ctx, namespace, tenant)
})
}
if err = group.Wait(); err != nil {
r.Log.Error(err, "Cannot sync Namespaces")
err = fmt.Errorf("cannot sync Namespaces: %s", err.Error())
err = fmt.Errorf("cannot sync Namespaces: %w", err)
}
return
}
func (r *Manager) syncNamespaceMetadata(namespace string, tnt *capsulev1beta1.Tenant) (err error) {
// nolint:gocognit
func (r *Manager) syncNamespaceMetadata(ctx context.Context, namespace string, tnt *capsulev1beta1.Tenant) (err error) {
var res controllerutil.OperationResult
err = retry.RetryOnConflict(retry.DefaultBackoff, func() (conflictErr error) {
ns := &corev1.Namespace{}
if conflictErr = r.Client.Get(context.TODO(), types.NamespacedName{Name: namespace}, ns); err != nil {
if conflictErr = r.Client.Get(ctx, types.NamespacedName{Name: namespace}, ns); err != nil {
return
}
capsuleLabel, _ := capsulev1beta1.GetTypeLabel(&capsulev1beta1.Tenant{})
res, conflictErr = controllerutil.CreateOrUpdate(context.TODO(), r.Client, ns, func() error {
res, conflictErr = controllerutil.CreateOrUpdate(ctx, r.Client, ns, func() error {
annotations := make(map[string]string)
labels := map[string]string{
"name": namespace,
@@ -144,28 +146,28 @@ func (r *Manager) syncNamespaceMetadata(namespace string, tnt *capsulev1beta1.Te
r.emitEvent(tnt, namespace, res, "Ensuring Namespace metadata", err)
return
return err
}
func (r *Manager) ensureNamespaceCount(tenant *capsulev1beta1.Tenant) error {
func (r *Manager) ensureNamespaceCount(ctx context.Context, tenant *capsulev1beta1.Tenant) error {
return retry.RetryOnConflict(retry.DefaultBackoff, func() error {
tenant.Status.Size = uint(len(tenant.Status.Namespaces))
found := &capsulev1beta1.Tenant{}
if err := r.Client.Get(context.TODO(), types.NamespacedName{Name: tenant.GetName()}, found); err != nil {
if err := r.Client.Get(ctx, types.NamespacedName{Name: tenant.GetName()}, found); err != nil {
return err
}
found.Status.Size = tenant.Status.Size
return r.Client.Status().Update(context.TODO(), found, &client.UpdateOptions{})
return r.Client.Status().Update(ctx, found, &client.UpdateOptions{})
})
}
func (r *Manager) collectNamespaces(tenant *capsulev1beta1.Tenant) error {
func (r *Manager) collectNamespaces(ctx context.Context, tenant *capsulev1beta1.Tenant) error {
return retry.RetryOnConflict(retry.DefaultBackoff, func() (err error) {
list := &corev1.NamespaceList{}
err = r.Client.List(context.TODO(), list, client.MatchingFieldsSelector{
err = r.Client.List(ctx, list, client.MatchingFieldsSelector{
Selector: fields.OneTermEqualSelector(".metadata.ownerReferences[*].capsule", tenant.GetName()),
})
@@ -173,11 +175,12 @@ func (r *Manager) collectNamespaces(tenant *capsulev1beta1.Tenant) error {
return
}
_, err = controllerutil.CreateOrUpdate(context.TODO(), r.Client, tenant.DeepCopy(), func() error {
_, err = controllerutil.CreateOrUpdate(ctx, r.Client, tenant.DeepCopy(), func() error {
tenant.AssignNamespaces(list.Items)
return r.Client.Status().Update(context.TODO(), tenant, &client.UpdateOptions{})
return r.Client.Status().Update(ctx, tenant, &client.UpdateOptions{})
})
return
})
}

View File

@@ -13,8 +13,9 @@ import (
capsulev1beta1 "github.com/clastix/capsule/api/v1beta1"
)
// nolint:dupl
// Ensuring all the NetworkPolicies are applied to each Namespace handled by the Tenant.
func (r *Manager) syncNetworkPolicies(tenant *capsulev1beta1.Tenant) error {
func (r *Manager) syncNetworkPolicies(ctx context.Context, tenant *capsulev1beta1.Tenant) error {
// getting requested NetworkPolicy keys
keys := make([]string, 0, len(tenant.Spec.NetworkPolicies.Items))
@@ -28,26 +29,26 @@ func (r *Manager) syncNetworkPolicies(tenant *capsulev1beta1.Tenant) error {
namespace := ns
group.Go(func() error {
return r.syncNetworkPolicy(tenant, namespace, keys)
return r.syncNetworkPolicy(ctx, tenant, namespace, keys)
})
}
return group.Wait()
}
func (r *Manager) syncNetworkPolicy(tenant *capsulev1beta1.Tenant, namespace string, keys []string) (err error) {
if err = r.pruningResources(namespace, keys, &networkingv1.NetworkPolicy{}); err != nil {
return
func (r *Manager) syncNetworkPolicy(ctx context.Context, tenant *capsulev1beta1.Tenant, namespace string, keys []string) (err error) {
if err = r.pruningResources(ctx, namespace, keys, &networkingv1.NetworkPolicy{}); err != nil {
return err
}
// getting NetworkPolicy labels for the mutateFn
var tenantLabel, networkPolicyLabel string
if tenantLabel, err = capsulev1beta1.GetTypeLabel(&capsulev1beta1.Tenant{}); err != nil {
return
return err
}
if networkPolicyLabel, err = capsulev1beta1.GetTypeLabel(&networkingv1.NetworkPolicy{}); err != nil {
return
return err
}
for i, spec := range tenant.Spec.NetworkPolicies.Items {
@@ -59,14 +60,14 @@ func (r *Manager) syncNetworkPolicy(tenant *capsulev1beta1.Tenant, namespace str
}
var res controllerutil.OperationResult
res, err = controllerutil.CreateOrUpdate(context.TODO(), r.Client, target, func() (err error) {
res, err = controllerutil.CreateOrUpdate(ctx, r.Client, target, func() (err error) {
target.SetLabels(map[string]string{
tenantLabel: tenant.Name,
networkPolicyLabel: strconv.Itoa(i),
})
target.Spec = spec
return controllerutil.SetControllerReference(tenant, target, r.Scheme)
return controllerutil.SetControllerReference(tenant, target, r.Client.Scheme())
})
r.emitEvent(tenant, target.GetNamespace(), res, fmt.Sprintf("Ensuring NetworkPolicy %s", target.GetName()), err)
@@ -74,9 +75,9 @@ func (r *Manager) syncNetworkPolicy(tenant *capsulev1beta1.Tenant, namespace str
r.Log.Info("Network Policy sync result: "+string(res), "name", target.Name, "namespace", target.Namespace)
if err != nil {
return
return err
}
}
return
return nil
}

View File

@@ -31,7 +31,8 @@ import (
// the mutateFn along with the CreateOrUpdate to don't perform the update since resources are identical.
//
// In case of Namespace-scoped Resource Budget, we're just replicating the resources across all registered Namespaces.
func (r *Manager) syncResourceQuotas(tenant *capsulev1beta1.Tenant) (err error) {
// nolint:gocognit
func (r *Manager) syncResourceQuotas(ctx context.Context, tenant *capsulev1beta1.Tenant) (err error) {
// getting ResourceQuota labels for the mutateFn
var tenantLabel, typeLabel string
@@ -42,7 +43,7 @@ func (r *Manager) syncResourceQuotas(tenant *capsulev1beta1.Tenant) (err error)
if typeLabel, err = capsulev1beta1.GetTypeLabel(&corev1.ResourceQuota{}); err != nil {
return err
}
// nolint:nestif
if tenant.Spec.ResourceQuota.Scope == capsulev1beta1.ResourceQuotaScopeTenant {
group := new(errgroup.Group)
@@ -67,8 +68,9 @@ func (r *Manager) syncResourceQuotas(tenant *capsulev1beta1.Tenant) (err error)
// These are required since Capsule is going to sum all the used quota to
// sum them and get the Tenant one.
list := &corev1.ResourceQuotaList{}
if scopeErr = r.List(context.TODO(), list, &client.ListOptions{LabelSelector: labels.NewSelector().Add(*tntRequirement).Add(*indexRequirement)}); scopeErr != nil {
if scopeErr = r.List(ctx, list, &client.ListOptions{LabelSelector: labels.NewSelector().Add(*tntRequirement).Add(*indexRequirement)}); scopeErr != nil {
r.Log.Error(scopeErr, "Cannot list ResourceQuota", "tenantFilter", tntRequirement.String(), "indexFilter", indexRequirement.String())
return
}
// Iterating over all the options declared for the ResourceQuota,
@@ -116,11 +118,13 @@ func (r *Manager) syncResourceQuotas(tenant *capsulev1beta1.Tenant) (err error)
list.Items[item].Spec.Hard[name] = resourceQuota.Hard[name]
}
}
if scopeErr = r.resourceQuotasUpdate(name, quantity, resourceQuota.Hard[name], list.Items...); scopeErr != nil {
if scopeErr = r.resourceQuotasUpdate(ctx, name, quantity, resourceQuota.Hard[name], list.Items...); scopeErr != nil {
r.Log.Error(scopeErr, "cannot proceed with outer ResourceQuota")
return
}
}
return
})
}
@@ -142,14 +146,14 @@ func (r *Manager) syncResourceQuotas(tenant *capsulev1beta1.Tenant) (err error)
namespace := ns
group.Go(func() error {
return r.syncResourceQuota(tenant, namespace, keys)
return r.syncResourceQuota(ctx, tenant, namespace, keys)
})
}
return group.Wait()
}
func (r *Manager) syncResourceQuota(tenant *capsulev1beta1.Tenant, namespace string, keys []string) (err error) {
func (r *Manager) syncResourceQuota(ctx context.Context, tenant *capsulev1beta1.Tenant, namespace string, keys []string) (err error) {
// getting ResourceQuota labels for the mutateFn
var tenantLabel, typeLabel string
@@ -161,7 +165,7 @@ func (r *Manager) syncResourceQuota(tenant *capsulev1beta1.Tenant, namespace str
return err
}
// Pruning resource of non-requested resources
if err = r.pruningResources(namespace, keys, &corev1.ResourceQuota{}); err != nil {
if err = r.pruningResources(ctx, namespace, keys, &corev1.ResourceQuota{}); err != nil {
return err
}
@@ -174,8 +178,9 @@ func (r *Manager) syncResourceQuota(tenant *capsulev1beta1.Tenant, namespace str
}
var res controllerutil.OperationResult
err = retry.RetryOnConflict(retry.DefaultBackoff, func() (retryErr error) {
res, retryErr = controllerutil.CreateOrUpdate(context.TODO(), r.Client, target, func() (err error) {
res, retryErr = controllerutil.CreateOrUpdate(ctx, r.Client, target, func() (err error) {
target.SetLabels(map[string]string{
tenantLabel: tenant.Name,
typeLabel: strconv.Itoa(index),
@@ -187,7 +192,7 @@ func (r *Manager) syncResourceQuota(tenant *capsulev1beta1.Tenant, namespace str
target.Spec.Hard = resQuota.Hard
}
return controllerutil.SetControllerReference(tenant, target, r.Scheme)
return controllerutil.SetControllerReference(tenant, target, r.Client.Scheme())
})
return retryErr
@@ -208,7 +213,7 @@ func (r *Manager) syncResourceQuota(tenant *capsulev1beta1.Tenant, namespace str
// Serial ResourceQuota processing is expensive: using Go routines we can speed it up.
// In case of multiple errors these are logged properly, returning a generic error since we have to repush back the
// reconciliation loop.
func (r *Manager) resourceQuotasUpdate(resourceName corev1.ResourceName, actual, limit resource.Quantity, list ...corev1.ResourceQuota) (err error) {
func (r *Manager) resourceQuotasUpdate(ctx context.Context, resourceName corev1.ResourceName, actual, limit resource.Quantity, list ...corev1.ResourceQuota) (err error) {
group := new(errgroup.Group)
for _, item := range list {
@@ -216,12 +221,12 @@ func (r *Manager) resourceQuotasUpdate(resourceName corev1.ResourceName, actual,
group.Go(func() (err error) {
found := &corev1.ResourceQuota{}
if err = r.Get(context.TODO(), types.NamespacedName{Namespace: rq.Namespace, Name: rq.Name}, found); err != nil {
if err = r.Get(ctx, types.NamespacedName{Namespace: rq.Namespace, Name: rq.Name}, found); err != nil {
return
}
return retry.RetryOnConflict(retry.DefaultBackoff, func() (retryErr error) {
_, retryErr = controllerutil.CreateOrUpdate(context.TODO(), r.Client, found, func() error {
_, retryErr = controllerutil.CreateOrUpdate(ctx, r.Client, found, func() error {
// Ensuring annotation map is there to avoid uninitialized map error and
// assigning the overall usage
if found.Annotations == nil {
@@ -232,6 +237,7 @@ func (r *Manager) resourceQuotasUpdate(resourceName corev1.ResourceName, actual,
found.Annotations[capsulev1beta1.HardQuotaFor(resourceName)] = limit.String()
// Updating the Resource according to the actual.Cmp result
found.Spec.Hard = rq.Spec.Hard
return nil
})
@@ -244,7 +250,7 @@ func (r *Manager) resourceQuotasUpdate(resourceName corev1.ResourceName, actual,
// We had an error and we mark the whole transaction as failed
// to process it another time according to the Tenant controller back-off factor.
r.Log.Error(err, "Cannot update outer ResourceQuotas", "resourceName", resourceName.String())
err = fmt.Errorf("update of outer ResourceQuota items has failed: %s", err.Error())
err = fmt.Errorf("update of outer ResourceQuota items has failed: %w", err)
}
return err

View File

@@ -22,7 +22,7 @@ func (r *Manager) syncCustomResourceQuotaUsages(ctx context.Context, tenant *cap
group string
version string
}
// nolint:prealloc
var resourceList []resource
for k := range tenant.GetAnnotations() {

View File

@@ -9,16 +9,43 @@ import (
"golang.org/x/sync/errgroup"
rbacv1 "k8s.io/api/rbac/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/types"
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
capsulev1beta1 "github.com/clastix/capsule/api/v1beta1"
"github.com/clastix/capsule/controllers/rbac"
)
// Additional Role Bindings can be used in many ways: applying Pod Security Policies or giving
// access to CRDs or specific API groups.
func (r *Manager) syncAdditionalRoleBindings(tenant *capsulev1beta1.Tenant) (err error) {
// ownerClusterRoleBindings generates a Capsule AdditionalRoleBinding object for the Owner dynamic clusterrole in order
// to take advantage of the additional role binding feature.
func (r *Manager) ownerClusterRoleBindings(owner capsulev1beta1.OwnerSpec, clusterRole string) capsulev1beta1.AdditionalRoleBindingsSpec {
var subject rbacv1.Subject
if owner.Kind == "ServiceAccount" {
splitName := strings.Split(owner.Name, ":")
subject = rbacv1.Subject{
Kind: owner.Kind.String(),
Name: splitName[len(splitName)-1],
Namespace: splitName[len(splitName)-2],
}
} else {
subject = rbacv1.Subject{
APIGroup: rbacv1.GroupName,
Kind: owner.Kind.String(),
Name: owner.Name,
}
}
return capsulev1beta1.AdditionalRoleBindingsSpec{
ClusterRoleName: clusterRole,
Subjects: []rbacv1.Subject{
subject,
},
}
}
// Sync the dynamic Tenant Owner specific cluster-roles and additional Role Bindings, which can be used in many ways:
// applying Pod Security Policies or giving access to CRDs or specific API groups.
func (r *Manager) syncRoleBindings(ctx context.Context, tenant *capsulev1beta1.Tenant) (err error) {
// hashing the RoleBinding name due to DNS RFC-1123 applied to Kubernetes labels
hashFn := func(binding capsulev1beta1.AdditionalRoleBindingsSpec) string {
h := fnv.New64a()
@@ -32,7 +59,16 @@ func (r *Manager) syncAdditionalRoleBindings(tenant *capsulev1beta1.Tenant) (err
return fmt.Sprintf("%x", h.Sum64())
}
// getting requested Role Binding keys
var keys []string
keys := make([]string, 0, len(tenant.Spec.Owners))
// Generating for dynamic tenant owners cluster roles
for index, owner := range tenant.Spec.Owners {
for _, clusterRoleName := range owner.GetRoles(*tenant, index) {
cr := r.ownerClusterRoleBindings(owner, clusterRoleName)
keys = append(keys, hashFn(cr))
}
}
// Generating hash of additional role bindings
for _, i := range tenant.Spec.AdditionalRoleBindings {
keys = append(keys, hashFn(i))
}
@@ -43,14 +79,14 @@ func (r *Manager) syncAdditionalRoleBindings(tenant *capsulev1beta1.Tenant) (err
namespace := ns
group.Go(func() error {
return r.syncAdditionalRoleBinding(tenant, namespace, keys, hashFn)
return r.syncAdditionalRoleBinding(ctx, tenant, namespace, keys, hashFn)
})
}
return group.Wait()
}
func (r *Manager) syncAdditionalRoleBinding(tenant *capsulev1beta1.Tenant, ns string, keys []string, hashFn func(binding capsulev1beta1.AdditionalRoleBindingsSpec) string) (err error) {
func (r *Manager) syncAdditionalRoleBinding(ctx context.Context, tenant *capsulev1beta1.Tenant, ns string, keys []string, hashFn func(binding capsulev1beta1.AdditionalRoleBindingsSpec) string) (err error) {
var tenantLabel, roleBindingLabel string
if tenantLabel, err = capsulev1beta1.GetTypeLabel(&capsulev1beta1.Tenant{}); err != nil {
@@ -61,11 +97,21 @@ func (r *Manager) syncAdditionalRoleBinding(tenant *capsulev1beta1.Tenant, ns st
return
}
if err = r.pruningResources(ns, keys, &rbacv1.RoleBinding{}); err != nil {
if err = r.pruningResources(ctx, ns, keys, &rbacv1.RoleBinding{}); err != nil {
return
}
for i, roleBinding := range tenant.Spec.AdditionalRoleBindings {
var roleBindings []capsulev1beta1.AdditionalRoleBindingsSpec
for index, owner := range tenant.Spec.Owners {
for _, clusterRoleName := range owner.GetRoles(*tenant, index) {
roleBindings = append(roleBindings, r.ownerClusterRoleBindings(owner, clusterRoleName))
}
}
roleBindings = append(roleBindings, tenant.Spec.AdditionalRoleBindings...)
for i, roleBinding := range roleBindings {
roleBindingHashLabel := hashFn(roleBinding)
target := &rbacv1.RoleBinding{
@@ -76,27 +122,29 @@ func (r *Manager) syncAdditionalRoleBinding(tenant *capsulev1beta1.Tenant, ns st
}
var res controllerutil.OperationResult
res, err = controllerutil.CreateOrUpdate(context.TODO(), r.Client, target, func() error {
res, err = controllerutil.CreateOrUpdate(ctx, r.Client, target, func() error {
target.ObjectMeta.Labels = map[string]string{
tenantLabel: tenant.Name,
roleBindingLabel: roleBindingHashLabel,
}
target.RoleRef = rbacv1.RoleRef{
APIGroup: "rbac.authorization.k8s.io",
APIGroup: rbacv1.GroupName,
Kind: "ClusterRole",
Name: roleBinding.ClusterRoleName,
}
target.Subjects = roleBinding.Subjects
return controllerutil.SetControllerReference(tenant, target, r.Scheme)
return controllerutil.SetControllerReference(tenant, target, r.Client.Scheme())
})
r.emitEvent(tenant, target.GetNamespace(), res, fmt.Sprintf("Ensuring additional RoleBinding %s", target.GetName()), err)
r.emitEvent(tenant, target.GetNamespace(), res, fmt.Sprintf("Ensuring RoleBinding %s", target.GetName()), err)
if err != nil {
r.Log.Error(err, "Cannot sync Additional RoleBinding")
r.Log.Error(err, "Cannot sync RoleBinding")
}
r.Log.Info(fmt.Sprintf("Additional RoleBindings sync result: %s", string(res)), "name", target.Name, "namespace", target.Namespace)
r.Log.Info(fmt.Sprintf("RoleBinding sync result: %s", string(res)), "name", target.Name, "namespace", target.Namespace)
if err != nil {
return
}
@@ -104,76 +152,3 @@ func (r *Manager) syncAdditionalRoleBinding(tenant *capsulev1beta1.Tenant, ns st
return nil
}
// Each Tenant owner needs the admin Role attached to each Namespace, otherwise no actions on it can be performed.
// Since RBAC is based on deny all first, some specific actions like editing Capsule resources are going to be blocked
// via Dynamic Admission Webhooks.
// TODO(prometherion): we could create a capsule:admin role rather than hitting webhooks for each action
func (r *Manager) ownerRoleBinding(tenant *capsulev1beta1.Tenant) error {
// getting RoleBinding label for the mutateFn
var subjects []rbacv1.Subject
tl, err := capsulev1beta1.GetTypeLabel(&capsulev1beta1.Tenant{})
if err != nil {
return err
}
newLabels := map[string]string{tl: tenant.Name}
for _, owner := range tenant.Spec.Owners {
if owner.Kind == "ServiceAccount" {
splitName := strings.Split(owner.Name, ":")
subjects = append(subjects, rbacv1.Subject{
Kind: owner.Kind.String(),
Name: splitName[len(splitName)-1],
Namespace: splitName[len(splitName)-2],
})
} else {
subjects = append(subjects, rbacv1.Subject{
APIGroup: "rbac.authorization.k8s.io",
Kind: owner.Kind.String(),
Name: owner.Name,
})
}
}
list := make(map[types.NamespacedName]rbacv1.RoleRef)
for _, i := range tenant.Status.Namespaces {
list[types.NamespacedName{Namespace: i, Name: "namespace:admin"}] = rbacv1.RoleRef{
APIGroup: "rbac.authorization.k8s.io",
Kind: "ClusterRole",
Name: "admin",
}
list[types.NamespacedName{Namespace: i, Name: "namespace-deleter"}] = rbacv1.RoleRef{
APIGroup: "rbac.authorization.k8s.io",
Kind: "ClusterRole",
Name: rbac.DeleterRoleName,
}
}
for namespacedName, roleRef := range list {
target := &rbacv1.RoleBinding{
ObjectMeta: metav1.ObjectMeta{
Name: namespacedName.Name,
Namespace: namespacedName.Namespace,
},
}
var res controllerutil.OperationResult
res, err = controllerutil.CreateOrUpdate(context.TODO(), r.Client, target, func() (err error) {
target.ObjectMeta.Labels = newLabels
target.Subjects = subjects
target.RoleRef = roleRef
return controllerutil.SetControllerReference(tenant, target, r.Scheme)
})
r.emitEvent(tenant, target.GetNamespace(), res, fmt.Sprintf("Ensuring Capsule RoleBinding %s", target.GetName()), err)
r.Log.Info("Role Binding sync result: "+string(res), "name", target.Name, "namespace", target.Namespace)
if err != nil {
return err
}
}
return nil
}

View File

@@ -14,10 +14,11 @@ import (
capsulev1beta1 "github.com/clastix/capsule/api/v1beta1"
)
// pruningResources is taking care of removing the no more requested sub-resources as LimitRange, ResourceQuota or
// pruningClusterResources is taking care of removing the no more requested sub-resources as LimitRange, ResourceQuota or
// NetworkPolicy using the "exists" and "notin" LabelSelector to perform an outer-join removal.
func (r *Manager) pruningResources(ns string, keys []string, obj client.Object) (err error) {
func (r *Manager) pruningClusterResources(ctx context.Context, keys []string, obj client.Object) (err error) {
var capsuleLabel string
if capsuleLabel, err = capsulev1beta1.GetTypeLabel(obj); err != nil {
return
}
@@ -25,13 +26,16 @@ func (r *Manager) pruningResources(ns string, keys []string, obj client.Object)
selector := labels.NewSelector()
var exists *labels.Requirement
if exists, err = labels.NewRequirement(capsuleLabel, selection.Exists, []string{}); err != nil {
return
}
selector = selector.Add(*exists)
if len(keys) > 0 {
var notIn *labels.Requirement
if notIn, err = labels.NewRequirement(capsuleLabel, selection.NotIn, keys); err != nil {
return err
}
@@ -42,7 +46,48 @@ func (r *Manager) pruningResources(ns string, keys []string, obj client.Object)
r.Log.Info("Pruning objects with label selector " + selector.String())
return retry.RetryOnConflict(retry.DefaultBackoff, func() error {
return r.DeleteAllOf(context.TODO(), obj, &client.DeleteAllOfOptions{
return r.DeleteAllOf(ctx, obj, &client.DeleteAllOfOptions{
ListOptions: client.ListOptions{
LabelSelector: selector,
},
DeleteOptions: client.DeleteOptions{},
})
})
}
// pruningResources is taking care of removing the no more requested sub-resources as LimitRange, ResourceQuota or
// NetworkPolicy using the "exists" and "notin" LabelSelector to perform an outer-join removal.
func (r *Manager) pruningResources(ctx context.Context, ns string, keys []string, obj client.Object) (err error) {
var capsuleLabel string
if capsuleLabel, err = capsulev1beta1.GetTypeLabel(obj); err != nil {
return
}
selector := labels.NewSelector()
var exists *labels.Requirement
if exists, err = labels.NewRequirement(capsuleLabel, selection.Exists, []string{}); err != nil {
return
}
selector = selector.Add(*exists)
if len(keys) > 0 {
var notIn *labels.Requirement
if notIn, err = labels.NewRequirement(capsuleLabel, selection.NotIn, keys); err != nil {
return err
}
selector = selector.Add(*notIn)
}
r.Log.Info("Pruning objects with label selector " + selector.String())
return retry.RetryOnConflict(retry.DefaultBackoff, func() error {
return r.DeleteAllOf(ctx, obj, &client.DeleteAllOfOptions{
ListOptions: client.ListOptions{
LabelSelector: selector,
Namespace: ns,
@@ -53,7 +98,8 @@ func (r *Manager) pruningResources(ns string, keys []string, obj client.Object)
}
func (r *Manager) emitEvent(object runtime.Object, namespace string, res controllerutil.OperationResult, msg string, err error) {
var eventType = corev1.EventTypeNormal
eventType := corev1.EventTypeNormal
if err != nil {
eventType = corev1.EventTypeWarning
res = "Error"

10
controllers/tls/errors.go Normal file
View File

@@ -0,0 +1,10 @@
// Copyright 2020-2021 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package tls
type RunningInOutOfClusterModeError struct{}
func (r RunningInOutOfClusterModeError) Error() string {
return "cannot retrieve the leader Pod, probably running in out of the cluster mode"
}

337
controllers/tls/manager.go Normal file
View File

@@ -0,0 +1,337 @@
// Copyright 2020-2021 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package tls
import (
"context"
"fmt"
"os"
"time"
"github.com/go-logr/logr"
"github.com/pkg/errors"
"golang.org/x/sync/errgroup"
admissionregistrationv1 "k8s.io/api/admissionregistration/v1"
corev1 "k8s.io/api/core/v1"
apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
apierrors "k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/types"
"k8s.io/client-go/util/retry"
"k8s.io/utils/pointer"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/builder"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
"sigs.k8s.io/controller-runtime/pkg/handler"
"sigs.k8s.io/controller-runtime/pkg/predicate"
"sigs.k8s.io/controller-runtime/pkg/reconcile"
"sigs.k8s.io/controller-runtime/pkg/source"
"github.com/clastix/capsule/controllers/utils"
"github.com/clastix/capsule/pkg/cert"
"github.com/clastix/capsule/pkg/configuration"
)
const (
certificateExpirationThreshold = 3 * 24 * time.Hour
certificateValidity = 6 * 30 * 24 * time.Hour
PodUpdateAnnotationName = "capsule.clastix.io/updated"
)
type Reconciler struct {
client.Client
Log logr.Logger
Scheme *runtime.Scheme
Namespace string
Configuration configuration.Configuration
}
func (r *Reconciler) SetupWithManager(mgr ctrl.Manager) error {
enqueueFn := handler.EnqueueRequestsFromMapFunc(func(client.Object) []reconcile.Request {
return []reconcile.Request{
{
NamespacedName: types.NamespacedName{
Namespace: r.Namespace,
Name: r.Configuration.TLSSecretName(),
},
},
}
})
return ctrl.NewControllerManagedBy(mgr).
For(&corev1.Secret{}, utils.NamesMatchingPredicate(r.Configuration.TLSSecretName())).
Watches(source.NewKindWithCache(&admissionregistrationv1.ValidatingWebhookConfiguration{}, mgr.GetCache()), enqueueFn, builder.WithPredicates(predicate.NewPredicateFuncs(func(object client.Object) bool {
return object.GetName() == r.Configuration.ValidatingWebhookConfigurationName()
}))).
Watches(source.NewKindWithCache(&admissionregistrationv1.MutatingWebhookConfiguration{}, mgr.GetCache()), enqueueFn, builder.WithPredicates(predicate.NewPredicateFuncs(func(object client.Object) bool {
return object.GetName() == r.Configuration.MutatingWebhookConfigurationName()
}))).
Watches(source.NewKindWithCache(&apiextensionsv1.CustomResourceDefinition{}, mgr.GetCache()), enqueueFn, builder.WithPredicates(predicate.NewPredicateFuncs(func(object client.Object) bool {
return object.GetName() == r.Configuration.TenantCRDName()
}))).
Complete(r)
}
func (r Reconciler) ReconcileCertificates(ctx context.Context, certSecret *corev1.Secret) error {
if r.shouldUpdateCertificate(certSecret) {
r.Log.Info("Generating new TLS certificate")
ca, err := cert.GenerateCertificateAuthority()
if err != nil {
return err
}
opts := cert.NewCertOpts(time.Now().Add(certificateValidity), fmt.Sprintf("capsule-webhook-service.%s.svc", r.Namespace))
crt, key, err := ca.GenerateCertificate(opts)
if err != nil {
r.Log.Error(err, "Cannot generate new TLS certificate")
return err
}
caCrt, _ := ca.CACertificatePem()
certSecret.Data = map[string][]byte{
corev1.TLSCertKey: crt.Bytes(),
corev1.TLSPrivateKeyKey: key.Bytes(),
corev1.ServiceAccountRootCAKey: caCrt.Bytes(),
}
t := &corev1.Secret{ObjectMeta: certSecret.ObjectMeta}
_, err = controllerutil.CreateOrUpdate(ctx, r.Client, t, func() error {
t.Data = certSecret.Data
return nil
})
if err != nil {
r.Log.Error(err, "cannot update Capsule TLS")
return err
}
}
var caBundle []byte
var ok bool
if caBundle, ok = certSecret.Data[corev1.ServiceAccountRootCAKey]; !ok {
return fmt.Errorf("missing %s field in %s secret", corev1.ServiceAccountRootCAKey, r.Configuration.TLSSecretName())
}
r.Log.Info("Updating caBundle in webhooks and crd")
group := new(errgroup.Group)
group.Go(func() error {
return r.updateMutatingWebhookConfiguration(ctx, caBundle)
})
group.Go(func() error {
return r.updateValidatingWebhookConfiguration(ctx, caBundle)
})
group.Go(func() error {
return r.updateCustomResourceDefinition(ctx, caBundle)
})
operatorPods, err := r.getOperatorPods(ctx)
if err != nil {
if errors.As(err, &RunningInOutOfClusterModeError{}) {
r.Log.Info("skipping annotation of Pods for cert-manager", "error", err.Error())
return nil
}
return err
}
r.Log.Info("Updating capsule operator pods")
for _, pod := range operatorPods.Items {
p := pod
group.Go(func() error {
return r.updateOperatorPod(ctx, p)
})
}
if err := group.Wait(); err != nil {
return err
}
return nil
}
func (r Reconciler) Reconcile(ctx context.Context, request ctrl.Request) (ctrl.Result, error) {
r.Log = r.Log.WithValues("Request.Namespace", request.Namespace, "Request.Name", request.Name)
certSecret := &corev1.Secret{}
if err := r.Client.Get(ctx, request.NamespacedName, certSecret); err != nil {
// Error reading the object - requeue the request.
return reconcile.Result{}, err
}
if err := r.ReconcileCertificates(ctx, certSecret); err != nil {
return reconcile.Result{}, err
}
certificate, err := cert.GetCertificateFromBytes(certSecret.Data[corev1.TLSCertKey])
if err != nil {
return reconcile.Result{}, err
}
now := time.Now()
requeueTime := certificate.NotAfter.Add(-(certificateExpirationThreshold - 1*time.Second))
rq := requeueTime.Sub(now)
r.Log.Info("Reconciliation completed, processing back in " + rq.String())
return reconcile.Result{Requeue: true, RequeueAfter: rq}, nil
}
func (r Reconciler) shouldUpdateCertificate(secret *corev1.Secret) bool {
if _, ok := secret.Data[corev1.ServiceAccountRootCAKey]; !ok {
return true
}
certificate, key, err := cert.GetCertificateWithPrivateKeyFromBytes(secret.Data[corev1.TLSCertKey], secret.Data[corev1.TLSPrivateKeyKey])
if err != nil {
return true
}
if err := cert.ValidateCertificate(certificate, key, certificateExpirationThreshold); err != nil {
r.Log.Error(err, "failed to validate certificate, generating new one")
return true
}
r.Log.Info("Skipping TLS certificate generation as it is still valid")
return false
}
// By default helm doesn't allow to use templates in CRD (https://helm.sh/docs/chart_best_practices/custom_resource_definitions/#method-1-let-helm-do-it-for-you).
// In order to overcome this, we are setting conversion strategy in helm chart to None, and then update it with CA and namespace information.
func (r *Reconciler) updateCustomResourceDefinition(ctx context.Context, caBundle []byte) error {
return retry.RetryOnConflict(retry.DefaultBackoff, func() (err error) {
crd := &apiextensionsv1.CustomResourceDefinition{}
err = r.Get(ctx, types.NamespacedName{Name: "tenants.capsule.clastix.io"}, crd)
if err != nil {
r.Log.Error(err, "cannot retrieve CustomResourceDefinition")
return err
}
_, err = controllerutil.CreateOrUpdate(ctx, r.Client, crd, func() error {
crd.Spec.Conversion = &apiextensionsv1.CustomResourceConversion{
Strategy: "Webhook",
Webhook: &apiextensionsv1.WebhookConversion{
ClientConfig: &apiextensionsv1.WebhookClientConfig{
Service: &apiextensionsv1.ServiceReference{
Namespace: r.Namespace,
Name: "capsule-webhook-service",
Path: pointer.StringPtr("/convert"),
Port: pointer.Int32Ptr(443),
},
CABundle: caBundle,
},
ConversionReviewVersions: []string{"v1alpha1", "v1beta1"},
},
}
return nil
})
return err
})
}
//nolint:dupl
func (r Reconciler) updateValidatingWebhookConfiguration(ctx context.Context, caBundle []byte) error {
return retry.RetryOnConflict(retry.DefaultBackoff, func() (err error) {
vw := &admissionregistrationv1.ValidatingWebhookConfiguration{}
err = r.Get(ctx, types.NamespacedName{Name: r.Configuration.ValidatingWebhookConfigurationName()}, vw)
if err != nil {
r.Log.Error(err, "cannot retrieve ValidatingWebhookConfiguration")
return err
}
for i, w := range vw.Webhooks {
// Updating CABundle only in case of an internal service reference
if w.ClientConfig.Service != nil {
vw.Webhooks[i].ClientConfig.CABundle = caBundle
}
}
return r.Update(ctx, vw, &client.UpdateOptions{})
})
}
//nolint:dupl
func (r Reconciler) updateMutatingWebhookConfiguration(ctx context.Context, caBundle []byte) error {
return retry.RetryOnConflict(retry.DefaultBackoff, func() (err error) {
mw := &admissionregistrationv1.MutatingWebhookConfiguration{}
err = r.Get(ctx, types.NamespacedName{Name: r.Configuration.MutatingWebhookConfigurationName()}, mw)
if err != nil {
r.Log.Error(err, "cannot retrieve MutatingWebhookConfiguration")
return err
}
for i, w := range mw.Webhooks {
// Updating CABundle only in case of an internal service reference
if w.ClientConfig.Service != nil {
mw.Webhooks[i].ClientConfig.CABundle = caBundle
}
}
return r.Update(ctx, mw, &client.UpdateOptions{})
})
}
func (r Reconciler) updateOperatorPod(ctx context.Context, pod corev1.Pod) error {
return retry.RetryOnConflict(retry.DefaultRetry, func() error {
// Need to get latest version of pod
p := &corev1.Pod{}
if err := r.Client.Get(ctx, types.NamespacedName{Namespace: pod.Namespace, Name: pod.Name}, p); err != nil && !apierrors.IsNotFound(err) {
r.Log.Error(err, "cannot get pod", "name", pod.Name, "namespace", pod.Namespace)
return err
}
if p.Annotations == nil {
p.Annotations = map[string]string{}
}
p.Annotations[PodUpdateAnnotationName] = time.Now().Format(time.RFC3339Nano)
if err := r.Client.Update(ctx, p, &client.UpdateOptions{}); err != nil {
r.Log.Error(err, "cannot update pod", "name", pod.Name, "namespace", pod.Namespace)
return err
}
return nil
})
}
func (r Reconciler) getOperatorPods(ctx context.Context) (*corev1.PodList, error) {
hostname, _ := os.Hostname()
leaderPod := &corev1.Pod{}
if err := r.Client.Get(ctx, types.NamespacedName{Namespace: os.Getenv("NAMESPACE"), Name: hostname}, leaderPod); err != nil {
return nil, RunningInOutOfClusterModeError{}
}
podList := &corev1.PodList{}
if err := r.Client.List(ctx, podList, client.MatchingLabels(leaderPod.ObjectMeta.Labels)); err != nil {
r.Log.Error(err, "cannot retrieve list of Capsule pods")
return nil, err
}
return podList, nil
}

View File

@@ -0,0 +1,19 @@
package utils
import (
"sigs.k8s.io/controller-runtime/pkg/builder"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/predicate"
)
func NamesMatchingPredicate(names ...string) builder.Predicates {
return builder.WithPredicates(predicate.NewPredicateFuncs(func(object client.Object) bool {
for _, name := range names {
if object.GetName() == name {
return true
}
}
return false
}))
}

View File

@@ -4,7 +4,7 @@
Make sure you have these tools installed:
- [Go 1.16+](https://golang.org/dl/)
- [Go 1.18+](https://golang.org/dl/)
- [Operator SDK 1.7.2+](https://github.com/operator-framework/operator-sdk), or [Kubebuilder](https://github.com/kubernetes-sigs/kubebuilder)
- [KinD](https://github.com/kubernetes-sigs/kind) or [k3d](https://k3d.io/), with `kubectl`
- [ngrok](https://ngrok.com/) (if you want to run locally with remote Kubernetes)
@@ -125,12 +125,12 @@ $ go mod download
$ make docker-build
# Retrieve the built image version
$ export CAPSULE_IMAGE_VESION=`docker images --format '{{.Tag}}' quay.io/clastix/capsule`
$ export CAPSULE_IMAGE_VESION=`docker images --format '{{.Tag}}' clastix/capsule`
# If k3s, load the image into cluster by
$ k3d image import --cluster k3s-capsule capsule quay.io/clastix/capsule:${CAPSULE_IMAGE_VESION}
$ k3d image import --cluster k3s-capsule capsule clastix/capsule:${CAPSULE_IMAGE_VESION}
# If Kind, load the image into cluster by
$ kind load docker-image --name kind-capsule quay.io/clastix/capsule:${CAPSULE_IMAGE_VESION}
$ kind load docker-image --name kind-capsule clastix/capsule:${CAPSULE_IMAGE_VESION}
# deploy all the required manifests
# Note: 1) please retry if you saw errors; 2) if you want to clean it up first, run: make remove

View File

@@ -17,6 +17,8 @@ In the context of Capsule project, we consider the following roles:
The release process will be governed by Maintainers.
Please, refer to the [maintainers file](https://github.com/clastix/capsule/blob/master/.github/blob/master/maintainers.yaml) available in the source code.
## Roadmap Planning
Maintainers will share roadmap and release versions as milestones in GitHub.

View File

@@ -450,24 +450,24 @@ As tenant owner list RBAC permissions set by Capsule
```bash
kubectl --kubeconfig alice get rolebindings
NAME ROLE AGE
namespace-deleter ClusterRole/capsule-namespace-deleter 11h
namespace:admin ClusterRole/admin 11h
NAME ROLE AGE
capsule-oil-0-admin ClusterRole/admin 11h
capsule-oil-1-capsule-namespace-deleter ClusterRole/capsule-namespace-deleter 11h
```
As tenant owner, try to change/delete the rolebinding to escalate permissions
```bash
kubectl --kubeconfig alice edit/delete rolebinding namespace:admin
kubectl --kubeconfig alice edit/delete rolebinding capsule-oil-0-admin
```
The rolebinding is immediately recreated by Capsule:
```
kubectl --kubeconfig alice get rolebindings
NAME ROLE AGE
namespace-deleter ClusterRole/capsule-namespace-deleter 11h
namespace:admin ClusterRole/admin 2s
NAME ROLE AGE
capsule-oil-0-admin ClusterRole/admin 2s
capsule-oil-1-capsule-namespace-deleter ClusterRole/capsule-namespace-deleter 11h
```
However, the tenant owner can create and assign permissions inside the namespace she owns

View File

@@ -30,7 +30,7 @@ The `capsule-proxy` implements a simple reverse proxy that intercepts only speci
Current implementation filters the following requests:
* `/api/scheduling.k8s.io/{v1}/priorityclasses{/name}`
* `/api/v1/namespaces`
* `/api/v1/namespaces{/name}`
* `/api/v1/nodes{/name}`
* `/api/v1/pods?fieldSelector=spec.nodeName%3D{name}`
* `/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/{name}`
@@ -151,6 +151,21 @@ oil-development Active 2m
oil-production Active 2m
```
Capsule Proxy supports applying a Namespace configuration using the `apply` command, as follows.
```
$: cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Namespace
metadata:
name: solar-development
EOF
namespace/solar-development unchanged
# or, in case of non existing Namespace:
namespace/solar-development created
```
### Nodes
The Capsule Proxy gives the owners the ability to access the nodes matching the `.spec.nodeSelector` in the Tenant manifest:
@@ -189,35 +204,6 @@ When issuing a `kubectl describe node`, some other endpoints are put in place:
These are mandatory in order to retrieve the list of the running Pods on the required node, and providing info about the lease status of it.
### Nodes
The Capsule Proxy gives the owners the ability to access the nodes matching the `.spec.nodeSelector` in the Tenant manifest:
```yaml
apiVersion: capsule.clastix.io/v1beta1
kind: Tenant
metadata:
name: oil
spec:
owners:
- kind: User
name: alice
proxySettings:
- kind: Nodes
operations:
- List
nodeSelector:
kubernetes.io/hostname: capsule-gold-qwerty
```
```bash
$ kubectl --context alice-oidc@mycluster get nodes
NAME STATUS ROLES AGE VERSION
capsule-gold-qwerty Ready <none> 43h v1.19.1
```
> Warning: when no `nodeSelector` is specified, the tenant owners has access to all the nodes, according to the permissions listed in the `proxySettings` specs.
### Storage Classes
A Tenant may be limited to use a set of allowed Storage Class resources, as follows.
@@ -399,6 +385,59 @@ globalDefault: false
description: "Priority class for Tenants"
```
### ProxySetting Use Case
Consider a scenario, where a cluster admin creates a tenant and assign ownership of the tenant to a user, so called tenant owner. Afterwards, tenant owner would in turn like to provide access to their cluster-scoped resources to a set of users (e.g. non-owners or tenant users), groups and service accounts, who doesn't require tenant owner level permissions.
Tenant Owner can provide access to following cluster-scoped resources to their tenant users, groups and service account by creating `ProxySetting` resource
- `Nodes`
- `StorageClasses`
- `IngressClasses`
- `PriorityClasses`
Each Resource kind can be granted with following verbs, such as:
- `List`
- `Update`
- `Delete`
These tenant users, groups and services accounts have less privileged access than tenant owners.
As a Tenant Owner `alice`, you can create a `ProxySetting` resources to allow `bob` to list nodes, storage classes, ingress classes and priority classes
```yaml
apiVersion: capsule.clastix.io/v1beta1
kind: ProxySetting
metadata:
name: sre-readers
namespace: solar-production
spec:
subjects:
- name: bob
kind: User
proxySettings:
- kind: Nodes
operations:
- List
- kind: StorageClasses
operations:
- List
- kind: IngressClasses
operations:
- List
- kind: PriorityClasses
operations:
- List
```
As a Tenant User `bob`, you can list nodes, storage classes, ingress classes and priority classes
```bash
$ kubectl auth can-i --context bob-oidc@mycluster get nodes
yes
$ kubectl auth can-i --context bob-oidc@mycluster get storageclasses
yes
$ kubectl auth can-i --context bob-oidc@mycluster get ingressclasses
yes
$ kubectl auth can-i --context bob-oidc@mycluster get priorityclasses
yes
```
## HTTP support
Capsule proxy supports `https` and `http`, although the latter is not recommended, we understand that it can be useful for some use cases (i.e. development, working behind a TLS-terminated reverse proxy and so on). As the default behaviour is to work with `https`, we need to use the flag `--enable-ssl=false` if we really want to work under `http`.
@@ -413,6 +452,45 @@ $ curl -H "Authorization: Bearer $TOKEN" http://localhost:9001/api/v1/namespaces
> NOTE: `kubectl` will not work against a `http` server.
## Metrics
Starting from the v0.3.0 release, Capsule Proxy exposes Prometheus metrics available at `http://0.0.0.0:8080/metrics`.
The offered metrics are related to the internal `controller-manager` code base, such as work-queue and REST client requests, and the Go runtime ones.
Along with these, metrics `capsule_proxy_response_time_seconds` and `capsule_proxy_requests_total` have been introduced and are specific to the Capsule Proxy code-base and functionalities.
`capsule_proxy_response_time_seconds` offers a bucket representation of the HTTP request duration.
The available variables for this metrics are the following ones:
- `path`: the HTTP path of each single request that Capsule Proxy passes to the upstream
`capsule_proxy_requests_total` counts the global requests that Capsule Proxy is passing to the upstream with the following labels.
- `path`: the HTTP path of each single request that Capsule Proxy passes to the upstream
- `status`: the HTTP status code of the request
> Example output of the metrics:
> ```
> # HELP capsule_proxy_requests_total Number of requests
> # TYPE capsule_proxy_requests_total counter
> capsule_proxy_requests_total{path="/api/v1/namespaces",status="403"} 1
> # HELP capsule_proxy_response_time_seconds Duration of capsule proxy requests.
> # TYPE capsule_proxy_response_time_seconds histogram
> capsule_proxy_response_time_seconds_bucket{path="/api/v1/namespaces",le="0.005"} 0
> capsule_proxy_response_time_seconds_bucket{path="/api/v1/namespaces",le="0.01"} 0
> capsule_proxy_response_time_seconds_bucket{path="/api/v1/namespaces",le="0.025"} 0
> capsule_proxy_response_time_seconds_bucket{path="/api/v1/namespaces",le="0.05"} 0
> capsule_proxy_response_time_seconds_bucket{path="/api/v1/namespaces",le="0.1"} 0
> capsule_proxy_response_time_seconds_bucket{path="/api/v1/namespaces",le="0.25"} 0
> capsule_proxy_response_time_seconds_bucket{path="/api/v1/namespaces",le="0.5"} 0
> capsule_proxy_response_time_seconds_bucket{path="/api/v1/namespaces",le="1"} 0
> capsule_proxy_response_time_seconds_bucket{path="/api/v1/namespaces",le="2.5"} 1
> capsule_proxy_response_time_seconds_bucket{path="/api/v1/namespaces",le="5"} 1
> capsule_proxy_response_time_seconds_bucket{path="/api/v1/namespaces",le="10"} 1
> capsule_proxy_response_time_seconds_bucket{path="/api/v1/namespaces",le="+Inf"} 1
> capsule_proxy_response_time_seconds_sum{path="/api/v1/namespaces"} 2.206192787
> capsule_proxy_response_time_seconds_count{path="/api/v1/namespaces"} 1
> ```
## Contributing
`capsule-proxy` is an open-source software released with Apache2 [license](https://github.com/clastix/capsule-proxy/blob/master/LICENSE).

View File

@@ -2,23 +2,21 @@
Capsule is a framework to implement multi-tenant and policy-driven scenarios in Kubernetes. In this tutorial, we'll focus on a hypothetical case covering the main features of the Capsule Operator.
***Acme Corp***, our sample organization, is building a Container as a Service platform (CaaS) to serve multiple lines of business. Each line of business has its team of engineers that are responsible for the development, deployment, and operating of their digital products. We'll work with the following actors:
***Acme Corp***, our sample organization, is building a Container as a Service platform (CaaS) to serve multiple lines of business, or departments, e.g. _Oil_, _Gas_, _Solar_, _Wind_, _Water_. Each department has its team of engineers that are responsible for the development, deployment, and operating of their digital products. We'll work with the following actors:
* ***Bill***: the cluster administrator from the operations department of Acme Corp.
* ***Bill***: the cluster administrator from the operations department of _Acme Corp_.
* ***Alice***: the IT Project Leader in the Oil & Gas Business Units. She is responsible for a team made of different job responsibilities (developers, administrators, SRE engineers, etc.) working in separate multiple departments.
* ***Alice***: the project leader in the _Oil_ & _Gas_ departments. She is responsible for a team made of different job responsibilities: e.g. developers, administrators, SRE engineers, etc.
* ***Joe***:
He works at Acme Corp, as a lead developer of a distributed team in Alice's organization.
* ***Joe***: works as a lead developer of a distributed team in Alice's organization.
* ***Bob***:
He is the head of Engineering for the Water Business Unit, the main and historical line of business at Acme Corp.
* ***Bob***: is the head of engineering for the _Water_ department, the main and historical line of business at _Acme Corp_.
## Assign Tenant ownership
### User as tenant owner
Bill, the cluster admin, receives a new request from Acme Corp.'s CTO asking for a new tenant to be onboarded and Alice user will be the tenant owner. Bill then assigns Alice's identity of `alice` in the Acme Corp. identity management system. Since Alice is a tenant owner, Bill needs to assign `alice` the Capsule group defined by `--capsule-user-group` option, which defaults to `capsule.clastix.io`.
Bill, the cluster admin, receives a new request from _Acme Corp_'s CTO asking for a new tenant to be onboarded and Alice user will be the tenant owner. Bill then assigns Alice's identity of `alice` in the _Acme Corp_. identity management system. Since Alice is a tenant owner, Bill needs to assign `alice` the Capsule group defined by `--capsule-user-group` option, which defaults to `capsule.clastix.io`.
To keep things simple, we assume that Bill just creates a client certificate for authentication using X.509 Certificate Signing Request, so Alice's certificate has `"/CN=alice/O=capsule.clastix.io"`.
@@ -173,26 +171,31 @@ system:serviceaccounts:{service-account-namespace}
> Please, pay attention when setting a service account acting as tenant owner. Make sure you're not using the group `system:serviceaccounts` or the group `system:serviceaccounts:{capsule-namespace}` as Capsule group, otherwise you'll create a short-circuit in the Capsule controller, being Capsule itself controlled by a serviceaccount.
### Roles assigned to Tenant Owners
## Create namespaces
Alice, once logged with her credentials, can create a new namespace in her tenant, as simply issuing:
By default, all Tenant Owners will be granted with two ClusterRole resources using the RoleBinding API:
1. the Kubernetes default one, [`admin`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles), that grants most of the namespace scoped resources
2. a custom one, created by Capsule, named `capsule-namespace-deleter`, allowing to delete the created namespaces
In the example below, assuming the tenant owner creates a namespace `oil-production` in Tenant `oil`, you'll see the Role Bindings giving the tenant owner full permissions on the tenant namespaces:
```
kubectl create ns oil-production
$: kubectl get rolebindings -n oil-production
NAME ROLE AGE
capsule-oil-0-admin ClusterRole/admin 6s
capsule-oil-1-capsule-namespace-deleter ClusterRole/capsule-namespace-deleter 5s
```
Alice started the name of the namespace prepended by the name of the tenant: this is not a strict requirement but it is highly suggested because it is likely that many different tenants would like to call their namespaces `production`, `test`, or `demo`, etc.
The enforcement of this naming convention is optional and can be controlled by the cluster administrator with the `--force-tenant-prefix` option as an argument of the Capsule controller.
When Alice creates the namespace, the Capsule controller listening for creation and deletion events assigns to Alice the following roles:
When Alice creates the namespaces, the Capsule controller assigns to Alice the following permissions, so that Alice can act as the admin of all the tenant namespaces.
```yaml
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: namespace:admin
name: capsule-oil-0-admin
namespace: oil-production
subjects:
- kind: User
@@ -205,7 +208,7 @@ roleRef:
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: namespace-deleter
name: capsule-oil-1-capsule-namespace-deleter
namespace: oil-production
subjects:
- kind: User
@@ -216,28 +219,191 @@ roleRef:
apiGroup: rbac.authorization.k8s.io
```
So Alice is the admin of the namespaces:
In some cases, the cluster admin needs to narrow the range of permissions assigned to tenant owners by assigning a Cluster Role with less permissions than above. Capsule supports the dynamic assignment of any ClusterRole resources for each Tenant Owner.
```
kubectl get rolebindings -n oil-development
NAME ROLE AGE
namespace:admin ClusterRole/admin 12s
namespace-deleter ClusterRole/capsule-namespace-deleter 12s
For example, assign user `Joe` the tenant ownership with only [view](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) permissions on tenant namespaces:
```yaml
kubectl apply -f - << EOF
apiVersion: capsule.clastix.io/v1beta1
kind: Tenant
metadata:
name: oil
annotations:
clusterrolenames.capsule.clastix.io/user.joe: view
spec:
owners:
- name: alice
kind: User
- name: joe
kind: User
EOF
```
The said Role Binding resources are automatically created by Capsule controller when the tenant owner Alice creates a namespace in the tenant.
you'll see the new Role Bindings assigned to Joe:
Alice can deploy any resource in the namespace, according to the predefined
[`admin` cluster role](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles).
```
kubectl -n oil-production get rolebindings
NAME ROLE AGE
capsule-oil-0-admin ClusterRole/admin 8d
capsule-oil-1-capsule-namespace-deleter ClusterRole/capsule-namespace-deleter 8d
capsule-oil-2-view ClusterRole/edit 5s
```
so that Joe can only view resources in the tenant namespaces:
```
kubectl --as joe --as-group capsule.clastix.io auth can-i delete pods -n oil-marketing
no
```
> Please, note that, despite created with more restricted permissions, a tenant owner can still create namespaces in the tenant because he belongs to the `capsule.clastix.io` group. If you want a user not acting as tenant owner, but still operating in the tenant, you can assign additional `RoleBindings` without assigning him the tenant ownership.
Custom ClusterRoles are also supported. Assuming the cluster admin creates:
```yaml
kubectl apply -f - << EOF
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: prometheus-servicemonitors-viewer
rules:
- apiGroups: ["monitoring.coreos.com"]
resources: ["servicemonitors"]
verbs: ["get", "list", "watch"]
EOF
```
These permissions can be granted to Joe
```yaml
kubectl apply -f - << EOF
apiVersion: capsule.clastix.io/v1beta1
kind: Tenant
metadata:
name: oil
annotations:
clusterrolenames.capsule.clastix.io/user.joe: view,prometheus-servicemonitors-viewer
spec:
owners:
- name: alice
kind: User
- name: joe
kind: User
EOF
```
For the given configuration, the resulting RoleBinding resources are the following ones:
```
kubectl -n oil-production get rolebindings
NAME ROLE AGE
capsule-oil-0-admin ClusterRole/admin 8d
capsule-oil-1-capsule-namespace-deleter ClusterRole/capsule-namespace-deleter 8d
capsule-oil-2-view ClusterRole/view 11m
capsule-oil-3-prometheus-servicemonitors-viewer ClusterRole/prometheus-servicemonitors-viewer 18s
```
> The pattern for the annotation is `clusterrolenames.capsule.clastix.io/${KIND}.${NAME}`.
> The placeholders `${KIND}` and `${NAME}` are referring to the Tenant Owner specification fields, both lower-cased.
>
> In the case of users that are identified using their email address, the symbol `@` wouldn't be supported by the RFC 1123.
> For such cases, the `@` symbol can be replaced with the placeholder `__AT__`.
>
> ```yaml
> apiVersion: capsule.clastix.io/v1beta1
> kind: Tenant
> metadata:
> annotations:
> clusterrolenames.capsule.clastix.io/alice__AT__clastix.io: editor,manager
> spec:
> owners:
> - kind: User
> name: alice@org.tld
> - kind: User
> name: alice@clastix.io
> ```
>
> Instead, with the resulting annotation key exceeding 63 characters length, the zero-based index of the owner can be specified as follows:
>
> ```yaml
> apiVersion: capsule.clastix.io/v1beta1
> kind: Tenant
> metadata:
> annotations:
> clusterrolenames.capsule.clastix.io/1: editor,manager
> spec:
> owners:
> - kind: User
> name: alice@org.tld
> - kind: User
> name: very-long-user-name-that-breaks-rfc-1123@org.tld
> ```
>
> This latter example will assign the roles `editor` and `manager`, assigned to the user `very-long-user-name-that-breaks-rfc-1123@org.tld`.
### Assign additional Role Bindings
The tenant owner acts as admin of tenant namespaces. Other users can operate inside the tenant namespaces with different levels of permissions and authorizations.
Assuming the cluster admin creates:
```yaml
kubectl apply -f - << EOF
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: prometheus-servicemonitors-viewer
rules:
- apiGroups: ["monitoring.coreos.com"]
resources: ["servicemonitors"]
verbs: ["get", "list", "watch"]
EOF
```
These permissions can be granted to a user without giving the role of tenant owner:
```yaml
kubectl apply -f - << EOF
apiVersion: capsule.clastix.io/v1beta1
kind: Tenant
metadata:
name: oil
spec:
owners:
- name: alice
kind: User
additionalRoleBindings:
- clusterRoleName: 'prometheus-servicemonitors-viewer'
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: User
name: joe
EOF
```
## Create namespaces
Alice, once logged with her credentials, can create a new namespace in her tenant, as simply issuing:
```
kubectl create ns oil-production
```
Alice started the name of the namespace prepended by the name of the tenant: this is not a strict requirement but it is highly suggested because it is likely that many different tenants would like to call their namespaces `production`, `test`, or `demo`, etc.
The enforcement of this naming convention is optional and can be controlled by the cluster administrator with the `--force-tenant-prefix` option as an argument of the Capsule controller.
Alice can deploy any resource in any of the namespaces
```
kubectl -n oil-development run nginx --image=docker.io/nginx
kubectl -n oil-development get pods
```
Bill, the cluster admin, can control how many namespaces Alice, creates by setting a quota in the tenant manifest `spec.namespaceOptions.quota`
The cluster admin, can control how many namespaces Alice, creates by setting a quota in the tenant manifest `spec.namespaceOptions.quota`
```yaml
kubectl apply -f - << EOF
apiVersion: capsule.clastix.io/v1beta1
kind: Tenant
metadata:
@@ -248,6 +414,7 @@ spec:
kind: User
namespaceOptions:
quota: 3
EOF
```
Alice can create additional namespaces according to the quota:
@@ -283,58 +450,6 @@ admission webhook "namespace.capsule.clastix.io" denied the request.
```
The enforcement on the maximum number of namespaces per Tenant is the responsibility of the Capsule controller via its Dynamic Admission Webhook capability.
## Assign permissions
Alice acts as the tenant admin. Other users can operate inside the tenant with different levels of permissions and authorizations. Alice is responsible for creating additional roles and assigning these roles to other users to work in the same tenant.
One of the key design principles of the Capsule is self-provisioning management from the tenant owner's perspective. Alice, the tenant owner, does not need to interact with Bill, the cluster admin, to complete her day-by-day duties. On the other side, Bill does not have to deal with multiple requests coming from multiple tenant owners that probably will overwhelm him.
Capsule leaves Alice, and the other tenant owners, the freedom to create RBAC roles at the namespace level, or using the pre-defined cluster roles already available in Kubernetes. Since roles and rolebindings are limited to a namespace scope, Alice can assign the roles to the other users accessing the same tenant only after the namespace is created. This gives Alice the power to administer the tenant without the intervention of the cluster admin.
From the cluster admin perspective, the only required action for Bill is to provide the other identities, eg. `joe` in the Identity Management system. This task can be done once when onboarding the tenant and the number of users accessing the tenant can be part of the tenant business profile.
Alice can create Roles and RoleBindings only in the namespaces she owns
```
kubectl auth can-i get roles -n oil-development
yes
kubectl auth can-i get rolebindings -n oil-development
yes
```
so she can assign the role of namespace `oil-development` admin to Joe, another user accessing the tenant `oil`
```yaml
kubectl --as alice --as-group capsule.clastix.io apply -f - << EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
name: oil-development:admin
namespace: oil-development
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: admin
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: User
name: joe
EOF
```
Joe now can operate on the namespace `oil-development` as admin but he has no access to the other namespaces `oil-production`, and `oil-test` that are part of the same tenant:
```
kubectl --as joe --as-group capsule.clastix.io auth can-i create pod -n oil-development
yes
kubectl --as joe --as-group capsule.clastix.io auth can-i create pod -n oil-production
no
```
> Please, note the user `joe`, in the example above, is not acting as tenant owner. He can just operate in `oil-development` namespace as admin.
## Assign multiple tenants
A single team is likely responsible for multiple lines of business. For example, in our sample organization Acme Corp., Alice is responsible for both the Oil and Gas lines of business. It's more likely that Alice requires two different tenants, for example, `oil` and `gas` to keep things isolated.
@@ -1000,7 +1115,7 @@ Also, Bill can make sure pods belonging to a tenant namespace cannot access othe
Bill can set network policies in the tenant manifest, according to the requirements:
```yaml
kubectl -n oil-production apply -f - << EOF
kubectl apply -f - << EOF
apiVersion: capsule.clastix.io/v1beta1
kind: Tenant
metadata:
@@ -1101,7 +1216,7 @@ Bob, an attacker, could try to schedule a Pod on the same node where Alice is ru
To avoid this kind of attack, Bill, the cluster admin, can force Alice, the tenant owner, to start her Pods using only the allowed values for `ImagePullPolicy`, enforcing the `kubelet` to check the authorization first.
```yaml
kubectl -n oil-production apply -f - << EOF
kubectl apply -f - << EOF
apiVersion: capsule.clastix.io/v1beta1
kind: Tenant
metadata:
@@ -1127,7 +1242,7 @@ The spec `containerRegistries` addresses this task and can provide a combination
```yaml
kubectl -n oil-production apply -f - << EOF
kubectl apply -f - << EOF
apiVersion: capsule.clastix.io/v1beta1
kind: Tenant
metadata:
@@ -1152,95 +1267,13 @@ A Pod running `internal.registry.foo.tld/capsule:latest` as registry will be all
Any attempt of Alice to use a not allowed `containerRegistries` value is denied by the Validation Webhook enforcing it.
## Assign Pod Security Policies
Bill, the cluster admin, can assign a dedicated Pod Security Policy (PSP) to Alice's tenant. This is likely to be a requirement in a multi-tenancy environment.
The cluster admin creates a PSP:
```yaml
kubectl -n oil-production apply -f - << EOF
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp:restricted
spec:
privileged: false
# Required to prevent escalations to root.
allowPrivilegeEscalation: false
...
EOF
```
Then create a _ClusterRole_ using or granting the said item
```yaml
kubectl -n oil-production apply -f - << EOF
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: psp:restricted
rules:
- apiGroups: ['policy']
resources: ['podsecuritypolicies']
resourceNames: ['psp:restricted']
verbs: ['use']
EOF
```
Bill can assign this role to all namespaces in the Alice's tenant by setting it in the tenant manifest:
```yaml
kubectl -n oil-production apply -f - << EOF
apiVersion: capsule.clastix.io/v1beta1
kind: Tenant
metadata:
name: oil
spec:
owners:
- name: alice
kind: User
additionalRoleBindings:
- clusterRoleName: psp:privileged
subjects:
- kind: "Group"
apiGroup: "rbac.authorization.k8s.io"
name: "system:authenticated"
EOF
```
With the given specification, Capsule will ensure that all Alice's namespaces will contain a _RoleBinding_ for the specified _Cluster Role_.
For example, in the `oil-production` namespace, Alice will see:
```yaml
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: 'capsule-oil-psp:privileged'
namespace: oil-production
labels:
capsule.clastix.io/role-binding: a10c4c8c48474963
capsule.clastix.io/tenant: oil
subjects:
- kind: Group
apiGroup: rbac.authorization.k8s.io
name: 'system:authenticated'
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: 'psp:privileged'
```
With the above example, Capsule is forbidding any authenticated user in `oil-production` namespace to run privileged pods and to perform privilege escalation as declared by the Cluster Role `psp:privileged`.
## Create Custom Resources
Capsule grants admin permissions to the tenant owners but is only limited to their namespaces. To achieve that, it assigns the ClusterRole [admin](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) to the tenant owner. This ClusterRole does not permit the installation of custom resources in the namespaces.
In order to leave the tenant owner to create Custom Resources in their namespaces, the cluster admin defines a proper Cluster Role. For example:
```yaml
kubectl -n oil-production apply -f - << EOF
kubectl apply -f - << EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
@@ -1265,7 +1298,7 @@ EOF
Bill can assign this role to any namespace in the Alice's tenant by setting it in the tenant manifest:
```yaml
kubectl -n oil-production apply -f - << EOF
kubectl apply -f - << EOF
apiVersion: capsule.clastix.io/v1beta1
kind: Tenant
metadata:
@@ -1363,10 +1396,10 @@ spec:
> This feature is still in an alpha stage and requires a high amount of computing resources due to the dynamic client requests.
## Taint namespaces
With Capsule, Bill can _"taint"_ the namespaces created by Alice with additional labels and/or annotations. There is no specific semantic assigned to these labels and annotations: they just will be assigned to the namespaces in the tenant as they are created by Alice. This can help the cluster admin to implement specific use cases. As it can be used to implement backup as a service for namespaces in the tenant.
## Assign Additional Metadata
The cluster admin can _"taint"_ the namespaces created by tenant onwers with additional metadata as labels and annotations. There is no specific semantic assigned to these labels and annotations: they will be assigned to the namespaces in the tenant as they are created. This can help the cluster admin to implement specific use cases as, for example, leave only a given tenant to be backuped by a backup service.
Bill assigns additional labels and annotations to all namespaces created in the `oil` tenant:
Assigns additional labels and annotations to all namespaces created in the `oil` tenant:
```yaml
kubectl apply -f - << EOF
@@ -1381,18 +1414,42 @@ spec:
namespaceOptions:
additionalMetadata:
annotations:
capsule.clastix.io/backup: "true"
storagelocationtype: s3
labels:
capsule.clastix.io/tenant: oil
capsule.clastix.io/backup: "true"
EOF
```
When Alice creates a namespace, this will inherit the given label and/or annotation.
When the tenant owner creates a namespace, it inherits the given label and/or annotation:
## Taint services
With Capsule, Bill can _"taint"_ the services created by Alice with additional labels and/or annotations. There is no specific semantic assigned to these labels and annotations: they just will be assigned to the services in the tenant as they are created by Alice. This can help the cluster admin to implement specific use cases.
```yaml
apiVersion: v1
kind: Namespace
metadata:
annotations:
storagelocationtype: s3
labels:
capsule.clastix.io/tenant: oil
kubernetes.io/metadata.name: oil-production
name: oil-production
capsule.clastix.io/backup: "true"
name: oil-production
ownerReferences:
- apiVersion: capsule.clastix.io/v1beta1
blockOwnerDeletion: true
controller: true
kind: Tenant
name: oil
spec:
finalizers:
- kubernetes
status:
phase: Active
```
Bill assigns additional labels and annotations to all services created in the `oil` tenant:
Additionally, the cluster admin can _"taint"_ the services created by the tenant owners with additional metadata as labels and annotations.
Assigns additional labels and annotations to all services created in the `oil` tenant:
```yaml
kubectl apply -f - << EOF
@@ -1406,14 +1463,30 @@ spec:
kind: User
serviceOptions:
additionalMetadata:
annotations:
capsule.clastix.io/backup: "true"
labels:
capsule.clastix.io/tenant: oil
capsule.clastix.io/backup: "true"
EOF
```
When Alice creates a service in a namespace, this will inherit the given label and/or annotation.
When the tenant owner creates a service in a tenant namespace, it inherits the given label and/or annotation:
```yaml
apiVersion: v1
kind: Service
metadata:
name: nginx
namespace: oil-production
labels:
capsule.clastix.io/backup: "true"
spec:
ports:
- protocol: TCP
port: 80
targetPort: 8080
selector:
run: nginx
type: ClusterIP
```
## Cordon a Tenant
@@ -1614,6 +1687,26 @@ EOF
>* v1.20.6
>* v1.21.0
## Protecting tenants from deletion
Sometimes it is important to protect business critical tenants from accidental deletion.
This can be achieved by adding `capsule.clastix.io/protected` annotation on the tenant:
```yaml
kubectl apply -f - << EOF
apiVersion: capsule.clastix.io/v1beta1
kind: Tenant
metadata:
name: oil
annotations:
capsule.clastix.io/protected: ""
spec:
owners:
- name: alice
kind: User
EOF
```
---
This ends our tutorial on how to implement complex multi-tenancy and policy-driven scenarios with Capsule. As we improve it, more use cases about multi-tenancy, policy admission control, and cluster governance will be covered in the future.

Binary file not shown.

After

Width:  |  Height:  |  Size: 85 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 72 KiB

Some files were not shown because too many files have changed in this diff Show More