Initial commit

This commit is contained in:
Dario Tranchitella
2020-06-29 22:27:53 +02:00
commit 812b16fcff
85 changed files with 7280 additions and 0 deletions

12
.github/FUNDING.yml vendored Normal file
View File

@@ -0,0 +1,12 @@
# These are supported funding model platforms
github: [prometherion]
patreon: # Replace with a single Patreon username
open_collective: # Replace with a single Open Collective username
ko_fi: # Replace with a single Ko-fi username
tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
liberapay: # Replace with a single Liberapay username
issuehunt: # Replace with a single IssueHunt username
otechie: # Replace with a single Otechie username
custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2']

43
.github/ISSUE_TEMPLATE/bug_report.md vendored Normal file
View File

@@ -0,0 +1,43 @@
---
name: Bug report
about: Create a report to help us improve Capsule
title: ''
labels: blocked-needs-validation, bug
assignees: ''
---
<!--
Thanks for taking time reporting a Capsule bug!
We do our best to keep it reliable and working, so don't hesitate adding
as many information as you can and keep in mind you can reach us on our
Clastix Slack workspace: https://clastix.slack.com, #capsule channel.
-->
# Bug description
A clear and concise description of what the bug is.
# How to reproduce
Steps to reproduce the behaviour:
1. Provide the Capsule Tenant YAML definitions
2. Provide all managed Kubernetes resources
# Expected behavior
A clear and concise description of what you expected to happen.
# Logs
If applicable, please provide logs of `capsule`.
In a standard stand-alone installation of Capsule,
you'd get this by running `kubectl -n capsule-system logs deploy/capsule`.
# Additional context
- Capsule version: (`capsule --version`)
- Kubernetes version: (`kubectl version`)

View File

@@ -0,0 +1,37 @@
---
name: Feature request
about: Suggest a new feature for Capsule
title: ''
labels: blocked-needs-validation, feature
assignees: ''
---
<!--
Yay, it look you're enjoying Capsule and, first, thanks for that!
We're trying to build a community drive Open Source project, so don't
hesitate proposing your enhancement ideas: keep in mind, since we would like
to keep it as agnostic as possible, to motivate all your assumptions.
If you need to reach the maintainers, please join the Clastix Slack workspace:
https://clastix.slack.com, #capsule channel.
-->
# Describe the feature
A clear and concise description of the feature.
# What would the new user story look like?
How would the new interaction with Capsule look like? E.g.
1. What are the prerequisites for this?
2. Tenant owner creates a new _Namespace_
3. This is going to be attached to the _Tenant_
4. All the magic happens in the background
Feel free to add a diagram if that helps explain things.
# Expected behavior
A clear and concise description of what you expect to happen.

20
.github/PULL_REQUEST_TEMPLATE.md vendored Normal file
View File

@@ -0,0 +1,20 @@
<!--
# General contribution criteria
Thanks for spending some time for improving and fixing Capsule!
We're still working on the outline of the contribution guidelines but we're
following ourselves these points:
- reference a previously opened issue: https://docs.github.com/en/github/writing-on-github/autolinked-references-and-urls#issues-and-pull-requests
- including a sentence or two in the commit description for the
changelog/release notes
- splitting changes into several and documented small commits
- limit the git subject to 50 characters and write as the continuation of the
sentence "If applied, this commit will ..."
- explain what and why in the body, if more than a trivial change, wrapping at
72 characters
If you have any issue or question, reach out us!
https://clastix.slack.com >>> #capsule channel
-->

13
.github/workflows/main.yaml vendored Normal file
View File

@@ -0,0 +1,13 @@
name: golangci-lint
on:
pull_request:
jobs:
golangci:
name: lint
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: golangci-lint
uses: golangci/golangci-lint-action@v1
with:
version: v1.29

78
.gitignore vendored Normal file
View File

@@ -0,0 +1,78 @@
# Temporary Build Files
build/_output
build/_test
# Created by https://www.gitignore.io/api/go,vim,emacs,visualstudiocode
### Emacs ###
# -*- mode: gitignore; -*-
*~
\#*\#
/.emacs.desktop
/.emacs.desktop.lock
*.elc
auto-save-list
tramp
.\#*
# Org-mode
.org-id-locations
*_archive
# flymake-mode
*_flymake.*
# eshell files
/eshell/history
/eshell/lastdir
# elpa packages
/elpa/
# reftex files
*.rel
# AUCTeX auto folder
/auto/
# cask packages
.cask/
dist/
# Flycheck
flycheck_*.el
# server auth directory
/server/
# projectiles files
.projectile
projectile-bookmarks.eld
# directory configuration
.dir-locals.el
# saveplace
places
# url cache
url/cache/
# cedet
ede-projects.el
# smex
smex-items
# company-statistics
company-statistics-cache.el
# anaconda-mode
anaconda-mode/
### Go ###
# Binaries for programs and plugins
*.exe
*.exe~
*.dll
*.so
*.dylib
# Test binary, build with 'go test -c'
*.test
# Output of the go coverage tool, specifically when used with LiteIDE
*.out
### Vim ###
# swap
.sw[a-p]
.*.sw[a-p]
# session
Session.vim
# temporary
.netrwhist
# auto-generated tag files
tags
### VisualStudioCode ###
.vscode/*
.history
# End of https://www.gitignore.io/api/go,vim,emacs,visualstudiocode
.idea

201
LICENSE Normal file
View File

@@ -0,0 +1,201 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [2020] Clastix Labs
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

7
Makefile Normal file
View File

@@ -0,0 +1,7 @@
.PHONY: k8s
k8s:
operator-sdk generate k8s
.PHONY: crds
crds:
operator-sdk generate crds

118
README.md Normal file
View File

@@ -0,0 +1,118 @@
# ![icon](assets/logo/space-capsule3.png) Capsule
# A Kubernetes multi-tenant operator
This project aims to provide a custom operator for implementing a strong
multi-tenant environment in _Kubernetes_, especially suited for public
_Container-as-a-Service_ (CaaS) platforms.
# tl;dr; How to install
As a Cluster Admin, ensure the `capsule-system` Namespace is already there.
```
# kubectl apply -f deploy
mutatingwebhookconfiguration.admissionregistration.k8s.io/capsule created
clusterrole.rbac.authorization.k8s.io/namespace:deleter created
clusterrole.rbac.authorization.k8s.io/namespace:provisioner created
clusterrolebinding.rbac.authorization.k8s.io/namespace:provisioner created
deployment.apps/capsule created
clusterrole.rbac.authorization.k8s.io/capsule created
clusterrolebinding.rbac.authorization.k8s.io/capsule-cluster-admin created
clusterrolebinding.rbac.authorization.k8s.io/capsule created
secret/capsule-ca created
secret/capsule-tls created
service/capsule created
serviceaccount/capsule created
# kubectl apply -f deploy/crds/capsule.clastix.io_tenants_crd.yaml
customresourcedefinition.apiextensions.k8s.io/tenants.capsule.clastix.io created
```
## Webhooks and CA Bundle
Capsule is leveraging Kubernetes Multi-Tenant capabilities using the
[Dynamic Admission Controller](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/),
providing callbacks to add further validation or resource patching.
All this requests must be server via HTTPS and a CA must be provided to ensure that
the API Server is communicating with the right client.
Capsule upon installation is setting its custom Certificate Authority as
client certificate as well, updating all the required resources to minimize
the operational tasks.
## Tenant users
All Tenant owner needs to be granted with a X.509 certificate with
`capsule.clastix.io` as _Organization_.
> the [hack/create-user.sh](hack/create-user.sh) can help you setting up a
> dummy kubeconfig
>
> ```
> #. /create-user.sh alice oil
> creating certs in TMPDIR /tmp/tmp.4CLgpuime3
> Generating RSA private key, 2048 bit long modulus (2 primes)
> ............+++++
> ........................+++++
> e is 65537 (0x010001)
> certificatesigningrequest.certificates.k8s.io/alice-oil created
> certificatesigningrequest.certificates.k8s.io/alice-oil approved
> kubeconfig file is: alice-oil.kubeconfig
> to use it as alice export KUBECONFIG=alice-oil.kubeconfig
> ```
## How to create a Tenant
Use the [scaffold Tenant](deploy/crds/capsule.clastix.io_v1alpha1_tenant_cr.yaml)
and simply apply as Cluster Admin.
```
# kubectl apply -f deploy/crds/capsule.clastix.io_v1alpha1_tenant_cr.yaml
tenant.capsule.clastix.io/oil created
```
The related Tenant owner can create Namespaces according to their quota:
happy Kubernetes cluster administration!
# Which is the problem to solve?
Kubernetes uses _Namespace_ resources to create logical partitions of the
cluster. A Kubernetes namespace provides the scope for some kind of resources
in the cluster. Users interacting with one namespace do not see the content in
another Namespace.
Kubernetes comes with few Namespace resources and leave the administrator to
create further namespaces in order to create sort of isolated *slices* of the
cluster: _Network and Security Policies_, _Resource Quota_, _Limit Ranges_, and
_RBAC_ are used to enforce isolation among namespaces.
Namespace isolation shines when Kubernetes is used as an enterprise container
platform, for example, to isolate the production environment from the
development and/or to isolate different types of applications.
Also it works well to isolate applications serving different users when
implementing the SaaS business model.
When implementing a public _CaaS_ platform, the flat namespace structure in
Kubernetes shows its main limitations. In this model, each new user receives
their own namespace where to deploy workloads. The user buys a limited amount
of resources (e.g.: _vCPU_, _RAM_, _ephemeral and persistent storage_) and
cannot use more than that.
If the user needs for multiple namespaces, they can buy other namespaces.
However, resources cannot shared easily between namespaces which still work as
fully isolated environments.
_Capsule_ aggregates multiple namespaces belonging to the same user by leaving
the user to freely share resources among all their namespaces.
All the constraints, defined by _Network and Security Policies_,
_Resource Quota_, _Limit Ranges_, and RBAC can be freely shared between
namespaces in a fully self-provisioning fashion without any intervention of the
cluster admin.
# Use cases for Capsule
Please refer to the corresponding [section](use_cases.md)
# Production Grade status
Capsule is still in an _alpha_ stage, so **don't use it in production**!

View File

@@ -0,0 +1 @@
Icons made by [Roundicons](https://www.flaticon.com/authors/roundicons) from [www.flaticon.com](https://www.flaticon.com).

Binary file not shown.

After

Width:  |  Height:  |  Size: 18 KiB

View File

@@ -0,0 +1,107 @@
<?xml version="1.0" encoding="iso-8859-1"?>
<!-- Generator: Adobe Illustrator 19.0.0, SVG Export Plug-In . SVG Version: 6.00 Build 0) -->
<svg version="1.1" id="Layer_1" xmlns="http://www.w3.org/2000/svg" x="0px" y="0px"
viewBox="0 0 504.123 504.123" style="enable-background:new 0 0 504.123 504.123;" xml:space="preserve">
<path style="fill:#2477AA;" d="M265.665,468.582c0.378-1.276,0.646-2.615,0.646-4.033v-63.827c0-7.483-5.805-13.564-13.162-14.131
l0,0c-0.354-0.039-0.709-0.118-1.087-0.11c-7.869,0-14.249,6.372-14.249,14.241v63.835c0,1.41,0.268,2.749,0.646,4.025
c-4.112,1.772-7.026,5.868-7.026,10.65v3.812v3.419c0,6.396,5.175,11.587,11.579,11.587h18.093c6.396,0,11.571-5.183,11.571-11.587
v-7.231C272.675,474.451,269.777,470.355,265.665,468.582z"/>
<ellipse style="fill:#7BC6C6;" cx="252.062" cy="85.851" rx="7.404" ry="67.245"/>
<circle style="fill:#FF7F00;" cx="252.062" cy="269.722" r="183.863"/>
<path style="fill:#FF5B00;" d="M252.062,85.858c101.541,0,183.863,82.322,183.863,183.863c0,101.557-82.322,183.879-183.863,183.879
"/>
<path style="fill:#25618E;" d="M110.222,386.733h283.672c25.647-31.051,41.173-70.719,41.874-113.971H68.348
C69.065,316.014,84.582,355.675,110.222,386.733z"/>
<g>
<path style="fill:#18456D;" d="M252.062,272.762v113.971h141.832c0-0.008,0.024-0.024,0.031-0.039
c0.055-0.063,0.095-0.142,0.165-0.197c3.411-4.151,6.609-8.476,9.657-12.926c1.166-1.686,2.198-3.45,3.277-5.167
c1.827-2.851,3.631-5.742,5.309-8.704c1.26-2.229,2.41-4.537,3.584-6.829c1.276-2.505,2.489-5.049,3.671-7.625
c1.197-2.67,2.355-5.38,3.426-8.113c0.874-2.221,1.678-4.474,2.465-6.735c1.087-3.119,2.15-6.246,3.072-9.429
c0.512-1.757,0.922-3.545,1.378-5.325c3.497-13.674,5.585-27.932,5.837-42.646c0-0.079,0-0.15,0.016-0.221H252.062V272.762z"/>
<path style="fill:#18456D;" d="M152.174,298.977c0,1.883-1.528,3.426-3.419,3.426h-28.499c-1.883,0-3.419-1.536-3.419-3.426l0,0
c0-1.883,1.528-3.426,3.419-3.426h28.499C150.646,295.55,152.174,297.094,152.174,298.977L152.174,298.977z"/>
<path style="fill:#18456D;" d="M152.174,318.354c0,1.883-1.528,3.419-3.419,3.419h-28.499c-1.883,0-3.419-1.528-3.419-3.419l0,0
c0-1.89,1.528-3.426,3.419-3.426h28.499C150.646,314.927,152.174,316.463,152.174,318.354L152.174,318.354z"/>
<path style="fill:#18456D;" d="M152.174,337.723c0,1.89-1.528,3.426-3.419,3.426h-28.499c-1.883,0-3.419-1.528-3.419-3.426l0,0
c0-1.883,1.528-3.419,3.419-3.419h28.499C150.646,334.305,152.174,335.841,152.174,337.723L152.174,337.723z"/>
</g>
<path style="fill:#25618E;" d="M380.109,352.54c0,10.075-8.153,18.235-18.219,18.235h-67.253c-10.059,0-18.219-8.16-18.219-18.235
v-45.584c0-10.067,8.16-18.227,18.219-18.227h67.253c10.067,0,18.219,8.16,18.219,18.227V352.54z"/>
<g>
<path style="fill:#3479A3;" d="M367.577,347.034c0,7.633-6.183,13.832-13.824,13.832h-50.987c-7.641,0-13.832-6.199-13.832-13.832
v-34.572c0-7.633,6.191-13.824,13.832-13.824h50.987c7.641,0,13.824,6.191,13.824,13.824V347.034z"/>
<path style="fill:#3479A3;" d="M289.666,81.865c0,7.239-5.868,13.107-13.107,13.107h-49.01c-7.231,0-13.099-5.868-13.099-13.107
l0,0c0-7.239,5.868-13.107,13.099-13.107h49.01C283.798,68.758,289.666,74.626,289.666,81.865L289.666,81.865z"/>
</g>
<path style="fill:#18456D;" d="M276.559,68.758c7.239,0,13.107,5.868,13.107,13.107l0,0c0,7.239-5.868,13.107-13.107,13.107h-49.01
c-7.231,0-13.099-5.868-13.099-13.107l0,0"/>
<circle style="fill:#B4E7ED;" cx="252.062" cy="152.718" r="14.438"/>
<path style="fill:#7BC6C6;" d="M252.062,138.279c7.979,0,14.438,6.467,14.438,14.438s-6.459,14.438-14.438,14.438"/>
<circle style="fill:#B4E7ED;" cx="252.062" cy="198.309" r="14.438"/>
<path style="fill:#7BC6C6;" d="M252.062,183.871c7.979,0,14.438,6.459,14.438,14.438c0,7.971-6.459,14.438-14.438,14.438"/>
<circle style="fill:#B4E7ED;" cx="252.069" cy="14.438" r="14.438"/>
<path style="fill:#7BC6C6;" d="M262.262,4.23c5.648,5.64,5.648,14.785,0.016,20.417c-5.64,5.632-14.785,5.632-20.417,0"/>
<circle style="fill:#B4E7ED;" cx="252.062" cy="243.893" r="14.438"/>
<g>
<path style="fill:#7BC6C6;" d="M252.062,229.455c7.979,0,14.438,6.467,14.438,14.438s-6.459,14.438-14.438,14.438"/>
<path style="fill:#7BC6C6;" d="M353.319,332.312c0,2.056-1.646,3.71-3.694,3.71h-13.107c-2.048,0-3.71-1.654-3.71-3.71l0,0
c0-2.048,1.662-3.702,3.71-3.702h13.107C351.673,328.609,353.319,330.264,353.319,332.312L353.319,332.312z"/>
</g>
<path style="fill:#FF5B00;" d="M185.194,440.95c-0.457-18.692-14.612-33.705-32.106-33.705c-6.231,0-12.012,2.001-16.951,5.309
C150.772,424.432,167.329,433.995,185.194,440.95z"/>
<path style="fill:#7BC6C6;" d="M161.225,412.782c5.561,5.569,5.561,14.588,0,20.157l-45.127,45.127
c-5.569,5.569-14.588,5.569-20.149,0l0,0c-5.561-5.553-5.561-14.58,0-20.149l45.135-45.127
C146.637,407.221,155.656,407.221,161.225,412.782L161.225,412.782z"/>
<path style="fill:#8DD8D6;" d="M141.084,412.782l-45.135,45.127c-1,1.008-1.78,2.143-2.41,3.34c0.228,0.276,0.433,0.583,0.693,0.851
c5.585,5.569,14.588,5.569,20.157,0l45.127-45.127c1.016-1.008,1.764-2.143,2.41-3.34c-0.236-0.284-0.433-0.583-0.701-0.851
C155.656,407.221,146.637,407.221,141.084,412.782z"/>
<path style="fill:#25618E;" d="M130.859,485.888c0,10.067-8.153,18.235-18.227,18.235H84.141c-10.059,0-18.235-8.168-18.235-18.235
v-11.39c0-10.075,8.176-18.235,18.235-18.235h28.491c10.075,0,18.227,8.16,18.227,18.235V485.888z"/>
<path style="fill:#2477AA;" d="M117.752,457.074c-1.638-0.488-3.332-0.819-5.12-0.819H84.141c-10.059,0-18.235,8.16-18.235,18.235
v6.018c1.638,0.48,3.332,0.819,5.128,0.819h28.491c10.075,0,18.227-8.16,18.227-18.227V457.074z"/>
<path style="fill:#FF7F00;" d="M318.921,440.95c0.465-18.692,14.62-33.705,32.114-33.705c6.223,0,12.012,2.001,16.951,5.309
C353.351,424.432,336.786,433.995,318.921,440.95z"/>
<path style="fill:#7BC6C6;" d="M342.898,412.782c-5.561,5.569-5.561,14.588,0,20.157l45.127,45.127
c5.577,5.569,14.588,5.569,20.157,0l0,0c5.561-5.553,5.561-14.58,0-20.149L363.04,412.79
C357.486,407.221,348.459,407.221,342.898,412.782L342.898,412.782z"/>
<path style="fill:#8DD8D6;" d="M363.032,412.782l45.143,45.127c1,1.008,1.772,2.143,2.41,3.34c-0.228,0.276-0.433,0.583-0.693,0.851
c-5.577,5.569-14.588,5.569-20.165,0L344.6,416.973c-1.016-1.008-1.764-2.143-2.41-3.34c0.236-0.284,0.433-0.583,0.701-0.851
C348.459,407.221,357.486,407.221,363.032,412.782z"/>
<path style="fill:#25618E;" d="M373.264,485.888c0,10.067,8.153,18.235,18.227,18.235h28.491c10.059,0,18.235-8.168,18.235-18.235
v-11.39c0-10.075-8.176-18.235-18.235-18.235h-28.491c-10.075,0-18.227,8.16-18.227,18.235V485.888z"/>
<path style="fill:#2477AA;" d="M386.371,457.074c1.638-0.488,3.332-0.819,5.12-0.819h28.491c10.059,0,18.235,8.16,18.235,18.235
v6.018c-1.638,0.48-3.332,0.819-5.128,0.819h-28.491c-10.075,0-18.227-8.16-18.227-18.227V457.074z"/>
<path style="fill:#B4E7ED;" d="M72.428,230.747c1.788,0.591,3.679,0.985,5.671,0.985h98.013c10.067,0,18.219-8.168,18.219-18.235
V95.232C133.152,115.468,86.221,166.896,72.428,230.747z"/>
<path style="fill:#7BC6C6;" d="M78.1,231.731h98.013c10.067,0,18.219-8.168,18.219-18.235V95.232"/>
<g>
</g>
<g>
</g>
<g>
</g>
<g>
</g>
<g>
</g>
<g>
</g>
<g>
</g>
<g>
</g>
<g>
</g>
<g>
</g>
<g>
</g>
<g>
</g>
<g>
</g>
<g>
</g>
<g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 7.1 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.4 KiB

15
build/Dockerfile Normal file
View File

@@ -0,0 +1,15 @@
FROM registry.access.redhat.com/ubi8/ubi-minimal:latest
ENV OPERATOR=/usr/local/bin/capsule \
USER_UID=0 \
USER_NAME=capsule
# install operator binary
COPY build/_output/bin/capsule ${OPERATOR}
COPY build/bin /usr/local/bin
RUN /usr/local/bin/user_setup
ENTRYPOINT ["/usr/local/bin/entrypoint"]
USER ${USER_UID}

3
build/bin/entrypoint Executable file
View File

@@ -0,0 +1,3 @@
#!/bin/sh -e
exec ${OPERATOR} $@

11
build/bin/user_setup Executable file
View File

@@ -0,0 +1,11 @@
#!/bin/sh
set -x
# ensure $HOME exists and is accessible by group 0 (we don't know what the runtime UID will be)
echo "${USER_NAME}:x:${USER_UID}:0:${USER_NAME} user:${HOME}:/sbin/nologin" >> /etc/passwd
mkdir -p "${HOME}"
chown "${USER_UID}:0" "${HOME}"
chmod ug+rwx "${HOME}"
# no need for this script to remain in the image after running
rm "$0"

246
cmd/manager/main.go Normal file
View File

@@ -0,0 +1,246 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package main
import (
"context"
"errors"
"flag"
"fmt"
"os"
"runtime"
"strings"
"github.com/operator-framework/operator-sdk/pkg/k8sutil"
kubemetrics "github.com/operator-framework/operator-sdk/pkg/kube-metrics"
"github.com/operator-framework/operator-sdk/pkg/leader"
"github.com/operator-framework/operator-sdk/pkg/log/zap"
"github.com/operator-framework/operator-sdk/pkg/metrics"
sdkVersion "github.com/operator-framework/operator-sdk/version"
"github.com/spf13/pflag"
v1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/util/intstr"
_ "k8s.io/client-go/plugin/pkg/client/auth"
"k8s.io/client-go/rest"
"sigs.k8s.io/controller-runtime/pkg/cache"
"sigs.k8s.io/controller-runtime/pkg/client/config"
logf "sigs.k8s.io/controller-runtime/pkg/log"
"sigs.k8s.io/controller-runtime/pkg/manager"
"sigs.k8s.io/controller-runtime/pkg/manager/signals"
"github.com/clastix/capsule/pkg/apis"
"github.com/clastix/capsule/pkg/controller"
// Import all Kubernetes client auth plugins (e.g. Azure, GCP, OIDC, etc.)
"github.com/clastix/capsule/pkg/indexer"
"github.com/clastix/capsule/pkg/webhook"
"github.com/clastix/capsule/version"
)
// Change below variables to serve metrics on different host or port.
var (
metricsHost = "0.0.0.0"
metricsPort int32 = 8383
operatorMetricsPort int32 = 8686
)
var log = logf.Log.WithName("cmd")
func printVersion() {
log.Info(fmt.Sprintf("Operator Version: %s", version.Version))
log.Info(fmt.Sprintf("Go Version: %s", runtime.Version()))
log.Info(fmt.Sprintf("Go OS/Arch: %s/%s", runtime.GOOS, runtime.GOARCH))
log.Info(fmt.Sprintf("Version of operator-sdk: %v", sdkVersion.Version))
}
func main() {
// Add the zap logger flag set to the CLI. The flag set must
// be added before calling pflag.Parse().
pflag.CommandLine.AddFlagSet(zap.FlagSet())
// Add flags registered by imported packages (e.g. glog and
// controller-runtime)
pflag.CommandLine.AddGoFlagSet(flag.CommandLine)
var v bool
pflag.BoolVarP(&v, "version", "v", false, "Print the Capsule version and exit")
pflag.Parse()
// Use a zap logr.Logger implementation. If none of the zap
// flags are configured (or if the zap flag set is not being
// used), this defaults to a production zap logger.
//
// The logger instantiated here can be changed to any logger
// implementing the logr.Logger interface. This logger will
// be propagated through the whole operator, generating
// uniform and structured logs.
logf.SetLogger(zap.Logger())
printVersion()
if v {
os.Exit(0)
}
namespace, err := k8sutil.GetWatchNamespace()
if err != nil {
log.Error(err, "Failed to get watch namespace")
os.Exit(1)
}
// Get a config to talk to the apiserver
cfg, err := config.GetConfig()
if err != nil {
log.Error(err, "")
os.Exit(1)
}
ctx := context.TODO()
// Become the leader before proceeding
err = leader.Become(ctx, "capsule-lock")
if err != nil {
log.Error(err, "")
os.Exit(1)
}
// Set default manager options
options := manager.Options{
Namespace: namespace,
MetricsBindAddress: fmt.Sprintf("%s:%d", metricsHost, metricsPort),
}
// Add support for MultiNamespace set in WATCH_NAMESPACE (e.g ns1,ns2)
// Note that this is not intended to be used for excluding namespaces, this is better done via a Predicate
// Also note that you may face performance issues when using this with a high number of namespaces.
// More Info: https://godoc.org/github.com/kubernetes-sigs/controller-runtime/pkg/cache#MultiNamespacedCacheBuilder
if strings.Contains(namespace, ",") {
options.Namespace = ""
options.NewCache = cache.MultiNamespacedCacheBuilder(strings.Split(namespace, ","))
}
stop := signals.SetupSignalHandler()
// Create a new manager to provide shared dependencies and start components
mgr, err := manager.New(cfg, options)
if err != nil {
log.Error(err, "")
os.Exit(1)
}
log.Info("Registering Components.")
// Setup Scheme for all resources
if err := apis.AddToScheme(mgr.GetScheme()); err != nil {
log.Error(err, "")
os.Exit(1)
}
// Setup all Controllers
if err := controller.AddToManager(mgr); err != nil {
log.Error(err, "")
os.Exit(1)
}
// Setup all Webhooks
if err := webhook.AddToServer(mgr); err != nil {
log.Error(err, "")
os.Exit(1)
}
// Setup all Custom Indexers
if err := indexer.AddToManager(mgr); err != nil {
log.Error(err, "")
os.Exit(1)
}
// Add the Metrics Service
addMetrics(ctx, cfg)
log.Info("Starting the Cmd.")
// Start the Cmd
if err := mgr.Start(stop); err != nil {
log.Error(err, "Manager exited non-zero")
os.Exit(1)
}
}
// addMetrics will create the Services and Service Monitors to allow the operator export the metrics by using
// the Prometheus operator
func addMetrics(ctx context.Context, cfg *rest.Config) {
// Get the namespace the operator is currently deployed in.
operatorNs, err := k8sutil.GetOperatorNamespace()
if err != nil {
if errors.Is(err, k8sutil.ErrRunLocal) {
log.Info("Skipping CR metrics server creation; not running in a cluster.")
return
}
}
if err := serveCRMetrics(cfg, operatorNs); err != nil {
log.Info("Could not generate and serve custom resource metrics", "error", err.Error())
}
// Add to the below struct any other metrics ports you want to expose.
servicePorts := []v1.ServicePort{
{Port: metricsPort, Name: metrics.OperatorPortName, Protocol: v1.ProtocolTCP, TargetPort: intstr.IntOrString{Type: intstr.Int, IntVal: metricsPort}},
{Port: operatorMetricsPort, Name: metrics.CRPortName, Protocol: v1.ProtocolTCP, TargetPort: intstr.IntOrString{Type: intstr.Int, IntVal: operatorMetricsPort}},
}
// Create Service object to expose the metrics port(s).
service, err := metrics.CreateMetricsService(ctx, cfg, servicePorts)
if err != nil {
log.Info("Could not create metrics Service", "error", err.Error())
}
// CreateServiceMonitors will automatically create the prometheus-operator ServiceMonitor resources
// necessary to configure Prometheus to scrape metrics from this operator.
services := []*v1.Service{service}
// The ServiceMonitor is created in the same namespace where the operator is deployed
_, err = metrics.CreateServiceMonitors(cfg, operatorNs, services)
if err != nil {
log.Info("Could not create ServiceMonitor object", "error", err.Error())
// If this operator is deployed to a cluster without the prometheus-operator running, it will return
// ErrServiceMonitorNotPresent, which can be used to safely skip ServiceMonitor creation.
if err == metrics.ErrServiceMonitorNotPresent {
log.Info("Install prometheus-operator in your cluster to create ServiceMonitor objects", "error", err.Error())
}
}
}
// serveCRMetrics gets the Operator/CustomResource GVKs and generates metrics based on those types.
// It serves those metrics on "http://metricsHost:operatorMetricsPort".
func serveCRMetrics(cfg *rest.Config, operatorNs string) error {
// The function below returns a list of filtered operator/CR specific GVKs. For more control, override the GVK list below
// with your own custom logic. Note that if you are adding third party API schemas, probably you will need to
// customize this implementation to avoid permissions issues.
filteredGVK, err := k8sutil.GetGVKsFromAddToScheme(apis.AddToScheme)
if err != nil {
return err
}
// The metrics will be generated from the namespaces which are returned here.
// NOTE that passing nil or an empty list of namespaces in GenerateAndServeCRMetrics will result in an error.
ns, err := kubemetrics.GetNamespacesForMetrics(operatorNs)
if err != nil {
return err
}
// Generate and serve custom resource specific metrics.
err = kubemetrics.GenerateAndServeCRMetrics(cfg, ns, filteredGVK, metricsHost, operatorMetricsPort)
if err != nil {
return err
}
return nil
}

View File

@@ -0,0 +1,710 @@
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: tenants.capsule.clastix.io
spec:
group: capsule.clastix.io
names:
kind: Tenant
listKind: TenantList
plural: tenants
singular: tenant
scope: Cluster
versions:
- additionalPrinterColumns:
- description: The max amount of Namespaces can be created
jsonPath: .spec.namespaceQuota
name: Namespace quota
type: integer
- description: The total amount of Namespaces in use
jsonPath: .status.size
name: Namespace count
type: integer
name: v1alpha1
schema:
openAPIV3Schema:
description: Tenant is the Schema for the tenants API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: TenantSpec defines the desired state of Tenant
properties:
ingressClasses:
items:
type: string
type: array
limitRanges:
items:
description: LimitRangeSpec defines a min/max usage limit for resources
that match on kind.
properties:
limits:
description: Limits is the list of LimitRangeItem objects that
are enforced.
items:
description: LimitRangeItem defines a min/max usage limit
for any resource that matches on kind.
properties:
default:
additionalProperties:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: Default resource requirement limit value
by resource name if resource limit is omitted.
type: object
defaultRequest:
additionalProperties:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: DefaultRequest is the default resource requirement
request value by resource name if resource request is
omitted.
type: object
max:
additionalProperties:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: Max usage constraints on this kind by resource
name.
type: object
maxLimitRequestRatio:
additionalProperties:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: MaxLimitRequestRatio if specified, the named
resource must have a request and limit that are both
non-zero where limit divided by request is less than
or equal to the enumerated value; this represents the
max burst for the named resource.
type: object
min:
additionalProperties:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: Min usage constraints on this kind by resource
name.
type: object
type:
description: Type of resource that this limit applies
to.
type: string
required:
- type
type: object
type: array
required:
- limits
type: object
type: array
namespaceQuota:
minimum: 1
type: integer
networkPolicies:
items:
description: NetworkPolicySpec provides the specification of a NetworkPolicy
properties:
egress:
description: List of egress rules to be applied to the selected
pods. Outgoing traffic is allowed if there are no NetworkPolicies
selecting the pod (and cluster policy otherwise allows the
traffic), OR if the traffic matches at least one egress rule
across all of the NetworkPolicy objects whose podSelector
matches the pod. If this field is empty then this NetworkPolicy
limits all outgoing traffic (and serves solely to ensure that
the pods it selects are isolated by default). This field is
beta-level in 1.8
items:
description: NetworkPolicyEgressRule describes a particular
set of traffic that is allowed out of pods matched by a
NetworkPolicySpec's podSelector. The traffic must match
both ports and to. This type is beta-level in 1.8
properties:
ports:
description: List of destination ports for outgoing traffic.
Each item in this list is combined using a logical OR.
If this field is empty or missing, this rule matches
all ports (traffic not restricted by port). If this
field is present and contains at least one item, then
this rule allows traffic only if the traffic matches
at least one port in the list.
items:
description: NetworkPolicyPort describes a port to allow
traffic on
properties:
port:
anyOf:
- type: integer
- type: string
description: The port on the given protocol. This
can either be a numerical or named port on a pod.
If this field is not provided, this matches all
port names and numbers.
x-kubernetes-int-or-string: true
protocol:
description: The protocol (TCP, UDP, or SCTP) which
traffic must match. If not specified, this field
defaults to TCP.
type: string
type: object
type: array
to:
description: List of destinations for outgoing traffic
of pods selected for this rule. Items in this list are
combined using a logical OR operation. If this field
is empty or missing, this rule matches all destinations
(traffic not restricted by destination). If this field
is present and contains at least one item, this rule
allows traffic only if the traffic matches at least
one item in the to list.
items:
description: NetworkPolicyPeer describes a peer to allow
traffic from. Only certain combinations of fields
are allowed
properties:
ipBlock:
description: IPBlock defines policy on a particular
IPBlock. If this field is set then neither of
the other fields can be.
properties:
cidr:
description: CIDR is a string representing the
IP Block Valid examples are "192.168.1.1/24"
or "2001:db9::/64"
type: string
except:
description: Except is a slice of CIDRs that
should not be included within an IP Block
Valid examples are "192.168.1.1/24" or "2001:db9::/64"
Except values will be rejected if they are
outside the CIDR range
items:
type: string
type: array
required:
- cidr
type: object
namespaceSelector:
description: "Selects Namespaces using cluster-scoped
labels. This field follows standard label selector
semantics; if present but empty, it selects all
namespaces. \n If PodSelector is also set, then
the NetworkPolicyPeer as a whole selects the Pods
matching PodSelector in the Namespaces selected
by NamespaceSelector. Otherwise it selects all
Pods in the Namespaces selected by NamespaceSelector."
properties:
matchExpressions:
description: matchExpressions is a list of label
selector requirements. The requirements are
ANDed.
items:
description: A label selector requirement
is a selector that contains values, a key,
and an operator that relates the key and
values.
properties:
key:
description: key is the label key that
the selector applies to.
type: string
operator:
description: operator represents a key's
relationship to a set of values. Valid
operators are In, NotIn, Exists and
DoesNotExist.
type: string
values:
description: values is an array of string
values. If the operator is In or NotIn,
the values array must be non-empty.
If the operator is Exists or DoesNotExist,
the values array must be empty. This
array is replaced during a strategic
merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value}
pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions,
whose key field is "key", the operator is
"In", and the values array contains only "value".
The requirements are ANDed.
type: object
type: object
podSelector:
description: "This is a label selector which selects
Pods. This field follows standard label selector
semantics; if present but empty, it selects all
pods. \n If NamespaceSelector is also set, then
the NetworkPolicyPeer as a whole selects the Pods
matching PodSelector in the Namespaces selected
by NamespaceSelector. Otherwise it selects the
Pods matching PodSelector in the policy's own
Namespace."
properties:
matchExpressions:
description: matchExpressions is a list of label
selector requirements. The requirements are
ANDed.
items:
description: A label selector requirement
is a selector that contains values, a key,
and an operator that relates the key and
values.
properties:
key:
description: key is the label key that
the selector applies to.
type: string
operator:
description: operator represents a key's
relationship to a set of values. Valid
operators are In, NotIn, Exists and
DoesNotExist.
type: string
values:
description: values is an array of string
values. If the operator is In or NotIn,
the values array must be non-empty.
If the operator is Exists or DoesNotExist,
the values array must be empty. This
array is replaced during a strategic
merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value}
pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions,
whose key field is "key", the operator is
"In", and the values array contains only "value".
The requirements are ANDed.
type: object
type: object
type: object
type: array
type: object
type: array
ingress:
description: List of ingress rules to be applied to the selected
pods. Traffic is allowed to a pod if there are no NetworkPolicies
selecting the pod (and cluster policy otherwise allows the
traffic), OR if the traffic source is the pod's local node,
OR if the traffic matches at least one ingress rule across
all of the NetworkPolicy objects whose podSelector matches
the pod. If this field is empty then this NetworkPolicy does
not allow any traffic (and serves solely to ensure that the
pods it selects are isolated by default)
items:
description: NetworkPolicyIngressRule describes a particular
set of traffic that is allowed to the pods matched by a
NetworkPolicySpec's podSelector. The traffic must match
both ports and from.
properties:
from:
description: List of sources which should be able to access
the pods selected for this rule. Items in this list
are combined using a logical OR operation. If this field
is empty or missing, this rule matches all sources (traffic
not restricted by source). If this field is present
and contains at least one item, this rule allows traffic
only if the traffic matches at least one item in the
from list.
items:
description: NetworkPolicyPeer describes a peer to allow
traffic from. Only certain combinations of fields
are allowed
properties:
ipBlock:
description: IPBlock defines policy on a particular
IPBlock. If this field is set then neither of
the other fields can be.
properties:
cidr:
description: CIDR is a string representing the
IP Block Valid examples are "192.168.1.1/24"
or "2001:db9::/64"
type: string
except:
description: Except is a slice of CIDRs that
should not be included within an IP Block
Valid examples are "192.168.1.1/24" or "2001:db9::/64"
Except values will be rejected if they are
outside the CIDR range
items:
type: string
type: array
required:
- cidr
type: object
namespaceSelector:
description: "Selects Namespaces using cluster-scoped
labels. This field follows standard label selector
semantics; if present but empty, it selects all
namespaces. \n If PodSelector is also set, then
the NetworkPolicyPeer as a whole selects the Pods
matching PodSelector in the Namespaces selected
by NamespaceSelector. Otherwise it selects all
Pods in the Namespaces selected by NamespaceSelector."
properties:
matchExpressions:
description: matchExpressions is a list of label
selector requirements. The requirements are
ANDed.
items:
description: A label selector requirement
is a selector that contains values, a key,
and an operator that relates the key and
values.
properties:
key:
description: key is the label key that
the selector applies to.
type: string
operator:
description: operator represents a key's
relationship to a set of values. Valid
operators are In, NotIn, Exists and
DoesNotExist.
type: string
values:
description: values is an array of string
values. If the operator is In or NotIn,
the values array must be non-empty.
If the operator is Exists or DoesNotExist,
the values array must be empty. This
array is replaced during a strategic
merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value}
pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions,
whose key field is "key", the operator is
"In", and the values array contains only "value".
The requirements are ANDed.
type: object
type: object
podSelector:
description: "This is a label selector which selects
Pods. This field follows standard label selector
semantics; if present but empty, it selects all
pods. \n If NamespaceSelector is also set, then
the NetworkPolicyPeer as a whole selects the Pods
matching PodSelector in the Namespaces selected
by NamespaceSelector. Otherwise it selects the
Pods matching PodSelector in the policy's own
Namespace."
properties:
matchExpressions:
description: matchExpressions is a list of label
selector requirements. The requirements are
ANDed.
items:
description: A label selector requirement
is a selector that contains values, a key,
and an operator that relates the key and
values.
properties:
key:
description: key is the label key that
the selector applies to.
type: string
operator:
description: operator represents a key's
relationship to a set of values. Valid
operators are In, NotIn, Exists and
DoesNotExist.
type: string
values:
description: values is an array of string
values. If the operator is In or NotIn,
the values array must be non-empty.
If the operator is Exists or DoesNotExist,
the values array must be empty. This
array is replaced during a strategic
merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value}
pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions,
whose key field is "key", the operator is
"In", and the values array contains only "value".
The requirements are ANDed.
type: object
type: object
type: object
type: array
ports:
description: List of ports which should be made accessible
on the pods selected for this rule. Each item in this
list is combined using a logical OR. If this field is
empty or missing, this rule matches all ports (traffic
not restricted by port). If this field is present and
contains at least one item, then this rule allows traffic
only if the traffic matches at least one port in the
list.
items:
description: NetworkPolicyPort describes a port to allow
traffic on
properties:
port:
anyOf:
- type: integer
- type: string
description: The port on the given protocol. This
can either be a numerical or named port on a pod.
If this field is not provided, this matches all
port names and numbers.
x-kubernetes-int-or-string: true
protocol:
description: The protocol (TCP, UDP, or SCTP) which
traffic must match. If not specified, this field
defaults to TCP.
type: string
type: object
type: array
type: object
type: array
podSelector:
description: Selects the pods to which this NetworkPolicy object
applies. The array of ingress rules is applied to any pods
selected by this field. Multiple network policies can select
the same set of pods. In this case, the ingress rules for
each are combined additively. This field is NOT optional and
follows standard label selector semantics. An empty podSelector
matches all pods in this namespace.
properties:
matchExpressions:
description: matchExpressions is a list of label selector
requirements. The requirements are ANDed.
items:
description: A label selector requirement is a selector
that contains values, a key, and an operator that relates
the key and values.
properties:
key:
description: key is the label key that the selector
applies to.
type: string
operator:
description: operator represents a key's relationship
to a set of values. Valid operators are In, NotIn,
Exists and DoesNotExist.
type: string
values:
description: values is an array of string values.
If the operator is In or NotIn, the values array
must be non-empty. If the operator is Exists or
DoesNotExist, the values array must be empty. This
array is replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs.
A single {key,value} in the matchLabels map is equivalent
to an element of matchExpressions, whose key field is
"key", the operator is "In", and the values array contains
only "value". The requirements are ANDed.
type: object
type: object
policyTypes:
description: List of rule types that the NetworkPolicy relates
to. Valid options are "Ingress", "Egress", or "Ingress,Egress".
If this field is not specified, it will default based on the
existence of Ingress or Egress rules; policies that contain
an Egress section are assumed to affect Egress, and all policies
(whether or not they contain an Ingress section) are assumed
to affect Ingress. If you want to write an egress-only policy,
you must explicitly specify policyTypes [ "Egress" ]. Likewise,
if you want to write a policy that specifies that no egress
is allowed, you must specify a policyTypes value that include
"Egress" (since such a policy would not include an Egress
section and would otherwise default to just [ "Ingress" ]).
This field is beta-level in 1.8
items:
description: Policy Type string describes the NetworkPolicy
type This type is beta-level in 1.8
type: string
type: array
required:
- podSelector
type: object
type: array
nodeSelector:
additionalProperties:
type: string
type: object
owner:
type: string
resourceQuotas:
items:
description: ResourceQuotaSpec defines the desired hard limits to
enforce for Quota.
properties:
hard:
additionalProperties:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'hard is the set of desired hard limits for each
named resource. More info: https://kubernetes.io/docs/concepts/policy/resource-quotas/'
type: object
scopeSelector:
description: scopeSelector is also a collection of filters like
scopes that must match each object tracked by a quota but
expressed using ScopeSelectorOperator in combination with
possible values. For a resource to match, both scopes AND
scopeSelector (if specified in spec), must be matched.
properties:
matchExpressions:
description: A list of scope selector requirements by scope
of the resources.
items:
description: A scoped-resource selector requirement is
a selector that contains values, a scope name, and an
operator that relates the scope name and values.
properties:
operator:
description: Represents a scope's relationship to
a set of values. Valid operators are In, NotIn,
Exists, DoesNotExist.
type: string
scopeName:
description: The name of the scope that the selector
applies to.
type: string
values:
description: An array of string values. If the operator
is In or NotIn, the values array must be non-empty.
If the operator is Exists or DoesNotExist, the values
array must be empty. This array is replaced during
a strategic merge patch.
items:
type: string
type: array
required:
- operator
- scopeName
type: object
type: array
type: object
scopes:
description: A collection of filters that must match each object
tracked by a quota. If not specified, the quota matches all
objects.
items:
description: A ResourceQuotaScope defines a filter that must
match each object tracked by a quota
type: string
type: array
type: object
type: array
storageClasses:
items:
type: string
type: array
required:
- ingressClasses
- limitRanges
- namespaceQuota
- owner
- storageClasses
type: object
status:
description: TenantStatus defines the observed state of Tenant
properties:
groups:
items:
type: string
type: array
namespaces:
items:
type: string
type: array
size:
type: integer
users:
items:
type: string
type: array
required:
- size
type: object
type: object
served: true
storage: true
subresources:
status: {}

View File

@@ -0,0 +1,86 @@
---
apiVersion: capsule.clastix.io/v1alpha1
kind: Tenant
metadata:
name: oil
spec:
ingressClasses:
- default
limitRanges:
-
limits:
-
max:
cpu: "1"
memory: 1Gi
min:
cpu: 50m
memory: 5Mi
type: Pod
-
default:
cpu: 200m
memory: 100Mi
defaultRequest:
cpu: 100m
memory: 10Mi
max:
cpu: "1"
memory: 1Gi
min:
cpu: 50m
memory: 5Mi
type: Container
-
max:
storage: 10Gi
min:
storage: 1Gi
type: PersistentVolumeClaim
namespaceQuota: 3
networkPolicies:
-
egress:
-
to:
-
ipBlock:
cidr: 0.0.0.0/0
except:
- 192.168.0.0/12
ingress:
-
from:
-
namespaceSelector:
matchLabels:
capsule.clastix.io/tenant: oil
-
podSelector: {}
-
ipBlock:
cidr: 192.168.0.0/12
podSelector: {}
policyTypes:
- Ingress
- Egress
nodeSelector:
kubernetes.io/os: linux
owner: alice
resourceQuotas:
-
hard:
limits.cpu: "8"
limits.memory: 16Gi
requests.cpu: "8"
requests.memory: 16Gi
scopes:
- NotTerminating
-
hard:
pods: "10"
-
hard:
requests.storage: 100Gi
storageClasses:
- standard

View File

@@ -0,0 +1,96 @@
apiVersion: admissionregistration.k8s.io/v1beta1
kind: MutatingWebhookConfiguration
metadata:
name: capsule
webhooks:
- name: owner.namespace.capsule.clastix.io
failurePolicy: Fail
rules:
- apiGroups: [""]
apiVersions: ["v1"]
operations: ["CREATE"]
resources: ["namespaces"]
clientConfig:
# use url if you're developing locally
# url: https://<FIXME>.ngrok.io/mutate-v1-namespace-owner-reference
caBundle:
service:
namespace: capsule-system
name: capsule
path: /mutate-v1-namespace-owner-reference
- name: quota.namespace.capsule.clastix.io
failurePolicy: Fail
rules:
- apiGroups: [""]
apiVersions: ["v1"]
operations: ["CREATE"]
resources: ["namespaces"]
clientConfig:
# use url if you're developing locally
# url: https://<FIXME>.ngrok.io/validate-v1-namespace-quota
caBundle:
service:
namespace: capsule-system
name: capsule
path: /validate-v1-namespace-quota
- name: validating.network-policy.capsule.clastix.io
failurePolicy: Fail
rules:
- apiGroups: ["networking.k8s.io"]
apiVersions: ["v1"]
operations: ["CREATE", "UPDATE", "DELETE"]
resources: ["networkpolicies"]
clientConfig:
# use url if you're developing locally
# url: https://<FIXME>.ngrok.io/validating-v1-network-policy
caBundle:
service:
namespace: capsule-system
name: capsule
path: /validating-v1-network-policy
- name: pvc.capsule.clastix.io
failurePolicy: Fail
rules:
- apiGroups: [""]
apiVersions: ["v1"]
operations: ["CREATE"]
resources: ["persistentvolumeclaims"]
clientConfig:
# use url if you're developing locally
# url: https://<FIXME>.ngrok.io/validating-v1-pvc
caBundle:
service:
namespace: capsule-system
name: capsule
path: /validating-v1-pvc
- name: extensions.ingress.capsule.clastix.io
failurePolicy: Fail
rules:
- apiGroups: ["extensions"]
apiVersions: ["v1beta1"]
operations: ["CREATE", "UPDATE"]
resources: ["ingresses"]
clientConfig:
# use url if you're developing locally
# url: https://<FIXME>.ngrok.io/validating-v1-extensions-ingress
caBundle:
service:
namespace: capsule-system
name: capsule
path: /validating-v1-extensions-ingress
- name: networking.ingress.capsule.clastix.io
failurePolicy: Fail
rules:
- apiGroups: ["networking.k8s.io"]
apiVersions: ["v1beta1"]
operations: ["CREATE", "UPDATE"]
resources: ["ingresses"]
clientConfig:
# use url if you're developing locally
# url: https://<FIXME>.ngrok.io/validating-v1-networking-ingress
caBundle:
service:
namespace: capsule-system
name: capsule
path: /validating-v1-networking-ingress

View File

@@ -0,0 +1,8 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: namespace:deleter
rules:
- apiGroups: [""]
resources: ["namespaces"]
verbs: ["delete"]

View File

@@ -0,0 +1,22 @@
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
labels:
name: namespace:provisioner
rules:
- apiGroups: [""]
resources: ["namespaces"]
verbs: ["create"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: namespace:provisioner
subjects:
- kind: Group
name: capsule.clastix.io
roleRef:
kind: ClusterRole
name: namespace:provisioner
apiGroup: rbac.authorization.k8s.io

38
deploy/operator.yaml Normal file
View File

@@ -0,0 +1,38 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: capsule
namespace: capsule-system
spec:
replicas: 1
selector:
matchLabels:
name: capsule
template:
metadata:
labels:
name: capsule
spec:
serviceAccountName: capsule
containers:
- name: capsule
image: quay.io/clastix/capsule:latest
command:
- capsule
imagePullPolicy: IfNotPresent
env:
- name: WATCH_NAMESPACE
value: ""
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: OPERATOR_NAME
value: "capsule"
volumeMounts:
- name: tls
mountPath: /tmp/k8s-webhook-server/serving-certs
volumes:
- name: tls
secret:
secretName: capsule-tls

96
deploy/role.yaml Normal file
View File

@@ -0,0 +1,96 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
creationTimestamp: null
name: capsule
rules:
- apiGroups:
- ""
resources:
- pods
- replicasets
verbs:
- get
- apiGroups:
- ""
resources:
- configmaps
verbs:
- get
- create
- apiGroups:
- admissionregistration.k8s.io
resources:
- mutatingwebhookconfigurations
verbs:
- get
- list
- watch
- update
- patch
- apiGroups:
- ""
resources:
- limitranges
- resourcequotas
- namespaces
- secrets
verbs:
- create
- delete
- deletecollection
- get
- list
- patch
- update
- watch
- apiGroups:
- rbac.authorization.k8s.io
resources:
- rolebindings
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- extensions.k8s.io
resources:
- ingresses
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- networking.k8s.io
resources:
- networkpolicies
- ingresses
verbs:
- create
- delete
- deletecollection
- get
- list
- patch
- update
- watch
- apiGroups:
- capsule.clastix.io
resources:
- '*'
verbs:
- create
- delete
- get
- list
- patch
- update
- watch

25
deploy/role_binding.yaml Normal file
View File

@@ -0,0 +1,25 @@
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: capsule-cluster-admin
subjects:
- kind: ServiceAccount
name: capsule
namespace: capsule-system
roleRef:
kind: ClusterRole
name: admin
apiGroup: rbac.authorization.k8s.io
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: capsule
subjects:
- kind: ServiceAccount
name: capsule
namespace: capsule-system
roleRef:
kind: ClusterRole
name: capsule
apiGroup: rbac.authorization.k8s.io

7
deploy/secret-ca.yaml Normal file
View File

@@ -0,0 +1,7 @@
apiVersion: v1
kind: Secret
metadata:
labels:
app: capsule
name: capsule-ca
namespace: capsule-system

7
deploy/secret-tls.yaml Normal file
View File

@@ -0,0 +1,7 @@
apiVersion: v1
kind: Secret
metadata:
labels:
app: capsule
name: capsule-tls
namespace: capsule-system

16
deploy/service.yaml Normal file
View File

@@ -0,0 +1,16 @@
apiVersion: v1
kind: Service
metadata:
labels:
app: capsule
name: capsule
namespace: capsule-system
spec:
ports:
- name: https
port: 443
protocol: TCP
targetPort: 443
selector:
name: capsule
type: ClusterIP

View File

@@ -0,0 +1,5 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: capsule
namespace: capsule-system

19
go.mod Normal file
View File

@@ -0,0 +1,19 @@
module github.com/clastix/capsule
go 1.13
require (
github.com/go-logr/logr v0.1.0
github.com/operator-framework/operator-sdk v0.18.1
github.com/spf13/pflag v1.0.5
github.com/stretchr/testify v1.5.1
k8s.io/api v0.18.2
k8s.io/apimachinery v0.18.2
k8s.io/client-go v12.0.0+incompatible
sigs.k8s.io/controller-runtime v0.6.0
)
replace (
github.com/Azure/go-autorest => github.com/Azure/go-autorest v13.3.2+incompatible // Required by OLM
k8s.io/client-go => k8s.io/client-go v0.18.2 // Required by prometheus-operator
)

1233
go.sum Normal file

File diff suppressed because it is too large Load Diff

1
hack/.gitignore vendored Normal file
View File

@@ -0,0 +1 @@
*.kubeconfig

98
hack/create-user.sh Executable file
View File

@@ -0,0 +1,98 @@
#!/bin/bash
# This script uses Kubernetes CertificateSigningRequest (CSR) to a generate a
# certificate signed by the Kubernetes CA itself.
# It requires cluster admin permission.
#
# e.g.: ./create-user.sh alice oil
# where `oil` is the Tenant and `alice` the owner
# Check if OpenSSL is installed
if [[ ! -x "$(command -v openssl)" ]]; then
echo "Error: openssl not found"
exit 1
fi
# Check if kubectl is installed
if [[ ! -x "$(command -v kubectl)" ]]; then
echo "Error: kubectl not found"
exit 1
fi
USER=$1
TENANT=$2
if [[ -z ${USER} ]]; then
echo "User has not been specified!"
exit 1
fi
if [[ -z ${TENANT} ]]; then
echo "Tenant has not been specified!"
exit 1
fi
GROUP=capsule.clastix.io
TMPDIR=$(mktemp -d)
echo "creating certs in TMPDIR ${TMPDIR} "
openssl genrsa -out ${TMPDIR}/tls.key 2048
openssl req -new -key ${TMPDIR}/tls.key -subj "/CN=${USER}/O=${GROUP}" -out ${TMPDIR}/${USER}-${TENANT}.csr
# Clean any previously created CSR for the same user.
kubectl delete csr ${USER}-${TENANT} 2>/dev/null || true
# Create a new CSR file.
cat <<EOF > ${TMPDIR}/${USER}-${TENANT}-csr.yaml
apiVersion: certificates.k8s.io/v1beta1
kind: CertificateSigningRequest
metadata:
name: ${USER}-${TENANT}
spec:
groups:
- system:authenticated
request: $(cat ${TMPDIR}/${USER}-${TENANT}.csr | base64 | tr -d '\n')
usages:
- digital signature
- key encipherment
- client auth
EOF
# Create the CSR
kubectl apply -f ${TMPDIR}/${USER}-${TENANT}-csr.yaml
# Approve and fetch the signed certificate
kubectl certificate approve ${USER}-${TENANT}
kubectl get csr ${USER}-${TENANT} -o jsonpath='{.status.certificate}' | base64 --decode > ${TMPDIR}/tls.crt
# Create the kubeconfig file
CONTEXT=$(kubectl config current-context)
CLUSTER=$(kubectl config view -o jsonpath="{.contexts[?(@.name == \"$CONTEXT\"})].context.cluster}")
SERVER=$(kubectl config view -o jsonpath="{.clusters[?(@.name == \"${CLUSTER}\"})].cluster.server}")
CA=$(kubectl config view --flatten -o jsonpath="{.clusters[?(@.name == \"${CLUSTER}\"})].cluster.certificate-authority-data}")
cat > ${USER}-${TENANT}.kubeconfig <<EOF
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: $CA
server: ${SERVER}
name: ${CLUSTER}
contexts:
- context:
cluster: ${CLUSTER}
user: ${USER}
name: ${USER}-${TENANT}
current-context: ${USER}-${TENANT}
kind: Config
preferences: {}
users:
- name: ${USER}
user:
client-certificate-data: $(cat ${TMPDIR}/tls.crt | base64 | tr -d '\n')
client-key-data: $(cat ${TMPDIR}/tls.key | base64 | tr -d '\n')
EOF
echo "kubeconfig file is:" ${USER}-${TENANT}.kubeconfig
echo "to use it as" ${USER} "export KUBECONFIG="${USER}-${TENANT}.kubeconfig

View File

@@ -0,0 +1,23 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package apis
import (
"github.com/clastix/capsule/pkg/apis/capsule/v1alpha1"
)
func init() {
// Register the types with the Scheme so the components can map objects to GroupVersionKinds and back
AddToSchemes = append(AddToSchemes, v1alpha1.SchemeBuilder.AddToScheme)
}

26
pkg/apis/apis.go Normal file
View File

@@ -0,0 +1,26 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package apis
import (
"k8s.io/apimachinery/pkg/runtime"
)
// AddToSchemes may be used to add all resources defined in the project to a Scheme
var AddToSchemes runtime.SchemeBuilder
// AddToScheme adds all Resources to the Scheme
func AddToScheme(s *runtime.Scheme) error {
return AddToSchemes.AddToScheme(s)
}

19
pkg/apis/capsule/group.go Normal file
View File

@@ -0,0 +1,19 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
// Package capsule contains capsule API versions.
//
// This file ensures Go source parsers acknowledge the capsule package
// and any child packages. It can be removed if any other Go source files are
// added to this package.
package capsule

View File

@@ -0,0 +1,17 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
// Package v1alpha1 contains API Schema definitions for the capsule v1alpha1 API group
// +k8s:deepcopy-gen=package,register
// +groupName=capsule.clastix.io
package v1alpha1

View File

@@ -0,0 +1,40 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package v1alpha1
import (
"sort"
"strings"
)
type IngressClassList []string
func (n IngressClassList) Len() int {
return len(n)
}
func (n IngressClassList) Swap(i, j int) {
n[i], n[j] = n[j], n[i]
}
func (n IngressClassList) Less(i, j int) bool {
return strings.ToLower(n[i]) < strings.ToLower(n[j])
}
func (n IngressClassList) IsStringInList(value string) (ok bool) {
sort.Sort(n)
i := sort.SearchStrings(n, value)
ok = i < n.Len() && n[i] == value
return
}

View File

@@ -0,0 +1,40 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package v1alpha1
import (
"sort"
"strings"
)
type NamespaceList []string
func (n NamespaceList) Len() int {
return len(n)
}
func (n NamespaceList) Swap(i, j int) {
n[i], n[j] = n[j], n[i]
}
func (n NamespaceList) Less(i, j int) bool {
return strings.ToLower(n[i]) < strings.ToLower(n[j])
}
func (n NamespaceList) IsStringInList(value string) (ok bool) {
sort.Sort(n)
i := sort.SearchStrings(n, value)
ok = i < n.Len() && n[i] == value
return
}

View File

@@ -0,0 +1,32 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
// NOTE: Boilerplate only. Ignore this file.
// Package v1alpha1 contains API Schema definitions for the capsule v1alpha1 API group
// +k8s:deepcopy-gen=package,register
// +groupName=capsule.clastix.io
package v1alpha1
import (
"k8s.io/apimachinery/pkg/runtime/schema"
"sigs.k8s.io/controller-runtime/pkg/scheme"
)
var (
// SchemeGroupVersion is group version used to register these objects
SchemeGroupVersion = schema.GroupVersion{Group: "capsule.clastix.io", Version: "v1alpha1"}
// SchemeBuilder is used to add go types to the GroupVersionKind scheme
SchemeBuilder = &scheme.Builder{GroupVersion: SchemeGroupVersion}
)

View File

@@ -0,0 +1,18 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package v1alpha1
type SearchIn interface {
IsStringInList(value string) bool
}

View File

@@ -0,0 +1,40 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package v1alpha1
import (
"sort"
"strings"
)
type StorageClassList []string
func (n StorageClassList) Len() int {
return len(n)
}
func (n StorageClassList) Swap(i, j int) {
n[i], n[j] = n[j], n[i]
}
func (n StorageClassList) Less(i, j int) bool {
return strings.ToLower(n[i]) < strings.ToLower(n[j])
}
func (n StorageClassList) IsStringInList(value string) (ok bool) {
sort.Sort(n)
i := sort.SearchStrings(n, value)
ok = i < n.Len() && n[i] == value
return
}

View File

@@ -0,0 +1,20 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package v1alpha1
import corev1 "k8s.io/api/core/v1"
func UsedQuotaFor(resource corev1.ResourceName) string {
return "quota.capsule.clastix.io/used-" + resource.String()
}

View File

@@ -0,0 +1,18 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package v1alpha1
func (t Tenant) IsFull() bool {
return t.Status.Namespaces.Len() >= int(t.Spec.NamespaceQuota)
}

View File

@@ -0,0 +1,38 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package v1alpha1
import (
"fmt"
corev1 "k8s.io/api/core/v1"
networkingv1 "k8s.io/api/networking/v1"
"k8s.io/apimachinery/pkg/runtime"
)
func GetTypeLabel(t runtime.Object) (label string, err error) {
switch v := t.(type) {
case *Tenant:
return "capsule.clastix.io/tenant", nil
case *corev1.LimitRange:
return "capsule.clastix.io/limit-range", nil
case *networkingv1.NetworkPolicy:
return "capsule.clastix.io/network-policy", nil
case *corev1.ResourceQuota:
return "capsule.clastix.io/resource-quota", nil
default:
err = fmt.Errorf("type %T is not mapped as Capsule label recognized", v)
}
return
}

View File

@@ -0,0 +1,73 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package v1alpha1
import (
corev1 "k8s.io/api/core/v1"
networkingv1 "k8s.io/api/networking/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
// +kubebuilder:validation:Minimum=1
type NamespaceQuota uint
// TenantSpec defines the desired state of Tenant
type TenantSpec struct {
Owner string `json:"owner"`
// +kubebuilder:validation:Required
StorageClasses StorageClassList `json:"storageClasses"`
IngressClasses IngressClassList `json:"ingressClasses"`
// +kubebuilder:validation:Optional
NodeSelector map[string]string `json:"nodeSelector"`
NamespaceQuota NamespaceQuota `json:"namespaceQuota"`
NetworkPolicies []networkingv1.NetworkPolicySpec `json:"networkPolicies,omitempty"`
LimitRanges []corev1.LimitRangeSpec `json:"limitRanges"`
// +kubebuilder:validation:Optional
ResourceQuota []corev1.ResourceQuotaSpec `json:"resourceQuotas"`
}
// TenantStatus defines the observed state of Tenant
type TenantStatus struct {
Size uint `json:"size"`
Namespaces NamespaceList `json:"namespaces,omitempty"`
Users []string `json:"users,omitempty"`
Groups []string `json:"groups,omitempty"`
}
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
// Tenant is the Schema for the tenants API
// +kubebuilder:subresource:status
// +kubebuilder:resource:path=tenants,scope=Cluster
// +kubebuilder:printcolumn:name="Namespace quota",type="integer",JSONPath=".spec.namespaceQuota",description="The max amount of Namespaces can be created"
// +kubebuilder:printcolumn:name="Namespace count",type="integer",JSONPath=".status.size",description="The total amount of Namespaces in use"
type Tenant struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty"`
Spec TenantSpec `json:"spec,omitempty"`
Status TenantStatus `json:"status,omitempty"`
}
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
// TenantList contains a list of Tenant
type TenantList struct {
metav1.TypeMeta `json:",inline"`
metav1.ListMeta `json:"metadata,omitempty"`
Items []Tenant `json:"items"`
}
func init() {
SchemeBuilder.Register(&Tenant{}, &TenantList{})
}

View File

@@ -0,0 +1,217 @@
// +build !ignore_autogenerated
// Code generated by operator-sdk. DO NOT EDIT.
package v1alpha1
import (
corev1 "k8s.io/api/core/v1"
v1 "k8s.io/api/networking/v1"
runtime "k8s.io/apimachinery/pkg/runtime"
)
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in IngressClassList) DeepCopyInto(out *IngressClassList) {
{
in := &in
*out = make(IngressClassList, len(*in))
copy(*out, *in)
return
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new IngressClassList.
func (in IngressClassList) DeepCopy() IngressClassList {
if in == nil {
return nil
}
out := new(IngressClassList)
in.DeepCopyInto(out)
return *out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in NamespaceList) DeepCopyInto(out *NamespaceList) {
{
in := &in
*out = make(NamespaceList, len(*in))
copy(*out, *in)
return
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new NamespaceList.
func (in NamespaceList) DeepCopy() NamespaceList {
if in == nil {
return nil
}
out := new(NamespaceList)
in.DeepCopyInto(out)
return *out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in StorageClassList) DeepCopyInto(out *StorageClassList) {
{
in := &in
*out = make(StorageClassList, len(*in))
copy(*out, *in)
return
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new StorageClassList.
func (in StorageClassList) DeepCopy() StorageClassList {
if in == nil {
return nil
}
out := new(StorageClassList)
in.DeepCopyInto(out)
return *out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *Tenant) DeepCopyInto(out *Tenant) {
*out = *in
out.TypeMeta = in.TypeMeta
in.ObjectMeta.DeepCopyInto(&out.ObjectMeta)
in.Spec.DeepCopyInto(&out.Spec)
in.Status.DeepCopyInto(&out.Status)
return
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Tenant.
func (in *Tenant) DeepCopy() *Tenant {
if in == nil {
return nil
}
out := new(Tenant)
in.DeepCopyInto(out)
return out
}
// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.
func (in *Tenant) DeepCopyObject() runtime.Object {
if c := in.DeepCopy(); c != nil {
return c
}
return nil
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *TenantList) DeepCopyInto(out *TenantList) {
*out = *in
out.TypeMeta = in.TypeMeta
in.ListMeta.DeepCopyInto(&out.ListMeta)
if in.Items != nil {
in, out := &in.Items, &out.Items
*out = make([]Tenant, len(*in))
for i := range *in {
(*in)[i].DeepCopyInto(&(*out)[i])
}
}
return
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new TenantList.
func (in *TenantList) DeepCopy() *TenantList {
if in == nil {
return nil
}
out := new(TenantList)
in.DeepCopyInto(out)
return out
}
// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.
func (in *TenantList) DeepCopyObject() runtime.Object {
if c := in.DeepCopy(); c != nil {
return c
}
return nil
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *TenantSpec) DeepCopyInto(out *TenantSpec) {
*out = *in
if in.StorageClasses != nil {
in, out := &in.StorageClasses, &out.StorageClasses
*out = make(StorageClassList, len(*in))
copy(*out, *in)
}
if in.IngressClasses != nil {
in, out := &in.IngressClasses, &out.IngressClasses
*out = make(IngressClassList, len(*in))
copy(*out, *in)
}
if in.NodeSelector != nil {
in, out := &in.NodeSelector, &out.NodeSelector
*out = make(map[string]string, len(*in))
for key, val := range *in {
(*out)[key] = val
}
}
if in.NetworkPolicies != nil {
in, out := &in.NetworkPolicies, &out.NetworkPolicies
*out = make([]v1.NetworkPolicySpec, len(*in))
for i := range *in {
(*in)[i].DeepCopyInto(&(*out)[i])
}
}
if in.LimitRanges != nil {
in, out := &in.LimitRanges, &out.LimitRanges
*out = make([]corev1.LimitRangeSpec, len(*in))
for i := range *in {
(*in)[i].DeepCopyInto(&(*out)[i])
}
}
if in.ResourceQuota != nil {
in, out := &in.ResourceQuota, &out.ResourceQuota
*out = make([]corev1.ResourceQuotaSpec, len(*in))
for i := range *in {
(*in)[i].DeepCopyInto(&(*out)[i])
}
}
return
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new TenantSpec.
func (in *TenantSpec) DeepCopy() *TenantSpec {
if in == nil {
return nil
}
out := new(TenantSpec)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *TenantStatus) DeepCopyInto(out *TenantStatus) {
*out = *in
if in.Namespaces != nil {
in, out := &in.Namespaces, &out.Namespaces
*out = make(NamespaceList, len(*in))
copy(*out, *in)
}
if in.Users != nil {
in, out := &in.Users, &out.Users
*out = make([]string, len(*in))
copy(*out, *in)
}
if in.Groups != nil {
in, out := &in.Groups, &out.Groups
*out = make([]string, len(*in))
copy(*out, *in)
}
return
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new TenantStatus.
func (in *TenantStatus) DeepCopy() *TenantStatus {
if in == nil {
return nil
}
out := new(TenantStatus)
in.DeepCopyInto(out)
return out
}

166
pkg/cert/ca.go Normal file
View File

@@ -0,0 +1,166 @@
package cert
import (
"bytes"
"crypto/rand"
"crypto/rsa"
"crypto/x509"
"crypto/x509/pkix"
"encoding/pem"
"math/big"
"time"
)
type Ca interface {
GenerateCertificate(opts CertificateOptions) (certificatePem *bytes.Buffer, certificateKey *bytes.Buffer, err error)
CaCertificatePem() (b *bytes.Buffer, err error)
CaPrivateKeyPem() (b *bytes.Buffer, err error)
ExpiresIn(now time.Time) (time.Duration, error)
}
type CapsuleCa struct {
ca *x509.Certificate
privateKey *rsa.PrivateKey
}
func (c CapsuleCa) isAlreadyValid(now time.Time) bool {
return now.After(c.ca.NotBefore)
}
func (c CapsuleCa) isExpired(now time.Time) bool {
return now.Before(c.ca.NotAfter)
}
func (c CapsuleCa) ExpiresIn(now time.Time) (time.Duration, error) {
if !c.isExpired(now) {
return time.Nanosecond, CaExpiredError{}
}
if !c.isAlreadyValid(now) {
return time.Nanosecond, CaNotYetValidError{}
}
return time.Duration(c.ca.NotAfter.Unix() - now.Unix()) * time.Second, nil
}
func (c CapsuleCa) CaCertificatePem() (b *bytes.Buffer, err error) {
var crtBytes []byte
crtBytes, err = x509.CreateCertificate(rand.Reader, c.ca, c.ca, &c.privateKey.PublicKey, c.privateKey)
if err != nil {
return
}
b = new(bytes.Buffer)
err = pem.Encode(b, &pem.Block{
Type: "CERTIFICATE",
Bytes: crtBytes,
})
return b, err
}
func (c CapsuleCa) CaPrivateKeyPem() (b *bytes.Buffer, err error) {
b = new(bytes.Buffer)
return b, pem.Encode(b, &pem.Block{
Type: "RSA PRIVATE KEY",
Bytes: x509.MarshalPKCS1PrivateKey(c.privateKey),
})
}
func GenerateCertificateAuthority() (s *CapsuleCa, err error) {
s = &CapsuleCa{
ca: &x509.Certificate{
SerialNumber: big.NewInt(2019),
Subject: pkix.Name{
Organization: []string{"Clastix"},
Country: []string{"UK"},
Province: []string{""},
Locality: []string{"London"},
StreetAddress: []string{"27, Old Gloucester Street"},
PostalCode: []string{"WC1N 3AX"},
},
NotBefore: time.Now(),
NotAfter: time.Now().AddDate(10, 0, 0),
IsCA: true,
ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageClientAuth, x509.ExtKeyUsageServerAuth},
KeyUsage: x509.KeyUsageDigitalSignature | x509.KeyUsageCertSign,
BasicConstraintsValid: true,
},
}
s.privateKey, err = rsa.GenerateKey(rand.Reader, 4096)
if err != nil {
return nil, err
}
return
}
func NewCertificateAuthorityFromBytes(certBytes, keyBytes []byte) (s *CapsuleCa, err error) {
var b *pem.Block
b, _ = pem.Decode(certBytes)
var cert *x509.Certificate
if cert, err = x509.ParseCertificate(b.Bytes); err != nil {
return
}
b, _ = pem.Decode(keyBytes)
var key *rsa.PrivateKey
if key, err = x509.ParsePKCS1PrivateKey(b.Bytes); err != nil {
return
}
s = &CapsuleCa{
ca: cert,
privateKey: key,
}
return
}
func (c *CapsuleCa) GenerateCertificate(opts CertificateOptions) (certificatePem *bytes.Buffer, certificateKey *bytes.Buffer, err error) {
certPrivKey, err := rsa.GenerateKey(rand.Reader, 4096)
if err != nil {
return nil, nil, err
}
cert := &x509.Certificate{
SerialNumber: big.NewInt(1658),
Subject: pkix.Name{
Organization: []string{"Clastix"},
Country: []string{"UK"},
Province: []string{""},
Locality: []string{"London"},
StreetAddress: []string{"27, Old Gloucester Street"},
PostalCode: []string{"WC1N 3AX"},
},
DNSNames: opts.DnsNames(),
NotBefore: time.Now().AddDate(0, 0, -1),
NotAfter: opts.ExpirationDate(),
SubjectKeyId: []byte{1, 2, 3, 4, 6},
ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageClientAuth, x509.ExtKeyUsageServerAuth},
KeyUsage: x509.KeyUsageDigitalSignature,
}
certBytes, err := x509.CreateCertificate(rand.Reader, cert, c.ca, &certPrivKey.PublicKey, c.privateKey)
if err != nil {
return nil, nil, err
}
certificatePem = new(bytes.Buffer)
err = pem.Encode(certificatePem, &pem.Block{
Type: "CERTIFICATE",
Bytes: certBytes,
})
if err != nil {
return
}
certificateKey = new(bytes.Buffer)
err = pem.Encode(certificateKey, &pem.Block{
Type: "RSA PRIVATE KEY",
Bytes: x509.MarshalPKCS1PrivateKey(certPrivKey),
})
if err != nil {
return
}
return
}

105
pkg/cert/ca_test.go Normal file
View File

@@ -0,0 +1,105 @@
package cert
import (
"bytes"
"crypto/tls"
"crypto/x509"
"encoding/pem"
"testing"
"time"
"github.com/stretchr/testify/assert"
)
func TestNewCertificateAuthorityFromBytes(t *testing.T) {
var ca *CapsuleCa
var err error
ca, err = GenerateCertificateAuthority()
assert.Nil(t, err)
var crt *bytes.Buffer
crt, err =ca.CaCertificatePem()
assert.Nil(t, err)
var key *bytes.Buffer
key, err = ca.CaPrivateKeyPem()
assert.Nil(t, err)
_, err = NewCertificateAuthorityFromBytes(crt.Bytes(), key.Bytes())
assert.Nil(t, err)
}
func TestCapsuleCa_GenerateCertificate(t *testing.T) {
type testCase struct {
dnsNames []string
}
for name, c := range map[string]testCase{
"foo.tld": {[]string{"foo.tld"}},
"SAN": {[]string{"capsule.capsule-system.svc", "capsule.capsule-system.default.cluster"}},
} {
t.Run(name, func(t *testing.T) {
var ca *CapsuleCa
var err error
e := time.Now().AddDate(1, 0, 0)
ca, err = GenerateCertificateAuthority()
assert.Nil(t, err)
var crt *bytes.Buffer
var key *bytes.Buffer
crt, key, err = ca.GenerateCertificate(NewCertOpts(e, c.dnsNames...))
assert.Nil(t, err)
var b *pem.Block
var c *x509.Certificate
b, _ = pem.Decode(crt.Bytes())
c, err = x509.ParseCertificate(b.Bytes)
assert.Nil(t, err)
assert.Equal(t, e.Unix(), c.NotAfter.Unix())
for _, i := range c.DNSNames {
assert.Contains(t, c.DNSNames, i)
}
_, err = tls.X509KeyPair(crt.Bytes(), key.Bytes())
assert.Nil(t, err)
})
}
}
func TestCapsuleCa_IsValid(t *testing.T) {
type testCase struct {
notBefore time.Time
notAfter time.Time
returnError bool
}
tc := map[string]testCase{
"ok": {time.Now().AddDate(0, 0, -1), time.Now().AddDate(0, 0, 1), false},
"expired": {time.Now().AddDate(1, 0, 0), time.Now(), true},
"notValid": {time.Now().AddDate(0, 0, 1), time.Now().AddDate(0, 0, 2), true},
}
for name, c := range tc {
t.Run(name, func(t *testing.T) {
var ca *CapsuleCa
var err error
ca, err = GenerateCertificateAuthority()
assert.Nil(t, err)
ca.ca.NotAfter = c.notAfter
ca.ca.NotBefore = c.notBefore
var w time.Duration
w, err = ca.ExpiresIn(time.Now())
if c.returnError {
assert.Error(t, err)
return
}
assert.Nil(t, err)
assert.WithinDuration(t, ca.ca.NotAfter, time.Now().Add(w), time.Minute)
})
}
}

13
pkg/cert/errors.go Normal file
View File

@@ -0,0 +1,13 @@
package cert
type CaNotYetValidError struct {}
func (CaNotYetValidError) Error() string {
return "The current CA is not yet valid"
}
type CaExpiredError struct {}
func (CaExpiredError) Error() string {
return "The current CA is expired"
}

25
pkg/cert/options.go Normal file
View File

@@ -0,0 +1,25 @@
package cert
import "time"
type CertificateOptions interface {
DnsNames() []string
ExpirationDate() time.Time
}
type certOpts struct {
dnsNames []string
expirationDate time.Time
}
func (c certOpts) DnsNames() []string {
return c.dnsNames
}
func (c certOpts) ExpirationDate() time.Time {
return c.expirationDate
}
func NewCertOpts(expirationDate time.Time, dnsNames ...string) *certOpts {
return &certOpts{dnsNames: dnsNames, expirationDate: expirationDate}
}

View File

@@ -0,0 +1,21 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package controller
import "github.com/clastix/capsule/pkg/controller/namespace"
func init() {
// AddToManagerFuncs is a list of functions to create controllers and add them to a manager.
AddToManagerFuncs = append(AddToManagerFuncs, namespace.Add)
}

View File

@@ -0,0 +1,23 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package controller
import (
"github.com/clastix/capsule/pkg/controller/secret"
)
func init() {
// AddToManagerFuncs is a list of functions to create controllers and add them to a manager.
AddToManagerFuncs = append(AddToManagerFuncs, secret.AddTls, secret.AddCa)
}

View File

@@ -0,0 +1,23 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package controller
import (
"github.com/clastix/capsule/pkg/controller/tenant"
)
func init() {
// AddToManagerFuncs is a list of functions to create controllers and add them to a manager.
AddToManagerFuncs = append(AddToManagerFuncs, tenant.Add)
}

View File

@@ -0,0 +1,31 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package controller
import (
"sigs.k8s.io/controller-runtime/pkg/manager"
)
// AddToManagerFuncs is a list of functions to add all Controllers to the Manager
var AddToManagerFuncs []func(manager.Manager) error
// AddToManager adds all Controllers to the Manager
func AddToManager(m manager.Manager) error {
for _, f := range AddToManagerFuncs {
if err := f(m); err != nil {
return err
}
}
return nil
}

View File

@@ -0,0 +1,183 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package namespace
import (
"context"
"github.com/go-logr/logr"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/types"
"k8s.io/client-go/util/retry"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/controller"
"sigs.k8s.io/controller-runtime/pkg/handler"
"sigs.k8s.io/controller-runtime/pkg/log"
"sigs.k8s.io/controller-runtime/pkg/manager"
"sigs.k8s.io/controller-runtime/pkg/reconcile"
"sigs.k8s.io/controller-runtime/pkg/source"
"sort"
"github.com/clastix/capsule/pkg/apis/capsule/v1alpha1"
)
// Add creates a new Namespace Controller and adds it to the Manager. The Manager will set fields on the Controller
// and Start it when the Manager is Started.
func Add(mgr manager.Manager) error {
return add(mgr, newReconciler(mgr))
}
// newReconciler returns a new reconcile.Reconciler
func newReconciler(mgr manager.Manager) reconcile.Reconciler {
return &ReconcileNamespace{
client: mgr.GetClient(),
scheme: mgr.GetScheme(),
}
}
// add adds a new Controller to mgr with r as the reconcile.Reconciler
func add(mgr manager.Manager, r reconcile.Reconciler) error {
// Create a new controller
c, err := controller.New("namespace-controller", mgr, controller.Options{Reconciler: r})
if err != nil {
return err
}
// Watch for changes to primary resource Namespace
err = c.Watch(&source.Kind{Type: &corev1.Namespace{}}, &handler.EnqueueRequestForObject{})
if err != nil {
return err
}
return nil
}
// ReconcileNamespace reconciles a Namespace object
type ReconcileNamespace struct {
client client.Client
scheme *runtime.Scheme
logger logr.Logger
}
func (r *ReconcileNamespace) removeNamespace(name string, tenant *v1alpha1.Tenant) {
c := tenant.Status.Namespaces.DeepCopy()
sort.Sort(c)
i := sort.SearchStrings(c, name)
// namespace already removed, do nothing
if i > c.Len() || i == c.Len() {
return
}
// namespace is there, removing it
tenant.Status.Namespaces = []string{}
tenant.Status.Namespaces = append(tenant.Status.Namespaces, c[:i]...)
tenant.Status.Namespaces = append(tenant.Status.Namespaces, c[i+1:]...)
}
func (r *ReconcileNamespace) addNamespace(name string, tenant *v1alpha1.Tenant) {
c := tenant.Status.Namespaces.DeepCopy()
sort.Sort(c)
i := sort.SearchStrings(c, name)
// namespace already there, nothing to do
if i < c.Len() && c[i] == name {
return
}
// missing namespace, let's append it
if i == 0 {
tenant.Status.Namespaces = []string{name}
} else {
tenant.Status.Namespaces = v1alpha1.NamespaceList{}
tenant.Status.Namespaces = append(tenant.Status.Namespaces, c[:i]...)
tenant.Status.Namespaces = append(tenant.Status.Namespaces, name)
}
tenant.Status.Namespaces = append(tenant.Status.Namespaces, c[i:]...)
}
func (r *ReconcileNamespace) updateNamespaceCount(tenant *v1alpha1.Tenant) error {
tenant.Status.Size = uint(len(tenant.Status.Namespaces))
return retry.RetryOnConflict(retry.DefaultBackoff, func() error {
return r.client.Status().Update(context.TODO(), tenant, &client.UpdateOptions{})
})
}
func (r *ReconcileNamespace) Reconcile(request reconcile.Request) (res reconcile.Result, err error) {
r.logger = log.Log.WithName("controller_namespace").WithValues("Request.Namespace", request.Namespace, "Request.Name", request.Name)
r.logger.Info("Reconciling Namespace")
// Fetch the Namespace instance
ns := &corev1.Namespace{}
if err := r.client.Get(context.TODO(), request.NamespacedName, ns); err != nil {
if errors.IsNotFound(err) {
// Request object not found, could have been deleted after reconcile request.
// Owned objects are automatically garbage collected. For additional cleanup logic use finalizers.
// Return and don't requeue
return reconcile.Result{}, nil
}
// Error reading the object - requeue the request.
return reconcile.Result{}, err
}
// Skipping NS non referenced to a Tenant
if len(ns.OwnerReferences) == 0 {
return reconcile.Result{}, nil
}
t := &v1alpha1.Tenant{}
if err := r.client.Get(context.TODO(), types.NamespacedName{Name: ns.OwnerReferences[0].Name}, t); err != nil {
// Error reading the object - requeue the request.
return reconcile.Result{}, err
}
if err := r.ensureLabel(ns, t.Name); err != nil {
r.logger.Error(err, "cannot update Namespace label")
return reconcile.Result{}, err
}
r.updateTenantStatus(ns, t)
if err := r.updateNamespaceCount(t); err != nil {
r.logger.Error(err, "cannot update Namespace list", "tenant", t.Name)
}
r.logger.Info("Namespace reconciliation processed")
return reconcile.Result{}, nil
}
func (r *ReconcileNamespace) ensureLabel(ns *corev1.Namespace, tenantName string) error {
capsuleLabel, err := v1alpha1.GetTypeLabel(&v1alpha1.Tenant{})
if err != nil {
return err
}
if ns.Labels == nil {
ns.Labels = make(map[string]string)
}
tl, ok := ns.Labels[capsuleLabel]
if !ok || tl != tenantName {
ns.Labels[capsuleLabel] = tenantName
return retry.RetryOnConflict(retry.DefaultBackoff, func() error {
return r.client.Update(context.TODO(), ns, &client.UpdateOptions{})
})
}
return nil
}
func (r *ReconcileNamespace) updateTenantStatus(ns *corev1.Namespace, tenant *v1alpha1.Tenant) {
switch ns.Status.Phase {
case corev1.NamespaceTerminating:
r.removeNamespace(ns.Name, tenant)
case corev1.NamespaceActive:
r.addNamespace(ns.Name, tenant)
}
}

View File

@@ -0,0 +1,22 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package secret
const (
Cert = "tls.crt"
PrivateKey = "tls.key"
CaSecretName = "capsule-ca"
TlsSecretName = "capsule-tls"
)

View File

@@ -0,0 +1,89 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package secret
import (
"context"
"fmt"
"k8s.io/apimachinery/pkg/types"
"github.com/clastix/capsule/pkg/cert"
"github.com/go-logr/logr"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/log"
"sigs.k8s.io/controller-runtime/pkg/manager"
"sigs.k8s.io/controller-runtime/pkg/reconcile"
)
type secretReconciliationFunc func(reconciler *ReconcileSecret, request reconcile.Request) (reconcile.Result, error)
// ReconcileSecret reconciles a Secret object
type ReconcileSecret struct {
client client.Client
scheme *runtime.Scheme
logger logr.Logger
reconcileFunc secretReconciliationFunc
}
func newReconciler(mgr manager.Manager, name string, f secretReconciliationFunc) reconcile.Reconciler {
return &ReconcileSecret{
client: mgr.GetClient(),
scheme: mgr.GetScheme(),
logger: log.Log.WithName(name),
reconcileFunc: f,
}
}
func (r *ReconcileSecret) Reconcile(request reconcile.Request) (reconcile.Result, error) {
return r.reconcileFunc(r, request)
}
func (r *ReconcileSecret) GetCertificateAuthority() (ca cert.Ca, err error) {
instance := &corev1.Secret{}
err = r.client.Get(context.TODO(), types.NamespacedName{
Namespace: "capsule-system",
Name: CaSecretName,
}, instance)
if err != nil {
return nil, fmt.Errorf("missing secret %s, cannot reconcile", CaSecretName)
}
if instance.Data == nil {
ca, err = cert.GenerateCertificateAuthority()
if err != nil {
return
}
instance.Data = map[string][]byte{}
crt, _ := ca.CaCertificatePem()
instance.Data[Cert] = crt.Bytes()
key, _ := ca.CaPrivateKeyPem()
instance.Data[PrivateKey] = key.Bytes()
}
ca, err = cert.NewCertificateAuthorityFromBytes(instance.Data[Cert], instance.Data[PrivateKey])
if err != nil {
return
}
return
}
func filterByName(objName, desired string) bool {
return objName == desired
}

View File

@@ -0,0 +1,174 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package secret
import (
"bytes"
"context"
"time"
v1 "k8s.io/api/admissionregistration/v1"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/types"
"k8s.io/client-go/util/retry"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/controller"
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
"sigs.k8s.io/controller-runtime/pkg/event"
"sigs.k8s.io/controller-runtime/pkg/handler"
"sigs.k8s.io/controller-runtime/pkg/manager"
"sigs.k8s.io/controller-runtime/pkg/predicate"
"sigs.k8s.io/controller-runtime/pkg/reconcile"
"sigs.k8s.io/controller-runtime/pkg/source"
"github.com/clastix/capsule/pkg/cert"
)
// Add creates a new Secret Controller and adds it to the Manager. The Manager will set fields on the Controller
// and Start it when the Manager is Started.
func AddCa(mgr manager.Manager) error {
r := newReconciler(mgr, "controller_secret", caReconcile)
return ca(mgr, r)
}
// add adds a new Controller to mgr with r as the reconcile.Reconciler
func ca(mgr manager.Manager, r reconcile.Reconciler) error {
// Create a new controller
c, err := controller.New("secret-controller", mgr, controller.Options{Reconciler: r})
if err != nil {
return err
}
// Watch for changes to CA Secret
err = c.Watch(&source.Kind{Type: &corev1.Secret{}}, &handler.EnqueueRequestForObject{}, predicate.Funcs{
CreateFunc: func(event event.CreateEvent) (ok bool) {
return filterByName(event.Meta.GetName(), CaSecretName)
},
DeleteFunc: func(deleteEvent event.DeleteEvent) bool {
return filterByName(deleteEvent.Meta.GetName(), CaSecretName)
},
UpdateFunc: func(updateEvent event.UpdateEvent) bool {
return filterByName(updateEvent.MetaNew.GetName(), CaSecretName)
},
GenericFunc: func(genericEvent event.GenericEvent) bool {
return filterByName(genericEvent.Meta.GetName(), CaSecretName)
},
})
if err != nil {
return err
}
return nil
}
func caReconcile(r *ReconcileSecret, request reconcile.Request) (reconcile.Result, error) {
var err error
r.logger = r.logger.WithValues("Request.Namespace", request.Namespace, "Request.Name", request.Name)
r.logger.Info("Reconciling CA Secret")
// Fetch the CA instance
instance := &corev1.Secret{}
err = r.client.Get(context.TODO(), request.NamespacedName, instance)
if err != nil {
// Error reading the object - requeue the request.
return reconcile.Result{}, err
}
var ca cert.Ca
var rq time.Duration
ca, err = r.GetCertificateAuthority()
if err != nil {
return reconcile.Result{}, err
}
r.logger.Info("Handling CA Secret")
rq, err = ca.ExpiresIn(time.Now())
if err != nil {
r.logger.Info("CA is expired, cleaning to obtain a new one")
instance.Data = map[string][]byte{}
} else {
r.logger.Info("Updating CA secret with new PEM and RSA")
var crt *bytes.Buffer
var key *bytes.Buffer
crt, _ = ca.CaCertificatePem()
key, _ = ca.CaPrivateKeyPem()
instance.Data = map[string][]byte{
Cert: crt.Bytes(),
PrivateKey: key.Bytes(),
}
wh := &v1.MutatingWebhookConfiguration{}
err = r.client.Get(context.TODO(), types.NamespacedName{
Name: "capsule",
}, wh)
if err != nil {
r.logger.Error(err, "cannot retrieve MutatingWebhookConfiguration")
return reconcile.Result{}, err
}
for i, w := range wh.Webhooks {
// Updating CABundle only in case of an internal service reference
if w.ClientConfig.Service != nil {
wh.Webhooks[i].ClientConfig.CABundle = instance.Data[Cert]
}
}
err := retry.RetryOnConflict(retry.DefaultBackoff, func() error {
return r.client.Update(context.TODO(), wh, &client.UpdateOptions{})
})
if err != nil {
r.logger.Error(err, "cannot update MutatingWebhookConfiguration webhooks CA bundle")
return reconcile.Result{}, err
}
}
var res controllerutil.OperationResult
t := &corev1.Secret{ObjectMeta: instance.ObjectMeta}
res, err = controllerutil.CreateOrUpdate(context.TODO(), r.client, t, func() error {
t.Data = instance.Data
return nil
})
if err != nil {
r.logger.Error(err, "cannot update Capsule TLS")
return reconcile.Result{}, err
}
if res == controllerutil.OperationResultUpdated {
r.logger.Info("Capsule CA has been updated, we need to trigger TLS update too")
tls := &corev1.Secret{}
err = r.client.Get(context.TODO(), types.NamespacedName{
Namespace: "capsuel-system",
Name: TlsSecretName,
}, tls)
if err != nil {
r.logger.Error(err, "Capsule TLS Secret missing")
}
err = retry.RetryOnConflict(retry.DefaultBackoff, func() error {
_, err = controllerutil.CreateOrUpdate(context.TODO(), r.client, tls, func() error {
tls.Data = map[string][]byte{}
return nil
})
return err
})
if err != nil {
r.logger.Error(err, "Cannot clean Capsule TLS Secret due to CA update")
return reconcile.Result{}, err
}
}
r.logger.Info("Reconciliation completed, processing back in " + rq.String())
return reconcile.Result{Requeue: true, RequeueAfter: rq}, nil
}

View File

@@ -0,0 +1,150 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package secret
import (
"context"
"crypto/x509"
"encoding/pem"
"os"
"time"
corev1 "k8s.io/api/core/v1"
"sigs.k8s.io/controller-runtime/pkg/controller"
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
"sigs.k8s.io/controller-runtime/pkg/event"
"sigs.k8s.io/controller-runtime/pkg/handler"
"sigs.k8s.io/controller-runtime/pkg/manager"
"sigs.k8s.io/controller-runtime/pkg/predicate"
"sigs.k8s.io/controller-runtime/pkg/reconcile"
"sigs.k8s.io/controller-runtime/pkg/source"
"github.com/clastix/capsule/pkg/cert"
)
// Add creates a new Secret Controller and adds it to the Manager. The Manager will set fields on the Controller
// and Start it when the Manager is Started.
func AddTls(mgr manager.Manager) error {
return tls(mgr, newReconciler(mgr, "controller_secret_tls", tlsReconcile))
}
// add adds a new Controller to mgr with r as the reconcile.Reconciler
func tls(mgr manager.Manager, r reconcile.Reconciler) error {
// Create a new controller
c, err := controller.New("secret-controller", mgr, controller.Options{Reconciler: r})
if err != nil {
return err
}
// Watch for changes to TLS Secret
err = c.Watch(&source.Kind{Type: &corev1.Secret{}}, &handler.EnqueueRequestForObject{}, predicate.Funcs{
CreateFunc: func(event event.CreateEvent) (ok bool) {
return filterByName(event.Meta.GetName(), TlsSecretName)
},
DeleteFunc: func(deleteEvent event.DeleteEvent) bool {
return filterByName(deleteEvent.Meta.GetName(), TlsSecretName)
},
UpdateFunc: func(updateEvent event.UpdateEvent) bool {
return filterByName(updateEvent.MetaNew.GetName(), TlsSecretName)
},
GenericFunc: func(genericEvent event.GenericEvent) bool {
return filterByName(genericEvent.Meta.GetName(), TlsSecretName)
},
})
if err != nil {
return err
}
return nil
}
func tlsReconcile(r *ReconcileSecret, request reconcile.Request) (reconcile.Result, error) {
var err error
r.logger = r.logger.WithValues("Request.Namespace", request.Namespace, "Request.Name", request.Name)
r.logger.Info("Reconciling TLS/CA Secret")
// Fetch the Secret instance
instance := &corev1.Secret{}
err = r.client.Get(context.TODO(), request.NamespacedName, instance)
if err != nil {
// Error reading the object - requeue the request.
return reconcile.Result{}, err
}
var ca cert.Ca
var rq time.Duration
ca, err = r.GetCertificateAuthority()
if err != nil {
return reconcile.Result{}, err
}
var shouldCreate bool
for _, key := range []string{Cert, PrivateKey} {
if _, ok := instance.Data[key]; !ok {
shouldCreate = true
}
}
if shouldCreate {
r.logger.Info("Missing Capsule TLS certificate")
rq = 6 * 30 * 24 * time.Hour
opts := cert.NewCertOpts(time.Now().Add(rq), "capsule.capsule-system.svc")
crt, key, err := ca.GenerateCertificate(opts)
if err != nil {
r.logger.Error(err, "Cannot generate new TLS certificate")
return reconcile.Result{}, err
}
instance.Data = map[string][]byte{
Cert: crt.Bytes(),
PrivateKey: key.Bytes(),
}
} else {
var c *x509.Certificate
var b *pem.Block
b, _ = pem.Decode(instance.Data[Cert])
c, err = x509.ParseCertificate(b.Bytes)
if err != nil {
r.logger.Error(err, "cannot parse Capsule TLS")
return reconcile.Result{}, err
}
rq = time.Duration(c.NotAfter.Unix()-time.Now().Unix()) * time.Second
if time.Now().After(c.NotAfter) {
r.logger.Info("Capsule TLS is expired, cleaning to obtain a new one")
instance.Data = map[string][]byte{}
}
}
var res controllerutil.OperationResult
t := &corev1.Secret{ObjectMeta: instance.ObjectMeta,}
res, err = controllerutil.CreateOrUpdate(context.TODO(), r.client, t, func() error {
t.Data = instance.Data
return nil
})
if err != nil {
r.logger.Error(err, "cannot update Capsule TLS")
return reconcile.Result{}, err
}
if instance.Name == TlsSecretName && res == controllerutil.OperationResultUpdated {
r.logger.Info("Capsule TLS certificates has been updated, we need to restart the Controller")
os.Exit(15)
}
r.logger.Info("Reconciliation completed, processing back in " + rq.String())
return reconcile.Result{Requeue: true, RequeueAfter: rq}, nil
}

View File

@@ -0,0 +1,485 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package tenant
import (
"context"
"fmt"
"strconv"
"strings"
"github.com/go-logr/logr"
corev1 "k8s.io/api/core/v1"
networkingv1 "k8s.io/api/networking/v1"
rbacv1 "k8s.io/api/rbac/v1"
"k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/api/resource"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/labels"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/selection"
"k8s.io/apimachinery/pkg/types"
"k8s.io/client-go/util/retry"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/controller"
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
"sigs.k8s.io/controller-runtime/pkg/handler"
"sigs.k8s.io/controller-runtime/pkg/log"
"sigs.k8s.io/controller-runtime/pkg/manager"
"sigs.k8s.io/controller-runtime/pkg/reconcile"
"sigs.k8s.io/controller-runtime/pkg/source"
capsulev1alpha1 "github.com/clastix/capsule/pkg/apis/capsule/v1alpha1"
)
// Add creates a new Tenant Controller and adds it to the Manager. The Manager will set fields on the Controller
// and Start it when the Manager is Started.
func Add(mgr manager.Manager) error {
return add(mgr, &ReconcileTenant{
client: mgr.GetClient(),
scheme: mgr.GetScheme(),
logger: log.Log.WithName("controller_tenant"),
})
}
// add adds a new Controller to mgr with r as the reconcile.Reconciler
func add(mgr manager.Manager, r reconcile.Reconciler) error {
// Create a new controller
c, err := controller.New("tenant-controller", mgr, controller.Options{Reconciler: r})
if err != nil {
return err
}
// Watch for changes to primary resource Tenant
err = c.Watch(&source.Kind{Type: &capsulev1alpha1.Tenant{}}, &handler.EnqueueRequestForObject{}, )
if err != nil {
return err
}
// Watch for controlled resources
for _, r := range []runtime.Object{&networkingv1.NetworkPolicy{}, &corev1.LimitRange{}, &corev1.ResourceQuota{}, &rbacv1.RoleBinding{}} {
err = c.Watch(&source.Kind{Type: r}, &handler.EnqueueRequestForOwner{
IsController: true,
OwnerType: &capsulev1alpha1.Tenant{},
})
if err != nil {
return err
}
}
return nil
}
// ReconcileTenant reconciles a Tenant object
type ReconcileTenant struct {
client client.Client
scheme *runtime.Scheme
logger logr.Logger
}
// Reconcile reads that state of the cluster for a Tenant object and makes changes based on the state read
// and what is in the Tenant.Spec
// The Controller will requeue the Request to be processed again if the returned error is non-nil or
// Result.Requeue is true, otherwise upon completion it will remove the work from the queue.
func (r *ReconcileTenant) Reconcile(request reconcile.Request) (reconcile.Result, error) {
r.logger = log.Log.WithName("controller_tenant").WithValues("Request.Name", request.Name)
// Fetch the Tenant instance
instance := &capsulev1alpha1.Tenant{}
err := r.client.Get(context.TODO(), request.NamespacedName, instance)
if err != nil {
if errors.IsNotFound(err) {
r.logger.Info("Request object not found, could have been deleted after reconcile request")
return reconcile.Result{}, nil
}
r.logger.Error(err, "Error reading the object")
return reconcile.Result{}, err
}
r.logger.Info("Starting processing of Network Policies", "items", len(instance.Spec.NetworkPolicies))
if err := r.syncNetworkPolicies(instance); err != nil {
r.logger.Error(err, "Cannot sync NetworkPolicy items")
return reconcile.Result{}, err
}
r.logger.Info("Starting processing of Node Selector")
if err := r.ensureNodeSelector(instance); err != nil {
r.logger.Error(err, "Cannot sync Namespaces Node Selector items")
return reconcile.Result{}, err
}
r.logger.Info("Starting processing of Limit Ranges", "items", len(instance.Spec.LimitRanges))
if err := r.syncLimitRanges(instance); err != nil {
r.logger.Error(err, "Cannot sync LimitRange items")
return reconcile.Result{}, err
}
r.logger.Info("Starting processing of Resource Quotas", "items", len(instance.Spec.ResourceQuota))
if err := r.syncResourceQuotas(instance); err != nil {
r.logger.Error(err, "Cannot sync ResourceQuota items")
return reconcile.Result{}, err
}
r.logger.Info("Ensuring RoleBinding for owner")
if err := r.ownerRoleBinding(instance); err != nil {
r.logger.Error(err, "Cannot sync owner RoleBinding")
return reconcile.Result{}, err
}
r.logger.Info("Tenant reconciling completed")
return reconcile.Result{}, nil
}
// pruningResources is taking care of removing the no more requested sub-resources as LimitRange, ResourceQuota or
// NetworkPolicy using the "notin" LabelSelector to perform an outer-join removal.
func (r *ReconcileTenant) pruningResources(ns string, keys []string, obj runtime.Object) error {
capsuleLabel, err := capsulev1alpha1.GetTypeLabel(obj)
if err != nil {
return err
}
req, err := labels.NewRequirement(capsuleLabel, selection.NotIn, keys)
if err != nil {
return err
}
r.logger.Info("Pruning objects with label selector " + req.String())
err = retry.RetryOnConflict(retry.DefaultBackoff, func() error {
return r.client.DeleteAllOf(context.TODO(), obj, &client.DeleteAllOfOptions{
ListOptions: client.ListOptions{
LabelSelector: labels.NewSelector().Add(*req),
Namespace: ns,
},
DeleteOptions: client.DeleteOptions{},
})
})
if err != nil {
return err
}
return nil
}
// We're relying on the ResourceQuota resource to represent the resource quota for the single Tenant rather than the
// single Namespace, so abusing of this API although its Namespaced scope.
// Since a Namespace could take-up all the available resource quota, the Namespace ResourceQuota will be a 1:1 mapping
// to the Tenant one: in a second time Capsule is going to sum all the analogous ResourceQuota resources on other Tenant
// namespaces to check if the Tenant quota has been exceeded or not, reusing the native Kubernetes policy putting the
// .Status.Used value as the .Hard value.
// This will trigger a following reconciliation but that's ok: the mutateFn will re-use the same business logic, letting
// the mutateFn along with the CreateOrUpdate to don't perform the update since resources are identical.
func (r *ReconcileTenant) syncResourceQuotas(tenant *capsulev1alpha1.Tenant) error {
// getting requested ResourceQuota keys
keys := make([]string, 0, len(tenant.Spec.ResourceQuota))
for i := range tenant.Spec.ResourceQuota {
keys = append(keys, strconv.Itoa(i))
}
// getting ResourceQuota labels for the mutateFn
tenantLabel, err := capsulev1alpha1.GetTypeLabel(&capsulev1alpha1.Tenant{})
if err != nil {
return err
}
typeLabel, err := capsulev1alpha1.GetTypeLabel(&corev1.ResourceQuota{})
if err != nil {
return err
}
for _, ns := range tenant.Status.Namespaces {
if err := r.pruningResources(ns, keys, &corev1.ResourceQuota{}); err != nil {
return err
}
for i, q := range tenant.Spec.ResourceQuota {
target := &corev1.ResourceQuota{
ObjectMeta: metav1.ObjectMeta{
Name: fmt.Sprintf("capsule-%s-%d", tenant.Name, i),
Namespace: ns,
Annotations: make(map[string]string),
Labels: map[string]string{
tenantLabel: tenant.Name,
typeLabel: strconv.Itoa(i),
},
},
}
res, err := controllerutil.CreateOrUpdate(context.TODO(), r.client, target, func() (err error) {
// Requirement to list ResourceQuota of the current Tenant
tr, err := labels.NewRequirement(tenantLabel, selection.Equals, []string{tenant.Name})
if err != nil {
r.logger.Error(err, "Cannot build ResourceQuota Tenant requirement")
}
// Requirement to list ResourceQuota for the current index
ir, err := labels.NewRequirement(typeLabel, selection.Equals, []string{strconv.Itoa(i)})
if err != nil {
r.logger.Error(err, "Cannot build ResourceQuota index requirement")
}
// Listing all the ResourceQuota according to the said requirements.
// These are required since Capsule is going to sum all the used quota to
// sum them and get the Tenant one.
rql := &corev1.ResourceQuotaList{}
err = r.client.List(context.TODO(), rql, &client.ListOptions{
LabelSelector: labels.NewSelector().Add(*tr).Add(*ir),
})
if err != nil {
r.logger.Error(err, "Cannot list ResourceQuota", "tenantFilter", tr.String(), "indexFilter", ir.String())
return err
}
// Iterating over all the options declared for the ResourceQuota,
// summing all the used quota across different Namespaces to determinate
// if we're hitting a Hard quota at Tenant level.
// For this case, we're going to block the Quota setting the Hard as the
// used one.
for rn, rq := range q.Hard {
r.logger.Info("Desired hard " + rn.String() + " quota is " + rq.String())
// Getting the whole usage across all the Tenant Namespaces
var qt resource.Quantity
for _, rq := range rql.Items {
qt.Add(rq.Status.Used[rn])
}
r.logger.Info("Computed " + rn.String() + " quota for the whole Tenant is " + qt.String())
switch qt.Cmp(q.Hard[rn]) {
case 1:
// The Tenant is OverQuota:
// updating all the related ResourceQuota with the current
// used Quota to block further creations.
for i := range rql.Items {
rql.Items[i].Spec.Hard[rn] = rql.Items[i].Status.Used[rn]
}
println("")
default:
// The Tenant is respecting the Hard quota:
// restoring the default one for all the elements,
// also for the reconciliated one.
for i := range rql.Items {
rql.Items[i].Spec.Hard[rn] = q.Hard[rn]
}
target.Spec = q
}
// Updating all outer join ResourceQuota adding the Used for the current Resource
// TODO(prometherion): this is too expensive, should be performed via a recursion
for _, oj := range rql.Items {
err := retry.RetryOnConflict(retry.DefaultBackoff, func() error {
_ = r.client.Get(context.TODO(), types.NamespacedName{Namespace: oj.Namespace, Name: oj.Name}, &oj)
if oj.Annotations == nil {
oj.Annotations = make(map[string]string)
}
oj.Annotations[capsulev1alpha1.UsedQuotaFor(rn)] = qt.String()
return r.client.Update(context.TODO(), &oj, &client.UpdateOptions{})
})
if err != nil {
return err
}
}
}
return controllerutil.SetControllerReference(tenant, target, r.scheme)
})
r.logger.Info("Resource Quota sync result: "+string(res), "name", target.Name, "namespace", target.Namespace)
if err != nil {
return err
}
}
}
return nil
}
// Ensuring all the LimitRange are applied to each Namespace handled by the Tenant.
func (r *ReconcileTenant) syncLimitRanges(tenant *capsulev1alpha1.Tenant) error {
// getting requested LimitRange keys
keys := make([]string, 0, len(tenant.Spec.LimitRanges))
for i := range tenant.Spec.LimitRanges {
keys = append(keys, strconv.Itoa(i))
}
// getting LimitRange labels for the mutateFn
tl, err := capsulev1alpha1.GetTypeLabel(&capsulev1alpha1.Tenant{})
if err != nil {
return err
}
ll, err := capsulev1alpha1.GetTypeLabel(&corev1.LimitRange{})
if err != nil {
return err
}
for _, ns := range tenant.Status.Namespaces {
if err := r.pruningResources(ns, keys, &corev1.LimitRange{}); err != nil {
return err
}
for i, spec := range tenant.Spec.LimitRanges {
t := &corev1.LimitRange{
ObjectMeta: metav1.ObjectMeta{
Name: fmt.Sprintf("capsule-%s-%d", tenant.Name, i),
Namespace: ns,
},
}
res, err := controllerutil.CreateOrUpdate(context.TODO(), r.client, t, func() (err error) {
t.ObjectMeta.Labels = map[string]string{
tl: tenant.Name,
ll: strconv.Itoa(i),
}
t.Spec = spec
return controllerutil.SetControllerReference(tenant, t, r.scheme)
})
r.logger.Info("LimitRange sync result: "+string(res), "name", t.Name, "namespace", t.Namespace)
if err != nil {
return err
}
}
}
return nil
}
// Ensuring all the NetworkPolicies are applied to each Namespace handled by the Tenant.
func (r *ReconcileTenant) syncNetworkPolicies(tenant *capsulev1alpha1.Tenant) error {
// getting requested NetworkPolicy keys
keys := make([]string, 0, len(tenant.Spec.NetworkPolicies))
for i := range tenant.Spec.NetworkPolicies {
keys = append(keys, strconv.Itoa(i))
}
// getting NetworkPolicy labels for the mutateFn
tl, err := capsulev1alpha1.GetTypeLabel(&capsulev1alpha1.Tenant{})
if err != nil {
return err
}
nl, err := capsulev1alpha1.GetTypeLabel(&networkingv1.NetworkPolicy{})
if err != nil {
return err
}
for _, ns := range tenant.Status.Namespaces {
if err := r.pruningResources(ns, keys, &networkingv1.NetworkPolicy{}); err != nil {
return err
}
for i, spec := range tenant.Spec.NetworkPolicies {
t := &networkingv1.NetworkPolicy{
ObjectMeta: metav1.ObjectMeta{
Name: fmt.Sprintf("capsule-%s-%d", tenant.Name, i),
Namespace: ns,
Labels: map[string]string{
tl: tenant.Name,
nl: strconv.Itoa(i),
},
},
}
res, err := controllerutil.CreateOrUpdate(context.TODO(), r.client, t, func() (err error) {
t.Spec = spec
return controllerutil.SetControllerReference(tenant, t, r.scheme)
})
r.logger.Info("Network Policy sync result: "+string(res), "name", t.Name, "namespace", t.Namespace)
if err != nil {
return err
}
}
}
return nil
}
// Each Tenant owner needs the admin Role attached to each Namespace, otherwise no actions on it can be performed.
// Since RBAC is based on deny all first, some specific actions like editing Capsule resources are going to be blocked
// via Dynamic Admission Webhooks.
// TODO(prometherion): we could create a capsule:admin role rather than hitting webhooks for each action
func (r *ReconcileTenant) ownerRoleBinding(tenant *capsulev1alpha1.Tenant) error {
// getting RoleBinding label for the mutateFn
tl, err := capsulev1alpha1.GetTypeLabel(&capsulev1alpha1.Tenant{})
if err != nil {
return err
}
l := map[string]string{tl: tenant.Name}
s := []rbacv1.Subject{
{
Kind: "User",
Name: tenant.Spec.Owner,
},
}
rbl := make(map[types.NamespacedName]rbacv1.RoleRef)
for _, i := range tenant.Status.Namespaces {
rbl[types.NamespacedName{Namespace: i, Name: "namespace:admin"}] = rbacv1.RoleRef{
APIGroup: "rbac.authorization.k8s.io",
Kind: "ClusterRole",
Name: "admin",
}
rbl[types.NamespacedName{Namespace: i, Name: "namespace:deleter"}] = rbacv1.RoleRef{
APIGroup: "rbac.authorization.k8s.io",
Kind: "ClusterRole",
Name: "namespace:deleter",
}
}
for nn, rr := range rbl {
target := &rbacv1.RoleBinding{
ObjectMeta: metav1.ObjectMeta{
Name: nn.Name,
Namespace: nn.Namespace,
},
}
var res controllerutil.OperationResult
res, err = controllerutil.CreateOrUpdate(context.TODO(), r.client, target, func() (err error) {
target.ObjectMeta.Labels = l
target.Subjects = s
target.RoleRef = rr
return controllerutil.SetControllerReference(tenant, target, r.scheme)
})
r.logger.Info("Role Binding sync result: "+string(res), "name", target.Name, "namespace", target.Namespace)
if err != nil {
return err
}
}
return nil
}
func (r *ReconcileTenant) ensureNodeSelector(tenant *capsulev1alpha1.Tenant) (err error) {
if tenant.Spec.NodeSelector == nil {
return
}
for _, namespace := range tenant.Status.Namespaces {
selectorMap := tenant.Spec.NodeSelector
if selectorMap == nil {
return
}
ns := &corev1.Namespace{
ObjectMeta: metav1.ObjectMeta{
Name: namespace,
},
}
var res controllerutil.OperationResult
res, err = controllerutil.CreateOrUpdate(context.TODO(), r.client, ns, func() error {
if ns.Annotations == nil {
ns.Annotations = make(map[string]string)
}
var selector []string
for k, v := range selectorMap {
selector = append(selector, fmt.Sprintf("%s=%s", k, v))
}
ns.Annotations["scheduler.alpha.kubernetes.io/node-selector"] = strings.Join(selector, ",")
return nil
})
r.logger.Info("Namespace Node sync result: "+string(res), "name", ns.Name)
if err != nil {
return err
}
}
return
}

View File

@@ -0,0 +1,20 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package indexer
import "github.com/clastix/capsule/pkg/indexer/tenant"
func init() {
AddToIndexerFuncs = append(AddToIndexerFuncs, tenant.OwnerReference{})
}

20
pkg/indexer/add_owner.go Normal file
View File

@@ -0,0 +1,20 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package indexer
import "github.com/clastix/capsule/pkg/indexer/tenant"
func init() {
AddToIndexerFuncs = append(AddToIndexerFuncs, tenant.NamespacesReference{})
}

38
pkg/indexer/indexer.go Normal file
View File

@@ -0,0 +1,38 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package indexer
import (
"context"
"k8s.io/apimachinery/pkg/runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/manager"
)
type CustomIndexer interface {
Object() runtime.Object
Field() string
Func() client.IndexerFunc
}
var AddToIndexerFuncs []CustomIndexer
func AddToManager(m manager.Manager) error {
for _, f := range AddToIndexerFuncs {
if err := m.GetFieldIndexer().IndexField(context.TODO(), f.Object(), f.Field(), f.Func()); err != nil {
return err
}
}
return nil
}

View File

@@ -0,0 +1,39 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package tenant
import (
"k8s.io/apimachinery/pkg/runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
"github.com/clastix/capsule/pkg/apis/capsule/v1alpha1"
)
type NamespacesReference struct {
}
func (o NamespacesReference) Object() runtime.Object {
return &v1alpha1.Tenant{}
}
func (o NamespacesReference) Field() string {
return ".status.namespaces"
}
func (o NamespacesReference) Func() client.IndexerFunc {
return func(object runtime.Object) (res []string) {
tenant := object.(*v1alpha1.Tenant)
return tenant.Status.Namespaces.DeepCopy()
}
}

View File

@@ -0,0 +1,39 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package tenant
import (
"k8s.io/apimachinery/pkg/runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
"github.com/clastix/capsule/pkg/apis/capsule/v1alpha1"
)
type OwnerReference struct {
}
func (o OwnerReference) Object() runtime.Object {
return &v1alpha1.Tenant{}
}
func (o OwnerReference) Field() string {
return ".spec.owner"
}
func (o OwnerReference) Func() client.IndexerFunc {
return func(object runtime.Object) []string {
tenant := object.(*v1alpha1.Tenant)
return []string{tenant.Spec.Owner}
}
}

View File

@@ -0,0 +1,22 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package webhook
import (
"github.com/clastix/capsule/pkg/webhook/ingress_class"
)
func init() {
AddToWebhookServer = append(AddToWebhookServer, ingress_class.AddExtensions, ingress_class.AddNetworking)
}

View File

@@ -0,0 +1,22 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package webhook
import (
"github.com/clastix/capsule/pkg/webhook/namespace_quota"
)
func init() {
AddToWebhookServer = append(AddToWebhookServer, namespace_quota.Add)
}

View File

@@ -0,0 +1,20 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package webhook
import "github.com/clastix/capsule/pkg/webhook/network_policies"
func init() {
AddToWebhookServer = append(AddToWebhookServer, network_policies.Add)
}

View File

@@ -0,0 +1,20 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package webhook
import "github.com/clastix/capsule/pkg/webhook/owner_reference"
func init() {
AddToWebhookServer = append(AddToWebhookServer, owner_reference.Add)
}

22
pkg/webhook/add_pvc.go Normal file
View File

@@ -0,0 +1,22 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package webhook
import (
"github.com/clastix/capsule/pkg/webhook/pvc"
)
func init() {
AddToWebhookServer = append(AddToWebhookServer, pvc.Add)
}

View File

@@ -0,0 +1,64 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package ingress_class
import (
"context"
"net/http"
extensionsv1beta1 "k8s.io/api/extensions/v1beta1"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/manager"
"sigs.k8s.io/controller-runtime/pkg/webhook"
"sigs.k8s.io/controller-runtime/pkg/webhook/admission"
"github.com/clastix/capsule/pkg/webhook/utils"
)
func AddExtensions(mgr manager.Manager) error {
mgr.GetWebhookServer().Register("/validating-v1-extensions-ingress", &webhook.Admission{
Handler: &extensionIngress{},
})
return nil
}
type extensionIngress struct {
client client.Client
decoder *admission.Decoder
}
func (r *extensionIngress) Handle(ctx context.Context, req admission.Request) admission.Response {
g := utils.UserGroupList(req.UserInfo.Groups)
if !g.IsInCapsuleGroup() {
// not a Capsule user, can be skipped
return admission.Allowed("")
}
i := &extensionsv1beta1.Ingress{}
if err := r.decoder.Decode(req, i); err != nil {
return admission.Errored(http.StatusBadRequest, err)
}
return handleIngress(ctx, i, i.Spec.IngressClassName, r.client)
}
func (r *extensionIngress) InjectDecoder(d *admission.Decoder) error {
r.decoder = d
return nil
}
func (r *extensionIngress) InjectClient(c client.Client) error {
r.client = c
return nil
}

View File

@@ -0,0 +1,51 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package ingress_class
import (
"context"
"fmt"
"net/http"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/fields"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/webhook/admission"
"github.com/clastix/capsule/pkg/apis/capsule/v1alpha1"
)
func handleIngress(ctx context.Context, object metav1.Object, ic *string, c client.Client) admission.Response {
if v, ok := object.GetAnnotations()["kubernetes.io/ingress.class"]; ok {
ic = &v
}
if ic == nil {
return admission.Errored(http.StatusBadRequest, fmt.Errorf("A valid Ingress Class must be used"))
}
tl := &v1alpha1.TenantList{}
if err := c.List(ctx, tl, client.MatchingFieldsSelector{
Selector: fields.OneTermEqualSelector(".status.namespaces", object.GetNamespace()),
}); err != nil {
return admission.Errored(http.StatusBadRequest, err)
}
if !tl.Items[0].Spec.IngressClasses.IsStringInList(*ic) {
err := fmt.Errorf("Ingress Class %s is forbidden for the current Tenant", *ic)
return admission.Errored(http.StatusBadRequest, err)
}
return admission.Allowed("")
}

View File

@@ -0,0 +1,64 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package ingress_class
import (
"context"
"net/http"
networkingv1beta1 "k8s.io/api/networking/v1beta1"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/manager"
"sigs.k8s.io/controller-runtime/pkg/webhook"
"sigs.k8s.io/controller-runtime/pkg/webhook/admission"
"github.com/clastix/capsule/pkg/webhook/utils"
)
func AddNetworking(mgr manager.Manager) error {
mgr.GetWebhookServer().Register("/validating-v1-networking-ingress", &webhook.Admission{
Handler: &validatingV1{},
})
return nil
}
type validatingV1 struct {
client client.Client
decoder *admission.Decoder
}
func (r *validatingV1) Handle(ctx context.Context, req admission.Request) admission.Response {
g := utils.UserGroupList(req.UserInfo.Groups)
if !g.IsInCapsuleGroup() {
// not a Capsule user, can be skipped
return admission.Allowed("")
}
i := &networkingv1beta1.Ingress{}
if err := r.decoder.Decode(req, i); err != nil {
return admission.Errored(http.StatusBadRequest, err)
}
return handleIngress(ctx, i, i.Spec.IngressClassName, r.client)
}
func (r *validatingV1) InjectDecoder(d *admission.Decoder) error {
r.decoder = d
return nil
}
func (r *validatingV1) InjectClient(c client.Client) error {
r.client = c
return nil
}

View File

@@ -0,0 +1,71 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package namespace_quota
import (
"context"
"net/http"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/types"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/manager"
"sigs.k8s.io/controller-runtime/pkg/webhook"
"sigs.k8s.io/controller-runtime/pkg/webhook/admission"
"github.com/clastix/capsule/pkg/apis/capsule/v1alpha1"
)
func Add(mgr manager.Manager) error {
mgr.GetWebhookServer().Register("/validate-v1-namespace-quota", &webhook.Admission{
Handler: &nsQuota{},
})
return nil
}
type nsQuota struct {
client client.Client
decoder *admission.Decoder
}
func (r *nsQuota) Handle(ctx context.Context, req admission.Request) admission.Response {
// Decoding the NS
ns := &corev1.Namespace{}
if err := r.decoder.Decode(req, ns); err != nil {
return admission.Errored(http.StatusBadRequest, err)
}
for _, or := range ns.ObjectMeta.OwnerReferences {
// retrieving the selected Tenant
t := &v1alpha1.Tenant{}
if err := r.client.Get(ctx, types.NamespacedName{Name: or.Name}, t); err != nil {
return admission.Errored(http.StatusBadRequest, err)
}
if t.IsFull() {
return admission.Denied("Cannot exceed Namespace quota: please, reach out the system administrators")
}
}
// creating NS that is not bounded to any Tenant
return admission.Allowed("")
}
func (r *nsQuota) InjectDecoder(d *admission.Decoder) error {
r.decoder = d
return nil
}
func (r *nsQuota) InjectClient(c client.Client) error {
r.client = c
return nil
}

View File

@@ -0,0 +1,95 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package network_policies
import (
"context"
"github.com/clastix/capsule/pkg/webhook/utils"
"net/http"
"k8s.io/api/admission/v1beta1"
networkingv1 "k8s.io/api/networking/v1"
"k8s.io/apimachinery/pkg/types"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/manager"
"sigs.k8s.io/controller-runtime/pkg/webhook"
"sigs.k8s.io/controller-runtime/pkg/webhook/admission"
"github.com/clastix/capsule/pkg/apis/capsule/v1alpha1"
)
func Add(mgr manager.Manager) error {
mgr.GetWebhookServer().Register("/validating-v1-network-policy", &webhook.Admission{
Handler: &validatingNetworkPolicy{},
})
return nil
}
type validatingNetworkPolicy struct {
client client.Client
decoder *admission.Decoder
}
func (r *validatingNetworkPolicy) Handle(ctx context.Context, req admission.Request) admission.Response {
var err error
g := utils.UserGroupList(req.UserInfo.Groups)
if !g.IsInCapsuleGroup() {
// not a Capsule user, can be skipped
return admission.Allowed("")
}
np := &networkingv1.NetworkPolicy{}
switch req.Operation {
case v1beta1.Delete:
err := r.client.Get(ctx, types.NamespacedName{
Namespace: req.AdmissionRequest.Namespace,
Name: req.AdmissionRequest.Name,
}, np)
if err != nil {
return admission.Errored(http.StatusInternalServerError, err)
}
default:
if err := r.decoder.Decode(req, np); err != nil {
return admission.Errored(http.StatusInternalServerError, err)
}
err = r.client.Get(ctx, types.NamespacedName{
Namespace: np.Namespace,
Name: np.Name,
}, np)
if err != nil {
return admission.Errored(http.StatusInternalServerError, err)
}
}
l, err := v1alpha1.GetTypeLabel(&v1alpha1.Tenant{})
if err != nil {
return admission.Errored(http.StatusInternalServerError, err)
}
if _, ok := np.GetLabels()[l]; ok {
return admission.Denied("Capsule Network Policies cannot be manipulated: please, reach out the system administrators")
}
// manipulating user Network Policy: it's safe
return admission.Allowed("")
}
func (r *validatingNetworkPolicy) InjectDecoder(d *admission.Decoder) error {
r.decoder = d
return nil
}
func (r *validatingNetworkPolicy) InjectClient(c client.Client) error {
r.client = c
return nil
}

View File

@@ -0,0 +1,120 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package owner_reference
import (
"context"
"encoding/json"
"github.com/clastix/capsule/pkg/webhook/utils"
"net/http"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/fields"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/types"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
"sigs.k8s.io/controller-runtime/pkg/manager"
"sigs.k8s.io/controller-runtime/pkg/webhook"
"sigs.k8s.io/controller-runtime/pkg/webhook/admission"
"github.com/clastix/capsule/pkg/apis/capsule/v1alpha1"
)
func Add(mgr manager.Manager) error {
mgr.GetWebhookServer()
mgr.GetWebhookServer().Register("/mutate-v1-namespace-owner-reference", &webhook.Admission{
Handler: &ownerRef{
schema: mgr.GetScheme(),
},
})
return nil
}
type ownerRef struct {
client client.Client
decoder *admission.Decoder
// injecting the runtime.Scheme for controllerutil.SetOwnerReference
schema *runtime.Scheme
}
func (r *ownerRef) Handle(ctx context.Context, req admission.Request) admission.Response {
// Decoding the NS
ns := &corev1.Namespace{}
if err := r.decoder.Decode(req, ns); err != nil {
return admission.Errored(http.StatusBadRequest, err)
}
g := utils.UserGroupList(req.UserInfo.Groups)
if !g.IsInCapsuleGroup() {
// user requested NS creation is not a Capsule user, so skipping the validation checks
return admission.Allowed("")
}
if len(ns.ObjectMeta.Labels) > 0 {
ln, err := v1alpha1.GetTypeLabel(&v1alpha1.Tenant{})
if err != nil {
return admission.Errored(http.StatusBadRequest, err)
}
l, ok := ns.ObjectMeta.Labels[ln]
// assigning namespace to Tenant in case of label
if ok {
// retrieving the selected Tenant
t := &v1alpha1.Tenant{}
if err := r.client.Get(ctx, types.NamespacedName{Name: l}, t); err != nil {
return admission.Errored(http.StatusBadRequest, err)
}
// Tenant owner must adhere to user that asked for NS creation
if t.Spec.Owner != req.UserInfo.Username {
return admission.Denied("Cannot assign the desired namespace to a non-owned Tenant")
}
// Patching the response
return r.patchResponseForOwnerRef(t, ns)
}
}
tl := &v1alpha1.TenantList{}
if err := r.client.List(ctx, tl, client.MatchingFieldsSelector{
Selector: fields.OneTermEqualSelector(".spec.owner", req.UserInfo.Username),
}); err != nil {
return admission.Errored(http.StatusBadRequest, err)
}
if len(tl.Items) > 0 {
return r.patchResponseForOwnerRef(&tl.Items[0], ns)
}
return admission.Denied("You do not have any Tenant assigned: please, reach out the system administrators")
}
func (r *ownerRef) patchResponseForOwnerRef(tenant *v1alpha1.Tenant, ns *corev1.Namespace) admission.Response {
o, _ := json.Marshal(ns.DeepCopy())
if err := controllerutil.SetControllerReference(tenant, ns, r.schema); err != nil {
return admission.Errored(http.StatusInternalServerError, err)
}
c, _ := json.Marshal(ns)
return admission.PatchResponseFromRaw(o, c)
}
func (r *ownerRef) InjectDecoder(d *admission.Decoder) error {
r.decoder = d
return nil
}
func (r *ownerRef) InjectClient(c client.Client) error {
r.client = c
return nil
}

View File

@@ -0,0 +1,84 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package pvc
import (
"context"
"fmt"
"net/http"
v1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/fields"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/manager"
"sigs.k8s.io/controller-runtime/pkg/webhook"
"sigs.k8s.io/controller-runtime/pkg/webhook/admission"
"github.com/clastix/capsule/pkg/apis/capsule/v1alpha1"
"github.com/clastix/capsule/pkg/webhook/utils"
)
func Add(mgr manager.Manager) error {
mgr.GetWebhookServer().Register("/validating-v1-pvc", &webhook.Admission{
Handler: &validatindPvc{},
})
return nil
}
type validatindPvc struct {
client client.Client
decoder *admission.Decoder
}
func (r *validatindPvc) Handle(ctx context.Context, req admission.Request) admission.Response {
g := utils.UserGroupList(req.UserInfo.Groups)
if !g.IsInCapsuleGroup() {
// not a Capsule user, can be skipped
return admission.Allowed("")
}
pvc := &v1.PersistentVolumeClaim{}
if err := r.decoder.Decode(req, pvc); err != nil {
return admission.Errored(http.StatusBadRequest, err)
}
if pvc.Spec.StorageClassName == nil {
return admission.Errored(http.StatusBadRequest, fmt.Errorf("A valid Strage Class must be used"))
}
sc := *pvc.Spec.StorageClassName
tl := &v1alpha1.TenantList{}
if err := r.client.List(ctx, tl, client.MatchingFieldsSelector{
Selector: fields.OneTermEqualSelector(".status.namespaces", pvc.Namespace),
}); err != nil {
return admission.Errored(http.StatusBadRequest, err)
}
if !tl.Items[0].Spec.StorageClasses.IsStringInList(sc) {
err := fmt.Errorf("Storage Class %s is forbidden for the current Tenant", *pvc.Spec.StorageClassName)
return admission.Errored(http.StatusBadRequest, err)
}
return admission.Allowed("")
}
func (r *validatindPvc) InjectDecoder(d *admission.Decoder) error {
r.decoder = d
return nil
}
func (r *validatindPvc) InjectClient(c client.Client) error {
r.client = c
return nil
}

View File

@@ -0,0 +1,44 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package utils
import (
"sort"
"strings"
"github.com/clastix/capsule/pkg/apis/capsule/v1alpha1"
)
type UserGroupList []string
func (u UserGroupList) Len() int {
return len(u)
}
func (u UserGroupList) Less(i, j int) bool {
return strings.ToLower(u[i]) < strings.ToLower(u[j])
}
func (u UserGroupList) Swap(i, j int) {
u[i], u[j] = u[j], u[i]
}
func (u UserGroupList) IsInCapsuleGroup() (ok bool) {
v := v1alpha1.SchemeGroupVersion.Group
sort.Sort(u)
i := sort.SearchStrings(u, v)
ok = i < u.Len() && u[i] == v
return
}

37
pkg/webhook/webhook.go Normal file
View File

@@ -0,0 +1,37 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package webhook
import (
"io/ioutil"
"sigs.k8s.io/controller-runtime/pkg/manager"
)
// AddToWebhookServer is a list of functions to create webhooks and add them to a manager.
var AddToWebhookServer []func(manager2 manager.Manager) error
// AddToServer adds all Controllers to the Manager
func AddToServer(mgr manager.Manager) error {
// skipping webhook setup if certificate is missing
dat, _ := ioutil.ReadFile("/tmp/k8s-webhook-server/serving-certs/tls.crt")
if len(dat) == 0 {
return nil
}
for _, f := range AddToWebhookServer {
if err := f(mgr); err != nil {
return err
}
}
return nil
}

5
tools.go Normal file
View File

@@ -0,0 +1,5 @@
// +build tools
// Place any runtime dependencies as imports in this file.
// Go modules will be forced to download and install them.
package tools

855
use_cases.md Normal file
View File

@@ -0,0 +1,855 @@
# Use cases for Capsule
## Acme Corp. Public Container as a Service (CaaS) platform
Acme Corp. is a cloud provider that wants to enhance their public offer with a
new CaaS service based on Kubernetes.
Acme Corp. already provides an _Infrastructure as a Service_ (IaaS) platform
with VMs, Storage, DBaaS, and other managed traditional services.
### The background
The new CaaS service from Acme Corp. will include:
- **Shared CaaS**:
* Shared infra and worker nodes.
* Shared embedded registry.
* Shared control plane.
* Shared Public IP addresses.
* Shared Persistent Storage.
* Automatic backup of volumes.
* Shared routing layer with shared wildcard certificate.
* Multiple Namespaces isolation.
* Single user account.
* Resources Quotas and Limits.
* Self Service Provisioning portal.
* Shared Application Catalog.
- **Private CaaS**:
* Dedicated infra and worker nodes.
* Dedicated registry.
* Dedicated routing layer with dedicated wildcard certificates.
* Dedicated Public IP addresses.
* Dedicated Persistent Storage.
* Automatic backup of volumes.
* Shared control plane.
* Multiple Namespaces isolation.
* Resources Quotas and Limits.
* Self Service Provisioning portal.
* Dedicated Application Catalog.
* Multiple user accounts.
* Optional access to VMs, Storage, Networks, DBaaS, and other managed
traditional services from the IaaS offer.
### Involved actors
To simplify the design of Capsule, we'll work with following actors:
* *Bill*:
he is the cluster administrator from the operations department of Acme Corp.
and he is in charge of admin and mantain the CaaS platform.
Bill is also responsible for the onboarding of new customers and of the
daily work to support all customers.
* *Joe*:
he works as DevOps engineer at Oil & Stracci Inc., a new customer of the
Shared CaaS service.
Joe is responsible for deploying and mantaining container based applications
on the CaaS platform.
* *Alice*:
she works as IT Project Leader at Bastard Bank Inc.,
a new Private CaaS customer. Alice is responsible for a stategic IT project
and she is responsible also for a large team made of different background
(developers, administrators, SRE engineers, etc.) and organised in separated
departments.
### Some scenarios:
* [onboarding of new customer](#onboarding-of-new-customer)
* [create namespaces in a tenant](#create-namespaces-in-a-tenant)
* [quota enforcement for a tenant](#quota-enforcement-for-a-tenant)
* [node selector for a tenant](#node-selector-for-a-tenant)
* [ingress selector for a tenant](#ingress-selector-for-a-tenant)
* [network policies for a tenant](#network-policies-for-a-tenant)
* [storage class for a tenant](#storage-class-for-a-tenant)
<!-- TODO: need to be implemented
* [access images registry from a tenant](#access-images-registry-from-a-tenant)
* [backup and restore in a tenant](#backup-and-restore-in-a-tenant)
* [user management](#user-management)
-->
### Onboarding of new Customer
Bill receives a new request from the CaaS onboarding system that a new
Shared CaaS customer "Oil & Stracci Inc." has to be on board. This request
reports the name of the tenant owner and the total amount of purchased
resources: namespaces, CPU, memory, storage, ...
Bill creates a new user account id `Joe` in the Acme Corp. identity management
system and assign Joe to the group of the Shared CaaS user. To keep the things
simple, we assume that Bill just creates a certificate for authentication on
the CaaS platform using X.509 certificate, so the Joe's certificate has
`"/CN=joe/O=capsule.clastix.io"`.
Bill creates a new tenant `oil-and-stracci-inc` in the CaaS manangement portal
according to the tenant's profile:
```yaml
apiVersion: capsule.clastix.io/v1alpha1
kind: Tenant
metadata:
labels:
annotations:
name: oil-and-stracci-inc
spec:
owner: joe
nodeSelector:
node-role.kubernetes.io/capsule: caas
storageClasses:
- ceph-rbd
namespaceQuota: 10
resourceQuotas:
- hard:
limits.cpu: "8"
limits.memory: 16Gi
requests.cpu: "8"
requests.memory: 16Gi
scopes: ["NotTerminating"]
- hard:
pods : "10"
services: "5"
deployments: "5"
- spec:
hard:
requests.storage: "100Gi"
limitRanges:
- limits:
- type: Pod
min:
cpu: "50m"
memory: "5Mi"
max:
cpu: "1"
memory: "1Gi"
- type: Container
defaultRequest:
cpu: "100m"
memory: "10Mi"
default:
cpu: "200m"
memory: "100Mi"
min:
cpu: "50m"
memory: "5Mi"
max:
cpu: "1"
memory: "1Gi"
- type: PersistentVolumeClaim
min:
storage: "1Gi"
max:
storage: "10Gi"
networkPolicies:
- policyTypes:
- Ingress
- Egress
podSelector: {}
ingress:
- from:
- namespaceSelector:
matchLabels:
tenant: oil-and-stracci-inc
- podSelector: {}
- ipBlock:
cidr: 192.168.0.0/16
egress:
- to:
- ipBlock:
cidr: 0.0.0.0/0
except:
- 192.168.0.0/16
```
> Note that namespaces are not yet assigned to the tenant.
> The CaaS users are free to create their namespaces in a self-service fashion
> and without any intervent from Bill.
Once the new tenant `oil-and-stracci-inc` is in place, Bill sends the login
credentials to Joe along with the tenant details, for logging into the CaaS.
Joe logs into the CaaS by using his credentials and being part of the
`capsule.clastix.io` users group, he inherits the following authorization:
```yaml
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
labels:
name: namespace:provisioner
rules:
- apiGroups: [""]
resources: ["namespaces"]
verbs: ["create"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: namespace:provisioner
subjects:
- kind: Group
name: capsule.clastix.io
roleRef:
kind: ClusterRole
name: namespace:provisioner
apiGroup: rbac.authorization.k8s.io
```
Joe can login to the CaaS platform and checks if he can create a namespace.
```
# kubectl auth can-i create namespaces
Warning: resource 'namespaces' is not namespace scoped
yes
```
However, cluster resources are not accessible to Joe
```
# kubectl auth can-i get namespaces
Warning: resource 'namespaces' is not namespace scoped
no
# kubectl auth can-i get nodes
Warning: resource 'nodes' is not namespace scoped
no
# kubectl auth can-i get persistentvolumes
Warning: resource 'persistentvolumes' is not namespace scoped
no
```
including the `Tenant` resources
```
# kubectl auth can-i get tenants
Warning: resource 'tenants' is not namespace scoped
no
```
### Create namespaces in a tenant
Joe can create a new namespace in his tenant, as simply:
```
# kubectl create ns oil-production
```
> Note that Joe started the name of his namespace with an identifier of his
> tenant: this is not a strict requirement but it is higly suggested because
> it is likely that many different users would like to call their namespaces
> as `production`, `test`, or `demo`, etc.
>
> The enforcement of this rule, however, is not in charge of the Capsule
> controller and it is left to a policy engine.
When Joe creates the namespace, the Capsule controller, listening for creation
and deletion events, assigns to Joe the following roles:
```yaml
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: namespace:admin
namespace: oil-production
subjects:
- kind: User
name: joe
roleRef:
kind: ClusterRole
name: admin
apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: namespace:deleter
namespace: oil-production
subjects:
- kind: User
name: joe
roleRef:
kind: ClusterRole
name: namespace:deleter
apiGroup: rbac.authorization.k8s.io
```
If Joe inspects the namespace, he will see something like this:
```yaml
# kubectl get ns oil-production -o yaml
apiVersion: v1
kind: Namespace
metadata:
annotations:
capsule.k8s/owner: joe
scheduler.alpha.kubernetes.io/node-selector: node-role.kubernetes.io/capsule=caas
creationTimestamp: "2020-05-27T13:49:30Z"
labels:
tenant: oil-and-stracci-inc
name: oil-production
resourceVersion: "1651593"
selfLink: /api/v1/namespaces/oil-production
uid: e3b2efd4-a020-11ea-bba9-566fc1cb01af
spec:
finalizers:
- kubernetes
status:
phase: Active
```
Joe is the admin of the namespace:
```
# kubectl get rolebindings -n oil-production
NAME ROLE AGE
namespace:admin ClusterRole/admin 9m5s
namespace:deleter ClusterRole/admin 9m5s
```
The said Role Binding resources are automatically created by the Capsule
controller when Joe creates a namespace in his tenant.
Joe can deploy any resource in his namespace, according to the predefined
[`admin` cluster role](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles).
Also, Joe can delete the namespace
```
# kubectl auth can-i delete ns -n oil-production
Warning: resource 'namespaces' is not namespace scoped
yes
```
or he can create additional namespaces, according to the `namespaceQuota` field of the tenant manifest:
```
# kubectl create ns oil-development
# kubectl create ns oil-test
```
The enforcement on the maximum number of Namespace resources per Tenant is in
charge of the Capsule controller via a Dynamic Admission Webhook created and
managed by the Capsule controller.
While Joe creates Namespace resources, the Capsule controller updates the
status of the tenant as following:
```yaml
...
status:
size: 3 # namespace count
namespaces:
- oil-production
- oil-development
- oil-test
...
```
### Quota enforcement for a tenant
When Joe creates the namespace `oil-production`, the Capsule controller creates
a set of namespaced objects, according to the Tenant's manifest.
For example, there are three resource quotas
```yaml
kind: ResourceQuota
apiVersion: v1
metadata:
name: compute
namespace: oil-production
labels:
tenant: oil-and-stracci-inc
spec:
hard:
limits.cpu: "8"
limits.memory: 16Gi
requests.cpu: "8"
requests.memory: 16Gi
scopes: ["NotTerminating"]
---
kind: ResourceQuota
apiVersion: v1
metadata:
name: count
namespace: oil-production
labels:
tenant: oil-and-stracci-inc
spec:
hard:
pods : "10"
services: "5"
---
kind: ResourceQuota
apiVersion: v1
metadata:
name: storage
namespace: oil-production
labels:
tenant: oil-and-stracci-inc
spec:
hard:
requests.storage: "10Gi"
```
and a Limit Range:
```yaml
kind: LimitRange
apiVersion: v1
metadata:
name: limits
namespace: oil-production
labels:
tenant: oil-and-stracci-inc
spec:
limits:
- type: Pod
min:
cpu: "50m"
memory: "5Mi"
max:
cpu: "1"
memory: "1Gi"
- type: Container
defaultRequest:
cpu: "100m"
memory: "10Mi"
default:
cpu: "200m"
memory: "100Mi"
min:
cpu: "50m"
memory: "5Mi"
max:
cpu: "1"
memory: "1Gi"
- type: PersistentVolumeClaim
min:
storage: "1Gi"
max:
storage: "10Gi"
```
In their Namespace, Joe can create any resource according to the assigned
Resource Quota:
```
# kubectl -n oil-production create deployment nginx --image=nginx:latest
```
To check the remaining quota in the `oil-production` namesapce, he can get the list of resource quotas:
```
# kubectl -n oil-production get resourcequota
NAME AGE REQUEST LIMIT
capsule-oil-0 42h requests.cpu: 1/8, requests.memory: 1/16Gi limits.cpu: 1/8, limits.memory: 1/16Gi
capsule-oil-1 42h pods: 2/10
capsule-oil-2 42h requests.storage: 0/100Gi
```
and inspecting the Quota annotations:
```yaml
# kubectl get resourcequotas capsule-oil-1 -o yaml
apiVersion: v1
kind: ResourceQuota
metadata:
annotations:
quota.capsule.clastix.io/used-pods: "0"
...
```
> Nota Bene:
> at Namespace level, the quota enforcement is under the control of the default
> _ResourceQuota Admission Controller_ enabled on the Kubernetes API server
> using the flag `--enable-admission-plugins=ResourceQuota`.
At tenant level, the Capsule operator watches the Resource Quota usage for each
Tenant's Namespace and adjusts it as an aggregate of all the namespaces using
the said annotation pattern (`quota.capsule.clastix.io/<quota_name>`)
The used Resource Quota counts all the used resources as aggregate of all the
Namespace resources in the `oil-and-stracci-inc` Tenant namespaces:
- `oil-production`
- `oil-development`
- `oil-test`
When the aggregate usage reaches the hard quota limits,
then the ResourceQuota Admission Controller denies the Joe's request.
> In addition to Resource Quota, the Capsule controller create limits ranges in
> each namespace according to the tenant manifest.
>
> Limit ranges enforcement for single pod, container, and persistent volume
> claim is done by the default _LimitRanger Admission Controller_ enabled on
> the Kubernetes API server: using the flag
> `--enable-admission-plugins=LimitRanger`.
Joe can inspect Limit Ranges for his namespaces:
```
# kubectl -n oil-production get limitranges
NAME CREATED AT
capsule-oil-0 2020-07-20T18:41:15Z
# kubectl -n oil-production describe limitranges limits
Name: capsule-oil-0
Namespace: oil-production
Type Resource Min Max Default Request Default Limit Max Limit/Request Ratio
---- -------- --- --- --------------- ------------- -----------------------
Pod cpu 50m 1 - - -
Pod memory 5Mi 1Gi - - -
Container cpu 50m 1 100m 200m -
Container memory 5Mi 1Gi 10Mi 100Mi -
PersistentVolumeClaim storage 1Gi 10Gi - - -
```
Being the limit range specific of single resources:
- Pod
- Container
- Persistent Volume Claim
there is no aggregate to count.
Having access to resource quota and limits, however Joe is not able to change
or delete it according to his RBAC profile.
```
# kubectl -n oil-production auth can-i patch resourcequota
no - no RBAC policy matched
# kubectl -n oil-production auth can-i patch limitranges
no - no RBAC policy matched
```
### Node selector for a Tenant
A Tenant assigned to a shared CaaS tenant, shares infra and worker nodes with
all the other shared CaaS tenants.
Bill, the cluster admin of the CaaS, dedicated a set of infra and worker nodes
to shared CaaS tenants.
These nodes have been previously labeled as `node-role.kubernetes.io/capsule=caas`
to be separated from nodes dedicated to private CaaS users
```
$ kubectl get nodes --show-labels
NAME STATUS ROLES AGE VERSION LABELS
master01.acme.com Ready master 8d v1.18.2 node-role.kubernetes.io/capsule=caas
master02.acme.com Ready master 8d v1.18.2 node-role.kubernetes.io/capsule=caas
master03.acme.com Ready master 8d v1.18.2 node-role.kubernetes.io/capsule=caas
infra01.acme.com Ready infra 8d v1.18.2 node-role.kubernetes.io/capsule=caas
infra02.acme.com Ready infra 8d v1.18.2 node-role.kubernetes.io/capsule=caas
infra03.acme.com Ready infra 8d v1.18.2 node-role.kubernetes.io/capsule=qos
infra04.acme.com Ready infra 8d v1.18.2 node-role.kubernetes.io/capsule=qos
infra05.acme.com Ready infra 8d v1.18.2 node-role.kubernetes.io/capsule=qos
infra06.acme.com Ready infra 8d v1.18.2 node-role.kubernetes.io/capsule=qos
storage01.acme.com Ready storage 8d v1.18.2 node-role.kubernetes.io/capsule=caas
storage02.acme.com Ready storage 8d v1.18.2 node-role.kubernetes.io/capsule=caas
storage03.acme.com Ready storage 8d v1.18.2 node-role.kubernetes.io/capsule=qos
storage04.acme.com Ready storage 8d v1.18.2 node-role.kubernetes.io/capsule=qos
storage05.acme.com Ready storage 8d v1.18.2 node-role.kubernetes.io/capsule=qos
storage06.acme.com Ready storage 8d v1.18.2 node-role.kubernetes.io/capsule=qos
worker01.acme.com Ready worker 8d v1.18.2 node-role.kubernetes.io/capsule=caas
worker02.acme.com Ready worker 8d v1.18.2 node-role.kubernetes.io/capsule=caas
worker03.acme.com Ready worker 8d v1.18.2 node-role.kubernetes.io/capsule=caas
worker04.acme.com Ready worker 8d v1.18.2 node-role.kubernetes.io/capsule=caas
worker05.acme.com Ready worker 8d v1.18.2 node-role.kubernetes.io/capsule=qos
worker06.acme.com Ready worker 8d v1.18.2 node-role.kubernetes.io/capsule=qos
worker07.acme.com Ready worker 8d v1.18.2 node-role.kubernetes.io/capsule=qos
worker08.acme.com Ready worker 8d v1.18.2 node-role.kubernetes.io/capsule=qos
```
Bill should assure that all workload deployed by a shared CaaS users are
assigned to worker nodes labeled as `node-role.kubernetes.io/capsule=caas`.
On the Kubernetes API servers of the CaaS platform, Bill must enable the
`--enable-admission-plugins=PodNodeSelector` Admission Controller plugin.
This forces the CaaS platform to assign a dedicated selector to all pods
created in any namespace of the Tenant.
To help Bill, the Capsule controller must assure that any namespace created in
the tenant has the annotation:
`scheduler.alpha.kubernetes.io/node-selector: node-role.kubernetes.io/capsule=caas`.
The Capsule controller must force the annotation above for each namespace
created by any shared CaaS user.
For example, in the `oil-and-stracci-inc` tenant,
all pods deployed by Joe will have the selector
```yaml
...
nodeSelector:
node-role.kubernetes.io/capsule: caas
...
```
Any temptative to change the selector, will result in the following error from
the `PodNodeSelector` Admission Controller plugin:
```
Error from server (Forbidden): error when creating "podshell.yaml": pods "busybox" is forbidden:
pod node label selector conflicts with its namespace node label selector
```
and no additional actions are required to the Capsule controller.
On the other side, a private CaaS tenant receives a dedicated set of infra e
worker nodes. Bill has to make sure that these nodes are labeled according,
for example `node-role.kubernetes.io/capsule=qos` to be separated from nodes
dedicated to other private CaaS tenants and the shared CaaS tenants.
The Capsule controller must assure that any namespace created in the tenant has
the annotation:
`scheduler.alpha.kubernetes.io/node-selector: node-role.kubernetes.io/capsule=qos`.
The Capsule controller must force the annotation above for each namespace created by any private CaaS user.
For example, in the `evil-corp` tenant, all pods deployed by Alice will have
the selector
```yaml
...
nodeSelector:
node-role.kubernetes.io/capsule: evil-corp
...
```
Any temptative to change the selector, will be denied byt the `PodNodeSelector`
Admission Controller plugin no additional actions are required to the
Capsule controller.
### Ingress selector for a tenant
A tenant assigned to a shared CaaS tenant shares the infra nodes with all the
other shared CaaS tenants. On these infra nodes, a single Ingress Controller is
installed and provisioned with a wildcard certificate.
All the applications within the tenant will be published as `*.caas.acme.com`
Bill provisioned an Ingress Controller on the shared CaaS to use a dedicated
ingress class: `--ingress-class=caas` as ingress selector.
All ingresses created in all the shared CaaS tenants must use this selector in
order to be published on the CaaS Ingress Controller.
The Capsule operator must assure that all ingresses created in any tenant
belonging to the shared CaaS, have the annotation
`kubernetes.io/ingress.class: caas` where the selector is specified in the
tenant resouce manifest:
```yaml
apiVersion: capsule.clastix.io/v1alpha1
kind: Tenant
metadata:
labels:
annotations:
name: oil-and-stracci-inc
spec:
...
ingressClass: caas
...
```
For example, in the `oil-production` namespace belonging to the
`oil-and-stracci-inc` tenant, Joe will see:
```yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
namespace: oil-production
name: wordpress
annotations:
kubernetes.io/ingress.class: caas
spec:
rules:
- host: blog.caas.acme.com
http:
paths:
- path: /
backend:
serviceName: wordpress
servicePort: 80
```
Joe can create, change and delete `Ingress` resources, but the Capsule
controller will always force any change to the ingress selector annotation to be
`kubernetes.io/ingress.class: caas`.
On the other side, a private CaaS tenant receives a dedicated Ingress Controller
running on the infra nodes dedicated to that tenant only.
Bill provisions the dedicated Ingress Controller to use a dedicated ingress
class: `--ingress-class=evil-corp` as ingress selector and a dedicated wildcard
certificate, for example `*.evilcorp.com`. All ingresses created in the private
tenant must use this selector in order to be published on the dedicated Ingress
Controller.
The Capsule operator must assure that all ingresses created in the tenant,
have the annotation `kubernetes.io/ingress.class: evil-corp` where the selector
is specified into the tenant resouce manifest.
### Network policies for a tenant
Kubernetes network policies allow to control network traffic between namespaces
and between pods in the same namespace. The CaaS platform must enforce network
traffic isolation between different tenants while leaving to the tenant user
the freedom to set isolation between namespaces in the same tenant or even
between pods in the same namespace.
To meet this requirement, Bill, the CaaS platform administrator, needs to
define network policies that deny pods belonging to a tenant namespace to
access pods in namespaces belonging to other tenants or in system namespaces,
(e.g. `kube-system`).
Also Bill must assure that pods belonging to a tenant namespace cannot access
other network infrastructure like cluster nodes, load balancers, and virtual
machines running other services.
Bill can specify network policies in the tenant manifest,
according to the CaaS platform requirements:
```yaml
...
networkPolicies:
- policyTypes:
- Ingress
- Egress
podSelector: {}
ingress:
- from:
- namespaceSelector:
matchLabels:
tenant: oil-and-stracci-inc
- podSelector: {}
- ipBlock:
cidr: 192.168.0.0/16
egress:
- to:
- ipBlock:
cidr: 0.0.0.0/0
except:
- 192.168.0.0/16
```
The Capsule controller, watching for Namespace creation,
creates the Network Policies for each Namespace in the tenant.
The tenat user (e.g. Joe) has access these network policies:
```
# kubectl -n oil-production get networkpolicies
NAME POD-SELECTOR AGE
capsule-oil-0 <none> 42h
# kubectl -n oil-production describe networkpolicy
Name: capsule-oil-0
Namespace: oil-production
Created on: 2020-07-20 20:40:28 +0200 CEST
Labels: capsule.clastix.io/network-policy=0
capsule.clastix.io/tenant=oil
Annotations: <none>
Spec:
PodSelector: <none> (Allowing the specific traffic to all pods in this namespace)
Allowing ingress traffic:
To Port: <any> (traffic allowed to all ports)
From:
NamespaceSelector: capsule.clastix.io/tenant=oil
From:
PodSelector: <none>
From:
IPBlock:
CIDR: 192.168.0.0/12
Except:
Allowing egress traffic:
To Port: <any> (traffic allowed to all ports)
To:
IPBlock:
CIDR: 0.0.0.0/0
Except: 192.168.0.0/12
Policy Types: Ingress, Egress
```
and he can create, patch, and delete Nework Policies
```
# kubectl -n oil-production auth can-i get networkpolicies
yes
# kubectl -n oil-production auth can-i delete networkpolicies
yes
# kubectl -n oil-production auth can-i patch networkpolicies
yes
```
However, the Caspule controller enforces the Tenant Network Policie resources
above and prevents Joe to change, or delete them.
### Storage Class for a tenant
The CaaS platform provides persistent storage infrastructure for shared and
private tenants. Different type of storage requirements, with different level
of QoS, eg. SSD versus HDD, can be provided by the platform according to the
tenants profile and needs. To meet these dirrerent requirements, Bill, the
admin of the CaaS platform, has to provision different storage classes and
assign a proper storage class to the tenants, by specifing it into the tenant
manifest:
```yaml
apiVersion: capsule.clastix.io/v1alpha1
kind: Tenant
metadata:
labels:
annotations:
name: oil-and-stracci-inc
spec:
storageClasses:
- ceph-rbd
...
```
The Capsule controller will ensure that all Persistent Volume Claims created in
a Tenant will use one of the available storage classes (`ceph-rbd`,
in this case).
For example:
```yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: pvc
namespace:
spec:
storageClassName: denied
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 12Gi
```
The creation of the said PVC will fail as following:
```
# kubectl apply -f my_pvc.yaml
Error from server: error when creating "/tmp/pvc.yaml":
admission webhook "pvc.capsule.clastix.io" denied the request:
Storage Class ceph-rbd is forbidden for the current Tenant
```

18
version/version.go Normal file
View File

@@ -0,0 +1,18 @@
/*
Copyright 2020 Clastix Labs.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package version
var (
Version = "0.0.1"
)