Compare commits

...

16 Commits

Author SHA1 Message Date
dependabot[bot]
faf26b2254 feat(deps): bump github.com/onsi/ginkgo/v2 from 2.27.4 to 2.27.5 (#1060)
Bumps [github.com/onsi/ginkgo/v2](https://github.com/onsi/ginkgo) from 2.27.4 to 2.27.5.
- [Release notes](https://github.com/onsi/ginkgo/releases)
- [Changelog](https://github.com/onsi/ginkgo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/ginkgo/compare/v2.27.4...v2.27.5)

---
updated-dependencies:
- dependency-name: github.com/onsi/ginkgo/v2
  dependency-version: 2.27.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-14 18:39:07 +01:00
Parth Yadav
87242ff005 feat: extend Gateway API support to Konnectivity addons (#1054)
This change extends Gateway API support to Konnectivity addons.
When `spec.controlPlane.gateway` is configured and Konnectivity addon is
enabled, Kamaji automatically creates two TLSRoutes:
1. A Control plane TLSRoute (port 6443, sectionName "kube-apiserver")
2. A Konnectivity TLSRoute (port 8132, sectionName "konnectivity-server")

Both routes use the hostname specified in `gateway.hostname` and reference
the same Gateway resource via `parentRefs`, with `port` and `sectionName`
set automatically by Kamaji.

This patch also adds CEL validation to prevent users from specifying
`port` or `sectionName` in Gateway `parentRefs`, as these fields are now
managed automatically by Kamaji.

Signed-off-by: Parth Yadav <parth@coredge.io>
2026-01-11 11:31:24 +01:00
dependabot[bot]
b6b4888177 feat(deps): bump github.com/onsi/gomega from 1.38.3 to 1.39.0 (#1056)
Bumps [github.com/onsi/gomega](https://github.com/onsi/gomega) from 1.38.3 to 1.39.0.
- [Release notes](https://github.com/onsi/gomega/releases)
- [Changelog](https://github.com/onsi/gomega/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/gomega/compare/v1.38.3...v1.39.0)

---
updated-dependencies:
- dependency-name: github.com/onsi/gomega
  dependency-version: 1.39.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-11 11:27:38 +01:00
dependabot[bot]
bd0c7d354d feat(deps): bump github.com/onsi/ginkgo/v2 from 2.27.3 to 2.27.4 (#1055)
Bumps [github.com/onsi/ginkgo/v2](https://github.com/onsi/ginkgo) from 2.27.3 to 2.27.4.
- [Release notes](https://github.com/onsi/ginkgo/releases)
- [Changelog](https://github.com/onsi/ginkgo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/ginkgo/compare/v2.27.3...v2.27.4)

---
updated-dependencies:
- dependency-name: github.com/onsi/ginkgo/v2
  dependency-version: 2.27.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-11 11:22:33 +01:00
Dario Tranchitella
e1c6aa8459 feat: kubelet configuration json patching (#1052)
* feat: kubelet configuration json patching

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

* chore(helm): kubelet configuration json patching

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

* docs(api): kubelet configuration json patching

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

* chore(samples): kubelet configuration json patching

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

---------

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>
2026-01-07 12:36:52 +01:00
Michał Matuszak
11c315289c feat: Make replicas optional for Konnectivity Agent mode Deployment (#1041) 2026-01-04 19:26:57 +01:00
Syed Azeez
0428024946 fix(metrics): resolve workqueue metrics initialization conflict (#1044)
Signed-off-by: Azeez Syed <syedazeez337@gmail.com>
2026-01-04 19:24:43 +01:00
Syed Azeez
f55df56eac fix(soot): add unique controller names to prevent metric conflicts (#1043)
Signed-off-by: Azeez Syed <syedazeez337@gmail.com>
2026-01-04 19:24:10 +01:00
daseul cho
88e08fa0ec fix(soot): correct TenantControlPlane name in trigger events (#1040) 2025-12-22 07:03:17 +01:00
Dario Tranchitella
01e07ab411 feat: kubernetes 1.35 support (#1038)
* feat: supporting k8s v1.35

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

* feat: upgrading deployment also in sleeping mode

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

* feat(deps): bumping ko to v0.18.1

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

* feat(deps): bumping controller-gen to v0.20.0

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

* chore(crds): aligning to k8s v1.35

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

* docs: alinging to k8s v1.35

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

* test: upgrading to k8s 1.35

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

* feat(helm): updating artifact hub changes

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

---------

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>
2025-12-20 12:06:48 +01:00
dependabot[bot]
e0d6865df3 feat(deps): bump github.com/nats-io/nats.go from 1.47.0 to 1.48.0 (#1036)
Bumps [github.com/nats-io/nats.go](https://github.com/nats-io/nats.go) from 1.47.0 to 1.48.0.
- [Release notes](https://github.com/nats-io/nats.go/releases)
- [Commits](https://github.com/nats-io/nats.go/compare/v1.47.0...v1.48.0)

---
updated-dependencies:
- dependency-name: github.com/nats-io/nats.go
  dependency-version: 1.48.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-19 16:42:21 +01:00
dependabot[bot]
57e3e12f09 feat(deps): bump the etcd group with 2 updates (#1035)
Bumps the etcd group with 2 updates: [go.etcd.io/etcd/api/v3](https://github.com/etcd-io/etcd) and [go.etcd.io/etcd/client/v3](https://github.com/etcd-io/etcd).


Updates `go.etcd.io/etcd/api/v3` from 3.6.6 to 3.6.7
- [Release notes](https://github.com/etcd-io/etcd/releases)
- [Commits](https://github.com/etcd-io/etcd/compare/v3.6.6...v3.6.7)

Updates `go.etcd.io/etcd/client/v3` from 3.6.6 to 3.6.7
- [Release notes](https://github.com/etcd-io/etcd/releases)
- [Commits](https://github.com/etcd-io/etcd/compare/v3.6.6...v3.6.7)

---
updated-dependencies:
- dependency-name: go.etcd.io/etcd/api/v3
  dependency-version: 3.6.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: etcd
- dependency-name: go.etcd.io/etcd/client/v3
  dependency-version: 3.6.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: etcd
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-18 07:08:07 +01:00
Léonard Suslian
d3fb03a752 feat: add support for multiple Datastores (#961)
* feat: add support for multiple Datastores

* docs: add guide for datastore overrides

* feat(datastore): add e2e test for dataStoreOverrides

* ci: reclaim disk space from runner to fix flaky tests
2025-12-12 12:10:02 +01:00
dependabot[bot]
eb86fec050 feat(deps): bump github.com/onsi/gomega from 1.38.2 to 1.38.3 (#1032)
Bumps [github.com/onsi/gomega](https://github.com/onsi/gomega) from 1.38.2 to 1.38.3.
- [Release notes](https://github.com/onsi/gomega/releases)
- [Changelog](https://github.com/onsi/gomega/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/gomega/compare/v1.38.2...v1.38.3)

---
updated-dependencies:
- dependency-name: github.com/onsi/gomega
  dependency-version: 1.38.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-12 12:03:28 +01:00
dependabot[bot]
35c83fbd4d feat(deps): bump k8s.io/kubernetes in the k8s group (#1034)
Bumps the k8s group with 1 update: [k8s.io/kubernetes](https://github.com/kubernetes/kubernetes).


Updates `k8s.io/kubernetes` from 1.34.2 to 1.34.3
- [Release notes](https://github.com/kubernetes/kubernetes/releases)
- [Commits](https://github.com/kubernetes/kubernetes/compare/v1.34.2...v1.34.3)

---
updated-dependencies:
- dependency-name: k8s.io/kubernetes
  dependency-version: 1.34.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: k8s
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-12 12:03:16 +01:00
dependabot[bot]
4ad4721965 feat(deps): bump github.com/onsi/ginkgo/v2 from 2.27.2 to 2.27.3 (#1033)
Bumps [github.com/onsi/ginkgo/v2](https://github.com/onsi/ginkgo) from 2.27.2 to 2.27.3.
- [Release notes](https://github.com/onsi/ginkgo/releases)
- [Changelog](https://github.com/onsi/ginkgo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/ginkgo/compare/v2.27.2...v2.27.3)

---
updated-dependencies:
- dependency-name: github.com/onsi/ginkgo/v2
  dependency-version: 2.27.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-09 10:04:49 +01:00
59 changed files with 4303 additions and 456 deletions

View File

@@ -41,6 +41,9 @@ jobs:
- uses: actions/setup-go@v6
with:
go-version-file: go.mod
- name: reclaim disk space from runner
run: |
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /opt/hostedtoolcache/CodeQL
- run: |
sudo apt-get update
sudo apt-get install -y golang-cfssl

View File

@@ -73,7 +73,7 @@ help: ## Display this help.
.PHONY: ko
ko: $(KO) ## Download ko locally if necessary.
$(KO): $(LOCALBIN)
test -s $(LOCALBIN)/ko || GOBIN=$(LOCALBIN) CGO_ENABLED=0 go install -ldflags="-s -w" github.com/google/ko@v0.14.1
test -s $(LOCALBIN)/ko || GOBIN=$(LOCALBIN) CGO_ENABLED=0 go install -ldflags="-s -w" github.com/google/ko@v0.18.1
.PHONY: yq
yq: $(YQ) ## Download yq locally if necessary.
@@ -98,7 +98,7 @@ $(KIND): $(LOCALBIN)
.PHONY: controller-gen
controller-gen: $(CONTROLLER_GEN) ## Download controller-gen locally if necessary.
$(CONTROLLER_GEN): $(LOCALBIN)
test -s $(LOCALBIN)/controller-gen || GOBIN=$(LOCALBIN) CGO_ENABLED=0 go install -ldflags="-s -w" sigs.k8s.io/controller-tools/cmd/controller-gen@v0.16.1
test -s $(LOCALBIN)/controller-gen || GOBIN=$(LOCALBIN) CGO_ENABLED=0 go install -ldflags="-s -w" sigs.k8s.io/controller-tools/cmd/controller-gen@v0.20.0
.PHONY: golangci-lint
golangci-lint: $(GOLANGCI_LINT) ## Download golangci-lint locally if necessary.
@@ -177,7 +177,7 @@ datastore-postgres:
$(MAKE) NAME=gold _datastore-postgres
_datastore-etcd:
$(HELM) upgrade --install etcd-$(NAME) clastix/kamaji-etcd --create-namespace -n etcd-system --set datastore.enabled=true --set fullnameOverride=etcd-$(NAME)
$(HELM) upgrade --install etcd-$(NAME) clastix/kamaji-etcd --create-namespace -n $(NAMESPACE) --set datastore.enabled=true --set fullnameOverride=etcd-$(NAME) $(EXTRA_ARGS)
_datastore-nats:
$(MAKE) NAME=$(NAME) NAMESPACE=nats-system -C deploy/kine/nats nats
@@ -186,9 +186,11 @@ _datastore-nats:
datastore-etcd: helm
$(HELM) repo add clastix https://clastix.github.io/charts
$(HELM) repo update
$(MAKE) NAME=bronze _datastore-etcd
$(MAKE) NAME=silver _datastore-etcd
$(MAKE) NAME=gold _datastore-etcd
$(MAKE) NAME=bronze NAMESPACE=etcd-system _datastore-etcd
$(MAKE) NAME=silver NAMESPACE=etcd-system _datastore-etcd
$(MAKE) NAME=gold NAMESPACE=etcd-system _datastore-etcd
$(MAKE) NAME=primary NAMESPACE=kamaji-system EXTRA_ARGS='--set certManager.enabled=true --set certManager.issuerRef.kind=Issuer --set certManager.issuerRef.name=kamaji-selfsigned-issuer --set selfSignedCertificates.enabled=false' _datastore-etcd
$(MAKE) NAME=secondary NAMESPACE=kamaji-system EXTRA_ARGS='--set certManager.enabled=true --set certManager.ca.create=false --set certManager.ca.nameOverride=etcd-primary-ca --set certManager.issuerRef.kind=Issuer --set certManager.issuerRef.name=kamaji-selfsigned-issuer --set selfSignedCertificates.enabled=false' _datastore-etcd
datastore-nats: helm
$(HELM) repo add nats https://nats-io.github.io/k8s/helm/charts/
@@ -246,6 +248,10 @@ gateway-api:
kubectl apply --server-side -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.4.0/experimental-install.yaml
kubectl wait --for=condition=Established crd/gateways.gateway.networking.k8s.io --timeout=60s
envoy-gateway: gateway-api helm ## Install Envoy Gateway for Gateway API tests.
$(HELM) upgrade --install eg oci://docker.io/envoyproxy/gateway-helm --version v1.6.1 -n envoy-gateway-system --create-namespace
kubectl wait --timeout=5m -n envoy-gateway-system deployment/envoy-gateway --for=condition=Available
load: kind
$(KIND) load docker-image --name kamaji ${CONTAINER_REPOSITORY}:${VERSION}
@@ -259,7 +265,7 @@ cleanup: kind
$(KIND) delete cluster --name kamaji
.PHONY: e2e
e2e: env build load helm ginkgo cert-manager gateway-api ## Create a KinD cluster, install Kamaji on it and run the test suite.
e2e: env build load helm ginkgo cert-manager gateway-api envoy-gateway ## Create a KinD cluster, install Kamaji on it and run the test suite.
$(HELM) upgrade --debug --install kamaji-crds ./charts/kamaji-crds --create-namespace --namespace kamaji-system
$(HELM) repo add clastix https://clastix.github.io/charts
$(HELM) dependency build ./charts/kamaji

View File

@@ -0,0 +1,83 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package v1alpha1
import (
apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
)
type JSONPatches []JSONPatch
type JSONPatch struct {
// Op is the RFC 6902 JSON Patch operation.
//+kubebuilder:validation:Enum=add;remove;replace;move;copy;test
Op string `json:"op"`
// Path specifies the target location in the JSON document. Use "/" to separate keys; "-" for appending to arrays.
Path string `json:"path"`
// From specifies the source location for move or copy operations.
From string `json:"from,omitempty"`
// Value is the operation value to be used when Op is add, replace, test.
Value *apiextensionsv1.JSON `json:"value,omitempty"`
}
func (p JSONPatches) ToJSON() ([]byte, error) {
if len(p) == 0 {
return []byte("[]"), nil
}
buf := make([]byte, 0, 256)
buf = append(buf, '[')
for i, patch := range p {
if i > 0 {
buf = append(buf, ',')
}
buf = append(buf, '{')
buf = append(buf, `"op":"`...)
buf = appendEscapedString(buf, patch.Op)
buf = append(buf, '"')
buf = append(buf, `,"path":"`...)
buf = appendEscapedString(buf, patch.Path)
buf = append(buf, '"')
if patch.From != "" {
buf = append(buf, `,"from":"`...)
buf = appendEscapedString(buf, patch.From)
buf = append(buf, '"')
}
if patch.Value != nil {
buf = append(buf, `,"value":`...)
buf = append(buf, patch.Value.Raw...)
}
buf = append(buf, '}')
}
buf = append(buf, ']')
return buf, nil
}
func appendEscapedString(dst []byte, s string) []byte {
for i := range s {
switch s[i] {
case '\\', '"':
dst = append(dst, '\\', s[i])
case '\n':
dst = append(dst, '\\', 'n')
case '\r':
dst = append(dst, '\\', 'r')
case '\t':
dst = append(dst, '\\', 't')
default:
dst = append(dst, s[i])
}
}
return dst
}

View File

@@ -139,6 +139,7 @@ type KonnectivityStatus struct {
ClusterRoleBinding ExternalKubernetesObjectStatus `json:"clusterrolebinding,omitempty"`
Agent KonnectivityAgentStatus `json:"agent,omitempty"`
Service KubernetesServiceStatus `json:"service,omitempty"`
Gateway *KubernetesGatewayStatus `json:"gateway,omitempty"`
}
type KonnectivityConfigMap struct {

View File

@@ -68,6 +68,12 @@ const (
)
type KubeletSpec struct {
// ConfigurationJSONPatches contains the RFC 6902 JSON patches to customise the kubeadm generate configuration,
// useful to customise and mangling the configuration according to your needs;
// e.g.: configuring the cgroup driver used by Kubelet is possible via the following patch:
//
// [{"op": "replace", "path": "/cgroupDriver", "value": "systemd"}]
ConfigurationJSONPatches JSONPatches `json:"configurationJSONPatches,omitempty"`
// Ordered list of the preferred NodeAddressTypes to use for kubelet connections.
// Default to InternalIP, ExternalIP, Hostname.
//+kubebuilder:default={"InternalIP","ExternalIP","Hostname"}
@@ -76,6 +82,8 @@ type KubeletSpec struct {
PreferredAddressTypes []KubeletPreferredAddressType `json:"preferredAddressTypes,omitempty"`
// CGroupFS defines the cgroup driver for Kubelet
// https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/configure-cgroup-driver/
//
// Deprecated: use ConfigurationJSONPatches.
CGroupFS CGroupDriver `json:"cgroupfs,omitempty"`
}
@@ -147,6 +155,7 @@ type IngressSpec struct {
}
// GatewaySpec defines the options for the Gateway which will expose API Server of the Tenant Control Plane.
// +kubebuilder:validation:XValidation:rule="!has(self.parentRefs) || size(self.parentRefs) == 0 || self.parentRefs.all(ref, !has(ref.port) && !has(ref.sectionName))",message="parentRefs must not specify port or sectionName, these are set automatically by Kamaji"
type GatewaySpec struct {
// AdditionalMetadata to add Labels and Annotations support.
AdditionalMetadata AdditionalMetadata `json:"additionalMetadata,omitempty"`
@@ -289,7 +298,7 @@ var (
KonnectivityAgentModeDeployment KonnectivityAgentMode = "Deployment"
)
//+kubebuilder:validation:XValidation:rule="!(self.mode == 'DaemonSet' && has(self.replicas) && self.replicas != 0) && !(self.mode == 'Deployment' && self.replicas == 0)",message="replicas must be 0 when mode is DaemonSet, and greater than 0 when mode is Deployment"
//+kubebuilder:validation:XValidation:rule="!(self.mode == 'DaemonSet' && has(self.replicas) && self.replicas != 0) && !(self.mode == 'Deployment' && has(self.replicas) && self.replicas == 0)",message="replicas must be 0 (or unset) when mode is DaemonSet, and greater than 0 (or unset) when mode is Deployment"
type KonnectivityAgentSpec struct {
// AgentImage defines the container image for Konnectivity's agent.
@@ -318,7 +327,7 @@ type KonnectivityAgentSpec struct {
// Replicas defines the number of replicas when Mode is Deployment.
// Must be 0 if Mode is DaemonSet.
//+kubebuilder:validation:Optional
Replicas int32 `json:"replicas,omitempty"`
Replicas *int32 `json:"replicas,omitempty"`
}
// KonnectivitySpec defines the spec for Konnectivity.
@@ -355,6 +364,14 @@ func (p *Permissions) HasAnyLimitation() bool {
return false
}
// DataStoreOverride defines which kubernetes resource will be stored in a dedicated datastore.
type DataStoreOverride struct {
// Resource specifies which kubernetes resource to target.
Resource string `json:"resource,omitempty"`
// DataStore specifies the DataStore that should be used to store the Kubernetes data for the given Resource.
DataStore string `json:"dataStore,omitempty"`
}
// TenantControlPlaneSpec defines the desired state of TenantControlPlane.
// +kubebuilder:validation:XValidation:rule="!has(oldSelf.dataStore) || has(self.dataStore)", message="unsetting the dataStore is not supported"
// +kubebuilder:validation:XValidation:rule="!has(oldSelf.dataStoreSchema) || has(self.dataStoreSchema)", message="unsetting the dataStoreSchema is not supported"
@@ -389,8 +406,10 @@ type TenantControlPlaneSpec struct {
// to the user to avoid clashes between different TenantControlPlanes. If not set upon creation, Kamaji will default the
// DataStoreUsername by concatenating the namespace and name of the TenantControlPlane.
// +kubebuilder:validation:XValidation:rule="self == oldSelf",message="changing the dataStoreUsername is not supported"
DataStoreUsername string `json:"dataStoreUsername,omitempty"`
ControlPlane ControlPlane `json:"controlPlane"`
DataStoreUsername string `json:"dataStoreUsername,omitempty"`
// DataStoreOverride defines which kubernetes resources will be stored in dedicated datastores.
DataStoreOverrides []DataStoreOverride `json:"dataStoreOverrides,omitempty"`
ControlPlane ControlPlane `json:"controlPlane"`
// Kubernetes specification for tenant control plane
Kubernetes KubernetesSpec `json:"kubernetes"`
// NetworkProfile specifies how the network is

View File

@@ -9,8 +9,9 @@ package v1alpha1
import (
corev1 "k8s.io/api/core/v1"
"k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
runtime "k8s.io/apimachinery/pkg/runtime"
"sigs.k8s.io/gateway-api/apis/v1"
apisv1 "sigs.k8s.io/gateway-api/apis/v1"
)
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
@@ -538,6 +539,21 @@ func (in *DataStoreList) DeepCopyObject() runtime.Object {
return nil
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *DataStoreOverride) DeepCopyInto(out *DataStoreOverride) {
*out = *in
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataStoreOverride.
func (in *DataStoreOverride) DeepCopy() *DataStoreOverride {
if in == nil {
return nil
}
out := new(DataStoreOverride)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *DataStoreSetupStatus) DeepCopyInto(out *DataStoreSetupStatus) {
*out = *in
@@ -797,7 +813,7 @@ func (in *GatewayAccessPoint) DeepCopyInto(out *GatewayAccessPoint) {
*out = *in
if in.Type != nil {
in, out := &in.Type, &out.Type
*out = new(v1.AddressType)
*out = new(apisv1.AddressType)
**out = **in
}
if in.URLs != nil {
@@ -838,7 +854,7 @@ func (in *GatewaySpec) DeepCopyInto(out *GatewaySpec) {
in.AdditionalMetadata.DeepCopyInto(&out.AdditionalMetadata)
if in.GatewayParentRefs != nil {
in, out := &in.GatewayParentRefs, &out.GatewayParentRefs
*out = make([]v1.ParentReference, len(*in))
*out = make([]apisv1.ParentReference, len(*in))
for i := range *in {
(*in)[i].DeepCopyInto(&(*out)[i])
}
@@ -886,6 +902,47 @@ func (in *IngressSpec) DeepCopy() *IngressSpec {
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *JSONPatch) DeepCopyInto(out *JSONPatch) {
*out = *in
if in.Value != nil {
in, out := &in.Value, &out.Value
*out = new(v1.JSON)
(*in).DeepCopyInto(*out)
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new JSONPatch.
func (in *JSONPatch) DeepCopy() *JSONPatch {
if in == nil {
return nil
}
out := new(JSONPatch)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in JSONPatches) DeepCopyInto(out *JSONPatches) {
{
in := &in
*out = make(JSONPatches, len(*in))
for i := range *in {
(*in)[i].DeepCopyInto(&(*out)[i])
}
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new JSONPatches.
func (in JSONPatches) DeepCopy() JSONPatches {
if in == nil {
return nil
}
out := new(JSONPatches)
in.DeepCopyInto(out)
return *out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *KonnectivityAgentSpec) DeepCopyInto(out *KonnectivityAgentSpec) {
*out = *in
@@ -901,6 +958,11 @@ func (in *KonnectivityAgentSpec) DeepCopyInto(out *KonnectivityAgentSpec) {
*out = make(ExtraArgs, len(*in))
copy(*out, *in)
}
if in.Replicas != nil {
in, out := &in.Replicas, &out.Replicas
*out = new(int32)
**out = **in
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new KonnectivityAgentSpec.
@@ -996,6 +1058,11 @@ func (in *KonnectivityStatus) DeepCopyInto(out *KonnectivityStatus) {
in.ClusterRoleBinding.DeepCopyInto(&out.ClusterRoleBinding)
in.Agent.DeepCopyInto(&out.Agent)
in.Service.DeepCopyInto(&out.Service)
if in.Gateway != nil {
in, out := &in.Gateway, &out.Gateway
*out = new(KubernetesGatewayStatus)
(*in).DeepCopyInto(*out)
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new KonnectivityStatus.
@@ -1210,6 +1277,13 @@ func (in *KubeconfigsStatus) DeepCopy() *KubeconfigsStatus {
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *KubeletSpec) DeepCopyInto(out *KubeletSpec) {
*out = *in
if in.ConfigurationJSONPatches != nil {
in, out := &in.ConfigurationJSONPatches, &out.ConfigurationJSONPatches
*out = make(JSONPatches, len(*in))
for i := range *in {
(*in)[i].DeepCopyInto(&(*out)[i])
}
}
if in.PreferredAddressTypes != nil {
in, out := &in.PreferredAddressTypes, &out.PreferredAddressTypes
*out = make([]KubeletPreferredAddressType, len(*in))
@@ -1591,6 +1665,11 @@ func (in *TenantControlPlaneList) DeepCopyObject() runtime.Object {
func (in *TenantControlPlaneSpec) DeepCopyInto(out *TenantControlPlaneSpec) {
*out = *in
out.WritePermissions = in.WritePermissions
if in.DataStoreOverrides != nil {
in, out := &in.DataStoreOverrides, &out.DataStoreOverrides
*out = make([]DataStoreOverride, len(*in))
copy(*out, *in)
}
in.ControlPlane.DeepCopyInto(&out.ControlPlane)
in.Kubernetes.DeepCopyInto(&out.Kubernetes)
in.NetworkProfile.DeepCopyInto(&out.NetworkProfile)

View File

@@ -33,5 +33,7 @@ annotations:
- name: support
url: https://clastix.io/support
artifacthub.io/changes: |
- kind: changed
description: Upgrading support to Kubernetes v1.35
- kind: added
description: First commit
description: Supporting multiple Datastore via etcd overrides

View File

@@ -149,9 +149,10 @@ versions:
operator:
description: |-
Operator represents a key's relationship to the value.
Valid operators are Exists and Equal. Defaults to Equal.
Valid operators are Exists, Equal, Lt, and Gt. Defaults to Equal.
Exists is equivalent to wildcard for value, so that a pod can
tolerate all taints of a particular category.
Lt and Gt perform numeric comparisons (requires feature gate TaintTolerationComparisonOperators).
type: string
tolerationSeconds:
description: |-
@@ -177,8 +178,8 @@ versions:
type: string
type: object
x-kubernetes-validations:
- message: replicas must be 0 when mode is DaemonSet, and greater than 0 when mode is Deployment
rule: '!(self.mode == ''DaemonSet'' && has(self.replicas) && self.replicas != 0) && !(self.mode == ''Deployment'' && self.replicas == 0)'
- message: replicas must be 0 (or unset) when mode is DaemonSet, and greater than 0 (or unset) when mode is Deployment
rule: '!(self.mode == ''DaemonSet'' && has(self.replicas) && self.replicas != 0) && !(self.mode == ''Deployment'' && has(self.replicas) && self.replicas == 0)'
server:
default:
image: registry.k8s.io/kas-network-proxy/proxy-server
@@ -1126,7 +1127,9 @@ versions:
type: integer
type: object
resizePolicy:
description: Resources resize policy for the container.
description: |-
Resources resize policy for the container.
This field cannot be set on ephemeral containers.
items:
description: ContainerResizePolicy represents resource resize policy for the container.
properties:
@@ -2588,7 +2591,9 @@ versions:
type: integer
type: object
resizePolicy:
description: Resources resize policy for the container.
description: |-
Resources resize policy for the container.
This field cannot be set on ephemeral containers.
items:
description: ContainerResizePolicy represents resource resize policy for the container.
properties:
@@ -3981,7 +3986,7 @@ versions:
resources:
description: |-
resources represents the minimum resources the volume should have.
If RecoverVolumeExpansionFailure feature is enabled users are allowed to specify resource requirements
Users are allowed to specify resource requirements
that are lower than previous value but must still be higher than capacity recorded in the
status field of the claim.
More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#resources
@@ -4816,6 +4821,24 @@ versions:
signerName:
description: Kubelet's generated CSRs will be addressed to this signer.
type: string
userAnnotations:
additionalProperties:
type: string
description: |-
userAnnotations allow pod authors to pass additional information to
the signer implementation. Kubernetes does not restrict or validate this
metadata in any way.
These values are copied verbatim into the `spec.unverifiedUserAnnotations` field of
the PodCertificateRequest objects that Kubelet creates.
Entries are subject to the same validation as object metadata annotations,
with the addition that all keys must be domain-prefixed. No restrictions
are placed on values, except an overall size limitation on the entire field.
Signers should document the keys and values they support. Signers should
deny requests that contain keys they do not recognize.
type: object
required:
- keyType
- signerName
@@ -6505,9 +6528,10 @@ versions:
operator:
description: |-
Operator represents a key's relationship to the value.
Valid operators are Exists and Equal. Defaults to Equal.
Valid operators are Exists, Equal, Lt, and Gt. Defaults to Equal.
Exists is equivalent to wildcard for value, so that a pod can
tolerate all taints of a particular category.
Lt and Gt perform numeric comparisons (requires feature gate TaintTolerationComparisonOperators).
type: string
tolerationSeconds:
description: |-
@@ -6872,6 +6896,9 @@ versions:
type: object
type: array
type: object
x-kubernetes-validations:
- message: parentRefs must not specify port or sectionName, these are set automatically by Kamaji
rule: '!has(self.parentRefs) || size(self.parentRefs) == 0 || self.parentRefs.all(ref, !has(ref.port) && !has(ref.sectionName))'
ingress:
description: Defining the options for an Optional Ingress which will expose API Server of the Tenant Control Plane
properties:
@@ -6985,6 +7012,19 @@ versions:
Migration from one DataStore to another backed by the same Driver is possible. See: https://kamaji.clastix.io/guides/datastore-migration/
Migration from one DataStore to another backed by a different Driver is not supported.
type: string
dataStoreOverrides:
description: DataStoreOverride defines which kubernetes resources will be stored in dedicated datastores.
items:
description: DataStoreOverride defines which kubernetes resource will be stored in a dedicated datastore.
properties:
dataStore:
description: DataStore specifies the DataStore that should be used to store the Kubernetes data for the given Resource.
type: string
resource:
description: Resource specifies which kubernetes resource to target.
type: string
type: object
type: array
dataStoreSchema:
description: |-
DataStoreSchema allows to specify the name of the database (for relational DataStores) or the key prefix (for etcd). This
@@ -7077,10 +7117,45 @@ versions:
description: |-
CGroupFS defines the cgroup driver for Kubelet
https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/configure-cgroup-driver/
Deprecated: use ConfigurationJSONPatches.
enum:
- systemd
- cgroupfs
type: string
configurationJSONPatches:
description: |-
ConfigurationJSONPatches contains the RFC 6902 JSON patches to customise the kubeadm generate configuration,
useful to customise and mangling the configuration according to your needs;
e.g.: configuring the cgroup driver used by Kubelet is possible via the following patch:
[{"op": "replace", "path": "/cgroupDriver", "value": "systemd"}]
items:
properties:
from:
description: From specifies the source location for move or copy operations.
type: string
op:
description: Op is the RFC 6902 JSON Patch operation.
enum:
- add
- remove
- replace
- move
- copy
- test
type: string
path:
description: Path specifies the target location in the JSON document. Use "/" to separate keys; "-" for appending to arrays.
type: string
value:
description: Value is the operation value to be used when Op is add, replace, test.
x-kubernetes-preserve-unknown-fields: true
required:
- op
- path
type: object
type: array
preferredAddressTypes:
default:
- InternalIP
@@ -7277,6 +7352,383 @@ versions:
type: object
enabled:
type: boolean
gateway:
description: KubernetesGatewayStatus defines the status for the Tenant Control Plane Gateway in the management cluster.
properties:
accessPoints:
description: A list of valid access points that the route exposes.
items:
properties:
port:
format: int32
type: integer
type:
description: |-
AddressType defines how a network address is represented as a text string.
This may take two possible forms:
* A predefined CamelCase string identifier (currently limited to `IPAddress` or `Hostname`)
* A domain-prefixed string identifier (like `acme.io/CustomAddressType`)
Values `IPAddress` and `Hostname` have Extended support.
The `NamedAddress` value has been deprecated in favor of implementation
specific domain-prefixed strings.
All other values, including domain-prefixed values have Implementation-specific support,
which are used in implementation-specific behaviors. Support for additional
predefined CamelCase identifiers may be added in future releases.
maxLength: 253
minLength: 1
pattern: ^Hostname|IPAddress|NamedAddress|[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*\/[A-Za-z0-9\/\-._~%!$&'()*+,;=:]+$
type: string
urls:
items:
type: string
type: array
value:
type: string
required:
- port
- type
- value
type: object
type: array
parents:
description: |-
Parents is a list of parent resources (usually Gateways) that are
associated with the route, and the status of the route with respect to
each parent. When this route attaches to a parent, the controller that
manages the parent must add an entry to this list when the controller
first sees the route and should update the entry as appropriate when the
route or gateway is modified.
Note that parent references that cannot be resolved by an implementation
of this API will not be added to this list. Implementations of this API
can only populate Route status for the Gateways/parent resources they are
responsible for.
A maximum of 32 Gateways will be represented in this list. An empty list
means the route has not been attached to any Gateway.
<gateway:util:excludeFromCRD>
Notes for implementors:
While parents is not a listType `map`, this is due to the fact that the
list key is not scalar, and Kubernetes is unable to represent this.
Parent status MUST be considered to be namespaced by the combination of
the parentRef and controllerName fields, and implementations should keep
the following rules in mind when updating this status:
* Implementations MUST update only entries that have a matching value of
`controllerName` for that implementation.
* Implementations MUST NOT update entries with non-matching `controllerName`
fields.
* Implementations MUST treat each `parentRef`` in the Route separately and
update its status based on the relationship with that parent.
* Implementations MUST perform a read-modify-write cycle on this field
before modifying it. That is, when modifying this field, implementations
must be confident they have fetched the most recent version of this field,
and ensure that changes they make are on that recent version.
</gateway:util:excludeFromCRD>
items:
description: |-
RouteParentStatus describes the status of a route with respect to an
associated Parent.
properties:
conditions:
description: |-
Conditions describes the status of the route with respect to the Gateway.
Note that the route's availability is also subject to the Gateway's own
status conditions and listener status.
If the Route's ParentRef specifies an existing Gateway that supports
Routes of this kind AND that Gateway's controller has sufficient access,
then that Gateway's controller MUST set the "Accepted" condition on the
Route, to indicate whether the route has been accepted or rejected by the
Gateway, and why.
A Route MUST be considered "Accepted" if at least one of the Route's
rules is implemented by the Gateway.
There are a number of cases where the "Accepted" condition may not be set
due to lack of controller visibility, that includes when:
* The Route refers to a nonexistent parent.
* The Route is of a type that the controller does not support.
* The Route is in a namespace the controller does not have access to.
<gateway:util:excludeFromCRD>
Notes for implementors:
Conditions are a listType `map`, which means that they function like a
map with a key of the `type` field _in the k8s apiserver_.
This means that implementations must obey some rules when updating this
section.
* Implementations MUST perform a read-modify-write cycle on this field
before modifying it. That is, when modifying this field, implementations
must be confident they have fetched the most recent version of this field,
and ensure that changes they make are on that recent version.
* Implementations MUST NOT remove or reorder Conditions that they are not
directly responsible for. For example, if an implementation sees a Condition
with type `special.io/SomeField`, it MUST NOT remove, change or update that
Condition.
* Implementations MUST always _merge_ changes into Conditions of the same Type,
rather than creating more than one Condition of the same Type.
* Implementations MUST always update the `observedGeneration` field of the
Condition to the `metadata.generation` of the Gateway at the time of update creation.
* If the `observedGeneration` of a Condition is _greater than_ the value the
implementation knows about, then it MUST NOT perform the update on that Condition,
but must wait for a future reconciliation and status update. (The assumption is that
the implementation's copy of the object is stale and an update will be re-triggered
if relevant.)
</gateway:util:excludeFromCRD>
items:
description: Condition contains details for one aspect of the current state of this API Resource.
properties:
lastTransitionTime:
description: |-
lastTransitionTime is the last time the condition transitioned from one status to another.
This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable.
format: date-time
type: string
message:
description: |-
message is a human readable message indicating details about the transition.
This may be an empty string.
maxLength: 32768
type: string
observedGeneration:
description: |-
observedGeneration represents the .metadata.generation that the condition was set based upon.
For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date
with respect to the current state of the instance.
format: int64
minimum: 0
type: integer
reason:
description: |-
reason contains a programmatic identifier indicating the reason for the condition's last transition.
Producers of specific condition types may define expected values and meanings for this field,
and whether the values are considered a guaranteed API.
The value should be a CamelCase string.
This field may not be empty.
maxLength: 1024
minLength: 1
pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$
type: string
status:
description: status of the condition, one of True, False, Unknown.
enum:
- "True"
- "False"
- Unknown
type: string
type:
description: type of condition in CamelCase or in foo.example.com/CamelCase.
maxLength: 316
pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$
type: string
required:
- lastTransitionTime
- message
- reason
- status
- type
type: object
maxItems: 8
minItems: 1
type: array
x-kubernetes-list-map-keys:
- type
x-kubernetes-list-type: map
controllerName:
description: |-
ControllerName is a domain/path string that indicates the name of the
controller that wrote this status. This corresponds with the
controllerName field on GatewayClass.
Example: "example.net/gateway-controller".
The format of this field is DOMAIN "/" PATH, where DOMAIN and PATH are
valid Kubernetes names
(https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names).
Controllers MUST populate this field when writing status. Controllers should ensure that
entries to status populated with their ControllerName are cleaned up when they are no
longer necessary.
maxLength: 253
minLength: 1
pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*\/[A-Za-z0-9\/\-._~%!$&'()*+,;=:]+$
type: string
parentRef:
description: |-
ParentRef corresponds with a ParentRef in the spec that this
RouteParentStatus struct describes the status of.
properties:
group:
default: gateway.networking.k8s.io
description: |-
Group is the group of the referent.
When unspecified, "gateway.networking.k8s.io" is inferred.
To set the core API group (such as for a "Service" kind referent),
Group must be explicitly set to "" (empty string).
Support: Core
maxLength: 253
pattern: ^$|^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$
type: string
kind:
default: Gateway
description: |-
Kind is kind of the referent.
There are two kinds of parent resources with "Core" support:
* Gateway (Gateway conformance profile)
* Service (Mesh conformance profile, ClusterIP Services only)
Support for other resources is Implementation-Specific.
maxLength: 63
minLength: 1
pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$
type: string
name:
description: |-
Name is the name of the referent.
Support: Core
maxLength: 253
minLength: 1
type: string
namespace:
description: |-
Namespace is the namespace of the referent. When unspecified, this refers
to the local namespace of the Route.
Note that there are specific rules for ParentRefs which cross namespace
boundaries. Cross-namespace references are only valid if they are explicitly
allowed by something in the namespace they are referring to. For example:
Gateway has the AllowedRoutes field, and ReferenceGrant provides a
generic way to enable any other kind of cross-namespace reference.
<gateway:experimental:description>
ParentRefs from a Route to a Service in the same namespace are "producer"
routes, which apply default routing rules to inbound connections from
any namespace to the Service.
ParentRefs from a Route to a Service in a different namespace are
"consumer" routes, and these routing rules are only applied to outbound
connections originating from the same namespace as the Route, for which
the intended destination of the connections are a Service targeted as a
ParentRef of the Route.
</gateway:experimental:description>
Support: Core
maxLength: 63
minLength: 1
pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$
type: string
port:
description: |-
Port is the network port this Route targets. It can be interpreted
differently based on the type of parent resource.
When the parent resource is a Gateway, this targets all listeners
listening on the specified port that also support this kind of Route(and
select this Route). It's not recommended to set `Port` unless the
networking behaviors specified in a Route must apply to a specific port
as opposed to a listener(s) whose port(s) may be changed. When both Port
and SectionName are specified, the name and port of the selected listener
must match both specified values.
<gateway:experimental:description>
When the parent resource is a Service, this targets a specific port in the
Service spec. When both Port (experimental) and SectionName are specified,
the name and port of the selected port must match both specified values.
</gateway:experimental:description>
Implementations MAY choose to support other parent resources.
Implementations supporting other types of parent resources MUST clearly
document how/if Port is interpreted.
For the purpose of status, an attachment is considered successful as
long as the parent resource accepts it partially. For example, Gateway
listeners can restrict which Routes can attach to them by Route kind,
namespace, or hostname. If 1 of 2 Gateway listeners accept attachment
from the referencing Route, the Route MUST be considered successfully
attached. If no Gateway listeners accept attachment from this Route,
the Route MUST be considered detached from the Gateway.
Support: Extended
format: int32
maximum: 65535
minimum: 1
type: integer
sectionName:
description: |-
SectionName is the name of a section within the target resource. In the
following resources, SectionName is interpreted as the following:
* Gateway: Listener name. When both Port (experimental) and SectionName
are specified, the name and port of the selected listener must match
both specified values.
* Service: Port name. When both Port (experimental) and SectionName
are specified, the name and port of the selected listener must match
both specified values.
Implementations MAY choose to support attaching Routes to other resources.
If that is the case, they MUST clearly document how SectionName is
interpreted.
When unspecified (empty string), this will reference the entire resource.
For the purpose of status, an attachment is considered successful if at
least one section in the parent resource accepts it. For example, Gateway
listeners can restrict which Routes can attach to them by Route kind,
namespace, or hostname. If 1 of 2 Gateway listeners accept attachment from
the referencing Route, the Route MUST be considered successfully
attached. If no Gateway listeners accept attachment from this Route, the
Route MUST be considered detached from the Gateway.
Support: Core
maxLength: 253
minLength: 1
pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$
type: string
required:
- name
type: object
required:
- conditions
- controllerName
- parentRef
type: object
maxItems: 32
type: array
x-kubernetes-list-type: atomic
routeRef:
description: Reference to the route created for this tenant.
properties:
name:
default: ""
description: |-
Name of the referent.
This field is effectively required, but due to backwards compatibility is
allowed to be empty. Instances of this type with an empty value here are
almost certainly wrong.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
type: string
type: object
x-kubernetes-map-type: atomic
required:
- parents
type: object
kubeconfig:
description: KubeconfigStatus contains information about the generated kubeconfig.
properties:
@@ -7707,7 +8159,7 @@ versions:
Total number of terminating pods targeted by this deployment. Terminating pods have a non-null
.metadata.deletionTimestamp and have not yet reached the Failed or Succeeded .status.phase.
This is an alpha field. Enable DeploymentReplicaSetTerminatingReplicas to be able to use this field.
This is a beta field and requires enabling DeploymentReplicaSetTerminatingReplicas feature (enabled by default).
format: int32
type: integer
unavailableReplicas:

View File

@@ -1,3 +1,17 @@
- apiGroups:
- ""
resources:
- configmaps
- secrets
- services
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- ""
resources:
@@ -28,20 +42,6 @@
- get
- list
- watch
- apiGroups:
- ""
resources:
- configmaps
- secrets
- services
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- gateway.networking.k8s.io
resources:

View File

@@ -4,7 +4,7 @@ kind: CustomResourceDefinition
metadata:
annotations:
cert-manager.io/inject-ca-from: kamaji-system/kamaji-serving-cert
controller-gen.kubebuilder.io/version: v0.16.1
controller-gen.kubebuilder.io/version: v0.20.0
name: datastores.kamaji.clastix.io
spec:
group: kamaji.clastix.io

View File

@@ -3,7 +3,7 @@ kind: CustomResourceDefinition
metadata:
annotations:
cert-manager.io/inject-ca-from: kamaji-system/kamaji-serving-cert
controller-gen.kubebuilder.io/version: v0.16.1
controller-gen.kubebuilder.io/version: v0.20.0
name: kubeconfiggenerators.kamaji.clastix.io
spec:
group: kamaji.clastix.io

View File

@@ -3,7 +3,7 @@ kind: CustomResourceDefinition
metadata:
annotations:
cert-manager.io/inject-ca-from: kamaji-system/kamaji-serving-cert
controller-gen.kubebuilder.io/version: v0.16.1
controller-gen.kubebuilder.io/version: v0.20.0
name: tenantcontrolplanes.kamaji.clastix.io
spec:
group: kamaji.clastix.io
@@ -157,9 +157,10 @@ spec:
operator:
description: |-
Operator represents a key's relationship to the value.
Valid operators are Exists and Equal. Defaults to Equal.
Valid operators are Exists, Equal, Lt, and Gt. Defaults to Equal.
Exists is equivalent to wildcard for value, so that a pod can
tolerate all taints of a particular category.
Lt and Gt perform numeric comparisons (requires feature gate TaintTolerationComparisonOperators).
type: string
tolerationSeconds:
description: |-
@@ -185,8 +186,8 @@ spec:
type: string
type: object
x-kubernetes-validations:
- message: replicas must be 0 when mode is DaemonSet, and greater than 0 when mode is Deployment
rule: '!(self.mode == ''DaemonSet'' && has(self.replicas) && self.replicas != 0) && !(self.mode == ''Deployment'' && self.replicas == 0)'
- message: replicas must be 0 (or unset) when mode is DaemonSet, and greater than 0 (or unset) when mode is Deployment
rule: '!(self.mode == ''DaemonSet'' && has(self.replicas) && self.replicas != 0) && !(self.mode == ''Deployment'' && has(self.replicas) && self.replicas == 0)'
server:
default:
image: registry.k8s.io/kas-network-proxy/proxy-server
@@ -1134,7 +1135,9 @@ spec:
type: integer
type: object
resizePolicy:
description: Resources resize policy for the container.
description: |-
Resources resize policy for the container.
This field cannot be set on ephemeral containers.
items:
description: ContainerResizePolicy represents resource resize policy for the container.
properties:
@@ -2596,7 +2599,9 @@ spec:
type: integer
type: object
resizePolicy:
description: Resources resize policy for the container.
description: |-
Resources resize policy for the container.
This field cannot be set on ephemeral containers.
items:
description: ContainerResizePolicy represents resource resize policy for the container.
properties:
@@ -3989,7 +3994,7 @@ spec:
resources:
description: |-
resources represents the minimum resources the volume should have.
If RecoverVolumeExpansionFailure feature is enabled users are allowed to specify resource requirements
Users are allowed to specify resource requirements
that are lower than previous value but must still be higher than capacity recorded in the
status field of the claim.
More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#resources
@@ -4824,6 +4829,24 @@ spec:
signerName:
description: Kubelet's generated CSRs will be addressed to this signer.
type: string
userAnnotations:
additionalProperties:
type: string
description: |-
userAnnotations allow pod authors to pass additional information to
the signer implementation. Kubernetes does not restrict or validate this
metadata in any way.
These values are copied verbatim into the `spec.unverifiedUserAnnotations` field of
the PodCertificateRequest objects that Kubelet creates.
Entries are subject to the same validation as object metadata annotations,
with the addition that all keys must be domain-prefixed. No restrictions
are placed on values, except an overall size limitation on the entire field.
Signers should document the keys and values they support. Signers should
deny requests that contain keys they do not recognize.
type: object
required:
- keyType
- signerName
@@ -6513,9 +6536,10 @@ spec:
operator:
description: |-
Operator represents a key's relationship to the value.
Valid operators are Exists and Equal. Defaults to Equal.
Valid operators are Exists, Equal, Lt, and Gt. Defaults to Equal.
Exists is equivalent to wildcard for value, so that a pod can
tolerate all taints of a particular category.
Lt and Gt perform numeric comparisons (requires feature gate TaintTolerationComparisonOperators).
type: string
tolerationSeconds:
description: |-
@@ -6880,6 +6904,9 @@ spec:
type: object
type: array
type: object
x-kubernetes-validations:
- message: parentRefs must not specify port or sectionName, these are set automatically by Kamaji
rule: '!has(self.parentRefs) || size(self.parentRefs) == 0 || self.parentRefs.all(ref, !has(ref.port) && !has(ref.sectionName))'
ingress:
description: Defining the options for an Optional Ingress which will expose API Server of the Tenant Control Plane
properties:
@@ -6993,6 +7020,19 @@ spec:
Migration from one DataStore to another backed by the same Driver is possible. See: https://kamaji.clastix.io/guides/datastore-migration/
Migration from one DataStore to another backed by a different Driver is not supported.
type: string
dataStoreOverrides:
description: DataStoreOverride defines which kubernetes resources will be stored in dedicated datastores.
items:
description: DataStoreOverride defines which kubernetes resource will be stored in a dedicated datastore.
properties:
dataStore:
description: DataStore specifies the DataStore that should be used to store the Kubernetes data for the given Resource.
type: string
resource:
description: Resource specifies which kubernetes resource to target.
type: string
type: object
type: array
dataStoreSchema:
description: |-
DataStoreSchema allows to specify the name of the database (for relational DataStores) or the key prefix (for etcd). This
@@ -7085,10 +7125,45 @@ spec:
description: |-
CGroupFS defines the cgroup driver for Kubelet
https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/configure-cgroup-driver/
Deprecated: use ConfigurationJSONPatches.
enum:
- systemd
- cgroupfs
type: string
configurationJSONPatches:
description: |-
ConfigurationJSONPatches contains the RFC 6902 JSON patches to customise the kubeadm generate configuration,
useful to customise and mangling the configuration according to your needs;
e.g.: configuring the cgroup driver used by Kubelet is possible via the following patch:
[{"op": "replace", "path": "/cgroupDriver", "value": "systemd"}]
items:
properties:
from:
description: From specifies the source location for move or copy operations.
type: string
op:
description: Op is the RFC 6902 JSON Patch operation.
enum:
- add
- remove
- replace
- move
- copy
- test
type: string
path:
description: Path specifies the target location in the JSON document. Use "/" to separate keys; "-" for appending to arrays.
type: string
value:
description: Value is the operation value to be used when Op is add, replace, test.
x-kubernetes-preserve-unknown-fields: true
required:
- op
- path
type: object
type: array
preferredAddressTypes:
default:
- InternalIP
@@ -7285,6 +7360,383 @@ spec:
type: object
enabled:
type: boolean
gateway:
description: KubernetesGatewayStatus defines the status for the Tenant Control Plane Gateway in the management cluster.
properties:
accessPoints:
description: A list of valid access points that the route exposes.
items:
properties:
port:
format: int32
type: integer
type:
description: |-
AddressType defines how a network address is represented as a text string.
This may take two possible forms:
* A predefined CamelCase string identifier (currently limited to `IPAddress` or `Hostname`)
* A domain-prefixed string identifier (like `acme.io/CustomAddressType`)
Values `IPAddress` and `Hostname` have Extended support.
The `NamedAddress` value has been deprecated in favor of implementation
specific domain-prefixed strings.
All other values, including domain-prefixed values have Implementation-specific support,
which are used in implementation-specific behaviors. Support for additional
predefined CamelCase identifiers may be added in future releases.
maxLength: 253
minLength: 1
pattern: ^Hostname|IPAddress|NamedAddress|[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*\/[A-Za-z0-9\/\-._~%!$&'()*+,;=:]+$
type: string
urls:
items:
type: string
type: array
value:
type: string
required:
- port
- type
- value
type: object
type: array
parents:
description: |-
Parents is a list of parent resources (usually Gateways) that are
associated with the route, and the status of the route with respect to
each parent. When this route attaches to a parent, the controller that
manages the parent must add an entry to this list when the controller
first sees the route and should update the entry as appropriate when the
route or gateway is modified.
Note that parent references that cannot be resolved by an implementation
of this API will not be added to this list. Implementations of this API
can only populate Route status for the Gateways/parent resources they are
responsible for.
A maximum of 32 Gateways will be represented in this list. An empty list
means the route has not been attached to any Gateway.
<gateway:util:excludeFromCRD>
Notes for implementors:
While parents is not a listType `map`, this is due to the fact that the
list key is not scalar, and Kubernetes is unable to represent this.
Parent status MUST be considered to be namespaced by the combination of
the parentRef and controllerName fields, and implementations should keep
the following rules in mind when updating this status:
* Implementations MUST update only entries that have a matching value of
`controllerName` for that implementation.
* Implementations MUST NOT update entries with non-matching `controllerName`
fields.
* Implementations MUST treat each `parentRef`` in the Route separately and
update its status based on the relationship with that parent.
* Implementations MUST perform a read-modify-write cycle on this field
before modifying it. That is, when modifying this field, implementations
must be confident they have fetched the most recent version of this field,
and ensure that changes they make are on that recent version.
</gateway:util:excludeFromCRD>
items:
description: |-
RouteParentStatus describes the status of a route with respect to an
associated Parent.
properties:
conditions:
description: |-
Conditions describes the status of the route with respect to the Gateway.
Note that the route's availability is also subject to the Gateway's own
status conditions and listener status.
If the Route's ParentRef specifies an existing Gateway that supports
Routes of this kind AND that Gateway's controller has sufficient access,
then that Gateway's controller MUST set the "Accepted" condition on the
Route, to indicate whether the route has been accepted or rejected by the
Gateway, and why.
A Route MUST be considered "Accepted" if at least one of the Route's
rules is implemented by the Gateway.
There are a number of cases where the "Accepted" condition may not be set
due to lack of controller visibility, that includes when:
* The Route refers to a nonexistent parent.
* The Route is of a type that the controller does not support.
* The Route is in a namespace the controller does not have access to.
<gateway:util:excludeFromCRD>
Notes for implementors:
Conditions are a listType `map`, which means that they function like a
map with a key of the `type` field _in the k8s apiserver_.
This means that implementations must obey some rules when updating this
section.
* Implementations MUST perform a read-modify-write cycle on this field
before modifying it. That is, when modifying this field, implementations
must be confident they have fetched the most recent version of this field,
and ensure that changes they make are on that recent version.
* Implementations MUST NOT remove or reorder Conditions that they are not
directly responsible for. For example, if an implementation sees a Condition
with type `special.io/SomeField`, it MUST NOT remove, change or update that
Condition.
* Implementations MUST always _merge_ changes into Conditions of the same Type,
rather than creating more than one Condition of the same Type.
* Implementations MUST always update the `observedGeneration` field of the
Condition to the `metadata.generation` of the Gateway at the time of update creation.
* If the `observedGeneration` of a Condition is _greater than_ the value the
implementation knows about, then it MUST NOT perform the update on that Condition,
but must wait for a future reconciliation and status update. (The assumption is that
the implementation's copy of the object is stale and an update will be re-triggered
if relevant.)
</gateway:util:excludeFromCRD>
items:
description: Condition contains details for one aspect of the current state of this API Resource.
properties:
lastTransitionTime:
description: |-
lastTransitionTime is the last time the condition transitioned from one status to another.
This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable.
format: date-time
type: string
message:
description: |-
message is a human readable message indicating details about the transition.
This may be an empty string.
maxLength: 32768
type: string
observedGeneration:
description: |-
observedGeneration represents the .metadata.generation that the condition was set based upon.
For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date
with respect to the current state of the instance.
format: int64
minimum: 0
type: integer
reason:
description: |-
reason contains a programmatic identifier indicating the reason for the condition's last transition.
Producers of specific condition types may define expected values and meanings for this field,
and whether the values are considered a guaranteed API.
The value should be a CamelCase string.
This field may not be empty.
maxLength: 1024
minLength: 1
pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$
type: string
status:
description: status of the condition, one of True, False, Unknown.
enum:
- "True"
- "False"
- Unknown
type: string
type:
description: type of condition in CamelCase or in foo.example.com/CamelCase.
maxLength: 316
pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$
type: string
required:
- lastTransitionTime
- message
- reason
- status
- type
type: object
maxItems: 8
minItems: 1
type: array
x-kubernetes-list-map-keys:
- type
x-kubernetes-list-type: map
controllerName:
description: |-
ControllerName is a domain/path string that indicates the name of the
controller that wrote this status. This corresponds with the
controllerName field on GatewayClass.
Example: "example.net/gateway-controller".
The format of this field is DOMAIN "/" PATH, where DOMAIN and PATH are
valid Kubernetes names
(https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names).
Controllers MUST populate this field when writing status. Controllers should ensure that
entries to status populated with their ControllerName are cleaned up when they are no
longer necessary.
maxLength: 253
minLength: 1
pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*\/[A-Za-z0-9\/\-._~%!$&'()*+,;=:]+$
type: string
parentRef:
description: |-
ParentRef corresponds with a ParentRef in the spec that this
RouteParentStatus struct describes the status of.
properties:
group:
default: gateway.networking.k8s.io
description: |-
Group is the group of the referent.
When unspecified, "gateway.networking.k8s.io" is inferred.
To set the core API group (such as for a "Service" kind referent),
Group must be explicitly set to "" (empty string).
Support: Core
maxLength: 253
pattern: ^$|^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$
type: string
kind:
default: Gateway
description: |-
Kind is kind of the referent.
There are two kinds of parent resources with "Core" support:
* Gateway (Gateway conformance profile)
* Service (Mesh conformance profile, ClusterIP Services only)
Support for other resources is Implementation-Specific.
maxLength: 63
minLength: 1
pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$
type: string
name:
description: |-
Name is the name of the referent.
Support: Core
maxLength: 253
minLength: 1
type: string
namespace:
description: |-
Namespace is the namespace of the referent. When unspecified, this refers
to the local namespace of the Route.
Note that there are specific rules for ParentRefs which cross namespace
boundaries. Cross-namespace references are only valid if they are explicitly
allowed by something in the namespace they are referring to. For example:
Gateway has the AllowedRoutes field, and ReferenceGrant provides a
generic way to enable any other kind of cross-namespace reference.
<gateway:experimental:description>
ParentRefs from a Route to a Service in the same namespace are "producer"
routes, which apply default routing rules to inbound connections from
any namespace to the Service.
ParentRefs from a Route to a Service in a different namespace are
"consumer" routes, and these routing rules are only applied to outbound
connections originating from the same namespace as the Route, for which
the intended destination of the connections are a Service targeted as a
ParentRef of the Route.
</gateway:experimental:description>
Support: Core
maxLength: 63
minLength: 1
pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$
type: string
port:
description: |-
Port is the network port this Route targets. It can be interpreted
differently based on the type of parent resource.
When the parent resource is a Gateway, this targets all listeners
listening on the specified port that also support this kind of Route(and
select this Route). It's not recommended to set `Port` unless the
networking behaviors specified in a Route must apply to a specific port
as opposed to a listener(s) whose port(s) may be changed. When both Port
and SectionName are specified, the name and port of the selected listener
must match both specified values.
<gateway:experimental:description>
When the parent resource is a Service, this targets a specific port in the
Service spec. When both Port (experimental) and SectionName are specified,
the name and port of the selected port must match both specified values.
</gateway:experimental:description>
Implementations MAY choose to support other parent resources.
Implementations supporting other types of parent resources MUST clearly
document how/if Port is interpreted.
For the purpose of status, an attachment is considered successful as
long as the parent resource accepts it partially. For example, Gateway
listeners can restrict which Routes can attach to them by Route kind,
namespace, or hostname. If 1 of 2 Gateway listeners accept attachment
from the referencing Route, the Route MUST be considered successfully
attached. If no Gateway listeners accept attachment from this Route,
the Route MUST be considered detached from the Gateway.
Support: Extended
format: int32
maximum: 65535
minimum: 1
type: integer
sectionName:
description: |-
SectionName is the name of a section within the target resource. In the
following resources, SectionName is interpreted as the following:
* Gateway: Listener name. When both Port (experimental) and SectionName
are specified, the name and port of the selected listener must match
both specified values.
* Service: Port name. When both Port (experimental) and SectionName
are specified, the name and port of the selected listener must match
both specified values.
Implementations MAY choose to support attaching Routes to other resources.
If that is the case, they MUST clearly document how SectionName is
interpreted.
When unspecified (empty string), this will reference the entire resource.
For the purpose of status, an attachment is considered successful if at
least one section in the parent resource accepts it. For example, Gateway
listeners can restrict which Routes can attach to them by Route kind,
namespace, or hostname. If 1 of 2 Gateway listeners accept attachment from
the referencing Route, the Route MUST be considered successfully
attached. If no Gateway listeners accept attachment from this Route, the
Route MUST be considered detached from the Gateway.
Support: Core
maxLength: 253
minLength: 1
pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$
type: string
required:
- name
type: object
required:
- conditions
- controllerName
- parentRef
type: object
maxItems: 32
type: array
x-kubernetes-list-type: atomic
routeRef:
description: Reference to the route created for this tenant.
properties:
name:
default: ""
description: |-
Name of the referent.
This field is effectively required, but due to backwards compatibility is
allowed to be empty. Instances of this type with an empty value here are
almost certainly wrong.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
type: string
type: object
x-kubernetes-map-type: atomic
required:
- parents
type: object
kubeconfig:
description: KubeconfigStatus contains information about the generated kubeconfig.
properties:
@@ -7715,7 +8167,7 @@ spec:
Total number of terminating pods targeted by this deployment. Terminating pods have a non-null
.metadata.deletionTimestamp and have not yet reached the Failed or Succeeded .status.phase.
This is an alpha field. Enable DeploymentReplicaSetTerminatingReplicas to be able to use this field.
This is a beta field and requires enabling DeploymentReplicaSetTerminatingReplicas feature (enabled by default).
format: int32
type: integer
unavailableReplicas:

View File

@@ -13,7 +13,15 @@ spec:
kubernetes:
version: "v1.33.0"
kubelet:
cgroupfs: systemd
configurationJSONPatches:
- op: add
path: /featureGates
value:
KubeletCrashLoopBackOffMax: false
KubeletEnsureSecretPulledImages: false
- op: replace
path: /cgroupDriver
value: systemd
networkProfile:
port: 6443
addons:

View File

@@ -0,0 +1,81 @@
# Copyright 2022 Clastix Labs
# SPDX-License-Identifier: Apache-2.0
# This example demonstrates how to configure Gateway API support for a Tenant Control Plane.
#
# Prerequisites:
# 1. Gateway API CRDs must be installed (GatewayClass, Gateway, TLSRoute)
# 2. A Gateway resource must exist with listeners for ports 6443 and 8132
# 3. DNS(or worker nodes hosts entries) must be configured to resolve the hostname to the Gateway's external address
#
# Example GatewayClass and Gateway configuration:
#
# apiVersion: gateway.networking.k8s.io/v1
# kind: GatewayClass
# metadata:
# name: envoy-gw-class
# spec:
# controllerName: gateway.envoyproxy.io/gatewayclass-controller
# ---
# apiVersion: gateway.networking.k8s.io/v1
# kind: Gateway
# metadata:
# name: gateway
# namespace: default
# spec:
# gatewayClassName: envoy-gw-class
# listeners:
# - allowedRoutes:
# kinds:
# - group: gateway.networking.k8s.io
# kind: TLSRoute
# namespaces:
# from: All
# hostname: '*.cluster.dev'
# name: kube-apiserver
# port: 6443
# protocol: TLS
# tls:
# mode: Passthrough
# - allowedRoutes:
# kinds:
# - group: gateway.networking.k8s.io
# kind: TLSRoute
# namespaces:
# from: All
# hostname: '*.cluster.dev'
# name: konnectivity-server
# port: 8132
# protocol: TLS
# tls:
# mode: Passthrough
apiVersion: kamaji.clastix.io/v1alpha1
kind: TenantControlPlane
metadata:
name: demo-tcp-1
spec:
addons:
coreDNS: {}
kubeProxy: {}
konnectivity: {}
dataStore: default
controlPlane:
gateway:
hostname: "c11.cluster.dev" # worker nodes or kubectl clients must be able to resolve this hostname to the Gateway's external address.
parentRefs:
- name: gateway
namespace: default
deployment:
replicas: 1
service:
serviceType: ClusterIP
kubernetes:
version: v1.32.0
kubelet:
cgroupfs: systemd
networkProfile:
port: 6443
certSANs:
- "c11.cluster.dev"

View File

@@ -26,18 +26,20 @@ import (
)
type GroupResourceBuilderConfiguration struct {
client client.Client
log logr.Logger
tcpReconcilerConfig TenantControlPlaneReconcilerConfig
tenantControlPlane kamajiv1alpha1.TenantControlPlane
ExpirationThreshold time.Duration
Connection datastore.Connection
DataStore kamajiv1alpha1.DataStore
KamajiNamespace string
KamajiServiceAccount string
KamajiService string
KamajiMigrateImage string
DiscoveryClient discovery.DiscoveryInterface
client client.Client
log logr.Logger
tcpReconcilerConfig TenantControlPlaneReconcilerConfig
tenantControlPlane kamajiv1alpha1.TenantControlPlane
ExpirationThreshold time.Duration
Connection datastore.Connection
DataStore kamajiv1alpha1.DataStore
DataStoreOverrides []builder.DataStoreOverrides
DataStoreOverriedsConnections map[string]datastore.Connection
KamajiNamespace string
KamajiServiceAccount string
KamajiService string
KamajiMigrateImage string
DiscoveryClient discovery.DiscoveryInterface
}
type GroupDeletableResourceBuilderConfiguration struct {
@@ -62,8 +64,9 @@ func GetResources(ctx context.Context, config GroupResourceBuilderConfiguration)
resources = append(resources, getKubernetesCertificatesResources(config.client, config.tcpReconcilerConfig, config.tenantControlPlane)...)
resources = append(resources, getKubeconfigResources(config.client, config.tcpReconcilerConfig, config.tenantControlPlane)...)
resources = append(resources, getKubernetesStorageResources(config.client, config.Connection, config.DataStore, config.ExpirationThreshold)...)
resources = append(resources, getKubernetesAdditionalStorageResources(config.client, config.DataStoreOverriedsConnections, config.DataStoreOverrides, config.ExpirationThreshold)...)
resources = append(resources, getKonnectivityServerRequirementsResources(config.client, config.ExpirationThreshold)...)
resources = append(resources, getKubernetesDeploymentResources(config.client, config.tcpReconcilerConfig, config.DataStore)...)
resources = append(resources, getKubernetesDeploymentResources(config.client, config.tcpReconcilerConfig, config.DataStore, config.DataStoreOverrides)...)
resources = append(resources, getKonnectivityServerPatchResources(config.client)...)
resources = append(resources, getDataStoreMigratingCleanup(config.client, config.KamajiNamespace)...)
resources = append(resources, getKubernetesIngressResources(config.client)...)
@@ -71,6 +74,7 @@ func GetResources(ctx context.Context, config GroupResourceBuilderConfiguration)
// Conditionally add Gateway resources
if utilities.AreGatewayResourcesAvailable(ctx, config.client, config.DiscoveryClient) {
resources = append(resources, getKubernetesGatewayResources(config.client)...)
resources = append(resources, getKonnectivityGatewayResources(config.client)...)
}
return resources
@@ -143,6 +147,14 @@ func getKubernetesGatewayResources(c client.Client) []resources.Resource {
}
}
func getKonnectivityGatewayResources(c client.Client) []resources.Resource {
return []resources.Resource{
&konnectivity.KubernetesKonnectivityGatewayResource{
Client: c,
},
}
}
func getKubeadmConfigResources(c client.Client, tmpDirectory string, dataStore kamajiv1alpha1.DataStore) []resources.Resource {
var endpoints []string
@@ -252,12 +264,42 @@ func getKubernetesStorageResources(c client.Client, dbConnection datastore.Conne
}
}
func getKubernetesDeploymentResources(c client.Client, tcpReconcilerConfig TenantControlPlaneReconcilerConfig, dataStore kamajiv1alpha1.DataStore) []resources.Resource {
func getKubernetesAdditionalStorageResources(c client.Client, dbConnections map[string]datastore.Connection, dataStoreOverrides []builder.DataStoreOverrides, threshold time.Duration) []resources.Resource {
res := make([]resources.Resource, 0, len(dataStoreOverrides))
for _, dso := range dataStoreOverrides {
datastore := dso.DataStore
res = append(res,
&ds.MultiTenancy{
DataStore: datastore,
},
&ds.Config{
Client: c,
ConnString: dbConnections[dso.Resource].GetConnectionString(),
DataStore: datastore,
IsOverride: true,
},
&ds.Setup{
Client: c,
Connection: dbConnections[dso.Resource],
DataStore: datastore,
},
&ds.Certificate{
Client: c,
DataStore: datastore,
CertExpirationThreshold: threshold,
})
}
return res
}
func getKubernetesDeploymentResources(c client.Client, tcpReconcilerConfig TenantControlPlaneReconcilerConfig, dataStore kamajiv1alpha1.DataStore, dataStoreOverrides []builder.DataStoreOverrides) []resources.Resource {
return []resources.Resource{
&resources.KubernetesDeploymentResource{
Client: c,
DataStore: dataStore,
KineContainerImage: tcpReconcilerConfig.KineContainerImage,
DataStoreOverrides: dataStoreOverrides,
},
}
}

View File

@@ -36,6 +36,7 @@ type CoreDNS struct {
AdminClient client.Client
GetTenantControlPlaneFunc utils.TenantControlPlaneRetrievalFn
TriggerChannel chan event.GenericEvent
ControllerName string
}
func (c *CoreDNS) Reconcile(ctx context.Context, _ reconcile.Request) (reconcile.Result, error) {
@@ -80,6 +81,7 @@ func (c *CoreDNS) Reconcile(ctx context.Context, _ reconcile.Request) (reconcile
func (c *CoreDNS) SetupWithManager(mgr manager.Manager) error {
return controllerruntime.NewControllerManagedBy(mgr).
Named(c.ControllerName).
WithOptions(controller.TypedOptions[reconcile.Request]{SkipNameValidation: ptr.To(true)}).
For(&rbacv1.ClusterRoleBinding{}, builder.WithPredicates(predicate.NewPredicateFuncs(func(object client.Object) bool {
return object.GetName() == kubeadm.CoreDNSClusterRoleBindingName

View File

@@ -37,6 +37,7 @@ type KonnectivityAgent struct {
AdminClient client.Client
GetTenantControlPlaneFunc utils.TenantControlPlaneRetrievalFn
TriggerChannel chan event.GenericEvent
ControllerName string
}
func (k *KonnectivityAgent) Reconcile(ctx context.Context, _ reconcile.Request) (reconcile.Result, error) {
@@ -87,6 +88,7 @@ func (k *KonnectivityAgent) Reconcile(ctx context.Context, _ reconcile.Request)
func (k *KonnectivityAgent) SetupWithManager(mgr manager.Manager) error {
return controllerruntime.NewControllerManagedBy(mgr).
Named(k.ControllerName).
WithOptions(controller.TypedOptions[reconcile.Request]{SkipNameValidation: ptr.To(true)}).
For(&appsv1.DaemonSet{}, builder.WithPredicates(predicate.NewPredicateFuncs(func(object client.Object) bool {
return object.GetName() == konnectivity.AgentName && object.GetNamespace() == konnectivity.AgentNamespace

View File

@@ -29,6 +29,7 @@ type KubeadmPhase struct {
GetTenantControlPlaneFunc utils.TenantControlPlaneRetrievalFn
TriggerChannel chan event.GenericEvent
Phase resources.KubeadmPhaseResource
ControllerName string
logger logr.Logger
}
@@ -75,6 +76,7 @@ func (k *KubeadmPhase) SetupWithManager(mgr manager.Manager) error {
k.logger = mgr.GetLogger().WithName(k.Phase.GetName())
return controllerruntime.NewControllerManagedBy(mgr).
Named(k.ControllerName).
WithOptions(controller.TypedOptions[reconcile.Request]{SkipNameValidation: ptr.To(true)}).
For(k.Phase.GetWatchedObject(), builder.WithPredicates(predicate.NewPredicateFuncs(k.Phase.GetPredicateFunc()))).
WatchesRawSource(source.Channel(k.TriggerChannel, &handler.EnqueueRequestForObject{})).

View File

@@ -36,6 +36,7 @@ type KubeProxy struct {
AdminClient client.Client
GetTenantControlPlaneFunc utils.TenantControlPlaneRetrievalFn
TriggerChannel chan event.GenericEvent
ControllerName string
}
func (k *KubeProxy) Reconcile(ctx context.Context, _ reconcile.Request) (reconcile.Result, error) {
@@ -82,6 +83,7 @@ func (k *KubeProxy) Reconcile(ctx context.Context, _ reconcile.Request) (reconci
func (k *KubeProxy) SetupWithManager(mgr manager.Manager) error {
return controllerruntime.NewControllerManagedBy(mgr).
Named(k.ControllerName).
WithOptions(controller.TypedOptions[reconcile.Request]{SkipNameValidation: ptr.To(true)}).
For(&rbacv1.ClusterRoleBinding{}, builder.WithPredicates(predicate.NewPredicateFuncs(func(object client.Object) bool {
return object.GetName() == kubeadm.KubeProxyClusterRoleBindingName

View File

@@ -39,6 +39,7 @@ type Migrate struct {
WebhookServiceName string
WebhookCABundle []byte
TriggerChannel chan event.GenericEvent
ControllerName string
}
func (m *Migrate) Reconcile(ctx context.Context, _ reconcile.Request) (reconcile.Result, error) {
@@ -189,6 +190,7 @@ func (m *Migrate) SetupWithManager(mgr manager.Manager) error {
m.TriggerChannel = make(chan event.GenericEvent)
return controllerruntime.NewControllerManagedBy(mgr).
Named(m.ControllerName).
WithOptions(controller.TypedOptions[reconcile.Request]{SkipNameValidation: pointer.To(true)}).
For(&admissionregistrationv1.ValidatingWebhookConfiguration{}, builder.WithPredicates(predicate.NewPredicateFuncs(func(object client.Object) bool {
vwc := m.object()

View File

@@ -39,6 +39,7 @@ type WritePermissions struct {
WebhookServiceName string
WebhookCABundle []byte
TriggerChannel chan event.GenericEvent
ControllerName string
}
func (r *WritePermissions) Reconcile(ctx context.Context, _ reconcile.Request) (reconcile.Result, error) {
@@ -190,6 +191,7 @@ func (r *WritePermissions) SetupWithManager(mgr manager.Manager) error {
r.TriggerChannel = make(chan event.GenericEvent)
return controllerruntime.NewControllerManagedBy(mgr).
Named(r.ControllerName).
WithOptions(controller.TypedOptions[reconcile.Request]{SkipNameValidation: ptr.To(true)}).
For(&admissionregistrationv1.ValidatingWebhookConfiguration{}, builder.WithPredicates(predicate.NewPredicateFuncs(func(object client.Object) bool {
return object.GetName() == r.object().GetName()

View File

@@ -193,7 +193,7 @@ func (m *Manager) Reconcile(ctx context.Context, request reconcile.Request) (res
for _, trigger := range v.triggers {
var shrunkTCP kamajiv1alpha1.TenantControlPlane
shrunkTCP.Name = tcp.Namespace
shrunkTCP.Name = tcp.Name
shrunkTCP.Namespace = tcp.Namespace
go utils.TriggerChannel(ctx, trigger, shrunkTCP)
@@ -253,6 +253,9 @@ func (m *Manager) Reconcile(ctx context.Context, request reconcile.Request) (res
//
// Register all the controllers of the soot here:
//
// Generate unique controller name prefix from TenantControlPlane to avoid metric conflicts
controllerNamePrefix := fmt.Sprintf("%s-%s", tcp.GetNamespace(), tcp.GetName())
writePermissions := &controllers.WritePermissions{
Logger: mgr.GetLogger().WithName("writePermissions"),
Client: mgr.GetClient(),
@@ -261,6 +264,7 @@ func (m *Manager) Reconcile(ctx context.Context, request reconcile.Request) (res
WebhookServiceName: m.MigrateServiceName,
WebhookCABundle: m.MigrateCABundle,
TriggerChannel: nil,
ControllerName: fmt.Sprintf("%s-writepermissions", controllerNamePrefix),
}
if err = writePermissions.SetupWithManager(mgr); err != nil {
return reconcile.Result{}, err
@@ -273,6 +277,7 @@ func (m *Manager) Reconcile(ctx context.Context, request reconcile.Request) (res
GetTenantControlPlaneFunc: m.retrieveTenantControlPlane(tcpCtx, request),
Client: mgr.GetClient(),
Logger: mgr.GetLogger().WithName("migrate"),
ControllerName: fmt.Sprintf("%s-migrate", controllerNamePrefix),
}
if err = migrate.SetupWithManager(mgr); err != nil {
return reconcile.Result{}, err
@@ -283,6 +288,7 @@ func (m *Manager) Reconcile(ctx context.Context, request reconcile.Request) (res
GetTenantControlPlaneFunc: m.retrieveTenantControlPlane(tcpCtx, request),
Logger: mgr.GetLogger().WithName("konnectivity_agent"),
TriggerChannel: make(chan event.GenericEvent),
ControllerName: fmt.Sprintf("%s-konnectivity", controllerNamePrefix),
}
if err = konnectivityAgent.SetupWithManager(mgr); err != nil {
return reconcile.Result{}, err
@@ -293,6 +299,7 @@ func (m *Manager) Reconcile(ctx context.Context, request reconcile.Request) (res
GetTenantControlPlaneFunc: m.retrieveTenantControlPlane(tcpCtx, request),
Logger: mgr.GetLogger().WithName("kube_proxy"),
TriggerChannel: make(chan event.GenericEvent),
ControllerName: fmt.Sprintf("%s-kubeproxy", controllerNamePrefix),
}
if err = kubeProxy.SetupWithManager(mgr); err != nil {
return reconcile.Result{}, err
@@ -303,6 +310,7 @@ func (m *Manager) Reconcile(ctx context.Context, request reconcile.Request) (res
GetTenantControlPlaneFunc: m.retrieveTenantControlPlane(tcpCtx, request),
Logger: mgr.GetLogger().WithName("coredns"),
TriggerChannel: make(chan event.GenericEvent),
ControllerName: fmt.Sprintf("%s-coredns", controllerNamePrefix),
}
if err = coreDNS.SetupWithManager(mgr); err != nil {
return reconcile.Result{}, err
@@ -315,6 +323,7 @@ func (m *Manager) Reconcile(ctx context.Context, request reconcile.Request) (res
Phase: resources.PhaseUploadConfigKubeadm,
},
TriggerChannel: make(chan event.GenericEvent),
ControllerName: fmt.Sprintf("%s-kubeadmconfig", controllerNamePrefix),
}
if err = uploadKubeadmConfig.SetupWithManager(mgr); err != nil {
return reconcile.Result{}, err
@@ -327,6 +336,7 @@ func (m *Manager) Reconcile(ctx context.Context, request reconcile.Request) (res
Phase: resources.PhaseUploadConfigKubelet,
},
TriggerChannel: make(chan event.GenericEvent),
ControllerName: fmt.Sprintf("%s-kubeletconfig", controllerNamePrefix),
}
if err = uploadKubeletConfig.SetupWithManager(mgr); err != nil {
return reconcile.Result{}, err
@@ -339,6 +349,7 @@ func (m *Manager) Reconcile(ctx context.Context, request reconcile.Request) (res
Phase: resources.PhaseBootstrapToken,
},
TriggerChannel: make(chan event.GenericEvent),
ControllerName: fmt.Sprintf("%s-bootstraptoken", controllerNamePrefix),
}
if err = bootstrapToken.SetupWithManager(mgr); err != nil {
return reconcile.Result{}, err
@@ -351,6 +362,7 @@ func (m *Manager) Reconcile(ctx context.Context, request reconcile.Request) (res
Phase: resources.PhaseClusterAdminRBAC,
},
TriggerChannel: make(chan event.GenericEvent),
ControllerName: fmt.Sprintf("%s-kubeadmrbac", controllerNamePrefix),
}
if err = kubeadmRbac.SetupWithManager(mgr); err != nil {
return reconcile.Result{}, err

View File

@@ -37,6 +37,7 @@ import (
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
"github.com/clastix/kamaji/controllers/finalizers"
"github.com/clastix/kamaji/controllers/utils"
controlplanebuilder "github.com/clastix/kamaji/internal/builders/controlplane"
"github.com/clastix/kamaji/internal/datastore"
kamajierrors "github.com/clastix/kamaji/internal/errors"
"github.com/clastix/kamaji/internal/resources"
@@ -157,6 +158,25 @@ func (r *TenantControlPlaneReconciler) Reconcile(ctx context.Context, req ctrl.R
}
defer dsConnection.Close()
dso, err := r.dataStoreOverride(ctx, tenantControlPlane)
if err != nil {
log.Error(err, "cannot retrieve the DataStoreOverrides for the given instance")
return ctrl.Result{}, err
}
dsoConnections := make(map[string]datastore.Connection, len(dso))
for _, ds := range dso {
dsoConnection, err := datastore.NewStorageConnection(ctx, r.Client, ds.DataStore)
if err != nil {
log.Error(err, "cannot generate the DataStoreOverride connection for the given instance")
return ctrl.Result{}, err
}
defer dsoConnection.Close()
dsoConnections[ds.Resource] = dsoConnection
}
if markedToBeDeleted && controllerutil.ContainsFinalizer(tenantControlPlane, finalizers.DatastoreFinalizer) {
log.Info("marked for deletion, performing clean-up")
@@ -183,17 +203,19 @@ func (r *TenantControlPlaneReconciler) Reconcile(ctx context.Context, req ctrl.R
}
groupResourceBuilderConfiguration := GroupResourceBuilderConfiguration{
client: r.Client,
log: log,
tcpReconcilerConfig: r.Config,
tenantControlPlane: *tenantControlPlane,
Connection: dsConnection,
DataStore: *ds,
KamajiNamespace: r.KamajiNamespace,
KamajiServiceAccount: r.KamajiServiceAccount,
KamajiService: r.KamajiService,
KamajiMigrateImage: r.KamajiMigrateImage,
DiscoveryClient: r.DiscoveryClient,
client: r.Client,
log: log,
tcpReconcilerConfig: r.Config,
tenantControlPlane: *tenantControlPlane,
Connection: dsConnection,
DataStore: *ds,
DataStoreOverrides: dso,
DataStoreOverriedsConnections: dsoConnections,
KamajiNamespace: r.KamajiNamespace,
KamajiServiceAccount: r.KamajiServiceAccount,
KamajiService: r.KamajiService,
KamajiMigrateImage: r.KamajiMigrateImage,
DiscoveryClient: r.DiscoveryClient,
}
registeredResources := GetResources(ctx, groupResourceBuilderConfiguration)
@@ -362,3 +384,21 @@ func (r *TenantControlPlaneReconciler) dataStore(ctx context.Context, tenantCont
return &ds, nil
}
func (r *TenantControlPlaneReconciler) dataStoreOverride(ctx context.Context, tenantControlPlane *kamajiv1alpha1.TenantControlPlane) ([]controlplanebuilder.DataStoreOverrides, error) {
datastores := make([]controlplanebuilder.DataStoreOverrides, 0, len(tenantControlPlane.Spec.DataStoreOverrides))
for _, dso := range tenantControlPlane.Spec.DataStoreOverrides {
var ds kamajiv1alpha1.DataStore
if err := r.Client.Get(ctx, k8stypes.NamespacedName{Name: dso.DataStore}, &ds); err != nil {
return nil, errors.Wrap(err, "cannot retrieve *kamajiv1alpha.DataStore object")
}
if ds.Spec.Driver != kamajiv1alpha1.EtcdDriver {
return nil, errors.New("DataStoreOverrides can only use ETCD driver")
}
datastores = append(datastores, controlplanebuilder.DataStoreOverrides{Resource: dso.Resource, DataStore: ds})
}
return datastores, nil
}

View File

@@ -0,0 +1,78 @@
# Datastore Overrides
Kamaji offers the possibility of having multiple ETCD clusters backing different resources of the k8s api server by configuring the [`--etcd-servers-overrides`](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-apiserver/#:~:text=%2D%2Detcd%2Dservers%2Doverrides%20strings) flag. This feature can be useful for massive clusters to store resources with high churn in a dedicated ETCD cluster.
## Install Datastores
Create a self-signed cert-manager `ClusterIssuer`.
```bash
echo 'apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: self-signed
spec:
selfSigned: {}
' | kubectl apply -f -
```
Install two Datastores, a primary and a secondary that will be used for `/events` resources.
```bash
helm install etcd-primary clastix/kamaji-etcd -n kamaji-etcd --create-namespace \
--set selfSignedCertificates.enabled=false \
--set certManager.enabled=true \
--set certManager.issuerRef.kind=ClusterIssuer \
--set certManager.issuerRef.name=self-signed
```
For the secondary Datastore, use the cert-manager CA created by the `etcd-primary` helm release.
```bash
helm install etcd-secondary clastix/kamaji-etcd -n kamaji-etcd --create-namespace \
--set selfSignedCertificates.enabled=false \
--set certManager.enabled=true \
--set certManager.ca.create=false \
--set certManager.ca.nameOverride=etcd-primary-kamaji-etcd-ca \
--set certManager.issuerRef.kind=ClusterIssuer \
--set certManager.issuerRef.name=self-signed
```
## Create a Tenant Control Plane
Using the `spec.dataStoreOverrides` field, Datastores different from the one used in `spec.dataStore` can be used to store specific resources.
```bash
echo 'apiVersion: kamaji.clastix.io/v1alpha1
kind: TenantControlPlane
metadata:
name: k8s-133
labels:
tenant.clastix.io: k8s-133
spec:
controlPlane:
deployment:
replicas: 2
service:
serviceType: LoadBalancer
kubernetes:
version: "v1.33.1"
kubelet:
cgroupfs: systemd
dataStore: etcd-primary-kamaji-etcd
dataStoreOverrides:
- resource: "/events" # Store events in the secondary ETCD
dataStore: etcd-secondary-kamaji-etcd
networkProfile:
port: 6443
addons:
coreDNS: {}
kubeProxy: {}
konnectivity:
server:
port: 8132
agent:
mode: DaemonSet
' | k apply -f -
```
## Considerations
Only built-in resources can be tagetted by `--etcd-servers-overrides`, it is currently not possible to target Custom Resources.

View File

@@ -0,0 +1,213 @@
# Gateway API Support
Kamaji provides built-in support for the [Gateway API](https://gateway-api.sigs.k8s.io/), allowing you to expose Tenant Control Planes using TLSRoute resources with SNI-based routing. This enables hostname-based routing to multiple Tenant Control Planes through a single Gateway resource, reducing the need for dedicated LoadBalancer services.
## Overview
Gateway API support in Kamaji automatically creates and manages TLSRoute resources for your Tenant Control Planes. When you configure a Gateway for a Tenant Control Plane, Kamaji automatically creates TLSRoutes for the Control Plane API Server. If konnectivity is enabled, a separate TLSRoute is created for it. Both TLSRoutes use the same hostname and Gateway resource, but route to different ports(listeners) using port-based routing and semantic `sectionName` values.
Therefore, the target `Gateway` resource must have right listener configurations (see the Gateway [example section](#gateway-resource-setup) below).
## How It Works
When you configure `spec.controlPlane.gateway` in a TenantControlPlane resource, Kamaji automatically:
1. **Creates a TLSRoute for the control plane** that routes for port 6443 (or `spec.networkProfile.port`) with sectionName `"kube-apiserver"`
2. **Creates a TLSRoute for Konnectivity** (if konnectivity addon is enabled) that routes for port 8132 (or `spec.addons.konnectivity.server.port`) with sectionName `"konnectivity-server"`
Both TLSRoutes:
- Use the same hostname from `spec.controlPlane.gateway.hostname`
- Reference the same parent Gateway resource via `parentRefs`
- The `port` and `sectionName` fields are set automatically by Kamaji
- Route to the appropriate Tenant Control Plane service
The Gateway resource must have listeners configured for both ports (6443 and 8132) to support both routes.
## Prerequisites
Before using Gateway API support, ensure:
1. **Gateway API CRDs are installed** in your cluster (Required CRDs: `GatewayClass`, `Gateway`, `TLSRoute`)
2. **A Gateway resource exists** with appropriate listeners configured:
- At minimum, listeners for ports 6443 (control plane) and 8132 (Konnectivity)
- TLS protocol with Passthrough mode
- Hostname pattern matching your Tenant Control Plane hostnames
3. **DNS is configured** to resolve your hostnames to the Gateway's external address
4. **Gateway controller is running** (e.g., Envoy Gateway, Istio Gateway, etc.)
## Configuration
### TenantControlPlane Gateway Configuration
Enable Gateway API mode by setting the `spec.controlPlane.gateway` field in your TenantControlPlane resource:
```yaml
apiVersion: kamaji.clastix.io/v1alpha1
kind: TenantControlPlane
metadata:
name: tcp-1
spec:
controlPlane:
# ... gateway configuration:
gateway:
hostname: "tcp1.cluster.dev"
parentRefs:
- name: gateway
namespace: default
additionalMetadata:
labels:
environment: production
annotations:
example.com/custom: "value"
# ... rest of the spec
deployment:
replicas: 1
service:
serviceType: ClusterIP
dataStore: default
kubernetes:
version: v1.29.0
kubelet:
cgroupfs: systemd
networkProfile:
port: 6443
certSANs:
- "c11.cluster.dev" # make sure to set this.
addons:
coreDNS: {}
kubeProxy: {}
konnectivity: {}
```
**Required fields:**
- `hostname`: The hostname that will be used for routing (must match Gateway listener hostname pattern)
- `parentRefs`: Array of Gateway references (name and namespace)
**Optional fields:**
- `additionalMetadata.labels`: Custom labels to add to TLSRoute resources
- `additionalMetadata.annotations`: Custom annotations to add to TLSRoute resources
!!! warning "Port and sectionName are set automatically"
Do not specify `port` or `sectionName` in `parentRefs`. Kamaji automatically sets these fields in TLSRoutes.
### Gateway Resource Setup
Your Gateway resource must have listeners configured for both the control plane and Konnectivity ports. Here's an example Gateway configuration:
```yaml
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
name: envoy-gw-class
spec:
controllerName: gateway.envoyproxy.io/gatewayclass-controller
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: gateway
namespace: default
spec:
gatewayClassName: envoy-gw-class
listeners:
- name: kube-apiserver
port: 6443
protocol: TLS
hostname: 'tcp1.cluster.dev'
tls:
mode: Passthrough
allowedRoutes:
kinds:
- group: gateway.networking.k8s.io
kind: TLSRoute
namespaces:
from: All
# if konnectivity addon is enabled:
- name: konnectivity-server
port: 8132
protocol: TLS
hostname: 'tcp1.cluster.dev'
tls:
mode: Passthrough
allowedRoutes:
kinds:
- group: gateway.networking.k8s.io
kind: TLSRoute
namespaces:
from: All
```
## Multiple Tenant Control Planes
You can use the same Gateway resource for multiple Tenant Control Planes by using different hostnames:
```yaml
# Gateway with wildcard hostname
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: gateway
spec:
listeners:
- hostname: '*.cluster.dev'
name: kube-apiserver
port: 6443
# ...
---
# Tenant Control Plane 1
apiVersion: kamaji.clastix.io/v1alpha1
kind: TenantControlPlane
metadata:
name: tcp-1
spec:
controlPlane:
gateway:
hostname: "tcp1.cluster.dev"
parentRefs:
- name: gateway
namespace: default
# ...
---
# Tenant Control Plane 2
apiVersion: kamaji.clastix.io/v1alpha1
kind: TenantControlPlane
metadata:
name: tcp-2
spec:
controlPlane:
gateway:
hostname: "tcp2.cluster.dev"
parentRefs:
- name: gateway
namespace: default
# ...
```
Each Tenant Control Plane will get its own TLSRoutes with the respective hostnames, all routing through the same Gateway resource.
You can check the Gateway status in the TenantControlPlane:
```bash
kubectl get tenantcontrolplane tcp-1 -o yaml
```
Look for the `status.kubernetesResources.gateway` and `status.addons.konnectivity.gateway` fields.
## Additional Resources
- [Gateway API Documentation](https://gateway-api.sigs.k8s.io/)
- [Quickstart with Envoy Gateway](https://gateway.envoyproxy.io/docs/tasks/quickstart/)

View File

@@ -28500,6 +28500,13 @@ Migration from one DataStore to another backed by the same Driver is possible. S
Migration from one DataStore to another backed by a different Driver is not supported.<br/>
</td>
<td>false</td>
</tr><tr>
<td><b><a href="#tenantcontrolplanespecdatastoreoverridesindex">dataStoreOverrides</a></b></td>
<td>[]object</td>
<td>
DataStoreOverride defines which kubernetes resources will be stored in dedicated datastores.<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>dataStoreSchema</b></td>
<td>string</td>
@@ -29034,7 +29041,8 @@ More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#cont
<td><b><a href="#tenantcontrolplanespeccontrolplanedeploymentadditionalcontainersindexresizepolicyindex">resizePolicy</a></b></td>
<td>[]object</td>
<td>
Resources resize policy for the container.<br/>
Resources resize policy for the container.
This field cannot be set on ephemeral containers.<br/>
</td>
<td>false</td>
</tr><tr>
@@ -31985,7 +31993,8 @@ More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#cont
<td><b><a href="#tenantcontrolplanespeccontrolplanedeploymentadditionalinitcontainersindexresizepolicyindex">resizePolicy</a></b></td>
<td>[]object</td>
<td>
Resources resize policy for the container.<br/>
Resources resize policy for the container.
This field cannot be set on ephemeral containers.<br/>
</td>
<td>false</td>
</tr><tr>
@@ -36443,7 +36452,7 @@ There are three important differences between dataSource and dataSourceRef:
<td>object</td>
<td>
resources represents the minimum resources the volume should have.
If RecoverVolumeExpansionFailure feature is enabled users are allowed to specify resource requirements
Users are allowed to specify resource requirements
that are lower than previous value but must still be higher than capacity recorded in the
status field of the claim.
More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#resources<br/>
@@ -36623,7 +36632,7 @@ Note that when a namespace is specified, a gateway.networking.k8s.io/ReferenceGr
resources represents the minimum resources the volume should have.
If RecoverVolumeExpansionFailure feature is enabled users are allowed to specify resource requirements
Users are allowed to specify resource requirements
that are lower than previous value but must still be higher than capacity recorded in the
status field of the claim.
More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#resources
@@ -38155,6 +38164,25 @@ longer than 24 hours.<br/>
<i>Format</i>: int32<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>userAnnotations</b></td>
<td>map[string]string</td>
<td>
userAnnotations allow pod authors to pass additional information to
the signer implementation. Kubernetes does not restrict or validate this
metadata in any way.
These values are copied verbatim into the `spec.unverifiedUserAnnotations` field of
the PodCertificateRequest objects that Kubelet creates.
Entries are subject to the same validation as object metadata annotations,
with the addition that all keys must be domain-prefixed. No restrictions
are placed on values, except an overall size limitation on the entire field.
Signers should document the keys and values they support. Signers should
deny requests that contain keys they do not recognize.<br/>
</td>
<td>false</td>
</tr></tbody>
</table>
@@ -41105,9 +41133,10 @@ If the key is empty, operator must be Exists; this combination means to match al
<td>string</td>
<td>
Operator represents a key's relationship to the value.
Valid operators are Exists and Equal. Defaults to Equal.
Valid operators are Exists, Equal, Lt, and Gt. Defaults to Equal.
Exists is equivalent to wildcard for value, so that a pod can
tolerate all taints of a particular category.<br/>
tolerate all taints of a particular category.
Lt and Gt perform numeric comparisons (requires feature gate TaintTolerationComparisonOperators).<br/>
</td>
<td>false</td>
</tr><tr>
@@ -41745,11 +41774,24 @@ Full reference available here: https://kubernetes.io/docs/reference/access-authn
<td>enum</td>
<td>
CGroupFS defines the cgroup driver for Kubelet
https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/configure-cgroup-driver/<br/>
https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/configure-cgroup-driver/
Deprecated: use ConfigurationJSONPatches.<br/>
<br/>
<i>Enum</i>: systemd, cgroupfs<br/>
</td>
<td>false</td>
</tr><tr>
<td><b><a href="#tenantcontrolplanespeckuberneteskubeletconfigurationjsonpatchesindex">configurationJSONPatches</a></b></td>
<td>[]object</td>
<td>
ConfigurationJSONPatches contains the RFC 6902 JSON patches to customise the kubeadm generate configuration,
useful to customise and mangling the configuration according to your needs;
e.g.: configuring the cgroup driver used by Kubelet is possible via the following patch:
[{"op": "replace", "path": "/cgroupDriver", "value": "systemd"}]<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>preferredAddressTypes</b></td>
<td>[]enum</td>
@@ -41765,6 +41807,54 @@ Default to InternalIP, ExternalIP, Hostname.<br/>
</table>
<span id="tenantcontrolplanespeckuberneteskubeletconfigurationjsonpatchesindex">`TenantControlPlane.spec.kubernetes.kubelet.configurationJSONPatches[index]`</span>
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody><tr>
<td><b>op</b></td>
<td>enum</td>
<td>
Op is the RFC 6902 JSON Patch operation.<br/>
<br/>
<i>Enum</i>: add, remove, replace, move, copy, test<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>path</b></td>
<td>string</td>
<td>
Path specifies the target location in the JSON document. Use "/" to separate keys; "-" for appending to arrays.<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>from</b></td>
<td>string</td>
<td>
From specifies the source location for move or copy operations.<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>value</b></td>
<td>JSON</td>
<td>
Value is the operation value to be used when Op is add, replace, test.<br/>
</td>
<td>false</td>
</tr></tbody>
</table>
<span id="tenantcontrolplanespecaddons">`TenantControlPlane.spec.addons`</span>
@@ -42002,9 +42092,10 @@ If the key is empty, operator must be Exists; this combination means to match al
<td>string</td>
<td>
Operator represents a key's relationship to the value.
Valid operators are Exists and Equal. Defaults to Equal.
Valid operators are Exists, Equal, Lt, and Gt. Defaults to Equal.
Exists is equivalent to wildcard for value, so that a pod can
tolerate all taints of a particular category.<br/>
tolerate all taints of a particular category.
Lt and Gt perform numeric comparisons (requires feature gate TaintTolerationComparisonOperators).<br/>
</td>
<td>false</td>
</tr><tr>
@@ -42214,6 +42305,38 @@ In case this value is set, kubeadm does not change automatically the version of
</table>
<span id="tenantcontrolplanespecdatastoreoverridesindex">`TenantControlPlane.spec.dataStoreOverrides[index]`</span>
DataStoreOverride defines which kubernetes resource will be stored in a dedicated datastore.
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody><tr>
<td><b>dataStore</b></td>
<td>string</td>
<td>
DataStore specifies the DataStore that should be used to store the Kubernetes data for the given Resource.<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>resource</b></td>
<td>string</td>
<td>
Resource specifies which kubernetes resource to target.<br/>
</td>
<td>false</td>
</tr></tbody>
</table>
<span id="tenantcontrolplanespecnetworkprofile">`TenantControlPlane.spec.networkProfile`</span>
@@ -42565,6 +42688,13 @@ KonnectivityStatus defines the status of Konnectivity as Addon.
<br/>
</td>
<td>false</td>
</tr><tr>
<td><b><a href="#tenantcontrolplanestatusaddonskonnectivitygateway">gateway</a></b></td>
<td>object</td>
<td>
KubernetesGatewayStatus defines the status for the Tenant Control Plane Gateway in the management cluster.<br/>
</td>
<td>false</td>
</tr><tr>
<td><b><a href="#tenantcontrolplanestatusaddonskonnectivitykubeconfig">kubeconfig</a></b></td>
<td>object</td>
@@ -42752,6 +42882,505 @@ CertificatePrivateKeyPairStatus defines the status.
</table>
<span id="tenantcontrolplanestatusaddonskonnectivitygateway">`TenantControlPlane.status.addons.konnectivity.gateway`</span>
KubernetesGatewayStatus defines the status for the Tenant Control Plane Gateway in the management cluster.
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody><tr>
<td><b><a href="#tenantcontrolplanestatusaddonskonnectivitygatewayparentsindex">parents</a></b></td>
<td>[]object</td>
<td>
Parents is a list of parent resources (usually Gateways) that are
associated with the route, and the status of the route with respect to
each parent. When this route attaches to a parent, the controller that
manages the parent must add an entry to this list when the controller
first sees the route and should update the entry as appropriate when the
route or gateway is modified.
Note that parent references that cannot be resolved by an implementation
of this API will not be added to this list. Implementations of this API
can only populate Route status for the Gateways/parent resources they are
responsible for.
A maximum of 32 Gateways will be represented in this list. An empty list
means the route has not been attached to any Gateway.
<gateway:util:excludeFromCRD>
Notes for implementors:
While parents is not a listType `map`, this is due to the fact that the
list key is not scalar, and Kubernetes is unable to represent this.
Parent status MUST be considered to be namespaced by the combination of
the parentRef and controllerName fields, and implementations should keep
the following rules in mind when updating this status:
* Implementations MUST update only entries that have a matching value of
`controllerName` for that implementation.
* Implementations MUST NOT update entries with non-matching `controllerName`
fields.
* Implementations MUST treat each `parentRef`` in the Route separately and
update its status based on the relationship with that parent.
* Implementations MUST perform a read-modify-write cycle on this field
before modifying it. That is, when modifying this field, implementations
must be confident they have fetched the most recent version of this field,
and ensure that changes they make are on that recent version.
</gateway:util:excludeFromCRD><br/>
</td>
<td>true</td>
</tr><tr>
<td><b><a href="#tenantcontrolplanestatusaddonskonnectivitygatewayaccesspointsindex">accessPoints</a></b></td>
<td>[]object</td>
<td>
A list of valid access points that the route exposes.<br/>
</td>
<td>false</td>
</tr><tr>
<td><b><a href="#tenantcontrolplanestatusaddonskonnectivitygatewayrouteref">routeRef</a></b></td>
<td>object</td>
<td>
Reference to the route created for this tenant.<br/>
</td>
<td>false</td>
</tr></tbody>
</table>
<span id="tenantcontrolplanestatusaddonskonnectivitygatewayparentsindex">`TenantControlPlane.status.addons.konnectivity.gateway.parents[index]`</span>
RouteParentStatus describes the status of a route with respect to an
associated Parent.
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody><tr>
<td><b><a href="#tenantcontrolplanestatusaddonskonnectivitygatewayparentsindexconditionsindex">conditions</a></b></td>
<td>[]object</td>
<td>
Conditions describes the status of the route with respect to the Gateway.
Note that the route's availability is also subject to the Gateway's own
status conditions and listener status.
If the Route's ParentRef specifies an existing Gateway that supports
Routes of this kind AND that Gateway's controller has sufficient access,
then that Gateway's controller MUST set the "Accepted" condition on the
Route, to indicate whether the route has been accepted or rejected by the
Gateway, and why.
A Route MUST be considered "Accepted" if at least one of the Route's
rules is implemented by the Gateway.
There are a number of cases where the "Accepted" condition may not be set
due to lack of controller visibility, that includes when:
* The Route refers to a nonexistent parent.
* The Route is of a type that the controller does not support.
* The Route is in a namespace the controller does not have access to.
<gateway:util:excludeFromCRD>
Notes for implementors:
Conditions are a listType `map`, which means that they function like a
map with a key of the `type` field _in the k8s apiserver_.
This means that implementations must obey some rules when updating this
section.
* Implementations MUST perform a read-modify-write cycle on this field
before modifying it. That is, when modifying this field, implementations
must be confident they have fetched the most recent version of this field,
and ensure that changes they make are on that recent version.
* Implementations MUST NOT remove or reorder Conditions that they are not
directly responsible for. For example, if an implementation sees a Condition
with type `special.io/SomeField`, it MUST NOT remove, change or update that
Condition.
* Implementations MUST always _merge_ changes into Conditions of the same Type,
rather than creating more than one Condition of the same Type.
* Implementations MUST always update the `observedGeneration` field of the
Condition to the `metadata.generation` of the Gateway at the time of update creation.
* If the `observedGeneration` of a Condition is _greater than_ the value the
implementation knows about, then it MUST NOT perform the update on that Condition,
but must wait for a future reconciliation and status update. (The assumption is that
the implementation's copy of the object is stale and an update will be re-triggered
if relevant.)
</gateway:util:excludeFromCRD><br/>
</td>
<td>true</td>
</tr><tr>
<td><b>controllerName</b></td>
<td>string</td>
<td>
ControllerName is a domain/path string that indicates the name of the
controller that wrote this status. This corresponds with the
controllerName field on GatewayClass.
Example: "example.net/gateway-controller".
The format of this field is DOMAIN "/" PATH, where DOMAIN and PATH are
valid Kubernetes names
(https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names).
Controllers MUST populate this field when writing status. Controllers should ensure that
entries to status populated with their ControllerName are cleaned up when they are no
longer necessary.<br/>
</td>
<td>true</td>
</tr><tr>
<td><b><a href="#tenantcontrolplanestatusaddonskonnectivitygatewayparentsindexparentref">parentRef</a></b></td>
<td>object</td>
<td>
ParentRef corresponds with a ParentRef in the spec that this
RouteParentStatus struct describes the status of.<br/>
</td>
<td>true</td>
</tr></tbody>
</table>
<span id="tenantcontrolplanestatusaddonskonnectivitygatewayparentsindexconditionsindex">`TenantControlPlane.status.addons.konnectivity.gateway.parents[index].conditions[index]`</span>
Condition contains details for one aspect of the current state of this API Resource.
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody><tr>
<td><b>lastTransitionTime</b></td>
<td>string</td>
<td>
lastTransitionTime is the last time the condition transitioned from one status to another.
This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable.<br/>
<br/>
<i>Format</i>: date-time<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>message</b></td>
<td>string</td>
<td>
message is a human readable message indicating details about the transition.
This may be an empty string.<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>reason</b></td>
<td>string</td>
<td>
reason contains a programmatic identifier indicating the reason for the condition's last transition.
Producers of specific condition types may define expected values and meanings for this field,
and whether the values are considered a guaranteed API.
The value should be a CamelCase string.
This field may not be empty.<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>status</b></td>
<td>enum</td>
<td>
status of the condition, one of True, False, Unknown.<br/>
<br/>
<i>Enum</i>: True, False, Unknown<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>type</b></td>
<td>string</td>
<td>
type of condition in CamelCase or in foo.example.com/CamelCase.<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>observedGeneration</b></td>
<td>integer</td>
<td>
observedGeneration represents the .metadata.generation that the condition was set based upon.
For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date
with respect to the current state of the instance.<br/>
<br/>
<i>Format</i>: int64<br/>
<i>Minimum</i>: 0<br/>
</td>
<td>false</td>
</tr></tbody>
</table>
<span id="tenantcontrolplanestatusaddonskonnectivitygatewayparentsindexparentref">`TenantControlPlane.status.addons.konnectivity.gateway.parents[index].parentRef`</span>
ParentRef corresponds with a ParentRef in the spec that this
RouteParentStatus struct describes the status of.
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody><tr>
<td><b>name</b></td>
<td>string</td>
<td>
Name is the name of the referent.
Support: Core<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>group</b></td>
<td>string</td>
<td>
Group is the group of the referent.
When unspecified, "gateway.networking.k8s.io" is inferred.
To set the core API group (such as for a "Service" kind referent),
Group must be explicitly set to "" (empty string).
Support: Core<br/>
<br/>
<i>Default</i>: gateway.networking.k8s.io<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>kind</b></td>
<td>string</td>
<td>
Kind is kind of the referent.
There are two kinds of parent resources with "Core" support:
* Gateway (Gateway conformance profile)
* Service (Mesh conformance profile, ClusterIP Services only)
Support for other resources is Implementation-Specific.<br/>
<br/>
<i>Default</i>: Gateway<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>namespace</b></td>
<td>string</td>
<td>
Namespace is the namespace of the referent. When unspecified, this refers
to the local namespace of the Route.
Note that there are specific rules for ParentRefs which cross namespace
boundaries. Cross-namespace references are only valid if they are explicitly
allowed by something in the namespace they are referring to. For example:
Gateway has the AllowedRoutes field, and ReferenceGrant provides a
generic way to enable any other kind of cross-namespace reference.
<gateway:experimental:description>
ParentRefs from a Route to a Service in the same namespace are "producer"
routes, which apply default routing rules to inbound connections from
any namespace to the Service.
ParentRefs from a Route to a Service in a different namespace are
"consumer" routes, and these routing rules are only applied to outbound
connections originating from the same namespace as the Route, for which
the intended destination of the connections are a Service targeted as a
ParentRef of the Route.
</gateway:experimental:description>
Support: Core<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>port</b></td>
<td>integer</td>
<td>
Port is the network port this Route targets. It can be interpreted
differently based on the type of parent resource.
When the parent resource is a Gateway, this targets all listeners
listening on the specified port that also support this kind of Route(and
select this Route). It's not recommended to set `Port` unless the
networking behaviors specified in a Route must apply to a specific port
as opposed to a listener(s) whose port(s) may be changed. When both Port
and SectionName are specified, the name and port of the selected listener
must match both specified values.
<gateway:experimental:description>
When the parent resource is a Service, this targets a specific port in the
Service spec. When both Port (experimental) and SectionName are specified,
the name and port of the selected port must match both specified values.
</gateway:experimental:description>
Implementations MAY choose to support other parent resources.
Implementations supporting other types of parent resources MUST clearly
document how/if Port is interpreted.
For the purpose of status, an attachment is considered successful as
long as the parent resource accepts it partially. For example, Gateway
listeners can restrict which Routes can attach to them by Route kind,
namespace, or hostname. If 1 of 2 Gateway listeners accept attachment
from the referencing Route, the Route MUST be considered successfully
attached. If no Gateway listeners accept attachment from this Route,
the Route MUST be considered detached from the Gateway.
Support: Extended<br/>
<br/>
<i>Format</i>: int32<br/>
<i>Minimum</i>: 1<br/>
<i>Maximum</i>: 65535<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>sectionName</b></td>
<td>string</td>
<td>
SectionName is the name of a section within the target resource. In the
following resources, SectionName is interpreted as the following:
* Gateway: Listener name. When both Port (experimental) and SectionName
are specified, the name and port of the selected listener must match
both specified values.
* Service: Port name. When both Port (experimental) and SectionName
are specified, the name and port of the selected listener must match
both specified values.
Implementations MAY choose to support attaching Routes to other resources.
If that is the case, they MUST clearly document how SectionName is
interpreted.
When unspecified (empty string), this will reference the entire resource.
For the purpose of status, an attachment is considered successful if at
least one section in the parent resource accepts it. For example, Gateway
listeners can restrict which Routes can attach to them by Route kind,
namespace, or hostname. If 1 of 2 Gateway listeners accept attachment from
the referencing Route, the Route MUST be considered successfully
attached. If no Gateway listeners accept attachment from this Route, the
Route MUST be considered detached from the Gateway.
Support: Core<br/>
</td>
<td>false</td>
</tr></tbody>
</table>
<span id="tenantcontrolplanestatusaddonskonnectivitygatewayaccesspointsindex">`TenantControlPlane.status.addons.konnectivity.gateway.accessPoints[index]`</span>
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody><tr>
<td><b>port</b></td>
<td>integer</td>
<td>
<br/>
<br/>
<i>Format</i>: int32<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>type</b></td>
<td>string</td>
<td>
AddressType defines how a network address is represented as a text string.
This may take two possible forms:
* A predefined CamelCase string identifier (currently limited to `IPAddress` or `Hostname`)
* A domain-prefixed string identifier (like `acme.io/CustomAddressType`)
Values `IPAddress` and `Hostname` have Extended support.
The `NamedAddress` value has been deprecated in favor of implementation
specific domain-prefixed strings.
All other values, including domain-prefixed values have Implementation-specific support,
which are used in implementation-specific behaviors. Support for additional
predefined CamelCase identifiers may be added in future releases.<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>value</b></td>
<td>string</td>
<td>
<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>urls</b></td>
<td>[]string</td>
<td>
<br/>
</td>
<td>false</td>
</tr></tbody>
</table>
<span id="tenantcontrolplanestatusaddonskonnectivitygatewayrouteref">`TenantControlPlane.status.addons.konnectivity.gateway.routeRef`</span>
Reference to the route created for this tenant.
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody><tr>
<td><b>name</b></td>
<td>string</td>
<td>
Name of the referent.
This field is effectively required, but due to backwards compatibility is
allowed to be empty. Instances of this type with an empty value here are
almost certainly wrong.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names<br/>
<br/>
<i>Default</i>: <br/>
</td>
<td>false</td>
</tr></tbody>
</table>
<span id="tenantcontrolplanestatusaddonskonnectivitykubeconfig">`TenantControlPlane.status.addons.konnectivity.kubeconfig`</span>
@@ -43976,7 +44605,7 @@ newest ReplicaSet.<br/>
Total number of terminating pods targeted by this deployment. Terminating pods have a non-null
.metadata.deletionTimestamp and have not yet reached the Failed or Succeeded .status.phase.
This is an alpha field. Enable DeploymentReplicaSetTerminatingReplicas to be able to use this field.<br/>
This is a beta field and requires enabling DeploymentReplicaSetTerminatingReplicas feature (enabled by default).<br/>
<br/>
<i>Format</i>: int32<br/>
</td>

View File

@@ -76,9 +76,11 @@ nav:
- guides/pausing.md
- guides/write-permissions.md
- guides/datastore-migration.md
- guides/datastore-overrides.md
- guides/gitops.md
- guides/console.md
- guides/kubeconfig-generator.md
- guides/gateway-api.md
- guides/upgrade.md
- guides/monitoring.md
- guides/terraform.md

View File

@@ -4,10 +4,12 @@
package e2e
import (
"context"
"testing"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes/scheme"
"k8s.io/client-go/rest"
pointer "k8s.io/utils/ptr"
@@ -68,9 +70,39 @@ var _ = BeforeSuite(func() {
k8sClient, err = client.New(cfg, client.Options{Scheme: scheme.Scheme})
Expect(err).NotTo(HaveOccurred())
Expect(k8sClient).NotTo(BeNil())
By("creating GatewayClass for Gateway API tests")
gatewayClass := &gatewayv1.GatewayClass{
ObjectMeta: metav1.ObjectMeta{
Name: "envoy-gw-class",
},
Spec: gatewayv1.GatewayClassSpec{
ControllerName: "gateway.envoyproxy.io/gatewayclass-controller",
},
}
Expect(k8sClient.Create(context.Background(), gatewayClass)).NotTo(HaveOccurred())
By("creating Gateway with kube-apiserver and konnectivity-server listeners")
CreateGatewayWithListeners("test-gateway", "default", "envoy-gw-class", "*.example.com")
})
var _ = AfterSuite(func() {
By("deleting Gateway resources")
gateway := &gatewayv1.Gateway{
ObjectMeta: metav1.ObjectMeta{
Name: "test-gateway",
Namespace: "default",
},
}
_ = k8sClient.Delete(context.Background(), gateway)
gatewayClass := &gatewayv1.GatewayClass{
ObjectMeta: metav1.ObjectMeta{
Name: "envoy-gw-class",
},
}
_ = k8sClient.Delete(context.Background(), gatewayClass)
By("tearing down the test environment")
err := testEnv.Stop()
Expect(err).NotTo(HaveOccurred())

View File

@@ -0,0 +1,68 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package e2e
import (
"context"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
pointer "k8s.io/utils/ptr"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
)
var _ = Describe("Deploy a TenantControlPlane resource with DataStoreOverrides", func() {
// Fill TenantControlPlane object
tcp := &kamajiv1alpha1.TenantControlPlane{
ObjectMeta: metav1.ObjectMeta{
Name: "tcp-datastore-overrides",
Namespace: "default",
},
Spec: kamajiv1alpha1.TenantControlPlaneSpec{
DataStore: "etcd-primary",
DataStoreOverrides: []kamajiv1alpha1.DataStoreOverride{{
Resource: "/events",
DataStore: "etcd-secondary",
}},
ControlPlane: kamajiv1alpha1.ControlPlane{
Deployment: kamajiv1alpha1.DeploymentSpec{
Replicas: pointer.To(int32(1)),
},
Service: kamajiv1alpha1.ServiceSpec{
ServiceType: "ClusterIP",
},
},
NetworkProfile: kamajiv1alpha1.NetworkProfileSpec{
Address: "172.18.0.2",
},
Kubernetes: kamajiv1alpha1.KubernetesSpec{
Version: "v1.23.6",
Kubelet: kamajiv1alpha1.KubeletSpec{
CGroupFS: "cgroupfs",
},
AdmissionControllers: kamajiv1alpha1.AdmissionControllers{
"LimitRanger",
"ResourceQuota",
},
},
Addons: kamajiv1alpha1.AddonsSpec{},
},
}
// Create a TenantControlPlane resource into the cluster
JustBeforeEach(func() {
Expect(k8sClient.Create(context.Background(), tcp)).NotTo(HaveOccurred())
})
// Delete the TenantControlPlane resource after test is finished
JustAfterEach(func() {
Expect(k8sClient.Delete(context.Background(), tcp)).Should(Succeed())
})
// Check if TenantControlPlane resource has been created
It("Should be Ready", func() {
StatusMustEqualTo(tcp, kamajiv1alpha1.VersionReady)
})
})

View File

@@ -0,0 +1,151 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package e2e
import (
"context"
"fmt"
"time"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/types"
pointer "k8s.io/utils/ptr"
gatewayv1 "sigs.k8s.io/gateway-api/apis/v1"
gatewayv1alpha2 "sigs.k8s.io/gateway-api/apis/v1alpha2"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
)
var _ = Describe("Deploy a TenantControlPlane with Gateway API and Konnectivity", func() {
var tcp *kamajiv1alpha1.TenantControlPlane
JustBeforeEach(func() {
tcp = &kamajiv1alpha1.TenantControlPlane{
ObjectMeta: metav1.ObjectMeta{
Name: "tcp-konnectivity-gateway",
Namespace: "default",
},
Spec: kamajiv1alpha1.TenantControlPlaneSpec{
ControlPlane: kamajiv1alpha1.ControlPlane{
Deployment: kamajiv1alpha1.DeploymentSpec{
Replicas: pointer.To(int32(1)),
},
Service: kamajiv1alpha1.ServiceSpec{
ServiceType: "ClusterIP",
},
Gateway: &kamajiv1alpha1.GatewaySpec{
Hostname: gatewayv1.Hostname("tcp-gateway-konnectivity.example.com"),
AdditionalMetadata: kamajiv1alpha1.AdditionalMetadata{
Labels: map[string]string{
"test.kamaji.io/gateway": "true",
},
Annotations: map[string]string{
"test.kamaji.io/created-by": "e2e-test",
},
},
GatewayParentRefs: []gatewayv1.ParentReference{
{
Name: "test-gateway",
},
},
},
},
NetworkProfile: kamajiv1alpha1.NetworkProfileSpec{
Address: "172.18.0.4",
},
Kubernetes: kamajiv1alpha1.KubernetesSpec{
Version: "v1.28.0",
Kubelet: kamajiv1alpha1.KubeletSpec{
CGroupFS: "cgroupfs",
},
AdmissionControllers: kamajiv1alpha1.AdmissionControllers{
"LimitRanger",
"ResourceQuota",
},
},
Addons: kamajiv1alpha1.AddonsSpec{
Konnectivity: &kamajiv1alpha1.KonnectivitySpec{
KonnectivityServerSpec: kamajiv1alpha1.KonnectivityServerSpec{
Port: 8132,
},
},
},
},
}
Expect(k8sClient.Create(context.Background(), tcp)).NotTo(HaveOccurred())
})
JustAfterEach(func() {
Expect(k8sClient.Delete(context.Background(), tcp)).Should(Succeed())
// Wait for the object to be completely deleted
Eventually(func() bool {
err := k8sClient.Get(context.Background(), types.NamespacedName{
Name: tcp.Name,
Namespace: tcp.Namespace,
}, &kamajiv1alpha1.TenantControlPlane{})
return err != nil // Returns true when object is not found (deleted)
}).WithTimeout(time.Minute).Should(BeTrue())
})
It("Should be Ready", func() {
StatusMustEqualTo(tcp, kamajiv1alpha1.VersionReady)
})
It("Should create Konnectivity TLSRoute with correct sectionName", func() {
Eventually(func() error {
route := &gatewayv1alpha2.TLSRoute{}
if err := k8sClient.Get(context.Background(), types.NamespacedName{
Name: tcp.Name + "-konnectivity",
Namespace: tcp.Namespace,
}, route); err != nil {
return err
}
if len(route.Spec.ParentRefs) == 0 {
return fmt.Errorf("parentRefs is empty")
}
if route.Spec.ParentRefs[0].SectionName == nil {
return fmt.Errorf("sectionName is nil")
}
if *route.Spec.ParentRefs[0].SectionName != gatewayv1.SectionName("konnectivity-server") {
return fmt.Errorf("expected sectionName 'konnectivity-server', got '%s'", *route.Spec.ParentRefs[0].SectionName)
}
return nil
}).WithTimeout(time.Minute).Should(Succeed())
})
It("Should use same hostname for both TLSRoutes", func() {
Eventually(func() error {
controlPlaneRoute := &gatewayv1alpha2.TLSRoute{}
if err := k8sClient.Get(context.Background(), types.NamespacedName{
Name: tcp.Name,
Namespace: tcp.Namespace,
}, controlPlaneRoute); err != nil {
return err
}
konnectivityRoute := &gatewayv1alpha2.TLSRoute{}
if err := k8sClient.Get(context.Background(), types.NamespacedName{
Name: tcp.Name + "-konnectivity",
Namespace: tcp.Namespace,
}, konnectivityRoute); err != nil {
return err
}
if len(controlPlaneRoute.Spec.Hostnames) == 0 || len(konnectivityRoute.Spec.Hostnames) == 0 {
return fmt.Errorf("hostnames are empty")
}
if controlPlaneRoute.Spec.Hostnames[0] != konnectivityRoute.Spec.Hostnames[0] {
return fmt.Errorf("hostnames do not match: control plane '%s', konnectivity '%s'",
controlPlaneRoute.Spec.Hostnames[0], konnectivityRoute.Spec.Hostnames[0])
}
return nil
}).WithTimeout(time.Minute).Should(Succeed())
})
})

View File

@@ -5,6 +5,7 @@ package e2e
import (
"context"
"fmt"
"time"
. "github.com/onsi/ginkgo/v2"
@@ -89,14 +90,39 @@ var _ = Describe("Deploy a TenantControlPlane with Gateway API", func() {
StatusMustEqualTo(tcp, kamajiv1alpha1.VersionReady)
})
It("Should create TLSRoute resource", func() {
It("Should create control plane TLSRoute with correct sectionName", func() {
Eventually(func() error {
route := &gatewayv1alpha2.TLSRoute{}
// TODO: Check ownership.
return k8sClient.Get(context.Background(), types.NamespacedName{
if err := k8sClient.Get(context.Background(), types.NamespacedName{
Name: tcp.Name,
Namespace: tcp.Namespace,
}, route)
}, route); err != nil {
return err
}
if len(route.Spec.ParentRefs) == 0 {
return fmt.Errorf("parentRefs is empty")
}
if route.Spec.ParentRefs[0].SectionName == nil {
return fmt.Errorf("sectionName is nil")
}
if *route.Spec.ParentRefs[0].SectionName != gatewayv1.SectionName("kube-apiserver") {
return fmt.Errorf("expected sectionName 'kube-apiserver', got '%s'", *route.Spec.ParentRefs[0].SectionName)
}
return nil
}).WithTimeout(time.Minute).Should(Succeed())
})
It("Should not create Konnectivity TLSRoute", func() {
// Verify Konnectivity route is not created
Consistently(func() error {
route := &gatewayv1alpha2.TLSRoute{}
return k8sClient.Get(context.Background(), types.NamespacedName{
Name: tcp.Name + "-konnectivity",
Namespace: tcp.Namespace,
}, route)
}, 10*time.Second, time.Second).Should(HaveOccurred())
})
})

View File

@@ -19,7 +19,9 @@ import (
"k8s.io/apimachinery/pkg/types"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/util/retry"
pointer "k8s.io/utils/ptr"
"sigs.k8s.io/controller-runtime/pkg/client"
gatewayv1 "sigs.k8s.io/gateway-api/apis/v1"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
)
@@ -209,3 +211,60 @@ func ScaleTenantControlPlane(tcp *kamajiv1alpha1.TenantControlPlane, replicas in
})
Expect(err).To(Succeed())
}
// CreateGatewayWithListeners creates a Gateway with both kube-apiserver and konnectivity-server listeners.
func CreateGatewayWithListeners(gatewayName, namespace, gatewayClassName, hostname string) {
GinkgoHelper()
gateway := &gatewayv1.Gateway{
ObjectMeta: metav1.ObjectMeta{
Name: gatewayName,
Namespace: namespace,
},
Spec: gatewayv1.GatewaySpec{
GatewayClassName: gatewayv1.ObjectName(gatewayClassName),
Listeners: []gatewayv1.Listener{
{
Name: "kube-apiserver",
Port: 6443,
Protocol: gatewayv1.TLSProtocolType,
Hostname: pointer.To(gatewayv1.Hostname(hostname)),
TLS: &gatewayv1.ListenerTLSConfig{
Mode: pointer.To(gatewayv1.TLSModeType("Passthrough")),
},
AllowedRoutes: &gatewayv1.AllowedRoutes{
Namespaces: &gatewayv1.RouteNamespaces{
From: pointer.To(gatewayv1.NamespacesFromAll),
},
Kinds: []gatewayv1.RouteGroupKind{
{
Group: pointer.To(gatewayv1.Group("gateway.networking.k8s.io")),
Kind: "TLSRoute",
},
},
},
},
{
Name: "konnectivity-server",
Port: 8132,
Protocol: gatewayv1.TLSProtocolType,
Hostname: pointer.To(gatewayv1.Hostname(hostname)),
TLS: &gatewayv1.ListenerTLSConfig{
Mode: pointer.To(gatewayv1.TLSModeType("Passthrough")),
},
AllowedRoutes: &gatewayv1.AllowedRoutes{
Namespaces: &gatewayv1.RouteNamespaces{
From: pointer.To(gatewayv1.NamespacesFromAll),
},
Kinds: []gatewayv1.RouteGroupKind{
{
Group: pointer.To(gatewayv1.Group("gateway.networking.k8s.io")),
Kind: "TLSRoute",
},
},
},
},
},
},
}
Expect(k8sClient.Create(context.Background(), gateway)).NotTo(HaveOccurred())
}

View File

@@ -59,7 +59,7 @@ var _ = Describe("starting a kind worker with kubeadm", func() {
Port: 31443,
},
Kubernetes: kamajiv1alpha1.KubernetesSpec{
Version: "v1.29.0",
Version: "v1.35.0",
Kubelet: kamajiv1alpha1.KubeletSpec{
CGroupFS: "cgroupfs",
},

127
go.mod
View File

@@ -1,12 +1,13 @@
module github.com/clastix/kamaji
go 1.24.1
go 1.25.0
require (
github.com/JamesStewy/go-mysqldump v0.2.2
github.com/blang/semver v3.5.1+incompatible
github.com/clastix/kamaji-telemetry v1.0.0
github.com/docker/docker v28.5.2+incompatible
github.com/evanphx/json-patch/v5 v5.9.11
github.com/go-logr/logr v1.4.3
github.com/go-pg/pg/v10 v10.15.0
github.com/go-sql-driver/mysql v1.9.3
@@ -14,28 +15,29 @@ require (
github.com/google/uuid v1.6.0
github.com/json-iterator/go v1.1.12
github.com/juju/mutex/v2 v2.0.0
github.com/nats-io/nats.go v1.47.0
github.com/onsi/ginkgo/v2 v2.27.2
github.com/onsi/gomega v1.38.2
github.com/nats-io/nats.go v1.48.0
github.com/onsi/ginkgo/v2 v2.27.5
github.com/onsi/gomega v1.39.0
github.com/pkg/errors v0.9.1
github.com/prometheus/client_golang v1.23.2
github.com/prometheus/client_model v0.6.2
github.com/spf13/cobra v1.10.2
github.com/spf13/pflag v1.0.10
github.com/spf13/viper v1.21.0
github.com/testcontainers/testcontainers-go v0.40.0
go.etcd.io/etcd/api/v3 v3.6.6
go.etcd.io/etcd/client/v3 v3.6.6
go.etcd.io/etcd/api/v3 v3.6.7
go.etcd.io/etcd/client/v3 v3.6.7
go.uber.org/automaxprocs v1.6.0
gomodules.xyz/jsonpatch/v2 v2.5.0
k8s.io/api v0.34.1
k8s.io/apimachinery v0.34.1
k8s.io/apiserver v0.34.1
k8s.io/client-go v0.34.1
k8s.io/api v0.35.0
k8s.io/apiextensions-apiserver v0.34.1
k8s.io/apimachinery v0.35.0
k8s.io/client-go v0.35.0
k8s.io/cluster-bootstrap v0.0.0
k8s.io/klog/v2 v2.130.1
k8s.io/kubelet v0.0.0
k8s.io/kubernetes v1.34.2
k8s.io/utils v0.0.0-20250820121507-0af2bda4dd1d
k8s.io/kubernetes v1.35.0
k8s.io/utils v0.0.0-20251002143259-bc988d571ff4
sigs.k8s.io/controller-runtime v0.22.4
sigs.k8s.io/gateway-api v1.4.1
)
@@ -59,7 +61,7 @@ require (
github.com/containerd/log v0.1.0 // indirect
github.com/containerd/platforms v0.2.1 // indirect
github.com/coredns/caddy v1.1.1 // indirect
github.com/coredns/corefile-migration v1.0.26 // indirect
github.com/coredns/corefile-migration v1.0.29 // indirect
github.com/coreos/go-semver v0.3.1 // indirect
github.com/coreos/go-systemd/v22 v22.5.0 // indirect
github.com/cpuguy83/dockercfg v0.3.2 // indirect
@@ -70,7 +72,6 @@ require (
github.com/ebitengine/purego v0.8.4 // indirect
github.com/emicklei/go-restful/v3 v3.13.0 // indirect
github.com/evanphx/json-patch v5.7.0+incompatible // indirect
github.com/evanphx/json-patch/v5 v5.9.11 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/fsnotify/fsnotify v1.9.0 // indirect
github.com/fxamacker/cbor/v2 v2.9.0 // indirect
@@ -125,9 +126,9 @@ require (
github.com/peterbourgon/diskv v2.0.1+incompatible // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c // indirect
github.com/prometheus/client_model v0.6.2 // indirect
github.com/prometheus/common v0.66.1 // indirect
github.com/prometheus/procfs v0.17.0 // indirect
github.com/robfig/cron/v3 v3.0.1 // indirect
github.com/sagikazarmark/locafero v0.11.0 // indirect
github.com/shirou/gopsutil/v4 v4.25.6 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
@@ -147,10 +148,10 @@ require (
github.com/x448/float16 v0.8.4 // indirect
github.com/xlab/treeprint v1.2.0 // indirect
github.com/yusufpapurcu/wmi v1.2.4 // indirect
go.etcd.io/etcd/client/pkg/v3 v3.6.6 // indirect
go.etcd.io/etcd/client/pkg/v3 v3.6.7 // indirect
go.opentelemetry.io/auto/sdk v1.1.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.60.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.58.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 // indirect
go.opentelemetry.io/otel v1.37.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.34.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.34.0 // indirect
@@ -160,19 +161,19 @@ require (
go.opentelemetry.io/proto/otlp v1.5.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.uber.org/zap v1.27.0 // indirect
go.yaml.in/yaml/v2 v2.4.2 // indirect
go.yaml.in/yaml/v2 v2.4.3 // indirect
go.yaml.in/yaml/v3 v3.0.4 // indirect
golang.org/x/crypto v0.43.0 // indirect
golang.org/x/crypto v0.45.0 // indirect
golang.org/x/exp v0.0.0-20240719175910-8a7402abbf56 // indirect
golang.org/x/mod v0.28.0 // indirect
golang.org/x/net v0.45.0 // indirect
golang.org/x/mod v0.29.0 // indirect
golang.org/x/net v0.47.0 // indirect
golang.org/x/oauth2 v0.30.0 // indirect
golang.org/x/sync v0.17.0 // indirect
golang.org/x/sys v0.37.0 // indirect
golang.org/x/term v0.36.0 // indirect
golang.org/x/text v0.30.0 // indirect
golang.org/x/sync v0.18.0 // indirect
golang.org/x/sys v0.38.0 // indirect
golang.org/x/term v0.37.0 // indirect
golang.org/x/text v0.31.0 // indirect
golang.org/x/time v0.12.0 // indirect
golang.org/x/tools v0.37.0 // indirect
golang.org/x/tools v0.38.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20250707201910-8d1bb00bc6a7 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250826171959-ef028d996bc1 // indirect
google.golang.org/grpc v1.75.1 // indirect
@@ -182,18 +183,18 @@ require (
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/natefinch/lumberjack.v2 v2.2.1 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
k8s.io/apiextensions-apiserver v0.34.1 // indirect
k8s.io/apiserver v0.35.0 // indirect
k8s.io/cli-runtime v0.0.0 // indirect
k8s.io/cloud-provider v0.0.0 // indirect
k8s.io/component-base v0.34.1 // indirect
k8s.io/component-helpers v0.34.0 // indirect
k8s.io/controller-manager v0.34.0 // indirect
k8s.io/cri-api v0.34.0 // indirect
k8s.io/component-base v0.35.0 // indirect
k8s.io/component-helpers v0.35.0 // indirect
k8s.io/controller-manager v0.35.0 // indirect
k8s.io/cri-api v0.35.0 // indirect
k8s.io/cri-client v0.0.0 // indirect
k8s.io/kms v0.34.0 // indirect
k8s.io/kube-openapi v0.0.0-20250814151709-d7b6acb124c3 // indirect
k8s.io/kms v0.35.0 // indirect
k8s.io/kube-openapi v0.0.0-20250910181357-589584f1c912 // indirect
k8s.io/kube-proxy v0.0.0 // indirect
k8s.io/system-validators v1.10.2 // indirect
k8s.io/system-validators v1.12.1 // indirect
mellium.im/sasl v0.3.1 // indirect
sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.31.2 // indirect
sigs.k8s.io/json v0.0.0-20250730193827-2d320260d730 // indirect
@@ -205,35 +206,35 @@ require (
)
replace (
k8s.io/api => k8s.io/api v0.34.0
k8s.io/apiextensions-apiserver => k8s.io/apiextensions-apiserver v0.34.0
k8s.io/apimachinery => k8s.io/apimachinery v0.34.0
k8s.io/apiserver => k8s.io/apiserver v0.34.0
k8s.io/cli-runtime => k8s.io/cli-runtime v0.34.0
k8s.io/client-go => k8s.io/client-go v0.34.0
k8s.io/cloud-provider => k8s.io/cloud-provider v0.34.0
k8s.io/cluster-bootstrap => k8s.io/cluster-bootstrap v0.34.0
k8s.io/code-generator => k8s.io/code-generator v0.34.0
k8s.io/component-base => k8s.io/component-base v0.34.0
k8s.io/component-helpers => k8s.io/component-helpers v0.34.0
k8s.io/controller-manager => k8s.io/controller-manager v0.34.0
k8s.io/cri-api => k8s.io/cri-api v0.34.0
k8s.io/cri-client => k8s.io/cri-client v0.34.0
k8s.io/csi-translation-lib => k8s.io/csi-translation-lib v0.34.0
k8s.io/dynamic-resource-allocation => k8s.io/dynamic-resource-allocation v0.34.0
k8s.io/endpointslice => k8s.io/endpointslice v0.34.0
k8s.io/externaljwt => k8s.io/externaljwt v0.34.0
k8s.io/kube-aggregator => k8s.io/kube-aggregator v0.34.0
k8s.io/kube-controller-manager => k8s.io/kube-controller-manager v0.34.0
k8s.io/kube-proxy => k8s.io/kube-proxy v0.34.0
k8s.io/kube-scheduler => k8s.io/kube-scheduler v0.34.0
k8s.io/kubectl => k8s.io/kubectl v0.34.0
k8s.io/kubelet => k8s.io/kubelet v0.34.0
k8s.io/legacy-cloud-providers => k8s.io/legacy-cloud-providers v0.34.0
k8s.io/metrics => k8s.io/metrics v0.34.0
k8s.io/mount-utils => k8s.io/mount-utils v0.34.0
k8s.io/pod-security-admission => k8s.io/pod-security-admission v0.34.0
k8s.io/sample-apiserver => k8s.io/sample-apiserver v0.34.0
k8s.io/api => k8s.io/api v0.35.0
k8s.io/apiextensions-apiserver => k8s.io/apiextensions-apiserver v0.35.0
k8s.io/apimachinery => k8s.io/apimachinery v0.35.0
k8s.io/apiserver => k8s.io/apiserver v0.35.0
k8s.io/cli-runtime => k8s.io/cli-runtime v0.35.0
k8s.io/client-go => k8s.io/client-go v0.35.0
k8s.io/cloud-provider => k8s.io/cloud-provider v0.35.0
k8s.io/cluster-bootstrap => k8s.io/cluster-bootstrap v0.35.0
k8s.io/code-generator => k8s.io/code-generator v0.35.0
k8s.io/component-base => k8s.io/component-base v0.35.0
k8s.io/component-helpers => k8s.io/component-helpers v0.35.0
k8s.io/controller-manager => k8s.io/controller-manager v0.35.0
k8s.io/cri-api => k8s.io/cri-api v0.35.0
k8s.io/cri-client => k8s.io/cri-client v0.35.0
k8s.io/csi-translation-lib => k8s.io/csi-translation-lib v0.35.0
k8s.io/dynamic-resource-allocation => k8s.io/dynamic-resource-allocation v0.35.0
k8s.io/endpointslice => k8s.io/endpointslice v0.35.0
k8s.io/externaljwt => k8s.io/externaljwt v0.35.0
k8s.io/kube-aggregator => k8s.io/kube-aggregator v0.35.0
k8s.io/kube-controller-manager => k8s.io/kube-controller-manager v0.35.0
k8s.io/kube-proxy => k8s.io/kube-proxy v0.35.0
k8s.io/kube-scheduler => k8s.io/kube-scheduler v0.35.0
k8s.io/kubectl => k8s.io/kubectl v0.35.0
k8s.io/kubelet => k8s.io/kubelet v0.35.0
k8s.io/legacy-cloud-providers => k8s.io/legacy-cloud-providers v0.35.0
k8s.io/metrics => k8s.io/metrics v0.35.0
k8s.io/mount-utils => k8s.io/mount-utils v0.35.0
k8s.io/pod-security-admission => k8s.io/pod-security-admission v0.35.0
k8s.io/sample-apiserver => k8s.io/sample-apiserver v0.35.0
)
replace github.com/JamesStewy/go-mysqldump => github.com/vtoma/go-mysqldump v1.0.0

162
go.sum
View File

@@ -40,8 +40,8 @@ github.com/containerd/platforms v0.2.1 h1:zvwtM3rz2YHPQsF2CHYM8+KtB5dvhISiXh5ZpS
github.com/containerd/platforms v0.2.1/go.mod h1:XHCb+2/hzowdiut9rkudds9bE5yJ7npe7dG/wG+uFPw=
github.com/coredns/caddy v1.1.1 h1:2eYKZT7i6yxIfGP3qLJoJ7HAsDJqYB+X68g4NYjSrE0=
github.com/coredns/caddy v1.1.1/go.mod h1:A6ntJQlAWuQfFlsd9hvigKbo2WS0VUs2l1e2F+BawD4=
github.com/coredns/corefile-migration v1.0.26 h1:xiiEkVB1Dwolb24pkeDUDBfygV9/XsOSq79yFCrhptY=
github.com/coredns/corefile-migration v1.0.26/go.mod h1:56DPqONc3njpVPsdilEnfijCwNGC3/kTJLl7i7SPavY=
github.com/coredns/corefile-migration v1.0.29 h1:g4cPYMXXDDs9uLE2gFYrJaPBuUAR07eEMGyh9JBE13w=
github.com/coredns/corefile-migration v1.0.29/go.mod h1:56DPqONc3njpVPsdilEnfijCwNGC3/kTJLl7i7SPavY=
github.com/coreos/go-semver v0.3.1 h1:yi21YpKnrx1gt5R+la8n5WgS0kCrsPp33dmEyHReZr4=
github.com/coreos/go-semver v0.3.1/go.mod h1:irMmmIw/7yzSRPWryHsK7EYSg09caPQL03VsM8rvUec=
github.com/coreos/go-systemd/v22 v22.5.0 h1:RrqgGjYQKalulkV8NGVIfkXQf6YYmOyiJKk8iXXhfZs=
@@ -243,8 +243,8 @@ github.com/morikuni/aec v1.0.0 h1:nP9CBfwrvYnBRgY6qfDQkygYDmYwOilePFkwzv4dU8A=
github.com/morikuni/aec v1.0.0/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7PXmsc=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/nats-io/nats.go v1.47.0 h1:YQdADw6J/UfGUd2Oy6tn4Hq6YHxCaJrVKayxxFqYrgM=
github.com/nats-io/nats.go v1.47.0/go.mod h1:iRWIPokVIFbVijxuMQq4y9ttaBTMe0SFdlZfMDd+33g=
github.com/nats-io/nats.go v1.48.0 h1:pSFyXApG+yWU/TgbKCjmm5K4wrHu86231/w84qRVR+U=
github.com/nats-io/nats.go v1.48.0/go.mod h1:iRWIPokVIFbVijxuMQq4y9ttaBTMe0SFdlZfMDd+33g=
github.com/nats-io/nkeys v0.4.11 h1:q44qGV008kYd9W1b1nEBkNzvnWxtRSQ7A8BoqRrcfa0=
github.com/nats-io/nkeys v0.4.11/go.mod h1:szDimtgmfOi9n25JpfIdGw12tZFYXqhGxjhVxsatHVE=
github.com/nats-io/nuid v1.0.1 h1:5iA8DT8V7q8WK2EScv2padNa/rTESc1KdnPw4TC2paw=
@@ -253,10 +253,10 @@ github.com/nxadm/tail v1.4.4 h1:DQuhQpB1tVlglWS2hLQ5OV6B5r8aGxSrPc5Qo6uTN78=
github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A=
github.com/onsi/ginkgo v1.14.2 h1:8mVmC9kjFFmA8H4pKMUhcblgifdkOIXPvbhN1T36q1M=
github.com/onsi/ginkgo v1.14.2/go.mod h1:iSB4RoI2tjJc9BBv4NKIKWKya62Rps+oPG/Lv9klQyY=
github.com/onsi/ginkgo/v2 v2.27.2 h1:LzwLj0b89qtIy6SSASkzlNvX6WktqurSHwkk2ipF/Ns=
github.com/onsi/ginkgo/v2 v2.27.2/go.mod h1:ArE1D/XhNXBXCBkKOLkbsb2c81dQHCRcF5zwn/ykDRo=
github.com/onsi/gomega v1.38.2 h1:eZCjf2xjZAqe+LeWvKb5weQ+NcPwX84kqJ0cZNxok2A=
github.com/onsi/gomega v1.38.2/go.mod h1:W2MJcYxRGV63b418Ai34Ud0hEdTVXq9NW9+Sx6uXf3k=
github.com/onsi/ginkgo/v2 v2.27.5 h1:ZeVgZMx2PDMdJm/+w5fE/OyG6ILo1Y3e+QX4zSR0zTE=
github.com/onsi/ginkgo/v2 v2.27.5/go.mod h1:ArE1D/XhNXBXCBkKOLkbsb2c81dQHCRcF5zwn/ykDRo=
github.com/onsi/gomega v1.39.0 h1:y2ROC3hKFmQZJNFeGAMeHZKkjBL65mIZcvrLQBF9k6Q=
github.com/onsi/gomega v1.39.0/go.mod h1:ZCU1pkQcXDO5Sl9/VVEGlDyp+zm0m1cmeG5TOzLgdh4=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
github.com/opencontainers/image-spec v1.1.1 h1:y0fUlFfIZhPF1W537XOLg0/fcx6zcHCJwooC2xJA040=
@@ -282,6 +282,8 @@ github.com/prometheus/common v0.66.1 h1:h5E0h5/Y8niHc5DlaLlWLArTQI7tMrsfQjHV+d9Z
github.com/prometheus/common v0.66.1/go.mod h1:gcaUsgf3KfRSwHY4dIMXLPV0K/Wg1oZ8+SbZk/HH/dA=
github.com/prometheus/procfs v0.17.0 h1:FuLQ+05u4ZI+SS/w9+BWEM2TXiHKsUQ9TADiRH7DuK0=
github.com/prometheus/procfs v0.17.0/go.mod h1:oPQLaDAMRbA+u8H5Pbfq+dl3VDAvHxMUOVhe0wYB2zw=
github.com/robfig/cron/v3 v3.0.1 h1:WdRxkvbJztn8LMz/QEvLN5sBU+xKpSqwwUO1Pjr4qDs=
github.com/robfig/cron/v3 v3.0.1/go.mod h1:eQICP3HwyT7UooqI/z+Ov+PtYAWygg1TEWWzGIFLtro=
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
@@ -365,26 +367,26 @@ github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9dec
github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
github.com/yusufpapurcu/wmi v1.2.4 h1:zFUKzehAFReQwLys1b/iSMl+JQGSCSjtVqQn9bBrPo0=
github.com/yusufpapurcu/wmi v1.2.4/go.mod h1:SBZ9tNy3G9/m5Oi98Zks0QjeHVDvuK0qfxQmPyzfmi0=
go.etcd.io/bbolt v1.4.2 h1:IrUHp260R8c+zYx/Tm8QZr04CX+qWS5PGfPdevhdm1I=
go.etcd.io/bbolt v1.4.2/go.mod h1:Is8rSHO/b4f3XigBC0lL0+4FwAQv3HXEEIgFMuKHceM=
go.etcd.io/etcd/api/v3 v3.6.6 h1:mcaMp3+7JawWv69p6QShYWS8cIWUOl32bFLb6qf8pOQ=
go.etcd.io/etcd/api/v3 v3.6.6/go.mod h1:f/om26iXl2wSkcTA1zGQv8reJRSLVdoEBsi4JdfMrx4=
go.etcd.io/etcd/client/pkg/v3 v3.6.6 h1:uoqgzSOv2H9KlIF5O1Lsd8sW+eMLuV6wzE3q5GJGQNs=
go.etcd.io/etcd/client/pkg/v3 v3.6.6/go.mod h1:YngfUVmvsvOJ2rRgStIyHsKtOt9SZI2aBJrZiWJhCbI=
go.etcd.io/etcd/client/v3 v3.6.6 h1:G5z1wMf5B9SNexoxOHUGBaULurOZPIgGPsW6CN492ec=
go.etcd.io/etcd/client/v3 v3.6.6/go.mod h1:36Qv6baQ07znPR3+n7t+Rk5VHEzVYPvFfGmfF4wBHV8=
go.etcd.io/etcd/pkg/v3 v3.6.4 h1:fy8bmXIec1Q35/jRZ0KOes8vuFxbvdN0aAFqmEfJZWA=
go.etcd.io/etcd/pkg/v3 v3.6.4/go.mod h1:kKcYWP8gHuBRcteyv6MXWSN0+bVMnfgqiHueIZnKMtE=
go.etcd.io/etcd/server/v3 v3.6.4 h1:LsCA7CzjVt+8WGrdsnh6RhC0XqCsLkBly3ve5rTxMAU=
go.etcd.io/etcd/server/v3 v3.6.4/go.mod h1:aYCL/h43yiONOv0QIR82kH/2xZ7m+IWYjzRmyQfnCAg=
go.etcd.io/bbolt v1.4.3 h1:dEadXpI6G79deX5prL3QRNP6JB8UxVkqo4UPnHaNXJo=
go.etcd.io/bbolt v1.4.3/go.mod h1:tKQlpPaYCVFctUIgFKFnAlvbmB3tpy1vkTnDWohtc0E=
go.etcd.io/etcd/api/v3 v3.6.7 h1:7BNJ2gQmc3DNM+9cRkv7KkGQDayElg8x3X+tFDYS+E0=
go.etcd.io/etcd/api/v3 v3.6.7/go.mod h1:xJ81TLj9hxrYYEDmXTeKURMeY3qEDN24hqe+q7KhbnI=
go.etcd.io/etcd/client/pkg/v3 v3.6.7 h1:vvzgyozz46q+TyeGBuFzVuI53/yd133CHceNb/AhBVs=
go.etcd.io/etcd/client/pkg/v3 v3.6.7/go.mod h1:2IVulJ3FZ/czIGl9T4lMF1uxzrhRahLqe+hSgy+Kh7Q=
go.etcd.io/etcd/client/v3 v3.6.7 h1:9WqA5RpIBtdMxAy1ukXLAdtg2pAxNqW5NUoO2wQrE6U=
go.etcd.io/etcd/client/v3 v3.6.7/go.mod h1:2XfROY56AXnUqGsvl+6k29wrwsSbEh1lAouQB1vHpeE=
go.etcd.io/etcd/pkg/v3 v3.6.5 h1:byxWB4AqIKI4SBmquZUG1WGtvMfMaorXFoCcFbVeoxM=
go.etcd.io/etcd/pkg/v3 v3.6.5/go.mod h1:uqrXrzmMIJDEy5j00bCqhVLzR5jEJIwDp5wTlLwPGOU=
go.etcd.io/etcd/server/v3 v3.6.5 h1:4RbUb1Bd4y1WkBHmuF+cZII83JNQMuNXzyjwigQ06y0=
go.etcd.io/etcd/server/v3 v3.6.5/go.mod h1:PLuhyVXz8WWRhzXDsl3A3zv/+aK9e4A9lpQkqawIaH0=
go.etcd.io/raft/v3 v3.6.0 h1:5NtvbDVYpnfZWcIHgGRk9DyzkBIXOi8j+DDp1IcnUWQ=
go.etcd.io/raft/v3 v3.6.0/go.mod h1:nLvLevg6+xrVtHUmVaTcTz603gQPHfh7kUAwV6YpfGo=
go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA=
go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A=
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.60.0 h1:x7wzEgXfnzJcHDwStJT+mxOz4etr2EcexjqhBvmoakw=
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.60.0/go.mod h1:rg+RlpR5dKwaS95IyyZqj5Wd4E13lk/msnTS0Xl9lJM=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.58.0 h1:yd02MEjBdJkG3uabWP9apV+OuWRIXGDuJEUJbOHmCFU=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.58.0/go.mod h1:umTcuxiv1n/s/S6/c2AT/g2CQ7u5C59sHDNmfSwgz7Q=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 h1:F7Jx+6hwnZ41NSFTO5q4LYDtJRXBf2PD0rNBkeB/lus=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0/go.mod h1:UHB22Z8QsdRDrnAtX4PntOl36ajSxcdUMt1sF7Y6E7Q=
go.opentelemetry.io/otel v1.37.0 h1:9zhNfelUvx0KBfu/gb+ZgeAfAgtWrfHJZcAqFC228wQ=
go.opentelemetry.io/otel v1.37.0/go.mod h1:ehE/umFRLnuLa/vSccNq9oS1ErUlkkK71gMcN34UG8I=
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.34.0 h1:OeNbIYk/2C15ckl7glBlOBp5+WlYsOElzTNmiPW/x60=
@@ -411,37 +413,37 @@ go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
go.uber.org/zap v1.27.0 h1:aJMhYGrd5QSmlpLMr2MftRKl7t8J8PTZPA732ud/XR8=
go.uber.org/zap v1.27.0/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=
go.yaml.in/yaml/v2 v2.4.2 h1:DzmwEr2rDGHl7lsFgAHxmNz/1NlQ7xLIrlN2h5d1eGI=
go.yaml.in/yaml/v2 v2.4.2/go.mod h1:081UH+NErpNdqlCXm3TtEran0rJZGxAYx9hb/ELlsPU=
go.yaml.in/yaml/v2 v2.4.3 h1:6gvOSjQoTB3vt1l+CU+tSyi/HOjfOjRLJ4YwYZGwRO0=
go.yaml.in/yaml/v2 v2.4.3/go.mod h1:zSxWcmIDjOzPXpjlTTbAsKokqkDNAVtZO0WOMiT90s8=
go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.43.0 h1:dduJYIi3A3KOfdGOHX8AVZ/jGiyPa3IbBozJ5kNuE04=
golang.org/x/crypto v0.43.0/go.mod h1:BFbav4mRNlXJL4wNeejLpWxB7wMbc79PdRGhWKncxR0=
golang.org/x/crypto v0.45.0 h1:jMBrvKuj23MTlT0bQEOBcAE0mjg8mK9RXFhRH6nyF3Q=
golang.org/x/crypto v0.45.0/go.mod h1:XTGrrkGJve7CYK7J8PEww4aY7gM3qMCElcJQ8n8JdX4=
golang.org/x/exp v0.0.0-20240719175910-8a7402abbf56 h1:2dVuKD2vS7b0QIHQbpyTISPd0LeHDbnYEryqj5Q1ug8=
golang.org/x/exp v0.0.0-20240719175910-8a7402abbf56/go.mod h1:M4RDyNAINzryxdtnbRXRL/OHtkFuWGRjvuhBJpk2IlY=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.28.0 h1:gQBtGhjxykdjY9YhZpSlZIsbnaE2+PgjfLWUQTnoZ1U=
golang.org/x/mod v0.28.0/go.mod h1:yfB/L0NOf/kmEbXjzCPOx1iK1fRutOydrCMsqRhEBxI=
golang.org/x/mod v0.29.0 h1:HV8lRxZC4l2cr3Zq1LvtOsi/ThTgWnUk/y64QSs8GwA=
golang.org/x/mod v0.29.0/go.mod h1:NyhrlYXJ2H4eJiRy/WDBO6HMqZQ6q9nk4JzS3NuCK+w=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM=
golang.org/x/net v0.45.0 h1:RLBg5JKixCy82FtLJpeNlVM0nrSqpCRYzVU1n8kj0tM=
golang.org/x/net v0.45.0/go.mod h1:ECOoLqd5U3Lhyeyo/QDCEVQ4sNgYsqvCZ722XogGieY=
golang.org/x/net v0.47.0 h1:Mx+4dIFzqraBXUugkia1OOvlD6LemFo1ALMHjrXDOhY=
golang.org/x/net v0.47.0/go.mod h1:/jNxtkgq5yWUGYkaZGqo27cfGZ1c5Nen03aYrrKpVRU=
golang.org/x/oauth2 v0.30.0 h1:dnDm7JmhM45NNpd8FDDeLhK6FwqbOf4MLCM9zb1BOHI=
golang.org/x/oauth2 v0.30.0/go.mod h1:B++QgG3ZKulg6sRPGD/mqlHQs5rB3Ml9erfeDY7xKlU=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.17.0 h1:l60nONMj9l5drqw6jlhIELNv9I0A4OFgRsG9k2oT9Ug=
golang.org/x/sync v0.17.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
golang.org/x/sync v0.18.0 h1:kr88TuHDroi+UVf+0hZnirlk8o8T+4MrK6mr60WkH/I=
golang.org/x/sync v0.18.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190916202348-b4ddaad3f8a3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@@ -455,15 +457,15 @@ golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.11.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.27.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.37.0 h1:fdNQudmxPjkdUTPnLn5mdQv7Zwvbvpaxqs831goi9kQ=
golang.org/x/sys v0.37.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/sys v0.38.0 h1:3yZWxaJjBmCWXqhN1qh02AkOnCQ1poK6oF+a7xWL6Gc=
golang.org/x/sys v0.38.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.36.0 h1:zMPR+aF8gfksFprF/Nc/rd1wRS1EI6nDBGyWAvDzx2Q=
golang.org/x/term v0.36.0/go.mod h1:Qu394IJq6V6dCBRgwqshf3mPF85AqzYEzofzRdZkWss=
golang.org/x/term v0.37.0 h1:8EGAD0qCmHYZg6J17DvsMy9/wJ7/D/4pV/wfnld5lTU=
golang.org/x/term v0.37.0/go.mod h1:5pB4lxRNYYVZuTLmy8oR2BH8dflOR+IbTYFD8fi3254=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.30.0 h1:yznKA/E9zq54KzlzBEAWn1NXSQ8DIp/NYMy88xJjl4k=
golang.org/x/text v0.30.0/go.mod h1:yDdHFIX9t+tORqspjENWgzaCVXgk0yYnYuSZ8UzzBVM=
golang.org/x/text v0.31.0 h1:aC8ghyu4JhP8VojJ2lEHBnochRno1sgL6nEi9WGFGMM=
golang.org/x/text v0.31.0/go.mod h1:tKRAlv61yKIjGGHX/4tP1LTbc13YSec1pxVEWXzfoeM=
golang.org/x/time v0.12.0 h1:ScB/8o8olJvc+CQPWrK3fPZNfh7qgwCrY0zJmoEQLSE=
golang.org/x/time v0.12.0/go.mod h1:CDIdPxbZBQxdj6cxyCIdrNogrJKMJ7pr37NYpMcMDSg=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
@@ -471,8 +473,8 @@ golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtn
golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.1.1/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.37.0 h1:DVSRzp7FwePZW356yEAChSdNcQo6Nsp+fex1SUW09lE=
golang.org/x/tools v0.37.0/go.mod h1:MBN5QPQtLMHVdvsbtarmTNukZDdgwdwlO5qGacAzF0w=
golang.org/x/tools v0.38.0 h1:Hx2Xv8hISq8Lm16jvBZ2VQf+RLmbd7wVUsALibYI/IQ=
golang.org/x/tools v0.38.0/go.mod h1:yEsQ/d/YK8cjh0L6rZlY8tgtlKiBNTL14pGDJPJpYQs=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
@@ -511,48 +513,48 @@ gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gotest.tools/v3 v3.5.2 h1:7koQfIKdy+I8UTetycgUqXWSDwpgv193Ka+qRsmBY8Q=
gotest.tools/v3 v3.5.2/go.mod h1:LtdLGcnqToBH83WByAAi/wiwSFCArdFIUV/xxN4pcjA=
k8s.io/api v0.34.0 h1:L+JtP2wDbEYPUeNGbeSa/5GwFtIA662EmT2YSLOkAVE=
k8s.io/api v0.34.0/go.mod h1:YzgkIzOOlhl9uwWCZNqpw6RJy9L2FK4dlJeayUoydug=
k8s.io/apiextensions-apiserver v0.34.0 h1:B3hiB32jV7BcyKcMU5fDaDxk882YrJ1KU+ZSkA9Qxoc=
k8s.io/apiextensions-apiserver v0.34.0/go.mod h1:hLI4GxE1BDBy9adJKxUxCEHBGZtGfIg98Q+JmTD7+g0=
k8s.io/apimachinery v0.34.0 h1:eR1WO5fo0HyoQZt1wdISpFDffnWOvFLOOeJ7MgIv4z0=
k8s.io/apimachinery v0.34.0/go.mod h1:/GwIlEcWuTX9zKIg2mbw0LRFIsXwrfoVxn+ef0X13lw=
k8s.io/apiserver v0.34.0 h1:Z51fw1iGMqN7uJ1kEaynf2Aec1Y774PqU+FVWCFV3Jg=
k8s.io/apiserver v0.34.0/go.mod h1:52ti5YhxAvewmmpVRqlASvaqxt0gKJxvCeW7ZrwgazQ=
k8s.io/cli-runtime v0.34.0 h1:N2/rUlJg6TMEBgtQ3SDRJwa8XyKUizwjlOknT1mB2Cw=
k8s.io/cli-runtime v0.34.0/go.mod h1:t/skRecS73Piv+J+FmWIQA2N2/rDjdYSQzEE67LUUs8=
k8s.io/client-go v0.34.0 h1:YoWv5r7bsBfb0Hs2jh8SOvFbKzzxyNo0nSb0zC19KZo=
k8s.io/client-go v0.34.0/go.mod h1:ozgMnEKXkRjeMvBZdV1AijMHLTh3pbACPvK7zFR+QQY=
k8s.io/cloud-provider v0.34.0 h1:OgrNE+WSgfvDBQf6WS9qFM7Xr37bc0Og5kkL4hyWDmU=
k8s.io/cloud-provider v0.34.0/go.mod h1:JbMa0t6JIGDMLI7Py6bdp9TN6cfuHrWGq+E/X+Ljkmo=
k8s.io/cluster-bootstrap v0.34.0 h1:fWH6cUXbocLYMtWuONVwQ8ayqdEWlyvu25gedMTYTDk=
k8s.io/cluster-bootstrap v0.34.0/go.mod h1:ZpbQwB+CDTYZIjDKM6Hnt081s0xswcFrlhW7mHVNc7k=
k8s.io/component-base v0.34.0 h1:bS8Ua3zlJzapklsB1dZgjEJuJEeHjj8yTu1gxE2zQX8=
k8s.io/component-base v0.34.0/go.mod h1:RSCqUdvIjjrEm81epPcjQ/DS+49fADvGSCkIP3IC6vg=
k8s.io/component-helpers v0.34.0 h1:5T7P9XGMoUy1JDNKzHf0p/upYbeUf8ZaSf9jbx0QlIo=
k8s.io/component-helpers v0.34.0/go.mod h1:kaOyl5tdtnymriYcVZg4uwDBe2d1wlIpXyDkt6sVnt4=
k8s.io/controller-manager v0.34.0 h1:oCHoqS8dcFp7zDSu7HUvTpakq3isSxil3GprGGlJMsE=
k8s.io/controller-manager v0.34.0/go.mod h1:XFto21U+Mm9BT8r/Jd5E4tHCGtwjKAUFOuDcqaj2VK0=
k8s.io/cri-api v0.34.0 h1:erzXelLqzDbNdryR7eVqxmR/1JfQeurE9U+HdKTgSpU=
k8s.io/cri-api v0.34.0/go.mod h1:4qVUjidMg7/Z9YGZpqIDygbkPWkg3mkS1PvOx/kpHTE=
k8s.io/cri-client v0.34.0 h1:tLZro2oYinVKS5CaMtCASLmOacqVlwoHPSs9e7sBFWI=
k8s.io/cri-client v0.34.0/go.mod h1:KkGaUJWMvCdpSTf15Wiqtf3WKl3qjcvkBcMApPCqpxQ=
k8s.io/api v0.35.0 h1:iBAU5LTyBI9vw3L5glmat1njFK34srdLmktWwLTprlY=
k8s.io/api v0.35.0/go.mod h1:AQ0SNTzm4ZAczM03QH42c7l3bih1TbAXYo0DkF8ktnA=
k8s.io/apiextensions-apiserver v0.35.0 h1:3xHk2rTOdWXXJM+RDQZJvdx0yEOgC0FgQ1PlJatA5T4=
k8s.io/apiextensions-apiserver v0.35.0/go.mod h1:E1Ahk9SADaLQ4qtzYFkwUqusXTcaV2uw3l14aqpL2LU=
k8s.io/apimachinery v0.35.0 h1:Z2L3IHvPVv/MJ7xRxHEtk6GoJElaAqDCCU0S6ncYok8=
k8s.io/apimachinery v0.35.0/go.mod h1:jQCgFZFR1F4Ik7hvr2g84RTJSZegBc8yHgFWKn//hns=
k8s.io/apiserver v0.35.0 h1:CUGo5o+7hW9GcAEF3x3usT3fX4f9r8xmgQeCBDaOgX4=
k8s.io/apiserver v0.35.0/go.mod h1:QUy1U4+PrzbJaM3XGu2tQ7U9A4udRRo5cyxkFX0GEds=
k8s.io/cli-runtime v0.35.0 h1:PEJtYS/Zr4p20PfZSLCbY6YvaoLrfByd6THQzPworUE=
k8s.io/cli-runtime v0.35.0/go.mod h1:VBRvHzosVAoVdP3XwUQn1Oqkvaa8facnokNkD7jOTMY=
k8s.io/client-go v0.35.0 h1:IAW0ifFbfQQwQmga0UdoH0yvdqrbwMdq9vIFEhRpxBE=
k8s.io/client-go v0.35.0/go.mod h1:q2E5AAyqcbeLGPdoRB+Nxe3KYTfPce1Dnu1myQdqz9o=
k8s.io/cloud-provider v0.35.0 h1:syiBCQbKh2gho/S1BkIl006Dc44pV8eAtGZmv5NMe7M=
k8s.io/cloud-provider v0.35.0/go.mod h1:7grN+/Nt5Hf7tnSGPT3aErt4K7aQpygyCrGpbrQbzNc=
k8s.io/cluster-bootstrap v0.35.0 h1:VXnil8zw+FikqvytJYLB8wcvjxbUCyqMkiC//k426Y0=
k8s.io/cluster-bootstrap v0.35.0/go.mod h1:X6sjEjVUFSfFNIzJ6VAIuwwh2QiDtsVX1xZgcGX4gD8=
k8s.io/component-base v0.35.0 h1:+yBrOhzri2S1BVqyVSvcM3PtPyx5GUxCK2tinZz1G94=
k8s.io/component-base v0.35.0/go.mod h1:85SCX4UCa6SCFt6p3IKAPej7jSnF3L8EbfSyMZayJR0=
k8s.io/component-helpers v0.35.0 h1:wcXv7HJRksgVjM4VlXJ1CNFBpyDHruRI99RrBtrJceA=
k8s.io/component-helpers v0.35.0/go.mod h1:ahX0m/LTYmu7fL3W8zYiIwnQ/5gT28Ex4o2pymF63Co=
k8s.io/controller-manager v0.35.0 h1:KteodmfVIRzfZ3RDaxhnHb72rswBxEngvdL9vuZOA9A=
k8s.io/controller-manager v0.35.0/go.mod h1:1bVuPNUG6/dpWpevsJpXioS0E0SJnZ7I/Wqc9Awyzm4=
k8s.io/cri-api v0.35.0 h1:fxLSKyJHqbyCSUsg1rW4DRpmjSEM/elZ1GXzYTSLoDQ=
k8s.io/cri-api v0.35.0/go.mod h1:Cnt29u/tYl1Se1cBRL30uSZ/oJ5TaIp4sZm1xDLvcMc=
k8s.io/cri-client v0.35.0 h1:U1K4bteO93yioUS38804ybN+kWaon9zrzVtB37I3fCs=
k8s.io/cri-client v0.35.0/go.mod h1:XG5GkuuSpxvungsJVzW58NyWBoGSQhMMJmE5c66m9N8=
k8s.io/klog/v2 v2.130.1 h1:n9Xl7H1Xvksem4KFG4PYbdQCQxqc/tTUyrgXaOhHSzk=
k8s.io/klog/v2 v2.130.1/go.mod h1:3Jpz1GvMt720eyJH1ckRHK1EDfpxISzJ7I9OYgaDtPE=
k8s.io/kms v0.34.0 h1:u+/rcxQ3Jr7gC9AY5nXuEnBcGEB7ZOIJ9cdLdyHyEjQ=
k8s.io/kms v0.34.0/go.mod h1:s1CFkLG7w9eaTYvctOxosx88fl4spqmixnNpys0JAtM=
k8s.io/kube-openapi v0.0.0-20250814151709-d7b6acb124c3 h1:liMHz39T5dJO1aOKHLvwaCjDbf07wVh6yaUlTpunnkE=
k8s.io/kube-openapi v0.0.0-20250814151709-d7b6acb124c3/go.mod h1:UZ2yyWbFTpuhSbFhv24aGNOdoRdJZgsIObGBUaYVsts=
k8s.io/kube-proxy v0.34.0 h1:gU7MVbJHiXyPX8bXnod4bANtSC7rZSKkkLmM8gUqwT4=
k8s.io/kube-proxy v0.34.0/go.mod h1:tfwI8dCKm5Q0r+aVIbrq/aC36Kk936w2LZu8/rvJzWI=
k8s.io/kubelet v0.34.0 h1:1nZt1Q6Kfx7xCaTS9vnqR9sjZDxf3cRSQkAFCczULmc=
k8s.io/kubelet v0.34.0/go.mod h1:NqbF8ViVettlZbf9hw9DJhubaWn7rGvDDTcLMDm6tQ0=
k8s.io/kubernetes v1.34.2 h1:WQdDvYJazkmkwSncgNwGvVtaCt4TYXIU3wSMRgvp3MI=
k8s.io/kubernetes v1.34.2/go.mod h1:m6pZk6a179pRo2wsTiCPORJ86iOEQmfIzUvtyEF8BwA=
k8s.io/system-validators v1.10.2 h1:7rC7VdrQCaM55E08Pw3I1v1Op9ObLxdKAu5Ff5hIPwY=
k8s.io/system-validators v1.10.2/go.mod h1:awfSS706v9R12VC7u7K89FKfqVy44G+E0L1A0FX9Wmw=
k8s.io/utils v0.0.0-20250820121507-0af2bda4dd1d h1:wAhiDyZ4Tdtt7e46e9M5ZSAJ/MnPGPs+Ki1gHw4w1R0=
k8s.io/utils v0.0.0-20250820121507-0af2bda4dd1d/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0=
k8s.io/kms v0.35.0 h1:/x87FED2kDSo66csKtcYCEHsxF/DBlNl7LfJ1fVQs1o=
k8s.io/kms v0.35.0/go.mod h1:VT+4ekZAdrZDMgShK37vvlyHUVhwI9t/9tvh0AyCWmQ=
k8s.io/kube-openapi v0.0.0-20250910181357-589584f1c912 h1:Y3gxNAuB0OBLImH611+UDZcmKS3g6CthxToOb37KgwE=
k8s.io/kube-openapi v0.0.0-20250910181357-589584f1c912/go.mod h1:kdmbQkyfwUagLfXIad1y2TdrjPFWp2Q89B3qkRwf/pQ=
k8s.io/kube-proxy v0.35.0 h1:erv2wYmGZ6nyu/FtmaIb+ORD3q2rfZ4Fhn7VXs/8cPQ=
k8s.io/kube-proxy v0.35.0/go.mod h1:bd9lpN3uLLOOWc/CFZbkPEi9DTkzQQymbE8FqSU4bWk=
k8s.io/kubelet v0.35.0 h1:8cgJHCBCKLYuuQ7/Pxb/qWbJfX1LXIw7790ce9xHq7c=
k8s.io/kubelet v0.35.0/go.mod h1:ciRzAXn7C4z5iB7FhG1L2CGPPXLTVCABDlbXt/Zz8YA=
k8s.io/kubernetes v1.35.0 h1:PUOojD8c8E3csMP5NX+nLLne6SGqZjrYCscptyBfWMY=
k8s.io/kubernetes v1.35.0/go.mod h1:Tzk9Y9W/XUFFFgTUVg+BAowoFe+Pc7koGLuaiLHdcFg=
k8s.io/system-validators v1.12.1 h1:AY1+COTLJN/Sj0w9QzH1H0yvyF3Kl6CguMnh32WlcUU=
k8s.io/system-validators v1.12.1/go.mod h1:awfSS706v9R12VC7u7K89FKfqVy44G+E0L1A0FX9Wmw=
k8s.io/utils v0.0.0-20251002143259-bc988d571ff4 h1:SjGebBtkBqHFOli+05xYbK8YF1Dzkbzn+gDM4X9T4Ck=
k8s.io/utils v0.0.0-20251002143259-bc988d571ff4/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0=
mellium.im/sasl v0.3.1 h1:wE0LW6g7U83vhvxjC1IY8DnXM+EU095yeo8XClvCdfo=
mellium.im/sasl v0.3.1/go.mod h1:xm59PUYpZHhgQ9ZqoJ5QaCqzWMi8IeS49dhp6plPCzw=
sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.31.2 h1:jpcvIRr3GLoUoEKRkHKSmGjxb6lWwrBlJsXc+eUYQHM=

View File

@@ -52,9 +52,15 @@ const (
kineInitContainerName = "chmod"
)
type DataStoreOverrides struct {
Resource string
DataStore kamajiv1alpha1.DataStore
}
type Deployment struct {
KineContainerImage string
DataStore kamajiv1alpha1.DataStore
DataStoreOverrides []DataStoreOverrides
Client client.Client
}
@@ -711,11 +717,29 @@ func (d Deployment) buildKubeAPIServerCommand(tenantControlPlane kamajiv1alpha1.
desiredArgs["--etcd-keyfile"] = "/etc/kubernetes/pki/etcd/server.key"
}
if len(d.DataStoreOverrides) != 0 {
desiredArgs["--etcd-servers-overrides"] = d.etcdServersOverrides()
}
// Order matters, here: extraArgs could try to overwrite some arguments managed by Kamaji and that would be crucial.
// Adding as first element of the array of maps, we're sure that these overrides will be sanitized by our configuration.
return utilities.MergeMaps(current, desiredArgs, extraArgs)
}
func (d Deployment) etcdServersOverrides() string {
dataStoreOverridesEndpoints := make([]string, 0, len(d.DataStoreOverrides))
for _, dso := range d.DataStoreOverrides {
httpsEndpoints := make([]string, 0, len(dso.DataStore.Spec.Endpoints))
for _, ep := range dso.DataStore.Spec.Endpoints {
httpsEndpoints = append(httpsEndpoints, fmt.Sprintf("https://%s", ep))
}
dataStoreOverridesEndpoints = append(dataStoreOverridesEndpoints, fmt.Sprintf("%s#%s", dso.Resource, strings.Join(httpsEndpoints, ";")))
}
return strings.Join(dataStoreOverridesEndpoints, ",")
}
func (d Deployment) secretProjection(secretName, certKeyName, keyName string) *corev1.SecretProjection {
return &corev1.SecretProjection{
LocalObjectReference: corev1.LocalObjectReference{

View File

@@ -0,0 +1,46 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package controlplane
import (
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
)
var _ = Describe("Controlplane Deployment", func() {
var d Deployment
BeforeEach(func() {
d = Deployment{
DataStoreOverrides: []DataStoreOverrides{{
Resource: "/events",
DataStore: kamajiv1alpha1.DataStore{
Spec: kamajiv1alpha1.DataStoreSpec{
Endpoints: kamajiv1alpha1.Endpoints{"etcd-0", "etcd-1", "etcd-2"},
},
},
}},
}
})
Describe("DataStoreOverrides flag generation", func() {
It("should generate valid --etcd-servers-overrides value", func() {
etcdSerVersOverrides := d.etcdServersOverrides()
Expect(etcdSerVersOverrides).To(Equal("/events#https://etcd-0;https://etcd-1;https://etcd-2"))
})
It("should generate valid --etcd-servers-overrides value with 2 DataStoreOverrides", func() {
d.DataStoreOverrides = append(d.DataStoreOverrides, DataStoreOverrides{
Resource: "/pods",
DataStore: kamajiv1alpha1.DataStore{
Spec: kamajiv1alpha1.DataStoreSpec{
Endpoints: kamajiv1alpha1.Endpoints{"etcd-3", "etcd-4", "etcd-5"},
},
},
})
etcdSerVersOverrides := d.etcdServersOverrides()
Expect(etcdSerVersOverrides).To(Equal("/events#https://etcd-0;https://etcd-1;https://etcd-2,/pods#https://etcd-3;https://etcd-4;https://etcd-5"))
})
})
})

View File

@@ -7,6 +7,7 @@ import (
"fmt"
"github.com/blang/semver"
jsonpatchv5 "github.com/evanphx/json-patch/v5"
"github.com/pkg/errors"
corev1 "k8s.io/api/core/v1"
rbacv1 "k8s.io/api/rbac/v1"
@@ -34,13 +35,14 @@ func UploadKubeadmConfig(client kubernetes.Interface, config *Configuration) ([]
return nil, uploadconfig.UploadConfiguration(&config.InitConfiguration, client)
}
func UploadKubeletConfig(client kubernetes.Interface, config *Configuration) ([]byte, error) {
func UploadKubeletConfig(client kubernetes.Interface, config *Configuration, patches jsonpatchv5.Patch) ([]byte, error) {
kubeletConfiguration := KubeletConfiguration{
TenantControlPlaneDomain: config.InitConfiguration.Networking.DNSDomain,
TenantControlPlaneDNSServiceIPs: config.Parameters.TenantDNSServiceIPs,
TenantControlPlaneCgroupDriver: config.Parameters.TenantControlPlaneCGroupDriver,
}
content, err := getKubeletConfigmapContent(kubeletConfiguration)
content, err := getKubeletConfigmapContent(kubeletConfiguration, patches)
if err != nil {
return nil, err
}
@@ -71,7 +73,7 @@ func UploadKubeletConfig(client kubernetes.Interface, config *Configuration) ([]
return nil, nil
}
func getKubeletConfigmapContent(kubeletConfiguration KubeletConfiguration) ([]byte, error) {
func getKubeletConfigmapContent(kubeletConfiguration KubeletConfiguration, patch jsonpatchv5.Patch) ([]byte, error) {
var kc kubelettypes.KubeletConfiguration
kubeletv1beta1.SetDefaults_KubeletConfiguration(&kc)
@@ -93,6 +95,21 @@ func getKubeletConfigmapContent(kubeletConfiguration KubeletConfiguration) ([]by
// determine the resolvConf location, as reported in clastix/kamaji#581.
kc.ResolverConfig = nil
if len(patch) > 0 {
kubeletConfig, patchErr := utilities.EncodeToJSON(&kc)
if patchErr != nil {
return nil, errors.Wrapf(patchErr, "unable to encode KubeletConfiguration to JSON for JSON patching")
}
if kubeletConfig, patchErr = patch.Apply(kubeletConfig); patchErr != nil {
return nil, errors.Wrapf(patchErr, "unable to apply JSON patching to KubeletConfiguration")
}
if patchErr = utilities.DecodeFromJSON(string(kubeletConfig), &kc); patchErr != nil {
return nil, errors.Wrapf(patchErr, "unable to decode JSON to KubeletConfiguration")
}
}
return utilities.EncodeToYaml(&kc)
}

View File

@@ -30,6 +30,7 @@ type Config struct {
Client client.Client
ConnString string
DataStore kamajiv1alpha1.DataStore
IsOverride bool
}
func (r *Config) GetHistogram() prometheus.Histogram {
@@ -103,10 +104,12 @@ func (r *Config) GetName() string {
}
func (r *Config) UpdateTenantControlPlaneStatus(_ context.Context, tenantControlPlane *kamajiv1alpha1.TenantControlPlane) error {
tenantControlPlane.Status.Storage.Driver = string(r.DataStore.Spec.Driver)
tenantControlPlane.Status.Storage.DataStoreName = r.DataStore.GetName()
tenantControlPlane.Status.Storage.Config.SecretName = r.resource.GetName()
tenantControlPlane.Status.Storage.Config.Checksum = utilities.GetObjectChecksum(r.resource)
if !r.IsOverride {
tenantControlPlane.Status.Storage.Driver = string(r.DataStore.Spec.Driver)
tenantControlPlane.Status.Storage.DataStoreName = r.DataStore.GetName()
tenantControlPlane.Status.Storage.Config.SecretName = r.resource.GetName()
tenantControlPlane.Status.Storage.Config.Checksum = utilities.GetObjectChecksum(r.resource)
}
return nil
}

View File

@@ -22,6 +22,7 @@ type KubernetesDeploymentResource struct {
resource *appsv1.Deployment
Client client.Client
DataStore kamajiv1alpha1.DataStore
DataStoreOverrides []builder.DataStoreOverrides
KineContainerImage string
}
@@ -66,6 +67,7 @@ func (r *KubernetesDeploymentResource) mutate(ctx context.Context, tenantControl
Client: r.Client,
DataStore: r.DataStore,
KineContainerImage: r.KineContainerImage,
DataStoreOverrides: r.DataStoreOverrides,
}).Build(ctx, r.resource, *tenantControlPlane)
return controllerutil.SetControllerReference(tenantControlPlane, r.resource, r.Client.Scheme())
@@ -101,7 +103,8 @@ func (r *KubernetesDeploymentResource) computeStatus(tenantControlPlane *kamajiv
func (r *KubernetesDeploymentResource) UpdateTenantControlPlaneStatus(_ context.Context, tenantControlPlane *kamajiv1alpha1.TenantControlPlane) error {
tenantControlPlane.Status.Kubernetes.Version.Status = r.computeStatus(tenantControlPlane)
if *tenantControlPlane.Status.Kubernetes.Version.Status == kamajiv1alpha1.VersionReady {
if *tenantControlPlane.Status.Kubernetes.Version.Status == kamajiv1alpha1.VersionReady ||
*tenantControlPlane.Status.Kubernetes.Version.Status == kamajiv1alpha1.VersionSleeping {
tenantControlPlane.Status.Kubernetes.Version.Version = tenantControlPlane.Spec.Kubernetes.Version
}

View File

@@ -6,14 +6,10 @@ package resources
import (
"context"
"fmt"
"net/url"
"github.com/prometheus/client_golang/prometheus"
v1 "k8s.io/api/core/v1"
k8serrors "k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/api/meta"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/fields"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
"sigs.k8s.io/controller-runtime/pkg/log"
@@ -56,66 +52,12 @@ func (r *KubernetesGatewayResource) gatewayStatusNeedsUpdate(tcp *kamajiv1alpha1
currentStatus := tcp.Status.Kubernetes.Gateway
// Check if route reference has changed
if currentStatus.RouteRef.Name != r.resource.Name {
if currentStatus != nil && currentStatus.RouteRef.Name != r.resource.Name {
return true
}
// Compare RouteStatus - check if number of parents changed
if len(currentStatus.RouteStatus.Parents) != len(r.resource.Status.RouteStatus.Parents) {
return true
}
// Compare individual parent statuses
// NOTE: Multiple Parent References are assumed.
for i, currentParent := range currentStatus.RouteStatus.Parents {
if i >= len(r.resource.Status.RouteStatus.Parents) {
return true
}
resourceParent := r.resource.Status.RouteStatus.Parents[i]
// Compare parent references
if currentParent.ParentRef.Name != resourceParent.ParentRef.Name ||
(currentParent.ParentRef.Namespace == nil) != (resourceParent.ParentRef.Namespace == nil) ||
(currentParent.ParentRef.Namespace != nil && resourceParent.ParentRef.Namespace != nil &&
*currentParent.ParentRef.Namespace != *resourceParent.ParentRef.Namespace) ||
(currentParent.ParentRef.SectionName == nil) != (resourceParent.ParentRef.SectionName == nil) ||
(currentParent.ParentRef.SectionName != nil && resourceParent.ParentRef.SectionName != nil &&
*currentParent.ParentRef.SectionName != *resourceParent.ParentRef.SectionName) {
return true
}
if len(currentParent.Conditions) != len(resourceParent.Conditions) {
return true
}
// Compare each condition
for j, currentCondition := range currentParent.Conditions {
if j >= len(resourceParent.Conditions) {
return true
}
resourceCondition := resourceParent.Conditions[j]
if currentCondition.Type != resourceCondition.Type ||
currentCondition.Status != resourceCondition.Status ||
currentCondition.Reason != resourceCondition.Reason ||
currentCondition.Message != resourceCondition.Message ||
!currentCondition.LastTransitionTime.Equal(&resourceCondition.LastTransitionTime) {
return true
}
}
}
// Since access points are derived from route status and gateway conditions,
// and we've already compared the route status above, we can assume that
// if the route status hasn't changed, the access points calculation
// will produce the same result. This avoids the need for complex
// gateway fetching in the status comparison.
//
// If there are edge cases where gateway state changes but route status doesn't,
// those will be caught in the next reconciliation cycle anyway.
return false
return IsGatewayRouteStatusChanged(currentStatus, r.resource.Status.RouteStatus)
}
func (r *KubernetesGatewayResource) ShouldCleanup(tcp *kamajiv1alpha1.TenantControlPlane) bool {
@@ -125,95 +67,18 @@ func (r *KubernetesGatewayResource) ShouldCleanup(tcp *kamajiv1alpha1.TenantCont
func (r *KubernetesGatewayResource) CleanUp(ctx context.Context, tcp *kamajiv1alpha1.TenantControlPlane) (bool, error) {
logger := log.FromContext(ctx, "resource", r.GetName())
route := gatewayv1alpha2.TLSRoute{}
if err := r.Client.Get(ctx, client.ObjectKey{
Namespace: r.resource.GetNamespace(),
Name: r.resource.GetName(),
}, &route); err != nil {
if !k8serrors.IsNotFound(err) {
logger.Error(err, "failed to get TLSRoute before cleanup")
cleaned, err := CleanupTLSRoute(ctx, r.Client, r.resource.GetName(), r.resource.GetNamespace(), tcp)
if err != nil {
logger.Error(err, "failed to cleanup tcp route")
return false, err
}
return false, nil
return false, err
}
if !metav1.IsControlledBy(&route, tcp) {
logger.Info("skipping cleanup: route is not managed by Kamaji", "name", route.Name, "namespace", route.Namespace)
return false, nil
if cleaned {
logger.V(1).Info("tcp route cleaned up successfully")
}
if err := r.Client.Delete(ctx, &route); err != nil {
if !k8serrors.IsNotFound(err) {
// TODO: Is that an error? Wanted to delete the resource anyways.
logger.Error(err, "cannot cleanup tcp route")
return false, err
}
return false, nil
}
logger.V(1).Info("tcp route cleaned up successfully")
return true, nil
}
// fetchGatewayByListener uses the indexer to efficiently find a gateway with a specific listener.
// This avoids the need to iterate through all listeners in a gateway.
func (r *KubernetesGatewayResource) fetchGatewayByListener(ctx context.Context, ref gatewayv1.ParentReference) (*gatewayv1.Gateway, error) {
if ref.Namespace == nil {
return nil, fmt.Errorf("missing namespace")
}
if ref.SectionName == nil {
return nil, fmt.Errorf("missing sectionName")
}
// Build the composite key that matches our indexer format: namespace/gatewayName/listenerName
listenerKey := fmt.Sprintf("%s/%s/%s", *ref.Namespace, ref.Name, *ref.SectionName)
// Query gateways using the indexer
gatewayList := &gatewayv1.GatewayList{}
if err := r.Client.List(ctx, gatewayList, client.MatchingFieldsSelector{
Selector: fields.OneTermEqualSelector(kamajiv1alpha1.GatewayListenerNameKey, listenerKey),
}); err != nil {
return nil, fmt.Errorf("failed to list gateways by listener: %w", err)
}
if len(gatewayList.Items) == 0 {
return nil, fmt.Errorf("no gateway found with listener '%s'", *ref.SectionName)
}
// Since we're using a composite key with namespace/name/listener, we should get exactly one result
if len(gatewayList.Items) > 1 {
return nil, fmt.Errorf("found multiple gateways with listener '%s', expected exactly one", *ref.SectionName)
}
return &gatewayList.Items[0], nil
}
func FindMatchingListener(listeners []gatewayv1.Listener, ref gatewayv1.ParentReference) (gatewayv1.Listener, error) {
if ref.SectionName == nil {
return gatewayv1.Listener{}, fmt.Errorf("missing sectionName")
}
name := *ref.SectionName
for _, listener := range listeners {
if listener.Name == name {
return listener, nil
}
}
// TODO: Handle the cases according to the spec:
// - When both Port (experimental) and SectionName are
// specified, the name and port of the selected listener
// must match both specified values.
// - When unspecified (empty string) this will reference
// the entire resource [...] an attachment is considered
// successful if at least one section in the parent resource accepts it
return gatewayv1.Listener{}, fmt.Errorf("could not find listener '%s'", name)
return cleaned, nil
}
func (r *KubernetesGatewayResource) UpdateTenantControlPlaneStatus(ctx context.Context, tcp *kamajiv1alpha1.TenantControlPlane) error {
@@ -251,53 +116,9 @@ func (r *KubernetesGatewayResource) UpdateTenantControlPlaneStatus(ctx context.C
}
logger.V(1).Info("updating TenantControlPlane status for Gateway routes")
accessPoints := []kamajiv1alpha1.GatewayAccessPoint{}
for _, routeStatus := range routeStatuses.Parents {
routeAccepted := meta.IsStatusConditionTrue(
routeStatus.Conditions,
string(gatewayv1.RouteConditionAccepted),
)
if !routeAccepted {
continue
}
// Use the indexer to efficiently find the gateway with the specific listener
gateway, err := r.fetchGatewayByListener(ctx, routeStatus.ParentRef)
if err != nil {
return fmt.Errorf("could not fetch gateway with listener '%v': %w",
routeStatus.ParentRef.SectionName, err)
}
gatewayProgrammed := meta.IsStatusConditionTrue(
gateway.Status.Conditions,
string(gatewayv1.GatewayConditionProgrammed),
)
if !gatewayProgrammed {
continue
}
// Since we fetched the gateway using the indexer, we know the listener exists
// but we still need to get its details from the gateway spec
listener, err := FindMatchingListener(
gateway.Spec.Listeners, routeStatus.ParentRef,
)
if err != nil {
return fmt.Errorf("failed to match listener: %w", err)
}
for _, hostname := range r.resource.Spec.Hostnames {
rawURL := fmt.Sprintf("https://%s:%d", hostname, listener.Port)
url, err := url.Parse(rawURL)
if err != nil {
return fmt.Errorf("invalid url: %w", err)
}
hostnameAddressType := gatewayv1.HostnameAddressType
accessPoints = append(accessPoints, kamajiv1alpha1.GatewayAccessPoint{
Type: &hostnameAddressType,
Value: url.String(),
Port: listener.Port,
})
}
accessPoints, err := BuildGatewayAccessPointsStatus(ctx, r.Client, r.resource, routeStatuses)
if err != nil {
return err
}
tcp.Status.Kubernetes.Gateway.AccessPoints = accessPoints
@@ -329,10 +150,6 @@ func (r *KubernetesGatewayResource) mutate(tcp *kamajiv1alpha1.TenantControlPlan
tcp.Spec.ControlPlane.Gateway.AdditionalMetadata.Annotations)
r.resource.SetAnnotations(annotations)
if tcp.Spec.ControlPlane.Gateway.GatewayParentRefs != nil {
r.resource.Spec.ParentRefs = tcp.Spec.ControlPlane.Gateway.GatewayParentRefs
}
serviceName := gatewayv1alpha2.ObjectName(tcp.Status.Kubernetes.Service.Name)
servicePort := tcp.Status.Kubernetes.Service.Port
@@ -340,6 +157,11 @@ func (r *KubernetesGatewayResource) mutate(tcp *kamajiv1alpha1.TenantControlPlan
return fmt.Errorf("service not ready, cannot create TLSRoute")
}
if tcp.Spec.ControlPlane.Gateway.GatewayParentRefs != nil {
// Copy parentRefs and explicitly set port and sectionName fields
r.resource.Spec.ParentRefs = NewParentRefsSpecWithPortAndSection(tcp.Spec.ControlPlane.Gateway.GatewayParentRefs, servicePort, "kube-apiserver")
}
rule := gatewayv1alpha2.TLSRouteRule{
BackendRefs: []gatewayv1alpha2.BackendRef{
{

View File

@@ -12,6 +12,7 @@ import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/client-go/kubernetes/scheme"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/client/fake"
gatewayv1 "sigs.k8s.io/gateway-api/apis/v1"
gatewayv1alpha2 "sigs.k8s.io/gateway-api/apis/v1alpha2"
@@ -100,6 +101,62 @@ var _ = Describe("KubernetesGatewayResource", func() {
shouldUpdate := resource.ShouldStatusBeUpdated(ctx, tcp)
Expect(shouldUpdate).To(BeTrue())
})
It("should set port and sectionName in parentRefs, overriding any user-provided values", func() {
customPort := gatewayv1.PortNumber(9999)
customSectionName := gatewayv1.SectionName("custom")
tcp.Spec.ControlPlane.Gateway.GatewayParentRefs[0].Port = &customPort
tcp.Spec.ControlPlane.Gateway.GatewayParentRefs[0].SectionName = &customSectionName
err := resource.Define(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
_, err = resource.CreateOrUpdate(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
route := &gatewayv1alpha2.TLSRoute{}
err = resource.Client.Get(ctx, client.ObjectKey{Name: tcp.Name, Namespace: tcp.Namespace}, route)
Expect(err).NotTo(HaveOccurred())
Expect(route.Spec.ParentRefs).To(HaveLen(1))
Expect(route.Spec.ParentRefs[0].Port).NotTo(BeNil())
Expect(*route.Spec.ParentRefs[0].Port).To(Equal(tcp.Status.Kubernetes.Service.Port))
Expect(*route.Spec.ParentRefs[0].Port).NotTo(Equal(customPort))
Expect(route.Spec.ParentRefs[0].SectionName).NotTo(BeNil())
Expect(*route.Spec.ParentRefs[0].SectionName).To(Equal(gatewayv1.SectionName("kube-apiserver")))
Expect(*route.Spec.ParentRefs[0].SectionName).NotTo(Equal(customSectionName))
})
It("should handle multiple parentRefs correctly", func() {
namespace := gatewayv1.Namespace("default")
tcp.Spec.ControlPlane.Gateway.GatewayParentRefs = []gatewayv1alpha2.ParentReference{
{
Name: "test-gateway-1",
Namespace: &namespace,
},
{
Name: "test-gateway-2",
Namespace: &namespace,
},
}
err := resource.Define(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
_, err = resource.CreateOrUpdate(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
route := &gatewayv1alpha2.TLSRoute{}
err = resource.Client.Get(ctx, client.ObjectKey{Name: tcp.Name, Namespace: tcp.Namespace}, route)
Expect(err).NotTo(HaveOccurred())
Expect(route.Spec.ParentRefs).To(HaveLen(2))
for i := range route.Spec.ParentRefs {
Expect(route.Spec.ParentRefs[i].Port).NotTo(BeNil())
Expect(*route.Spec.ParentRefs[i].Port).To(Equal(tcp.Status.Kubernetes.Service.Port))
Expect(route.Spec.ParentRefs[i].SectionName).NotTo(BeNil())
Expect(*route.Spec.ParentRefs[i].SectionName).To(Equal(gatewayv1.SectionName("kube-apiserver")))
}
})
})
Context("When GatewayRoutes is not configured", func() {
@@ -235,4 +292,81 @@ var _ = Describe("KubernetesGatewayResource", func() {
Expect(listener.Port).To(Equal(gatewayv1.PortNumber(80)))
})
})
Describe("NewParentRefsSpecWithPortAndSection", func() {
var (
parentRefs []gatewayv1.ParentReference
testPort int32
testSectionName string
)
BeforeEach(func() {
namespace := gatewayv1.Namespace("default")
namespace2 := gatewayv1.Namespace("other")
testPort = int32(6443)
testSectionName = "kube-apiserver"
originalPort := gatewayv1.PortNumber(9999)
originalSectionName := gatewayv1.SectionName("original")
parentRefs = []gatewayv1.ParentReference{
{
Name: "test-gateway-1",
Namespace: &namespace,
Port: &originalPort,
SectionName: &originalSectionName,
},
{
Name: "test-gateway-2",
Namespace: &namespace2,
},
}
})
It("should create copy of parentRefs with port and sectionName set", func() {
result := resources.NewParentRefsSpecWithPortAndSection(parentRefs, testPort, testSectionName)
Expect(result).To(HaveLen(2))
for i := range result {
Expect(result[i].Name).To(Equal(parentRefs[i].Name))
Expect(result[i].Namespace).To(Equal(parentRefs[i].Namespace))
Expect(result[i].Port).NotTo(BeNil())
Expect(*result[i].Port).To(Equal(testPort))
Expect(result[i].SectionName).NotTo(BeNil())
Expect(*result[i].SectionName).To(Equal(gatewayv1.SectionName(testSectionName)))
}
})
It("should not modify original parentRefs", func() {
// Store original values for verification
originalFirstPort := parentRefs[0].Port
originalFirstSectionName := parentRefs[0].SectionName
originalSecondPort := parentRefs[1].Port
originalSecondSectionName := parentRefs[1].SectionName
result := resources.NewParentRefsSpecWithPortAndSection(parentRefs, testPort, testSectionName)
// Original should remain unchanged
Expect(parentRefs[0].Port).To(Equal(originalFirstPort))
Expect(parentRefs[0].SectionName).To(Equal(originalFirstSectionName))
Expect(parentRefs[1].Port).To(Equal(originalSecondPort))
Expect(parentRefs[1].SectionName).To(Equal(originalSecondSectionName))
// Result should have new values
Expect(result[0].Port).NotTo(BeNil())
Expect(*result[0].Port).To(Equal(testPort))
Expect(result[0].SectionName).NotTo(BeNil())
Expect(*result[0].SectionName).To(Equal(gatewayv1.SectionName(testSectionName)))
Expect(result[1].Port).NotTo(BeNil())
Expect(*result[1].Port).To(Equal(testPort))
Expect(result[1].SectionName).NotTo(BeNil())
Expect(*result[1].SectionName).To(Equal(gatewayv1.SectionName(testSectionName)))
})
It("should handle empty parentRefs slice", func() {
parentRefs = []gatewayv1.ParentReference{}
result := resources.NewParentRefsSpecWithPortAndSection(parentRefs, testPort, testSectionName)
Expect(result).To(BeEmpty())
})
})
})

View File

@@ -0,0 +1,241 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package resources
import (
"context"
"fmt"
"net/url"
k8serrors "k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/api/meta"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/fields"
"sigs.k8s.io/controller-runtime/pkg/client"
gatewayv1 "sigs.k8s.io/gateway-api/apis/v1"
gatewayv1alpha2 "sigs.k8s.io/gateway-api/apis/v1alpha2"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
)
// fetchGatewayByListener uses the indexer to efficiently find a gateway with a specific listener.
// This avoids the need to iterate through all listeners in a gateway.
func fetchGatewayByListener(ctx context.Context, c client.Client, ref gatewayv1.ParentReference) (*gatewayv1.Gateway, error) {
if ref.SectionName == nil {
return nil, fmt.Errorf("missing sectionName")
}
// Build the composite key that matches our indexer format: namespace/gatewayName/listenerName
listenerKey := fmt.Sprintf("%s/%s/%s", *ref.Namespace, ref.Name, *ref.SectionName)
// Query gateways using the indexer
gatewayList := &gatewayv1.GatewayList{}
if err := c.List(ctx, gatewayList, client.MatchingFieldsSelector{
Selector: fields.OneTermEqualSelector(kamajiv1alpha1.GatewayListenerNameKey, listenerKey),
}); err != nil {
return nil, fmt.Errorf("failed to list gateways by listener: %w", err)
}
if len(gatewayList.Items) == 0 {
return nil, fmt.Errorf("no gateway found with listener '%s'", *ref.SectionName)
}
// Since we're using a composite key with namespace/name/listener, we should get exactly one result
if len(gatewayList.Items) > 1 {
return nil, fmt.Errorf("found multiple gateways with listener '%s', expected exactly one", *ref.SectionName)
}
return &gatewayList.Items[0], nil
}
// FindMatchingListener finds a listener in the given list that matches the parent reference.
func FindMatchingListener(listeners []gatewayv1.Listener, ref gatewayv1.ParentReference) (gatewayv1.Listener, error) {
if ref.SectionName == nil {
return gatewayv1.Listener{}, fmt.Errorf("missing sectionName")
}
name := *ref.SectionName
for _, listener := range listeners {
if listener.Name == name {
return listener, nil
}
}
// TODO: Handle the cases according to the spec:
// - When both Port (experimental) and SectionName are
// specified, the name and port of the selected listener
// must match both specified values.
// - When unspecified (empty string) this will reference
// the entire resource [...] an attachment is considered
// successful if at least one section in the parent resource accepts it
return gatewayv1.Listener{}, fmt.Errorf("could not find listener '%s'", name)
}
// IsGatewayRouteStatusChanged checks if the gateway route status has changed compared to the stored status.
// Returns true if the status has changed (update needed), false if it's the same.
func IsGatewayRouteStatusChanged(currentStatus *kamajiv1alpha1.KubernetesGatewayStatus, resourceStatus gatewayv1alpha2.RouteStatus) bool {
if currentStatus == nil {
return true
}
// Compare RouteStatus - check if number of parents changed
if len(currentStatus.RouteStatus.Parents) != len(resourceStatus.Parents) {
return true
}
// Compare individual parent statuses
// NOTE: Multiple Parent References are assumed.
for i, currentParent := range currentStatus.RouteStatus.Parents {
if i >= len(resourceStatus.Parents) {
return true
}
resourceParent := resourceStatus.Parents[i]
// Compare parent references
if currentParent.ParentRef.Name != resourceParent.ParentRef.Name ||
(currentParent.ParentRef.Namespace == nil) != (resourceParent.ParentRef.Namespace == nil) ||
(currentParent.ParentRef.Namespace != nil && resourceParent.ParentRef.Namespace != nil &&
*currentParent.ParentRef.Namespace != *resourceParent.ParentRef.Namespace) ||
(currentParent.ParentRef.SectionName == nil) != (resourceParent.ParentRef.SectionName == nil) ||
(currentParent.ParentRef.SectionName != nil && resourceParent.ParentRef.SectionName != nil &&
*currentParent.ParentRef.SectionName != *resourceParent.ParentRef.SectionName) {
return true
}
if len(currentParent.Conditions) != len(resourceParent.Conditions) {
return true
}
// Compare each condition
for j, currentCondition := range currentParent.Conditions {
if j >= len(resourceParent.Conditions) {
return true
}
resourceCondition := resourceParent.Conditions[j]
if currentCondition.Type != resourceCondition.Type ||
currentCondition.Status != resourceCondition.Status ||
currentCondition.Reason != resourceCondition.Reason ||
currentCondition.Message != resourceCondition.Message ||
!currentCondition.LastTransitionTime.Equal(&resourceCondition.LastTransitionTime) {
return true
}
}
}
// Since access points are derived from route status and gateway conditions,
// and we've already compared the route status above, we can assume that
// if the route status hasn't changed, the access points calculation
// will produce the same result. This avoids the need for complex
// gateway fetching in the status comparison.
//
// If there are edge cases where gateway state changes but route status doesn't,
// those will be caught in the next reconciliation cycle anyway.
return false
}
// CleanupTLSRoute cleans up a TLSRoute resource if it's managed by the given TenantControlPlane.
func CleanupTLSRoute(ctx context.Context, c client.Client, routeName, routeNamespace string, tcp metav1.Object) (bool, error) {
route := gatewayv1alpha2.TLSRoute{}
if err := c.Get(ctx, client.ObjectKey{
Namespace: routeNamespace,
Name: routeName,
}, &route); err != nil {
if !k8serrors.IsNotFound(err) {
return false, fmt.Errorf("failed to get TLSRoute before cleanup: %w", err)
}
return false, nil
}
if !metav1.IsControlledBy(&route, tcp) {
return false, nil
}
if err := c.Delete(ctx, &route); err != nil {
if !k8serrors.IsNotFound(err) {
return false, fmt.Errorf("cannot delete TLSRoute route: %w", err)
}
return false, nil
}
return true, nil
}
// BuildGatewayAccessPointsStatus builds access points from route statuses.
func BuildGatewayAccessPointsStatus(ctx context.Context, c client.Client, route *gatewayv1alpha2.TLSRoute, routeStatuses gatewayv1alpha2.RouteStatus) ([]kamajiv1alpha1.GatewayAccessPoint, error) {
accessPoints := []kamajiv1alpha1.GatewayAccessPoint{}
routeNamespace := gatewayv1.Namespace(route.Namespace)
for _, routeStatus := range routeStatuses.Parents {
routeAccepted := meta.IsStatusConditionTrue(
routeStatus.Conditions,
string(gatewayv1.RouteConditionAccepted),
)
if !routeAccepted {
continue
}
if routeStatus.ParentRef.Namespace == nil {
// Set the namespace to the route namespace if not set
routeStatus.ParentRef.Namespace = &routeNamespace
}
// Use the indexer to efficiently find the gateway with the specific listener
gateway, err := fetchGatewayByListener(ctx, c, routeStatus.ParentRef)
if err != nil {
return nil, fmt.Errorf("could not fetch gateway with listener '%v': %w",
routeStatus.ParentRef.SectionName, err)
}
gatewayProgrammed := meta.IsStatusConditionTrue(
gateway.Status.Conditions,
string(gatewayv1.GatewayConditionProgrammed),
)
if !gatewayProgrammed {
continue
}
// Since we fetched the gateway using the indexer, we know the listener exists
// but we still need to get its details from the gateway spec
listener, err := FindMatchingListener(
gateway.Spec.Listeners, routeStatus.ParentRef,
)
if err != nil {
return nil, fmt.Errorf("failed to match listener: %w", err)
}
for _, hostname := range route.Spec.Hostnames {
rawURL := fmt.Sprintf("https://%s:%d", hostname, listener.Port)
parsedURL, err := url.Parse(rawURL)
if err != nil {
return nil, fmt.Errorf("invalid url: %w", err)
}
hostnameAddressType := gatewayv1.HostnameAddressType
accessPoints = append(accessPoints, kamajiv1alpha1.GatewayAccessPoint{
Type: &hostnameAddressType,
Value: parsedURL.String(),
Port: listener.Port,
})
}
}
return accessPoints, nil
}
// NewParentRefsSpecWithPortAndSection creates a copy of parentRefs with port and sectionName set for each reference.
func NewParentRefsSpecWithPortAndSection(parentRefs []gatewayv1.ParentReference, port int32, sectionName string) []gatewayv1.ParentReference {
result := make([]gatewayv1.ParentReference, len(parentRefs))
sectionNamePtr := gatewayv1.SectionName(sectionName)
for i, parentRef := range parentRefs {
result[i] = *parentRef.DeepCopy()
result[i].Port = &port
result[i].SectionName = &sectionNamePtr
}
return result
}

View File

@@ -182,6 +182,21 @@ func (r *Agent) mutate(ctx context.Context, tenantControlPlane *kamajiv1alpha1.T
return err
}
// Override address with control plane gateway hostname if configured
// Konnectivity TLSRoute uses the same hostname as control plane gateway
if tenantControlPlane.Spec.ControlPlane.Gateway != nil &&
len(tenantControlPlane.Spec.ControlPlane.Gateway.Hostname) > 0 {
hostname := tenantControlPlane.Spec.ControlPlane.Gateway.Hostname
// Extract hostname
if len(hostname) > 0 {
konnectivityHostname, _ := utilities.GetControlPlaneAddressAndPortFromHostname(
string(hostname),
tenantControlPlane.Spec.Addons.Konnectivity.KonnectivityServerSpec.Port)
address = konnectivityHostname
}
}
r.resource.SetLabels(utilities.MergeMaps(r.resource.GetLabels(), utilities.KamajiLabels(tenantControlPlane.GetName(), r.GetName())))
specSelector := &metav1.LabelSelector{
@@ -281,8 +296,10 @@ func (r *Agent) mutate(ctx context.Context, tenantControlPlane *kamajiv1alpha1.T
case kamajiv1alpha1.KonnectivityAgentModeDeployment:
//nolint:forcetypeassert
r.resource.(*appsv1.Deployment).Spec.Template = *podTemplateSpec
//nolint:forcetypeassert
r.resource.(*appsv1.Deployment).Spec.Replicas = pointer.To(tenantControlPlane.Spec.Addons.Konnectivity.KonnectivityAgentSpec.Replicas)
if tenantControlPlane.Spec.Addons.Konnectivity.KonnectivityAgentSpec.Replicas != nil {
//nolint:forcetypeassert
r.resource.(*appsv1.Deployment).Spec.Replicas = tenantControlPlane.Spec.Addons.Konnectivity.KonnectivityAgentSpec.Replicas
}
}
return nil

View File

@@ -10,7 +10,6 @@ import (
corev1 "k8s.io/api/core/v1"
k8serrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
apiserverv1alpha1 "k8s.io/apiserver/pkg/apis/apiserver/v1alpha1"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
@@ -94,18 +93,18 @@ func (r *EgressSelectorConfigurationResource) mutate(_ context.Context, tenantCo
return func() error {
r.resource.SetLabels(utilities.MergeMaps(r.resource.GetLabels(), utilities.KamajiLabels(tenantControlPlane.GetName(), r.GetName())))
configuration := &apiserverv1alpha1.EgressSelectorConfiguration{
configuration := &EgressSelectorConfiguration{
TypeMeta: metav1.TypeMeta{
Kind: egressSelectorConfigurationKind,
APIVersion: apiServerAPIVersion,
},
EgressSelections: []apiserverv1alpha1.EgressSelection{
EgressSelections: []EgressSelection{
{
Name: egressSelectorConfigurationName,
Connection: apiserverv1alpha1.Connection{
ProxyProtocol: apiserverv1alpha1.ProtocolGRPC,
Transport: &apiserverv1alpha1.Transport{
UDS: &apiserverv1alpha1.UDSTransport{
Connection: Connection{
ProxyProtocol: ProtocolGRPC,
Transport: &Transport{
UDS: &UDSTransport{
UDSName: defaultUDSName,
},
},

View File

@@ -0,0 +1,71 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package konnectivity
import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
// Local type definitions for EgressSelectorConfiguration to avoid importing
// k8s.io/apiserver which conflicts with controller-runtime workqueue metrics.
// These types are based on k8s.io/apiserver/pkg/apis/apiserver/v1alpha1.
// ProtocolType is the type of the proxy protocol used for egress selection.
type ProtocolType string
const (
// ProtocolHTTPConnect uses HTTP CONNECT as the proxy protocol.
ProtocolHTTPConnect ProtocolType = "HTTPConnect"
// ProtocolGRPC uses GRPC as the proxy protocol.
ProtocolGRPC ProtocolType = "GRPC"
// ProtocolDirect establishes a direct connection without proxy.
ProtocolDirect ProtocolType = "Direct"
)
// +kubebuilder:object:root=true
// +kubebuilder:object:generate=true
// EgressSelectorConfiguration provides versioned configuration for egress selector clients.
type EgressSelectorConfiguration struct {
metav1.TypeMeta `json:",inline"`
EgressSelections []EgressSelection `json:"egressSelections"`
}
// +kubebuilder:object:generate=true
// EgressSelection provides the configuration for a single egress selection client.
type EgressSelection struct {
Name string `json:"name"`
Connection Connection `json:"connection"`
}
// +kubebuilder:object:generate=true
// Connection provides the configuration for a single egress selection client connection.
type Connection struct {
ProxyProtocol ProtocolType `json:"proxyProtocol,omitempty"`
Transport *Transport `json:"transport,omitempty"`
}
// +kubebuilder:object:generate=true
// Transport defines the transport configurations we support for egress selector.
type Transport struct {
TCP *TCPTransport `json:"tcp,omitempty"`
UDS *UDSTransport `json:"uds,omitempty"`
}
// +kubebuilder:object:generate=true
// TCPTransport provides the information to connect to a TCP endpoint.
type TCPTransport struct {
URL string `json:"url,omitempty"`
}
// +kubebuilder:object:generate=true
// UDSTransport provides the information to connect to a Unix Domain Socket endpoint.
type UDSTransport struct {
UDSName string `json:"udsName,omitempty"`
}

View File

@@ -0,0 +1,232 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package konnectivity
import (
"context"
"fmt"
"github.com/prometheus/client_golang/prometheus"
v1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
"sigs.k8s.io/controller-runtime/pkg/log"
gatewayv1 "sigs.k8s.io/gateway-api/apis/v1"
gatewayv1alpha2 "sigs.k8s.io/gateway-api/apis/v1alpha2"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
"github.com/clastix/kamaji/internal/resources"
"github.com/clastix/kamaji/internal/utilities"
)
type KubernetesKonnectivityGatewayResource struct {
resource *gatewayv1alpha2.TLSRoute
Client client.Client
}
func (r *KubernetesKonnectivityGatewayResource) GetHistogram() prometheus.Histogram {
gatewayCollector = resources.LazyLoadHistogramFromResource(gatewayCollector, r)
return gatewayCollector
}
func (r *KubernetesKonnectivityGatewayResource) ShouldStatusBeUpdated(_ context.Context, tcp *kamajiv1alpha1.TenantControlPlane) bool {
switch {
case !r.shouldHaveGateway(tcp) && (tcp.Status.Addons.Konnectivity.Gateway == nil):
return false
case r.shouldHaveGateway(tcp) && (tcp.Status.Addons.Konnectivity.Gateway == nil):
return true
case !r.shouldHaveGateway(tcp) && (tcp.Status.Addons.Konnectivity.Gateway != nil):
return true
case r.shouldHaveGateway(tcp) && (tcp.Status.Addons.Konnectivity.Gateway != nil):
return r.gatewayStatusNeedsUpdate(tcp)
}
return false
}
// shouldHaveGateway checks if Konnectivity gateway should be configured.
// Create when Konnectivity addon is enabled and control plane gateway is configured.
func (r *KubernetesKonnectivityGatewayResource) shouldHaveGateway(tcp *kamajiv1alpha1.TenantControlPlane) bool {
if tcp.Spec.Addons.Konnectivity == nil { // konnectivity addon is disabled
return false
}
// Create when control plane gateway is configured
return tcp.Spec.ControlPlane.Gateway != nil
}
// gatewayStatusNeedsUpdate compares the current gateway resource status with the stored status.
func (r *KubernetesKonnectivityGatewayResource) gatewayStatusNeedsUpdate(tcp *kamajiv1alpha1.TenantControlPlane) bool {
currentStatus := tcp.Status.Addons.Konnectivity.Gateway
// Check if route reference has changed
if currentStatus != nil && currentStatus.RouteRef.Name != r.resource.Name {
return true
}
// Compare RouteStatus - check if number of parents changed
return resources.IsGatewayRouteStatusChanged(currentStatus, r.resource.Status.RouteStatus)
}
func (r *KubernetesKonnectivityGatewayResource) ShouldCleanup(tcp *kamajiv1alpha1.TenantControlPlane) bool {
return !r.shouldHaveGateway(tcp) && tcp.Status.Addons.Konnectivity.Gateway != nil
}
func (r *KubernetesKonnectivityGatewayResource) CleanUp(ctx context.Context, tcp *kamajiv1alpha1.TenantControlPlane) (bool, error) {
logger := log.FromContext(ctx, "resource", r.GetName())
cleaned, err := resources.CleanupTLSRoute(ctx, r.Client, r.resource.GetName(), r.resource.GetNamespace(), tcp)
if err != nil {
logger.Error(err, "failed to cleanup konnectivity route")
return false, err
}
if cleaned {
logger.V(1).Info("konnectivity route cleaned up successfully")
}
return cleaned, nil
}
func (r *KubernetesKonnectivityGatewayResource) UpdateTenantControlPlaneStatus(ctx context.Context, tcp *kamajiv1alpha1.TenantControlPlane) error {
logger := log.FromContext(ctx, "resource", r.GetName())
// Clean up status if Gateway routes are no longer configured
if !r.shouldHaveGateway(tcp) {
tcp.Status.Addons.Konnectivity.Gateway = nil
return nil
}
tcp.Status.Addons.Konnectivity.Gateway = &kamajiv1alpha1.KubernetesGatewayStatus{
RouteStatus: r.resource.Status.RouteStatus,
RouteRef: v1.LocalObjectReference{
Name: r.resource.Name,
},
}
routeStatuses := tcp.Status.Addons.Konnectivity.Gateway.RouteStatus
// TODO: Investigate the implications of having multiple parents / hostnames
// TODO: Use condition to report?
if len(routeStatuses.Parents) == 0 {
return fmt.Errorf("no gateway attached to the konnectivity route")
}
if len(routeStatuses.Parents) > 1 {
return fmt.Errorf("too many gateways attached to the konnectivity route")
}
if len(r.resource.Spec.Hostnames) == 0 {
return fmt.Errorf("no hostname in the konnectivity route")
}
if len(r.resource.Spec.Hostnames) > 1 {
return fmt.Errorf("too many hostnames in the konnectivity route")
}
logger.V(1).Info("updating TenantControlPlane status for Konnectivity Gateway routes")
accessPoints, err := resources.BuildGatewayAccessPointsStatus(ctx, r.Client, r.resource, routeStatuses)
if err != nil {
return err
}
tcp.Status.Addons.Konnectivity.Gateway.AccessPoints = accessPoints
return nil
}
func (r *KubernetesKonnectivityGatewayResource) Define(_ context.Context, tcp *kamajiv1alpha1.TenantControlPlane) error {
r.resource = &gatewayv1alpha2.TLSRoute{
ObjectMeta: metav1.ObjectMeta{
Name: fmt.Sprintf("%s-konnectivity", tcp.GetName()),
Namespace: tcp.GetNamespace(),
},
}
return nil
}
func (r *KubernetesKonnectivityGatewayResource) mutate(tcp *kamajiv1alpha1.TenantControlPlane) controllerutil.MutateFn {
return func() error {
// Use control plane gateway configuration
if tcp.Spec.ControlPlane.Gateway == nil {
return fmt.Errorf("control plane gateway is not configured")
}
labels := utilities.MergeMaps(
r.resource.GetLabels(),
utilities.KamajiLabels(tcp.GetName(), r.GetName()),
tcp.Spec.ControlPlane.Gateway.AdditionalMetadata.Labels,
)
r.resource.SetLabels(labels)
annotations := utilities.MergeMaps(
r.resource.GetAnnotations(),
tcp.Spec.ControlPlane.Gateway.AdditionalMetadata.Annotations,
)
r.resource.SetAnnotations(annotations)
// Use hostname from control plane gateway
if len(tcp.Spec.ControlPlane.Gateway.Hostname) == 0 {
return fmt.Errorf("control plane gateway hostname is not set")
}
serviceName := gatewayv1alpha2.ObjectName(tcp.Status.Addons.Konnectivity.Service.Name)
servicePort := tcp.Status.Addons.Konnectivity.Service.Port
if serviceName == "" || servicePort == 0 {
return fmt.Errorf("konnectivity service not ready, cannot create TLSRoute")
}
// Copy parentRefs from control plane gateway and explicitly set port and sectionName fields
if tcp.Spec.ControlPlane.Gateway.GatewayParentRefs == nil {
return fmt.Errorf("control plane gateway parentRefs are not specified")
}
r.resource.Spec.ParentRefs = resources.NewParentRefsSpecWithPortAndSection(tcp.Spec.ControlPlane.Gateway.GatewayParentRefs, servicePort, "konnectivity-server")
rule := gatewayv1alpha2.TLSRouteRule{
BackendRefs: []gatewayv1alpha2.BackendRef{
{
BackendObjectReference: gatewayv1alpha2.BackendObjectReference{
Name: serviceName,
Port: &servicePort,
},
},
},
}
r.resource.Spec.Hostnames = []gatewayv1.Hostname{tcp.Spec.ControlPlane.Gateway.Hostname}
r.resource.Spec.Rules = []gatewayv1alpha2.TLSRouteRule{rule}
return controllerutil.SetControllerReference(tcp, r.resource, r.Client.Scheme())
}
}
func (r *KubernetesKonnectivityGatewayResource) CreateOrUpdate(ctx context.Context, tenantControlPlane *kamajiv1alpha1.TenantControlPlane) (controllerutil.OperationResult, error) {
logger := log.FromContext(ctx, "resource", r.GetName())
if !r.shouldHaveGateway(tenantControlPlane) {
return controllerutil.OperationResultNone, nil
}
if tenantControlPlane.Spec.ControlPlane.Gateway == nil {
return controllerutil.OperationResultNone, nil
}
if len(tenantControlPlane.Spec.ControlPlane.Gateway.Hostname) == 0 {
return controllerutil.OperationResultNone, fmt.Errorf("missing hostname to expose Konnectivity using a Gateway resource")
}
logger.V(1).Info("creating or updating resource konnectivity gateway routes")
result, err := utilities.CreateOrUpdateWithConflict(ctx, r.Client, r.resource, r.mutate(tenantControlPlane))
if err != nil {
return result, err
}
return result, nil
}
func (r *KubernetesKonnectivityGatewayResource) GetName() string {
return "konnectivity_gateway_routes"
}

View File

@@ -0,0 +1,218 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package konnectivity_test
import (
"context"
"testing"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/client-go/kubernetes/scheme"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/client/fake"
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
gatewayv1 "sigs.k8s.io/gateway-api/apis/v1"
gatewayv1alpha2 "sigs.k8s.io/gateway-api/apis/v1alpha2"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
"github.com/clastix/kamaji/internal/resources/konnectivity"
)
func TestKonnectivityGatewayResource(t *testing.T) {
RegisterFailHandler(Fail)
RunSpecs(t, "Konnectivity Gateway Resource Suite")
}
var runtimeScheme *runtime.Scheme
var _ = BeforeSuite(func() {
runtimeScheme = runtime.NewScheme()
Expect(scheme.AddToScheme(runtimeScheme)).To(Succeed())
Expect(kamajiv1alpha1.AddToScheme(runtimeScheme)).To(Succeed())
Expect(gatewayv1alpha2.Install(runtimeScheme)).To(Succeed())
})
var _ = Describe("KubernetesKonnectivityGatewayResource", func() {
var (
tcp *kamajiv1alpha1.TenantControlPlane
resource *konnectivity.KubernetesKonnectivityGatewayResource
ctx context.Context
)
BeforeEach(func() {
ctx = context.Background()
fakeClient := fake.NewClientBuilder().
WithScheme(runtimeScheme).
Build()
resource = &konnectivity.KubernetesKonnectivityGatewayResource{
Client: fakeClient,
}
namespace := gatewayv1.Namespace("default")
tcp = &kamajiv1alpha1.TenantControlPlane{
ObjectMeta: metav1.ObjectMeta{
Name: "test-tcp",
Namespace: "default",
},
Spec: kamajiv1alpha1.TenantControlPlaneSpec{
ControlPlane: kamajiv1alpha1.ControlPlane{
Gateway: &kamajiv1alpha1.GatewaySpec{
Hostname: gatewayv1alpha2.Hostname("test.example.com"),
GatewayParentRefs: []gatewayv1alpha2.ParentReference{
{
Name: "test-gateway",
Namespace: &namespace,
},
},
},
},
Addons: kamajiv1alpha1.AddonsSpec{
Konnectivity: &kamajiv1alpha1.KonnectivitySpec{
KonnectivityServerSpec: kamajiv1alpha1.KonnectivityServerSpec{
Port: 8132,
},
},
},
},
Status: kamajiv1alpha1.TenantControlPlaneStatus{
Addons: kamajiv1alpha1.AddonsStatus{
Konnectivity: kamajiv1alpha1.KonnectivityStatus{
Service: kamajiv1alpha1.KubernetesServiceStatus{
Name: "test-konnectivity-service",
Port: 8132,
},
},
},
},
}
})
Describe("shouldHaveGateway logic", func() {
It("should return false when Konnectivity addon is disabled", func() {
tcp.Spec.Addons.Konnectivity = nil
shouldUpdate := resource.ShouldStatusBeUpdated(ctx, tcp)
Expect(shouldUpdate).To(BeFalse())
Expect(resource.ShouldCleanup(tcp)).To(BeFalse())
})
It("should return false when control plane gateway is not configured", func() {
tcp.Spec.ControlPlane.Gateway = nil
shouldUpdate := resource.ShouldStatusBeUpdated(ctx, tcp)
Expect(shouldUpdate).To(BeFalse())
Expect(resource.ShouldCleanup(tcp)).To(BeFalse())
})
It("should return true when both Konnectivity and gateway are configured", func() {
shouldUpdate := resource.ShouldStatusBeUpdated(ctx, tcp)
Expect(shouldUpdate).To(BeTrue())
Expect(resource.ShouldCleanup(tcp)).To(BeFalse())
})
})
Context("When Konnectivity gateway should be configured", func() {
It("should set correct TLSRoute name with -konnectivity suffix", func() {
err := resource.Define(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
_, err = resource.CreateOrUpdate(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
route := &gatewayv1alpha2.TLSRoute{}
err = resource.Client.Get(ctx, client.ObjectKey{Name: "test-tcp-konnectivity", Namespace: tcp.Namespace}, route)
Expect(err).NotTo(HaveOccurred())
Expect(route.Name).To(Equal("test-tcp-konnectivity"))
})
It("should set sectionName to \"konnectivity-server\" and port from Konnectivity service status", func() {
err := resource.Define(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
_, err = resource.CreateOrUpdate(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
route := &gatewayv1alpha2.TLSRoute{}
err = resource.Client.Get(ctx, client.ObjectKey{Name: "test-tcp-konnectivity", Namespace: tcp.Namespace}, route)
Expect(err).NotTo(HaveOccurred())
Expect(route.Spec.ParentRefs).To(HaveLen(1))
Expect(route.Spec.ParentRefs[0].SectionName).NotTo(BeNil())
Expect(*route.Spec.ParentRefs[0].SectionName).To(Equal(gatewayv1.SectionName("konnectivity-server")))
Expect(route.Spec.ParentRefs[0].Port).NotTo(BeNil())
Expect(*route.Spec.ParentRefs[0].Port).To(Equal(tcp.Status.Addons.Konnectivity.Service.Port))
})
It("should use control plane gateway hostname", func() {
err := resource.Define(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
_, err = resource.CreateOrUpdate(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
route := &gatewayv1alpha2.TLSRoute{}
err = resource.Client.Get(ctx, client.ObjectKey{Name: "test-tcp-konnectivity", Namespace: tcp.Namespace}, route)
Expect(err).NotTo(HaveOccurred())
Expect(route.Spec.Hostnames).To(HaveLen(1))
Expect(route.Spec.Hostnames[0]).To(Equal(tcp.Spec.ControlPlane.Gateway.Hostname))
})
})
Context("Konnectivity-specific error cases", func() {
It("should return early without error when control plane gateway is not configured", func() {
tcp.Spec.ControlPlane.Gateway = nil
err := resource.Define(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
result, err := resource.CreateOrUpdate(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
Expect(result).To(Equal(controllerutil.OperationResultNone))
})
It("should fail when Konnectivity service is not ready", func() {
tcp.Status.Addons.Konnectivity.Service.Name = ""
tcp.Status.Addons.Konnectivity.Service.Port = 0
err := resource.Define(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
_, err = resource.CreateOrUpdate(ctx, tcp)
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("konnectivity service not ready"))
})
It("should fail when control plane gateway parentRefs are not specified", func() {
tcp.Spec.ControlPlane.Gateway.GatewayParentRefs = nil
err := resource.Define(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
_, err = resource.CreateOrUpdate(ctx, tcp)
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("control plane gateway parentRefs are not specified"))
})
})
Context("When Konnectivity gateway should not be configured", func() {
BeforeEach(func() {
tcp.Spec.Addons.Konnectivity = nil
tcp.Status.Addons.Konnectivity = kamajiv1alpha1.KonnectivityStatus{
Gateway: &kamajiv1alpha1.KubernetesGatewayStatus{
AccessPoints: nil,
},
}
})
It("should cleanup when gateway is removed", func() {
Expect(resource.ShouldCleanup(tcp)).To(BeTrue())
})
})
It("should return correct resource name", func() {
Expect(resource.GetName()).To(Equal("konnectivity_gateway_routes"))
})
})

View File

@@ -13,6 +13,7 @@ var (
clusterrolebindingCollector prometheus.Histogram
deploymentCollector prometheus.Histogram
egressCollector prometheus.Histogram
gatewayCollector prometheus.Histogram
kubeconfigCollector prometheus.Histogram
serviceaccountCollector prometheus.Histogram
serviceCollector prometheus.Histogram

View File

@@ -0,0 +1,134 @@
//go:build !ignore_autogenerated
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
// Code generated by controller-gen. DO NOT EDIT.
package konnectivity
import (
runtime "k8s.io/apimachinery/pkg/runtime"
)
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *Connection) DeepCopyInto(out *Connection) {
*out = *in
if in.Transport != nil {
in, out := &in.Transport, &out.Transport
*out = new(Transport)
(*in).DeepCopyInto(*out)
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Connection.
func (in *Connection) DeepCopy() *Connection {
if in == nil {
return nil
}
out := new(Connection)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *EgressSelection) DeepCopyInto(out *EgressSelection) {
*out = *in
in.Connection.DeepCopyInto(&out.Connection)
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new EgressSelection.
func (in *EgressSelection) DeepCopy() *EgressSelection {
if in == nil {
return nil
}
out := new(EgressSelection)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *EgressSelectorConfiguration) DeepCopyInto(out *EgressSelectorConfiguration) {
*out = *in
out.TypeMeta = in.TypeMeta
if in.EgressSelections != nil {
in, out := &in.EgressSelections, &out.EgressSelections
*out = make([]EgressSelection, len(*in))
for i := range *in {
(*in)[i].DeepCopyInto(&(*out)[i])
}
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new EgressSelectorConfiguration.
func (in *EgressSelectorConfiguration) DeepCopy() *EgressSelectorConfiguration {
if in == nil {
return nil
}
out := new(EgressSelectorConfiguration)
in.DeepCopyInto(out)
return out
}
// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.
func (in *EgressSelectorConfiguration) DeepCopyObject() runtime.Object {
if c := in.DeepCopy(); c != nil {
return c
}
return nil
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *TCPTransport) DeepCopyInto(out *TCPTransport) {
*out = *in
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new TCPTransport.
func (in *TCPTransport) DeepCopy() *TCPTransport {
if in == nil {
return nil
}
out := new(TCPTransport)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *Transport) DeepCopyInto(out *Transport) {
*out = *in
if in.TCP != nil {
in, out := &in.TCP, &out.TCP
*out = new(TCPTransport)
**out = **in
}
if in.UDS != nil {
in, out := &in.UDS, &out.UDS
*out = new(UDSTransport)
**out = **in
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Transport.
func (in *Transport) DeepCopy() *Transport {
if in == nil {
return nil
}
out := new(Transport)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *UDSTransport) DeepCopyInto(out *UDSTransport) {
*out = *in
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new UDSTransport.
func (in *UDSTransport) DeepCopy() *UDSTransport {
if in == nil {
return nil
}
out := new(UDSTransport)
in.DeepCopyInto(out)
return out
}

View File

@@ -10,6 +10,8 @@ import (
"strings"
"time"
jsonpatchv5 "github.com/evanphx/json-patch/v5"
"github.com/pkg/errors"
"github.com/prometheus/client_golang/prometheus"
corev1 "k8s.io/api/core/v1"
rbacv1 "k8s.io/api/rbac/v1"
@@ -150,7 +152,21 @@ func (r *KubeadmPhase) GetKubeadmFunction(ctx context.Context, tcp *kamajiv1alph
case PhaseUploadConfigKubeadm:
return kubeadm.UploadKubeadmConfig, nil
case PhaseUploadConfigKubelet:
return kubeadm.UploadKubeletConfig, nil
return func(c clientset.Interface, configuration *kubeadm.Configuration) ([]byte, error) {
var patch jsonpatchv5.Patch
if len(tcp.Spec.Kubernetes.Kubelet.ConfigurationJSONPatches) > 0 {
jsonP, patchErr := tcp.Spec.Kubernetes.Kubelet.ConfigurationJSONPatches.ToJSON()
if patchErr != nil {
return nil, errors.Wrap(patchErr, "cannot encode JSON Patches to JSON")
}
if patch, patchErr = jsonpatchv5.DecodePatch(jsonP); patchErr != nil {
return nil, errors.Wrap(patchErr, "cannot decode JSON Patches")
}
}
return kubeadm.UploadKubeletConfig(c, configuration, patch)
}, nil
case PhaseBootstrapToken:
return func(client clientset.Interface, config *kubeadm.Configuration) ([]byte, error) {
config.InitConfiguration.BootstrapTokens = nil

View File

@@ -11,6 +11,7 @@ import (
"github.com/prometheus/client_golang/prometheus"
"k8s.io/apimachinery/pkg/util/version"
"k8s.io/kubernetes/cmd/kubeadm/app/phases/upgrade"
"k8s.io/utils/ptr"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
@@ -77,9 +78,14 @@ func (k *KubernetesUpgrade) CreateOrUpdate(ctx context.Context, tenantControlPla
return controllerutil.OperationResultNone, errors.Wrap(err, "cannot create REST client required for Kubernetes upgrade plan")
}
versionGetter := kamajiupgrade.NewKamajiKubeVersionGetter(clientSet, tenantControlPlane.Status.Kubernetes.Version.Version)
var coreDNSVersion string
if tenantControlPlane.Spec.Addons.CoreDNS != nil {
coreDNSVersion = tenantControlPlane.Spec.Addons.CoreDNS.ImageTag
}
if _, err = upgrade.GetAvailableUpgrades(versionGetter, false, false, clientSet, &printers.Discard{}); err != nil {
versionGetter := kamajiupgrade.NewKamajiKubeVersionGetter(clientSet, tenantControlPlane.Status.Kubernetes.Version.Version, coreDNSVersion, tenantControlPlane.Status.Kubernetes.Version.Status)
if _, err = upgrade.GetAvailableUpgrades(versionGetter, false, false, &printers.Discard{}); err != nil {
return controllerutil.OperationResultNone, errors.Wrap(err, "cannot retrieve available Upgrades for Kubernetes upgrade plan")
}
@@ -87,7 +93,9 @@ func (k *KubernetesUpgrade) CreateOrUpdate(ctx context.Context, tenantControlPla
return controllerutil.OperationResultNone, fmt.Errorf("the required upgrade plan is not available")
}
k.inProgress = true
if ptr.Deref(tenantControlPlane.Spec.ControlPlane.Deployment.Replicas, 0) > 0 {
k.inProgress = true
}
return controllerutil.OperationResultNone, nil
}

View File

@@ -47,7 +47,7 @@ func GetKubeadmManifestDeps(ctx context.Context, client client.Client, tenantCon
TenantControlPlaneAddress: address,
TenantControlPlaneCertSANs: tenantControlPlane.Spec.NetworkProfile.CertSANs,
TenantControlPlanePort: tenantControlPlane.Spec.NetworkProfile.Port,
TenantControlPlaneCGroupDriver: tenantControlPlane.Spec.Kubernetes.Kubelet.CGroupFS.String(),
TenantControlPlaneCGroupDriver: tenantControlPlane.Spec.Kubernetes.Kubelet.CGroupFS.String(), //nolint:staticcheck
}
// If CoreDNS addon is enabled and with an override, adding these to the kubeadm init configuration
if coreDNS := tenantControlPlane.Spec.Addons.CoreDNS; coreDNS != nil {
@@ -200,7 +200,7 @@ func KubeadmPhaseCreate(ctx context.Context, r KubeadmPhaseResource, logger logr
TenantControlPlaneAddress: address,
TenantControlPlaneCertSANs: tenantControlPlane.Spec.NetworkProfile.CertSANs,
TenantControlPlanePort: tenantControlPlane.Spec.NetworkProfile.Port,
TenantControlPlaneCGroupDriver: tenantControlPlane.Spec.Kubernetes.Kubelet.CGroupFS.String(),
TenantControlPlaneCGroupDriver: tenantControlPlane.Spec.Kubernetes.Kubelet.CGroupFS.String(), //nolint:staticcheck
}
var checksum string

View File

@@ -12,23 +12,47 @@ import (
apimachineryversion "k8s.io/apimachinery/pkg/version"
"k8s.io/client-go/kubernetes"
"k8s.io/kubernetes/cmd/kubeadm/app/phases/upgrade"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
)
type kamajiKubeVersionGetter struct {
upgrade.VersionGetter
Version string
k8sVersion string
coreDNSVersion string
status *kamajiv1alpha1.KubernetesVersionStatus
}
func NewKamajiKubeVersionGetter(restClient kubernetes.Interface, version string) upgrade.VersionGetter {
func NewKamajiKubeVersionGetter(restClient kubernetes.Interface, version, coreDNSVersion string, status *kamajiv1alpha1.KubernetesVersionStatus) upgrade.VersionGetter {
kubeVersionGetter := upgrade.NewOfflineVersionGetter(upgrade.NewKubeVersionGetter(restClient), KubeadmVersion)
return &kamajiKubeVersionGetter{VersionGetter: kubeVersionGetter, Version: version}
return &kamajiKubeVersionGetter{
VersionGetter: kubeVersionGetter,
k8sVersion: version,
coreDNSVersion: coreDNSVersion,
status: status,
}
}
func (k kamajiKubeVersionGetter) ClusterVersion() (string, *versionutil.Version, error) {
if k.status != nil && *k.status == kamajiv1alpha1.VersionSleeping {
parsedVersion, parsedErr := versionutil.ParseGeneric(k.k8sVersion)
return k.k8sVersion, parsedVersion, parsedErr
}
return k.VersionGetter.ClusterVersion()
}
func (k kamajiKubeVersionGetter) DNSAddonVersion() (string, error) {
if k.status != nil && *k.status == kamajiv1alpha1.VersionSleeping {
return k.coreDNSVersion, nil
}
return k.VersionGetter.DNSAddonVersion()
}
func (k kamajiKubeVersionGetter) KubeadmVersion() (string, *versionutil.Version, error) {
kubeadmVersionInfo := apimachineryversion.Info{
GitVersion: KubeadmVersion,
@@ -50,11 +74,15 @@ func (k kamajiKubeVersionGetter) VersionFromCILabel(ciVersionLabel, description
}
func (k kamajiKubeVersionGetter) KubeletVersions() (map[string][]string, error) {
if k.status != nil && *k.status == kamajiv1alpha1.VersionSleeping {
return map[string][]string{}, nil
}
return k.VersionGetter.KubeletVersions()
}
func (k kamajiKubeVersionGetter) ComponentVersions(string) (map[string][]string, error) {
return map[string][]string{
k.Version: {"kamaji"},
k.k8sVersion: {"kamaji"},
}, nil
}

View File

@@ -4,5 +4,5 @@
package upgrade
const (
KubeadmVersion = "v1.34.0"
KubeadmVersion = "v1.35.0"
)

View File

@@ -30,7 +30,7 @@ func (t TenantControlPlaneDataStore) OnCreate(object runtime.Object) AdmissionRe
return nil, t.check(ctx, tcp.Spec.DataStore)
}
return nil, nil
return nil, t.checkDataStoreOverrides(ctx, tcp)
}
}
@@ -61,3 +61,19 @@ func (t TenantControlPlaneDataStore) check(ctx context.Context, dataStoreName st
return nil
}
func (t TenantControlPlaneDataStore) checkDataStoreOverrides(ctx context.Context, tcp *kamajiv1alpha1.TenantControlPlane) error {
overrideCheck := make(map[string]struct{}, 0)
for _, ds := range tcp.Spec.DataStoreOverrides {
if _, exists := overrideCheck[ds.Resource]; !exists {
overrideCheck[ds.Resource] = struct{}{}
} else {
return fmt.Errorf("duplicate resource override in Spec.DataStoreOverrides")
}
if err := t.check(ctx, ds.DataStore); err != nil {
return err
}
}
return nil
}

View File

@@ -0,0 +1,94 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package handlers
import (
"context"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime"
utilruntime "k8s.io/apimachinery/pkg/util/runtime"
fakeclient "sigs.k8s.io/controller-runtime/pkg/client/fake"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
)
var _ = Describe("TCP Datastore webhook", func() {
var (
ctx context.Context
t TenantControlPlaneDataStore
tcp *kamajiv1alpha1.TenantControlPlane
)
BeforeEach(func() {
scheme := runtime.NewScheme()
utilruntime.Must(kamajiv1alpha1.AddToScheme(scheme))
ctx = context.Background()
t = TenantControlPlaneDataStore{
Client: fakeclient.NewClientBuilder().WithScheme(scheme).WithObjects(&kamajiv1alpha1.DataStore{
ObjectMeta: metav1.ObjectMeta{
Name: "foo",
},
}, &kamajiv1alpha1.DataStore{
ObjectMeta: metav1.ObjectMeta{
Name: "bar",
},
}).Build(),
}
tcp = &kamajiv1alpha1.TenantControlPlane{
ObjectMeta: metav1.ObjectMeta{
Name: "tcp",
Namespace: "default",
},
Spec: kamajiv1alpha1.TenantControlPlaneSpec{},
}
})
Describe("validation should succeed without DataStoreOverrides", func() {
It("should validate TCP without DataStoreOverrides", func() {
err := t.checkDataStoreOverrides(ctx, tcp)
Expect(err).ToNot(HaveOccurred())
})
})
Describe("validation should fail with duplicate resources in DataStoreOverrides", func() {
It("should fail to validate TCP with duplicate resources in DataStoreOverrides", func() {
tcp.Spec.DataStoreOverrides = []kamajiv1alpha1.DataStoreOverride{{
Resource: "/event",
DataStore: "foo",
}, {
Resource: "/event",
DataStore: "bar",
}}
err := t.checkDataStoreOverrides(ctx, tcp)
Expect(err).To(HaveOccurred())
})
})
Describe("validation should succeed with valid DataStoreOverrides", func() {
It("should validate TCP with valid DataStoreOverrides", func() {
tcp.Spec.DataStoreOverrides = []kamajiv1alpha1.DataStoreOverride{{
Resource: "/leases",
DataStore: "foo",
}, {
Resource: "/event",
DataStore: "bar",
}}
err := t.checkDataStoreOverrides(ctx, tcp)
Expect(err).ToNot(HaveOccurred())
})
})
Describe("validation should fail with nonexistent DataStoreOverrides", func() {
It("should fail to validate TCP with nonexistent DataStoreOverrides", func() {
tcp.Spec.DataStoreOverrides = []kamajiv1alpha1.DataStoreOverride{{
Resource: "/leases",
DataStore: "baz",
}}
err := t.checkDataStoreOverrides(ctx, tcp)
Expect(err).To(HaveOccurred())
})
})
})

View File

@@ -67,6 +67,20 @@ func (t TenantControlPlaneDeployment) OnUpdate(newObject runtime.Object, oldObje
}
t.DeploymentBuilder.DataStore = ds
dataStoreOverrides := make([]controlplane.DataStoreOverrides, 0, len(tcp.Spec.DataStoreOverrides))
for _, dso := range tcp.Spec.DataStoreOverrides {
ds := kamajiv1alpha1.DataStore{}
if err := t.Client.Get(ctx, types.NamespacedName{Name: dso.DataStore}, &ds); err != nil {
return nil, err
}
dataStoreOverrides = append(dataStoreOverrides, controlplane.DataStoreOverrides{
Resource: dso.Resource,
DataStore: ds,
})
}
t.DeploymentBuilder.DataStoreOverrides = dataStoreOverrides
deployment := appsv1.Deployment{}
deployment.Name = tcp.Name
deployment.Namespace = tcp.Namespace

View File

@@ -0,0 +1,156 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package internal
import (
"context"
"testing"
"time"
dto "github.com/prometheus/client_model/go"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/client-go/rest"
"sigs.k8s.io/controller-runtime/pkg/controller"
"sigs.k8s.io/controller-runtime/pkg/manager"
"sigs.k8s.io/controller-runtime/pkg/metrics"
metricsserver "sigs.k8s.io/controller-runtime/pkg/metrics/server"
"sigs.k8s.io/controller-runtime/pkg/reconcile"
)
// dummyReconciler is a minimal reconciler for testing.
type dummyReconciler struct{}
func (r *dummyReconciler) Reconcile(ctx context.Context, req reconcile.Request) (reconcile.Result, error) {
return reconcile.Result{}, nil
}
// TestWorkqueueMetricsRegistration verifies that controller-runtime workqueue
// metrics are properly registered after removing the k8s.io/apiserver import.
// This test proves that issue #1026 is fixed.
func TestWorkqueueMetricsRegistration(t *testing.T) {
// Create a minimal scheme and manager
scheme := runtime.NewScheme()
// Create a manager with a fake config - this will trigger controller-runtime initialization
mgr, err := manager.New(&rest.Config{
Host: "https://localhost:6443",
}, manager.Options{
Scheme: scheme,
Metrics: metricsserver.Options{
BindAddress: "0", // Disable metrics server binding
},
})
if err != nil {
// If we can't create a manager (e.g., no cluster), skip the full test
// but still verify basic metrics registration
t.Logf("Could not create manager (expected in unit test): %v", err)
t.Log("Falling back to basic metrics check...")
checkBasicMetrics(t)
return
}
// Create a controller with the manager - this triggers workqueue creation
_, err = controller.New("test-controller", mgr, controller.Options{
Reconciler: &dummyReconciler{},
})
if err != nil {
t.Fatalf("Failed to create controller: %v", err)
}
// Start the manager in background
ctx, cancel := context.WithTimeout(t.Context(), 2*time.Second)
defer cancel()
go func() {
_ = mgr.Start(ctx)
}()
// Give it a moment to initialize
time.Sleep(100 * time.Millisecond)
// Gather all registered metrics from controller-runtime's registry
metricFamilies, err := metrics.Registry.Gather()
if err != nil {
t.Fatalf("Failed to gather metrics: %v", err)
}
// Collect all workqueue metrics
workqueueMetrics := make(map[string]*dto.MetricFamily)
for _, mf := range metricFamilies {
name := mf.GetName()
if len(name) > 10 && name[:10] == "workqueue_" {
workqueueMetrics[name] = mf
}
}
t.Logf("Total metrics registered: %d", len(metricFamilies))
t.Logf("Workqueue metrics found: %d", len(workqueueMetrics))
// Verify we have workqueue metrics
if len(workqueueMetrics) == 0 {
t.Fatal("FAILED: No workqueue metrics found! The initialization conflict is still present.")
}
// List all found workqueue metrics
t.Log("Found workqueue metrics:")
for name := range workqueueMetrics {
t.Logf(" - %s", name)
}
// Check for specific expected metrics from controller-runtime
expectedMetrics := []string{
"workqueue_depth",
"workqueue_adds_total",
"workqueue_queue_duration_seconds",
"workqueue_work_duration_seconds",
"workqueue_retries_total",
"workqueue_unfinished_work_seconds",
"workqueue_longest_running_processor_seconds",
}
missingMetrics := []string{}
for _, expected := range expectedMetrics {
if _, found := workqueueMetrics[expected]; !found {
missingMetrics = append(missingMetrics, expected)
}
}
if len(missingMetrics) > 0 {
t.Errorf("Missing expected workqueue metrics: %v", missingMetrics)
} else {
t.Log("✅ SUCCESS: All expected workqueue metrics are present!")
t.Log("The fix successfully resolved issue #1026 - workqueue metrics are now registered.")
}
}
// checkBasicMetrics is a fallback check when we can't create a full manager.
func checkBasicMetrics(t *testing.T) {
t.Helper()
// Gather metrics
metricFamilies, err := metrics.Registry.Gather()
if err != nil {
t.Fatalf("Failed to gather metrics: %v", err)
}
// Count workqueue metrics
workqueueCount := 0
for _, mf := range metricFamilies {
name := mf.GetName()
if len(name) > 10 && name[:10] == "workqueue_" {
workqueueCount++
t.Logf("Found: %s", name)
}
}
t.Logf("Total metrics: %d", len(metricFamilies))
t.Logf("Workqueue metrics: %d", workqueueCount)
if workqueueCount > 0 {
t.Log("✅ Workqueue metrics are being registered!")
} else {
t.Log(" No workqueue metrics yet (this is expected without an actual controller)")
t.Log("The fix removed the import conflict - metrics will appear when controllers run")
}
}