Compare commits

..

9 Commits

Author SHA1 Message Date
Léonard Suslian
d3fb03a752 feat: add support for multiple Datastores (#961)
* feat: add support for multiple Datastores

* docs: add guide for datastore overrides

* feat(datastore): add e2e test for dataStoreOverrides

* ci: reclaim disk space from runner to fix flaky tests
2025-12-12 12:10:02 +01:00
dependabot[bot]
eb86fec050 feat(deps): bump github.com/onsi/gomega from 1.38.2 to 1.38.3 (#1032)
Bumps [github.com/onsi/gomega](https://github.com/onsi/gomega) from 1.38.2 to 1.38.3.
- [Release notes](https://github.com/onsi/gomega/releases)
- [Changelog](https://github.com/onsi/gomega/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/gomega/compare/v1.38.2...v1.38.3)

---
updated-dependencies:
- dependency-name: github.com/onsi/gomega
  dependency-version: 1.38.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-12 12:03:28 +01:00
dependabot[bot]
35c83fbd4d feat(deps): bump k8s.io/kubernetes in the k8s group (#1034)
Bumps the k8s group with 1 update: [k8s.io/kubernetes](https://github.com/kubernetes/kubernetes).


Updates `k8s.io/kubernetes` from 1.34.2 to 1.34.3
- [Release notes](https://github.com/kubernetes/kubernetes/releases)
- [Commits](https://github.com/kubernetes/kubernetes/compare/v1.34.2...v1.34.3)

---
updated-dependencies:
- dependency-name: k8s.io/kubernetes
  dependency-version: 1.34.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: k8s
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-12 12:03:16 +01:00
dependabot[bot]
4ad4721965 feat(deps): bump github.com/onsi/ginkgo/v2 from 2.27.2 to 2.27.3 (#1033)
Bumps [github.com/onsi/ginkgo/v2](https://github.com/onsi/ginkgo) from 2.27.2 to 2.27.3.
- [Release notes](https://github.com/onsi/ginkgo/releases)
- [Changelog](https://github.com/onsi/ginkgo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/ginkgo/compare/v2.27.2...v2.27.3)

---
updated-dependencies:
- dependency-name: github.com/onsi/ginkgo/v2
  dependency-version: 2.27.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-09 10:04:49 +01:00
dependabot[bot]
f4b6de4c40 feat(deps): bump github.com/spf13/cobra from 1.10.1 to 1.10.2 (#1030)
Bumps [github.com/spf13/cobra](https://github.com/spf13/cobra) from 1.10.1 to 1.10.2.
- [Release notes](https://github.com/spf13/cobra/releases)
- [Commits](https://github.com/spf13/cobra/compare/v1.10.1...v1.10.2)

---
updated-dependencies:
- dependency-name: github.com/spf13/cobra
  dependency-version: 1.10.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-07 16:50:05 +01:00
dependabot[bot]
54e795323e feat(deps): bump sigs.k8s.io/gateway-api from 1.4.0 to 1.4.1 (#1029)
Bumps [sigs.k8s.io/gateway-api](https://github.com/kubernetes-sigs/gateway-api) from 1.4.0 to 1.4.1.
- [Release notes](https://github.com/kubernetes-sigs/gateway-api/releases)
- [Changelog](https://github.com/kubernetes-sigs/gateway-api/blob/main/RELEASE.md)
- [Commits](https://github.com/kubernetes-sigs/gateway-api/compare/v1.4.0...v1.4.1)

---
updated-dependencies:
- dependency-name: sigs.k8s.io/gateway-api
  dependency-version: 1.4.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-06 16:31:13 +01:00
Mateusz Kwiatkowski
61a4c152b3 fix: reduce database privileges for kine user (#860)
* Reduce MySQL privileges for kine user

* Reduce PostgreSQL privileges for kine user
2025-12-05 19:36:03 +01:00
Domonkos Cinke
eface0f792 chore: add Namecheap as an adopter (#1027) 2025-12-02 19:31:07 +01:00
Wojciech Urbański
2316af9731 fix: correct argument order in GrantPrivileges and RevokePrivileges methods (#1028)
Updated the argument order in the GrantPrivileges and RevokePrivileges methods to ensure proper execution of SQL statements.
2025-12-02 19:17:38 +01:00
24 changed files with 577 additions and 57 deletions

View File

@@ -41,6 +41,9 @@ jobs:
- uses: actions/setup-go@v6
with:
go-version-file: go.mod
- name: reclaim disk space from runner
run: |
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /opt/hostedtoolcache/CodeQL
- run: |
sudo apt-get update
sudo apt-get install -y golang-cfssl

View File

@@ -15,7 +15,8 @@ Feel free to open a Pull-Request to get yours listed.
| Vendor | Dinova | 2025 | [link](https://dinova.one/) | Dinova is an Italian cloud services provider that integrates Kamaji in its datacenters to offer fully managed Kubernetes clusters. |
| Vendor | Hikube | 2024 | [link](https://hikube.cloud/) | Hikube.cloud is a Swiss sovereign cloud platform with triple replication across three Swiss datacenters, offering enterprise-grade infrastructure with full data sovereignty. |
| End-user | KINX | 2024 | [link](https://kinx.net/?lang=en) | KINX is an Internet infrastructure service provider and will use kamaji for its new [Managed Kubernetes Service](https://kinx.net/service/cloud/kubernetes/intro/?lang=en). |
| Vendor | Netalia | 2025 | [link](https://www.netalia.it) | Netalia uses Kamaji for the Italian cloud
| End-user | Namecheap | 2025 | [link](https://www.namecheap.com/) | Namecheap is an ICANN-accredited domain registrar and web hosting company that provides a wide range of internet-related services and uses Kamaji for both internal and external services. |
| Vendor | Netalia | 2025 | [link](https://www.netalia.it) | Netalia uses Kamaji for the Italian cloud
| Vendor | Netsons | 2023 | [link](https://www.netsons.com) | Netsons is an Italian hosting and cloud provider and uses Kamaji in its [Managed Kubernetes](https://www.netsons.com/kubernetes) offering. |
| Vendor | NVIDIA | 2024 | [link](https://github.com/NVIDIA/doca-platform) | DOCA Platform Framework manages provisioning and service orchestration for NVIDIA Bluefield DPUs. |
| R&D | Orange | 2024 | [link](https://gitlab.com/Orange-OpenSource/kanod) | Orange is a French telecommunications company using Kamaji for experimental research purpose, with Kanod research solution. |

View File

@@ -177,7 +177,7 @@ datastore-postgres:
$(MAKE) NAME=gold _datastore-postgres
_datastore-etcd:
$(HELM) upgrade --install etcd-$(NAME) clastix/kamaji-etcd --create-namespace -n etcd-system --set datastore.enabled=true --set fullnameOverride=etcd-$(NAME)
$(HELM) upgrade --install etcd-$(NAME) clastix/kamaji-etcd --create-namespace -n $(NAMESPACE) --set datastore.enabled=true --set fullnameOverride=etcd-$(NAME) $(EXTRA_ARGS)
_datastore-nats:
$(MAKE) NAME=$(NAME) NAMESPACE=nats-system -C deploy/kine/nats nats
@@ -186,9 +186,11 @@ _datastore-nats:
datastore-etcd: helm
$(HELM) repo add clastix https://clastix.github.io/charts
$(HELM) repo update
$(MAKE) NAME=bronze _datastore-etcd
$(MAKE) NAME=silver _datastore-etcd
$(MAKE) NAME=gold _datastore-etcd
$(MAKE) NAME=bronze NAMESPACE=etcd-system _datastore-etcd
$(MAKE) NAME=silver NAMESPACE=etcd-system _datastore-etcd
$(MAKE) NAME=gold NAMESPACE=etcd-system _datastore-etcd
$(MAKE) NAME=primary NAMESPACE=kamaji-system EXTRA_ARGS='--set certManager.enabled=true --set certManager.issuerRef.kind=Issuer --set certManager.issuerRef.name=kamaji-selfsigned-issuer --set selfSignedCertificates.enabled=false' _datastore-etcd
$(MAKE) NAME=secondary NAMESPACE=kamaji-system EXTRA_ARGS='--set certManager.enabled=true --set certManager.ca.create=false --set certManager.ca.nameOverride=etcd-primary-ca --set certManager.issuerRef.kind=Issuer --set certManager.issuerRef.name=kamaji-selfsigned-issuer --set selfSignedCertificates.enabled=false' _datastore-etcd
datastore-nats: helm
$(HELM) repo add nats https://nats-io.github.io/k8s/helm/charts/

View File

@@ -355,6 +355,14 @@ func (p *Permissions) HasAnyLimitation() bool {
return false
}
// DataStoreOverride defines which kubernetes resource will be stored in a dedicated datastore.
type DataStoreOverride struct {
// Resource specifies which kubernetes resource to target.
Resource string `json:"resource,omitempty"`
// DataStore specifies the DataStore that should be used to store the Kubernetes data for the given Resource.
DataStore string `json:"dataStore,omitempty"`
}
// TenantControlPlaneSpec defines the desired state of TenantControlPlane.
// +kubebuilder:validation:XValidation:rule="!has(oldSelf.dataStore) || has(self.dataStore)", message="unsetting the dataStore is not supported"
// +kubebuilder:validation:XValidation:rule="!has(oldSelf.dataStoreSchema) || has(self.dataStoreSchema)", message="unsetting the dataStoreSchema is not supported"
@@ -389,8 +397,10 @@ type TenantControlPlaneSpec struct {
// to the user to avoid clashes between different TenantControlPlanes. If not set upon creation, Kamaji will default the
// DataStoreUsername by concatenating the namespace and name of the TenantControlPlane.
// +kubebuilder:validation:XValidation:rule="self == oldSelf",message="changing the dataStoreUsername is not supported"
DataStoreUsername string `json:"dataStoreUsername,omitempty"`
ControlPlane ControlPlane `json:"controlPlane"`
DataStoreUsername string `json:"dataStoreUsername,omitempty"`
// DataStoreOverride defines which kubernetes resources will be stored in dedicated datastores.
DataStoreOverrides []DataStoreOverride `json:"dataStoreOverrides,omitempty"`
ControlPlane ControlPlane `json:"controlPlane"`
// Kubernetes specification for tenant control plane
Kubernetes KubernetesSpec `json:"kubernetes"`
// NetworkProfile specifies how the network is

View File

@@ -538,6 +538,21 @@ func (in *DataStoreList) DeepCopyObject() runtime.Object {
return nil
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *DataStoreOverride) DeepCopyInto(out *DataStoreOverride) {
*out = *in
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataStoreOverride.
func (in *DataStoreOverride) DeepCopy() *DataStoreOverride {
if in == nil {
return nil
}
out := new(DataStoreOverride)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *DataStoreSetupStatus) DeepCopyInto(out *DataStoreSetupStatus) {
*out = *in
@@ -1591,6 +1606,11 @@ func (in *TenantControlPlaneList) DeepCopyObject() runtime.Object {
func (in *TenantControlPlaneSpec) DeepCopyInto(out *TenantControlPlaneSpec) {
*out = *in
out.WritePermissions = in.WritePermissions
if in.DataStoreOverrides != nil {
in, out := &in.DataStoreOverrides, &out.DataStoreOverrides
*out = make([]DataStoreOverride, len(*in))
copy(*out, *in)
}
in.ControlPlane.DeepCopyInto(&out.ControlPlane)
in.Kubernetes.DeepCopyInto(&out.Kubernetes)
in.NetworkProfile.DeepCopyInto(&out.NetworkProfile)

View File

@@ -6985,6 +6985,19 @@ versions:
Migration from one DataStore to another backed by the same Driver is possible. See: https://kamaji.clastix.io/guides/datastore-migration/
Migration from one DataStore to another backed by a different Driver is not supported.
type: string
dataStoreOverrides:
description: DataStoreOverride defines which kubernetes resources will be stored in dedicated datastores.
items:
description: DataStoreOverride defines which kubernetes resource will be stored in a dedicated datastore.
properties:
dataStore:
description: DataStore specifies the DataStore that should be used to store the Kubernetes data for the given Resource.
type: string
resource:
description: Resource specifies which kubernetes resource to target.
type: string
type: object
type: array
dataStoreSchema:
description: |-
DataStoreSchema allows to specify the name of the database (for relational DataStores) or the key prefix (for etcd). This

View File

@@ -6993,6 +6993,19 @@ spec:
Migration from one DataStore to another backed by the same Driver is possible. See: https://kamaji.clastix.io/guides/datastore-migration/
Migration from one DataStore to another backed by a different Driver is not supported.
type: string
dataStoreOverrides:
description: DataStoreOverride defines which kubernetes resources will be stored in dedicated datastores.
items:
description: DataStoreOverride defines which kubernetes resource will be stored in a dedicated datastore.
properties:
dataStore:
description: DataStore specifies the DataStore that should be used to store the Kubernetes data for the given Resource.
type: string
resource:
description: Resource specifies which kubernetes resource to target.
type: string
type: object
type: array
dataStoreSchema:
description: |-
DataStoreSchema allows to specify the name of the database (for relational DataStores) or the key prefix (for etcd). This

View File

@@ -26,18 +26,20 @@ import (
)
type GroupResourceBuilderConfiguration struct {
client client.Client
log logr.Logger
tcpReconcilerConfig TenantControlPlaneReconcilerConfig
tenantControlPlane kamajiv1alpha1.TenantControlPlane
ExpirationThreshold time.Duration
Connection datastore.Connection
DataStore kamajiv1alpha1.DataStore
KamajiNamespace string
KamajiServiceAccount string
KamajiService string
KamajiMigrateImage string
DiscoveryClient discovery.DiscoveryInterface
client client.Client
log logr.Logger
tcpReconcilerConfig TenantControlPlaneReconcilerConfig
tenantControlPlane kamajiv1alpha1.TenantControlPlane
ExpirationThreshold time.Duration
Connection datastore.Connection
DataStore kamajiv1alpha1.DataStore
DataStoreOverrides []builder.DataStoreOverrides
DataStoreOverriedsConnections map[string]datastore.Connection
KamajiNamespace string
KamajiServiceAccount string
KamajiService string
KamajiMigrateImage string
DiscoveryClient discovery.DiscoveryInterface
}
type GroupDeletableResourceBuilderConfiguration struct {
@@ -62,8 +64,9 @@ func GetResources(ctx context.Context, config GroupResourceBuilderConfiguration)
resources = append(resources, getKubernetesCertificatesResources(config.client, config.tcpReconcilerConfig, config.tenantControlPlane)...)
resources = append(resources, getKubeconfigResources(config.client, config.tcpReconcilerConfig, config.tenantControlPlane)...)
resources = append(resources, getKubernetesStorageResources(config.client, config.Connection, config.DataStore, config.ExpirationThreshold)...)
resources = append(resources, getKubernetesAdditionalStorageResources(config.client, config.DataStoreOverriedsConnections, config.DataStoreOverrides, config.ExpirationThreshold)...)
resources = append(resources, getKonnectivityServerRequirementsResources(config.client, config.ExpirationThreshold)...)
resources = append(resources, getKubernetesDeploymentResources(config.client, config.tcpReconcilerConfig, config.DataStore)...)
resources = append(resources, getKubernetesDeploymentResources(config.client, config.tcpReconcilerConfig, config.DataStore, config.DataStoreOverrides)...)
resources = append(resources, getKonnectivityServerPatchResources(config.client)...)
resources = append(resources, getDataStoreMigratingCleanup(config.client, config.KamajiNamespace)...)
resources = append(resources, getKubernetesIngressResources(config.client)...)
@@ -252,12 +255,42 @@ func getKubernetesStorageResources(c client.Client, dbConnection datastore.Conne
}
}
func getKubernetesDeploymentResources(c client.Client, tcpReconcilerConfig TenantControlPlaneReconcilerConfig, dataStore kamajiv1alpha1.DataStore) []resources.Resource {
func getKubernetesAdditionalStorageResources(c client.Client, dbConnections map[string]datastore.Connection, dataStoreOverrides []builder.DataStoreOverrides, threshold time.Duration) []resources.Resource {
res := make([]resources.Resource, 0, len(dataStoreOverrides))
for _, dso := range dataStoreOverrides {
datastore := dso.DataStore
res = append(res,
&ds.MultiTenancy{
DataStore: datastore,
},
&ds.Config{
Client: c,
ConnString: dbConnections[dso.Resource].GetConnectionString(),
DataStore: datastore,
IsOverride: true,
},
&ds.Setup{
Client: c,
Connection: dbConnections[dso.Resource],
DataStore: datastore,
},
&ds.Certificate{
Client: c,
DataStore: datastore,
CertExpirationThreshold: threshold,
})
}
return res
}
func getKubernetesDeploymentResources(c client.Client, tcpReconcilerConfig TenantControlPlaneReconcilerConfig, dataStore kamajiv1alpha1.DataStore, dataStoreOverrides []builder.DataStoreOverrides) []resources.Resource {
return []resources.Resource{
&resources.KubernetesDeploymentResource{
Client: c,
DataStore: dataStore,
KineContainerImage: tcpReconcilerConfig.KineContainerImage,
DataStoreOverrides: dataStoreOverrides,
},
}
}

View File

@@ -37,6 +37,7 @@ import (
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
"github.com/clastix/kamaji/controllers/finalizers"
"github.com/clastix/kamaji/controllers/utils"
controlplanebuilder "github.com/clastix/kamaji/internal/builders/controlplane"
"github.com/clastix/kamaji/internal/datastore"
kamajierrors "github.com/clastix/kamaji/internal/errors"
"github.com/clastix/kamaji/internal/resources"
@@ -157,6 +158,25 @@ func (r *TenantControlPlaneReconciler) Reconcile(ctx context.Context, req ctrl.R
}
defer dsConnection.Close()
dso, err := r.dataStoreOverride(ctx, tenantControlPlane)
if err != nil {
log.Error(err, "cannot retrieve the DataStoreOverrides for the given instance")
return ctrl.Result{}, err
}
dsoConnections := make(map[string]datastore.Connection, len(dso))
for _, ds := range dso {
dsoConnection, err := datastore.NewStorageConnection(ctx, r.Client, ds.DataStore)
if err != nil {
log.Error(err, "cannot generate the DataStoreOverride connection for the given instance")
return ctrl.Result{}, err
}
defer dsoConnection.Close()
dsoConnections[ds.Resource] = dsoConnection
}
if markedToBeDeleted && controllerutil.ContainsFinalizer(tenantControlPlane, finalizers.DatastoreFinalizer) {
log.Info("marked for deletion, performing clean-up")
@@ -183,17 +203,19 @@ func (r *TenantControlPlaneReconciler) Reconcile(ctx context.Context, req ctrl.R
}
groupResourceBuilderConfiguration := GroupResourceBuilderConfiguration{
client: r.Client,
log: log,
tcpReconcilerConfig: r.Config,
tenantControlPlane: *tenantControlPlane,
Connection: dsConnection,
DataStore: *ds,
KamajiNamespace: r.KamajiNamespace,
KamajiServiceAccount: r.KamajiServiceAccount,
KamajiService: r.KamajiService,
KamajiMigrateImage: r.KamajiMigrateImage,
DiscoveryClient: r.DiscoveryClient,
client: r.Client,
log: log,
tcpReconcilerConfig: r.Config,
tenantControlPlane: *tenantControlPlane,
Connection: dsConnection,
DataStore: *ds,
DataStoreOverrides: dso,
DataStoreOverriedsConnections: dsoConnections,
KamajiNamespace: r.KamajiNamespace,
KamajiServiceAccount: r.KamajiServiceAccount,
KamajiService: r.KamajiService,
KamajiMigrateImage: r.KamajiMigrateImage,
DiscoveryClient: r.DiscoveryClient,
}
registeredResources := GetResources(ctx, groupResourceBuilderConfiguration)
@@ -362,3 +384,21 @@ func (r *TenantControlPlaneReconciler) dataStore(ctx context.Context, tenantCont
return &ds, nil
}
func (r *TenantControlPlaneReconciler) dataStoreOverride(ctx context.Context, tenantControlPlane *kamajiv1alpha1.TenantControlPlane) ([]controlplanebuilder.DataStoreOverrides, error) {
datastores := make([]controlplanebuilder.DataStoreOverrides, 0, len(tenantControlPlane.Spec.DataStoreOverrides))
for _, dso := range tenantControlPlane.Spec.DataStoreOverrides {
var ds kamajiv1alpha1.DataStore
if err := r.Client.Get(ctx, k8stypes.NamespacedName{Name: dso.DataStore}, &ds); err != nil {
return nil, errors.Wrap(err, "cannot retrieve *kamajiv1alpha.DataStore object")
}
if ds.Spec.Driver != kamajiv1alpha1.EtcdDriver {
return nil, errors.New("DataStoreOverrides can only use ETCD driver")
}
datastores = append(datastores, controlplanebuilder.DataStoreOverrides{Resource: dso.Resource, DataStore: ds})
}
return datastores, nil
}

View File

@@ -0,0 +1,78 @@
# Datastore Overrides
Kamaji offers the possibility of having multiple ETCD clusters backing different resources of the k8s api server by configuring the [`--etcd-servers-overrides`](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-apiserver/#:~:text=%2D%2Detcd%2Dservers%2Doverrides%20strings) flag. This feature can be useful for massive clusters to store resources with high churn in a dedicated ETCD cluster.
## Install Datastores
Create a self-signed cert-manager `ClusterIssuer`.
```bash
echo 'apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: self-signed
spec:
selfSigned: {}
' | kubectl apply -f -
```
Install two Datastores, a primary and a secondary that will be used for `/events` resources.
```bash
helm install etcd-primary clastix/kamaji-etcd -n kamaji-etcd --create-namespace \
--set selfSignedCertificates.enabled=false \
--set certManager.enabled=true \
--set certManager.issuerRef.kind=ClusterIssuer \
--set certManager.issuerRef.name=self-signed
```
For the secondary Datastore, use the cert-manager CA created by the `etcd-primary` helm release.
```bash
helm install etcd-secondary clastix/kamaji-etcd -n kamaji-etcd --create-namespace \
--set selfSignedCertificates.enabled=false \
--set certManager.enabled=true \
--set certManager.ca.create=false \
--set certManager.ca.nameOverride=etcd-primary-kamaji-etcd-ca \
--set certManager.issuerRef.kind=ClusterIssuer \
--set certManager.issuerRef.name=self-signed
```
## Create a Tenant Control Plane
Using the `spec.dataStoreOverrides` field, Datastores different from the one used in `spec.dataStore` can be used to store specific resources.
```bash
echo 'apiVersion: kamaji.clastix.io/v1alpha1
kind: TenantControlPlane
metadata:
name: k8s-133
labels:
tenant.clastix.io: k8s-133
spec:
controlPlane:
deployment:
replicas: 2
service:
serviceType: LoadBalancer
kubernetes:
version: "v1.33.1"
kubelet:
cgroupfs: systemd
dataStore: etcd-primary-kamaji-etcd
dataStoreOverrides:
- resource: "/events" # Store events in the secondary ETCD
dataStore: etcd-secondary-kamaji-etcd
networkProfile:
port: 6443
addons:
coreDNS: {}
kubeProxy: {}
konnectivity:
server:
port: 8132
agent:
mode: DaemonSet
' | k apply -f -
```
## Considerations
Only built-in resources can be tagetted by `--etcd-servers-overrides`, it is currently not possible to target Custom Resources.

View File

@@ -28500,6 +28500,13 @@ Migration from one DataStore to another backed by the same Driver is possible. S
Migration from one DataStore to another backed by a different Driver is not supported.<br/>
</td>
<td>false</td>
</tr><tr>
<td><b><a href="#tenantcontrolplanespecdatastoreoverridesindex">dataStoreOverrides</a></b></td>
<td>[]object</td>
<td>
DataStoreOverride defines which kubernetes resources will be stored in dedicated datastores.<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>dataStoreSchema</b></td>
<td>string</td>
@@ -42214,6 +42221,38 @@ In case this value is set, kubeadm does not change automatically the version of
</table>
<span id="tenantcontrolplanespecdatastoreoverridesindex">`TenantControlPlane.spec.dataStoreOverrides[index]`</span>
DataStoreOverride defines which kubernetes resource will be stored in a dedicated datastore.
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody><tr>
<td><b>dataStore</b></td>
<td>string</td>
<td>
DataStore specifies the DataStore that should be used to store the Kubernetes data for the given Resource.<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>resource</b></td>
<td>string</td>
<td>
Resource specifies which kubernetes resource to target.<br/>
</td>
<td>false</td>
</tr></tbody>
</table>
<span id="tenantcontrolplanespecnetworkprofile">`TenantControlPlane.spec.networkProfile`</span>

View File

@@ -76,6 +76,7 @@ nav:
- guides/pausing.md
- guides/write-permissions.md
- guides/datastore-migration.md
- guides/datastore-overrides.md
- guides/gitops.md
- guides/console.md
- guides/kubeconfig-generator.md

View File

@@ -0,0 +1,68 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package e2e
import (
"context"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
pointer "k8s.io/utils/ptr"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
)
var _ = Describe("Deploy a TenantControlPlane resource with DataStoreOverrides", func() {
// Fill TenantControlPlane object
tcp := &kamajiv1alpha1.TenantControlPlane{
ObjectMeta: metav1.ObjectMeta{
Name: "tcp-clusterip",
Namespace: "default",
},
Spec: kamajiv1alpha1.TenantControlPlaneSpec{
DataStore: "etcd-primary",
DataStoreOverrides: []kamajiv1alpha1.DataStoreOverride{{
Resource: "/events",
DataStore: "etcd-secondary",
}},
ControlPlane: kamajiv1alpha1.ControlPlane{
Deployment: kamajiv1alpha1.DeploymentSpec{
Replicas: pointer.To(int32(1)),
},
Service: kamajiv1alpha1.ServiceSpec{
ServiceType: "ClusterIP",
},
},
NetworkProfile: kamajiv1alpha1.NetworkProfileSpec{
Address: "172.18.0.2",
},
Kubernetes: kamajiv1alpha1.KubernetesSpec{
Version: "v1.23.6",
Kubelet: kamajiv1alpha1.KubeletSpec{
CGroupFS: "cgroupfs",
},
AdmissionControllers: kamajiv1alpha1.AdmissionControllers{
"LimitRanger",
"ResourceQuota",
},
},
Addons: kamajiv1alpha1.AddonsSpec{},
},
}
// Create a TenantControlPlane resource into the cluster
JustBeforeEach(func() {
Expect(k8sClient.Create(context.Background(), tcp)).NotTo(HaveOccurred())
})
// Delete the TenantControlPlane resource after test is finished
JustAfterEach(func() {
Expect(k8sClient.Delete(context.Background(), tcp)).Should(Succeed())
})
// Check if TenantControlPlane resource has been created
It("Should be Ready", func() {
StatusMustEqualTo(tcp, kamajiv1alpha1.VersionReady)
})
})

10
go.mod
View File

@@ -15,11 +15,11 @@ require (
github.com/json-iterator/go v1.1.12
github.com/juju/mutex/v2 v2.0.0
github.com/nats-io/nats.go v1.47.0
github.com/onsi/ginkgo/v2 v2.27.2
github.com/onsi/gomega v1.38.2
github.com/onsi/ginkgo/v2 v2.27.3
github.com/onsi/gomega v1.38.3
github.com/pkg/errors v0.9.1
github.com/prometheus/client_golang v1.23.2
github.com/spf13/cobra v1.10.1
github.com/spf13/cobra v1.10.2
github.com/spf13/pflag v1.0.10
github.com/spf13/viper v1.21.0
github.com/testcontainers/testcontainers-go v0.40.0
@@ -34,10 +34,10 @@ require (
k8s.io/cluster-bootstrap v0.0.0
k8s.io/klog/v2 v2.130.1
k8s.io/kubelet v0.0.0
k8s.io/kubernetes v1.34.2
k8s.io/kubernetes v1.34.3
k8s.io/utils v0.0.0-20250820121507-0af2bda4dd1d
sigs.k8s.io/controller-runtime v0.22.4
sigs.k8s.io/gateway-api v1.4.0
sigs.k8s.io/gateway-api v1.4.1
)
require (

20
go.sum
View File

@@ -253,10 +253,10 @@ github.com/nxadm/tail v1.4.4 h1:DQuhQpB1tVlglWS2hLQ5OV6B5r8aGxSrPc5Qo6uTN78=
github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A=
github.com/onsi/ginkgo v1.14.2 h1:8mVmC9kjFFmA8H4pKMUhcblgifdkOIXPvbhN1T36q1M=
github.com/onsi/ginkgo v1.14.2/go.mod h1:iSB4RoI2tjJc9BBv4NKIKWKya62Rps+oPG/Lv9klQyY=
github.com/onsi/ginkgo/v2 v2.27.2 h1:LzwLj0b89qtIy6SSASkzlNvX6WktqurSHwkk2ipF/Ns=
github.com/onsi/ginkgo/v2 v2.27.2/go.mod h1:ArE1D/XhNXBXCBkKOLkbsb2c81dQHCRcF5zwn/ykDRo=
github.com/onsi/gomega v1.38.2 h1:eZCjf2xjZAqe+LeWvKb5weQ+NcPwX84kqJ0cZNxok2A=
github.com/onsi/gomega v1.38.2/go.mod h1:W2MJcYxRGV63b418Ai34Ud0hEdTVXq9NW9+Sx6uXf3k=
github.com/onsi/ginkgo/v2 v2.27.3 h1:ICsZJ8JoYafeXFFlFAG75a7CxMsJHwgKwtO+82SE9L8=
github.com/onsi/ginkgo/v2 v2.27.3/go.mod h1:ArE1D/XhNXBXCBkKOLkbsb2c81dQHCRcF5zwn/ykDRo=
github.com/onsi/gomega v1.38.3 h1:eTX+W6dobAYfFeGC2PV6RwXRu/MyT+cQguijutvkpSM=
github.com/onsi/gomega v1.38.3/go.mod h1:ZCU1pkQcXDO5Sl9/VVEGlDyp+zm0m1cmeG5TOzLgdh4=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
github.com/opencontainers/image-spec v1.1.1 h1:y0fUlFfIZhPF1W537XOLg0/fcx6zcHCJwooC2xJA040=
@@ -301,8 +301,8 @@ github.com/spf13/afero v1.15.0 h1:b/YBCLWAJdFWJTN9cLhiXXcD7mzKn9Dm86dNnfyQw1I=
github.com/spf13/afero v1.15.0/go.mod h1:NC2ByUVxtQs4b3sIUphxK0NioZnmxgyCrfzeuq8lxMg=
github.com/spf13/cast v1.10.0 h1:h2x0u2shc1QuLHfxi+cTJvs30+ZAHOGRic8uyGTDWxY=
github.com/spf13/cast v1.10.0/go.mod h1:jNfB8QC9IA6ZuY2ZjDp0KtFO2LZZlg4S/7bzP6qqeHo=
github.com/spf13/cobra v1.10.1 h1:lJeBwCfmrnXthfAupyUTzJ/J4Nc1RsHC/mSRU2dll/s=
github.com/spf13/cobra v1.10.1/go.mod h1:7SmJGaTHFVBY0jW4NXGluQoLvhqFQM+6XSKD+P4XaB0=
github.com/spf13/cobra v1.10.2 h1:DMTTonx5m65Ic0GOoRY2c16WCbHxOOw6xxezuLaBpcU=
github.com/spf13/cobra v1.10.2/go.mod h1:7C1pvHqHw5A4vrJfjNwvOdzYu0Gml16OCs2GRiTUUS4=
github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk=
github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
@@ -547,8 +547,8 @@ k8s.io/kube-proxy v0.34.0 h1:gU7MVbJHiXyPX8bXnod4bANtSC7rZSKkkLmM8gUqwT4=
k8s.io/kube-proxy v0.34.0/go.mod h1:tfwI8dCKm5Q0r+aVIbrq/aC36Kk936w2LZu8/rvJzWI=
k8s.io/kubelet v0.34.0 h1:1nZt1Q6Kfx7xCaTS9vnqR9sjZDxf3cRSQkAFCczULmc=
k8s.io/kubelet v0.34.0/go.mod h1:NqbF8ViVettlZbf9hw9DJhubaWn7rGvDDTcLMDm6tQ0=
k8s.io/kubernetes v1.34.2 h1:WQdDvYJazkmkwSncgNwGvVtaCt4TYXIU3wSMRgvp3MI=
k8s.io/kubernetes v1.34.2/go.mod h1:m6pZk6a179pRo2wsTiCPORJ86iOEQmfIzUvtyEF8BwA=
k8s.io/kubernetes v1.34.3 h1:0TfljWbhEF5DBks+WFMSrvKfxBLo4vnZuqORjLMiyT4=
k8s.io/kubernetes v1.34.3/go.mod h1:m6pZk6a179pRo2wsTiCPORJ86iOEQmfIzUvtyEF8BwA=
k8s.io/system-validators v1.10.2 h1:7rC7VdrQCaM55E08Pw3I1v1Op9ObLxdKAu5Ff5hIPwY=
k8s.io/system-validators v1.10.2/go.mod h1:awfSS706v9R12VC7u7K89FKfqVy44G+E0L1A0FX9Wmw=
k8s.io/utils v0.0.0-20250820121507-0af2bda4dd1d h1:wAhiDyZ4Tdtt7e46e9M5ZSAJ/MnPGPs+Ki1gHw4w1R0=
@@ -559,8 +559,8 @@ sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.31.2 h1:jpcvIRr3GLoUo
sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.31.2/go.mod h1:Ve9uj1L+deCXFrPOk1LpFXqTg7LCFzFso6PA48q/XZw=
sigs.k8s.io/controller-runtime v0.22.4 h1:GEjV7KV3TY8e+tJ2LCTxUTanW4z/FmNB7l327UfMq9A=
sigs.k8s.io/controller-runtime v0.22.4/go.mod h1:+QX1XUpTXN4mLoblf4tqr5CQcyHPAki2HLXqQMY6vh8=
sigs.k8s.io/gateway-api v1.4.0 h1:ZwlNM6zOHq0h3WUX2gfByPs2yAEsy/EenYJB78jpQfQ=
sigs.k8s.io/gateway-api v1.4.0/go.mod h1:AR5RSqciWP98OPckEjOjh2XJhAe2Na4LHyXD2FUY7Qk=
sigs.k8s.io/gateway-api v1.4.1 h1:NPxFutNkKNa8UfLd2CMlEuhIPMQgDQ6DXNKG9sHbJU8=
sigs.k8s.io/gateway-api v1.4.1/go.mod h1:AR5RSqciWP98OPckEjOjh2XJhAe2Na4LHyXD2FUY7Qk=
sigs.k8s.io/json v0.0.0-20250730193827-2d320260d730 h1:IpInykpT6ceI+QxKBbEflcR5EXP7sU1kvOlxwZh5txg=
sigs.k8s.io/json v0.0.0-20250730193827-2d320260d730/go.mod h1:mdzfpAEoE6DHQEN0uh9ZbOCuHbLK5wOm7dK4ctXE9Tg=
sigs.k8s.io/kustomize/api v0.20.1 h1:iWP1Ydh3/lmldBnH/S5RXgT98vWYMaTUL1ADcr+Sv7I=

View File

@@ -52,9 +52,15 @@ const (
kineInitContainerName = "chmod"
)
type DataStoreOverrides struct {
Resource string
DataStore kamajiv1alpha1.DataStore
}
type Deployment struct {
KineContainerImage string
DataStore kamajiv1alpha1.DataStore
DataStoreOverrides []DataStoreOverrides
Client client.Client
}
@@ -711,11 +717,29 @@ func (d Deployment) buildKubeAPIServerCommand(tenantControlPlane kamajiv1alpha1.
desiredArgs["--etcd-keyfile"] = "/etc/kubernetes/pki/etcd/server.key"
}
if len(d.DataStoreOverrides) != 0 {
desiredArgs["--etcd-servers-overrides"] = d.etcdServersOverrides()
}
// Order matters, here: extraArgs could try to overwrite some arguments managed by Kamaji and that would be crucial.
// Adding as first element of the array of maps, we're sure that these overrides will be sanitized by our configuration.
return utilities.MergeMaps(current, desiredArgs, extraArgs)
}
func (d Deployment) etcdServersOverrides() string {
dataStoreOverridesEndpoints := make([]string, 0, len(d.DataStoreOverrides))
for _, dso := range d.DataStoreOverrides {
httpsEndpoints := make([]string, 0, len(dso.DataStore.Spec.Endpoints))
for _, ep := range dso.DataStore.Spec.Endpoints {
httpsEndpoints = append(httpsEndpoints, fmt.Sprintf("https://%s", ep))
}
dataStoreOverridesEndpoints = append(dataStoreOverridesEndpoints, fmt.Sprintf("%s#%s", dso.Resource, strings.Join(httpsEndpoints, ";")))
}
return strings.Join(dataStoreOverridesEndpoints, ",")
}
func (d Deployment) secretProjection(secretName, certKeyName, keyName string) *corev1.SecretProjection {
return &corev1.SecretProjection{
LocalObjectReference: corev1.LocalObjectReference{

View File

@@ -0,0 +1,46 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package controlplane
import (
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
)
var _ = Describe("Controlplane Deployment", func() {
var d Deployment
BeforeEach(func() {
d = Deployment{
DataStoreOverrides: []DataStoreOverrides{{
Resource: "/events",
DataStore: kamajiv1alpha1.DataStore{
Spec: kamajiv1alpha1.DataStoreSpec{
Endpoints: kamajiv1alpha1.Endpoints{"etcd-0", "etcd-1", "etcd-2"},
},
},
}},
}
})
Describe("DataStoreOverrides flag generation", func() {
It("should generate valid --etcd-servers-overrides value", func() {
etcdSerVersOverrides := d.etcdServersOverrides()
Expect(etcdSerVersOverrides).To(Equal("/events#https://etcd-0;https://etcd-1;https://etcd-2"))
})
It("should generate valid --etcd-servers-overrides value with 2 DataStoreOverrides", func() {
d.DataStoreOverrides = append(d.DataStoreOverrides, DataStoreOverrides{
Resource: "/pods",
DataStore: kamajiv1alpha1.DataStore{
Spec: kamajiv1alpha1.DataStoreSpec{
Endpoints: kamajiv1alpha1.Endpoints{"etcd-3", "etcd-4", "etcd-5"},
},
},
})
etcdSerVersOverrides := d.etcdServersOverrides()
Expect(etcdSerVersOverrides).To(Equal("/events#https://etcd-0;https://etcd-1;https://etcd-2,/pods#https://etcd-3;https://etcd-4;https://etcd-5"))
})
})
})

View File

@@ -29,7 +29,7 @@ const (
mysqlShowGrantsStatement = "SHOW GRANTS FOR `%s`@`%%`"
mysqlCreateDBStatement = "CREATE DATABASE IF NOT EXISTS %s"
mysqlCreateUserStatement = "CREATE USER `%s`@`%%` IDENTIFIED BY '%s'"
mysqlGrantPrivilegesStatement = "GRANT ALL PRIVILEGES ON `%s`.* TO `%s`@`%%`"
mysqlGrantPrivilegesStatement = "GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, ALTER, INDEX ON `%s`.* TO `%s`@`%%`"
mysqlDropDBStatement = "DROP DATABASE IF EXISTS `%s`"
mysqlDropUserStatement = "DROP USER IF EXISTS `%s`"
mysqlRevokePrivilegesStatement = "REVOKE ALL PRIVILEGES ON `%s`.* FROM `%s`"
@@ -167,7 +167,7 @@ func (c *MySQLConnection) CreateDB(ctx context.Context, dbName string) error {
}
func (c *MySQLConnection) GrantPrivileges(ctx context.Context, user, dbName string) error {
if err := c.mutate(ctx, mysqlGrantPrivilegesStatement, user, dbName); err != nil {
if err := c.mutate(ctx, mysqlGrantPrivilegesStatement, dbName, user); err != nil {
return errors.NewGrantPrivilegesError(err)
}
@@ -229,7 +229,7 @@ func (c *MySQLConnection) GrantPrivilegesExists(_ context.Context, user, dbName
return false, errors.NewGrantPrivilegesError(err)
}
expected := fmt.Sprintf(mysqlGrantPrivilegesStatement, user, dbName)
expected := fmt.Sprintf(mysqlGrantPrivilegesStatement, dbName, user)
var grant string
for rows.Next() {
@@ -262,7 +262,7 @@ func (c *MySQLConnection) DeleteDB(ctx context.Context, dbName string) error {
}
func (c *MySQLConnection) RevokePrivileges(ctx context.Context, user, dbName string) error {
if err := c.mutate(ctx, mysqlRevokePrivilegesStatement, user, dbName); err != nil {
if err := c.mutate(ctx, mysqlRevokePrivilegesStatement, dbName, user); err != nil {
return errors.NewRevokePrivilegesError(err)
}

View File

@@ -25,7 +25,7 @@ const (
postgresqlShowOwnershipStatement = "SELECT 't' FROM pg_catalog.pg_database AS d WHERE d.datname = ? AND pg_catalog.pg_get_userbyid(d.datdba) = ?"
postgresqlShowTableOwnershipStatement = "SELECT 't' from pg_tables where tableowner = ? AND tablename = ?"
postgresqlKineTableExistsStatement = "SELECT 't' FROM pg_tables WHERE schemaname = ? AND tablename = ?"
postgresqlGrantPrivilegesStatement = "GRANT ALL PRIVILEGES ON DATABASE %s TO %s"
postgresqlGrantPrivilegesStatement = "GRANT CONNECT, CREATE ON DATABASE %s TO %s"
postgresqlChangeOwnerStatement = "ALTER DATABASE %s OWNER TO %s"
postgresqlRevokePrivilegesStatement = "REVOKE ALL PRIVILEGES ON DATABASE %s FROM %s"
postgresqlDropRoleStatement = "DROP ROLE %s"

View File

@@ -30,6 +30,7 @@ type Config struct {
Client client.Client
ConnString string
DataStore kamajiv1alpha1.DataStore
IsOverride bool
}
func (r *Config) GetHistogram() prometheus.Histogram {
@@ -103,10 +104,12 @@ func (r *Config) GetName() string {
}
func (r *Config) UpdateTenantControlPlaneStatus(_ context.Context, tenantControlPlane *kamajiv1alpha1.TenantControlPlane) error {
tenantControlPlane.Status.Storage.Driver = string(r.DataStore.Spec.Driver)
tenantControlPlane.Status.Storage.DataStoreName = r.DataStore.GetName()
tenantControlPlane.Status.Storage.Config.SecretName = r.resource.GetName()
tenantControlPlane.Status.Storage.Config.Checksum = utilities.GetObjectChecksum(r.resource)
if !r.IsOverride {
tenantControlPlane.Status.Storage.Driver = string(r.DataStore.Spec.Driver)
tenantControlPlane.Status.Storage.DataStoreName = r.DataStore.GetName()
tenantControlPlane.Status.Storage.Config.SecretName = r.resource.GetName()
tenantControlPlane.Status.Storage.Config.Checksum = utilities.GetObjectChecksum(r.resource)
}
return nil
}

View File

@@ -22,6 +22,7 @@ type KubernetesDeploymentResource struct {
resource *appsv1.Deployment
Client client.Client
DataStore kamajiv1alpha1.DataStore
DataStoreOverrides []builder.DataStoreOverrides
KineContainerImage string
}
@@ -66,6 +67,7 @@ func (r *KubernetesDeploymentResource) mutate(ctx context.Context, tenantControl
Client: r.Client,
DataStore: r.DataStore,
KineContainerImage: r.KineContainerImage,
DataStoreOverrides: r.DataStoreOverrides,
}).Build(ctx, r.resource, *tenantControlPlane)
return controllerutil.SetControllerReference(tenantControlPlane, r.resource, r.Client.Scheme())

View File

@@ -30,7 +30,7 @@ func (t TenantControlPlaneDataStore) OnCreate(object runtime.Object) AdmissionRe
return nil, t.check(ctx, tcp.Spec.DataStore)
}
return nil, nil
return nil, t.checkDataStoreOverrides(ctx, tcp)
}
}
@@ -61,3 +61,19 @@ func (t TenantControlPlaneDataStore) check(ctx context.Context, dataStoreName st
return nil
}
func (t TenantControlPlaneDataStore) checkDataStoreOverrides(ctx context.Context, tcp *kamajiv1alpha1.TenantControlPlane) error {
overrideCheck := make(map[string]struct{}, 0)
for _, ds := range tcp.Spec.DataStoreOverrides {
if _, exists := overrideCheck[ds.Resource]; !exists {
overrideCheck[ds.Resource] = struct{}{}
} else {
return fmt.Errorf("duplicate resource override in Spec.DataStoreOverrides")
}
if err := t.check(ctx, ds.DataStore); err != nil {
return err
}
}
return nil
}

View File

@@ -0,0 +1,94 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package handlers
import (
"context"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime"
utilruntime "k8s.io/apimachinery/pkg/util/runtime"
fakeclient "sigs.k8s.io/controller-runtime/pkg/client/fake"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
)
var _ = Describe("TCP Datastore webhook", func() {
var (
ctx context.Context
t TenantControlPlaneDataStore
tcp *kamajiv1alpha1.TenantControlPlane
)
BeforeEach(func() {
scheme := runtime.NewScheme()
utilruntime.Must(kamajiv1alpha1.AddToScheme(scheme))
ctx = context.Background()
t = TenantControlPlaneDataStore{
Client: fakeclient.NewClientBuilder().WithScheme(scheme).WithObjects(&kamajiv1alpha1.DataStore{
ObjectMeta: metav1.ObjectMeta{
Name: "foo",
},
}, &kamajiv1alpha1.DataStore{
ObjectMeta: metav1.ObjectMeta{
Name: "bar",
},
}).Build(),
}
tcp = &kamajiv1alpha1.TenantControlPlane{
ObjectMeta: metav1.ObjectMeta{
Name: "tcp",
Namespace: "default",
},
Spec: kamajiv1alpha1.TenantControlPlaneSpec{},
}
})
Describe("validation should succeed without DataStoreOverrides", func() {
It("should validate TCP without DataStoreOverrides", func() {
err := t.checkDataStoreOverrides(ctx, tcp)
Expect(err).ToNot(HaveOccurred())
})
})
Describe("validation should fail with duplicate resources in DataStoreOverrides", func() {
It("should fail to validate TCP with duplicate resources in DataStoreOverrides", func() {
tcp.Spec.DataStoreOverrides = []kamajiv1alpha1.DataStoreOverride{{
Resource: "/event",
DataStore: "foo",
}, {
Resource: "/event",
DataStore: "bar",
}}
err := t.checkDataStoreOverrides(ctx, tcp)
Expect(err).To(HaveOccurred())
})
})
Describe("validation should succeed with valid DataStoreOverrides", func() {
It("should validate TCP with valid DataStoreOverrides", func() {
tcp.Spec.DataStoreOverrides = []kamajiv1alpha1.DataStoreOverride{{
Resource: "/leases",
DataStore: "foo",
}, {
Resource: "/event",
DataStore: "bar",
}}
err := t.checkDataStoreOverrides(ctx, tcp)
Expect(err).ToNot(HaveOccurred())
})
})
Describe("validation should fail with nonexistent DataStoreOverrides", func() {
It("should fail to validate TCP with nonexistent DataStoreOverrides", func() {
tcp.Spec.DataStoreOverrides = []kamajiv1alpha1.DataStoreOverride{{
Resource: "/leases",
DataStore: "baz",
}}
err := t.checkDataStoreOverrides(ctx, tcp)
Expect(err).To(HaveOccurred())
})
})
})

View File

@@ -67,6 +67,20 @@ func (t TenantControlPlaneDeployment) OnUpdate(newObject runtime.Object, oldObje
}
t.DeploymentBuilder.DataStore = ds
dataStoreOverrides := make([]controlplane.DataStoreOverrides, 0, len(tcp.Spec.DataStoreOverrides))
for _, dso := range tcp.Spec.DataStoreOverrides {
ds := kamajiv1alpha1.DataStore{}
if err := t.Client.Get(ctx, types.NamespacedName{Name: dso.DataStore}, &ds); err != nil {
return nil, err
}
dataStoreOverrides = append(dataStoreOverrides, controlplane.DataStoreOverrides{
Resource: dso.Resource,
DataStore: ds,
})
}
t.DeploymentBuilder.DataStoreOverrides = dataStoreOverrides
deployment := appsv1.Deployment{}
deployment.Name = tcp.Name
deployment.Namespace = tcp.Namespace