Compare commits

...

8 Commits

Author SHA1 Message Date
Dario Tranchitella
309d9889e8 refactor!: datastore conditions (#1087)
* feat(api)!: readiness and conditions for datastore

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

* docs: readiness and conditions for datastore

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

* feat(controller): readiness and conditions for datastore

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

* refactor: default datastore checked at reconciliation time

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

* chore(helm): upgrading to v4.1.1

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

---------

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>
2026-02-21 12:55:10 +01:00
Patryk Rostkowski
1d05f2bb4c fix: strip load balancer ports before updating TenantControlPlane status (#1082)
Signed-off-by: Patryk Rostkowski <patrostkowski@gmail.com>
2026-02-21 10:45:43 +01:00
Parth Yadav
cc8a8e14fd fix(gateway): allow explicit parentRefs for kube-apiserver TLSRoute (#1074)
Previously, when Kamaji created TLSRoutes for the kube-apiserver and
konnectivity, it automatically set the parentRefs `port` and `sectionName`,
overriding any user-provided values via TCP spec. This prevented users
from targeting specific Gateway listeners.

This change allows users to explicitly define parentRefs for the
kube-apiserver TLSRoute through `TCP.spec.controlPlane.gateway.parentRefs`.

The konnectivity TLSRoute behavior remains unchanged.

Signed-off-by: Parth Yadav <parth@coredge.io>
2026-02-18 18:45:23 +01:00
dependabot[bot]
13a3aa70f5 feat(deps): bump k8s.io/kubernetes in the k8s group (#1079)
Bumps the k8s group with 1 update: [k8s.io/kubernetes](https://github.com/kubernetes/kubernetes).


Updates `k8s.io/kubernetes` from 1.35.0 to 1.35.1
- [Release notes](https://github.com/kubernetes/kubernetes/releases)
- [Commits](https://github.com/kubernetes/kubernetes/compare/v1.35.0...v1.35.1)

---
updated-dependencies:
- dependency-name: k8s.io/kubernetes
  dependency-version: 1.35.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: k8s
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-18 18:45:06 +01:00
dependabot[bot]
49ea678047 feat(deps): bump the etcd group with 2 updates (#1078)
Bumps the etcd group with 2 updates: [go.etcd.io/etcd/api/v3](https://github.com/etcd-io/etcd) and [go.etcd.io/etcd/client/v3](https://github.com/etcd-io/etcd).


Updates `go.etcd.io/etcd/api/v3` from 3.6.7 to 3.6.8
- [Release notes](https://github.com/etcd-io/etcd/releases)
- [Commits](https://github.com/etcd-io/etcd/compare/v3.6.7...v3.6.8)

Updates `go.etcd.io/etcd/client/v3` from 3.6.7 to 3.6.8
- [Release notes](https://github.com/etcd-io/etcd/releases)
- [Commits](https://github.com/etcd-io/etcd/compare/v3.6.7...v3.6.8)

---
updated-dependencies:
- dependency-name: go.etcd.io/etcd/api/v3
  dependency-version: 3.6.8
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: etcd
- dependency-name: go.etcd.io/etcd/client/v3
  dependency-version: 3.6.8
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: etcd
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-16 14:23:13 +01:00
Dario Tranchitella
830d56dac1 chore(makefile): using only ipv4 for metallb in ci (#1076)
Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>
2026-02-13 11:48:29 +01:00
Matteo Ruina
a9c2c0de89 fix(style): migrate from deprecated github.com/pkg/errors package (#1071)
* refactor: migrate error packages from pkg/errors to stdlib

Replace github.com/pkg/errors with Go standard library error handling
in foundation error packages:

- internal/datastore/errors: errors.Wrap -> fmt.Errorf with %w
- internal/errors: errors.As -> stdlib errors.As
- controllers/soot/controllers/errors: errors.New -> stdlib errors.New

Part 1 of 4 in the pkg/errors migration.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: migrate datastore package from pkg/errors to stdlib

Replace github.com/pkg/errors with Go standard library error handling
in the datastore layer:

- connection.go: errors.Wrap -> fmt.Errorf with %w
- datastore.go: errors.Wrap -> fmt.Errorf with %w
- etcd.go: goerrors alias removed, use stdlib errors.As
- nats.go: errors.Wrap/Is/New -> stdlib equivalents
- postgresql.go: goerrors.Wrap -> fmt.Errorf with %w

Part 2 of 4 in the pkg/errors migration.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: migrate internal packages from pkg/errors to stdlib (partial)

Replace github.com/pkg/errors with Go standard library error handling
in internal packages:

- internal/builders/controlplane: errors.Wrap -> fmt.Errorf
- internal/crypto: errors.Wrap -> fmt.Errorf
- internal/kubeadm: errors.Wrap/Wrapf -> fmt.Errorf
- internal/upgrade: errors.Wrap -> fmt.Errorf
- internal/webhook: errors.Wrap -> fmt.Errorf

Part 3 of 4 in the pkg/errors migration.

Remaining files: internal/resources/*.go (8 files, 42 occurrences)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(resources): migrate from pkg/errors to stdlib

Replace github.com/pkg/errors with Go standard library:
- errors.Wrap(err, msg) → fmt.Errorf("msg: %w", err)
- errors.New(msg) → errors.New(msg)

Files migrated:
- internal/resources/kubeadm_phases.go
- internal/resources/kubeadm_upgrade.go
- internal/resources/kubeadm_utils.go
- internal/resources/datastore/datastore_multitenancy.go
- internal/resources/datastore/datastore_setup.go
- internal/resources/datastore/datastore_storage_config.go
- internal/resources/addons/coredns.go
- internal/resources/addons/kube_proxy.go

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(controllers): migrate from pkg/errors to stdlib

Replace github.com/pkg/errors with Go standard library:
- errors.Wrap(err, msg) → fmt.Errorf("msg: %w", err)
- errors.New(msg) → errors.New(msg) (stdlib)
- errors.Is/As → errors.Is/As (stdlib)

Files migrated:
- controllers/datastore_controller.go
- controllers/kubeconfiggenerator_controller.go
- controllers/tenantcontrolplane_controller.go
- controllers/telemetry_controller.go
- controllers/certificate_lifecycle_controller.go

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(soot): migrate from pkg/errors to stdlib

Replace github.com/pkg/errors with Go standard library:
- errors.Is() now uses stdlib errors.Is()

Files migrated:
- controllers/soot/controllers/kubeproxy.go
- controllers/soot/controllers/migrate.go
- controllers/soot/controllers/coredns.go
- controllers/soot/controllers/konnectivity.go
- controllers/soot/controllers/kubeadm_phase.go

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(api,cmd): migrate from pkg/errors to stdlib

Replace github.com/pkg/errors with Go standard library:
- errors.Wrap(err, msg) → fmt.Errorf("msg: %w", err)

Files migrated:
- api/v1alpha1/tenantcontrolplane_funcs.go
- cmd/utils/k8s_version.go

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: run go mod tidy after pkg/errors migration

The github.com/pkg/errors package moved from direct to indirect
dependency. It remains as an indirect dependency because other
packages in the dependency tree still use it.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(datastore): use errors.Is for sentinel error comparison

The stdlib errors.As expects a pointer to a concrete error type, not
a pointer to an error value. For comparing against sentinel errors
like rpctypes.ErrGRPCUserNotFound, errors.Is should be used instead.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: resolve golangci-lint errors

- Fix GCI import formatting (remove extra blank lines between groups)
- Use errors.Is instead of errors.As for mutex sentinel errors

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(errors): use proper variable declarations for errors.As

The errors.As function requires a pointer to an assignable variable,
not a pointer to a composite literal. The previous pattern
`errors.As(err, &SomeError{})` creates a pointer to a temporary value
which errors.As cannot reliably use for assignment.

This fix declares proper variables for each error type and passes
their addresses to errors.As, ensuring correct error chain matching.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(datastore/etcd): use rpctypes.Error() for gRPC error comparison

The etcd gRPC status errors (ErrGRPCUserNotFound, ErrGRPCRoleNotFound)
cannot be compared directly using errors.Is() because they are wrapped
in gRPC status errors during transmission.

The etcd rpctypes package provides:
- ErrGRPC* constants: server-side gRPC status errors
- Err* constants (without GRPC prefix): client-side comparable errors
- Error() function: converts gRPC errors to comparable EtcdError values

The correct pattern is to use rpctypes.Error(err) to normalize the
received error, then compare against client-side error constants
like rpctypes.ErrUserNotFound.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-05 06:11:14 +01:00
Matteo Ruina
b40428825d feat: add ObservedGeneration (#1069)
* feat: add ObservedGeneration to all status types

Add ObservedGeneration field to DataStoreStatus, KubeconfigGeneratorStatus,
and TenantControlPlaneStatus to track which generation the controller has
processed. This enables clients and tools like kstatus to determine if the
controller has reconciled the latest spec changes.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: follow Cluster API pattern for ObservedGeneration

Move ObservedGeneration setting for TenantControlPlane from intermediate
status updates to the final successful reconciliation completion. This
follows Cluster API conventions where ObservedGeneration indicates the
controller has fully processed the given generation.

Previously, ObservedGeneration was set on every status update during
resource processing, which could mislead clients into thinking the spec
was fully reconciled when the controller was still mid-reconciliation
or had hit a transient error.

Now:
- DataStore: Sets ObservedGeneration before single status update (simple controller)
- KubeconfigGenerator: Sets ObservedGeneration before single status update (simple controller)
- TenantControlPlane: Sets ObservedGeneration only after ALL resources processed successfully

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: verify ObservedGeneration equals Generation after reconciliation

Add assertion to e2e test to verify that status.observedGeneration
equals metadata.generation after a TenantControlPlane is successfully
reconciled.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: regenerate CRDs with ObservedGeneration field

Run make crds to regenerate CRDs with the new ObservedGeneration
field in status types.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Run make manifests

* Run make apidoc

* Remove rbac role

* Remove webhook manifest

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 12:18:46 +01:00
73 changed files with 1126 additions and 769 deletions

View File

@@ -83,7 +83,7 @@ $(YQ): $(LOCALBIN)
.PHONY: helm
helm: $(HELM) ## Download helm locally if necessary.
$(HELM): $(LOCALBIN)
test -s $(LOCALBIN)/helm || GOBIN=$(LOCALBIN) CGO_ENABLED=0 go install -ldflags="-s -w" helm.sh/helm/v3/cmd/helm@v3.9.0
test -s $(LOCALBIN)/helm || GOBIN=$(LOCALBIN) CGO_ENABLED=0 go install -ldflags="-s -w" helm.sh/helm/v4/cmd/helm@v4.1.1
.PHONY: ginkgo
ginkgo: $(GINKGO) ## Download ginkgo locally if necessary.
@@ -236,16 +236,29 @@ metallb:
kubectl apply -f "https://raw.githubusercontent.com/metallb/metallb/$$(curl "https://api.github.com/repos/metallb/metallb/releases/latest" | jq -r ".tag_name")/config/manifests/metallb-native.yaml"
kubectl wait pods -n metallb-system -l app=metallb,component=controller --for=condition=Ready --timeout=10m
kubectl wait pods -n metallb-system -l app=metallb,component=speaker --for=condition=Ready --timeout=2m
cat hack/metallb.yaml | sed -E "s|172.19|$$(docker network inspect -f '{{range .IPAM.Config}}{{.Gateway}}{{end}}' kind | sed -E 's|^([0-9]+\.[0-9]+)\..*$$|\1|g')|g" | kubectl apply -f -
@IPV4_PREFIX=$$(docker network inspect kind \
-f '{{range .IPAM.Config}}{{println .Subnet " " .Gateway}}{{end}}' \
| grep -v ':' \
| awk '{print $$2}' \
| sed -E 's|^([0-9]+\.[0-9]+)\..*$$|\1|'); \
sed -E "s|172\.19|$$IPV4_PREFIX|g" hack/metallb.yaml | kubectl apply -f -
cert-manager:
$(HELM) repo add jetstack https://charts.jetstack.io
$(HELM) upgrade --install cert-manager jetstack/cert-manager --namespace certmanager-system --create-namespace --set "installCRDs=true"
gateway-api:
kubectl apply --server-side -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.4.0/standard-install.yaml
# Required for the TLSRoutes. Experimentals.
kubectl apply --server-side -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.4.0/experimental-install.yaml
kubectl apply \
--server-side \
--force-conflicts \
--field-manager=helm \
-f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.4.0/standard-install.yaml
# Required for the TLSRoutes. Experimentals.
kubectl apply \
--server-side \
--force-conflicts \
--field-manager=helm \
-f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.4.0/experimental-install.yaml
kubectl wait --for=condition=Established crd/gateways.gateway.networking.k8s.io --timeout=60s
envoy-gateway: gateway-api helm ## Install Envoy Gateway for Gateway API tests.

View File

@@ -32,6 +32,7 @@ type Endpoints []string
// +kubebuilder:validation:XValidation:rule="(self.driver != \"etcd\" && has(self.basicAuth)) ? ((has(self.basicAuth.username.secretReference) || has(self.basicAuth.username.content))) : true", message="When driver is not etcd and basicAuth exists, username must have secretReference or content"
// +kubebuilder:validation:XValidation:rule="(self.driver != \"etcd\" && has(self.basicAuth)) ? ((has(self.basicAuth.password.secretReference) || has(self.basicAuth.password.content))) : true", message="When driver is not etcd and basicAuth exists, password must have secretReference or content"
// +kubebuilder:validation:XValidation:rule="(self.driver != \"etcd\") ? (has(self.tlsConfig) || has(self.basicAuth)) : true", message="When driver is not etcd, either tlsConfig or basicAuth must be provided"
// +kubebuilder:validation:XValidation:rule="oldSelf == null || self.driver == oldSelf.driver", message="driver is immutable and cannot be changed after creation"
type DataStoreSpec struct {
// The driver to use to connect to the shared datastore.
Driver Driver `json:"driver"`
@@ -88,16 +89,32 @@ type SecretReference struct {
KeyPath secretReferKeyPath `json:"keyPath"`
}
const (
DataStoreTCPFinalizer = "kamaji.clastix.io/TenantControlPlane"
DataStoreConditionValidType = "kamaji.clastix.io/DataStoreValidation"
DataStoreConditionAllowedDeletionType = "kamaji.clastix.io/DataStoreAllowedDeletion"
)
// DataStoreStatus defines the observed state of DataStore.
type DataStoreStatus struct {
// ObservedGeneration represents the .metadata.generation that was last reconciled.
// +optional
ObservedGeneration int64 `json:"observedGeneration,omitempty"`
// List of the Tenant Control Planes, namespaced named, using this data store.
UsedBy []string `json:"usedBy,omitempty"`
// Conditions contains the validation conditions for the given Datastore.
Conditions []metav1.Condition `json:"conditions,omitempty"`
// Ready returns if the DataStore is accepted and ready to get used:
// Kamaji will ensure certificates are available, or correctly referenced.
Ready bool `json:"ready"`
}
//+kubebuilder:object:root=true
//+kubebuilder:subresource:status
//+kubebuilder:resource:scope=Cluster
//+kubebuilder:printcolumn:name="Driver",type="string",JSONPath=".spec.driver",description="Kamaji data store driver"
//+kubebuilder:printcolumn:name="Ready",type="boolean",JSONPath=".status.ready",description="DataStore validated and ready for use"
//+kubebuilder:printcolumn:name="Age",type="date",JSONPath=".metadata.creationTimestamp",description="Age"
//+kubebuilder:metadata:annotations={"cert-manager.io/inject-ca-from=kamaji-system/kamaji-serving-cert"}

View File

@@ -66,6 +66,9 @@ type KubeconfigGeneratorStatusError struct {
// KubeconfigGeneratorStatus defines the observed state of KubeconfigGenerator.
type KubeconfigGeneratorStatus struct {
// ObservedGeneration represents the .metadata.generation that was last reconciled.
// +optional
ObservedGeneration int64 `json:"observedGeneration,omitempty"`
// Resources is the sum of targeted TenantControlPlane objects.
//+kubebuilder:default=0
Resources int `json:"resources"`

View File

@@ -10,7 +10,6 @@ import (
"strconv"
"strings"
"github.com/pkg/errors"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/types"
"sigs.k8s.io/controller-runtime/pkg/client"
@@ -27,12 +26,12 @@ func (in *TenantControlPlane) AssignedControlPlaneAddress() (string, int32, erro
address, portString, err := net.SplitHostPort(in.Status.ControlPlaneEndpoint)
if err != nil {
return "", 0, errors.Wrap(err, "cannot split host port from Tenant Control Plane endpoint")
return "", 0, fmt.Errorf("cannot split host port from Tenant Control Plane endpoint: %w", err)
}
port, err := strconv.Atoi(portString)
if err != nil {
return "", 0, errors.Wrap(err, "cannot convert Tenant Control Plane port from endpoint")
return "", 0, fmt.Errorf("cannot convert Tenant Control Plane port from endpoint: %w", err)
}
return address, int32(port), nil
@@ -47,7 +46,7 @@ func (in *TenantControlPlane) DeclaredControlPlaneAddress(ctx context.Context, c
svc := &corev1.Service{}
err := client.Get(ctx, types.NamespacedName{Namespace: in.GetNamespace(), Name: in.GetName()}, svc)
if err != nil {
return "", errors.Wrap(err, "cannot retrieve Service for the TenantControlPlane")
return "", fmt.Errorf("cannot retrieve Service for the TenantControlPlane: %w", err)
}
switch {

View File

@@ -162,6 +162,9 @@ type AddonsStatus struct {
// TenantControlPlaneStatus defines the observed state of TenantControlPlane.
type TenantControlPlaneStatus struct {
// ObservedGeneration represents the .metadata.generation that was last reconciled.
// +optional
ObservedGeneration int64 `json:"observedGeneration,omitempty"`
// Storage Status contains information about Kubernetes storage system
Storage StorageStatus `json:"storage,omitempty"`
// Certificates contains information about the different certificates

View File

@@ -155,7 +155,6 @@ type IngressSpec struct {
}
// GatewaySpec defines the options for the Gateway which will expose API Server of the Tenant Control Plane.
// +kubebuilder:validation:XValidation:rule="!has(self.parentRefs) || size(self.parentRefs) == 0 || self.parentRefs.all(ref, !has(ref.port) && !has(ref.sectionName))",message="parentRefs must not specify port or sectionName, these are set automatically by Kamaji"
type GatewaySpec struct {
// AdditionalMetadata to add Labels and Annotations support.
AdditionalMetadata AdditionalMetadata `json:"additionalMetadata,omitempty"`

View File

@@ -11,6 +11,10 @@ versions:
jsonPath: .spec.driver
name: Driver
type: string
- description: DataStore validated and ready for use
jsonPath: .status.ready
name: Ready
type: boolean
- description: Age
jsonPath: .metadata.creationTimestamp
name: Age
@@ -272,14 +276,83 @@ versions:
rule: '(self.driver != "etcd" && has(self.basicAuth)) ? ((has(self.basicAuth.password.secretReference) || has(self.basicAuth.password.content))) : true'
- message: When driver is not etcd, either tlsConfig or basicAuth must be provided
rule: '(self.driver != "etcd") ? (has(self.tlsConfig) || has(self.basicAuth)) : true'
- message: driver is immutable and cannot be changed after creation
rule: oldSelf == null || self.driver == oldSelf.driver
status:
description: DataStoreStatus defines the observed state of DataStore.
properties:
conditions:
description: Conditions contains the validation conditions for the given Datastore.
items:
description: Condition contains details for one aspect of the current state of this API Resource.
properties:
lastTransitionTime:
description: |-
lastTransitionTime is the last time the condition transitioned from one status to another.
This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable.
format: date-time
type: string
message:
description: |-
message is a human readable message indicating details about the transition.
This may be an empty string.
maxLength: 32768
type: string
observedGeneration:
description: |-
observedGeneration represents the .metadata.generation that the condition was set based upon.
For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date
with respect to the current state of the instance.
format: int64
minimum: 0
type: integer
reason:
description: |-
reason contains a programmatic identifier indicating the reason for the condition's last transition.
Producers of specific condition types may define expected values and meanings for this field,
and whether the values are considered a guaranteed API.
The value should be a CamelCase string.
This field may not be empty.
maxLength: 1024
minLength: 1
pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$
type: string
status:
description: status of the condition, one of True, False, Unknown.
enum:
- "True"
- "False"
- Unknown
type: string
type:
description: type of condition in CamelCase or in foo.example.com/CamelCase.
maxLength: 316
pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$
type: string
required:
- lastTransitionTime
- message
- reason
- status
- type
type: object
type: array
observedGeneration:
description: ObservedGeneration represents the .metadata.generation that was last reconciled.
format: int64
type: integer
ready:
description: |-
Ready returns if the DataStore is accepted and ready to get used:
Kamaji will ensure certificates are available, or correctly referenced.
type: boolean
usedBy:
description: List of the Tenant Control Planes, namespaced named, using this data store.
items:
type: string
type: array
required:
- ready
type: object
type: object
served: true

View File

@@ -199,6 +199,10 @@ versions:
- resource
type: object
type: array
observedGeneration:
description: ObservedGeneration represents the .metadata.generation that was last reconciled.
format: int64
type: integer
resources:
default: 0
description: Resources is the sum of targeted TenantControlPlane objects.

View File

@@ -6896,9 +6896,6 @@ versions:
type: object
type: array
type: object
x-kubernetes-validations:
- message: parentRefs must not specify port or sectionName, these are set automatically by Kamaji
rule: '!has(self.parentRefs) || size(self.parentRefs) == 0 || self.parentRefs.all(ref, !has(ref.port) && !has(ref.sectionName))'
ingress:
description: Defining the options for an Optional Ingress which will expose API Server of the Tenant Control Plane
properties:
@@ -8788,6 +8785,10 @@ versions:
type: string
type: object
type: object
observedGeneration:
description: ObservedGeneration represents the .metadata.generation that was last reconciled.
format: int64
type: integer
storage:
description: Storage Status contains information about Kubernetes storage system
properties:

View File

@@ -19,46 +19,6 @@
resources:
- tenantcontrolplanes
sideEffects: None
- admissionReviewVersions:
- v1
clientConfig:
service:
name: '{{ include "kamaji.webhookServiceName" . }}'
namespace: '{{ .Release.Namespace }}'
path: /validate-kamaji-clastix-io-v1alpha1-datastore
failurePolicy: Fail
name: vdatastore.kb.io
rules:
- apiGroups:
- kamaji.clastix.io
apiVersions:
- v1alpha1
operations:
- CREATE
- UPDATE
- DELETE
resources:
- datastores
sideEffects: None
- admissionReviewVersions:
- v1
clientConfig:
service:
name: '{{ include "kamaji.webhookServiceName" . }}'
namespace: '{{ .Release.Namespace }}'
path: /validate--v1-secret
failurePolicy: Ignore
name: vdatastoresecrets.kb.io
rules:
- apiGroups:
- ""
apiVersions:
- v1
operations:
- DELETE
resources:
- secrets
sideEffects: None
- admissionReviewVersions:
- v1
clientConfig:

View File

@@ -20,6 +20,10 @@ spec:
jsonPath: .spec.driver
name: Driver
type: string
- description: DataStore validated and ready for use
jsonPath: .status.ready
name: Ready
type: boolean
- description: Age
jsonPath: .metadata.creationTimestamp
name: Age
@@ -281,14 +285,83 @@ spec:
rule: '(self.driver != "etcd" && has(self.basicAuth)) ? ((has(self.basicAuth.password.secretReference) || has(self.basicAuth.password.content))) : true'
- message: When driver is not etcd, either tlsConfig or basicAuth must be provided
rule: '(self.driver != "etcd") ? (has(self.tlsConfig) || has(self.basicAuth)) : true'
- message: driver is immutable and cannot be changed after creation
rule: oldSelf == null || self.driver == oldSelf.driver
status:
description: DataStoreStatus defines the observed state of DataStore.
properties:
conditions:
description: Conditions contains the validation conditions for the given Datastore.
items:
description: Condition contains details for one aspect of the current state of this API Resource.
properties:
lastTransitionTime:
description: |-
lastTransitionTime is the last time the condition transitioned from one status to another.
This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable.
format: date-time
type: string
message:
description: |-
message is a human readable message indicating details about the transition.
This may be an empty string.
maxLength: 32768
type: string
observedGeneration:
description: |-
observedGeneration represents the .metadata.generation that the condition was set based upon.
For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date
with respect to the current state of the instance.
format: int64
minimum: 0
type: integer
reason:
description: |-
reason contains a programmatic identifier indicating the reason for the condition's last transition.
Producers of specific condition types may define expected values and meanings for this field,
and whether the values are considered a guaranteed API.
The value should be a CamelCase string.
This field may not be empty.
maxLength: 1024
minLength: 1
pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$
type: string
status:
description: status of the condition, one of True, False, Unknown.
enum:
- "True"
- "False"
- Unknown
type: string
type:
description: type of condition in CamelCase or in foo.example.com/CamelCase.
maxLength: 316
pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$
type: string
required:
- lastTransitionTime
- message
- reason
- status
- type
type: object
type: array
observedGeneration:
description: ObservedGeneration represents the .metadata.generation that was last reconciled.
format: int64
type: integer
ready:
description: |-
Ready returns if the DataStore is accepted and ready to get used:
Kamaji will ensure certificates are available, or correctly referenced.
type: boolean
usedBy:
description: List of the Tenant Control Planes, namespaced named, using this data store.
items:
type: string
type: array
required:
- ready
type: object
type: object
served: true

View File

@@ -207,6 +207,10 @@ spec:
- resource
type: object
type: array
observedGeneration:
description: ObservedGeneration represents the .metadata.generation that was last reconciled.
format: int64
type: integer
resources:
default: 0
description: Resources is the sum of targeted TenantControlPlane objects.

View File

@@ -6904,9 +6904,6 @@ spec:
type: object
type: array
type: object
x-kubernetes-validations:
- message: parentRefs must not specify port or sectionName, these are set automatically by Kamaji
rule: '!has(self.parentRefs) || size(self.parentRefs) == 0 || self.parentRefs.all(ref, !has(ref.port) && !has(ref.sectionName))'
ingress:
description: Defining the options for an Optional Ingress which will expose API Server of the Tenant Control Plane
properties:
@@ -8796,6 +8793,10 @@ spec:
type: string
type: object
type: object
observedGeneration:
description: ObservedGeneration represents the .metadata.generation that was last reconciled.
format: int64
type: integer
storage:
description: Storage Status contains information about Kubernetes storage system
properties:

View File

@@ -4,7 +4,6 @@
package manager
import (
"context"
"flag"
"fmt"
"io"
@@ -34,7 +33,6 @@ import (
"github.com/clastix/kamaji/controllers/soot"
"github.com/clastix/kamaji/internal"
"github.com/clastix/kamaji/internal/builders/controlplane"
datastoreutils "github.com/clastix/kamaji/internal/datastore/utils"
"github.com/clastix/kamaji/internal/utilities"
"github.com/clastix/kamaji/internal/webhook"
"github.com/clastix/kamaji/internal/webhook/handlers"
@@ -87,10 +85,6 @@ func NewCmd(scheme *runtime.Scheme) *cobra.Command {
return fmt.Errorf("unable to read webhook CA: %w", err)
}
if err = datastoreutils.CheckExists(context.Background(), scheme, datastore); err != nil {
return err
}
if controllerReconcileTimeout.Seconds() == 0 {
return fmt.Errorf("the controller reconcile timeout must be greater than zero")
}
@@ -276,12 +270,6 @@ func NewCmd(scheme *runtime.Scheme) *cobra.Command {
KubernetesVersion: k8sVersion,
},
},
routes.DataStoreValidate{}: {
handlers.DataStoreValidation{Client: mgr.GetClient()},
},
routes.DataStoreSecrets{}: {
handlers.DataStoreSecretValidation{Client: mgr.GetClient()},
},
})
if err != nil {
setupLog.Error(err, "unable to create webhook")

View File

@@ -4,7 +4,8 @@
package utils
import (
"github.com/pkg/errors"
"fmt"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/rest"
)
@@ -12,12 +13,12 @@ import (
func KubernetesVersion(config *rest.Config) (string, error) {
cs, csErr := kubernetes.NewForConfig(config)
if csErr != nil {
return "", errors.Wrap(csErr, "cannot create kubernetes clientset")
return "", fmt.Errorf("cannot create kubernetes clientset: %w", csErr)
}
sv, svErr := cs.ServerVersion()
if svErr != nil {
return "", errors.Wrap(svErr, "cannot get Kubernetes version")
return "", fmt.Errorf("cannot get Kubernetes version: %w", svErr)
}
return sv.GitVersion, nil

View File

@@ -9,7 +9,6 @@ import (
"fmt"
"time"
"github.com/pkg/errors"
corev1 "k8s.io/api/core/v1"
k8serrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
@@ -168,7 +167,7 @@ func (s *CertificateLifecycle) extractCertificateFromKubeconfig(secret corev1.Se
crt, err := crypto.ParseCertificateBytes(kc.AuthInfos[0].AuthInfo.ClientCertificateData)
if err != nil {
return nil, errors.Wrap(err, "cannot parse kubeconfig certificate bytes")
return nil, fmt.Errorf("cannot parse kubeconfig certificate bytes: %w", err)
}
return crt, nil

View File

@@ -5,21 +5,22 @@ package controllers
import (
"context"
"fmt"
"github.com/pkg/errors"
corev1 "k8s.io/api/core/v1"
k8serrors "k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/api/meta"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/fields"
k8stypes "k8s.io/apimachinery/pkg/types"
"k8s.io/apimachinery/pkg/util/sets"
"k8s.io/client-go/util/retry"
"k8s.io/client-go/util/workqueue"
controllerruntime "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/builder"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
"sigs.k8s.io/controller-runtime/pkg/event"
"sigs.k8s.io/controller-runtime/pkg/handler"
"sigs.k8s.io/controller-runtime/pkg/log"
"sigs.k8s.io/controller-runtime/pkg/predicate"
"sigs.k8s.io/controller-runtime/pkg/reconcile"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
@@ -38,19 +39,21 @@ type DataStore struct {
//+kubebuilder:rbac:groups=kamaji.clastix.io,resources=datastores/status,verbs=get;update;patch
func (r *DataStore) Reconcile(ctx context.Context, request reconcile.Request) (reconcile.Result, error) {
var err error
logger := log.FromContext(ctx)
var ds kamajiv1alpha1.DataStore
if err := r.Client.Get(ctx, request.NamespacedName, &ds); err != nil {
if k8serrors.IsNotFound(err) {
if dsErr := r.Client.Get(ctx, request.NamespacedName, &ds); dsErr != nil {
if k8serrors.IsNotFound(dsErr) {
logger.Info("resource may have been deleted, skipping")
return reconcile.Result{}, nil
}
logger.Error(err, "cannot retrieve the required resource")
logger.Error(dsErr, "cannot retrieve the required resource")
return reconcile.Result{}, err
return reconcile.Result{}, dsErr
}
if utils.IsPaused(&ds) {
@@ -59,44 +62,179 @@ func (r *DataStore) Reconcile(ctx context.Context, request reconcile.Request) (r
return reconcile.Result{}, nil
}
if ds.GetDeletionTimestamp() == nil && !controllerutil.ContainsFinalizer(&ds, kamajiv1alpha1.DataStoreTCPFinalizer) {
logger.Info("missing finalizer, adding it")
ds.SetFinalizers(append(ds.GetFinalizers(), kamajiv1alpha1.DataStoreTCPFinalizer))
if uErr := r.Client.Update(ctx, &ds); uErr != nil {
return reconcile.Result{}, uErr
}
return reconcile.Result{}, nil
}
// Extracting the list of TenantControlPlane objects referenced by the given DataStore:
// this data is used to reference these in the Status, as well as propagating changes
// that would be required, such as changing TLS Configuration, or Basic Auth.
var tcpList kamajiv1alpha1.TenantControlPlaneList
updateErr := retry.RetryOnConflict(retry.DefaultRetry, func() error {
if lErr := r.Client.List(ctx, &tcpList, client.MatchingFieldsSelector{
Selector: fields.OneTermEqualSelector(kamajiv1alpha1.TenantControlPlaneUsedDataStoreKey, ds.GetName()),
}); lErr != nil {
return errors.Wrap(lErr, "cannot retrieve list of the Tenant Control Plane using the following instance")
}
// Updating the status with the list of Tenant Control Plane using the following Data Source
tcpSets := sets.NewString()
for _, tcp := range tcpList.Items {
tcpSets.Insert(getNamespacedName(tcp.GetNamespace(), tcp.GetName()).String())
if lErr := r.Client.List(ctx, &tcpList, client.MatchingFieldsSelector{
Selector: fields.OneTermEqualSelector(kamajiv1alpha1.TenantControlPlaneUsedDataStoreKey, ds.GetName()),
}); lErr != nil {
return reconcile.Result{}, fmt.Errorf("cannot retrieve list of the Tenant Control Plane using the following instance: %w", lErr)
}
tcpSets := sets.NewString()
for _, tcp := range tcpList.Items {
tcpSets.Insert(getNamespacedName(tcp.GetNamespace(), tcp.GetName()).String())
}
ds.Status.UsedBy = tcpSets.List()
// Performing the status update only at the end of the reconciliation loop:
// this is performed in defer to avoid duplication of code,
// and triggering a reconciliation of depending on TenantControlPlanes only if the update was successful.
defer func() {
if meta.IsStatusConditionTrue(ds.Status.Conditions, kamajiv1alpha1.DataStoreConditionAllowedDeletionType) {
logger.Info("removing finalizer upon true condition")
controllerutil.RemoveFinalizer(&ds, kamajiv1alpha1.DataStoreTCPFinalizer)
if uErr := r.Client.Update(ctx, &ds); uErr != nil {
logger.Error(uErr, "cannot update object")
return
}
logger.Info("finalizer removed successfully")
return
}
ds.Status.UsedBy = tcpSets.List()
ds.Status.ObservedGeneration = ds.Generation
ds.Status.Ready = meta.IsStatusConditionTrue(ds.Status.Conditions, kamajiv1alpha1.DataStoreConditionValidType)
if err = r.Client.Status().Update(ctx, &ds); err != nil {
logger.Error(err, "cannot update the status for the given instance")
return
}
if !ds.Status.Ready {
logger.Info("skipping triggering, DataStore is not ready")
return
}
logger.Info("triggering cascading reconciliation for TenantControlPlanes")
for _, tcp := range tcpList.Items {
var shrunkTCP kamajiv1alpha1.TenantControlPlane
shrunkTCP.Name = tcp.Name
shrunkTCP.Namespace = tcp.Namespace
go utils.TriggerChannel(ctx, r.TenantControlPlaneTrigger, shrunkTCP)
}
}()
if ds.GetDeletionTimestamp() != nil {
if len(tcpList.Items) > 0 {
logger.Info("deletion is blocked due to DataStore still being referenced")
meta.SetStatusCondition(&ds.Status.Conditions, metav1.Condition{
Type: kamajiv1alpha1.DataStoreConditionAllowedDeletionType,
Status: metav1.ConditionFalse,
ObservedGeneration: ds.Generation,
Reason: "DataStoreStillUsed",
Message: "The DataStore is still used and referenced by one (or more) TenantControlPlane objects.",
})
return reconcile.Result{}, nil
}
if meta.IsStatusConditionFalse(ds.Status.Conditions, kamajiv1alpha1.DataStoreConditionAllowedDeletionType) {
logger.Info("DataStore is not used by any TenantControlPlane object")
meta.SetStatusCondition(&ds.Status.Conditions, metav1.Condition{
Type: kamajiv1alpha1.DataStoreConditionAllowedDeletionType,
Status: metav1.ConditionTrue,
ObservedGeneration: ds.Generation,
Reason: "DataStoreUnused",
Message: "",
})
return reconcile.Result{}, nil
}
logger.Info("DataStore can be safely deleted")
return reconcile.Result{}, nil
}
if exists := meta.FindStatusCondition(ds.Status.Conditions, kamajiv1alpha1.DataStoreConditionValidType); exists == nil {
logger.Info("missing starting condition")
meta.SetStatusCondition(&ds.Status.Conditions, metav1.Condition{
Type: kamajiv1alpha1.DataStoreConditionValidType,
Status: metav1.ConditionUnknown,
ObservedGeneration: ds.Generation,
Reason: "MissingCondition",
Message: "Controller will process the validation.",
})
if sErr := r.Client.Status().Update(ctx, &ds); sErr != nil {
return errors.Wrap(sErr, "cannot update the status for the given instance")
return reconcile.Result{}, fmt.Errorf("cannot update the status for the given instance: %w", sErr)
}
return nil
return reconcile.Result{}, nil
}
if ds.Spec.BasicAuth != nil {
logger.Info("validating basic authentication")
if vErr := r.validateBasicAuth(ctx, ds); vErr != nil {
meta.SetStatusCondition(&ds.Status.Conditions, metav1.Condition{
Type: kamajiv1alpha1.DataStoreConditionValidType,
Status: metav1.ConditionFalse,
ObservedGeneration: ds.Generation,
Reason: "BasicAuthValidationFailed",
Message: vErr.Error(),
})
logger.Info("invalid basic authentication")
return reconcile.Result{}, nil
}
logger.Info("basic authentication is valid")
}
logger.Info("validating TLS configuration")
if vErr := r.validateTLSConfig(ctx, ds); vErr != nil {
meta.SetStatusCondition(&ds.Status.Conditions, metav1.Condition{
Type: kamajiv1alpha1.DataStoreConditionValidType,
Status: metav1.ConditionFalse,
ObservedGeneration: ds.Generation,
Reason: "TLSConfigurationValidationFailed",
Message: vErr.Error(),
})
logger.Info("invalid TLS configuration")
return reconcile.Result{}, nil
}
logger.Info("TLS configuration is valid")
meta.SetStatusCondition(&ds.Status.Conditions, metav1.Condition{
Type: kamajiv1alpha1.DataStoreConditionValidType,
Status: metav1.ConditionTrue,
ObservedGeneration: ds.Status.ObservedGeneration,
Reason: "DataStoreIsValid",
Message: "",
})
if updateErr != nil {
logger.Error(updateErr, "cannot update DataStore status")
return reconcile.Result{}, updateErr
}
// Triggering the reconciliation of the Tenant Control Plane upon a Secret change
for _, tcp := range tcpList.Items {
var shrunkTCP kamajiv1alpha1.TenantControlPlane
shrunkTCP.Name = tcp.Name
shrunkTCP.Namespace = tcp.Namespace
go utils.TriggerChannel(ctx, r.TenantControlPlaneTrigger, shrunkTCP)
}
return reconcile.Result{}, nil
return reconcile.Result{}, err
}
func (r *DataStore) SetupWithManager(mgr controllerruntime.Manager) error {
@@ -111,9 +249,7 @@ func (r *DataStore) SetupWithManager(mgr controllerruntime.Manager) error {
}
//nolint:forcetypeassert
return controllerruntime.NewControllerManagedBy(mgr).
For(&kamajiv1alpha1.DataStore{}, builder.WithPredicates(
predicate.GenerationChangedPredicate{},
)).
For(&kamajiv1alpha1.DataStore{}).
Watches(&kamajiv1alpha1.TenantControlPlane{}, handler.Funcs{
CreateFunc: func(_ context.Context, createEvent event.TypedCreateEvent[client.Object], w workqueue.TypedRateLimitingInterface[reconcile.Request]) {
enqueueFn(createEvent.Object.(*kamajiv1alpha1.TenantControlPlane), w)
@@ -128,3 +264,76 @@ func (r *DataStore) SetupWithManager(mgr controllerruntime.Manager) error {
}).
Complete(r)
}
func (r *DataStore) validateBasicAuth(ctx context.Context, ds kamajiv1alpha1.DataStore) error {
if err := r.validateContentReference(ctx, ds.Spec.BasicAuth.Password); err != nil {
return fmt.Errorf("basic-auth password is not valid, %w", err)
}
if err := r.validateContentReference(ctx, ds.Spec.BasicAuth.Username); err != nil {
return fmt.Errorf("basic-auth username is not valid, %w", err)
}
return nil
}
func (r *DataStore) validateTLSConfig(ctx context.Context, ds kamajiv1alpha1.DataStore) error {
if ds.Spec.TLSConfig == nil && ds.Spec.Driver != kamajiv1alpha1.EtcdDriver {
return nil
}
if err := r.validateContentReference(ctx, ds.Spec.TLSConfig.CertificateAuthority.Certificate); err != nil {
return fmt.Errorf("CA certificate is not valid, %w", err)
}
if ds.Spec.Driver == kamajiv1alpha1.EtcdDriver {
if ds.Spec.TLSConfig.CertificateAuthority.PrivateKey == nil {
return fmt.Errorf("CA private key is required when using the etcd driver")
}
if ds.Spec.TLSConfig.ClientCertificate == nil {
return fmt.Errorf("client certificate is required when using the etcd driver")
}
}
if ds.Spec.TLSConfig.CertificateAuthority.PrivateKey != nil {
if err := r.validateContentReference(ctx, *ds.Spec.TLSConfig.CertificateAuthority.PrivateKey); err != nil {
return fmt.Errorf("CA private key is not valid, %w", err)
}
}
if ds.Spec.TLSConfig.ClientCertificate != nil {
if err := r.validateContentReference(ctx, ds.Spec.TLSConfig.ClientCertificate.Certificate); err != nil {
return fmt.Errorf("client certificate is not valid, %w", err)
}
if err := r.validateContentReference(ctx, ds.Spec.TLSConfig.ClientCertificate.PrivateKey); err != nil {
return fmt.Errorf("client private key is not valid, %w", err)
}
}
return nil
}
func (r *DataStore) validateContentReference(ctx context.Context, ref kamajiv1alpha1.ContentRef) error {
switch {
case len(ref.Content) > 0:
return nil
case ref.SecretRef == nil:
return fmt.Errorf("the Secret reference is mandatory when bare content is not specified")
case len(ref.SecretRef.SecretReference.Name) == 0:
return fmt.Errorf("the Secret reference name is mandatory")
case len(ref.SecretRef.SecretReference.Namespace) == 0:
return fmt.Errorf("the Secret reference namespace is mandatory")
}
if err := r.Client.Get(ctx, k8stypes.NamespacedName{Name: ref.SecretRef.SecretReference.Name, Namespace: ref.SecretRef.SecretReference.Namespace}, &corev1.Secret{}); err != nil {
if k8serrors.IsNotFound(err) {
return fmt.Errorf("secret %s/%s is not found", ref.SecretRef.SecretReference.Namespace, ref.SecretRef.SecretReference.Name)
}
return err
}
return nil
}

View File

@@ -12,7 +12,6 @@ import (
"strings"
"time"
"github.com/pkg/errors"
corev1 "k8s.io/api/core/v1"
apierrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
@@ -88,6 +87,7 @@ func (r *KubeconfigGeneratorReconciler) Reconcile(ctx context.Context, req ctrl.
}
generator.Status = status
generator.Status.ObservedGeneration = generator.Generation
if statusErr := r.Client.Status().Update(ctx, &generator); statusErr != nil {
logger.Error(statusErr, "cannot update resource status")
@@ -103,12 +103,12 @@ func (r *KubeconfigGeneratorReconciler) Reconcile(ctx context.Context, req ctrl.
func (r *KubeconfigGeneratorReconciler) handle(ctx context.Context, generator *kamajiv1alpha1.KubeconfigGenerator) (kamajiv1alpha1.KubeconfigGeneratorStatus, error) {
nsSelector, nsErr := metav1.LabelSelectorAsSelector(&generator.Spec.NamespaceSelector)
if nsErr != nil {
return kamajiv1alpha1.KubeconfigGeneratorStatus{}, errors.Wrap(nsErr, "NamespaceSelector contains an error")
return kamajiv1alpha1.KubeconfigGeneratorStatus{}, fmt.Errorf("NamespaceSelector contains an error: %w", nsErr)
}
var namespaceList corev1.NamespaceList
if err := r.Client.List(ctx, &namespaceList, &client.ListOptions{LabelSelector: nsSelector}); err != nil {
return kamajiv1alpha1.KubeconfigGeneratorStatus{}, errors.Wrap(err, "cannot filter Namespace objects using provided selector")
return kamajiv1alpha1.KubeconfigGeneratorStatus{}, fmt.Errorf("cannot filter Namespace objects using provided selector: %w", err)
}
var targets []kamajiv1alpha1.TenantControlPlane
@@ -116,12 +116,12 @@ func (r *KubeconfigGeneratorReconciler) handle(ctx context.Context, generator *k
for _, ns := range namespaceList.Items {
tcpSelector, tcpErr := metav1.LabelSelectorAsSelector(&generator.Spec.TenantControlPlaneSelector)
if tcpErr != nil {
return kamajiv1alpha1.KubeconfigGeneratorStatus{}, errors.Wrap(tcpErr, "TenantControlPlaneSelector contains an error")
return kamajiv1alpha1.KubeconfigGeneratorStatus{}, fmt.Errorf("TenantControlPlaneSelector contains an error: %w", tcpErr)
}
var tcpList kamajiv1alpha1.TenantControlPlaneList
if err := r.Client.List(ctx, &tcpList, &client.ListOptions{Namespace: ns.GetName(), LabelSelector: tcpSelector}); err != nil {
return kamajiv1alpha1.KubeconfigGeneratorStatus{}, errors.Wrap(err, "cannot filter TenantControlPlane objects using provided selector")
return kamajiv1alpha1.KubeconfigGeneratorStatus{}, fmt.Errorf("cannot filter TenantControlPlane objects using provided selector: %w", err)
}
targets = append(targets, tcpList.Items...)
@@ -290,17 +290,17 @@ func (r *KubeconfigGeneratorReconciler) generate(ctx context.Context, generator
var caSecret corev1.Secret
if caErr := r.Client.Get(ctx, types.NamespacedName{Namespace: tcp.Namespace, Name: tcp.Status.Certificates.CA.SecretName}, &caSecret); caErr != nil {
return errors.Wrap(caErr, "cannot retrieve Certificate Authority")
return fmt.Errorf("cannot retrieve Certificate Authority: %w", caErr)
}
caCert, crtErr := crypto.ParseCertificateBytes(caSecret.Data[kubeadmconstants.CACertName])
if crtErr != nil {
return errors.Wrap(crtErr, "cannot parse Certificate Authority certificate")
return fmt.Errorf("cannot parse Certificate Authority certificate: %w", crtErr)
}
caKey, keyErr := crypto.ParsePrivateKeyBytes(caSecret.Data[kubeadmconstants.CAKeyName])
if keyErr != nil {
return errors.Wrap(keyErr, "cannot parse Certificate Authority key")
return fmt.Errorf("cannot parse Certificate Authority key: %w", keyErr)
}
clientCert, clientKey, err := pkiutil.NewCertAndKey(caCert, caKey, &clientCertConfig)
@@ -312,7 +312,7 @@ func (r *KubeconfigGeneratorReconciler) generate(ctx context.Context, generator
tmpl.AuthInfos[name].AuthInfo.ClientCertificateData = pkiutil.EncodeCertPEM(clientCert)
tmpl.AuthInfos[name].AuthInfo.ClientKeyData, err = keyutil.MarshalPrivateKeyToPEM(clientKey)
if err != nil {
return errors.Wrap(err, "cannot marshal private key to PEM")
return fmt.Errorf("cannot marshal private key to PEM: %w", err)
}
}
@@ -341,7 +341,7 @@ func (r *KubeconfigGeneratorReconciler) generate(ctx context.Context, generator
secret.Data["value"], err = utilities.EncodeToYaml(tmpl)
if err != nil {
return errors.Wrap(err, "cannot encode generated Kubeconfig to YAML")
return fmt.Errorf("cannot encode generated Kubeconfig to YAML: %w", err)
}
if utilities.IsRotationRequested(secret) {
@@ -355,7 +355,7 @@ func (r *KubeconfigGeneratorReconciler) generate(ctx context.Context, generator
return ctrl.SetControllerReference(tcp, secret, r.Client.Scheme())
})
if err != nil {
return errors.Wrap(err, "cannot create or update generated Kubeconfig")
return fmt.Errorf("cannot create or update generated Kubeconfig: %w", err)
}
return nil

View File

@@ -5,9 +5,9 @@ package controllers
import (
"context"
"errors"
"github.com/go-logr/logr"
"github.com/pkg/errors"
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
rbacv1 "k8s.io/api/rbac/v1"

View File

@@ -3,8 +3,6 @@
package errors
import (
"github.com/pkg/errors"
)
import "errors"
var ErrPausedReconciliation = errors.New("paused reconciliation, no further actions")

View File

@@ -5,9 +5,9 @@ package controllers
import (
"context"
"errors"
"github.com/go-logr/logr"
"github.com/pkg/errors"
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
v1 "k8s.io/api/rbac/v1"

View File

@@ -5,9 +5,9 @@ package controllers
import (
"context"
"errors"
"github.com/go-logr/logr"
"github.com/pkg/errors"
"k8s.io/utils/ptr"
controllerruntime "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/builder"

View File

@@ -5,9 +5,9 @@ package controllers
import (
"context"
"errors"
"github.com/go-logr/logr"
"github.com/pkg/errors"
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
rbacv1 "k8s.io/api/rbac/v1"

View File

@@ -5,11 +5,11 @@ package controllers
import (
"context"
"errors"
"fmt"
"time"
"github.com/go-logr/logr"
"github.com/pkg/errors"
admissionregistrationv1 "k8s.io/api/admissionregistration/v1"
apierrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"

View File

@@ -5,11 +5,11 @@ package controllers
import (
"context"
"fmt"
"time"
"github.com/clastix/kamaji-telemetry/api"
telemetry "github.com/clastix/kamaji-telemetry/pkg/client"
"github.com/pkg/errors"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/types"
"k8s.io/utils/ptr"
@@ -31,7 +31,7 @@ type TelemetryController struct {
func (m *TelemetryController) retrieveControllerUID(ctx context.Context) (string, error) {
var defaultSvc corev1.Service
if err := m.Client.Get(ctx, types.NamespacedName{Name: "kubernetes", Namespace: "default"}, &defaultSvc); err != nil {
return "", errors.Wrap(err, "cannot start the telemetry controller")
return "", fmt.Errorf("cannot start the telemetry controller: %w", err)
}
return string(defaultSvc.UID), nil

View File

@@ -5,12 +5,12 @@ package controllers
import (
"context"
"errors"
"fmt"
"strings"
"time"
"github.com/juju/mutex/v2"
"github.com/pkg/errors"
appsv1 "k8s.io/api/apps/v1"
batchv1 "k8s.io/api/batch/v1"
corev1 "k8s.io/api/core/v1"
@@ -18,6 +18,7 @@ import (
k8serrors "k8s.io/apimachinery/pkg/api/errors"
k8stypes "k8s.io/apimachinery/pkg/types"
"k8s.io/client-go/discovery"
"k8s.io/client-go/util/retry"
"k8s.io/client-go/util/workqueue"
"k8s.io/utils/clock"
ctrl "sigs.k8s.io/controller-runtime"
@@ -87,6 +88,7 @@ type TenantControlPlaneReconcilerConfig struct {
//+kubebuilder:rbac:groups=gateway.networking.k8s.io,resources=tlsroutes,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=gateway.networking.k8s.io,resources=gateways,verbs=get;list;watch
//nolint:maintidx
func (r *TenantControlPlaneReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
log := log.FromContext(ctx)
@@ -115,11 +117,11 @@ func (r *TenantControlPlaneReconciler) Reconcile(ctx context.Context, req ctrl.R
releaser, err := mutex.Acquire(r.mutexSpec(tenantControlPlane))
if err != nil {
switch {
case errors.As(err, &mutex.ErrTimeout):
case errors.Is(err, mutex.ErrTimeout):
log.Info("acquire timed out, current process is blocked by another reconciliation")
return ctrl.Result{RequeueAfter: time.Second}, nil
case errors.As(err, &mutex.ErrCancelled):
case errors.Is(err, mutex.ErrCancelled):
log.Info("acquire cancelled")
return ctrl.Result{RequeueAfter: time.Second}, nil
@@ -136,11 +138,13 @@ func (r *TenantControlPlaneReconciler) Reconcile(ctx context.Context, req ctrl.R
if markedToBeDeleted && !controllerutil.ContainsFinalizer(tenantControlPlane, finalizers.DatastoreFinalizer) {
return ctrl.Result{}, nil
}
// Retrieving the DataStore to use for the current reconciliation
ds, err := r.dataStore(ctx, tenantControlPlane)
if err != nil {
if errors.Is(err, ErrMissingDataStore) {
log.Info(err.Error())
// DataStore preflight checks:
// 1. DataStore must exist
// 2. it must be ready
ds, dsErr := r.dataStore(ctx, tenantControlPlane)
if dsErr != nil {
if errors.Is(dsErr, ErrMissingDataStore) {
log.Info(dsErr.Error())
return ctrl.Result{RequeueAfter: time.Second}, nil
}
@@ -150,6 +154,12 @@ func (r *TenantControlPlaneReconciler) Reconcile(ctx context.Context, req ctrl.R
return ctrl.Result{}, err
}
if !ds.Status.Ready {
log.Info("cannot reconcile since DataStore is not ready")
return ctrl.Result{RequeueAfter: time.Second}, nil
}
dsConnection, err := datastore.NewStorageConnection(ctx, r.Client, *ds)
if err != nil {
log.Error(err, "cannot generate the DataStore connection for the given instance")
@@ -260,6 +270,23 @@ func (r *TenantControlPlaneReconciler) Reconcile(ctx context.Context, req ctrl.R
log.Info(fmt.Sprintf("%s has been reconciled", tenantControlPlane.GetName()))
// Set ObservedGeneration only on successful reconciliation completion.
// This follows Cluster API conventions where ObservedGeneration indicates
// the controller has fully processed the given generation.
if err := retry.RetryOnConflict(retry.DefaultRetry, func() error {
if getErr := r.Client.Get(ctx, req.NamespacedName, tenantControlPlane); getErr != nil {
return getErr
}
tenantControlPlane.Status.ObservedGeneration = tenantControlPlane.Generation
return r.Client.Status().Update(ctx, tenantControlPlane)
}); err != nil {
log.Error(err, "failed to update ObservedGeneration")
return ctrl.Result{}, err
}
return ctrl.Result{}, nil
}
@@ -379,7 +406,7 @@ func (r *TenantControlPlaneReconciler) dataStore(ctx context.Context, tenantCont
var ds kamajiv1alpha1.DataStore
if err := r.Client.Get(ctx, k8stypes.NamespacedName{Name: tenantControlPlane.Spec.DataStore}, &ds); err != nil {
return nil, errors.Wrap(err, "cannot retrieve *kamajiv1alpha.DataStore object")
return nil, fmt.Errorf("cannot retrieve *kamajiv1alpha.DataStore object: %w", err)
}
return &ds, nil
@@ -391,7 +418,7 @@ func (r *TenantControlPlaneReconciler) dataStoreOverride(ctx context.Context, te
for _, dso := range tenantControlPlane.Spec.DataStoreOverrides {
var ds kamajiv1alpha1.DataStore
if err := r.Client.Get(ctx, k8stypes.NamespacedName{Name: dso.DataStore}, &ds); err != nil {
return nil, errors.Wrap(err, "cannot retrieve *kamajiv1alpha.DataStore object")
return nil, fmt.Errorf("cannot retrieve *kamajiv1alpha.DataStore object: %w", err)
}
if ds.Spec.Driver != kamajiv1alpha1.EtcdDriver {
return nil, errors.New("DataStoreOverrides can only use ETCD driver")

View File

@@ -5,7 +5,7 @@ NAMESPACE:=nats-system
.PHONY: helm
HELM = $(shell pwd)/../../../bin/helm
helm: ## Download helm locally if necessary.
$(call go-install-tool,$(HELM),helm.sh/helm/v3/cmd/helm@v3.9.0)
$(call go-install-tool,$(HELM),helm.sh/helm/v3/cmd/helm@v4.1.1)
nats: nats-certificates nats-secret nats-deployment

View File

@@ -1,106 +1,34 @@
# Gateway API Support
# Gateway API
Kamaji provides built-in support for the [Gateway API](https://gateway-api.sigs.k8s.io/), allowing you to expose Tenant Control Planes using TLSRoute resources with SNI-based routing. This enables hostname-based routing to multiple Tenant Control Planes through a single Gateway resource, reducing the need for dedicated LoadBalancer services.
Kamaji provides built-in support for the [Gateway API](https://gateway-api.sigs.k8s.io/), allowing Tenant Control Planes to be exposed as SNI-based addresses/urls. This eliminates the need for a dedicated LoadBalancer IP per TCP. A single Gateway resource can be used for multiple Tenant Control Planes and provide access to them with hostname-based routing (like `https://mycluster.xyz.com:6443`).
## Overview
You can configure Gateway in Tenant Control Plane via `tcp.spec.controlPlane.gateway`, Kamaji will automatically create a `TLSRoute` resource with corresponding spec. To make this configuration work, you need to ensure `gateway` exists (is created by you) and `tcp.spec.controlPlane.gateway` points to your gateway.
Gateway API support in Kamaji automatically creates and manages TLSRoute resources for your Tenant Control Planes. When you configure a Gateway for a Tenant Control Plane, Kamaji automatically creates TLSRoutes for the Control Plane API Server. If konnectivity is enabled, a separate TLSRoute is created for it. Both TLSRoutes use the same hostname and Gateway resource, but route to different ports(listeners) using port-based routing and semantic `sectionName` values.
We will cover a few examples below on how this is done.
Therefore, the target `Gateway` resource must have right listener configurations (see the Gateway [example section](#gateway-resource-setup) below).
## How It Works
When you configure `spec.controlPlane.gateway` in a TenantControlPlane resource, Kamaji automatically:
1. **Creates a TLSRoute for the control plane** that routes for port 6443 (or `spec.networkProfile.port`) with sectionName `"kube-apiserver"`
2. **Creates a TLSRoute for Konnectivity** (if konnectivity addon is enabled) that routes for port 8132 (or `spec.addons.konnectivity.server.port`) with sectionName `"konnectivity-server"`
Both TLSRoutes:
- Use the same hostname from `spec.controlPlane.gateway.hostname`
- Reference the same parent Gateway resource via `parentRefs`
- The `port` and `sectionName` fields are set automatically by Kamaji
- Route to the appropriate Tenant Control Plane service
The Gateway resource must have listeners configured for both ports (6443 and 8132) to support both routes.
## Prerequisites
Before using Gateway API support, ensure:
Before using Gateway API mode, please ensure:
1. **Gateway API CRDs are installed** in your cluster (Required CRDs: `GatewayClass`, `Gateway`, `TLSRoute`)
2. **A Gateway resource exists** with appropriate listeners configured:
- At minimum, listeners for ports 6443 (control plane) and 8132 (Konnectivity)
- TLS protocol with Passthrough mode
- Hostname pattern matching your Tenant Control Plane hostnames
2. **A Gateway resource exists** with appropriate configuration (see examples in this guide):
- Listeners for kube-apiserver.
- Use TLS protocol with Passthrough mode
- Hostname (or Hostname pattern) matching your Tenant Control Plane hostname
3. **DNS is configured** to resolve your hostnames to the Gateway's external address
3. (optional) **DNS is configured** to resolve hostnames (or hostname pattern) to the Gateway's LoadBalancer IP address. (This is needed for worker nodes to join, for testing we will use host entries in `/etc/hosts` for this guide)
4. **Gateway controller is running** (e.g., Envoy Gateway, Istio Gateway, etc.)
## Configuration
To replicate the guide below, please install [Envoy Gateway](https://gateway.envoyproxy.io/docs/tasks/quickstart/).
### TenantControlPlane Gateway Configuration
Next, create a gateway resource:
Enable Gateway API mode by setting the `spec.controlPlane.gateway` field in your TenantControlPlane resource:
#### Gateway Resource Setup
```yaml
apiVersion: kamaji.clastix.io/v1alpha1
kind: TenantControlPlane
metadata:
name: tcp-1
spec:
controlPlane:
# ... gateway configuration:
gateway:
hostname: "tcp1.cluster.dev"
parentRefs:
- name: gateway
namespace: default
additionalMetadata:
labels:
environment: production
annotations:
example.com/custom: "value"
# ... rest of the spec
deployment:
replicas: 1
service:
serviceType: ClusterIP
dataStore: default
kubernetes:
version: v1.29.0
kubelet:
cgroupfs: systemd
networkProfile:
port: 6443
certSANs:
- "c11.cluster.dev" # make sure to set this.
addons:
coreDNS: {}
kubeProxy: {}
konnectivity: {}
```
**Required fields:**
- `hostname`: The hostname that will be used for routing (must match Gateway listener hostname pattern)
- `parentRefs`: Array of Gateway references (name and namespace)
**Optional fields:**
- `additionalMetadata.labels`: Custom labels to add to TLSRoute resources
- `additionalMetadata.annotations`: Custom annotations to add to TLSRoute resources
!!! warning "Port and sectionName are set automatically"
Do not specify `port` or `sectionName` in `parentRefs`. Kamaji automatically sets these fields in TLSRoutes.
### Gateway Resource Setup
Your Gateway resource must have listeners configured for both the control plane and Konnectivity ports. Here's an example Gateway configuration:
Your Gateway resource must have listeners configured for the control plane. Here's an example Gateway configuration:
```yaml
apiVersion: gateway.networking.k8s.io/v1
@@ -130,27 +58,94 @@ spec:
kind: TLSRoute
namespaces:
from: All
```
The above gateway is configured with envoy gateway controller. You can achieve the same results with any other gateway controller that supports `TLSRoutes` and TLS passthrough mode.
The rest of this guide focuses on TCP.
## TenantControlPlane Gateway Configuration
Enable Gateway API mode by setting the `spec.controlPlane.gateway` field in your TenantControlPlane resource:
```yaml
apiVersion: kamaji.clastix.io/v1alpha1
kind: TenantControlPlane
metadata:
name: tcp-1
spec:
controlPlane:
# ... gateway configuration:
gateway:
hostname: "tcp1.cluster.dev"
parentRefs:
- name: gateway
namespace: default
sectionName: kube-apiserver
port: 6443
additionalMetadata:
labels:
environment: production
annotations:
example.com/custom: "value"
# ... rest of the spec
deployment:
replicas: 1
service:
serviceType: ClusterIP
dataStore: default
kubernetes:
version: v1.29.0
kubelet:
cgroupfs: systemd
networkProfile:
port: 6443
certSANs:
- "tcp1.cluster.dev" # make sure to set this.
addons:
coreDNS: {}
kubeProxy: {}
# if konnectivity addon is enabled:
- name: konnectivity-server
port: 8132
protocol: TLS
hostname: 'tcp1.cluster.dev'
tls:
mode: Passthrough
allowedRoutes:
kinds:
- group: gateway.networking.k8s.io
kind: TLSRoute
namespaces:
from: All
```
**Required fields:**
## Multiple Tenant Control Planes
- `hostname`: The hostname that will be used for routing (must match Gateway listener hostname pattern)
- `parentRefs`: Array of Gateway references
You can use the same Gateway resource for multiple Tenant Control Planes by using different hostnames:
**Optional fields:**
- `additionalMetadata.labels`: Custom labels to add to TLSRoute resources
- `additionalMetadata.annotations`: Custom annotations to add to TLSRoute resources
!!! info
### Verify
From our kubectl client machine / remote machines we can access the cluster above with the hostname.
**Step 1:** Fetch load balancer IP of the gateway.
`kubectl get gateway gateway -n default -o jsonpath='{.status.addresses[0].value}'`
**Step 2:** Add host entries in `/etc/hosts` with the above hostname and gateway LB IP.
`echo "<LB_IP> tcp1.cluster.dev" | sudo tee -a /etc/hosts`
**Step 3:** Fetch kubeconfig of tcp cluster.
`kubectl get secrets tcp-1-admin-kubeconfig -o jsonpath='{.data.admin\.conf}' | base64 -d > kubeconfig`
**Step 4:** Test connectivity:
`kubectl --kubeconfig kubeconfig cluster-info`
### Multiple Tenant Control Planes
We can use the same Gateway resource for multiple Tenant Control Planes by using different hostnames per tenant cluster.
Let us extend the above example for multiple tenant control planes behind single gateway (and LB IP).
```yaml
# Gateway with wildcard hostname
@@ -160,10 +155,13 @@ metadata:
name: gateway
spec:
listeners:
- hostname: '*.cluster.dev'
name: kube-apiserver
- name: kube-apiserver
port: 6443
# ...
# note: we changed to wildcard hostname pattern matching
# for cluster.dev
hostname: '*.cluster.dev'
# ...
---
# Tenant Control Plane 1
apiVersion: kamaji.clastix.io/v1alpha1
@@ -194,17 +192,37 @@ spec:
# ...
```
Each Tenant Control Plane will get its own TLSRoutes with the respective hostnames, all routing through the same Gateway resource.
Each Tenant Control Plane needs to use a different hostname. For each TCP, Kamaji creates a `TLSRoutes` resource with the respective hostnames, all `TLSRoutes` routing through the same Gateway resource.
You can check the Gateway status in the TenantControlPlane:
```bash
kubectl get tenantcontrolplane tcp-1 -o yaml
### Konnectivity
If konnectivity addon is enabled, Kamaji creates a separate TLSRoute for it. But this is hardcoded with the listener name `konnectivity-server` and port `8132`. All gateways mentioned in `spec.controlPlane.gateway.parentRefs` must contain a listener with the same configuration for the given hostname. Below is example configuration:
```yaml
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: gateway
namespace: default
spec:
gatewayClassName: envoy-gw-class
listeners:
- ...
- ...
- name: konnectivity-server
port: 8132
protocol: TLS
hostname: 'tcp1.cluster.dev'
tls:
mode: Passthrough
allowedRoutes:
kinds:
- group: gateway.networking.k8s.io
kind: TLSRoute
namespaces:
from: All
```
Look for the `status.kubernetesResources.gateway` and `status.addons.konnectivity.gateway` fields.
## Additional Resources
- [Gateway API Documentation](https://gateway-api.sigs.k8s.io/)

View File

@@ -27982,6 +27982,30 @@ DataStoreStatus defines the observed state of DataStore.
</tr>
</thead>
<tbody><tr>
<td><b>ready</b></td>
<td>boolean</td>
<td>
Ready returns if the DataStore is accepted and ready to get used:
Kamaji will ensure certificates are available, or correctly referenced.<br/>
</td>
<td>true</td>
</tr><tr>
<td><b><a href="#datastorestatusconditionsindex">conditions</a></b></td>
<td>[]object</td>
<td>
Conditions contains the validation conditions for the given Datastore.<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>observedGeneration</b></td>
<td>integer</td>
<td>
ObservedGeneration represents the .metadata.generation that was last reconciled.<br/>
<br/>
<i>Format</i>: int64<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>usedBy</b></td>
<td>[]string</td>
<td>
@@ -27991,6 +28015,81 @@ DataStoreStatus defines the observed state of DataStore.
</tr></tbody>
</table>
<span id="datastorestatusconditionsindex">`DataStore.status.conditions[index]`</span>
Condition contains details for one aspect of the current state of this API Resource.
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody><tr>
<td><b>lastTransitionTime</b></td>
<td>string</td>
<td>
lastTransitionTime is the last time the condition transitioned from one status to another.
This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable.<br/>
<br/>
<i>Format</i>: date-time<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>message</b></td>
<td>string</td>
<td>
message is a human readable message indicating details about the transition.
This may be an empty string.<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>reason</b></td>
<td>string</td>
<td>
reason contains a programmatic identifier indicating the reason for the condition's last transition.
Producers of specific condition types may define expected values and meanings for this field,
and whether the values are considered a guaranteed API.
The value should be a CamelCase string.
This field may not be empty.<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>status</b></td>
<td>enum</td>
<td>
status of the condition, one of True, False, Unknown.<br/>
<br/>
<i>Enum</i>: True, False, Unknown<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>type</b></td>
<td>string</td>
<td>
type of condition in CamelCase or in foo.example.com/CamelCase.<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>observedGeneration</b></td>
<td>integer</td>
<td>
observedGeneration represents the .metadata.generation that the condition was set based upon.
For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date
with respect to the current state of the instance.<br/>
<br/>
<i>Format</i>: int64<br/>
<i>Minimum</i>: 0<br/>
</td>
<td>false</td>
</tr></tbody>
</table>
### KubeconfigGenerator
@@ -28365,6 +28464,15 @@ In case of a different value compared to Resources, check the field errors.<br/>
Errors is the list of failed kubeconfig generations.<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>observedGeneration</b></td>
<td>integer</td>
<td>
ObservedGeneration represents the .metadata.generation that was last reconciled.<br/>
<br/>
<i>Format</i>: int64<br/>
</td>
<td>false</td>
</tr></tbody>
</table>
@@ -42555,6 +42663,15 @@ that are necessary to run a kubernetes control plane<br/>
Kubernetes contains information about the reconciliation of the required Kubernetes resources deployed in the admin cluster<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>observedGeneration</b></td>
<td>integer</td>
<td>
ObservedGeneration represents the .metadata.generation that was last reconciled.<br/>
<br/>
<i>Format</i>: int64<br/>
</td>
<td>false</td>
</tr><tr>
<td><b><a href="#tenantcontrolplanestatusstorage">storage</a></b></td>
<td>object</td>

View File

@@ -48,7 +48,9 @@ var _ = Describe("Deploy a TenantControlPlane with Gateway API and Konnectivity"
},
GatewayParentRefs: []gatewayv1.ParentReference{
{
Name: "test-gateway",
Name: "test-gateway",
Port: pointer.To(gatewayv1.PortNumber(6443)),
SectionName: pointer.To(gatewayv1.SectionName("cp-listener")),
},
},
},
@@ -96,6 +98,35 @@ var _ = Describe("Deploy a TenantControlPlane with Gateway API and Konnectivity"
StatusMustEqualTo(tcp, kamajiv1alpha1.VersionReady)
})
It("Should create control plane TLSRoute preserving user-provided parentRef fields", func() {
Eventually(func() error {
route := &gatewayv1alpha2.TLSRoute{}
if err := k8sClient.Get(context.Background(), types.NamespacedName{
Name: tcp.Name,
Namespace: tcp.Namespace,
}, route); err != nil {
return err
}
if len(route.Spec.ParentRefs) == 0 {
return fmt.Errorf("parentRefs is empty")
}
if route.Spec.ParentRefs[0].SectionName == nil {
return fmt.Errorf("sectionName is nil")
}
if *route.Spec.ParentRefs[0].SectionName != gatewayv1.SectionName("cp-listener") {
return fmt.Errorf("expected sectionName 'cp-listener', got '%s'", *route.Spec.ParentRefs[0].SectionName)
}
if route.Spec.ParentRefs[0].Port == nil {
return fmt.Errorf("port is nil")
}
if *route.Spec.ParentRefs[0].Port != gatewayv1.PortNumber(6443) {
return fmt.Errorf("expected port 6443, got '%d'", *route.Spec.ParentRefs[0].Port)
}
return nil
}).WithTimeout(time.Minute).Should(Succeed())
})
It("Should create Konnectivity TLSRoute with correct sectionName", func() {
Eventually(func() error {
route := &gatewayv1alpha2.TLSRoute{}

View File

@@ -48,7 +48,9 @@ var _ = Describe("Deploy a TenantControlPlane with Gateway API", func() {
},
GatewayParentRefs: []gatewayv1.ParentReference{
{
Name: "test-gateway",
Name: "test-gateway",
Port: pointer.To(gatewayv1.PortNumber(6443)),
SectionName: pointer.To(gatewayv1.SectionName("cp-listener")),
},
},
},
@@ -90,7 +92,7 @@ var _ = Describe("Deploy a TenantControlPlane with Gateway API", func() {
StatusMustEqualTo(tcp, kamajiv1alpha1.VersionReady)
})
It("Should create control plane TLSRoute with correct sectionName", func() {
It("Should create control plane TLSRoute preserving user-provided parentRef fields", func() {
Eventually(func() error {
route := &gatewayv1alpha2.TLSRoute{}
// TODO: Check ownership.
@@ -106,8 +108,14 @@ var _ = Describe("Deploy a TenantControlPlane with Gateway API", func() {
if route.Spec.ParentRefs[0].SectionName == nil {
return fmt.Errorf("sectionName is nil")
}
if *route.Spec.ParentRefs[0].SectionName != gatewayv1.SectionName("kube-apiserver") {
return fmt.Errorf("expected sectionName 'kube-apiserver', got '%s'", *route.Spec.ParentRefs[0].SectionName)
if *route.Spec.ParentRefs[0].SectionName != gatewayv1.SectionName("cp-listener") {
return fmt.Errorf("expected sectionName 'cp-listener', got '%s'", *route.Spec.ParentRefs[0].SectionName)
}
if route.Spec.ParentRefs[0].Port == nil {
return fmt.Errorf("port is nil")
}
if *route.Spec.ParentRefs[0].Port != gatewayv1.PortNumber(6443) {
return fmt.Errorf("expected port 6443, got '%d'", *route.Spec.ParentRefs[0].Port)
}
return nil

View File

@@ -5,10 +5,12 @@ package e2e
import (
"context"
"time"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/types"
pointer "k8s.io/utils/ptr"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
@@ -59,5 +61,15 @@ var _ = Describe("Deploy a TenantControlPlane resource", func() {
// Check if TenantControlPlane resource has been created
It("Should be Ready", func() {
StatusMustEqualTo(tcp, kamajiv1alpha1.VersionReady)
// ObservedGeneration is set at the end of successful reconciliation,
// after status becomes Ready, so we need to wait for it.
Eventually(func() int64 {
if err := k8sClient.Get(context.Background(), types.NamespacedName{Name: tcp.GetName(), Namespace: tcp.GetNamespace()}, tcp); err != nil {
return -1
}
return tcp.Status.ObservedGeneration
}, 30*time.Second, time.Second).Should(Equal(tcp.Generation),
"ObservedGeneration should equal Generation after successful reconciliation")
})
})

View File

@@ -212,7 +212,7 @@ func ScaleTenantControlPlane(tcp *kamajiv1alpha1.TenantControlPlane, replicas in
Expect(err).To(Succeed())
}
// CreateGatewayWithListeners creates a Gateway with both kube-apiserver and konnectivity-server listeners.
// CreateGatewayWithListeners creates a Gateway with control plane and konnectivity-server listeners.
func CreateGatewayWithListeners(gatewayName, namespace, gatewayClassName, hostname string) {
GinkgoHelper()
gateway := &gatewayv1.Gateway{
@@ -224,7 +224,7 @@ func CreateGatewayWithListeners(gatewayName, namespace, gatewayClassName, hostna
GatewayClassName: gatewayv1.ObjectName(gatewayClassName),
Listeners: []gatewayv1.Listener{
{
Name: "kube-apiserver",
Name: "cp-listener",
Port: 6443,
Protocol: gatewayv1.TLSProtocolType,
Hostname: pointer.To(gatewayv1.Hostname(hostname)),

10
go.mod
View File

@@ -18,15 +18,14 @@ require (
github.com/nats-io/nats.go v1.48.0
github.com/onsi/ginkgo/v2 v2.28.1
github.com/onsi/gomega v1.39.1
github.com/pkg/errors v0.9.1
github.com/prometheus/client_golang v1.23.2
github.com/prometheus/client_model v0.6.2
github.com/spf13/cobra v1.10.2
github.com/spf13/pflag v1.0.10
github.com/spf13/viper v1.21.0
github.com/testcontainers/testcontainers-go v0.40.0
go.etcd.io/etcd/api/v3 v3.6.7
go.etcd.io/etcd/client/v3 v3.6.7
go.etcd.io/etcd/api/v3 v3.6.8
go.etcd.io/etcd/client/v3 v3.6.8
go.uber.org/automaxprocs v1.6.0
gomodules.xyz/jsonpatch/v2 v2.5.0
k8s.io/api v0.35.0
@@ -36,7 +35,7 @@ require (
k8s.io/cluster-bootstrap v0.0.0
k8s.io/klog/v2 v2.130.1
k8s.io/kubelet v0.0.0
k8s.io/kubernetes v1.35.0
k8s.io/kubernetes v1.35.1
k8s.io/utils v0.0.0-20251002143259-bc988d571ff4
sigs.k8s.io/controller-runtime v0.22.4
sigs.k8s.io/gateway-api v1.4.1
@@ -124,6 +123,7 @@ require (
github.com/opencontainers/image-spec v1.1.1 // indirect
github.com/pelletier/go-toml/v2 v2.2.4 // indirect
github.com/peterbourgon/diskv v2.0.1+incompatible // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c // indirect
github.com/prometheus/common v0.66.1 // indirect
@@ -148,7 +148,7 @@ require (
github.com/x448/float16 v0.8.4 // indirect
github.com/xlab/treeprint v1.2.0 // indirect
github.com/yusufpapurcu/wmi v1.2.4 // indirect
go.etcd.io/etcd/client/pkg/v3 v3.6.7 // indirect
go.etcd.io/etcd/client/pkg/v3 v3.6.8 // indirect
go.opentelemetry.io/auto/sdk v1.1.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.60.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 // indirect

16
go.sum
View File

@@ -369,12 +369,12 @@ github.com/yusufpapurcu/wmi v1.2.4 h1:zFUKzehAFReQwLys1b/iSMl+JQGSCSjtVqQn9bBrPo
github.com/yusufpapurcu/wmi v1.2.4/go.mod h1:SBZ9tNy3G9/m5Oi98Zks0QjeHVDvuK0qfxQmPyzfmi0=
go.etcd.io/bbolt v1.4.3 h1:dEadXpI6G79deX5prL3QRNP6JB8UxVkqo4UPnHaNXJo=
go.etcd.io/bbolt v1.4.3/go.mod h1:tKQlpPaYCVFctUIgFKFnAlvbmB3tpy1vkTnDWohtc0E=
go.etcd.io/etcd/api/v3 v3.6.7 h1:7BNJ2gQmc3DNM+9cRkv7KkGQDayElg8x3X+tFDYS+E0=
go.etcd.io/etcd/api/v3 v3.6.7/go.mod h1:xJ81TLj9hxrYYEDmXTeKURMeY3qEDN24hqe+q7KhbnI=
go.etcd.io/etcd/client/pkg/v3 v3.6.7 h1:vvzgyozz46q+TyeGBuFzVuI53/yd133CHceNb/AhBVs=
go.etcd.io/etcd/client/pkg/v3 v3.6.7/go.mod h1:2IVulJ3FZ/czIGl9T4lMF1uxzrhRahLqe+hSgy+Kh7Q=
go.etcd.io/etcd/client/v3 v3.6.7 h1:9WqA5RpIBtdMxAy1ukXLAdtg2pAxNqW5NUoO2wQrE6U=
go.etcd.io/etcd/client/v3 v3.6.7/go.mod h1:2XfROY56AXnUqGsvl+6k29wrwsSbEh1lAouQB1vHpeE=
go.etcd.io/etcd/api/v3 v3.6.8 h1:gqb1VN92TAI6G2FiBvWcqKtHiIjr4SU2GdXxTwyexbM=
go.etcd.io/etcd/api/v3 v3.6.8/go.mod h1:qyQj1HZPUV3B5cbAL8scG62+fyz5dSxxu0w8pn28N6Q=
go.etcd.io/etcd/client/pkg/v3 v3.6.8 h1:Qs/5C0LNFiqXxYf2GU8MVjYUEXJ6sZaYOz0zEqQgy50=
go.etcd.io/etcd/client/pkg/v3 v3.6.8/go.mod h1:GsiTRUZE2318PggZkAo6sWb6l8JLVrnckTNfbG8PWtw=
go.etcd.io/etcd/client/v3 v3.6.8 h1:B3G76t1UykqAOrbio7s/EPatixQDkQBevN8/mwiplrY=
go.etcd.io/etcd/client/v3 v3.6.8/go.mod h1:MVG4BpSIuumPi+ELF7wYtySETmoTWBHVcDoHdVupwt8=
go.etcd.io/etcd/pkg/v3 v3.6.5 h1:byxWB4AqIKI4SBmquZUG1WGtvMfMaorXFoCcFbVeoxM=
go.etcd.io/etcd/pkg/v3 v3.6.5/go.mod h1:uqrXrzmMIJDEy5j00bCqhVLzR5jEJIwDp5wTlLwPGOU=
go.etcd.io/etcd/server/v3 v3.6.5 h1:4RbUb1Bd4y1WkBHmuF+cZII83JNQMuNXzyjwigQ06y0=
@@ -549,8 +549,8 @@ k8s.io/kube-proxy v0.35.0 h1:erv2wYmGZ6nyu/FtmaIb+ORD3q2rfZ4Fhn7VXs/8cPQ=
k8s.io/kube-proxy v0.35.0/go.mod h1:bd9lpN3uLLOOWc/CFZbkPEi9DTkzQQymbE8FqSU4bWk=
k8s.io/kubelet v0.35.0 h1:8cgJHCBCKLYuuQ7/Pxb/qWbJfX1LXIw7790ce9xHq7c=
k8s.io/kubelet v0.35.0/go.mod h1:ciRzAXn7C4z5iB7FhG1L2CGPPXLTVCABDlbXt/Zz8YA=
k8s.io/kubernetes v1.35.0 h1:PUOojD8c8E3csMP5NX+nLLne6SGqZjrYCscptyBfWMY=
k8s.io/kubernetes v1.35.0/go.mod h1:Tzk9Y9W/XUFFFgTUVg+BAowoFe+Pc7koGLuaiLHdcFg=
k8s.io/kubernetes v1.35.1 h1:qmjXSCDPnOuXPuJb5pv+eLzpXhhlD09Jid1pG/OvFU8=
k8s.io/kubernetes v1.35.1/go.mod h1:AaPpCpiS8oAqRbEwpY5r3RitLpwpVp5lVXKFkJril58=
k8s.io/system-validators v1.12.1 h1:AY1+COTLJN/Sj0w9QzH1H0yvyF3Kl6CguMnh32WlcUU=
k8s.io/system-validators v1.12.1/go.mod h1:awfSS706v9R12VC7u7K89FKfqVy44G+E0L1A0FX9Wmw=
k8s.io/utils v0.0.0-20251002143259-bc988d571ff4 h1:SjGebBtkBqHFOli+05xYbK8YF1Dzkbzn+gDM4X9T4Ck=

View File

@@ -12,7 +12,6 @@ import (
"strconv"
"strings"
"github.com/pkg/errors"
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
@@ -1022,7 +1021,7 @@ func (d Deployment) templateLabels(ctx context.Context, tenantControlPlane *kama
func (d Deployment) secretHashValue(ctx context.Context, client client.Client, namespace, name string) (string, error) {
secret := &corev1.Secret{}
if err := client.Get(ctx, types.NamespacedName{Namespace: namespace, Name: name}, secret); err != nil {
return "", errors.Wrap(err, "cannot retrieve *corev1.Secret for resource version retrieval")
return "", fmt.Errorf("cannot retrieve *corev1.Secret for resource version retrieval: %w", err)
}
return d.hashValue(*secret), nil

View File

@@ -17,7 +17,6 @@ import (
"net"
"time"
"github.com/pkg/errors"
"k8s.io/apimachinery/pkg/util/sets"
)
@@ -93,7 +92,7 @@ func GenerateCertificatePrivateKeyPair(template *x509.Certificate, caCertificate
caPrivKeyBytes, err := ParsePrivateKeyBytes(caPrivateKey)
if err != nil {
return nil, nil, errors.Wrap(err, "provided CA private key for certificate generation cannot be parsed")
return nil, nil, fmt.Errorf("provided CA private key for certificate generation cannot be parsed: %w", err)
}
return generateCertificateKeyPairBytes(template, caCertBytes, caPrivKeyBytes)
@@ -108,7 +107,7 @@ func ParseCertificateBytes(content []byte) (*x509.Certificate, error) {
crt, err := x509.ParseCertificate(pemContent.Bytes)
if err != nil {
return nil, errors.Wrap(err, "cannot parse x509 Certificate")
return nil, fmt.Errorf("cannot parse x509 Certificate: %w", err)
}
return crt, nil
@@ -124,7 +123,7 @@ func ParsePrivateKeyBytes(content []byte) (crypto.Signer, error) {
if pemContent.Type == "EC PRIVATE KEY" {
privateKey, err := x509.ParseECPrivateKey(pemContent.Bytes)
if err != nil {
return nil, errors.Wrap(err, "cannot parse EC Private Key")
return nil, fmt.Errorf("cannot parse EC Private Key: %w", err)
}
return privateKey, nil
@@ -132,7 +131,7 @@ func ParsePrivateKeyBytes(content []byte) (crypto.Signer, error) {
privateKey, err := x509.ParsePKCS1PrivateKey(pemContent.Bytes)
if err != nil {
return nil, errors.Wrap(err, "cannot parse PKCS1 Private Key")
return nil, fmt.Errorf("cannot parse PKCS1 Private Key: %w", err)
}
return privateKey, nil
@@ -209,12 +208,12 @@ func VerifyCertificate(cert, ca []byte, usages ...x509.ExtKeyUsage) (bool, error
func generateCertificateKeyPairBytes(template *x509.Certificate, caCert *x509.Certificate, caKey crypto.Signer) (*bytes.Buffer, *bytes.Buffer, error) {
certPrivKey, err := rsa.GenerateKey(cryptorand.Reader, 2048)
if err != nil {
return nil, nil, errors.Wrap(err, "cannot generate an RSA key")
return nil, nil, fmt.Errorf("cannot generate an RSA key: %w", err)
}
certBytes, err := x509.CreateCertificate(cryptorand.Reader, template, caCert, &certPrivKey.PublicKey, caKey)
if err != nil {
return nil, nil, errors.Wrap(err, "cannot create the certificate")
return nil, nil, fmt.Errorf("cannot create the certificate: %w", err)
}
certPEM := &bytes.Buffer{}
@@ -223,7 +222,7 @@ func generateCertificateKeyPairBytes(template *x509.Certificate, caCert *x509.Ce
Headers: nil,
Bytes: certBytes,
}); err != nil {
return nil, nil, errors.Wrap(err, "cannot encode the generate certificate bytes")
return nil, nil, fmt.Errorf("cannot encode the generate certificate bytes: %w", err)
}
certPrivKeyPEM := &bytes.Buffer{}
@@ -232,7 +231,7 @@ func generateCertificateKeyPairBytes(template *x509.Certificate, caCert *x509.Ce
Headers: nil,
Bytes: x509.MarshalPKCS1PrivateKey(certPrivKey),
}); err != nil {
return nil, nil, errors.Wrap(err, "cannot encode private key")
return nil, nil, fmt.Errorf("cannot encode private key: %w", err)
}
return certPEM, certPrivKeyPEM, nil

View File

@@ -7,7 +7,6 @@ import (
"context"
"fmt"
"github.com/pkg/errors"
"sigs.k8s.io/controller-runtime/pkg/client"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
@@ -16,7 +15,7 @@ import (
func NewStorageConnection(ctx context.Context, client client.Client, ds kamajiv1alpha1.DataStore) (Connection, error) {
cc, err := NewConnectionConfig(ctx, client, ds)
if err != nil {
return nil, errors.Wrap(err, "unable to create connection config object")
return nil, fmt.Errorf("unable to create connection config object: %w", err)
}
switch ds.Spec.Driver {

View File

@@ -11,7 +11,6 @@ import (
"net"
"strconv"
"github.com/pkg/errors"
"sigs.k8s.io/controller-runtime/pkg/client"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
@@ -67,7 +66,7 @@ func NewConnectionConfig(ctx context.Context, client client.Client, ds kamajiv1a
certificate, err := tls.X509KeyPair(crt, key)
if err != nil {
return nil, errors.Wrap(err, "cannot retrieve x.509 key pair from the Kine Secret")
return nil, fmt.Errorf("cannot retrieve x.509 key pair from the Kine Secret: %w", err)
}
tlsConfig.Certificates = []tls.Certificate{certificate}
@@ -93,12 +92,12 @@ func NewConnectionConfig(ctx context.Context, client client.Client, ds kamajiv1a
for _, ep := range ds.Spec.Endpoints {
host, stringPort, err := net.SplitHostPort(ep)
if err != nil {
return nil, errors.Wrap(err, "cannot retrieve host-port pair from DataStore endpoints")
return nil, fmt.Errorf("cannot retrieve host-port pair from DataStore endpoints: %w", err)
}
port, err := strconv.Atoi(stringPort)
if err != nil {
return nil, errors.Wrap(err, "cannot convert port from string for the given DataStore")
return nil, fmt.Errorf("cannot convert port from string for the given DataStore: %w", err)
}
eps = append(eps, ConnectionEndpoint{

View File

@@ -3,48 +3,48 @@
package errors
import "github.com/pkg/errors"
import "fmt"
func NewCreateUserError(err error) error {
return errors.Wrap(err, "cannot create user")
return fmt.Errorf("cannot create user: %w", err)
}
func NewGrantPrivilegesError(err error) error {
return errors.Wrap(err, "cannot grant privileges")
return fmt.Errorf("cannot grant privileges: %w", err)
}
func NewCheckUserExistsError(err error) error {
return errors.Wrap(err, "cannot check if user exists")
return fmt.Errorf("cannot check if user exists: %w", err)
}
func NewCheckGrantExistsError(err error) error {
return errors.Wrap(err, "cannot check if grant exists")
return fmt.Errorf("cannot check if grant exists: %w", err)
}
func NewDeleteUserError(err error) error {
return errors.Wrap(err, "cannot delete user")
return fmt.Errorf("cannot delete user: %w", err)
}
func NewCannotDeleteDatabaseError(err error) error {
return errors.Wrap(err, "cannot delete database")
return fmt.Errorf("cannot delete database: %w", err)
}
func NewCheckDatabaseExistError(err error) error {
return errors.Wrap(err, "cannot check if database exists")
return fmt.Errorf("cannot check if database exists: %w", err)
}
func NewRevokePrivilegesError(err error) error {
return errors.Wrap(err, "cannot revoke privileges")
return fmt.Errorf("cannot revoke privileges: %w", err)
}
func NewCloseConnectionError(err error) error {
return errors.Wrap(err, "cannot close connection")
return fmt.Errorf("cannot close connection: %w", err)
}
func NewCheckConnectionError(err error) error {
return errors.Wrap(err, "cannot check connection")
return fmt.Errorf("cannot check connection: %w", err)
}
func NewCreateDBError(err error) error {
return errors.Wrap(err, "cannot create database")
return fmt.Errorf("cannot create database: %w", err)
}

View File

@@ -7,13 +7,12 @@ import (
"context"
"fmt"
goerrors "github.com/pkg/errors"
"go.etcd.io/etcd/api/v3/authpb"
"go.etcd.io/etcd/api/v3/v3rpc/rpctypes"
etcdclient "go.etcd.io/etcd/client/v3"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
"github.com/clastix/kamaji/internal/datastore/errors"
dserrors "github.com/clastix/kamaji/internal/datastore/errors"
)
func NewETCDConnection(config ConnectionConfig) (Connection, error) {
@@ -44,7 +43,7 @@ type EtcdClient struct {
func (e *EtcdClient) CreateUser(ctx context.Context, user, password string) error {
if _, err := e.Client.Auth.UserAddWithOptions(ctx, user, password, &etcdclient.UserAddOptions{NoPassword: true}); err != nil {
return errors.NewCreateUserError(err)
return dserrors.NewCreateUserError(err)
}
return nil
@@ -56,18 +55,18 @@ func (e *EtcdClient) CreateDB(context.Context, string) error {
func (e *EtcdClient) GrantPrivileges(ctx context.Context, user, dbName string) error {
if _, err := e.Client.Auth.RoleAdd(ctx, dbName); err != nil {
return errors.NewGrantPrivilegesError(err)
return dserrors.NewGrantPrivilegesError(err)
}
permission := etcdclient.PermissionType(authpb.READWRITE)
key := e.buildKey(dbName)
if _, err := e.Client.RoleGrantPermission(ctx, dbName, key, etcdclient.GetPrefixRangeEnd(key), permission); err != nil {
return errors.NewGrantPrivilegesError(err)
return dserrors.NewGrantPrivilegesError(err)
}
if _, err := e.Client.UserGrantRole(ctx, user, dbName); err != nil {
return errors.NewGrantPrivilegesError(err)
return dserrors.NewGrantPrivilegesError(err)
}
return nil
@@ -75,11 +74,15 @@ func (e *EtcdClient) GrantPrivileges(ctx context.Context, user, dbName string) e
func (e *EtcdClient) UserExists(ctx context.Context, user string) (bool, error) {
if _, err := e.Client.UserGet(ctx, user); err != nil {
if goerrors.As(err, &rpctypes.ErrGRPCUserNotFound) {
// Convert gRPC error to comparable EtcdError using rpctypes.Error(),
// then compare against the client-side error constant.
// The == comparison is correct here as rpctypes.Error() normalizes
// gRPC status errors to comparable EtcdError struct values.
if rpctypes.Error(err) == rpctypes.ErrUserNotFound { //nolint:errorlint
return false, nil
}
return false, errors.NewCheckUserExistsError(err)
return false, dserrors.NewCheckUserExistsError(err)
}
return true, nil
@@ -92,16 +95,20 @@ func (e *EtcdClient) DBExists(context.Context, string) (bool, error) {
func (e *EtcdClient) GrantPrivilegesExists(ctx context.Context, username, dbName string) (bool, error) {
_, err := e.Client.RoleGet(ctx, dbName)
if err != nil {
if goerrors.As(err, &rpctypes.ErrGRPCRoleNotFound) {
// Convert gRPC error to comparable EtcdError using rpctypes.Error(),
// then compare against the client-side error constant.
// The == comparison is correct here as rpctypes.Error() normalizes
// gRPC status errors to comparable EtcdError struct values.
if rpctypes.Error(err) == rpctypes.ErrRoleNotFound { //nolint:errorlint
return false, nil
}
return false, errors.NewCheckGrantExistsError(err)
return false, dserrors.NewCheckGrantExistsError(err)
}
user, err := e.Client.UserGet(ctx, username)
if err != nil {
return false, errors.NewCheckGrantExistsError(err)
return false, dserrors.NewCheckGrantExistsError(err)
}
for _, i := range user.Roles {
@@ -115,7 +122,7 @@ func (e *EtcdClient) GrantPrivilegesExists(ctx context.Context, username, dbName
func (e *EtcdClient) DeleteUser(ctx context.Context, user string) error {
if _, err := e.Client.Auth.UserDelete(ctx, user); err != nil {
return errors.NewDeleteUserError(err)
return dserrors.NewDeleteUserError(err)
}
return nil
@@ -124,7 +131,7 @@ func (e *EtcdClient) DeleteUser(ctx context.Context, user string) error {
func (e *EtcdClient) DeleteDB(ctx context.Context, dbName string) error {
prefix := e.buildKey(dbName)
if _, err := e.Client.Delete(ctx, prefix, etcdclient.WithPrefix()); err != nil {
return errors.NewCannotDeleteDatabaseError(err)
return dserrors.NewCannotDeleteDatabaseError(err)
}
return nil
@@ -132,7 +139,7 @@ func (e *EtcdClient) DeleteDB(ctx context.Context, dbName string) error {
func (e *EtcdClient) RevokePrivileges(ctx context.Context, _, dbName string) error {
if _, err := e.Client.Auth.RoleDelete(ctx, dbName); err != nil {
return errors.NewRevokePrivilegesError(err)
return dserrors.NewRevokePrivilegesError(err)
}
return nil
@@ -146,7 +153,7 @@ func (e *EtcdClient) GetConnectionString() string {
func (e *EtcdClient) Close() error {
if err := e.Client.Close(); err != nil {
return errors.NewCloseConnectionError(err)
return dserrors.NewCloseConnectionError(err)
}
return nil
@@ -154,7 +161,7 @@ func (e *EtcdClient) Close() error {
func (e *EtcdClient) Check(ctx context.Context) error {
if _, err := e.Client.AuthStatus(ctx); err != nil {
return errors.NewCheckConnectionError(err)
return dserrors.NewCheckConnectionError(err)
}
return nil

View File

@@ -5,11 +5,11 @@ package datastore
import (
"context"
"errors"
"fmt"
"strings"
"github.com/nats-io/nats.go"
"github.com/pkg/errors"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
)
@@ -73,7 +73,7 @@ func (nc *NATSConnection) CreateUser(_ context.Context, _, _ string) error {
func (nc *NATSConnection) CreateDB(_ context.Context, dbName string) error {
_, err := nc.js.CreateKeyValue(&nats.KeyValueConfig{Bucket: dbName})
if err != nil {
return errors.Wrap(err, "unable to create the datastore")
return fmt.Errorf("unable to create the datastore: %w", err)
}
return nil

View File

@@ -10,7 +10,6 @@ import (
"strings"
"github.com/go-pg/pg/v10"
goerrors "github.com/pkg/errors"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
"github.com/clastix/kamaji/internal/datastore/errors"
@@ -236,7 +235,7 @@ func (r *PostgreSQLConnection) DeleteUser(ctx context.Context, user string) erro
func (r *PostgreSQLConnection) DeleteDB(ctx context.Context, dbName string) error {
if err := r.GrantPrivileges(ctx, r.rootUser, dbName); err != nil {
return errors.NewCannotDeleteDatabaseError(goerrors.Wrap(err, "cannot grant privileges to root user"))
return errors.NewCannotDeleteDatabaseError(fmt.Errorf("cannot grant privileges to root user: %w", err))
}
if _, err := r.db.ExecContext(ctx, fmt.Sprintf(postgresqlDropDBStatement, dbName)); err != nil {

View File

@@ -1,39 +0,0 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package utils
import (
"context"
"fmt"
"k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/types"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
)
// CheckExists ensures that the default Datastore exists before starting the manager.
func CheckExists(ctx context.Context, scheme *runtime.Scheme, datastoreName string) error {
if datastoreName == "" {
return nil
}
ctrlClient, err := client.New(ctrl.GetConfigOrDie(), client.Options{Scheme: scheme})
if err != nil {
return fmt.Errorf("unable to create controlerruntime.Client: %w", err)
}
if err = ctrlClient.Get(ctx, types.NamespacedName{Name: datastoreName}, &kamajiv1alpha1.DataStore{}); err != nil {
if errors.IsNotFound(err) {
return fmt.Errorf("the default Datastore %s doesn't exist", datastoreName)
}
return fmt.Errorf("an error occurred during datastore retrieval: %w", err)
}
return nil
}

View File

@@ -3,15 +3,21 @@
package errors
import "github.com/pkg/errors"
import "errors"
func ShouldReconcileErrorBeIgnored(err error) bool {
var (
nonExposedLBErr NonExposedLoadBalancerError
missingValidIPErr MissingValidIPError
migrationErr MigrationInProcessError
)
switch {
case errors.As(err, &NonExposedLoadBalancerError{}):
case errors.As(err, &nonExposedLBErr):
return true
case errors.As(err, &MissingValidIPError{}):
case errors.As(err, &missingValidIPErr):
return true
case errors.As(err, &MigrationInProcessError{}):
case errors.As(err, &migrationErr):
return true
default:
return false

View File

@@ -4,7 +4,8 @@
package kubeadm
import (
"github.com/pkg/errors"
"fmt"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
@@ -20,19 +21,19 @@ func BootstrapToken(client kubernetes.Interface, config *Configuration) error {
initConfiguration := config.InitConfiguration
if err := node.UpdateOrCreateTokens(client, false, initConfiguration.BootstrapTokens); err != nil {
return errors.Wrap(err, "error updating or creating token")
return fmt.Errorf("error updating or creating token: %w", err)
}
if err := node.AllowBootstrapTokensToGetNodes(client); err != nil {
return errors.Wrap(err, "error allowing bootstrap tokens to get Nodes")
return fmt.Errorf("error allowing bootstrap tokens to get Nodes: %w", err)
}
if err := node.AllowBootstrapTokensToPostCSRs(client); err != nil {
return errors.Wrap(err, "error allowing bootstrap tokens to post CSRs")
return fmt.Errorf("error allowing bootstrap tokens to post CSRs: %w", err)
}
if err := node.AutoApproveNodeBootstrapTokens(client); err != nil {
return errors.Wrap(err, "error auto-approving node bootstrap tokens")
return fmt.Errorf("error auto-approving node bootstrap tokens: %w", err)
}
if err := node.AutoApproveNodeCertificateRotation(client); err != nil {
@@ -66,7 +67,7 @@ func BootstrapToken(client kubernetes.Interface, config *Configuration) error {
}
if err := clusterinfo.CreateClusterInfoRBACRules(client); err != nil {
return errors.Wrap(err, "error creating clusterinfo RBAC rules")
return fmt.Errorf("error creating clusterinfo RBAC rules: %w", err)
}
return nil

View File

@@ -8,7 +8,6 @@ import (
"github.com/blang/semver"
jsonpatchv5 "github.com/evanphx/json-patch/v5"
"github.com/pkg/errors"
corev1 "k8s.io/api/core/v1"
rbacv1 "k8s.io/api/rbac/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
@@ -67,7 +66,7 @@ func UploadKubeletConfig(client kubernetes.Interface, config *Configuration, pat
}
if err = createConfigMapRBACRules(client, configMapName); err != nil {
return nil, errors.Wrap(err, "error creating kubelet configuration configmap RBAC rules")
return nil, fmt.Errorf("error creating kubelet configuration configmap RBAC rules: %w", err)
}
return nil, nil
@@ -98,15 +97,15 @@ func getKubeletConfigmapContent(kubeletConfiguration KubeletConfiguration, patch
if len(patch) > 0 {
kubeletConfig, patchErr := utilities.EncodeToJSON(&kc)
if patchErr != nil {
return nil, errors.Wrapf(patchErr, "unable to encode KubeletConfiguration to JSON for JSON patching")
return nil, fmt.Errorf("unable to encode KubeletConfiguration to JSON for JSON patching: %w", patchErr)
}
if kubeletConfig, patchErr = patch.Apply(kubeletConfig); patchErr != nil {
return nil, errors.Wrapf(patchErr, "unable to apply JSON patching to KubeletConfiguration")
return nil, fmt.Errorf("unable to apply JSON patching to KubeletConfiguration: %w", patchErr)
}
if patchErr = utilities.DecodeFromJSON(string(kubeletConfig), &kc); patchErr != nil {
return nil, errors.Wrapf(patchErr, "unable to decode JSON to KubeletConfiguration")
return nil, fmt.Errorf("unable to decode JSON to KubeletConfiguration: %w", patchErr)
}
}
@@ -159,7 +158,7 @@ func createConfigMapRBACRules(client kubernetes.Interface, configMapName string)
func generateKubeletConfigMapName(version string) (string, error) {
parsedVersion, err := semver.ParseTolerant(version)
if err != nil {
return "", errors.Wrapf(err, "failed to parse kubernetes version %q", version)
return "", fmt.Errorf("failed to parse kubernetes version %q: %w", version, err)
}
majorMinor := semver.Version{Major: parsedVersion.Major, Minor: parsedVersion.Minor}

View File

@@ -6,8 +6,8 @@ package addons
import (
"bytes"
"context"
"fmt"
"github.com/pkg/errors"
"github.com/prometheus/client_golang/prometheus"
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
@@ -219,7 +219,7 @@ func (c *CoreDNS) UpdateTenantControlPlaneStatus(_ context.Context, tcp *kamajiv
func (c *CoreDNS) decodeManifests(ctx context.Context, tcp *kamajiv1alpha1.TenantControlPlane) error {
tcpClient, config, err := resources.GetKubeadmManifestDeps(ctx, c.Client, tcp)
if err != nil {
return errors.Wrap(err, "unable to create manifests dependencies")
return fmt.Errorf("unable to create manifests dependencies: %w", err)
}
// If CoreDNS addon is enabled and with an override, adding these to the kubeadm init configuration
@@ -235,38 +235,38 @@ func (c *CoreDNS) decodeManifests(ctx context.Context, tcp *kamajiv1alpha1.Tenan
manifests, err := kubeadm.AddCoreDNS(tcpClient, config)
if err != nil {
return errors.Wrap(err, "unable to generate manifests")
return fmt.Errorf("unable to generate manifests: %w", err)
}
parts := bytes.Split(manifests, []byte("---"))
if err = utilities.DecodeFromYAML(string(parts[1]), c.deployment); err != nil {
return errors.Wrap(err, "unable to decode Deployment manifest")
return fmt.Errorf("unable to decode Deployment manifest: %w", err)
}
addons_utils.SetKamajiManagedLabels(c.deployment)
if err = utilities.DecodeFromYAML(string(parts[2]), c.configMap); err != nil {
return errors.Wrap(err, "unable to decode ConfigMap manifest")
return fmt.Errorf("unable to decode ConfigMap manifest: %w", err)
}
addons_utils.SetKamajiManagedLabels(c.configMap)
if err = utilities.DecodeFromYAML(string(parts[3]), c.service); err != nil {
return errors.Wrap(err, "unable to decode Service manifest")
return fmt.Errorf("unable to decode Service manifest: %w", err)
}
addons_utils.SetKamajiManagedLabels(c.service)
if err = utilities.DecodeFromYAML(string(parts[4]), c.clusterRole); err != nil {
return errors.Wrap(err, "unable to decode ClusterRole manifest")
return fmt.Errorf("unable to decode ClusterRole manifest: %w", err)
}
addons_utils.SetKamajiManagedLabels(c.clusterRole)
if err = utilities.DecodeFromYAML(string(parts[5]), c.clusterRoleBinding); err != nil {
return errors.Wrap(err, "unable to decode ClusterRoleBinding manifest")
return fmt.Errorf("unable to decode ClusterRoleBinding manifest: %w", err)
}
addons_utils.SetKamajiManagedLabels(c.clusterRoleBinding)
if err = utilities.DecodeFromYAML(string(parts[6]), c.serviceAccount); err != nil {
return errors.Wrap(err, "unable to decode ServiceAccount manifest")
return fmt.Errorf("unable to decode ServiceAccount manifest: %w", err)
}
addons_utils.SetKamajiManagedLabels(c.serviceAccount)

View File

@@ -6,8 +6,8 @@ package addons
import (
"bytes"
"context"
"fmt"
"github.com/pkg/errors"
"github.com/prometheus/client_golang/prometheus"
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
@@ -321,7 +321,7 @@ func (k *KubeProxy) mutateDaemonSet(ctx context.Context, tenantClient client.Cli
func (k *KubeProxy) decodeManifests(ctx context.Context, tcp *kamajiv1alpha1.TenantControlPlane) error {
tcpClient, config, err := resources.GetKubeadmManifestDeps(ctx, k.Client, tcp)
if err != nil {
return errors.Wrap(err, "unable to create manifests dependencies")
return fmt.Errorf("unable to create manifests dependencies: %w", err)
}
// If the kube-proxy addon has overrides, adding it to the kubeadm parameters
config.Parameters.KubeProxyOptions = &kubeadm.AddonOptions{}
@@ -340,38 +340,38 @@ func (k *KubeProxy) decodeManifests(ctx context.Context, tcp *kamajiv1alpha1.Ten
manifests, err := kubeadm.AddKubeProxy(tcpClient, config)
if err != nil {
return errors.Wrap(err, "unable to generate manifests")
return fmt.Errorf("unable to generate manifests: %w", err)
}
parts := bytes.Split(manifests, []byte("---"))
if err = utilities.DecodeFromYAML(string(parts[1]), k.serviceAccount); err != nil {
return errors.Wrap(err, "unable to decode ServiceAccount manifest")
return fmt.Errorf("unable to decode ServiceAccount manifest: %w", err)
}
addon_utils.SetKamajiManagedLabels(k.serviceAccount)
if err = utilities.DecodeFromYAML(string(parts[2]), k.clusterRoleBinding); err != nil {
return errors.Wrap(err, "unable to decode ClusterRoleBinding manifest")
return fmt.Errorf("unable to decode ClusterRoleBinding manifest: %w", err)
}
addon_utils.SetKamajiManagedLabels(k.clusterRoleBinding)
if err = utilities.DecodeFromYAML(string(parts[3]), k.role); err != nil {
return errors.Wrap(err, "unable to decode Role manifest")
return fmt.Errorf("unable to decode Role manifest: %w", err)
}
addon_utils.SetKamajiManagedLabels(k.role)
if err = utilities.DecodeFromYAML(string(parts[4]), k.roleBinding); err != nil {
return errors.Wrap(err, "unable to decode RoleBinding manifest")
return fmt.Errorf("unable to decode RoleBinding manifest: %w", err)
}
addon_utils.SetKamajiManagedLabels(k.roleBinding)
if err = utilities.DecodeFromYAML(string(parts[5]), k.configMap); err != nil {
return errors.Wrap(err, "unable to decode ConfigMap manifest")
return fmt.Errorf("unable to decode ConfigMap manifest: %w", err)
}
addon_utils.SetKamajiManagedLabels(k.configMap)
if err = utilities.DecodeFromYAML(string(parts[6]), k.daemonSet); err != nil {
return errors.Wrap(err, "unable to decode DaemonSet manifest")
return fmt.Errorf("unable to decode DaemonSet manifest: %w", err)
}
addon_utils.SetKamajiManagedLabels(k.daemonSet)

View File

@@ -5,8 +5,8 @@ package datastore
import (
"context"
"errors"
"github.com/pkg/errors"
"github.com/prometheus/client_golang/prometheus"
"k8s.io/apimachinery/pkg/util/sets"
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"

View File

@@ -5,8 +5,8 @@ package datastore
import (
"context"
"fmt"
"github.com/pkg/errors"
"github.com/prometheus/client_golang/prometheus"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
@@ -192,7 +192,7 @@ func (r *Setup) UpdateTenantControlPlaneStatus(_ context.Context, tenantControlP
func (r *Setup) createDB(ctx context.Context, _ *kamajiv1alpha1.TenantControlPlane) (controllerutil.OperationResult, error) {
exists, err := r.Connection.DBExists(ctx, r.resource.schema)
if err != nil {
return controllerutil.OperationResultNone, errors.Wrap(err, "unable to check if datastore exists")
return controllerutil.OperationResultNone, fmt.Errorf("unable to check if datastore exists: %w", err)
}
if exists {
@@ -200,7 +200,7 @@ func (r *Setup) createDB(ctx context.Context, _ *kamajiv1alpha1.TenantControlPla
}
if err := r.Connection.CreateDB(ctx, r.resource.schema); err != nil {
return controllerutil.OperationResultNone, errors.Wrap(err, "unable to create the datastore")
return controllerutil.OperationResultNone, fmt.Errorf("unable to create the datastore: %w", err)
}
return controllerutil.OperationResultCreated, nil
@@ -209,7 +209,7 @@ func (r *Setup) createDB(ctx context.Context, _ *kamajiv1alpha1.TenantControlPla
func (r *Setup) deleteDB(ctx context.Context, _ *kamajiv1alpha1.TenantControlPlane) error {
exists, err := r.Connection.DBExists(ctx, r.resource.schema)
if err != nil {
return errors.Wrap(err, "unable to check if datastore exists")
return fmt.Errorf("unable to check if datastore exists: %w", err)
}
if !exists {
@@ -217,7 +217,7 @@ func (r *Setup) deleteDB(ctx context.Context, _ *kamajiv1alpha1.TenantControlPla
}
if err := r.Connection.DeleteDB(ctx, r.resource.schema); err != nil {
return errors.Wrap(err, "unable to delete the datastore")
return fmt.Errorf("unable to delete the datastore: %w", err)
}
return nil
@@ -226,7 +226,7 @@ func (r *Setup) deleteDB(ctx context.Context, _ *kamajiv1alpha1.TenantControlPla
func (r *Setup) createUser(ctx context.Context, _ *kamajiv1alpha1.TenantControlPlane) (controllerutil.OperationResult, error) {
exists, err := r.Connection.UserExists(ctx, r.resource.user)
if err != nil {
return controllerutil.OperationResultNone, errors.Wrap(err, "unable to check if user exists")
return controllerutil.OperationResultNone, fmt.Errorf("unable to check if user exists: %w", err)
}
if exists {
@@ -234,7 +234,7 @@ func (r *Setup) createUser(ctx context.Context, _ *kamajiv1alpha1.TenantControlP
}
if err := r.Connection.CreateUser(ctx, r.resource.user, r.resource.password); err != nil {
return controllerutil.OperationResultNone, errors.Wrap(err, "unable to create the user")
return controllerutil.OperationResultNone, fmt.Errorf("unable to create the user: %w", err)
}
return controllerutil.OperationResultCreated, nil
@@ -243,7 +243,7 @@ func (r *Setup) createUser(ctx context.Context, _ *kamajiv1alpha1.TenantControlP
func (r *Setup) deleteUser(ctx context.Context, _ *kamajiv1alpha1.TenantControlPlane) error {
exists, err := r.Connection.UserExists(ctx, r.resource.user)
if err != nil {
return errors.Wrap(err, "unable to check if user exists")
return fmt.Errorf("unable to check if user exists: %w", err)
}
if !exists {
@@ -251,7 +251,7 @@ func (r *Setup) deleteUser(ctx context.Context, _ *kamajiv1alpha1.TenantControlP
}
if err := r.Connection.DeleteUser(ctx, r.resource.user); err != nil {
return errors.Wrap(err, "unable to remove the user")
return fmt.Errorf("unable to remove the user: %w", err)
}
return nil
@@ -260,7 +260,7 @@ func (r *Setup) deleteUser(ctx context.Context, _ *kamajiv1alpha1.TenantControlP
func (r *Setup) createGrantPrivileges(ctx context.Context, _ *kamajiv1alpha1.TenantControlPlane) (controllerutil.OperationResult, error) {
exists, err := r.Connection.GrantPrivilegesExists(ctx, r.resource.user, r.resource.schema)
if err != nil {
return controllerutil.OperationResultNone, errors.Wrap(err, "unable to check if privileges exist")
return controllerutil.OperationResultNone, fmt.Errorf("unable to check if privileges exist: %w", err)
}
if exists {
@@ -268,7 +268,7 @@ func (r *Setup) createGrantPrivileges(ctx context.Context, _ *kamajiv1alpha1.Ten
}
if err := r.Connection.GrantPrivileges(ctx, r.resource.user, r.resource.schema); err != nil {
return controllerutil.OperationResultNone, errors.Wrap(err, "unable to grant privileges")
return controllerutil.OperationResultNone, fmt.Errorf("unable to grant privileges: %w", err)
}
return controllerutil.OperationResultCreated, nil
@@ -277,7 +277,7 @@ func (r *Setup) createGrantPrivileges(ctx context.Context, _ *kamajiv1alpha1.Ten
func (r *Setup) revokeGrantPrivileges(ctx context.Context, _ *kamajiv1alpha1.TenantControlPlane) error {
exists, err := r.Connection.GrantPrivilegesExists(ctx, r.resource.user, r.resource.schema)
if err != nil {
return errors.Wrap(err, "unable to check if privileges exist")
return fmt.Errorf("unable to check if privileges exist: %w", err)
}
if !exists {
@@ -285,7 +285,7 @@ func (r *Setup) revokeGrantPrivileges(ctx context.Context, _ *kamajiv1alpha1.Ten
}
if err := r.Connection.RevokePrivileges(ctx, r.resource.user, r.resource.schema); err != nil {
return errors.Wrap(err, "unable to revoke privileges")
return fmt.Errorf("unable to revoke privileges: %w", err)
}
return nil

View File

@@ -5,9 +5,9 @@ package datastore
import (
"context"
"fmt"
"github.com/google/uuid"
"github.com/pkg/errors"
"github.com/prometheus/client_golang/prometheus"
corev1 "k8s.io/api/core/v1"
kubeerrors "k8s.io/apimachinery/pkg/api/errors"
@@ -82,7 +82,7 @@ func (r *Config) Delete(ctx context.Context, _ *kamajiv1alpha1.TenantControlPlan
return nil
}
return errors.Wrap(err, "cannot retrieve the DataStore Secret for removal")
return fmt.Errorf("cannot retrieve the DataStore Secret for removal: %w", err)
}
secret.SetFinalizers(nil)
@@ -92,7 +92,7 @@ func (r *Config) Delete(ctx context.Context, _ *kamajiv1alpha1.TenantControlPlan
return nil
}
return errors.Wrap(err, "cannot remove DataStore Secret finalizers")
return fmt.Errorf("cannot remove DataStore Secret finalizers: %w", err)
}
return nil
@@ -138,12 +138,12 @@ func (r *Config) mutate(ctx context.Context, tenantControlPlane *kamajiv1alpha1.
// set username and password to the basicAuth values of the NATS datastore
u, err := r.DataStore.Spec.BasicAuth.Username.GetContent(ctx, r.Client)
if err != nil {
return errors.Wrap(err, "failed to retrieve the username for the NATS datastore")
return fmt.Errorf("failed to retrieve the username for the NATS datastore: %w", err)
}
p, err := r.DataStore.Spec.BasicAuth.Password.GetContent(ctx, r.Client)
if err != nil {
return errors.Wrap(err, "failed to retrieve the password for the NATS datastore")
return fmt.Errorf("failed to retrieve the password for the NATS datastore: %w", err)
}
username = u

View File

@@ -150,6 +150,10 @@ func (r *KubernetesGatewayResource) mutate(tcp *kamajiv1alpha1.TenantControlPlan
tcp.Spec.ControlPlane.Gateway.AdditionalMetadata.Annotations)
r.resource.SetAnnotations(annotations)
if tcp.Spec.ControlPlane.Gateway.GatewayParentRefs != nil {
r.resource.Spec.ParentRefs = tcp.Spec.ControlPlane.Gateway.GatewayParentRefs
}
serviceName := gatewayv1alpha2.ObjectName(tcp.Status.Kubernetes.Service.Name)
servicePort := tcp.Status.Kubernetes.Service.Port
@@ -157,11 +161,6 @@ func (r *KubernetesGatewayResource) mutate(tcp *kamajiv1alpha1.TenantControlPlan
return fmt.Errorf("service not ready, cannot create TLSRoute")
}
if tcp.Spec.ControlPlane.Gateway.GatewayParentRefs != nil {
// Copy parentRefs and explicitly set port and sectionName fields
r.resource.Spec.ParentRefs = NewParentRefsSpecWithPortAndSection(tcp.Spec.ControlPlane.Gateway.GatewayParentRefs, servicePort, "kube-apiserver")
}
rule := gatewayv1alpha2.TLSRouteRule{
BackendRefs: []gatewayv1alpha2.BackendRef{
{

View File

@@ -102,30 +102,6 @@ var _ = Describe("KubernetesGatewayResource", func() {
Expect(shouldUpdate).To(BeTrue())
})
It("should set port and sectionName in parentRefs, overriding any user-provided values", func() {
customPort := gatewayv1.PortNumber(9999)
customSectionName := gatewayv1.SectionName("custom")
tcp.Spec.ControlPlane.Gateway.GatewayParentRefs[0].Port = &customPort
tcp.Spec.ControlPlane.Gateway.GatewayParentRefs[0].SectionName = &customSectionName
err := resource.Define(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
_, err = resource.CreateOrUpdate(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
route := &gatewayv1alpha2.TLSRoute{}
err = resource.Client.Get(ctx, client.ObjectKey{Name: tcp.Name, Namespace: tcp.Namespace}, route)
Expect(err).NotTo(HaveOccurred())
Expect(route.Spec.ParentRefs).To(HaveLen(1))
Expect(route.Spec.ParentRefs[0].Port).NotTo(BeNil())
Expect(*route.Spec.ParentRefs[0].Port).To(Equal(tcp.Status.Kubernetes.Service.Port))
Expect(*route.Spec.ParentRefs[0].Port).NotTo(Equal(customPort))
Expect(route.Spec.ParentRefs[0].SectionName).NotTo(BeNil())
Expect(*route.Spec.ParentRefs[0].SectionName).To(Equal(gatewayv1.SectionName("kube-apiserver")))
Expect(*route.Spec.ParentRefs[0].SectionName).NotTo(Equal(customSectionName))
})
It("should handle multiple parentRefs correctly", func() {
namespace := gatewayv1.Namespace("default")
tcp.Spec.ControlPlane.Gateway.GatewayParentRefs = []gatewayv1alpha2.ParentReference{
@@ -149,13 +125,12 @@ var _ = Describe("KubernetesGatewayResource", func() {
err = resource.Client.Get(ctx, client.ObjectKey{Name: tcp.Name, Namespace: tcp.Namespace}, route)
Expect(err).NotTo(HaveOccurred())
Expect(route.Spec.ParentRefs).To(HaveLen(2))
for i := range route.Spec.ParentRefs {
Expect(route.Spec.ParentRefs[i].Port).NotTo(BeNil())
Expect(*route.Spec.ParentRefs[i].Port).To(Equal(tcp.Status.Kubernetes.Service.Port))
Expect(route.Spec.ParentRefs[i].SectionName).NotTo(BeNil())
Expect(*route.Spec.ParentRefs[i].SectionName).To(Equal(gatewayv1.SectionName("kube-apiserver")))
}
Expect(route.Spec.ParentRefs[0].Name).To(Equal(gatewayv1alpha2.ObjectName("test-gateway-1")))
Expect(route.Spec.ParentRefs[0].Namespace).NotTo(BeNil())
Expect(*route.Spec.ParentRefs[0].Namespace).To(Equal(namespace))
Expect(route.Spec.ParentRefs[1].Name).To(Equal(gatewayv1alpha2.ObjectName("test-gateway-2")))
Expect(route.Spec.ParentRefs[1].Namespace).NotTo(BeNil())
Expect(*route.Spec.ParentRefs[1].Namespace).To(Equal(namespace))
})
})
@@ -292,81 +267,4 @@ var _ = Describe("KubernetesGatewayResource", func() {
Expect(listener.Port).To(Equal(gatewayv1.PortNumber(80)))
})
})
Describe("NewParentRefsSpecWithPortAndSection", func() {
var (
parentRefs []gatewayv1.ParentReference
testPort int32
testSectionName string
)
BeforeEach(func() {
namespace := gatewayv1.Namespace("default")
namespace2 := gatewayv1.Namespace("other")
testPort = int32(6443)
testSectionName = "kube-apiserver"
originalPort := gatewayv1.PortNumber(9999)
originalSectionName := gatewayv1.SectionName("original")
parentRefs = []gatewayv1.ParentReference{
{
Name: "test-gateway-1",
Namespace: &namespace,
Port: &originalPort,
SectionName: &originalSectionName,
},
{
Name: "test-gateway-2",
Namespace: &namespace2,
},
}
})
It("should create copy of parentRefs with port and sectionName set", func() {
result := resources.NewParentRefsSpecWithPortAndSection(parentRefs, testPort, testSectionName)
Expect(result).To(HaveLen(2))
for i := range result {
Expect(result[i].Name).To(Equal(parentRefs[i].Name))
Expect(result[i].Namespace).To(Equal(parentRefs[i].Namespace))
Expect(result[i].Port).NotTo(BeNil())
Expect(*result[i].Port).To(Equal(testPort))
Expect(result[i].SectionName).NotTo(BeNil())
Expect(*result[i].SectionName).To(Equal(gatewayv1.SectionName(testSectionName)))
}
})
It("should not modify original parentRefs", func() {
// Store original values for verification
originalFirstPort := parentRefs[0].Port
originalFirstSectionName := parentRefs[0].SectionName
originalSecondPort := parentRefs[1].Port
originalSecondSectionName := parentRefs[1].SectionName
result := resources.NewParentRefsSpecWithPortAndSection(parentRefs, testPort, testSectionName)
// Original should remain unchanged
Expect(parentRefs[0].Port).To(Equal(originalFirstPort))
Expect(parentRefs[0].SectionName).To(Equal(originalFirstSectionName))
Expect(parentRefs[1].Port).To(Equal(originalSecondPort))
Expect(parentRefs[1].SectionName).To(Equal(originalSecondSectionName))
// Result should have new values
Expect(result[0].Port).NotTo(BeNil())
Expect(*result[0].Port).To(Equal(testPort))
Expect(result[0].SectionName).NotTo(BeNil())
Expect(*result[0].SectionName).To(Equal(gatewayv1.SectionName(testSectionName)))
Expect(result[1].Port).NotTo(BeNil())
Expect(*result[1].Port).To(Equal(testPort))
Expect(result[1].SectionName).NotTo(BeNil())
Expect(*result[1].SectionName).To(Equal(gatewayv1.SectionName(testSectionName)))
})
It("should handle empty parentRefs slice", func() {
parentRefs = []gatewayv1.ParentReference{}
result := resources.NewParentRefsSpecWithPortAndSection(parentRefs, testPort, testSectionName)
Expect(result).To(BeEmpty())
})
})
})

View File

@@ -226,16 +226,3 @@ func BuildGatewayAccessPointsStatus(ctx context.Context, c client.Client, route
return accessPoints, nil
}
// NewParentRefsSpecWithPortAndSection creates a copy of parentRefs with port and sectionName set for each reference.
func NewParentRefsSpecWithPortAndSection(parentRefs []gatewayv1.ParentReference, port int32, sectionName string) []gatewayv1.ParentReference {
result := make([]gatewayv1.ParentReference, len(parentRefs))
sectionNamePtr := gatewayv1.SectionName(sectionName)
for i, parentRef := range parentRefs {
result[i] = *parentRef.DeepCopy()
result[i].Port = &port
result[i].SectionName = &sectionNamePtr
}
return result
}

View File

@@ -48,7 +48,7 @@ func (r *KubernetesServiceResource) CleanUp(context.Context, *kamajiv1alpha1.Ten
}
func (r *KubernetesServiceResource) UpdateTenantControlPlaneStatus(ctx context.Context, tenantControlPlane *kamajiv1alpha1.TenantControlPlane) error {
tenantControlPlane.Status.Kubernetes.Service.ServiceStatus = r.resource.Status
tenantControlPlane.Status.Kubernetes.Service.ServiceStatus = StripLoadBalancerPortsFromServiceStatus(r.resource.Status)
tenantControlPlane.Status.Kubernetes.Service.Name = r.resource.GetName()
tenantControlPlane.Status.Kubernetes.Service.Namespace = r.resource.GetNamespace()
tenantControlPlane.Status.Kubernetes.Service.Port = r.resource.Spec.Ports[0].Port

View File

@@ -182,7 +182,7 @@ func (r *KubernetesKonnectivityGatewayResource) mutate(tcp *kamajiv1alpha1.Tenan
if tcp.Spec.ControlPlane.Gateway.GatewayParentRefs == nil {
return fmt.Errorf("control plane gateway parentRefs are not specified")
}
r.resource.Spec.ParentRefs = resources.NewParentRefsSpecWithPortAndSection(tcp.Spec.ControlPlane.Gateway.GatewayParentRefs, servicePort, "konnectivity-server")
r.resource.Spec.ParentRefs = newParentRefsSpecWithPortAndSection(tcp.Spec.ControlPlane.Gateway.GatewayParentRefs, servicePort, "konnectivity-server")
rule := gatewayv1alpha2.TLSRouteRule{
BackendRefs: []gatewayv1alpha2.BackendRef{
@@ -230,3 +230,16 @@ func (r *KubernetesKonnectivityGatewayResource) CreateOrUpdate(ctx context.Conte
func (r *KubernetesKonnectivityGatewayResource) GetName() string {
return "konnectivity_gateway_routes"
}
// newParentRefsSpecWithPortAndSection creates a copy of parentRefs with port and sectionName set for each reference.
func newParentRefsSpecWithPortAndSection(parentRefs []gatewayv1.ParentReference, port int32, sectionName string) []gatewayv1.ParentReference {
result := make([]gatewayv1.ParentReference, len(parentRefs))
sectionNamePtr := gatewayv1.SectionName(sectionName)
for i, parentRef := range parentRefs {
result[i] = *parentRef.DeepCopy()
result[i].Port = &port
result[i].SectionName = &sectionNamePtr
}
return result
}

View File

@@ -113,7 +113,7 @@ func (r *ServiceResource) UpdateTenantControlPlaneStatus(_ context.Context, tena
tenantControlPlane.Status.Addons.Konnectivity.Service.Name = r.resource.GetName()
tenantControlPlane.Status.Addons.Konnectivity.Service.Namespace = r.resource.GetNamespace()
tenantControlPlane.Status.Addons.Konnectivity.Service.Port = r.resource.Spec.Ports[1].Port
tenantControlPlane.Status.Addons.Konnectivity.Service.ServiceStatus = r.resource.Status
tenantControlPlane.Status.Addons.Konnectivity.Service.ServiceStatus = resources.StripLoadBalancerPortsFromServiceStatus(r.resource.Status)
}
return nil

View File

@@ -11,7 +11,6 @@ import (
"time"
jsonpatchv5 "github.com/evanphx/json-patch/v5"
"github.com/pkg/errors"
"github.com/prometheus/client_golang/prometheus"
corev1 "k8s.io/api/core/v1"
rbacv1 "k8s.io/api/rbac/v1"
@@ -158,10 +157,10 @@ func (r *KubeadmPhase) GetKubeadmFunction(ctx context.Context, tcp *kamajiv1alph
if len(tcp.Spec.Kubernetes.Kubelet.ConfigurationJSONPatches) > 0 {
jsonP, patchErr := tcp.Spec.Kubernetes.Kubelet.ConfigurationJSONPatches.ToJSON()
if patchErr != nil {
return nil, errors.Wrap(patchErr, "cannot encode JSON Patches to JSON")
return nil, fmt.Errorf("cannot encode JSON Patches to JSON: %w", patchErr)
}
if patch, patchErr = jsonpatchv5.DecodePatch(jsonP); patchErr != nil {
return nil, errors.Wrap(patchErr, "cannot decode JSON Patches")
return nil, fmt.Errorf("cannot decode JSON Patches: %w", patchErr)
}
}

View File

@@ -7,7 +7,6 @@ import (
"context"
"fmt"
"github.com/pkg/errors"
"github.com/prometheus/client_golang/prometheus"
"k8s.io/apimachinery/pkg/util/version"
"k8s.io/kubernetes/cmd/kubeadm/app/phases/upgrade"
@@ -75,7 +74,7 @@ func (k *KubernetesUpgrade) CreateOrUpdate(ctx context.Context, tenantControlPla
// Checking if the upgrade is allowed, or not
clientSet, err := utilities.GetTenantClientSet(ctx, k.Client, tenantControlPlane)
if err != nil {
return controllerutil.OperationResultNone, errors.Wrap(err, "cannot create REST client required for Kubernetes upgrade plan")
return controllerutil.OperationResultNone, fmt.Errorf("cannot create REST client required for Kubernetes upgrade plan: %w", err)
}
var coreDNSVersion string
@@ -86,7 +85,7 @@ func (k *KubernetesUpgrade) CreateOrUpdate(ctx context.Context, tenantControlPla
versionGetter := kamajiupgrade.NewKamajiKubeVersionGetter(clientSet, tenantControlPlane.Status.Kubernetes.Version.Version, coreDNSVersion, tenantControlPlane.Status.Kubernetes.Version.Status)
if _, err = upgrade.GetAvailableUpgrades(versionGetter, false, false, &printers.Discard{}); err != nil {
return controllerutil.OperationResultNone, errors.Wrap(err, "cannot retrieve available Upgrades for Kubernetes upgrade plan")
return controllerutil.OperationResultNone, fmt.Errorf("cannot retrieve available Upgrades for Kubernetes upgrade plan: %w", err)
}
if err = k.isUpgradable(); err != nil {
@@ -123,12 +122,12 @@ func (k *KubernetesUpgrade) UpdateTenantControlPlaneStatus(_ context.Context, te
func (k *KubernetesUpgrade) isUpgradable() error {
newK8sVersion, err := version.ParseSemantic(k.upgrade.After.KubeVersion)
if err != nil {
return errors.Wrap(err, fmt.Sprintf("unable to parse normalized version %q as a semantic version", k.upgrade.After.KubeVersion))
return fmt.Errorf("unable to parse normalized version %q as a semantic version: %w", k.upgrade.After.KubeVersion, err)
}
oldK8sVersion, err := version.ParseSemantic(k.upgrade.Before.KubeVersion)
if err != nil {
return errors.Wrap(err, fmt.Sprintf("unable to parse normalized version %q as a semantic version", k.upgrade.After.KubeVersion))
return fmt.Errorf("unable to parse normalized version %q as a semantic version: %w", k.upgrade.After.KubeVersion, err)
}
if newK8sVersion.Minor() < oldK8sVersion.Minor() {

View File

@@ -5,9 +5,9 @@ package resources
import (
"context"
"fmt"
"github.com/go-logr/logr"
"github.com/pkg/errors"
corev1 "k8s.io/api/core/v1"
k8serrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
@@ -25,17 +25,17 @@ import (
func GetKubeadmManifestDeps(ctx context.Context, client client.Client, tenantControlPlane *kamajiv1alpha1.TenantControlPlane) (*clientset.Clientset, *kubeadm.Configuration, error) {
config, err := getStoredKubeadmConfiguration(ctx, client, "", tenantControlPlane)
if err != nil {
return nil, nil, errors.Wrap(err, "cannot retrieve kubeadm configuration")
return nil, nil, fmt.Errorf("cannot retrieve kubeadm configuration: %w", err)
}
kubeconfig, err := utilities.GetTenantKubeconfig(ctx, client, tenantControlPlane)
if err != nil {
return nil, nil, errors.Wrap(err, "cannot retrieve kubeconfig configuration")
return nil, nil, fmt.Errorf("cannot retrieve kubeconfig configuration: %w", err)
}
address, _, err := tenantControlPlane.AssignedControlPlaneAddress()
if err != nil {
return nil, nil, errors.Wrap(err, "cannot retrieve Tenant Control Plane address")
return nil, nil, fmt.Errorf("cannot retrieve Tenant Control Plane address: %w", err)
}
config.Kubeconfig = *kubeconfig
@@ -80,7 +80,7 @@ func GetKubeadmManifestDeps(ctx context.Context, client client.Client, tenantCon
tenantClient, err := utilities.GetTenantClientSet(ctx, client, tenantControlPlane)
if err != nil {
return nil, nil, errors.Wrap(err, "cannot generate tenant client")
return nil, nil, fmt.Errorf("cannot generate tenant client: %w", err)
}
return tenantClient, config, nil

View File

@@ -132,3 +132,18 @@ func getStoredKubeadmConfiguration(ctx context.Context, client client.Client, tm
return config, nil
}
func StripLoadBalancerPortsFromServiceStatus(s corev1.ServiceStatus) corev1.ServiceStatus {
sanitized := s
if len(s.LoadBalancer.Ingress) > 0 {
sanitized.LoadBalancer.Ingress = make([]corev1.LoadBalancerIngress, len(s.LoadBalancer.Ingress))
copy(sanitized.LoadBalancer.Ingress, s.LoadBalancer.Ingress)
}
for i := range sanitized.LoadBalancer.Ingress {
sanitized.LoadBalancer.Ingress[i].Ports = nil
}
return sanitized
}

View File

@@ -0,0 +1,111 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package resources
import (
"testing"
corev1 "k8s.io/api/core/v1"
)
func TestStripLoadBalancerPortsFromServiceStatus(t *testing.T) {
ipModeProxy := corev1.LoadBalancerIPModeProxy
tests := []struct {
name string
input corev1.ServiceStatus
assert func(t *testing.T, orig, got corev1.ServiceStatus)
}{
{
name: "ip ingress with ports and ipMode",
input: corev1.ServiceStatus{
LoadBalancer: corev1.LoadBalancerStatus{
Ingress: []corev1.LoadBalancerIngress{
{
IP: "172.18.0.3",
IPMode: &ipModeProxy,
Ports: []corev1.PortStatus{
{Port: 6443, Protocol: corev1.ProtocolTCP},
{Port: 8132, Protocol: corev1.ProtocolTCP},
},
},
},
},
},
assert: func(t *testing.T, orig, got corev1.ServiceStatus) {
t.Helper()
if got.LoadBalancer.Ingress[0].Ports != nil {
t.Fatalf("expected ports stripped, got %#v", got.LoadBalancer.Ingress[0].Ports)
}
if got.LoadBalancer.Ingress[0].IP != "172.18.0.3" {
t.Fatalf("IP not preserved")
}
if got.LoadBalancer.Ingress[0].IPMode == nil || *got.LoadBalancer.Ingress[0].IPMode != ipModeProxy {
t.Fatalf("IPMode not preserved")
}
if orig.LoadBalancer.Ingress[0].Ports == nil {
t.Fatalf("original ports mutated")
}
},
},
{
name: "hostname ingress with ports",
input: corev1.ServiceStatus{
LoadBalancer: corev1.LoadBalancerStatus{
Ingress: []corev1.LoadBalancerIngress{
{
Hostname: "example.local",
Ports: []corev1.PortStatus{
{Port: 6443, Protocol: corev1.ProtocolTCP},
},
},
},
},
},
assert: func(t *testing.T, orig, got corev1.ServiceStatus) {
t.Helper()
if got.LoadBalancer.Ingress[0].Ports != nil {
t.Fatalf("expected ports stripped")
}
if got.LoadBalancer.Ingress[0].Hostname != "example.local" {
t.Fatalf("hostname not preserved")
}
},
},
{
name: "no ingress",
input: corev1.ServiceStatus{},
assert: func(t *testing.T, _, got corev1.ServiceStatus) {
t.Helper()
if len(got.LoadBalancer.Ingress) != 0 {
t.Fatalf("expected no ingress")
}
},
},
{
name: "ingress with nil ports",
input: corev1.ServiceStatus{
LoadBalancer: corev1.LoadBalancerStatus{
Ingress: []corev1.LoadBalancerIngress{
{IP: "10.0.0.1"},
},
},
},
assert: func(t *testing.T, _, got corev1.ServiceStatus) {
t.Helper()
if got.LoadBalancer.Ingress[0].Ports != nil {
t.Fatalf("expected ports to stay nil")
}
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
orig := tt.input
got := StripLoadBalancerPortsFromServiceStatus(tt.input)
tt.assert(t, orig, got)
})
}
}

View File

@@ -7,7 +7,6 @@ import (
"fmt"
"runtime"
"github.com/pkg/errors"
versionutil "k8s.io/apimachinery/pkg/util/version"
apimachineryversion "k8s.io/apimachinery/pkg/version"
"k8s.io/client-go/kubernetes"
@@ -63,7 +62,7 @@ func (k kamajiKubeVersionGetter) KubeadmVersion() (string, *versionutil.Version,
kubeadmVersion, err := versionutil.ParseSemantic(kubeadmVersionInfo.String())
if err != nil {
return "", nil, errors.Wrap(err, "Couldn't parse kubeadm version")
return "", nil, fmt.Errorf("couldn't parse kubeadm version: %w", err)
}
return kubeadmVersionInfo.String(), kubeadmVersion, nil

View File

@@ -9,7 +9,6 @@ import (
"net/http"
"strings"
"github.com/pkg/errors"
"gomodules.xyz/jsonpatch/v2"
admissionv1 "k8s.io/api/admission/v1"
"k8s.io/apimachinery/pkg/runtime"
@@ -34,11 +33,11 @@ func (h handlersChainer) Handler(object runtime.Object, routeHandlers ...handler
// When deleting the OldObject struct field contains the object being deleted:
// https://github.com/kubernetes/kubernetes/pull/76346
if err := h.decoder.DecodeRaw(req.OldObject, decodedObj); err != nil {
return admission.Errored(http.StatusInternalServerError, errors.Wrap(err, fmt.Sprintf("unable to decode deleted object into %T", object)))
return admission.Errored(http.StatusInternalServerError, fmt.Errorf("unable to decode deleted object into %T: %w", object, err))
}
default:
if err := h.decoder.Decode(req, decodedObj); err != nil {
return admission.Errored(http.StatusInternalServerError, errors.Wrap(err, fmt.Sprintf("unable to decode into %T", object)))
return admission.Errored(http.StatusInternalServerError, fmt.Errorf("unable to decode into %T: %w", object, err))
}
}
}
@@ -70,7 +69,7 @@ func (h handlersChainer) Handler(object runtime.Object, routeHandlers ...handler
}
case admissionv1.Update:
if err := h.decoder.DecodeRaw(req.OldObject, oldDecodedObj); err != nil {
return admission.Errored(http.StatusInternalServerError, errors.Wrap(err, fmt.Sprintf("unable to decode old object into %T", object)))
return admission.Errored(http.StatusInternalServerError, fmt.Errorf("unable to decode old object into %T: %w", object, err))
}
for _, routeHandler := range routeHandlers {

View File

@@ -8,7 +8,6 @@ import (
"fmt"
"strings"
"github.com/pkg/errors"
"gomodules.xyz/jsonpatch/v2"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/fields"
@@ -39,7 +38,7 @@ func (d DataStoreSecretValidation) OnUpdate(object runtime.Object, _ runtime.Obj
dsList := &kamajiv1alpha1.DataStoreList{}
if err := d.Client.List(ctx, dsList, client.MatchingFieldsSelector{Selector: fields.OneTermEqualSelector(kamajiv1alpha1.DatastoreUsedSecretNamespacedNameKey, fmt.Sprintf("%s/%s", secret.GetNamespace(), secret.GetName()))}); err != nil {
return nil, errors.Wrap(err, "cannot list Tenant Control Plane using the provided Secret")
return nil, fmt.Errorf("cannot list Tenant Control Plane using the provided Secret: %w", err)
}
if len(dsList.Items) > 0 {

View File

@@ -1,145 +0,0 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package handlers
import (
"context"
"fmt"
"github.com/pkg/errors"
"gomodules.xyz/jsonpatch/v2"
corev1 "k8s.io/api/core/v1"
k8serrors "k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/fields"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/types"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/webhook/admission"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
)
type DataStoreValidation struct {
Client client.Client
}
func (d DataStoreValidation) OnCreate(object runtime.Object) AdmissionResponse {
return func(ctx context.Context, _ admission.Request) ([]jsonpatch.JsonPatchOperation, error) {
ds := object.(*kamajiv1alpha1.DataStore) //nolint:forcetypeassert
return nil, d.validate(ctx, *ds)
}
}
func (d DataStoreValidation) OnDelete(object runtime.Object) AdmissionResponse {
return func(ctx context.Context, _ admission.Request) ([]jsonpatch.JsonPatchOperation, error) {
ds := object.(*kamajiv1alpha1.DataStore) //nolint:forcetypeassert
tcpList := &kamajiv1alpha1.TenantControlPlaneList{}
if err := d.Client.List(ctx, tcpList, client.MatchingFieldsSelector{Selector: fields.OneTermEqualSelector(kamajiv1alpha1.TenantControlPlaneUsedDataStoreKey, ds.GetName())}); err != nil {
return nil, errors.Wrap(err, "cannot retrieve TenantControlPlane list used by the DataStore")
}
if len(tcpList.Items) > 0 {
return nil, fmt.Errorf("the DataStore is used by multiple TenantControlPlanes and cannot be removed")
}
return nil, nil
}
}
func (d DataStoreValidation) OnUpdate(object runtime.Object, oldObj runtime.Object) AdmissionResponse {
return func(ctx context.Context, _ admission.Request) ([]jsonpatch.JsonPatchOperation, error) {
newDs, oldDs := object.(*kamajiv1alpha1.DataStore), oldObj.(*kamajiv1alpha1.DataStore) //nolint:forcetypeassert
if oldDs.Spec.Driver != newDs.Spec.Driver {
return nil, fmt.Errorf("driver of a DataStore cannot be changed")
}
return nil, d.validate(ctx, *newDs)
}
}
func (d DataStoreValidation) validate(ctx context.Context, ds kamajiv1alpha1.DataStore) error {
if ds.Spec.BasicAuth != nil {
if err := d.validateBasicAuth(ctx, ds); err != nil {
return err
}
}
return d.validateTLSConfig(ctx, ds)
}
func (d DataStoreValidation) validateBasicAuth(ctx context.Context, ds kamajiv1alpha1.DataStore) error {
if err := d.validateContentReference(ctx, ds.Spec.BasicAuth.Password); err != nil {
return fmt.Errorf("basic-auth password is not valid, %w", err)
}
if err := d.validateContentReference(ctx, ds.Spec.BasicAuth.Username); err != nil {
return fmt.Errorf("basic-auth username is not valid, %w", err)
}
return nil
}
func (d DataStoreValidation) validateTLSConfig(ctx context.Context, ds kamajiv1alpha1.DataStore) error {
if ds.Spec.TLSConfig == nil && ds.Spec.Driver != kamajiv1alpha1.EtcdDriver {
return nil
}
if err := d.validateContentReference(ctx, ds.Spec.TLSConfig.CertificateAuthority.Certificate); err != nil {
return fmt.Errorf("CA certificate is not valid, %w", err)
}
if ds.Spec.Driver == kamajiv1alpha1.EtcdDriver {
if ds.Spec.TLSConfig.CertificateAuthority.PrivateKey == nil {
return fmt.Errorf("CA private key is required when using the etcd driver")
}
if ds.Spec.TLSConfig.ClientCertificate == nil {
return fmt.Errorf("client certificate is required when using the etcd driver")
}
}
if ds.Spec.TLSConfig.CertificateAuthority.PrivateKey != nil {
if err := d.validateContentReference(ctx, *ds.Spec.TLSConfig.CertificateAuthority.PrivateKey); err != nil {
return fmt.Errorf("CA private key is not valid, %w", err)
}
}
if ds.Spec.TLSConfig.ClientCertificate != nil {
if err := d.validateContentReference(ctx, ds.Spec.TLSConfig.ClientCertificate.Certificate); err != nil {
return fmt.Errorf("client certificate is not valid, %w", err)
}
if err := d.validateContentReference(ctx, ds.Spec.TLSConfig.ClientCertificate.PrivateKey); err != nil {
return fmt.Errorf("client private key is not valid, %w", err)
}
}
return nil
}
func (d DataStoreValidation) validateContentReference(ctx context.Context, ref kamajiv1alpha1.ContentRef) error {
switch {
case len(ref.Content) > 0:
return nil
case ref.SecretRef == nil:
return fmt.Errorf("the Secret reference is mandatory when bare content is not specified")
case len(ref.SecretRef.SecretReference.Name) == 0:
return fmt.Errorf("the Secret reference name is mandatory")
case len(ref.SecretRef.SecretReference.Namespace) == 0:
return fmt.Errorf("the Secret reference namespace is mandatory")
}
if err := d.Client.Get(ctx, types.NamespacedName{Name: ref.SecretRef.SecretReference.Name, Namespace: ref.SecretRef.SecretReference.Namespace}, &corev1.Secret{}); err != nil {
if k8serrors.IsNotFound(err) {
return fmt.Errorf("secret %s/%s is not found", ref.SecretRef.SecretReference.Namespace, ref.SecretRef.SecretReference.Name)
}
return err
}
return nil
}

View File

@@ -5,9 +5,9 @@ package handlers
import (
"context"
"fmt"
"net"
"github.com/pkg/errors"
"gomodules.xyz/jsonpatch/v2"
"k8s.io/apimachinery/pkg/runtime"
pointer "k8s.io/utils/ptr"
@@ -31,7 +31,7 @@ func (t TenantControlPlaneDefaults) OnCreate(object runtime.Object) AdmissionRes
if len(defaulted.Spec.NetworkProfile.DNSServiceIPs) == 0 {
ip, _, err := net.ParseCIDR(defaulted.Spec.NetworkProfile.ServiceCIDR)
if err != nil {
return nil, errors.Wrap(err, "cannot define resulting DNS Service IP")
return nil, fmt.Errorf("cannot define resulting DNS Service IP: %w", err)
}
switch {
case ip.To4() != nil:
@@ -45,7 +45,7 @@ func (t TenantControlPlaneDefaults) OnCreate(object runtime.Object) AdmissionRes
operations, err := utils.JSONPatch(original, defaulted)
if err != nil {
return nil, errors.Wrap(err, "cannot create patch responses upon Tenant Control Plane creation")
return nil, fmt.Errorf("cannot create patch responses upon Tenant Control Plane creation: %w", err)
}
return operations, nil

View File

@@ -5,9 +5,9 @@ package handlers
import (
"context"
"fmt"
"github.com/google/go-cmp/cmp"
"github.com/pkg/errors"
"gomodules.xyz/jsonpatch/v2"
appsv1 "k8s.io/api/apps/v1"
k8serrors "k8s.io/apimachinery/pkg/api/errors"
@@ -103,7 +103,7 @@ func (t TenantControlPlaneDeployment) OnUpdate(newObject runtime.Object, oldObje
}
if err != nil {
return nil, errors.Wrap(err, "the resulting Deployment will generate a configuration error, cannot proceed")
return nil, fmt.Errorf("the resulting Deployment will generate a configuration error, cannot proceed: %w", err)
}
return nil, nil

View File

@@ -9,7 +9,6 @@ import (
"strings"
"github.com/blang/semver"
"github.com/pkg/errors"
"gomodules.xyz/jsonpatch/v2"
"k8s.io/apimachinery/pkg/runtime"
"sigs.k8s.io/controller-runtime/pkg/webhook/admission"
@@ -27,14 +26,14 @@ func (t TenantControlPlaneVersion) OnCreate(object runtime.Object) AdmissionResp
ver, err := semver.New(t.normalizeKubernetesVersion(tcp.Spec.Kubernetes.Version))
if err != nil {
return nil, errors.Wrap(err, "unable to parse the desired Kubernetes version")
return nil, fmt.Errorf("unable to parse the desired Kubernetes version: %w", err)
}
// No need to check if the patch version
ver.Patch = 0
supportedVer, supportedErr := semver.Make(t.normalizeKubernetesVersion(upgrade.KubeadmVersion))
if supportedErr != nil {
return nil, errors.Wrap(supportedErr, "unable to parse the Kamaji supported Kubernetes version")
return nil, fmt.Errorf("unable to parse the Kamaji supported Kubernetes version: %w", supportedErr)
}
if ver.GT(supportedVer) {
@@ -67,21 +66,21 @@ func (t TenantControlPlaneVersion) OnUpdate(object runtime.Object, oldObject run
oldVer, oldErr := semver.Make(t.normalizeKubernetesVersion(oldTCP.Spec.Kubernetes.Version))
if oldErr != nil {
return nil, errors.Wrap(oldErr, "unable to parse the previous Kubernetes version")
return nil, fmt.Errorf("unable to parse the previous Kubernetes version: %w", oldErr)
}
// No need to check if the patch version
oldVer.Patch = 0
newVer, newErr := semver.New(t.normalizeKubernetesVersion(newTCP.Spec.Kubernetes.Version))
if newErr != nil {
return nil, errors.Wrap(newErr, "unable to parse the desired Kubernetes version")
return nil, fmt.Errorf("unable to parse the desired Kubernetes version: %w", newErr)
}
// No need to check if the patch version
newVer.Patch = 0
supportedVer, supportedErr := semver.Make(t.normalizeKubernetesVersion(upgrade.KubeadmVersion))
if supportedErr != nil {
return nil, errors.Wrap(supportedErr, "unable to parse the Kamaji supported Kubernetes version")
return nil, fmt.Errorf("unable to parse the Kamaji supported Kubernetes version: %w", supportedErr)
}
switch {

View File

@@ -1,21 +0,0 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package routes
import (
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/runtime"
)
//+kubebuilder:webhook:path=/validate--v1-secret,mutating=false,failurePolicy=ignore,sideEffects=None,groups="",resources=secrets,verbs=delete,versions=v1,name=vdatastoresecrets.kb.io,admissionReviewVersions=v1
type DataStoreSecrets struct{}
func (d DataStoreSecrets) GetPath() string {
return "/validate--v1-secret"
}
func (d DataStoreSecrets) GetObject() runtime.Object {
return &corev1.Secret{}
}

View File

@@ -1,22 +0,0 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package routes
import (
"k8s.io/apimachinery/pkg/runtime"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
)
//+kubebuilder:webhook:path=/validate-kamaji-clastix-io-v1alpha1-datastore,mutating=false,failurePolicy=fail,sideEffects=None,groups=kamaji.clastix.io,resources=datastores,verbs=create;update;delete,versions=v1alpha1,name=vdatastore.kb.io,admissionReviewVersions=v1
type DataStoreValidate struct{}
func (d DataStoreValidate) GetPath() string {
return "/validate-kamaji-clastix-io-v1alpha1-datastore"
}
func (d DataStoreValidate) GetObject() runtime.Object {
return &kamajiv1alpha1.DataStore{}
}

View File

@@ -4,8 +4,9 @@
package utils
import (
"fmt"
json "github.com/json-iterator/go"
"github.com/pkg/errors"
"gomodules.xyz/jsonpatch/v2"
"sigs.k8s.io/controller-runtime/pkg/client"
)
@@ -13,12 +14,12 @@ import (
func JSONPatch(original, modified client.Object) ([]jsonpatch.Operation, error) {
originalJSON, err := json.Marshal(original)
if err != nil {
return nil, errors.Wrap(err, "cannot marshal original object")
return nil, fmt.Errorf("cannot marshal original object: %w", err)
}
modifiedJSON, err := json.Marshal(modified)
if err != nil {
return nil, errors.Wrap(err, "cannot marshal modified object")
return nil, fmt.Errorf("cannot marshal modified object: %w", err)
}
return jsonpatch.CreatePatch(originalJSON, modifiedJSON)