Compare commits

..

32 Commits

Author SHA1 Message Date
Matteo Ruina
a9c2c0de89 fix(style): migrate from deprecated github.com/pkg/errors package (#1071)
* refactor: migrate error packages from pkg/errors to stdlib

Replace github.com/pkg/errors with Go standard library error handling
in foundation error packages:

- internal/datastore/errors: errors.Wrap -> fmt.Errorf with %w
- internal/errors: errors.As -> stdlib errors.As
- controllers/soot/controllers/errors: errors.New -> stdlib errors.New

Part 1 of 4 in the pkg/errors migration.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: migrate datastore package from pkg/errors to stdlib

Replace github.com/pkg/errors with Go standard library error handling
in the datastore layer:

- connection.go: errors.Wrap -> fmt.Errorf with %w
- datastore.go: errors.Wrap -> fmt.Errorf with %w
- etcd.go: goerrors alias removed, use stdlib errors.As
- nats.go: errors.Wrap/Is/New -> stdlib equivalents
- postgresql.go: goerrors.Wrap -> fmt.Errorf with %w

Part 2 of 4 in the pkg/errors migration.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: migrate internal packages from pkg/errors to stdlib (partial)

Replace github.com/pkg/errors with Go standard library error handling
in internal packages:

- internal/builders/controlplane: errors.Wrap -> fmt.Errorf
- internal/crypto: errors.Wrap -> fmt.Errorf
- internal/kubeadm: errors.Wrap/Wrapf -> fmt.Errorf
- internal/upgrade: errors.Wrap -> fmt.Errorf
- internal/webhook: errors.Wrap -> fmt.Errorf

Part 3 of 4 in the pkg/errors migration.

Remaining files: internal/resources/*.go (8 files, 42 occurrences)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(resources): migrate from pkg/errors to stdlib

Replace github.com/pkg/errors with Go standard library:
- errors.Wrap(err, msg) → fmt.Errorf("msg: %w", err)
- errors.New(msg) → errors.New(msg)

Files migrated:
- internal/resources/kubeadm_phases.go
- internal/resources/kubeadm_upgrade.go
- internal/resources/kubeadm_utils.go
- internal/resources/datastore/datastore_multitenancy.go
- internal/resources/datastore/datastore_setup.go
- internal/resources/datastore/datastore_storage_config.go
- internal/resources/addons/coredns.go
- internal/resources/addons/kube_proxy.go

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(controllers): migrate from pkg/errors to stdlib

Replace github.com/pkg/errors with Go standard library:
- errors.Wrap(err, msg) → fmt.Errorf("msg: %w", err)
- errors.New(msg) → errors.New(msg) (stdlib)
- errors.Is/As → errors.Is/As (stdlib)

Files migrated:
- controllers/datastore_controller.go
- controllers/kubeconfiggenerator_controller.go
- controllers/tenantcontrolplane_controller.go
- controllers/telemetry_controller.go
- controllers/certificate_lifecycle_controller.go

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(soot): migrate from pkg/errors to stdlib

Replace github.com/pkg/errors with Go standard library:
- errors.Is() now uses stdlib errors.Is()

Files migrated:
- controllers/soot/controllers/kubeproxy.go
- controllers/soot/controllers/migrate.go
- controllers/soot/controllers/coredns.go
- controllers/soot/controllers/konnectivity.go
- controllers/soot/controllers/kubeadm_phase.go

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(api,cmd): migrate from pkg/errors to stdlib

Replace github.com/pkg/errors with Go standard library:
- errors.Wrap(err, msg) → fmt.Errorf("msg: %w", err)

Files migrated:
- api/v1alpha1/tenantcontrolplane_funcs.go
- cmd/utils/k8s_version.go

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: run go mod tidy after pkg/errors migration

The github.com/pkg/errors package moved from direct to indirect
dependency. It remains as an indirect dependency because other
packages in the dependency tree still use it.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(datastore): use errors.Is for sentinel error comparison

The stdlib errors.As expects a pointer to a concrete error type, not
a pointer to an error value. For comparing against sentinel errors
like rpctypes.ErrGRPCUserNotFound, errors.Is should be used instead.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: resolve golangci-lint errors

- Fix GCI import formatting (remove extra blank lines between groups)
- Use errors.Is instead of errors.As for mutex sentinel errors

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(errors): use proper variable declarations for errors.As

The errors.As function requires a pointer to an assignable variable,
not a pointer to a composite literal. The previous pattern
`errors.As(err, &SomeError{})` creates a pointer to a temporary value
which errors.As cannot reliably use for assignment.

This fix declares proper variables for each error type and passes
their addresses to errors.As, ensuring correct error chain matching.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(datastore/etcd): use rpctypes.Error() for gRPC error comparison

The etcd gRPC status errors (ErrGRPCUserNotFound, ErrGRPCRoleNotFound)
cannot be compared directly using errors.Is() because they are wrapped
in gRPC status errors during transmission.

The etcd rpctypes package provides:
- ErrGRPC* constants: server-side gRPC status errors
- Err* constants (without GRPC prefix): client-side comparable errors
- Error() function: converts gRPC errors to comparable EtcdError values

The correct pattern is to use rpctypes.Error(err) to normalize the
received error, then compare against client-side error constants
like rpctypes.ErrUserNotFound.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-05 06:11:14 +01:00
Matteo Ruina
b40428825d feat: add ObservedGeneration (#1069)
* feat: add ObservedGeneration to all status types

Add ObservedGeneration field to DataStoreStatus, KubeconfigGeneratorStatus,
and TenantControlPlaneStatus to track which generation the controller has
processed. This enables clients and tools like kstatus to determine if the
controller has reconciled the latest spec changes.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: follow Cluster API pattern for ObservedGeneration

Move ObservedGeneration setting for TenantControlPlane from intermediate
status updates to the final successful reconciliation completion. This
follows Cluster API conventions where ObservedGeneration indicates the
controller has fully processed the given generation.

Previously, ObservedGeneration was set on every status update during
resource processing, which could mislead clients into thinking the spec
was fully reconciled when the controller was still mid-reconciliation
or had hit a transient error.

Now:
- DataStore: Sets ObservedGeneration before single status update (simple controller)
- KubeconfigGenerator: Sets ObservedGeneration before single status update (simple controller)
- TenantControlPlane: Sets ObservedGeneration only after ALL resources processed successfully

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: verify ObservedGeneration equals Generation after reconciliation

Add assertion to e2e test to verify that status.observedGeneration
equals metadata.generation after a TenantControlPlane is successfully
reconciled.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: regenerate CRDs with ObservedGeneration field

Run make crds to regenerate CRDs with the new ObservedGeneration
field in status types.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Run make manifests

* Run make apidoc

* Remove rbac role

* Remove webhook manifest

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 12:18:46 +01:00
Dario Tranchitella
c0316956a8 fix(psql): trailing junk after numeric literal (#1070)
Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>
2026-02-01 11:01:28 +01:00
dependabot[bot]
4c8f77e883 feat(deps): bump github.com/onsi/ginkgo/v2 from 2.27.5 to 2.28.1 (#1067)
Bumps [github.com/onsi/ginkgo/v2](https://github.com/onsi/ginkgo) from 2.27.5 to 2.28.1.
- [Release notes](https://github.com/onsi/ginkgo/releases)
- [Changelog](https://github.com/onsi/ginkgo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/ginkgo/compare/v2.27.5...v2.28.1)

---
updated-dependencies:
- dependency-name: github.com/onsi/ginkgo/v2
  dependency-version: 2.28.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-31 18:06:10 +01:00
dependabot[bot]
490697ec55 feat(deps): bump github.com/onsi/gomega from 1.39.0 to 1.39.1 (#1068)
Bumps [github.com/onsi/gomega](https://github.com/onsi/gomega) from 1.39.0 to 1.39.1.
- [Release notes](https://github.com/onsi/gomega/releases)
- [Changelog](https://github.com/onsi/gomega/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/gomega/compare/v1.39.0...v1.39.1)

---
updated-dependencies:
- dependency-name: github.com/onsi/gomega
  dependency-version: 1.39.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-31 17:32:42 +01:00
Alejo Morell
3b44dfc210 docs: updating cert-manager and metallb charts (#1064)
* docs: Modified bitnami cert-manager reference because images do not exist anymore

* docs: update MetalLB version

* docs: added podman specific command for retrieving GW_IP
2026-01-25 12:38:03 +01:00
dependabot[bot]
faf26b2254 feat(deps): bump github.com/onsi/ginkgo/v2 from 2.27.4 to 2.27.5 (#1060)
Bumps [github.com/onsi/ginkgo/v2](https://github.com/onsi/ginkgo) from 2.27.4 to 2.27.5.
- [Release notes](https://github.com/onsi/ginkgo/releases)
- [Changelog](https://github.com/onsi/ginkgo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/ginkgo/compare/v2.27.4...v2.27.5)

---
updated-dependencies:
- dependency-name: github.com/onsi/ginkgo/v2
  dependency-version: 2.27.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-14 18:39:07 +01:00
Parth Yadav
87242ff005 feat: extend Gateway API support to Konnectivity addons (#1054)
This change extends Gateway API support to Konnectivity addons.
When `spec.controlPlane.gateway` is configured and Konnectivity addon is
enabled, Kamaji automatically creates two TLSRoutes:
1. A Control plane TLSRoute (port 6443, sectionName "kube-apiserver")
2. A Konnectivity TLSRoute (port 8132, sectionName "konnectivity-server")

Both routes use the hostname specified in `gateway.hostname` and reference
the same Gateway resource via `parentRefs`, with `port` and `sectionName`
set automatically by Kamaji.

This patch also adds CEL validation to prevent users from specifying
`port` or `sectionName` in Gateway `parentRefs`, as these fields are now
managed automatically by Kamaji.

Signed-off-by: Parth Yadav <parth@coredge.io>
2026-01-11 11:31:24 +01:00
dependabot[bot]
b6b4888177 feat(deps): bump github.com/onsi/gomega from 1.38.3 to 1.39.0 (#1056)
Bumps [github.com/onsi/gomega](https://github.com/onsi/gomega) from 1.38.3 to 1.39.0.
- [Release notes](https://github.com/onsi/gomega/releases)
- [Changelog](https://github.com/onsi/gomega/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/gomega/compare/v1.38.3...v1.39.0)

---
updated-dependencies:
- dependency-name: github.com/onsi/gomega
  dependency-version: 1.39.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-11 11:27:38 +01:00
dependabot[bot]
bd0c7d354d feat(deps): bump github.com/onsi/ginkgo/v2 from 2.27.3 to 2.27.4 (#1055)
Bumps [github.com/onsi/ginkgo/v2](https://github.com/onsi/ginkgo) from 2.27.3 to 2.27.4.
- [Release notes](https://github.com/onsi/ginkgo/releases)
- [Changelog](https://github.com/onsi/ginkgo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/ginkgo/compare/v2.27.3...v2.27.4)

---
updated-dependencies:
- dependency-name: github.com/onsi/ginkgo/v2
  dependency-version: 2.27.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-11 11:22:33 +01:00
Dario Tranchitella
e1c6aa8459 feat: kubelet configuration json patching (#1052)
* feat: kubelet configuration json patching

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

* chore(helm): kubelet configuration json patching

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

* docs(api): kubelet configuration json patching

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

* chore(samples): kubelet configuration json patching

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

---------

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>
2026-01-07 12:36:52 +01:00
Michał Matuszak
11c315289c feat: Make replicas optional for Konnectivity Agent mode Deployment (#1041) 2026-01-04 19:26:57 +01:00
Syed Azeez
0428024946 fix(metrics): resolve workqueue metrics initialization conflict (#1044)
Signed-off-by: Azeez Syed <syedazeez337@gmail.com>
2026-01-04 19:24:43 +01:00
Syed Azeez
f55df56eac fix(soot): add unique controller names to prevent metric conflicts (#1043)
Signed-off-by: Azeez Syed <syedazeez337@gmail.com>
2026-01-04 19:24:10 +01:00
daseul cho
88e08fa0ec fix(soot): correct TenantControlPlane name in trigger events (#1040) 2025-12-22 07:03:17 +01:00
Dario Tranchitella
01e07ab411 feat: kubernetes 1.35 support (#1038)
* feat: supporting k8s v1.35

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

* feat: upgrading deployment also in sleeping mode

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

* feat(deps): bumping ko to v0.18.1

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

* feat(deps): bumping controller-gen to v0.20.0

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

* chore(crds): aligning to k8s v1.35

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

* docs: alinging to k8s v1.35

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

* test: upgrading to k8s 1.35

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

* feat(helm): updating artifact hub changes

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

---------

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>
2025-12-20 12:06:48 +01:00
dependabot[bot]
e0d6865df3 feat(deps): bump github.com/nats-io/nats.go from 1.47.0 to 1.48.0 (#1036)
Bumps [github.com/nats-io/nats.go](https://github.com/nats-io/nats.go) from 1.47.0 to 1.48.0.
- [Release notes](https://github.com/nats-io/nats.go/releases)
- [Commits](https://github.com/nats-io/nats.go/compare/v1.47.0...v1.48.0)

---
updated-dependencies:
- dependency-name: github.com/nats-io/nats.go
  dependency-version: 1.48.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-19 16:42:21 +01:00
dependabot[bot]
57e3e12f09 feat(deps): bump the etcd group with 2 updates (#1035)
Bumps the etcd group with 2 updates: [go.etcd.io/etcd/api/v3](https://github.com/etcd-io/etcd) and [go.etcd.io/etcd/client/v3](https://github.com/etcd-io/etcd).


Updates `go.etcd.io/etcd/api/v3` from 3.6.6 to 3.6.7
- [Release notes](https://github.com/etcd-io/etcd/releases)
- [Commits](https://github.com/etcd-io/etcd/compare/v3.6.6...v3.6.7)

Updates `go.etcd.io/etcd/client/v3` from 3.6.6 to 3.6.7
- [Release notes](https://github.com/etcd-io/etcd/releases)
- [Commits](https://github.com/etcd-io/etcd/compare/v3.6.6...v3.6.7)

---
updated-dependencies:
- dependency-name: go.etcd.io/etcd/api/v3
  dependency-version: 3.6.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: etcd
- dependency-name: go.etcd.io/etcd/client/v3
  dependency-version: 3.6.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: etcd
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-18 07:08:07 +01:00
Léonard Suslian
d3fb03a752 feat: add support for multiple Datastores (#961)
* feat: add support for multiple Datastores

* docs: add guide for datastore overrides

* feat(datastore): add e2e test for dataStoreOverrides

* ci: reclaim disk space from runner to fix flaky tests
2025-12-12 12:10:02 +01:00
dependabot[bot]
eb86fec050 feat(deps): bump github.com/onsi/gomega from 1.38.2 to 1.38.3 (#1032)
Bumps [github.com/onsi/gomega](https://github.com/onsi/gomega) from 1.38.2 to 1.38.3.
- [Release notes](https://github.com/onsi/gomega/releases)
- [Changelog](https://github.com/onsi/gomega/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/gomega/compare/v1.38.2...v1.38.3)

---
updated-dependencies:
- dependency-name: github.com/onsi/gomega
  dependency-version: 1.38.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-12 12:03:28 +01:00
dependabot[bot]
35c83fbd4d feat(deps): bump k8s.io/kubernetes in the k8s group (#1034)
Bumps the k8s group with 1 update: [k8s.io/kubernetes](https://github.com/kubernetes/kubernetes).


Updates `k8s.io/kubernetes` from 1.34.2 to 1.34.3
- [Release notes](https://github.com/kubernetes/kubernetes/releases)
- [Commits](https://github.com/kubernetes/kubernetes/compare/v1.34.2...v1.34.3)

---
updated-dependencies:
- dependency-name: k8s.io/kubernetes
  dependency-version: 1.34.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: k8s
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-12 12:03:16 +01:00
dependabot[bot]
4ad4721965 feat(deps): bump github.com/onsi/ginkgo/v2 from 2.27.2 to 2.27.3 (#1033)
Bumps [github.com/onsi/ginkgo/v2](https://github.com/onsi/ginkgo) from 2.27.2 to 2.27.3.
- [Release notes](https://github.com/onsi/ginkgo/releases)
- [Changelog](https://github.com/onsi/ginkgo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/ginkgo/compare/v2.27.2...v2.27.3)

---
updated-dependencies:
- dependency-name: github.com/onsi/ginkgo/v2
  dependency-version: 2.27.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-09 10:04:49 +01:00
dependabot[bot]
f4b6de4c40 feat(deps): bump github.com/spf13/cobra from 1.10.1 to 1.10.2 (#1030)
Bumps [github.com/spf13/cobra](https://github.com/spf13/cobra) from 1.10.1 to 1.10.2.
- [Release notes](https://github.com/spf13/cobra/releases)
- [Commits](https://github.com/spf13/cobra/compare/v1.10.1...v1.10.2)

---
updated-dependencies:
- dependency-name: github.com/spf13/cobra
  dependency-version: 1.10.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-07 16:50:05 +01:00
dependabot[bot]
54e795323e feat(deps): bump sigs.k8s.io/gateway-api from 1.4.0 to 1.4.1 (#1029)
Bumps [sigs.k8s.io/gateway-api](https://github.com/kubernetes-sigs/gateway-api) from 1.4.0 to 1.4.1.
- [Release notes](https://github.com/kubernetes-sigs/gateway-api/releases)
- [Changelog](https://github.com/kubernetes-sigs/gateway-api/blob/main/RELEASE.md)
- [Commits](https://github.com/kubernetes-sigs/gateway-api/compare/v1.4.0...v1.4.1)

---
updated-dependencies:
- dependency-name: sigs.k8s.io/gateway-api
  dependency-version: 1.4.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-06 16:31:13 +01:00
Mateusz Kwiatkowski
61a4c152b3 fix: reduce database privileges for kine user (#860)
* Reduce MySQL privileges for kine user

* Reduce PostgreSQL privileges for kine user
2025-12-05 19:36:03 +01:00
Domonkos Cinke
eface0f792 chore: add Namecheap as an adopter (#1027) 2025-12-02 19:31:07 +01:00
Wojciech Urbański
2316af9731 fix: correct argument order in GrantPrivileges and RevokePrivileges methods (#1028)
Updated the argument order in the GrantPrivileges and RevokePrivileges methods to ensure proper execution of SQL statements.
2025-12-02 19:17:38 +01:00
Alfredo Suarez
880b36e0fa feat: gateway api support (#1000)
* Feat: Gateway Routes Specs, plus resource and status init progress

* Generated content, RBAC and start of e2e

* latest code POC Working but e2e fails

* Use Gateway API v1.2.0

* Remove draft comment

* Use TCPRoute

* Revert the charts folder to reduce noise

* Use the correct controller-gen version

* Rename fields and fix tcp/tls typos

* Rename TLSRouteSpec to GatewayRouteSpec

* Remove last instance of tcproute

* Renaming more fields to match the gateway api naming

* Remove ownership of the gateway

* Revert Ko to 0.14.1 and makefile comments

* service discovery, webhooks, and deadcode removal.

* add conditional check for gateway api resources and mark is as owned!

* removing duplicated code and note for maybe a refactor later

* E2E now works!

* e2e suite modifications to support Gateway API v1alpha2 TLSRoute

* Suggestions commit, naming and other related.

* First pass at the status update

* Rename route to gateway

* Only allow one hostname in gateway

* Update status types

* WIP: testing conditions

* Update status API

* Add tests

* Detect endpoint

* Update manifests

* Remove old code and use proper condition check

* Fix compilation error

* Watch the Gateway resources

* Rename fields

* Add missing port

* Add ingress endpoint to the kubeadm

* Error if access points are empty

* Check the spec and status to delay the creation of the kubeadm

* Use the spec for the hostname

* Update api/v1alpha1/tenantcontrolplane_types.go

Co-authored-by: Dario Tranchitella <dario@tranchitella.eu>

* PR fixes, CEL k8s validations, proper status updates checks

* more context and separation of functions

* resolve all pr comments, with indexer

* merge master - go {sum,mod} updates dependabot

* Feat: Gateway Routes Specs, plus resource and status init progress

* Use Gateway API v1.2.0

* merge master - go {sum,mod} updates dependabot

* sum go mod tidy

* leftover comments

* clean go.sum

* fix: missing generated crds spec

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

* docs: gateway api support

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

* golint comments

* linting and test fix.

* Gateway API resource watching was made conditional to prevent crashes when CRDs are absent, and TLSRoute creation now returns an error when the service isn't ready instead of creating invalid resources with empty rules.

* unit test was incorrect after all the fixes we did, gracefull errors are not expected due to conditional adds

* fix(conditional-indexer): Gateway Indexer should also be conditional

* fix(conditional-indexer): Gateway Indexer should also be conditional

---------

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>
Co-authored-by: Hadrien Kohl <hadrien.kohl@gmail.com>
Co-authored-by: Dario Tranchitella <dario@tranchitella.eu>
2025-11-26 10:34:09 +01:00
dependabot[bot]
ac7da57454 chore(ci): bump actions/checkout from 5 to 6 (#1016)
Bumps [actions/checkout](https://github.com/actions/checkout) from 5 to 6.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v5...v6)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-11-23 11:21:10 +01:00
Matthieu ROBIN
20cc50b748 chore(adopters): add Hikube as a new vendor 2025-11-23 11:20:54 +01:00
Dario Tranchitella
4956790e2b feat: loose control over k8s patch version (#1014)
* feat: loose control over k8s patch version

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

* chore(ci): fixing no space left on device

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

---------

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>
2025-11-20 10:18:37 +01:00
Dario Tranchitella
9069c9be47 docs: cluster autoscaler provisioning request (#1015)
* docs: refactoring cluster autoscaler docs

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

* fix(docs): missing required cluster autoscaler annotations

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

* docs: cluster autoscaler provisioningrequest support

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

* docs: updating to latest capi cp provider spec

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>

---------

Signed-off-by: Dario Tranchitella <dario@tranchitella.eu>
2025-11-19 19:55:49 +01:00
108 changed files with 7912 additions and 589 deletions

View File

@@ -11,7 +11,7 @@ jobs:
name: integration
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v6
- uses: actions/setup-go@v6
with:
go-version-file: go.mod
@@ -20,7 +20,7 @@ jobs:
name: lint
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v6
- uses: actions/setup-go@v6
with:
go-version-file: go.mod
@@ -36,7 +36,7 @@ jobs:
name: diff
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v6
with:
fetch-depth: 0
- uses: actions/setup-go@v6

View File

@@ -35,16 +35,23 @@ jobs:
name: Kubernetes
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v6
with:
fetch-depth: 0
- uses: actions/setup-go@v6
with:
go-version-file: go.mod
- name: reclaim disk space from runner
run: |
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /opt/hostedtoolcache/CodeQL
- run: |
sudo apt-get update
sudo apt-get install -y golang-cfssl
sudo swapoff -a
sudo modprobe br_netfilter
- name: install required Go tools
run: make kind ko helm ginkgo
- name: cleaning up go mod
run: go clean -modcache
- name: e2e testing
run: make e2e

View File

@@ -11,7 +11,7 @@ jobs:
name: diff
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v6
with:
fetch-depth: 0
- run: make -C charts/kamaji docs
@@ -23,7 +23,7 @@ jobs:
lint:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v6
- uses: azure/setup-helm@v4
with:
version: 3.3.4
@@ -40,7 +40,7 @@ jobs:
needs: [ "lint", "diff" ]
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v6
- name: Publish Helm chart
uses: stefanprodan/helm-gh-pages@master
with:

View File

@@ -18,7 +18,7 @@ jobs:
ko:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v6
with:
fetch-depth: 0
- uses: actions/setup-go@v6

View File

@@ -12,7 +12,7 @@ jobs:
release:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v6
with:
fetch-depth: 0
- name: generating date metadata

View File

@@ -13,8 +13,10 @@ Feel free to open a Pull-Request to get yours listed.
| Vendor | Coredge | 2025 | [link](https://coredge.io/) | Coredge uses Kamaji in its K8saaS offering to save infrastructure costs in its Sovereign Cloud & AI Infrastructure Platform for end-user organisations. |
| Vendor | DCloud | 2024 | [link](https://dcloud.co.id) | DCloud is an Indonesian Cloud Provider using Kamaji to build and offer [Managed Kubernetes Service](https://dcloud.co.id/dkubes.html). |
| Vendor | Dinova | 2025 | [link](https://dinova.one/) | Dinova is an Italian cloud services provider that integrates Kamaji in its datacenters to offer fully managed Kubernetes clusters. |
| Vendor | Hikube | 2024 | [link](https://hikube.cloud/) | Hikube.cloud is a Swiss sovereign cloud platform with triple replication across three Swiss datacenters, offering enterprise-grade infrastructure with full data sovereignty. |
| End-user | KINX | 2024 | [link](https://kinx.net/?lang=en) | KINX is an Internet infrastructure service provider and will use kamaji for its new [Managed Kubernetes Service](https://kinx.net/service/cloud/kubernetes/intro/?lang=en). |
| Vendor | Netalia | 2025 | [link](https://www.netalia.it) | Netalia uses Kamaji for the Italian cloud
| End-user | Namecheap | 2025 | [link](https://www.namecheap.com/) | Namecheap is an ICANN-accredited domain registrar and web hosting company that provides a wide range of internet-related services and uses Kamaji for both internal and external services. |
| Vendor | Netalia | 2025 | [link](https://www.netalia.it) | Netalia uses Kamaji for the Italian cloud
| Vendor | Netsons | 2023 | [link](https://www.netsons.com) | Netsons is an Italian hosting and cloud provider and uses Kamaji in its [Managed Kubernetes](https://www.netsons.com/kubernetes) offering. |
| Vendor | NVIDIA | 2024 | [link](https://github.com/NVIDIA/doca-platform) | DOCA Platform Framework manages provisioning and service orchestration for NVIDIA Bluefield DPUs. |
| R&D | Orange | 2024 | [link](https://gitlab.com/Orange-OpenSource/kanod) | Orange is a French telecommunications company using Kamaji for experimental research purpose, with Kanod research solution. |

View File

@@ -73,7 +73,7 @@ help: ## Display this help.
.PHONY: ko
ko: $(KO) ## Download ko locally if necessary.
$(KO): $(LOCALBIN)
test -s $(LOCALBIN)/ko || GOBIN=$(LOCALBIN) CGO_ENABLED=0 go install -ldflags="-s -w" github.com/google/ko@v0.14.1
test -s $(LOCALBIN)/ko || GOBIN=$(LOCALBIN) CGO_ENABLED=0 go install -ldflags="-s -w" github.com/google/ko@v0.18.1
.PHONY: yq
yq: $(YQ) ## Download yq locally if necessary.
@@ -98,7 +98,7 @@ $(KIND): $(LOCALBIN)
.PHONY: controller-gen
controller-gen: $(CONTROLLER_GEN) ## Download controller-gen locally if necessary.
$(CONTROLLER_GEN): $(LOCALBIN)
test -s $(LOCALBIN)/controller-gen || GOBIN=$(LOCALBIN) CGO_ENABLED=0 go install -ldflags="-s -w" sigs.k8s.io/controller-tools/cmd/controller-gen@v0.16.1
test -s $(LOCALBIN)/controller-gen || GOBIN=$(LOCALBIN) CGO_ENABLED=0 go install -ldflags="-s -w" sigs.k8s.io/controller-tools/cmd/controller-gen@v0.20.0
.PHONY: golangci-lint
golangci-lint: $(GOLANGCI_LINT) ## Download golangci-lint locally if necessary.
@@ -177,7 +177,7 @@ datastore-postgres:
$(MAKE) NAME=gold _datastore-postgres
_datastore-etcd:
$(HELM) upgrade --install etcd-$(NAME) clastix/kamaji-etcd --create-namespace -n etcd-system --set datastore.enabled=true --set fullnameOverride=etcd-$(NAME)
$(HELM) upgrade --install etcd-$(NAME) clastix/kamaji-etcd --create-namespace -n $(NAMESPACE) --set datastore.enabled=true --set fullnameOverride=etcd-$(NAME) $(EXTRA_ARGS)
_datastore-nats:
$(MAKE) NAME=$(NAME) NAMESPACE=nats-system -C deploy/kine/nats nats
@@ -186,9 +186,11 @@ _datastore-nats:
datastore-etcd: helm
$(HELM) repo add clastix https://clastix.github.io/charts
$(HELM) repo update
$(MAKE) NAME=bronze _datastore-etcd
$(MAKE) NAME=silver _datastore-etcd
$(MAKE) NAME=gold _datastore-etcd
$(MAKE) NAME=bronze NAMESPACE=etcd-system _datastore-etcd
$(MAKE) NAME=silver NAMESPACE=etcd-system _datastore-etcd
$(MAKE) NAME=gold NAMESPACE=etcd-system _datastore-etcd
$(MAKE) NAME=primary NAMESPACE=kamaji-system EXTRA_ARGS='--set certManager.enabled=true --set certManager.issuerRef.kind=Issuer --set certManager.issuerRef.name=kamaji-selfsigned-issuer --set selfSignedCertificates.enabled=false' _datastore-etcd
$(MAKE) NAME=secondary NAMESPACE=kamaji-system EXTRA_ARGS='--set certManager.enabled=true --set certManager.ca.create=false --set certManager.ca.nameOverride=etcd-primary-ca --set certManager.issuerRef.kind=Issuer --set certManager.issuerRef.name=kamaji-selfsigned-issuer --set selfSignedCertificates.enabled=false' _datastore-etcd
datastore-nats: helm
$(HELM) repo add nats https://nats-io.github.io/k8s/helm/charts/
@@ -240,6 +242,16 @@ cert-manager:
$(HELM) repo add jetstack https://charts.jetstack.io
$(HELM) upgrade --install cert-manager jetstack/cert-manager --namespace certmanager-system --create-namespace --set "installCRDs=true"
gateway-api:
kubectl apply --server-side -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.4.0/standard-install.yaml
# Required for the TLSRoutes. Experimentals.
kubectl apply --server-side -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.4.0/experimental-install.yaml
kubectl wait --for=condition=Established crd/gateways.gateway.networking.k8s.io --timeout=60s
envoy-gateway: gateway-api helm ## Install Envoy Gateway for Gateway API tests.
$(HELM) upgrade --install eg oci://docker.io/envoyproxy/gateway-helm --version v1.6.1 -n envoy-gateway-system --create-namespace
kubectl wait --timeout=5m -n envoy-gateway-system deployment/envoy-gateway --for=condition=Available
load: kind
$(KIND) load docker-image --name kamaji ${CONTAINER_REPOSITORY}:${VERSION}
@@ -249,8 +261,11 @@ load: kind
env: kind
$(KIND) create cluster --name kamaji
cleanup: kind
$(KIND) delete cluster --name kamaji
.PHONY: e2e
e2e: env build load helm ginkgo cert-manager ## Create a KinD cluster, install Kamaji on it and run the test suite.
e2e: env build load helm ginkgo cert-manager gateway-api envoy-gateway ## Create a KinD cluster, install Kamaji on it and run the test suite.
$(HELM) upgrade --debug --install kamaji-crds ./charts/kamaji-crds --create-namespace --namespace kamaji-system
$(HELM) repo add clastix https://clastix.github.io/charts
$(HELM) dependency build ./charts/kamaji

View File

@@ -90,6 +90,9 @@ type SecretReference struct {
// DataStoreStatus defines the observed state of DataStore.
type DataStoreStatus struct {
// ObservedGeneration represents the .metadata.generation that was last reconciled.
// +optional
ObservedGeneration int64 `json:"observedGeneration,omitempty"`
// List of the Tenant Control Planes, namespaced named, using this data store.
UsedBy []string `json:"usedBy,omitempty"`
}

View File

@@ -4,7 +4,6 @@
// Package v1alpha1 contains API Schema definitions for the kamaji v1alpha1 API group
// +kubebuilder:object:generate=true
// +groupName=kamaji.clastix.io
//nolint
package v1alpha1
import (

View File

@@ -0,0 +1,47 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package v1alpha1
import (
"context"
"fmt"
controllerruntime "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
gatewayv1 "sigs.k8s.io/gateway-api/apis/v1"
)
const (
GatewayListenerNameKey = "spec.listeners.name"
)
type GatewayListener struct{}
func (g *GatewayListener) Object() client.Object {
return &gatewayv1.Gateway{}
}
func (g *GatewayListener) Field() string {
return GatewayListenerNameKey
}
func (g *GatewayListener) ExtractValue() client.IndexerFunc {
return func(object client.Object) []string {
gateway := object.(*gatewayv1.Gateway) //nolint:forcetypeassert
listenerNames := make([]string, 0, len(gateway.Spec.Listeners))
for _, listener := range gateway.Spec.Listeners {
// Create a composite key: namespace/gatewayName/listenerName
// This allows us to look up gateways by listener name while ensuring uniqueness
key := fmt.Sprintf("%s/%s/%s", gateway.Namespace, gateway.Name, listener.Name)
listenerNames = append(listenerNames, key)
}
return listenerNames
}
}
func (g *GatewayListener) SetupWithManager(ctx context.Context, mgr controllerruntime.Manager) error {
return mgr.GetFieldIndexer().IndexField(ctx, g.Object(), g.Field(), g.ExtractValue())
}

View File

@@ -66,6 +66,9 @@ type KubeconfigGeneratorStatusError struct {
// KubeconfigGeneratorStatus defines the observed state of KubeconfigGenerator.
type KubeconfigGeneratorStatus struct {
// ObservedGeneration represents the .metadata.generation that was last reconciled.
// +optional
ObservedGeneration int64 `json:"observedGeneration,omitempty"`
// Resources is the sum of targeted TenantControlPlane objects.
//+kubebuilder:default=0
Resources int `json:"resources"`

View File

@@ -10,7 +10,6 @@ import (
"strconv"
"strings"
"github.com/pkg/errors"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/types"
"sigs.k8s.io/controller-runtime/pkg/client"
@@ -27,12 +26,12 @@ func (in *TenantControlPlane) AssignedControlPlaneAddress() (string, int32, erro
address, portString, err := net.SplitHostPort(in.Status.ControlPlaneEndpoint)
if err != nil {
return "", 0, errors.Wrap(err, "cannot split host port from Tenant Control Plane endpoint")
return "", 0, fmt.Errorf("cannot split host port from Tenant Control Plane endpoint: %w", err)
}
port, err := strconv.Atoi(portString)
if err != nil {
return "", 0, errors.Wrap(err, "cannot convert Tenant Control Plane port from endpoint")
return "", 0, fmt.Errorf("cannot convert Tenant Control Plane port from endpoint: %w", err)
}
return address, int32(port), nil
@@ -47,7 +46,7 @@ func (in *TenantControlPlane) DeclaredControlPlaneAddress(ctx context.Context, c
svc := &corev1.Service{}
err := client.Get(ctx, types.NamespacedName{Namespace: in.GetNamespace(), Name: in.GetName()}, svc)
if err != nil {
return "", errors.Wrap(err, "cannot retrieve Service for the TenantControlPlane")
return "", fmt.Errorf("cannot retrieve Service for the TenantControlPlane: %w", err)
}
switch {

View File

@@ -0,0 +1,83 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package v1alpha1
import (
apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
)
type JSONPatches []JSONPatch
type JSONPatch struct {
// Op is the RFC 6902 JSON Patch operation.
//+kubebuilder:validation:Enum=add;remove;replace;move;copy;test
Op string `json:"op"`
// Path specifies the target location in the JSON document. Use "/" to separate keys; "-" for appending to arrays.
Path string `json:"path"`
// From specifies the source location for move or copy operations.
From string `json:"from,omitempty"`
// Value is the operation value to be used when Op is add, replace, test.
Value *apiextensionsv1.JSON `json:"value,omitempty"`
}
func (p JSONPatches) ToJSON() ([]byte, error) {
if len(p) == 0 {
return []byte("[]"), nil
}
buf := make([]byte, 0, 256)
buf = append(buf, '[')
for i, patch := range p {
if i > 0 {
buf = append(buf, ',')
}
buf = append(buf, '{')
buf = append(buf, `"op":"`...)
buf = appendEscapedString(buf, patch.Op)
buf = append(buf, '"')
buf = append(buf, `,"path":"`...)
buf = appendEscapedString(buf, patch.Path)
buf = append(buf, '"')
if patch.From != "" {
buf = append(buf, `,"from":"`...)
buf = appendEscapedString(buf, patch.From)
buf = append(buf, '"')
}
if patch.Value != nil {
buf = append(buf, `,"value":`...)
buf = append(buf, patch.Value.Raw...)
}
buf = append(buf, '}')
}
buf = append(buf, ']')
return buf, nil
}
func appendEscapedString(dst []byte, s string) []byte {
for i := range s {
switch s[i] {
case '\\', '"':
dst = append(dst, '\\', s[i])
case '\n':
dst = append(dst, '\\', 'n')
case '\r':
dst = append(dst, '\\', 'r')
case '\t':
dst = append(dst, '\\', 't')
default:
dst = append(dst, s[i])
}
}
return dst
}

View File

@@ -8,6 +8,7 @@ import (
corev1 "k8s.io/api/core/v1"
networkingv1 "k8s.io/api/networking/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
gatewayv1 "sigs.k8s.io/gateway-api/apis/v1"
)
// APIServerCertificatesStatus defines the observed state of ETCD Certificate for API server.
@@ -138,6 +139,7 @@ type KonnectivityStatus struct {
ClusterRoleBinding ExternalKubernetesObjectStatus `json:"clusterrolebinding,omitempty"`
Agent KonnectivityAgentStatus `json:"agent,omitempty"`
Service KubernetesServiceStatus `json:"service,omitempty"`
Gateway *KubernetesGatewayStatus `json:"gateway,omitempty"`
}
type KonnectivityConfigMap struct {
@@ -160,6 +162,9 @@ type AddonsStatus struct {
// TenantControlPlaneStatus defines the observed state of TenantControlPlane.
type TenantControlPlaneStatus struct {
// ObservedGeneration represents the .metadata.generation that was last reconciled.
// +optional
ObservedGeneration int64 `json:"observedGeneration,omitempty"`
// Storage Status contains information about Kubernetes storage system
Storage StorageStatus `json:"storage,omitempty"`
// Certificates contains information about the different certificates
@@ -187,6 +192,7 @@ type KubernetesStatus struct {
Deployment KubernetesDeploymentStatus `json:"deployment,omitempty"`
Service KubernetesServiceStatus `json:"service,omitempty"`
Ingress *KubernetesIngressStatus `json:"ingress,omitempty"`
Gateway *KubernetesGatewayStatus `json:"gateway,omitempty"`
}
// +kubebuilder:validation:Enum=Unknown;Provisioning;CertificateAuthorityRotating;Upgrading;Migrating;Ready;NotReady;Sleeping;WriteLimited
@@ -244,3 +250,25 @@ type KubernetesIngressStatus struct {
// The namespace which the Ingress for the given cluster is deployed.
Namespace string `json:"namespace"`
}
type GatewayAccessPoint struct {
Type *gatewayv1.AddressType `json:"type"`
Value string `json:"value"`
Port int32 `json:"port"`
URLs []string `json:"urls,omitempty"`
}
// +k8s:deepcopy-gen=false
type RouteStatus = gatewayv1.RouteStatus
// KubernetesGatewayStatus defines the status for the Tenant Control Plane Gateway in the management cluster.
type KubernetesGatewayStatus struct {
// The TLSRoute status as resported by the gateway controllers.
RouteStatus `json:",inline"`
// Reference to the route created for this tenant.
RouteRef corev1.LocalObjectReference `json:"routeRef,omitempty"`
// A list of valid access points that the route exposes.
AccessPoints []GatewayAccessPoint `json:"accessPoints,omitempty"`
}

View File

@@ -8,6 +8,7 @@ import (
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/util/intstr"
gatewayv1 "sigs.k8s.io/gateway-api/apis/v1"
)
// NetworkProfileSpec defines the desired state of NetworkProfile.
@@ -67,6 +68,12 @@ const (
)
type KubeletSpec struct {
// ConfigurationJSONPatches contains the RFC 6902 JSON patches to customise the kubeadm generate configuration,
// useful to customise and mangling the configuration according to your needs;
// e.g.: configuring the cgroup driver used by Kubelet is possible via the following patch:
//
// [{"op": "replace", "path": "/cgroupDriver", "value": "systemd"}]
ConfigurationJSONPatches JSONPatches `json:"configurationJSONPatches,omitempty"`
// Ordered list of the preferred NodeAddressTypes to use for kubelet connections.
// Default to InternalIP, ExternalIP, Hostname.
//+kubebuilder:default={"InternalIP","ExternalIP","Hostname"}
@@ -75,6 +82,8 @@ type KubeletSpec struct {
PreferredAddressTypes []KubeletPreferredAddressType `json:"preferredAddressTypes,omitempty"`
// CGroupFS defines the cgroup driver for Kubelet
// https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/configure-cgroup-driver/
//
// Deprecated: use ConfigurationJSONPatches.
CGroupFS CGroupDriver `json:"cgroupfs,omitempty"`
}
@@ -124,6 +133,7 @@ type AdditionalMetadata struct {
// ControlPlane defines how the Tenant Control Plane Kubernetes resources must be created in the Admin Cluster,
// such as the number of Pod replicas, the Service resource, or the Ingress.
// +kubebuilder:validation:XValidation:rule="!(has(self.ingress) && has(self.gateway))",message="using both ingress and gateway is not supported"
type ControlPlane struct {
// Defining the options for the deployed Tenant Control Plane as Deployment resource.
Deployment DeploymentSpec `json:"deployment,omitempty"`
@@ -131,6 +141,8 @@ type ControlPlane struct {
Service ServiceSpec `json:"service"`
// Defining the options for an Optional Ingress which will expose API Server of the Tenant Control Plane
Ingress *IngressSpec `json:"ingress,omitempty"`
// Defining the options for an Optional Gateway which will expose API Server of the Tenant Control Plane
Gateway *GatewaySpec `json:"gateway,omitempty"`
}
// IngressSpec defines the options for the ingress which will expose API Server of the Tenant Control Plane.
@@ -142,6 +154,17 @@ type IngressSpec struct {
Hostname string `json:"hostname,omitempty"`
}
// GatewaySpec defines the options for the Gateway which will expose API Server of the Tenant Control Plane.
// +kubebuilder:validation:XValidation:rule="!has(self.parentRefs) || size(self.parentRefs) == 0 || self.parentRefs.all(ref, !has(ref.port) && !has(ref.sectionName))",message="parentRefs must not specify port or sectionName, these are set automatically by Kamaji"
type GatewaySpec struct {
// AdditionalMetadata to add Labels and Annotations support.
AdditionalMetadata AdditionalMetadata `json:"additionalMetadata,omitempty"`
// GatewayParentRefs is the class of the Gateway resource to use.
GatewayParentRefs []gatewayv1.ParentReference `json:"parentRefs,omitempty"`
// Hostname is an optional field which will be used as a route hostname.
Hostname gatewayv1.Hostname `json:"hostname,omitempty"`
}
type ControlPlaneComponentsResources struct {
APIServer *corev1.ResourceRequirements `json:"apiServer,omitempty"`
ControllerManager *corev1.ResourceRequirements `json:"controllerManager,omitempty"`
@@ -275,7 +298,7 @@ var (
KonnectivityAgentModeDeployment KonnectivityAgentMode = "Deployment"
)
//+kubebuilder:validation:XValidation:rule="!(self.mode == 'DaemonSet' && has(self.replicas) && self.replicas != 0) && !(self.mode == 'Deployment' && self.replicas == 0)",message="replicas must be 0 when mode is DaemonSet, and greater than 0 when mode is Deployment"
//+kubebuilder:validation:XValidation:rule="!(self.mode == 'DaemonSet' && has(self.replicas) && self.replicas != 0) && !(self.mode == 'Deployment' && has(self.replicas) && self.replicas == 0)",message="replicas must be 0 (or unset) when mode is DaemonSet, and greater than 0 (or unset) when mode is Deployment"
type KonnectivityAgentSpec struct {
// AgentImage defines the container image for Konnectivity's agent.
@@ -304,7 +327,7 @@ type KonnectivityAgentSpec struct {
// Replicas defines the number of replicas when Mode is Deployment.
// Must be 0 if Mode is DaemonSet.
//+kubebuilder:validation:Optional
Replicas int32 `json:"replicas,omitempty"`
Replicas *int32 `json:"replicas,omitempty"`
}
// KonnectivitySpec defines the spec for Konnectivity.
@@ -341,6 +364,14 @@ func (p *Permissions) HasAnyLimitation() bool {
return false
}
// DataStoreOverride defines which kubernetes resource will be stored in a dedicated datastore.
type DataStoreOverride struct {
// Resource specifies which kubernetes resource to target.
Resource string `json:"resource,omitempty"`
// DataStore specifies the DataStore that should be used to store the Kubernetes data for the given Resource.
DataStore string `json:"dataStore,omitempty"`
}
// TenantControlPlaneSpec defines the desired state of TenantControlPlane.
// +kubebuilder:validation:XValidation:rule="!has(oldSelf.dataStore) || has(self.dataStore)", message="unsetting the dataStore is not supported"
// +kubebuilder:validation:XValidation:rule="!has(oldSelf.dataStoreSchema) || has(self.dataStoreSchema)", message="unsetting the dataStoreSchema is not supported"
@@ -375,8 +406,10 @@ type TenantControlPlaneSpec struct {
// to the user to avoid clashes between different TenantControlPlanes. If not set upon creation, Kamaji will default the
// DataStoreUsername by concatenating the namespace and name of the TenantControlPlane.
// +kubebuilder:validation:XValidation:rule="self == oldSelf",message="changing the dataStoreUsername is not supported"
DataStoreUsername string `json:"dataStoreUsername,omitempty"`
ControlPlane ControlPlane `json:"controlPlane"`
DataStoreUsername string `json:"dataStoreUsername,omitempty"`
// DataStoreOverride defines which kubernetes resources will be stored in dedicated datastores.
DataStoreOverrides []DataStoreOverride `json:"dataStoreOverrides,omitempty"`
ControlPlane ControlPlane `json:"controlPlane"`
// Kubernetes specification for tenant control plane
Kubernetes KubernetesSpec `json:"kubernetes"`
// NetworkProfile specifies how the network is

View File

@@ -8,8 +8,10 @@
package v1alpha1
import (
"k8s.io/api/core/v1"
corev1 "k8s.io/api/core/v1"
"k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
runtime "k8s.io/apimachinery/pkg/runtime"
apisv1 "sigs.k8s.io/gateway-api/apis/v1"
)
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
@@ -83,21 +85,21 @@ func (in *AdditionalVolumeMounts) DeepCopyInto(out *AdditionalVolumeMounts) {
*out = *in
if in.APIServer != nil {
in, out := &in.APIServer, &out.APIServer
*out = make([]v1.VolumeMount, len(*in))
*out = make([]corev1.VolumeMount, len(*in))
for i := range *in {
(*in)[i].DeepCopyInto(&(*out)[i])
}
}
if in.ControllerManager != nil {
in, out := &in.ControllerManager, &out.ControllerManager
*out = make([]v1.VolumeMount, len(*in))
*out = make([]corev1.VolumeMount, len(*in))
for i := range *in {
(*in)[i].DeepCopyInto(&(*out)[i])
}
}
if in.Scheduler != nil {
in, out := &in.Scheduler, &out.Scheduler
*out = make([]v1.VolumeMount, len(*in))
*out = make([]corev1.VolumeMount, len(*in))
for i := range *in {
(*in)[i].DeepCopyInto(&(*out)[i])
}
@@ -360,6 +362,11 @@ func (in *ControlPlane) DeepCopyInto(out *ControlPlane) {
*out = new(IngressSpec)
(*in).DeepCopyInto(*out)
}
if in.Gateway != nil {
in, out := &in.Gateway, &out.Gateway
*out = new(GatewaySpec)
(*in).DeepCopyInto(*out)
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ControlPlane.
@@ -377,22 +384,22 @@ func (in *ControlPlaneComponentsResources) DeepCopyInto(out *ControlPlaneCompone
*out = *in
if in.APIServer != nil {
in, out := &in.APIServer, &out.APIServer
*out = new(v1.ResourceRequirements)
*out = new(corev1.ResourceRequirements)
(*in).DeepCopyInto(*out)
}
if in.ControllerManager != nil {
in, out := &in.ControllerManager, &out.ControllerManager
*out = new(v1.ResourceRequirements)
*out = new(corev1.ResourceRequirements)
(*in).DeepCopyInto(*out)
}
if in.Scheduler != nil {
in, out := &in.Scheduler, &out.Scheduler
*out = new(v1.ResourceRequirements)
*out = new(corev1.ResourceRequirements)
(*in).DeepCopyInto(*out)
}
if in.Kine != nil {
in, out := &in.Kine, &out.Kine
*out = new(v1.ResourceRequirements)
*out = new(corev1.ResourceRequirements)
(*in).DeepCopyInto(*out)
}
}
@@ -532,6 +539,21 @@ func (in *DataStoreList) DeepCopyObject() runtime.Object {
return nil
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *DataStoreOverride) DeepCopyInto(out *DataStoreOverride) {
*out = *in
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataStoreOverride.
func (in *DataStoreOverride) DeepCopy() *DataStoreOverride {
if in == nil {
return nil
}
out := new(DataStoreOverride)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *DataStoreSetupStatus) DeepCopyInto(out *DataStoreSetupStatus) {
*out = *in
@@ -632,19 +654,19 @@ func (in *DeploymentSpec) DeepCopyInto(out *DeploymentSpec) {
in.Strategy.DeepCopyInto(&out.Strategy)
if in.Tolerations != nil {
in, out := &in.Tolerations, &out.Tolerations
*out = make([]v1.Toleration, len(*in))
*out = make([]corev1.Toleration, len(*in))
for i := range *in {
(*in)[i].DeepCopyInto(&(*out)[i])
}
}
if in.Affinity != nil {
in, out := &in.Affinity, &out.Affinity
*out = new(v1.Affinity)
*out = new(corev1.Affinity)
(*in).DeepCopyInto(*out)
}
if in.TopologySpreadConstraints != nil {
in, out := &in.TopologySpreadConstraints, &out.TopologySpreadConstraints
*out = make([]v1.TopologySpreadConstraint, len(*in))
*out = make([]corev1.TopologySpreadConstraint, len(*in))
for i := range *in {
(*in)[i].DeepCopyInto(&(*out)[i])
}
@@ -663,21 +685,21 @@ func (in *DeploymentSpec) DeepCopyInto(out *DeploymentSpec) {
in.PodAdditionalMetadata.DeepCopyInto(&out.PodAdditionalMetadata)
if in.AdditionalInitContainers != nil {
in, out := &in.AdditionalInitContainers, &out.AdditionalInitContainers
*out = make([]v1.Container, len(*in))
*out = make([]corev1.Container, len(*in))
for i := range *in {
(*in)[i].DeepCopyInto(&(*out)[i])
}
}
if in.AdditionalContainers != nil {
in, out := &in.AdditionalContainers, &out.AdditionalContainers
*out = make([]v1.Container, len(*in))
*out = make([]corev1.Container, len(*in))
for i := range *in {
(*in)[i].DeepCopyInto(&(*out)[i])
}
}
if in.AdditionalVolumes != nil {
in, out := &in.AdditionalVolumes, &out.AdditionalVolumes
*out = make([]v1.Volume, len(*in))
*out = make([]corev1.Volume, len(*in))
for i := range *in {
(*in)[i].DeepCopyInto(&(*out)[i])
}
@@ -786,6 +808,69 @@ func (in ExtraArgs) DeepCopy() ExtraArgs {
return *out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *GatewayAccessPoint) DeepCopyInto(out *GatewayAccessPoint) {
*out = *in
if in.Type != nil {
in, out := &in.Type, &out.Type
*out = new(apisv1.AddressType)
**out = **in
}
if in.URLs != nil {
in, out := &in.URLs, &out.URLs
*out = make([]string, len(*in))
copy(*out, *in)
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GatewayAccessPoint.
func (in *GatewayAccessPoint) DeepCopy() *GatewayAccessPoint {
if in == nil {
return nil
}
out := new(GatewayAccessPoint)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *GatewayListener) DeepCopyInto(out *GatewayListener) {
*out = *in
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GatewayListener.
func (in *GatewayListener) DeepCopy() *GatewayListener {
if in == nil {
return nil
}
out := new(GatewayListener)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *GatewaySpec) DeepCopyInto(out *GatewaySpec) {
*out = *in
in.AdditionalMetadata.DeepCopyInto(&out.AdditionalMetadata)
if in.GatewayParentRefs != nil {
in, out := &in.GatewayParentRefs, &out.GatewayParentRefs
*out = make([]apisv1.ParentReference, len(*in))
for i := range *in {
(*in)[i].DeepCopyInto(&(*out)[i])
}
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GatewaySpec.
func (in *GatewaySpec) DeepCopy() *GatewaySpec {
if in == nil {
return nil
}
out := new(GatewaySpec)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *ImageOverrideTrait) DeepCopyInto(out *ImageOverrideTrait) {
*out = *in
@@ -817,12 +902,53 @@ func (in *IngressSpec) DeepCopy() *IngressSpec {
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *JSONPatch) DeepCopyInto(out *JSONPatch) {
*out = *in
if in.Value != nil {
in, out := &in.Value, &out.Value
*out = new(v1.JSON)
(*in).DeepCopyInto(*out)
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new JSONPatch.
func (in *JSONPatch) DeepCopy() *JSONPatch {
if in == nil {
return nil
}
out := new(JSONPatch)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in JSONPatches) DeepCopyInto(out *JSONPatches) {
{
in := &in
*out = make(JSONPatches, len(*in))
for i := range *in {
(*in)[i].DeepCopyInto(&(*out)[i])
}
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new JSONPatches.
func (in JSONPatches) DeepCopy() JSONPatches {
if in == nil {
return nil
}
out := new(JSONPatches)
in.DeepCopyInto(out)
return *out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *KonnectivityAgentSpec) DeepCopyInto(out *KonnectivityAgentSpec) {
*out = *in
if in.Tolerations != nil {
in, out := &in.Tolerations, &out.Tolerations
*out = make([]v1.Toleration, len(*in))
*out = make([]corev1.Toleration, len(*in))
for i := range *in {
(*in)[i].DeepCopyInto(&(*out)[i])
}
@@ -832,6 +958,11 @@ func (in *KonnectivityAgentSpec) DeepCopyInto(out *KonnectivityAgentSpec) {
*out = make(ExtraArgs, len(*in))
copy(*out, *in)
}
if in.Replicas != nil {
in, out := &in.Replicas, &out.Replicas
*out = new(int32)
**out = **in
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new KonnectivityAgentSpec.
@@ -880,7 +1011,7 @@ func (in *KonnectivityServerSpec) DeepCopyInto(out *KonnectivityServerSpec) {
*out = *in
if in.Resources != nil {
in, out := &in.Resources, &out.Resources
*out = new(v1.ResourceRequirements)
*out = new(corev1.ResourceRequirements)
(*in).DeepCopyInto(*out)
}
if in.ExtraArgs != nil {
@@ -927,6 +1058,11 @@ func (in *KonnectivityStatus) DeepCopyInto(out *KonnectivityStatus) {
in.ClusterRoleBinding.DeepCopyInto(&out.ClusterRoleBinding)
in.Agent.DeepCopyInto(&out.Agent)
in.Service.DeepCopyInto(&out.Service)
if in.Gateway != nil {
in, out := &in.Gateway, &out.Gateway
*out = new(KubernetesGatewayStatus)
(*in).DeepCopyInto(*out)
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new KonnectivityStatus.
@@ -1141,6 +1277,13 @@ func (in *KubeconfigsStatus) DeepCopy() *KubeconfigsStatus {
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *KubeletSpec) DeepCopyInto(out *KubeletSpec) {
*out = *in
if in.ConfigurationJSONPatches != nil {
in, out := &in.ConfigurationJSONPatches, &out.ConfigurationJSONPatches
*out = make(JSONPatches, len(*in))
for i := range *in {
(*in)[i].DeepCopyInto(&(*out)[i])
}
}
if in.PreferredAddressTypes != nil {
in, out := &in.PreferredAddressTypes, &out.PreferredAddressTypes
*out = make([]KubeletPreferredAddressType, len(*in))
@@ -1175,6 +1318,30 @@ func (in *KubernetesDeploymentStatus) DeepCopy() *KubernetesDeploymentStatus {
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *KubernetesGatewayStatus) DeepCopyInto(out *KubernetesGatewayStatus) {
*out = *in
in.RouteStatus.DeepCopyInto(&out.RouteStatus)
out.RouteRef = in.RouteRef
if in.AccessPoints != nil {
in, out := &in.AccessPoints, &out.AccessPoints
*out = make([]GatewayAccessPoint, len(*in))
for i := range *in {
(*in)[i].DeepCopyInto(&(*out)[i])
}
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new KubernetesGatewayStatus.
func (in *KubernetesGatewayStatus) DeepCopy() *KubernetesGatewayStatus {
if in == nil {
return nil
}
out := new(KubernetesGatewayStatus)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *KubernetesIngressStatus) DeepCopyInto(out *KubernetesIngressStatus) {
*out = *in
@@ -1239,6 +1406,11 @@ func (in *KubernetesStatus) DeepCopyInto(out *KubernetesStatus) {
*out = new(KubernetesIngressStatus)
(*in).DeepCopyInto(*out)
}
if in.Gateway != nil {
in, out := &in.Gateway, &out.Gateway
*out = new(KubernetesGatewayStatus)
(*in).DeepCopyInto(*out)
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new KubernetesStatus.
@@ -1493,6 +1665,11 @@ func (in *TenantControlPlaneList) DeepCopyObject() runtime.Object {
func (in *TenantControlPlaneSpec) DeepCopyInto(out *TenantControlPlaneSpec) {
*out = *in
out.WritePermissions = in.WritePermissions
if in.DataStoreOverrides != nil {
in, out := &in.DataStoreOverrides, &out.DataStoreOverrides
*out = make([]DataStoreOverride, len(*in))
copy(*out, *in)
}
in.ControlPlane.DeepCopyInto(&out.ControlPlane)
in.Kubernetes.DeepCopyInto(&out.Kubernetes)
in.NetworkProfile.DeepCopyInto(&out.NetworkProfile)

View File

@@ -33,5 +33,7 @@ annotations:
- name: support
url: https://clastix.io/support
artifacthub.io/changes: |
- kind: changed
description: Upgrading support to Kubernetes v1.35
- kind: added
description: First commit
description: Supporting multiple Datastore via etcd overrides

View File

@@ -275,6 +275,10 @@ versions:
status:
description: DataStoreStatus defines the observed state of DataStore.
properties:
observedGeneration:
description: ObservedGeneration represents the .metadata.generation that was last reconciled.
format: int64
type: integer
usedBy:
description: List of the Tenant Control Planes, namespaced named, using this data store.
items:

View File

@@ -199,6 +199,10 @@ versions:
- resource
type: object
type: array
observedGeneration:
description: ObservedGeneration represents the .metadata.generation that was last reconciled.
format: int64
type: integer
resources:
default: 0
description: Resources is the sum of targeted TenantControlPlane objects.

View File

@@ -1,3 +1,17 @@
- apiGroups:
- ""
resources:
- configmaps
- secrets
- services
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- ""
resources:
@@ -29,11 +43,19 @@
- list
- watch
- apiGroups:
- ""
- gateway.networking.k8s.io
resources:
- configmaps
- secrets
- services
- gateways
verbs:
- get
- list
- watch
- apiGroups:
- gateway.networking.k8s.io
resources:
- grpcroutes
- httproutes
- tlsroutes
verbs:
- create
- delete

View File

@@ -4,7 +4,7 @@ kind: CustomResourceDefinition
metadata:
annotations:
cert-manager.io/inject-ca-from: kamaji-system/kamaji-serving-cert
controller-gen.kubebuilder.io/version: v0.16.1
controller-gen.kubebuilder.io/version: v0.20.0
name: datastores.kamaji.clastix.io
spec:
group: kamaji.clastix.io
@@ -284,6 +284,10 @@ spec:
status:
description: DataStoreStatus defines the observed state of DataStore.
properties:
observedGeneration:
description: ObservedGeneration represents the .metadata.generation that was last reconciled.
format: int64
type: integer
usedBy:
description: List of the Tenant Control Planes, namespaced named, using this data store.
items:

View File

@@ -3,7 +3,7 @@ kind: CustomResourceDefinition
metadata:
annotations:
cert-manager.io/inject-ca-from: kamaji-system/kamaji-serving-cert
controller-gen.kubebuilder.io/version: v0.16.1
controller-gen.kubebuilder.io/version: v0.20.0
name: kubeconfiggenerators.kamaji.clastix.io
spec:
group: kamaji.clastix.io
@@ -207,6 +207,10 @@ spec:
- resource
type: object
type: array
observedGeneration:
description: ObservedGeneration represents the .metadata.generation that was last reconciled.
format: int64
type: integer
resources:
default: 0
description: Resources is the sum of targeted TenantControlPlane objects.

File diff suppressed because it is too large Load Diff

View File

@@ -17,6 +17,7 @@ import (
"github.com/spf13/cobra"
"github.com/spf13/viper"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/client-go/discovery"
"k8s.io/client-go/rest"
"k8s.io/klog/v2"
ctrl "sigs.k8s.io/controller-runtime"
@@ -34,6 +35,7 @@ import (
"github.com/clastix/kamaji/internal"
"github.com/clastix/kamaji/internal/builders/controlplane"
datastoreutils "github.com/clastix/kamaji/internal/datastore/utils"
"github.com/clastix/kamaji/internal/utilities"
"github.com/clastix/kamaji/internal/webhook"
"github.com/clastix/kamaji/internal/webhook/handlers"
"github.com/clastix/kamaji/internal/webhook/routes"
@@ -146,6 +148,13 @@ func NewCmd(scheme *runtime.Scheme) *cobra.Command {
return err
}
discoveryClient, err := discovery.NewDiscoveryClientForConfig(mgr.GetConfig())
if err != nil {
setupLog.Error(err, "unable to create discovery client")
return err
}
reconciler := &controllers.TenantControlPlaneReconciler{
Client: mgr.GetClient(),
APIReader: mgr.GetAPIReader(),
@@ -163,9 +172,10 @@ func NewCmd(scheme *runtime.Scheme) *cobra.Command {
KamajiService: managerServiceName,
KamajiMigrateImage: migrateJobImage,
MaxConcurrentReconciles: maxConcurrentReconciles,
DiscoveryClient: discoveryClient,
}
if err = reconciler.SetupWithManager(mgr); err != nil {
if err = reconciler.SetupWithManager(ctx, mgr); err != nil {
setupLog.Error(err, "unable to create controller", "controller", "Namespace")
return err
@@ -215,6 +225,15 @@ func NewCmd(scheme *runtime.Scheme) *cobra.Command {
return err
}
// Only requires to look for the core api group.
if utilities.AreGatewayResourcesAvailable(ctx, mgr.GetClient(), discoveryClient) {
if err = (&kamajiv1alpha1.GatewayListener{}).SetupWithManager(ctx, mgr); err != nil {
setupLog.Error(err, "unable to create indexer", "indexer", "GatewayListener")
return err
}
}
err = webhook.Register(mgr, map[routes.Route][]handlers.Handler{
routes.TenantControlPlaneMigrate{}: {
handlers.Freeze{},
@@ -244,6 +263,10 @@ func NewCmd(scheme *runtime.Scheme) *cobra.Command {
},
handlers.TenantControlPlaneServiceCIDR{},
handlers.TenantControlPlaneLoadBalancerSourceRanges{},
handlers.TenantControlPlaneGatewayValidation{
Client: mgr.GetClient(),
DiscoveryClient: discoveryClient,
},
},
routes.TenantControlPlaneTelemetry{}: {
handlers.TenantControlPlaneTelemetry{

View File

@@ -10,6 +10,8 @@ import (
utilruntime "k8s.io/apimachinery/pkg/util/runtime"
clientgoscheme "k8s.io/client-go/kubernetes/scheme"
appsv1 "k8s.io/kubernetes/pkg/apis/apps/v1"
gatewayv1 "sigs.k8s.io/gateway-api/apis/v1"
gatewayv1alpha2 "sigs.k8s.io/gateway-api/apis/v1alpha2"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
)
@@ -22,6 +24,10 @@ func NewCmd(scheme *runtime.Scheme) *cobra.Command {
utilruntime.Must(clientgoscheme.AddToScheme(scheme))
utilruntime.Must(kamajiv1alpha1.AddToScheme(scheme))
utilruntime.Must(appsv1.RegisterDefaults(scheme))
// NOTE: This will succeed even if Gateway API is not installed in the cluster.
// Only registers the go types.
utilruntime.Must(gatewayv1.Install(scheme))
utilruntime.Must(gatewayv1alpha2.Install(scheme))
},
}
}

View File

@@ -4,7 +4,8 @@
package utils
import (
"github.com/pkg/errors"
"fmt"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/rest"
)
@@ -12,12 +13,12 @@ import (
func KubernetesVersion(config *rest.Config) (string, error) {
cs, csErr := kubernetes.NewForConfig(config)
if csErr != nil {
return "", errors.Wrap(csErr, "cannot create kubernetes clientset")
return "", fmt.Errorf("cannot create kubernetes clientset: %w", csErr)
}
sv, svErr := cs.ServerVersion()
if svErr != nil {
return "", errors.Wrap(svErr, "cannot get Kubernetes version")
return "", fmt.Errorf("cannot get Kubernetes version: %w", svErr)
}
return sv.GitVersion, nil

View File

@@ -13,7 +13,15 @@ spec:
kubernetes:
version: "v1.33.0"
kubelet:
cgroupfs: systemd
configurationJSONPatches:
- op: add
path: /featureGates
value:
KubeletCrashLoopBackOffMax: false
KubeletEnsureSecretPulledImages: false
- op: replace
path: /cgroupDriver
value: systemd
networkProfile:
port: 6443
addons:

View File

@@ -0,0 +1,81 @@
# Copyright 2022 Clastix Labs
# SPDX-License-Identifier: Apache-2.0
# This example demonstrates how to configure Gateway API support for a Tenant Control Plane.
#
# Prerequisites:
# 1. Gateway API CRDs must be installed (GatewayClass, Gateway, TLSRoute)
# 2. A Gateway resource must exist with listeners for ports 6443 and 8132
# 3. DNS(or worker nodes hosts entries) must be configured to resolve the hostname to the Gateway's external address
#
# Example GatewayClass and Gateway configuration:
#
# apiVersion: gateway.networking.k8s.io/v1
# kind: GatewayClass
# metadata:
# name: envoy-gw-class
# spec:
# controllerName: gateway.envoyproxy.io/gatewayclass-controller
# ---
# apiVersion: gateway.networking.k8s.io/v1
# kind: Gateway
# metadata:
# name: gateway
# namespace: default
# spec:
# gatewayClassName: envoy-gw-class
# listeners:
# - allowedRoutes:
# kinds:
# - group: gateway.networking.k8s.io
# kind: TLSRoute
# namespaces:
# from: All
# hostname: '*.cluster.dev'
# name: kube-apiserver
# port: 6443
# protocol: TLS
# tls:
# mode: Passthrough
# - allowedRoutes:
# kinds:
# - group: gateway.networking.k8s.io
# kind: TLSRoute
# namespaces:
# from: All
# hostname: '*.cluster.dev'
# name: konnectivity-server
# port: 8132
# protocol: TLS
# tls:
# mode: Passthrough
apiVersion: kamaji.clastix.io/v1alpha1
kind: TenantControlPlane
metadata:
name: demo-tcp-1
spec:
addons:
coreDNS: {}
kubeProxy: {}
konnectivity: {}
dataStore: default
controlPlane:
gateway:
hostname: "c11.cluster.dev" # worker nodes or kubectl clients must be able to resolve this hostname to the Gateway's external address.
parentRefs:
- name: gateway
namespace: default
deployment:
replicas: 1
service:
serviceType: ClusterIP
kubernetes:
version: v1.32.0
kubelet:
cgroupfs: systemd
networkProfile:
port: 6443
certSANs:
- "c11.cluster.dev"

View File

@@ -9,7 +9,6 @@ import (
"fmt"
"time"
"github.com/pkg/errors"
corev1 "k8s.io/api/core/v1"
k8serrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
@@ -168,7 +167,7 @@ func (s *CertificateLifecycle) extractCertificateFromKubeconfig(secret corev1.Se
crt, err := crypto.ParseCertificateBytes(kc.AuthInfos[0].AuthInfo.ClientCertificateData)
if err != nil {
return nil, errors.Wrap(err, "cannot parse kubeconfig certificate bytes")
return nil, fmt.Errorf("cannot parse kubeconfig certificate bytes: %w", err)
}
return crt, nil

View File

@@ -5,8 +5,8 @@ package controllers
import (
"context"
"fmt"
"github.com/pkg/errors"
k8serrors "k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/fields"
k8stypes "k8s.io/apimachinery/pkg/types"
@@ -65,7 +65,7 @@ func (r *DataStore) Reconcile(ctx context.Context, request reconcile.Request) (r
if lErr := r.Client.List(ctx, &tcpList, client.MatchingFieldsSelector{
Selector: fields.OneTermEqualSelector(kamajiv1alpha1.TenantControlPlaneUsedDataStoreKey, ds.GetName()),
}); lErr != nil {
return errors.Wrap(lErr, "cannot retrieve list of the Tenant Control Plane using the following instance")
return fmt.Errorf("cannot retrieve list of the Tenant Control Plane using the following instance: %w", lErr)
}
// Updating the status with the list of Tenant Control Plane using the following Data Source
tcpSets := sets.NewString()
@@ -73,10 +73,11 @@ func (r *DataStore) Reconcile(ctx context.Context, request reconcile.Request) (r
tcpSets.Insert(getNamespacedName(tcp.GetNamespace(), tcp.GetName()).String())
}
ds.Status.ObservedGeneration = ds.Generation
ds.Status.UsedBy = tcpSets.List()
if sErr := r.Client.Status().Update(ctx, &ds); sErr != nil {
return errors.Wrap(sErr, "cannot update the status for the given instance")
return fmt.Errorf("cannot update the status for the given instance: %w", sErr)
}
return nil

View File

@@ -12,7 +12,6 @@ import (
"strings"
"time"
"github.com/pkg/errors"
corev1 "k8s.io/api/core/v1"
apierrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
@@ -88,6 +87,7 @@ func (r *KubeconfigGeneratorReconciler) Reconcile(ctx context.Context, req ctrl.
}
generator.Status = status
generator.Status.ObservedGeneration = generator.Generation
if statusErr := r.Client.Status().Update(ctx, &generator); statusErr != nil {
logger.Error(statusErr, "cannot update resource status")
@@ -103,12 +103,12 @@ func (r *KubeconfigGeneratorReconciler) Reconcile(ctx context.Context, req ctrl.
func (r *KubeconfigGeneratorReconciler) handle(ctx context.Context, generator *kamajiv1alpha1.KubeconfigGenerator) (kamajiv1alpha1.KubeconfigGeneratorStatus, error) {
nsSelector, nsErr := metav1.LabelSelectorAsSelector(&generator.Spec.NamespaceSelector)
if nsErr != nil {
return kamajiv1alpha1.KubeconfigGeneratorStatus{}, errors.Wrap(nsErr, "NamespaceSelector contains an error")
return kamajiv1alpha1.KubeconfigGeneratorStatus{}, fmt.Errorf("NamespaceSelector contains an error: %w", nsErr)
}
var namespaceList corev1.NamespaceList
if err := r.Client.List(ctx, &namespaceList, &client.ListOptions{LabelSelector: nsSelector}); err != nil {
return kamajiv1alpha1.KubeconfigGeneratorStatus{}, errors.Wrap(err, "cannot filter Namespace objects using provided selector")
return kamajiv1alpha1.KubeconfigGeneratorStatus{}, fmt.Errorf("cannot filter Namespace objects using provided selector: %w", err)
}
var targets []kamajiv1alpha1.TenantControlPlane
@@ -116,12 +116,12 @@ func (r *KubeconfigGeneratorReconciler) handle(ctx context.Context, generator *k
for _, ns := range namespaceList.Items {
tcpSelector, tcpErr := metav1.LabelSelectorAsSelector(&generator.Spec.TenantControlPlaneSelector)
if tcpErr != nil {
return kamajiv1alpha1.KubeconfigGeneratorStatus{}, errors.Wrap(tcpErr, "TenantControlPlaneSelector contains an error")
return kamajiv1alpha1.KubeconfigGeneratorStatus{}, fmt.Errorf("TenantControlPlaneSelector contains an error: %w", tcpErr)
}
var tcpList kamajiv1alpha1.TenantControlPlaneList
if err := r.Client.List(ctx, &tcpList, &client.ListOptions{Namespace: ns.GetName(), LabelSelector: tcpSelector}); err != nil {
return kamajiv1alpha1.KubeconfigGeneratorStatus{}, errors.Wrap(err, "cannot filter TenantControlPlane objects using provided selector")
return kamajiv1alpha1.KubeconfigGeneratorStatus{}, fmt.Errorf("cannot filter TenantControlPlane objects using provided selector: %w", err)
}
targets = append(targets, tcpList.Items...)
@@ -290,17 +290,17 @@ func (r *KubeconfigGeneratorReconciler) generate(ctx context.Context, generator
var caSecret corev1.Secret
if caErr := r.Client.Get(ctx, types.NamespacedName{Namespace: tcp.Namespace, Name: tcp.Status.Certificates.CA.SecretName}, &caSecret); caErr != nil {
return errors.Wrap(caErr, "cannot retrieve Certificate Authority")
return fmt.Errorf("cannot retrieve Certificate Authority: %w", caErr)
}
caCert, crtErr := crypto.ParseCertificateBytes(caSecret.Data[kubeadmconstants.CACertName])
if crtErr != nil {
return errors.Wrap(crtErr, "cannot parse Certificate Authority certificate")
return fmt.Errorf("cannot parse Certificate Authority certificate: %w", crtErr)
}
caKey, keyErr := crypto.ParsePrivateKeyBytes(caSecret.Data[kubeadmconstants.CAKeyName])
if keyErr != nil {
return errors.Wrap(keyErr, "cannot parse Certificate Authority key")
return fmt.Errorf("cannot parse Certificate Authority key: %w", keyErr)
}
clientCert, clientKey, err := pkiutil.NewCertAndKey(caCert, caKey, &clientCertConfig)
@@ -312,7 +312,7 @@ func (r *KubeconfigGeneratorReconciler) generate(ctx context.Context, generator
tmpl.AuthInfos[name].AuthInfo.ClientCertificateData = pkiutil.EncodeCertPEM(clientCert)
tmpl.AuthInfos[name].AuthInfo.ClientKeyData, err = keyutil.MarshalPrivateKeyToPEM(clientKey)
if err != nil {
return errors.Wrap(err, "cannot marshal private key to PEM")
return fmt.Errorf("cannot marshal private key to PEM: %w", err)
}
}
@@ -341,7 +341,7 @@ func (r *KubeconfigGeneratorReconciler) generate(ctx context.Context, generator
secret.Data["value"], err = utilities.EncodeToYaml(tmpl)
if err != nil {
return errors.Wrap(err, "cannot encode generated Kubeconfig to YAML")
return fmt.Errorf("cannot encode generated Kubeconfig to YAML: %w", err)
}
if utilities.IsRotationRequested(secret) {
@@ -355,7 +355,7 @@ func (r *KubeconfigGeneratorReconciler) generate(ctx context.Context, generator
return ctrl.SetControllerReference(tcp, secret, r.Client.Scheme())
})
if err != nil {
return errors.Wrap(err, "cannot create or update generated Kubeconfig")
return fmt.Errorf("cannot create or update generated Kubeconfig: %w", err)
}
return nil

View File

@@ -4,12 +4,14 @@
package controllers
import (
"context"
"fmt"
"time"
"github.com/go-logr/logr"
"github.com/google/uuid"
k8stypes "k8s.io/apimachinery/pkg/types"
"k8s.io/client-go/discovery"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
@@ -20,20 +22,24 @@ import (
"github.com/clastix/kamaji/internal/resources"
ds "github.com/clastix/kamaji/internal/resources/datastore"
"github.com/clastix/kamaji/internal/resources/konnectivity"
"github.com/clastix/kamaji/internal/utilities"
)
type GroupResourceBuilderConfiguration struct {
client client.Client
log logr.Logger
tcpReconcilerConfig TenantControlPlaneReconcilerConfig
tenantControlPlane kamajiv1alpha1.TenantControlPlane
ExpirationThreshold time.Duration
Connection datastore.Connection
DataStore kamajiv1alpha1.DataStore
KamajiNamespace string
KamajiServiceAccount string
KamajiService string
KamajiMigrateImage string
client client.Client
log logr.Logger
tcpReconcilerConfig TenantControlPlaneReconcilerConfig
tenantControlPlane kamajiv1alpha1.TenantControlPlane
ExpirationThreshold time.Duration
Connection datastore.Connection
DataStore kamajiv1alpha1.DataStore
DataStoreOverrides []builder.DataStoreOverrides
DataStoreOverriedsConnections map[string]datastore.Connection
KamajiNamespace string
KamajiServiceAccount string
KamajiService string
KamajiMigrateImage string
DiscoveryClient discovery.DiscoveryInterface
}
type GroupDeletableResourceBuilderConfiguration struct {
@@ -48,8 +54,30 @@ type GroupDeletableResourceBuilderConfiguration struct {
// GetResources returns a list of resources that will be used to provide tenant control planes
// Currently there is only a default approach
// TODO: the idea of this function is to become a factory to return the group of resources according to the given configuration.
func GetResources(config GroupResourceBuilderConfiguration) []resources.Resource {
return getDefaultResources(config)
func GetResources(ctx context.Context, config GroupResourceBuilderConfiguration) []resources.Resource {
resources := []resources.Resource{}
resources = append(resources, getDataStoreMigratingResources(config.client, config.KamajiNamespace, config.KamajiMigrateImage, config.KamajiServiceAccount, config.KamajiService)...)
resources = append(resources, getUpgradeResources(config.client)...)
resources = append(resources, getKubernetesServiceResources(config.client)...)
resources = append(resources, getKubeadmConfigResources(config.client, getTmpDirectory(config.tcpReconcilerConfig.TmpBaseDirectory, config.tenantControlPlane), config.DataStore)...)
resources = append(resources, getKubernetesCertificatesResources(config.client, config.tcpReconcilerConfig, config.tenantControlPlane)...)
resources = append(resources, getKubeconfigResources(config.client, config.tcpReconcilerConfig, config.tenantControlPlane)...)
resources = append(resources, getKubernetesStorageResources(config.client, config.Connection, config.DataStore, config.ExpirationThreshold)...)
resources = append(resources, getKubernetesAdditionalStorageResources(config.client, config.DataStoreOverriedsConnections, config.DataStoreOverrides, config.ExpirationThreshold)...)
resources = append(resources, getKonnectivityServerRequirementsResources(config.client, config.ExpirationThreshold)...)
resources = append(resources, getKubernetesDeploymentResources(config.client, config.tcpReconcilerConfig, config.DataStore, config.DataStoreOverrides)...)
resources = append(resources, getKonnectivityServerPatchResources(config.client)...)
resources = append(resources, getDataStoreMigratingCleanup(config.client, config.KamajiNamespace)...)
resources = append(resources, getKubernetesIngressResources(config.client)...)
// Conditionally add Gateway resources
if utilities.AreGatewayResourcesAvailable(ctx, config.client, config.DiscoveryClient) {
resources = append(resources, getKubernetesGatewayResources(config.client)...)
resources = append(resources, getKonnectivityGatewayResources(config.client)...)
}
return resources
}
// GetDeletableResources returns a list of resources that have to be deleted when tenant control planes are deleted
@@ -73,23 +101,6 @@ func GetDeletableResources(tcp *kamajiv1alpha1.TenantControlPlane, config GroupD
return res
}
func getDefaultResources(config GroupResourceBuilderConfiguration) []resources.Resource {
resources := getDataStoreMigratingResources(config.client, config.KamajiNamespace, config.KamajiMigrateImage, config.KamajiServiceAccount, config.KamajiService)
resources = append(resources, getUpgradeResources(config.client)...)
resources = append(resources, getKubernetesServiceResources(config.client)...)
resources = append(resources, getKubeadmConfigResources(config.client, getTmpDirectory(config.tcpReconcilerConfig.TmpBaseDirectory, config.tenantControlPlane), config.DataStore)...)
resources = append(resources, getKubernetesCertificatesResources(config.client, config.tcpReconcilerConfig, config.tenantControlPlane)...)
resources = append(resources, getKubeconfigResources(config.client, config.tcpReconcilerConfig, config.tenantControlPlane)...)
resources = append(resources, getKubernetesStorageResources(config.client, config.Connection, config.DataStore, config.ExpirationThreshold)...)
resources = append(resources, getKonnectivityServerRequirementsResources(config.client, config.ExpirationThreshold)...)
resources = append(resources, getKubernetesDeploymentResources(config.client, config.tcpReconcilerConfig, config.DataStore)...)
resources = append(resources, getKonnectivityServerPatchResources(config.client)...)
resources = append(resources, getDataStoreMigratingCleanup(config.client, config.KamajiNamespace)...)
resources = append(resources, getKubernetesIngressResources(config.client)...)
return resources
}
func getDataStoreMigratingCleanup(c client.Client, kamajiNamespace string) []resources.Resource {
return []resources.Resource{
&ds.Migrate{
@@ -128,6 +139,22 @@ func getKubernetesServiceResources(c client.Client) []resources.Resource {
}
}
func getKubernetesGatewayResources(c client.Client) []resources.Resource {
return []resources.Resource{
&resources.KubernetesGatewayResource{
Client: c,
},
}
}
func getKonnectivityGatewayResources(c client.Client) []resources.Resource {
return []resources.Resource{
&konnectivity.KubernetesKonnectivityGatewayResource{
Client: c,
},
}
}
func getKubeadmConfigResources(c client.Client, tmpDirectory string, dataStore kamajiv1alpha1.DataStore) []resources.Resource {
var endpoints []string
@@ -237,12 +264,42 @@ func getKubernetesStorageResources(c client.Client, dbConnection datastore.Conne
}
}
func getKubernetesDeploymentResources(c client.Client, tcpReconcilerConfig TenantControlPlaneReconcilerConfig, dataStore kamajiv1alpha1.DataStore) []resources.Resource {
func getKubernetesAdditionalStorageResources(c client.Client, dbConnections map[string]datastore.Connection, dataStoreOverrides []builder.DataStoreOverrides, threshold time.Duration) []resources.Resource {
res := make([]resources.Resource, 0, len(dataStoreOverrides))
for _, dso := range dataStoreOverrides {
datastore := dso.DataStore
res = append(res,
&ds.MultiTenancy{
DataStore: datastore,
},
&ds.Config{
Client: c,
ConnString: dbConnections[dso.Resource].GetConnectionString(),
DataStore: datastore,
IsOverride: true,
},
&ds.Setup{
Client: c,
Connection: dbConnections[dso.Resource],
DataStore: datastore,
},
&ds.Certificate{
Client: c,
DataStore: datastore,
CertExpirationThreshold: threshold,
})
}
return res
}
func getKubernetesDeploymentResources(c client.Client, tcpReconcilerConfig TenantControlPlaneReconcilerConfig, dataStore kamajiv1alpha1.DataStore, dataStoreOverrides []builder.DataStoreOverrides) []resources.Resource {
return []resources.Resource{
&resources.KubernetesDeploymentResource{
Client: c,
DataStore: dataStore,
KineContainerImage: tcpReconcilerConfig.KineContainerImage,
DataStoreOverrides: dataStoreOverrides,
},
}
}

View File

@@ -5,9 +5,9 @@ package controllers
import (
"context"
"errors"
"github.com/go-logr/logr"
"github.com/pkg/errors"
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
rbacv1 "k8s.io/api/rbac/v1"
@@ -36,6 +36,7 @@ type CoreDNS struct {
AdminClient client.Client
GetTenantControlPlaneFunc utils.TenantControlPlaneRetrievalFn
TriggerChannel chan event.GenericEvent
ControllerName string
}
func (c *CoreDNS) Reconcile(ctx context.Context, _ reconcile.Request) (reconcile.Result, error) {
@@ -80,6 +81,7 @@ func (c *CoreDNS) Reconcile(ctx context.Context, _ reconcile.Request) (reconcile
func (c *CoreDNS) SetupWithManager(mgr manager.Manager) error {
return controllerruntime.NewControllerManagedBy(mgr).
Named(c.ControllerName).
WithOptions(controller.TypedOptions[reconcile.Request]{SkipNameValidation: ptr.To(true)}).
For(&rbacv1.ClusterRoleBinding{}, builder.WithPredicates(predicate.NewPredicateFuncs(func(object client.Object) bool {
return object.GetName() == kubeadm.CoreDNSClusterRoleBindingName

View File

@@ -3,8 +3,6 @@
package errors
import (
"github.com/pkg/errors"
)
import "errors"
var ErrPausedReconciliation = errors.New("paused reconciliation, no further actions")

View File

@@ -5,9 +5,9 @@ package controllers
import (
"context"
"errors"
"github.com/go-logr/logr"
"github.com/pkg/errors"
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
v1 "k8s.io/api/rbac/v1"
@@ -37,6 +37,7 @@ type KonnectivityAgent struct {
AdminClient client.Client
GetTenantControlPlaneFunc utils.TenantControlPlaneRetrievalFn
TriggerChannel chan event.GenericEvent
ControllerName string
}
func (k *KonnectivityAgent) Reconcile(ctx context.Context, _ reconcile.Request) (reconcile.Result, error) {
@@ -87,6 +88,7 @@ func (k *KonnectivityAgent) Reconcile(ctx context.Context, _ reconcile.Request)
func (k *KonnectivityAgent) SetupWithManager(mgr manager.Manager) error {
return controllerruntime.NewControllerManagedBy(mgr).
Named(k.ControllerName).
WithOptions(controller.TypedOptions[reconcile.Request]{SkipNameValidation: ptr.To(true)}).
For(&appsv1.DaemonSet{}, builder.WithPredicates(predicate.NewPredicateFuncs(func(object client.Object) bool {
return object.GetName() == konnectivity.AgentName && object.GetNamespace() == konnectivity.AgentNamespace

View File

@@ -5,9 +5,9 @@ package controllers
import (
"context"
"errors"
"github.com/go-logr/logr"
"github.com/pkg/errors"
"k8s.io/utils/ptr"
controllerruntime "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/builder"
@@ -29,6 +29,7 @@ type KubeadmPhase struct {
GetTenantControlPlaneFunc utils.TenantControlPlaneRetrievalFn
TriggerChannel chan event.GenericEvent
Phase resources.KubeadmPhaseResource
ControllerName string
logger logr.Logger
}
@@ -75,6 +76,7 @@ func (k *KubeadmPhase) SetupWithManager(mgr manager.Manager) error {
k.logger = mgr.GetLogger().WithName(k.Phase.GetName())
return controllerruntime.NewControllerManagedBy(mgr).
Named(k.ControllerName).
WithOptions(controller.TypedOptions[reconcile.Request]{SkipNameValidation: ptr.To(true)}).
For(k.Phase.GetWatchedObject(), builder.WithPredicates(predicate.NewPredicateFuncs(k.Phase.GetPredicateFunc()))).
WatchesRawSource(source.Channel(k.TriggerChannel, &handler.EnqueueRequestForObject{})).

View File

@@ -5,9 +5,9 @@ package controllers
import (
"context"
"errors"
"github.com/go-logr/logr"
"github.com/pkg/errors"
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
rbacv1 "k8s.io/api/rbac/v1"
@@ -36,6 +36,7 @@ type KubeProxy struct {
AdminClient client.Client
GetTenantControlPlaneFunc utils.TenantControlPlaneRetrievalFn
TriggerChannel chan event.GenericEvent
ControllerName string
}
func (k *KubeProxy) Reconcile(ctx context.Context, _ reconcile.Request) (reconcile.Result, error) {
@@ -82,6 +83,7 @@ func (k *KubeProxy) Reconcile(ctx context.Context, _ reconcile.Request) (reconci
func (k *KubeProxy) SetupWithManager(mgr manager.Manager) error {
return controllerruntime.NewControllerManagedBy(mgr).
Named(k.ControllerName).
WithOptions(controller.TypedOptions[reconcile.Request]{SkipNameValidation: ptr.To(true)}).
For(&rbacv1.ClusterRoleBinding{}, builder.WithPredicates(predicate.NewPredicateFuncs(func(object client.Object) bool {
return object.GetName() == kubeadm.KubeProxyClusterRoleBindingName

View File

@@ -5,11 +5,11 @@ package controllers
import (
"context"
"errors"
"fmt"
"time"
"github.com/go-logr/logr"
"github.com/pkg/errors"
admissionregistrationv1 "k8s.io/api/admissionregistration/v1"
apierrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
@@ -39,6 +39,7 @@ type Migrate struct {
WebhookServiceName string
WebhookCABundle []byte
TriggerChannel chan event.GenericEvent
ControllerName string
}
func (m *Migrate) Reconcile(ctx context.Context, _ reconcile.Request) (reconcile.Result, error) {
@@ -189,6 +190,7 @@ func (m *Migrate) SetupWithManager(mgr manager.Manager) error {
m.TriggerChannel = make(chan event.GenericEvent)
return controllerruntime.NewControllerManagedBy(mgr).
Named(m.ControllerName).
WithOptions(controller.TypedOptions[reconcile.Request]{SkipNameValidation: pointer.To(true)}).
For(&admissionregistrationv1.ValidatingWebhookConfiguration{}, builder.WithPredicates(predicate.NewPredicateFuncs(func(object client.Object) bool {
vwc := m.object()

View File

@@ -39,6 +39,7 @@ type WritePermissions struct {
WebhookServiceName string
WebhookCABundle []byte
TriggerChannel chan event.GenericEvent
ControllerName string
}
func (r *WritePermissions) Reconcile(ctx context.Context, _ reconcile.Request) (reconcile.Result, error) {
@@ -190,6 +191,7 @@ func (r *WritePermissions) SetupWithManager(mgr manager.Manager) error {
r.TriggerChannel = make(chan event.GenericEvent)
return controllerruntime.NewControllerManagedBy(mgr).
Named(r.ControllerName).
WithOptions(controller.TypedOptions[reconcile.Request]{SkipNameValidation: ptr.To(true)}).
For(&admissionregistrationv1.ValidatingWebhookConfiguration{}, builder.WithPredicates(predicate.NewPredicateFuncs(func(object client.Object) bool {
return object.GetName() == r.object().GetName()

View File

@@ -193,7 +193,7 @@ func (m *Manager) Reconcile(ctx context.Context, request reconcile.Request) (res
for _, trigger := range v.triggers {
var shrunkTCP kamajiv1alpha1.TenantControlPlane
shrunkTCP.Name = tcp.Namespace
shrunkTCP.Name = tcp.Name
shrunkTCP.Namespace = tcp.Namespace
go utils.TriggerChannel(ctx, trigger, shrunkTCP)
@@ -253,6 +253,9 @@ func (m *Manager) Reconcile(ctx context.Context, request reconcile.Request) (res
//
// Register all the controllers of the soot here:
//
// Generate unique controller name prefix from TenantControlPlane to avoid metric conflicts
controllerNamePrefix := fmt.Sprintf("%s-%s", tcp.GetNamespace(), tcp.GetName())
writePermissions := &controllers.WritePermissions{
Logger: mgr.GetLogger().WithName("writePermissions"),
Client: mgr.GetClient(),
@@ -261,6 +264,7 @@ func (m *Manager) Reconcile(ctx context.Context, request reconcile.Request) (res
WebhookServiceName: m.MigrateServiceName,
WebhookCABundle: m.MigrateCABundle,
TriggerChannel: nil,
ControllerName: fmt.Sprintf("%s-writepermissions", controllerNamePrefix),
}
if err = writePermissions.SetupWithManager(mgr); err != nil {
return reconcile.Result{}, err
@@ -273,6 +277,7 @@ func (m *Manager) Reconcile(ctx context.Context, request reconcile.Request) (res
GetTenantControlPlaneFunc: m.retrieveTenantControlPlane(tcpCtx, request),
Client: mgr.GetClient(),
Logger: mgr.GetLogger().WithName("migrate"),
ControllerName: fmt.Sprintf("%s-migrate", controllerNamePrefix),
}
if err = migrate.SetupWithManager(mgr); err != nil {
return reconcile.Result{}, err
@@ -283,6 +288,7 @@ func (m *Manager) Reconcile(ctx context.Context, request reconcile.Request) (res
GetTenantControlPlaneFunc: m.retrieveTenantControlPlane(tcpCtx, request),
Logger: mgr.GetLogger().WithName("konnectivity_agent"),
TriggerChannel: make(chan event.GenericEvent),
ControllerName: fmt.Sprintf("%s-konnectivity", controllerNamePrefix),
}
if err = konnectivityAgent.SetupWithManager(mgr); err != nil {
return reconcile.Result{}, err
@@ -293,6 +299,7 @@ func (m *Manager) Reconcile(ctx context.Context, request reconcile.Request) (res
GetTenantControlPlaneFunc: m.retrieveTenantControlPlane(tcpCtx, request),
Logger: mgr.GetLogger().WithName("kube_proxy"),
TriggerChannel: make(chan event.GenericEvent),
ControllerName: fmt.Sprintf("%s-kubeproxy", controllerNamePrefix),
}
if err = kubeProxy.SetupWithManager(mgr); err != nil {
return reconcile.Result{}, err
@@ -303,6 +310,7 @@ func (m *Manager) Reconcile(ctx context.Context, request reconcile.Request) (res
GetTenantControlPlaneFunc: m.retrieveTenantControlPlane(tcpCtx, request),
Logger: mgr.GetLogger().WithName("coredns"),
TriggerChannel: make(chan event.GenericEvent),
ControllerName: fmt.Sprintf("%s-coredns", controllerNamePrefix),
}
if err = coreDNS.SetupWithManager(mgr); err != nil {
return reconcile.Result{}, err
@@ -315,6 +323,7 @@ func (m *Manager) Reconcile(ctx context.Context, request reconcile.Request) (res
Phase: resources.PhaseUploadConfigKubeadm,
},
TriggerChannel: make(chan event.GenericEvent),
ControllerName: fmt.Sprintf("%s-kubeadmconfig", controllerNamePrefix),
}
if err = uploadKubeadmConfig.SetupWithManager(mgr); err != nil {
return reconcile.Result{}, err
@@ -327,6 +336,7 @@ func (m *Manager) Reconcile(ctx context.Context, request reconcile.Request) (res
Phase: resources.PhaseUploadConfigKubelet,
},
TriggerChannel: make(chan event.GenericEvent),
ControllerName: fmt.Sprintf("%s-kubeletconfig", controllerNamePrefix),
}
if err = uploadKubeletConfig.SetupWithManager(mgr); err != nil {
return reconcile.Result{}, err
@@ -339,6 +349,7 @@ func (m *Manager) Reconcile(ctx context.Context, request reconcile.Request) (res
Phase: resources.PhaseBootstrapToken,
},
TriggerChannel: make(chan event.GenericEvent),
ControllerName: fmt.Sprintf("%s-bootstraptoken", controllerNamePrefix),
}
if err = bootstrapToken.SetupWithManager(mgr); err != nil {
return reconcile.Result{}, err
@@ -351,6 +362,7 @@ func (m *Manager) Reconcile(ctx context.Context, request reconcile.Request) (res
Phase: resources.PhaseClusterAdminRBAC,
},
TriggerChannel: make(chan event.GenericEvent),
ControllerName: fmt.Sprintf("%s-kubeadmrbac", controllerNamePrefix),
}
if err = kubeadmRbac.SetupWithManager(mgr); err != nil {
return reconcile.Result{}, err

View File

@@ -5,11 +5,11 @@ package controllers
import (
"context"
"fmt"
"time"
"github.com/clastix/kamaji-telemetry/api"
telemetry "github.com/clastix/kamaji-telemetry/pkg/client"
"github.com/pkg/errors"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/types"
"k8s.io/utils/ptr"
@@ -31,7 +31,7 @@ type TelemetryController struct {
func (m *TelemetryController) retrieveControllerUID(ctx context.Context) (string, error) {
var defaultSvc corev1.Service
if err := m.Client.Get(ctx, types.NamespacedName{Name: "kubernetes", Namespace: "default"}, &defaultSvc); err != nil {
return "", errors.Wrap(err, "cannot start the telemetry controller")
return "", fmt.Errorf("cannot start the telemetry controller: %w", err)
}
return string(defaultSvc.UID), nil

View File

@@ -5,18 +5,20 @@ package controllers
import (
"context"
"errors"
"fmt"
"strings"
"time"
"github.com/juju/mutex/v2"
"github.com/pkg/errors"
appsv1 "k8s.io/api/apps/v1"
batchv1 "k8s.io/api/batch/v1"
corev1 "k8s.io/api/core/v1"
networkingv1 "k8s.io/api/networking/v1"
k8serrors "k8s.io/apimachinery/pkg/api/errors"
k8stypes "k8s.io/apimachinery/pkg/types"
"k8s.io/client-go/discovery"
"k8s.io/client-go/util/retry"
"k8s.io/client-go/util/workqueue"
"k8s.io/utils/clock"
ctrl "sigs.k8s.io/controller-runtime"
@@ -30,13 +32,17 @@ import (
"sigs.k8s.io/controller-runtime/pkg/predicate"
"sigs.k8s.io/controller-runtime/pkg/reconcile"
"sigs.k8s.io/controller-runtime/pkg/source"
gatewayv1 "sigs.k8s.io/gateway-api/apis/v1"
gatewayv1alpha2 "sigs.k8s.io/gateway-api/apis/v1alpha2"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
"github.com/clastix/kamaji/controllers/finalizers"
"github.com/clastix/kamaji/controllers/utils"
controlplanebuilder "github.com/clastix/kamaji/internal/builders/controlplane"
"github.com/clastix/kamaji/internal/datastore"
kamajierrors "github.com/clastix/kamaji/internal/errors"
"github.com/clastix/kamaji/internal/resources"
"github.com/clastix/kamaji/internal/utilities"
)
// TenantControlPlaneReconciler reconciles a TenantControlPlane object.
@@ -51,6 +57,7 @@ type TenantControlPlaneReconciler struct {
KamajiMigrateImage string
MaxConcurrentReconciles int
ReconcileTimeout time.Duration
DiscoveryClient discovery.DiscoveryInterface
// CertificateChan is the channel used by the CertificateLifecycleController that is checking for
// certificates and kubeconfig user certs validity: a generic event for the given TCP will be triggered
// once the validity threshold for the given certificate is reached.
@@ -76,6 +83,10 @@ type TenantControlPlaneReconcilerConfig struct {
//+kubebuilder:rbac:groups=apps,resources=deployments,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=networking.k8s.io,resources=ingresses,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=batch,resources=jobs,verbs=get;list;watch;create;delete
//+kubebuilder:rbac:groups=gateway.networking.k8s.io,resources=httproutes,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=gateway.networking.k8s.io,resources=grpcroutes,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=gateway.networking.k8s.io,resources=tlsroutes,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=gateway.networking.k8s.io,resources=gateways,verbs=get;list;watch
func (r *TenantControlPlaneReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
log := log.FromContext(ctx)
@@ -105,11 +116,11 @@ func (r *TenantControlPlaneReconciler) Reconcile(ctx context.Context, req ctrl.R
releaser, err := mutex.Acquire(r.mutexSpec(tenantControlPlane))
if err != nil {
switch {
case errors.As(err, &mutex.ErrTimeout):
case errors.Is(err, mutex.ErrTimeout):
log.Info("acquire timed out, current process is blocked by another reconciliation")
return ctrl.Result{RequeueAfter: time.Second}, nil
case errors.As(err, &mutex.ErrCancelled):
case errors.Is(err, mutex.ErrCancelled):
log.Info("acquire cancelled")
return ctrl.Result{RequeueAfter: time.Second}, nil
@@ -148,6 +159,25 @@ func (r *TenantControlPlaneReconciler) Reconcile(ctx context.Context, req ctrl.R
}
defer dsConnection.Close()
dso, err := r.dataStoreOverride(ctx, tenantControlPlane)
if err != nil {
log.Error(err, "cannot retrieve the DataStoreOverrides for the given instance")
return ctrl.Result{}, err
}
dsoConnections := make(map[string]datastore.Connection, len(dso))
for _, ds := range dso {
dsoConnection, err := datastore.NewStorageConnection(ctx, r.Client, ds.DataStore)
if err != nil {
log.Error(err, "cannot generate the DataStoreOverride connection for the given instance")
return ctrl.Result{}, err
}
defer dsoConnection.Close()
dsoConnections[ds.Resource] = dsoConnection
}
if markedToBeDeleted && controllerutil.ContainsFinalizer(tenantControlPlane, finalizers.DatastoreFinalizer) {
log.Info("marked for deletion, performing clean-up")
@@ -174,18 +204,21 @@ func (r *TenantControlPlaneReconciler) Reconcile(ctx context.Context, req ctrl.R
}
groupResourceBuilderConfiguration := GroupResourceBuilderConfiguration{
client: r.Client,
log: log,
tcpReconcilerConfig: r.Config,
tenantControlPlane: *tenantControlPlane,
Connection: dsConnection,
DataStore: *ds,
KamajiNamespace: r.KamajiNamespace,
KamajiServiceAccount: r.KamajiServiceAccount,
KamajiService: r.KamajiService,
KamajiMigrateImage: r.KamajiMigrateImage,
client: r.Client,
log: log,
tcpReconcilerConfig: r.Config,
tenantControlPlane: *tenantControlPlane,
Connection: dsConnection,
DataStore: *ds,
DataStoreOverrides: dso,
DataStoreOverriedsConnections: dsoConnections,
KamajiNamespace: r.KamajiNamespace,
KamajiServiceAccount: r.KamajiServiceAccount,
KamajiService: r.KamajiService,
KamajiMigrateImage: r.KamajiMigrateImage,
DiscoveryClient: r.DiscoveryClient,
}
registeredResources := GetResources(groupResourceBuilderConfiguration)
registeredResources := GetResources(ctx, groupResourceBuilderConfiguration)
for _, resource := range registeredResources {
result, err := resources.Handle(ctx, resource, tenantControlPlane)
@@ -228,6 +261,23 @@ func (r *TenantControlPlaneReconciler) Reconcile(ctx context.Context, req ctrl.R
log.Info(fmt.Sprintf("%s has been reconciled", tenantControlPlane.GetName()))
// Set ObservedGeneration only on successful reconciliation completion.
// This follows Cluster API conventions where ObservedGeneration indicates
// the controller has fully processed the given generation.
if err := retry.RetryOnConflict(retry.DefaultRetry, func() error {
if getErr := r.Client.Get(ctx, req.NamespacedName, tenantControlPlane); getErr != nil {
return getErr
}
tenantControlPlane.Status.ObservedGeneration = tenantControlPlane.Generation
return r.Client.Status().Update(ctx, tenantControlPlane)
}); err != nil {
log.Error(err, "failed to update ObservedGeneration")
return ctrl.Result{}, err
}
return ctrl.Result{}, nil
}
@@ -242,10 +292,10 @@ func (r *TenantControlPlaneReconciler) mutexSpec(obj client.Object) mutex.Spec {
}
// SetupWithManager sets up the controller with the Manager.
func (r *TenantControlPlaneReconciler) SetupWithManager(mgr ctrl.Manager) error {
func (r *TenantControlPlaneReconciler) SetupWithManager(ctx context.Context, mgr ctrl.Manager) error {
r.clock = clock.RealClock{}
return ctrl.NewControllerManagedBy(mgr).
controllerBuilder := ctrl.NewControllerManagedBy(mgr).
WatchesRawSource(source.Channel(r.CertificateChan, handler.Funcs{GenericFunc: func(_ context.Context, genericEvent event.TypedGenericEvent[client.Object], w workqueue.TypedRateLimitingInterface[reconcile.Request]) {
w.AddRateLimited(ctrl.Request{
NamespacedName: k8stypes.NamespacedName{
@@ -295,7 +345,20 @@ func (r *TenantControlPlaneReconciler) SetupWithManager(mgr ctrl.Manager) error
v, ok := labels["kamaji.clastix.io/component"]
return ok && v == "migrate"
}))).
})))
// Conditionally add Gateway API ownership if available
if utilities.AreGatewayResourcesAvailable(ctx, r.Client, r.DiscoveryClient) {
controllerBuilder = controllerBuilder.
Owns(&gatewayv1.HTTPRoute{}).
Owns(&gatewayv1.GRPCRoute{}).
Owns(&gatewayv1alpha2.TLSRoute{}).
Watches(&gatewayv1.Gateway{}, handler.EnqueueRequestsFromMapFunc(func(_ context.Context, object client.Object) []reconcile.Request {
return nil
}))
}
return controllerBuilder.
WithOptions(controller.Options{
MaxConcurrentReconciles: r.MaxConcurrentReconciles,
}).
@@ -334,8 +397,26 @@ func (r *TenantControlPlaneReconciler) dataStore(ctx context.Context, tenantCont
var ds kamajiv1alpha1.DataStore
if err := r.Client.Get(ctx, k8stypes.NamespacedName{Name: tenantControlPlane.Spec.DataStore}, &ds); err != nil {
return nil, errors.Wrap(err, "cannot retrieve *kamajiv1alpha.DataStore object")
return nil, fmt.Errorf("cannot retrieve *kamajiv1alpha.DataStore object: %w", err)
}
return &ds, nil
}
func (r *TenantControlPlaneReconciler) dataStoreOverride(ctx context.Context, tenantControlPlane *kamajiv1alpha1.TenantControlPlane) ([]controlplanebuilder.DataStoreOverrides, error) {
datastores := make([]controlplanebuilder.DataStoreOverrides, 0, len(tenantControlPlane.Spec.DataStoreOverrides))
for _, dso := range tenantControlPlane.Spec.DataStoreOverrides {
var ds kamajiv1alpha1.DataStore
if err := r.Client.Get(ctx, k8stypes.NamespacedName{Name: dso.DataStore}, &ds); err != nil {
return nil, fmt.Errorf("cannot retrieve *kamajiv1alpha.DataStore object: %w", err)
}
if ds.Spec.Driver != kamajiv1alpha1.EtcdDriver {
return nil, errors.New("DataStoreOverrides can only use ETCD driver")
}
datastores = append(datastores, controlplanebuilder.DataStoreOverrides{Resource: dso.Resource, DataStore: ds})
}
return datastores, nil
}

View File

@@ -1,24 +1,24 @@
# Cluster Autoscaler
The [Cluster Autoscaler](https://github.com/kubernetes/autoscaler) is a tool that automatically adjusts the size of a Kubernetes cluster so that all pods have a place to run and there are no unneeded nodes.
The [Cluster Autoscaler](https://github.com/kubernetes/autoscaler) is a tool that automatically adjusts the size of a Kubernetes cluster so that all pods have a place to run and no unneeded nodes remain.
When pods are unschedulable because there are not enough resources, Cluster Autoscaler scales up the cluster. When nodes are underutilized, Cluster Autoscaler scales down the cluster.
When pods are unschedulable because there are not enough resources, the Cluster Autoscaler scales up the cluster. When nodes are underutilized, the Cluster Autoscaler scales the cluster down.
Cluster API supports the Cluster Autoscaler. See the [Cluster Autoscaler on Cluster API](https://cluster-api.sigs.k8s.io/tasks/automated-machine-management/autoscaling) for more information.
## Getting started with the Cluster Autoscaler on Kamaji
Kamaji supports the Cluster Autoscaler through Cluster API. There are several way to run the Cluster autoscaler with Cluster API. In this guide, we're leveraging the unique features of Kamaji to run the Cluster Autoscaler as part of Hosted Control Plane.
Kamaji supports the Cluster Autoscaler through Cluster API. There are several ways to run the Cluster Autoscaler with Cluster API. In this guide, we leverage the unique features of Kamaji to run the Cluster Autoscaler as part of the Hosted Control Plane.
In other words, the Cluster Autoscaler is running as a pod in the Kamaji Management Cluster, side by side with the Tenant Control Plane pods, and connecting directly to the apiserver of the workload cluster, hiding sensitive data and information from the tenant: this can be done by mounting the kubeconfig of the tenant cluster in the Cluster Autoscaler pod.
In other words, the Cluster Autoscaler runs as a pod in the Kamaji Management Cluster, alongside the Tenant Control Plane pods, and connects directly to the API server of the workload cluster. This approach hides sensitive data from the tenant. It works by mounting the kubeconfig of the tenant cluster into the Cluster Autoscaler pod.
### Create the workload cluster
Create a workload cluster using the Kamaji Control Plane Provider and the Infrastructure Provider of choice. The following example creates a workload cluster using the vSphere Infrastructure Provider:
Create a workload cluster using the Kamaji Control Plane Provider and the Infrastructure Provider of your choice. The following example creates a workload cluster using the vSphere Infrastructure Provider.
The template file [`capi-kamaji-vsphere-autoscaler-template.yaml`](https://raw.githubusercontent.com/clastix/cluster-api-control-plane-provider-kamaji/master/templates/vsphere/capi-kamaji-vsphere-autoscaler-template.yaml) provides a full example of a cluster with autoscaler enabled. You can generate the cluster manifest using `clusterctl`.
The template file [`capi-kamaji-vsphere-autoscaler-template.yaml`](https://raw.githubusercontent.com/clastix/cluster-api-control-plane-provider-kamaji/master/templates/vsphere/capi-kamaji-vsphere-autoscaler-template.yaml) provides a full example of a cluster with the autoscaler enabled. You can generate the cluster manifest using `clusterctl`.
Before you need to list all the variables in the template file:
Before doing so, list all the variables in the template file:
```bash
cat capi-kamaji-vsphere-autoscaler-template.yaml | clusterctl generate yaml --list-variables
@@ -40,10 +40,10 @@ kubectl apply -f capi-kamaji-vsphere-cluster.yaml
### Install the Cluster Autoscaler
Install the Cluster Autoscaler via Helm in the Management Cluster, in the same namespace where workload cluster is deployed.
Install the Cluster Autoscaler via Helm in the Management Cluster, in the same namespace where the workload cluster is deployed.
!!! info "Options for install Cluster Autoscaler"
Cluster Autoscaler works on a single cluster: it means every cluster must have its own Cluster Autoscaler instance. This could be solved by leveraging on Project Sveltos automations, by deploying a Cluster Autoscaler instance for each Kamaji Cluster API instance.
!!! info "Options for installing the Cluster Autoscaler"
The Cluster Autoscaler works on a single cluster, meaning every cluster must have its own Cluster Autoscaler instance. This can be addressed by leveraging Project Sveltos automations to deploy a Cluster Autoscaler instance for each Kamaji Cluster API instance.
```bash
helm repo add autoscaler https://kubernetes.github.io/autoscaler
@@ -56,9 +56,9 @@ helm upgrade --install ${CLUSTER_NAME}-autoscaler autoscaler/cluster-autoscaler
--set clusterAPIMode=kubeconfig-incluster
```
The `autoDiscovery.labels` values are used to pick dynamically clusters to autoscale.
The `autoDiscovery.labels` values are used to dynamically select clusters to autoscale.
Such labels must be set on the workload cluster, in the `Cluster` and `MachineDeployment` resources.
These labels must be set on the workload cluster, specifically in the `Cluster` and `MachineDeployment` resources.
```yaml
apiVersion: cluster.x-k8s.io/v1beta1
@@ -79,6 +79,9 @@ metadata:
# Cluster Autoscaler annotations
cluster.x-k8s.io/cluster-api-autoscaler-node-group-min-size: "0"
cluster.x-k8s.io/cluster-api-autoscaler-node-group-max-size: "6"
capacity.cluster-autoscaler.kubernetes.io/cpu: "2" # YMMV
capacity.cluster-autoscaler.kubernetes.io/memory: 4Gi # YMMV
capacity.cluster-autoscaler.kubernetes.io/maxPods: "110" # YMMV
labels:
cluster.x-k8s.io/cluster-name: sample
# Cluster Autoscaler labels
@@ -90,10 +93,9 @@ metadata:
# other Cluster API resources omitted for brevity
```
### Verify the Cluster Autoscaler
To verify the Cluster Autoscaler is working as expected, you can deploy a workload in the Tenant cluster with some CPU requirements in order to simulate workload requiring resources.
To verify that the Cluster Autoscaler is working as expected, deploy a workload in the Tenant cluster with specific CPU requirements to simulate resource demand.
```yaml
apiVersion: apps/v1
@@ -122,7 +124,51 @@ spec:
cpu: 500m
```
Apply the workload to the Tenant cluster and simulate the load spike by increasing the replicas. The Cluster Autoscaler should scale up the cluster to accommodate the workload. Cooldown time must be configured properly on a cluster basis.
Apply the workload to the Tenant cluster and simulate a load spike by increasing the number of replicas. The Cluster Autoscaler should scale up the cluster to accommodate the workload. Cooldown times must be configured correctly on a per-cluster basis.
!!! warning "Possible Resource Wasting"
With Cluster Autoscaler, new machines are automatically created in a very short time, ending up with some up-provisioning and potentially wasting resources. The official Cluster Autosclaler documentation must be understood to provide correct values according to the infrastructure and provisioning times.
!!! warning "Possible Resource Wastage"
With the Cluster Autoscaler, new machines may be created very quickly, which can lead to over-provisioning and potentially wasted resources. The official Cluster Autoscaler documentation should be consulted to configure appropriate values based on your infrastructure and provisioning times.
## `ProvisioningRequest` support
The [ProvisioningRequest](https://github.com/kubernetes/autoscaler/blob/cluster-autoscaler-1.34.1/cluster-autoscaler/proposals/provisioning-request.md) introduces a Kubernetes-native way for Cluster Autoscaler to request new capacity without talking directly to cloud provider APIs.
Instead of embedding provider-specific logic, the autoscaler simply describes the capacity it needs, and an external provisioner decides how to create the required nodes.
This makes scaling portable across clouds, on-prem platforms, and custom provisioning systems, while greatly reducing complexity inside the autoscaler.
Once the cluster has been provisioned, install the `ProvisioningRequest` definition.
```
kubectl kamaji kubeconfig get capi-quickstart-kubevirt > /tmp/capi-quickstart-kubevirt
KUBECONFIG=/tmp/capi-quickstart-kubevirt kubectl apply -f https://raw.githubusercontent.com/kubernetes/autoscaler/refs/tags/cluster-autoscaler-1.34.1/cluster-autoscaler/apis/config/crd/autoscaling.x-k8s.io_provisioningrequests.yaml
```
Proceed with the installation of Cluster Autoscaler by enabling some additional parameters: YMMV.
```yaml
cloudProvider: clusterapi
autoDiscovery:
namespace: default
labels:
- autoscaling.x-k8s.io: enabled
clusterAPIKubeconfigSecret: capi-quickstart-kubeconfig
clusterAPIMode: kubeconfig-incluster
extraArgs:
enable-provisioning-requests: true
kube-api-content-type: "application/json"
cloud-config: /etc/kubernetes/management/kubeconfig
extraVolumeSecrets:
# Mount the management kubeconfig to talk with the management cluster:
# the in-rest configuration doesn't work
management-kubeconfig:
name: management-kubeconfig
mountPath: /etc/kubernetes/management
items:
- key: kubeconfig
path: kubeconfig
```
The Cluster Autoscaler should be up and running, enabled to connect to the management and tenant cluster API Server:
follow the [official example](https://github.com/kubernetes/autoscaler/blob/cluster-autoscaler-1.34.1/cluster-autoscaler/FAQ.md#example-usage) from the repository to assess the `ProvisioningRequest` feature.

View File

@@ -32,16 +32,18 @@ Kamaji has a dependency on Cert Manager, as it uses dynamic admission control, v
Add the Bitnami Repo to the Helm Manager.
```
helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo add jetstack https://charts.jetstack.io
helm repo update
```
Install Cert Manager using Helm
```
helm upgrade --install cert-manager bitnami/cert-manager \
--namespace certmanager-system \
helm install \
cert-manager jetstack/cert-manager \
--namespace cert-manager \
--create-namespace \
--set "installCRDs=true"
--set installCRDs=true
```
This will install cert-manager to the cluster. You can watch the progress of the installation on the cluster using the command
@@ -55,7 +57,7 @@ kubectl get pods -Aw
MetalLB is used in order to dynamically assign IP addresses to the components, and also define custom IP Address Pools. Install MetalLb using the `kubectl` command for apply the manifest:
```
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.13.7/config/manifests/metallb-native.yaml
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.15.3/config/manifests/metallb-native.yaml
```
This will install MetalLb onto the cluster with all the necessary resources.
@@ -65,7 +67,11 @@ This will install MetalLb onto the cluster with all the necessary resources.
Extract the Gateway IP of the network Kind is running on.
```
# docker
GW_IP=$(docker network inspect -f '{{range .IPAM.Config}}{{.Gateway}}{{end}}' kind)
# podman
GW_IP=$(podman network inspect kind --format '{{(index .Subnets 1).Gateway}}')
```
Modify the IP Address, and create the resource to be added to the cluster to create the IP Address Pool

View File

@@ -0,0 +1,78 @@
# Datastore Overrides
Kamaji offers the possibility of having multiple ETCD clusters backing different resources of the k8s api server by configuring the [`--etcd-servers-overrides`](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-apiserver/#:~:text=%2D%2Detcd%2Dservers%2Doverrides%20strings) flag. This feature can be useful for massive clusters to store resources with high churn in a dedicated ETCD cluster.
## Install Datastores
Create a self-signed cert-manager `ClusterIssuer`.
```bash
echo 'apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: self-signed
spec:
selfSigned: {}
' | kubectl apply -f -
```
Install two Datastores, a primary and a secondary that will be used for `/events` resources.
```bash
helm install etcd-primary clastix/kamaji-etcd -n kamaji-etcd --create-namespace \
--set selfSignedCertificates.enabled=false \
--set certManager.enabled=true \
--set certManager.issuerRef.kind=ClusterIssuer \
--set certManager.issuerRef.name=self-signed
```
For the secondary Datastore, use the cert-manager CA created by the `etcd-primary` helm release.
```bash
helm install etcd-secondary clastix/kamaji-etcd -n kamaji-etcd --create-namespace \
--set selfSignedCertificates.enabled=false \
--set certManager.enabled=true \
--set certManager.ca.create=false \
--set certManager.ca.nameOverride=etcd-primary-kamaji-etcd-ca \
--set certManager.issuerRef.kind=ClusterIssuer \
--set certManager.issuerRef.name=self-signed
```
## Create a Tenant Control Plane
Using the `spec.dataStoreOverrides` field, Datastores different from the one used in `spec.dataStore` can be used to store specific resources.
```bash
echo 'apiVersion: kamaji.clastix.io/v1alpha1
kind: TenantControlPlane
metadata:
name: k8s-133
labels:
tenant.clastix.io: k8s-133
spec:
controlPlane:
deployment:
replicas: 2
service:
serviceType: LoadBalancer
kubernetes:
version: "v1.33.1"
kubelet:
cgroupfs: systemd
dataStore: etcd-primary-kamaji-etcd
dataStoreOverrides:
- resource: "/events" # Store events in the secondary ETCD
dataStore: etcd-secondary-kamaji-etcd
networkProfile:
port: 6443
addons:
coreDNS: {}
kubeProxy: {}
konnectivity:
server:
port: 8132
agent:
mode: DaemonSet
' | k apply -f -
```
## Considerations
Only built-in resources can be tagetted by `--etcd-servers-overrides`, it is currently not possible to target Custom Resources.

View File

@@ -0,0 +1,213 @@
# Gateway API Support
Kamaji provides built-in support for the [Gateway API](https://gateway-api.sigs.k8s.io/), allowing you to expose Tenant Control Planes using TLSRoute resources with SNI-based routing. This enables hostname-based routing to multiple Tenant Control Planes through a single Gateway resource, reducing the need for dedicated LoadBalancer services.
## Overview
Gateway API support in Kamaji automatically creates and manages TLSRoute resources for your Tenant Control Planes. When you configure a Gateway for a Tenant Control Plane, Kamaji automatically creates TLSRoutes for the Control Plane API Server. If konnectivity is enabled, a separate TLSRoute is created for it. Both TLSRoutes use the same hostname and Gateway resource, but route to different ports(listeners) using port-based routing and semantic `sectionName` values.
Therefore, the target `Gateway` resource must have right listener configurations (see the Gateway [example section](#gateway-resource-setup) below).
## How It Works
When you configure `spec.controlPlane.gateway` in a TenantControlPlane resource, Kamaji automatically:
1. **Creates a TLSRoute for the control plane** that routes for port 6443 (or `spec.networkProfile.port`) with sectionName `"kube-apiserver"`
2. **Creates a TLSRoute for Konnectivity** (if konnectivity addon is enabled) that routes for port 8132 (or `spec.addons.konnectivity.server.port`) with sectionName `"konnectivity-server"`
Both TLSRoutes:
- Use the same hostname from `spec.controlPlane.gateway.hostname`
- Reference the same parent Gateway resource via `parentRefs`
- The `port` and `sectionName` fields are set automatically by Kamaji
- Route to the appropriate Tenant Control Plane service
The Gateway resource must have listeners configured for both ports (6443 and 8132) to support both routes.
## Prerequisites
Before using Gateway API support, ensure:
1. **Gateway API CRDs are installed** in your cluster (Required CRDs: `GatewayClass`, `Gateway`, `TLSRoute`)
2. **A Gateway resource exists** with appropriate listeners configured:
- At minimum, listeners for ports 6443 (control plane) and 8132 (Konnectivity)
- TLS protocol with Passthrough mode
- Hostname pattern matching your Tenant Control Plane hostnames
3. **DNS is configured** to resolve your hostnames to the Gateway's external address
4. **Gateway controller is running** (e.g., Envoy Gateway, Istio Gateway, etc.)
## Configuration
### TenantControlPlane Gateway Configuration
Enable Gateway API mode by setting the `spec.controlPlane.gateway` field in your TenantControlPlane resource:
```yaml
apiVersion: kamaji.clastix.io/v1alpha1
kind: TenantControlPlane
metadata:
name: tcp-1
spec:
controlPlane:
# ... gateway configuration:
gateway:
hostname: "tcp1.cluster.dev"
parentRefs:
- name: gateway
namespace: default
additionalMetadata:
labels:
environment: production
annotations:
example.com/custom: "value"
# ... rest of the spec
deployment:
replicas: 1
service:
serviceType: ClusterIP
dataStore: default
kubernetes:
version: v1.29.0
kubelet:
cgroupfs: systemd
networkProfile:
port: 6443
certSANs:
- "c11.cluster.dev" # make sure to set this.
addons:
coreDNS: {}
kubeProxy: {}
konnectivity: {}
```
**Required fields:**
- `hostname`: The hostname that will be used for routing (must match Gateway listener hostname pattern)
- `parentRefs`: Array of Gateway references (name and namespace)
**Optional fields:**
- `additionalMetadata.labels`: Custom labels to add to TLSRoute resources
- `additionalMetadata.annotations`: Custom annotations to add to TLSRoute resources
!!! warning "Port and sectionName are set automatically"
Do not specify `port` or `sectionName` in `parentRefs`. Kamaji automatically sets these fields in TLSRoutes.
### Gateway Resource Setup
Your Gateway resource must have listeners configured for both the control plane and Konnectivity ports. Here's an example Gateway configuration:
```yaml
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
name: envoy-gw-class
spec:
controllerName: gateway.envoyproxy.io/gatewayclass-controller
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: gateway
namespace: default
spec:
gatewayClassName: envoy-gw-class
listeners:
- name: kube-apiserver
port: 6443
protocol: TLS
hostname: 'tcp1.cluster.dev'
tls:
mode: Passthrough
allowedRoutes:
kinds:
- group: gateway.networking.k8s.io
kind: TLSRoute
namespaces:
from: All
# if konnectivity addon is enabled:
- name: konnectivity-server
port: 8132
protocol: TLS
hostname: 'tcp1.cluster.dev'
tls:
mode: Passthrough
allowedRoutes:
kinds:
- group: gateway.networking.k8s.io
kind: TLSRoute
namespaces:
from: All
```
## Multiple Tenant Control Planes
You can use the same Gateway resource for multiple Tenant Control Planes by using different hostnames:
```yaml
# Gateway with wildcard hostname
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: gateway
spec:
listeners:
- hostname: '*.cluster.dev'
name: kube-apiserver
port: 6443
# ...
---
# Tenant Control Plane 1
apiVersion: kamaji.clastix.io/v1alpha1
kind: TenantControlPlane
metadata:
name: tcp-1
spec:
controlPlane:
gateway:
hostname: "tcp1.cluster.dev"
parentRefs:
- name: gateway
namespace: default
# ...
---
# Tenant Control Plane 2
apiVersion: kamaji.clastix.io/v1alpha1
kind: TenantControlPlane
metadata:
name: tcp-2
spec:
controlPlane:
gateway:
hostname: "tcp2.cluster.dev"
parentRefs:
- name: gateway
namespace: default
# ...
```
Each Tenant Control Plane will get its own TLSRoutes with the respective hostnames, all routing through the same Gateway resource.
You can check the Gateway status in the TenantControlPlane:
```bash
kubectl get tenantcontrolplane tcp-1 -o yaml
```
Look for the `status.kubernetesResources.gateway` and `status.addons.konnectivity.gateway` fields.
## Additional Resources
- [Gateway API Documentation](https://gateway-api.sigs.k8s.io/)
- [Quickstart with Envoy Gateway](https://gateway.envoyproxy.io/docs/tasks/quickstart/)

File diff suppressed because it is too large Load Diff

View File

@@ -76,9 +76,11 @@ nav:
- guides/pausing.md
- guides/write-permissions.md
- guides/datastore-migration.md
- guides/datastore-overrides.md
- guides/gitops.md
- guides/console.md
- guides/kubeconfig-generator.md
- guides/gateway-api.md
- guides/upgrade.md
- guides/monitoring.md
- guides/terraform.md

View File

@@ -4,10 +4,12 @@
package e2e
import (
"context"
"testing"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes/scheme"
"k8s.io/client-go/rest"
pointer "k8s.io/utils/ptr"
@@ -15,6 +17,8 @@ import (
"sigs.k8s.io/controller-runtime/pkg/envtest"
logf "sigs.k8s.io/controller-runtime/pkg/log"
"sigs.k8s.io/controller-runtime/pkg/log/zap"
gatewayv1 "sigs.k8s.io/gateway-api/apis/v1"
gatewayv1alpha2 "sigs.k8s.io/gateway-api/apis/v1alpha2"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
)
@@ -56,13 +60,49 @@ var _ = BeforeSuite(func() {
err = kamajiv1alpha1.AddToScheme(scheme.Scheme)
Expect(err).NotTo(HaveOccurred())
err = gatewayv1.Install(scheme.Scheme)
Expect(err).NotTo(HaveOccurred())
err = gatewayv1alpha2.Install(scheme.Scheme)
Expect(err).NotTo(HaveOccurred())
//+kubebuilder:scaffold:scheme
k8sClient, err = client.New(cfg, client.Options{Scheme: scheme.Scheme})
Expect(err).NotTo(HaveOccurred())
Expect(k8sClient).NotTo(BeNil())
By("creating GatewayClass for Gateway API tests")
gatewayClass := &gatewayv1.GatewayClass{
ObjectMeta: metav1.ObjectMeta{
Name: "envoy-gw-class",
},
Spec: gatewayv1.GatewayClassSpec{
ControllerName: "gateway.envoyproxy.io/gatewayclass-controller",
},
}
Expect(k8sClient.Create(context.Background(), gatewayClass)).NotTo(HaveOccurred())
By("creating Gateway with kube-apiserver and konnectivity-server listeners")
CreateGatewayWithListeners("test-gateway", "default", "envoy-gw-class", "*.example.com")
})
var _ = AfterSuite(func() {
By("deleting Gateway resources")
gateway := &gatewayv1.Gateway{
ObjectMeta: metav1.ObjectMeta{
Name: "test-gateway",
Namespace: "default",
},
}
_ = k8sClient.Delete(context.Background(), gateway)
gatewayClass := &gatewayv1.GatewayClass{
ObjectMeta: metav1.ObjectMeta{
Name: "envoy-gw-class",
},
}
_ = k8sClient.Delete(context.Background(), gatewayClass)
By("tearing down the test environment")
err := testEnv.Stop()
Expect(err).NotTo(HaveOccurred())

View File

@@ -0,0 +1,68 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package e2e
import (
"context"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
pointer "k8s.io/utils/ptr"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
)
var _ = Describe("Deploy a TenantControlPlane resource with DataStoreOverrides", func() {
// Fill TenantControlPlane object
tcp := &kamajiv1alpha1.TenantControlPlane{
ObjectMeta: metav1.ObjectMeta{
Name: "tcp-datastore-overrides",
Namespace: "default",
},
Spec: kamajiv1alpha1.TenantControlPlaneSpec{
DataStore: "etcd-primary",
DataStoreOverrides: []kamajiv1alpha1.DataStoreOverride{{
Resource: "/events",
DataStore: "etcd-secondary",
}},
ControlPlane: kamajiv1alpha1.ControlPlane{
Deployment: kamajiv1alpha1.DeploymentSpec{
Replicas: pointer.To(int32(1)),
},
Service: kamajiv1alpha1.ServiceSpec{
ServiceType: "ClusterIP",
},
},
NetworkProfile: kamajiv1alpha1.NetworkProfileSpec{
Address: "172.18.0.2",
},
Kubernetes: kamajiv1alpha1.KubernetesSpec{
Version: "v1.23.6",
Kubelet: kamajiv1alpha1.KubeletSpec{
CGroupFS: "cgroupfs",
},
AdmissionControllers: kamajiv1alpha1.AdmissionControllers{
"LimitRanger",
"ResourceQuota",
},
},
Addons: kamajiv1alpha1.AddonsSpec{},
},
}
// Create a TenantControlPlane resource into the cluster
JustBeforeEach(func() {
Expect(k8sClient.Create(context.Background(), tcp)).NotTo(HaveOccurred())
})
// Delete the TenantControlPlane resource after test is finished
JustAfterEach(func() {
Expect(k8sClient.Delete(context.Background(), tcp)).Should(Succeed())
})
// Check if TenantControlPlane resource has been created
It("Should be Ready", func() {
StatusMustEqualTo(tcp, kamajiv1alpha1.VersionReady)
})
})

View File

@@ -0,0 +1,151 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package e2e
import (
"context"
"fmt"
"time"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/types"
pointer "k8s.io/utils/ptr"
gatewayv1 "sigs.k8s.io/gateway-api/apis/v1"
gatewayv1alpha2 "sigs.k8s.io/gateway-api/apis/v1alpha2"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
)
var _ = Describe("Deploy a TenantControlPlane with Gateway API and Konnectivity", func() {
var tcp *kamajiv1alpha1.TenantControlPlane
JustBeforeEach(func() {
tcp = &kamajiv1alpha1.TenantControlPlane{
ObjectMeta: metav1.ObjectMeta{
Name: "tcp-konnectivity-gateway",
Namespace: "default",
},
Spec: kamajiv1alpha1.TenantControlPlaneSpec{
ControlPlane: kamajiv1alpha1.ControlPlane{
Deployment: kamajiv1alpha1.DeploymentSpec{
Replicas: pointer.To(int32(1)),
},
Service: kamajiv1alpha1.ServiceSpec{
ServiceType: "ClusterIP",
},
Gateway: &kamajiv1alpha1.GatewaySpec{
Hostname: gatewayv1.Hostname("tcp-gateway-konnectivity.example.com"),
AdditionalMetadata: kamajiv1alpha1.AdditionalMetadata{
Labels: map[string]string{
"test.kamaji.io/gateway": "true",
},
Annotations: map[string]string{
"test.kamaji.io/created-by": "e2e-test",
},
},
GatewayParentRefs: []gatewayv1.ParentReference{
{
Name: "test-gateway",
},
},
},
},
NetworkProfile: kamajiv1alpha1.NetworkProfileSpec{
Address: "172.18.0.4",
},
Kubernetes: kamajiv1alpha1.KubernetesSpec{
Version: "v1.28.0",
Kubelet: kamajiv1alpha1.KubeletSpec{
CGroupFS: "cgroupfs",
},
AdmissionControllers: kamajiv1alpha1.AdmissionControllers{
"LimitRanger",
"ResourceQuota",
},
},
Addons: kamajiv1alpha1.AddonsSpec{
Konnectivity: &kamajiv1alpha1.KonnectivitySpec{
KonnectivityServerSpec: kamajiv1alpha1.KonnectivityServerSpec{
Port: 8132,
},
},
},
},
}
Expect(k8sClient.Create(context.Background(), tcp)).NotTo(HaveOccurred())
})
JustAfterEach(func() {
Expect(k8sClient.Delete(context.Background(), tcp)).Should(Succeed())
// Wait for the object to be completely deleted
Eventually(func() bool {
err := k8sClient.Get(context.Background(), types.NamespacedName{
Name: tcp.Name,
Namespace: tcp.Namespace,
}, &kamajiv1alpha1.TenantControlPlane{})
return err != nil // Returns true when object is not found (deleted)
}).WithTimeout(time.Minute).Should(BeTrue())
})
It("Should be Ready", func() {
StatusMustEqualTo(tcp, kamajiv1alpha1.VersionReady)
})
It("Should create Konnectivity TLSRoute with correct sectionName", func() {
Eventually(func() error {
route := &gatewayv1alpha2.TLSRoute{}
if err := k8sClient.Get(context.Background(), types.NamespacedName{
Name: tcp.Name + "-konnectivity",
Namespace: tcp.Namespace,
}, route); err != nil {
return err
}
if len(route.Spec.ParentRefs) == 0 {
return fmt.Errorf("parentRefs is empty")
}
if route.Spec.ParentRefs[0].SectionName == nil {
return fmt.Errorf("sectionName is nil")
}
if *route.Spec.ParentRefs[0].SectionName != gatewayv1.SectionName("konnectivity-server") {
return fmt.Errorf("expected sectionName 'konnectivity-server', got '%s'", *route.Spec.ParentRefs[0].SectionName)
}
return nil
}).WithTimeout(time.Minute).Should(Succeed())
})
It("Should use same hostname for both TLSRoutes", func() {
Eventually(func() error {
controlPlaneRoute := &gatewayv1alpha2.TLSRoute{}
if err := k8sClient.Get(context.Background(), types.NamespacedName{
Name: tcp.Name,
Namespace: tcp.Namespace,
}, controlPlaneRoute); err != nil {
return err
}
konnectivityRoute := &gatewayv1alpha2.TLSRoute{}
if err := k8sClient.Get(context.Background(), types.NamespacedName{
Name: tcp.Name + "-konnectivity",
Namespace: tcp.Namespace,
}, konnectivityRoute); err != nil {
return err
}
if len(controlPlaneRoute.Spec.Hostnames) == 0 || len(konnectivityRoute.Spec.Hostnames) == 0 {
return fmt.Errorf("hostnames are empty")
}
if controlPlaneRoute.Spec.Hostnames[0] != konnectivityRoute.Spec.Hostnames[0] {
return fmt.Errorf("hostnames do not match: control plane '%s', konnectivity '%s'",
controlPlaneRoute.Spec.Hostnames[0], konnectivityRoute.Spec.Hostnames[0])
}
return nil
}).WithTimeout(time.Minute).Should(Succeed())
})
})

View File

@@ -0,0 +1,128 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package e2e
import (
"context"
"fmt"
"time"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/types"
pointer "k8s.io/utils/ptr"
gatewayv1 "sigs.k8s.io/gateway-api/apis/v1"
gatewayv1alpha2 "sigs.k8s.io/gateway-api/apis/v1alpha2"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
)
var _ = Describe("Deploy a TenantControlPlane with Gateway API", func() {
var tcp *kamajiv1alpha1.TenantControlPlane
JustBeforeEach(func() {
tcp = &kamajiv1alpha1.TenantControlPlane{
ObjectMeta: metav1.ObjectMeta{
Name: "tcp-gateway",
Namespace: "default",
},
Spec: kamajiv1alpha1.TenantControlPlaneSpec{
ControlPlane: kamajiv1alpha1.ControlPlane{
Deployment: kamajiv1alpha1.DeploymentSpec{
Replicas: pointer.To(int32(1)),
},
Service: kamajiv1alpha1.ServiceSpec{
ServiceType: "ClusterIP",
},
Gateway: &kamajiv1alpha1.GatewaySpec{
Hostname: gatewayv1.Hostname("tcp-gateway.example.com"),
AdditionalMetadata: kamajiv1alpha1.AdditionalMetadata{
Labels: map[string]string{
"test.kamaji.io/gateway": "true",
},
Annotations: map[string]string{
"test.kamaji.io/created-by": "e2e-test",
},
},
GatewayParentRefs: []gatewayv1.ParentReference{
{
Name: "test-gateway",
},
},
},
},
NetworkProfile: kamajiv1alpha1.NetworkProfileSpec{
Address: "172.18.0.3",
},
Kubernetes: kamajiv1alpha1.KubernetesSpec{
Version: "v1.23.6",
Kubelet: kamajiv1alpha1.KubeletSpec{
CGroupFS: "cgroupfs",
},
AdmissionControllers: kamajiv1alpha1.AdmissionControllers{
"LimitRanger",
"ResourceQuota",
},
},
Addons: kamajiv1alpha1.AddonsSpec{},
},
}
Expect(k8sClient.Create(context.Background(), tcp)).NotTo(HaveOccurred())
})
JustAfterEach(func() {
Expect(k8sClient.Delete(context.Background(), tcp)).Should(Succeed())
// Wait for the object to be completely deleted
Eventually(func() bool {
err := k8sClient.Get(context.Background(), types.NamespacedName{
Name: tcp.Name,
Namespace: tcp.Namespace,
}, &kamajiv1alpha1.TenantControlPlane{})
return err != nil // Returns true when object is not found (deleted)
}).WithTimeout(time.Minute).Should(BeTrue())
})
It("Should be Ready", func() {
StatusMustEqualTo(tcp, kamajiv1alpha1.VersionReady)
})
It("Should create control plane TLSRoute with correct sectionName", func() {
Eventually(func() error {
route := &gatewayv1alpha2.TLSRoute{}
// TODO: Check ownership.
if err := k8sClient.Get(context.Background(), types.NamespacedName{
Name: tcp.Name,
Namespace: tcp.Namespace,
}, route); err != nil {
return err
}
if len(route.Spec.ParentRefs) == 0 {
return fmt.Errorf("parentRefs is empty")
}
if route.Spec.ParentRefs[0].SectionName == nil {
return fmt.Errorf("sectionName is nil")
}
if *route.Spec.ParentRefs[0].SectionName != gatewayv1.SectionName("kube-apiserver") {
return fmt.Errorf("expected sectionName 'kube-apiserver', got '%s'", *route.Spec.ParentRefs[0].SectionName)
}
return nil
}).WithTimeout(time.Minute).Should(Succeed())
})
It("Should not create Konnectivity TLSRoute", func() {
// Verify Konnectivity route is not created
Consistently(func() error {
route := &gatewayv1alpha2.TLSRoute{}
return k8sClient.Get(context.Background(), types.NamespacedName{
Name: tcp.Name + "-konnectivity",
Namespace: tcp.Namespace,
}, route)
}, 10*time.Second, time.Second).Should(HaveOccurred())
})
})

View File

@@ -5,10 +5,12 @@ package e2e
import (
"context"
"time"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/types"
pointer "k8s.io/utils/ptr"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
@@ -59,5 +61,15 @@ var _ = Describe("Deploy a TenantControlPlane resource", func() {
// Check if TenantControlPlane resource has been created
It("Should be Ready", func() {
StatusMustEqualTo(tcp, kamajiv1alpha1.VersionReady)
// ObservedGeneration is set at the end of successful reconciliation,
// after status becomes Ready, so we need to wait for it.
Eventually(func() int64 {
if err := k8sClient.Get(context.Background(), types.NamespacedName{Name: tcp.GetName(), Namespace: tcp.GetNamespace()}, tcp); err != nil {
return -1
}
return tcp.Status.ObservedGeneration
}, 30*time.Second, time.Second).Should(Equal(tcp.Generation),
"ObservedGeneration should equal Generation after successful reconciliation")
})
})

View File

@@ -23,14 +23,14 @@ var _ = Describe("using an unsupported TenantControlPlane Kubernetes version", f
v, err := semver.Make(upgrade.KubeadmVersion[1:])
Expect(err).ToNot(HaveOccurred())
unsupported, err := semver.Make(fmt.Sprintf("%d.%d.%d", v.Major, v.Minor, v.Patch+1))
unsupported, err := semver.Make(fmt.Sprintf("%d.%d.%d", v.Major, v.Minor+1, 0))
Expect(err).ToNot(HaveOccurred())
It("should be blocked on creation", func() {
Consistently(func() error {
tcp := kamajiv1alpha1.TenantControlPlane{
ObjectMeta: metav1.ObjectMeta{
Name: "non-linear-update",
Name: "unsupported-version",
Namespace: "default",
},
Spec: kamajiv1alpha1.TenantControlPlaneSpec{

View File

@@ -19,7 +19,9 @@ import (
"k8s.io/apimachinery/pkg/types"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/util/retry"
pointer "k8s.io/utils/ptr"
"sigs.k8s.io/controller-runtime/pkg/client"
gatewayv1 "sigs.k8s.io/gateway-api/apis/v1"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
)
@@ -209,3 +211,60 @@ func ScaleTenantControlPlane(tcp *kamajiv1alpha1.TenantControlPlane, replicas in
})
Expect(err).To(Succeed())
}
// CreateGatewayWithListeners creates a Gateway with both kube-apiserver and konnectivity-server listeners.
func CreateGatewayWithListeners(gatewayName, namespace, gatewayClassName, hostname string) {
GinkgoHelper()
gateway := &gatewayv1.Gateway{
ObjectMeta: metav1.ObjectMeta{
Name: gatewayName,
Namespace: namespace,
},
Spec: gatewayv1.GatewaySpec{
GatewayClassName: gatewayv1.ObjectName(gatewayClassName),
Listeners: []gatewayv1.Listener{
{
Name: "kube-apiserver",
Port: 6443,
Protocol: gatewayv1.TLSProtocolType,
Hostname: pointer.To(gatewayv1.Hostname(hostname)),
TLS: &gatewayv1.ListenerTLSConfig{
Mode: pointer.To(gatewayv1.TLSModeType("Passthrough")),
},
AllowedRoutes: &gatewayv1.AllowedRoutes{
Namespaces: &gatewayv1.RouteNamespaces{
From: pointer.To(gatewayv1.NamespacesFromAll),
},
Kinds: []gatewayv1.RouteGroupKind{
{
Group: pointer.To(gatewayv1.Group("gateway.networking.k8s.io")),
Kind: "TLSRoute",
},
},
},
},
{
Name: "konnectivity-server",
Port: 8132,
Protocol: gatewayv1.TLSProtocolType,
Hostname: pointer.To(gatewayv1.Hostname(hostname)),
TLS: &gatewayv1.ListenerTLSConfig{
Mode: pointer.To(gatewayv1.TLSModeType("Passthrough")),
},
AllowedRoutes: &gatewayv1.AllowedRoutes{
Namespaces: &gatewayv1.RouteNamespaces{
From: pointer.To(gatewayv1.NamespacesFromAll),
},
Kinds: []gatewayv1.RouteGroupKind{
{
Group: pointer.To(gatewayv1.Group("gateway.networking.k8s.io")),
Kind: "TLSRoute",
},
},
},
},
},
},
}
Expect(k8sClient.Create(context.Background(), gateway)).NotTo(HaveOccurred())
}

View File

@@ -59,7 +59,7 @@ var _ = Describe("starting a kind worker with kubeadm", func() {
Port: 31443,
},
Kubernetes: kamajiv1alpha1.KubernetesSpec{
Version: "v1.29.0",
Version: "v1.35.0",
Kubelet: kamajiv1alpha1.KubeletSpec{
CGroupFS: "cgroupfs",
},

168
go.mod
View File

@@ -1,12 +1,13 @@
module github.com/clastix/kamaji
go 1.24.1
go 1.25.0
require (
github.com/JamesStewy/go-mysqldump v0.2.2
github.com/blang/semver v3.5.1+incompatible
github.com/clastix/kamaji-telemetry v1.0.0
github.com/docker/docker v28.5.2+incompatible
github.com/evanphx/json-patch/v5 v5.9.11
github.com/go-logr/logr v1.4.3
github.com/go-pg/pg/v10 v10.15.0
github.com/go-sql-driver/mysql v1.9.3
@@ -14,29 +15,30 @@ require (
github.com/google/uuid v1.6.0
github.com/json-iterator/go v1.1.12
github.com/juju/mutex/v2 v2.0.0
github.com/nats-io/nats.go v1.47.0
github.com/onsi/ginkgo/v2 v2.27.2
github.com/onsi/gomega v1.38.2
github.com/pkg/errors v0.9.1
github.com/nats-io/nats.go v1.48.0
github.com/onsi/ginkgo/v2 v2.28.1
github.com/onsi/gomega v1.39.1
github.com/prometheus/client_golang v1.23.2
github.com/spf13/cobra v1.10.1
github.com/prometheus/client_model v0.6.2
github.com/spf13/cobra v1.10.2
github.com/spf13/pflag v1.0.10
github.com/spf13/viper v1.21.0
github.com/testcontainers/testcontainers-go v0.40.0
go.etcd.io/etcd/api/v3 v3.6.6
go.etcd.io/etcd/client/v3 v3.6.6
go.etcd.io/etcd/api/v3 v3.6.7
go.etcd.io/etcd/client/v3 v3.6.7
go.uber.org/automaxprocs v1.6.0
gomodules.xyz/jsonpatch/v2 v2.5.0
k8s.io/api v0.34.1
k8s.io/apimachinery v0.34.1
k8s.io/apiserver v0.34.1
k8s.io/client-go v0.34.1
k8s.io/api v0.35.0
k8s.io/apiextensions-apiserver v0.34.1
k8s.io/apimachinery v0.35.0
k8s.io/client-go v0.35.0
k8s.io/cluster-bootstrap v0.0.0
k8s.io/klog/v2 v2.130.1
k8s.io/kubelet v0.0.0
k8s.io/kubernetes v1.34.2
k8s.io/utils v0.0.0-20250604170112-4c0f3b243397
k8s.io/kubernetes v1.35.0
k8s.io/utils v0.0.0-20251002143259-bc988d571ff4
sigs.k8s.io/controller-runtime v0.22.4
sigs.k8s.io/gateway-api v1.4.1
)
require (
@@ -58,7 +60,7 @@ require (
github.com/containerd/log v0.1.0 // indirect
github.com/containerd/platforms v0.2.1 // indirect
github.com/coredns/caddy v1.1.1 // indirect
github.com/coredns/corefile-migration v1.0.26 // indirect
github.com/coredns/corefile-migration v1.0.29 // indirect
github.com/coreos/go-semver v0.3.1 // indirect
github.com/coreos/go-systemd/v22 v22.5.0 // indirect
github.com/cpuguy83/dockercfg v0.3.2 // indirect
@@ -67,9 +69,8 @@ require (
github.com/docker/go-connections v0.6.0 // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/ebitengine/purego v0.8.4 // indirect
github.com/emicklei/go-restful/v3 v3.12.2 // indirect
github.com/evanphx/json-patch v4.12.0+incompatible // indirect
github.com/evanphx/json-patch/v5 v5.9.11 // indirect
github.com/emicklei/go-restful/v3 v3.13.0 // indirect
github.com/evanphx/json-patch v5.7.0+incompatible // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/fsnotify/fsnotify v1.9.0 // indirect
github.com/fxamacker/cbor/v2 v2.9.0 // indirect
@@ -77,9 +78,9 @@ require (
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-logr/zapr v1.3.0 // indirect
github.com/go-ole/go-ole v1.2.6 // indirect
github.com/go-openapi/jsonpointer v0.21.0 // indirect
github.com/go-openapi/jsonreference v0.20.2 // indirect
github.com/go-openapi/swag v0.23.0 // indirect
github.com/go-openapi/jsonpointer v0.21.2 // indirect
github.com/go-openapi/jsonreference v0.21.0 // indirect
github.com/go-openapi/swag v0.23.1 // indirect
github.com/go-pg/zerochecker v0.2.0 // indirect
github.com/go-task/slim-sprig/v3 v3.0.0 // indirect
github.com/go-viper/mapstructure/v2 v2.4.0 // indirect
@@ -89,7 +90,7 @@ require (
github.com/google/btree v1.1.3 // indirect
github.com/google/cel-go v0.26.0 // indirect
github.com/google/gnostic-models v0.7.0 // indirect
github.com/google/pprof v0.0.0-20250820193118-f64d9cf942d6 // indirect
github.com/google/pprof v0.0.0-20260115054156-294ebfa9ad83 // indirect
github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79 // indirect
github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.3 // indirect
@@ -103,7 +104,7 @@ require (
github.com/lithammer/dedent v1.1.0 // indirect
github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 // indirect
github.com/magiconair/properties v1.8.10 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/mailru/easyjson v0.9.0 // indirect
github.com/moby/docker-image-spec v1.3.1 // indirect
github.com/moby/go-archive v0.1.0 // indirect
github.com/moby/patternmatcher v0.6.0 // indirect
@@ -122,11 +123,12 @@ require (
github.com/opencontainers/image-spec v1.1.1 // indirect
github.com/pelletier/go-toml/v2 v2.2.4 // indirect
github.com/peterbourgon/diskv v2.0.1+incompatible // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c // indirect
github.com/prometheus/client_model v0.6.2 // indirect
github.com/prometheus/common v0.66.1 // indirect
github.com/prometheus/procfs v0.16.1 // indirect
github.com/prometheus/procfs v0.17.0 // indirect
github.com/robfig/cron/v3 v3.0.1 // indirect
github.com/sagikazarmark/locafero v0.11.0 // indirect
github.com/shirou/gopsutil/v4 v4.25.6 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
@@ -146,56 +148,56 @@ require (
github.com/x448/float16 v0.8.4 // indirect
github.com/xlab/treeprint v1.2.0 // indirect
github.com/yusufpapurcu/wmi v1.2.4 // indirect
go.etcd.io/etcd/client/pkg/v3 v3.6.6 // indirect
go.etcd.io/etcd/client/pkg/v3 v3.6.7 // indirect
go.opentelemetry.io/auto/sdk v1.1.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.60.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.58.0 // indirect
go.opentelemetry.io/otel v1.35.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 // indirect
go.opentelemetry.io/otel v1.37.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.34.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.34.0 // indirect
go.opentelemetry.io/otel/metric v1.35.0 // indirect
go.opentelemetry.io/otel/sdk v1.34.0 // indirect
go.opentelemetry.io/otel/trace v1.35.0 // indirect
go.opentelemetry.io/otel/metric v1.37.0 // indirect
go.opentelemetry.io/otel/sdk v1.37.0 // indirect
go.opentelemetry.io/otel/trace v1.37.0 // indirect
go.opentelemetry.io/proto/otlp v1.5.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.uber.org/zap v1.27.0 // indirect
go.yaml.in/yaml/v2 v2.4.2 // indirect
go.yaml.in/yaml/v2 v2.4.3 // indirect
go.yaml.in/yaml/v3 v3.0.4 // indirect
golang.org/x/crypto v0.43.0 // indirect
golang.org/x/crypto v0.47.0 // indirect
golang.org/x/exp v0.0.0-20240719175910-8a7402abbf56 // indirect
golang.org/x/mod v0.28.0 // indirect
golang.org/x/net v0.45.0 // indirect
golang.org/x/mod v0.32.0 // indirect
golang.org/x/net v0.49.0 // indirect
golang.org/x/oauth2 v0.30.0 // indirect
golang.org/x/sync v0.17.0 // indirect
golang.org/x/sys v0.37.0 // indirect
golang.org/x/term v0.36.0 // indirect
golang.org/x/text v0.30.0 // indirect
golang.org/x/time v0.9.0 // indirect
golang.org/x/tools v0.37.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20250303144028-a0af3efb3deb // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250303144028-a0af3efb3deb // indirect
google.golang.org/grpc v1.72.1 // indirect
golang.org/x/sync v0.19.0 // indirect
golang.org/x/sys v0.40.0 // indirect
golang.org/x/term v0.39.0 // indirect
golang.org/x/text v0.33.0 // indirect
golang.org/x/time v0.12.0 // indirect
golang.org/x/tools v0.41.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20250707201910-8d1bb00bc6a7 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250826171959-ef028d996bc1 // indirect
google.golang.org/grpc v1.75.1 // indirect
google.golang.org/protobuf v1.36.8 // indirect
gopkg.in/evanphx/json-patch.v4 v4.12.0 // indirect
gopkg.in/evanphx/json-patch.v4 v4.13.0 // indirect
gopkg.in/go-jose/go-jose.v2 v2.6.3 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/natefinch/lumberjack.v2 v2.2.1 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
k8s.io/apiextensions-apiserver v0.34.1 // indirect
k8s.io/apiserver v0.35.0 // indirect
k8s.io/cli-runtime v0.0.0 // indirect
k8s.io/cloud-provider v0.0.0 // indirect
k8s.io/component-base v0.34.1 // indirect
k8s.io/component-helpers v0.34.0 // indirect
k8s.io/controller-manager v0.34.0 // indirect
k8s.io/cri-api v0.34.0 // indirect
k8s.io/component-base v0.35.0 // indirect
k8s.io/component-helpers v0.35.0 // indirect
k8s.io/controller-manager v0.35.0 // indirect
k8s.io/cri-api v0.35.0 // indirect
k8s.io/cri-client v0.0.0 // indirect
k8s.io/kms v0.34.0 // indirect
k8s.io/kube-openapi v0.0.0-20250710124328-f3f2b991d03b // indirect
k8s.io/kms v0.35.0 // indirect
k8s.io/kube-openapi v0.0.0-20250910181357-589584f1c912 // indirect
k8s.io/kube-proxy v0.0.0 // indirect
k8s.io/system-validators v1.10.2 // indirect
k8s.io/system-validators v1.12.1 // indirect
mellium.im/sasl v0.3.1 // indirect
sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.31.2 // indirect
sigs.k8s.io/json v0.0.0-20241014173422-cfa47c3a1cc8 // indirect
sigs.k8s.io/json v0.0.0-20250730193827-2d320260d730 // indirect
sigs.k8s.io/kustomize/api v0.20.1 // indirect
sigs.k8s.io/kustomize/kyaml v0.20.1 // indirect
sigs.k8s.io/randfill v1.0.0 // indirect
@@ -204,35 +206,35 @@ require (
)
replace (
k8s.io/api => k8s.io/api v0.34.0
k8s.io/apiextensions-apiserver => k8s.io/apiextensions-apiserver v0.34.0
k8s.io/apimachinery => k8s.io/apimachinery v0.34.0
k8s.io/apiserver => k8s.io/apiserver v0.34.0
k8s.io/cli-runtime => k8s.io/cli-runtime v0.34.0
k8s.io/client-go => k8s.io/client-go v0.34.0
k8s.io/cloud-provider => k8s.io/cloud-provider v0.34.0
k8s.io/cluster-bootstrap => k8s.io/cluster-bootstrap v0.34.0
k8s.io/code-generator => k8s.io/code-generator v0.34.0
k8s.io/component-base => k8s.io/component-base v0.34.0
k8s.io/component-helpers => k8s.io/component-helpers v0.34.0
k8s.io/controller-manager => k8s.io/controller-manager v0.34.0
k8s.io/cri-api => k8s.io/cri-api v0.34.0
k8s.io/cri-client => k8s.io/cri-client v0.34.0
k8s.io/csi-translation-lib => k8s.io/csi-translation-lib v0.34.0
k8s.io/dynamic-resource-allocation => k8s.io/dynamic-resource-allocation v0.34.0
k8s.io/endpointslice => k8s.io/endpointslice v0.34.0
k8s.io/externaljwt => k8s.io/externaljwt v0.34.0
k8s.io/kube-aggregator => k8s.io/kube-aggregator v0.34.0
k8s.io/kube-controller-manager => k8s.io/kube-controller-manager v0.34.0
k8s.io/kube-proxy => k8s.io/kube-proxy v0.34.0
k8s.io/kube-scheduler => k8s.io/kube-scheduler v0.34.0
k8s.io/kubectl => k8s.io/kubectl v0.34.0
k8s.io/kubelet => k8s.io/kubelet v0.34.0
k8s.io/legacy-cloud-providers => k8s.io/legacy-cloud-providers v0.34.0
k8s.io/metrics => k8s.io/metrics v0.34.0
k8s.io/mount-utils => k8s.io/mount-utils v0.34.0
k8s.io/pod-security-admission => k8s.io/pod-security-admission v0.34.0
k8s.io/sample-apiserver => k8s.io/sample-apiserver v0.34.0
k8s.io/api => k8s.io/api v0.35.0
k8s.io/apiextensions-apiserver => k8s.io/apiextensions-apiserver v0.35.0
k8s.io/apimachinery => k8s.io/apimachinery v0.35.0
k8s.io/apiserver => k8s.io/apiserver v0.35.0
k8s.io/cli-runtime => k8s.io/cli-runtime v0.35.0
k8s.io/client-go => k8s.io/client-go v0.35.0
k8s.io/cloud-provider => k8s.io/cloud-provider v0.35.0
k8s.io/cluster-bootstrap => k8s.io/cluster-bootstrap v0.35.0
k8s.io/code-generator => k8s.io/code-generator v0.35.0
k8s.io/component-base => k8s.io/component-base v0.35.0
k8s.io/component-helpers => k8s.io/component-helpers v0.35.0
k8s.io/controller-manager => k8s.io/controller-manager v0.35.0
k8s.io/cri-api => k8s.io/cri-api v0.35.0
k8s.io/cri-client => k8s.io/cri-client v0.35.0
k8s.io/csi-translation-lib => k8s.io/csi-translation-lib v0.35.0
k8s.io/dynamic-resource-allocation => k8s.io/dynamic-resource-allocation v0.35.0
k8s.io/endpointslice => k8s.io/endpointslice v0.35.0
k8s.io/externaljwt => k8s.io/externaljwt v0.35.0
k8s.io/kube-aggregator => k8s.io/kube-aggregator v0.35.0
k8s.io/kube-controller-manager => k8s.io/kube-controller-manager v0.35.0
k8s.io/kube-proxy => k8s.io/kube-proxy v0.35.0
k8s.io/kube-scheduler => k8s.io/kube-scheduler v0.35.0
k8s.io/kubectl => k8s.io/kubectl v0.35.0
k8s.io/kubelet => k8s.io/kubelet v0.35.0
k8s.io/legacy-cloud-providers => k8s.io/legacy-cloud-providers v0.35.0
k8s.io/metrics => k8s.io/metrics v0.35.0
k8s.io/mount-utils => k8s.io/mount-utils v0.35.0
k8s.io/pod-security-admission => k8s.io/pod-security-admission v0.35.0
k8s.io/sample-apiserver => k8s.io/sample-apiserver v0.35.0
)
replace github.com/JamesStewy/go-mysqldump => github.com/vtoma/go-mysqldump v1.0.0

250
go.sum
View File

@@ -40,8 +40,8 @@ github.com/containerd/platforms v0.2.1 h1:zvwtM3rz2YHPQsF2CHYM8+KtB5dvhISiXh5ZpS
github.com/containerd/platforms v0.2.1/go.mod h1:XHCb+2/hzowdiut9rkudds9bE5yJ7npe7dG/wG+uFPw=
github.com/coredns/caddy v1.1.1 h1:2eYKZT7i6yxIfGP3qLJoJ7HAsDJqYB+X68g4NYjSrE0=
github.com/coredns/caddy v1.1.1/go.mod h1:A6ntJQlAWuQfFlsd9hvigKbo2WS0VUs2l1e2F+BawD4=
github.com/coredns/corefile-migration v1.0.26 h1:xiiEkVB1Dwolb24pkeDUDBfygV9/XsOSq79yFCrhptY=
github.com/coredns/corefile-migration v1.0.26/go.mod h1:56DPqONc3njpVPsdilEnfijCwNGC3/kTJLl7i7SPavY=
github.com/coredns/corefile-migration v1.0.29 h1:g4cPYMXXDDs9uLE2gFYrJaPBuUAR07eEMGyh9JBE13w=
github.com/coredns/corefile-migration v1.0.29/go.mod h1:56DPqONc3njpVPsdilEnfijCwNGC3/kTJLl7i7SPavY=
github.com/coreos/go-semver v0.3.1 h1:yi21YpKnrx1gt5R+la8n5WgS0kCrsPp33dmEyHReZr4=
github.com/coreos/go-semver v0.3.1/go.mod h1:irMmmIw/7yzSRPWryHsK7EYSg09caPQL03VsM8rvUec=
github.com/coreos/go-systemd/v22 v22.5.0 h1:RrqgGjYQKalulkV8NGVIfkXQf6YYmOyiJKk8iXXhfZs=
@@ -49,7 +49,6 @@ github.com/coreos/go-systemd/v22 v22.5.0/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSV
github.com/cpuguy83/dockercfg v0.3.2 h1:DlJTyZGBDlXqUZ2Dk2Q3xHs/FtnooJJVaad2S9GKorA=
github.com/cpuguy83/dockercfg v0.3.2/go.mod h1:sugsbF4//dDlL/i+S+rtpIWp+5h0BHJHfjj5/jFyUJc=
github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/creack/pty v1.1.18 h1:n56/Zwd5o6whRC5PMGretI4IdRLlmBXYNjScPaBgsbY=
github.com/creack/pty v1.1.18/go.mod h1:MOBLtS5ELjhRRrroQr9kyvTxUAFNvYEK993ew/Vr4O4=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
@@ -68,10 +67,10 @@ github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkp
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/ebitengine/purego v0.8.4 h1:CF7LEKg5FFOsASUj0+QwaXf8Ht6TlFxg09+S9wz0omw=
github.com/ebitengine/purego v0.8.4/go.mod h1:iIjxzd6CiRiOG0UyXP+V1+jWqUXVjPKLAI0mRfJZTmQ=
github.com/emicklei/go-restful/v3 v3.12.2 h1:DhwDP0vY3k8ZzE0RunuJy8GhNpPL6zqLkDf9B/a0/xU=
github.com/emicklei/go-restful/v3 v3.12.2/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc=
github.com/evanphx/json-patch v4.12.0+incompatible h1:4onqiflcdA9EOZ4RxV643DvftH5pOlLGNtQ5lPWQu84=
github.com/evanphx/json-patch v4.12.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk=
github.com/emicklei/go-restful/v3 v3.13.0 h1:C4Bl2xDndpU6nJ4bc1jXd+uTmYPVUwkD6bFY/oTyCes=
github.com/emicklei/go-restful/v3 v3.13.0/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc=
github.com/evanphx/json-patch v5.7.0+incompatible h1:vgGkfT/9f8zE6tvSCe74nfpAVDQ2tG6yudJd8LBksgI=
github.com/evanphx/json-patch v5.7.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk=
github.com/evanphx/json-patch/v5 v5.9.11 h1:/8HVnzMq13/3x9TPvjG08wUGqBTmZBsCWzjTM0wiaDU=
github.com/evanphx/json-patch/v5 v5.9.11/go.mod h1:3j+LviiESTElxA4p3EMKAB9HXj3/XEtnUf6OZxqIQTM=
github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
@@ -100,14 +99,12 @@ github.com/go-logr/zapr v1.3.0 h1:XGdV8XW8zdwFiwOA2Dryh1gj2KRQyOOoNmBy4EplIcQ=
github.com/go-logr/zapr v1.3.0/go.mod h1:YKepepNBd1u/oyhd/yQmtjVXmm9uML4IXUgMOwR8/Gg=
github.com/go-ole/go-ole v1.2.6 h1:/Fpf6oFPoeFik9ty7siob0G6Ke8QvQEuVcuChpwXzpY=
github.com/go-ole/go-ole v1.2.6/go.mod h1:pprOEPIfldk/42T2oK7lQ4v4JSDwmV0As9GaiUsvbm0=
github.com/go-openapi/jsonpointer v0.19.6/go.mod h1:osyAmYz/mB/C3I+WsTTSgw1ONzaLJoLCyoi6/zppojs=
github.com/go-openapi/jsonpointer v0.21.0 h1:YgdVicSA9vH5RiHs9TZW5oyafXZFc6+2Vc1rr/O9oNQ=
github.com/go-openapi/jsonpointer v0.21.0/go.mod h1:IUyH9l/+uyhIYQ/PXVA41Rexl+kOkAPDdXEYns6fzUY=
github.com/go-openapi/jsonreference v0.20.2 h1:3sVjiK66+uXK/6oQ8xgcRKcFgQ5KXa2KvnJRumpMGbE=
github.com/go-openapi/jsonreference v0.20.2/go.mod h1:Bl1zwGIM8/wsvqjsOQLJ/SH+En5Ap4rVB5KVcIDZG2k=
github.com/go-openapi/swag v0.22.3/go.mod h1:UzaqsxGiab7freDnrUUra0MwWfN/q7tE4j+VcZ0yl14=
github.com/go-openapi/swag v0.23.0 h1:vsEVJDUo2hPJ2tu0/Xc+4noaxyEffXNIs3cOULZ+GrE=
github.com/go-openapi/swag v0.23.0/go.mod h1:esZ8ITTYEsH1V2trKHjAN8Ai7xHb8RV+YSZ577vPjgQ=
github.com/go-openapi/jsonpointer v0.21.2 h1:AqQaNADVwq/VnkCmQg6ogE+M3FOsKTytwges0JdwVuA=
github.com/go-openapi/jsonpointer v0.21.2/go.mod h1:50I1STOfbY1ycR8jGz8DaMeLCdXiI6aDteEdRNNzpdk=
github.com/go-openapi/jsonreference v0.21.0 h1:Rs+Y7hSXT83Jacb7kFyjn4ijOuVGSvOdF2+tg1TRrwQ=
github.com/go-openapi/jsonreference v0.21.0/go.mod h1:LmZmgsrTkVg9LG4EaHeY8cBDslNPMo06cago5JNLkm4=
github.com/go-openapi/swag v0.23.1 h1:lpsStH0n2ittzTnbaSloVZLuB5+fvSY/+hnagBjSNZU=
github.com/go-openapi/swag v0.23.1/go.mod h1:STZs8TbRvEQQKUA+JZNAm3EWlgaOBGpyFDqQnDHMef0=
github.com/go-pg/pg/v10 v10.15.0 h1:6DQwbaxJz/e4wvgzbxBkBLiL/Uuk87MGgHhkURtzx24=
github.com/go-pg/pg/v10 v10.15.0/go.mod h1:FIn/x04hahOf9ywQ1p68rXqaDVbTRLYlu4MQR0lhoB8=
github.com/go-pg/zerochecker v0.2.0 h1:pp7f72c3DobMWOb2ErtZsnrPaSvHd2W4o9//8HtF4mU=
@@ -141,8 +138,8 @@ github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/gofuzz v1.2.0 h1:xRy4A+RhZaiKjJ1bPfwQ8sedCA+YS2YcCHW6ec7JMi0=
github.com/google/gofuzz v1.2.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/pprof v0.0.0-20250820193118-f64d9cf942d6 h1:EEHtgt9IwisQ2AZ4pIsMjahcegHh6rmhqxzIRQIyepY=
github.com/google/pprof v0.0.0-20250820193118-f64d9cf942d6/go.mod h1:I6V7YzU0XDpsHqbsyrghnFZLO1gwK6NPTNvmetQIk9U=
github.com/google/pprof v0.0.0-20260115054156-294ebfa9ad83 h1:z2ogiKUYzX5Is6zr/vP9vJGqPwcdqsWjOt+V8J7+bTc=
github.com/google/pprof v0.0.0-20260115054156-294ebfa9ad83/go.mod h1:MxpfABSjhmINe3F1It9d+8exIHFvUqtLIRCdOGNXqiI=
github.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
@@ -196,7 +193,6 @@ github.com/kisielk/sqlstruct v0.0.0-20201105191214-5f3e10d3ab46/go.mod h1:yyMNCy
github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo=
github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
@@ -213,8 +209,8 @@ github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 h1:6E+4a0GO5zZEnZ
github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0/go.mod h1:zJYVVT2jmtg6P3p1VtQj7WsuWi/y4VnjVBn7F8KPB3I=
github.com/magiconair/properties v1.8.10 h1:s31yESBquKXCV9a/ScB3ESkOjUYYv+X0rg8SYxI99mE=
github.com/magiconair/properties v1.8.10/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0=
github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0=
github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=
github.com/mailru/easyjson v0.9.0 h1:PrnmzHw7262yW8sTBwxi1PdJA3Iw/EKBa8psRf7d9a4=
github.com/mailru/easyjson v0.9.0/go.mod h1:1+xMtQp2MRNVL/V1bOzuP3aP8VNwRW55fQUto+XFtTU=
github.com/maruel/natural v1.1.1 h1:Hja7XhhmvEFhcByqDoHz9QZbkWey+COd9xWfCfn1ioo=
github.com/maruel/natural v1.1.1/go.mod h1:v+Rfd79xlw1AgVBjbO0BEQmptqb5HvL/k9GRHB7ZKEg=
github.com/mfridman/tparse v0.18.0 h1:wh6dzOKaIwkUGyKgOntDW4liXSo37qg5AXbIhkMV3vE=
@@ -247,8 +243,8 @@ github.com/morikuni/aec v1.0.0 h1:nP9CBfwrvYnBRgY6qfDQkygYDmYwOilePFkwzv4dU8A=
github.com/morikuni/aec v1.0.0/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7PXmsc=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/nats-io/nats.go v1.47.0 h1:YQdADw6J/UfGUd2Oy6tn4Hq6YHxCaJrVKayxxFqYrgM=
github.com/nats-io/nats.go v1.47.0/go.mod h1:iRWIPokVIFbVijxuMQq4y9ttaBTMe0SFdlZfMDd+33g=
github.com/nats-io/nats.go v1.48.0 h1:pSFyXApG+yWU/TgbKCjmm5K4wrHu86231/w84qRVR+U=
github.com/nats-io/nats.go v1.48.0/go.mod h1:iRWIPokVIFbVijxuMQq4y9ttaBTMe0SFdlZfMDd+33g=
github.com/nats-io/nkeys v0.4.11 h1:q44qGV008kYd9W1b1nEBkNzvnWxtRSQ7A8BoqRrcfa0=
github.com/nats-io/nkeys v0.4.11/go.mod h1:szDimtgmfOi9n25JpfIdGw12tZFYXqhGxjhVxsatHVE=
github.com/nats-io/nuid v1.0.1 h1:5iA8DT8V7q8WK2EScv2padNa/rTESc1KdnPw4TC2paw=
@@ -257,10 +253,10 @@ github.com/nxadm/tail v1.4.4 h1:DQuhQpB1tVlglWS2hLQ5OV6B5r8aGxSrPc5Qo6uTN78=
github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A=
github.com/onsi/ginkgo v1.14.2 h1:8mVmC9kjFFmA8H4pKMUhcblgifdkOIXPvbhN1T36q1M=
github.com/onsi/ginkgo v1.14.2/go.mod h1:iSB4RoI2tjJc9BBv4NKIKWKya62Rps+oPG/Lv9klQyY=
github.com/onsi/ginkgo/v2 v2.27.2 h1:LzwLj0b89qtIy6SSASkzlNvX6WktqurSHwkk2ipF/Ns=
github.com/onsi/ginkgo/v2 v2.27.2/go.mod h1:ArE1D/XhNXBXCBkKOLkbsb2c81dQHCRcF5zwn/ykDRo=
github.com/onsi/gomega v1.38.2 h1:eZCjf2xjZAqe+LeWvKb5weQ+NcPwX84kqJ0cZNxok2A=
github.com/onsi/gomega v1.38.2/go.mod h1:W2MJcYxRGV63b418Ai34Ud0hEdTVXq9NW9+Sx6uXf3k=
github.com/onsi/ginkgo/v2 v2.28.1 h1:S4hj+HbZp40fNKuLUQOYLDgZLwNUVn19N3Atb98NCyI=
github.com/onsi/ginkgo/v2 v2.28.1/go.mod h1:CLtbVInNckU3/+gC8LzkGUb9oF+e8W8TdUsxPwvdOgE=
github.com/onsi/gomega v1.39.1 h1:1IJLAad4zjPn2PsnhH70V4DKRFlrCzGBNrNaru+Vf28=
github.com/onsi/gomega v1.39.1/go.mod h1:hL6yVALoTOxeWudERyfppUcZXjMwIMLnuSfruD2lcfg=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
github.com/opencontainers/image-spec v1.1.1 h1:y0fUlFfIZhPF1W537XOLg0/fcx6zcHCJwooC2xJA040=
@@ -284,8 +280,10 @@ github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNw
github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE=
github.com/prometheus/common v0.66.1 h1:h5E0h5/Y8niHc5DlaLlWLArTQI7tMrsfQjHV+d9ZoGs=
github.com/prometheus/common v0.66.1/go.mod h1:gcaUsgf3KfRSwHY4dIMXLPV0K/Wg1oZ8+SbZk/HH/dA=
github.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg=
github.com/prometheus/procfs v0.16.1/go.mod h1:teAbpZRB1iIAJYREa1LsoWUXykVXA1KlTmWl8x/U+Is=
github.com/prometheus/procfs v0.17.0 h1:FuLQ+05u4ZI+SS/w9+BWEM2TXiHKsUQ9TADiRH7DuK0=
github.com/prometheus/procfs v0.17.0/go.mod h1:oPQLaDAMRbA+u8H5Pbfq+dl3VDAvHxMUOVhe0wYB2zw=
github.com/robfig/cron/v3 v3.0.1 h1:WdRxkvbJztn8LMz/QEvLN5sBU+xKpSqwwUO1Pjr4qDs=
github.com/robfig/cron/v3 v3.0.1/go.mod h1:eQICP3HwyT7UooqI/z+Ov+PtYAWygg1TEWWzGIFLtro=
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
@@ -305,8 +303,8 @@ github.com/spf13/afero v1.15.0 h1:b/YBCLWAJdFWJTN9cLhiXXcD7mzKn9Dm86dNnfyQw1I=
github.com/spf13/afero v1.15.0/go.mod h1:NC2ByUVxtQs4b3sIUphxK0NioZnmxgyCrfzeuq8lxMg=
github.com/spf13/cast v1.10.0 h1:h2x0u2shc1QuLHfxi+cTJvs30+ZAHOGRic8uyGTDWxY=
github.com/spf13/cast v1.10.0/go.mod h1:jNfB8QC9IA6ZuY2ZjDp0KtFO2LZZlg4S/7bzP6qqeHo=
github.com/spf13/cobra v1.10.1 h1:lJeBwCfmrnXthfAupyUTzJ/J4Nc1RsHC/mSRU2dll/s=
github.com/spf13/cobra v1.10.1/go.mod h1:7SmJGaTHFVBY0jW4NXGluQoLvhqFQM+6XSKD+P4XaB0=
github.com/spf13/cobra v1.10.2 h1:DMTTonx5m65Ic0GOoRY2c16WCbHxOOw6xxezuLaBpcU=
github.com/spf13/cobra v1.10.2/go.mod h1:7C1pvHqHw5A4vrJfjNwvOdzYu0Gml16OCs2GRiTUUS4=
github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk=
github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
@@ -369,42 +367,42 @@ github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9dec
github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
github.com/yusufpapurcu/wmi v1.2.4 h1:zFUKzehAFReQwLys1b/iSMl+JQGSCSjtVqQn9bBrPo0=
github.com/yusufpapurcu/wmi v1.2.4/go.mod h1:SBZ9tNy3G9/m5Oi98Zks0QjeHVDvuK0qfxQmPyzfmi0=
go.etcd.io/bbolt v1.4.2 h1:IrUHp260R8c+zYx/Tm8QZr04CX+qWS5PGfPdevhdm1I=
go.etcd.io/bbolt v1.4.2/go.mod h1:Is8rSHO/b4f3XigBC0lL0+4FwAQv3HXEEIgFMuKHceM=
go.etcd.io/etcd/api/v3 v3.6.6 h1:mcaMp3+7JawWv69p6QShYWS8cIWUOl32bFLb6qf8pOQ=
go.etcd.io/etcd/api/v3 v3.6.6/go.mod h1:f/om26iXl2wSkcTA1zGQv8reJRSLVdoEBsi4JdfMrx4=
go.etcd.io/etcd/client/pkg/v3 v3.6.6 h1:uoqgzSOv2H9KlIF5O1Lsd8sW+eMLuV6wzE3q5GJGQNs=
go.etcd.io/etcd/client/pkg/v3 v3.6.6/go.mod h1:YngfUVmvsvOJ2rRgStIyHsKtOt9SZI2aBJrZiWJhCbI=
go.etcd.io/etcd/client/v3 v3.6.6 h1:G5z1wMf5B9SNexoxOHUGBaULurOZPIgGPsW6CN492ec=
go.etcd.io/etcd/client/v3 v3.6.6/go.mod h1:36Qv6baQ07znPR3+n7t+Rk5VHEzVYPvFfGmfF4wBHV8=
go.etcd.io/etcd/pkg/v3 v3.6.4 h1:fy8bmXIec1Q35/jRZ0KOes8vuFxbvdN0aAFqmEfJZWA=
go.etcd.io/etcd/pkg/v3 v3.6.4/go.mod h1:kKcYWP8gHuBRcteyv6MXWSN0+bVMnfgqiHueIZnKMtE=
go.etcd.io/etcd/server/v3 v3.6.4 h1:LsCA7CzjVt+8WGrdsnh6RhC0XqCsLkBly3ve5rTxMAU=
go.etcd.io/etcd/server/v3 v3.6.4/go.mod h1:aYCL/h43yiONOv0QIR82kH/2xZ7m+IWYjzRmyQfnCAg=
go.etcd.io/bbolt v1.4.3 h1:dEadXpI6G79deX5prL3QRNP6JB8UxVkqo4UPnHaNXJo=
go.etcd.io/bbolt v1.4.3/go.mod h1:tKQlpPaYCVFctUIgFKFnAlvbmB3tpy1vkTnDWohtc0E=
go.etcd.io/etcd/api/v3 v3.6.7 h1:7BNJ2gQmc3DNM+9cRkv7KkGQDayElg8x3X+tFDYS+E0=
go.etcd.io/etcd/api/v3 v3.6.7/go.mod h1:xJ81TLj9hxrYYEDmXTeKURMeY3qEDN24hqe+q7KhbnI=
go.etcd.io/etcd/client/pkg/v3 v3.6.7 h1:vvzgyozz46q+TyeGBuFzVuI53/yd133CHceNb/AhBVs=
go.etcd.io/etcd/client/pkg/v3 v3.6.7/go.mod h1:2IVulJ3FZ/czIGl9T4lMF1uxzrhRahLqe+hSgy+Kh7Q=
go.etcd.io/etcd/client/v3 v3.6.7 h1:9WqA5RpIBtdMxAy1ukXLAdtg2pAxNqW5NUoO2wQrE6U=
go.etcd.io/etcd/client/v3 v3.6.7/go.mod h1:2XfROY56AXnUqGsvl+6k29wrwsSbEh1lAouQB1vHpeE=
go.etcd.io/etcd/pkg/v3 v3.6.5 h1:byxWB4AqIKI4SBmquZUG1WGtvMfMaorXFoCcFbVeoxM=
go.etcd.io/etcd/pkg/v3 v3.6.5/go.mod h1:uqrXrzmMIJDEy5j00bCqhVLzR5jEJIwDp5wTlLwPGOU=
go.etcd.io/etcd/server/v3 v3.6.5 h1:4RbUb1Bd4y1WkBHmuF+cZII83JNQMuNXzyjwigQ06y0=
go.etcd.io/etcd/server/v3 v3.6.5/go.mod h1:PLuhyVXz8WWRhzXDsl3A3zv/+aK9e4A9lpQkqawIaH0=
go.etcd.io/raft/v3 v3.6.0 h1:5NtvbDVYpnfZWcIHgGRk9DyzkBIXOi8j+DDp1IcnUWQ=
go.etcd.io/raft/v3 v3.6.0/go.mod h1:nLvLevg6+xrVtHUmVaTcTz603gQPHfh7kUAwV6YpfGo=
go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA=
go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A=
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.60.0 h1:x7wzEgXfnzJcHDwStJT+mxOz4etr2EcexjqhBvmoakw=
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.60.0/go.mod h1:rg+RlpR5dKwaS95IyyZqj5Wd4E13lk/msnTS0Xl9lJM=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.58.0 h1:yd02MEjBdJkG3uabWP9apV+OuWRIXGDuJEUJbOHmCFU=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.58.0/go.mod h1:umTcuxiv1n/s/S6/c2AT/g2CQ7u5C59sHDNmfSwgz7Q=
go.opentelemetry.io/otel v1.35.0 h1:xKWKPxrxB6OtMCbmMY021CqC45J+3Onta9MqjhnusiQ=
go.opentelemetry.io/otel v1.35.0/go.mod h1:UEqy8Zp11hpkUrL73gSlELM0DupHoiq72dR+Zqel/+Y=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 h1:F7Jx+6hwnZ41NSFTO5q4LYDtJRXBf2PD0rNBkeB/lus=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0/go.mod h1:UHB22Z8QsdRDrnAtX4PntOl36ajSxcdUMt1sF7Y6E7Q=
go.opentelemetry.io/otel v1.37.0 h1:9zhNfelUvx0KBfu/gb+ZgeAfAgtWrfHJZcAqFC228wQ=
go.opentelemetry.io/otel v1.37.0/go.mod h1:ehE/umFRLnuLa/vSccNq9oS1ErUlkkK71gMcN34UG8I=
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.34.0 h1:OeNbIYk/2C15ckl7glBlOBp5+WlYsOElzTNmiPW/x60=
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.34.0/go.mod h1:7Bept48yIeqxP2OZ9/AqIpYS94h2or0aB4FypJTc8ZM=
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.34.0 h1:tgJ0uaNS4c98WRNUEx5U3aDlrDOI5Rs+1Vifcw4DJ8U=
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.34.0/go.mod h1:U7HYyW0zt/a9x5J1Kjs+r1f/d4ZHnYFclhYY2+YbeoE=
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.19.0 h1:IeMeyr1aBvBiPVYihXIaeIZba6b8E1bYp7lbdxK8CQg=
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.19.0/go.mod h1:oVdCUtjq9MK9BlS7TtucsQwUcXcymNiEDjgDD2jMtZU=
go.opentelemetry.io/otel/metric v1.35.0 h1:0znxYu2SNyuMSQT4Y9WDWej0VpcsxkuklLa4/siN90M=
go.opentelemetry.io/otel/metric v1.35.0/go.mod h1:nKVFgxBZ2fReX6IlyW28MgZojkoAkJGaE8CpgeAU3oE=
go.opentelemetry.io/otel/sdk v1.34.0 h1:95zS4k/2GOy069d321O8jWgYsW3MzVV+KuSPKp7Wr1A=
go.opentelemetry.io/otel/sdk v1.34.0/go.mod h1:0e/pNiaMAqaykJGKbi+tSjWfNNHMTxoC9qANsCzbyxU=
go.opentelemetry.io/otel/sdk/metric v1.34.0 h1:5CeK9ujjbFVL5c1PhLuStg1wxA7vQv7ce1EK0Gyvahk=
go.opentelemetry.io/otel/sdk/metric v1.34.0/go.mod h1:jQ/r8Ze28zRKoNRdkjCZxfs6YvBTG1+YIqyFVFYec5w=
go.opentelemetry.io/otel/trace v1.35.0 h1:dPpEfJu1sDIqruz7BHFG3c7528f6ddfSWfFDVt/xgMs=
go.opentelemetry.io/otel/trace v1.35.0/go.mod h1:WUk7DtFp1Aw2MkvqGdwiXYDZZNvA/1J8o6xRXLrIkyc=
go.opentelemetry.io/otel/metric v1.37.0 h1:mvwbQS5m0tbmqML4NqK+e3aDiO02vsf/WgbsdpcPoZE=
go.opentelemetry.io/otel/metric v1.37.0/go.mod h1:04wGrZurHYKOc+RKeye86GwKiTb9FKm1WHtO+4EVr2E=
go.opentelemetry.io/otel/sdk v1.37.0 h1:ItB0QUqnjesGRvNcmAcU0LyvkVyGJ2xftD29bWdDvKI=
go.opentelemetry.io/otel/sdk v1.37.0/go.mod h1:VredYzxUvuo2q3WRcDnKDjbdvmO0sCzOvVAiY+yUkAg=
go.opentelemetry.io/otel/sdk/metric v1.37.0 h1:90lI228XrB9jCMuSdA0673aubgRobVZFhbjxHHspCPc=
go.opentelemetry.io/otel/sdk/metric v1.37.0/go.mod h1:cNen4ZWfiD37l5NhS+Keb5RXVWZWpRE+9WyVCpbo5ps=
go.opentelemetry.io/otel/trace v1.37.0 h1:HLdcFNbRQBE2imdSEgm/kwqmQj1Or1l/7bW6mxVK7z4=
go.opentelemetry.io/otel/trace v1.37.0/go.mod h1:TlgrlQ+PtQO5XFerSPUYG0JSgGyryXewPGyayAWSBS0=
go.opentelemetry.io/proto/otlp v1.5.0 h1:xJvq7gMzB31/d406fB8U5CBdyQGw4P399D1aQWU/3i4=
go.opentelemetry.io/proto/otlp v1.5.0/go.mod h1:keN8WnHxOy8PG0rQZjJJ5A2ebUoafqWp0eVQ4yIXvJ4=
go.uber.org/automaxprocs v1.6.0 h1:O3y2/QNTOdbF+e/dpXNNW7Rx2hZ4sTIPyybbxyNqTUs=
@@ -415,37 +413,37 @@ go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
go.uber.org/zap v1.27.0 h1:aJMhYGrd5QSmlpLMr2MftRKl7t8J8PTZPA732ud/XR8=
go.uber.org/zap v1.27.0/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=
go.yaml.in/yaml/v2 v2.4.2 h1:DzmwEr2rDGHl7lsFgAHxmNz/1NlQ7xLIrlN2h5d1eGI=
go.yaml.in/yaml/v2 v2.4.2/go.mod h1:081UH+NErpNdqlCXm3TtEran0rJZGxAYx9hb/ELlsPU=
go.yaml.in/yaml/v2 v2.4.3 h1:6gvOSjQoTB3vt1l+CU+tSyi/HOjfOjRLJ4YwYZGwRO0=
go.yaml.in/yaml/v2 v2.4.3/go.mod h1:zSxWcmIDjOzPXpjlTTbAsKokqkDNAVtZO0WOMiT90s8=
go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.43.0 h1:dduJYIi3A3KOfdGOHX8AVZ/jGiyPa3IbBozJ5kNuE04=
golang.org/x/crypto v0.43.0/go.mod h1:BFbav4mRNlXJL4wNeejLpWxB7wMbc79PdRGhWKncxR0=
golang.org/x/crypto v0.47.0 h1:V6e3FRj+n4dbpw86FJ8Fv7XVOql7TEwpHapKoMJ/GO8=
golang.org/x/crypto v0.47.0/go.mod h1:ff3Y9VzzKbwSSEzWqJsJVBnWmRwRSHt/6Op5n9bQc4A=
golang.org/x/exp v0.0.0-20240719175910-8a7402abbf56 h1:2dVuKD2vS7b0QIHQbpyTISPd0LeHDbnYEryqj5Q1ug8=
golang.org/x/exp v0.0.0-20240719175910-8a7402abbf56/go.mod h1:M4RDyNAINzryxdtnbRXRL/OHtkFuWGRjvuhBJpk2IlY=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.28.0 h1:gQBtGhjxykdjY9YhZpSlZIsbnaE2+PgjfLWUQTnoZ1U=
golang.org/x/mod v0.28.0/go.mod h1:yfB/L0NOf/kmEbXjzCPOx1iK1fRutOydrCMsqRhEBxI=
golang.org/x/mod v0.32.0 h1:9F4d3PHLljb6x//jOyokMv3eX+YDeepZSEo3mFJy93c=
golang.org/x/mod v0.32.0/go.mod h1:SgipZ/3h2Ci89DlEtEXWUk/HteuRin+HHhN+WbNhguU=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM=
golang.org/x/net v0.45.0 h1:RLBg5JKixCy82FtLJpeNlVM0nrSqpCRYzVU1n8kj0tM=
golang.org/x/net v0.45.0/go.mod h1:ECOoLqd5U3Lhyeyo/QDCEVQ4sNgYsqvCZ722XogGieY=
golang.org/x/net v0.49.0 h1:eeHFmOGUTtaaPSGNmjBKpbng9MulQsJURQUAfUwY++o=
golang.org/x/net v0.49.0/go.mod h1:/ysNB2EvaqvesRkuLAyjI1ycPZlQHM3q01F02UY/MV8=
golang.org/x/oauth2 v0.30.0 h1:dnDm7JmhM45NNpd8FDDeLhK6FwqbOf4MLCM9zb1BOHI=
golang.org/x/oauth2 v0.30.0/go.mod h1:B++QgG3ZKulg6sRPGD/mqlHQs5rB3Ml9erfeDY7xKlU=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.17.0 h1:l60nONMj9l5drqw6jlhIELNv9I0A4OFgRsG9k2oT9Ug=
golang.org/x/sync v0.17.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
golang.org/x/sync v0.19.0 h1:vV+1eWNmZ5geRlYjzm2adRgW2/mcpevXNg50YZtPCE4=
golang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190916202348-b4ddaad3f8a3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@@ -459,44 +457,46 @@ golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.11.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.27.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.37.0 h1:fdNQudmxPjkdUTPnLn5mdQv7Zwvbvpaxqs831goi9kQ=
golang.org/x/sys v0.37.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/sys v0.40.0 h1:DBZZqJ2Rkml6QMQsZywtnjnnGvHza6BTfYFWY9kjEWQ=
golang.org/x/sys v0.40.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.36.0 h1:zMPR+aF8gfksFprF/Nc/rd1wRS1EI6nDBGyWAvDzx2Q=
golang.org/x/term v0.36.0/go.mod h1:Qu394IJq6V6dCBRgwqshf3mPF85AqzYEzofzRdZkWss=
golang.org/x/term v0.39.0 h1:RclSuaJf32jOqZz74CkPA9qFuVTX7vhLlpfj/IGWlqY=
golang.org/x/term v0.39.0/go.mod h1:yxzUCTP/U+FzoxfdKmLaA0RV1WgE0VY7hXBwKtY/4ww=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.30.0 h1:yznKA/E9zq54KzlzBEAWn1NXSQ8DIp/NYMy88xJjl4k=
golang.org/x/text v0.30.0/go.mod h1:yDdHFIX9t+tORqspjENWgzaCVXgk0yYnYuSZ8UzzBVM=
golang.org/x/time v0.9.0 h1:EsRrnYcQiGH+5FfbgvV4AP7qEZstoyrHB0DzarOQ4ZY=
golang.org/x/time v0.9.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
golang.org/x/text v0.33.0 h1:B3njUFyqtHDUI5jMn1YIr5B0IE2U0qck04r6d4KPAxE=
golang.org/x/text v0.33.0/go.mod h1:LuMebE6+rBincTi9+xWTY8TztLzKHc/9C1uBCG27+q8=
golang.org/x/time v0.12.0 h1:ScB/8o8olJvc+CQPWrK3fPZNfh7qgwCrY0zJmoEQLSE=
golang.org/x/time v0.12.0/go.mod h1:CDIdPxbZBQxdj6cxyCIdrNogrJKMJ7pr37NYpMcMDSg=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.1.1/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.37.0 h1:DVSRzp7FwePZW356yEAChSdNcQo6Nsp+fex1SUW09lE=
golang.org/x/tools v0.37.0/go.mod h1:MBN5QPQtLMHVdvsbtarmTNukZDdgwdwlO5qGacAzF0w=
golang.org/x/tools v0.41.0 h1:a9b8iMweWG+S0OBnlU36rzLp20z1Rp10w+IY2czHTQc=
golang.org/x/tools v0.41.0/go.mod h1:XSY6eDqxVNiYgezAVqqCeihT4j1U2CCsqvH3WhQpnlg=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
gomodules.xyz/jsonpatch/v2 v2.5.0 h1:JELs8RLM12qJGXU4u/TO3V25KW8GreMKl9pdkk14RM0=
gomodules.xyz/jsonpatch/v2 v2.5.0/go.mod h1:AH3dM2RI6uoBZxn3LVrfvJ3E0/9dG4cSrbuBJT4moAY=
google.golang.org/genproto/googleapis/api v0.0.0-20250303144028-a0af3efb3deb h1:p31xT4yrYrSM/G4Sn2+TNUkVhFCbG9y8itM2S6Th950=
google.golang.org/genproto/googleapis/api v0.0.0-20250303144028-a0af3efb3deb/go.mod h1:jbe3Bkdp+Dh2IrslsFCklNhweNTBgSYanP1UXhJDhKg=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250303144028-a0af3efb3deb h1:TLPQVbx1GJ8VKZxz52VAxl1EBgKXXbTiU9Fc5fZeLn4=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250303144028-a0af3efb3deb/go.mod h1:LuRYeWDFV6WOn90g357N17oMCaxpgCnbi/44qJvDn2I=
google.golang.org/grpc v1.72.1 h1:HR03wO6eyZ7lknl75XlxABNVLLFc2PAb6mHlYh756mA=
google.golang.org/grpc v1.72.1/go.mod h1:wH5Aktxcg25y1I3w7H69nHfXdOG3UiadoBtjh3izSDM=
gonum.org/v1/gonum v0.16.0 h1:5+ul4Swaf3ESvrOnidPp4GZbzf0mxVQpDCYUQE7OJfk=
gonum.org/v1/gonum v0.16.0/go.mod h1:fef3am4MQ93R2HHpKnLk4/Tbh/s0+wqD5nfa6Pnwy4E=
google.golang.org/genproto/googleapis/api v0.0.0-20250707201910-8d1bb00bc6a7 h1:FiusG7LWj+4byqhbvmB+Q93B/mOxJLN2DTozDuZm4EU=
google.golang.org/genproto/googleapis/api v0.0.0-20250707201910-8d1bb00bc6a7/go.mod h1:kXqgZtrWaf6qS3jZOCnCH7WYfrvFjkC51bM8fz3RsCA=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250826171959-ef028d996bc1 h1:pmJpJEvT846VzausCQ5d7KreSROcDqmO388w5YbnltA=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250826171959-ef028d996bc1/go.mod h1:GmFNa4BdJZ2a8G+wCe9Bg3wwThLrJun751XstdJt5Og=
google.golang.org/grpc v1.75.1 h1:/ODCNEuf9VghjgO3rqLcfg8fiOP0nSluljWFlDxELLI=
google.golang.org/grpc v1.75.1/go.mod h1:JtPAzKiq4v1xcAB2hydNlWI2RnF85XXcV0mhKXr2ecQ=
google.golang.org/protobuf v1.36.8 h1:xHScyCOEuuwZEc6UtSOvPbAT4zRh0xcNRYekJwfqyMc=
google.golang.org/protobuf v1.36.8/go.mod h1:fuxRtAxBytpl4zzqUh6/eyUujkJdNiuEkXntxiD/uRU=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/evanphx/json-patch.v4 v4.12.0 h1:n6jtcsulIzXPJaxegRbvFNNrZDjbij7ny3gmSPG+6V4=
gopkg.in/evanphx/json-patch.v4 v4.12.0/go.mod h1:p8EYWUEYMpynmqDbY58zCKCFZw8pRWMG4EsWvDvM72M=
gopkg.in/evanphx/json-patch.v4 v4.13.0 h1:czT3CmqEaQ1aanPc5SdlgQrrEIb8w/wwCvWWnfEbYzo=
gopkg.in/evanphx/json-patch.v4 v4.13.0/go.mod h1:p8EYWUEYMpynmqDbY58zCKCFZw8pRWMG4EsWvDvM72M=
gopkg.in/go-jose/go-jose.v2 v2.6.3 h1:nt80fvSDlhKWQgSWyHyy5CfmlQr+asih51R8PTWNKKs=
gopkg.in/go-jose/go-jose.v2 v2.6.3/go.mod h1:zzZDPkNNw/c9IE7Z9jr11mBZQhKQTMzoEEIoEdZlFBI=
gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc=
@@ -513,56 +513,58 @@ gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gotest.tools/v3 v3.5.2 h1:7koQfIKdy+I8UTetycgUqXWSDwpgv193Ka+qRsmBY8Q=
gotest.tools/v3 v3.5.2/go.mod h1:LtdLGcnqToBH83WByAAi/wiwSFCArdFIUV/xxN4pcjA=
k8s.io/api v0.34.0 h1:L+JtP2wDbEYPUeNGbeSa/5GwFtIA662EmT2YSLOkAVE=
k8s.io/api v0.34.0/go.mod h1:YzgkIzOOlhl9uwWCZNqpw6RJy9L2FK4dlJeayUoydug=
k8s.io/apiextensions-apiserver v0.34.0 h1:B3hiB32jV7BcyKcMU5fDaDxk882YrJ1KU+ZSkA9Qxoc=
k8s.io/apiextensions-apiserver v0.34.0/go.mod h1:hLI4GxE1BDBy9adJKxUxCEHBGZtGfIg98Q+JmTD7+g0=
k8s.io/apimachinery v0.34.0 h1:eR1WO5fo0HyoQZt1wdISpFDffnWOvFLOOeJ7MgIv4z0=
k8s.io/apimachinery v0.34.0/go.mod h1:/GwIlEcWuTX9zKIg2mbw0LRFIsXwrfoVxn+ef0X13lw=
k8s.io/apiserver v0.34.0 h1:Z51fw1iGMqN7uJ1kEaynf2Aec1Y774PqU+FVWCFV3Jg=
k8s.io/apiserver v0.34.0/go.mod h1:52ti5YhxAvewmmpVRqlASvaqxt0gKJxvCeW7ZrwgazQ=
k8s.io/cli-runtime v0.34.0 h1:N2/rUlJg6TMEBgtQ3SDRJwa8XyKUizwjlOknT1mB2Cw=
k8s.io/cli-runtime v0.34.0/go.mod h1:t/skRecS73Piv+J+FmWIQA2N2/rDjdYSQzEE67LUUs8=
k8s.io/client-go v0.34.0 h1:YoWv5r7bsBfb0Hs2jh8SOvFbKzzxyNo0nSb0zC19KZo=
k8s.io/client-go v0.34.0/go.mod h1:ozgMnEKXkRjeMvBZdV1AijMHLTh3pbACPvK7zFR+QQY=
k8s.io/cloud-provider v0.34.0 h1:OgrNE+WSgfvDBQf6WS9qFM7Xr37bc0Og5kkL4hyWDmU=
k8s.io/cloud-provider v0.34.0/go.mod h1:JbMa0t6JIGDMLI7Py6bdp9TN6cfuHrWGq+E/X+Ljkmo=
k8s.io/cluster-bootstrap v0.34.0 h1:fWH6cUXbocLYMtWuONVwQ8ayqdEWlyvu25gedMTYTDk=
k8s.io/cluster-bootstrap v0.34.0/go.mod h1:ZpbQwB+CDTYZIjDKM6Hnt081s0xswcFrlhW7mHVNc7k=
k8s.io/component-base v0.34.0 h1:bS8Ua3zlJzapklsB1dZgjEJuJEeHjj8yTu1gxE2zQX8=
k8s.io/component-base v0.34.0/go.mod h1:RSCqUdvIjjrEm81epPcjQ/DS+49fADvGSCkIP3IC6vg=
k8s.io/component-helpers v0.34.0 h1:5T7P9XGMoUy1JDNKzHf0p/upYbeUf8ZaSf9jbx0QlIo=
k8s.io/component-helpers v0.34.0/go.mod h1:kaOyl5tdtnymriYcVZg4uwDBe2d1wlIpXyDkt6sVnt4=
k8s.io/controller-manager v0.34.0 h1:oCHoqS8dcFp7zDSu7HUvTpakq3isSxil3GprGGlJMsE=
k8s.io/controller-manager v0.34.0/go.mod h1:XFto21U+Mm9BT8r/Jd5E4tHCGtwjKAUFOuDcqaj2VK0=
k8s.io/cri-api v0.34.0 h1:erzXelLqzDbNdryR7eVqxmR/1JfQeurE9U+HdKTgSpU=
k8s.io/cri-api v0.34.0/go.mod h1:4qVUjidMg7/Z9YGZpqIDygbkPWkg3mkS1PvOx/kpHTE=
k8s.io/cri-client v0.34.0 h1:tLZro2oYinVKS5CaMtCASLmOacqVlwoHPSs9e7sBFWI=
k8s.io/cri-client v0.34.0/go.mod h1:KkGaUJWMvCdpSTf15Wiqtf3WKl3qjcvkBcMApPCqpxQ=
k8s.io/api v0.35.0 h1:iBAU5LTyBI9vw3L5glmat1njFK34srdLmktWwLTprlY=
k8s.io/api v0.35.0/go.mod h1:AQ0SNTzm4ZAczM03QH42c7l3bih1TbAXYo0DkF8ktnA=
k8s.io/apiextensions-apiserver v0.35.0 h1:3xHk2rTOdWXXJM+RDQZJvdx0yEOgC0FgQ1PlJatA5T4=
k8s.io/apiextensions-apiserver v0.35.0/go.mod h1:E1Ahk9SADaLQ4qtzYFkwUqusXTcaV2uw3l14aqpL2LU=
k8s.io/apimachinery v0.35.0 h1:Z2L3IHvPVv/MJ7xRxHEtk6GoJElaAqDCCU0S6ncYok8=
k8s.io/apimachinery v0.35.0/go.mod h1:jQCgFZFR1F4Ik7hvr2g84RTJSZegBc8yHgFWKn//hns=
k8s.io/apiserver v0.35.0 h1:CUGo5o+7hW9GcAEF3x3usT3fX4f9r8xmgQeCBDaOgX4=
k8s.io/apiserver v0.35.0/go.mod h1:QUy1U4+PrzbJaM3XGu2tQ7U9A4udRRo5cyxkFX0GEds=
k8s.io/cli-runtime v0.35.0 h1:PEJtYS/Zr4p20PfZSLCbY6YvaoLrfByd6THQzPworUE=
k8s.io/cli-runtime v0.35.0/go.mod h1:VBRvHzosVAoVdP3XwUQn1Oqkvaa8facnokNkD7jOTMY=
k8s.io/client-go v0.35.0 h1:IAW0ifFbfQQwQmga0UdoH0yvdqrbwMdq9vIFEhRpxBE=
k8s.io/client-go v0.35.0/go.mod h1:q2E5AAyqcbeLGPdoRB+Nxe3KYTfPce1Dnu1myQdqz9o=
k8s.io/cloud-provider v0.35.0 h1:syiBCQbKh2gho/S1BkIl006Dc44pV8eAtGZmv5NMe7M=
k8s.io/cloud-provider v0.35.0/go.mod h1:7grN+/Nt5Hf7tnSGPT3aErt4K7aQpygyCrGpbrQbzNc=
k8s.io/cluster-bootstrap v0.35.0 h1:VXnil8zw+FikqvytJYLB8wcvjxbUCyqMkiC//k426Y0=
k8s.io/cluster-bootstrap v0.35.0/go.mod h1:X6sjEjVUFSfFNIzJ6VAIuwwh2QiDtsVX1xZgcGX4gD8=
k8s.io/component-base v0.35.0 h1:+yBrOhzri2S1BVqyVSvcM3PtPyx5GUxCK2tinZz1G94=
k8s.io/component-base v0.35.0/go.mod h1:85SCX4UCa6SCFt6p3IKAPej7jSnF3L8EbfSyMZayJR0=
k8s.io/component-helpers v0.35.0 h1:wcXv7HJRksgVjM4VlXJ1CNFBpyDHruRI99RrBtrJceA=
k8s.io/component-helpers v0.35.0/go.mod h1:ahX0m/LTYmu7fL3W8zYiIwnQ/5gT28Ex4o2pymF63Co=
k8s.io/controller-manager v0.35.0 h1:KteodmfVIRzfZ3RDaxhnHb72rswBxEngvdL9vuZOA9A=
k8s.io/controller-manager v0.35.0/go.mod h1:1bVuPNUG6/dpWpevsJpXioS0E0SJnZ7I/Wqc9Awyzm4=
k8s.io/cri-api v0.35.0 h1:fxLSKyJHqbyCSUsg1rW4DRpmjSEM/elZ1GXzYTSLoDQ=
k8s.io/cri-api v0.35.0/go.mod h1:Cnt29u/tYl1Se1cBRL30uSZ/oJ5TaIp4sZm1xDLvcMc=
k8s.io/cri-client v0.35.0 h1:U1K4bteO93yioUS38804ybN+kWaon9zrzVtB37I3fCs=
k8s.io/cri-client v0.35.0/go.mod h1:XG5GkuuSpxvungsJVzW58NyWBoGSQhMMJmE5c66m9N8=
k8s.io/klog/v2 v2.130.1 h1:n9Xl7H1Xvksem4KFG4PYbdQCQxqc/tTUyrgXaOhHSzk=
k8s.io/klog/v2 v2.130.1/go.mod h1:3Jpz1GvMt720eyJH1ckRHK1EDfpxISzJ7I9OYgaDtPE=
k8s.io/kms v0.34.0 h1:u+/rcxQ3Jr7gC9AY5nXuEnBcGEB7ZOIJ9cdLdyHyEjQ=
k8s.io/kms v0.34.0/go.mod h1:s1CFkLG7w9eaTYvctOxosx88fl4spqmixnNpys0JAtM=
k8s.io/kube-openapi v0.0.0-20250710124328-f3f2b991d03b h1:MloQ9/bdJyIu9lb1PzujOPolHyvO06MXG5TUIj2mNAA=
k8s.io/kube-openapi v0.0.0-20250710124328-f3f2b991d03b/go.mod h1:UZ2yyWbFTpuhSbFhv24aGNOdoRdJZgsIObGBUaYVsts=
k8s.io/kube-proxy v0.34.0 h1:gU7MVbJHiXyPX8bXnod4bANtSC7rZSKkkLmM8gUqwT4=
k8s.io/kube-proxy v0.34.0/go.mod h1:tfwI8dCKm5Q0r+aVIbrq/aC36Kk936w2LZu8/rvJzWI=
k8s.io/kubelet v0.34.0 h1:1nZt1Q6Kfx7xCaTS9vnqR9sjZDxf3cRSQkAFCczULmc=
k8s.io/kubelet v0.34.0/go.mod h1:NqbF8ViVettlZbf9hw9DJhubaWn7rGvDDTcLMDm6tQ0=
k8s.io/kubernetes v1.34.2 h1:WQdDvYJazkmkwSncgNwGvVtaCt4TYXIU3wSMRgvp3MI=
k8s.io/kubernetes v1.34.2/go.mod h1:m6pZk6a179pRo2wsTiCPORJ86iOEQmfIzUvtyEF8BwA=
k8s.io/system-validators v1.10.2 h1:7rC7VdrQCaM55E08Pw3I1v1Op9ObLxdKAu5Ff5hIPwY=
k8s.io/system-validators v1.10.2/go.mod h1:awfSS706v9R12VC7u7K89FKfqVy44G+E0L1A0FX9Wmw=
k8s.io/utils v0.0.0-20250604170112-4c0f3b243397 h1:hwvWFiBzdWw1FhfY1FooPn3kzWuJ8tmbZBHi4zVsl1Y=
k8s.io/utils v0.0.0-20250604170112-4c0f3b243397/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0=
k8s.io/kms v0.35.0 h1:/x87FED2kDSo66csKtcYCEHsxF/DBlNl7LfJ1fVQs1o=
k8s.io/kms v0.35.0/go.mod h1:VT+4ekZAdrZDMgShK37vvlyHUVhwI9t/9tvh0AyCWmQ=
k8s.io/kube-openapi v0.0.0-20250910181357-589584f1c912 h1:Y3gxNAuB0OBLImH611+UDZcmKS3g6CthxToOb37KgwE=
k8s.io/kube-openapi v0.0.0-20250910181357-589584f1c912/go.mod h1:kdmbQkyfwUagLfXIad1y2TdrjPFWp2Q89B3qkRwf/pQ=
k8s.io/kube-proxy v0.35.0 h1:erv2wYmGZ6nyu/FtmaIb+ORD3q2rfZ4Fhn7VXs/8cPQ=
k8s.io/kube-proxy v0.35.0/go.mod h1:bd9lpN3uLLOOWc/CFZbkPEi9DTkzQQymbE8FqSU4bWk=
k8s.io/kubelet v0.35.0 h1:8cgJHCBCKLYuuQ7/Pxb/qWbJfX1LXIw7790ce9xHq7c=
k8s.io/kubelet v0.35.0/go.mod h1:ciRzAXn7C4z5iB7FhG1L2CGPPXLTVCABDlbXt/Zz8YA=
k8s.io/kubernetes v1.35.0 h1:PUOojD8c8E3csMP5NX+nLLne6SGqZjrYCscptyBfWMY=
k8s.io/kubernetes v1.35.0/go.mod h1:Tzk9Y9W/XUFFFgTUVg+BAowoFe+Pc7koGLuaiLHdcFg=
k8s.io/system-validators v1.12.1 h1:AY1+COTLJN/Sj0w9QzH1H0yvyF3Kl6CguMnh32WlcUU=
k8s.io/system-validators v1.12.1/go.mod h1:awfSS706v9R12VC7u7K89FKfqVy44G+E0L1A0FX9Wmw=
k8s.io/utils v0.0.0-20251002143259-bc988d571ff4 h1:SjGebBtkBqHFOli+05xYbK8YF1Dzkbzn+gDM4X9T4Ck=
k8s.io/utils v0.0.0-20251002143259-bc988d571ff4/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0=
mellium.im/sasl v0.3.1 h1:wE0LW6g7U83vhvxjC1IY8DnXM+EU095yeo8XClvCdfo=
mellium.im/sasl v0.3.1/go.mod h1:xm59PUYpZHhgQ9ZqoJ5QaCqzWMi8IeS49dhp6plPCzw=
sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.31.2 h1:jpcvIRr3GLoUoEKRkHKSmGjxb6lWwrBlJsXc+eUYQHM=
sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.31.2/go.mod h1:Ve9uj1L+deCXFrPOk1LpFXqTg7LCFzFso6PA48q/XZw=
sigs.k8s.io/controller-runtime v0.22.4 h1:GEjV7KV3TY8e+tJ2LCTxUTanW4z/FmNB7l327UfMq9A=
sigs.k8s.io/controller-runtime v0.22.4/go.mod h1:+QX1XUpTXN4mLoblf4tqr5CQcyHPAki2HLXqQMY6vh8=
sigs.k8s.io/json v0.0.0-20241014173422-cfa47c3a1cc8 h1:gBQPwqORJ8d8/YNZWEjoZs7npUVDpVXUUOFfW6CgAqE=
sigs.k8s.io/json v0.0.0-20241014173422-cfa47c3a1cc8/go.mod h1:mdzfpAEoE6DHQEN0uh9ZbOCuHbLK5wOm7dK4ctXE9Tg=
sigs.k8s.io/gateway-api v1.4.1 h1:NPxFutNkKNa8UfLd2CMlEuhIPMQgDQ6DXNKG9sHbJU8=
sigs.k8s.io/gateway-api v1.4.1/go.mod h1:AR5RSqciWP98OPckEjOjh2XJhAe2Na4LHyXD2FUY7Qk=
sigs.k8s.io/json v0.0.0-20250730193827-2d320260d730 h1:IpInykpT6ceI+QxKBbEflcR5EXP7sU1kvOlxwZh5txg=
sigs.k8s.io/json v0.0.0-20250730193827-2d320260d730/go.mod h1:mdzfpAEoE6DHQEN0uh9ZbOCuHbLK5wOm7dK4ctXE9Tg=
sigs.k8s.io/kustomize/api v0.20.1 h1:iWP1Ydh3/lmldBnH/S5RXgT98vWYMaTUL1ADcr+Sv7I=
sigs.k8s.io/kustomize/api v0.20.1/go.mod h1:t6hUFxO+Ph0VxIk1sKp1WS0dOjbPCtLJ4p8aADLwqjM=
sigs.k8s.io/kustomize/kyaml v0.20.1 h1:PCMnA2mrVbRP3NIB6v9kYCAc38uvFLVs8j/CD567A78=

View File

@@ -12,7 +12,6 @@ import (
"strconv"
"strings"
"github.com/pkg/errors"
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
@@ -52,9 +51,15 @@ const (
kineInitContainerName = "chmod"
)
type DataStoreOverrides struct {
Resource string
DataStore kamajiv1alpha1.DataStore
}
type Deployment struct {
KineContainerImage string
DataStore kamajiv1alpha1.DataStore
DataStoreOverrides []DataStoreOverrides
Client client.Client
}
@@ -711,11 +716,29 @@ func (d Deployment) buildKubeAPIServerCommand(tenantControlPlane kamajiv1alpha1.
desiredArgs["--etcd-keyfile"] = "/etc/kubernetes/pki/etcd/server.key"
}
if len(d.DataStoreOverrides) != 0 {
desiredArgs["--etcd-servers-overrides"] = d.etcdServersOverrides()
}
// Order matters, here: extraArgs could try to overwrite some arguments managed by Kamaji and that would be crucial.
// Adding as first element of the array of maps, we're sure that these overrides will be sanitized by our configuration.
return utilities.MergeMaps(current, desiredArgs, extraArgs)
}
func (d Deployment) etcdServersOverrides() string {
dataStoreOverridesEndpoints := make([]string, 0, len(d.DataStoreOverrides))
for _, dso := range d.DataStoreOverrides {
httpsEndpoints := make([]string, 0, len(dso.DataStore.Spec.Endpoints))
for _, ep := range dso.DataStore.Spec.Endpoints {
httpsEndpoints = append(httpsEndpoints, fmt.Sprintf("https://%s", ep))
}
dataStoreOverridesEndpoints = append(dataStoreOverridesEndpoints, fmt.Sprintf("%s#%s", dso.Resource, strings.Join(httpsEndpoints, ";")))
}
return strings.Join(dataStoreOverridesEndpoints, ",")
}
func (d Deployment) secretProjection(secretName, certKeyName, keyName string) *corev1.SecretProjection {
return &corev1.SecretProjection{
LocalObjectReference: corev1.LocalObjectReference{
@@ -998,7 +1021,7 @@ func (d Deployment) templateLabels(ctx context.Context, tenantControlPlane *kama
func (d Deployment) secretHashValue(ctx context.Context, client client.Client, namespace, name string) (string, error) {
secret := &corev1.Secret{}
if err := client.Get(ctx, types.NamespacedName{Namespace: namespace, Name: name}, secret); err != nil {
return "", errors.Wrap(err, "cannot retrieve *corev1.Secret for resource version retrieval")
return "", fmt.Errorf("cannot retrieve *corev1.Secret for resource version retrieval: %w", err)
}
return d.hashValue(*secret), nil

View File

@@ -0,0 +1,46 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package controlplane
import (
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
)
var _ = Describe("Controlplane Deployment", func() {
var d Deployment
BeforeEach(func() {
d = Deployment{
DataStoreOverrides: []DataStoreOverrides{{
Resource: "/events",
DataStore: kamajiv1alpha1.DataStore{
Spec: kamajiv1alpha1.DataStoreSpec{
Endpoints: kamajiv1alpha1.Endpoints{"etcd-0", "etcd-1", "etcd-2"},
},
},
}},
}
})
Describe("DataStoreOverrides flag generation", func() {
It("should generate valid --etcd-servers-overrides value", func() {
etcdSerVersOverrides := d.etcdServersOverrides()
Expect(etcdSerVersOverrides).To(Equal("/events#https://etcd-0;https://etcd-1;https://etcd-2"))
})
It("should generate valid --etcd-servers-overrides value with 2 DataStoreOverrides", func() {
d.DataStoreOverrides = append(d.DataStoreOverrides, DataStoreOverrides{
Resource: "/pods",
DataStore: kamajiv1alpha1.DataStore{
Spec: kamajiv1alpha1.DataStoreSpec{
Endpoints: kamajiv1alpha1.Endpoints{"etcd-3", "etcd-4", "etcd-5"},
},
},
})
etcdSerVersOverrides := d.etcdServersOverrides()
Expect(etcdSerVersOverrides).To(Equal("/events#https://etcd-0;https://etcd-1;https://etcd-2,/pods#https://etcd-3;https://etcd-4;https://etcd-5"))
})
})
})

View File

@@ -17,7 +17,6 @@ import (
"net"
"time"
"github.com/pkg/errors"
"k8s.io/apimachinery/pkg/util/sets"
)
@@ -93,7 +92,7 @@ func GenerateCertificatePrivateKeyPair(template *x509.Certificate, caCertificate
caPrivKeyBytes, err := ParsePrivateKeyBytes(caPrivateKey)
if err != nil {
return nil, nil, errors.Wrap(err, "provided CA private key for certificate generation cannot be parsed")
return nil, nil, fmt.Errorf("provided CA private key for certificate generation cannot be parsed: %w", err)
}
return generateCertificateKeyPairBytes(template, caCertBytes, caPrivKeyBytes)
@@ -108,7 +107,7 @@ func ParseCertificateBytes(content []byte) (*x509.Certificate, error) {
crt, err := x509.ParseCertificate(pemContent.Bytes)
if err != nil {
return nil, errors.Wrap(err, "cannot parse x509 Certificate")
return nil, fmt.Errorf("cannot parse x509 Certificate: %w", err)
}
return crt, nil
@@ -124,7 +123,7 @@ func ParsePrivateKeyBytes(content []byte) (crypto.Signer, error) {
if pemContent.Type == "EC PRIVATE KEY" {
privateKey, err := x509.ParseECPrivateKey(pemContent.Bytes)
if err != nil {
return nil, errors.Wrap(err, "cannot parse EC Private Key")
return nil, fmt.Errorf("cannot parse EC Private Key: %w", err)
}
return privateKey, nil
@@ -132,7 +131,7 @@ func ParsePrivateKeyBytes(content []byte) (crypto.Signer, error) {
privateKey, err := x509.ParsePKCS1PrivateKey(pemContent.Bytes)
if err != nil {
return nil, errors.Wrap(err, "cannot parse PKCS1 Private Key")
return nil, fmt.Errorf("cannot parse PKCS1 Private Key: %w", err)
}
return privateKey, nil
@@ -209,12 +208,12 @@ func VerifyCertificate(cert, ca []byte, usages ...x509.ExtKeyUsage) (bool, error
func generateCertificateKeyPairBytes(template *x509.Certificate, caCert *x509.Certificate, caKey crypto.Signer) (*bytes.Buffer, *bytes.Buffer, error) {
certPrivKey, err := rsa.GenerateKey(cryptorand.Reader, 2048)
if err != nil {
return nil, nil, errors.Wrap(err, "cannot generate an RSA key")
return nil, nil, fmt.Errorf("cannot generate an RSA key: %w", err)
}
certBytes, err := x509.CreateCertificate(cryptorand.Reader, template, caCert, &certPrivKey.PublicKey, caKey)
if err != nil {
return nil, nil, errors.Wrap(err, "cannot create the certificate")
return nil, nil, fmt.Errorf("cannot create the certificate: %w", err)
}
certPEM := &bytes.Buffer{}
@@ -223,7 +222,7 @@ func generateCertificateKeyPairBytes(template *x509.Certificate, caCert *x509.Ce
Headers: nil,
Bytes: certBytes,
}); err != nil {
return nil, nil, errors.Wrap(err, "cannot encode the generate certificate bytes")
return nil, nil, fmt.Errorf("cannot encode the generate certificate bytes: %w", err)
}
certPrivKeyPEM := &bytes.Buffer{}
@@ -232,7 +231,7 @@ func generateCertificateKeyPairBytes(template *x509.Certificate, caCert *x509.Ce
Headers: nil,
Bytes: x509.MarshalPKCS1PrivateKey(certPrivKey),
}); err != nil {
return nil, nil, errors.Wrap(err, "cannot encode private key")
return nil, nil, fmt.Errorf("cannot encode private key: %w", err)
}
return certPEM, certPrivKeyPEM, nil

View File

@@ -7,7 +7,6 @@ import (
"context"
"fmt"
"github.com/pkg/errors"
"sigs.k8s.io/controller-runtime/pkg/client"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
@@ -16,7 +15,7 @@ import (
func NewStorageConnection(ctx context.Context, client client.Client, ds kamajiv1alpha1.DataStore) (Connection, error) {
cc, err := NewConnectionConfig(ctx, client, ds)
if err != nil {
return nil, errors.Wrap(err, "unable to create connection config object")
return nil, fmt.Errorf("unable to create connection config object: %w", err)
}
switch ds.Spec.Driver {

View File

@@ -11,7 +11,6 @@ import (
"net"
"strconv"
"github.com/pkg/errors"
"sigs.k8s.io/controller-runtime/pkg/client"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
@@ -67,7 +66,7 @@ func NewConnectionConfig(ctx context.Context, client client.Client, ds kamajiv1a
certificate, err := tls.X509KeyPair(crt, key)
if err != nil {
return nil, errors.Wrap(err, "cannot retrieve x.509 key pair from the Kine Secret")
return nil, fmt.Errorf("cannot retrieve x.509 key pair from the Kine Secret: %w", err)
}
tlsConfig.Certificates = []tls.Certificate{certificate}
@@ -93,12 +92,12 @@ func NewConnectionConfig(ctx context.Context, client client.Client, ds kamajiv1a
for _, ep := range ds.Spec.Endpoints {
host, stringPort, err := net.SplitHostPort(ep)
if err != nil {
return nil, errors.Wrap(err, "cannot retrieve host-port pair from DataStore endpoints")
return nil, fmt.Errorf("cannot retrieve host-port pair from DataStore endpoints: %w", err)
}
port, err := strconv.Atoi(stringPort)
if err != nil {
return nil, errors.Wrap(err, "cannot convert port from string for the given DataStore")
return nil, fmt.Errorf("cannot convert port from string for the given DataStore: %w", err)
}
eps = append(eps, ConnectionEndpoint{

View File

@@ -3,48 +3,48 @@
package errors
import "github.com/pkg/errors"
import "fmt"
func NewCreateUserError(err error) error {
return errors.Wrap(err, "cannot create user")
return fmt.Errorf("cannot create user: %w", err)
}
func NewGrantPrivilegesError(err error) error {
return errors.Wrap(err, "cannot grant privileges")
return fmt.Errorf("cannot grant privileges: %w", err)
}
func NewCheckUserExistsError(err error) error {
return errors.Wrap(err, "cannot check if user exists")
return fmt.Errorf("cannot check if user exists: %w", err)
}
func NewCheckGrantExistsError(err error) error {
return errors.Wrap(err, "cannot check if grant exists")
return fmt.Errorf("cannot check if grant exists: %w", err)
}
func NewDeleteUserError(err error) error {
return errors.Wrap(err, "cannot delete user")
return fmt.Errorf("cannot delete user: %w", err)
}
func NewCannotDeleteDatabaseError(err error) error {
return errors.Wrap(err, "cannot delete database")
return fmt.Errorf("cannot delete database: %w", err)
}
func NewCheckDatabaseExistError(err error) error {
return errors.Wrap(err, "cannot check if database exists")
return fmt.Errorf("cannot check if database exists: %w", err)
}
func NewRevokePrivilegesError(err error) error {
return errors.Wrap(err, "cannot revoke privileges")
return fmt.Errorf("cannot revoke privileges: %w", err)
}
func NewCloseConnectionError(err error) error {
return errors.Wrap(err, "cannot close connection")
return fmt.Errorf("cannot close connection: %w", err)
}
func NewCheckConnectionError(err error) error {
return errors.Wrap(err, "cannot check connection")
return fmt.Errorf("cannot check connection: %w", err)
}
func NewCreateDBError(err error) error {
return errors.Wrap(err, "cannot create database")
return fmt.Errorf("cannot create database: %w", err)
}

View File

@@ -7,13 +7,12 @@ import (
"context"
"fmt"
goerrors "github.com/pkg/errors"
"go.etcd.io/etcd/api/v3/authpb"
"go.etcd.io/etcd/api/v3/v3rpc/rpctypes"
etcdclient "go.etcd.io/etcd/client/v3"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
"github.com/clastix/kamaji/internal/datastore/errors"
dserrors "github.com/clastix/kamaji/internal/datastore/errors"
)
func NewETCDConnection(config ConnectionConfig) (Connection, error) {
@@ -44,7 +43,7 @@ type EtcdClient struct {
func (e *EtcdClient) CreateUser(ctx context.Context, user, password string) error {
if _, err := e.Client.Auth.UserAddWithOptions(ctx, user, password, &etcdclient.UserAddOptions{NoPassword: true}); err != nil {
return errors.NewCreateUserError(err)
return dserrors.NewCreateUserError(err)
}
return nil
@@ -56,18 +55,18 @@ func (e *EtcdClient) CreateDB(context.Context, string) error {
func (e *EtcdClient) GrantPrivileges(ctx context.Context, user, dbName string) error {
if _, err := e.Client.Auth.RoleAdd(ctx, dbName); err != nil {
return errors.NewGrantPrivilegesError(err)
return dserrors.NewGrantPrivilegesError(err)
}
permission := etcdclient.PermissionType(authpb.READWRITE)
key := e.buildKey(dbName)
if _, err := e.Client.RoleGrantPermission(ctx, dbName, key, etcdclient.GetPrefixRangeEnd(key), permission); err != nil {
return errors.NewGrantPrivilegesError(err)
return dserrors.NewGrantPrivilegesError(err)
}
if _, err := e.Client.UserGrantRole(ctx, user, dbName); err != nil {
return errors.NewGrantPrivilegesError(err)
return dserrors.NewGrantPrivilegesError(err)
}
return nil
@@ -75,11 +74,15 @@ func (e *EtcdClient) GrantPrivileges(ctx context.Context, user, dbName string) e
func (e *EtcdClient) UserExists(ctx context.Context, user string) (bool, error) {
if _, err := e.Client.UserGet(ctx, user); err != nil {
if goerrors.As(err, &rpctypes.ErrGRPCUserNotFound) {
// Convert gRPC error to comparable EtcdError using rpctypes.Error(),
// then compare against the client-side error constant.
// The == comparison is correct here as rpctypes.Error() normalizes
// gRPC status errors to comparable EtcdError struct values.
if rpctypes.Error(err) == rpctypes.ErrUserNotFound { //nolint:errorlint
return false, nil
}
return false, errors.NewCheckUserExistsError(err)
return false, dserrors.NewCheckUserExistsError(err)
}
return true, nil
@@ -92,16 +95,20 @@ func (e *EtcdClient) DBExists(context.Context, string) (bool, error) {
func (e *EtcdClient) GrantPrivilegesExists(ctx context.Context, username, dbName string) (bool, error) {
_, err := e.Client.RoleGet(ctx, dbName)
if err != nil {
if goerrors.As(err, &rpctypes.ErrGRPCRoleNotFound) {
// Convert gRPC error to comparable EtcdError using rpctypes.Error(),
// then compare against the client-side error constant.
// The == comparison is correct here as rpctypes.Error() normalizes
// gRPC status errors to comparable EtcdError struct values.
if rpctypes.Error(err) == rpctypes.ErrRoleNotFound { //nolint:errorlint
return false, nil
}
return false, errors.NewCheckGrantExistsError(err)
return false, dserrors.NewCheckGrantExistsError(err)
}
user, err := e.Client.UserGet(ctx, username)
if err != nil {
return false, errors.NewCheckGrantExistsError(err)
return false, dserrors.NewCheckGrantExistsError(err)
}
for _, i := range user.Roles {
@@ -115,7 +122,7 @@ func (e *EtcdClient) GrantPrivilegesExists(ctx context.Context, username, dbName
func (e *EtcdClient) DeleteUser(ctx context.Context, user string) error {
if _, err := e.Client.Auth.UserDelete(ctx, user); err != nil {
return errors.NewDeleteUserError(err)
return dserrors.NewDeleteUserError(err)
}
return nil
@@ -124,7 +131,7 @@ func (e *EtcdClient) DeleteUser(ctx context.Context, user string) error {
func (e *EtcdClient) DeleteDB(ctx context.Context, dbName string) error {
prefix := e.buildKey(dbName)
if _, err := e.Client.Delete(ctx, prefix, etcdclient.WithPrefix()); err != nil {
return errors.NewCannotDeleteDatabaseError(err)
return dserrors.NewCannotDeleteDatabaseError(err)
}
return nil
@@ -132,7 +139,7 @@ func (e *EtcdClient) DeleteDB(ctx context.Context, dbName string) error {
func (e *EtcdClient) RevokePrivileges(ctx context.Context, _, dbName string) error {
if _, err := e.Client.Auth.RoleDelete(ctx, dbName); err != nil {
return errors.NewRevokePrivilegesError(err)
return dserrors.NewRevokePrivilegesError(err)
}
return nil
@@ -146,7 +153,7 @@ func (e *EtcdClient) GetConnectionString() string {
func (e *EtcdClient) Close() error {
if err := e.Client.Close(); err != nil {
return errors.NewCloseConnectionError(err)
return dserrors.NewCloseConnectionError(err)
}
return nil
@@ -154,7 +161,7 @@ func (e *EtcdClient) Close() error {
func (e *EtcdClient) Check(ctx context.Context) error {
if _, err := e.Client.AuthStatus(ctx); err != nil {
return errors.NewCheckConnectionError(err)
return dserrors.NewCheckConnectionError(err)
}
return nil

View File

@@ -29,7 +29,7 @@ const (
mysqlShowGrantsStatement = "SHOW GRANTS FOR `%s`@`%%`"
mysqlCreateDBStatement = "CREATE DATABASE IF NOT EXISTS %s"
mysqlCreateUserStatement = "CREATE USER `%s`@`%%` IDENTIFIED BY '%s'"
mysqlGrantPrivilegesStatement = "GRANT ALL PRIVILEGES ON `%s`.* TO `%s`@`%%`"
mysqlGrantPrivilegesStatement = "GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, ALTER, INDEX ON `%s`.* TO `%s`@`%%`"
mysqlDropDBStatement = "DROP DATABASE IF EXISTS `%s`"
mysqlDropUserStatement = "DROP USER IF EXISTS `%s`"
mysqlRevokePrivilegesStatement = "REVOKE ALL PRIVILEGES ON `%s`.* FROM `%s`"
@@ -167,7 +167,7 @@ func (c *MySQLConnection) CreateDB(ctx context.Context, dbName string) error {
}
func (c *MySQLConnection) GrantPrivileges(ctx context.Context, user, dbName string) error {
if err := c.mutate(ctx, mysqlGrantPrivilegesStatement, user, dbName); err != nil {
if err := c.mutate(ctx, mysqlGrantPrivilegesStatement, dbName, user); err != nil {
return errors.NewGrantPrivilegesError(err)
}
@@ -229,7 +229,7 @@ func (c *MySQLConnection) GrantPrivilegesExists(_ context.Context, user, dbName
return false, errors.NewGrantPrivilegesError(err)
}
expected := fmt.Sprintf(mysqlGrantPrivilegesStatement, user, dbName)
expected := fmt.Sprintf(mysqlGrantPrivilegesStatement, dbName, user)
var grant string
for rows.Next() {
@@ -262,7 +262,7 @@ func (c *MySQLConnection) DeleteDB(ctx context.Context, dbName string) error {
}
func (c *MySQLConnection) RevokePrivileges(ctx context.Context, user, dbName string) error {
if err := c.mutate(ctx, mysqlRevokePrivilegesStatement, user, dbName); err != nil {
if err := c.mutate(ctx, mysqlRevokePrivilegesStatement, dbName, user); err != nil {
return errors.NewRevokePrivilegesError(err)
}

View File

@@ -5,11 +5,11 @@ package datastore
import (
"context"
"errors"
"fmt"
"strings"
"github.com/nats-io/nats.go"
"github.com/pkg/errors"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
)
@@ -73,7 +73,7 @@ func (nc *NATSConnection) CreateUser(_ context.Context, _, _ string) error {
func (nc *NATSConnection) CreateDB(_ context.Context, dbName string) error {
_, err := nc.js.CreateKeyValue(&nats.KeyValueConfig{Bucket: dbName})
if err != nil {
return errors.Wrap(err, "unable to create the datastore")
return fmt.Errorf("unable to create the datastore: %w", err)
}
return nil

View File

@@ -10,7 +10,6 @@ import (
"strings"
"github.com/go-pg/pg/v10"
goerrors "github.com/pkg/errors"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
"github.com/clastix/kamaji/internal/datastore/errors"
@@ -18,18 +17,18 @@ import (
const (
postgresqlFetchDBStatement = "SELECT FROM pg_database WHERE datname = ?"
postgresqlCreateDBStatement = "CREATE DATABASE %s"
postgresqlCreateDBStatement = `CREATE DATABASE "%s"`
postgresqlUserExists = "SELECT 1 FROM pg_roles WHERE rolname = ?"
postgresqlCreateUserStatement = "CREATE ROLE %s LOGIN PASSWORD ?"
postgresqlCreateUserStatement = `CREATE ROLE "%s" LOGIN PASSWORD ?`
postgresqlShowGrantsStatement = "SELECT has_database_privilege(rolname, ?, 'create') from pg_roles where rolcanlogin and rolname = ?"
postgresqlShowOwnershipStatement = "SELECT 't' FROM pg_catalog.pg_database AS d WHERE d.datname = ? AND pg_catalog.pg_get_userbyid(d.datdba) = ?"
postgresqlShowTableOwnershipStatement = "SELECT 't' from pg_tables where tableowner = ? AND tablename = ?"
postgresqlKineTableExistsStatement = "SELECT 't' FROM pg_tables WHERE schemaname = ? AND tablename = ?"
postgresqlGrantPrivilegesStatement = "GRANT ALL PRIVILEGES ON DATABASE %s TO %s"
postgresqlChangeOwnerStatement = "ALTER DATABASE %s OWNER TO %s"
postgresqlRevokePrivilegesStatement = "REVOKE ALL PRIVILEGES ON DATABASE %s FROM %s"
postgresqlDropRoleStatement = "DROP ROLE %s"
postgresqlDropDBStatement = "DROP DATABASE %s WITH (FORCE)"
postgresqlGrantPrivilegesStatement = `GRANT CONNECT, CREATE ON DATABASE "%s" TO "%s"`
postgresqlChangeOwnerStatement = `ALTER DATABASE "%s" OWNER TO "%s"`
postgresqlRevokePrivilegesStatement = `REVOKE ALL PRIVILEGES ON DATABASE "%s" FROM "%s"`
postgresqlDropRoleStatement = `DROP ROLE "%s"`
postgresqlDropDBStatement = `DROP DATABASE "%s" WITH (FORCE)`
)
type PostgreSQLConnection struct {
@@ -236,7 +235,7 @@ func (r *PostgreSQLConnection) DeleteUser(ctx context.Context, user string) erro
func (r *PostgreSQLConnection) DeleteDB(ctx context.Context, dbName string) error {
if err := r.GrantPrivileges(ctx, r.rootUser, dbName); err != nil {
return errors.NewCannotDeleteDatabaseError(goerrors.Wrap(err, "cannot grant privileges to root user"))
return errors.NewCannotDeleteDatabaseError(fmt.Errorf("cannot grant privileges to root user: %w", err))
}
if _, err := r.db.ExecContext(ctx, fmt.Sprintf(postgresqlDropDBStatement, dbName)); err != nil {

View File

@@ -3,15 +3,21 @@
package errors
import "github.com/pkg/errors"
import "errors"
func ShouldReconcileErrorBeIgnored(err error) bool {
var (
nonExposedLBErr NonExposedLoadBalancerError
missingValidIPErr MissingValidIPError
migrationErr MigrationInProcessError
)
switch {
case errors.As(err, &NonExposedLoadBalancerError{}):
case errors.As(err, &nonExposedLBErr):
return true
case errors.As(err, &MissingValidIPError{}):
case errors.As(err, &missingValidIPErr):
return true
case errors.As(err, &MigrationInProcessError{}):
case errors.As(err, &migrationErr):
return true
default:
return false

View File

@@ -4,7 +4,8 @@
package kubeadm
import (
"github.com/pkg/errors"
"fmt"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
@@ -20,19 +21,19 @@ func BootstrapToken(client kubernetes.Interface, config *Configuration) error {
initConfiguration := config.InitConfiguration
if err := node.UpdateOrCreateTokens(client, false, initConfiguration.BootstrapTokens); err != nil {
return errors.Wrap(err, "error updating or creating token")
return fmt.Errorf("error updating or creating token: %w", err)
}
if err := node.AllowBootstrapTokensToGetNodes(client); err != nil {
return errors.Wrap(err, "error allowing bootstrap tokens to get Nodes")
return fmt.Errorf("error allowing bootstrap tokens to get Nodes: %w", err)
}
if err := node.AllowBootstrapTokensToPostCSRs(client); err != nil {
return errors.Wrap(err, "error allowing bootstrap tokens to post CSRs")
return fmt.Errorf("error allowing bootstrap tokens to post CSRs: %w", err)
}
if err := node.AutoApproveNodeBootstrapTokens(client); err != nil {
return errors.Wrap(err, "error auto-approving node bootstrap tokens")
return fmt.Errorf("error auto-approving node bootstrap tokens: %w", err)
}
if err := node.AutoApproveNodeCertificateRotation(client); err != nil {
@@ -66,7 +67,7 @@ func BootstrapToken(client kubernetes.Interface, config *Configuration) error {
}
if err := clusterinfo.CreateClusterInfoRBACRules(client); err != nil {
return errors.Wrap(err, "error creating clusterinfo RBAC rules")
return fmt.Errorf("error creating clusterinfo RBAC rules: %w", err)
}
return nil

View File

@@ -7,7 +7,7 @@ import (
"fmt"
"github.com/blang/semver"
"github.com/pkg/errors"
jsonpatchv5 "github.com/evanphx/json-patch/v5"
corev1 "k8s.io/api/core/v1"
rbacv1 "k8s.io/api/rbac/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
@@ -34,13 +34,14 @@ func UploadKubeadmConfig(client kubernetes.Interface, config *Configuration) ([]
return nil, uploadconfig.UploadConfiguration(&config.InitConfiguration, client)
}
func UploadKubeletConfig(client kubernetes.Interface, config *Configuration) ([]byte, error) {
func UploadKubeletConfig(client kubernetes.Interface, config *Configuration, patches jsonpatchv5.Patch) ([]byte, error) {
kubeletConfiguration := KubeletConfiguration{
TenantControlPlaneDomain: config.InitConfiguration.Networking.DNSDomain,
TenantControlPlaneDNSServiceIPs: config.Parameters.TenantDNSServiceIPs,
TenantControlPlaneCgroupDriver: config.Parameters.TenantControlPlaneCGroupDriver,
}
content, err := getKubeletConfigmapContent(kubeletConfiguration)
content, err := getKubeletConfigmapContent(kubeletConfiguration, patches)
if err != nil {
return nil, err
}
@@ -65,13 +66,13 @@ func UploadKubeletConfig(client kubernetes.Interface, config *Configuration) ([]
}
if err = createConfigMapRBACRules(client, configMapName); err != nil {
return nil, errors.Wrap(err, "error creating kubelet configuration configmap RBAC rules")
return nil, fmt.Errorf("error creating kubelet configuration configmap RBAC rules: %w", err)
}
return nil, nil
}
func getKubeletConfigmapContent(kubeletConfiguration KubeletConfiguration) ([]byte, error) {
func getKubeletConfigmapContent(kubeletConfiguration KubeletConfiguration, patch jsonpatchv5.Patch) ([]byte, error) {
var kc kubelettypes.KubeletConfiguration
kubeletv1beta1.SetDefaults_KubeletConfiguration(&kc)
@@ -93,6 +94,21 @@ func getKubeletConfigmapContent(kubeletConfiguration KubeletConfiguration) ([]by
// determine the resolvConf location, as reported in clastix/kamaji#581.
kc.ResolverConfig = nil
if len(patch) > 0 {
kubeletConfig, patchErr := utilities.EncodeToJSON(&kc)
if patchErr != nil {
return nil, fmt.Errorf("unable to encode KubeletConfiguration to JSON for JSON patching: %w", patchErr)
}
if kubeletConfig, patchErr = patch.Apply(kubeletConfig); patchErr != nil {
return nil, fmt.Errorf("unable to apply JSON patching to KubeletConfiguration: %w", patchErr)
}
if patchErr = utilities.DecodeFromJSON(string(kubeletConfig), &kc); patchErr != nil {
return nil, fmt.Errorf("unable to decode JSON to KubeletConfiguration: %w", patchErr)
}
}
return utilities.EncodeToYaml(&kc)
}
@@ -142,7 +158,7 @@ func createConfigMapRBACRules(client kubernetes.Interface, configMapName string)
func generateKubeletConfigMapName(version string) (string, error) {
parsedVersion, err := semver.ParseTolerant(version)
if err != nil {
return "", errors.Wrapf(err, "failed to parse kubernetes version %q", version)
return "", fmt.Errorf("failed to parse kubernetes version %q: %w", version, err)
}
majorMinor := semver.Version{Major: parsedVersion.Major, Minor: parsedVersion.Minor}

View File

@@ -6,8 +6,8 @@ package addons
import (
"bytes"
"context"
"fmt"
"github.com/pkg/errors"
"github.com/prometheus/client_golang/prometheus"
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
@@ -219,7 +219,7 @@ func (c *CoreDNS) UpdateTenantControlPlaneStatus(_ context.Context, tcp *kamajiv
func (c *CoreDNS) decodeManifests(ctx context.Context, tcp *kamajiv1alpha1.TenantControlPlane) error {
tcpClient, config, err := resources.GetKubeadmManifestDeps(ctx, c.Client, tcp)
if err != nil {
return errors.Wrap(err, "unable to create manifests dependencies")
return fmt.Errorf("unable to create manifests dependencies: %w", err)
}
// If CoreDNS addon is enabled and with an override, adding these to the kubeadm init configuration
@@ -235,38 +235,38 @@ func (c *CoreDNS) decodeManifests(ctx context.Context, tcp *kamajiv1alpha1.Tenan
manifests, err := kubeadm.AddCoreDNS(tcpClient, config)
if err != nil {
return errors.Wrap(err, "unable to generate manifests")
return fmt.Errorf("unable to generate manifests: %w", err)
}
parts := bytes.Split(manifests, []byte("---"))
if err = utilities.DecodeFromYAML(string(parts[1]), c.deployment); err != nil {
return errors.Wrap(err, "unable to decode Deployment manifest")
return fmt.Errorf("unable to decode Deployment manifest: %w", err)
}
addons_utils.SetKamajiManagedLabels(c.deployment)
if err = utilities.DecodeFromYAML(string(parts[2]), c.configMap); err != nil {
return errors.Wrap(err, "unable to decode ConfigMap manifest")
return fmt.Errorf("unable to decode ConfigMap manifest: %w", err)
}
addons_utils.SetKamajiManagedLabels(c.configMap)
if err = utilities.DecodeFromYAML(string(parts[3]), c.service); err != nil {
return errors.Wrap(err, "unable to decode Service manifest")
return fmt.Errorf("unable to decode Service manifest: %w", err)
}
addons_utils.SetKamajiManagedLabels(c.service)
if err = utilities.DecodeFromYAML(string(parts[4]), c.clusterRole); err != nil {
return errors.Wrap(err, "unable to decode ClusterRole manifest")
return fmt.Errorf("unable to decode ClusterRole manifest: %w", err)
}
addons_utils.SetKamajiManagedLabels(c.clusterRole)
if err = utilities.DecodeFromYAML(string(parts[5]), c.clusterRoleBinding); err != nil {
return errors.Wrap(err, "unable to decode ClusterRoleBinding manifest")
return fmt.Errorf("unable to decode ClusterRoleBinding manifest: %w", err)
}
addons_utils.SetKamajiManagedLabels(c.clusterRoleBinding)
if err = utilities.DecodeFromYAML(string(parts[6]), c.serviceAccount); err != nil {
return errors.Wrap(err, "unable to decode ServiceAccount manifest")
return fmt.Errorf("unable to decode ServiceAccount manifest: %w", err)
}
addons_utils.SetKamajiManagedLabels(c.serviceAccount)

View File

@@ -6,8 +6,8 @@ package addons
import (
"bytes"
"context"
"fmt"
"github.com/pkg/errors"
"github.com/prometheus/client_golang/prometheus"
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
@@ -321,7 +321,7 @@ func (k *KubeProxy) mutateDaemonSet(ctx context.Context, tenantClient client.Cli
func (k *KubeProxy) decodeManifests(ctx context.Context, tcp *kamajiv1alpha1.TenantControlPlane) error {
tcpClient, config, err := resources.GetKubeadmManifestDeps(ctx, k.Client, tcp)
if err != nil {
return errors.Wrap(err, "unable to create manifests dependencies")
return fmt.Errorf("unable to create manifests dependencies: %w", err)
}
// If the kube-proxy addon has overrides, adding it to the kubeadm parameters
config.Parameters.KubeProxyOptions = &kubeadm.AddonOptions{}
@@ -340,38 +340,38 @@ func (k *KubeProxy) decodeManifests(ctx context.Context, tcp *kamajiv1alpha1.Ten
manifests, err := kubeadm.AddKubeProxy(tcpClient, config)
if err != nil {
return errors.Wrap(err, "unable to generate manifests")
return fmt.Errorf("unable to generate manifests: %w", err)
}
parts := bytes.Split(manifests, []byte("---"))
if err = utilities.DecodeFromYAML(string(parts[1]), k.serviceAccount); err != nil {
return errors.Wrap(err, "unable to decode ServiceAccount manifest")
return fmt.Errorf("unable to decode ServiceAccount manifest: %w", err)
}
addon_utils.SetKamajiManagedLabels(k.serviceAccount)
if err = utilities.DecodeFromYAML(string(parts[2]), k.clusterRoleBinding); err != nil {
return errors.Wrap(err, "unable to decode ClusterRoleBinding manifest")
return fmt.Errorf("unable to decode ClusterRoleBinding manifest: %w", err)
}
addon_utils.SetKamajiManagedLabels(k.clusterRoleBinding)
if err = utilities.DecodeFromYAML(string(parts[3]), k.role); err != nil {
return errors.Wrap(err, "unable to decode Role manifest")
return fmt.Errorf("unable to decode Role manifest: %w", err)
}
addon_utils.SetKamajiManagedLabels(k.role)
if err = utilities.DecodeFromYAML(string(parts[4]), k.roleBinding); err != nil {
return errors.Wrap(err, "unable to decode RoleBinding manifest")
return fmt.Errorf("unable to decode RoleBinding manifest: %w", err)
}
addon_utils.SetKamajiManagedLabels(k.roleBinding)
if err = utilities.DecodeFromYAML(string(parts[5]), k.configMap); err != nil {
return errors.Wrap(err, "unable to decode ConfigMap manifest")
return fmt.Errorf("unable to decode ConfigMap manifest: %w", err)
}
addon_utils.SetKamajiManagedLabels(k.configMap)
if err = utilities.DecodeFromYAML(string(parts[6]), k.daemonSet); err != nil {
return errors.Wrap(err, "unable to decode DaemonSet manifest")
return fmt.Errorf("unable to decode DaemonSet manifest: %w", err)
}
addon_utils.SetKamajiManagedLabels(k.daemonSet)

View File

@@ -5,8 +5,8 @@ package datastore
import (
"context"
"errors"
"github.com/pkg/errors"
"github.com/prometheus/client_golang/prometheus"
"k8s.io/apimachinery/pkg/util/sets"
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"

View File

@@ -5,8 +5,8 @@ package datastore
import (
"context"
"fmt"
"github.com/pkg/errors"
"github.com/prometheus/client_golang/prometheus"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
@@ -192,7 +192,7 @@ func (r *Setup) UpdateTenantControlPlaneStatus(_ context.Context, tenantControlP
func (r *Setup) createDB(ctx context.Context, _ *kamajiv1alpha1.TenantControlPlane) (controllerutil.OperationResult, error) {
exists, err := r.Connection.DBExists(ctx, r.resource.schema)
if err != nil {
return controllerutil.OperationResultNone, errors.Wrap(err, "unable to check if datastore exists")
return controllerutil.OperationResultNone, fmt.Errorf("unable to check if datastore exists: %w", err)
}
if exists {
@@ -200,7 +200,7 @@ func (r *Setup) createDB(ctx context.Context, _ *kamajiv1alpha1.TenantControlPla
}
if err := r.Connection.CreateDB(ctx, r.resource.schema); err != nil {
return controllerutil.OperationResultNone, errors.Wrap(err, "unable to create the datastore")
return controllerutil.OperationResultNone, fmt.Errorf("unable to create the datastore: %w", err)
}
return controllerutil.OperationResultCreated, nil
@@ -209,7 +209,7 @@ func (r *Setup) createDB(ctx context.Context, _ *kamajiv1alpha1.TenantControlPla
func (r *Setup) deleteDB(ctx context.Context, _ *kamajiv1alpha1.TenantControlPlane) error {
exists, err := r.Connection.DBExists(ctx, r.resource.schema)
if err != nil {
return errors.Wrap(err, "unable to check if datastore exists")
return fmt.Errorf("unable to check if datastore exists: %w", err)
}
if !exists {
@@ -217,7 +217,7 @@ func (r *Setup) deleteDB(ctx context.Context, _ *kamajiv1alpha1.TenantControlPla
}
if err := r.Connection.DeleteDB(ctx, r.resource.schema); err != nil {
return errors.Wrap(err, "unable to delete the datastore")
return fmt.Errorf("unable to delete the datastore: %w", err)
}
return nil
@@ -226,7 +226,7 @@ func (r *Setup) deleteDB(ctx context.Context, _ *kamajiv1alpha1.TenantControlPla
func (r *Setup) createUser(ctx context.Context, _ *kamajiv1alpha1.TenantControlPlane) (controllerutil.OperationResult, error) {
exists, err := r.Connection.UserExists(ctx, r.resource.user)
if err != nil {
return controllerutil.OperationResultNone, errors.Wrap(err, "unable to check if user exists")
return controllerutil.OperationResultNone, fmt.Errorf("unable to check if user exists: %w", err)
}
if exists {
@@ -234,7 +234,7 @@ func (r *Setup) createUser(ctx context.Context, _ *kamajiv1alpha1.TenantControlP
}
if err := r.Connection.CreateUser(ctx, r.resource.user, r.resource.password); err != nil {
return controllerutil.OperationResultNone, errors.Wrap(err, "unable to create the user")
return controllerutil.OperationResultNone, fmt.Errorf("unable to create the user: %w", err)
}
return controllerutil.OperationResultCreated, nil
@@ -243,7 +243,7 @@ func (r *Setup) createUser(ctx context.Context, _ *kamajiv1alpha1.TenantControlP
func (r *Setup) deleteUser(ctx context.Context, _ *kamajiv1alpha1.TenantControlPlane) error {
exists, err := r.Connection.UserExists(ctx, r.resource.user)
if err != nil {
return errors.Wrap(err, "unable to check if user exists")
return fmt.Errorf("unable to check if user exists: %w", err)
}
if !exists {
@@ -251,7 +251,7 @@ func (r *Setup) deleteUser(ctx context.Context, _ *kamajiv1alpha1.TenantControlP
}
if err := r.Connection.DeleteUser(ctx, r.resource.user); err != nil {
return errors.Wrap(err, "unable to remove the user")
return fmt.Errorf("unable to remove the user: %w", err)
}
return nil
@@ -260,7 +260,7 @@ func (r *Setup) deleteUser(ctx context.Context, _ *kamajiv1alpha1.TenantControlP
func (r *Setup) createGrantPrivileges(ctx context.Context, _ *kamajiv1alpha1.TenantControlPlane) (controllerutil.OperationResult, error) {
exists, err := r.Connection.GrantPrivilegesExists(ctx, r.resource.user, r.resource.schema)
if err != nil {
return controllerutil.OperationResultNone, errors.Wrap(err, "unable to check if privileges exist")
return controllerutil.OperationResultNone, fmt.Errorf("unable to check if privileges exist: %w", err)
}
if exists {
@@ -268,7 +268,7 @@ func (r *Setup) createGrantPrivileges(ctx context.Context, _ *kamajiv1alpha1.Ten
}
if err := r.Connection.GrantPrivileges(ctx, r.resource.user, r.resource.schema); err != nil {
return controllerutil.OperationResultNone, errors.Wrap(err, "unable to grant privileges")
return controllerutil.OperationResultNone, fmt.Errorf("unable to grant privileges: %w", err)
}
return controllerutil.OperationResultCreated, nil
@@ -277,7 +277,7 @@ func (r *Setup) createGrantPrivileges(ctx context.Context, _ *kamajiv1alpha1.Ten
func (r *Setup) revokeGrantPrivileges(ctx context.Context, _ *kamajiv1alpha1.TenantControlPlane) error {
exists, err := r.Connection.GrantPrivilegesExists(ctx, r.resource.user, r.resource.schema)
if err != nil {
return errors.Wrap(err, "unable to check if privileges exist")
return fmt.Errorf("unable to check if privileges exist: %w", err)
}
if !exists {
@@ -285,7 +285,7 @@ func (r *Setup) revokeGrantPrivileges(ctx context.Context, _ *kamajiv1alpha1.Ten
}
if err := r.Connection.RevokePrivileges(ctx, r.resource.user, r.resource.schema); err != nil {
return errors.Wrap(err, "unable to revoke privileges")
return fmt.Errorf("unable to revoke privileges: %w", err)
}
return nil

View File

@@ -5,9 +5,9 @@ package datastore
import (
"context"
"fmt"
"github.com/google/uuid"
"github.com/pkg/errors"
"github.com/prometheus/client_golang/prometheus"
corev1 "k8s.io/api/core/v1"
kubeerrors "k8s.io/apimachinery/pkg/api/errors"
@@ -30,6 +30,7 @@ type Config struct {
Client client.Client
ConnString string
DataStore kamajiv1alpha1.DataStore
IsOverride bool
}
func (r *Config) GetHistogram() prometheus.Histogram {
@@ -81,7 +82,7 @@ func (r *Config) Delete(ctx context.Context, _ *kamajiv1alpha1.TenantControlPlan
return nil
}
return errors.Wrap(err, "cannot retrieve the DataStore Secret for removal")
return fmt.Errorf("cannot retrieve the DataStore Secret for removal: %w", err)
}
secret.SetFinalizers(nil)
@@ -91,7 +92,7 @@ func (r *Config) Delete(ctx context.Context, _ *kamajiv1alpha1.TenantControlPlan
return nil
}
return errors.Wrap(err, "cannot remove DataStore Secret finalizers")
return fmt.Errorf("cannot remove DataStore Secret finalizers: %w", err)
}
return nil
@@ -103,10 +104,12 @@ func (r *Config) GetName() string {
}
func (r *Config) UpdateTenantControlPlaneStatus(_ context.Context, tenantControlPlane *kamajiv1alpha1.TenantControlPlane) error {
tenantControlPlane.Status.Storage.Driver = string(r.DataStore.Spec.Driver)
tenantControlPlane.Status.Storage.DataStoreName = r.DataStore.GetName()
tenantControlPlane.Status.Storage.Config.SecretName = r.resource.GetName()
tenantControlPlane.Status.Storage.Config.Checksum = utilities.GetObjectChecksum(r.resource)
if !r.IsOverride {
tenantControlPlane.Status.Storage.Driver = string(r.DataStore.Spec.Driver)
tenantControlPlane.Status.Storage.DataStoreName = r.DataStore.GetName()
tenantControlPlane.Status.Storage.Config.SecretName = r.resource.GetName()
tenantControlPlane.Status.Storage.Config.Checksum = utilities.GetObjectChecksum(r.resource)
}
return nil
}
@@ -135,12 +138,12 @@ func (r *Config) mutate(ctx context.Context, tenantControlPlane *kamajiv1alpha1.
// set username and password to the basicAuth values of the NATS datastore
u, err := r.DataStore.Spec.BasicAuth.Username.GetContent(ctx, r.Client)
if err != nil {
return errors.Wrap(err, "failed to retrieve the username for the NATS datastore")
return fmt.Errorf("failed to retrieve the username for the NATS datastore: %w", err)
}
p, err := r.DataStore.Spec.BasicAuth.Password.GetContent(ctx, r.Client)
if err != nil {
return errors.Wrap(err, "failed to retrieve the password for the NATS datastore")
return fmt.Errorf("failed to retrieve the password for the NATS datastore: %w", err)
}
username = u

View File

@@ -22,6 +22,7 @@ type KubernetesDeploymentResource struct {
resource *appsv1.Deployment
Client client.Client
DataStore kamajiv1alpha1.DataStore
DataStoreOverrides []builder.DataStoreOverrides
KineContainerImage string
}
@@ -66,6 +67,7 @@ func (r *KubernetesDeploymentResource) mutate(ctx context.Context, tenantControl
Client: r.Client,
DataStore: r.DataStore,
KineContainerImage: r.KineContainerImage,
DataStoreOverrides: r.DataStoreOverrides,
}).Build(ctx, r.resource, *tenantControlPlane)
return controllerutil.SetControllerReference(tenantControlPlane, r.resource, r.Client.Scheme())
@@ -101,7 +103,8 @@ func (r *KubernetesDeploymentResource) computeStatus(tenantControlPlane *kamajiv
func (r *KubernetesDeploymentResource) UpdateTenantControlPlaneStatus(_ context.Context, tenantControlPlane *kamajiv1alpha1.TenantControlPlane) error {
tenantControlPlane.Status.Kubernetes.Version.Status = r.computeStatus(tenantControlPlane)
if *tenantControlPlane.Status.Kubernetes.Version.Status == kamajiv1alpha1.VersionReady {
if *tenantControlPlane.Status.Kubernetes.Version.Status == kamajiv1alpha1.VersionReady ||
*tenantControlPlane.Status.Kubernetes.Version.Status == kamajiv1alpha1.VersionSleeping {
tenantControlPlane.Status.Kubernetes.Version.Version = tenantControlPlane.Spec.Kubernetes.Version
}

View File

@@ -0,0 +1,206 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package resources
import (
"context"
"fmt"
"github.com/prometheus/client_golang/prometheus"
v1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
"sigs.k8s.io/controller-runtime/pkg/log"
gatewayv1 "sigs.k8s.io/gateway-api/apis/v1"
gatewayv1alpha2 "sigs.k8s.io/gateway-api/apis/v1alpha2"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
"github.com/clastix/kamaji/internal/utilities"
)
type KubernetesGatewayResource struct {
resource *gatewayv1alpha2.TLSRoute
Client client.Client
}
func (r *KubernetesGatewayResource) GetHistogram() prometheus.Histogram {
gatewayCollector = LazyLoadHistogramFromResource(gatewayCollector, r)
return gatewayCollector
}
func (r *KubernetesGatewayResource) ShouldStatusBeUpdated(_ context.Context, tcp *kamajiv1alpha1.TenantControlPlane) bool {
switch {
case tcp.Spec.ControlPlane.Gateway == nil && tcp.Status.Kubernetes.Gateway == nil:
return false
case tcp.Spec.ControlPlane.Gateway != nil && tcp.Status.Kubernetes.Gateway == nil:
return true
case tcp.Spec.ControlPlane.Gateway == nil && tcp.Status.Kubernetes.Gateway != nil:
return true
// Could be an alias for default here since the other cases are covered.
case tcp.Spec.ControlPlane.Gateway != nil && tcp.Status.Kubernetes.Gateway != nil:
return r.gatewayStatusNeedsUpdate(tcp)
}
return false
}
// gatewayStatusNeedsUpdate compares the current gateway resource status with the stored status.
func (r *KubernetesGatewayResource) gatewayStatusNeedsUpdate(tcp *kamajiv1alpha1.TenantControlPlane) bool {
currentStatus := tcp.Status.Kubernetes.Gateway
// Check if route reference has changed
if currentStatus != nil && currentStatus.RouteRef.Name != r.resource.Name {
return true
}
// Compare RouteStatus - check if number of parents changed
return IsGatewayRouteStatusChanged(currentStatus, r.resource.Status.RouteStatus)
}
func (r *KubernetesGatewayResource) ShouldCleanup(tcp *kamajiv1alpha1.TenantControlPlane) bool {
return tcp.Spec.ControlPlane.Gateway == nil && tcp.Status.Kubernetes.Gateway != nil
}
func (r *KubernetesGatewayResource) CleanUp(ctx context.Context, tcp *kamajiv1alpha1.TenantControlPlane) (bool, error) {
logger := log.FromContext(ctx, "resource", r.GetName())
cleaned, err := CleanupTLSRoute(ctx, r.Client, r.resource.GetName(), r.resource.GetNamespace(), tcp)
if err != nil {
logger.Error(err, "failed to cleanup tcp route")
return false, err
}
if cleaned {
logger.V(1).Info("tcp route cleaned up successfully")
}
return cleaned, nil
}
func (r *KubernetesGatewayResource) UpdateTenantControlPlaneStatus(ctx context.Context, tcp *kamajiv1alpha1.TenantControlPlane) error {
logger := log.FromContext(ctx, "resource", r.GetName())
// Clean up status if Gateway routes are no longer configured
if tcp.Spec.ControlPlane.Gateway == nil {
tcp.Status.Kubernetes.Gateway = nil
return nil
}
tcp.Status.Kubernetes.Gateway = &kamajiv1alpha1.KubernetesGatewayStatus{
RouteStatus: r.resource.Status.RouteStatus,
RouteRef: v1.LocalObjectReference{
Name: r.resource.Name,
},
}
routeStatuses := tcp.Status.Kubernetes.Gateway.RouteStatus
// TODO: Investigate the implications of having multiple parents / hostnames
// TODO: Use condition to report?
if len(routeStatuses.Parents) == 0 {
return fmt.Errorf("no gateway attached to the route")
}
if len(routeStatuses.Parents) > 1 {
return fmt.Errorf("too many gateway attached to the route")
}
if len(r.resource.Spec.Hostnames) == 0 {
return fmt.Errorf("no hostname in the route")
}
if len(r.resource.Spec.Hostnames) > 1 {
return fmt.Errorf("too many hostnames in the route")
}
logger.V(1).Info("updating TenantControlPlane status for Gateway routes")
accessPoints, err := BuildGatewayAccessPointsStatus(ctx, r.Client, r.resource, routeStatuses)
if err != nil {
return err
}
tcp.Status.Kubernetes.Gateway.AccessPoints = accessPoints
return nil
}
func (r *KubernetesGatewayResource) Define(_ context.Context, tcp *kamajiv1alpha1.TenantControlPlane) error {
r.resource = &gatewayv1alpha2.TLSRoute{
ObjectMeta: metav1.ObjectMeta{
Name: tcp.GetName(),
Namespace: tcp.GetNamespace(),
},
}
return nil
}
func (r *KubernetesGatewayResource) mutate(tcp *kamajiv1alpha1.TenantControlPlane) controllerutil.MutateFn {
return func() error {
labels := utilities.MergeMaps(
r.resource.GetLabels(),
utilities.KamajiLabels(tcp.GetName(), r.GetName()),
tcp.Spec.ControlPlane.Gateway.AdditionalMetadata.Labels,
)
r.resource.SetLabels(labels)
annotations := utilities.MergeMaps(
r.resource.GetAnnotations(),
tcp.Spec.ControlPlane.Gateway.AdditionalMetadata.Annotations)
r.resource.SetAnnotations(annotations)
serviceName := gatewayv1alpha2.ObjectName(tcp.Status.Kubernetes.Service.Name)
servicePort := tcp.Status.Kubernetes.Service.Port
if serviceName == "" || servicePort == 0 {
return fmt.Errorf("service not ready, cannot create TLSRoute")
}
if tcp.Spec.ControlPlane.Gateway.GatewayParentRefs != nil {
// Copy parentRefs and explicitly set port and sectionName fields
r.resource.Spec.ParentRefs = NewParentRefsSpecWithPortAndSection(tcp.Spec.ControlPlane.Gateway.GatewayParentRefs, servicePort, "kube-apiserver")
}
rule := gatewayv1alpha2.TLSRouteRule{
BackendRefs: []gatewayv1alpha2.BackendRef{
{
BackendObjectReference: gatewayv1alpha2.BackendObjectReference{
Name: serviceName,
Port: &servicePort,
},
},
},
}
r.resource.Spec.Hostnames = []gatewayv1.Hostname{tcp.Spec.ControlPlane.Gateway.Hostname}
r.resource.Spec.Rules = []gatewayv1alpha2.TLSRouteRule{rule}
return controllerutil.SetControllerReference(tcp, r.resource, r.Client.Scheme())
}
}
func (r *KubernetesGatewayResource) CreateOrUpdate(ctx context.Context, tenantControlPlane *kamajiv1alpha1.TenantControlPlane) (controllerutil.OperationResult, error) {
logger := log.FromContext(ctx, "resource", r.GetName())
if tenantControlPlane.Spec.ControlPlane.Gateway == nil {
return controllerutil.OperationResultNone, nil
}
if len(tenantControlPlane.Spec.ControlPlane.Gateway.Hostname) == 0 {
return controllerutil.OperationResultNone, fmt.Errorf("missing hostname to expose the Tenant Control Plane using a Gateway resource")
}
logger.V(1).Info("creating or updating resource gateway routes")
result, err := utilities.CreateOrUpdateWithConflict(ctx, r.Client, r.resource, r.mutate(tenantControlPlane))
if err != nil {
return result, err
}
return result, nil
}
func (r *KubernetesGatewayResource) GetName() string {
return "gateway_routes"
}

View File

@@ -0,0 +1,372 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package resources_test
import (
"context"
"testing"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/client-go/kubernetes/scheme"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/client/fake"
gatewayv1 "sigs.k8s.io/gateway-api/apis/v1"
gatewayv1alpha2 "sigs.k8s.io/gateway-api/apis/v1alpha2"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
"github.com/clastix/kamaji/internal/resources"
)
func TestGatewayResource(t *testing.T) {
RegisterFailHandler(Fail)
RunSpecs(t, "Gateway Resource Suite")
}
var runtimeScheme *runtime.Scheme
var _ = BeforeSuite(func() {
runtimeScheme = runtime.NewScheme()
Expect(scheme.AddToScheme(runtimeScheme)).To(Succeed())
Expect(kamajiv1alpha1.AddToScheme(runtimeScheme)).To(Succeed())
Expect(gatewayv1alpha2.Install(runtimeScheme)).To(Succeed())
})
var _ = Describe("KubernetesGatewayResource", func() {
var (
tcp *kamajiv1alpha1.TenantControlPlane
resource *resources.KubernetesGatewayResource
ctx context.Context
)
BeforeEach(func() {
ctx = context.Background()
fakeClient := fake.NewClientBuilder().
WithScheme(runtimeScheme).
Build()
resource = &resources.KubernetesGatewayResource{
Client: fakeClient,
}
tcp = &kamajiv1alpha1.TenantControlPlane{
ObjectMeta: metav1.ObjectMeta{
Name: "test-tcp",
Namespace: "default",
},
Spec: kamajiv1alpha1.TenantControlPlaneSpec{
ControlPlane: kamajiv1alpha1.ControlPlane{
Gateway: &kamajiv1alpha1.GatewaySpec{
Hostname: gatewayv1alpha2.Hostname("test.example.com"),
AdditionalMetadata: kamajiv1alpha1.AdditionalMetadata{
Labels: map[string]string{
"test-label": "test-value",
},
},
GatewayParentRefs: []gatewayv1alpha2.ParentReference{
{
Name: "test-gateway",
},
},
},
},
},
Status: kamajiv1alpha1.TenantControlPlaneStatus{
Kubernetes: kamajiv1alpha1.KubernetesStatus{
Service: kamajiv1alpha1.KubernetesServiceStatus{
Name: "test-service",
Port: 6443,
},
},
},
}
})
Context("When GatewayRoutes is configured", func() {
It("should not cleanup", func() {
Expect(resource.ShouldCleanup(tcp)).To(BeFalse())
})
It("should define route resources", func() {
err := resource.Define(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
})
It("should require status update when GatewayRoutes is configured but status is nil", func() {
tcp.Status.Kubernetes.Gateway = nil
shouldUpdate := resource.ShouldStatusBeUpdated(ctx, tcp)
Expect(shouldUpdate).To(BeTrue())
})
It("should set port and sectionName in parentRefs, overriding any user-provided values", func() {
customPort := gatewayv1.PortNumber(9999)
customSectionName := gatewayv1.SectionName("custom")
tcp.Spec.ControlPlane.Gateway.GatewayParentRefs[0].Port = &customPort
tcp.Spec.ControlPlane.Gateway.GatewayParentRefs[0].SectionName = &customSectionName
err := resource.Define(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
_, err = resource.CreateOrUpdate(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
route := &gatewayv1alpha2.TLSRoute{}
err = resource.Client.Get(ctx, client.ObjectKey{Name: tcp.Name, Namespace: tcp.Namespace}, route)
Expect(err).NotTo(HaveOccurred())
Expect(route.Spec.ParentRefs).To(HaveLen(1))
Expect(route.Spec.ParentRefs[0].Port).NotTo(BeNil())
Expect(*route.Spec.ParentRefs[0].Port).To(Equal(tcp.Status.Kubernetes.Service.Port))
Expect(*route.Spec.ParentRefs[0].Port).NotTo(Equal(customPort))
Expect(route.Spec.ParentRefs[0].SectionName).NotTo(BeNil())
Expect(*route.Spec.ParentRefs[0].SectionName).To(Equal(gatewayv1.SectionName("kube-apiserver")))
Expect(*route.Spec.ParentRefs[0].SectionName).NotTo(Equal(customSectionName))
})
It("should handle multiple parentRefs correctly", func() {
namespace := gatewayv1.Namespace("default")
tcp.Spec.ControlPlane.Gateway.GatewayParentRefs = []gatewayv1alpha2.ParentReference{
{
Name: "test-gateway-1",
Namespace: &namespace,
},
{
Name: "test-gateway-2",
Namespace: &namespace,
},
}
err := resource.Define(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
_, err = resource.CreateOrUpdate(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
route := &gatewayv1alpha2.TLSRoute{}
err = resource.Client.Get(ctx, client.ObjectKey{Name: tcp.Name, Namespace: tcp.Namespace}, route)
Expect(err).NotTo(HaveOccurred())
Expect(route.Spec.ParentRefs).To(HaveLen(2))
for i := range route.Spec.ParentRefs {
Expect(route.Spec.ParentRefs[i].Port).NotTo(BeNil())
Expect(*route.Spec.ParentRefs[i].Port).To(Equal(tcp.Status.Kubernetes.Service.Port))
Expect(route.Spec.ParentRefs[i].SectionName).NotTo(BeNil())
Expect(*route.Spec.ParentRefs[i].SectionName).To(Equal(gatewayv1.SectionName("kube-apiserver")))
}
})
})
Context("When GatewayRoutes is not configured", func() {
BeforeEach(func() {
tcp.Spec.ControlPlane.Gateway = nil
tcp.Status.Kubernetes.Gateway = &kamajiv1alpha1.KubernetesGatewayStatus{
AccessPoints: nil,
}
})
It("should cleanup", func() {
Expect(resource.ShouldCleanup(tcp)).To(BeTrue())
})
It("should not require status update when both spec and status are nil", func() {
tcp.Status.Kubernetes.Gateway = nil
shouldUpdate := resource.ShouldStatusBeUpdated(ctx, tcp)
Expect(shouldUpdate).To(BeFalse())
})
})
Context("When hostname is missing", func() {
BeforeEach(func() {
tcp.Spec.ControlPlane.Gateway.Hostname = ""
})
It("should fail to create or update", func() {
err := resource.Define(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
_, err = resource.CreateOrUpdate(ctx, tcp)
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("missing hostname"))
})
})
Context("When service is not ready", func() {
BeforeEach(func() {
tcp.Status.Kubernetes.Service.Name = ""
tcp.Status.Kubernetes.Service.Port = 0
})
It("should fail to create or update", func() {
err := resource.Define(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
_, err = resource.CreateOrUpdate(ctx, tcp)
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("service not ready"))
})
})
It("should return correct resource name", func() {
Expect(resource.GetName()).To(Equal("gateway_routes"))
})
Describe("findMatchingListener", func() {
var (
listeners []gatewayv1.Listener
ref gatewayv1.ParentReference
)
BeforeEach(func() {
listeners = []gatewayv1.Listener{
{
Name: "first",
Port: gatewayv1.PortNumber(443),
},
{
Name: "middle",
Port: gatewayv1.PortNumber(6443),
},
{
Name: "last",
Port: gatewayv1.PortNumber(80),
},
}
ref = gatewayv1.ParentReference{
Name: "test-gateway",
}
})
It("should return an error when sectionName is nil", func() {
listener, err := resources.FindMatchingListener(listeners, ref)
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("missing sectionName"))
Expect(listener).To(Equal(gatewayv1.Listener{}))
})
It("should return an error when sectionName is an empty string", func() {
sectionName := gatewayv1.SectionName("")
ref.SectionName = &sectionName
listener, err := resources.FindMatchingListener(listeners, ref)
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("could not find listener ''"))
Expect(listener).To(Equal(gatewayv1.Listener{}))
})
It("should return the matching listener when sectionName points to an existing listener", func() {
sectionName := gatewayv1.SectionName("middle")
ref.SectionName = &sectionName
listener, err := resources.FindMatchingListener(listeners, ref)
Expect(err).NotTo(HaveOccurred())
Expect(listener.Port).To(Equal(gatewayv1.PortNumber(6443)))
})
It("should return an error when sectionName points to a non-existent listener", func() {
sectionName := gatewayv1.SectionName("non-existent")
ref.SectionName = &sectionName
listener, err := resources.FindMatchingListener(listeners, ref)
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("could not find listener 'non-existent'"))
Expect(listener).To(Equal(gatewayv1.Listener{}))
})
It("should return the first listener", func() {
sectionName := gatewayv1.SectionName("first")
ref.SectionName = &sectionName
listener, err := resources.FindMatchingListener(listeners, ref)
Expect(err).NotTo(HaveOccurred())
Expect(listener.Port).To(Equal(gatewayv1.PortNumber(443)))
})
It("should return the last listener when matching by name", func() {
sectionName := gatewayv1.SectionName("last")
ref.SectionName = &sectionName
listener, err := resources.FindMatchingListener(listeners, ref)
Expect(err).NotTo(HaveOccurred())
Expect(listener.Port).To(Equal(gatewayv1.PortNumber(80)))
})
})
Describe("NewParentRefsSpecWithPortAndSection", func() {
var (
parentRefs []gatewayv1.ParentReference
testPort int32
testSectionName string
)
BeforeEach(func() {
namespace := gatewayv1.Namespace("default")
namespace2 := gatewayv1.Namespace("other")
testPort = int32(6443)
testSectionName = "kube-apiserver"
originalPort := gatewayv1.PortNumber(9999)
originalSectionName := gatewayv1.SectionName("original")
parentRefs = []gatewayv1.ParentReference{
{
Name: "test-gateway-1",
Namespace: &namespace,
Port: &originalPort,
SectionName: &originalSectionName,
},
{
Name: "test-gateway-2",
Namespace: &namespace2,
},
}
})
It("should create copy of parentRefs with port and sectionName set", func() {
result := resources.NewParentRefsSpecWithPortAndSection(parentRefs, testPort, testSectionName)
Expect(result).To(HaveLen(2))
for i := range result {
Expect(result[i].Name).To(Equal(parentRefs[i].Name))
Expect(result[i].Namespace).To(Equal(parentRefs[i].Namespace))
Expect(result[i].Port).NotTo(BeNil())
Expect(*result[i].Port).To(Equal(testPort))
Expect(result[i].SectionName).NotTo(BeNil())
Expect(*result[i].SectionName).To(Equal(gatewayv1.SectionName(testSectionName)))
}
})
It("should not modify original parentRefs", func() {
// Store original values for verification
originalFirstPort := parentRefs[0].Port
originalFirstSectionName := parentRefs[0].SectionName
originalSecondPort := parentRefs[1].Port
originalSecondSectionName := parentRefs[1].SectionName
result := resources.NewParentRefsSpecWithPortAndSection(parentRefs, testPort, testSectionName)
// Original should remain unchanged
Expect(parentRefs[0].Port).To(Equal(originalFirstPort))
Expect(parentRefs[0].SectionName).To(Equal(originalFirstSectionName))
Expect(parentRefs[1].Port).To(Equal(originalSecondPort))
Expect(parentRefs[1].SectionName).To(Equal(originalSecondSectionName))
// Result should have new values
Expect(result[0].Port).NotTo(BeNil())
Expect(*result[0].Port).To(Equal(testPort))
Expect(result[0].SectionName).NotTo(BeNil())
Expect(*result[0].SectionName).To(Equal(gatewayv1.SectionName(testSectionName)))
Expect(result[1].Port).NotTo(BeNil())
Expect(*result[1].Port).To(Equal(testPort))
Expect(result[1].SectionName).NotTo(BeNil())
Expect(*result[1].SectionName).To(Equal(gatewayv1.SectionName(testSectionName)))
})
It("should handle empty parentRefs slice", func() {
parentRefs = []gatewayv1.ParentReference{}
result := resources.NewParentRefsSpecWithPortAndSection(parentRefs, testPort, testSectionName)
Expect(result).To(BeEmpty())
})
})
})

View File

@@ -0,0 +1,241 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package resources
import (
"context"
"fmt"
"net/url"
k8serrors "k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/api/meta"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/fields"
"sigs.k8s.io/controller-runtime/pkg/client"
gatewayv1 "sigs.k8s.io/gateway-api/apis/v1"
gatewayv1alpha2 "sigs.k8s.io/gateway-api/apis/v1alpha2"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
)
// fetchGatewayByListener uses the indexer to efficiently find a gateway with a specific listener.
// This avoids the need to iterate through all listeners in a gateway.
func fetchGatewayByListener(ctx context.Context, c client.Client, ref gatewayv1.ParentReference) (*gatewayv1.Gateway, error) {
if ref.SectionName == nil {
return nil, fmt.Errorf("missing sectionName")
}
// Build the composite key that matches our indexer format: namespace/gatewayName/listenerName
listenerKey := fmt.Sprintf("%s/%s/%s", *ref.Namespace, ref.Name, *ref.SectionName)
// Query gateways using the indexer
gatewayList := &gatewayv1.GatewayList{}
if err := c.List(ctx, gatewayList, client.MatchingFieldsSelector{
Selector: fields.OneTermEqualSelector(kamajiv1alpha1.GatewayListenerNameKey, listenerKey),
}); err != nil {
return nil, fmt.Errorf("failed to list gateways by listener: %w", err)
}
if len(gatewayList.Items) == 0 {
return nil, fmt.Errorf("no gateway found with listener '%s'", *ref.SectionName)
}
// Since we're using a composite key with namespace/name/listener, we should get exactly one result
if len(gatewayList.Items) > 1 {
return nil, fmt.Errorf("found multiple gateways with listener '%s', expected exactly one", *ref.SectionName)
}
return &gatewayList.Items[0], nil
}
// FindMatchingListener finds a listener in the given list that matches the parent reference.
func FindMatchingListener(listeners []gatewayv1.Listener, ref gatewayv1.ParentReference) (gatewayv1.Listener, error) {
if ref.SectionName == nil {
return gatewayv1.Listener{}, fmt.Errorf("missing sectionName")
}
name := *ref.SectionName
for _, listener := range listeners {
if listener.Name == name {
return listener, nil
}
}
// TODO: Handle the cases according to the spec:
// - When both Port (experimental) and SectionName are
// specified, the name and port of the selected listener
// must match both specified values.
// - When unspecified (empty string) this will reference
// the entire resource [...] an attachment is considered
// successful if at least one section in the parent resource accepts it
return gatewayv1.Listener{}, fmt.Errorf("could not find listener '%s'", name)
}
// IsGatewayRouteStatusChanged checks if the gateway route status has changed compared to the stored status.
// Returns true if the status has changed (update needed), false if it's the same.
func IsGatewayRouteStatusChanged(currentStatus *kamajiv1alpha1.KubernetesGatewayStatus, resourceStatus gatewayv1alpha2.RouteStatus) bool {
if currentStatus == nil {
return true
}
// Compare RouteStatus - check if number of parents changed
if len(currentStatus.RouteStatus.Parents) != len(resourceStatus.Parents) {
return true
}
// Compare individual parent statuses
// NOTE: Multiple Parent References are assumed.
for i, currentParent := range currentStatus.RouteStatus.Parents {
if i >= len(resourceStatus.Parents) {
return true
}
resourceParent := resourceStatus.Parents[i]
// Compare parent references
if currentParent.ParentRef.Name != resourceParent.ParentRef.Name ||
(currentParent.ParentRef.Namespace == nil) != (resourceParent.ParentRef.Namespace == nil) ||
(currentParent.ParentRef.Namespace != nil && resourceParent.ParentRef.Namespace != nil &&
*currentParent.ParentRef.Namespace != *resourceParent.ParentRef.Namespace) ||
(currentParent.ParentRef.SectionName == nil) != (resourceParent.ParentRef.SectionName == nil) ||
(currentParent.ParentRef.SectionName != nil && resourceParent.ParentRef.SectionName != nil &&
*currentParent.ParentRef.SectionName != *resourceParent.ParentRef.SectionName) {
return true
}
if len(currentParent.Conditions) != len(resourceParent.Conditions) {
return true
}
// Compare each condition
for j, currentCondition := range currentParent.Conditions {
if j >= len(resourceParent.Conditions) {
return true
}
resourceCondition := resourceParent.Conditions[j]
if currentCondition.Type != resourceCondition.Type ||
currentCondition.Status != resourceCondition.Status ||
currentCondition.Reason != resourceCondition.Reason ||
currentCondition.Message != resourceCondition.Message ||
!currentCondition.LastTransitionTime.Equal(&resourceCondition.LastTransitionTime) {
return true
}
}
}
// Since access points are derived from route status and gateway conditions,
// and we've already compared the route status above, we can assume that
// if the route status hasn't changed, the access points calculation
// will produce the same result. This avoids the need for complex
// gateway fetching in the status comparison.
//
// If there are edge cases where gateway state changes but route status doesn't,
// those will be caught in the next reconciliation cycle anyway.
return false
}
// CleanupTLSRoute cleans up a TLSRoute resource if it's managed by the given TenantControlPlane.
func CleanupTLSRoute(ctx context.Context, c client.Client, routeName, routeNamespace string, tcp metav1.Object) (bool, error) {
route := gatewayv1alpha2.TLSRoute{}
if err := c.Get(ctx, client.ObjectKey{
Namespace: routeNamespace,
Name: routeName,
}, &route); err != nil {
if !k8serrors.IsNotFound(err) {
return false, fmt.Errorf("failed to get TLSRoute before cleanup: %w", err)
}
return false, nil
}
if !metav1.IsControlledBy(&route, tcp) {
return false, nil
}
if err := c.Delete(ctx, &route); err != nil {
if !k8serrors.IsNotFound(err) {
return false, fmt.Errorf("cannot delete TLSRoute route: %w", err)
}
return false, nil
}
return true, nil
}
// BuildGatewayAccessPointsStatus builds access points from route statuses.
func BuildGatewayAccessPointsStatus(ctx context.Context, c client.Client, route *gatewayv1alpha2.TLSRoute, routeStatuses gatewayv1alpha2.RouteStatus) ([]kamajiv1alpha1.GatewayAccessPoint, error) {
accessPoints := []kamajiv1alpha1.GatewayAccessPoint{}
routeNamespace := gatewayv1.Namespace(route.Namespace)
for _, routeStatus := range routeStatuses.Parents {
routeAccepted := meta.IsStatusConditionTrue(
routeStatus.Conditions,
string(gatewayv1.RouteConditionAccepted),
)
if !routeAccepted {
continue
}
if routeStatus.ParentRef.Namespace == nil {
// Set the namespace to the route namespace if not set
routeStatus.ParentRef.Namespace = &routeNamespace
}
// Use the indexer to efficiently find the gateway with the specific listener
gateway, err := fetchGatewayByListener(ctx, c, routeStatus.ParentRef)
if err != nil {
return nil, fmt.Errorf("could not fetch gateway with listener '%v': %w",
routeStatus.ParentRef.SectionName, err)
}
gatewayProgrammed := meta.IsStatusConditionTrue(
gateway.Status.Conditions,
string(gatewayv1.GatewayConditionProgrammed),
)
if !gatewayProgrammed {
continue
}
// Since we fetched the gateway using the indexer, we know the listener exists
// but we still need to get its details from the gateway spec
listener, err := FindMatchingListener(
gateway.Spec.Listeners, routeStatus.ParentRef,
)
if err != nil {
return nil, fmt.Errorf("failed to match listener: %w", err)
}
for _, hostname := range route.Spec.Hostnames {
rawURL := fmt.Sprintf("https://%s:%d", hostname, listener.Port)
parsedURL, err := url.Parse(rawURL)
if err != nil {
return nil, fmt.Errorf("invalid url: %w", err)
}
hostnameAddressType := gatewayv1.HostnameAddressType
accessPoints = append(accessPoints, kamajiv1alpha1.GatewayAccessPoint{
Type: &hostnameAddressType,
Value: parsedURL.String(),
Port: listener.Port,
})
}
}
return accessPoints, nil
}
// NewParentRefsSpecWithPortAndSection creates a copy of parentRefs with port and sectionName set for each reference.
func NewParentRefsSpecWithPortAndSection(parentRefs []gatewayv1.ParentReference, port int32, sectionName string) []gatewayv1.ParentReference {
result := make([]gatewayv1.ParentReference, len(parentRefs))
sectionNamePtr := gatewayv1.SectionName(sectionName)
for i, parentRef := range parentRefs {
result[i] = *parentRef.DeepCopy()
result[i].Port = &port
result[i].SectionName = &sectionNamePtr
}
return result
}

View File

@@ -182,6 +182,21 @@ func (r *Agent) mutate(ctx context.Context, tenantControlPlane *kamajiv1alpha1.T
return err
}
// Override address with control plane gateway hostname if configured
// Konnectivity TLSRoute uses the same hostname as control plane gateway
if tenantControlPlane.Spec.ControlPlane.Gateway != nil &&
len(tenantControlPlane.Spec.ControlPlane.Gateway.Hostname) > 0 {
hostname := tenantControlPlane.Spec.ControlPlane.Gateway.Hostname
// Extract hostname
if len(hostname) > 0 {
konnectivityHostname, _ := utilities.GetControlPlaneAddressAndPortFromHostname(
string(hostname),
tenantControlPlane.Spec.Addons.Konnectivity.KonnectivityServerSpec.Port)
address = konnectivityHostname
}
}
r.resource.SetLabels(utilities.MergeMaps(r.resource.GetLabels(), utilities.KamajiLabels(tenantControlPlane.GetName(), r.GetName())))
specSelector := &metav1.LabelSelector{
@@ -281,8 +296,10 @@ func (r *Agent) mutate(ctx context.Context, tenantControlPlane *kamajiv1alpha1.T
case kamajiv1alpha1.KonnectivityAgentModeDeployment:
//nolint:forcetypeassert
r.resource.(*appsv1.Deployment).Spec.Template = *podTemplateSpec
//nolint:forcetypeassert
r.resource.(*appsv1.Deployment).Spec.Replicas = pointer.To(tenantControlPlane.Spec.Addons.Konnectivity.KonnectivityAgentSpec.Replicas)
if tenantControlPlane.Spec.Addons.Konnectivity.KonnectivityAgentSpec.Replicas != nil {
//nolint:forcetypeassert
r.resource.(*appsv1.Deployment).Spec.Replicas = tenantControlPlane.Spec.Addons.Konnectivity.KonnectivityAgentSpec.Replicas
}
}
return nil

View File

@@ -10,7 +10,6 @@ import (
corev1 "k8s.io/api/core/v1"
k8serrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
apiserverv1alpha1 "k8s.io/apiserver/pkg/apis/apiserver/v1alpha1"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
@@ -94,18 +93,18 @@ func (r *EgressSelectorConfigurationResource) mutate(_ context.Context, tenantCo
return func() error {
r.resource.SetLabels(utilities.MergeMaps(r.resource.GetLabels(), utilities.KamajiLabels(tenantControlPlane.GetName(), r.GetName())))
configuration := &apiserverv1alpha1.EgressSelectorConfiguration{
configuration := &EgressSelectorConfiguration{
TypeMeta: metav1.TypeMeta{
Kind: egressSelectorConfigurationKind,
APIVersion: apiServerAPIVersion,
},
EgressSelections: []apiserverv1alpha1.EgressSelection{
EgressSelections: []EgressSelection{
{
Name: egressSelectorConfigurationName,
Connection: apiserverv1alpha1.Connection{
ProxyProtocol: apiserverv1alpha1.ProtocolGRPC,
Transport: &apiserverv1alpha1.Transport{
UDS: &apiserverv1alpha1.UDSTransport{
Connection: Connection{
ProxyProtocol: ProtocolGRPC,
Transport: &Transport{
UDS: &UDSTransport{
UDSName: defaultUDSName,
},
},

View File

@@ -0,0 +1,71 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package konnectivity
import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
// Local type definitions for EgressSelectorConfiguration to avoid importing
// k8s.io/apiserver which conflicts with controller-runtime workqueue metrics.
// These types are based on k8s.io/apiserver/pkg/apis/apiserver/v1alpha1.
// ProtocolType is the type of the proxy protocol used for egress selection.
type ProtocolType string
const (
// ProtocolHTTPConnect uses HTTP CONNECT as the proxy protocol.
ProtocolHTTPConnect ProtocolType = "HTTPConnect"
// ProtocolGRPC uses GRPC as the proxy protocol.
ProtocolGRPC ProtocolType = "GRPC"
// ProtocolDirect establishes a direct connection without proxy.
ProtocolDirect ProtocolType = "Direct"
)
// +kubebuilder:object:root=true
// +kubebuilder:object:generate=true
// EgressSelectorConfiguration provides versioned configuration for egress selector clients.
type EgressSelectorConfiguration struct {
metav1.TypeMeta `json:",inline"`
EgressSelections []EgressSelection `json:"egressSelections"`
}
// +kubebuilder:object:generate=true
// EgressSelection provides the configuration for a single egress selection client.
type EgressSelection struct {
Name string `json:"name"`
Connection Connection `json:"connection"`
}
// +kubebuilder:object:generate=true
// Connection provides the configuration for a single egress selection client connection.
type Connection struct {
ProxyProtocol ProtocolType `json:"proxyProtocol,omitempty"`
Transport *Transport `json:"transport,omitempty"`
}
// +kubebuilder:object:generate=true
// Transport defines the transport configurations we support for egress selector.
type Transport struct {
TCP *TCPTransport `json:"tcp,omitempty"`
UDS *UDSTransport `json:"uds,omitempty"`
}
// +kubebuilder:object:generate=true
// TCPTransport provides the information to connect to a TCP endpoint.
type TCPTransport struct {
URL string `json:"url,omitempty"`
}
// +kubebuilder:object:generate=true
// UDSTransport provides the information to connect to a Unix Domain Socket endpoint.
type UDSTransport struct {
UDSName string `json:"udsName,omitempty"`
}

View File

@@ -0,0 +1,232 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package konnectivity
import (
"context"
"fmt"
"github.com/prometheus/client_golang/prometheus"
v1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
"sigs.k8s.io/controller-runtime/pkg/log"
gatewayv1 "sigs.k8s.io/gateway-api/apis/v1"
gatewayv1alpha2 "sigs.k8s.io/gateway-api/apis/v1alpha2"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
"github.com/clastix/kamaji/internal/resources"
"github.com/clastix/kamaji/internal/utilities"
)
type KubernetesKonnectivityGatewayResource struct {
resource *gatewayv1alpha2.TLSRoute
Client client.Client
}
func (r *KubernetesKonnectivityGatewayResource) GetHistogram() prometheus.Histogram {
gatewayCollector = resources.LazyLoadHistogramFromResource(gatewayCollector, r)
return gatewayCollector
}
func (r *KubernetesKonnectivityGatewayResource) ShouldStatusBeUpdated(_ context.Context, tcp *kamajiv1alpha1.TenantControlPlane) bool {
switch {
case !r.shouldHaveGateway(tcp) && (tcp.Status.Addons.Konnectivity.Gateway == nil):
return false
case r.shouldHaveGateway(tcp) && (tcp.Status.Addons.Konnectivity.Gateway == nil):
return true
case !r.shouldHaveGateway(tcp) && (tcp.Status.Addons.Konnectivity.Gateway != nil):
return true
case r.shouldHaveGateway(tcp) && (tcp.Status.Addons.Konnectivity.Gateway != nil):
return r.gatewayStatusNeedsUpdate(tcp)
}
return false
}
// shouldHaveGateway checks if Konnectivity gateway should be configured.
// Create when Konnectivity addon is enabled and control plane gateway is configured.
func (r *KubernetesKonnectivityGatewayResource) shouldHaveGateway(tcp *kamajiv1alpha1.TenantControlPlane) bool {
if tcp.Spec.Addons.Konnectivity == nil { // konnectivity addon is disabled
return false
}
// Create when control plane gateway is configured
return tcp.Spec.ControlPlane.Gateway != nil
}
// gatewayStatusNeedsUpdate compares the current gateway resource status with the stored status.
func (r *KubernetesKonnectivityGatewayResource) gatewayStatusNeedsUpdate(tcp *kamajiv1alpha1.TenantControlPlane) bool {
currentStatus := tcp.Status.Addons.Konnectivity.Gateway
// Check if route reference has changed
if currentStatus != nil && currentStatus.RouteRef.Name != r.resource.Name {
return true
}
// Compare RouteStatus - check if number of parents changed
return resources.IsGatewayRouteStatusChanged(currentStatus, r.resource.Status.RouteStatus)
}
func (r *KubernetesKonnectivityGatewayResource) ShouldCleanup(tcp *kamajiv1alpha1.TenantControlPlane) bool {
return !r.shouldHaveGateway(tcp) && tcp.Status.Addons.Konnectivity.Gateway != nil
}
func (r *KubernetesKonnectivityGatewayResource) CleanUp(ctx context.Context, tcp *kamajiv1alpha1.TenantControlPlane) (bool, error) {
logger := log.FromContext(ctx, "resource", r.GetName())
cleaned, err := resources.CleanupTLSRoute(ctx, r.Client, r.resource.GetName(), r.resource.GetNamespace(), tcp)
if err != nil {
logger.Error(err, "failed to cleanup konnectivity route")
return false, err
}
if cleaned {
logger.V(1).Info("konnectivity route cleaned up successfully")
}
return cleaned, nil
}
func (r *KubernetesKonnectivityGatewayResource) UpdateTenantControlPlaneStatus(ctx context.Context, tcp *kamajiv1alpha1.TenantControlPlane) error {
logger := log.FromContext(ctx, "resource", r.GetName())
// Clean up status if Gateway routes are no longer configured
if !r.shouldHaveGateway(tcp) {
tcp.Status.Addons.Konnectivity.Gateway = nil
return nil
}
tcp.Status.Addons.Konnectivity.Gateway = &kamajiv1alpha1.KubernetesGatewayStatus{
RouteStatus: r.resource.Status.RouteStatus,
RouteRef: v1.LocalObjectReference{
Name: r.resource.Name,
},
}
routeStatuses := tcp.Status.Addons.Konnectivity.Gateway.RouteStatus
// TODO: Investigate the implications of having multiple parents / hostnames
// TODO: Use condition to report?
if len(routeStatuses.Parents) == 0 {
return fmt.Errorf("no gateway attached to the konnectivity route")
}
if len(routeStatuses.Parents) > 1 {
return fmt.Errorf("too many gateways attached to the konnectivity route")
}
if len(r.resource.Spec.Hostnames) == 0 {
return fmt.Errorf("no hostname in the konnectivity route")
}
if len(r.resource.Spec.Hostnames) > 1 {
return fmt.Errorf("too many hostnames in the konnectivity route")
}
logger.V(1).Info("updating TenantControlPlane status for Konnectivity Gateway routes")
accessPoints, err := resources.BuildGatewayAccessPointsStatus(ctx, r.Client, r.resource, routeStatuses)
if err != nil {
return err
}
tcp.Status.Addons.Konnectivity.Gateway.AccessPoints = accessPoints
return nil
}
func (r *KubernetesKonnectivityGatewayResource) Define(_ context.Context, tcp *kamajiv1alpha1.TenantControlPlane) error {
r.resource = &gatewayv1alpha2.TLSRoute{
ObjectMeta: metav1.ObjectMeta{
Name: fmt.Sprintf("%s-konnectivity", tcp.GetName()),
Namespace: tcp.GetNamespace(),
},
}
return nil
}
func (r *KubernetesKonnectivityGatewayResource) mutate(tcp *kamajiv1alpha1.TenantControlPlane) controllerutil.MutateFn {
return func() error {
// Use control plane gateway configuration
if tcp.Spec.ControlPlane.Gateway == nil {
return fmt.Errorf("control plane gateway is not configured")
}
labels := utilities.MergeMaps(
r.resource.GetLabels(),
utilities.KamajiLabels(tcp.GetName(), r.GetName()),
tcp.Spec.ControlPlane.Gateway.AdditionalMetadata.Labels,
)
r.resource.SetLabels(labels)
annotations := utilities.MergeMaps(
r.resource.GetAnnotations(),
tcp.Spec.ControlPlane.Gateway.AdditionalMetadata.Annotations,
)
r.resource.SetAnnotations(annotations)
// Use hostname from control plane gateway
if len(tcp.Spec.ControlPlane.Gateway.Hostname) == 0 {
return fmt.Errorf("control plane gateway hostname is not set")
}
serviceName := gatewayv1alpha2.ObjectName(tcp.Status.Addons.Konnectivity.Service.Name)
servicePort := tcp.Status.Addons.Konnectivity.Service.Port
if serviceName == "" || servicePort == 0 {
return fmt.Errorf("konnectivity service not ready, cannot create TLSRoute")
}
// Copy parentRefs from control plane gateway and explicitly set port and sectionName fields
if tcp.Spec.ControlPlane.Gateway.GatewayParentRefs == nil {
return fmt.Errorf("control plane gateway parentRefs are not specified")
}
r.resource.Spec.ParentRefs = resources.NewParentRefsSpecWithPortAndSection(tcp.Spec.ControlPlane.Gateway.GatewayParentRefs, servicePort, "konnectivity-server")
rule := gatewayv1alpha2.TLSRouteRule{
BackendRefs: []gatewayv1alpha2.BackendRef{
{
BackendObjectReference: gatewayv1alpha2.BackendObjectReference{
Name: serviceName,
Port: &servicePort,
},
},
},
}
r.resource.Spec.Hostnames = []gatewayv1.Hostname{tcp.Spec.ControlPlane.Gateway.Hostname}
r.resource.Spec.Rules = []gatewayv1alpha2.TLSRouteRule{rule}
return controllerutil.SetControllerReference(tcp, r.resource, r.Client.Scheme())
}
}
func (r *KubernetesKonnectivityGatewayResource) CreateOrUpdate(ctx context.Context, tenantControlPlane *kamajiv1alpha1.TenantControlPlane) (controllerutil.OperationResult, error) {
logger := log.FromContext(ctx, "resource", r.GetName())
if !r.shouldHaveGateway(tenantControlPlane) {
return controllerutil.OperationResultNone, nil
}
if tenantControlPlane.Spec.ControlPlane.Gateway == nil {
return controllerutil.OperationResultNone, nil
}
if len(tenantControlPlane.Spec.ControlPlane.Gateway.Hostname) == 0 {
return controllerutil.OperationResultNone, fmt.Errorf("missing hostname to expose Konnectivity using a Gateway resource")
}
logger.V(1).Info("creating or updating resource konnectivity gateway routes")
result, err := utilities.CreateOrUpdateWithConflict(ctx, r.Client, r.resource, r.mutate(tenantControlPlane))
if err != nil {
return result, err
}
return result, nil
}
func (r *KubernetesKonnectivityGatewayResource) GetName() string {
return "konnectivity_gateway_routes"
}

View File

@@ -0,0 +1,218 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package konnectivity_test
import (
"context"
"testing"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/client-go/kubernetes/scheme"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/client/fake"
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
gatewayv1 "sigs.k8s.io/gateway-api/apis/v1"
gatewayv1alpha2 "sigs.k8s.io/gateway-api/apis/v1alpha2"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
"github.com/clastix/kamaji/internal/resources/konnectivity"
)
func TestKonnectivityGatewayResource(t *testing.T) {
RegisterFailHandler(Fail)
RunSpecs(t, "Konnectivity Gateway Resource Suite")
}
var runtimeScheme *runtime.Scheme
var _ = BeforeSuite(func() {
runtimeScheme = runtime.NewScheme()
Expect(scheme.AddToScheme(runtimeScheme)).To(Succeed())
Expect(kamajiv1alpha1.AddToScheme(runtimeScheme)).To(Succeed())
Expect(gatewayv1alpha2.Install(runtimeScheme)).To(Succeed())
})
var _ = Describe("KubernetesKonnectivityGatewayResource", func() {
var (
tcp *kamajiv1alpha1.TenantControlPlane
resource *konnectivity.KubernetesKonnectivityGatewayResource
ctx context.Context
)
BeforeEach(func() {
ctx = context.Background()
fakeClient := fake.NewClientBuilder().
WithScheme(runtimeScheme).
Build()
resource = &konnectivity.KubernetesKonnectivityGatewayResource{
Client: fakeClient,
}
namespace := gatewayv1.Namespace("default")
tcp = &kamajiv1alpha1.TenantControlPlane{
ObjectMeta: metav1.ObjectMeta{
Name: "test-tcp",
Namespace: "default",
},
Spec: kamajiv1alpha1.TenantControlPlaneSpec{
ControlPlane: kamajiv1alpha1.ControlPlane{
Gateway: &kamajiv1alpha1.GatewaySpec{
Hostname: gatewayv1alpha2.Hostname("test.example.com"),
GatewayParentRefs: []gatewayv1alpha2.ParentReference{
{
Name: "test-gateway",
Namespace: &namespace,
},
},
},
},
Addons: kamajiv1alpha1.AddonsSpec{
Konnectivity: &kamajiv1alpha1.KonnectivitySpec{
KonnectivityServerSpec: kamajiv1alpha1.KonnectivityServerSpec{
Port: 8132,
},
},
},
},
Status: kamajiv1alpha1.TenantControlPlaneStatus{
Addons: kamajiv1alpha1.AddonsStatus{
Konnectivity: kamajiv1alpha1.KonnectivityStatus{
Service: kamajiv1alpha1.KubernetesServiceStatus{
Name: "test-konnectivity-service",
Port: 8132,
},
},
},
},
}
})
Describe("shouldHaveGateway logic", func() {
It("should return false when Konnectivity addon is disabled", func() {
tcp.Spec.Addons.Konnectivity = nil
shouldUpdate := resource.ShouldStatusBeUpdated(ctx, tcp)
Expect(shouldUpdate).To(BeFalse())
Expect(resource.ShouldCleanup(tcp)).To(BeFalse())
})
It("should return false when control plane gateway is not configured", func() {
tcp.Spec.ControlPlane.Gateway = nil
shouldUpdate := resource.ShouldStatusBeUpdated(ctx, tcp)
Expect(shouldUpdate).To(BeFalse())
Expect(resource.ShouldCleanup(tcp)).To(BeFalse())
})
It("should return true when both Konnectivity and gateway are configured", func() {
shouldUpdate := resource.ShouldStatusBeUpdated(ctx, tcp)
Expect(shouldUpdate).To(BeTrue())
Expect(resource.ShouldCleanup(tcp)).To(BeFalse())
})
})
Context("When Konnectivity gateway should be configured", func() {
It("should set correct TLSRoute name with -konnectivity suffix", func() {
err := resource.Define(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
_, err = resource.CreateOrUpdate(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
route := &gatewayv1alpha2.TLSRoute{}
err = resource.Client.Get(ctx, client.ObjectKey{Name: "test-tcp-konnectivity", Namespace: tcp.Namespace}, route)
Expect(err).NotTo(HaveOccurred())
Expect(route.Name).To(Equal("test-tcp-konnectivity"))
})
It("should set sectionName to \"konnectivity-server\" and port from Konnectivity service status", func() {
err := resource.Define(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
_, err = resource.CreateOrUpdate(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
route := &gatewayv1alpha2.TLSRoute{}
err = resource.Client.Get(ctx, client.ObjectKey{Name: "test-tcp-konnectivity", Namespace: tcp.Namespace}, route)
Expect(err).NotTo(HaveOccurred())
Expect(route.Spec.ParentRefs).To(HaveLen(1))
Expect(route.Spec.ParentRefs[0].SectionName).NotTo(BeNil())
Expect(*route.Spec.ParentRefs[0].SectionName).To(Equal(gatewayv1.SectionName("konnectivity-server")))
Expect(route.Spec.ParentRefs[0].Port).NotTo(BeNil())
Expect(*route.Spec.ParentRefs[0].Port).To(Equal(tcp.Status.Addons.Konnectivity.Service.Port))
})
It("should use control plane gateway hostname", func() {
err := resource.Define(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
_, err = resource.CreateOrUpdate(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
route := &gatewayv1alpha2.TLSRoute{}
err = resource.Client.Get(ctx, client.ObjectKey{Name: "test-tcp-konnectivity", Namespace: tcp.Namespace}, route)
Expect(err).NotTo(HaveOccurred())
Expect(route.Spec.Hostnames).To(HaveLen(1))
Expect(route.Spec.Hostnames[0]).To(Equal(tcp.Spec.ControlPlane.Gateway.Hostname))
})
})
Context("Konnectivity-specific error cases", func() {
It("should return early without error when control plane gateway is not configured", func() {
tcp.Spec.ControlPlane.Gateway = nil
err := resource.Define(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
result, err := resource.CreateOrUpdate(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
Expect(result).To(Equal(controllerutil.OperationResultNone))
})
It("should fail when Konnectivity service is not ready", func() {
tcp.Status.Addons.Konnectivity.Service.Name = ""
tcp.Status.Addons.Konnectivity.Service.Port = 0
err := resource.Define(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
_, err = resource.CreateOrUpdate(ctx, tcp)
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("konnectivity service not ready"))
})
It("should fail when control plane gateway parentRefs are not specified", func() {
tcp.Spec.ControlPlane.Gateway.GatewayParentRefs = nil
err := resource.Define(ctx, tcp)
Expect(err).NotTo(HaveOccurred())
_, err = resource.CreateOrUpdate(ctx, tcp)
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("control plane gateway parentRefs are not specified"))
})
})
Context("When Konnectivity gateway should not be configured", func() {
BeforeEach(func() {
tcp.Spec.Addons.Konnectivity = nil
tcp.Status.Addons.Konnectivity = kamajiv1alpha1.KonnectivityStatus{
Gateway: &kamajiv1alpha1.KubernetesGatewayStatus{
AccessPoints: nil,
},
}
})
It("should cleanup when gateway is removed", func() {
Expect(resource.ShouldCleanup(tcp)).To(BeTrue())
})
})
It("should return correct resource name", func() {
Expect(resource.GetName()).To(Equal("konnectivity_gateway_routes"))
})
})

View File

@@ -13,6 +13,7 @@ var (
clusterrolebindingCollector prometheus.Histogram
deploymentCollector prometheus.Histogram
egressCollector prometheus.Histogram
gatewayCollector prometheus.Histogram
kubeconfigCollector prometheus.Histogram
serviceaccountCollector prometheus.Histogram
serviceCollector prometheus.Histogram

View File

@@ -0,0 +1,134 @@
//go:build !ignore_autogenerated
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
// Code generated by controller-gen. DO NOT EDIT.
package konnectivity
import (
runtime "k8s.io/apimachinery/pkg/runtime"
)
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *Connection) DeepCopyInto(out *Connection) {
*out = *in
if in.Transport != nil {
in, out := &in.Transport, &out.Transport
*out = new(Transport)
(*in).DeepCopyInto(*out)
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Connection.
func (in *Connection) DeepCopy() *Connection {
if in == nil {
return nil
}
out := new(Connection)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *EgressSelection) DeepCopyInto(out *EgressSelection) {
*out = *in
in.Connection.DeepCopyInto(&out.Connection)
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new EgressSelection.
func (in *EgressSelection) DeepCopy() *EgressSelection {
if in == nil {
return nil
}
out := new(EgressSelection)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *EgressSelectorConfiguration) DeepCopyInto(out *EgressSelectorConfiguration) {
*out = *in
out.TypeMeta = in.TypeMeta
if in.EgressSelections != nil {
in, out := &in.EgressSelections, &out.EgressSelections
*out = make([]EgressSelection, len(*in))
for i := range *in {
(*in)[i].DeepCopyInto(&(*out)[i])
}
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new EgressSelectorConfiguration.
func (in *EgressSelectorConfiguration) DeepCopy() *EgressSelectorConfiguration {
if in == nil {
return nil
}
out := new(EgressSelectorConfiguration)
in.DeepCopyInto(out)
return out
}
// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.
func (in *EgressSelectorConfiguration) DeepCopyObject() runtime.Object {
if c := in.DeepCopy(); c != nil {
return c
}
return nil
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *TCPTransport) DeepCopyInto(out *TCPTransport) {
*out = *in
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new TCPTransport.
func (in *TCPTransport) DeepCopy() *TCPTransport {
if in == nil {
return nil
}
out := new(TCPTransport)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *Transport) DeepCopyInto(out *Transport) {
*out = *in
if in.TCP != nil {
in, out := &in.TCP, &out.TCP
*out = new(TCPTransport)
**out = **in
}
if in.UDS != nil {
in, out := &in.UDS, &out.UDS
*out = new(UDSTransport)
**out = **in
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Transport.
func (in *Transport) DeepCopy() *Transport {
if in == nil {
return nil
}
out := new(Transport)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *UDSTransport) DeepCopyInto(out *UDSTransport) {
*out = *in
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new UDSTransport.
func (in *UDSTransport) DeepCopy() *UDSTransport {
if in == nil {
return nil
}
out := new(UDSTransport)
in.DeepCopyInto(out)
return out
}

View File

@@ -77,14 +77,6 @@ func (r *KubeadmConfigResource) UpdateTenantControlPlaneStatus(_ context.Context
return nil
}
func (r *KubeadmConfigResource) getControlPlaneEndpoint(ingress *kamajiv1alpha1.IngressSpec, address string, port int32) string {
if ingress != nil && len(ingress.Hostname) > 0 {
address, port = utilities.GetControlPlaneAddressAndPortFromHostname(ingress.Hostname, port)
}
return net.JoinHostPort(address, strconv.FormatInt(int64(port), 10))
}
func (r *KubeadmConfigResource) mutate(ctx context.Context, tenantControlPlane *kamajiv1alpha1.TenantControlPlane) controllerutil.MutateFn {
return func() error {
logger := log.FromContext(ctx, "resource", r.GetName())
@@ -98,12 +90,27 @@ func (r *KubeadmConfigResource) mutate(ctx context.Context, tenantControlPlane *
r.resource.SetLabels(utilities.MergeMaps(r.resource.GetLabels(), utilities.KamajiLabels(tenantControlPlane.GetName(), r.GetName())))
endpoint := net.JoinHostPort(address, strconv.FormatInt(int64(port), 10))
spec := tenantControlPlane.Spec.ControlPlane
if spec.Gateway != nil {
if len(spec.Gateway.Hostname) > 0 {
gaddr, gport := utilities.GetControlPlaneAddressAndPortFromHostname(string(spec.Gateway.Hostname), port)
endpoint = net.JoinHostPort(gaddr, strconv.FormatInt(int64(gport), 10))
}
}
if spec.Ingress != nil {
if len(spec.Ingress.Hostname) > 0 {
iaddr, iport := utilities.GetControlPlaneAddressAndPortFromHostname(spec.Ingress.Hostname, port)
endpoint = net.JoinHostPort(iaddr, strconv.FormatInt(int64(iport), 10))
}
}
params := kubeadm.Parameters{
TenantControlPlaneAddress: address,
TenantControlPlanePort: port,
TenantControlPlaneName: tenantControlPlane.GetName(),
TenantControlPlaneNamespace: tenantControlPlane.GetNamespace(),
TenantControlPlaneEndpoint: r.getControlPlaneEndpoint(tenantControlPlane.Spec.ControlPlane.Ingress, address, port),
TenantControlPlaneEndpoint: endpoint,
TenantControlPlaneCertSANs: tenantControlPlane.Spec.NetworkProfile.CertSANs,
TenantControlPlaneClusterDomain: tenantControlPlane.Spec.NetworkProfile.ClusterDomain,
TenantControlPlanePodCIDR: tenantControlPlane.Spec.NetworkProfile.PodCIDR,

View File

@@ -10,6 +10,7 @@ import (
"strings"
"time"
jsonpatchv5 "github.com/evanphx/json-patch/v5"
"github.com/prometheus/client_golang/prometheus"
corev1 "k8s.io/api/core/v1"
rbacv1 "k8s.io/api/rbac/v1"
@@ -150,7 +151,21 @@ func (r *KubeadmPhase) GetKubeadmFunction(ctx context.Context, tcp *kamajiv1alph
case PhaseUploadConfigKubeadm:
return kubeadm.UploadKubeadmConfig, nil
case PhaseUploadConfigKubelet:
return kubeadm.UploadKubeletConfig, nil
return func(c clientset.Interface, configuration *kubeadm.Configuration) ([]byte, error) {
var patch jsonpatchv5.Patch
if len(tcp.Spec.Kubernetes.Kubelet.ConfigurationJSONPatches) > 0 {
jsonP, patchErr := tcp.Spec.Kubernetes.Kubelet.ConfigurationJSONPatches.ToJSON()
if patchErr != nil {
return nil, fmt.Errorf("cannot encode JSON Patches to JSON: %w", patchErr)
}
if patch, patchErr = jsonpatchv5.DecodePatch(jsonP); patchErr != nil {
return nil, fmt.Errorf("cannot decode JSON Patches: %w", patchErr)
}
}
return kubeadm.UploadKubeletConfig(c, configuration, patch)
}, nil
case PhaseBootstrapToken:
return func(client clientset.Interface, config *kubeadm.Configuration) ([]byte, error) {
config.InitConfiguration.BootstrapTokens = nil

View File

@@ -7,10 +7,10 @@ import (
"context"
"fmt"
"github.com/pkg/errors"
"github.com/prometheus/client_golang/prometheus"
"k8s.io/apimachinery/pkg/util/version"
"k8s.io/kubernetes/cmd/kubeadm/app/phases/upgrade"
"k8s.io/utils/ptr"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
@@ -74,20 +74,27 @@ func (k *KubernetesUpgrade) CreateOrUpdate(ctx context.Context, tenantControlPla
// Checking if the upgrade is allowed, or not
clientSet, err := utilities.GetTenantClientSet(ctx, k.Client, tenantControlPlane)
if err != nil {
return controllerutil.OperationResultNone, errors.Wrap(err, "cannot create REST client required for Kubernetes upgrade plan")
return controllerutil.OperationResultNone, fmt.Errorf("cannot create REST client required for Kubernetes upgrade plan: %w", err)
}
versionGetter := kamajiupgrade.NewKamajiKubeVersionGetter(clientSet, tenantControlPlane.Status.Kubernetes.Version.Version)
var coreDNSVersion string
if tenantControlPlane.Spec.Addons.CoreDNS != nil {
coreDNSVersion = tenantControlPlane.Spec.Addons.CoreDNS.ImageTag
}
if _, err = upgrade.GetAvailableUpgrades(versionGetter, false, false, clientSet, &printers.Discard{}); err != nil {
return controllerutil.OperationResultNone, errors.Wrap(err, "cannot retrieve available Upgrades for Kubernetes upgrade plan")
versionGetter := kamajiupgrade.NewKamajiKubeVersionGetter(clientSet, tenantControlPlane.Status.Kubernetes.Version.Version, coreDNSVersion, tenantControlPlane.Status.Kubernetes.Version.Status)
if _, err = upgrade.GetAvailableUpgrades(versionGetter, false, false, &printers.Discard{}); err != nil {
return controllerutil.OperationResultNone, fmt.Errorf("cannot retrieve available Upgrades for Kubernetes upgrade plan: %w", err)
}
if err = k.isUpgradable(); err != nil {
return controllerutil.OperationResultNone, fmt.Errorf("the required upgrade plan is not available")
}
k.inProgress = true
if ptr.Deref(tenantControlPlane.Spec.ControlPlane.Deployment.Replicas, 0) > 0 {
k.inProgress = true
}
return controllerutil.OperationResultNone, nil
}
@@ -115,12 +122,12 @@ func (k *KubernetesUpgrade) UpdateTenantControlPlaneStatus(_ context.Context, te
func (k *KubernetesUpgrade) isUpgradable() error {
newK8sVersion, err := version.ParseSemantic(k.upgrade.After.KubeVersion)
if err != nil {
return errors.Wrap(err, fmt.Sprintf("unable to parse normalized version %q as a semantic version", k.upgrade.After.KubeVersion))
return fmt.Errorf("unable to parse normalized version %q as a semantic version: %w", k.upgrade.After.KubeVersion, err)
}
oldK8sVersion, err := version.ParseSemantic(k.upgrade.Before.KubeVersion)
if err != nil {
return errors.Wrap(err, fmt.Sprintf("unable to parse normalized version %q as a semantic version", k.upgrade.After.KubeVersion))
return fmt.Errorf("unable to parse normalized version %q as a semantic version: %w", k.upgrade.After.KubeVersion, err)
}
if newK8sVersion.Minor() < oldK8sVersion.Minor() {

View File

@@ -5,9 +5,9 @@ package resources
import (
"context"
"fmt"
"github.com/go-logr/logr"
"github.com/pkg/errors"
corev1 "k8s.io/api/core/v1"
k8serrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
@@ -25,17 +25,17 @@ import (
func GetKubeadmManifestDeps(ctx context.Context, client client.Client, tenantControlPlane *kamajiv1alpha1.TenantControlPlane) (*clientset.Clientset, *kubeadm.Configuration, error) {
config, err := getStoredKubeadmConfiguration(ctx, client, "", tenantControlPlane)
if err != nil {
return nil, nil, errors.Wrap(err, "cannot retrieve kubeadm configuration")
return nil, nil, fmt.Errorf("cannot retrieve kubeadm configuration: %w", err)
}
kubeconfig, err := utilities.GetTenantKubeconfig(ctx, client, tenantControlPlane)
if err != nil {
return nil, nil, errors.Wrap(err, "cannot retrieve kubeconfig configuration")
return nil, nil, fmt.Errorf("cannot retrieve kubeconfig configuration: %w", err)
}
address, _, err := tenantControlPlane.AssignedControlPlaneAddress()
if err != nil {
return nil, nil, errors.Wrap(err, "cannot retrieve Tenant Control Plane address")
return nil, nil, fmt.Errorf("cannot retrieve Tenant Control Plane address: %w", err)
}
config.Kubeconfig = *kubeconfig
@@ -47,7 +47,7 @@ func GetKubeadmManifestDeps(ctx context.Context, client client.Client, tenantCon
TenantControlPlaneAddress: address,
TenantControlPlaneCertSANs: tenantControlPlane.Spec.NetworkProfile.CertSANs,
TenantControlPlanePort: tenantControlPlane.Spec.NetworkProfile.Port,
TenantControlPlaneCGroupDriver: tenantControlPlane.Spec.Kubernetes.Kubelet.CGroupFS.String(),
TenantControlPlaneCGroupDriver: tenantControlPlane.Spec.Kubernetes.Kubelet.CGroupFS.String(), //nolint:staticcheck
}
// If CoreDNS addon is enabled and with an override, adding these to the kubeadm init configuration
if coreDNS := tenantControlPlane.Spec.Addons.CoreDNS; coreDNS != nil {
@@ -80,7 +80,7 @@ func GetKubeadmManifestDeps(ctx context.Context, client client.Client, tenantCon
tenantClient, err := utilities.GetTenantClientSet(ctx, client, tenantControlPlane)
if err != nil {
return nil, nil, errors.Wrap(err, "cannot generate tenant client")
return nil, nil, fmt.Errorf("cannot generate tenant client: %w", err)
}
return tenantClient, config, nil
@@ -200,7 +200,7 @@ func KubeadmPhaseCreate(ctx context.Context, r KubeadmPhaseResource, logger logr
TenantControlPlaneAddress: address,
TenantControlPlaneCertSANs: tenantControlPlane.Spec.NetworkProfile.CertSANs,
TenantControlPlanePort: tenantControlPlane.Spec.NetworkProfile.Port,
TenantControlPlaneCGroupDriver: tenantControlPlane.Spec.Kubernetes.Kubelet.CGroupFS.String(),
TenantControlPlaneCGroupDriver: tenantControlPlane.Spec.Kubernetes.Kubelet.CGroupFS.String(), //nolint:staticcheck
}
var checksum string

View File

@@ -16,6 +16,7 @@ var (
frontproxycaCollector prometheus.Histogram
deploymentCollector prometheus.Histogram
ingressCollector prometheus.Histogram
gatewayCollector prometheus.Histogram
serviceCollector prometheus.Histogram
kubeadmconfigCollector prometheus.Histogram
kubeadmupgradeCollector prometheus.Histogram

View File

@@ -7,28 +7,51 @@ import (
"fmt"
"runtime"
"github.com/pkg/errors"
versionutil "k8s.io/apimachinery/pkg/util/version"
apimachineryversion "k8s.io/apimachinery/pkg/version"
"k8s.io/client-go/kubernetes"
"k8s.io/kubernetes/cmd/kubeadm/app/phases/upgrade"
kamajiv1alpha1 "github.com/clastix/kamaji/api/v1alpha1"
)
type kamajiKubeVersionGetter struct {
upgrade.VersionGetter
Version string
k8sVersion string
coreDNSVersion string
status *kamajiv1alpha1.KubernetesVersionStatus
}
func NewKamajiKubeVersionGetter(restClient kubernetes.Interface, version string) upgrade.VersionGetter {
func NewKamajiKubeVersionGetter(restClient kubernetes.Interface, version, coreDNSVersion string, status *kamajiv1alpha1.KubernetesVersionStatus) upgrade.VersionGetter {
kubeVersionGetter := upgrade.NewOfflineVersionGetter(upgrade.NewKubeVersionGetter(restClient), KubeadmVersion)
return &kamajiKubeVersionGetter{VersionGetter: kubeVersionGetter, Version: version}
return &kamajiKubeVersionGetter{
VersionGetter: kubeVersionGetter,
k8sVersion: version,
coreDNSVersion: coreDNSVersion,
status: status,
}
}
func (k kamajiKubeVersionGetter) ClusterVersion() (string, *versionutil.Version, error) {
if k.status != nil && *k.status == kamajiv1alpha1.VersionSleeping {
parsedVersion, parsedErr := versionutil.ParseGeneric(k.k8sVersion)
return k.k8sVersion, parsedVersion, parsedErr
}
return k.VersionGetter.ClusterVersion()
}
func (k kamajiKubeVersionGetter) DNSAddonVersion() (string, error) {
if k.status != nil && *k.status == kamajiv1alpha1.VersionSleeping {
return k.coreDNSVersion, nil
}
return k.VersionGetter.DNSAddonVersion()
}
func (k kamajiKubeVersionGetter) KubeadmVersion() (string, *versionutil.Version, error) {
kubeadmVersionInfo := apimachineryversion.Info{
GitVersion: KubeadmVersion,
@@ -39,7 +62,7 @@ func (k kamajiKubeVersionGetter) KubeadmVersion() (string, *versionutil.Version,
kubeadmVersion, err := versionutil.ParseSemantic(kubeadmVersionInfo.String())
if err != nil {
return "", nil, errors.Wrap(err, "Couldn't parse kubeadm version")
return "", nil, fmt.Errorf("couldn't parse kubeadm version: %w", err)
}
return kubeadmVersionInfo.String(), kubeadmVersion, nil
@@ -50,11 +73,15 @@ func (k kamajiKubeVersionGetter) VersionFromCILabel(ciVersionLabel, description
}
func (k kamajiKubeVersionGetter) KubeletVersions() (map[string][]string, error) {
if k.status != nil && *k.status == kamajiv1alpha1.VersionSleeping {
return map[string][]string{}, nil
}
return k.VersionGetter.KubeletVersions()
}
func (k kamajiKubeVersionGetter) ComponentVersions(string) (map[string][]string, error) {
return map[string][]string{
k.Version: {"kamaji"},
k.k8sVersion: {"kamaji"},
}, nil
}

View File

@@ -4,5 +4,5 @@
package upgrade
const (
KubeadmVersion = "v1.34.0"
KubeadmVersion = "v1.35.0"
)

View File

@@ -0,0 +1,109 @@
// Copyright 2022 Clastix Labs
// SPDX-License-Identifier: Apache-2.0
package utilities
import (
"context"
"k8s.io/apimachinery/pkg/api/meta"
"k8s.io/apimachinery/pkg/runtime/schema"
"k8s.io/client-go/discovery"
"sigs.k8s.io/controller-runtime/pkg/client"
gatewayv1 "sigs.k8s.io/gateway-api/apis/v1"
gatewayv1alpha2 "sigs.k8s.io/gateway-api/apis/v1alpha2"
)
// AreGatewayResourcesAvailable checks if Gateway API is available in the cluster through a discovery Client
// with fallback to client-based check.
func AreGatewayResourcesAvailable(ctx context.Context, c client.Client, discoveryClient discovery.DiscoveryInterface) bool {
if discoveryClient == nil {
return IsGatewayAPIAvailableViaClient(ctx, c)
}
available, err := GatewayAPIResourcesAvailable(ctx, discoveryClient)
if err != nil {
return false
}
return available
}
// NOTE: These functions are extremely similar, maybe they can be merged and accept a GVK.
// Explicit for now.
// GatewayAPIResourcesAvailable checks if Gateway API is available in the cluster.
func GatewayAPIResourcesAvailable(ctx context.Context, discoveryClient discovery.DiscoveryInterface) (bool, error) {
gatewayAPIGroup := gatewayv1.GroupName
serverGroups, err := discoveryClient.ServerGroups()
if err != nil {
return false, err
}
for _, group := range serverGroups.Groups {
if group.Name == gatewayAPIGroup {
return true, nil
}
}
return false, nil
}
// TLSRouteAPIAvailable checks specifically for TLSRoute resource availability.
func TLSRouteAPIAvailable(ctx context.Context, discoveryClient discovery.DiscoveryInterface) (bool, error) {
gv := gatewayv1alpha2.SchemeGroupVersion
resourceList, err := discoveryClient.ServerResourcesForGroupVersion(gv.String())
if err != nil {
return false, err
}
for _, resource := range resourceList.APIResources {
if resource.Kind == "TLSRoute" {
return true, nil
}
}
return false, nil
}
// IsTLSRouteAvailable checks if TLSRoute is available with fallback to client-based check.
func IsTLSRouteAvailable(ctx context.Context, c client.Client, discoveryClient discovery.DiscoveryInterface) bool {
if discoveryClient == nil {
return IsTLSRouteAvailableViaClient(ctx, c)
}
available, err := TLSRouteAPIAvailable(ctx, discoveryClient)
if err != nil {
return false
}
return available
}
// IsTLSRouteAvailableViaClient uses client to check TLSRoute availability.
func IsTLSRouteAvailableViaClient(ctx context.Context, c client.Client) bool {
// Try to check if TLSRoute GVK can be resolved
gvk := schema.GroupVersionKind{
Group: gatewayv1alpha2.GroupName,
Version: "v1alpha2",
Kind: "TLSRoute",
}
restMapper := c.RESTMapper()
_, err := restMapper.RESTMapping(gvk.GroupKind(), gvk.Version)
if err != nil {
if meta.IsNoMatchError(err) {
return false
}
// Other errors might be transient, assume available
return true
}
return true
}
// IsGatewayAPIAvailableViaClient uses client to check Gateway API availability.
func IsGatewayAPIAvailableViaClient(ctx context.Context, c client.Client) bool {
return IsTLSRouteAvailableViaClient(ctx, c)
}

View File

@@ -9,7 +9,6 @@ import (
"net/http"
"strings"
"github.com/pkg/errors"
"gomodules.xyz/jsonpatch/v2"
admissionv1 "k8s.io/api/admission/v1"
"k8s.io/apimachinery/pkg/runtime"
@@ -34,11 +33,11 @@ func (h handlersChainer) Handler(object runtime.Object, routeHandlers ...handler
// When deleting the OldObject struct field contains the object being deleted:
// https://github.com/kubernetes/kubernetes/pull/76346
if err := h.decoder.DecodeRaw(req.OldObject, decodedObj); err != nil {
return admission.Errored(http.StatusInternalServerError, errors.Wrap(err, fmt.Sprintf("unable to decode deleted object into %T", object)))
return admission.Errored(http.StatusInternalServerError, fmt.Errorf("unable to decode deleted object into %T: %w", object, err))
}
default:
if err := h.decoder.Decode(req, decodedObj); err != nil {
return admission.Errored(http.StatusInternalServerError, errors.Wrap(err, fmt.Sprintf("unable to decode into %T", object)))
return admission.Errored(http.StatusInternalServerError, fmt.Errorf("unable to decode into %T: %w", object, err))
}
}
}
@@ -70,7 +69,7 @@ func (h handlersChainer) Handler(object runtime.Object, routeHandlers ...handler
}
case admissionv1.Update:
if err := h.decoder.DecodeRaw(req.OldObject, oldDecodedObj); err != nil {
return admission.Errored(http.StatusInternalServerError, errors.Wrap(err, fmt.Sprintf("unable to decode old object into %T", object)))
return admission.Errored(http.StatusInternalServerError, fmt.Errorf("unable to decode old object into %T: %w", object, err))
}
for _, routeHandler := range routeHandlers {

View File

@@ -8,7 +8,6 @@ import (
"fmt"
"strings"
"github.com/pkg/errors"
"gomodules.xyz/jsonpatch/v2"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/fields"
@@ -39,7 +38,7 @@ func (d DataStoreSecretValidation) OnUpdate(object runtime.Object, _ runtime.Obj
dsList := &kamajiv1alpha1.DataStoreList{}
if err := d.Client.List(ctx, dsList, client.MatchingFieldsSelector{Selector: fields.OneTermEqualSelector(kamajiv1alpha1.DatastoreUsedSecretNamespacedNameKey, fmt.Sprintf("%s/%s", secret.GetNamespace(), secret.GetName()))}); err != nil {
return nil, errors.Wrap(err, "cannot list Tenant Control Plane using the provided Secret")
return nil, fmt.Errorf("cannot list Tenant Control Plane using the provided Secret: %w", err)
}
if len(dsList.Items) > 0 {

View File

@@ -7,7 +7,6 @@ import (
"context"
"fmt"
"github.com/pkg/errors"
"gomodules.xyz/jsonpatch/v2"
corev1 "k8s.io/api/core/v1"
k8serrors "k8s.io/apimachinery/pkg/api/errors"
@@ -38,7 +37,7 @@ func (d DataStoreValidation) OnDelete(object runtime.Object) AdmissionResponse {
tcpList := &kamajiv1alpha1.TenantControlPlaneList{}
if err := d.Client.List(ctx, tcpList, client.MatchingFieldsSelector{Selector: fields.OneTermEqualSelector(kamajiv1alpha1.TenantControlPlaneUsedDataStoreKey, ds.GetName())}); err != nil {
return nil, errors.Wrap(err, "cannot retrieve TenantControlPlane list used by the DataStore")
return nil, fmt.Errorf("cannot retrieve TenantControlPlane list used by the DataStore: %w", err)
}
if len(tcpList.Items) > 0 {

View File

@@ -30,7 +30,7 @@ func (t TenantControlPlaneDataStore) OnCreate(object runtime.Object) AdmissionRe
return nil, t.check(ctx, tcp.Spec.DataStore)
}
return nil, nil
return nil, t.checkDataStoreOverrides(ctx, tcp)
}
}
@@ -61,3 +61,19 @@ func (t TenantControlPlaneDataStore) check(ctx context.Context, dataStoreName st
return nil
}
func (t TenantControlPlaneDataStore) checkDataStoreOverrides(ctx context.Context, tcp *kamajiv1alpha1.TenantControlPlane) error {
overrideCheck := make(map[string]struct{}, 0)
for _, ds := range tcp.Spec.DataStoreOverrides {
if _, exists := overrideCheck[ds.Resource]; !exists {
overrideCheck[ds.Resource] = struct{}{}
} else {
return fmt.Errorf("duplicate resource override in Spec.DataStoreOverrides")
}
if err := t.check(ctx, ds.DataStore); err != nil {
return err
}
}
return nil
}

Some files were not shown because too many files have changed in this diff Show More