# Certificates Lifecycle Kamaji is responsible for creating the required certificates, such as: - the Kubernetes API Server certificate - the Kubernetes API Server kubelet client certificate - the Datastore certificate - the front proxy client certificate - the konnectivity certificate (if enabled) Also, the following `kubeconfig` resources contain client certificates, which are created by Kamaji, such as: - `admin` - `controller-manager` - `konnectivity` (if enabled) - `scheduler` All the certificates are created with the `kubeadm` defaults, thus their validity is set to 1 year. ## How to rotate certificates All certificates can be rotated at the same time, or one by one: this is possible by annotating resources using the well-known annotation `certs.kamaji.clastix.io/rotate`. ``` $: kubectl get secret NAME TYPE DATA AGE k8s-133-admin-kubeconfig Opaque 1 12m k8s-133-api-server-certificate Opaque 2 12m k8s-133-api-server-kubelet-client-certificate Opaque 2 3h45m k8s-133-ca Opaque 4 3h45m k8s-133-controller-manager-kubeconfig Opaque 1 3h45m k8s-133-datastore-certificate Opaque 3 3h45m k8s-133-datastore-config Opaque 4 3h45m k8s-133-front-proxy-ca-certificate Opaque 2 3h45m k8s-133-front-proxy-client-certificate Opaque 2 3h45m k8s-133-konnectivity-certificate kubernetes.io/tls 2 3h45m k8s-133-konnectivity-kubeconfig Opaque 1 3h45m k8s-133-sa-certificate Opaque 2 3h45m k8s-133-scheduler-kubeconfig Opaque 1 3h45m ``` Once this operation is performed, Kamaji will trigger a certificate renewal, reporting the rotation date time as the annotation `certs.kamaji.clastix.io/rotate` value in the [RFC3339](https://pkg.go.dev/time#RFC3339) format. ``` $: kubectl annotate secret -l kamaji.clastix.io/certificate_lifecycle_controller=x509 certs.kamaji.clastix.io/rotate="" secret/k8s-133-api-server-certificate annotated secret/k8s-133-api-server-kubelet-client-certificate annotated secret/k8s-133-datastore-certificate annotated secret/k8s-133-front-proxy-client-certificate annotated secret/k8s-133-konnectivity-certificate annotated $: kubectl get secrets -l kamaji.clastix.io/certificate_lifecycle_controller=x509 -ojson | jq -r '.items[] | "\(.metadata.name) rotated at \(.metadata.annotations["certs.kamaji.clastix.io/rotate"])"' k8s-133-api-server-certificate rotated at 2025-07-15T15:15:08Z02:00 k8s-133-api-server-kubelet-client-certificate rotated at 2025-07-15T15:15:10Z0200 k8s-133-datastore-certificate rotated at 2025-07-15T15:15:15Z0200 k8s-133-front-proxy-client-certificate rotated at 2025-07-15T15:15:13Z0200 k8s-133-konnectivity-certificate rotated at 2025-07-15T15:15:17Z0200 ``` You can notice the secrets have been automatically created back, as well as a TenantControlPlane rollout with the updated certificates. ``` $: kubectl get pods NAME READY STATUS RESTARTS AGE k8s-133-67bf496c8c-27bmp 4/4 Running 0 4m52s k8s-133-67bf496c8c-x4t76 4/4 Running 0 4m52s ``` The same occurs with the `kubeconfig` ones. ``` $: kubectl annotate secret -l kamaji.clastix.io/certificate_lifecycle_controller=kubeconfig certs.kamaji.clastix.io/rotate="" secret/k8s-133-admin-kubeconfig annotated secret/k8s-133-controller-manager-kubeconfig annotated secret/k8s-133-konnectivity-kubeconfig annotated secret/k8s-133-scheduler-kubeconfig annotated $: kubectl get secrets -l kamaji.clastix.io/certificate_lifecycle_controller=kubeconfig -ojson | jq -r '.items[] | "\(.metadata.name) rotated at \(.metadata.annotations["certs.kamaji.clastix.io/rotate"])"' k8s-133-admin-kubeconfig rotated at 2025-07-15 15:20:41.688181782 +0200 CEST m=+658.630990441 k8s-133-controller-manager-kubeconfig rotated at 2025-07-15 15:20:42.712211056 +0200 CEST m=+659.655019677 k8s-133-konnectivity-kubeconfig rotated at 2025-07-15 15:20:46.405567865 +0200 CEST m=+663.348376504 k8s-133-scheduler-kubeconfig rotated at 2025-07-15 15:20:46.333718563 +0200 CEST m=+663.276527216 ``` ## Automatic certificates rotation The Kamaji operator will run a controller which processes all the Secrets to determine their expiration, both for the `kubeconfig`, as well as for the certificates. The controller, named `CertificateLifecycle`, will extract the certificates from the _Secret_ objects notifying the `TenantControlPlaneReconciler` controller which will start a new certificate rotation. By default, the rotation will occur the day before their expiration. This rotation deadline can be dynamically configured using the Kamaji CLI flag `--certificate-expiration-deadline` using the Go _Duration_ syntax: e.g.: set the value `7d` to trigger the renewal a week before the effective expiration date. !!! info "Other Datastore Drivers" Kamaji is responsible for creating the `etcd` client certificate, and the generation of a new one will occur. For other Datastore drivers, such as MySQL, PostgreSQL, or NATS, the referenced Secret will always be deleted by the Controller to trigger the rotation: the PKI management, since it's offloaded externally, must provide the renewed certificates. ## Certificate Authority rotation Kamaji is also taking care of your Tenant Clusters Certificate Authority. This can be rotated manually like other certificates by using the annotation `certs.kamaji.clastix.io/rotate` ``` $: kubectl annotate secret k8s-133-ca certs.kamaji.clastix.io/rotate="" secret/k8s-133-ca annotated ``` Once this occurs the TenantControlPlane will enter in the `CertificateAuthorityRotating` status. ``` $: kubectl get tcp -w NAME VERSION STATUS CONTROL-PLANE ENDPOINT KUBECONFIG DATASTORE AGE k8s-133 v1.33.0 Ready 172.18.255.200:6443 k8s-133-admin-kubeconfig default 3h58m k8s-133 v1.33.0 CertificateAuthorityRotating 172.18.255.200:6443 k8s-133-admin-kubeconfig default 3h58m k8s-133 v1.33.0 CertificateAuthorityRotating 172.18.255.200:6443 k8s-133-admin-kubeconfig default 3h58m k8s-133 v1.33.0 CertificateAuthorityRotating 172.18.255.200:6443 k8s-133-admin-kubeconfig default 3h58m k8s-133 v1.33.0 CertificateAuthorityRotating 172.18.255.200:6443 k8s-133-admin-kubeconfig default 3h58m k8s-133 v1.33.0 CertificateAuthorityRotating 172.18.255.200:6443 k8s-133-admin-kubeconfig default 3h58m k8s-133 v1.33.0 CertificateAuthorityRotating 172.18.255.200:6443 k8s-133-admin-kubeconfig default 3h58m k8s-133 v1.33.0 CertificateAuthorityRotating 172.18.255.200:6443 k8s-133-admin-kubeconfig default 3h58m k8s-133 v1.33.0 CertificateAuthorityRotating 172.18.255.200:6443 k8s-133-admin-kubeconfig default 3h58m k8s-133 v1.33.0 CertificateAuthorityRotating 172.18.255.200:6443 k8s-133-admin-kubeconfig default 3h58m k8s-133 v1.33.0 CertificateAuthorityRotating 172.18.255.200:6443 k8s-133-admin-kubeconfig default 3h58m k8s-133 v1.33.0 CertificateAuthorityRotating 172.18.255.200:6443 k8s-133-admin-kubeconfig default 3h58m k8s-133 v1.33.0 Ready 172.18.255.200:6443 k8s-133-admin-kubeconfig default 3h58m k8s-133 v1.33.0 Ready 172.18.255.200:6443 k8s-133-admin-kubeconfig default 3h58m k8s-133 v1.33.0 Ready 172.18.255.200:6443 k8s-133-admin-kubeconfig default 3h58m k8s-133 v1.33.0 Ready 172.18.255.200:6443 k8s-133-admin-kubeconfig default 3h58m k8s-133 v1.33.0 Ready 172.18.255.200:6443 k8s-133-admin-kubeconfig default 3h58m ``` This operation is intended to be performed manually since a new Certificate Authority requires the restart of all the components, as well as of the nodes: in such a case, you will need to distribute the new Certificate Authority and the new nodes certificates. Given the sensibility of such operation, the `Secret` controller will not check the _CA_, which is offering validity of 10 years as `kubeadm` default values.