mirror of
https://github.com/enix/x509-certificate-exporter.git
synced 2026-08-18 19:47:28 +00:00
docs(examples): add curated values.yaml starters for generic and per-distro setups
This commit is contained in:
Thibault VINCENT
@@ -0,0 +1,51 @@
|
||||
# Helm values examples
|
||||
|
||||
Curated `values.yaml` files for the x509-certificate-exporter chart. Each
|
||||
file is a self-contained, valid input for:
|
||||
|
||||
```sh
|
||||
helm install x509-certificate-exporter \
|
||||
oci://quay.io/enix/charts/x509-certificate-exporter \
|
||||
--values <one-of-the-files>.yaml
|
||||
```
|
||||
|
||||
Pick the closest starting point, copy it into your cluster's configuration,
|
||||
**then adapt**.
|
||||
|
||||
> [!WARNING]
|
||||
> These files are **documentation, not turn-key configurations**.
|
||||
> Read each setting, understand what it does, and decide if it fits your
|
||||
> cluster before applying. Drop what you don't need; layer multiple
|
||||
> `--values` flags or merge pieces of different examples freely.
|
||||
|
||||
---
|
||||
|
||||
## Generic examples
|
||||
|
||||
| File | What it covers |
|
||||
| --- | --- |
|
||||
| [`secrets-tuned.values.yaml`](./secrets-tuned.values.yaml) | Secrets-only deployment with **every reasonable knob** turned on: namespace and label filtering, PKCS#12 with sibling-key passphrases, surfaced Secret labels, ConfigMap watching, HA via 2 replicas + anti-affinity, mTLS on `/metrics` via `webConfiguration`, custom alert thresholds, Grafana dashboard, API-server rate limits. Use as a reference for what's tunable on the cluster-watching side. |
|
||||
| [`hostpath-rich.values.yaml`](./hostpath-rich.values.yaml) | `secretsExporter` left at chart defaults **paired with three hostPath DaemonSets** carved up by node OS role: `controlplane` (kubeadm CP PKI), `storage-nodes` (Ceph daemons running as systemd units, PKI under `/etc/ceph/`), and `edge-nodes` (strongSwan IPsec terminator, X.509 in `/etc/ipsec.d/{certs,cacerts}/`). Demonstrates per-DS `nodeSelector`, conditional `tolerations:` (only where the pool is tainted), and the `watchFiles` / `watchDirectories` / `watchKubeconfFiles` triplet. Common thread: the PKI is on disk because the *node OS* manages a daemon — never a Pod misusing the host filesystem. |
|
||||
|
||||
## Distribution-specific examples
|
||||
|
||||
These target the on-node PKI layout of common Kubernetes distributions.
|
||||
They configure only `hostPathsExporter` (the cluster-level
|
||||
`secretsExporter` is distribution-agnostic — pair them with one of the
|
||||
generic examples or with the chart defaults).
|
||||
|
||||
| Distribution | File | Notes |
|
||||
| --- | --- | --- |
|
||||
| **kubeadm** (vanilla upstream) | [`distros/kubeadm.values.yaml`](./distros/kubeadm.values.yaml) | Reference layout for the kubernetes/kubernetes apiserver/etcd/kubelet PKI under `/etc/kubernetes/pki/` + `/var/lib/kubelet/pki/`. The other distro examples are mostly variations on this one. |
|
||||
| **Talos** | [`distros/talos.values.yaml`](./distros/talos.values.yaml) | Full control-plane PKI surfaced via Talos's `/system/secrets/{kubernetes,etcd}` tree (apiserver + aggregator + etcd-client material, etcd peer/server, plus the kubeadm-shaped `/etc/kubernetes/pki/ca.crt` convenience copy). Two specifics: `hostPathVolumeType: null` to skip the kubelet's pre-mount type-check (Talos's bind mounts confuse it), and **don't add `privileged: true`** — the chart's defaults already match PSA `baseline`, escalation would push past it and Talos PSA would reject the Pod. |
|
||||
| **RKE2** (Rancher) | [`distros/rke2.values.yaml`](./distros/rke2.values.yaml) | Server PKI under `/var/lib/rancher/rke2/server/tls/`, agent PKI under `/var/lib/rancher/rke2/agent/`, kubeconfig at `/etc/rancher/rke2/rke2.yaml`. Sets `hostPathVolumeType: null` because RKE2's embedded kubelet runs in its own mount namespace and the chart's default `Directory` type-check fails on the redirected paths. |
|
||||
| **k3s** (Rancher) | [`distros/k3s.values.yaml`](./distros/k3s.values.yaml) | `/var/lib/rancher/k3s/server/tls/` + `/var/lib/rancher/k3s/agent/`. Same shape as RKE2; smaller surface (single-node-friendly). |
|
||||
| **k0s** (Mirantis) | [`distros/k0s.values.yaml`](./distros/k0s.values.yaml) | `/var/lib/k0s/pki/` for the control plane, `/var/lib/k0s/kubelet/pki/` for the kubelet. Embedded etcd PKI under `/var/lib/k0s/pki/etcd/`. |
|
||||
| **OpenShift** | [`distros/openshift.values.yaml`](./distros/openshift.values.yaml) | Control-plane static-pod resources at `/etc/kubernetes/static-pod-resources/` and a kubelet client at `/var/lib/kubelet/pki/`. Requires the `hostmount-anyuid` (or `privileged`) SCC — annotated in the file. |
|
||||
|
||||
## Spot a missing distro or a wrong path?
|
||||
|
||||
Open an issue or a PR. The PKI layouts evolve from one minor release to
|
||||
the next — sometimes a path moves, sometimes a new file appears. These
|
||||
examples are best-effort snapshots tested against current releases of
|
||||
each distribution; we'd rather hear about a stale path than ship one.
|
||||
@@ -0,0 +1,43 @@
|
||||
# k0s (Mirantis) — control plane PKI under /var/lib/k0s/pki/, kubelet
|
||||
# data under /var/lib/k0s/kubelet/pki/. Embedded etcd lives at
|
||||
# /var/lib/k0s/pki/etcd/. The bundled k0s "controller+worker" mode
|
||||
# materialises both directories on the same node, in which case both
|
||||
# DaemonSets land on the same Pod (harmless — same PKI seen twice).
|
||||
#
|
||||
# References:
|
||||
# https://docs.k0sproject.io/stable/configuration/
|
||||
---
|
||||
hostPathsExporter:
|
||||
daemonSets:
|
||||
controller:
|
||||
nodeSelector:
|
||||
node-role.kubernetes.io/control-plane: "true"
|
||||
tolerations:
|
||||
- key: node-role.kubernetes.io/control-plane
|
||||
operator: Exists
|
||||
effect: NoSchedule
|
||||
watchFiles:
|
||||
# Cluster CAs
|
||||
- /var/lib/k0s/pki/ca.crt
|
||||
- /var/lib/k0s/pki/front-proxy-ca.crt
|
||||
# apiserver
|
||||
- /var/lib/k0s/pki/apiserver.crt
|
||||
- /var/lib/k0s/pki/apiserver-kubelet-client.crt
|
||||
- /var/lib/k0s/pki/front-proxy-client.crt
|
||||
# Service-account / scheduler / controller-manager client certs
|
||||
# are k0s-specific: it embeds them in admin.conf rather than
|
||||
# writing them as separate files. See watchKubeconfFiles below.
|
||||
# Embedded etcd
|
||||
- /var/lib/k0s/pki/etcd/ca.crt
|
||||
- /var/lib/k0s/pki/etcd/server.crt
|
||||
- /var/lib/k0s/pki/etcd/peer.crt
|
||||
- /var/lib/k0s/pki/etcd/apiserver-etcd-client.crt
|
||||
watchKubeconfFiles:
|
||||
- /var/lib/k0s/pki/admin.conf
|
||||
|
||||
workers:
|
||||
# nodeSelector:
|
||||
# node-role.kubernetes.io/worker: "true"
|
||||
watchFiles:
|
||||
- /var/lib/k0s/kubelet/pki/kubelet-client-current.pem
|
||||
- /var/lib/k0s/kubelet/pki/kubelet.crt
|
||||
@@ -0,0 +1,52 @@
|
||||
# k3s (Rancher) — same shape as RKE2, smaller surface. Single-node
|
||||
# clusters are common with k3s; the example keeps both DaemonSets
|
||||
# anyway so it scales when more nodes join.
|
||||
#
|
||||
# k3s rotates the kubelet client cert under
|
||||
# /var/lib/rancher/k3s/agent/client-kubelet.crt — it's a regular file,
|
||||
# not a "current.pem" symlink, so no symlink-following is needed.
|
||||
#
|
||||
# References:
|
||||
# https://docs.k3s.io/cli/certificate
|
||||
---
|
||||
hostPathsExporter:
|
||||
daemonSets:
|
||||
server:
|
||||
nodeSelector:
|
||||
node-role.kubernetes.io/control-plane: "true"
|
||||
tolerations:
|
||||
- key: node-role.kubernetes.io/control-plane
|
||||
operator: Exists
|
||||
effect: NoSchedule
|
||||
watchFiles:
|
||||
# Server-side CAs and bundles
|
||||
- /var/lib/rancher/k3s/server/tls/server-ca.crt
|
||||
- /var/lib/rancher/k3s/server/tls/client-ca.crt
|
||||
- /var/lib/rancher/k3s/server/tls/request-header-ca.crt
|
||||
# Serving certs for the apiserver
|
||||
- /var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crt
|
||||
# Client certs
|
||||
- /var/lib/rancher/k3s/server/tls/client-kube-apiserver.crt
|
||||
- /var/lib/rancher/k3s/server/tls/client-controller.crt
|
||||
- /var/lib/rancher/k3s/server/tls/client-scheduler.crt
|
||||
- /var/lib/rancher/k3s/server/tls/client-kube-proxy.crt
|
||||
# etcd — only present when k3s is configured with embedded
|
||||
# etcd (default is sqlite). Drop these lines on sqlite-backed
|
||||
# single-server installs.
|
||||
- /var/lib/rancher/k3s/server/tls/etcd/server-ca.crt
|
||||
- /var/lib/rancher/k3s/server/tls/etcd/peer-ca.crt
|
||||
- /var/lib/rancher/k3s/server/tls/etcd/client-ca.crt
|
||||
# Agent role on the server node
|
||||
- /var/lib/rancher/k3s/agent/client-kubelet.crt
|
||||
- /var/lib/rancher/k3s/agent/serving-kubelet.crt
|
||||
watchKubeconfFiles:
|
||||
- /etc/rancher/k3s/k3s.yaml
|
||||
|
||||
agents:
|
||||
# nodeSelector:
|
||||
# node-role.kubernetes.io/worker: "true"
|
||||
watchFiles:
|
||||
- /var/lib/rancher/k3s/agent/client-kubelet.crt
|
||||
- /var/lib/rancher/k3s/agent/serving-kubelet.crt
|
||||
- /var/lib/rancher/k3s/agent/client-ca.crt
|
||||
- /var/lib/rancher/k3s/agent/server-ca.crt
|
||||
@@ -0,0 +1,49 @@
|
||||
# Vanilla upstream kubeadm — control plane and worker PKI layout as
|
||||
# produced by `kubeadm init` / `kubeadm join`. Reference layout that
|
||||
# the other distro-specific files in this directory derive from.
|
||||
#
|
||||
# Two DaemonSets:
|
||||
# - controlplane: apiserver + etcd + admin/scheduler/controller
|
||||
# kubeconfigs + kubelet client
|
||||
# - workers: kubelet client only (kubeadm doesn't drop anything
|
||||
# else cert-shaped on a worker)
|
||||
#
|
||||
# Mix with a generic secrets-* file for the cluster-watching side.
|
||||
---
|
||||
hostPathsExporter:
|
||||
daemonSets:
|
||||
controlplane:
|
||||
nodeSelector:
|
||||
node-role.kubernetes.io/control-plane: ""
|
||||
tolerations:
|
||||
- key: node-role.kubernetes.io/control-plane
|
||||
operator: Exists
|
||||
effect: NoSchedule
|
||||
watchFiles:
|
||||
# apiserver
|
||||
- /etc/kubernetes/pki/apiserver.crt
|
||||
- /etc/kubernetes/pki/apiserver-etcd-client.crt
|
||||
- /etc/kubernetes/pki/apiserver-kubelet-client.crt
|
||||
- /etc/kubernetes/pki/front-proxy-client.crt
|
||||
# CAs
|
||||
- /etc/kubernetes/pki/ca.crt
|
||||
- /etc/kubernetes/pki/front-proxy-ca.crt
|
||||
- /etc/kubernetes/pki/etcd/ca.crt
|
||||
# etcd
|
||||
- /etc/kubernetes/pki/etcd/server.crt
|
||||
- /etc/kubernetes/pki/etcd/peer.crt
|
||||
- /etc/kubernetes/pki/etcd/healthcheck-client.crt
|
||||
# kubelet client (rotated symlink)
|
||||
- /var/lib/kubelet/pki/kubelet-client-current.pem
|
||||
watchKubeconfFiles:
|
||||
- /etc/kubernetes/admin.conf
|
||||
- /etc/kubernetes/super-admin.conf
|
||||
- /etc/kubernetes/controller-manager.conf
|
||||
- /etc/kubernetes/scheduler.conf
|
||||
|
||||
workers:
|
||||
# nodeSelector:
|
||||
# node-role.kubernetes.io/worker: ""
|
||||
watchFiles:
|
||||
- /etc/kubernetes/pki/ca.crt
|
||||
- /var/lib/kubelet/pki/kubelet-client-current.pem
|
||||
@@ -0,0 +1,65 @@
|
||||
# OpenShift / OKD — control plane operates as static Pods whose
|
||||
# resources are materialised under /etc/kubernetes/static-pod-resources/.
|
||||
# Kubelet client lives at /var/lib/kubelet/pki/.
|
||||
#
|
||||
# IMPORTANT — SCC: hostPathsExporter mounts host directories and runs
|
||||
# its container as root (UID 0) to read kubelet/etcd files owned by
|
||||
# root. OpenShift's default `restricted-v2` SCC forbids that. The
|
||||
# DaemonSet's ServiceAccount must be granted one of:
|
||||
# - hostmount-anyuid (recommended — minimum scope for hostPath + root)
|
||||
# - privileged (over-permissive but the easy path)
|
||||
#
|
||||
# Bind it once with:
|
||||
# oc adm policy add-scc-to-user hostmount-anyuid \
|
||||
# -z $RELEASE_NAME-node -n $NAMESPACE
|
||||
#
|
||||
# (Replace $RELEASE_NAME with your `helm install` release name and
|
||||
# $NAMESPACE with the namespace you installed into. The chart's
|
||||
# DaemonSet ServiceAccount is `<release>-node`.)
|
||||
#
|
||||
# Mix with a generic secrets-* file for the cluster-watching side. Same
|
||||
# SCC caveat does NOT apply to secretsExporter — that one runs unprivileged.
|
||||
#
|
||||
# References:
|
||||
# https://docs.openshift.com/container-platform/latest/security/certificates/api-server.html
|
||||
# https://docs.openshift.com/container-platform/latest/security/certificate_types_descriptions/etcd-certificates.html
|
||||
---
|
||||
hostPathsExporter:
|
||||
daemonSets:
|
||||
masters:
|
||||
nodeSelector:
|
||||
node-role.kubernetes.io/master: ""
|
||||
tolerations:
|
||||
- key: node-role.kubernetes.io/master
|
||||
operator: Exists
|
||||
effect: NoSchedule
|
||||
podAnnotations:
|
||||
# Required when SCC enforcement is the openshift PSA mode.
|
||||
# Has no effect on stock kubernetes.
|
||||
openshift.io/required-scc: hostmount-anyuid
|
||||
# OpenShift's static-pod resources tree is deeply nested
|
||||
# (per-revision subdirectories with the date in the name). A
|
||||
# broad watchDirectories cast catches each revision as it
|
||||
# rotates without us hard-coding paths.
|
||||
watchDirectories:
|
||||
- /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/loadbalancer-serving-ca
|
||||
- /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/localhost-serving-ca
|
||||
- /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/internal-loadbalancer-serving-ca
|
||||
- /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/service-network-serving-ca
|
||||
- /etc/kubernetes/static-pod-resources/etcd-certs/secrets/etcd-all-certs
|
||||
watchFiles:
|
||||
- /etc/kubernetes/kubelet-ca.crt
|
||||
- /var/lib/kubelet/pki/kubelet-client-current.pem
|
||||
watchKubeconfFiles:
|
||||
- /etc/kubernetes/kubeconfig
|
||||
|
||||
workers:
|
||||
# nodeSelector:
|
||||
# node-role.kubernetes.io/worker: ""
|
||||
podAnnotations:
|
||||
openshift.io/required-scc: hostmount-anyuid
|
||||
watchFiles:
|
||||
- /etc/kubernetes/kubelet-ca.crt
|
||||
- /var/lib/kubelet/pki/kubelet-client-current.pem
|
||||
watchKubeconfFiles:
|
||||
- /etc/kubernetes/kubeconfig
|
||||
@@ -0,0 +1,62 @@
|
||||
# RKE2 (Rancher) — server PKI under /var/lib/rancher/rke2/server/tls/,
|
||||
# agent PKI under /var/lib/rancher/rke2/agent/, kubeconfig in
|
||||
# /etc/rancher/rke2/. RKE2 keeps the kube-apiserver / etcd PKI in its
|
||||
# own directory tree rather than the upstream /etc/kubernetes/pki path.
|
||||
#
|
||||
# Mix with a generic secrets-* file for the cluster-watching side.
|
||||
#
|
||||
# References:
|
||||
# https://docs.rke2.io/security/certificates
|
||||
---
|
||||
hostPathsExporter:
|
||||
# RKE2 ships an embedded kubelet that runs with its own mount
|
||||
# namespace; `/var/lib/kubelet/` and `/etc/kubernetes/` are
|
||||
# redirected via bind mounts/symlinks to /var/lib/rancher/rke2/...
|
||||
# The kubelet's stat of those paths can return a mode that doesn't
|
||||
# match `type: Directory`, so the chart's default validation fails
|
||||
# ("hostPath type check failed"). Setting `null` skips that
|
||||
# pre-check (no extra access granted, just a sanity check disabled).
|
||||
hostPathVolumeType: null
|
||||
|
||||
daemonSets:
|
||||
controlplane:
|
||||
nodeSelector:
|
||||
node-role.kubernetes.io/control-plane: "true"
|
||||
tolerations:
|
||||
- key: node-role.kubernetes.io/control-plane
|
||||
operator: Exists
|
||||
effect: NoSchedule
|
||||
- key: node-role.kubernetes.io/etcd
|
||||
operator: Exists
|
||||
effect: NoExecute
|
||||
watchFiles:
|
||||
# Server-side CAs and bundles
|
||||
- /var/lib/rancher/rke2/server/tls/server-ca.crt
|
||||
- /var/lib/rancher/rke2/server/tls/client-ca.crt
|
||||
- /var/lib/rancher/rke2/server/tls/request-header-ca.crt
|
||||
# Serving certs for the apiserver
|
||||
- /var/lib/rancher/rke2/server/tls/serving-kube-apiserver.crt
|
||||
# Client certs
|
||||
- /var/lib/rancher/rke2/server/tls/client-kube-apiserver.crt
|
||||
- /var/lib/rancher/rke2/server/tls/client-controller.crt
|
||||
- /var/lib/rancher/rke2/server/tls/client-scheduler.crt
|
||||
- /var/lib/rancher/rke2/server/tls/client-kube-proxy.crt
|
||||
# etcd — RKE2 writes per-node files; the kube-apiserver client
|
||||
# cert lives next to the rest of the server TLS.
|
||||
- /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt
|
||||
- /var/lib/rancher/rke2/server/tls/etcd/peer-ca.crt
|
||||
- /var/lib/rancher/rke2/server/tls/etcd/client-ca.crt
|
||||
# kubelet client + serving (agent role on the CP node too)
|
||||
- /var/lib/rancher/rke2/agent/client-kubelet.crt
|
||||
- /var/lib/rancher/rke2/agent/serving-kubelet.crt
|
||||
watchKubeconfFiles:
|
||||
- /etc/rancher/rke2/rke2.yaml
|
||||
|
||||
agents:
|
||||
# nodeSelector:
|
||||
# node-role.kubernetes.io/worker: "true"
|
||||
watchFiles:
|
||||
- /var/lib/rancher/rke2/agent/client-kubelet.crt
|
||||
- /var/lib/rancher/rke2/agent/serving-kubelet.crt
|
||||
- /var/lib/rancher/rke2/agent/client-ca.crt
|
||||
- /var/lib/rancher/rke2/agent/server-ca.crt
|
||||
@@ -0,0 +1,75 @@
|
||||
# Talos Linux — full control-plane PKI coverage via hostPath.
|
||||
#
|
||||
# Talos keeps the cluster PKI under its own /system/secrets/... tree
|
||||
# (apiserver + aggregator + etcd-client material under
|
||||
# /system/secrets/kubernetes/, etcd peer/server certs under
|
||||
# /system/secrets/etcd/), with a convenience `/etc/kubernetes/pki/ca.crt`
|
||||
# copy for kubeadm-shaped tooling. All of those paths are reachable
|
||||
# from a hostPath mount when the kubelet's pre-mount type-check is
|
||||
# disabled — which is what `hostPathVolumeType: null` below does.
|
||||
#
|
||||
# Two things to know:
|
||||
#
|
||||
# 1. The chart's default `hostPath.type: Directory` makes the
|
||||
# kubelet stat each watched parent directory before mounting.
|
||||
# Talos's mount namespace returns a type that doesn't match
|
||||
# "vanilla directory", so the check fails. `null` removes the
|
||||
# check (no extra privilege granted, just a sanity check skipped).
|
||||
#
|
||||
# 2. PSA enforcement: Talos runs Pod Security Admission on user
|
||||
# namespaces. The chart's defaults (runAsUser: 0 +
|
||||
# readOnlyRootFilesystem + drop ALL caps + seccompProfile:
|
||||
# RuntimeDefault + allowPrivilegeEscalation: false) sit at the
|
||||
# `baseline` profile, which Talos accepts in any namespace. Do
|
||||
# NOT add `privileged: true` — it would push the Pod beyond
|
||||
# `baseline` and Talos's PSA would reject it.
|
||||
#
|
||||
#
|
||||
# References:
|
||||
# https://www.talos.dev/latest/learn-more/control-plane/
|
||||
# https://www.talos.dev/latest/talos-guides/howto/cert-management/
|
||||
---
|
||||
hostPathsExporter:
|
||||
# Talos's kubelet runs in its own mount namespace; many host paths
|
||||
# are bind mounts or symlinks back to /system/... — kubelet's stat
|
||||
# of those returns something that doesn't match `type: Directory`,
|
||||
# so the chart's default validation fails ("hostPath type check
|
||||
# failed"). Setting `null` skips that pre-check (no extra access
|
||||
# granted, just a sanity check disabled).
|
||||
hostPathVolumeType: null
|
||||
|
||||
daemonSets:
|
||||
controlplane:
|
||||
nodeSelector:
|
||||
node-role.kubernetes.io/control-plane: ""
|
||||
tolerations:
|
||||
- key: node-role.kubernetes.io/control-plane
|
||||
operator: Exists
|
||||
effect: NoSchedule
|
||||
watchFiles:
|
||||
# apiserver — serving + client certs to other components
|
||||
- /system/secrets/kubernetes/kube-apiserver/apiserver.crt
|
||||
- /system/secrets/kubernetes/kube-apiserver/apiserver-kubelet-client.crt
|
||||
- /system/secrets/kubernetes/kube-apiserver/etcd-client.crt
|
||||
- /system/secrets/kubernetes/kube-apiserver/front-proxy-client.crt
|
||||
# CAs — the canonical copies live under /system/secrets/...;
|
||||
# /etc/kubernetes/pki/ca.crt is a convenience copy Talos
|
||||
# exposes for compatibility with kubeadm-shaped tooling.
|
||||
- /etc/kubernetes/pki/ca.crt
|
||||
- /system/secrets/kubernetes/kube-apiserver/ca.crt
|
||||
- /system/secrets/kubernetes/kube-apiserver/aggregator-ca.crt
|
||||
- /system/secrets/kubernetes/kube-apiserver/etcd-client-ca.crt
|
||||
- /system/secrets/etcd/ca.crt
|
||||
# etcd — server + peer-mesh certs, plus the etcdctl admin cert
|
||||
- /system/secrets/etcd/server.crt
|
||||
- /system/secrets/etcd/peer.crt
|
||||
- /system/secrets/etcd/admin.crt
|
||||
# kubelet client (rotated symlink, resolved on every walk)
|
||||
- /var/lib/kubelet/pki/kubelet-client-current.pem
|
||||
|
||||
workers:
|
||||
# nodeSelector:
|
||||
# node-role.kubernetes.io/worker: ""
|
||||
watchFiles:
|
||||
- /etc/kubernetes/pki/ca.crt
|
||||
- /var/lib/kubelet/pki/kubelet-client-current.pem
|
||||
@@ -0,0 +1,122 @@
|
||||
# Classic secretsExporter + rich on-node coverage.
|
||||
#
|
||||
# The cluster-watching side runs with chart defaults (sane for a typical
|
||||
# cluster). The interesting part is `hostPathsExporter`, with three
|
||||
# DaemonSets carved up by node OS role — each role hosts different
|
||||
# systemd-managed daemons whose PKI lives on disk by design:
|
||||
#
|
||||
# - `controlplane` → vanilla kubeadm PKI on the control-plane nodes
|
||||
# (apiserver, etcd, kubelet client, cluster-admin
|
||||
# / scheduler / controller-manager kubeconfigs).
|
||||
# - `storage-nodes` → workers that host Ceph daemons (mon / OSD /
|
||||
# mgr) as systemd units co-located with the
|
||||
# kubelet. Ceph's TLS lives under /etc/ceph/.
|
||||
# - `edge-nodes` → workers in remote sites that terminate a
|
||||
# site-to-site IPsec tunnel via strongSwan
|
||||
# (systemd unit). X.509 material is in
|
||||
# /etc/ipsec.d/certs/.
|
||||
#
|
||||
# Common thread: in all three, the PKI is on disk because the *node OS*
|
||||
# manages a daemon — not because a Pod is misusing the host filesystem.
|
||||
# Each DaemonSet sets its own `nodeSelector`; `controlplane` and
|
||||
# `storage-nodes` also carry a `tolerations:` block to clear the
|
||||
# matching node taint (control-plane convention + dedicated-storage
|
||||
# pool). Edge nodes are typically untainted, so they get a selector
|
||||
# only.
|
||||
---
|
||||
# ─── Cluster-watching side: keep it boring ────────────────────────────────────
|
||||
secretsExporter:
|
||||
# Tautological override — kept as a visible anchor. The chart already
|
||||
# defaults to `enabled: true` (cluster-wide `kubernetes.io/tls` watch
|
||||
# + ServiceMonitor + PrometheusRule). For filtering, PKCS#12 and
|
||||
# mTLS scrape tuning, see `secrets-tuned.yaml`.
|
||||
enabled: true
|
||||
|
||||
# ─── On-node PKI: three role-specific DaemonSets ──────────────────────────────
|
||||
hostPathsExporter:
|
||||
# Cluster-wide defaults inherited by every entry under daemonSets:.
|
||||
# Each DS can override individually.
|
||||
resources:
|
||||
requests:
|
||||
cpu: 5m
|
||||
memory: 16Mi
|
||||
limits:
|
||||
memory: 40Mi
|
||||
|
||||
daemonSets:
|
||||
# ─── 1. Control plane ─────────────────────────────────────────────────────
|
||||
controlplane:
|
||||
nodeSelector:
|
||||
node-role.kubernetes.io/control-plane: ""
|
||||
tolerations:
|
||||
- key: node-role.kubernetes.io/control-plane
|
||||
operator: Exists
|
||||
effect: NoSchedule
|
||||
podExtraLabels:
|
||||
x509-exporter.example.com/role: controlplane
|
||||
watchFiles:
|
||||
# apiserver
|
||||
- /etc/kubernetes/pki/apiserver.crt
|
||||
- /etc/kubernetes/pki/apiserver-etcd-client.crt
|
||||
- /etc/kubernetes/pki/apiserver-kubelet-client.crt
|
||||
- /etc/kubernetes/pki/front-proxy-client.crt
|
||||
# CAs
|
||||
- /etc/kubernetes/pki/ca.crt
|
||||
- /etc/kubernetes/pki/front-proxy-ca.crt
|
||||
- /etc/kubernetes/pki/etcd/ca.crt
|
||||
# etcd
|
||||
- /etc/kubernetes/pki/etcd/server.crt
|
||||
- /etc/kubernetes/pki/etcd/peer.crt
|
||||
- /etc/kubernetes/pki/etcd/healthcheck-client.crt
|
||||
# kubelet client (rotated symlink)
|
||||
- /var/lib/kubelet/pki/kubelet-client-current.pem
|
||||
watchKubeconfFiles:
|
||||
- /etc/kubernetes/admin.conf
|
||||
- /etc/kubernetes/super-admin.conf
|
||||
- /etc/kubernetes/controller-manager.conf
|
||||
- /etc/kubernetes/scheduler.conf
|
||||
|
||||
# ─── 2. Storage nodes (Ceph daemons in systemd) ───────────────────────────
|
||||
# Workers tagged as Ceph hosts run mon / OSD / mgr daemons directly
|
||||
# on the node OS (cephadm or manual install — NOT Rook, which is
|
||||
# Pod-level and uses Secrets). Their TLS lives under /etc/ceph/.
|
||||
# The exact filenames depend on the deployment style; the
|
||||
# directory-watch covers every PEM that ends up there.
|
||||
storage-nodes:
|
||||
nodeSelector:
|
||||
node-role.kubernetes.io/storage: ""
|
||||
# Storage pools are typically tainted to keep general workloads
|
||||
# off the disks; clear the taint so the DaemonSet can run there.
|
||||
tolerations:
|
||||
- key: node-role.kubernetes.io/storage
|
||||
operator: Exists
|
||||
effect: NoSchedule
|
||||
podExtraLabels:
|
||||
x509-exporter.example.com/role: storage
|
||||
watchDirectories:
|
||||
- /etc/ceph
|
||||
# Plus the kubelet client on every worker.
|
||||
watchFiles:
|
||||
- /var/lib/kubelet/pki/kubelet-client-current.pem
|
||||
|
||||
# ─── 3. Edge nodes (strongSwan / IPsec site-to-site) ──────────────────────
|
||||
# Workers deployed in remote sites terminate an IKEv2 tunnel back to
|
||||
# the cluster. strongSwan runs as a systemd unit and reads its X.509
|
||||
# material from /etc/ipsec.d/certs/ (peer certs) and
|
||||
# /etc/ipsec.d/cacerts/ (CAs). Surfacing those alongside the
|
||||
# kubelet's own client cert gives one-stop-shop coverage of the
|
||||
# node-side cryptographic surface.
|
||||
edge-nodes:
|
||||
nodeSelector:
|
||||
x509-exporter.example.com/role: edge
|
||||
podExtraLabels:
|
||||
x509-exporter.example.com/role: edge
|
||||
watchDirectories:
|
||||
- /etc/ipsec.d/certs
|
||||
- /etc/ipsec.d/cacerts
|
||||
watchFiles:
|
||||
- /var/lib/kubelet/pki/kubelet-client-current.pem
|
||||
|
||||
# ─── Grafana dashboard ────────────────────────────────────────────────────────
|
||||
grafana:
|
||||
createDashboard: true
|
||||
@@ -0,0 +1,223 @@
|
||||
# Secrets-only, fully tuned.
|
||||
#
|
||||
# Everything the cluster-watching side of the chart can do, in one
|
||||
# place. No `hostPathsExporter` block — the chart's default
|
||||
# (`daemonSets: {}`) produces nothing on its own, so there's nothing
|
||||
# to disable here.
|
||||
#
|
||||
# Walk through every section before applying — most fields are
|
||||
# placeholders (made-up node-selector labels, sample namespace opt-in
|
||||
# labels, illustrative resources, Alertmanager routing labels,
|
||||
# Secret names for the mTLS / BasicAuth scrape credentials, …).
|
||||
# Inline comments call out what each one expects.
|
||||
---
|
||||
# ─── secretsExporter: workload + scope + hardening ────────────────────────────
|
||||
secretsExporter:
|
||||
enabled: true
|
||||
|
||||
# Two replicas for HA. The exporter is idempotent against the same
|
||||
# informer state, so two replicas double-count nothing — they just
|
||||
# double the scrape capacity and survive a node drain.
|
||||
replicas: 2
|
||||
|
||||
# Anti-affinity so the two replicas never co-locate on one node.
|
||||
affinity:
|
||||
podAntiAffinity:
|
||||
preferredDuringSchedulingIgnoredDuringExecution:
|
||||
- weight: 100
|
||||
podAffinityTerm:
|
||||
topologyKey: kubernetes.io/hostname
|
||||
labelSelector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: x509-certificate-exporter
|
||||
app.kubernetes.io/component: secrets
|
||||
|
||||
# Schedule on observability-tier nodes if you have that pool.
|
||||
nodeSelector:
|
||||
workload-tier: observability
|
||||
|
||||
tolerations:
|
||||
- key: workload-tier
|
||||
operator: Equal
|
||||
value: observability
|
||||
effect: NoSchedule
|
||||
|
||||
priorityClassName: monitoring-cluster-critical
|
||||
|
||||
resources:
|
||||
requests:
|
||||
cpu: 50m
|
||||
memory: 64Mi
|
||||
limits:
|
||||
memory: 256Mi
|
||||
|
||||
# ─── What to watch ──────────────────────────────────────────────────────────
|
||||
# Multiple Secret types, demonstrating every passphrase strategy.
|
||||
secretTypes:
|
||||
# Standard PEM in kubernetes.io/tls (cert-manager output, ingress, …)
|
||||
- type: kubernetes.io/tls
|
||||
key: tls.crt
|
||||
|
||||
# Opaque PEM under a non-conventional key (legacy bundles)
|
||||
- type: Opaque
|
||||
key: cert.pem
|
||||
|
||||
# Multiple keys via regex — handy for full-chain bundles next to
|
||||
# individual leaves
|
||||
- type: Opaque
|
||||
keyPatterns:
|
||||
- '^.*\.crt$'
|
||||
- '^.*\.pem$'
|
||||
|
||||
# PKCS#12 with passphrase as a sibling key in the same Secret
|
||||
- type: Opaque
|
||||
key: keystore.p12
|
||||
format: pkcs12
|
||||
pkcs12:
|
||||
passphraseKey: keystore-passphrase
|
||||
|
||||
# PKCS#12 truststore — same passphrase pattern, different filename
|
||||
- type: Opaque
|
||||
key: truststore.p12
|
||||
format: pkcs12
|
||||
pkcs12:
|
||||
passphraseKey: keystore-passphrase
|
||||
|
||||
# PKCS#12 with passphrase pulled from a separate Secret in another
|
||||
# namespace — useful when the keystore lives in a workload namespace
|
||||
# but the passphrase is centrally managed in `secrets-pkcs12-vault`.
|
||||
- type: Opaque
|
||||
key: vault.p12
|
||||
format: pkcs12
|
||||
pkcs12:
|
||||
passphraseSecretRef:
|
||||
namespace: secrets-pkcs12-vault
|
||||
name: shared-keystore-passphrase
|
||||
key: passphrase
|
||||
|
||||
# Passwordless PKCS#12 — try the empty passphrase before erroring out.
|
||||
- type: Opaque
|
||||
key: keystore-empty.p12
|
||||
format: pkcs12
|
||||
pkcs12:
|
||||
tryEmptyPassphrase: true
|
||||
|
||||
# ConfigMaps holding PEM material (e.g. cluster-issuer CA bundles)
|
||||
configMapKeys:
|
||||
- tls.crt
|
||||
- ca.crt
|
||||
|
||||
# ─── Filtering ──────────────────────────────────────────────────────────────
|
||||
# Excluding kube-system / kube-public is conventional — those Secrets
|
||||
# are mostly bootstrap tokens whose expiry the cluster manages itself.
|
||||
excludeNamespaces:
|
||||
- kube-system
|
||||
- kube-public
|
||||
- kube-node-lease
|
||||
|
||||
# Opt-in via namespace label: only scan namespaces that have explicitly
|
||||
# opted in. Comment out if you want cluster-wide-by-default.
|
||||
includeNamespaceLabels:
|
||||
- "x509-certificate-exporter/watch=true"
|
||||
|
||||
# Per-Secret opt-out — even in an opted-in namespace, individual
|
||||
# Secrets can be skipped.
|
||||
excludeLabels:
|
||||
- "x509-certificate-exporter/ignore=true"
|
||||
|
||||
# ─── Surface Secret metadata onto metrics ───────────────────────────────────
|
||||
# Lifts these labels from each Secret onto its emitted series, prefixed
|
||||
# with `secret_label_`. Useful for routing alerts by team / service.
|
||||
exposeSecretLabels:
|
||||
- app.kubernetes.io/name
|
||||
- app.kubernetes.io/instance
|
||||
- team
|
||||
- environment
|
||||
|
||||
# ─── API-server load shaping ────────────────────────────────────────────────
|
||||
# Throttle client-go to friendly defaults; matters on clusters with
|
||||
# 100k+ Secrets and a beefy `--include-namespace-labels` selector that
|
||||
# forces a wide initial list.
|
||||
kubeApiRateLimits:
|
||||
enabled: true
|
||||
queriesPerSecond: 10
|
||||
burstQueries: 20
|
||||
|
||||
# Internal parse cache — keeps re-parsing identical Secret payloads
|
||||
# cheap when watch events fire on unrelated Secret fields. Chart
|
||||
# defaults (enabled: true, maxDuration: 300) suit most clusters; bump
|
||||
# `maxDuration` if Secrets are very chatty on non-cert fields and you
|
||||
# want to lower CPU at the cost of slightly staler error counters.
|
||||
cache:
|
||||
maxDuration: 3600 # seconds
|
||||
|
||||
# ─── Pod customisation ──────────────────────────────────────────────────────
|
||||
# Mount server TLS material referenced by webConfiguration below.
|
||||
# Adapt the Secret name to your cert-manager Certificate /
|
||||
# SealedSecret / ExternalSecrets setup.
|
||||
extraVolumes:
|
||||
- name: server-tls
|
||||
secret:
|
||||
secretName: x509-exporter-server-tls
|
||||
extraVolumeMounts:
|
||||
- name: server-tls
|
||||
mountPath: /mnt/tls
|
||||
readOnly: true
|
||||
|
||||
# ─── Securing the /metrics endpoint (mTLS + BasicAuth) ────────────────────────
|
||||
# Web configuration consumed by prometheus/exporter-toolkit. The chart
|
||||
# wires this as a Secret + volume mount + `--web.config.file`.
|
||||
webConfiguration: |
|
||||
basic_auth_users:
|
||||
# Generate with: htpasswd -nBC 12 prometheus
|
||||
prometheus: $2y$12$REPLACE_WITH_BCRYPT_HASH
|
||||
|
||||
tls_server_config:
|
||||
cert_file: /mnt/tls/tls.crt
|
||||
key_file: /mnt/tls/tls.key
|
||||
client_auth_type: RequireAndVerifyClientCert
|
||||
client_ca_file: /mnt/tls/ca.crt
|
||||
|
||||
prometheusServiceMonitor:
|
||||
# mTLS + BasicAuth wiring — tell Prometheus how to reach /metrics.
|
||||
# The Secret named here must contain `username`, `password`, and the
|
||||
# CA / client cert / client key — see
|
||||
# https://prometheus-operator.dev/docs/api-reference/api/#monitoring.coreos.com/v1.Endpoint
|
||||
basicAuth:
|
||||
username:
|
||||
name: x509-exporter-scrape-creds
|
||||
key: username
|
||||
password:
|
||||
name: x509-exporter-scrape-creds
|
||||
key: password
|
||||
tlsConfig:
|
||||
caFile: /etc/prometheus/secrets/x509-exporter-scrape-ca/ca.crt
|
||||
serverName: x509-certificate-exporter.monitoring.svc
|
||||
|
||||
# ─── PrometheusRule (alerting thresholds + routing labels) ────────────────────
|
||||
prometheusRules:
|
||||
# Tighter renewal alert (default: 28 days). Tune to match your cert
|
||||
# rotation SLO — for cert-manager + LE, 14 days is plenty.
|
||||
warningDaysLeft: 14
|
||||
criticalDaysLeft: 7
|
||||
alertOnReadErrors: true
|
||||
alertOnCertificateError: true
|
||||
alertOnCertificateRenewal: true
|
||||
alertOnCertificateExpiration: true
|
||||
# Add labels that your Alertmanager router uses to route to the right
|
||||
# team channel.
|
||||
extraLabels:
|
||||
team: platform
|
||||
service: x509-certificate-exporter
|
||||
|
||||
# ─── Metrics shape ────────────────────────────────────────────────────────────
|
||||
# Per-cert error metrics: enable when you want one failed-cert series
|
||||
# per offending Secret in addition to the source-level counters. Useful
|
||||
# in dashboards but inflates cardinality on ~thousands of bad certs.
|
||||
exposePerCertificateErrorMetrics: true
|
||||
|
||||
# ─── Grafana dashboard ────────────────────────────────────────────────────────
|
||||
# Materialise the bundled dashboard as a sidecar-discovered ConfigMap.
|
||||
# Requires Grafana running with the dashboards sidecar enabled.
|
||||
grafana:
|
||||
createDashboard: true
|
||||
Reference in New Issue
Block a user