Five user-facing counters had a UX hit: prometheus.NewCounterVec only
materialises a series the first time WithLabelValues(...).Inc() is
called, so an exporter running cleanly produced no series at all for
x509_source_errors_total, x509_kube_transport_errors_total,
x509_cert_collision_dropped_total and the two
x509_{pkcs12,jks}_passphrase_failures_total. Dashboards couldn't tell
'healthy' from 'metric not reporting', rate()/increase() needed two
real events to compute anything, and the chart's
SourceErrors[Sustained] / KubeTransportErrors[Sustained] alerts had
ambiguous PromQL semantics on first event.
PreInitBundleSource(kind, name) and PreInitKubeTransport(name,
resources, namespaceInformer) on Registry materialise the expected
series at zero by calling WithLabelValues without .Inc(). The static
reason sets they iterate (cert.BundleReasons, cert.KubeTransportPerResourceReasons,
cert.ReasonNamespaceSyncFail) live in pkg/cert/reason.go for a single
source of truth — dynamic 'http_NNN' reasons are deliberately
excluded since they can't be enumerated.
cmd/x509-certificate-exporter/main.go calls the right combination
for each config.Source kind right after buildSource: kubernetes
sources get both bundle init (kube-secret, kube-configmap) and
transport init (per-resource + namespace informer when label rules
require it); file/kubeconfig/cabundle sources just get the bundle
init for their respective kind. Cardinality cost is bounded by
declared reasons × source count — about 50 series per source — and
negligible against the per-cert series the exporter produces in
normal operation.
TestPreInitMaterializesZeroSeries in pkg/registry locks in the exact
series count materialised by a representative call and asserts every
emitted counter value is 0.
The Kubernetes source's transport layer had 9 distinct failure sites
that surfaced only as logs: LIST failure (with retry/backoff), WATCH
start failure, WATCH stream Error event, watch flap (close within 5s
of open), and namespace informer sync timeout — once for secrets, once
for configmaps. None had a corresponding metric, so an exporter that
was functionally up but operationally degraded (token rotation drift,
apiserver throttling, intermittent network) was invisible to ops.
Introduce x509_kube_transport_errors_total{source_name,resource,reason}
incremented at each of the 9 sites via a new Recorder interface on
k8s.Options. The interface keeps the source decoupled from
*registry.Registry; nil is valid (the unmetered path keeps the old
log-only behaviour) and tests plug a fake. main.go wires the actual
registry through.
Ship a paired KubeTransportErrors PrometheusRule alert (opt-in via
alertOnKubeTransportErrors, default true). docs/metrics.md documents
the new metric, the five reason codes, and cross-links the alert.
Detect-only x509_cert_collision_total ticks on every scrape under the
default CollisionAuto policy, even when the discriminator label
silently resolves the overlap and no certificate is dropped. Alerting
on it produced perma-firing pages for a benign config quirk.
Introduce x509_cert_collision_dropped_total: counts only the items
the registry actually threw away (CollisionNever policy). The
CertificateCollision PrometheusRule now points at the new counter so
the alert fires only when data is genuinely lost. The detect-only
counter stays available for dashboards and diagnostics.
Tests in pkg/registry cement the contract: dropped stays at 0 under
CollisionAuto and is positive under CollisionNever. docs/metrics.md
documents both counters and cross-links them.
Same root cause as the passphraseKey fix: when resolveRef fails and
tryEmptyPassphrase is false, the parser was being called with an empty
passphrase, producing a misleading digest error.
resolveRef now returns bool; call sites propagate failure into
passphraseUnavailable using the relevant TryEmpty flag. Tests updated
to assert the new (correct) skip behavior and cover the tryEmpty:true
path that must still fall through.
When passphraseKey is configured but the key is absent from the Secret
and tryEmptyPassphrase is false, the parser was called with an empty
passphrase anyway, producing a misleading 'invalid digest' error. Now
the source emits a bad_passphrase bundle error directly, and the
preceding debug log already names the missing key.
Thread ctx through onSecret→fetchPassphrase (contextcheck); suppress
SHA-1 gosec warnings in jceksHMAC with nosec (protocol-mandated by
JKS/JCEKS wire format); drop dead skip initialiser in decodeJCEKS
(ineffassign); remove redundant int64 cast on UnixMilli (unconvert).
Complete rewrite of the codebase, the build pipeline, the dev loop,
and the release pipeline.
For the exporter itself, refer to the updated README and Helm chart
documentation to discover the new functionality and assess the impact
of the breaking changes on your existing setup.
Build & release:
- QA/CI pipelines now run through a Dagger Module, wrapped by
Taskfile.yml for the developer interface.
- Releases run through GoReleaser: cross-compiled binaries × OS/arch,
archives, checksums, multi-arch container images (busybox + scratch
variants on linux/amd64,arm64,riscv64), pushed to ghcr/quay/docker.io.
- Everything is cosign-signed (binaries, images, Helm chart). Image
CycloneDX SBOMs are attached as cosign attestations. SLSA-3
provenance is attached to every GitHub Release.
- The Helm chart is published as a cosign-signed OCI artifact.
- Versioning and changelog are automated by release-please from
Conventional Commits.
Dev experience:
- Local loop driven by Tilt + k3d + Dagger; one command brings up an
exporter with seeded fixtures and a Prometheus scraping it.
- End-to-end tests run on a throwaway k3d cluster against the real
rendered chart.
BREAKING CHANGE: the Helm chart is now published exclusively as an OCI
artifact at oci://quay.io/enix/charts/x509-certificate-exporter. The
legacy Helm repository at https://charts.enix.io is no longer updated;
users must switch to the OCI reference (Helm 3.8+ required).
Installation: `helm install x509-certificate-exporter
oci://quay.io/enix/charts/x509-certificate-exporter --version <vX.Y.Z>`.
BREAKING CHANGE: the Helm chart's values schema may diverge from v3 in
edge cases despite a best-effort to preserve backwards compatibility.
Review your existing values against the updated chart/values.yaml
before upgrading. A JSON schema (chart/values.schema.json) is shipped
with the chart so `helm install` / `helm upgrade` will reject any
values that no longer match the expected shape, surfacing regressions
early instead of at runtime.
BREAKING CHANGE: Alpine-based container images are no longer published.
The release pipeline now ships only the `busybox` and `scratch` variants
on linux/amd64,arm64,riscv64. Users pulling `*-alpine` tags must switch
to one of the new variants — `busybox` is the closest functional
replacement (still has a shell), `scratch` is the minimal distroless
option.
