mirror of
https://github.com/open-cluster-management-io/ocm.git
synced 2026-08-23 22:26:49 +00:00
* 🌱 Add TLS profile configuration support via flags and ConfigMap Add pkg/common/tls library to support TLS profile compliance for OCM components. This enables components to receive TLS configuration via command-line flags (--tls-min-version and --tls-cipher-suites) from operators, aligning with the upstream enhancement proposal for TLS profile configuration. Key features: - TLS version and cipher suite parsing from flags or ConfigMap - ConfigMap-based TLS configuration for operator use - ConfigMap watcher for operators to detect profile changes - OpenSSL cipher name mapping to Go crypto/tls constants - Safe defaults (TLS 1.2) when no configuration provided Updated pkg/common/options/webhook.go to use TLS library instead of hardcoded TLS 1.2, enabling webhook components to respect TLS flags injected by operators. This is the foundation for OCM TLS profile compliance, keeping upstream code OpenShift-agnostic while supporting dynamic TLS configuration. Related: open-cluster-management-io/enhancements#175 Signed-off-by: Jia Zhu <jiazhu@redhat.com> Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> Signed-off-by: zhujian <jiazhu@redhat.com> * 🌱 Add TLS ConfigMap watch and restart to cluster-manager operator Implement ConfigMap-based TLS profile compliance for cluster-manager operator with hash comparison to prevent infinite restart loops. Changes: - Add TLS ConfigMap informer to watch ocm-tls-profile ConfigMap - Load current TLS config at startup and compute hash - Add event handlers that compare ConfigMap hash with current hash - Only restart if ConfigMap content actually differs from current config - Add comprehensive logging for all scenarios Scenarios handled: ✅ ConfigMap exists at startup (hash matches) → no restart ✅ ConfigMap created after startup (hash differs) → restart to apply ✅ ConfigMap updated (new hash differs) → restart to apply ✅ ConfigMap deleted (was using it) → restart to use defaults Leader election behavior: - This code only runs on the leader pod (due to controllercmd framework) - Non-leader pods wait idle until they acquire leadership - New leaders load current ConfigMap state when they start, ensuring latest config - Only the active leader monitors ConfigMap changes and restarts 🤖 Generated with Claude Code Signed-off-by: zhujian <jiazhu@redhat.com> * 🌱 Inject TLS config flags into addon-webhook deployment Implement Case 2 pattern for addon-webhook TLS configuration: cluster-manager-operator loads TLS config from ConfigMap and injects it as flags into the addon-webhook deployment. Changes: - Add AddonWebhookTLSMinVersion and AddonWebhookTLSCipherSuites fields to HubConfig - Load TLS config once when creating ClusterManagerController - Pass TLS config strings as parameters to controller - Inject --tls-min-version and --tls-cipher-suites flags into addon-webhook deployment template This approach ensures addon-webhook receives TLS configuration via flags without needing to watch the ConfigMap itself. When the ConfigMap changes, cluster-manager-operator restarts, reloads the config, and updates the deployment with new flags. 🤖 Generated with Claude Code Signed-off-by: zhujian <jiazhu@redhat.com> * 🌱 Log TLS min version and cipher suites on startup Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: zhujian <jiazhu@redhat.com> * 🌱 Move TLS library to sdk-go and update vendor dependencies Relocates TLS config and cipher helpers from pkg/common/tls into the vendored open-cluster-management.io/sdk-go/pkg/tls package, adds a generic watcher utility, and updates all import references accordingly. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: zhujian <jiazhu@redhat.com> * 🌱 Inject TLS flags into all hub component deployments Extend TLS flag injection from addon-webhook-only to all seven hub deployments managed by cluster-manager-operator: Manifests (operator → deployment args): - Rename HubConfig.AddonWebhookTLS* → TLS* so the same fields drive all deployments rather than only the addon webhook - Add {{- if .TLSMinVersion }} blocks to all six remaining deployment manifests (registration/work/placement controllers and registration/work webhook servers) Controller binaries (registration, work, placement, addon-manager): - Add --tls-min-version and --tls-cipher-suites flags to the common Options struct so the binaries accept the injected flags without failing; the flags are stored for future use Note: library-go's NewCommandWithContext uses cmd.Run (not RunE), so there is no clean programmatic hook to inject TLS into the 8443 health server without bypassing library-go's own boilerplate (signal handling, log init, profiling). Upstream library-go also has no native TLS configuration API on ControllerCommandConfig or ControllerBuilder. The 8443 health server defaults to TLS 1.2 via SetRecommendedHTTPServingInfoDefaults; configuring it further requires an upstream library-go enhancement. Webhook binaries already fully support these flags via WebhookOptions; no binary changes are needed there. Signed-off-by: Jian Zhu <zhujian@redhat.com> Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: zhujian <jiazhu@redhat.com> * 🌱 Wire --tls-min-version to library-go 8443 health server via WithServingTLSConfig Now that library-go has WithServingTLSConfig (ServingMinTLSVersion / ServingCipherSuites fields + injection in StartController before WithServer is called), wire the --tls-min-version and --tls-cipher-suites flags from Options into it. ApplyTLSToCommand installs a PersistentPreRunE hook that calls CmdConfig.WithServingTLSConfig after cobra flag parsing completes. PersistentPreRunE runs before cmd.Run, so all library-go boilerplate (signal handling, logging, profiling) is preserved - unlike the previous approach of replacing RunE which silently bypassed it. Uses go mod replace → /Users/jiazhu/go/src/github.com/openshift/library-go for local development/testing; replace directive to be removed once the library-go PR is merged and vendored. Signed-off-by: Jian Zhu <zhujian@redhat.com> Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: zhujian <jiazhu@redhat.com> * 🌱 Switch to --config file for controller 8443 TLS configuration Replace the WithServingTLSConfig approach with library-go's native --config flag mechanism: ApplyTLSToCommand now installs a PersistentPreRunE hook that: 1. Writes a minimal GenericOperatorConfig YAML to a temp file under /tmp (which is mounted as an emptyDir in all hub controller deployments, so writing is safe even with readOnlyRootFilesystem) 2. Sets --config to point at the temp file before cmd.Run executes All library-go boilerplate in cmd.Run (signal handling, log init, profiling, basicFlags.Validate) is fully preserved because PersistentPreRunE runs before Run, not replacing it. Inside StartController, Config() reads the temp file; the TLS values survive SetRecommendedHTTPServingInfoDefaults because DefaultString only sets fields that are currently empty. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: zhujian <jiazhu@redhat.com> * 🌱 Add tests for TLS profile compliance Unit tests (pkg/common/options): - TestApplyTLSToCommand: table-driven test covering all flag combinations: no flags (no-op), min-version only, cipher-suites only, both set, and --config pre-set by user (injection skipped). Unit tests (clustermanager_controller): - TestSyncDeployWithTLSConfig: verifies that when tlsMinVersion / tlsCipherSuites are set on the controller, the --tls-min-version and --tls-cipher-suites flags appear in the args of every managed hub deployment (registration, registration-webhook, placement, work-webhook). Also verifies the flags are absent when TLS config is not set. Integration tests (test/integration/operator): - "should inject tls-min-version into all hub deployments when ocm-tls-profile ConfigMap exists": creates the ocm-tls-profile ConfigMap with minTLSVersion=VersionTLS13 in the operator namespace and verifies all six hub deployments gain --tls-min-version=VersionTLS13 in their container args. Signed-off-by: Jian Zhu <zhujian@redhat.com> Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: zhujian <jiazhu@redhat.com> * 🌱 Switch TLS cipher suite format from OpenSSL to IANA Update vendored sdk-go to use IANA cipher suite names (e.g. TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) instead of OpenSSL names (e.g. ECDHE-RSA-AES128-GCM-SHA256). IANA is the canonical format used by Go's crypto/tls, the Kubernetes apiserver --tls-cipher-suites flag, and library-go's ServingInfo.CipherSuites. Using IANA names end-to-end eliminates the format mismatch that caused library-go's 8443 health server to reject cipher suite names written by ApplyTLSToCommand. The ocm-tls-profile ConfigMap now accepts IANA names only. The downstream tls-profile-sync sidecar is responsible for converting OpenShift TLSSecurityProfile (OpenSSL-style) names to IANA before writing the ConfigMap. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: zhujian <jiazhu@redhat.com> * 🌱 Fix TLS ConfigMap test: create ConfigMap before operator startup The previous test created ocm-tls-profile ConfigMap after the operator started, which triggered the watcher's hash-change detection and called os.Exit(0), killing the test process. Move the test into a dedicated Describe with BeforeEach that creates the ConfigMap before starting the operator so the watcher seeds its hash at startup and no restart is triggered. Also add hubWorkControllerDeployment to the tlsDeployments list since its manifest includes tls-min-version injection. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: zhujian <jiazhu@redhat.com> --------- Signed-off-by: Jia Zhu <jiazhu@redhat.com> Signed-off-by: zhujian <jiazhu@redhat.com> Signed-off-by: Jian Zhu <zhujian@redhat.com> Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
360 lines
9.8 KiB
Go
360 lines
9.8 KiB
Go
package options
|
|
|
|
import (
|
|
"context"
|
|
"os"
|
|
"path"
|
|
"strings"
|
|
"testing"
|
|
"time"
|
|
|
|
"github.com/openshift/library-go/pkg/controller/controllercmd"
|
|
"github.com/spf13/cobra"
|
|
corev1 "k8s.io/api/core/v1"
|
|
"k8s.io/apimachinery/pkg/runtime"
|
|
kubefake "k8s.io/client-go/kubernetes/fake"
|
|
clocktesting "k8s.io/utils/clock/testing"
|
|
|
|
testinghelpers "open-cluster-management.io/ocm/pkg/registration/helpers/testing"
|
|
"open-cluster-management.io/ocm/pkg/registration/register"
|
|
"open-cluster-management.io/ocm/pkg/registration/spoke/registration"
|
|
"open-cluster-management.io/ocm/pkg/version"
|
|
)
|
|
|
|
func TestComplete(t *testing.T) {
|
|
// get component namespace
|
|
var componentNamespace string
|
|
nsBytes, err := os.ReadFile("/var/run/secrets/kubernetes.io/serviceaccount/namespace")
|
|
if err != nil {
|
|
componentNamespace = defaultSpokeComponentNamespace
|
|
} else {
|
|
componentNamespace = string(nsBytes)
|
|
}
|
|
|
|
cases := []struct {
|
|
name string
|
|
clusterName string
|
|
secret *corev1.Secret
|
|
expectedClusterName string
|
|
expectedAgentName string
|
|
}{
|
|
{
|
|
name: "generate random cluster/agent name",
|
|
},
|
|
{
|
|
name: "specify cluster name",
|
|
clusterName: "cluster1",
|
|
expectedClusterName: "cluster1",
|
|
},
|
|
{
|
|
name: "override cluster name in secret with specified value",
|
|
clusterName: "cluster1",
|
|
secret: testinghelpers.NewHubKubeconfigSecret(componentNamespace, "hub-kubeconfig-secret", "", nil, map[string][]byte{
|
|
"cluster-name": []byte("cluster2"),
|
|
"agent-name": []byte("agent2"),
|
|
}),
|
|
expectedClusterName: "cluster1",
|
|
expectedAgentName: "agent2",
|
|
},
|
|
{
|
|
name: "take cluster/agent name from secret",
|
|
secret: testinghelpers.NewHubKubeconfigSecret(
|
|
componentNamespace, "hub-kubeconfig-secret", "", nil, map[string][]byte{
|
|
"cluster-name": []byte("cluster1"),
|
|
"agent-name": []byte("agent1"),
|
|
}),
|
|
expectedClusterName: "cluster1",
|
|
expectedAgentName: "agent1",
|
|
},
|
|
}
|
|
|
|
for _, c := range cases {
|
|
t.Run(c.name, func(t *testing.T) {
|
|
// setup kube client
|
|
var objects []runtime.Object
|
|
if c.secret != nil {
|
|
objects = append(objects, c.secret)
|
|
}
|
|
kubeClient := kubefake.NewSimpleClientset(objects...)
|
|
|
|
// create a tmp dir to dump hub kubeconfig
|
|
dir, err := os.MkdirTemp("", "hub-kubeconfig")
|
|
if err != nil {
|
|
t.Error("unable to create a tmp dir")
|
|
}
|
|
defer os.RemoveAll(dir)
|
|
|
|
options := NewAgentOptions()
|
|
options.SpokeClusterName = c.clusterName
|
|
options.HubKubeconfigDir = dir
|
|
|
|
err = registration.DumpSecret(
|
|
context.TODO(), kubeClient.CoreV1(), componentNamespace, "hub-kubeconfig-secret",
|
|
options.HubKubeconfigDir)
|
|
if err != nil {
|
|
t.Error(err)
|
|
}
|
|
|
|
if err := options.Complete(); err != nil {
|
|
t.Errorf("unexpected error: %v", err)
|
|
}
|
|
if options.ComponentNamespace == "" {
|
|
t.Error("component namespace should not be empty")
|
|
}
|
|
if options.SpokeClusterName == "" {
|
|
t.Error("cluster name should not be empty")
|
|
}
|
|
if options.AgentID == "" {
|
|
t.Error("agent name should not be empty")
|
|
}
|
|
if len(c.expectedClusterName) > 0 && options.SpokeClusterName != c.expectedClusterName {
|
|
t.Errorf("expect cluster name %q but got %q", c.expectedClusterName, options.SpokeClusterName)
|
|
}
|
|
if len(c.expectedAgentName) > 0 && options.AgentID != c.expectedAgentName {
|
|
t.Errorf("expect agent name %q but got %q", c.expectedAgentName, options.AgentID)
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
func TestValidate(t *testing.T) {
|
|
cases := []struct {
|
|
name string
|
|
clusterName string
|
|
expectedErr bool
|
|
}{
|
|
{
|
|
name: "empty cluster name",
|
|
expectedErr: true,
|
|
},
|
|
{
|
|
name: "invalid cluster name format",
|
|
clusterName: "test.cluster",
|
|
expectedErr: true,
|
|
},
|
|
{
|
|
name: "valid passed",
|
|
clusterName: "cluster-1",
|
|
expectedErr: false,
|
|
},
|
|
}
|
|
|
|
for _, c := range cases {
|
|
t.Run(c.name, func(t *testing.T) {
|
|
options := NewAgentOptions()
|
|
options.SpokeClusterName = c.clusterName
|
|
err := options.Validate()
|
|
if err == nil && c.expectedErr {
|
|
t.Errorf("expect to get err")
|
|
}
|
|
if err != nil && !c.expectedErr {
|
|
t.Errorf("expect not error but got %v", err)
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
func TestGetOrGenerateClusterAgentNames(t *testing.T) {
|
|
tempDir, err := os.MkdirTemp("", "testgetorgenerateclusteragentnames")
|
|
if err != nil {
|
|
t.Errorf("unexpected error: %v", err)
|
|
}
|
|
defer os.RemoveAll(tempDir)
|
|
|
|
cases := []struct {
|
|
name string
|
|
options *AgentOptions
|
|
expectedClusterName string
|
|
expectedAgentName string
|
|
}{
|
|
{
|
|
name: "cluster name is specified",
|
|
options: &AgentOptions{SpokeClusterName: "cluster0"},
|
|
expectedClusterName: "cluster0",
|
|
},
|
|
{
|
|
name: "cluster name and agent name are in file",
|
|
options: &AgentOptions{HubKubeconfigDir: tempDir},
|
|
expectedClusterName: "cluster1",
|
|
expectedAgentName: "agent1",
|
|
},
|
|
}
|
|
for _, c := range cases {
|
|
t.Run(c.name, func(t *testing.T) {
|
|
if c.options.HubKubeconfigDir != "" {
|
|
testinghelpers.WriteFile(path.Join(tempDir, register.ClusterNameFile), []byte(c.expectedClusterName))
|
|
testinghelpers.WriteFile(path.Join(tempDir, register.AgentNameFile), []byte(c.expectedAgentName))
|
|
}
|
|
clusterName, agentName := c.options.getOrGenerateClusterAgentID()
|
|
if clusterName != c.expectedClusterName {
|
|
t.Errorf("expect cluster name %q but got %q", c.expectedClusterName, clusterName)
|
|
}
|
|
|
|
// agent name cannot be empty, it is either generated or from file
|
|
if agentName == "" {
|
|
t.Error("agent name should not be empty")
|
|
}
|
|
|
|
if c.expectedAgentName != "" && c.expectedAgentName != agentName {
|
|
t.Errorf("expect agent name %q but got %q", c.expectedAgentName, agentName)
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
func TestNewOptions(t *testing.T) {
|
|
opts := NewOptions()
|
|
cmd := &cobra.Command{
|
|
Use: "test",
|
|
Short: "test Controller",
|
|
Run: func(cmd *cobra.Command, args []string) {
|
|
_ = cmd.Help()
|
|
os.Exit(1)
|
|
},
|
|
}
|
|
|
|
opts.NewControllerCommandConfig("test", version.Get(), func(ctx context.Context, controllerCtx *controllercmd.ControllerContext) error {
|
|
return nil
|
|
}, clocktesting.NewFakeClock(time.Now()))
|
|
|
|
opts.AddFlags(cmd.Flags())
|
|
if err := cmd.Flags().Set("kube-api-qps", "10"); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if err := cmd.Flags().Set("kube-api-burst", "20"); err != nil {
|
|
t.Fatal(err)
|
|
|
|
}
|
|
if err := cmd.Flags().Set("disable-leader-election", "true"); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if err := cmd.Flags().Set("unsupported-flag", "true"); err == nil {
|
|
t.Errorf("Should return err")
|
|
}
|
|
}
|
|
|
|
func TestApplyTLSToCommand(t *testing.T) {
|
|
cases := []struct {
|
|
name string
|
|
tlsMinVersion string
|
|
tlsCipherSuites string
|
|
configPreSet bool
|
|
wantConfigSet bool
|
|
wantError bool
|
|
wantMinVersion string
|
|
wantCiphers []string
|
|
}{
|
|
{
|
|
name: "no TLS flags set — no-op",
|
|
wantConfigSet: false,
|
|
},
|
|
{
|
|
name: "only tls-min-version set",
|
|
tlsMinVersion: "VersionTLS13",
|
|
wantConfigSet: true,
|
|
wantMinVersion: "VersionTLS13",
|
|
},
|
|
{
|
|
name: "only tls-cipher-suites set",
|
|
tlsCipherSuites: "TLS_AES_256_GCM_SHA384",
|
|
wantConfigSet: true,
|
|
wantCiphers: []string{"TLS_AES_256_GCM_SHA384"},
|
|
},
|
|
{
|
|
name: "both tls-min-version and tls-cipher-suites set",
|
|
tlsMinVersion: "VersionTLS13",
|
|
tlsCipherSuites: "TLS_AES_256_GCM_SHA384,TLS_CHACHA20_POLY1305_SHA256",
|
|
wantConfigSet: true,
|
|
wantMinVersion: "VersionTLS13",
|
|
wantCiphers: []string{"TLS_AES_256_GCM_SHA384", "TLS_CHACHA20_POLY1305_SHA256"},
|
|
},
|
|
{
|
|
name: "user already provided --config — skip injection",
|
|
tlsMinVersion: "VersionTLS13",
|
|
configPreSet: true,
|
|
wantConfigSet: false,
|
|
},
|
|
{
|
|
name: "invalid tls-min-version — error before file creation",
|
|
tlsMinVersion: "BadVersion",
|
|
wantError: true,
|
|
},
|
|
{
|
|
name: "invalid tls-cipher-suites — error before file creation",
|
|
tlsCipherSuites: "NOT_A_REAL_CIPHER",
|
|
wantError: true,
|
|
},
|
|
}
|
|
|
|
for _, c := range cases {
|
|
t.Run(c.name, func(t *testing.T) {
|
|
opts := NewOptions()
|
|
opts.TLSMinVersion = c.tlsMinVersion
|
|
opts.TLSCipherSuites = c.tlsCipherSuites
|
|
|
|
// Simulate library-go's --config flag.
|
|
cmd := &cobra.Command{Use: "test"}
|
|
var configFile string
|
|
cmd.Flags().StringVar(&configFile, "config", "", "")
|
|
|
|
if c.configPreSet {
|
|
if err := cmd.Flags().Set("config", "/existing/config.yaml"); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
}
|
|
|
|
opts.ApplyTLSToCommand(cmd)
|
|
|
|
// PersistentPreRunE is set by ApplyTLSToCommand; invoke it directly.
|
|
err := cmd.PersistentPreRunE(cmd, nil)
|
|
if c.wantError {
|
|
if err == nil {
|
|
t.Fatal("expected an error but got none")
|
|
}
|
|
return
|
|
}
|
|
if err != nil {
|
|
t.Fatalf("unexpected error: %v", err)
|
|
}
|
|
|
|
if c.configPreSet {
|
|
// --config should remain as the user set it, not overwritten.
|
|
if configFile != "/existing/config.yaml" {
|
|
t.Errorf("expected --config to remain %q, got %q", "/existing/config.yaml", configFile)
|
|
}
|
|
return
|
|
}
|
|
|
|
if !c.wantConfigSet {
|
|
if configFile != "" {
|
|
t.Errorf("expected --config to be empty, got %q", configFile)
|
|
}
|
|
return
|
|
}
|
|
|
|
if configFile == "" {
|
|
t.Fatal("expected --config to be set, but it is empty")
|
|
}
|
|
defer os.Remove(configFile)
|
|
|
|
content, err := os.ReadFile(configFile)
|
|
if err != nil {
|
|
t.Fatalf("failed to read generated config file: %v", err)
|
|
}
|
|
s := string(content)
|
|
|
|
if c.wantMinVersion != "" && !strings.Contains(s, "minTLSVersion: "+c.wantMinVersion) {
|
|
t.Errorf("expected config to contain minTLSVersion %q, got:\n%s", c.wantMinVersion, s)
|
|
}
|
|
if c.wantMinVersion == "" && strings.Contains(s, "minTLSVersion") {
|
|
t.Errorf("expected config to NOT contain minTLSVersion, got:\n%s", s)
|
|
}
|
|
for _, cipher := range c.wantCiphers {
|
|
if !strings.Contains(s, cipher) {
|
|
t.Errorf("expected config to contain cipher %q, got:\n%s", cipher, s)
|
|
}
|
|
}
|
|
})
|
|
}
|
|
}
|