Added HCP (Hosted Control Plane) mode (#876)

* Add HCP (Hosted Control Plane) support

Introduce hosted control plane mode for k3k virtual clusters, including
API types, controller logic, server endpoint handling, CLI flags,
CRD updates, kubeconfig generation, and examples.

Co-Authored-By: RuFlo <ruv@ruv.net>

* removed hcpRegitration command

added HCP conformance tests

warning for hcp

fix multi-VM HCP conformance test networking

  Both QEMU workers booted with `-net user` and ended up registering the
  same InternalIP (10.0.2.15) because each VM gets its own isolated NAT
  slirp. Flannel propagated this to `public-ip` on both nodes, so VXLAN
  could not tunnel between workers and any cross-node pod traffic broke
  (89 failed / 335 passed of 424 conformance specs).

  Replace user-mode networking with a Linux bridge (k3kbr0,
  192.168.100.0/24) and one TAP device per VM, so the two workers share
  an L2 segment with unique routable IPs. NAT outbound from the bridge
  keeps internet access working for image pulls.

  Also set unique hostnames via cloud-init (worker-1/worker-2) and drop
  the `--node-name` flag from INSTALL_K3S_EXEC, since k3s now picks the
  correct node name from the OS hostname on its own.

  Bump hydrophone back to `--parallel 4` to match the single-VM job
  (parallelism was reduced earlier when the failure was thought to be
  resource-related).

added HCP print command

updated crds

adding e2e tests

Refactor selectNonLoopbackSAN function to accept SANs directly and update related logic in ensureHCPRegistration

* Update agent flag validation and enhance ingress host check with a warning log

Refactor descriptions for cluster provisioning mode and role in CRDs and documentation

Refactor logging in ServerURL function to use controller-runtime logger

Rename selectNonLoopbackSAN to findNonLoopbackSAN for clarity and update references

Refactor ServerURL function and related code to remove unused parameters and improve clarity

Remove unused imports from kubeconfig.go to improve code clarity

* suggested changes

* fix comment

* fix test

---------

Co-authored-by: jpgouin <jeanphilippe.gouin@suse.com>
Co-authored-by: RuFlo <ruv@ruv.net>
This commit is contained in:
Enrico Candino
2026-07-07 14:12:19 +02:00
committed by GitHub
co-authored by RuFlo jpgouin
parent 33216d49ae
commit 39cc69f3e3
28 changed files with 1355 additions and 212 deletions
+15 -11
View File
@@ -45,11 +45,10 @@ type ClusterSpec struct {
// +optional
Version string `json:"version,omitempty"`
// Mode specifies the cluster provisioning mode: "shared" or "virtual".
// Defaults to "shared". This field is immutable.
// Mode specifies the cluster provisioning mode. This field is immutable.
//
// +kubebuilder:default="shared"
// +kubebuilder:validation:Enum=shared;virtual
// +kubebuilder:default=shared
// +kubebuilder:validation:Enum=shared;virtual;hcp
// +kubebuilder:validation:XValidation:message="mode is immutable",rule="self == oldSelf"
// +optional
Mode ClusterMode `json:"mode,omitempty"`
@@ -262,10 +261,9 @@ type SecretMount struct {
// +optional
SubPath string `json:"subPath,omitempty"`
// Role is the type of the k3k pod that will be used to mount the secret.
// This can be 'server', 'agent', or 'all' (for both).
//
// +optional
// +kubebuilder:validation:Enum=server;agent;all
// +optional
Role string `json:"role,omitempty"`
}
@@ -422,8 +420,7 @@ type StorageClassSyncConfig struct {
// ClusterMode is the possible provisioning mode of a Cluster.
//
// +kubebuilder:validation:Enum=shared;virtual
// +kubebuilder:default="shared"
// Supported values: `shared`, `virtual`, `hcp`.
type ClusterMode string
const (
@@ -432,6 +429,11 @@ const (
// VirtualClusterMode represents a cluster that runs in a virtual environment.
VirtualClusterMode = ClusterMode("virtual")
// HCPClusterMode represents a Hosted Control Plane: an agentless K3s control
// plane managed by k3k inside the host cluster. End users join their own
// external nodes (BYO) using the standard K3s installer command.
HCPClusterMode = ClusterMode("hcp")
)
// PersistenceMode is the storage mode of a Cluster.
@@ -640,7 +642,7 @@ type ClusterStatus struct {
// Phase is a high-level summary of the cluster's current lifecycle state.
//
// +kubebuilder:default="Unknown"
// +kubebuilder:default=Unknown
// +kubebuilder:validation:Enum=Pending;Provisioning;Ready;Failed;Terminating;Unknown
// +optional
Phase ClusterPhase `json:"phase,omitempty"`
@@ -785,9 +787,10 @@ type VirtualClusterPolicySpec struct {
// +optional
DefaultAgentAffinity *corev1.Affinity `json:"defaultAgentAffinity,omitempty"`
// AllowedMode specifies the allowed cluster provisioning mode. Defaults to "shared".
// AllowedMode specifies the allowed cluster provisioning mode.
//
// +kubebuilder:default=shared
// +kubebuilder:validation:Enum=shared;virtual;hcp
// +kubebuilder:validation:XValidation:message="mode is immutable",rule="self == oldSelf"
// +optional
AllowedMode ClusterMode `json:"allowedMode,omitempty"`
@@ -799,6 +802,7 @@ type VirtualClusterPolicySpec struct {
// PodSecurityAdmissionLevel specifies the pod security admission level applied to the pods in the namespace.
//
// +kubebuilder:validation:Enum=privileged;baseline;restricted
// +optional
PodSecurityAdmissionLevel *PodSecurityAdmissionLevel `json:"podSecurityAdmissionLevel,omitempty"`
@@ -831,7 +835,7 @@ type VirtualClusterPolicySpec struct {
// PodSecurityAdmissionLevel is the policy level applied to the pods in the namespace.
//
// +kubebuilder:validation:Enum=privileged;baseline;restricted
// Supported values: `privileged`, `baseline`, `restricted`.
type PodSecurityAdmissionLevel string
const (