Commit Graph
74 Commits
Author SHA1 Message Date
e1ae07c836 Isolate only virtual cluster workload Pods (#989)
* Enhance network policy to isolate synced workload pods and improve cross-cluster pod isolation handling

* Add test for label update on synced Pod to ensure isolation label persistence

* Derive host pod CIDRs dynamically for the isolation NetworkPolicy

Compute the egress-exclude CIDRs from the --cluster-cidr flag or the live
Node PodCIDR(s) via FindPodCIDRs, instead of a hardcoded guess, so cross-cluster
pod isolation is enforced against the host's real pod network. Adds unit,
integration, and e2e coverage.

* update comment

* Sort CIDR list in FindPodCIDRs function to ensure consistent order for egress rules

* Use `t.Context()` instead of `context.Background()`

Co-authored-by: Kevin McDermott <bigkevmcd@gmail.com>

* Use Ginkgo provided context

* Refactor FindPodCIDRs to use sets for CIDR collection and simplify logic

* fix lint

---------

Co-authored-by: Kevin McDermott <bigkevmcd@gmail.com>
2026-07-16 15:18:04 +02:00
Enrico CandinoandGitHub 3a4d93f5fb Fix dangling Pods (#1021)
* Add tests asserting GetPods is scoped to the node's own Pods

Adds a unit test (fake host + virtual clients, Pods across multiple nodes plus
a dangling one) and a multi-node e2e test (a Deployment with one nginx Pod per
node via required anti-affinity; restart every k3k-kubelet agent Pod; assert no
workload Pod is deleted from the host or virtual cluster). Both pin the intended
behavior and fail against the current code; the fix follows in the next commit.

* Scope Provider.GetPods to the node's own Pods to prevent cross-node dangling-pod deletion

GetPods() listed host Pods cluster-wide (by the k3k.io/clusterName label only).
The vendored virtual-kubelet library's deleteDanglingPods reconciliation deletes
any Pod returned here that is missing from this instance's virtual Pod lister,
which is scoped to spec.nodeName == agentHostname. On a multi-node host cluster,
every restarting k3k-kubelet instance therefore saw Pods owned by other nodes as
'dangling' and deleted them from the host (and, in turn, the virtual cluster).

Scope GetPods by the *virtual* Pod's spec.nodeName -- the same ownership signal
the framework uses -- excluding Pods owned by other nodes while still returning
own-node Pods and genuinely dangling ones. The host Pod's physical node is not a
reliable owner (it is scheduled with only a soft, sometimes-absent affinity), so
it must not be used. Virtual Pods are read live to avoid a startup cache-sync
race. Makes the previous commit's tests pass.

* Refactor GetPods to scope to the cluster namespace and update tests accordingly

* Add AgentNameLabel to track Pods synced by the k3k-kubelet agent and update tests accordingly

* Add failing test for updatePod
2026-07-15 17:02:30 +02:00
Enrico CandinoandGitHub 07cfe77318 Move e2e tests to a multi-node cluster, add kubelet restart test (#1010)
* Add E2E tests for multi-node setup and kubelet restart scenarios

* Update Kubernetes version options in conformance tests workflow

* Enhance VM provisioning action to support dynamic bridge CIDR prefix and pin package versions
2026-07-14 11:21:17 +02:00
Enrico CandinoandGitHub 3d0438edc9 Update k8s dependencies to v1.36 (main) (#979)
* Update k3s version in test suites to v1.36.2-k3s1

- Updated the k3sVersion and k3sOldVersion constants in both cli and e2e test suites to reflect the new version v1.36.2-k3s1 and the previous version v1.36.0-k3s1.

* updated README.md
2026-07-08 15:58:06 +02:00
39cc69f3e3 Added HCP (Hosted Control Plane) mode (#876)
* Add HCP (Hosted Control Plane) support

Introduce hosted control plane mode for k3k virtual clusters, including
API types, controller logic, server endpoint handling, CLI flags,
CRD updates, kubeconfig generation, and examples.

Co-Authored-By: RuFlo <ruv@ruv.net>

* removed hcpRegitration command

added HCP conformance tests

warning for hcp

fix multi-VM HCP conformance test networking

  Both QEMU workers booted with `-net user` and ended up registering the
  same InternalIP (10.0.2.15) because each VM gets its own isolated NAT
  slirp. Flannel propagated this to `public-ip` on both nodes, so VXLAN
  could not tunnel between workers and any cross-node pod traffic broke
  (89 failed / 335 passed of 424 conformance specs).

  Replace user-mode networking with a Linux bridge (k3kbr0,
  192.168.100.0/24) and one TAP device per VM, so the two workers share
  an L2 segment with unique routable IPs. NAT outbound from the bridge
  keeps internet access working for image pulls.

  Also set unique hostnames via cloud-init (worker-1/worker-2) and drop
  the `--node-name` flag from INSTALL_K3S_EXEC, since k3s now picks the
  correct node name from the OS hostname on its own.

  Bump hydrophone back to `--parallel 4` to match the single-VM job
  (parallelism was reduced earlier when the failure was thought to be
  resource-related).

added HCP print command

updated crds

adding e2e tests

Refactor selectNonLoopbackSAN function to accept SANs directly and update related logic in ensureHCPRegistration

* Update agent flag validation and enhance ingress host check with a warning log

Refactor descriptions for cluster provisioning mode and role in CRDs and documentation

Refactor logging in ServerURL function to use controller-runtime logger

Rename selectNonLoopbackSAN to findNonLoopbackSAN for clarity and update references

Refactor ServerURL function and related code to remove unused parameters and improve clarity

Remove unused imports from kubeconfig.go to improve code clarity

* suggested changes

* fix comment

* fix test

---------

Co-authored-by: jpgouin <jeanphilippe.gouin@suse.com>
Co-authored-by: RuFlo <ruv@ruv.net>
2026-07-07 14:12:19 +02:00
Kevin McDermott f1dbf33435 Support annotations on exposed Services
This adds a new .spec.expose.annotations field which is applied to the
service when it's created.
2026-07-06 08:19:30 +01:00
7f2cf6fe98 Add --tls-sans flag to k3kcli (#925)
* Add --tls-sans flag to k3kcli cluster create

* Refine --tls-sans flag description in k3kcli documentation

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Fix TLS SANs handling in cluster creation by using hostname directly

* Implement extractHost function and update TLS SAN handling in cluster creation

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-07-02 13:23:13 +02:00
Enrico CandinoandGitHub 3db4cc0652 Refactor test suite to remove K3s container dependency (#971)
- Removed K3s container setup and teardown logic from CLI and E2E test suites.
- Simplified Kubernetes client initialization by eliminating the need for K3s container.
- Updated InitFromKubeconfig function to no longer require K3s container as a parameter.
- Deleted unused HelmInstaller and RESTClientGetter implementations.
- Cleaned up logging functions related to K3s and Helm operations.
2026-07-02 10:10:48 +02:00
Enrico CandinoandGitHub ed3e901229 Add K3K prefix to k3kcli env vars, align cli test, add failCount (#962)
* align test-cli

* test context

* increase timeout

* Enhance cluster readiness check with failure count limit
Add prefix K3K to env vars
2026-07-01 11:14:37 +02:00
Enrico CandinoandGitHub 244011e68d Refactor kubeconfig URL generation (#938)
* Refactor kubeconfig generation to remove unused port parameter and update related functions

* Refactor kubeconfig generation to streamline error handling and remove unused imports

* Refactor kubeconfig URL generation functions and deprecate old implementation

* restore old behavior

* Add 'k3kcli kubeconfig get' command and update documentation

* Refactor URL generation by removing deprecated getURLFromService function and updating tests to use new implementation

* Fix expected URL for LoadBalancer test case to include hostname

* Refactor kubeconfig test documentation to clarify URL generation behavior for ClusterIP, NodePort, and LoadBalancer service types

* Refactor kubeconfig URL generation functions to improve clarity and maintainability

* Remove deprecated 'k3kcli kubeconfig get' command and update related documentation

* Set logger to discard in NewRootCmd for improved logging control

* Refactor getURLFromService to streamline ingress key retrieval
2026-06-26 11:16:54 +02:00
Enrico CandinoandGitHub 651da42ef0 Added provider.cattle.io=k3k label to Clusters (#901)
* added provider k3k label

* small change to check  build
2026-06-12 16:12:38 +02:00
Enrico CandinoandGitHub 737dcf998e Added Policy finalizer (#781)
* added policy finalizer

* check with a small timeout

* comments

* removed requeue, renamed log
2026-04-15 11:14:22 +02:00
Enrico CandinoandGitHub cb985ec5ea fix pvc race condition (#786) 2026-04-14 17:37:27 +02:00
Enrico CandinoandGitHub 06c5aee7c9 add importas linter (#779) 2026-04-13 17:25:17 +02:00
Enrico CandinoandGitHub ba1648e560 Refactoring of tests moving common logic in tests/framework (#769)
* first commit

* refactor: update golangci configuration and reorder imports in namespace.go

* refactor: update golangci configuration for linters and formatters

* check restart for logs fetch

* rverted a bit the structure

* requested changes

* WriteLogs rename
2026-04-13 12:50:38 +02:00
Gaurav MehtaandGitHub 8add97934e minor fix to translated ingress tls secrets in ingress object (#672)
* minor fix to translated ingress tls secrets in ingress object

updated cluster ingress sync options to include extra field for SyncTLSSecrets

Signed-off-by: Gaurav Mehta <gaurav.mehta@suse.com>

* rebase changes from main, include feedback for renaming ingress secret sync field to DisableTLSSecretTranslation, and add kubelet integration tests to verify secret name translation

Signed-off-by: Gaurav Mehta <gaurav.mehta@suse.com>

---------

Signed-off-by: Gaurav Mehta <gaurav.mehta@suse.com>
2026-04-08 11:39:23 +02:00
Enrico CandinoandGitHub ba819618e3 Cleanup tests and small fixes (#741)
* Added missing k3k pod logs export

* fix env variables

* bump codecov action to v6 (node 24 support)

* remove azure/setup-helm dependency, fixed k3s image

* fix shasum

* fix helm check

* added kubernetes version matrix

* requested changes, and small fixes
2026-04-04 02:50:03 +02:00
Enrico CandinoandGitHub ef2bb0339a Schedule Pods in the same Node with a preferred affinity (#724)
* Add integration tests for Deployment and StatefulSet creation with PVC in shared cluster

* Add affinity settings for pod scheduling based on agent hostname

* increased timeout

* focus test

* rmeove cleanup

* check for existing pvc

* remove focus

* add affinity tests for Pods in shared cluster

* refactor restartServerPod to improve pod restart checks and timeout handling

* unfocus

* fix test description
2026-03-31 11:22:10 +02:00
Enrico CandinoandGitHub 93e8ab6d8f update K3S_HOST_VERSION and use variable in cli tests (#734) 2026-03-31 11:12:25 +02:00
Enrico CandinoandGitHub 18889ba6b7 Removed Pod mutating Webhook (shared mode) (#699)
* removed all webhook references

* fix tests

* removal of webhook

* updated doc

* add downward API test

* small refactor of virtual-kubelet

* split tests
2026-03-31 10:29:22 +02:00
Jonathan CrowtherandGitHub 624fee98d7 Refactor tests to their own directories (#723)
* Move cli tests

* Move e2e tests to their own directory

* Move integration tests

* Fix path within the cli tests

* Move k3k-kubelet tests

* Improve the various make test- options

* Remove dead code from cli tests

* Update development.md with the new make commands
2026-03-26 09:45:31 -04:00
Enrico CandinoandGitHub 9554628fc5 Update virtual-kubelet (v1.12) and Kubernetes deps (v1.35) (#716)
* bump virtual-kubelet and k8s

* bump controller-manager

* fix upgrade-downgrade

* fix kubernetes version

* Update tests_suite_test.go

* removed direct dep of yaml.v2, bump etcd modules
2026-03-23 12:51:43 +01:00
Enrico CandinoandGitHub 7b0f695248 Bump some tes dependencies and fix lint (#708) 2026-03-18 17:43:41 +01:00
Kevin McDermottandGitHub 9d38388c80 Merge pull request #662 from bigkevmcd/change-role-to-worker
Change the nodes type to worker.
2026-03-18 08:02:31 +00:00
Enrico CandinoandGitHub 7641a1c9c5 Add sync of Host StorageClasses (#681)
* initial impl

* wip test

* fix

* wip tests

* Refactor storage class sync logic and enhance test coverage

* fix test

* remove storageclass sync test

* removed commented code

* added sync to cluster status to apply policy configuration

* fix for storageClass policy indexes

* fix for missing indexed field, and label sync

* - update sync options descriptions for resource types
- added storage class tests sync with policy
- requested changes

* fix for nil map
2026-03-17 16:53:29 +01:00
Kevin McDermottandEnrico Candino d975171920 Change the nodes type to worker.
This modifies the configuration of the created nodes via virtual-kubelet
to set the node type to be worker instead of agent.

Bump the Ginkgo version to the latest - allows use of `t.Context()`
rather than creating contexts.

Co-authored-by: Enrico Candino <enrico.candino@gmail.com>
2026-03-17 09:28:30 +00:00
Enrico CandinoandGitHub 83b4415f02 refactor: streamline K3S Docker installation and chart setup with dynamic repository handling (#692) 2026-03-12 11:11:12 +01:00
Enrico CandinoandGitHub 9836f8376d Added policy in Cluster Status (#663)
* initial implementation

restored policyName

* added test, fixed priority scheduling

* requested changes from review

- wrapped errors
- fixed some kube-api-linter issues to match k8s conventions
- moved policy namespace check in the same condition branch
2026-02-17 16:15:13 +01:00
Enrico CandinoandGitHub 004e177ac1 Bump kubernetes dependencies (v1.33) (#647)
* bump kubernetes to v0.33.7

* updated kuberneets api versions

* bump tests

* fix k3s version

* fix test

* centralize k8s version

* remove focus

* revert GetPodCondition, GetContainerStatus and pin of k8s.io/controller-manager
2026-01-27 22:28:56 +01:00
Enrico CandinoandGitHub 0164c785ab Show correct allocatable resources when a Policy is applied (#638)
* wip

* wip

* wip

* fix lint and tests

* fixed bugs for missing resources

* cleanup and refactor

* removed coreClient from configureNode

* added comments to distribute algorithm
2026-01-27 15:56:37 +01:00
Hussein GalalandGitHub c1b7da4c72 SecretMounts feature and private registries (#570)
* Add SecretMounts field

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>
2026-01-26 21:47:40 +02:00
Hussein GalalandGitHub fcc7191ab3 CLI cluster update (#595)
* CLI cluster update

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>
2026-01-20 14:00:24 +02:00
Enrico CandinoandGitHub 5f42eafd2a Dev doc update (#611)
* update development.md

* fix tests

* fix cli test
2026-01-16 14:56:43 +01:00
Hussein GalalandGitHub a871917aec Refactor startup command to wait for node IP changes (#598)
* Patch node ip when server pod restarts

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* wsl

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* Refactor startup command and adding safe mode

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* Add date/time logging to the startup script

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* fixes

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

---------

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>
2026-01-09 16:29:47 +02:00
Enrico CandinoandGitHub fc6bcedc5f fix for missing label update in creation, added tests (#592) 2025-12-16 11:34:50 +01:00
Enrico CandinoandGitHub c9bb1bcf46 Fixed CreatePod and UpdatePod in Virtual Kubelet for Downward API (#573)
* wip

* fix lint

* ephemerals container

* remove unused

* add retry

* volumes refactor

* added configmap and secret keyRef translation

* set debug logger to virtual kubelet logger

* added tests

* fix lint, removed unused func

* added test file locally
2025-12-10 13:47:34 +01:00
Enrico CandinoandGitHub e385ceb66f Fixed missing Kubernetes host version when specified (#585)
* fix for missing host version

* added test

* fix test

* fix test
2025-12-03 09:21:27 +01:00
Enrico CandinoandGitHub 521ff17ef6 Added test for SubPathExpr (#569)
* small fixes

* added test for subpathexpr

* removed old comment
2025-11-21 16:28:59 +01:00
Enrico CandinoandGitHub 8760afd5bc Added --namespace flag to k3kcli policy create (#564)
* added --namespace flag to policy create to actually bind the new policy to existing namespaces

* fix lint

* fix tests

* added overwrite flag

* updated cli docs

* fix tests 2

* moved double quotes to single quote

* fix test
2025-11-14 21:45:28 +01:00
Enrico CandinoandGitHub 2e6de51dab Improve tests resiliency (#539)
* fix missing namespaces cleanup

* fix conflict namespace

* fix PVC already created error, patch for existing volume, and check with hardcoded k3k name

* removed useless test

* fix for dump covdata from external pod

* keep namespaces flag

* fix for multi-node clusters

* fix for hanging pod in isolated namespace
2025-10-31 21:51:37 +01:00
Enrico CandinoandGitHub af9e1d6ca7 Cleanup orphaned resources after Cluster deletion (#540)
* adding controller reference for garbage collection, delete API lease

* added test

* fix lint
2025-10-31 15:25:38 +01:00
Enrico CandinoandGitHub 075d72df5d Cleanup of customCAs spec (#527)
* cleanup spec from customCAs when omitted

* add enabled default for customCAs
2025-10-23 22:11:44 +02:00
Enrico CandinoandGitHub 514fdf6b86 Fix for flaky test (#523)
* fix for flaky test

* fix lint

* check ContainersReady condition
2025-10-21 18:19:36 +02:00
Hussein GalalandGitHub 89dc352bea Scale up/down tests for virtual and shared mode (#508)
* Scale up/down tests for virtual and shared mode

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* defer cleanup and more fixes

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* add labels to e2e tests and divide the workload

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* add labels to e2e tests and divide the workload

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* add validate job to e2e test

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* fix label filters for e2e tests

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* fix makefile

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* use constants for e2e tests labels

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* fix typo

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* fix labels

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

---------

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>
2025-10-15 17:01:14 +03:00
Enrico CandinoandGitHub 7644406eeb Fix for flaky test (#509)
* fix for flaky test and fix for PVC creation

* fix lint
2025-10-15 11:31:45 +02:00
Enrico CandinoandGitHub 594c2571c3 promoted v1alpha1 resources to v1beta1 (#505) 2025-10-13 17:24:56 +02:00
Hussein GalalandGitHub 12971f55a6 Add k8s version upgrade test (#503)
* Add k8s version upgrade test

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* lint

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* remove unused functions

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

---------

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>
2025-10-13 17:14:25 +03:00
Hussein GalalandGitHub a0fd472841 Use K3S host cluster for E2E tests (#492)
* Add kubeconfig to e2e_tests

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* add E2E_KUBECONFIG env variable

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* fix yaml permissions for kubeconfig

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* Fix image name and use ttl.sh

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* add uuidgen result to a file

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* add hostIP

* Add k3s version to e2e test

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* remove comment

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* remove virtual mode tests

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* fix failed test

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* Fixes

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* fixes

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* fixes

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* Fixes

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* Add KUBECONFIG env variable to the make install

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* add k3kcli to github_path

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* Use docker installation for testing the cli

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* Fixes

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* fixes

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* typo

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* fix test cli

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* lint

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* fixes

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* typo

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

---------

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>
2025-10-08 15:39:35 +03:00
Enrico CandinoandGitHub 9f265c73d9 Fix for HA server deletion (#493)
* wip

* wip

* wip

* removed todo
2025-10-08 13:23:15 +02:00
Enrico CandinoandGitHub 5c95ca3dfa Fix for pod eviction in host cluster (#484)
* update statefulset controller

* fix for single pod

* adding pod controller

* added test

* removed comment

* merged service controller

* revert statefulset

* added test

* added common owner filter
2025-10-03 16:22:54 +02:00