bcraft and GitHub
120d964281
set machine id on server pod if not exists ( #1011 )
2026-07-10 12:58:37 +02:00
39cc69f3e3
Added HCP (Hosted Control Plane) mode ( #876 )
...
* Add HCP (Hosted Control Plane) support
Introduce hosted control plane mode for k3k virtual clusters, including
API types, controller logic, server endpoint handling, CLI flags,
CRD updates, kubeconfig generation, and examples.
Co-Authored-By: RuFlo <ruv@ruv.net >
* removed hcpRegitration command
added HCP conformance tests
warning for hcp
fix multi-VM HCP conformance test networking
Both QEMU workers booted with `-net user` and ended up registering the
same InternalIP (10.0.2.15) because each VM gets its own isolated NAT
slirp. Flannel propagated this to `public-ip` on both nodes, so VXLAN
could not tunnel between workers and any cross-node pod traffic broke
(89 failed / 335 passed of 424 conformance specs).
Replace user-mode networking with a Linux bridge (k3kbr0,
192.168.100.0/24) and one TAP device per VM, so the two workers share
an L2 segment with unique routable IPs. NAT outbound from the bridge
keeps internet access working for image pulls.
Also set unique hostnames via cloud-init (worker-1/worker-2) and drop
the `--node-name` flag from INSTALL_K3S_EXEC, since k3s now picks the
correct node name from the OS hostname on its own.
Bump hydrophone back to `--parallel 4` to match the single-VM job
(parallelism was reduced earlier when the failure was thought to be
resource-related).
added HCP print command
updated crds
adding e2e tests
Refactor selectNonLoopbackSAN function to accept SANs directly and update related logic in ensureHCPRegistration
* Update agent flag validation and enhance ingress host check with a warning log
Refactor descriptions for cluster provisioning mode and role in CRDs and documentation
Refactor logging in ServerURL function to use controller-runtime logger
Rename selectNonLoopbackSAN to findNonLoopbackSAN for clarity and update references
Refactor ServerURL function and related code to remove unused parameters and improve clarity
Remove unused imports from kubeconfig.go to improve code clarity
* suggested changes
* fix comment
* fix test
---------
Co-authored-by: jpgouin <jeanphilippe.gouin@suse.com >
Co-authored-by: RuFlo <ruv@ruv.net >
2026-07-07 14:12:19 +02:00
Kevin McDermott
4cd5bd85d0
Refactor the service tests.
2026-07-06 08:42:20 +01:00
Kevin McDermott
f1dbf33435
Support annotations on exposed Services
...
This adds a new .spec.expose.annotations field which is applied to the
service when it's created.
2026-07-06 08:19:30 +01:00
fa16898689
fix EXTRA_ARGS parsing in the startup command ( #951 )
...
* fix EXTRA_ARGS parsing in the startup command and add multi server args test
* fix lint
* Refactor server config tests to improve readability and maintainability
---------
Co-authored-by: Enrico Candino <enrico.candino@suse.com >
2026-07-01 18:17:58 +02:00
Hussein Galal and GitHub
3abffae92d
External datastore support ( #879 )
...
* Adding new k3s function to get the config
* Adding a new way to get the bootstrap data directly from the server Pod
* Gate the bootstrap fetch if external datastore is in use
---------
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
2026-06-26 17:22:00 +03:00
Enrico Candino and GitHub
244011e68d
Refactor kubeconfig URL generation ( #938 )
...
* Refactor kubeconfig generation to remove unused port parameter and update related functions
* Refactor kubeconfig generation to streamline error handling and remove unused imports
* Refactor kubeconfig URL generation functions and deprecate old implementation
* restore old behavior
* Add 'k3kcli kubeconfig get' command and update documentation
* Refactor URL generation by removing deprecated getURLFromService function and updating tests to use new implementation
* Fix expected URL for LoadBalancer test case to include hostname
* Refactor kubeconfig test documentation to clarify URL generation behavior for ClusterIP, NodePort, and LoadBalancer service types
* Refactor kubeconfig URL generation functions to improve clarity and maintainability
* Remove deprecated 'k3kcli kubeconfig get' command and update related documentation
* Set logger to discard in NewRootCmd for improved logging control
* Refactor getURLFromService to streamline ingress key retrieval
2026-06-26 11:16:54 +02:00
ff02eb5473
Initial support for kata runtimes ( #814 )
...
* update shell script to exec and remove pipe, add conditional kmsg mount for kata
update logging config
Apply suggestions from code review
Co-authored-by: Enrico Candino <enrico.candino@gmail.com >
pr comments, remove isKata, fix agent unit tests, update config
fix validation
fix template
revert log changes
update init script
remove emptydir mounts from kata
remove emptydir mounts from kata
fix tests
lint
* add quickstart docs
* centralise kata spotspec modifications
* update doc
* update docs
2026-06-12 13:57:42 +03:00
Hussein Galal and GitHub
5bd779cdaa
Refactor bootstrap data management ( #869 )
...
* Refactor bootstrap data management
* Change errors from errors.New to fmt.Errorf
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
2026-06-03 19:40:31 +03:00
Hussein Galal and GitHub
8457d3a20b
Refactor servers and agents configuration management ( #827 )
...
* Refactor server config secret
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* lint fixes
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* Refactor Agents configuration for virtual and shared mode
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* wsl
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* fixes
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* fix typo
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* fix unit tests
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* Remove go assert depednecny
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
---------
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
2026-05-12 15:57:24 +03:00
Hussein Galal and GitHub
1ca9e7274c
Add server and worker resource request and limits ( #832 )
...
* Add server and worker resource request and limits
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* docs
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
---------
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
2026-04-30 16:16:11 +03:00
Hussein Galal and GitHub
d4c8f56639
Fix cgroup dirs for virtual mode clusters ( #792 )
...
* Fix cgroup dirs for virtual mode clusters
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
2026-04-28 16:06:30 +03:00
Hussein Galal and GitHub
28acb94d8a
Add hostUsers to cluster spec ( #787 )
...
* Add hostUsers to cluster spec
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* wsl
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* Fix the do comment for hostusers
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* Fix the do comment for hostusers
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
---------
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
2026-04-15 15:46:23 +02:00
Enrico Candino and GitHub
06c5aee7c9
add importas linter ( #779 )
2026-04-13 17:25:17 +02:00
Hussein Galal and GitHub
96d812a3cb
Custom runtime class and security context ( #742 )
...
* Add experimental runtimeClass and securityContext to spec
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
2026-04-09 10:22:46 +02:00
Jonathan Crowther and GitHub
25e910ccaf
Add initial affinity to podspecs ( #696 )
...
* Add initial affinity to podspecs
* Fix go generate
* Add field to the policy and prioritize it over the cluster spec
* Fix linter issue
* Add docs
* Address comments
* Fix the tests and improve the field descriptions
* Fix formatter issues
* Change logs to info level
* run validation
* undo pandoc changes
2026-03-23 16:16:30 -04:00
Hussein Galal and GitHub
c1b7da4c72
SecretMounts feature and private registries ( #570 )
...
* Add SecretMounts field
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
2026-01-26 21:47:40 +02:00
Kevin McDermott
9ea81c861b
Use resource.Quantity for storageRequestSize
...
Previously the resource.Quantity was stored as string which allowed
invalid values to be created.
This performs validation on the strings using the standard K8s resource
mechanism.
2026-01-21 14:50:28 +00:00
Hussein Galal and GitHub
a871917aec
Refactor startup command to wait for node IP changes ( #598 )
...
* Patch node ip when server pod restarts
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* wsl
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* Refactor startup command and adding safe mode
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* Add date/time logging to the startup script
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* fixes
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
---------
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
2026-01-09 16:29:47 +02:00
Enrico Candino and GitHub
af9e1d6ca7
Cleanup orphaned resources after Cluster deletion ( #540 )
...
* adding controller reference for garbage collection, delete API lease
* added test
* fix lint
2025-10-31 15:25:38 +01:00
Enrico Candino and GitHub
075d72df5d
Cleanup of customCAs spec ( #527 )
...
* cleanup spec from customCAs when omitted
* add enabled default for customCAs
2025-10-23 22:11:44 +02:00
Enrico Candino and GitHub
594c2571c3
promoted v1alpha1 resources to v1beta1 ( #505 )
2025-10-13 17:24:56 +02:00
Hussein Galal and GitHub
72eb819216
Add imagepullsecrets to controller, server, and agents ( #455 )
...
* Add imagepullsecrets to controller, server, and agents
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* Fix tests
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* Fix tests
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* fix test cli
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* fxing tests
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* Add agent section to helm chart values
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* Fix charts values
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* Fixing chart and refactoring cluster config
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* default lists to the values of imagepullsecrets
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* fixes
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* more fixes
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* fixes
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* wsl
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* fix version image function and add unit tests
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* simplify arguments and remove registry from the code
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
---------
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
2025-09-17 11:29:01 +03:00
Enrico Candino and GitHub
98d17cdb50
Added new golangci-lint formatters ( #425 )
...
* add gci formatter
* gofmt and gofumpt
* rewrite rule
* added make fmt
2025-07-22 10:42:41 +02:00
Enrico Candino and GitHub
2047a600ed
Migrate golangci-lint to v2 ( #424 )
...
* golangci-lint upgrade
* fix lint
2025-07-22 10:10:26 +02:00
Hussein Galal and GitHub
a98c49b59a
Adding custom certificate to the virtual clusters ( #409 )
...
* Adding custom certificate to the virtual clusters
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* wsl
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* wsl
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* fix tests
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* Fix tests
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* wsl
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* wsl
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* wsl
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* docs update
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* integrate cert-manager
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* Add individual cert tests
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* wsl
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* docs
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* docs
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* wsl
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* wsl
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* Fixes
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
---------
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
2025-07-21 19:23:11 +03:00
Enrico Candino and GitHub
2ab71df139
Add Conditions and current status to Cluster ( #408 )
...
* Added Cluster Conditions
* added e2e tests
* fix lint
* cli polling
* update tests
2025-07-14 15:53:37 +02:00
Enrico Candino and GitHub
753b31b52a
Adding configurable maxConcurrentReconcilers and small CRD cleanup ( #410 )
...
* removed Persistence from Status, fixed default for StorageSize and StorageDefault
* added configurable maxConcurrentReconciles
* fix concurrent issues
* add validate as prereq for tests
2025-07-10 14:46:33 +02:00
jpgouin and GitHub
2b1448ffb8
add air-gap support ( #359 )
...
* add airgap support
* add airgap howto guide
2025-05-27 10:13:07 +02:00
Hussein Galal and GitHub
37fe4493e7
Fix HA init server scaling ( #333 )
...
* Fix HA init server scaling
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* increase timeout in e2e test
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
---------
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
2025-05-12 11:51:35 +03:00
Enrico Candino and GitHub
96a4341dfb
Services updates (LoadBalancerConfig and NodePortConfig) ( #329 )
...
* updates to services
- added loadBalancerConfig
- removed service-port
- added logic to not expose services
* Refactor cluster tests to improve readability and maintainability
- Simplified service port expectations by directly accessing elements instead of using `ContainElement`.
- Enhanced clarity of test assertions for `k3s-server-port` and `k3s-etcd-port` attributes.
- Removed redundant code for checking service ports.
* fix ports for ingress expose, update kubeconfig generate
2025-04-22 11:52:18 +02:00
Hussein Galal and GitHub
510ab4bb8a
Add extra env for servers/agents ( #324 )
...
* Add extra env for servers/agents
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* wsl
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* cli docs
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* Fix container env
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
---------
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
2025-04-21 12:25:51 +02:00
Hussein Galal and GitHub
0843a9e313
Initial support for ResourceQuotas in clustersets ( #308 )
...
* Add ResourceQuota to clusterset
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* wsl fixes
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* Generate docs
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* fix tests
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* Add a defualt limitRange for ClusterSets
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* fix linting
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* fix docs
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* Add test for clusterset limitRange
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* Add server and worker limits
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* make charts
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* add default limits and fixes to resourcesquota
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* wsl lint
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* wsl lint
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* make docs
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* make build-crds
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* make build-crds
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* make spec as pointer
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* fix tests
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* fix tests
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* delete default limit
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* wsl lint
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* wsl
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* wsl
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* Update tests
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* Update tests
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* return on delete in limitrange
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* wsl
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
---------
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
2025-04-03 12:30:48 +02:00
Enrico Candino and GitHub
c4cc1e69cd
requeue if server not ready ( #318 )
2025-04-03 10:45:18 +02:00
Enrico Candino and GitHub
5e8bc0d3cd
Update CRDs documentation ( #279 )
...
* complete CRD documentation
* fix missing rebuild of CRDs
2025-03-03 11:47:53 +01:00
Enrico Candino and GitHub
430e18bf30
Added wsl linter, and fixed related issues ( #275 )
...
* added wsl linter
* fixed issues
2025-02-27 10:59:02 +01:00
Hussein Galal and GitHub
fa553d25d4
Default to dynamic persistence and fix HA restarts ( #250 )
...
* Default to dynamic persistence and fix HA restarts
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
---------
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
2025-02-14 14:26:10 +02:00
Enrico Candino and GitHub
51a8fd8a8d
Fix and enhancements to IngressExposeConfig (annotations) ( #248 )
...
* ingress fixes
* added annotations to IngressConfig
* sync annotations with CR
* removed hosts
* small doc for ingress
2025-02-14 12:38:42 +01:00
Enrico Candino and GitHub
fdb133ad4a
Added Ports to NodePortConfig and expose fixes ( #247 )
...
* fix NodePort service update
* updated crd docs
2025-02-11 14:47:01 +01:00
jpgouin and GitHub
3df5a5b780
Merge pull request #213 from jp-gouin/fix-ingress
...
fix ingress creation, use the ingress host in Kubeconfig when enabled
2025-02-04 09:31:49 +01:00
Enrico Candino and GitHub
2a7541cdca
Fix missing updates of server certificates ( #219 )
...
* merge
* wip test
* added test for restart
* tests reorg
* simplified tests
2025-02-04 09:17:56 +01:00
Enrico Candino and GitHub
bc3f906280
Fix status update, updated k3s default version, updated CRDs ( #218 )
...
* fix status update
* fix schema and default image
* removed retry in controller
* removed fmt
2025-01-30 12:56:42 +01:00
Enrico Candino and GitHub
72b5a98dff
Fix typos and adding spellcheck linter ( #215 )
...
* adding spellcheck linter
* fix typos
2025-01-28 17:47:45 +01:00
jpgouin and GitHub
c88890e502
Merge branch 'main' into fix-ingress
2025-01-28 15:11:09 +01:00
Enrico Candino and GitHub
44045c5592
Added test (virtual cluster creation, with pod) and small kubeconfig refactor ( #211 )
...
* added virtual cluster and pod test
* moved ClusterCreate
* match patch k8s host version
2025-01-24 22:26:01 +01:00
jpgouin
e6db5a34c8
fix ingress creation, use the ingress host in Kubeconfig when enabled
2025-01-24 18:48:31 +00:00
Hussein Galal and GitHub
931c7c5fcb
Fix secret tokens and DNS translation ( #200 )
...
* Include init containers in token translation
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* Fix kubernetes.defaul service DNS translation
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* Add skip test var to dapper
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* Add kubelet version and image pull policy to the shared agent
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* fixes
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
---------
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
2025-01-23 01:55:05 +02:00
Enrico Candino and GitHub
fd6ed8184f
removed antiaffinity ( #199 )
2025-01-22 18:34:30 +01:00
Hussein Galal and GitHub
ccaa09fa4a
Add PVC syncing support ( #179 )
...
* Add pvc syncing support
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
* fixes
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
---------
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com >
2025-01-14 20:57:04 +02:00
Enrico Candino and GitHub
f9ddec53b7
Added priorityClass to Clusters and ClusterSets ( #180 )
...
* added priorityClass to Clusters and ClusterSets
* fixed comment
2025-01-14 11:05:48 +01:00