github/k3k
1
0
Fork 0
mirror of https://github.com/rancher/k3k.git synced 2026-08-14 01:46:17 +00:00
Code Issues Packages Projects Releases Wiki Activity

(docs) Remove ambiguity of virtual mode isolation (#1122)

Browse Source 
* Remove ambiguity of virtual mode isolation

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>
This commit is contained in:
Hussein Galal
2026-08-06 12:25:57 +03:00
committed by GitHub
parent b7dc666748
commit 98cd3ee92e
2 changed files with 21 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+2 -2
View File
@@ -8,7 +8,7 @@
[![Conformance Tests - Virtual Mode](https://github.com/rancher/k3k/actions/workflows/test-conformance-virtual.yaml/badge.svg)](https://github.com/rancher/k3k/actions/workflows/test-conformance-virtual.yaml) [![Conformance Tests - HCP Mode](https://github.com/rancher/k3k/actions/workflows/test-conformance-hcp.yaml/badge.svg)](https://github.com/rancher/k3k/actions/workflows/test-conformance-hcp.yaml)
K3k, Kubernetes in Kubernetes, is a tool that empowers you to create and manage isolated K3s clusters within your existing Kubernetes environment. It enables efficient multi-tenancy, streamlined experimentation, and robust resource isolation, minimizing infrastructure costs by allowing you to run multiple lightweight Kubernetes clusters on the same physical host. K3k offers both "shared" mode, optimizing resource utilization, and "virtual" mode, providing complete isolation with dedicated K3s server pods. This allows you to access a full Kubernetes experience without the overhead of managing separate physical resources.
K3k, Kubernetes in Kubernetes, is a tool that empowers you to create and manage isolated K3s clusters within your existing Kubernetes environment. It enables efficient multi-tenancy, streamlined experimentation, and robust resource isolation, minimizing infrastructure costs by allowing you to run multiple lightweight Kubernetes clusters on the same physical host. K3k offers both "shared" mode, optimizing resource utilization, and "virtual" mode, providing more isolation with dedicated K3s server and agent pods. This allows you to access a full Kubernetes experience without the overhead of managing separate physical resources.
K3k integrates seamlessly with Rancher for simplified management of your embedded clusters.
@@ -24,7 +24,7 @@ K3k integrates seamlessly with Rancher for simplified management of your embedde
- **Optimized Resource Utilization (Shared Mode):** Maximize your infrastructure investment by running multiple K3s clusters on the same physical host. K3k's shared mode allows you to efficiently share underlying resources, reducing overhead and minimizing costs.
- **Complete Isolation (Virtual Mode):** For enhanced security and isolation, K3k's virtual mode provides dedicated K3s server pods for each embedded cluster. This ensures complete separation of workloads and eliminates any potential resource contention or security risks.
- **Stronger Isolation (Virtual Mode):** For enhanced isolation, K3k's virtual mode provides dedicated K3s server and agent pods for each embedded cluster, so every cluster gets its own control plane, CNI, and nodes, without resource contention from other clusters. Note that the agent pods currently run privileged on the host node, so a pod that is privileged inside a virtual cluster is also privileged on the real host node. A sandboxed runtime such as [Kata Containers](./docs/howtos/using-kata-containers.md) can be configured with `spec.runtimeClassName` to provide stronger isolation. See [Isolation and Security](./docs/architecture.md#isolation-and-security-1) for details.
- **Rancher Integration:** Simplify the management of your K3k clusters with Rancher. Leverage Rancher's intuitive UI and powerful features to monitor, manage, and scale your embedded clusters with ease.
docs/architecture.md
+19 -3
View File
@@ -73,11 +73,27 @@ This approach provides a clear and direct way to control the resources available
### Isolation and Security
The `virtual` mode offers strong isolation due to the dedicated K3s clusters deployed for each virtual cluster. Because each virtual cluster runs its own separate control plane and worker nodes, workloads are effectively isolated from each other and from the host cluster. This architecture minimizes the risk of one virtual cluster impacting others or the host cluster.
The `virtual` mode offers more isolation than `shared` mode due to the dedicated K3s clusters deployed for each virtual cluster. Because each virtual cluster runs its own separate control plane and worker nodes, workloads are effectively isolated from each other and from the host cluster. This architecture minimizes the risk of one virtual cluster impacting others or the host cluster.
Security in `virtual` mode benefits from the inherent isolation provided by the separate K3s clusters. However, standard Kubernetes security best practices still apply, and K3k emphasizes a layered security approach. While the K3s server pods often run with elevated privileges (due to the nature of their function, requiring access to system resources), K3k recommends minimizing these privileges whenever possible and adhering to the principle of least privilege. This can be achieved by carefully configuring the necessary capabilities instead of relying on full `privileged` mode. Further information on K3s security best practices can be found in the official K3s documentation: [https://docs.k3s.io/security](https://docs.k3s.io/security) (This link provides general security guidance, including discussions of capabilities and other relevant topics).
Currently, `virtual` mode runs the server and agent pods with a `Privileged` securityContext to allow the kubelet and other components to function properly. As a result of this drawback, any pod that is privileged inside a virtual cluster also holds full privileges on the real host node.
Currently security in virtual mode has a risk of privilege escalation as the server pods run with elevated privileges (due to the nature of their function, requiring access to system resources).
Pod Security Admission configured *inside* a virtual cluster is administered by the owner of that virtual cluster, who can relax or remove it. It restricts the workloads of that cluster; it does not protect the host cluster from the owner of the virtual cluster.
To close this gap, however, the server and agent pods can be run under a sandboxed runtime by setting `runtimeClassName` on the `Cluster` (or on a `VirtualClusterPolicy`, to apply it to every cluster in the bound Namespaces):
```yaml
apiVersion: k3k.io/v1beta1
kind: Cluster
metadata:
name: my-cluster
spec:
mode: virtual
runtimeClassName: kata-qemu
```
Runtimes such as [Kata Containers](./howtos/using-kata-containers.md), which runs each pod in a lightweight VM, or [Sysbox](https://github.com/nestybox/sysbox) provide a kernel or hardware boundary between the virtual cluster and the host node. Support for these runtimes is experimental; see the [Kata Containers how-to](./howtos/using-kata-containers.md) for a full setup walkthrough.
Beyond that, standard Kubernetes security best practices still apply, and K3k emphasizes a layered security approach. K3k recommends minimizing the privileges of the virtual cluster pods whenever possible and adhering to the principle of least privilege; `securityContext` and `podSecurityContext` on the `Cluster` can be used to override the defaults, keeping in mind that the nested kubelet and container runtime require most of these privileges to function. Further information on K3s security best practices can be found in the official K3s documentation: [https://docs.k3s.io/security](https://docs.k3s.io/security) (This link provides general security guidance, including discussions of capabilities and other relevant topics).
## K3k Components