k3k/docs/howtos/create-virtual-clusters.md

How to: Create a Virtual Cluster

This guide walks through the various ways to create and manage virtual clusters in K3K. We'll cover common use cases using both the Custom Resource Definitions (CRDs) and the K3K CLI, so you can choose the method that fits your workflow.

📘 For full reference:

• CRD Reference Documentation
• CLI Reference Documentation
• Full example

Note

🚧 Some features are currently only available via the CRD interface. CLI support may be added in the future.

---

Use Case: Create and Expose a Basic Virtual Cluster

CRD Method

apiVersion: k3k.io/v1beta1
kind: Cluster
metadata:
  name: k3kcluster-ingress
spec:
  tlsSANs:
    - my-cluster.example.com
  expose:
    ingress:
      ingressClassName: nginx
      annotations:
        nginx.ingress.kubernetes.io/ssl-passthrough: "true"
        nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
        nginx.ingress.kubernetes.io/ssl-redirect: "HTTPS"

This will create a virtual cluster in shared mode and expose it via an ingress with the specified hostname.

CLI Method

No CLI method available yet

---

Use Case: Create a Virtual Cluster with Persistent Storage (Default)

CRD Method

apiVersion: k3k.io/v1beta1
kind: Cluster
metadata:
  name: k3kcluster-persistent
spec:
  persistence:
    type: dynamic
    storageClassName: local-path
    storageRequestSize: 30Gi

This ensures that the virtual cluster stores its state persistently with a 30Gi volume.
If storageClassName is not set it will default to the default StorageClass.
If storageRequestSize is not set it will request a 1Gi volume by default.

CLI Method

k3kcli cluster create \
  --persistence-type dynamic \
  --storage-class-name local-path \
  k3kcluster-persistent

Note

The k3kcli does not support configuring the storageRequestSize yet.

---

Use Case: Create a Highly Available Virtual Cluster in shared mode

CRD Method

apiVersion: k3k.io/v1beta1
kind: Cluster
metadata:
  name: k3kcluster-ha
spec:
  servers: 3

This will create a virtual cluster with 3 servers and a default 1Gi volume for persistence.

CLI Method

k3kcli cluster create \
  --servers 3 \
  k3kcluster-ha

---

Use Case: Create a Highly Available Virtual Cluster in virtual mode

CRD Method

apiVersion: k3k.io/v1beta1
kind: Cluster
metadata:
  name: k3kcluster-virtual
spec:
  mode: virtual
  servers: 3
  agents: 3

This will create a virtual cluster with 3 servers and 3 agents and a default 1Gi volume for persistence.

Note

Agents only exist for virtual mode.

CLI Method

k3kcli cluster create \
  --agents 3 \
  --servers 3 \
  --mode virtual \
  k3kcluster-virtual

---

Use Case: Create an Ephemeral Virtual Cluster

CRD Method

apiVersion: k3k.io/v1beta1
kind: Cluster
metadata:
  name: k3kcluster-ephemeral
spec:
  persistence:
    type: ephemeral

This will create an ephemeral virtual cluster with no persistence and a single server.

CLI Method

k3kcli cluster create \
  --persistence-type ephemeral \
  k3kcluster-ephemeral

---

Use Case: Create a Virtual Cluster with a Custom Kubernetes Version

CRD Method

apiVersion: k3k.io/v1beta1
kind: Cluster
metadata:
  name: k3kcluster-custom-k8s
spec:
  version: "v1.33.1-k3s1"

This sets the virtual cluster's Kubernetes version explicitly.

Note

Only K3s distributions are supported. You can find compatible versions on the K3s GitHub release page.

CLI Method

k3kcli cluster create \
  --version v1.33.1-k3s1 \
  k3kcluster-custom-k8s

---

Use Case: Create a Virtual Cluster with Custom Resource Limits

CRD Method

apiVersion: k3k.io/v1beta1
kind: Cluster
metadata:
  name: k3kcluster-resourced
spec:
  mode: virtual
  serverLimit:
    cpu: "1"
    memory: "2Gi"
  workerLimit:
    cpu: "1"
    memory: "2Gi"

This configures the CPU and memory limit for the virtual cluster.

CLI Method

No CLI method available yet

---

Use Case: Create a Virtual Cluster on specific host nodes

CRD Method

apiVersion: k3k.io/v1beta1
kind: Cluster
metadata:
  name: k3kcluster-node-placed
spec:
  nodeSelector:
    disktype: ssd

This places the virtual cluster on nodes with the label disktype: ssd.

Note

In shared mode workloads are also scheduled on the selected nodes

CLI Method

No CLI method available yet

---

Use Case: Create a Virtual Cluster with a Rancher Host Cluster Kubeconfig

When using a kubeconfig generated with Rancher, you need to specify with the CLI the desired host for the virtual cluster kubeconfig.
By default, k3kcli uses the current host kubeconfig to determine the target cluster.

CRD Method

Not applicable

CLI Method

k3kcli cluster create \
  --kubeconfig-server https://abc.xyz \
  k3kcluster-host-rancher

---

Use Case: Create a Virtual Cluster Behind an HTTP Proxy

CRD Method

apiVersion: k3k.io/v1beta1
kind: Cluster
metadata:
  name: k3kcluster-http-proxy
spec:
  serverEnvs:
    - name: HTTP_PROXY
      value: "http://abc.xyz"
  agentEnvs:
    - name: HTTP_PROXY
      value: "http://abc.xyz"

This configures an HTTP proxy for both servers and agents in the virtual cluster.

Note

This can be leveraged to pass any custom environment variables to the servers and agents — not just proxy settings.

CLI Method

k3kcli cluster create  \
  --server-envs HTTP_PROXY=http://abc.xyz \
  --agent-envs HTTP_PROXY=http://abc.xyz \
  k3kcluster-http-proxy

---

How to: Connect to a Virtual Cluster

Once the virtual cluster is running, you can connect to it using the CLI:

CLI Method

k3kcli kubeconfig generate --namespace k3k-mycluster --name mycluster 
export KUBECONFIG=$PWD/mycluster-kubeconfig.yaml
kubectl get nodes

This command generates a kubeconfig file, which you can use to access your virtual cluster via kubectl.