github/kamaji
1
0
Fork 0
mirror of https://github.com/clastix/kamaji.git synced 2026-04-15 06:56:47 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
master
kamaji/docs/content/cluster-api/openstack-infra-provider.md
T
Patryk RostkowskiGitHubPatryk Rostkowski
 e27202e0e0 docs: add OpenStack infra provider guide for Cluster API (#1103) 
Signed-off-by: Patryk Rostkowski <patrostkowski@gmail.com>
Co-authored-by: Patryk Rostkowski <prostkowski@cloudferro.com>
2026-03-23 11:34:57 +01:00

852 lines
25 KiB
Markdown
Raw Permalink Blame History

# OpenStack Infra Provider
Use the Cluster API [OpenStack Infra Provider (CAPO)](https://github.com/kubernetes-sigs/cluster-api-provider-openstack) with the Cluster API [Kamaji Control Plane Provider](https://github.com/clastix/cluster-api-control-plane-provider-kamaji) to create Kubernetes clusters.
!!! warning "Important Notes"
This walkthrough uses an advanced `externalClusterReference` setup. `kind` cluster is used as the Cluster API management cluster for ease of PoC, and it assumes you start with no existing Kubernetes infrastructure, only a working OpenStack environment.
Because a local `kind` cluster is typically not reachable from OpenStack instances, Kamaji is installed on a kubeadm-based Kubernetes cluster running inside OpenStack (the "control plane cluster"). The tenant control plane is then deployed there using `KamajiControlPlane.spec.deployment.externalClusterReference`.
If you already have a Kubernetes cluster reachable from your OpenStack nodes (for example, a Magnum-bootstrapped cluster, a kubeadm cluster in OpenStack, or any managed Kubernetes service), you can run both Cluster API controllers and Kamaji on that single management cluster and avoid `externalClusterReference`.
## Topology
```text
+--------------------+
| management cluster |
+---------+----------+
/ \
/ \
v v
+-----------------------+ +-------------------------+
| control plane cluster | | workload tenant cluster |
+-----------+-----------+ +------------+------------+
\ /
+--------------------------+
```
- The management cluster is a local `kind` cluster and runs Cluster API core, Kubeadm providers, CAPO, CAPH (Cluster API Helm add-on provider), and the Kamaji Cluster API control plane provider, and it reconciles both OpenStack `Cluster` resources.
- The control plane cluster is a kubeadm-based cluster on OpenStack that runs Kamaji and add-ons such as CCM, CNI, and CSI.
- The workload tenant cluster runs worker nodes on OpenStack, while its tenant control plane is hosted by Kamaji on the control plane cluster.
## Prerequisites
- `kind`, `kubectl`, `clusterctl`, `openstack` CLI, `openssl`, and `base64`
- Host images built with [image-builder](https://image-builder.sigs.k8s.io/) (or equivalent) and configured with `cloud-init` using the [OpenStack datasource](https://docs.cloud-init.io/en/latest/reference/datasources/openstack.html)
- An existing OpenStack environment with network connectivity, flavors, host images, and sufficient quotas for control plane and worker nodes
- OpenStack load balancer support (Octavia) for Service type `LoadBalancer` and API endpoint/public IP management for the kubeadm-based control plane cluster
!!! warning "ProviderID and cloud-init behavior"
CAPI host images must have the OpenStack cloud-init datasource configured. Otherwise, kubelet `provider-id` injection can resolve to a Nova-style non-UUID value and Cluster API reconciliation can stall. In this configuration, `kubeletExtraArgs` does not set `provider-id`; OpenStack CCM sets `providerID` after cluster start. See [cloud-init OpenStack datasource docs](https://docs.cloud-init.io/en/latest/reference/datasources/openstack.html) and [CAPO external cloud provider notes](https://cluster-api-openstack.sigs.k8s.io/topics/external-cloud-provider).
## Setup management cluster
```bash
kind create cluster --name management-cluster
kubectl cluster-info --context kind-management-cluster
```
### Install providers
Install ORC before initializing Cluster API providers:
```bash
export ORC_VERSION=v2.0.3
kubectl apply -f "https://github.com/k-orc/openstack-resource-controller/releases/download/${ORC_VERSION}/install.yaml"
```
Initialize Cluster API providers:
```bash
clusterctl init \
--core cluster-api \
--bootstrap kubeadm \
--control-plane kubeadm \
--infrastructure openstack \
--addon helm \
--control-plane kamaji
```
Enable feature gates required by `KamajiControlPlane.spec.deployment.externalClusterReference`:
```bash
kubectl -n kamaji-system patch deployment capi-kamaji-controller-manager \
--type='json' \
-p='[
{
"op": "replace",
"path": "/spec/template/spec/containers/0/args/1",
"value": "--feature-gates=ExternalClusterReference=true,ExternalClusterReferenceCrossNamespace=true"
}
]'
```
### Prepare OpenStack application credentials
Application credentials are used instead of username/password.
!!! warning "Credential Scope"
The same application credential is reused for both tenant clusters. In production, separate credentials per cluster or environment are preferred.
!!! note "Project selection"
Application credentials are created in the currently scoped project. Set `TARGET_PROJECT_ID` explicitly so credential creation is project-scoped.
Set and review variables:
```bash
export OPENSTACK_CLOUD_NAME=example-openstack
export TARGET_PROJECT_ID=<REPLACE_WITH_PROJECT_ID>
export OPENSTACK_APP_CREDENTIAL_NAME=capi-kamaji
export OPENSTACK_APP_CREDENTIAL_SECRET="$(openssl rand -hex 24)"
```
Create the application credential:
```bash
openstack --os-cloud "$OPENSTACK_CLOUD_NAME" application credential create \
--os-project-id "$TARGET_PROJECT_ID" \
--secret "$OPENSTACK_APP_CREDENTIAL_SECRET" \
"$OPENSTACK_APP_CREDENTIAL_NAME"
```
Store the generated credential ID:
```bash
export OPENSTACK_APP_CREDENTIAL_ID="$(openstack --os-cloud "$OPENSTACK_CLOUD_NAME" --os-project-id "$TARGET_PROJECT_ID" application credential show "$OPENSTACK_APP_CREDENTIAL_NAME" -f value -c id)"
```
### Prepare `clouds.yaml` and `cloud.conf`
Set OpenStack connection variables:
```bash
export OPENSTACK_AUTH_URL=https://openstack.example.com:5000/v3
export OPENSTACK_REGION_NAME=RegionOne
export OPENSTACK_INTERFACE=public
export OPENSTACK_IDENTITY_API_VERSION=3
export OPENSTACK_TLS_INSECURE=false
```
Build `clouds.yaml` and `cloud.conf` content in environment variables:
```bash
export CLOUDS_YAML_CONTENT="$(cat <<EOF
clouds:
${OPENSTACK_CLOUD_NAME}:
auth:
auth_url: ${OPENSTACK_AUTH_URL}
application_credential_id: ${OPENSTACK_APP_CREDENTIAL_ID}
application_credential_secret: ${OPENSTACK_APP_CREDENTIAL_SECRET}
auth_type: v3applicationcredential
region_name: ${OPENSTACK_REGION_NAME}
interface: ${OPENSTACK_INTERFACE}
identity_api_version: ${OPENSTACK_IDENTITY_API_VERSION}
EOF
)"
export CLOUD_CONF_CONTENT="$(cat <<EOF
[Global]
auth-url=${OPENSTACK_AUTH_URL}
application-credential-id=${OPENSTACK_APP_CREDENTIAL_ID}
application-credential-secret=${OPENSTACK_APP_CREDENTIAL_SECRET}
region=${OPENSTACK_REGION_NAME}
tls-insecure=${OPENSTACK_TLS_INSECURE}
interface=public
identity-api-version=3
auth-type=v3applicationcredential
EOF
)"
```
Encode both values for Kubernetes Secret `data` fields:
```bash
export REPLACE_BASE64_CLOUD_CONF="$(printf '%s' "$CLOUD_CONF_CONTENT" | base64 | tr -d '\n')"
export REPLACE_BASE64_CLOUDS_YAML="$(printf '%s' "$CLOUDS_YAML_CONTENT" | base64 | tr -d '\n')"
```
## Bootstrap the control plane cluster
### Set variables
Set shared variables used by the next commands:
!!! note "Variable Values"
Names, versions, network IDs, flavors, and image names in this section are example values. Replace them with environment-specific OpenStack values.
!!! info "Namespace Layout"
All resources are created in `kamaji-system`. Resources can also be separated by namespace with corresponding RBAC and policy controls.
```bash
export CLUSTER_NAMESPACE=kamaji-system
export CONTROL_PLANE_CLUSTER_NAME=kamaji-control-plane
export WORKLOAD_TENANT_CLUSTER_NAME=tenant-cluster-01
export KUBERNETES_VERSION=v1.33.0
export OPENSTACK_EXTERNAL_NETWORK_ID=00000000-0000-0000-0000-000000000000
export OPENSTACK_FLAVOR=m1.large
export OPENSTACK_IMAGE_NAME=ubuntu-2404-kube-v1.33.0
export OPENSTACK_SSH_KEY_NAME=default
```
### Apply cluster resources
```bash
kubectl apply -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
name: ${CONTROL_PLANE_CLUSTER_NAME}
namespace: ${CLUSTER_NAMESPACE}
labels:
clusterctl.cluster.x-k8s.io/move: "true"
type: Opaque
data:
cacert: ""
clouds.yaml: ${REPLACE_BASE64_CLOUDS_YAML}
---
apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
name: ${CONTROL_PLANE_CLUSTER_NAME}
namespace: ${CLUSTER_NAMESPACE}
labels:
addons.cluster.x-k8s.io/ccm: "true"
addons.cluster.x-k8s.io/cilium: "true"
addons.cluster.x-k8s.io/cinder-csi: "true"
addons.cluster.x-k8s.io/cert-manager: "true"
addons.cluster.x-k8s.io/kamaji: "true"
spec:
clusterNetwork:
services:
cidrBlocks:
- 10.96.0.0/12
pods:
cidrBlocks:
- 10.244.0.0/16
serviceDomain: cluster.local
controlPlaneRef:
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
kind: KubeadmControlPlane
name: ${CONTROL_PLANE_CLUSTER_NAME}-control-plane
namespace: ${CLUSTER_NAMESPACE}
infrastructureRef:
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: OpenStackCluster
name: ${CONTROL_PLANE_CLUSTER_NAME}
namespace: ${CLUSTER_NAMESPACE}
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: OpenStackCluster
metadata:
name: ${CONTROL_PLANE_CLUSTER_NAME}
namespace: ${CLUSTER_NAMESPACE}
spec:
apiServerLoadBalancer:
enabled: true
externalNetwork:
id: ${OPENSTACK_EXTERNAL_NETWORK_ID}
identityRef:
cloudName: ${OPENSTACK_CLOUD_NAME}
name: ${CONTROL_PLANE_CLUSTER_NAME}
managedSecurityGroups:
allowAllInClusterTraffic: true
managedSubnets:
- cidr: 10.0.0.0/24
dnsNameservers:
- 8.8.8.8
---
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
kind: KubeadmControlPlane
metadata:
name: ${CONTROL_PLANE_CLUSTER_NAME}-control-plane
namespace: ${CLUSTER_NAMESPACE}
spec:
replicas: 1
version: ${KUBERNETES_VERSION}
machineTemplate:
infrastructureRef:
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: OpenStackMachineTemplate
name: ${CONTROL_PLANE_CLUSTER_NAME}-control-plane
namespace: ${CLUSTER_NAMESPACE}
kubeadmConfigSpec:
format: cloud-config
files:
- path: /etc/kubernetes/cloud.conf
owner: root:root
permissions: "0600"
encoding: base64
content: ${REPLACE_BASE64_CLOUD_CONF}
clusterConfiguration:
apiServer:
extraArgs:
cloud-provider: external
controllerManager:
extraArgs:
cloud-provider: external
initConfiguration:
nodeRegistration:
kubeletExtraArgs:
cloud-provider: external
name: '{{ local_hostname }}'
joinConfiguration:
nodeRegistration:
kubeletExtraArgs:
cloud-provider: external
name: '{{ local_hostname }}'
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: OpenStackMachineTemplate
metadata:
name: ${CONTROL_PLANE_CLUSTER_NAME}-control-plane
namespace: ${CLUSTER_NAMESPACE}
spec:
template:
spec:
flavor: ${OPENSTACK_FLAVOR}
image:
filter:
name: ${OPENSTACK_IMAGE_NAME}
sshKeyName: ${OPENSTACK_SSH_KEY_NAME}
---
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
kind: KubeadmConfigTemplate
metadata:
name: ${CONTROL_PLANE_CLUSTER_NAME}-bootstrap
namespace: ${CLUSTER_NAMESPACE}
spec:
template:
spec:
format: cloud-config
files:
- path: /etc/kubernetes/cloud.conf
owner: root:root
permissions: "0600"
encoding: base64
content: ${REPLACE_BASE64_CLOUD_CONF}
joinConfiguration:
nodeRegistration:
imagePullPolicy: IfNotPresent
kubeletExtraArgs:
cloud-provider: external
name: '{{ local_hostname }}'
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: OpenStackMachineTemplate
metadata:
name: ${CONTROL_PLANE_CLUSTER_NAME}-worker
namespace: ${CLUSTER_NAMESPACE}
spec:
template:
spec:
flavor: ${OPENSTACK_FLAVOR}
image:
filter:
name: ${OPENSTACK_IMAGE_NAME}
sshKeyName: ${OPENSTACK_SSH_KEY_NAME}
---
apiVersion: cluster.x-k8s.io/v1beta1
kind: MachineDeployment
metadata:
name: ${CONTROL_PLANE_CLUSTER_NAME}-md-0
namespace: ${CLUSTER_NAMESPACE}
spec:
clusterName: ${CONTROL_PLANE_CLUSTER_NAME}
replicas: 1
selector:
matchLabels:
nodepool: worker
template:
metadata:
labels:
nodepool: worker
spec:
clusterName: ${CONTROL_PLANE_CLUSTER_NAME}
version: ${KUBERNETES_VERSION}
bootstrap:
configRef:
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
kind: KubeadmConfigTemplate
name: ${CONTROL_PLANE_CLUSTER_NAME}-bootstrap
namespace: ${CLUSTER_NAMESPACE}
infrastructureRef:
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: OpenStackMachineTemplate
name: ${CONTROL_PLANE_CLUSTER_NAME}-worker
namespace: ${CLUSTER_NAMESPACE}
EOF
```
### Apply add-on Helm release resources
!!! info "Add-on scheduling profile"
HelmChartProxy resources are configured to match both the control plane cluster and the workload tenant cluster. The chart values are set to work on both kubeadm-based and Kamaji-based clusters (for example, `nodeSelector: null`, broad tolerations, and `dnsPolicy: Default`).
```bash
kubectl apply -f - <<EOF
apiVersion: addons.cluster.x-k8s.io/v1alpha1
kind: HelmChartProxy
metadata:
name: openstack-cloud-controller-manager
namespace: ${CLUSTER_NAMESPACE}
spec:
clusterSelector:
matchLabels:
addons.cluster.x-k8s.io/ccm: "true"
repoURL: https://kubernetes.github.io/cloud-provider-openstack
chartName: openstack-cloud-controller-manager
version: 2.30.1
namespace: kube-system
releaseName: openstack-cloud-controller-manager
options:
enableClientCache: false
timeout: 10m0s
install:
createNamespace: true
upgrade:
maxHistory: 10
valuesTemplate: |
secret:
enabled: false
create: false
nodeSelector: null
tolerations:
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
- key: node.cloudprovider.kubernetes.io/uninitialized
operator: Equal
value: "true"
effect: NoSchedule
- key: node.kubernetes.io/not-ready
operator: Exists
effect: NoSchedule
- key: node.cluster.x-k8s.io/uninitialized
operator: Exists
effect: NoSchedule
podSecurityContext:
runAsUser: 0
dnsPolicy: Default
extraVolumes:
- name: cloud-config
hostPath:
path: /etc/kubernetes/cloud.conf
type: File
- name: k8s-certs
hostPath:
path: /etc/kubernetes/pki
extraVolumeMounts:
- name: cloud-config
mountPath: /etc/config/cloud.conf
readOnly: true
- name: k8s-certs
mountPath: /etc/kubernetes/pki
readOnly: true
---
apiVersion: addons.cluster.x-k8s.io/v1alpha1
kind: HelmChartProxy
metadata:
name: cilium
namespace: ${CLUSTER_NAMESPACE}
spec:
clusterSelector:
matchLabels:
addons.cluster.x-k8s.io/cilium: "true"
repoURL: https://helm.cilium.io
chartName: cilium
version: 1.18.4
namespace: kube-system
releaseName: cilium
options:
enableClientCache: false
timeout: 10m0s
install:
createNamespace: true
upgrade:
maxHistory: 10
valuesTemplate: |
cni:
chainingMode: portmap
prometheus:
enabled: false
operator:
replicas: 1
prometheus:
enabled: false
ipam:
operator:
clusterPoolIPv4PodCIDRList:
- "10.244.0.0/16"
clusterPoolIPv4MaskSize: 24
kubeProxyReplacement: false
sessionAffinity: true
tolerations:
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
- key: node.cloudprovider.kubernetes.io/uninitialized
operator: Equal
value: "true"
effect: NoSchedule
- key: node.kubernetes.io/not-ready
operator: Exists
effect: NoSchedule
---
apiVersion: addons.cluster.x-k8s.io/v1alpha1
kind: HelmChartProxy
metadata:
name: openstack-cinder-csi
namespace: ${CLUSTER_NAMESPACE}
spec:
clusterSelector:
matchLabels:
addons.cluster.x-k8s.io/cinder-csi: "true"
repoURL: https://kubernetes.github.io/cloud-provider-openstack
chartName: openstack-cinder-csi
version: 2.31.7
namespace: kube-system
releaseName: openstack-cinder-csi
options:
enableClientCache: false
timeout: 10m0s
install:
createNamespace: true
upgrade:
maxHistory: 10
valuesTemplate: |
csi:
plugin:
nodePlugin:
dnsPolicy: Default
controllerPlugin:
dnsPolicy: Default
nodeSelector: null
tolerations:
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
- key: node.cloudprovider.kubernetes.io/uninitialized
operator: Equal
value: "true"
effect: NoSchedule
- key: node.kubernetes.io/not-ready
operator: Exists
effect: NoSchedule
- key: CriticalAddonsOnly
operator: Exists
- key: node.cluster.x-k8s.io/uninitialized
operator: Exists
effect: NoSchedule
secret:
enabled: false
create: false
hostMount: true
filename: cloud.conf
storageClass:
enabled: false
custom: |-
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
annotations:
storageclass.kubernetes.io/is-default-class: "true"
labels:
name: cinder-ssd
name: cinder-ssd
allowVolumeExpansion: true
provisioner: cinder.csi.openstack.org
reclaimPolicy: Delete
volumeBindingMode: Immediate
parameters:
type: ssd
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
labels:
name: cinder-hdd
name: cinder-hdd
allowVolumeExpansion: true
provisioner: cinder.csi.openstack.org
reclaimPolicy: Delete
volumeBindingMode: Immediate
parameters:
type: hdd
---
apiVersion: addons.cluster.x-k8s.io/v1alpha1
kind: HelmChartProxy
metadata:
name: cert-manager
namespace: ${CLUSTER_NAMESPACE}
spec:
clusterSelector:
matchLabels:
addons.cluster.x-k8s.io/cert-manager: "true"
repoURL: https://charts.jetstack.io
chartName: cert-manager
namespace: cert-manager
releaseName: cert-manager
options:
enableClientCache: false
timeout: 10m0s
install:
createNamespace: true
upgrade:
maxHistory: 10
valuesTemplate: |
installCRDs: true
---
apiVersion: addons.cluster.x-k8s.io/v1alpha1
kind: HelmChartProxy
metadata:
name: kamaji
namespace: ${CLUSTER_NAMESPACE}
spec:
clusterSelector:
matchLabels:
addons.cluster.x-k8s.io/kamaji: "true"
repoURL: https://clastix.github.io/charts
chartName: kamaji
version: 0.0.0+latest
namespace: ${CLUSTER_NAMESPACE}
releaseName: kamaji
options:
enableClientCache: false
timeout: 10m0s
install:
createNamespace: true
upgrade:
maxHistory: 10
valuesTemplate: |
resources: null
EOF
```
### Verify control plane cluster
Apply and monitor:
```bash
clusterctl describe cluster "$CONTROL_PLANE_CLUSTER_NAME" -n "$CLUSTER_NAMESPACE"
clusterctl get kubeconfig "$CONTROL_PLANE_CLUSTER_NAME" -n "$CLUSTER_NAMESPACE" > ~/.kube/${CONTROL_PLANE_CLUSTER_NAME}.kubeconfig
KUBECONFIG=~/.kube/${CONTROL_PLANE_CLUSTER_NAME}.kubeconfig kubectl get nodes
```
## Bootstrap the tenant cluster with Kamaji control plane
### Set cross-cluster reference
Set the reference to the first cluster kubeconfig Secret:
```bash
export CONTROL_PLANE_CLUSTER_KUBECONFIG_SECRET_NAME="${CONTROL_PLANE_CLUSTER_NAME}-kubeconfig"
```
### Apply tenant cluster resources
```bash
kubectl apply -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
name: ${WORKLOAD_TENANT_CLUSTER_NAME}
namespace: ${CLUSTER_NAMESPACE}
labels:
clusterctl.cluster.x-k8s.io/move: "true"
type: Opaque
data:
cacert: ""
clouds.yaml: ${REPLACE_BASE64_CLOUDS_YAML}
---
apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
name: ${WORKLOAD_TENANT_CLUSTER_NAME}
namespace: ${CLUSTER_NAMESPACE}
labels:
addons.cluster.x-k8s.io/ccm: "true"
addons.cluster.x-k8s.io/cilium: "true"
addons.cluster.x-k8s.io/cinder-csi: "true"
spec:
clusterNetwork:
services:
cidrBlocks:
- 10.96.0.0/12
pods:
cidrBlocks:
- 10.244.0.0/16
serviceDomain: cluster.local
controlPlaneRef:
apiVersion: controlplane.cluster.x-k8s.io/v1alpha1
kind: KamajiControlPlane
name: ${WORKLOAD_TENANT_CLUSTER_NAME}
namespace: ${CLUSTER_NAMESPACE}
infrastructureRef:
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: OpenStackCluster
name: ${WORKLOAD_TENANT_CLUSTER_NAME}
namespace: ${CLUSTER_NAMESPACE}
---
apiVersion: controlplane.cluster.x-k8s.io/v1alpha1
kind: KamajiControlPlane
metadata:
name: ${WORKLOAD_TENANT_CLUSTER_NAME}
namespace: ${CLUSTER_NAMESPACE}
spec:
version: ${KUBERNETES_VERSION}
dataStoreName: default
replicas: 1
apiServer:
extraArgs:
- --cloud-provider=external
controllerManager:
extraArgs:
- --cloud-provider=external
kubelet:
cgroupfs: systemd
preferredAddressTypes:
- InternalIP
configurationJSONPatches:
- op: remove
path: /imagePullCredentialsVerificationPolicy
- op: remove
path: /mergeDefaultEvictionSettings
- op: remove
path: /crashLoopBackOff
- op: add
path: /cgroupDriver
value: systemd
network:
serviceType: LoadBalancer
addons:
coreDNS: {}
kubeProxy: {}
konnectivity: {}
deployment:
externalClusterReference:
deploymentNamespace: ${CLUSTER_NAMESPACE}
kubeconfigSecretName: ${CONTROL_PLANE_CLUSTER_KUBECONFIG_SECRET_NAME}
kubeconfigSecretKey: value
kubeconfigSecretNamespace: ${CLUSTER_NAMESPACE}
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: OpenStackCluster
metadata:
name: ${WORKLOAD_TENANT_CLUSTER_NAME}
namespace: ${CLUSTER_NAMESPACE}
spec:
apiServerLoadBalancer:
enabled: false
disableAPIServerFloatingIP: true
externalNetwork:
id: ${OPENSTACK_EXTERNAL_NETWORK_ID}
identityRef:
cloudName: ${OPENSTACK_CLOUD_NAME}
name: ${WORKLOAD_TENANT_CLUSTER_NAME}
managedSecurityGroups:
allowAllInClusterTraffic: true
managedSubnets:
- cidr: 10.0.0.0/24
dnsNameservers:
- 8.8.8.8
---
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
kind: KubeadmConfigTemplate
metadata:
name: ${WORKLOAD_TENANT_CLUSTER_NAME}-bootstrap
namespace: ${CLUSTER_NAMESPACE}
spec:
template:
spec:
format: cloud-config
files:
- path: /etc/kubernetes/cloud.conf
owner: root:root
permissions: "0600"
encoding: base64
content: ${REPLACE_BASE64_CLOUD_CONF}
joinConfiguration:
nodeRegistration:
imagePullPolicy: IfNotPresent
kubeletExtraArgs:
cloud-provider: external
name: '{{ local_hostname }}'
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: OpenStackMachineTemplate
metadata:
name: ${WORKLOAD_TENANT_CLUSTER_NAME}-worker
namespace: ${CLUSTER_NAMESPACE}
spec:
template:
spec:
flavor: ${OPENSTACK_FLAVOR}
image:
filter:
name: ${OPENSTACK_IMAGE_NAME}
sshKeyName: ${OPENSTACK_SSH_KEY_NAME}
---
apiVersion: cluster.x-k8s.io/v1beta1
kind: MachineDeployment
metadata:
name: ${WORKLOAD_TENANT_CLUSTER_NAME}-md-0
namespace: ${CLUSTER_NAMESPACE}
spec:
clusterName: ${WORKLOAD_TENANT_CLUSTER_NAME}
replicas: 1
selector:
matchLabels:
nodepool: worker
template:
metadata:
labels:
nodepool: worker
spec:
clusterName: ${WORKLOAD_TENANT_CLUSTER_NAME}
version: ${KUBERNETES_VERSION}
bootstrap:
configRef:
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
kind: KubeadmConfigTemplate
name: ${WORKLOAD_TENANT_CLUSTER_NAME}-bootstrap
namespace: ${CLUSTER_NAMESPACE}
infrastructureRef:
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: OpenStackMachineTemplate
name: ${WORKLOAD_TENANT_CLUSTER_NAME}-worker
namespace: ${CLUSTER_NAMESPACE}
EOF
```
### Verify tenant cluster
Apply and monitor:
```bash
clusterctl describe cluster "$WORKLOAD_TENANT_CLUSTER_NAME" -n "$CLUSTER_NAMESPACE"
clusterctl get kubeconfig "$WORKLOAD_TENANT_CLUSTER_NAME" -n "$CLUSTER_NAMESPACE" > ~/.kube/${WORKLOAD_TENANT_CLUSTER_NAME}.kubeconfig
KUBECONFIG=~/.kube/${WORKLOAD_TENANT_CLUSTER_NAME}.kubeconfig kubectl get nodes
```
## Clean up
```bash
kubectl delete cluster "$WORKLOAD_TENANT_CLUSTER_NAME" -n "$CLUSTER_NAMESPACE"
kubectl delete cluster "$CONTROL_PLANE_CLUSTER_NAME" -n "$CLUSTER_NAMESPACE"
kind delete cluster --name management-cluster
```
Reference in New Issue View Git Blame Copy Permalink