diff --git a/k8s/M6-rocky-cluster-role.yaml b/k8s/M6-rocky-cluster-role.yaml new file mode 100644 index 00000000..a2ab2f47 --- /dev/null +++ b/k8s/M6-rocky-cluster-role.yaml @@ -0,0 +1,10 @@ +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + namespace: rocky-test + name: rocky-full-access +rules: +- apiGroups: ["", extensions, apps] + resources: [deployments, replicasets, pods, services, ingresses, statefulsets] + verbs: [get, list, watch, create, update, patch, delete] # You can also use [*] diff --git a/k8s/M6-rocky-test-kustomization.yaml b/k8s/M6-rocky-test-kustomization.yaml new file mode 100644 index 00000000..ccaf2d90 --- /dev/null +++ b/k8s/M6-rocky-test-kustomization.yaml @@ -0,0 +1,8 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - ../../base/rocky +patches: + - path: M6-rocky-test-patch.yaml + target: + kind: Kustomization diff --git a/k8s/M6-rocky-test-patch.yaml b/k8s/M6-rocky-test-patch.yaml new file mode 100644 index 00000000..b70fbb4e --- /dev/null +++ b/k8s/M6-rocky-test-patch.yaml @@ -0,0 +1,7 @@ +apiVersion: kustomize.toolkit.fluxcd.io/v1beta1 +kind: Kustomization +metadata: + name: rocky + namespace: rocky-test +spec: + path: ./k8s/plain diff --git a/slides/images/M6-R01-config-files.png b/slides/images/M6-R01-config-files.png new file mode 100644 index 00000000..9d839043 Binary files /dev/null and b/slides/images/M6-R01-config-files.png differ diff --git a/slides/images/M6-flux-controllers.png b/slides/images/M6-flux-controllers.png new file mode 100644 index 00000000..dd641260 Binary files /dev/null and b/slides/images/M6-flux-controllers.png differ diff --git a/slides/images/M6-flux-schema.png b/slides/images/M6-flux-schema.png deleted file mode 100644 index cc5447b6..00000000 Binary files a/slides/images/M6-flux-schema.png and /dev/null differ diff --git a/slides/k8s/M6-R01-flux_configure-ROCKY-deployment.md b/slides/k8s/M6-R01-flux_configure-ROCKY-deployment.md new file mode 100644 index 00000000..17eb0f7f --- /dev/null +++ b/slides/k8s/M6-R01-flux_configure-ROCKY-deployment.md @@ -0,0 +1,265 @@ +# R01- Configuring **_🎸ROCKY_** deployment with Flux + +The **_⚙️OPS_** team manages 2 distinct envs: _**⚗️TEST**_ et _**🚜PROD**_ +Thanks to _Kustomize_ + 1. it creates a **_base_** common config + 2. this common config is overwritten with a _**⚗️TEST**_ _tenant_-specific configuration + 3. the same applies with a _**🚜PROD**_-specific configuration + +> 💡 This seems complex, but no worries: Flux's CLI handles the essentials. + +--- + +## Creating the **_🎸ROCKY_**-dedicated _tenant_ in _**⚗️TEST**_ env + +- Using the `flux` _CLI_, we create the file configuring the **_🎸ROCKY_** team's dedicated _tenant_… +- … this file takes place in the `base` common configuration for both envs + +.lab[ + +```bash +k8s@shpod:~/fleet-config-using-flux-XXXXX$ \ + mkdir -p ./tenants/base/rocky && \ + flux create tenant rocky \ + --with-namespace=rocky-test \ + --cluster-role=rocky-full-access \ + --export > ./tenants/base/rocky/rbac.yaml +``` + +] + +--- + +### 📂 ./tenants/base/rocky/rbac.yaml + +Let's see our file… + +3 resources are created: + +- `Namespace`, +- a `ServiceAccount` and +- a `ClusterRoleBinding` + +`Flux` impersonates as this `ServiceAccount` when it applies any resources found in this tenant-dedicated source(s) +By default, the `ServiceAccount` is bound to a `ClusterRole` named `cluster-admin` + +It means that, any team that maintain the sourced `Github` repository is able to apply Kubernetes resources as `cluster-admin` +A not that much isolated tenant! 😕 + +That's why the **_⚙️OPS_** team forces a binding to a specific `ClusterRole` +Let's create this ClusterRole permissions! + +--- + +## _namespace_ isolation for **_🎸ROCKY_** + + +.lab[ + +- Here is a `ClusterRole` with permissions restricted to the dedicated `Namespace` +```bash +k8s@shpod:~/fleet-config-using-flux-XXXXX$ \ + cp ~/container.training/k8s/M6-rocky-cluster-role.yaml ./tenants/base/rocky/ +``` + +] + +--- + +## Creating `Github` source in Flux for **_🎸ROCKY_** app repository + +A specific _branch_ of the `Github` repository is monitored by the `Flux` source + +.lab[ + +- ⚠️ you may change the **repository URL** to the one of your own clone +```bash +k8s@shpod:~/fleet-config-using-flux-XXXXX$ flux create source git rocky-app \ + --namespace=rocky-test \ + --url=https://github.com/Musk8teers/container.training-spring-music/ \ + --branch=rocky --export > ./tenants/base/rocky/sync.yaml +k8s@shpod:~/fleet-config-using-flux-XXXXX$ flux create kustomization rocky \ + --namespace=rocky-test \ + --service-account=rocky \ + --source=GitRepository/rocky-app \ + --path="./k8s/" --export >> ./tenants/base/rocky/sync.yaml +k8s@shpod:~/fleet-config-using-flux-XXXXX$ cd ./tenants/base/rocky/ && \ + kustomize create --autodetect && cd - +``` + +] + +--- + +### 📂 Flux config files + +Let's review our `Flux` configuration files + +.lab[ + +```bash +k8s@shpod:~/fleet-config-using-flux-XXXXX$ \ + cat ./tenants/base/rocky/sync.yaml && \ + cat ./tenants/base/rocky/kustomization.yaml +``` + +] + +--- + +## Adding a kustomize patch for _**⚗️TEST**_ cluster deployment + +Remember! +The `Flux` tenant-dedicated configuration is looking for this file `.tenants/test/rocky/kustomization.yaml` +It has been configured here: `clusters/CLOUDY/tenants.yaml` + +All the files we just created are located in `.tenants/base/rocky` (remember the DRY strategy) + +So we have to create a specific kustomization in the right location + +```bash +k8s@shpod:~/fleet-config-using-flux-XXXXX$ \ + mkdir -p ./tenants/test/rocky && \ + cp ~/container.training/k8s/M6-rocky-test-patch.yaml ./tenants/test/rocky/ && \ + cp ~/container.training/k8s/M6-rocky-test-kustomization.yaml ./tenants/test/rocky/kustomization.yaml +``` + +--- + +### Synchronizing Flux config with its Github repo + +Locally, our `Flux` config repo is ready +The ops team has to push it to `Github` + +.lab[ + +```bash +k8s@shpod:~/fleet-config-using-flux-XXXXX$ \ + git add . && \ + git commit -m':wrench: :construction_worker: add ROCKY tenant configuration' && \ + git push +``` + +] + +--- + +class: pic + +![rocky config files](images/M6-R01-config-files.png) + +--- + +class: extra-details + +### Flux resources for ROCKY tenant 1/2 + +.lab[ + +```bash +k8s@shpod:~$ flux get all -A +NAMESPACE NAME REVISION SUSPENDED + READY MESSAGE +flux-system gitrepository/flux-system main@sha1:8ffd72cf False + True stored artifact for revision 'main@sha1:8ffd72cf' +rocky-test gitrepository/rocky-app rocky@sha1:ffe9f3fe False + True stored artifact for revision 'rocky@sha1:ffe9f3fe' +(…) +``` + +] + +--- + +class: extra-details + +### Flux resources for ROCKY tenant 2/2 + +.lab[ + +```bash +k8s@shpod:~$ flux get all -A +(…) +NAMESPACE NAME REVISION SUSPENDED + READY MESSAGE +flux-system kustomization/flux-system main@sha1:8ffd72cf False + True Applied revision: main@sha1:8ffd72cf +flux-system kustomization/tenant-prod False + False kustomization path not found: stat /tmp/kustomization-1164119282/tenants/prod: no such file or directory +flux-system kustomization/tenant-test main@sha1:8ffd72cf False + True Applied revision: main@sha1:8ffd72cf +rocky-test kustomization/rocky False + False StatefulSet/db dry-run failed (Forbidden): statefulsets.apps "db" is forbidden: User "system:serviceaccount:rocky-test:rocky" cannot patch resource "statefulsets" in API group "apps" at the cluster scope +``` + +] + +And here is our 2nd Flux error(s)! 😅 + +--- + +class: extra-details + +### Flux Kustomization, mutability, … + +🔍 Notice that none of the expected resources is created: +the whole kustomization is rejected, even if the `StatefulSet` is this only resource that fails! + +🔍 Flux Kustomization uses the dry-run feature to templatize the resources and then applying patches onto them +Good but some resources are not completely mutable, such as `StatefulSets` + +We have to fix the mutation by applying the change without having to patch the resource. + +🔍 Simply add the `spec.targetNamespace: rocky-test` to the `Kustomization` named `rocky` + +--- + +class: extra-details + +## And then it's deployed + +You should see the following resources in the `rocky-test` namespace + +.lab[ + +```bash +k8s@shpod-578d64468-tp7r2 ~/$ k get all -n rocky-test +NAME READY STATUS RESTARTS AGE +pod/db-0 1/1 Running 0 47s +pod/web-6c677bf97f-c7pkv 0/1 Running 1 (22s ago) 47s +pod/web-6c677bf97f-p7b4r 0/1 Running 1 (19s ago) 47s + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/db ClusterIP 10.32.6.128 5432/TCP 48s +service/web ClusterIP 10.32.2.202 80/TCP 48s + +NAME READY UP-TO-DATE AVAILABLE AGE +deployment.apps/web 0/2 2 0 47s + +NAME DESIRED CURRENT READY AGE +replicaset.apps/web-6c677bf97f 2 2 0 47s + +NAME READY AGE +statefulset.apps/db 1/1 47s +``` + +] + +--- + +## Upgrading ROCKY app + +The Git source named `rocky-app` is pointing at +- a Github repository named [https://github.com/Musk8teers/container.training-spring-music/] +- on its branch named `rocky` + +This branch deploy the v1.0.0 of the _Web_ app: +`spec.template.spec.containers.image: ghcr.io/musk8teers/container.training-spring-music:1.0.0` + +What happens if we upgrade this branch to deploy `v1.0.1` of the _Web_ app? + +--- + +## tenant **_🏭PROD_** + +**_🏭PROD_** tenant is still waiting for its `Flux` configuration, but don't bother for it right now. diff --git a/slides/k8s/M6-T02-flux-install.md b/slides/k8s/M6-T02-flux-install.md index a249a398..9b09084b 100644 --- a/slides/k8s/M6-T02-flux-install.md +++ b/slides/k8s/M6-T02-flux-install.md @@ -9,12 +9,22 @@ This managed cluster comes preinstalled with specific features: - specific _Storage Classes_ based on Scaleway _IaaS_ block storage offerings - a `Cilium` _CNI_ stack already set up +--- + +## Accessing the managed Kubernetes cluster + To access our cluster, we'll connect via [`shpod`](https://github.com/jpetazzo/shpod) .lab[ +- If you already have a kubectl on your desktop computer ```bash -kubectl run shpod --image=jpetazzo/shpod --overrides='{ "spec": { "serviceAccountName": "" } }' +kubectl -n shpod run shpod --image=jpetazzo/shpod +kubectl -n shpod exec -it shpod -- bash +``` +- or directly via ssh +```bash +ssh -p myPort k8s@mySHPODSvcIpAddress ``` ] @@ -45,10 +55,10 @@ Before installation, we need to check that: .lab[ ```bash -shpod:~# flux --version +k8s@shpod:~$ flux --version flux version 2.5.1 -shpod:~# flux check --pre +k8s@shpod:~$ flux check --pre ► checking prerequisites ✔ Kubernetes 1.32.3 >=1.30.0-0 ✔ prerequisites checks passed @@ -85,42 +95,45 @@ class: pic ### Creating dedicated `Github` repo to host Flux config +.lab[ + - let's replace the `GITHUB_TOKEN` value by our _Personal Access Token_ - and the `GITHUB_REPO` value by our specific repository name -.lab[ - ```bash -shpod:~# export GITHUB_TOKEN="my-token" -shpod:~# export GITHUB_USER="container-training-fleet" -shpod:~# export GITHUB_REPO="fleet-config-using-flux-XXXXX" -shpod:~# flux bootstrap github \ - --owner=${GITHUB_USER} \ - --repository=${GITHUB_REPO} \ - --team=OPS \ - --team=ROCKY --team=MOVY \ - --path=clusters/CLOUDY +k8s@shpod:~$ export GITHUB_TOKEN="my-token" && \ + export GITHUB_USER="container-training-fleet" && \ + export GITHUB_REPO="fleet-config-using-flux-XXXXX" + +k8s@shpod:~$ flux bootstrap github \ + --owner=${GITHUB_USER} \ + --repository=${GITHUB_REPO} \ + --team=OPS \ + --team=ROCKY --team=MOVY \ + --path=clusters/CLOUDY ``` ] --- +class: extra-details + Here is the result ```bash -✔ repository "https://github.com/container-training-fleet/fleet-config-using-flux-lpiot" created +✔ repository "https://github.com/container-training-fleet/fleet-config-using-flux-XXXXX" created ► reconciling repository permissions ✔ granted "maintain" permissions to "OPS" ✔ granted "maintain" permissions to "ROCKY" ✔ granted "maintain" permissions to "MOVY" ► reconciling repository permissions ✔ reconciled repository permissions -► cloning branch "main" from Git repository "https://github.com/container-training-fleet/fleet-config-using-flux-lpiot.git" +► cloning branch "main" from Git repository "https://github.com/container-training-fleet/fleet-config-using-flux-XXXXX.git" ✔ cloned repository ► generating component manifests ✔ generated component manifests ✔ committed component manifests to "main" ("7c97bdeb5b932040fd8d8a65fe1dc84c66664cbf") -► pushing component manifests to "https://github.com/container-training-fleet/fleet-config-using-flux-lpiot.git" +► pushing component manifests to "https://github.com/container-training-fleet/fleet-config-using-flux-XXXXX.git" ✔ component manifests are up to date ► installing components in "flux-system" namespace ✔ installed components @@ -128,13 +141,13 @@ Here is the result ► determining if source secret "flux-system/flux-system" exists ► generating source secret ✔ public key: ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFqaT8B8SezU92qoE+bhnv9xONv9oIGuy7yVAznAZfyoWWEVkgP2dYDye5lMbgl6MorG/yjfkyo75ETieAE49/m9D2xvL4esnSx9zsOLdnfS9W99XSfFpC2n6soL+Exodw== -✔ configured deploy key "flux-system-main-flux-system-./clusters/CLOUDY" for "https://github.com/container-training-fleet/fleet-config-using-flux-lpiot" +✔ configured deploy key "flux-system-main-flux-system-./clusters/CLOUDY" for "https://github.com/container-training-fleet/fleet-config-using-flux-XXXXX" ► applying source secret "flux-system/flux-system" ✔ reconciled source secret ► generating sync manifests ✔ generated sync manifests ✔ committed sync manifests to "main" ("11035e19cabd9fd2c7c94f6e93707f22d69a5ff2") -► pushing sync manifests to "https://github.com/container-training-fleet/fleet-config-using-flux-lpiot.git" +► pushing sync manifests to "https://github.com/container-training-fleet/fleet-config-using-flux-XXXXX.git" ► applying sync manifests ✔ reconciled sync configuration ◎ waiting for GitRepository "flux-system/flux-system" to be reconciled @@ -184,17 +197,19 @@ Let's review our `Flux` configuration files we've created and pushed into the `G --- class: pic - -![Flux architecture](images/M6-flux-schema.png) + +![Flux architecture](images/M6-flux-controllers.png) --- +class: extra-details + ### Flux resources 1/2 .lab[ ```bash -shpod:~# kubectl get all --namespace flux-system +k8s@shpod:~$ kubectl get all --namespace flux-system NAME READY STATUS RESTARTS AGE pod/helm-controller-b6767d66-h6qhk 1/1 Running 0 5m pod/kustomize-controller-57c7ff5596-94rnd 1/1 Running 0 5m @@ -212,12 +227,14 @@ service/webhook-receiver ClusterIP 10.96.28.236 80/ --- +class: extra-details + ### Flux resources 2/2 .lab[ ```bash -shpod:~# kubectl get all --namespace flux-system +k8s@shpod:~$ kubectl get all --namespace flux-system (…) NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/helm-controller 1/1 1 1 5m @@ -246,12 +263,14 @@ replicaset.apps/source-controller-6ff87cb475 1 1 1 --- +class: extra-details + ### Flux resources that have been created .lab[ ```bash -shpod:~# flux get all --all-namespaces +k8s@shpod:~$ flux get all --all-namespaces NAMESPACE NAME REVISION SUSPENDED READY MESSAGE flux-system gitrepository/flux-system main@sha1:d48291a8 False @@ -302,6 +321,8 @@ The `Flux` component named `kustomize controller` look for `Kustomize` resources --- +class: extra-details + ### 2 different kustomization resources ⚠️ `Flux` uses 2 distinct resources with `kind: kustomization` @@ -318,7 +339,7 @@ apiVersion: kustomize.toolkit.fluxcd.io/v1 group kind: Kustomization ``` -describes where `Flux kustomize-controller` look for a `kustomization.yaml` file in a given `Flux` code-based source +describes where `Flux kustomize-controller` looks for a `kustomization.yaml` file in a given `Flux` code-based source --- diff --git a/slides/k8s/M6-T03-installing-tenants.md b/slides/k8s/M6-T03-installing-tenants.md index 6a55d945..a7264ccc 100644 --- a/slides/k8s/M6-T03-installing-tenants.md +++ b/slides/k8s/M6-T03-installing-tenants.md @@ -24,11 +24,11 @@ While basic `Flux` behavior is to use a single configuration directory applied b Several tenants are created - per env - - for **_⚗️TEST_** - - and **_🏭PROD_** + - for **_⚗️TEST_** + - and **_🏭PROD_** - per team - - for **_🎸ROCKY_** - - and **_🎬MOVY_** + - for **_🎸ROCKY_** + - and **_🎬MOVY_** --- @@ -38,21 +38,32 @@ class: pic --- +### Flux CLI works locally + +First, we have to **locally** clone your `Flux` configuration `Github` repository + +- create an ssh key pair +- add the **public** key to your `Github` repository (**with write access**) +- and git clone the repository + +--- + ### The command line 1/2 Creating the **_⚗️TEST_** tenant .lab[ +- ⚠️ Think about renaming the repo with your own suffix ```bash -shpod:~# cd fleet-config-using-flux-XXXXX/ -shpod:~/fleet-config-using-flux-lpiot# flux create kustomization tenants \ - --namespace=flux-system \ - --source=GitRepository/flux-system \ - --path ./tenants/test \ - --prune \ - --interval=1m \ - --export >> clusters/CLOUDY/tenants.yaml +k8s@shpod:~$ cd fleet-config-using-flux-XXXXX/ +k8s@shpod:~/fleet-config-using-flux-XXXXX$ \ + flux create kustomization tenant-test \ + --namespace=flux-system \ + --source=GitRepository/flux-system \ + --path ./tenants/test \ + --interval=1m \ + --prune --export >> clusters/CLOUDY/tenants.yaml ``` ] @@ -66,13 +77,13 @@ Then we create the **_🏭PROD_** tenant .lab[ ```bash -shpod:~/fleet-config-using-flux-lpiot# flux create kustomization tenants \ - --namespace=flux-system \ - --source=GitRepository/flux-system \ - --path ./tenants/prod \ - --prune \ - --interval=3m \ - --export >> clusters/CLOUDY/tenants.yaml +k8s@shpod:~/fleet-config-using-flux-XXXXX$ \ + flux create kustomization tenant-prod \ + --namespace=flux-system \ + --source=GitRepository/flux-system \ + --path ./tenants/prod \ + --interval=3m \ + --prune --export >> clusters/CLOUDY/tenants.yaml ``` ] @@ -97,16 +108,20 @@ Let's review the `fleet-config-using-flux-XXXXX/clusters/CLOUDY/tenants.yaml` fi .lab[ ```bash -shpod:~/fleet-config-using-flux-lpiot# flux get all +k8s@shpod:~/fleet-config-using-flux-XXXXX$ flux get all NAMESPACE NAME REVISION SUSPENDED - READY MESSAGE -flux-system gitrepository/flux-system main@sha1:4db19114 False - True stored artifact for revision 'main@sha1:4db19114' + READY MESSAGE +flux-system gitrepository/flux-system main@sha1:0466652e False + True stored artifact for revision 'main@sha1:0466652e' NAMESPACE NAME REVISION SUSPENDED - READY MESSAGE -flux-system kustomization/flux-system main@sha1:d48291a8 False - False kustomize build failed: accumulating resources: accumulation err='accumulating resources from './tenants.yaml': may not add resource with an already registered id: Kustomization.v1.kustomize.toolkit.fluxcd.io/tenants.flux-system': must build at directory: '/tmp/kustomization-689086759/clusters/CLOUDY/tenants.yaml': file is not directory + READY MESSAGE +kustomization/flux-system main@sha1:0466652e False True + Applied revision: main@sha1:0466652e +kustomization/tenant-prod False False + kustomization path not found: stat /tmp/kustomization-417981261/tenants/prod: no such file or directory +kustomization/tenant-test False False + kustomization path not found: stat /tmp/kustomization-2532810750/tenants/test: no such file or directory ``` ] diff --git a/slides/m6.yml b/slides/m6.yml index c73b4b76..385c37ba 100644 --- a/slides/m6.yml +++ b/slides/m6.yml @@ -17,8 +17,8 @@ exclude: content: - k8s/M6-START-a-company-scenario.md - - k8s/M6-T02-flux-install.md + - k8S/M6-T02-flux-install.md - k8s/M6-T03-installing-tenants.md - # - k8s/M6-R01-flux_configure-ROCKY-deployment.md + - k8s/M6-R01-flux_configure-ROCKY-deployment.md # - k8S/M6-T01-TEST-cluster-creation.md # - k8S/M6-T02-1-flux-overview.md