Merge pull request #664 from lpiot/main

The missing slides…😅
This commit is contained in:
Jérôme Petazzoni
2025-06-13 08:48:44 +02:00
committed by GitHub
12 changed files with 626 additions and 12 deletions

View File

@@ -0,0 +1,10 @@
apiVersion: v1
kind: Namespace
metadata:
labels:
app.kubernetes.io/instance: flux-system
app.kubernetes.io/part-of: flux
app.kubernetes.io/version: v2.5.1
pod-security.kubernetes.io/warn: restricted
pod-security.kubernetes.io/warn-version: latest
name: kyverno

View File

@@ -0,0 +1,72 @@
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: flux-multi-tenancy
spec:
validationFailureAction: enforce
rules:
- name: serviceAccountName
exclude:
resources:
namespaces:
- flux-system
match:
resources:
kinds:
- Kustomization
- HelmRelease
validate:
message: ".spec.serviceAccountName is required"
pattern:
spec:
serviceAccountName: "?*"
- name: kustomizationSourceRefNamespace
exclude:
resources:
namespaces:
- flux-system
- ingress-nginx
- kyverno
- monitoring
- openebs
match:
resources:
kinds:
- Kustomization
preconditions:
any:
- key: "{{request.object.spec.sourceRef.namespace}}"
operator: NotEquals
value: ""
validate:
message: "spec.sourceRef.namespace must be the same as metadata.namespace"
deny:
conditions:
- key: "{{request.object.spec.sourceRef.namespace}}"
operator: NotEquals
value: "{{request.object.metadata.namespace}}"
- name: helmReleaseSourceRefNamespace
exclude:
resources:
namespaces:
- flux-system
- ingress-nginx
- kyverno
- monitoring
- openebs
match:
resources:
kinds:
- HelmRelease
preconditions:
any:
- key: "{{request.object.spec.chart.spec.sourceRef.namespace}}"
operator: NotEquals
value: ""
validate:
message: "spec.chart.spec.sourceRef.namespace must be the same as metadata.namespace"
deny:
conditions:
- key: "{{request.object.spec.chart.spec.sourceRef.namespace}}"
operator: NotEquals
value: "{{request.object.metadata.namespace}}"

View File

@@ -0,0 +1,29 @@
apiVersion: v1
kind: Namespace
metadata:
labels:
app.kubernetes.io/instance: flux-system
app.kubernetes.io/part-of: flux
app.kubernetes.io/version: v2.5.1
pod-security.kubernetes.io/warn: restricted
pod-security.kubernetes.io/warn-version: latest
name: monitoring
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: grafana
namespace: monitoring
spec:
ingressClassName: nginx
rules:
- host: grafana.test.metal.mybestdomain.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: kube-prometheus-stack-grafana
port:
number: 80

View File

@@ -4,7 +4,7 @@ metadata:
name: rocky
namespace: rocky-test
spec:
ingressClassName: "nginx"
ingressClassName: nginx
rules:
- host: rocky.test.mybestdomain.com
http:

Binary file not shown.

After

Width:  |  Height:  |  Size: 570 KiB

View File

@@ -192,6 +192,8 @@ pod network is still full mesh and any connection is authorized.
---
class: extra-details
## Adding NetworkPolicies to **_🎸ROCKY_** and **_🎬MOVY_** namespaces
`Network policies` may be seen as the firewall feature in the pod network.
@@ -307,12 +309,12 @@ gitGraph
checkout OPS
commit id:'Prometheus install'
checkout METAL_TEST-PROD
checkout TEST-env
merge OPS type: HIGHLIGHT
checkout OPS
commit id:'Kyverno install'
commit id:'Kyverno rules'
checkout METAL_TEST-PROD
checkout TEST-env
merge OPS type: HIGHLIGHT
</pre>

View File

@@ -236,7 +236,6 @@ The **_⚙OPS_** team aims to provide clusters offering the following feature
### What our scenario might look like…
<!-- TODO: to upgrade according to the actual scenario that is eventually build -->
<pre class="mermaid">
%%{init:
{
@@ -328,13 +327,13 @@ gitGraph
checkout OPS
commit id:'Prometheus install'
checkout METAL_TEST-PROD
checkout TEST-env
merge OPS type: HIGHLIGHT
checkout OPS
commit id:'Kyverno install'
commit id:'Kyverno rules'
checkout METAL_TEST-PROD
checkout TEST-env
merge OPS type: HIGHLIGHT
checkout OPS

View File

@@ -305,7 +305,9 @@ flux-system kustomization/flux-system main@sha1:d48291a8 False
---
### 🔍 Flux -- for more info
class: extra-details
### Flux -- for more info
Please, refer to the [`Flux` chapter in the High Five M3 module](./3.yml.html#toc-helm-chart-format)
@@ -343,13 +345,17 @@ describes where `Flux kustomize-controller` looks for a `kustomization.yaml` fil
---
### 🔍 Kustomize -- for more info
class: extra-details
### Kustomize -- for more info
Please, refer to the [`Kustomize` chapter in the High Five M3 module](./3.yml.html#toc-kustomize)
---
### 🔍 Group / Version / Kind -- for more info
class: extra-details
### Group / Version / Kind -- for more info
For more info about how Kubernetes resource natures are identified by their `Group / Version / Kind` triplet…
… please, refer to the [`Kubernetes API` chapter in the High Five M5 module](./5.yml.html#toc-the-kubernetes-api)

View File

@@ -13,7 +13,9 @@ The regular way to do it in Kubernetes is to configure an `Ingress` resource.
---
## 🔍 Ingress -- for more info
class: extra-details
## Ingress -- for more info
Please, refer to the [`Ingress` chapter in the High Five M2 module](./2.yml.html#toc-exposing-http-services-with-ingress-resources)
@@ -143,7 +145,7 @@ class: pic
---
### Here is the result!
### Here is the result
After Flux reconciled the whole bunch of sources and kustomizations, you should see

View File

@@ -0,0 +1,241 @@
## introducing Kyverno
Kyverno is a tool to extend Kubernetes permission management to express complex policies…
</br>… and override manifests delivered by client teams.
---
class: extra-details
### Kyverno -- for more info
Please, refer to the [`Setting up Kubernetes` chapter in the High Five M4 module](./4.yml.html#toc-policy-management-with-kyverno) for more infos about `Kyverno`.
---
## Creating an `Helm` source in Flux for OpenEBS Helm chart
.lab[
```bash
k8s@shpod:~/fleet-config-using-flux-XXXXX$ \
mkdir -p clusters/CLOUDY/kyverno && \
cp -pr ~/container.training/k8s/
k8s@shpod ~$ flux create source helm kyverno \
--namespace=kyverno \
--url=https://kyverno.github.io/kyverno/ \
--interval=3m \
--export > ./clusters/CLOUDY/kyverno/sync2.yaml
```
]
---
## Creating the `HelmRelease` in Flux
.lab[
```bash
k8s@shpod ~$ flux create helmrelease kyverno \
--namespace=kyverno \
--source=HelmRepository/kyverno.flux-system \
--target-namespace=kyverno \
--create-target-namespace=true \
--chart-version=">=3.4.2" \
--chart=kyverno \
--export >> ./clusters/CLOUDY/kyverno/sync.yaml
```
]
---
## Add Kyverno policy
This polivy is just an example.
It enforces the use of a `Service Account` in `Flux` configurations
```bash
k8s@shpod:~/fleet-config-using-flux-XXXXX$ \
mkdir -p clusters/CLOUDY/kyverno-policies && \
cp -pr ~/container.training/k8s/M6-kyverno-enforce-service-account.yaml \
./clusters/CLOUDY/kyverno-policies/
---
### Creating `kustomization` in Flux for Kyverno policies
.lab[
```bash
k8s@shpod:~/fleet-config-using-flux-XXXXX$ \
flux create kustomization kyverno-policies \
--namespace=kyverno \
--source=GitRepository/flux-system \
--path="./clusters/CLOUDY/kyverno-policies/" \
--prune true --interval 5m \
--depends-on kyverno \
--export >> ./clusters/CLOUDY/kyverno-policies/sync.yaml
```
]
## Apply Kyverno policy
```bash
flux create kustomization
--path
--source GitRepository/
--export > ./clusters/CLOUDY/kyverno-policies/sync.yaml
```
---
## Add Kyverno dependency for **_⚗TEST_** cluster
- Now that we've got `Kyverno` policies,
- ops team will enforce any upgrade from any kustomization in our dev team tenants
- to wait for the `kyverno` policies to be reconciled (in a `Flux` perspective)
- upgrade file `./clusters/CLOUDY/tenants.yaml`,
- by adding this property: `spec.dependsOn.{name: kyverno-policies}`
---
class: pic
![Running Mario](images/M6-running-Mario.gif)
---
### Debugging
`Kyverno-policies` `Kustomization` failed because `spec.dependsOn` property can only target a resource from the same `Kind`.
- Let's suppress the `spec.dependsOn` property.
Now `Kustomizations` for **_🎸ROCKY_** and **_🎬MOVY_** tenants failed because of our policies.
---
### 🗺️ Where are we in our scenario?
<pre class="mermaid">
%%{init:
{
"theme": "default",
"gitGraph": {
"mainBranchName": "OPS",
"mainBranchOrder": 0
}
}
}%%
gitGraph
commit id:"0" tag:"start"
branch ROCKY order:4
branch MOVY order:5
branch YouRHere order:6
checkout OPS
commit id:'Flux install on CLOUDY cluster' tag:'T01'
branch TEST-env order:1
commit id:'FLUX install on TEST' tag:'T02' type: HIGHLIGHT
checkout OPS
commit id:'Flux config. for TEST tenant' tag:'T03'
commit id:'namespace isolation by RBAC'
checkout TEST-env
merge OPS id:'ROCKY tenant creation' tag:'T04'
checkout OPS
commit id:'ROCKY deploy. config.' tag:'R01'
checkout TEST-env
merge OPS id:'TEST ready to deploy ROCKY' type: HIGHLIGHT tag:'R02'
checkout ROCKY
commit id:'ROCKY' tag:'v1.0.0'
checkout TEST-env
merge ROCKY tag:'ROCKY v1.0.0'
checkout OPS
commit id:'Ingress-controller config.' tag:'T05'
checkout TEST-env
merge OPS id:'Ingress-controller install' type: HIGHLIGHT tag:'T06'
checkout OPS
commit id:'ROCKY patch for ingress config.' tag:'R03'
checkout TEST-env
merge OPS id:'ingress config. for ROCKY app'
checkout ROCKY
commit id:'blue color' tag:'v1.0.1'
checkout TEST-env
merge ROCKY tag:'ROCKY v1.0.1'
checkout ROCKY
commit id:'pink color' tag:'v1.0.2'
checkout TEST-env
merge ROCKY tag:'ROCKY v1.0.2'
checkout OPS
commit id:'FLUX config for MOVY deployment' tag:'M01'
checkout TEST-env
merge OPS id:'FLUX ready to deploy MOVY' type: HIGHLIGHT tag:'M02'
checkout MOVY
commit id:'MOVY' tag:'v1.0.3'
checkout TEST-env
merge MOVY tag:'MOVY v1.0.3' type: REVERSE
checkout OPS
commit id:'Network policies'
checkout TEST-env
merge OPS type: HIGHLIGHT tag:'T07'
checkout OPS
commit id:'k0s install on METAL cluster' tag:'K01'
commit id:'Flux config. for METAL cluster' tag:'K02'
branch METAL_TEST-PROD order:3
commit id:'ROCKY/MOVY tenants on METAL' type: HIGHLIGHT
checkout OPS
commit id:'Flux config. for OpenEBS' tag:'K03'
checkout METAL_TEST-PROD
merge OPS id:'openEBS on METAL' type: HIGHLIGHT
checkout OPS
commit id:'Prometheus install'
checkout TEST-env
merge OPS type: HIGHLIGHT
checkout OPS
commit id:'Kyverno install'
commit id:'Kyverno rules'
checkout TEST-env
merge OPS type: HIGHLIGHT
checkout YouRHere
commit id:'x'
checkout OPS
merge YouRHere id:'YOU ARE HERE'
checkout OPS
commit id:'Flux config. for PROD tenant' tag:'P01'
branch PROD-env order:2
commit id:'ROCKY tenant on PROD'
checkout OPS
commit id:'ROCKY patch for PROD' tag:'R04'
checkout PROD-env
merge OPS id:'PROD ready to deploy ROCKY' type: HIGHLIGHT
checkout PROD-env
merge ROCKY tag:'ROCKY v1.0.2'
checkout MOVY
commit id:'MOVY HELM chart' tag:'M03'
checkout TEST-env
merge MOVY tag:'MOVY v1.0'
</pre>

View File

@@ -0,0 +1,251 @@
# Install monitoring stack
The **_⚙OPS_** team wants to have a real monitoring stack for its clusters.
Let's deploy `Prometheus` and `Grafana` onto the clusters.
Note:
---
## Creating `Github` source in Flux for monitoring components install repository
.lab[
```bash
k8s@shpod:~/fleet-config-using-flux-XXXXX$ mkdir -p clusters/CLOUDY/kube-prometheus-stack
k8s@shpod:~/fleet-config-using-flux-XXXXX$ flux create source git monitoring \
--namespace=monitoring \
--url=https://github.com/fluxcd/flux2-monitoring-example.git \
--branch=main --export > ./clusters/CLOUDY/kube-prometheus-stack/sync.yaml
```
]
---
### Creating `kustomization` in Flux for monitoring stack
.lab[
```bash
k8s@shpod:~/fleet-config-using-flux-XXXXX$ flux create kustomization monitoring \
--namespace=monitoring \
--source=GitRepository/monitoring \
--path="./monitoring/controllers/kube-prometheus-stack/" \
--export >> ./clusters/CLOUDY/kube-prometheus-stack/sync.yaml
```
]
---
### Install Flux Grafana dashboards
.lab[
```bash
k8s@shpod:~/fleet-config-using-flux-XXXXX$ flux create kustomization dashboards \
--namespace=monitoring \
--source=GitRepository/monitoring \
--path="./monitoring/configs/" \
--export >> ./clusters/CLOUDY/kube-prometheus-stack/sync.yaml
```
]
---
class: pic
![Running Mario](images/M6-running-Mario.gif)
---
## Flux repository synchro is broken😅
It seems that `Flux` on **_☁CLOUDY_** cluster is not able to authenticate with `ssh` on its `Github` config repository!
What happened?
When we install `Flux` on **_🤘METAL_** cluster, it generates a new `ssh` keypair and override the one used by **_☁CLOUDY_** among the "deployment keys" of the `Github` repository.
⚠️ Beware of flux bootstrap command!
We have to
- generate a new keypair (or reuse an already existing one)
- add the private key to the Flux-dedicated secrets in **_☁CLOUDY_** cluster
- add it to the "deployment keys" of the `Github` repository
---
### the command
.lab[
- `Flux` _CLI_ helps to recreate the secret holding the `ssh` **private** key.
```bash
k8s@shpod:~$ flux create secret git flux-system \
--url=ssh://git@github.com/container-training-fleet/fleet-config-using-flux-XXXXX \
--private-key-file=/home/k8s/.ssh/id_ed25519
```
- copy the **public** key into the deployment keys of the `Github` repository
]
---
class: pic
![Running Mario](images/M6-running-Mario.gif)
---
## Access the Grafana dashboard
.lab[
- Get the `Host` and `IP` address to request
```bash
k8s@shpod:~$ kubectl -n monitoring get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
grafana nginx grafana.test.metal.mybestdomain.com 62.210.39.83 80 6m30s
```
- Get the `Grafana` admin password
```bash
k8s@shpod:~$ k get secret kube-prometheus-stack-grafana -n monitoring \
-o jsonpath='{.data.admin-password}' | base64 -d
```
]
## And browse…
class: pic
![Grafana dashboard screenshot](images/M6-grafana-dashboard.png)
---
### 🗺️ Where are we in our scenario?
<pre class="mermaid">
%%{init:
{
"theme": "default",
"gitGraph": {
"mainBranchName": "OPS",
"mainBranchOrder": 0
}
}
}%%
gitGraph
commit id:"0" tag:"start"
branch ROCKY order:4
branch MOVY order:5
branch YouRHere order:6
checkout OPS
commit id:'Flux install on CLOUDY cluster' tag:'T01'
branch TEST-env order:1
commit id:'FLUX install on TEST' tag:'T02' type: HIGHLIGHT
checkout OPS
commit id:'Flux config. for TEST tenant' tag:'T03'
commit id:'namespace isolation by RBAC'
checkout TEST-env
merge OPS id:'ROCKY tenant creation' tag:'T04'
checkout OPS
commit id:'ROCKY deploy. config.' tag:'R01'
checkout TEST-env
merge OPS id:'TEST ready to deploy ROCKY' type: HIGHLIGHT tag:'R02'
checkout ROCKY
commit id:'ROCKY' tag:'v1.0.0'
checkout TEST-env
merge ROCKY tag:'ROCKY v1.0.0'
checkout OPS
commit id:'Ingress-controller config.' tag:'T05'
checkout TEST-env
merge OPS id:'Ingress-controller install' type: HIGHLIGHT tag:'T06'
checkout OPS
commit id:'ROCKY patch for ingress config.' tag:'R03'
checkout TEST-env
merge OPS id:'ingress config. for ROCKY app'
checkout ROCKY
commit id:'blue color' tag:'v1.0.1'
checkout TEST-env
merge ROCKY tag:'ROCKY v1.0.1'
checkout ROCKY
commit id:'pink color' tag:'v1.0.2'
checkout TEST-env
merge ROCKY tag:'ROCKY v1.0.2'
checkout OPS
commit id:'FLUX config for MOVY deployment' tag:'M01'
checkout TEST-env
merge OPS id:'FLUX ready to deploy MOVY' type: HIGHLIGHT tag:'M02'
checkout MOVY
commit id:'MOVY' tag:'v1.0.3'
checkout TEST-env
merge MOVY tag:'MOVY v1.0.3' type: REVERSE
checkout OPS
commit id:'Network policies'
checkout TEST-env
merge OPS type: HIGHLIGHT tag:'T07'
checkout OPS
commit id:'k0s install on METAL cluster' tag:'K01'
commit id:'Flux config. for METAL cluster' tag:'K02'
branch METAL_TEST-PROD order:3
commit id:'ROCKY/MOVY tenants on METAL' type: HIGHLIGHT
checkout OPS
commit id:'Flux config. for OpenEBS' tag:'K03'
checkout METAL_TEST-PROD
merge OPS id:'openEBS on METAL' type: HIGHLIGHT
checkout OPS
commit id:'Prometheus install'
checkout TEST-env
merge OPS type: HIGHLIGHT
checkout YouRHere
commit id:'x'
checkout OPS
merge YouRHere id:'YOU ARE HERE'
checkout OPS
commit id:'Kyverno install'
commit id:'Kyverno rules'
checkout TEST-env
merge OPS type: HIGHLIGHT
checkout OPS
commit id:'Flux config. for PROD tenant' tag:'P01'
branch PROD-env order:2
commit id:'ROCKY tenant on PROD'
checkout OPS
commit id:'ROCKY patch for PROD' tag:'R04'
checkout PROD-env
merge OPS id:'PROD ready to deploy ROCKY' type: HIGHLIGHT
checkout PROD-env
merge ROCKY tag:'ROCKY v1.0.2'
checkout MOVY
commit id:'MOVY HELM chart' tag:'M03'
checkout TEST-env
merge MOVY tag:'MOVY v1.0'
</pre>

View File

@@ -17,10 +17,12 @@ exclude:
content:
- k8s/M6-START-a-company-scenario.md
- k8S/M6-T02-flux-install.md
- k8s/M6-T02-flux-install.md
- k8s/M6-T03-installing-tenants.md
- k8s/M6-R01-flux_configure-ROCKY-deployment.md
- k8s/M6-T05-ingress-config.md
- k8s/M6-M01-adding-MOVY-tenant.md
- k8s/M6-K01-METAL-install.md
- k8s/M6-K03-openebs-install.md
- k8s/M6-monitoring-stack-install.md
- k8s/M6-kyverno-install.md