From df103fb2570ff3886d38359919d2761f7690fd97 Mon Sep 17 00:00:00 2001 From: Cosmin Mogos Date: Wed, 27 May 2020 18:45:43 +0200 Subject: [PATCH 1/4] Add example RBAC for `helm test` --- docs/gitbook/usage/webhooks.md | 51 ++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/docs/gitbook/usage/webhooks.md b/docs/gitbook/usage/webhooks.md index f99a0905..9139e2c2 100644 --- a/docs/gitbook/usage/webhooks.md +++ b/docs/gitbook/usage/webhooks.md @@ -276,6 +276,9 @@ If you are using Helm v3, you'll have to create a dedicated service account and cmd: "test {{ .Release.Name }} --timeout 3m -n {{ .Release.Namespace }}" ``` +If the test hangs or logs error messages hinting to insufficient permissions it can be related to RBAC, +check the [Troubleshooting](#Troubleshooting) section for an example configuration. + As an alternative to Helm you can use the [Bash Automated Testing System](https://github.com/bats-core/bats-core) to run your tests. ```yaml @@ -423,3 +426,51 @@ curl -d '{"name": "podinfo","namespace":"test"}' http://localhost:8080/rollback/ ``` If you have notifications enabled, Flagger will post a message to Slack or MS Teams if a canary has been rolled back. + +### Troubleshooting + +#### Manually check if helm test is running + +To debug in depth any issues with helm tests, you can execute commands on the flagger-loadtester pod. +```bash +kubectl -n linkerd exec -it flagger-loadtester -- bash +helmv3 test .... +``` + +#### Helm tests hang during canary deployment + +If test execution hangs or displays insufficient permissions, check your RBAC settings. + +```yaml +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: helm-smoke-tester +rules: + - apiGroups: [""] + resources: ["secrets"] + verbs: ["get", "watch", "list", "update"] + # choose the permission based on helm test type (Pod or Job) + - apiGroups: [""] + resources: ["pods", "pods/log"] + verbs: ["create", "list", "delete", "watch"] + - apiGroups: ["batch"] + resources: ["jobs", "jobs/log"] + verbs: ["create", "list", "delete", "watch"] +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: helm-smoke-tester + # Don't forget to update accordingly + namespace: namespace-of-the-tested-release +subjects: + - kind: User + name: system:serviceaccount:linkerd:default + apiGroup: rbac.authorization.k8s.io +roleRef: + kind: ClusterRole + name: helm-smoke-tester + apiGroup: rbac.authorization.k8s.io +``` \ No newline at end of file From ada9288f8843a5e9bf048c6d5b8c35193ef26a51 Mon Sep 17 00:00:00 2001 From: Cosmin Mogos <35903525+cosmin-mogos@users.noreply.github.com> Date: Sat, 30 May 2020 13:57:57 +0200 Subject: [PATCH 2/4] Update docs/gitbook/usage/webhooks.md Co-authored-by: Stefan Prodan --- docs/gitbook/usage/webhooks.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/gitbook/usage/webhooks.md b/docs/gitbook/usage/webhooks.md index 9139e2c2..32cdccde 100644 --- a/docs/gitbook/usage/webhooks.md +++ b/docs/gitbook/usage/webhooks.md @@ -433,7 +433,7 @@ If you have notifications enabled, Flagger will post a message to Slack or MS Te To debug in depth any issues with helm tests, you can execute commands on the flagger-loadtester pod. ```bash -kubectl -n linkerd exec -it flagger-loadtester -- bash +kubectl exec -it deploy/flagger-loadtester -- bash helmv3 test .... ``` @@ -473,4 +473,4 @@ roleRef: kind: ClusterRole name: helm-smoke-tester apiGroup: rbac.authorization.k8s.io -``` \ No newline at end of file +``` From df50c32c09aaeec6eafc2089fc0cf74cf7e39829 Mon Sep 17 00:00:00 2001 From: Cosmin Mogos <35903525+cosmin-mogos@users.noreply.github.com> Date: Sat, 30 May 2020 13:58:25 +0200 Subject: [PATCH 3/4] Update docs/gitbook/usage/webhooks.md Co-authored-by: Stefan Prodan --- docs/gitbook/usage/webhooks.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/gitbook/usage/webhooks.md b/docs/gitbook/usage/webhooks.md index 32cdccde..5edbe510 100644 --- a/docs/gitbook/usage/webhooks.md +++ b/docs/gitbook/usage/webhooks.md @@ -434,7 +434,7 @@ If you have notifications enabled, Flagger will post a message to Slack or MS Te To debug in depth any issues with helm tests, you can execute commands on the flagger-loadtester pod. ```bash kubectl exec -it deploy/flagger-loadtester -- bash -helmv3 test .... +helmv3 test -n ``` #### Helm tests hang during canary deployment From fe58b32d9b4eb06c158d42649fe5fe9f384dbe5c Mon Sep 17 00:00:00 2001 From: Cosmin Mogos <35903525+cosmin-mogos@users.noreply.github.com> Date: Sat, 30 May 2020 18:07:40 +0200 Subject: [PATCH 4/4] Add --debug to helm command --- docs/gitbook/usage/webhooks.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/gitbook/usage/webhooks.md b/docs/gitbook/usage/webhooks.md index 5edbe510..b34f9cae 100644 --- a/docs/gitbook/usage/webhooks.md +++ b/docs/gitbook/usage/webhooks.md @@ -434,7 +434,7 @@ If you have notifications enabled, Flagger will post a message to Slack or MS Te To debug in depth any issues with helm tests, you can execute commands on the flagger-loadtester pod. ```bash kubectl exec -it deploy/flagger-loadtester -- bash -helmv3 test -n +helmv3 test -n --debug ``` #### Helm tests hang during canary deployment