diff --git a/docs/gitbook/README.md b/docs/gitbook/README.md index a5e983ee..4f6c4ed7 100644 --- a/docs/gitbook/README.md +++ b/docs/gitbook/README.md @@ -1,5 +1,5 @@ --- -description: Flagger is an Istio progressive delivery Kubernetes operator +description: Flagger is a progressive delivery Kubernetes operator --- # Introduction diff --git a/docs/gitbook/SUMMARY.md b/docs/gitbook/SUMMARY.md index 771a49be..4ce8974e 100644 --- a/docs/gitbook/SUMMARY.md +++ b/docs/gitbook/SUMMARY.md @@ -11,8 +11,9 @@ ## Usage -* [Canary Deployments](usage/progressive-delivery.md) -* [A/B Testing](usage/ab-testing.md) +* [Istio Canary Deployments](usage/progressive-delivery.md) +* [Istio A/B Testing](usage/ab-testing.md) +* [App Mesh Canary Deployments](usage/appmesh-progressive-delivery.md) * [Monitoring](usage/monitoring.md) * [Alerting](usage/alerting.md) diff --git a/docs/gitbook/usage/ab-testing.md b/docs/gitbook/usage/ab-testing.md index 01142969..fdb08c5d 100644 --- a/docs/gitbook/usage/ab-testing.md +++ b/docs/gitbook/usage/ab-testing.md @@ -1,6 +1,6 @@ -# A/B Testing +# Istio A/B Testing -This guide shows you how to automate A/B testing with Flagger. +This guide shows you how to automate A/B testing with Istio and Flagger. Besides weighted routing, Flagger can be configured to route traffic to the canary based on HTTP match conditions. In an A/B testing scenario, you'll be using HTTP headers or cookies to target a certain segment of your users. diff --git a/docs/gitbook/usage/appmesh-progressive-delivery.md b/docs/gitbook/usage/appmesh-progressive-delivery.md new file mode 100644 index 00000000..e83e1a26 --- /dev/null +++ b/docs/gitbook/usage/appmesh-progressive-delivery.md @@ -0,0 +1,206 @@ +# App Mesh Canary Deployments + +This guide shows you how to use App Mesh and Flagger to automate canary deployments. + +Create a mesh called `global` in the `appmesh-system` namespace: + +```bash +export REPO=https://raw.githubusercontent.com/weaveworks/flagger/master + +kubectl apply -f ${REPO}/artifacts/appmesh/global-mesh.yaml +``` + +Create a test namespace with App Mesh sidecar injection enabled: + +```bash +kubectl apply -f ${REPO}/artifacts/namespaces/test.yaml +``` + +Create a deployment and a horizontal pod autoscaler: + +```bash +kubectl apply -f ${REPO}/artifacts/appmesh/deployment.yaml +kubectl apply -f ${REPO}/artifacts/appmesh/hpa.yaml +``` + +Deploy the load testing service to generate traffic during the canary analysis: + +```bash +helm upgrade -i flagger-loadtester flagger/loadtester \ +--namespace=test \ +--set meshName=global.appmesh-system \ +--set backends[0]=frontend.test +``` + +Create a canary custom resource: + +```yaml +apiVersion: flagger.app/v1alpha3 +kind: Canary +metadata: + name: podinfo + namespace: test +spec: + # deployment reference + targetRef: + apiVersion: apps/v1 + kind: Deployment + name: podinfo + # the maximum time in seconds for the canary deployment + # to make progress before it is rollback (default 600s) + progressDeadlineSeconds: 60 + # HPA reference (optional) + autoscalerRef: + apiVersion: autoscaling/v2beta1 + kind: HorizontalPodAutoscaler + name: podinfo + service: + # container port + port: 9898 + # App Mesh reference + meshName: global.appmesh-system + # define the canary analysis timing and KPIs + canaryAnalysis: + # schedule interval (default 60s) + interval: 10s + # max number of failed metric checks before rollback + threshold: 10 + # max traffic percentage routed to canary + # percentage (0-100) + maxWeight: 50 + # canary increment step + # percentage (0-100) + stepWeight: 5 + # App Mesh Prometheus checks + metrics: + - name: envoy_cluster_upstream_rq + # minimum req success rate (non 5xx responses) + # percentage (0-100) + threshold: 99 + interval: 1m + # external checks (optional) + webhooks: + - name: load-test + url: http://flagger-loadtester.test/ + timeout: 5s + metadata: + cmd: "hey -z 1m -q 10 -c 2 http://podinfo.test:9898/" +``` + +Save the above resource as podinfo-canary.yaml and then apply it: + +```bash +kubectl apply -f ./podinfo-canary.yaml +``` + +After a couple of seconds Flagger will create the canary objects: + +```bash +# applied +deployment.apps/podinfo +horizontalpodautoscaler.autoscaling/podinfo +canary.flagger.app/podinfo + +# generated Kubernetes objects +deployment.apps/podinfo-primary +horizontalpodautoscaler.autoscaling/podinfo-primary +service/podinfo +service/podinfo-canary +service/podinfo-primary + +# generated App Mesh objects +virtualnode.appmesh.k8s.aws/podinfo +virtualnode.appmesh.k8s.aws/podinfo-canary +virtualnode.appmesh.k8s.aws/podinfo-primary +virtualservice.appmesh.k8s.aws/podinfo.test +``` + +Trigger a canary deployment by updating the container image: + +```bash +kubectl -n test set image deployment/podinfo \ +podinfod=quay.io/stefanprodan/podinfo:1.4.1 +``` + +Flagger detects that the deployment revision changed and starts a new rollout: + +```text +kubectl -n test describe canary/podinfo + +Status: + Canary Weight: 0 + Failed Checks: 0 + Phase: Succeeded +Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal Synced 3m flagger New revision detected podinfo.test + Normal Synced 3m flagger Scaling up podinfo.test + Warning Synced 3m flagger Waiting for podinfo.test rollout to finish: 0 of 1 updated replicas are available + Normal Synced 3m flagger Advance podinfo.test canary weight 5 + Normal Synced 3m flagger Advance podinfo.test canary weight 10 + Normal Synced 3m flagger Advance podinfo.test canary weight 15 + Normal Synced 2m flagger Advance podinfo.test canary weight 20 + Normal Synced 2m flagger Advance podinfo.test canary weight 25 + Normal Synced 1m flagger Advance podinfo.test canary weight 30 + Normal Synced 1m flagger Advance podinfo.test canary weight 35 + Normal Synced 55s flagger Advance podinfo.test canary weight 40 + Normal Synced 45s flagger Advance podinfo.test canary weight 45 + Normal Synced 35s flagger Advance podinfo.test canary weight 50 + Normal Synced 25s flagger Copying podinfo.test template spec to podinfo-primary.test + Warning Synced 15s flagger Waiting for podinfo-primary.test rollout to finish: 1 of 2 updated replicas are available + Normal Synced 5s flagger Promotion completed! Scaling down podinfo.test +``` + +**Note** that if you apply new changes to the deployment during the canary analysis, Flagger will restart the analysis. + +You can monitor all canaries with: + +```bash +watch kubectl get canaries --all-namespaces + +NAMESPACE NAME STATUS WEIGHT LASTTRANSITIONTIME +test podinfo Progressing 15 2019-03-16T14:05:07Z +prod frontend Succeeded 0 2019-03-15T16:15:07Z +prod backend Failed 0 2019-03-14T17:05:07Z +``` + +During the canary analysis you can generate HTTP 500 errors to test if Flagger pauses the rollout. + +Exec into the load tester pod with: + +```bash +kubectl -n test exec -it flagger-loadtester-xx-xx sh +``` + +Generate HTTP 500 errors: + +```bash +hey -z 1m -c 5 -q 5 http://podinfo.test:9898/status/500 +``` + +When the number of failed checks reaches the canary analysis threshold, the traffic is routed back to the primary, +the canary is scaled to zero and the rollout is marked as failed. + +```text +kubectl -n test describe canary/podinfo + +Status: + Canary Weight: 0 + Failed Checks: 10 + Phase: Failed +Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal Synced 3m flagger Starting canary deployment for podinfo.test + Normal Synced 3m flagger Advance podinfo.test canary weight 5 + Normal Synced 3m flagger Advance podinfo.test canary weight 10 + Normal Synced 3m flagger Advance podinfo.test canary weight 15 + Normal Synced 3m flagger Halt podinfo.test advancement success rate 69.17% < 99% + Normal Synced 2m flagger Halt podinfo.test advancement success rate 61.39% < 99% + Normal Synced 2m flagger Halt podinfo.test advancement success rate 55.06% < 99% + Normal Synced 2m flagger Halt podinfo.test advancement success rate 47.00% < 99% + Normal Synced 2m flagger (combined from similar events): Halt podinfo.test advancement success rate 38.08% < 99% + Warning Synced 1m flagger Rolling back podinfo.test failed checks threshold reached 10 + Warning Synced 1m flagger Canary failed! Scaling down podinfo.test +``` diff --git a/docs/gitbook/usage/progressive-delivery.md b/docs/gitbook/usage/progressive-delivery.md index 9ac4e309..f0660e63 100644 --- a/docs/gitbook/usage/progressive-delivery.md +++ b/docs/gitbook/usage/progressive-delivery.md @@ -1,4 +1,4 @@ -# Canary Deployments +# Istio Canary Deployments This guide shows you how to use Istio and Flagger to automate canary deployments.