From c1067967510de65c1a60158d2db8466eaa7ad64f Mon Sep 17 00:00:00 2001 From: stefanprodan Date: Sun, 16 Jun 2019 10:31:25 +0300 Subject: [PATCH] Add A/B testing to FAQ --- README.md | 1 + docs/gitbook/SUMMARY.md | 1 + docs/gitbook/faq.md | 64 ++++++++++++++++++++++++++++++----------- 3 files changed, 50 insertions(+), 16 deletions(-) diff --git a/README.md b/README.md index f00a7baf..087f6d3d 100644 --- a/README.md +++ b/README.md @@ -35,6 +35,7 @@ Flagger documentation can be found at [docs.flagger.app](https://docs.flagger.ap * [Custom metrics](https://docs.flagger.app/how-it-works#custom-metrics) * [Webhooks](https://docs.flagger.app/how-it-works#webhooks) * [Load testing](https://docs.flagger.app/how-it-works#load-testing) + * [FAQ](https://docs.flagger.app/faq) * Usage * [Istio canary deployments](https://docs.flagger.app/usage/progressive-delivery) * [Istio A/B testing](https://docs.flagger.app/usage/ab-testing) diff --git a/docs/gitbook/SUMMARY.md b/docs/gitbook/SUMMARY.md index 346d7c4d..4a674a2d 100644 --- a/docs/gitbook/SUMMARY.md +++ b/docs/gitbook/SUMMARY.md @@ -2,6 +2,7 @@ * [Introduction](README.md) * [How it works](how-it-works.md) +* [FAQ](faq.md) ## Install diff --git a/docs/gitbook/faq.md b/docs/gitbook/faq.md index bc2e1b8a..b0eb69dc 100644 --- a/docs/gitbook/faq.md +++ b/docs/gitbook/faq.md @@ -1,21 +1,53 @@ # Frequently asked questions -**Can Flagger be part of my integration tests?** -> Yes, Flagger supports webhooks to do integration testing. +### A/B Testing -**What if I only want to target beta testers?** -> That's a feature in Flagger, not in App Mesh. It's on the App Mesh roadmap. +When should I use A/B testing instead of progressive traffic shifting? -**When do I use A/B testing when Canary?** -> One advantage of using A/B testing is that each version remains separated and routes aren't mixed. -> -> Using a Canary deployment can lead to behaviour like this one observed by a -> user: -> -> [..] during a canary deployment of our nodejs app, the version that is being served <50% traffic reports mime type mismatch errors in the browser (js as "text/html") -> When the deployment Passes/ Fails (doesn't really matter) the version that stays alive works as expected. If anyone has any tips or direction I would greatly appreciate it. Even if its as simple as I'm looking in the wrong place. Thanks in advance! -> -> The issue was that we were not maintaining session affinity while serving files for our frontend. Which resulted in any redirects or refreshes occasionally returning a mismatched app.*.js file (generated from vue) -> -> Read up on [A/B testing](https://docs.flagger.app/usage/ab-testing). +For frontend applications that require session affinity you should use HTTP headers or cookies match conditions +to ensure a set of users will stay on the same version for the whole duration of the canary analysis. +A/B testing is supported by Istio and NGINX only. + +Istio example: + +```yaml + canaryAnalysis: + # schedule interval (default 60s) + interval: 1m + # total number of iterations + iterations: 10 + # max number of failed iterations before rollback + threshold: 2 + # canary match condition + match: + - headers: + x-canary: + regex: ".*insider.*" + - headers: + cookie: + regex: "^(.*?;)?(canary=always)(;.*)?$" +``` + +NGINX example: + +```yaml + canaryAnalysis: + interval: 1m + threshold: 10 + iterations: 2 + match: + - headers: + x-canary: + exact: "insider" + - headers: + cookie: + exact: "canary" +``` + +The above configurations will route users with the x-canary header or canary cookie to the canary instance during analysis: + +```bash +curl -H 'X-Canary: insider' http://app.example.com +curl -b 'canary=always' http://app.example.com +```