mirror of
https://github.com/fluxcd/flagger.git
synced 2026-04-15 06:57:34 +00:00
193 lines
30 KiB
HTML
193 lines
30 KiB
HTML
<!DOCTYPE html>
|
||
<html lang="en-US">
|
||
<head>
|
||
<meta charset="utf-8">
|
||
<meta name="viewport" content="width=device-width,initial-scale=1">
|
||
<title>SMI Istio Canary Deployments | Flagger</title>
|
||
<meta name="description" content="Progressive Delivery operator for Kubernetes (Canary, A/B Testing and Blue/Green deployments)">
|
||
<link rel="icon" href="/favicon.png">
|
||
<link rel="stylesheet" href="/website.css">
|
||
<meta name="keywords" content="gitops kubernetes flagger istio linkerd appmesh contour gloo nginx">
|
||
<meta name="twitter:card" content="summary_large_image">
|
||
<meta name="twitter:title" content="Flagger">
|
||
<meta name="twitter:description" content="Progressive delivery Kubernetes operator (Canary, A/B Testing and Blue/Green deployments)">
|
||
<meta name="twitter:image:src" content="https://flagger.app/flagger-overview.png">
|
||
|
||
<link rel="preload" href="/assets/css/0.styles.a0ada997.css" as="style"><link rel="preload" href="/assets/js/app.9b46ae46.js" as="script"><link rel="preload" href="/assets/js/2.ba5bf77c.js" as="script"><link rel="preload" href="/assets/js/21.e1068843.js" as="script"><link rel="prefetch" href="/assets/js/10.bf6f38d4.js"><link rel="prefetch" href="/assets/js/11.3af38cfe.js"><link rel="prefetch" href="/assets/js/12.eca4b89c.js"><link rel="prefetch" href="/assets/js/13.9104349a.js"><link rel="prefetch" href="/assets/js/14.a4f3d810.js"><link rel="prefetch" href="/assets/js/15.aada2caa.js"><link rel="prefetch" href="/assets/js/16.3012a10b.js"><link rel="prefetch" href="/assets/js/17.a98b6aa9.js"><link rel="prefetch" href="/assets/js/18.e5188926.js"><link rel="prefetch" href="/assets/js/19.bb9498f7.js"><link rel="prefetch" href="/assets/js/20.2b1bebf5.js"><link rel="prefetch" href="/assets/js/22.1c5d14b9.js"><link rel="prefetch" href="/assets/js/23.b3dcc355.js"><link rel="prefetch" href="/assets/js/24.4f54d289.js"><link rel="prefetch" href="/assets/js/25.914acd1d.js"><link rel="prefetch" href="/assets/js/26.b2d4f402.js"><link rel="prefetch" href="/assets/js/27.ea9f1f9c.js"><link rel="prefetch" href="/assets/js/28.deae273d.js"><link rel="prefetch" href="/assets/js/29.ce4011f4.js"><link rel="prefetch" href="/assets/js/3.8b254d5d.js"><link rel="prefetch" href="/assets/js/30.edda19d9.js"><link rel="prefetch" href="/assets/js/31.fbfb336c.js"><link rel="prefetch" href="/assets/js/32.591c3f2b.js"><link rel="prefetch" href="/assets/js/33.c456709f.js"><link rel="prefetch" href="/assets/js/34.5a071e36.js"><link rel="prefetch" href="/assets/js/35.fc27c09e.js"><link rel="prefetch" href="/assets/js/36.144098aa.js"><link rel="prefetch" href="/assets/js/4.26a3069c.js"><link rel="prefetch" href="/assets/js/5.0c32bdf9.js"><link rel="prefetch" href="/assets/js/6.ee5ffca3.js"><link rel="prefetch" href="/assets/js/7.6bf27a87.js"><link rel="prefetch" href="/assets/js/8.8b72a8d3.js"><link rel="prefetch" href="/assets/js/9.7919fc1d.js">
|
||
<link rel="stylesheet" href="/assets/css/0.styles.a0ada997.css">
|
||
</head>
|
||
<body>
|
||
<div id="app" data-server-rendered="true"><div class="theme-container"><header class="navbar"><div class="sidebar-button"><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" role="img" viewBox="0 0 448 512" class="icon"><path fill="currentColor" d="M436 124H12c-6.627 0-12-5.373-12-12V80c0-6.627 5.373-12 12-12h424c6.627 0 12 5.373 12 12v32c0 6.627-5.373 12-12 12zm0 160H12c-6.627 0-12-5.373-12-12v-32c0-6.627 5.373-12 12-12h424c6.627 0 12 5.373 12 12v32c0 6.627-5.373 12-12 12zm0 160H12c-6.627 0-12-5.373-12-12v-32c0-6.627 5.373-12 12-12h424c6.627 0 12 5.373 12 12v32c0 6.627-5.373 12-12 12z"></path></svg></div> <a href="/" class="home-link router-link-active"><!----> <span class="site-name">Flagger</span></a> <div class="links"><div class="search-box"><input aria-label="Search" autocomplete="off" spellcheck="false" value=""> <!----></div> <nav class="nav-links can-hide"><div class="nav-item"><a href="https://docs.flagger.app" target="_blank" rel="noopener noreferrer" class="nav-link external">
|
||
Docs
|
||
<svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg></a></div><div class="nav-item"><a href="https://github.com/weaveworks/flagger/blob/master/CHANGELOG.md" target="_blank" rel="noopener noreferrer" class="nav-link external">
|
||
Changelog
|
||
<svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg></a></div> <a href="https://github.com/weaveworks/flagger" target="_blank" rel="noopener noreferrer" class="repo-link">
|
||
GitHub
|
||
<svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg></a></nav></div></header> <div class="sidebar-mask"></div> <aside class="sidebar"><nav class="nav-links"><div class="nav-item"><a href="https://docs.flagger.app" target="_blank" rel="noopener noreferrer" class="nav-link external">
|
||
Docs
|
||
<svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg></a></div><div class="nav-item"><a href="https://github.com/weaveworks/flagger/blob/master/CHANGELOG.md" target="_blank" rel="noopener noreferrer" class="nav-link external">
|
||
Changelog
|
||
<svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg></a></div> <a href="https://github.com/weaveworks/flagger" target="_blank" rel="noopener noreferrer" class="repo-link">
|
||
GitHub
|
||
<svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg></a></nav> <ul class="sidebar-links"><li><a href="/" class="sidebar-link">Home</a></li><li><section class="sidebar-group depth-0"><a href="/intro/" class="sidebar-heading clickable"><span>Introduction</span> <!----></a> <ul class="sidebar-links sidebar-group-items"><li><a href="/intro/" class="sidebar-link">Get Started</a></li><li><a href="/intro/faq.html" class="sidebar-link">FAQ</a></li></ul></section></li><li><section class="sidebar-group depth-0"><a href="/install/flagger-install-on-kubernetes" class="sidebar-heading clickable"><span>Install</span> <!----></a> <ul class="sidebar-links sidebar-group-items"><li><a href="/install/flagger-install-on-kubernetes.html" class="sidebar-link">On Kubernetes</a></li><li><a href="/install/flagger-install-on-google-cloud.html" class="sidebar-link">On GKE Istio</a></li><li><a href="/install/flagger-install-on-eks-appmesh.html" class="sidebar-link">On EKS App Mesh</a></li></ul></section></li><li><section class="sidebar-group depth-0"><a href="/usage/how-it-works" class="sidebar-heading clickable"><span>Usage</span> <!----></a> <ul class="sidebar-links sidebar-group-items"><li><a href="/usage/how-it-works.html" class="sidebar-link">How it works</a></li><li><a href="/usage/deployment-strategies.html" class="sidebar-link">Deployment Strategies</a></li><li><a href="/usage/metrics.html" class="sidebar-link">Metrics Analysis</a></li><li><a href="/usage/webhooks.html" class="sidebar-link">Webhooks</a></li><li><a href="/usage/alerting.html" class="sidebar-link">Alerting</a></li><li><a href="/usage/monitoring.html" class="sidebar-link">Monitoring</a></li></ul></section></li><li><section class="sidebar-group depth-0"><a href="/tutorials/istio-progressive-delivery" class="sidebar-heading clickable"><span>Tutorials</span> <!----></a> <ul class="sidebar-links sidebar-group-items"><li><a href="/tutorials/istio-progressive-delivery.html" class="sidebar-link">Istio Canaries</a></li><li><a href="/tutorials/istio-ab-testing.html" class="sidebar-link">Istio A/B Testing</a></li><li><a href="/tutorials/linkerd-progressive-delivery.html" class="sidebar-link">Linkerd Canaries</a></li><li><a href="/tutorials/appmesh-progressive-delivery.html" class="sidebar-link">App Mesh Canaries</a></li><li><a href="/tutorials/nginx-progressive-delivery.html" class="sidebar-link">NGINX Ingress Canaries</a></li><li><a href="/tutorials/gloo-progressive-delivery.html" class="sidebar-link">Gloo Canaries</a></li><li><a href="/tutorials/contour-progressive-delivery.html" class="sidebar-link">Contour Canaries</a></li><li><a href="/tutorials/kubernetes-blue-green.html" class="sidebar-link">Kubernetes Blue/Green</a></li><li><a href="/tutorials/canary-helm-gitops.html" class="sidebar-link">Canaries with Helm charts and GitOps</a></li></ul></section></li><li><section class="sidebar-group depth-0"><a href="/dev/dev-guide" class="sidebar-heading clickable"><span>Dev</span> <!----></a> <ul class="sidebar-links sidebar-group-items"><li><a href="/dev/dev-guide.html" class="sidebar-link">Development Guide</a></li><li><a href="/dev/release-guide.html" class="sidebar-link">Release Guide</a></li><li><a href="/dev/upgrade-guide.html" class="sidebar-link">Upgrade Guide</a></li></ul></section></li></ul> </aside> <main class="page"> <div class="theme-default-content content__default"><h1 id="smi-istio-canary-deployments"><a href="#smi-istio-canary-deployments" aria-hidden="true" class="header-anchor">#</a> SMI Istio Canary Deployments</h1> <p>This guide shows you how to use the SMI Istio adapter and Flagger to automate canary deployments.</p> <h2 id="prerequisites"><a href="#prerequisites" aria-hidden="true" class="header-anchor">#</a> Prerequisites</h2> <ul><li>Kubernetes > 1.13</li> <li>Istio > 1.0</li></ul> <h2 id="install-istio-smi-adapter"><a href="#install-istio-smi-adapter" aria-hidden="true" class="header-anchor">#</a> Install Istio SMI adapter</h2> <p>Install the SMI adapter:</p> <div class="language-bash extra-class"><pre class="language-bash"><code>kubectl apply -f https://raw.githubusercontent.com/deislabs/smi-adapter-istio/master/deploy/crds/crds.yaml
|
||
kubectl apply -f https://raw.githubusercontent.com/deislabs/smi-adapter-istio/master/deploy/operator-and-rbac.yaml
|
||
</code></pre></div><p>Create a generic Istio gateway to expose services outside the mesh on HTTP:</p> <div class="language-yaml extra-class"><pre class="language-yaml"><code><span class="token key atrule">apiVersion</span><span class="token punctuation">:</span> networking.istio.io/v1alpha3
|
||
<span class="token key atrule">kind</span><span class="token punctuation">:</span> Gateway
|
||
<span class="token key atrule">metadata</span><span class="token punctuation">:</span>
|
||
<span class="token key atrule">name</span><span class="token punctuation">:</span> public<span class="token punctuation">-</span>gateway
|
||
<span class="token key atrule">namespace</span><span class="token punctuation">:</span> istio<span class="token punctuation">-</span>system
|
||
<span class="token key atrule">spec</span><span class="token punctuation">:</span>
|
||
<span class="token key atrule">selector</span><span class="token punctuation">:</span>
|
||
<span class="token key atrule">istio</span><span class="token punctuation">:</span> ingressgateway
|
||
<span class="token key atrule">servers</span><span class="token punctuation">:</span>
|
||
<span class="token punctuation">-</span> <span class="token key atrule">port</span><span class="token punctuation">:</span>
|
||
<span class="token key atrule">number</span><span class="token punctuation">:</span> <span class="token number">80</span>
|
||
<span class="token key atrule">name</span><span class="token punctuation">:</span> http
|
||
<span class="token key atrule">protocol</span><span class="token punctuation">:</span> HTTP
|
||
<span class="token key atrule">hosts</span><span class="token punctuation">:</span>
|
||
<span class="token punctuation">-</span> <span class="token string">"*"</span>
|
||
</code></pre></div><p>Save the above resource as public-gateway.yaml and then apply it:</p> <div class="language-bash extra-class"><pre class="language-bash"><code>kubectl apply -f ./public-gateway.yaml
|
||
</code></pre></div><p>Find the Gateway load balancer IP and add a DNS record for it:</p> <div class="language-bash extra-class"><pre class="language-bash"><code>kubectl -n istio-system get svc/istio-ingressgateway -ojson <span class="token operator">|</span> jq -r .status.loadBalancer.ingress<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span>.ip
|
||
</code></pre></div><h2 id="install-flagger-and-grafana"><a href="#install-flagger-and-grafana" aria-hidden="true" class="header-anchor">#</a> Install Flagger and Grafana</h2> <p>Add Flagger Helm repository:</p> <div class="language-bash extra-class"><pre class="language-bash"><code>helm repo <span class="token function">add</span> flagger https://flagger.app
|
||
</code></pre></div><p>Deploy Flagger in the <em><strong>istio-system</strong></em> namespace:</p> <div class="language-bash extra-class"><pre class="language-bash"><code>helm upgrade -i flagger flagger/flagger <span class="token punctuation">\</span>
|
||
--namespace<span class="token operator">=</span>istio-system <span class="token punctuation">\</span>
|
||
--set <span class="token assign-left variable">meshProvider</span><span class="token operator">=</span>smi:istio
|
||
</code></pre></div><p>Flagger comes with a Grafana dashboard made for monitoring the canary deployments.</p> <p>Deploy Grafana in the <em><strong>istio-system</strong></em> namespace:</p> <div class="language-bash extra-class"><pre class="language-bash"><code>helm upgrade -i flagger-grafana flagger/grafana <span class="token punctuation">\</span>
|
||
--namespace<span class="token operator">=</span>istio-system <span class="token punctuation">\</span>
|
||
--set <span class="token assign-left variable">url</span><span class="token operator">=</span>http://prometheus.istio-system:9090
|
||
</code></pre></div><p>You can access Grafana using port forwarding:</p> <div class="language-bash extra-class"><pre class="language-bash"><code>kubectl -n istio-system port-forward svc/flagger-grafana <span class="token number">3000</span>:80
|
||
</code></pre></div><h2 id="workloads-bootstrap"><a href="#workloads-bootstrap" aria-hidden="true" class="header-anchor">#</a> Workloads bootstrap</h2> <p>Create a test namespace with Istio sidecar injection enabled:</p> <p>Create a test namespace and enable Linkerd proxy injection:</p> <div class="language-bash extra-class"><pre class="language-bash"><code>kubectl create ns <span class="token builtin class-name">test</span>
|
||
kubectl label namespace <span class="token builtin class-name">test</span> istio-injection<span class="token operator">=</span>enabled
|
||
</code></pre></div><p>Create a deployment and a horizontal pod autoscaler:</p> <div class="language-bash extra-class"><pre class="language-bash"><code>kubectl apply -k github.com/weaveworks/flagger//kustomize/podinfo
|
||
</code></pre></div><p>Deploy the load testing service to generate traffic during the canary analysis:</p> <div class="language-bash extra-class"><pre class="language-bash"><code>kubectl apply -k github.com/weaveworks/flagger//kustomize/tester
|
||
</code></pre></div><p>Create a canary custom resource (replace example.com with your own domain):</p> <div class="language-yaml extra-class"><pre class="language-yaml"><code><span class="token key atrule">apiVersion</span><span class="token punctuation">:</span> flagger.app/v1beta1
|
||
<span class="token key atrule">kind</span><span class="token punctuation">:</span> Canary
|
||
<span class="token key atrule">metadata</span><span class="token punctuation">:</span>
|
||
<span class="token key atrule">name</span><span class="token punctuation">:</span> podinfo
|
||
<span class="token key atrule">namespace</span><span class="token punctuation">:</span> test
|
||
<span class="token key atrule">spec</span><span class="token punctuation">:</span>
|
||
<span class="token comment"># deployment reference</span>
|
||
<span class="token key atrule">targetRef</span><span class="token punctuation">:</span>
|
||
<span class="token key atrule">apiVersion</span><span class="token punctuation">:</span> apps/v1
|
||
<span class="token key atrule">kind</span><span class="token punctuation">:</span> Deployment
|
||
<span class="token key atrule">name</span><span class="token punctuation">:</span> podinfo
|
||
<span class="token comment"># the maximum time in seconds for the canary deployment</span>
|
||
<span class="token comment"># to make progress before it is rollback (default 600s)</span>
|
||
<span class="token key atrule">progressDeadlineSeconds</span><span class="token punctuation">:</span> <span class="token number">60</span>
|
||
<span class="token comment"># HPA reference (optional)</span>
|
||
<span class="token key atrule">autoscalerRef</span><span class="token punctuation">:</span>
|
||
<span class="token key atrule">apiVersion</span><span class="token punctuation">:</span> autoscaling/v2beta1
|
||
<span class="token key atrule">kind</span><span class="token punctuation">:</span> HorizontalPodAutoscaler
|
||
<span class="token key atrule">name</span><span class="token punctuation">:</span> podinfo
|
||
<span class="token key atrule">service</span><span class="token punctuation">:</span>
|
||
<span class="token comment"># container port</span>
|
||
<span class="token key atrule">port</span><span class="token punctuation">:</span> <span class="token number">9898</span>
|
||
<span class="token comment"># Istio gateways (optional)</span>
|
||
<span class="token key atrule">gateways</span><span class="token punctuation">:</span>
|
||
<span class="token punctuation">-</span> public<span class="token punctuation">-</span>gateway.istio<span class="token punctuation">-</span>system.svc.cluster.local
|
||
<span class="token comment"># Istio virtual service host names (optional)</span>
|
||
<span class="token key atrule">hosts</span><span class="token punctuation">:</span>
|
||
<span class="token punctuation">-</span> app.example.com
|
||
<span class="token key atrule">analysis</span><span class="token punctuation">:</span>
|
||
<span class="token comment"># schedule interval (default 60s)</span>
|
||
<span class="token key atrule">interval</span><span class="token punctuation">:</span> 10s
|
||
<span class="token comment"># max number of failed metric checks before rollback</span>
|
||
<span class="token key atrule">threshold</span><span class="token punctuation">:</span> <span class="token number">5</span>
|
||
<span class="token comment"># max traffic percentage routed to canary</span>
|
||
<span class="token comment"># percentage (0-100)</span>
|
||
<span class="token key atrule">maxWeight</span><span class="token punctuation">:</span> <span class="token number">50</span>
|
||
<span class="token comment"># canary increment step</span>
|
||
<span class="token comment"># percentage (0-100)</span>
|
||
<span class="token key atrule">stepWeight</span><span class="token punctuation">:</span> <span class="token number">10</span>
|
||
<span class="token key atrule">metrics</span><span class="token punctuation">:</span>
|
||
<span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> request<span class="token punctuation">-</span>success<span class="token punctuation">-</span>rate
|
||
<span class="token comment"># minimum req success rate (non 5xx responses)</span>
|
||
<span class="token comment"># percentage (0-100)</span>
|
||
<span class="token key atrule">thresholdRange</span><span class="token punctuation">:</span>
|
||
<span class="token key atrule">min</span><span class="token punctuation">:</span> <span class="token number">99</span>
|
||
<span class="token key atrule">interval</span><span class="token punctuation">:</span> 1m
|
||
<span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> request<span class="token punctuation">-</span>duration
|
||
<span class="token comment"># maximum req duration P99</span>
|
||
<span class="token comment"># milliseconds</span>
|
||
<span class="token key atrule">thresholdRange</span><span class="token punctuation">:</span>
|
||
<span class="token key atrule">max</span><span class="token punctuation">:</span> <span class="token number">500</span>
|
||
<span class="token key atrule">interval</span><span class="token punctuation">:</span> 30s
|
||
<span class="token comment"># generate traffic during analysis</span>
|
||
<span class="token key atrule">webhooks</span><span class="token punctuation">:</span>
|
||
<span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> load<span class="token punctuation">-</span>test
|
||
<span class="token key atrule">url</span><span class="token punctuation">:</span> http<span class="token punctuation">:</span>//flagger<span class="token punctuation">-</span>loadtester.test/
|
||
<span class="token key atrule">timeout</span><span class="token punctuation">:</span> 5s
|
||
<span class="token key atrule">metadata</span><span class="token punctuation">:</span>
|
||
<span class="token key atrule">cmd</span><span class="token punctuation">:</span> <span class="token string">"hey -z 1m -q 10 -c 2 http://podinfo.test:9898/"</span>
|
||
</code></pre></div><p>Save the above resource as podinfo-canary.yaml and then apply it:</p> <div class="language-bash extra-class"><pre class="language-bash"><code>kubectl apply -f ./podinfo-canary.yaml
|
||
</code></pre></div><p>After a couple of seconds Flagger will create the canary objects:</p> <div class="language-bash extra-class"><pre class="language-bash"><code><span class="token comment"># applied </span>
|
||
deployment.apps/podinfo
|
||
horizontalpodautoscaler.autoscaling/podinfo
|
||
canary.flagger.app/podinfo
|
||
|
||
<span class="token comment"># generated </span>
|
||
deployment.apps/podinfo-primary
|
||
horizontalpodautoscaler.autoscaling/podinfo-primary
|
||
service/podinfo
|
||
service/podinfo-canary
|
||
service/podinfo-primary
|
||
trafficsplits.split.smi-spec.io/podinfo
|
||
</code></pre></div><h2 id="automated-canary-promotion"><a href="#automated-canary-promotion" aria-hidden="true" class="header-anchor">#</a> Automated canary promotion</h2> <p>Flagger implements a control loop that gradually shifts traffic to the canary while measuring key performance indicators like HTTP requests success rate, requests average duration and pod health. Based on analysis of the KPIs a canary is promoted or aborted, and the analysis result is published to Slack.</p> <p><img src="https://raw.githubusercontent.com/weaveworks/flagger/master/docs/diagrams/flagger-canary-steps.png" alt="Flagger Canary Stages"></p> <p>Trigger a canary deployment by updating the container image:</p> <div class="language-bash extra-class"><pre class="language-bash"><code>kubectl -n <span class="token builtin class-name">test</span> <span class="token builtin class-name">set</span> image deployment/podinfo <span class="token punctuation">\</span>
|
||
<span class="token assign-left variable">podinfod</span><span class="token operator">=</span>quay.io/stefanprodan/podinfo:3.1.1
|
||
</code></pre></div><p>Flagger detects that the deployment revision changed and starts a new rollout:</p> <div class="language-text extra-class"><pre class="language-text"><code>kubectl -n istio-system logs deployment/flagger -f | jq .msg
|
||
|
||
|
||
New revision detected podinfo.test
|
||
Scaling up podinfo.test
|
||
Waiting for podinfo.test rollout to finish: 0 of 1 updated replicas are available
|
||
Advance podinfo.test canary weight 5
|
||
Advance podinfo.test canary weight 10
|
||
Advance podinfo.test canary weight 15
|
||
Advance podinfo.test canary weight 20
|
||
Advance podinfo.test canary weight 25
|
||
Advance podinfo.test canary weight 30
|
||
Advance podinfo.test canary weight 35
|
||
Advance podinfo.test canary weight 40
|
||
Advance podinfo.test canary weight 45
|
||
Advance podinfo.test canary weight 50
|
||
Copying podinfo.test template spec to podinfo-primary.test
|
||
Waiting for podinfo-primary.test rollout to finish: 1 of 2 updated replicas are available
|
||
Promotion completed! Scaling down podinfo.test
|
||
</code></pre></div><p><strong>Note</strong> that if you apply new changes to the deployment during the canary analysis, Flagger will restart the analysis.</p> <p>During the analysis the canary’s progress can be monitored with Grafana. The Istio dashboard URL is <a href="http://localhost:3000/d/flagger-istio/istio-canary?refresh=10s&orgId=1&var-namespace=test&var-primary=podinfo-primary&var-canary=podinfo" target="_blank" rel="noopener noreferrer">http://localhost:3000/d/flagger-istio/istio-canary?refresh=10s&orgId=1&var-namespace=test&var-primary=podinfo-primary&var-canary=podinfo<svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg></a></p> <p>You can monitor all canaries with:</p> <div class="language-bash extra-class"><pre class="language-bash"><code><span class="token function">watch</span> kubectl get canaries --all-namespaces
|
||
|
||
NAMESPACE NAME STATUS WEIGHT LASTTRANSITIONTIME
|
||
<span class="token builtin class-name">test</span> podinfo Progressing <span class="token number">15</span> <span class="token number">2019</span>-05-16T14:05:07Z
|
||
prod frontend Succeeded <span class="token number">0</span> <span class="token number">2019</span>-05-15T16:15:07Z
|
||
prod backend Failed <span class="token number">0</span> <span class="token number">2019</span>-05-14T17:05:07Z
|
||
</code></pre></div><h2 id="automated-rollback"><a href="#automated-rollback" aria-hidden="true" class="header-anchor">#</a> Automated rollback</h2> <p>During the canary analysis you can generate HTTP 500 errors and high latency to test if Flagger pauses the rollout.</p> <p>Create a tester pod and exec into it:</p> <div class="language-bash extra-class"><pre class="language-bash"><code>kubectl -n <span class="token builtin class-name">test</span> run tester <span class="token punctuation">\</span>
|
||
--image<span class="token operator">=</span>quay.io/stefanprodan/podinfo:3.1.2 <span class="token punctuation">\</span>
|
||
-- ./podinfo --port<span class="token operator">=</span><span class="token number">9898</span>
|
||
|
||
kubectl -n <span class="token builtin class-name">test</span> <span class="token builtin class-name">exec</span> -it tester-xx-xx <span class="token function">sh</span>
|
||
</code></pre></div><p>Generate HTTP 500 errors:</p> <div class="language-bash extra-class"><pre class="language-bash"><code><span class="token function">watch</span> <span class="token function">curl</span> http://podinfo-canary:9898/status/500
|
||
</code></pre></div><p>Generate latency:</p> <div class="language-bash extra-class"><pre class="language-bash"><code><span class="token function">watch</span> <span class="token function">curl</span> http://podinfo-canary:9898/delay/1
|
||
</code></pre></div><p>When the number of failed checks reaches the canary analysis threshold, the traffic is routed back to the primary, the canary is scaled to zero and the rollout is marked as failed.</p> <div class="language-text extra-class"><pre class="language-text"><code>kubectl -n test describe canary/podinfo
|
||
|
||
Status:
|
||
Canary Weight: 0
|
||
Failed Checks: 10
|
||
Phase: Failed
|
||
Events:
|
||
Type Reason Age From Message
|
||
---- ------ ---- ---- -------
|
||
Normal Synced 3m flagger Starting canary deployment for podinfo.test
|
||
Normal Synced 3m flagger Advance podinfo.test canary weight 5
|
||
Normal Synced 3m flagger Advance podinfo.test canary weight 10
|
||
Normal Synced 3m flagger Advance podinfo.test canary weight 15
|
||
Normal Synced 3m flagger Halt podinfo.test advancement success rate 69.17% < 99%
|
||
Normal Synced 2m flagger Halt podinfo.test advancement success rate 61.39% < 99%
|
||
Normal Synced 2m flagger Halt podinfo.test advancement success rate 55.06% < 99%
|
||
Normal Synced 2m flagger Halt podinfo.test advancement success rate 47.00% < 99%
|
||
Normal Synced 2m flagger (combined from similar events): Halt podinfo.test advancement success rate 38.08% < 99%
|
||
Warning Synced 1m flagger Rolling back podinfo.test failed checks threshold reached 10
|
||
Warning Synced 1m flagger Canary failed! Scaling down podinfo.test
|
||
</code></pre></div></div> <footer class="page-edit"><!----> <!----></footer> <!----> </main></div><div class="global-ui"></div></div>
|
||
<script src="/assets/js/app.9b46ae46.js" defer></script><script src="/assets/js/2.ba5bf77c.js" defer></script><script src="/assets/js/21.e1068843.js" defer></script>
|
||
</body>
|
||
</html>
|