github/book-cicd-docker-kubernetes
1
0
Fork 0
mirror of https://github.com/semaphoreci/book-cicd-docker-kubernetes.git synced 2026-02-14 15:59:50 +00:00
Code Issues Packages Projects Releases Wiki Activity

update deployment steps for DO and DO cloud setup

Browse Source
This commit is contained in:
tomas f
2021-07-19 19:35:59 -03:00
parent 52bf3cd765
commit 30eb91fff1
2 changed files with 74 additions and 62 deletions
Download Patch File Download Diff File Expand all files Collapse all files

55
chapters/07-tutorial-clouds.md
View File

@@ -8,19 +8,20 @@ We’ll deploy the application in a three-node Kubernetes cluster. You can pick
#### 4.5.1 DigitalOcean Cluster
DigitalOcean provides a managed Kubernetes service but lacks a private Docker registry[^do-private-reg], so we’ll use Docker Hub for the images.
**DigitalOcean provides a managed Kubernetes service but lacks a private Docker registry[^do-private-reg], so we’ll use Docker Hub for the images.**
[^do-private-reg]: At the time of writing, DigitalOcean announced a beta for a private registry offering. For more information, consult the available documentation: _<https://www.digitalocean.com/docs/kubernetes/how-to/set-up-registry>_
- Sign up for a free account on [hub.docker.com](https://hub.docker.com).
- Create a public repository called “semaphore-demo-cicd-kubernetes”.
- **Sign up for a free account on [hub.docker.com](https://hub.docker.com).**
- **Create a public repository called “semaphore-demo-cicd-kubernetes”.**
To create the Kubernetes cluster:
- Sign up or log in to your account on [digitalocean.com](https://www.digitalocean.com).
- Create a *New Project*.
- Create a *Kubernetes* cluster: select the latest version and choose one of the available regions. Name your cluster “semaphore-demo-cicd-kubernetes”.
- Go to the *API* menu and generate a *Personal Access Token*.
- While DigitalOcean is working on the cluster, go to *API* menu and generate a *Personal Access Token* with Read & Write permissions.
- To finalize, go to *Container Registry* and create a starter repository. Set the repository name, which is globally unique, and remember it for the next chapter.
On Semaphore, store the DigitalOcean Access Token as a secret:
@@ -28,12 +29,12 @@ On Semaphore, store the DigitalOcean Access Token as a secret:
2. In the sidebar on the left-hand side, under *Configuration*, select *Secrets* and click on the *Create New Secret* button.
3. The name of the secret is “do-key”.
4. Add the `DO_ACCESS_TOKEN` variable and set its value with your personal token.
5. Click on *Save changes*.
5. Click on *Save Secret*.
Repeat the last steps to add the second secret, call it “dockerhub” and add the following variables:
**Repeat the last steps to add the second secret, call it “dockerhub” and add the following variables:**
- `DOCKER_USERNAME` for your DockerHub user name.
- `DOCKER_PASSWORD` with the corresponding password.
- **`DOCKER_USERNAME` for your DockerHub user name.**
- **`DOCKER_PASSWORD` with the corresponding password.**
#### 4.5.2 Google Cloud Cluster
@@ -41,19 +42,20 @@ Google Cloud calls its service *Kubernetes Engine*. To create the services:
- Sign up or log in to your Google Cloud account on [cloud.google.com](https://cloud.google.com).
- Create a *New Project*. In *Project ID* type “semaphore-demo-cicd-kubernetes”.
- Go to *Kubernetes Engine* \> *Clusters* and create a cluster. Select “Zonal” in *Location Type* and select one of the available zones.
- Go to *Kubernetes Engine* \> *Clusters* and enable the service. Create a public **autopilot** cluster in one of the available zones.
- Name your cluster “semaphore-demo-cicd-kubernetes”.
- Go to *IAM* \> *Service Accounts*.
- Generate an account with “Project Owner” permissions.
- Generate and download a JSON Access Key file.
- Generate an account Basic > Owner role.
- Click on the menu for the new roles, select *Manage Keys* > *Add Keys*.
- Generate and download a **JSON** Access Key file.
On Semaphore, create a secret for your Google Cloud Access Key file:
1. Log in to your organization on [id.semaphoreci.com](https://id.semaphoreci.com).
2. In the sidebar on the left-hand side, under *Cconfiguration*, select *Secrets* and click on the *Create New Secret* button.
2. Open your account menu and click on Settings. Go to *Secrets* > *New Secret*.
3. Name the secret “gcp-key”.
4. Add this file: `/home/semaphore/gcp-key.json` and upload the Google Cloud Access JSON from your computer.
5. Click on *Save changes*.
5. Click on *Save Secret*.
#### 4.5.3 AWS Cluster
@@ -100,7 +102,7 @@ On Semaphore, create a secret to store the AWS Secret Access Key and the kubecon
- `AWS_SECRET_ACCESS_KEY` has the AWS Access Secret Key string.
5. Add the following file:
- `/home/semaphore/aws-key.yml` and upload the Kubeconfig file created by eksctl earlier.
6. Click on *Save changes*.
6. Click on *Save Secret*.
### 4.6 Provisioning a Database
@@ -110,21 +112,23 @@ We’ll need a database to store data. For that, we’ll use a managed PostgreSQ
- Go to *Databases*.
- Create a PostgreSQL database. Select the same region where the cluster is running.
- In the *Connectivity* tab, whitelist the `0.0.0.0/0` network[^network-whitelist].
- Go to the *Users & Databases* tab and create a database called “demo” and a user named “demouser”.
- Once the database is ready, go to the *Users & Databases* tab and create a database called “demo” and a user named “demouser”.
- In the *Overview* tab, take note of the PostgreSQL IP address and port.
[^network-whitelist]: Later, when everything is working, you can restrict access to the Kubernetes nodes to increase security.
#### 4.6.2 Google Cloud Database
- Select *SQL* on the console menu.
- Create a new PostgreSQL database instance.
- Create a new **PostgreSQL** database instance.
- Select the same region and zone where the Kubernetes cluster is running.
- Enable the *Private IP* network.
- Go to the *Users* tab and create a new user called “demouser”.
- Go to the *Databases* tab and create a new DB called “demo”.
- In the *Overview* tab, take note of the database IP address and port.
- Open the *Customize your instance* section.
- Enable the *Private IP* network with the default options and an automatically allocated IP range.
- Create the instance.
Once the cloud database is running:
- Open the left-side menu and select *Users*. Create a new built-in user called “demouser”.
- Go to the *Databases* and create a new DB called “demo”.
- In the *Overview* tab (you can skip the getting started part), take note of the database IP address and port.
#### 4.6.3 AWS Database
@@ -143,10 +147,11 @@ The database secret is the same for all clouds. Create a secret to store the dat
2. On the main page, under *Configuration* select *Secrets* and click on the *Create New Secret* button.
3. The secret name is “db-params”.
4. Add the following variables:
- `DB_HOST` with the database hostname or IP.
- `DB_HOST` with the database hostname or private IP.
- `DB_PORT` points to the database port (default is 5432).
- `DB_SCHEMA` for AWS should be called “postgres”, for the other clouds its value should be “demo”.
- `DB_USER` for the database user.
- `DB_PASSWORD` with the password.
- `DB_SSL` should be “true” for DigitalOcean, it can be left empty for the rest.
5. Click on *Save changes*.
5. Click on *Save Secret*.

81
chapters/08-tutorial-deployment.md
View File

@@ -22,58 +22,59 @@ result = 'passed' and (branch = 'master' or tag =~ '^hotfix*')
![Automatic promotion](./figures/05-sem-canary-auto-promotion.png){ width=95% }
In the new pipeline, click on the first block. Let's call it “Push”. The push block takes the Docker image that we built earlier and uploads it to Docker Hub. The secrets and the login command will vary depending on the cloud of choice. For DigitalOcean, we’ll use Docker Hub as a repository:
In the new pipeline, click on the first block. Let's call it “Push”. The push block takes the Docker image that we built earlier and uploads it to Docker Hub. The secrets and the login command will vary depending on the cloud of choice.
Open the *Secrets* section and check the `dockerhub` secret.
Open the *Secrets* section and check the `do-key` secret.
Type the following commands in the job:
```bash
docker login \
-u $SEMAPHORE_REGISTRY_USERNAME \
-p $SEMAPHORE_REGISTRY_PASSWORD $SEMAPHORE_REGISTRY_URL
-p $SEMAPHORE_REGISTRY_PASSWORD \
$SEMAPHORE_REGISTRY_URL
docker pull \
$SEMAPHORE_REGISTRY_URL/demo:$SEMAPHORE_WORKFLOW_ID
echo "${DOCKER_PASSWORD}" | \
docker login -u "${DOCKER_USERNAME}" --password-stdin
docker tag \
$SEMAPHORE_REGISTRY_URL/demo:$SEMAPHORE_WORKFLOW_ID \
$DOCKER_USERNAME/demo:$SEMAPHORE_WORKFLOW_ID
registry.digitalocean.com/$REGISTRY_NAME/demo:$SEMAPHORE_WORKFLOW_ID
doctl auth init -t $DO_ACCESS_TOKEN
doctl registry login
docker push \
$DOCKER_USERNAME/demo:$SEMAPHORE_WORKFLOW_ID
registry.digitalocean.com/$REGISTRY_NAME/demo:$SEMAPHORE_WORKFLOW_ID
```
![Push block](./figures/05-sem-canary-push-block.png){ width=95% }
Create a new block called “Deploy” and enable secrets:
- `dockerhub` to communicate with Docker Hub;
- `db-params` to use the cloud database;
- `do-key` which is the cloud-specific access token.
Open the *Environment Variables* section and create a variable called `CLUSTER_NAME` with the DigitalOcean cluster name (`semaphore-demo-cicd-kubernetes`).
Open the *Environment Variables* section:
- Create a variable called `CLUSTER_NAME` with the DigitalOcean cluster name (`semaphore-demo-cicd-kubernetes`)
- Create a variable called `REGISTRY_NAME` with the name of the DigitalOcean container registry name.
To connect with the DigitalOcean cluster, we can use the official `doctl` tool, which comes preinstalled in Semaphore.
First, type these commands in the *prologue*:
Add the following commands to the *job*:
```bash
doctl auth init --access-token $DO_ACCESS_TOKEN
doctl kubernetes cluster kubeconfig save "${CLUSTER_NAME}"
checkout
```
Then, add the following commands to the *job*:
```bash
kubectl apply -f manifests/service.yml
./apply.sh manifests/deployment.yml addressbook-canary 1 \
$DOCKER_USERNAME/demo:$SEMAPHORE_WORKFLOW_ID
./apply.sh \
manifests/deployment.yml \
addressbook-canary 1 \
registry.digitalocean.com/$REGISTRY_NAME/demo:$SEMAPHORE_WORKFLOW_ID
if kubectl get deployment addressbook-stable; then \
kubectl scale --replicas=2 deployment/addressbook-stable; \
@@ -88,15 +89,15 @@ This is the canary job sequence:
![Deploy block](./figures/05-sem-canary-deploy-block.png){ width=95% }
Create a third block called “Functional test and migration” and enable the `do-key` secret. Repeat the environment variables and prologue steps from the previous block. This is the last block in the pipeline and it runs some automated tests on the canary. By combining `kubectl get pod` and `kubectl exec`, we can run commands inside the pod.
Create a third block called “Functional test and migration” and enable the `do-key` secret. Repeat the environment variables. This is the last block in the pipeline and it runs some automated tests on the canary. By combining `kubectl get pod` and `kubectl exec`, we can run commands inside the pod.
Type the following commands in the job:
```bash
POD=$(kubectl get pod \
-l deployment=addressbook-canary \
-o name \ | head -n 1)
doctl auth init --access-token $DO_ACCESS_TOKEN
doctl kubernetes cluster kubeconfig save "${CLUSTER_NAME}"
checkout
POD=$(kubectl get pod -l deployment=addressbook-canary -o name | head -n 1)
kubectl exec -it "$POD" -- npm run ping
kubectl exec -it "$POD" -- npm run migrate
```
@@ -115,16 +116,23 @@ Create a new pipeline (using the *Add promotion* button) branching out from the
![Stable promotion](./figures/05-sem-stable-promotion.png){ width=95% }
Create the “Deploy to Kubernetes” block with the `do-key`, `db-params`, and `dockerhub` secrets. Also, create the `CLUSTER_NAME` variable and repeat the same commands in the prologue as we did in the previous step.
Create the “Deploy to Kubernetes” block with the `do-key` and `db-params` secrets. Also, create the `CLUSTER_NAME` and `REGISTRY_NAME` variables as we did in the previous step.
In the job command box, type the following lines to make the rolling deployment and delete the canary pods:
```bash
./apply.sh manifests/deployment.yml addressbook-stable 3 \
$DOCKER_USERNAME/demo:$SEMAPHORE_WORKFLOW_ID
doctl auth init --access-token $DO_ACCESS_TOKEN
doctl kubernetes cluster kubeconfig save "${CLUSTER_NAME}"
checkout
kubectl apply -f manifests/service.yml
./apply.sh \
manifests/deployment.yml \
addressbook-stable 3 \
registry.digitalocean.com/$REGISTRY_NAME/demo:$SEMAPHORE_WORKFLOW_ID
if kubectl get deployment addressbook-canary; then \
kubectl delete deployment/addressbook-canary; \
kubectl delete deployment/addressbook-canary; \
fi
```
@@ -247,25 +255,24 @@ result = 'failed'
![Rollback promotion](./figures/05-sem-rollback-promotion.png){ width=95% }
The rollback job collects information to help diagnose the problem. Create a new block called “Rollback Canary”, import the `do-ctl` secret, and create `CLUSTER_NAME`. Repeat the prologue commands like we did before and type these lines in the job:
The rollback job collects information to help diagnose the problem. Create a new block called “Rollback Canary”, import the `do-ctl` secret, and create `CLUSTER_NAME` and `REGISTRY_NAME`. Type these lines in the job:
```bash
doctl auth init --access-token $DO_ACCESS_TOKEN
doctl kubernetes cluster kubeconfig save "${CLUSTER_NAME}"
kubectl get all -o wide
kubectl get events
kubectl describe deployment addressbook-canary || true
POD=$(kubectl get pod \
-l deployment=addressbook-canary \
-o name \ | head -n 1)
POD=$(kubectl get pod -l deployment=addressbook-canary -o name | head -n 1)
kubectl logs "$POD" || true
if kubectl get deployment addressbook-stable; then \
kubectl scale --replicas=3 deployment/addressbook-stable; \
kubectl scale --replicas=3 \
deployment/addressbook-stable; \
fi
if kubectl get deployment addressbook-canary; then \
kubectl delete deployment/addressbook-canary; \
kubectl delete deployment/addressbook-canary; \
fi
```