Add Buildkite agent chart (#1797)

* Add Buildkite agent chart * update wording
2026-08-23 22:37:45 +00:00 · 2017-09-04 15:12:01 +01:00
parent 12c17bff3a
commit b050cd57d8
14 changed files with 444 additions and 0 deletions
@@ -0,0 +1,21 @@
+# Patterns to ignore when building packages.
+# This supports shell glob matching, relative path matching, and
+# negation (prefixed with !). Only one pattern per line.
+.DS_Store
+# Common VCS dirs
+.git/
+.gitignore
+.bzr/
+.bzrignore
+.hg/
+.hgignore
+.svn/
+# Common backup files
+*.swp
+*.bak
+*.tmp
+*~
+# Various IDEs
+.project
+.idea/
+*.tmproj
@@ -0,0 +1,17 @@
+apiVersion: v1
+description: Agent for Buildkite
+name: buildkite
+version: 0.1.0
+appVersion: 3.0
+icon: https://github.com/buildkite/media/blob/master/marks/Buildkite%20-%20Mark%20-%20colour.png
+keywords:
+- kubernetes deployment
+- helm release
+- continuous deployment
+- continuous integration
+home: https://buildkite.com
+sources:
+- https://github.com/buildkite/agent
+maintainers:
+- name: rimusz
+  email: rmocius@gmail.com
@@ -0,0 +1,68 @@
+# Running Buildkite agent
+
+The [buildkite agent](https://buildkite.com/docs/agent) is a small, reliable and cross-platform build runner that makes it easy to run automated builds on your own infrastructure. Its main responsibilities are polling buildkite.com for work, running build jobs, reporting back the status code and output log of the job, and uploading the job's artefacts.
+It is simple, lightweight hosted [Buildkite](https://buildkite.com) CI/CD system which only requires to host agents in your Kubernetes cluster.
+
+## Introduction
+
+This chart bootstraps a [buildkite agent](https://github.com/buildkite/docker-buildkite-agent) builder on a [Kubernetes](http://kubernetes.io) cluster using the [Helm](https://helm.sh) package manager.
+As it sets `service account` it can be used to build Docker images and deploy them using `kubectl` and `helm` clients in the same cluster where agents run, without any extra setup.
+
+
+## Installing the Chart
+
+In order for the chart to configure the Buildkite Agent properly during the installation process, you must provide some minimal configuration which can't rely on defaults. This includes at least one element in the _agent_ list `token`:
+
+To install the chart with the release name `bk-agent`:
+
+```bash
+$ helm install stable/buildkite --name bk-agent --namespace buildkite --set agent.token="BUILDKITE_AGENT_TOKEN"
+```
+
+To install the chart with the release name `bk-agent` and set Agent meta-data and git repo SSH key:
+```console
+$ helm install stable/buildkite --name bk-agent --namespace buildkite \
+  --set agent.token="$(cat buildkite.token)",agent.meta="role=production",privateSshKey="$(cat buildkite.key)"
+```
+
+Where `--set` values contain:
+```
+agentToken: Buildkite token read from file
+agentMeta: tagging agent with - role=production
+privateSshKey: private SSH key read from file
+```
+
+> **Tip**: List all releases using `helm list`
+
+## Uninstalling the Chart
+
+To uninstall/delete the `bk-agent` release:
+
+```bash
+$ helm delete bk-agent
+```
+
+The command removes all the Kubernetes components associated with the chart and deletes the release.
+
+## Configuration
+
+The following table lists the configurable parameters of the `buildkite` chart and their default values.
+
+Parameter | Description | Default
+--- | --- | ---
+`replicaCount` | Replicas count | 1
+`image.repository` | Image | `buildkite/agent`
+`image.tag` | Image tag | `3.0`
+`image.pullPolicy` | Image pull policy | `IfNotPresent`
+`agent.token` | Agent token | Must be specified
+`agent.meta` | Agent meta-data | `role=agent`
+`extraEnv` | Agent extra env vars | `nil`
+`privateSshKey` | Agent ssh key for git access | `nil`
+`registryCreds.gcrServiceAccountKey` | GCP Service account json key | `nil`
+`registryCreds.dockerConfig` | Private registry docker config.json | `nil`
+`resources` | pod resource requests & limits | `{}`
+`nodeSelector` | node labels for pod assignment | `{}`
+
+## Buildkite pipeline examples
+
+Check for examples of `pipeline.yml` and `build/deploy` scripts [here](pipeline-examples).
@@ -0,0 +1,42 @@
+#!/bin/bash
+
+# set docker repository
+DOCKER_REPO=quay.io/your_user_name
+
+# image name
+IMAGE=my_image
+
+# use buildkite commit hash as a TAG
+TAG=${BUILDKITE_COMMIT::8}
+
+# make tmp folder
+mkdir /tmp
+cd /tmp
+
+#  clone repo
+env SSH_AUTH_SOCK= GIT_SSH_COMMAND='ssh -v -i ./buildkite' git clone ${BUILDKITE_REPO}
+
+# cd to pulled repo folder
+cd ${BUILDKITE_PIPELINE_SLUG}
+
+# checkout branch
+git checkout ${BUILDKITE_BRANCH}
+
+# build docker image
+echo -e "\n--- Building :docker: image ${IMAGE}:${TAG}"
+docker build -t ${IMAGE}:${TAG} .
+
+# cleaning up repo folder
+echo "--- Cleaning up git repo folder ${BUILDKITE_PIPELINE_SLUG}"
+rm -rf /tmp/${BUILDKITE_PIPELINE_SLUG}
+
+# tag docker image
+docker tag ${IMAGE}:${TAG} ${DOCKER_REPO}/${IMAGE}:${TAG}
+
+# push to repository
+echo "--- Pushing :docker: image ${DOCKER_REPO}/${IMAGE}:${TAG} to registry"
+docker push ${DOCKER_REPO}/${IMAGE}:${TAG}
+
+# local clean up
+echo "--- Cleaning up :docker: image ${DOCKER_REPO}/${IMAGE}:${TAG}"
+docker rmi -f ${DOCKER_REPO}/${IMAGE}:${TAG}
@@ -0,0 +1,21 @@
+#!/bin/bash
+
+# setup helm
+echo "Installing helm"
+curl https://raw.githubusercontent.com/kubernetes/helm/master/scripts/get | bash
+echo "--- Configuring Helm cli :rocket:"
+export HELM_HOME="${PWD}/.buildkite/.helm"
+helm init -c
+helm repo add charts https://my_charts.com/charts
+helm repo update
+
+# use buildkite commit hash as a TAG
+TAG=${BUILDKITE_COMMIT::8}
+
+# app name
+APP=some_app
+
+# deploy/upgrade app with helm
+echo "--- Deploying $APP :rocket:"
+helm upgrade --install ${APP} charts/my_app --namespace ${SOME_NAMESPACE} --reuse-values \
+  --set image.tag="${GIT_TAG}"
@@ -0,0 +1,54 @@
+steps:
+
+# Test release
+  - name: "Run :docker: compose tests :testflight:"
+    command: docker-compose run --rm test
+    branches: test
+    agents:
+      role: builder-test
+    plugins:
+      docker-compose:
+        run: test
+  - wait
+  - name: "Build :docker: image and push it to registry :rocket:"
+    command: .buildkite/build.sh
+    branches: test
+    agents:
+      role: builder-test
+  - wait
+  - name: "Deploy :docker: image to Test Workflow"
+    command: .buildkite/deploy.sh
+    branches: test
+    agents:
+      role: builder-test
+
+# Staging release
+  - name: "Run :docker: compose tests :testflight:"
+    command: docker-compose run --rm test
+    branches: staging
+    agents:
+      role: builder-staging
+  - wait
+  - name: "Build :docker: image and push it to registry :rocket:"
+    command: .buildkite/build.sh
+    branches: staging
+    agents:
+      role: builder-staging
+  - wait
+  - name: "Deploy :docker: image to Staging"
+    command: .buildkite/deploy.sh
+    branches: staging
+    agents:
+      role: builder-staging
+  - wait
+
+# Production release
+  # wait for unblock by team member
+  - block: 'Trigger Production Release :red_button:'
+    branches: production
+
+  - name: ":rocket: Deploying to Production"
+    command: .buildkite/deploy.sh
+    branches: production
+    agents:
+      role: deploy-production
@@ -0,0 +1,22 @@
+{{- if .Values.agent.token }}
+
+The {{ template "fullname" . }} is getting provisioned in your cluster. After a few minutes, you can run the following to verify.
+
+To verify that {{ template "fullname" . }} has started, run:
+
+  kubectl --namespace={{ .Release.Namespace }} get pods -l "app={{ template "name" . }},release={{ .Release.Name }}"
+
+{{- else }}
+
+##############################################################################
+####   ERROR: You did not provide Buildkite Agent Token.                  ####
+##############################################################################
+
+Pods will not go to the running state if the Buildkite Agent Token was not provided.
+
+This deployment will be incomplete until you provide Buildkite Agent Token:
+
+  helm upgrade {{ template "fullname" . }} stable/builkite \
+      --set agent.token="YOUR_BUILKITE_TOKEN"
+
+{{- end }}
@@ -0,0 +1,16 @@
+{{/* vim: set filetype=mustache: */}}
+{{/*
+Expand the name of the chart.
+*/}}
+{{- define "name" -}}
+{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
+{{- end -}}
+
+{{/*
+Create a default fully qualified app name.
+We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
+*/}}
+{{- define "fullname" -}}
+{{- $name := default .Chart.Name .Values.nameOverride -}}
+{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
+{{- end -}}
@@ -0,0 +1,75 @@
+{{- if .Values.agent.token }}
+apiVersion: extensions/v1beta1
+kind: Deployment
+metadata:
+  name: {{ template "fullname" . }}
+  labels:
+    app: {{ template "name" . }}
+    chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
+    release: {{ .Release.Name }}
+    heritage: {{ .Release.Service }}
+spec:
+  replicas: {{ .Values.replicaCount }}
+  template:
+    metadata:
+      labels:
+        app: {{ template "name" . }}
+        release: {{ .Release.Name }}
+    spec:
+      serviceAccount: {{ template "fullname" . }}
+      containers:
+        - name: {{ .Chart.Name }}
+          image: "{{ .Values.image.repository }}:{{ .Values.image.tag  }}"
+          imagePullPolicy: {{ .Values.image.pullPolicy }}
+          securityContext:
+            privileged: true
+          env:
+            # BUILDKITE AGENT ENV VARS
+            - name: BUILDKITE_AGENT_TOKEN
+              valueFrom:
+                secretKeyRef:
+                  name: {{ template "fullname" . }}
+                  key: agent-token
+            - name: BUILDKITE_AGENT_META_DATA
+              value: "{{ .Values.agent.meta }}"
+            {{- if .Values.privateSshKey }}
+            - name: SSH_PRIVATE_RSA_KEY
+              valueFrom:
+                secretKeyRef:
+                  name: {{ template "fullname" . }}
+                  key: agent-ssh
+            {{- end }}
+            # EXTRA BUILDKITE AGENT ENV VARS
+{{- if .Values.extraEnv }}{{ toYaml .Values.extraEnv | indent 12 }}{{- end }}
+          resources:
+{{ toYaml .Values.resources | indent 12 }}
+          volumeMounts:
+            - name: var-run-docker-sock
+              mountPath: /var/run/docker.sock
+            {{- if .Values.registryCreds.gcrServiceAccountKey }}
+            - name: service-key
+              mountPath: /etc/service_key
+            {{- end }}
+            {{- if .Values.registryCreds.dockerConfig }}
+            - name: docker-config
+              mountPath: /root/.docker
+            {{- end }}
+      volumes:
+        - name: var-run-docker-sock
+          hostPath:
+            path: /var/run/docker.sock
+        {{- if .Values.registryCreds.gcrServiceAccountKey }}
+        - name: service-key
+          secret:
+            secretName: {{ template "fullname" . }}-gcr
+        {{- end }}
+        {{- if .Values.registryCreds.dockerConfig }}
+        - name: docker-config
+          secret:
+            secretName: {{ template "fullname" . }}-registry
+        {{- end }}
+    {{- if .Values.nodeSelector }}
+      nodeSelector:
+{{ toYaml .Values.nodeSelector | indent 8 }}
+    {{- end }}
+{{- end }}
@@ -0,0 +1,15 @@
+{{- if .Values.registryCreds.gcrServiceAccountKey }}
+apiVersion: v1
+kind: Secret
+metadata:
+  name: {{ template "fullname" . }}-gcr
+  labels:
+    app: {{ template "name" . }}
+    chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
+    release: {{ .Release.Name }}
+    heritage: {{ .Release.Service }}
+type: Opaque
+data:
+  service-account-key.json: |-
+    {{ .Values.registryCreds.gcrServiceAccountKey }}
+{{- end }}
@@ -0,0 +1,15 @@
+{{- if .Values.registryCreds.dockerConfig }}
+apiVersion: v1
+kind: Secret
+metadata:
+  name: {{ template "fullname" . }}-registry
+  labels:
+    app: {{ template "name" . }}
+    chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
+    release: {{ .Release.Name }}
+    heritage: {{ .Release.Service }}
+type: Opaque
+data:
+  .dockerconfigjson: |-
+    {{ .Values.registryCreds.dockerConfig }}
+{{- end }}
@@ -0,0 +1,15 @@
+apiVersion: v1
+kind: Secret
+metadata:
+  name: {{ template "fullname" . }}
+  labels:
+    app: {{ template "name" . }}
+    chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
+    release: {{ .Release.Name }}
+    heritage: {{ .Release.Service }}
+type: Opaque
+data:
+  agent-token: {{ .Values.agent.token | b64enc }}
+{{- if .Values.privateSshKey }}
+  agent-ssh: {{ .Values.privateSshKey | b64enc }}
+{{- end }}
@@ -0,0 +1,9 @@
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: {{ template "fullname" . }}
+  labels:
+    app: {{ template "name" . }}
+    chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
+    release: {{ .Release.Name }}
+    heritage: {{ .Release.Service }}
@@ -0,0 +1,54 @@
+# Default values for bk-agent.
+# This is a YAML-formatted file.
+# Declare variables to be passed into your templates.
+
+replicaCount: 1
+
+image:
+  repository: buildkite/agent
+  tag: 3.0
+  pullPolicy: IfNotPresent
+
+# Buildkite Agent settings
+agent:
+  # Your Buildkite agent token, it must be set
+  token: ""
+  # Agent meta-data, which can be used to assign jobs
+  meta: "role=agent"
+
+# Extra env vars to be passed
+# If you do want to xxtra env vars to pass to agent, uncomment the following
+# lines, adjust them as necessary.
+#extraEnv:
+#  - name: test1
+#    value: "test1"
+#  - name: test2
+#    value: "test2"
+
+# Your ssh private key if you want to access private git repository
+privateSshKey: ""
+
+# Docker registries credentials
+registryCreds:
+  # GCP credentials for GCR
+  # base64 encoded GCP Service account json key file
+  gcrServiceAccountKey: ""
+  # base64 encoded private registry docker config.json file
+  # for quay.io, docker hub, ecr and etc
+  dockerConfig: ""
+
+resources: {}
+  # We usually recommend not to specify default resources and to leave this as a conscious
+  # choice for the user. This also increases chances charts run on environments with little
+  # resources, such as Minikube. If you do want to specify resources, uncomment the following
+  # lines, adjust them as necessary, and remove the curly braces after 'resources:'.
+  # limits:
+  #  cpu: 500m
+  #  memory: 1024Mi
+  # requests:
+  #  cpu: 100m
+  #  memory: 128Mi
+
+## Node labels for pod assignment
+## Ref: https://kubernetes.io/docs/user-guide/node-selection/
+nodeSelector: {}