Added mongodb helm chart. (#74)

* Added mongodb helm chart.

* Fixing readme comments.

* Added additional values to parameterize installer image.

* Renamed, moved, bumped up image version.

* Parameterizing storage class.

* Added notes.txt and updated README.
This commit is contained in:
Anirudh Ramanathan
2016-09-21 14:43:03 -07:00
committed by Vic Iglesias
parent d1b51bf761
commit bd1a11caae
12 changed files with 616 additions and 0 deletions
+21
View File
@@ -0,0 +1,21 @@
# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*~
# Various IDEs
.project
.idea/
*.tmproj
+9
View File
@@ -0,0 +1,9 @@
name: mongodb
home: https://github.com/mongodb/mongo
version: 0.1.0
description: MongoDB Helm chart for Kubernetes.
sources:
- https://github.com/mongodb/mongo
maintainers:
- name: Anirudh Ramanathan
email: ramanathana@google.com
+201
View File
@@ -0,0 +1,201 @@
# MongoDB Helm Chart
## Prerequisites Details
* Kubernetes 1.3 or higher with alpha APIs enabled.
* Petsets are in alpha in 1.3.
* Init containers are alpha in 1.3 and are moving to beta in 1.4.
* PV support on the underlying infrastructure.
## PetSet Details
* http://kubernetes.io/docs/user-guide/petset/
## PetSet Caveats
* http://kubernetes.io/docs/user-guide/petset/#alpha-limitations
# TODO
* Set up authorization between replicaset peers.
* Add a liveliness check.
* Make the `test.sh` script more robust.
## Chart Details
This chart implements a dynamically scalable [mongoDB replica set](https://docs.mongodb.com/manual/tutorial/deploy-replica-set/)
using Kubernetes PetSets and Init Containers.
## Get this chart
Download the latest release of the chart from the [releases](../../../releases) page.
Alternatively, clone the repo if you wish to use the development snapshot:
```console
$ git clone https://github.com/kubernetes/charts.git
```
## Installing the Chart
To install the chart with the release name `my-release`:
```console
$ helm install --name my-release mongodb-x.x.x.tgz
```
## Configuration
The following tables lists the configurable parameters of the mongodb chart and their default values.
| Parameter | Description | Default |
|-----------------------|----------------------------------|----------------------------------------------------------|
| `Name` | Name of the chart | `mongodb` |
| `Image` | Container image name | `mongo` |
| `ImageTag` | Container image tag | `3.2` |
| `ImagePullPolicy` | Container pull policy | `Always` |
| `Replicas` | k8s petset replicas | `3` |
| `Component` | k8s selector key | `mongodb` |
| `Cpu` | container requested cpu | `100m` |
| `Memory` |container requested memory | `512Mi` |
| `PeerPort`| Container listening port | `27017` |
| `Storage`| Persistent volume size | `10Gi` |
Specify each parameter using the `--set key=value[,key=value]` argument to `helm install`.
Alternatively, a YAML file that specifies the values for the parameters can be provided while installing the chart. For example,
```console
$ helm install --name my-release -f values.yaml mongodb-x.x.x.tgz
```
> **Tip**: You can use the default [values.yaml](values.yaml)
Once you have all 3 nodes in running, you can run the "test.sh" script in this directory, which will insert a key into the primary and check the secondaries for output. This script requires that the `$RELEASE_NAME` environment variable be set, in order to access the pods.
# Deep dive
Because the pod names are dependent on the name chosen for it, the following examples use the
environment variable `RELEASENAME`. For example, if the helm release name is `messy-hydra`, one would need to set the following before proceeding. The example scripts below assume 3 pods only.
```console
export RELEASE_NAME=messy-hydra
```
## Cluster Health
```console
$ for i in 0 1 2; do kubectl exec $RELEASE_NAME-mongodb-$i -- sh -c '/usr/bin/mongo --eval="printjson(db.serverStatus())"'; done
```
## Failover
One can check the roles being played by each node by using the following:
```console
$ for i in 0 1 2; do kubectl exec $RELEASE_NAME-mongodb-$i -- sh -c '/usr/bin/mongo --eval="printjson(rs.isMaster())"'; done
MongoDB shell version: 3.2.9
connecting to: test
{
"hosts" : [
"messy-hydra-mongodb-0.messy-hydra-mongodb.default.svc.cluster.local:27017",
"messy-hydra-mongodb-1.messy-hydra-mongodb.default.svc.cluster.local:27017",
"messy-hydra-mongodb-2.messy-hydra-mongodb.default.svc.cluster.local:27017"
],
"setName" : "rs0",
"setVersion" : 3,
"ismaster" : true,
"secondary" : false,
"primary" : "messy-hydra-mongodb-0.messy-hydra-mongodb.default.svc.cluster.local:27017",
"me" : "messy-hydra-mongodb-0.messy-hydra-mongodb.default.svc.cluster.local:27017",
"electionId" : ObjectId("7fffffff0000000000000001"),
"maxBsonObjectSize" : 16777216,
"maxMessageSizeBytes" : 48000000,
"maxWriteBatchSize" : 1000,
"localTime" : ISODate("2016-09-13T01:10:12.680Z"),
"maxWireVersion" : 4,
"minWireVersion" : 0,
"ok" : 1
}
```
This lets us see which member is primary.
Let us now test persistence and failover. First, we insert a key (in the below example, we assume pod 0 is the master):
```console
$ kubectl exec $RELEASE_NAME-mongodb-0 -- /usr/bin/mongo --eval="printjson(db.test.insert({key1: 'value1'}))"
MongoDB shell version: 3.2.8
connecting to: test
{ "nInserted" : 1 }
```
Watch existing members:
```console
$ kubectl run --attach bbox --image=mongo:3.2 --restart=Never -- sh -c 'while true; do for i in 0 1 2; do echo <$release-podname-$i> $(mongo --host=$RELEASE_NAME-mongodb-$i.$RELEASE_NAME-mongodb --eval="printjson(rs.isMaster())" | grep primary); sleep 1; done; done';
Waiting for pod default/bbox2 to be running, status is Pending, pod ready: false
If you don't see a command prompt, try pressing enter.
messy-hydra-mongodb-2 "primary" : "messy-hydra-mongodb-0.messy-hydra-mongodb.default.svc.cluster.local:27017",
messy-hydra-mongodb-0 "primary" : "messy-hydra-mongodb-0.messy-hydra-mongodb.default.svc.cluster.local:27017",
messy-hydra-mongodb-1 "primary" : "messy-hydra-mongodb-0.messy-hydra-mongodb.default.svc.cluster.local:27017",
messy-hydra-mongodb-2 "primary" : "messy-hydra-mongodb-0.messy-hydra-mongodb.default.svc.cluster.local:27017",
messy-hydra-mongodb-0 "primary" : "messy-hydra-mongodb-0.messy-hydra-mongodb.default.svc.cluster.local:27017",
```
Kill the primary and watch as a new master getting elected.
```console
$ kubectl delete pod $RELEASE_NAME-mongodb-0
pod "messy-hydra-mongodb-0" deleted
```
Delete all pods and let the petset controller bring it up.
```console
$ kubectl delete po -l app=mongodb
$ kubectl get po --watch-only
NAME READY STATUS RESTARTS AGE
messy-hydra-mongodb-0 0/1 Pending 0 0s
messy-hydra-mongodb-0 0/1 Pending 0 0s
messy-hydra-mongodb-0 0/1 Pending 0 7s
messy-hydra-mongodb-0 0/1 Init:0/2 0 7s
messy-hydra-mongodb-0 0/1 Init:1/2 0 27s
messy-hydra-mongodb-0 0/1 Init:1/2 0 28s
messy-hydra-mongodb-0 0/1 PodInitializing 0 31s
messy-hydra-mongodb-0 0/1 Running 0 32s
messy-hydra-mongodb-0 1/1 Running 0 37s
messy-hydra-mongodb-1 0/1 Pending 0 0s
messy-hydra-mongodb-1 0/1 Pending 0 0s
messy-hydra-mongodb-1 0/1 Init:0/2 0 0s
messy-hydra-mongodb-1 0/1 Init:1/2 0 20s
messy-hydra-mongodb-1 0/1 Init:1/2 0 21s
messy-hydra-mongodb-1 0/1 PodInitializing 0 24s
messy-hydra-mongodb-1 0/1 Running 0 25s
messy-hydra-mongodb-1 1/1 Running 0 30s
messy-hydra-mongodb-2 0/1 Pending 0 0s
messy-hydra-mongodb-2 0/1 Pending 0 0s
messy-hydra-mongodb-2 0/1 Init:0/2 0 0s
messy-hydra-mongodb-2 0/1 Init:1/2 0 21s
messy-hydra-mongodb-2 0/1 Init:1/2 0 22s
messy-hydra-mongodb-2 0/1 PodInitializing 0 25s
messy-hydra-mongodb-2 0/1 Running 0 26s
messy-hydra-mongodb-2 1/1 Running 0 30s
...
messy-hydra-mongodb-0 "primary" : "messy-hydra-mongodb-0.messy-hydra-mongodb.default.svc.cluster.local:27017",
messy-hydra-mongodb-1 "primary" : "messy-hydra-mongodb-0.messy-hydra-mongodb.default.svc.cluster.local:27017",
messy-hydra-mongodb-2 "primary" : "messy-hydra-mongodb-0.messy-hydra-mongodb.default.svc.cluster.local:27017",
```
Check the previously inserted key:
```console
$ kubectl exec $RELEASE_NAME-mongodb-1 -- /usr/bin/mongo --eval="rs.slaveOk(); db.test.find({key1:{\$exists:true}}).forEach(printjson)"
MongoDB shell version: 3.2.8
connecting to: test
{ "_id" : ObjectId("57b180b1a7311d08f2bfb617"), "key1" : "value1" }
```
## Scaling
Scaling should be managed by `helm upgrade`, which is the recommended way.
You can also scale up by modifying the number of replicas on the PetSet using `kubectl patch` or `kubectl apply`. Deleting the various pods created and the associated resources needs some care and is described in the [petset documentation](http://kubernetes.io/docs/user-guide/petset/#deleting-a-pet-set).
+47
View File
@@ -0,0 +1,47 @@
# Copyright 2016 The Kubernetes Authors All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
FROM debian:jessie
MAINTAINER Anirudh Ramanathan <foxish@google.com>
ADD files/* /
RUN set -x \
&& echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections \
&& apt-get update \
&& apt-get install --no-install-recommends -y wget localepurge ca-certificates \
&& wget -qO /peer-finder http://storage.googleapis.com/kubernetes-release/pets/peer-finder \
&& chmod -c 755 /install.sh /on-start.sh /peer-finder
RUN apt-get -y purge localepurge wget \
&& apt-get clean \
&& cd / \
&& rm -rf \
doc \
man \
info \
locale \
/var/lib/apt/lists/* \
/var/log/* \
/var/cache/debconf/* \
common-licenses \
~/.bashrc \
/etc/systemd \
/lib/lsb \
/lib/udev \
/usr/share/doc/ \
/usr/share/doc-base/ \
/usr/share/man/ \
/tmp/*
ENTRYPOINT ["/install.sh"]
+27
View File
@@ -0,0 +1,27 @@
# Copyright 2016 The Kubernetes Authors All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
all: push
TAG = 0.2
PREFIX = gcr.io/google_containers/mongodb-install
container:
docker build -t $(PREFIX):$(TAG) .
push: container
gcloud docker push $(PREFIX):$(TAG)
clean:
docker rmi $(PREFIX):$(TAG)
+51
View File
@@ -0,0 +1,51 @@
#!/usr/bin/env bash
# Copyright 2016 The Kubernetes Authors All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This volume is assumed to exist and is shared with parent of the init
# container. It contains the mongodb config.
CONFIG_VOLUME="/config"
# This volume is assumed to exist and is shared with the peer-finder
# init container. It contains on-start/change configuration scripts.
WORKDIR_VOLUME="/work-dir"
for i in "$@"
do
case $i in
-c=*|--config=*)
CONFIG_VOLUME="${i#*=}"
shift
;;
-w=*|--work-dir=*)
WORKDIR_VOLUME="${i#*=}"
shift
;;
*)
# unknown option
;;
esac
done
echo installing config scripts into "${WORKDIR_VOLUME}"
mkdir -p "${WORKDIR_VOLUME}"
cp /on-start.sh "${WORKDIR_VOLUME}"/
cp /peer-finder "${WORKDIR_VOLUME}"/
echo installing mongod.conf into "${CONFIG_VOLUME}"
mkdir -p "${CONFIG_VOLUME}"
chown -R mongodb:mongodb "${CONFIG_VOLUME}"
cp /mongod.conf "${CONFIG_VOLUME}"/
chmod 644 /${CONFIG_VOLUME}/mongod.conf
+13
View File
@@ -0,0 +1,13 @@
# mongod.conf
# Where to store the data.
# Note: if you run mongodb as a non-root user (recommended) you may
# need to create and set permissions for this directory manually,
# e.g., if the parent directory isn't mutable by the mongodb user.
dbpath=/data/db
port = 27017
# Replication Options
# in replicated mongo databases, specify the replica set name here
replSet=rs0
+58
View File
@@ -0,0 +1,58 @@
#!/usr/bin/env bash
# Copyright 2016 The Kubernetes Authors All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
function join {
local IFS="$1"; shift; echo "$*";
}
HOSTNAME=$(hostname)
# Parse out cluster name, formatted as: petset_name-index
IFS='-' read -ra ADDR <<< "$(hostname)"
CLUSTER_NAME="${ADDR[0]}"
while read -ra LINE; do
if [[ "${LINE}" == *"${HOSTNAME}"* ]]; then
MY_NAME=$LINE
continue
fi
PEERS=("${PEERS[@]}" $LINE)
done
# start a mongodb instance
/usr/bin/mongod --config=/config/mongod.conf &> /work-dir/log.txt &
until /usr/bin/mongo --eval 'printjson(db.serverStatus())'
do
echo "Retrying..." >> /work-dir/log.txt
sleep 2
done
echo "Initialized." >> /work-dir/log.txt
# try to find a master and add yourself to its replicaset.
for f in "${PEERS[@]}"
do
/usr/bin/mongo --host=${f} --eval="printjson(rs.isMaster())" | grep "\"ismaster\" : true"
if [ $? -eq 0 ]; then
echo ${f} MASTER >> /work-dir/log.txt
/usr/bin/mongo --host=${f} --eval="printjson(rs.add('${MY_NAME}'))"
exit 0
fi
done
# else initiate a replicaset with yourself.
/usr/bin/mongo --eval="printjson(rs.status())" | grep "no replset config has been received"
if [ $? -eq 0 ]; then
/usr/bin/mongo --eval="printjson(rs.initiate({'_id': 'rs0', 'members': [{'_id': 0, 'host': '${MY_NAME}'}]}))"
fi
+13
View File
@@ -0,0 +1,13 @@
Getting Started:
1. After the petset is created completely, one can check which instance is primary by running:
$ for i in `seq 0 2`; do kubectl exec {{ printf "%s-%s" .Release.Name .Values.Name | trunc 24 }}-$i -- sh -c '/usr/bin/mongo --eval="printjson(rs.isMaster())"'; done.
This assumes 3 replicas, 0 through 2. It should be modified to reflect the actual number of replicas specified.
2. One can insert a key into the primary instance of the mongodb replica set by running the following:
$ kubectl exec MASTER_POD_NAME -- /usr/bin/mongo --eval="printjson(db.test.insert({key1: 'value1'}))"
MASTER_POD_NAME must be replaced with the name of the master found from the previous step.
3. One can fetch the keys stored in the primary or any of the slave nodes in the following manner.
$ kubectl exec POD_NAME -- /usr/bin/mongo --eval="rs.slaveOk(); db.test.find().forEach(printjson)"
POD_NAME must be replaced by the name of the pod being queried.
@@ -0,0 +1,136 @@
# A headless service to create DNS records
apiVersion: v1
kind: Service
metadata:
name: "{{ printf "%s-%s" .Release.Name .Values.Name | trunc 24 }}"
labels:
heritage: {{.Release.Service | quote }}
release: {{.Release.Name | quote }}
chart: "{{.Chart.Name}}-{{.Chart.Version}}"
component: "{{.Release.Name}}-{{.Values.Component}}"
annotations:
"helm.sh/created": {{.Release.Time.Seconds | quote }}
service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"
spec:
ports:
- port: {{.Values.PeerPort}}
name: peer
clusterIP: None
selector:
component: "{{.Release.Name}}-{{.Values.Component}}"
---
apiVersion: apps/v1alpha1
kind: PetSet
metadata:
name: "{{ printf "%s-%s" .Release.Name .Values.Name | trunc 24 }}"
labels:
heritage: {{.Release.Service | quote }}
release: {{.Release.Name | quote }}
chart: "{{.Chart.Name}}-{{.Chart.Version}}"
component: "{{.Release.Name}}-{{.Values.Component}}"
annotations:
"helm.sh/created": {{.Release.Time.Seconds | quote }}
spec:
serviceName: "{{ printf "%s-%s" .Release.Name .Values.Name | trunc 24 }}"
replicas: {{default 3 .Values.Replicas | quote }}
template:
metadata:
labels:
heritage: {{.Release.Service | quote }}
release: {{.Release.Name | quote }}
chart: "{{.Chart.Name}}-{{.Chart.Version}}"
component: "{{ printf "%s-%s" .Release.Name .Values.Name | trunc 24 }}"
annotations:
"helm.sh/created": {{.Release.Time.Seconds | quote }}
pod.alpha.kubernetes.io/initialized: "true"
pod.alpha.kubernetes.io/init-containers: '[
{
"name": "install",
"image": "{{.Values.InstallImage}}:{{.Values.InstallImageTag}}",
"args": ["--work-dir=/work-dir"],
"volumeMounts": [
{
"name": "workdir",
"mountPath": "/work-dir"
},
{
"name": "config",
"mountPath": "/config"
}
]
},
{
"name": "bootstrap",
"image": "{{.Values.Image}}:{{.Values.ImageTag}}",
"command": ["/work-dir/peer-finder"],
"args": ["-on-start=\"/work-dir/on-start.sh\"", "-service={{ printf "%s-%s" .Release.Name .Values.Name | trunc 24 }}"],
"env": [
{
"name": "POD_NAMESPACE",
"valueFrom": {
"fieldRef": {
"apiVersion": "v1",
"fieldPath": "metadata.namespace"
}
}
}
],
"volumeMounts": [
{
"name": "workdir",
"mountPath": "/work-dir"
},
{
"name": "config",
"mountPath": "/config"
},
{
"name": "datadir",
"mountPath": "/data/db"
}
]
}
]'
spec:
containers:
- name: "{{ printf "%s-%s" .Release.Name .Values.Name | trunc 24 }}"
image: "{{.Values.Image}}:{{.Values.ImageTag}}"
imagePullPolicy: "{{.Values.ImagePullPolicy}}"
ports:
- containerPort: {{.Values.PeerPort}}
name: peer
resources:
requests:
cpu: "{{.Values.Cpu}}"
memory: "{{.Values.Memory}}"
command:
- /usr/bin/mongod
- --config=/config/mongod.conf
readinessProbe:
exec:
command:
- sh
- -c
- "/usr/bin/mongo --eval 'printjson(db.serverStatus())'"
initialDelaySeconds: 5
timeoutSeconds: 5
volumeMounts:
- name: datadir
mountPath: /data/db
- name: config
mountPath: /config
volumes:
- name: config
emptyDir: {}
- name: workdir
emptyDir: {}
volumeClaimTemplates:
- metadata:
name: datadir
annotations:
volume.alpha.kubernetes.io/storage-class: {{.Values.StorageClass}}
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: {{.Values.Storage}}
+21
View File
@@ -0,0 +1,21 @@
#! /bin/bash
# Copyright 2016 The Kubernetes Authors All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
kubectl exec $RELEASE_NAME-mongodb-0 -- /usr/bin/mongo --eval="printjson(db.test.insert({\"status\": \"success\"}))"
# TODO: find maximum duration to wait for slaves to be up-to-date with master.
sleep 2
kubectl exec $RELEASE_NAME-mongodb-1 -- /usr/bin/mongo --eval="rs.slaveOk(); db.test.find().forEach(printjson)"
kubectl exec $RELEASE_NAME-mongodb-2 -- /usr/bin/mongo --eval="rs.slaveOk(); db.test.find().forEach(printjson)"
+19
View File
@@ -0,0 +1,19 @@
# Default values for etcd.
# This is a YAML-formatted file.
# Declare name/value pairs to be passed into your templates.
# name: value
Name: mongodb
PeerPort: 27017
Component: "mongodb"
Replicas: 3
Image: "mongo"
ImageTag: "3.2"
InstallImage: "gcr.io/google_containers/mongodb-install"
InstallImageTag: "0.2"
ImagePullPolicy: "Always"
Cpu: "100m"
Memory: "512Mi"
Storage: "10Gi"
StorageClass: generic