From 9eecded9d243e981bd78a2b72cbd9e419af9766c Mon Sep 17 00:00:00 2001 From: wgj Date: Thu, 16 Jan 2020 20:14:56 -0700 Subject: [PATCH] [stable/influxdb] Adds support for InfluxDB Enterprise (#19625) * Adds support for InfluxDB Enterprise Follows the newly introduced pattern of StatefulSets for pods, extending InfluxDB's sts and allowing parameter for it's ReplicaSet, and adds a new sts for InfluxDB "meta nodes" which hold distribution state for InfluxDB's high availability and horizontal scaling. Documentation for these new features has been added, but I can not stress enough how important it is to follow, as Enterprise not only uses different container images for meta containers, but also expects new container images for InfluxDB (appropriately labeled `data` in Docker Hub) which in my experience was easy to miss if a user is following one set of [instructions](https://docs.influxdata.com/enterprise_influxdb/v1.7/install-and-deploy/production_installation/data_node_installation/) over [another](https://hub.docker.com/_/influxdb). A caveat of trade off of deploying Enterprise is an expectation of human interaction when creating or scaling the cluster (`influxd-ctl` usage). This makes Enterprise decidedly not "cloud native", but is less obnoxious when using a StatefulSet with PVCs, and no pod auto-scaling, of which I suppose is unlikely given InfluxData's approach to licensing. Still forcing k8s operator involvement feels "bad", and I left in provisions to improve this expectation in a later version, namely a headless service and using `publishNotReadyAddresses`, such that a pod can use the Service to find available `influxdb-meta` and `influxdb` pods to create/join an InfluxDB cluster. Signed-off-by: Weston Johnson * Adds missing newline at end of file Signed-off-by: Weston Johnson * Add reference links for InfluxDB Enterprise Signed-off-by: Weston Johnson * Bump version Signed-off-by: Weston Johnson * Remove bad whitespace that sneaked in Signed-off-by: Weston Johnson * Hard code headless service for metadata Metadata service is a StatefulSet, so it doesn't make sense to let users to set the k8s service to anything but ClusterIP::None. This is true for the StatefulSet for InfluxDB data pods, but that will be fixed in a subsequent PR. When that happens, the Enterprise section of README.md should be updated to remove references to ClusterIP::None. Signed-off-by: Weston Johnson --- stable/influxdb/Chart.yaml | 2 +- stable/influxdb/README.md | 33 +++++ stable/influxdb/templates/config.yaml | 13 +- stable/influxdb/templates/meta-config.yaml | 36 +++++ stable/influxdb/templates/meta-service.yaml | 26 ++++ .../influxdb/templates/meta-statefulset.yaml | 132 ++++++++++++++++++ stable/influxdb/templates/statefulset.yaml | 14 ++ stable/influxdb/values.yaml | 30 +++- 8 files changed, 283 insertions(+), 3 deletions(-) create mode 100644 stable/influxdb/templates/meta-config.yaml create mode 100644 stable/influxdb/templates/meta-service.yaml create mode 100644 stable/influxdb/templates/meta-statefulset.yaml diff --git a/stable/influxdb/Chart.yaml b/stable/influxdb/Chart.yaml index a07a462ae3..bd213ecd43 100755 --- a/stable/influxdb/Chart.yaml +++ b/stable/influxdb/Chart.yaml @@ -1,6 +1,6 @@ apiVersion: v1 name: influxdb -version: 3.1.1 +version: 3.2.0 appVersion: 1.7.6 description: Scalable datastore for metrics, events, and real-time analytics. keywords: diff --git a/stable/influxdb/README.md b/stable/influxdb/README.md index 357d328525..0384cb9909 100644 --- a/stable/influxdb/README.md +++ b/stable/influxdb/README.md @@ -65,6 +65,39 @@ $ helm install --name my-release -f values.yaml stable/influxdb > **Tip**: You can use the default [values.yaml](values.yaml) +### Enterprise +[InfluxDB Enterprise](https://www.influxdata.com/products/influxdb-enterprise/) is a hardened version of the open source core InfluxDB that includes additional closed source features designed for production workloads, featuring high availability and horizontal scaling. InfluxDB Enterprise features require a InfluxDB Enterprise license. + +#### Configuration +To enable InfluxDB Enterprise, set the following keys and values in a values file provided to Helm. + +| Key | Description | Recommended value | +| --- | --- | --- | +| `livenessProbe.initalDelaySeconds` | Used to allow enough time to join meta nodes to a cluster | `3600` | +| `image.tag` | Set to a `data` image. See https://hub.docker.com/_/influxdb for details | `data` | +| `service.ClusterIP` | Use a headless service for StatefulSets | `"None"` | +| `env.name[_HOSTNAME]` | Used to provide a unique `name.service` for InfluxDB. See [values.yaml]() for an example | `valueFrom.fieldRef.fieldPath: metadata.name` | +| `enterprise.enabled` | Create StatefulSets for use with `influx-data` and `influx-meta` images | `true` | +| `enterprise.licensekey` | License for InfluxDB Enterprise | | +| `enterprise.clusterSize` | Replicas for `influx` StatefulSet | Dependent on license | +| `enterprise.meta.image.tag` | Set to an `meta` image. See https://hub.docker.com/_/influxdb for details | `meta` | +| `enterprise.meta.clusterSize` | Replicas for `influxdb-meta` StatefulSet. | `3` | +| `enterprise.meta.resources` | Resources requests and limits for meta `influxdb-meta` pods | See `values.yaml` | + +#### Join pods to InfluxDB Enterprise cluster +Meta and data pods need to be joined together using the command `influxd-ctl` found on meta pods. +It is recommended you run `influxd-ctl` on one and only one meta pod, and to join meta pods together before data pods. +For each meta pod, run `influxd-ctl`. With default settings it should look something like this: +```shell script +kubectl exec influxdb-meta-0 influxd-ctl add-meta influxdb-meta-0.influxdb-meta:8091 +``` +From the same meta pod, for each data pod, run `influxd-ctl`. With default settings it should look something like this: +```shell script +kubectl exec influxdb-meta-0 influxd-ctl add-data influxdb-0.influxdb:8088 +``` +When using `influxd-ctl` be sure to use the appropriate DNS name for your pods, following the naming scheme of `pod.service`. +In the above examples, the pod names were `influxdb-meta-0` and `influxdb-0` respectively, and the service name was `influxdb` + ## Persistence The [InfluxDB](https://hub.docker.com/_/influxdb/) image stores data in the `/var/lib/influxdb` directory in the container. diff --git a/stable/influxdb/templates/config.yaml b/stable/influxdb/templates/config.yaml index 2ecf68b7e3..69b1219c5c 100644 --- a/stable/influxdb/templates/config.yaml +++ b/stable/influxdb/templates/config.yaml @@ -1,4 +1,3 @@ - apiVersion: v1 kind: ConfigMap metadata: @@ -13,10 +12,22 @@ data: reporting-disabled = {{ .Values.config.reporting_disabled | default false }} bind-address = ":{{ .Values.config.rpc.bind_address }}" + {{ if .Values.enterprise.enabled -}} + [enterprise] + license-key = {{ .Values.enterprise.licensekey | quote }} + [hinted-handoff] + enabled = true + dir = "{{ .Values.config.storage_directory }}/hh" + {{- end }} + [meta] dir = "{{ .Values.config.storage_directory }}/meta" retention-autocreate = {{ .Values.config.meta.retention_autocreate }} logging-enabled = {{ .Values.config.meta.logging_enabled }} + {{- if .Values.enterprise.enabled }} + internal-shared-secret = "{{ sha256sum .Values.enterprise.meta.seed }}" + meta-auth-enabled = {{ .Values.config.meta.authEnabled }} + {{- end }} [data] dir = "{{ .Values.config.storage_directory }}/data" diff --git a/stable/influxdb/templates/meta-config.yaml b/stable/influxdb/templates/meta-config.yaml new file mode 100644 index 0000000000..21ab491338 --- /dev/null +++ b/stable/influxdb/templates/meta-config.yaml @@ -0,0 +1,36 @@ +{{ if .Values.enterprise.enabled -}} +apiVersion: v1 +kind: ConfigMap +metadata: + name: "{{ template "influxdb.fullname" . }}-meta" + labels: + app: "{{ template "influxdb.fullname" . }}-meta" + chart: "{{ template "influxdb.chart" . }}" + release: "{{ .Release.Name }}" + heritage: "{{ .Release.Service }}" +data: + influxdb-meta.conf: |+ + reporting-disabled = {{ .Values.config.reporting_disabled | default false }} + bind-address = ":{{ .Values.config.meta.bind_address }}" + + [enterprise] + license-key = {{ .Values.enterprise.licensekey | quote }} + + [meta] + dir = "{{ .Values.config.storage_directory }}/meta" + retention-autocreate = {{ .Values.config.meta.retention_autocreate }} + logging-enabled = {{ .Values.config.meta.logging_enabled }} + internal-shared-secret = "{{ sha256sum .Values.enterprise.meta.seed }}" + {{- if .Values.enterprise.enabled }} + meta-auth-enabled = {{ .Values.config.meta.authEnabled }} + {{- end }} + + [logging] + format = "{{ .Values.config.logging.format }}" + level = "{{ .Values.config.logging.level }}" + supress-logo = {{ .Values.config.logging.supress_logo }} + + [tls] + min-version = "" + max-version = "" +{{- end }} diff --git a/stable/influxdb/templates/meta-service.yaml b/stable/influxdb/templates/meta-service.yaml new file mode 100644 index 0000000000..aaf67a372d --- /dev/null +++ b/stable/influxdb/templates/meta-service.yaml @@ -0,0 +1,26 @@ +{{ if .Values.enterprise.enabled -}} +apiVersion: v1 +kind: Service +metadata: +{{- if .Values.service.annotations }} + annotations: +{{ toYaml .Values.service.annotations | indent 4 }} +{{- end }} + name: "{{ template "influxdb.fullname" . }}-meta" + labels: + app: "{{ template "influxdb.fullname" . }}-meta" + chart: "{{ template "influxdb.chart" . }}" + release: "{{ .Release.Name }}" + heritage: "{{ .Release.Service }}" +spec: + type: ClusterIP + clusterIP: None + # publishNotReadyAddresses is used for service discovery of meta and data nodes by querying the service's SRV record. + publishNotReadyAddresses: true + ports: + - name: meta + port: {{ .Values.config.meta.bind_address }} + targetPort: {{ .Values.config.meta.bind_address }} + selector: + app: "{{ template "influxdb.fullname" . }}-meta" +{{- end }} diff --git a/stable/influxdb/templates/meta-statefulset.yaml b/stable/influxdb/templates/meta-statefulset.yaml new file mode 100644 index 0000000000..601b529837 --- /dev/null +++ b/stable/influxdb/templates/meta-statefulset.yaml @@ -0,0 +1,132 @@ +{{- if .Values.enterprise.enabled }} +apiVersion: apps/v1 +kind: StatefulSet +metadata: + name: "{{ template "influxdb.fullname" . }}-meta" + labels: + app: "{{ template "influxdb.fullname" . }}-meta" + chart: "{{ template "influxdb.chart" . }}" + release: "{{ .Release.Name }}" + heritage: "{{ .Release.Service }}" +spec: + replicas: {{ .Values.enterprise.meta.clusterSize }} + selector: + matchLabels: + app: "{{ template "influxdb.fullname" . }}-meta" + serviceName: "{{ template "influxdb.fullname" . }}-meta" + template: + metadata: + labels: + app: "{{ template "influxdb.fullname" . }}-meta" + release: "{{ .Release.Name }}" + {{- if .Values.podAnnotations }} + annotations: +{{ toYaml .Values.podAnnotations | indent 8 }} + {{- end }} + spec: + {{- if .Values.image.pullSecrets }} + imagePullSecrets: + {{- range .Values.image.pullSecrets }} + - name: {{ . }} + {{- end}} + {{- end }} + containers: + - name: "{{ template "influxdb.fullname" . }}-meta" + image: "{{ .Values.image.repository }}:{{ .Values.enterprise.meta.image.tag }}" + imagePullPolicy: {{ .Values.image.pullPolicy | quote }} + resources: +{{ toYaml .Values.enterprise.meta.resources | indent 10 }} + ports: + - name: udp + containerPort: {{ .Values.config.udp.bind_address }} + - name: rpc + containerPort: {{ .Values.config.rpc.bind_address }} + - name: meta + containerPort: {{ .Values.config.meta.bind_address }} + {{- if .Values.env }} + env: +{{ toYaml .Values.env | indent 10 }} + # Values.env's HOSTNAME isn't fundamentally different from $HOSTNAME, but this way we get a distinguished name for InfluxDB at runtime. + - name: INFLUXDB_HOSTNAME + value: "$(_HOSTNAME).{{ template "influxdb.fullname" . }}-meta" + {{- end }} + livenessProbe: + httpGet: + path: {{ .Values.livenessProbe.probePath }} + port: meta + initialDelaySeconds: {{ .Values.livenessProbe.initialDelaySeconds | default 30 }} + timeoutSeconds: {{ .Values.livenessProbe.timeoutSeconds | default 5 }} + readinessProbe: + httpGet: + path: {{ .Values.readinessProbe.probePath }} + port: meta + initialDelaySeconds: {{ .Values.readinessProbe.initialDelaySeconds | default 5 }} + timeoutSeconds: {{ .Values.readinessProbe.timeoutSeconds | default 1 }} + {{- if .Values.startupProbe.enabled }} + startupProbe: + httpGet: + path: {{ .Values.startupProbe.probePath }} + port: meta + failureThreshold: {{ .Values.startupProbe.failureThreshold | default 6 }} + periodSeconds: {{ .Values.startupProbe.periodSeconds | default 5 }} + {{- end }} + volumeMounts: + - name: {{ template "influxdb.fullname" . }}-meta + mountPath: {{ .Values.config.storage_directory }} + - name: config + mountPath: /etc/influxdb + {{- if .Values.initScripts.enabled }} + - name: init + mountPath: /docker-entrypoint-initdb.d + {{- end }} + volumes: + - name: config + configMap: + name: {{ template "influxdb.fullname" . }}-meta + {{- if .Values.initScripts.enabled }} + - name: init + configMap: + name: {{ template "influxdb.fullname" . }}-init + {{- end }} + {{- if (not .Values.persistence.enabled ) }} + - name: {{ template "influxdb.fullname" . }}-meta + emptyDir: {} + {{- end }} + {{- if .Values.schedulerName }} + schedulerName: "{{ .Values.schedulerName }}" + {{- end }} + {{- if .Values.nodeSelector }} + nodeSelector: +{{ toYaml .Values.nodeSelector | indent 8 }} + {{- end -}} + {{- if .Values.affinity }} + affinity: +{{ toYaml .Values.affinity | indent 8 }} + {{- end }} + {{- if .Values.tolerations }} + tolerations: +{{ toYaml .Values.tolerations | indent 8 }} + {{- end }} + {{- if .Values.persistence.enabled }} + volumeClaimTemplates: + - metadata: + name: {{ template "influxdb.fullname" . }}-meta + annotations: + {{- range $key, $value := .Values.persistence.annotations }} + {{ $key }}: "{{ $value }}" + {{- end }} + spec: + accessModes: + - {{ .Values.persistence.accessMode | quote}} + resources: + requests: + storage: {{ .Values.persistence.size | quote }} + {{- if .Values.persistence.storageClass }} + {{- if (eq "-" .Values.persistence.storageClass) }} + storageClassName: "" + {{- else }} + storageClassName: "{{ .Values.persistence.storageClass }}" + {{- end }} + {{- end }} + {{- end }} +{{- end }} diff --git a/stable/influxdb/templates/statefulset.yaml b/stable/influxdb/templates/statefulset.yaml index 95e9a8c57d..ae1de7d2fa 100644 --- a/stable/influxdb/templates/statefulset.yaml +++ b/stable/influxdb/templates/statefulset.yaml @@ -8,7 +8,11 @@ metadata: release: "{{ .Release.Name }}" heritage: "{{ .Release.Service }}" spec: + {{- if .Values.enterprise.enabled }} + replicas: {{ .Values.enterprise.clusterSize }} + {{ else }} replicas: 1 + {{- end}} selector: matchLabels: app: {{ template "influxdb.fullname" . }} @@ -75,9 +79,19 @@ spec: - name: opentsdb containerPort: {{ .Values.config.opentsdb.bind_address }} {{- end }} + {{- if .Values.enterprise.enabled }} + - name: rpc + containerPort: {{ .Values.config.rpc.bind_address }} + - name: meta + containerPort: {{ .Values.config.meta.bind_address }} + {{- end }} {{- if .Values.env }} env: {{ toYaml .Values.env | indent 10 }} + {{- if .Values.enterprise.enabled }} + - name: INFLUXDB_HOSTNAME # Values.env's HOSTNAME isn't fundamentally different from $HOSTNAME, but this way weg get a distinguished name at runtime. + value: "$(_HOSTNAME).{{ template "influxdb.fullname" . }}" + {{- end }} {{- end }} livenessProbe: httpGet: diff --git a/stable/influxdb/values.yaml b/stable/influxdb/values.yaml index 228eeacecb..ba5def8413 100644 --- a/stable/influxdb/values.yaml +++ b/stable/influxdb/values.yaml @@ -56,6 +56,32 @@ persistence: accessMode: ReadWriteOnce size: 8Gi +## Deploy InfluxDB Enterprise - License required +## ref: https://www.influxdata.com/products/influxdb-enterprise/ +enterprise: + enabled: false + licensekey: {} + clusterSize: 4 + meta: + image: + ## This image contains the enterprise meta node package for clustering. + ## It is meant to be used in conjunction with the influxdb:data package of the same version. + ## ref: https://hub.docker.com/_/influxdb + tag: meta + clusterSize: 3 + ## seed is hashed and used as `internal-shared-secret` for Meta service. + seed: dead-beef-cafe-bae + ## Configure resource requests and limits + ## ref: http://kubernetes.io/docs/user-guide/compute-resources/ + resources: {} + # resources: + # requests: + # memory: 512Mi + # cpu: 2 + # limits: + # memory: 1Gi + # cpu: 4 + ## Create default user through Kubernetes job ## Defaults indicated below ## @@ -106,7 +132,7 @@ resources: {} # memory: 16Gi # cpu: 8 -# Annotations to be added to InfuxDB pods +# Annotations to be added to InfluxDB pods podAnnotations: {} ingress: @@ -159,8 +185,10 @@ config: enabled: true bind_address: 8088 meta: + bind_address: 8091 retention_autocreate: true logging_enabled: true + authEnabled: false data: query_log_enabled: true cache_max_memory_size: 1073741824