[stable/metricbeat] metricbeat chart (#7347)

* [stable/metricbeat] add metricbeat chart

Signed-off-by: Atsushi Kawamura <akawamura@c-fo.com>

* [stable/metricbeat] add starting space in comment

Signed-off-by: Atsushi Kawamura <akawamura@c-fo.com>

* [stable/metricbeat] modify api version

Signed-off-by: Atsushi Kawamura <akawamura@c-fo.com>

* [stable/metricbeat] set latest beat version and update role settings based on official

Signed-off-by: Atsushi Kawamura <akawamura@c-fo.com>

* [stable/metricbeat] split common config file into daemonset and deployment

Signed-off-by: Atsushi Kawamura <akawamura@c-fo.com>

* [stable/metricbeat] modify checksum target

Signed-off-by: Atsushi Kawamura <akawamura@c-fo.com>

* [stable/metricbeat] cleanup daemonset.yaml

Signed-off-by: Atsushi Kawamura <akawamura@c-fo.com>

* [stable/metricbeat] make serviceAccount.name configurable

Signed-off-by: Atsushi Kawamura <akawamura@c-fo.com>

* [stable/metricbeat] add matchLabels at deployment.yaml

Signed-off-by: Atsushi Kawamura <akawamura@c-fo.com>

* [stable/metricbeat] set api ver to v1

Signed-off-by: Atsushi Kawamura <akawamura@c-fo.com>

* [stable/metricbeat] add OWNERS file

Signed-off-by: Atsushi Kawamura <akawamura@c-fo.com>

* [stable/metricbeat] modify README

Signed-off-by: Atsushi Kawamura <akawamura@c-fo.com>

* [stable/metricbeat] extraVars -> extraEnv

Signed-off-by: Atsushi Kawamura <akawamura@c-fo.com>

* [stable/metricbeat] modify readme

Signed-off-by: Atsushi Kawamura <akawamura@c-fo.com>
This commit is contained in:
Atsushi Kawamura
2018-09-23 04:11:28 -07:00
committed by k8s-ci-robot
parent ad307f6596
commit e421d6c289
13 changed files with 607 additions and 0 deletions
+23
View File
@@ -0,0 +1,23 @@
# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
# OWNERS file for Kubernetes
OWNERS
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*~
# Various IDEs
.project
.idea/
*.tmproj
+14
View File
@@ -0,0 +1,14 @@
apiVersion: v1
description: A Helm chart to collect Kubernetes logs with metricbeat
icon: https://www.elastic.co/assets/blt47799dcdcf08438d/logo-elastic-beats-lt.svg
name: metricbeat
version: 0.3.0
appVersion: 6.4.0
home: https://www.elastic.co/products/beats/metricbeat
sources:
- https://www.elastic.co/guide/en/beats/metricbeat/current/index.html
maintainers:
- name: at-k
email: atushi.k@gmail.com
- name: mumoshu
email: ykuoka@gmail.com
+6
View File
@@ -0,0 +1,6 @@
approvers:
- at-k
- mumoshu
reviewers:
- at-k
- mumoshu
+70
View File
@@ -0,0 +1,70 @@
# Metricbeat
[metricbeat](https://www.elastic.co/guide/en/beats/metricbeat/current/index.html) is used to ship Kubernetes and host metrics to multiple outputs.
## Prerequisites
- Kubernetes 1.9+
## Installing the Chart
To install the chart with the release name `my-release`:
```bash
$ helm install --name my-release stable/metricbeat
```
> **Tip**: List all releases using `helm list`
## Uninstalling the Chart
To uninstall/delete the `my-release` deployment:
```bash
$ helm delete my-release
```
The command removes all the Kubernetes components associated with the chart and deletes the release.
## Configuration
The following table lists the configurable parameters of the metricbeat chart and their default values.
| Parameter | Description | Default |
|-------------------------------------|------------------------------------|-------------------------------------------|
| `image.repository` | The image repository to pull from | `docker.elastic.co/beats/metricbeat` |
| `image.tag` | The image tag to pull | `6.4.0` |
| `image.pullPolicy` | Image pull policy | `IfNotPresent` |
| `rbac.create` | If true, create & use RBAC resources | `true` |
| `serviceAccount.create` | If true, create & use ServiceAccount | `true` |
| `serviceAccount.name` | The name of the ServiceAccount to use. If not set and create is true, a name is generated using the fullname template | |
| `config` | The content of the configuration file consumed by metricbeat. See the [metricbeat.reference.yml](https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-reference-yml.html) for full details | |
| `plugins` | List of beat plugins | |
| `extraEnv` | Additional environment | |
| `extraVolumes`, `extraVolumeMounts` | Additional volumes and mounts, for example to provide other configuration files | |
| `resources.requests.cpu` | CPU resource requests | |
| `resources.limits.cpu` | CPU resource limits | |
| `resources.requests.memory` | Memory resource requests | |
| `resources.limits.memory` | Memory resource limits | |
| `daemonset.modules.<name>.config` | The content of the modules configuration file consumed by metricbeat deployed as daemonset, which is assumed to collect metrics in each nodes. See the [metricbeat.reference.yml](https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-reference-yml.html) for full details |
| `daemonset.modules.<name>.enabled` | If true, enable configuration | |
| `daemonset.podAnnotations` | Pod annotations for daemonset | |
| `deployment.modules.<name>.config` | The content of the modules configuration file consumed by metricbeat deployed as deployment, which is assumed to collect cluster-level metrics. See the [metricbeat.reference.yml](https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-reference-yml.html) for full details ||
| `deployment.modules.<name>.enabled` | If true, enable configuration ||
| `deployment.podAnnotations` | Pod annotations for deployment | |
Specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. For example,
```bash
$ helm install --name my-release \
--set rbac.create=true \
stable/metricbeat
```
Alternatively, a YAML file that specifies the values for the parameters can be provided while installing the chart. For example,
```bash
$ helm install --name my-release -f values.yaml stable/metricbeat
```
> **Tip**: You can use the default [values.yaml](values.yaml)
+3
View File
@@ -0,0 +1,3 @@
To verify that Metricbeat has started, run:
kubectl --namespace={{ .Release.Namespace }} get pods -l "app={{ template "metricbeat.name" . }},release={{ .Release.Name }}"
+43
View File
@@ -0,0 +1,43 @@
{{/* vim: set filetype=mustache: */}}
{{/*
Expand the name of the chart.
*/}}
{{- define "metricbeat.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{{/*
Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
If release name contains chart name it will be used as a full name.
*/}}
{{- define "metricbeat.fullname" -}}
{{- if .Values.fullnameOverride -}}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}}
{{- else -}}
{{- $name := default .Chart.Name .Values.nameOverride -}}
{{- if contains $name .Release.Name -}}
{{- .Release.Name | trunc 63 | trimSuffix "-" -}}
{{- else -}}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{{- end -}}
{{- end -}}
{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "metricbeat.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{{/*
Create the name of the service account to use
*/}}
{{- define "metricbeat.serviceAccountName" -}}
{{- if .Values.serviceAccount.create -}}
{{ default (include "metricbeat.fullname" .) .Values.serviceAccount.name }}
{{- else -}}
{{ default "default" .Values.serviceAccount.name }}
{{- end -}}
{{- end -}}
@@ -0,0 +1,28 @@
{{- if .Values.rbac.create -}}
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: {{ template "metricbeat.fullname" . }}
labels:
app: {{ template "metricbeat.name" . }}
chart: {{ template "metricbeat.chart" . }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
rules:
- apiGroups: [""]
resources:
- nodes
- namespaces
- events
- pods
verbs: ["get", "list", "watch"]
- apiGroups: ["extensions"]
resources:
- replicasets
verbs: ["get", "list", "watch"]
- apiGroups: ["apps"]
resources:
- statefulsets
- deployments
verbs: ["get", "list", "watch"]
{{- end -}}
@@ -0,0 +1,19 @@
{{- if .Values.rbac.create -}}
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: {{ template "metricbeat.fullname" . }}
labels:
app: {{ template "metricbeat.name" . }}
chart: {{ template "metricbeat.chart" . }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: {{ template "metricbeat.fullname" . }}
subjects:
- kind: ServiceAccount
name: {{ template "metricbeat.serviceAccountName" . }}
namespace: {{ .Release.Namespace }}
{{- end -}}
+119
View File
@@ -0,0 +1,119 @@
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: {{ template "metricbeat.fullname" . }}
labels:
app: {{ template "metricbeat.name" . }}
chart: {{ template "metricbeat.chart" . }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
spec:
selector:
matchLabels:
app: {{ template "metricbeat.name" . }}
release: {{ .Release.Name }}
minReadySeconds: 10
updateStrategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
template:
metadata:
labels:
app: {{ template "metricbeat.name" . }}
release: {{ .Release.Name }}
annotations:
checksum/daemonset: {{ toYaml .Values.daemonset.modules | sha256sum }}
checksum/config: {{ toYaml .Values.daemonset.config | sha256sum }}
{{- if .Values.daemonset.podAnnotations }}
{{- range $key, $value := .Values.daemonset.podAnnotations }}
{{ $key }}: {{ $value }}
{{- end }}
{{- end }}
spec:
containers:
- name: {{ .Chart.Name }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
args:
- "-e"
{{- if .Values.plugins }}
- "--plugin"
- {{ .Values.plugins | join "," | quote }}
{{- end }}
- "-system.hostfs=/hostfs"
{{- if .Values.daemonset.debug }}
- "-d"
- "*"
{{- end }}
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
{{- if .Values.extraEnv }}
{{ toYaml .Values.extraEnv | indent 8 }}
{{- end }}
securityContext:
runAsUser: 0
resources:
{{ toYaml .Values.resources | indent 10 }}
volumeMounts:
- name: config
mountPath: /usr/share/metricbeat/metricbeat.yml
readOnly: true
subPath: metricbeat.yml
- name: modules
mountPath: /usr/share/metricbeat/modules.d
readOnly: true
- name: data
mountPath: /usr/share/metricbeat/data
- name: proc
mountPath: /hostfs/proc
readOnly: true
- name: cgroup
mountPath: /hostfs/sys/fs/cgroup
readOnly: true
- name: dockersock
mountPath: /var/run/docker.sock
{{- if .Values.extraVolumeMounts }}
{{ toYaml .Values.extraVolumeMounts | indent 8 }}
{{- end }}
volumes:
- name: config
secret:
secretName: {{ template "metricbeat.fullname" . }}-daemonset-config
- name: modules
secret:
secretName: {{ template "metricbeat.fullname" . }}-daemonset-modules
- name: data
hostPath:
path: /var/lib/metricbeat
type: DirectoryOrCreate
- name: proc
hostPath:
path: /proc
- name: cgroup
hostPath:
path: /sys/fs/cgroup
- name: dockersock
hostPath:
path: /var/run/docker.sock
{{- if .Values.extraVolumes }}
{{ toYaml .Values.extraVolumes | indent 6 }}
{{- end }}
terminationGracePeriodSeconds: 60
serviceAccountName: {{ template "metricbeat.serviceAccountName" . }}
hostNetwork: true
tolerations:
- key: node-role.kubernetes.io/master
operator: Exists
effect: NoSchedule
{{- if .Values.nodeSelector }}
nodeSelector:
{{ toYaml .Values.nodeSelector | indent 8 }}
{{- end }}
@@ -0,0 +1,78 @@
# Deploy singleton instance in the whole cluster for some unique data sources, like kube-state-metrics
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ template "metricbeat.fullname" . }}
labels:
app: {{ template "metricbeat.name" . }}
chart: {{ template "metricbeat.chart" . }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
spec:
selector:
matchLabels:
app: {{ template "metricbeat.name" . }}
release: {{ .Release.Name }}
template:
metadata:
labels:
app: {{ template "metricbeat.name" . }}
release: {{ .Release.Name }}
annotations:
checksum/modules: {{ toYaml .Values.deployment.modules | sha256sum }}
checksum/config: {{ toYaml .Values.deployment.config | sha256sum }}
{{- if .Values.deployment.podAnnotations }}
{{- range $key, $value := .Values.deployment.podAnnotations }}
{{ $key }}: {{ $value }}
{{- end }}
{{- end }}
spec:
serviceAccountName: {{ template "metricbeat.serviceAccountName" . }}
containers:
- name: {{ .Chart.Name }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
args:
- "-e"
{{- if .Values.plugins }}
- "--plugin"
- {{ .Values.plugins | join "," | quote }}
{{- end }}
{{- if .Values.deployment.debug }}
- "-d"
- "*"
{{- end }}
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
{{- if .Values.extraEnv }}
{{ toYaml .Values.extraEnv | indent 8 }}
{{- end }}
securityContext:
runAsUser: 0
resources:
{{ toYaml .Values.resources | indent 10 }}
volumeMounts:
- name: metricbeat-config
mountPath: /usr/share/metricbeat/metricbeat.yml
readOnly: true
subPath: metricbeat.yml
- name: modules
mountPath: /usr/share/metricbeat/modules.d
readOnly: true
{{- if .Values.extraVolumeMounts }}
{{ toYaml .Values.extraVolumeMounts | indent 8 }}
{{- end }}
volumes:
- name: metricbeat-config
secret:
secretName: {{ template "metricbeat.fullname" . }}-deployment-config
- name: modules
secret:
secretName: {{ template "metricbeat.fullname" . }}-deployment-modules
+59
View File
@@ -0,0 +1,59 @@
apiVersion: v1
kind: Secret
metadata:
name: {{ template "metricbeat.fullname" . }}-daemonset-config
labels:
app: {{ template "metricbeat.name" . }}
chart: {{ template "metricbeat.chart" . }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
type: Opaque
data:
metricbeat.yml: {{ toYaml .Values.daemonset.config | indent 4 | b64enc }}
---
apiVersion: v1
kind: Secret
metadata:
name: {{ template "metricbeat.fullname" . }}-deployment-config
labels:
app: {{ template "metricbeat.name" . }}
chart: {{ template "metricbeat.chart" . }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
type: Opaque
data:
metricbeat.yml: {{ toYaml .Values.deployment.config | indent 4 | b64enc }}
---
apiVersion: v1
kind: Secret
metadata:
name: {{ template "metricbeat.fullname" . }}-daemonset-modules
labels:
app: {{ template "metricbeat.name" . }}
chart: {{ template "metricbeat.chart" . }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
type: Opaque
data:
{{- range $key, $value := .Values.daemonset.modules }}
{{- if eq $value.enabled true }}
{{ $key }}.yml: {{ toYaml $value.config | indent 4 | b64enc }}
{{- end }}
{{- end }}
---
apiVersion: v1
kind: Secret
metadata:
name: {{ template "metricbeat.fullname" . }}-deployment-modules
labels:
app: {{ template "metricbeat.name" . }}
chart: {{ template "metricbeat.chart" . }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
type: Opaque
data:
{{- range $key, $value := .Values.deployment.modules }}
{{- if eq $value.enabled true }}
{{ $key }}.yml: {{ toYaml $value.config | indent 4 | b64enc }}
{{- end }}
{{- end }}
@@ -0,0 +1,11 @@
{{- if .Values.serviceAccount.create -}}
apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ template "metricbeat.serviceAccountName" . }}
labels:
app: {{ template "metricbeat.name" . }}
chart: {{ template "metricbeat.chart" . }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
{{- end -}}
+134
View File
@@ -0,0 +1,134 @@
image:
repository: docker.elastic.co/beats/metricbeat
tag: 6.4.0
pullPolicy: IfNotPresent
# The instances created by daemonset retrieve most metrics from the host
daemonset:
podAnnotations: []
config:
metricbeat.config:
modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false
processors:
- add_cloud_metadata:
output.file:
path: "/usr/share/metricbeat/data"
filename: metricbeat
rotate_every_kb: 10000
number_of_files: 5
modules:
system:
enabled: true
config:
- module: system
period: 10s
metricsets:
- cpu
- load
- memory
- network
- process
- process_summary
# - core
# - diskio
# - socket
processes: ['.*']
process.include_top_n:
by_cpu: 5 # include top 5 processes by CPU
by_memory: 5 # include top 5 processes by memory
- module: system
period: 1m
metricsets:
- filesystem
- fsstat
processors:
- drop_event.when.regexp:
system.filesystem.mount_point: '^/(sys|cgroup|proc|dev|etc|host|lib)($|/)'
kubernetes:
enabled: true
config:
- module: kubernetes
metricsets:
- node
- system
- pod
- container
- volume
period: 10s
hosts: ["localhost:10255"]
# The instance created by deployment retrieves metrics that are unique for the whole cluster, like Kubernetes events or kube-state-metrics
deployment:
podAnnotations: []
config:
metricbeat.config:
modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false
processors:
- add_cloud_metadata:
output.file:
path: "/usr/share/metricbeat/data"
filename: metricbeat
rotate_every_kb: 10000
number_of_files: 5
modules:
kubernetes:
enabled: true
config:
- module: kubernetes
metricsets:
- state_node
- state_deployment
- state_replicaset
- state_pod
- state_container
# Uncomment this to get k8s events:
# - event
period: 10s
hosts: ["kube-state-metrics:8080"]
# List of beat plugins
plugins: []
# - kinesis.so
# additional environment
# extraEnv:
# - name: test1
# value: "test1"
# - name: test2
# value: "test2"
# Add additional volumes and mounts, for example to read other log files on the host
extraVolumes: []
# - hostPath:
# path: /var/log
# name: varlog
extraVolumeMounts: []
# - name: varlog
# mountPath: /host/var/log
# readOnly: true
resources: {}
# We usually recommend not to specify default resources and to leave this as a conscious
# choice for the user. This also increases chances charts run on environments with little
# resources, such as Minikube. If you do want to specify resources, uncomment the following
# lines, adjust them as necessary, and remove the curly braces after 'resources:'.
# limits:
# cpu: 100m
# memory: 200Mi
# requests:
# cpu: 100m
# memory: 100Mi
rbac:
# Specifies whether RBAC resources should be created
create: true
serviceAccount:
# Specifies whether a ServiceAccount should be created
create: true
# The name of the ServiceAccount to use.
# If not set and create is true, a name is generated using the fullname template
name: