[stable/prometheus-operator] allow definition of affinity for alertmanagerSpec/prometheusSpec (#12936)

* [stable/prometheus-operator] add affinity configuration

Extends the prometheus-operator to allow the definition of additional
affinity rules for alertmanager and prometheus without breaking the
existing podAntiAffinity functionality.

Signed-off-by: Lukas Grossar <lukas.grossar@adfinis-sygroup.ch>

* [stable/prometheus-operator] update ci/test-values.yaml

Got forgotten in some other PRs.

Signed-off-by: Lukas Grossar <lukas.grossar@adfinis-sygroup.ch>
This commit is contained in:
Lukas Grossar
2019-05-21 03:00:45 -07:00
committed by Kubernetes Prow Robot
parent 0f0069dfb7
commit 0269736bfd
6 changed files with 165 additions and 26 deletions
+1 -1
View File
@@ -9,7 +9,7 @@ name: prometheus-operator
sources:
- https://github.com/coreos/prometheus-operator
- https://coreos.com/operators/prometheus
version: 5.8.0
version: 5.9.0
appVersion: 0.29.0
home: https://github.com/coreos/prometheus-operator
keywords:
+3 -1
View File
@@ -155,7 +155,7 @@ The following tables list the configurable parameters of the prometheus-operator
| `prometheusOperator.securityContext` | SecurityContext for prometheus operator | `{"runAsNonRoot": true, "runAsUser": 65534}` |
| `prometheusOperator.nodeSelector` | Prometheus operator node selector https://kubernetes.io/docs/user-guide/node-selection/ | `{}` |
| `prometheusOperator.tolerations` | Tolerations for use with node taints https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/ | `[]` |
| `prometheusOperator.affinity` | Assign the prometheus operator to run on specific nodes https://kubernetes.io/docs/concepts/configuration/assign-pod-node/ | `{}` |
| `prometheusOperator.affinity` | Assign custom affinity rules to the prometheus operator https://kubernetes.io/docs/concepts/configuration/assign-pod-node/ | `{}` |
| `prometheusOperator.image.repository` | Repository for prometheus operator image | `quay.io/coreos/prometheus-operator` |
| `prometheusOperator.image.tag` | Tag for prometheus operator image | `v0.29.0` |
| `prometheusOperator.image.pullPolicy` | Pull policy for prometheus operator image | `IfNotPresent` |
@@ -229,6 +229,7 @@ The following tables list the configurable parameters of the prometheus-operator
| `prometheus.prometheusSpec.query` | QuerySpec defines the query command line flags when starting Prometheus. Not all parameters are supported by the operator - [see coreos documentation](https://github.com/coreos/prometheus-operator/blob/master/Documentation/api.md#queryspec) | `{}` |
| `prometheus.prometheusSpec.podAntiAffinity` | Pod anti-affinity can prevent the scheduler from placing Prometheus replicas on the same node. The default value "soft" means that the scheduler should *prefer* to not schedule two replica pods onto the same node but no guarantee is provided. The value "hard" means that the scheduler is *required* to not schedule two replica pods onto the same node. The value "" will disable pod anti-affinity so that no anti-affinity rules will be configured. | `""` |
| `prometheus.prometheusSpec.podAntiAffinityTopologyKey` | If anti-affinity is enabled sets the topologyKey to use for anti-affinity. This can be changed to, for example `failure-domain.beta.kubernetes.io/zone`| `kubernetes.io/hostname` |
| `prometheus.prometheusSpec.affinity` | Assign custom affinity rules to the prometheus instance https://kubernetes.io/docs/concepts/configuration/assign-pod-node/ | `{}` |
| `prometheus.prometheusSpec.tolerations` | If specified, the pod's tolerations. | `[]` |
| `prometheus.prometheusSpec.remoteWrite` | If specified, the remote_write spec. This is an experimental feature, it may change in any upcoming release in a breaking way. | `[]` |
| `prometheus.prometheusSpec.remoteRead` | If specified, the remote_read spec. This is an experimental feature, it may change in any upcoming release in a breaking way. | `[]` |
@@ -287,6 +288,7 @@ The following tables list the configurable parameters of the prometheus-operator
| `alertmanager.alertmanagerSpec.resources` | Define resources requests and limits for single Pods. | `{}` |
| `alertmanager.alertmanagerSpec.podAntiAffinity` | Pod anti-affinity can prevent the scheduler from placing Prometheus replicas on the same node. The default value "soft" means that the scheduler should *prefer* to not schedule two replica pods onto the same node but no guarantee is provided. The value "hard" means that the scheduler is *required* to not schedule two replica pods onto the same node. The value "" will disable pod anti-affinity so that no anti-affinity rules will be configured. | `""` |
| `alertmanager.alertmanagerSpec.podAntiAffinityTopologyKey` | If anti-affinity is enabled sets the topologyKey to use for anti-affinity. This can be changed to, for example `failure-domain.beta.kubernetes.io/zone`| `kubernetes.io/hostname` |
| `alertmanager.alertmanagerSpec.affinity` | Assign custom affinity rules to the alertmanager instance https://kubernetes.io/docs/concepts/configuration/assign-pod-node/ | `{}` |
| `alertmanager.alertmanagerSpec.tolerations` | If specified, the pod's tolerations. | `[]` |
| `alertmanager.alertmanagerSpec.securityContext` | SecurityContext holds pod-level security attributes and common container settings. This defaults to non root user with uid 1000 and gid 2000 in order to support migration from operator version < 0.26 | `{"runAsNonRoot": true, "runAsUser": 1000, "fsGroup": 2000}` |
| `alertmanager.alertmanagerSpec.listenLocal` | ListenLocal makes the Alertmanager server listen on loopback, so that it does not bind against the Pod IP. Note this is only for the Alertmanager UI, not the gossip communication. | `false` |
+111 -10
View File
@@ -186,6 +186,23 @@ alertmanager:
interval: ""
selfMonitor: true
## metric relabel configs to apply to samples before ingestion.
##
metricRelabelings: []
# - action: keep
# regex: 'kube_(daemonset|deployment|pod|namespace|node|statefulset).+'
# sourceLabels: [__name__]
# relabel configs to apply to samples before ingestion.
##
relabelings: []
# - sourceLabels: [__meta_kubernetes_pod_node_name]
# separator: ;
# regex: ^(.*)$
# target_label: nodename
# replacement: $1
# action: replace
## Settings affecting alertmanagerSpec
## ref: https://github.com/coreos/prometheus-operator/blob/master/Documentation/api.md#alertmanagerspec
##
@@ -280,6 +297,20 @@ alertmanager:
##
podAntiAffinityTopologyKey: kubernetes.io/hostname
## Assign custom affinity rules to the alertmanager instance
## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/
##
affinity: {}
# nodeAffinity:
# requiredDuringSchedulingIgnoredDuringExecution:
# nodeSelectorTerms:
# - matchExpressions:
# - key: kubernetes.io/e2e-az-name
# operator: In
# values:
# - e2e-az1
# - e2e-az2
## If specified, the pod's tolerations.
## ref: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
##
@@ -398,6 +429,23 @@ grafana:
interval: ""
selfMonitor: true
## metric relabel configs to apply to samples before ingestion.
##
metricRelabelings: []
# - action: keep
# regex: 'kube_(daemonset|deployment|pod|namespace|node|statefulset).+'
# sourceLabels: [__name__]
# relabel configs to apply to samples before ingestion.
##
relabelings: []
# - sourceLabels: [__meta_kubernetes_pod_node_name]
# separator: ;
# regex: ^(.*)$
# target_label: nodename
# replacement: $1
# action: replace
## Component scraping the kube api server
##
kubeApiServer:
@@ -858,6 +906,23 @@ prometheusOperator:
interval: ""
selfMonitor: true
## metric relabel configs to apply to samples before ingestion.
##
metricRelabelings: []
# - action: keep
# regex: 'kube_(daemonset|deployment|pod|namespace|node|statefulset).+'
# sourceLabels: [__name__]
# relabel configs to apply to samples before ingestion.
##
relabelings: []
# - sourceLabels: [__meta_kubernetes_pod_node_name]
# separator: ;
# regex: ^(.*)$
# target_label: nodename
# replacement: $1
# action: replace
## Resource limits & requests
##
resources: {}
@@ -882,18 +947,19 @@ prometheusOperator:
# value: "value"
# effect: "NoSchedule"
## Assign the prometheus operator to run on specific nodes
## Assign custom affinity rules to the prometheus operator
## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/
##
affinity: {}
# requiredDuringSchedulingIgnoredDuringExecution:
# nodeSelectorTerms:
# - matchExpressions:
# - key: kubernetes.io/e2e-az-name
# operator: In
# values:
# - e2e-az1
# - e2e-az2
# nodeAffinity:
# requiredDuringSchedulingIgnoredDuringExecution:
# nodeSelectorTerms:
# - matchExpressions:
# - key: kubernetes.io/e2e-az-name
# operator: In
# values:
# - e2e-az1
# - e2e-az2
securityContext:
runAsNonRoot: true
@@ -1025,6 +1091,23 @@ prometheus:
interval: ""
selfMonitor: true
## metric relabel configs to apply to samples before ingestion.
##
metricRelabelings: []
# - action: keep
# regex: 'kube_(daemonset|deployment|pod|namespace|node|statefulset).+'
# sourceLabels: [__name__]
# relabel configs to apply to samples before ingestion.
##
relabelings: []
# - sourceLabels: [__meta_kubernetes_pod_node_name]
# separator: ;
# regex: ^(.*)$
# target_label: nodename
# replacement: $1
# action: replace
## Settings affecting prometheusSpec
## ref: https://github.com/coreos/prometheus-operator/blob/master/Documentation/api.md#prometheusspec
##
@@ -1046,7 +1129,7 @@ prometheus:
##
image:
repository: quay.io/prometheus/prometheus
tag: v2.7.2
tag: v2.9.1
## Tolerations for use with node taints
## ref: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
@@ -1165,6 +1248,10 @@ prometheus:
##
logLevel: info
## Log format for Prometheus be configured in
##
logFormat: logfmt
## Prefix used to register routes, overriding externalUrl route.
## Useful for proxies that rewrite URLs.
##
@@ -1189,6 +1276,20 @@ prometheus:
##
podAntiAffinityTopologyKey: kubernetes.io/hostname
## Assign custom affinity rules to the prometheus instance
## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/
##
affinity: {}
# nodeAffinity:
# requiredDuringSchedulingIgnoredDuringExecution:
# nodeSelectorTerms:
# - matchExpressions:
# - key: kubernetes.io/e2e-az-name
# operator: In
# values:
# - e2e-az1
# - e2e-az2
## The remote_read spec configuration for Prometheus.
## ref: https://github.com/coreos/prometheus-operator/blob/master/Documentation/api.md#remotereadspec
remoteRead: []
@@ -55,8 +55,12 @@ spec:
podMetadata:
{{ toYaml .Values.alertmanager.alertmanagerSpec.podMetadata | indent 4 }}
{{- end }}
{{- if eq .Values.alertmanager.alertmanagerSpec.podAntiAffinity "hard" }}
{{- if or .Values.alertmanager.alertmanagerSpec.podAntiAffinity .Values.alertmanager.alertmanagerSpec.affinity }}
affinity:
{{- if .Values.alertmanager.alertmanagerSpec.affinity }}
{{ toYaml .Values.alertmanager.alertmanagerSpec.affinity | indent 4 }}
{{- end }}
{{- if eq .Values.alertmanager.alertmanagerSpec.podAntiAffinity "hard" }}
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- topologyKey: {{ .Values.alertmanager.alertmanagerSpec.podAntiAffinityTopologyKey }}
@@ -65,7 +69,6 @@ spec:
app: alertmanager
alertmanager: {{ template "prometheus-operator.fullname" . }}-alertmanager
{{- else if eq .Values.alertmanager.alertmanagerSpec.podAntiAffinity "soft" }}
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
@@ -76,6 +79,7 @@ spec:
app: alertmanager
alertmanager: {{ template "prometheus-operator.fullname" . }}-alertmanager
{{- end }}
{{- end }}
{{- if .Values.alertmanager.alertmanagerSpec.tolerations }}
tolerations:
{{ toYaml .Values.alertmanager.alertmanagerSpec.tolerations | indent 4 }}
@@ -123,8 +123,12 @@ spec:
query:
{{ toYaml .Values.prometheus.prometheusSpec.query | indent 4}}
{{- end }}
{{- if eq .Values.prometheus.prometheusSpec.podAntiAffinity "hard" }}
{{- if or .Values.prometheus.prometheusSpec.podAntiAffinity .Values.prometheus.prometheusSpec.affinity }}
affinity:
{{- if .Values.prometheus.prometheusSpec.affinity }}
{{ toYaml .Values.prometheus.prometheusSpec.affinity | indent 4 }}
{{- end }}
{{- if eq .Values.prometheus.prometheusSpec.podAntiAffinity "hard" }}
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- topologyKey: {{ .Values.prometheus.prometheusSpec.podAntiAffinityTopologyKey }}
@@ -133,7 +137,6 @@ spec:
app: prometheus
prometheus: {{ template "prometheus-operator.fullname" . }}-prometheus
{{- else if eq .Values.prometheus.prometheusSpec.podAntiAffinity "soft" }}
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
@@ -144,6 +147,7 @@ spec:
app: prometheus
prometheus: {{ template "prometheus-operator.fullname" . }}-prometheus
{{- end }}
{{- end }}
{{- if .Values.prometheus.prometheusSpec.tolerations }}
tolerations:
{{ toYaml .Values.prometheus.prometheusSpec.tolerations | indent 4 }}
+38 -10
View File
@@ -223,7 +223,6 @@ alertmanager:
##
useExistingSecret: false
## Secrets is a list of Secrets in the same namespace as the Alertmanager object, which shall be mounted into the
## Alertmanager Pods. The Secrets are mounted into /etc/alertmanager/secrets/.
##
@@ -298,6 +297,20 @@ alertmanager:
##
podAntiAffinityTopologyKey: kubernetes.io/hostname
## Assign custom affinity rules to the alertmanager instance
## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/
##
affinity: {}
# nodeAffinity:
# requiredDuringSchedulingIgnoredDuringExecution:
# nodeSelectorTerms:
# - matchExpressions:
# - key: kubernetes.io/e2e-az-name
# operator: In
# values:
# - e2e-az1
# - e2e-az2
## If specified, the pod's tolerations.
## ref: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
##
@@ -934,18 +947,19 @@ prometheusOperator:
# value: "value"
# effect: "NoSchedule"
## Assign the prometheus operator to run on specific nodes
## Assign custom affinity rules to the prometheus operator
## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/
##
affinity: {}
# requiredDuringSchedulingIgnoredDuringExecution:
# nodeSelectorTerms:
# - matchExpressions:
# - key: kubernetes.io/e2e-az-name
# operator: In
# values:
# - e2e-az1
# - e2e-az2
# nodeAffinity:
# requiredDuringSchedulingIgnoredDuringExecution:
# nodeSelectorTerms:
# - matchExpressions:
# - key: kubernetes.io/e2e-az-name
# operator: In
# values:
# - e2e-az1
# - e2e-az2
securityContext:
runAsNonRoot: true
@@ -1262,6 +1276,20 @@ prometheus:
##
podAntiAffinityTopologyKey: kubernetes.io/hostname
## Assign custom affinity rules to the prometheus instance
## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/
##
affinity: {}
# nodeAffinity:
# requiredDuringSchedulingIgnoredDuringExecution:
# nodeSelectorTerms:
# - matchExpressions:
# - key: kubernetes.io/e2e-az-name
# operator: In
# values:
# - e2e-az1
# - e2e-az2
## The remote_read spec configuration for Prometheus.
## ref: https://github.com/coreos/prometheus-operator/blob/master/Documentation/api.md#remotereadspec
remoteRead: []