From 52b019a467a6730a582eb52d84d3ceeb2f49b6cf Mon Sep 17 00:00:00 2001 From: Florent Delannoy Date: Wed, 16 Oct 2019 14:25:42 +0100 Subject: [PATCH] [stable/elasticsearch-curator] Make restartPolicy configurable (#17748) We're currently in a situation where our ES _sometimes_ fails to clear up indexes, especially under load, but will eventually succeed when the job is restarted. There are two approaches to this kind of issues as far as I understand: - Let the CronJob recreate a new Job resource, which will create a new Pod and succeed - Set the Job's restartPolicy to Always or OnFailure, and let the Job finish The first option is currently doable through this chart, but hits the default Prometheus alerting rules that will warn you if a Job resource has failed and is left in failed state for a while. This is of course a spurious warning as the CronJob has eventually succeeded... But it's there :( By giving the user the possibility to change the restartPolicy, we make that second option available to go around Prometheus' over-sensitivity. Note: I relaise it's also possible to delete failed Jobs by setting the `failedJobsHistoryLimit` to zero, but that would hide _actual_ failures as well as temporary flakiness. Signed-off-by: Florent Delannoy --- stable/elasticsearch-curator/Chart.yaml | 2 +- stable/elasticsearch-curator/README.md | 1 + stable/elasticsearch-curator/templates/cronjob.yaml | 2 +- stable/elasticsearch-curator/values.yaml | 1 + 4 files changed, 4 insertions(+), 2 deletions(-) diff --git a/stable/elasticsearch-curator/Chart.yaml b/stable/elasticsearch-curator/Chart.yaml index 6def65efd6..3754ca976a 100644 --- a/stable/elasticsearch-curator/Chart.yaml +++ b/stable/elasticsearch-curator/Chart.yaml @@ -2,7 +2,7 @@ apiVersion: v1 appVersion: "5.7.6" description: A Helm chart for Elasticsearch Curator name: elasticsearch-curator -version: 2.0.3 +version: 2.1.0 home: https://github.com/elastic/curator keywords: - curator diff --git a/stable/elasticsearch-curator/README.md b/stable/elasticsearch-curator/README.md index 2057b85dd3..7a83f12048 100644 --- a/stable/elasticsearch-curator/README.md +++ b/stable/elasticsearch-curator/README.md @@ -50,6 +50,7 @@ their default values. | `cronjob.concurrencyPolicy` | `Allow|Forbid|Replace` concurrent jobs | `nil` | | `cronjob.failedJobsHistoryLimit` | Specify the number of failed Jobs to keep | `nil` | | `cronjob.successfulJobsHistoryLimit` | Specify the number of completed Jobs to keep | `nil` | +| `cronjob.jobRestartPolicy` | Control the Job restartPolicy | `Never` | | `pod.annotations` | Annotations to add to the pod | {} | | `dryrun` | Run Curator in dry-run mode | `false` | | `env` | Environment variables to add to the cronjob container | {} | diff --git a/stable/elasticsearch-curator/templates/cronjob.yaml b/stable/elasticsearch-curator/templates/cronjob.yaml index 37274f6a80..52feefb2bf 100644 --- a/stable/elasticsearch-curator/templates/cronjob.yaml +++ b/stable/elasticsearch-curator/templates/cronjob.yaml @@ -45,7 +45,7 @@ spec: {{- if .Values.extraVolumes }} {{ toYaml .Values.extraVolumes | indent 12 }} {{- end }} - restartPolicy: Never + restartPolicy: {{ .Values.cronjob.jobRestartPolicy }} {{- if .Values.priorityClassName }} priorityClassName: "{{ .Values.priorityClassName }}" {{- end }} diff --git a/stable/elasticsearch-curator/values.yaml b/stable/elasticsearch-curator/values.yaml index 460f2a46ed..320c65404a 100644 --- a/stable/elasticsearch-curator/values.yaml +++ b/stable/elasticsearch-curator/values.yaml @@ -9,6 +9,7 @@ cronjob: concurrencyPolicy: "" failedJobsHistoryLimit: "" successfulJobsHistoryLimit: "" + jobRestartPolicy: Never pod: annotations: {}