Commit Graph
61 Commits
Author SHA1 Message Date
Ted Kaplan 8c00680aaf [stable/airflow] Allow specifying nodePort for Web service (#16099)
* Allow specifying nodePort for Web service

Signed-off-by: Ted Kaplan <edward.kaplan@nielsen.com>

* Update README.md

Signed-off-by: Ted Kaplan <edward.kaplan@nielsen.com>

* Only use nodePort value if service type is NodePort

Signed-off-by: Ted Kaplan <edward.kaplan@nielsen.com>

* Bump version

Signed-off-by: Ted Kaplan <edward.kaplan@nielsen.com>

* Fix template variable names

Signed-off-by: Ted Kaplan <edward.kaplan@nielsen.com>

* Update nodePort template variable

Update airflow.service.nodePort to airflow.service.nodePort.http as
requested by @gsemet

Signed-off-by: Ted Kaplan <edward.kaplan@nielsen.com>
2019-10-01 22:15:06 -07:00
Marcin Szymański 6f52cc5428 [stable/airflow] fix typo in environment variable name (#17558)
Signed-off-by: Marcin Szymanski <ms32035@gmail.com>
2019-10-01 01:59:32 -07:00
Maha Gamal 0c09a6dde2 [stable/airflow]: Added extraInitContainers option (#15690)
Signed-off-by: MahaGamal <mohaamer5@google.com>
2019-09-22 02:11:22 -07:00
Tin H Kyaw 18a742014f [stable/airflow] proper initializatons of variables for airflow chart (#17132)
* proper initializatons of variables

* `ingress.web.annotations` and `ingress.flower.annotations` should be initialized as maps
* `dags.git.secret` should be initialized as empty string

Signed-off-by: Tin Kyaw <tin@lsvp.com>

* bumped version to 4.0.9

Signed-off-by: Tin Kyaw <tin@lsvp.com>
2019-09-16 14:14:39 -07:00
Michael Cherepnev a8c435ee48 Add command line parameters quoting in create connections script in Airflow chart (#17062)
Signed-off-by: Mikhail Cherepnev <mikhail.cherepnev@gmail.com>
2019-09-11 02:05:24 -07:00
Joost Döbken 2b8b84f3fb bump version airflow container (#16646)
Signed-off-by: Joost Dobken <joost.dobken@pon.com>
2019-08-28 01:17:12 -07:00
Mohannad Albanayosy 87756a9f7f [stable/airflow] Option to specify postgres/redis secret key (#16615)
Signed-off-by: Mohannad Albanayosy <m.banayosi@gmail.com>
2019-08-26 08:28:21 -07:00
Mohannad Albanayosy ef50e8c711 [stable/airflow] Mount password when Postgres or Redis are enabled (#16030)
Signed-off-by: Mohannad Albanayosy <m.banayosi@gmail.com>
2019-08-23 02:47:18 -07:00
Sudip 59e43ac076 For adding pools, the if check should be on pools field and not variables field (#16521)
Signed-off-by: Sudip Datta <sudipdatta10@gmail.com>
2019-08-21 12:56:33 -07:00
Marcin Szymański abc9f77bcc [stable/airflow] add support for secrets on webserver (#16288)
Signed-off-by: Marcin Szymanski <ms32035@gmail.com>
2019-08-14 05:52:25 -07:00
Khwunchai Jaengsawang 3899135dd2 [stable/airflow] Add readinessProbe (#15138)
* Add readinessProbe

Signed-off-by: Khwunchai Jaengsawang <khwunchai.j@ku.th>

* Bump chart version

Signed-off-by: Khwunchai Jaengsawang <khwunchai.j@ku.th>
2019-08-13 01:12:00 -07:00
Maxim Mironenko 52bbcb787f [stable/airflow] Airflow connections values fix (#16159)
* [stable/airflow] Hotfix for `connections` default value

* connections should be a list by default, to avoid error trying overwrite map with list when passing actual list with connections. See: https://github.com/helm/charts/pull/10317#pullrequestreview-194739611

Signed-off-by: Maxim Mironenko <maxim@cloudposse.com>

* [stable/airflow] Hotfix for `connections` default value

* version bumped

Signed-off-by: Maxim Mironenko <maxim@cloudposse.com>
2019-08-07 23:15:17 -07:00
Samuel BéginandSamuel Begin c4b8c1140b [stable/airflow] Split node/affinity/toleration features for each component (#15898)
* make the affinity, nodeselector and tolerations features component specific instead of global

Signed-off-by: Sam Begin <samuel.begin@optelgroup.com>
Co-authored-by: Samuel Begin <begs1307@gmail.com>

* bump chart version and added migration steps

Signed-off-by: Sam Begin <samuel.begin@optelgroup.com>
2019-07-26 01:09:59 -07:00
Marcin Szymański 2bf5e63aa4 [stable/airflow] remove secret subpath (#15450)
Signed-off-by: Marcin Szymanski <ms32035@gmail.com>
2019-07-23 01:12:54 -07:00
Maha Gamal 27fb92ad0f [stable/airflow] update condition (#15712)
Signed-off-by: MahaGamal <mohaamer5@google.com>
2019-07-21 03:04:53 -07:00
Maha Gamal 3bc6e63734 [stable/airflow]: Added checking redis and postgres env when setting enable (#15667)
Signed-off-by: MahaGamal <mohaamer5@google.com>
2019-07-18 02:58:11 -07:00
Andrei 645f87f544 Fixed typos and mistakes in README (#15571)
Signed-off-by: Andrei Loginov <loginov.andrei@gmail.com>
2019-07-17 07:26:36 -07:00
Steven Anton c77af8ccfb Update to Airflow 1.10.3 (#14989)
Signed-off-by: Steven Anton <steven.m.anton@gmail.com>
2019-06-21 23:54:11 -07:00
Marcin Szymański 95572b7ff5 [stable/airflow] add option for initdb (#14853)
Signed-off-by: Marcin Szymanski <ms32035@gmail.com>
2019-06-19 04:30:34 -07:00
Steven Anton 158265ea08 [stable/airflow] Improve handling of existing secrets (#14715)
* Cleaning up the existing-secret handling for Airflow

Signed-off-by: Steven Anton <steven.m.anton@gmail.com>

* Bumping chart version

Signed-off-by: Steven Anton <steven.m.anton@gmail.com>

* Cleaning up the existing-secret handling for Airflow

Signed-off-by: Steven Anton <steven.m.anton@gmail.com>

* Bumping chart version

Signed-off-by: Steven Anton <steven.m.anton@gmail.com>

* Update the README

Signed-off-by: Steven Anton <steven.m.anton@gmail.com>

* A major overhaul of how secrets are handled

Signed-off-by: Steven Anton <steven.m.anton@gmail.com>

* Major version bump; adding documentation

Signed-off-by: Steven Anton <steven.m.anton@gmail.com>
2019-06-13 13:36:17 -07:00
Kevin Pullin 2de596cd1d Add livenessPath config to airflow web deployment (#14575)
Signed-off-by: Kevin Pullin <kevin.pullin@gmail.com>
2019-06-06 13:58:02 -07:00
Bernhard Rode fa7ba8d155 [stable/airflow] Fix unknown field scheduler (#13992)
* fix: unknown field "scheduler" in io.k8s.api.core.v1.ResourceRequirements

Signed-off-by: Bernhard Rode <mail@bernhardrode.de>

* chore: bump version

Signed-off-by: Bernhard Rode <mail@bernhardrode.de>
2019-06-03 07:42:33 -07:00
Jonathan Miles 32181cdc75 [stable/airflow]: align log path directories (#14339)
- Add missing config overrides for scheduler.child_process_log_directory and core.dag_processor_manager_log_location

Signed-off-by: Jonathan Miles <jonathan.miles@thetradedesk.com>
2019-05-30 13:24:23 -07:00
Jonathan Miles d837d0c9dc [stable/airflow]: allow subPath to be specified for DAG and log volumes + documentation (#14221)
- Fixed base_log_folder to be set to logs.path via environment variable

- Allow persistence.subPath to be specified for the DAG volume

- Allow logsPersistence.subPath to be specified for the log volume

- Documented how to use a single volume for both DAGs and logs

Signed-off-by: Jonathan Miles <jonathan.miles@thetradedesk.com>
2019-05-30 04:44:22 -07:00
Steven Anton 3c63cbe00a [stable/airflow] Airflow worker log persistence (#12803)
* Adding log persistence

Signed-off-by: Steven Anton <steven.m.anton@gmail.com>

* Adding a note on persistent logs in response to @maver1ck

Signed-off-by: Steven Anton <steven.m.anton@gmail.com>

* Updating Chart version

Signed-off-by: Steven Anton <steven.m.anton@gmail.com>
2019-05-29 08:52:33 -07:00
Jonathan Miles a7baff7cde [stable/airflow]: pass through annotations to service-web and service-flower (#14222)
- Added airflow.service.annotations and airflow.service.externalPort

- Added flower.service.{annotations,type,externalPort}

- Above decouples service-web and service-flower configuration

Signed-off-by: Jonathan Miles <jonathan.miles@thetradedesk.com>
2019-05-28 14:59:45 -07:00
Carlos Tadeu Panato Junior 6cd7ad6299 add apiVersion (#13769)
Signed-off-by: Carlos Panato <ctadeu@gmail.com>
2019-05-14 13:57:00 -07:00
Stijn De Haes 40637f335b [stable/airflow] Fix prometheusRule additionalLabels (#13683)
Signed-off-by: Stijn De Haes <stijndehaes@gmail.com>
2019-05-10 07:29:59 -07:00
Stijn De Haes eb9d6a8e54 [stable/airflow] Add support for prometheus rules (#13662)
Signed-off-by: Stijn De Haes <stijndehaes@gmail.com>
2019-05-09 14:04:52 -07:00
Paulo George Gomes Bezerra 82251b8b24 Fix configuration name (#12968)
Documentation is wrong.

Signed-off-by: Paulo George Gomes Bezerra <pgomes@xerpa.com.br>
2019-05-02 06:02:21 -07:00
Chia-Chun Tai 641f77402b [stable/airflow] Add checksum annotations to roll deployments when configmaps or secrets change (#13432)
Signed-off-by: Chia-Chun Tai <darkgerm@gmail.com>
2019-05-01 08:16:58 -07:00
Stijn De Haes bfb219ec4e Add support for configuring pools in airflow (#13305)
Also removes ingress from the minikube values for easier local testing.
And added a small fix to the Notes.txt.

Signed-off-by: Stijn De Haes <stijndehaes@gmail.com>
2019-04-27 06:46:21 -07:00
Naseem 302585baad Various (#13313)
Signed-off-by: Naseem <naseemkullah@gmail.com>
2019-04-26 08:24:30 -07:00
Stijn De Haes ef428cead7 Added support for configmap subPath (#12873)
Signed-off-by: Stijn De Haes <stijndehaes@gmail.com>
2019-04-06 13:04:26 -07:00
Maha Gamal 8f2a843af6 [stable/airflow] Update NOTES file (#12793)
Signed-off-by: MahaGamal <mohaamer5@google.com>
2019-04-03 11:06:11 -07:00
Maha Gamal 47d9a04cd6 [stable/airflow] Support affinity, nodeSelector and toleration for a… (#11690)
*  [stable/airflow] Support affinity, nodeSelector and toleration for airflow & worker

Signed-off-by: Maha <you@example.com>

* update chart version

Signed-off-by: MahaGamal <mohaamer5@google.com>
2019-04-03 08:37:43 -07:00
Brian Nutt bf708989c1 [stable/airflow] Update redis requirements version (#12740)
Enables the ability to set sysctlImage.enabled values can make some changes in the kernel of the host machine to make redis work as expected, in particular increasing the somaxconn value and disabling transparent huge pages.

Signed-off-by: Brian Nutt <brian@slice.com>
2019-04-03 02:25:43 -07:00
Brian Nutt 0691f0f3e2 [stable/airflow]: Configurable initialDelay for starting webserver and livenessprobe (#12683)
When running locally, it is nice to have a delay initially to make sure postgres and redis are available. However, in the case you are interacting with an external redis or postgres, it is expected that they're already available, so the wait time isn't necessary.

This change makes the intial sleep delay before starting airflow configurable and the initialDelay before a healthcheck configurable so it possible to start up the webserver faster.

Signed-off-by: Brian Nutt <brian@slice.com>
2019-03-30 01:09:08 -07:00
Stijn De Haes 9287ccf7cf Added support for servicemonitor and updated the documentation accordingly. (#12276)
Signed-off-by: Stijn De Haes <stijndehaes@gmail.com>
2019-03-15 07:04:57 -07:00
Stijn De Haes 3a7e22cd4a added resources to airflow components (#11921)
Signed-off-by: Stijn De Haes <stijndehaes@gmail.com>
2019-03-14 07:02:26 -07:00
Stijn De Haes 3a8e1c6ead Added airflow variables support (#11990)
Signed-off-by: Stijn De Haes <stijndehaes@gmail.com>
2019-03-13 18:00:41 -07:00
Steve Larkin cfdc2bded7 [stable/airflow] Add container and volume extension points (#11775)
* [stable/airflow] Add extraContainers/VolumeMounts/Volumes

Signed-off-by: Steve Larkin <steve.larkin@sony.com>

* [stable/airflow] Step version

Signed-off-by: Steve Larkin <steve.larkin@sony.com>
2019-02-27 08:18:49 -08:00
Ryan Greget 71530eb172 Airflow role fix (#11762)
* Update to stable/airflow role.yaml to grant permissions needed for airflow xcom_push to stream out of the sidecar.

Signed-off-by: Ryan Greget <rgreget@gmail.com>

* Version bump.

Signed-off-by: Ryan Greget <rgreget@gmail.com>
2019-02-26 15:47:58 -08:00
Arthur Clément 9f10c2d71d [stable/airflow] fix expired documentation link in values.yml (#11700)
Signed-off-by: Arthur Clément <arthur.clement@protonmail.com>
2019-02-25 07:08:06 -08:00
Steve Larkin ccecf3559e [stable/airflow] Configurable web and scheduler pod annotaitons (#11520)
* [stable/airflow] Fix whitespace in configuration table

Signed-off-by: Steve Larkin <steve.larkin@sony.com>

* [stable/airflow] workers.pod.annotations => workers.podAnnotations

This follows the standard used elsewhere, such as in the Kibana chart.
Note that this introduces a break in backwards compatibility.

Signed-off-by: Steve Larkin <steve.larkin@sony.com>

* [stable/airflow] Fix typo in README

Signed-off-by: Steve Larkin <steve.larkin@sony.com>

* [stable/airflow] Add annotations for Web/Scheduler/Worker Pods

Signed-off-by: Steve Larkin <steve.larkin@sony.com>

* [stable/airflow] Step Chart version and describe upgrade path

Signed-off-by: Steve Larkin <steve.larkin@sony.com>
2019-02-24 13:01:34 -08:00
Andrew Meier 24d26fed48 Fix/airflow helper (#11614)
* fix/airflow-helper: closes #5341 update assignment operator

Signed-off-by: Andrew Meier <ameier38@gmail.com>

* fix/airflow-helper: bump chart version

Signed-off-by: Andrew Meier <ameier38@gmail.com>

* fix/airflow-helper: bump chart version

Signed-off-by: Andrew Meier <ameier38@gmail.com>
2019-02-23 12:01:43 -08:00
Ken Wronkiewicz b1e6434763 Adding optional additional airflow ConfigMaps (#11632)
* Adding optional additional airflow ConfigMaps

- Cribbed off of how the Grafana chart works
- I needed a way to deploy a set of config files to the DAGs but separate.

Signed-off-by: Ken Wronkiewicz <kwronkiewicz@salesforce.com>

* Accidentally a mistake.

Signed-off-by: Ken Wronkiewicz <kwronkiewicz@salesforce.com>

* Upgrade to 1.0

Signed-off-by: Wirehead <wh-github@wirewd.com>
2019-02-22 08:29:30 -08:00
Steve Larkin e785f60807 [stable/airflow] Remove dependency on initContainer for creating connections (#11495)
Signed-off-by: Steve Larkin <steve.larkin@sony.com>
2019-02-18 08:35:53 -08:00
Jason White 5863186a81 [airflow] allow multiple existing secrets to be used (#11428)
* [airflow] allow multiple existing secrets to be used
Signed-off-by: Jason White <jwhite@neighborly.com>

Signed-off-by: Jason White <jwhite@neighborly.com>

* remove trailing spaces

Signed-off-by: Jason White <jwhite@neighborly.com>
2019-02-14 09:31:54 -08:00
Massimo Gaggero e0119b61ca Implements #11353 : [stable/airflow] install-requirements.sh script s… (#11357)
* Implements #11353 : [stable/airflow] install-requirements.sh script should be always available

Signed-off-by: Massimo Gaggero <massimo.gaggero@crs4.it>

* - fixes scripts volume creation issue

Signed-off-by: Massimo Gaggero <massimo.gaggero@crs4.it>

* - updates version

Signed-off-by: Massimo Gaggero <massimo.gaggero@crs4.it>
2019-02-12 09:09:42 -08:00