Nathan J Mehl
fb18b68bc8
Update redis.existingSecretKey option to align w/ stable/redis ( #23725 )
...
Addresses https://github.com/helm/charts/issues/23719
Signed-off-by: Nathan J. Mehl <n@oden.io >
2020-09-08 23:23:54 -07:00
Mathew Wicks
6853cae0d1
[stable/airflow] version 7.6.0 ( #23651 )
...
* [stable/airflow] refactor templates
Signed-off-by: Mathew Wicks <thesuperzapper@users.noreply.github.com >
* [stable/airflow] allow multiple flower replicas
Signed-off-by: Mathew Wicks <thesuperzapper@users.noreply.github.com >
* [stable/airflow] use percentage for maxSurge on webserver
Signed-off-by: Mathew Wicks <thesuperzapper@users.noreply.github.com >
* [stable/airflow] implement PodDisruptionBudget for all resources
Signed-off-by: Mathew Wicks <thesuperzapper@users.noreply.github.com >
* [stable/airflow] ensure correct service ports are used
Signed-off-by: Mathew Wicks <thesuperzapper@users.noreply.github.com >
* [stable/airflow] create `flower.minReadySeconds` value
Signed-off-by: Mathew Wicks <thesuperzapper@users.noreply.github.com >
* [stable/airflow] increase default celery worker concurrency to 16
Signed-off-by: Mathew Wicks <thesuperzapper@users.noreply.github.com >
* [stable/airflow] make pods safe-to-evict by default
Signed-off-by: Mathew Wicks <thesuperzapper@users.noreply.github.com >
* [stable/airflow] update google-gke example
Signed-off-by: Mathew Wicks <thesuperzapper@users.noreply.github.com >
* [stable/airflow] fix incorrect example annotation syntax
Signed-off-by: Mathew Wicks <thesuperzapper@users.noreply.github.com >
* [stable/airflow] update autoscaling docs
Signed-off-by: Mathew Wicks <thesuperzapper@users.noreply.github.com >
* [stable/airflow] fix helm typos from refactor
Signed-off-by: Mathew Wicks <thesuperzapper@users.noreply.github.com >
* [stable/airflow] release 7.6.0
Signed-off-by: Mathew Wicks <thesuperzapper@users.noreply.github.com >
2020-09-01 00:55:51 -07:00
Mathew Wicks
e1d968475d
[stable/airflow] improve celery worker graceful shutdown ( #23504 )
...
Signed-off-by: Mathew Wicks <thesuperzapper@users.noreply.github.com >
2020-08-11 02:46:17 -07:00
Alex Begg
77c2f8c632
[stable/airflow] fixed flower extraConfigmapMounts ( #22747 )
...
Signed-off-by: Alex Begg <abegg@gocurrency.com >
2020-06-12 09:37:56 -07:00
Mathew Wicks
3e7dde8921
[stable/airflow] 7.1.1 hotfix ( #22523 )
...
Signed-off-by: Mathew Wicks <thesuperzapper@users.noreply.github.com >
2020-05-26 00:27:11 -07:00
Mathew Wicks
10c341f488
[stable/airflow] 7.1.0 release ( #22506 )
...
Signed-off-by: Mathew Wicks <thesuperzapper@users.noreply.github.com >
2020-05-25 05:35:11 -07:00
Mathew Wicks
23f3a53569
[stable/airflow] update to airflow 1.10.10 ( #22442 )
...
Signed-off-by: Mathew Wicks <thesuperzapper@users.noreply.github.com >
2020-05-20 03:38:19 -07:00
Stijn De Haes
bfb219ec4e
Add support for configuring pools in airflow ( #13305 )
...
Also removes ingress from the minikube values for easier local testing.
And added a small fix to the Notes.txt.
Signed-off-by: Stijn De Haes <stijndehaes@gmail.com >
2019-04-27 06:46:21 -07:00
Naseem
302585baad
Various ( #13313 )
...
Signed-off-by: Naseem <naseemkullah@gmail.com >
2019-04-26 08:24:30 -07:00
Stijn De Haes
3a8e1c6ead
Added airflow variables support ( #11990 )
...
Signed-off-by: Stijn De Haes <stijndehaes@gmail.com >
2019-03-13 18:00:41 -07:00
Gaetan Semet
b0e88db1dd
[incubator/airflow] Airflow Helm Chart ( #3959 )
...
* Naive import from gh:gsemet/kube-airflow
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* update chart with recent fixes
from https://github.com/gsemet/kube-airflow
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Update the templates from kube-airflow
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Update after review
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Sync with gsemet/kube-airflow
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Move to secrets
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Fixed some (find/replace?) errors.
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* fix env vars
add missing var name REDIS_USER
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* apply base64 encoding to secrets
delete obsolete var/secret REDIS_USER
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Remove trailing whitespace
Fix comment (postgres -> Redis)
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Add configurable serviceAccountName to workers
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Fix templates API versions
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Update instructions for embedded DAGs
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Rename worker/celery vars
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Fix NOTES.txt
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Remove 'v' from chart version
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Fix field serviceAccountName
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Fix a templating typo
There's a small typo in the template which I noticed when I added a custom Secrets volume for the scheduler. The YAML config accidentally works if the typo is on the last line. :)
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* fix celery replica config
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Add quotes around the numbers for celery workers
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Parametrize cpu and mem req/limits for worker statefulset
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Fix 'trailing spaces' lint issue
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* added the ability to toggle the scheduler -p option
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* missed the airflow namespace
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* moved command arguments to separate lines
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Headless service name should match statefulset serviceName
I had issues where Airflow could not find the log files, so I started looking into it and concluded that everything was working (hostname correct in task instance, could retrieve logs from worker internally) except for the DNS resolution.
This commit changes the StatefulSet serviceName suffix to *-worker* instead of *-workers*, so that it matches the Service name.
Now all is fine, the logs are shown.
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* compatibility with kube 2.7
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Configure persistance
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Configure persistance
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Support initcontainer git synchro
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Update configuration description in README
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Adding documentation for persistence.existingClaim
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* airflow.scheduler_do_pickle => dags.donot_pickle
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Injecting ~/.local/bin in PATH
so that the commands pip installed can be accessed by
a BashOperator
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Force disable xcom pickling
will be disabled in airflow 2.0
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* mkdir ~/.local/bin
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Use real use as base URL
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* more docs examples
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Typo fixes
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* export local PATH before pip install
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Hardcode local path
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* revert BASE_URL
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Reference proper enabled parameter for values.dags.init_container
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* bad order or install in scheduler
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* increase initial delay for webui to 6 min
maybe we might want it in the values.yaml?
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Fixed some more spots for Values.dags.init_container.enabled
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Doc for donot_pickle
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* notes about num_runs
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* remove load_examples from example
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* remove load_examples from example
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* airflow: Add support for ingress TLS termination
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* airflow: Allow customisation of worker Pod annotations
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* feature/secrets: add secrets as volume mount
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* feature/secrets: change variable to camel case; add worker secrets; add image pull secret
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* feature/secrets: change variables to camel case
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* feature/secrets: update array spacing
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* feature/secrets: update README
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* feature/secrets: add newline to ingresses
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* feature/secrets: update values
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Update airflow chart with:
- RBAC setup
- Ability to disable the redis+celery systems and allow for setting an alternative `executor`, such as the kubernetes executor
- Wire up the `serviceAccountName` to additional resources
- Update to latest airflow release
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Bump versions
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* 'existingClaim' must not be declared for the pvc template to work
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Support setting a custom postgresHost
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Fix apps/v1beta1
Signed-off-by: Maciej Bryński <maciej.brynski@xcaliber.com >
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Change minikube values
Signed-off-by: Maciej Bryński <maciej.brynski@xcaliber.com >
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Fix indendation
Signed-off-by: Maciej Bryński <maciej.brynski@xcaliber.com >
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* OWNERS file
Signed-off-by: Maciej Bryński <maciej.brynski@xcaliber.com >
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* chore/cleanup: fix spelling
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Move to stable
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* add std labels and component label
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* use template for chart label
Signed-off-by: Gaetan Semet <gaetan@xeberon.net >
* Fix reference to incubator
Signed-off-by: Reinhard Nägele <unguiculus@gmail.com >
2018-12-04 10:17:30 -08:00