Files
deprecated-helm-charts/stable/airflow/examples
Gaetan SemetandKubernetes Prow Robot b0e88db1dd [incubator/airflow] Airflow Helm Chart (#3959)
* Naive import from gh:gsemet/kube-airflow

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* update chart with recent fixes

from https://github.com/gsemet/kube-airflow

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Update the templates from kube-airflow

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Update after review

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Sync with gsemet/kube-airflow

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Move to secrets

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Fixed some (find/replace?) errors.


Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* fix env vars
add missing var name REDIS_USER

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* apply base64 encoding to secrets
delete obsolete var/secret REDIS_USER

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Remove trailing whitespace
Fix comment (postgres -> Redis)

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Add configurable serviceAccountName to workers

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Fix templates API versions

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Update instructions for embedded DAGs

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Rename worker/celery vars

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Fix NOTES.txt

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Remove 'v' from chart version

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Fix field serviceAccountName

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Fix a templating typo

There's a small typo in the template which I noticed when I added a custom Secrets volume for the scheduler. The YAML config accidentally works if the typo is on the last line. :)
Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* fix celery replica config

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Add quotes around the numbers for celery workers

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Parametrize cpu and mem req/limits for worker statefulset

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Fix 'trailing spaces' lint issue

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* added the ability to toggle the scheduler -p option

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* missed the airflow namespace

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* moved command arguments to separate lines

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Headless service name should match statefulset serviceName

I had issues where Airflow could not find the log files, so I started looking into it and concluded that everything was working (hostname correct in task instance, could retrieve logs from worker internally) except for the DNS resolution.

This commit changes the StatefulSet serviceName suffix to *-worker* instead of *-workers*, so that it matches the Service name.

Now all is fine, the logs are shown.
Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* compatibility with kube 2.7

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Configure persistance

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Configure persistance

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Support initcontainer git synchro

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Update configuration description in README

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Adding documentation for persistence.existingClaim

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* airflow.scheduler_do_pickle => dags.donot_pickle

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Injecting ~/.local/bin in PATH

so that the commands pip installed can be accessed by
a BashOperator

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Force disable xcom pickling

will be disabled in airflow 2.0

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* mkdir ~/.local/bin

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Use real use as base URL

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* more docs examples

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Typo fixes

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* export local PATH before pip install

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Hardcode local path

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* revert BASE_URL

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Reference proper enabled parameter for values.dags.init_container


Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* bad order or install in scheduler

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* increase initial delay for webui to 6 min

maybe we might want it in the values.yaml?

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Fixed some more spots for Values.dags.init_container.enabled


Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Doc for donot_pickle

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* notes about num_runs

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* remove load_examples from example

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* remove load_examples from example

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* airflow: Add support for ingress TLS termination

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* airflow: Allow customisation of worker Pod annotations

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* feature/secrets: add secrets as volume mount

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* feature/secrets: change variable to camel case; add worker secrets; add image pull secret

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* feature/secrets: change variables to camel case

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* feature/secrets: update array spacing

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* feature/secrets: update README

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* feature/secrets: add newline to ingresses

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* feature/secrets: update values

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Update airflow chart with:
- RBAC setup
- Ability to disable the redis+celery systems and allow for setting an alternative `executor`, such as the kubernetes executor
- Wire up the `serviceAccountName` to additional resources
- Update to latest airflow release

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Bump versions

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* 'existingClaim' must not be declared for the pvc template to work

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Support setting a custom postgresHost

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Fix apps/v1beta1

Signed-off-by: Maciej Bryński <maciej.brynski@xcaliber.com>

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Change minikube values

Signed-off-by: Maciej Bryński <maciej.brynski@xcaliber.com>

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Fix indendation

Signed-off-by: Maciej Bryński <maciej.brynski@xcaliber.com>

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* OWNERS file

Signed-off-by: Maciej Bryński <maciej.brynski@xcaliber.com>

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* chore/cleanup: fix spelling

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Move to stable

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* add std labels and component label

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* use template for chart label

Signed-off-by: Gaetan Semet <gaetan@xeberon.net>

* Fix reference to incubator

Signed-off-by: Reinhard Nägele <unguiculus@gmail.com>
2018-12-04 10:17:30 -08:00
..