* Adding airflow scheduler existing secret connections option
Signed-off-by: Lucas Garcia <kartgarcia@gmail.com>
* Updating Airflow README with information about scheduler.existingSecretConnections
Signed-off-by: Lucas Garcia <kartgarcia@gmail.com>
* Adding more examples and references to existingSecretConnections README
Signed-off-by: Lucas Garcia <kartgarcia@gmail.com>
* Bumping minor section in airflow Chart
Signed-off-by: Lucas Garcia <kartgarcia@gmail.com>
If the airflow chart is installed in a non-default namespace
in (at least) GKE, tasks using the KubernetesPodOperator will
be marked failed because the worker does not have the ability
to watch the event stream.
Signed-off-by: Nathan J. Mehl <n@oden.io>
* [stable/airflow] Remove connection before adding it
fixes#23499
The --delete command exits 0 (success) even if the connection does not
exist.
Signed-off-by: Vicken Simonian <vsimon@gmail.com>
* [stable/airflow] whitespace
Signed-off-by: Vicken Simonian <vsimon@gmail.com>
Changes:
* There was a security issue introduced in #22162 which caused the remote git ssh host to be implicitly trusted by default.
* In addition to the security issue, #22162 broke any user with a non-standard ssh repo port.
* This commit keeps the implicit trust if the user specifies git.sshKeyscan=true (with a default of false), in addition to generally cleaning up this part of the codebase.
Signed-off-by: Mathew Wicks <thesuperzapper@users.noreply.github.com>
* fix: postgres to postgresql for all variables
* fix: postgres to postgresql for all variables
* refactor: use recommended default function
Signed-off-by: carlosescura <carlosescura@gmail.com>
* Bump Airflow chart version
Signed-off-by: carlosescura <carlosescura@gmail.com>
* Revert README.md modification
Signed-off-by: carlosescura <carlosescura@gmail.com>
* fix: Add new Postgresql variables to readme
Signed-off-by: carlosescura <carlosescura@gmail.com>
* Added horizontal pod autoscaler for Celery workers.
Also added possibility to set "resources" for git-sync container, to be able to use hpa with pods it is attached to.
Signed-off-by: Maksym Shalak <mshalak@n-ix.com>
* Described how to use an autoscaler with memory metric
Signed-off-by: Maksym Shalak <mshalak@n-ix.com>
Fix#21951
Using `command` to call the script was overridding the
puckel/docker-airflow ENTRYPOINT that is building Airflow
specific env variables used to connect Airflow to a
PostgreSQL database.
Bypassing this ENTRYPOINT it thus preventing airflow to
connect to the proper database and the `airflow initdb`
command is not run on the proper database.
Using `args` as done at other place in the same chart
before calling `airflow initdb` is fixing this issue.
Signed-off-by: Pierre de Poulpiquet <pierre@whaly.io>
Co-authored-by: Pierre de Poulpiquet <pierre@whaly.io>
- Add an `airflow.preinitdb` option: if true, the initdb step
will be run as an initcontainer rather than inline in the
scheduler startup; this can work around dependencies in custom
db hooks that prevent initdb from running.
- fix a bug wherein if the git-clone initcontainer was not activated,
but the git-sync sidecar container was, the web and scheduler
deployments would fail due to not correctly creating the
git-clone-secret volume, which is needed by both the initcontainer
and the sidecar.
- Fix a similar bug for dag path mounting.
- Fix another cosmetic data type mismatch bug in values.yaml:
`airflow.pools` should be a string containing a JSON
object, not a yaml map.
Signed-off-by: Nathan J. Mehl <n@oden.io>
* Update README.md Postgres details to match values
The default value for postgresql.existingSecretKey in values.yaml does
not mention the key that the README.md example uses to create the
secret. This creates unnecessary confusion for anyone trying to use the
chart as it will install correctly but the pods will not run because the
secret cannot be found.
By updating the README.md content to match the values.yaml, installation
should hopefully be a little smoother for new users.
Signed-off-by: Remy Suen <remy.suen@gmail.com>
* Increase chart version in Chart.yaml
Signed-off-by: Remy Suen <remy.suen@gmail.com>
The init order change in https://github.com/helm/charts/pull/21368
was the wrong solution to the problem at hand (circular dependencies
between a custom db hook and initdb) and can create init failures
when installing with a blank db. Reverting.
Signed-off-by: Nathan J. Mehl <n@oden.io>
Matt Farina
Mathew Wicks
Scott Rigby
Dave Cavaletto
Cory Murphy
Lucas Garcia
Nathan J Mehl
Wei Sun
Vicken Simonian
Alex Begg
Ales Nosek
Alex Begg
Edward Vella
mshalak-nix
YehloandJoshua Leuenberger
Ihor KaharlichenkoandIhor Kaharlichenko
rjarsenault1101
Bernd Hansen
Carlos Escura
Pierre de PoulpiquetandPierre de Poulpiquet
memory
Remy Suen