diff --git a/stable/airflow/Chart.yaml b/stable/airflow/Chart.yaml index b80292c386..42a215dbe8 100644 --- a/stable/airflow/Chart.yaml +++ b/stable/airflow/Chart.yaml @@ -1,7 +1,7 @@ apiVersion: v1 description: Airflow is a platform to programmatically author, schedule and monitor workflows name: airflow -version: 7.1.3 +version: 7.1.4 appVersion: 1.10.10 icon: https://airflow.apache.org/_images/pin_large.png home: https://airflow.apache.org/ diff --git a/stable/airflow/README.md b/stable/airflow/README.md index f14030ebeb..150af46eb4 100644 --- a/stable/airflow/README.md +++ b/stable/airflow/README.md @@ -1,4 +1,4 @@ -# Airflow / Celery +# Airflow Helm Chart [Airflow](https://airflow.apache.org/) is a platform to programmatically author, schedule and monitor workflows. @@ -92,7 +92,7 @@ We expose the `scheduler.connections` value to allow specifying [Airflow Connect For example, to add a connection called `my_aws`: ```yaml -airflow: +scheduler: connections: - id: my_aws type: aws @@ -114,18 +114,18 @@ We expose the `scheduler.variables` value to allow specifying [Airflow Variables For example, to specify a variable called `environment`: ```yaml -airflow: +scheduler: variables: | { "environment": "dev" } ``` ### Airflow-Configs/Pools -We expose the `airflow.pools` value to allow specifying [Airflow Variables](https://airflow.apache.org/docs/stable/concepts.html#pools) at deployment time, these pools will be automatically imported by the Airflow scheduler when it starts up. +We expose the `scheduler.pools` value to allow specifying [Airflow Variables](https://airflow.apache.org/docs/stable/concepts.html#pools) at deployment time, these pools will be automatically imported by the Airflow scheduler when it starts up. For example, to create a pool called `example`: ```yaml -airflow: +scheduler: pools: | { "example": { @@ -326,9 +326,9 @@ extraManifests: ### Database-Configs/Initialization -If the value `airflow.initdb` is set to `true`, the airflow-scheduler container will run `airflow initdb` before starting the scheduler as part of its startup script. +If the value `scheduler.initdb` is set to `true`, the airflow-scheduler container will run `airflow initdb` before starting the scheduler as part of its startup script. -If the value `airflow.preinitdb` is set to `true`, the airflow-scheduler pod will run `airflow initdb` as an initContainer, before the git-clone initContainer (if that is enabled). +If the value `scheduler.preinitdb` is set to `true`, the airflow-scheduler pod will run `airflow initdb` as an initContainer, before the git-clone initContainer (if that is enabled). This is rarely necessary but can be so under certain conditions if your synced DAGs include custom database hooks that prevent `initdb` from running successfully. For example, if they have dependencies on variables that won't be present yet. The initdb initcontainer will retry up to 5 times before giving up. @@ -624,7 +624,7 @@ __Airflow DAGs Values:__ | Parameter | Description | Default | | --- | --- | --- | -| `dags.path` | the airflow dags folder | `/opt/airflow/logs` | +| `dags.path` | the airflow dags folder | `/opt/airflow/dags` | | `dags.doNotPickle` | whether to disable pickling dags from the scheduler to workers | `false` | | `dags.installRequirements` | install any Python `requirements.txt` at the root of `dags.path` automatically | `false` | | `dags.persistence.*` | configs for the dags PVC | `` | diff --git a/stable/airflow/values.yaml b/stable/airflow/values.yaml index 55a64dd4a4..0dd385f814 100644 --- a/stable/airflow/values.yaml +++ b/stable/airflow/values.yaml @@ -706,6 +706,10 @@ dags: ## install any Python `requirements.txt` at the root of `dags.path` automatically ## + ## WARNING: + ## - if set to true, and you are using `dags.git.gitSync`, you must also enable + ## `dags.initContainer` to ensure the requirements.txt is available at Pod start + ## installRequirements: false ## configs for the dags PVC @@ -850,7 +854,8 @@ dags: ## ## NOTE: ## - this is NOT required for the git-sync sidecar to work - ## - in most environments, you can leave this as false + ## - this is mostly used for when `dags.installRequirements` is true to ensure that + ## requirements.txt is available at Pod start ## enabled: false