mirror of
https://github.com/helm/charts.git
synced 2026-08-23 22:37:45 +00:00
[stable/airflow] be able to change git sync target (#21217)
Helpful when you want to sync to a subdir Signed-off-by: Steffen Tautenhahn <stevie-@users.noreply.github.com>
This commit is contained in:
Steffen Tautenhahn
@@ -1,7 +1,7 @@
|
||||
apiVersion: v1
|
||||
description: Airflow is a platform to programmatically author, schedule and monitor workflows
|
||||
name: airflow
|
||||
version: 6.1.1
|
||||
version: 6.2.1
|
||||
appVersion: 1.10.4
|
||||
icon: https://airflow.apache.org/_images/pin_large.png
|
||||
home: https://airflow.apache.org/
|
||||
|
||||
@@ -223,7 +223,7 @@ This approach has the additional advantage of keeping secrets outside of the Hel
|
||||
|
||||
It is possible to specify additional environment variables using the same format as in a pod's `.spec.containers.env` definition.
|
||||
These environment variables will be mounted in the web, scheduler, and worker pods.
|
||||
You can use this feature to pass additional secret environment variables to Airflow.
|
||||
You can use this feature to pass additional secret environment variables to Airflow.
|
||||
|
||||
Here is a simple example showing how to pass in a Fernet key and LDAP password.
|
||||
Of course, for this example to work, both the `airflow` and `ldap` Kubernetes secrets must already exist in the proper namespace; be sure to create those before running Helm.
|
||||
@@ -293,7 +293,7 @@ For example, this will create a secret named `my-git-secret` from your ed25519 k
|
||||
|
||||
This set of instructions will enable you to clone your repository in the initContainer git-clone.sh script through an ssh connection.
|
||||
|
||||
To do this you must have `dags.initContainer.enabled` set to true. Then you need to have the following set. `dags.git.url` This is the repository of your dags. `dags.git.ref` This is the branch with your dags on your repo. `dags.git.secret` This is the name of the secret cotaining your private ssh key. With this you need `dags.git.privateKeyName`, this is the name of the private key in the secret mounted in the keys directory on the initContainer. The last variable you need is this `dags.git.repoHost`. This is the host of your repo, for example if hosted on gitlab set the value as gitlab.com and if github put github.com. This enables us to tell ssh to use our private key when cloning otherwise we would recieve an unable to recognize host bug.
|
||||
To do this you must have `dags.initContainer.enabled` set to true. Then you need to have the following set. `dags.git.url` This is the repository of your dags. `dags.git.ref` This is the branch with your dags on your repo. `dags.git.secret` This is the name of the secret cotaining your private ssh key. With this you need `dags.git.privateKeyName`, this is the name of the private key in the secret mounted in the keys directory on the initContainer. The last variable you need is this `dags.git.repoHost`. This is the host of your repo, for example if hosted on gitlab set the value as gitlab.com and if github put github.com. This enables us to tell ssh to use our private key when cloning otherwise we would recieve an unable to recognize host bug.
|
||||
|
||||
### Embedded DAGs
|
||||
|
||||
@@ -321,7 +321,7 @@ You can also persist logs using a `PersistentVolume` using `logsPersistence.enab
|
||||
To use an existing volume, pass the name to `logsPersistence.existingClaim`.
|
||||
Otherwise, a new volume will be created.
|
||||
|
||||
Note that it is also possible to persist logs by mounting a `PersistentVolume` to the log directory (`/usr/local/airflow/logs` by default) using `airflow.extraVolumes` and `airflow.extraVolumeMounts`.
|
||||
Note that it is also possible to persist logs by mounting a `PersistentVolume` to the log directory (`/usr/local/airflow/logs` by default) using `airflow.extraVolumes` and `airflow.extraVolumeMounts`.
|
||||
|
||||
Refer to the `Mount a Shared Persistent Volume` section above for details on using persistent volumes.
|
||||
|
||||
@@ -343,7 +343,7 @@ To be able to expose metrics to prometheus you need install a plugin, this can b
|
||||
This exposes dag and task based metrics from Airflow.
|
||||
For service monitor configuration see the generic [Helm chart Configuration](#helm-chart-configuration).
|
||||
|
||||
## Additional manifests
|
||||
## Additional manifests
|
||||
|
||||
It is possible to add additional manifests into a deployment, to extend the chart. One of the reason is to deploy a manifest specific to a cloud provider ( BackendConfig on GKE for example ).
|
||||
|
||||
@@ -469,6 +469,8 @@ The following table lists the configurable parameters of the Airflow chart and t
|
||||
| `dags.initContainer.image.repository` | Init container Docker image. | `alpine/git` |
|
||||
| `dags.initContainer.image.tag` | Init container Docker image tag. | `1.0.4` |
|
||||
| `dags.initContainer.installRequirements` | auto install requirements.txt deps | `true` |
|
||||
| `dags.initContainer.mountPath` | Mountpath inside init container for dags | `/dags` |
|
||||
| `dags.initContainer.syncSubPath` | Path inside init container used to sync/clone git repo to; appended to `dags.initContainer.mountPath` | `` |
|
||||
| `dags.git.url` | url to clone the git repository | nil |
|
||||
| `dags.git.ref` | branch name, tag or sha1 to reset to | `master` |
|
||||
| `dags.git.secret` | name of a secret containing an ssh deploy key | nil |
|
||||
|
||||
@@ -84,14 +84,14 @@ spec:
|
||||
args:
|
||||
- "{{ .Values.dags.git.url }}"
|
||||
- "{{ .Values.dags.git.ref }}"
|
||||
- "/dags"
|
||||
- "{{ .Values.dags.initContainer.mountPath }}{{ .Values.dags.initContainer.syncSubPath | default "" }}"
|
||||
- "{{ .Values.dags.git.repoHost}}"
|
||||
- "{{ .Values.dags.git.privateKeyName }}"
|
||||
volumeMounts:
|
||||
- name: git-clone
|
||||
mountPath: /usr/local/git
|
||||
- name: dags-data
|
||||
mountPath: /dags
|
||||
mountPath: "{{ .Values.dags.initContainer.mountPath }}"
|
||||
{{- if .Values.dags.git.secret }}
|
||||
- name: git-clone-secret
|
||||
mountPath: /keys
|
||||
@@ -118,7 +118,7 @@ spec:
|
||||
args:
|
||||
- "{{ .Values.dags.git.url }}"
|
||||
- "{{ .Values.dags.git.ref }}"
|
||||
- "/dags"
|
||||
- "{{ .Values.dags.initContainer.mountPath }}"
|
||||
- "{{ .Values.dags.git.repoHost}}"
|
||||
- "{{ .Values.dags.git.privateKeyName }}"
|
||||
- "{{ .Values.dags.git.gitSync.refreshTime }}"
|
||||
@@ -126,7 +126,7 @@ spec:
|
||||
- name: git-clone
|
||||
mountPath: /usr/local/git
|
||||
- name: dags-data
|
||||
mountPath: /dags
|
||||
mountPath: "{{ .Values.dags.initContainer.mountPath }}"
|
||||
{{- if .Values.dags.git.secret }}
|
||||
- name: git-clone-secret
|
||||
mountPath: /keys
|
||||
|
||||
@@ -81,14 +81,14 @@ spec:
|
||||
args:
|
||||
- "{{ .Values.dags.git.url }}"
|
||||
- "{{ .Values.dags.git.ref }}"
|
||||
- "/dags"
|
||||
- "{{ .Values.dags.initContainer.mountPath }}{{ .Values.dags.initContainer.syncSubPath | default "" }}"
|
||||
- "{{ .Values.dags.git.repoHost}}"
|
||||
- "{{ .Values.dags.git.privateKeyName }}"
|
||||
volumeMounts:
|
||||
- name: git-clone
|
||||
mountPath: /usr/local/git
|
||||
- name: dags-data
|
||||
mountPath: /dags
|
||||
mountPath: "{{ .Values.dags.initContainer.mountPath }}"
|
||||
{{- if .Values.dags.git.secret }}
|
||||
- name: git-clone-secret
|
||||
mountPath: /keys
|
||||
@@ -109,7 +109,7 @@ spec:
|
||||
args:
|
||||
- "{{ .Values.dags.git.url }}"
|
||||
- "{{ .Values.dags.git.ref }}"
|
||||
- "/dags"
|
||||
- "{{ .Values.dags.initContainer.mountPath }}{{ .Values.dags.initContainer.syncSubPath | default "" }}"
|
||||
- "{{ .Values.dags.git.repoHost}}"
|
||||
- "{{ .Values.dags.git.privateKeyName }}"
|
||||
- "{{ .Values.dags.git.gitSync.refreshTime }}"
|
||||
@@ -117,7 +117,7 @@ spec:
|
||||
- name: git-clone
|
||||
mountPath: /usr/local/git
|
||||
- name: dags-data
|
||||
mountPath: /dags
|
||||
mountPath: "{{ .Values.dags.initContainer.mountPath }}"
|
||||
{{- if .Values.dags.git.secret }}
|
||||
- name: git-clone-secret
|
||||
mountPath: /keys
|
||||
|
||||
@@ -85,14 +85,14 @@ spec:
|
||||
args:
|
||||
- "{{ .Values.dags.git.url }}"
|
||||
- "{{ .Values.dags.git.ref }}"
|
||||
- "/dags"
|
||||
- "{{ .Values.dags.initContainer.mountPath }}{{ .Values.dags.initContainer.syncSubPath | default "" }}"
|
||||
- "{{ .Values.dags.git.repoHost}}"
|
||||
- "{{ .Values.dags.git.privateKeyName }}"
|
||||
volumeMounts:
|
||||
- name: git-clone
|
||||
mountPath: /usr/local/git
|
||||
- name: dags-data
|
||||
mountPath: /dags
|
||||
mountPath: "{{ .Values.dags.initContainer.mountPath }}"
|
||||
{{- if .Values.dags.git.secret }}
|
||||
- name: git-clone-secret
|
||||
mountPath: /keys
|
||||
@@ -113,7 +113,7 @@ spec:
|
||||
args:
|
||||
- "{{ .Values.dags.git.url }}"
|
||||
- "{{ .Values.dags.git.ref }}"
|
||||
- "/dags"
|
||||
- "{{ .Values.dags.initContainer.mountPath }}{{ .Values.dags.initContainer.syncSubPath | default "" }}"
|
||||
- "{{ .Values.dags.git.repoHost}}"
|
||||
- "{{ .Values.dags.git.privateKeyName }}"
|
||||
- "{{ .Values.dags.git.gitSync.refreshTime }}"
|
||||
@@ -121,7 +121,7 @@ spec:
|
||||
- name: git-clone
|
||||
mountPath: /usr/local/git
|
||||
- name: dags-data
|
||||
mountPath: /dags
|
||||
mountPath: "{{ .Values.dags.initContainer.mountPath }}"
|
||||
{{- if .Values.dags.git.secret }}
|
||||
- name: git-clone-secret
|
||||
mountPath: /keys
|
||||
|
||||
@@ -569,6 +569,11 @@ dags:
|
||||
pullPolicy: IfNotPresent
|
||||
## install requirements.txt dependencies automatically
|
||||
installRequirements: true
|
||||
## Path to mount dags-data volume to
|
||||
mountPath: "/dags"
|
||||
## Path to add to `mountPath` where we clone/sync git to. Include leading /
|
||||
## example: "/subdirWithDags"
|
||||
syncSubPath: ""
|
||||
|
||||
##
|
||||
## Configure logs
|
||||
|
||||
Reference in New Issue
Block a user