The original quoting prevented the globbing expansion to happen,
making workdir content persist across volume restarts. This PR
modifies the behavior to allow for the expansion.
Note: this is not a full-featured fix to this issue (e.g. doesn't
handle dotfiles) and depending on how/whether concourse does any
shell escaping, this might need some further work.
Signed-off-by: Milan Plzik <milan.plzik@ceai.io>
Previously (https://github.com/helm/charts/pull/14545), configuration
was added to allow folks to leverage CredHub as a credential management
for Concourse.
While that allowed people to have that functionality, it was only
partially implemented - mutual TLS was still missing.
This commit aims at addressing that by introducing a
`authenticationMode` key that allows one to either specify certificates
for the authentication, or certificates:
```yaml
concourse:
web:
credhub:
enabled: true
authenticationMode: "secrets"
secrets:
credhubClientId: $client_id
credhubClientSecret: $client_secret
```
or
```yaml
concourse:
web:
credhub:
enabled: true
authenticationMode: "mtls"
secrets:
credhubClientKey: $key
credhubClientCert: $cert
```
In case none of them are set, it fails to render.
Signed-off-by: Ciro S. Costa <cscosta@pivotal.io>
Co-authored-by: Bishoy Youssef <byoussef@pivotal.io>
- removed references to the worker public key from the worker deployment
- on the Concourse side, added more tests to make sure this is not used
anywhere (see: concourse/concourse#4133)
fixes: #13019
Signed-off-by: Zoe Tian <ztian@pivotal.io>
Co-authored-by: Bishoy Youssef <byoussef@pivotal.io>
Co-authored-by: Ciro S. Costa <cscosta@pivotal.io>
Previously, the `concourse.web.tsa.address` template used to take
`tsaNodePort` (the `nodePort` set for the TSA endpoint in the web
service) as part of the address that the worker should use to connect to
it, but that is just wrong as the `worker`s are fully able to connect
through regular target ports within the cluster (which is the case when
running a full deployment).
Signed-off-by: Ciro S. Costa <cscosta@pivotal.io>
Co-authored-by: Bishoy Youssef <byoussef@pivotal.io>
When a `dind` (Docker in Docker) image is used with btrfs to e.g. run
integration tests as per https://hub.docker.com/r/amidos/dcind/ in some
occasions like job errors or interruptions the btrfs subvolumes are left
not cleaned.
So what happens then is that when the `rm -rf /concourse-worker-dir`
runs it fails with `Operation not permitted` error which then causes
`Init:Error` and ends in `Init:CrashLoopBackOff`.
The solution is to take that into account and properly delete all of the
btrfs subvolumes.
This can be achieved either with the suggested script or with the mount
option [user_subvol_rm_allowed](https://askubuntu.com/questions/509292/how-to-set-user-subvol-rm-allowed-capability)
that is tricky to apply or with that delete script that seems as a
better option.
Signed-off-by: Ciro S. Costa <cscosta@pivotal.io>
Co-authored-by: Radoslav Kirilov <rkirilow@gmail.com>
Co-authored-by: Taylor Silva <tsilva@pivotal.io>
- Changed web.tsa.host to web.tsa.hosts which now takes a list of
hostnames and ports
- For web & worker deployments (default state) we read the port number
from the web's config values (bindPort)
Signed-off-by: Ciro S. Costa <cscosta@pivotal.io>
Co-authored-by: Taylor Silva <tsilva@pivotal.io>
Signed-off-by: Zoe Tian <ztian@pivotal.io>
* [stable/concourse]: adding web default values to values.yml
- setting the default values as per the concourse binary
- for the issue: helm/charts#12221
Signed-off-by: Sameer Vohra <svohra@pivotal.io>
Co-authored-by: Bishoy Youssef <byoussef@pivotal.io>
* [stable/concourse] Add default worker values to values.yml
- Added the default values as per the concourse binary
- for the issue: helm/charts#12221
Signed-off-by: Sameer Vohra <svohra@pivotal.io>
Co-authored-by: Bishoy Youssef <byoussef@pivotal.io>
* [stable/concourse] Update Chart, values & deployment
- bump chart version
- add defaults for postgres, syslog, credentials
- removed GC override flag as defaults are set
- fixed syslog hostname typo
Signed-off-by: Bishoy Youssef <byoussef@pivotal.io>
Co-authored-by: Sameer Vohra <svohra@pivotal.io>
There's been a long time since the last upgrade that we did to the
subchart that Concourse optionally depends on.
With the upgrade, some values set in the `postgresql` chart changed. As
those are exposed in `concourse`'s `values` too (and we version our
Chart based on its public interface - values.yaml), this made it deserve
a major version bump.
For those users not making use of the subchart (or changing the default
values), no changes ar necessary.
For those who *are* configuring `postgresql.*` values - changes will
almost certainly be necessary.
closes#11610
Signed-off-by: Ciro S. Costa <cscosta@pivotal.io>
* [stable/concourse] Update flags for next Concourse release
- Adds `CONCOURSE_CLUSTER_NAME` (concourse.web.clusterName)
- https://github.com/concourse/concourse/pull/3736<Paste>
Signed-off-by: Ciro S. Costa <cscosta@pivotal.io>
* [stable/concourse] add generic secret configurations
With the introduction generic caching, we no longer need the
vault-specific cacheing flags, while at the same time, having the need
of providing new ones (generic).
https://github.com/concourse/concourse/pull/3628
Signed-off-by: Ciro S. Costa <cscosta@pivotal.io>
* [stable/concourse] add auditing flags
With the introduction of auditing capabilities (see Auditor#3577 [1]),
it's now possible to configure in a per-subsystem basis which API
requests an installation should audit.
This commit adds the necessary support for such flags.
[1]: https://github.com/concourse/concourse/pull/3577
Signed-off-by: Ciro S. Costa <cscosta@pivotal.io>
* [stable/concourse] update image version to 5.2.0
Signed-off-by: Ciro S. Costa <cscosta@pivotal.io>
Co-authored-by: Krishna Mannem <kmannem@pivotal.io>
* [stable/concourse] Add time based build log retention flags
concourse/concourse#3560
Signed-off-by: Ciro S. Costa <cscosta@pivotal.io>
Co-authored-by: Krishna Mannem <kmannem@pivotal.io>
* [stable/concourse] add support for externalGardenUrl
With the addition of `external-garden-url` to the `concourse worker`
command, one is now able to reference a non-embedded garden server to be
used as the target for container creation.
https://github.com/concourse/concourse/pull/3806
Signed-off-by: Ciro S. Costa <cscosta@pivotal.io>
* Add possibility to set strategy for rollingUpdates (web)
Signed-off-by: Yann David <davidyann88@gmail.com>
* Bump minor instead of patch
Signed-off-by: Yann David <davidyann88@gmail.com>
* stable/concourse: separate worker, web deployments
- Added .Values.Web.enabled and .Values.worker.enabled (with default to
true).
- if only .Values.web.enable is enabled: only web resources are going to
be created, as well as secrets namespace.
- if only .Values.worker.enable is enabled: only worker resources are going to
be created.
- moved the worker specific and web specific secrets each to a seprate
file and secrets object.
- added .Values.concourse.worker.tsa.port and utilised
.Values.concourse.worker.host in order to alow the user to set the
`CONCOURSE_TSA_HOST` easily.
- removed the template `concourse.concourse.fullname` as it is not used
anymore.
- bumped the chart version to 6.0.0, as this adds new ways to use the
chart.
- added the new variables to the README doc.
for the issue: helm/charts#11280
Signed-off-by: Taylor Silva <tsilva@pivotal.io>
Co-authored-by: Bishoy Youssef <byoussef@pivotal.io>
Signed-off-by: Taylor Silva <tsilva@pivotal.io>
* [stable/concourse] bumping chart version and updating the Readme
Signed-off-by: Taylor Silva <tsilva@pivotal.io>
Co-authored-by: Bishoy Youssef <byoussef@pivotal.io>
* Adds a required check yaml for having either the web or worker enabled
Signed-off-by: Bishoy Youssef <byoussef@pivotal.io>
* Adds required for concourse.worker.tsa in case of worker only deployments
Signed-off-by: Bishoy Youssef <byoussef@pivotal.io>
* updates readme
Signed-off-by: Bishoy Youssef <byoussef@pivotal.io>
* adds changelog file
Signed-off-by: Bishoy Youssef <byoussef@pivotal.io>
* Apply suggestions from code review
Signed-off-by: Bishoy Youssef <byoussef@pivotal.io>
- move the worker preStop script to a configMap
- add a wait for the worker process to exit in the preStop script
Signed-off-by: Bishoy Youssef <byoussef@pivotal.io>
Co-authored-by: Taylor Silva <tsilva@pivotal.io>
Signed-off-by: Bishoy Youssef <byoussef@pivotal.io>
* Bumped major chart version
Signed-off-by: Tarek Mokhtar <tareks01@yahoo.co.nz>
* Added documentation to explain how authentication paramters work.
Signed-off-by: Tarek Mokhtar <tareks01@yahoo.co.nz>
* Added new keyauth variables and conditionals to match them for secrets manager and systems manager. Session token must now be explictly enabled if used.
Signed-off-by: Tarek Mokhtar <tareks01@yahoo.co.nz>
* Fixed typo with awsSsm paramter name
Signed-off-by: Tarek Mokhtar <tareks01@yahoo.co.nz>
* Fixed minor version
Signed-off-by: Tarek Mokhtar <tareks01@yahoo.co.nz>
* Uncommented session token flag and set default to false.
Signed-off-by: Tarek Mokhtar <tareks01@yahoo.co.nz>
* Added missing backticks that broke formatting
Signed-off-by: Tarek Mokhtar <tareks01@yahoo.co.nz>
* [stable/concourse] Easier way to add team workers
- Previously, in order to add an external worker to the web deployment,
worker keys were to be added as a seprate secret or a config map and
then mounted as an addtional volume to the web pod.
- With this feature, only team names and public keys are to be added to
the secrets as an array and the creation of secrets and mounting the
volumes is done automatically.
- Also, removed `.Values.concourse.Web.teamAuthorizedKeys` as it won't
be used anymore.
Signed-off-by: Bishoy Youssef <byoussef@pivotal.io>
* fixing per Ciro's comments
Signed-off-by: Bishoy Youssef <byoussef@pivotal.io>
* concourse/stable bump major version
- also, fixing linting error.
Signed-off-by: Bishoy Youssef <byoussef@pivotal.io>
* adds security caveats and extra reads
Signed-off-by: Bishoy Youssef <byoussef@pivotal.io>
* rename teamKeys to teamAuthorizedKeys
Signed-off-by: Bishoy Youssef <byoussef@pivotal.io>
* [stable/concourse] Upgrade to 5.0 (breaking)
This commit provides the necessary updates to fully support the latest
version of Concourse: 5.0.
BREAKING:
- `concourse.web.auth.mainTeam.allowAllUsers`, which was previously used
to allow any logged user to be part of `main`, has now been removed;
- `concourse.worker.bindDebugPort` is now
`concourse.worker.debugBindPort`;
- `concourse.worker.baggageclaim.bindDebugPort` is now
`concourse.worker.baggageclaim.debugBindPort`;
- `concourse.worker.garden.*` variables that would previously turn into
environment variables that got translated to `gdn`-specfic tunings are
not values that can be set through `concourse.worker.garden` anymore.
These can still be provided through environment variables though -
`CONCOURSE_GARDEN_...` works as before.
DEPRECATIONS:
- `concourse.worker.peerIp` is not used anymore;
- `worker.fatalErrors` are not used anymore in favor of worker
healthchecks through a specific port (`worker-hc`) in the worker
container.
ENHANCEMENTS:
- reduced complexity in the liveness probe for the worker, removing the
need for custom scripts to verify its health;
- bitbucket cloud is now supported through
`secrets.bitbucket-cloud-client*` and other `concourse.web.*`
configurations;
- AWS region settings can now be configured at environment-var level
instead of arguments;
- garden configuration can be specified through `ini` files that can be
passed through `configmap`s;
- RBAC configuration for local users can be specified through
`configmap`s
- ability to opt-out of the automatic removal of
`concourse.worker.workDir` contents
- ability to specify whether `land`ing or `retire`ing should be
performed when terminating the worker pod (through the configuration
of `worker.shutdownSignal`);
- ability to specify custom readinessProbe for the worker;
- more consistent `values.yaml` examples and documentation.
Concourse-specific release notes (that are do not affect the Chart
structure itself) can be found in the Concourse website:
- https://concourse-ci.org/download.html#v500
Signed-off-by: Ciro S. Costa <cscosta@pivotal.io>
* [stable/concourse] Updates README to match values for 5.0
Signed-off-by: Ciro S. Costa <cscosta@pivotal.io>
The `values.yaml` for `stable/concourse` didn't look much consistent,
having some values with totally different commenting formats compared
to others.
This commit also improves the documentation around some of those values
that are not very easy to infer what they're all about.
By making the values uncommented we can leverage tools that parse yaml
files to make sure we have all variables documented.
He we also remove an unused debug values file and fix few values check.
With the use of default values under `values.yaml` as opposed to
commented fields, few values (like default `storageClass`) needed to be
updated, as well as adding few checks.
Also, Previously we allowed the creation of objects that wouldn't pass
`kubeval`'s validation. Now `kubeval` is ok with all of our objects.
Signed-off-by: Ciro S. Costa <cscosta@pivotal.io>
* Allow namespace creation independently of rbac
At the moment, when defining `concourse.web.kubernetes.teams`, the helm
chart will take care of namespace generation. Although this is very
useful in most cases, we believe some people may find it problematic.
Our use case, is to create the namespaces ahead of time and fill them
with `Pipeline` type resources defining Concourse pipelines. These are
then picked by our `pipeline-operator` and continuously applied to
Concourse for specific teams.
A hacky way around it, would be to set the
`concourse.web.kubernetes.teams` value to an empty array, and create the
role bindings manually. It feels a little like cheating, and a cleaner
way to accomplish that would be to have a separate flag responsible for
namespace creation in the Concourse helm chart.
Signed-off-by: Rafal Proszowski <paroxp@gmail.com>
* Bump patch
Signed-off-by: Rafal Proszowski <paroxp@gmail.com>
Previously, it was not possible to include neither additional volumes
nor volume mounts to the web instance.
This ends up prohibiting and operator of specifying configuration for
team authorized keys, or any other future configuration that might
depend on files on disk.
Signed-off-by: Ciro S. Costa <cscosta@pivotal.io>
This commit improves:
- notes regarding the use of the default `test:test` default user that
concourse adds; and
- README instructions
- so that we don't use the future tense too much; and
- instructions regarding credential management and postgresql match the
actual `values.yml` structure.
Signed-off-by: Ciro S. Costa <cscosta@pivotal.io>
* specify additional labels for web service
Signed-off-by: Richard Godden <goddenrich@gmail.com>
* added web service labels to README
Signed-off-by: Richard Godden <goddenrich@gmail.com>
* bump version
Signed-off-by: Richard Godden <goddenrich@gmail.com>
* bump version
Signed-off-by: David J. M. Karlsen <david@davidkarlsen.com>
This commit allows to use AWS Secrets Manager without simultaneously
activating AWS SSM, and vice versa.
This should not be a problem in concourse 5.0.0, but the 4.x.x is still
affected.
Signed-off-by: Vladimir Yarotsky <vladimir@pubnub.com>
* control the mount paths of the secret keys and certs
Signed-off-by: Richard Godden <goddenrich@gmail.com>
* added variables to readme
Signed-off-by: Richard Godden <goddenrich@gmail.com>
* secrets mountpath now has defaults
Signed-off-by: Richard Godden <goddenrich@gmail.com>
* using web and worker keys to specify secrets mount path
Signed-off-by: Richard Godden <goddenrich@gmail.com>
* lint
Signed-off-by: Richard Godden <goddenrich@gmail.com>
* changed secrets path keys in README
Signed-off-by: Richard Godden <goddenrich@gmail.com>
* setting secret mount path defaults in the values.yaml instead of the templates
Signed-off-by: rgodden <rgodden@thoughtmachine.net>
* specifying mount path for tls
Signed-off-by: rgodden <rgodden@thoughtmachine.net>
* bump version
Signed-off-by: rgodden <rgodden@thoughtmachine.net>
`imagePullSecrets` can now be specified in order to
be able to pull the image from a registry that requires
auth.
closeshelm/charts#9228.
Signed-off-by: Ciro S. Costa <cscosta@pivotal.io>
Previously, Prometheus annotations were added to the web service,
which is meant to represent the whole replica set of pods, leaving
no pod-specific labels to the metrics gathered, defeating the
purpose of some of them (that are meant to be container-specific).
By moving them to `deployment`, we're able to collect them per-pod,
instead of per-service.
Signed-off-by: Ciro S. Costa <cscosta@pivotal.io>
Following the example set by the grafana chart (see [1]), this commit
turns `web` probes into configurable entities.
It also makes use of `/api/v1/info` as the default endpoint for
performing the requests against so `web` doesn't return the whole
initial page as a return value (instead, just a very small json from
`/api/v1/info`).
Signed-off-by: Ciro S. Costa <cscosta@pivotal.io>
Previously, configuring ATC as an HTTPS-enabled service wasn't
an easy thing to do.
After this commit, such capability is achieved by configuring
the `tls` object under `concourse.web`:
concourse:
web:
tls: { enabled: true , bindPort: 443 }
With that configured, a cert and a key gets injected into the
containers via secrets:
secrets:
webTlsCert: |
<my_cert_chain>
webTlsKey: |
<my_private_key>
Which are then referenced in the right environment variables and
exposed in the service using the proper port.
Signed-off-by: Ciro S. Costa <cscosta@pivotal.io>
`.Values.concourse.atcPort` has been replaced with `.Values.concourse.web.bindPort`
in 263bc80322.
So the references in NOTES.txt should also be updated.
Signed-off-by: Noburou TANIGUCHI <dev@nota.m001.jp>
vault auth params are sensitive credentials and should be treated as
secret
This returns auth params functionality closer to concourse chart v1.16.1,
but it is a breaking change compared to that version
Signed-off-by: Chris Hiestand <chrishiestand@gmail.com>
* stable/concourse: allow taking over the `concourse-work-dir` volume
This allows using a user-defined volume spec (set in
`worker.additionalVolumes` with `name: concourse-work-dir`) instead of
the default `emptyDir` when `persistence.enabled: false` is set.
It changes the behaviour in a backward-compatible way: previously using
`name: concourse-work-dir` in any `worker.additionalVolumes` resulted in
an error (two volumes defined with the same name), now instead of the
error it will use that additionalVolume spec to back the
`concourse-work-dir` volume mount (given that `persistence.enabled` is
false).
Signed-off-by: Norbert Buchmueller <norbert.buchmueller@optiopay.com>
* stable/concourse: bump chart version
Signed-off-by: Norbert Buchmueller <norbert.buchmueller@optiopay.com>
* Fixed typos in the AWS SecretsManager and SSM sections from concoures to concourse.
Also, corrected what appeared to be a misplaced SSM variable in the SecretsManager section.
Signed-off-by: Tarek Mokhtar <tareks01@yahoo.co.nz>
* Bumped chart version.
Signed-off-by: Tarek Mokhtar <tareks01@yahoo.co.nz>
instead of directly from helm chart. k8s secret will use the chart value
as a default, so functionality is not lost
Signed-off-by: Chris Hiestand <chrishiestand@gmail.com>
* Allow the use of a specific image digest
This allows testing pre-release concourse/dev images without
having to tag them.
Signed-off-by: Topher Bullock <cbullock@pivotal.io>
* [stable/concourse] bump chart version
Signed-off-by: Rui Yang <ryang@pivotal.io>
* Bump chart minor version to 2.1.0
Signed-off-by: Topher Bullock <cbullock@pivotal.io>
* Make postgres ssl related variables independent
Signed-off-by: Anton Ustyuzhanin <au@hellofresh.com>
* Bump chart version
Signed-off-by: Anton Ustyuzhanin <au@hellofresh.com>
Kubernetes (but not help) will explode when trying to apply any
container with env vars that are not strings. This commit pipes all
Values based env vars to quote to ensure this doesn't happen.
There are a few variables for both the web and worker that will break
(bools, ints) if not quoted.
Signed-off-by: Dylan Arbour <dylan.arbour@gmail.com>
* [stable/concourse] Fix Broken Tutorial Link
The Concourse docs have a new section with a list of tutorials rather than a single hello-world
Signed-off-by: Topher Bullock <cbullock@pivotal.io>
* Bump Chart version number
Signed-off-by: Topher Bullock <cbullock@pivotal.io>
Taylor Silva
Milan Plžík
Jamie KlassenandJames Thomson
Zoe TianandZoe Tian
Aaron Gershman
Ciro S. CostaandBishoy Youssef
taha-au
Taylor SilvaandBishoy Youssef
Yann David
Sameer VohraandBishoy Youssef
tareks
Ciro S. Costa
Jan Brauer
Rafal Proszowski
Richard Godden
Vlad Yarotsky
Jeppler
Richard Alberto
nota-ja
Chris Hiestand
Norbert Buchmüller
Topher Bullock
Tatsuya Kitagawa
Anton Ustyuzhanin
Dylan Arbour