diff --git a/docs/src/pages/versions.md b/docs/src/pages/versions.md index a74641b2f..abd17596d 100644 --- a/docs/src/pages/versions.md +++ b/docs/src/pages/versions.md @@ -33,6 +33,7 @@ Here you can find documentation for previous versions of Woodpecker. | | | | | ------- | ---------- | ------------------------------------------------------------------------------------- | +| 3.15.0 | 2026-05-28 | [Documentation](https://github.com/woodpecker-ci/woodpecker/tree/v3.15.0/docs/docs/) | | 3.14.0 | 2026-05-01 | [Documentation](https://github.com/woodpecker-ci/woodpecker/tree/v3.14.0/docs/docs/) | | 3.13.0 | 2026-01-14 | [Documentation](https://github.com/woodpecker-ci/woodpecker/tree/v3.13.0/docs/docs/) | | 3.12.0 | 2025-11-18 | [Documentation](https://github.com/woodpecker-ci/woodpecker/tree/v3.12.0/docs/docs/) | diff --git a/docs/versioned_docs/version-3.13/20-usage/100-troubleshooting.md b/docs/versioned_docs/version-3.13/20-usage/100-troubleshooting.md deleted file mode 100644 index b961530f4..000000000 --- a/docs/versioned_docs/version-3.13/20-usage/100-troubleshooting.md +++ /dev/null @@ -1,37 +0,0 @@ -# Troubleshooting - -## How to debug clone issues - -(And what to do with an error message like `fatal: could not read Username for 'https://': No such device or address`) - -This error can have multiple causes. If you use internal repositories you might have to enable `WOODPECKER_AUTHENTICATE_PUBLIC_REPOS`: - -```ini -WOODPECKER_AUTHENTICATE_PUBLIC_REPOS=true -``` - -If that does not work, try to make sure the container can reach your git server. In order to do that disable git checkout and make the container "hang": - -```yaml -skip_clone: true - -steps: - build: - image: debian:stable-backports - commands: - - apt update - - apt install -y inetutils-ping wget - - ping -c 4 git.example.com - - wget git.example.com - - sleep 9999999 -``` - -Get the container id using `docker ps` and copy the id from the first column. Enter the container with: `docker exec -it 1234asdf bash` (replace `1234asdf` with the docker id). Then try to clone the git repository with the commands from the failing pipeline: - -```bash -git init -git remote add origin https://git.example.com/username/repo.git -git fetch --no-tags origin +refs/heads/branch: -``` - -(replace the url AND the branch with the correct values, use your username and password as log in values) diff --git a/docs/versioned_docs/version-3.13/92-development/05-architecture.md b/docs/versioned_docs/version-3.13/92-development/05-architecture.md deleted file mode 100644 index 535ec6a80..000000000 --- a/docs/versioned_docs/version-3.13/92-development/05-architecture.md +++ /dev/null @@ -1,48 +0,0 @@ -# Architecture - -## Package architecture - -![Woodpecker architecture](./woodpecker-architecture.png) - -## System architecture - -### main package hierarchy - -| package | meaning | imports | -| ------------------ | -------------------------------------------------------------- | ------------------------------------- | -| `cmd/**` | parse command-line args & environment to stat server/cli/agent | all other | -| `agent/**` | code only agent (remote worker) will need | `pipeline`, `shared` | -| `cli/**` | code only cli tool does need | `pipeline`, `shared`, `woodpecker-go` | -| `server/**` | code only server will need | `pipeline`, `shared` | -| `shared/**` | code shared for all three main tools (go help utils) | only std and external libs | -| `woodpecker-go/**` | go client for server rest api | std | - -### Server - -| package | meaning | imports | -| -------------------- | ----------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `server/api/**` | handle web requests from `server/router` | `pipeline`, `../badges`, `../ccmenu`, `../logging`, `../model`, `../pubsub`, `../queue`, `../forge`, `../shared`, `../store`, `shared`, (TODO: mv `server/router/middleware/session`) | -| `server/badges/**` | generate svg badges for pipelines | `../model` | -| `server/ccmenu/**` | generate xml ccmenu for pipelines | `../model` | -| `server/grpc/**` | gRPC server agents can connect to | `pipeline/rpc/**`, `../logging`, `../model`, `../pubsub`, `../queue`, `../forge`, `../pipeline`, `../store` | -| `server/logging/**` | logging lib for gPRC server to stream logs while running | std | -| `server/model/**` | structs for store (db) and api (json) | std | -| `server/plugins/**` | plugins for server | `../model`, `../forge` | -| `server/pipeline/**` | orchestrate pipelines | `pipeline`, `../model`, `../pubsub`, `../queue`, `../forge`, `../store`, `../plugins` | -| `server/pubsub/**` | pubsub lib for server to push changes to the WebUI | std | -| `server/queue/**` | queue lib for server where agents pull new pipelines from via gRPC | `server/model` | -| `server/forge/**` | forge lib for server to connect and handle forge specific stuff | `shared`, `server/model` | -| `server/router/**` | handle requests to REST API (and all middleware) and serve UI and WebUI config | `shared`, `../api`, `../model`, `../forge`, `../store`, `../web` | -| `server/store/**` | handle database | `server/model` | -| `server/shared/**` | TODO: move and split [#974](https://github.com/woodpecker-ci/woodpecker/issues/974) | | -| `server/web/**` | server SPA | | - -- `../` = `server/` - -### Agent - -TODO - -### CLI - -TODO diff --git a/docs/versioned_docs/version-3.13/92-development/woodpecker-architecture.png b/docs/versioned_docs/version-3.13/92-development/woodpecker-architecture.png deleted file mode 100644 index 22f6a054a..000000000 Binary files a/docs/versioned_docs/version-3.13/92-development/woodpecker-architecture.png and /dev/null differ diff --git a/docs/versioned_docs/version-3.13/10-intro/index.md b/docs/versioned_docs/version-3.16/10-intro/index.md similarity index 89% rename from docs/versioned_docs/version-3.13/10-intro/index.md rename to docs/versioned_docs/version-3.16/10-intro/index.md index 025bd1f85..a8e00c742 100644 --- a/docs/versioned_docs/version-3.13/10-intro/index.md +++ b/docs/versioned_docs/version-3.16/10-intro/index.md @@ -24,3 +24,5 @@ Then you might want to jump directly into it and [start creating your first pipe ## Want to start from scratch and deploy your own Woodpecker instance? Woodpecker is lightweight and even runs on a Raspberry Pi. You can follow the [deployment guide](../30-administration/00-general.md) to set up your own Woodpecker instance. + +If you want to try a pipeline before installing a server, you can also run workflow files locally with [`woodpecker-cli exec`](../20-usage/73-local-execution.md). diff --git a/docs/versioned_docs/version-3.13/20-usage/10-intro.md b/docs/versioned_docs/version-3.16/20-usage/10-intro.md similarity index 88% rename from docs/versioned_docs/version-3.13/20-usage/10-intro.md rename to docs/versioned_docs/version-3.16/20-usage/10-intro.md index 095528b4e..affe61b47 100644 --- a/docs/versioned_docs/version-3.13/20-usage/10-intro.md +++ b/docs/versioned_docs/version-3.16/20-usage/10-intro.md @@ -66,7 +66,17 @@ You can use any image from registries like the [Docker Hub](https://hub.docker.c - aws help ``` -## 3. Push the file and trigger first pipeline +## 3. Run the workflow locally + +If you have `woodpecker-cli` and a supported backend installed, you can run the workflow before pushing it: + +```shell +woodpecker-cli exec .woodpecker/my-first-workflow.yaml +``` + +This is useful for checking workflow syntax, command output, and metadata conditions while you are still editing the file. For more examples, including secrets and downloaded metadata, see [local pipeline execution](./73-local-execution.md). + +## 4. Push the file and trigger first pipeline If you push this file to your repository now, Woodpecker will already execute your first pipeline. @@ -79,7 +89,7 @@ As you probably noticed, there is another step in called `clone` which is execut This for example allows the first step to build your application using your source code and as the second step will receive the same workspace it can use the previously built binary and test it. -## 4. Use a plugin for reusable tasks +## 5. Use a plugin for reusable tasks Sometimes you have some tasks that you need to do in every project. For example, deploying to Kubernetes or sending a Slack message. Therefore you can use one of the [official and community plugins](/plugins) or simply [create your own](./51-plugins/20-creating-plugins.md). diff --git a/docs/versioned_docs/version-3.16/20-usage/100-troubleshooting.md b/docs/versioned_docs/version-3.16/20-usage/100-troubleshooting.md new file mode 100644 index 000000000..2885c134d --- /dev/null +++ b/docs/versioned_docs/version-3.16/20-usage/100-troubleshooting.md @@ -0,0 +1,97 @@ +# Troubleshooting + +## How to debug clone issues + +(And what to do with an error message like `fatal: could not read Username for 'https://': No such device or address`) + +This error can have multiple causes. If you use internal repositories you might have to enable `WOODPECKER_AUTHENTICATE_PUBLIC_REPOS`: + +```ini +WOODPECKER_AUTHENTICATE_PUBLIC_REPOS=true +``` + +If that does not work, try to make sure the container can reach your git server. In order to do that disable git checkout and make the container "hang": + +```yaml +skip_clone: true + +steps: + build: + image: debian:stable-backports + commands: + - apt update + - apt install -y inetutils-ping wget + - ping -c 4 git.example.com + - wget git.example.com + - sleep 9999999 +``` + +Get the container id using `docker ps` and copy the id from the first column. Enter the container with: `docker exec -it 1234asdf bash` (replace `1234asdf` with the docker id). Then try to clone the git repository with the commands from the failing pipeline: + +```bash +git init +git remote add origin https://git.example.com/username/repo.git +git fetch --no-tags origin +refs/heads/branch: +``` + +(replace the url AND the branch with the correct values, use your username and password as log in values) + +## SELinux Issues + +When running Woodpecker on systems with SELinux enabled (such as RHEL, CentOS, Fedora, or other Enterprise Linux distributions), SELinux may prevent the agent from accessing the Docker socket. + +### Symptoms + +If SELinux is blocking access, you may see errors like: + +```text +permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock +``` + +### Solutions + +There are several ways to resolve this: + +#### Option 1: Set SELinux to Permissive Mode (For Testing Only) + +Set SELinux to permissive mode temporarily to verify it's the issue: + +```bash +setenforce 0 +``` + +To permanently set SELinux to permissive mode: + +```bash +# Edit /etc/selinux/config +SELINUX=permissive +``` + +#### Option 2: Configure SELinux Policy (Recommended) + +Create a custom SELinux policy to allow Woodpecker agent to access Docker: + +```bash +# Generate the policy module +ausearch -c 'docker' -avc | audit2allow -R -o woodpecker-docker.te +# Build the policy module +checkmodule -M -m -o woodpecker-docker.mod woodpecker-docker.te +semodule_package -o woodpecker-docker.pp -m woodpecker-docker.mod +# Load the policy module +semodule -i woodpecker-docker.pp +``` + +#### Option 3: Use Docker Volume with SELinux Options + +When using Docker Compose or Docker, add the `:z` or `:Z` option to volume mounts: + +```yaml +volumes: + - /var/run/docker.sock:/var/run/docker.sock:z +``` + +The `:z` option tells Docker to automatically relabel the volume content for SELinux. Use `:Z` with caution as it relabels the volume exclusively for this container. + +#### Option 4: Use Podman (Alternative) + +If you prefer to avoid SELinux configuration issues, consider using Podman instead of Docker, as it has better SELinux integration. diff --git a/docs/versioned_docs/version-3.13/20-usage/15-terminology/architecture.excalidraw b/docs/versioned_docs/version-3.16/20-usage/15-terminology/architecture.excalidraw similarity index 100% rename from docs/versioned_docs/version-3.13/20-usage/15-terminology/architecture.excalidraw rename to docs/versioned_docs/version-3.16/20-usage/15-terminology/architecture.excalidraw diff --git a/docs/versioned_docs/version-3.13/20-usage/15-terminology/architecture.svg b/docs/versioned_docs/version-3.16/20-usage/15-terminology/architecture.svg similarity index 100% rename from docs/versioned_docs/version-3.13/20-usage/15-terminology/architecture.svg rename to docs/versioned_docs/version-3.16/20-usage/15-terminology/architecture.svg diff --git a/docs/versioned_docs/version-3.13/20-usage/15-terminology/index.md b/docs/versioned_docs/version-3.16/20-usage/15-terminology/index.md similarity index 95% rename from docs/versioned_docs/version-3.13/20-usage/15-terminology/index.md rename to docs/versioned_docs/version-3.16/20-usage/15-terminology/index.md index 08376f06c..222b2dc34 100644 --- a/docs/versioned_docs/version-3.13/20-usage/15-terminology/index.md +++ b/docs/versioned_docs/version-3.16/20-usage/15-terminology/index.md @@ -9,6 +9,7 @@ - **Container**: A lightweight and isolated environment where commands are executed. - **Dependency**: [Workflows][Workflow] can depend on each other, and if possible, they are executed in parallel. - **[Event][Event]**: Triggers the execution of a [pipeline][Pipeline], such as a [forge][Forge] event like `push`, or `manual` triggered manually from the UI. +- **[Extension][Extension]**: Some parts of Woodpecker internal services like secrets storage or config fetcher can be replaced through extensions. - **[Forge][Forge]**: The hosting platform or service where the repositories are hosted. - **[Matrix][Matrix]**: A configuration option that allows the execution of [workflows][Workflow] for each value in the matrix. - **[Pipeline][Pipeline]**: A sequence of [workflows][Workflow] that are executed on the code. Pipelines are triggered by events. @@ -16,7 +17,6 @@ - **Repos**: Short for repositories, these are storage locations where code is stored. - **Server**: The component of Woodpecker that handles webhooks from forges, orchestrates agents, and sends status back. It also serves the API and web UI for administration and configuration. - **Service**: A service is a step that is executed from the start of a [workflow][Workflow] until its end. It can be accessed by name via the network from other steps within the same [workflow][Workflow]. -- **Service extension**: Some parts of Woodpecker internal services like secrets storage or config fetcher can be replaced through service extensions. - **Status**: Status refers to the outcome of a step or [workflow][Workflow] after it has been executed, determined by the internal command exit code. At the end of a [workflow][Workflow], its status is sent to the [forge][Forge]. - **Steps**: Individual commands, actions or tasks within a [workflow][Workflow]. - **Task**: A task is a [workflow][Workflow] that's currently waiting for its execution in the task queue. @@ -54,3 +54,4 @@ Sometimes there are multiple terms that can be used to describe something. This [Matrix]: ../30-matrix-workflows.md [Docker]: ../../30-administration/10-configuration/11-backends/10-docker.md [Local]: ../../30-administration/10-configuration/11-backends/30-local.md +[Extension]: ../72-extensions/index.md diff --git a/docs/versioned_docs/version-3.13/20-usage/15-terminology/pipeline-workflow-step.excalidraw b/docs/versioned_docs/version-3.16/20-usage/15-terminology/pipeline-workflow-step.excalidraw similarity index 100% rename from docs/versioned_docs/version-3.13/20-usage/15-terminology/pipeline-workflow-step.excalidraw rename to docs/versioned_docs/version-3.16/20-usage/15-terminology/pipeline-workflow-step.excalidraw diff --git a/docs/versioned_docs/version-3.13/20-usage/15-terminology/pipeline-workflow-step.svg b/docs/versioned_docs/version-3.16/20-usage/15-terminology/pipeline-workflow-step.svg similarity index 100% rename from docs/versioned_docs/version-3.13/20-usage/15-terminology/pipeline-workflow-step.svg rename to docs/versioned_docs/version-3.16/20-usage/15-terminology/pipeline-workflow-step.svg diff --git a/docs/versioned_docs/version-3.13/20-usage/20-workflow-syntax.md b/docs/versioned_docs/version-3.16/20-usage/20-workflow-syntax.md similarity index 92% rename from docs/versioned_docs/version-3.13/20-usage/20-workflow-syntax.md rename to docs/versioned_docs/version-3.16/20-usage/20-workflow-syntax.md index f97d87e4f..282fe22f7 100644 --- a/docs/versioned_docs/version-3.13/20-usage/20-workflow-syntax.md +++ b/docs/versioned_docs/version-3.16/20-usage/20-workflow-syntax.md @@ -196,6 +196,8 @@ Some of the steps may be allowed to fail without causing the whole workflow and + failure: ignore ``` +If you would like to cancel the full pipeline once the step fails, you can set `failure: cancel`. For the default behaviour, use `failure: fail`. + ### `when` - Conditional Execution Woodpecker supports defining a list of conditions for a step by using a `when` block. If at least one of the conditions in the `when` block evaluate to true the step is executed, otherwise it is skipped. A condition is evaluated to true if _all_ sub-conditions are true. @@ -340,7 +342,14 @@ when: #### `status` -There are use cases for executing steps on failure, such as sending notifications for failed workflow/pipeline. Use the status constraint to execute steps even when the workflow fails: +By default, steps only run when the workflow has succeeded up to that point,
+which is equivalent to `status: [ success ]`. + +The `status` filter lets you override this behavior. +The only accepted values are `success` and `failure`. + +A common use case is executing a step on failure, such as sending notifications for a failed workflow/pipeline. +To run a step regardless of outcome, list both values: ```diff steps: @@ -350,6 +359,18 @@ There are use cases for executing steps on failure, such as sending notification + - status: [ success, failure ] ``` +The filter is aware of the other filters. If you want to run on failures if the event is `tag`, but if it's a `pull_request`, run it on both success and failure: + +```diff + when: ++ - event: tag ++ status: [ failure ] ++ - event: pull_request ++ status: [ success, failure ] +``` + +If there's no matching filter at all or all matching filters don't have set `status`, it will use the default, which means it runs on success only. In the example above this will happen if the event is neither `tag` nor `pull_request`. + #### `platform` :::note @@ -736,7 +757,10 @@ skip_clone: true ## `when` - Global workflow conditions -Woodpecker gives the ability to skip whole workflows ([not just steps](#when---conditional-execution)) based on certain conditions by a `when` block. If all conditions in the `when` block evaluate to true the workflow is executed, otherwise it is skipped, but treated as successful and other workflows depending on it will still continue. +Woodpecker gives the ability to skip whole workflows ([not just steps](#when---conditional-execution)) based on certain conditions by a `when` block. +If all conditions in the `when` block evaluate to true the workflow is executed, otherwise it is not included in the pipeline. +Other workflows that have a `depends_on` referencing a skipped workflow will also be excluded. +Use [`optional: true` on a dependency](./25-workflows.md#optional-dependencies) if you want it to be ignored when the referenced workflow is not part of the pipeline. For more information about the specific filters, take a look at the [step-specific `when` filters](#when---conditional-execution). @@ -761,9 +785,18 @@ The workflow now triggers on `main`, but also if the target branch of a pull req Woodpecker supports to define multiple workflows for a repository. Those workflows will run independent from each other. To depend them on each other you can use the [`depends_on`](./25-workflows.md#flow-control) keyword. -## `runs_on` +### Optional dependencies in `depends_on` -Workflows that should run even on failure should set the `runs_on` tag. See [here](./25-workflows.md#flow-control) for an example. +Each entry in `depends_on` can be a string (required dependency) or an object with `name` and `optional: true`. +Optional dependencies are silently ignored when the referenced workflow or step is not part of the pipeline (e.g. filtered out by `when` conditions). +If present, the workflow waits for them as usual. See [optional dependencies](./25-workflows.md#optional-dependencies) for details. + +```yaml +depends_on: + - check-a + - name: check-b + optional: true +``` ## Advanced network options for steps diff --git a/docs/versioned_docs/version-3.13/20-usage/25-workflows.md b/docs/versioned_docs/version-3.16/20-usage/25-workflows.md similarity index 51% rename from docs/versioned_docs/version-3.13/20-usage/25-workflows.md rename to docs/versioned_docs/version-3.16/20-usage/25-workflows.md index ef09d485e..919705b5f 100644 --- a/docs/versioned_docs/version-3.13/20-usage/25-workflows.md +++ b/docs/versioned_docs/version-3.16/20-usage/25-workflows.md @@ -97,7 +97,7 @@ The name for a `depends_on` entry is the filename without the path, leading dots + - test ``` -Workflows that need to run even on failures should set the `runs_on` tag. +Workflows that need to run even on failures should set the `status` filter. ```diff steps: @@ -109,10 +109,80 @@ Workflows that need to run even on failures should set the `runs_on` tag. depends_on: - deploy -+runs_on: [ success, failure ] ++when: ++ - status: [ success, failure ] ``` +This works just like the [`status` filter for steps](./20-workflow-syntax.md#status). + +### Optional dependencies + +In a monorepo, workflows often use `when: path` to only run when relevant files change. A deploy workflow may need to wait for all check workflows, but some of them might not run because their path filter didn't match. With `depends_on`, this would block the deploy workflow entirely. + +Mark a dependency as `optional: true` so it is only enforced when the referenced workflow is part of the pipeline. If the dependency is not built (e.g. its `when` conditions don't match), it is silently ignored. + +```diff + steps: + - name: deploy + image: debian:stable-slim + commands: + - echo deploying app a + + depends_on: + - check-a ++ - name: check-b ++ optional: true ++ - name: check-c ++ optional: true +``` + +In this example, `deploy` always waits for `check-a`. It also waits for `check-b` and `check-c` if they are part of the pipeline, but runs without them if they were filtered out. + +The same syntax works at the step level within a workflow: if a step uses `depends_on` with `optional: true` on another step that was filtered out by a `when` condition, the dependency is silently dropped. + :::info Some workflows don't need the source code, like creating a notification on failure. Read more about `skip_clone` at [pipeline syntax](./20-workflow-syntax.md#skip_clone) ::: + +## Concurrency + +By default workflows run with no concurrency limit. Some workflows, however, must not run more than a given number of times at once. A typical example is a deployment workflow: running two deployments at the same time can cause race conditions or corrupt state. Cancelling the previous pipeline is often not an option either, since it could interrupt an ongoing deployment. + +The `concurrency` setting limits how many instances of a workflow may run at the same time. When the limit is reached, additional instances stay queued and start only once a running one has finished. Nothing is cancelled. + +```yaml title=".woodpecker/deploy.yaml" +steps: + - name: deploy + image: debian:stable-slim + commands: + - echo deploying + +depends_on: + - test + +concurrency: + limit: 1 +``` + +You can also use the shorthand form to only set the limit: + +```yaml +concurrency: 1 +``` + +### Ordering + +Queued workflows of the same group start in the order their pipelines were created, **not** in the order they become ready to run. This matters when a workflow depends on other workflows (via `depends_on`) whose duration varies: even if a later pipeline's checks finish first, its limited workflow will not overtake an earlier pipeline that is still waiting. This guarantees that, for example, deployments happen in commit order. + +### Groups + +By default, the limit applies per workflow within a repository. Different runs of the same workflow are limited against each other, while different workflows (and other repositories) are unaffected. + +Setting a `group` is optional. You can set a custom `group` to share a limit across workflows or to make the limit more specific. The group supports [environment variable substitution](./50-environment.md), so you can, for example, limit concurrency per branch or per deployment target: + +```yaml +concurrency: + limit: 1 + group: deploy-${CI_COMMIT_BRANCH} +``` diff --git a/docs/versioned_docs/version-3.13/20-usage/30-matrix-workflows.md b/docs/versioned_docs/version-3.16/20-usage/30-matrix-workflows.md similarity index 100% rename from docs/versioned_docs/version-3.13/20-usage/30-matrix-workflows.md rename to docs/versioned_docs/version-3.16/20-usage/30-matrix-workflows.md diff --git a/docs/versioned_docs/version-3.13/20-usage/40-secrets.md b/docs/versioned_docs/version-3.16/20-usage/40-secrets.md similarity index 100% rename from docs/versioned_docs/version-3.13/20-usage/40-secrets.md rename to docs/versioned_docs/version-3.16/20-usage/40-secrets.md diff --git a/docs/versioned_docs/version-3.13/20-usage/41-registries.md b/docs/versioned_docs/version-3.16/20-usage/41-registries.md similarity index 100% rename from docs/versioned_docs/version-3.13/20-usage/41-registries.md rename to docs/versioned_docs/version-3.16/20-usage/41-registries.md diff --git a/docs/versioned_docs/version-3.13/20-usage/45-cron.md b/docs/versioned_docs/version-3.16/20-usage/45-cron.md similarity index 100% rename from docs/versioned_docs/version-3.13/20-usage/45-cron.md rename to docs/versioned_docs/version-3.16/20-usage/45-cron.md diff --git a/docs/versioned_docs/version-3.13/20-usage/50-environment.md b/docs/versioned_docs/version-3.16/20-usage/50-environment.md similarity index 57% rename from docs/versioned_docs/version-3.13/20-usage/50-environment.md rename to docs/versioned_docs/version-3.16/20-usage/50-environment.md index b6d2690d9..4306a7167 100644 --- a/docs/versioned_docs/version-3.13/20-usage/50-environment.md +++ b/docs/versioned_docs/version-3.16/20-usage/50-environment.md @@ -48,99 +48,105 @@ Please note that the environment section is not able to expand environment varia This is the reference list of all environment variables available to your pipeline containers. These are injected into your pipeline step and plugins containers, at runtime. -| NAME | Description | Example | -| ---------------------------------- | ------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------- | -| `CI` | CI environment name | `woodpecker` | -| | **Repository** | | -| `CI_REPO` | repository full name `/` | `john-doe/my-repo` | -| `CI_REPO_OWNER` | repository owner | `john-doe` | -| `CI_REPO_NAME` | repository name | `my-repo` | -| `CI_REPO_REMOTE_ID` | repository remote ID, is the UID it has in the forge | `82` | -| `CI_REPO_URL` | repository web URL | `https://git.example.com/john-doe/my-repo` | -| `CI_REPO_CLONE_URL` | repository clone URL | `https://git.example.com/john-doe/my-repo.git` | -| `CI_REPO_CLONE_SSH_URL` | repository SSH clone URL | `git@git.example.com:john-doe/my-repo.git` | -| `CI_REPO_DEFAULT_BRANCH` | repository default branch | `main` | -| `CI_REPO_PRIVATE` | repository is private | `true` | -| `CI_REPO_TRUSTED_NETWORK` | repository has trusted network access | `false` | -| `CI_REPO_TRUSTED_VOLUMES` | repository has trusted volumes access | `false` | -| `CI_REPO_TRUSTED_SECURITY` | repository has trusted security access | `false` | -| | **Current Commit** | | -| `CI_COMMIT_SHA` | commit SHA | `eba09b46064473a1d345da7abf28b477468e8dbd` | -| `CI_COMMIT_REF` | commit ref | `refs/heads/main` | -| `CI_COMMIT_REFSPEC` | commit ref spec | `issue-branch:main` | -| `CI_COMMIT_BRANCH` | commit branch (equals target branch for pull requests) | `main` | -| `CI_COMMIT_SOURCE_BRANCH` | commit source branch (set only for pull request events) | `issue-branch` | -| `CI_COMMIT_TARGET_BRANCH` | commit target branch (set only for pull request events) | `main` | -| `CI_COMMIT_TAG` | commit tag name (empty if event is not `tag`) | `v1.10.3` | -| `CI_COMMIT_PULL_REQUEST` | commit pull request number (set only for pull request events) | `1` | -| `CI_COMMIT_PULL_REQUEST_LABELS` | labels assigned to pull request (set only for pull request events) | `server` | -| `CI_COMMIT_PULL_REQUEST_MILESTONE` | milestone assigned to pull request (set only for `pull_request` and `pull_request_closed` events) | `summer-sprint` | -| `CI_COMMIT_MESSAGE` | commit message | `Initial commit` | -| `CI_COMMIT_AUTHOR` | commit author username | `john-doe` | -| `CI_COMMIT_AUTHOR_EMAIL` | commit author email address | `john-doe@example.com` | -| `CI_COMMIT_PRERELEASE` | release is a pre-release (empty if event is not `release`) | `false` | -| | **Current pipeline** | | -| `CI_PIPELINE_NUMBER` | pipeline number | `8` | -| `CI_PIPELINE_PARENT` | number of parent pipeline | `0` | -| `CI_PIPELINE_EVENT` | pipeline event (see [`event`](../20-usage/20-workflow-syntax.md#event)) | `push`, `pull_request`, `pull_request_closed`, `pull_request_metadata`, `tag`, `release`, `manual`, `cron` | -| `CI_PIPELINE_EVENT_REASON` | exact reason why `pull_request_metadata` event was send. it is forge instance specific and can change | `label_updated`, `milestoned`, `demilestoned`, `assigned`, `edited`, ... | -| `CI_PIPELINE_URL` | link to the web UI for the pipeline | `https://ci.example.com/repos/7/pipeline/8` | -| `CI_PIPELINE_FORGE_URL` | link to the forge's web UI for the commit(s) or tag that triggered the pipeline | `https://git.example.com/john-doe/my-repo/commit/eba09b46064473a1d345da7abf28b477468e8dbd` | -| `CI_PIPELINE_DEPLOY_TARGET` | pipeline deploy target for `deployment` events | `production` | -| `CI_PIPELINE_DEPLOY_TASK` | pipeline deploy task for `deployment` events | `migration` | -| `CI_PIPELINE_CREATED` | pipeline created UNIX timestamp | `1722617519` | -| `CI_PIPELINE_STARTED` | pipeline started UNIX timestamp | `1722617519` | -| `CI_PIPELINE_FILES` | changed files (empty if event is not `push` or `pull_request`), it is undefined if more than 500 files are touched | `[]`, `[".woodpecker.yml","README.md"]` | -| `CI_PIPELINE_AUTHOR` | pipeline author username | `octocat` | -| `CI_PIPELINE_AVATAR` | pipeline author avatar | `https://git.example.com/avatars/5dcbcadbce6f87f8abef` | -| | **Current workflow** | | -| `CI_WORKFLOW_NAME` | workflow name | `release` | -| | **Current step** | | -| `CI_STEP_NAME` | step name | `build package` | -| `CI_STEP_NUMBER` | step number | `0` | -| `CI_STEP_STARTED` | step started UNIX timestamp | `1722617519` | -| `CI_STEP_URL` | URL to step in UI | `https://ci.example.com/repos/7/pipeline/8` | -| | **Previous commit** | | -| `CI_PREV_COMMIT_SHA` | previous commit SHA | `15784117e4e103f36cba75a9e29da48046eb82c4` | -| `CI_PREV_COMMIT_REF` | previous commit ref | `refs/heads/main` | -| `CI_PREV_COMMIT_REFSPEC` | previous commit ref spec | `issue-branch:main` | -| `CI_PREV_COMMIT_BRANCH` | previous commit branch | `main` | -| `CI_PREV_COMMIT_SOURCE_BRANCH` | previous commit source branch (set only for pull request events) | `issue-branch` | -| `CI_PREV_COMMIT_TARGET_BRANCH` | previous commit target branch (set only for pull request events) | `main` | -| `CI_PREV_COMMIT_URL` | previous commit link in forge | `https://git.example.com/john-doe/my-repo/commit/15784117e4e103f36cba75a9e29da48046eb82c4` | -| `CI_PREV_COMMIT_MESSAGE` | previous commit message | `test` | -| `CI_PREV_COMMIT_AUTHOR` | previous commit author username | `john-doe` | -| `CI_PREV_COMMIT_AUTHOR_EMAIL` | previous commit author email address | `john-doe@example.com` | -| | **Previous pipeline** | | -| `CI_PREV_PIPELINE_NUMBER` | previous pipeline number | `7` | -| `CI_PREV_PIPELINE_PARENT` | previous pipeline number of parent pipeline | `0` | -| `CI_PREV_PIPELINE_EVENT` | previous pipeline event (see [`event`](../20-usage/20-workflow-syntax.md#event)) | `push`, `pull_request`, `pull_request_closed`, `pull_request_metadata`, `tag`, `release`, `manual`, `cron` | -| `CI_PREV_PIPELINE_EVENT_REASON` | previous exact reason `pull_request_metadata` event was send. it is forge instance specific and can change | `label_updated`, `milestoned`, `demilestoned`, `assigned`, `edited`, ... | -| `CI_PREV_PIPELINE_URL` | previous pipeline link in CI | `https://ci.example.com/repos/7/pipeline/7` | -| `CI_PREV_PIPELINE_FORGE_URL` | previous pipeline link to event in forge | `https://git.example.com/john-doe/my-repo/commit/15784117e4e103f36cba75a9e29da48046eb82c4` | -| `CI_PREV_PIPELINE_DEPLOY_TARGET` | previous pipeline deploy target for `deployment` events | `production` | -| `CI_PREV_PIPELINE_DEPLOY_TASK` | previous pipeline deploy task for `deployment` events | `migration` | -| `CI_PREV_PIPELINE_STATUS` | previous pipeline status | `success`, `failure` | -| `CI_PREV_PIPELINE_CREATED` | previous pipeline created UNIX timestamp | `1722610173` | -| `CI_PREV_PIPELINE_STARTED` | previous pipeline started UNIX timestamp | `1722610173` | -| `CI_PREV_PIPELINE_FINISHED` | previous pipeline finished UNIX timestamp | `1722610383` | -| `CI_PREV_PIPELINE_AUTHOR` | previous pipeline author username | `octocat` | -| `CI_PREV_PIPELINE_AVATAR` | previous pipeline author avatar | `https://git.example.com/avatars/5dcbcadbce6f87f8abef` | -| |   | | -| `CI_WORKSPACE` | Path of the workspace where source code gets cloned to | `/woodpecker/src/git.example.com/john-doe/my-repo` | -| | **System** | | -| `CI_SYSTEM_NAME` | name of the CI system | `woodpecker` | -| `CI_SYSTEM_URL` | link to CI system | `https://ci.example.com` | -| `CI_SYSTEM_HOST` | hostname of CI server | `ci.example.com` | -| `CI_SYSTEM_VERSION` | version of the server | `2.7.0` | -| | **Forge** | | -| `CI_FORGE_TYPE` | name of forge | `bitbucket` , `bitbucket_dc` , `forgejo` , `gitea` , `github` , `gitlab` | -| `CI_FORGE_URL` | root URL of configured forge | `https://git.example.com` | -| | **Internal** - Please don't use! | | -| `CI_SCRIPT` | Internal script path. Used to call pipeline step commands. | | -| `CI_NETRC_USERNAME` | Credentials for private repos to be able to clone data. (Only available for specific images) | | -| `CI_NETRC_PASSWORD` | Credentials for private repos to be able to clone data. (Only available for specific images) | | -| `CI_NETRC_MACHINE` | Credentials for private repos to be able to clone data. (Only available for specific images) | | +| NAME | Description | Example | +| ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------- | +| `CI` | CI environment name | `woodpecker` | +| | **Repository** | | +| `CI_REPO` | repository full name `/` | `john-doe/my-repo` | +| `CI_REPO_OWNER` | repository owner | `john-doe` | +| `CI_REPO_NAME` | repository name | `my-repo` | +| `CI_REPO_REMOTE_ID` | repository remote ID, is the UID it has in the forge | `82` | +| `CI_REPO_URL` | repository web URL | `https://git.example.com/john-doe/my-repo` | +| `CI_REPO_CLONE_URL` | repository clone URL | `https://git.example.com/john-doe/my-repo.git` | +| `CI_REPO_CLONE_SSH_URL` | repository SSH clone URL | `git@git.example.com:john-doe/my-repo.git` | +| `CI_REPO_DEFAULT_BRANCH` | repository default branch | `main` | +| `CI_REPO_PRIVATE` | repository is private | `true` | +| `CI_REPO_TRUSTED_NETWORK` | repository has trusted network access | `false` | +| `CI_REPO_TRUSTED_VOLUMES` | repository has trusted volumes access | `false` | +| `CI_REPO_TRUSTED_SECURITY` | repository has trusted security access | `false` | +| | **Current Commit** | | +| `CI_COMMIT_SHA` | commit SHA | `eba09b46064473a1d345da7abf28b477468e8dbd` | +| `CI_COMMIT_REF` | commit ref | `refs/heads/main` | +| `CI_COMMIT_REFSPEC` | commit ref spec | `issue-branch:main` | +| `CI_COMMIT_BRANCH` | commit branch (equals target branch for pull requests) | `main` | +| `CI_COMMIT_SOURCE_BRANCH` | commit source branch (set only for pull request events) | `issue-branch` | +| `CI_COMMIT_TARGET_BRANCH` | commit target branch (set only for pull request events) | `main` | +| `CI_COMMIT_TAG` | commit tag name (empty if event is not `tag`) | `v1.10.3` | +| `CI_COMMIT_PULL_REQUEST` | commit pull request number (set only for pull request events) | `1` | +| `CI_COMMIT_PULL_REQUEST_LABELS` | labels assigned to pull request (set only for pull request events) | `server` | +| `CI_COMMIT_PULL_REQUEST_MILESTONE` | milestone assigned to pull request (set only for `pull_request` and `pull_request_closed` events) | `summer-sprint` | +| `CI_COMMIT_PULL_REQUEST_DRAFT` | whether the pull request is a draft (set only for pull request events; see [forge support](#ci_commit_pull_request_draft-forge-support)) | `true`, `false` | +| `CI_COMMIT_MESSAGE` | commit message | `Initial commit` | +| `CI_COMMIT_TIMESTAMP` | commit UNIX timestamp | `1722617519` | +| `CI_COMMIT_AUTHOR` | commit author username | `john-doe` | +| `CI_COMMIT_AUTHOR_EMAIL` | commit author email address | `john-doe@example.com` | +| `CI_COMMIT_PRERELEASE` | release is a pre-release (empty if event is not `release`) | `false` | +| | **Current pipeline** | | +| `CI_PIPELINE_NUMBER` | pipeline number | `8` | +| `CI_PIPELINE_PARENT` | number of parent pipeline | `0` | +| `CI_PIPELINE_STATUS` | state of the workflow right before the step was started | `success`, `failure` | +| `CI_PIPELINE_EVENT` | pipeline event (see [`event`](../20-usage/20-workflow-syntax.md#event)) | `push`, `pull_request`, `pull_request_closed`, `pull_request_metadata`, `tag`, `release`, `manual`, `cron` | +| `CI_PIPELINE_EVENT_REASON` | exact reason why `pull_request_metadata` event was send. it is forge instance specific and can change | `label_updated`, `milestoned`, `demilestoned`, `assigned`, `edited`, ... | +| `CI_PIPELINE_URL` | link to the web UI for the pipeline | `https://ci.example.com/repos/7/pipeline/8` | +| `CI_PIPELINE_FORGE_URL` | link to the forge's web UI for the commit(s) or tag that triggered the pipeline | `https://git.example.com/john-doe/my-repo/commit/eba09b46064473a1d345da7abf28b477468e8dbd` | +| `CI_PIPELINE_DEPLOY_TARGET` | pipeline deploy target for `deployment` events | `production` | +| `CI_PIPELINE_DEPLOY_TASK` | pipeline deploy task for `deployment` events | `migration` | +| `CI_PIPELINE_CREATED` | pipeline created UNIX timestamp | `1722617519` | +| `CI_PIPELINE_STARTED` | pipeline started UNIX timestamp | `1722617519` | +| `CI_PIPELINE_FILES` | changed files (empty if event is not `push` or `pull_request`), it is undefined if more than 500 files are touched | `[]`, `[".woodpecker.yml","README.md"]` | +| `CI_PIPELINE_AUTHOR` | pipeline author username | `octocat` | +| `CI_PIPELINE_AVATAR` | pipeline author avatar | `https://git.example.com/avatars/5dcbcadbce6f87f8abef` | +| `CI_PIPELINE_RERUNS` | number of times the pipeline has been restarted; not set on the initial run, `1` after the first restart, incremented on each subsequent restart | `1` | +| | **Current workflow** | | +| `CI_WORKFLOW_NAME` | workflow name | `release` | +| | **Current step** | | +| `CI_STEP_NAME` | step name | `build package` | +| `CI_STEP_TYPE` | step type (`commands`, `plugin`, `service`, `clone` or `cache`) | `commands` | +| `CI_STEP_NUMBER` | step number | `0` | +| `CI_STEP_STARTED` | step started UNIX timestamp | `1722617519` | +| `CI_STEP_URL` | URL to step in UI | `https://ci.example.com/repos/7/pipeline/8` | +| | **Previous commit** | | +| `CI_PREV_COMMIT_SHA` | previous commit SHA | `15784117e4e103f36cba75a9e29da48046eb82c4` | +| `CI_PREV_COMMIT_REF` | previous commit ref | `refs/heads/main` | +| `CI_PREV_COMMIT_REFSPEC` | previous commit ref spec | `issue-branch:main` | +| `CI_PREV_COMMIT_BRANCH` | previous commit branch | `main` | +| `CI_PREV_COMMIT_SOURCE_BRANCH` | previous commit source branch (set only for pull request events) | `issue-branch` | +| `CI_PREV_COMMIT_TARGET_BRANCH` | previous commit target branch (set only for pull request events) | `main` | +| `CI_PREV_COMMIT_URL` | previous commit link in forge | `https://git.example.com/john-doe/my-repo/commit/15784117e4e103f36cba75a9e29da48046eb82c4` | +| `CI_PREV_COMMIT_MESSAGE` | previous commit message | `test` | +| `CI_PREV_COMMIT_TIMESTAMP` | previous commit UNIX timestamp | `1722617519` | +| `CI_PREV_COMMIT_AUTHOR` | previous commit author username | `john-doe` | +| `CI_PREV_COMMIT_AUTHOR_EMAIL` | previous commit author email address | `john-doe@example.com` | +| | **Previous pipeline** | | +| `CI_PREV_PIPELINE_NUMBER` | previous pipeline number | `7` | +| `CI_PREV_PIPELINE_PARENT` | previous pipeline number of parent pipeline | `0` | +| `CI_PREV_PIPELINE_EVENT` | previous pipeline event (see [`event`](../20-usage/20-workflow-syntax.md#event)) | `push`, `pull_request`, `pull_request_closed`, `pull_request_metadata`, `tag`, `release`, `manual`, `cron` | +| `CI_PREV_PIPELINE_EVENT_REASON` | previous exact reason `pull_request_metadata` event was send. it is forge instance specific and can change | `label_updated`, `milestoned`, `demilestoned`, `assigned`, `edited`, ... | +| `CI_PREV_PIPELINE_URL` | previous pipeline link in CI | `https://ci.example.com/repos/7/pipeline/7` | +| `CI_PREV_PIPELINE_FORGE_URL` | previous pipeline link to event in forge | `https://git.example.com/john-doe/my-repo/commit/15784117e4e103f36cba75a9e29da48046eb82c4` | +| `CI_PREV_PIPELINE_DEPLOY_TARGET` | previous pipeline deploy target for `deployment` events | `production` | +| `CI_PREV_PIPELINE_DEPLOY_TASK` | previous pipeline deploy task for `deployment` events | `migration` | +| `CI_PREV_PIPELINE_STATUS` | previous pipeline status | `success`, `failure` | +| `CI_PREV_PIPELINE_CREATED` | previous pipeline created UNIX timestamp | `1722610173` | +| `CI_PREV_PIPELINE_STARTED` | previous pipeline started UNIX timestamp | `1722610173` | +| `CI_PREV_PIPELINE_FINISHED` | previous pipeline finished UNIX timestamp | `1722610383` | +| `CI_PREV_PIPELINE_AUTHOR` | previous pipeline author username | `octocat` | +| `CI_PREV_PIPELINE_AVATAR` | previous pipeline author avatar | `https://git.example.com/avatars/5dcbcadbce6f87f8abef` | +| |   | | +| `CI_WORKSPACE` | Path of the workspace where source code gets cloned to | `/woodpecker/src/git.example.com/john-doe/my-repo` | +| | **System** | | +| `CI_SYSTEM_NAME` | name of the CI system | `woodpecker` | +| `CI_SYSTEM_URL` | link to CI system | `https://ci.example.com` | +| `CI_SYSTEM_HOST` | hostname of CI server | `ci.example.com` | +| `CI_SYSTEM_VERSION` | version of the server | `2.7.0` | +| | **Forge** | | +| `CI_FORGE_TYPE` | name of forge | `bitbucket` , `bitbucket_dc` , `forgejo` , `gitea` , `github` , `gitlab` | +| `CI_FORGE_URL` | root URL of configured forge | `https://git.example.com` | +| | **Internal** - Please don't use! | | +| `CI_SCRIPT` | Internal script path. Used to call pipeline step commands. | | +| `CI_NETRC_USERNAME` | Credentials for private repos to be able to clone data. (Only available for specific images) | | +| `CI_NETRC_PASSWORD` | Credentials for private repos to be able to clone data. (Only available for specific images) | | +| `CI_NETRC_MACHINE` | Credentials for private repos to be able to clone data. (Only available for specific images) | | ## Global environment variables @@ -227,6 +233,21 @@ Example variable substitution strips `v` prefix from `v.1.0.0`: + target: /target/${CI_COMMIT_TAG##v} ``` +## `CI_COMMIT_PULL_REQUEST_DRAFT` forge support + +For pull request events, `CI_COMMIT_PULL_REQUEST_DRAFT` is set to `true` or `false` depending on whether the pull request is a draft. + +| Forge | Supported | Notes | +| -------------------- | ------------------ | ----------------------------------------------------------------- | +| GitHub | :white_check_mark: | | +| Gitea | :white_check_mark: | | +| GitLab | :white_check_mark: | Uses `draft`; falls back to legacy `work_in_progress` when needed | +| Forgejo | :x: | Webhook payloads include draft status, but it is not exposed yet | +| Bitbucket | :x: | Webhook payloads include draft status, but it is not exposed yet | +| Bitbucket Datacenter | :x: | Webhook payloads include draft status, but it is not exposed yet | + +On unsupported forges the variable is still set to `false`. + ## `pull_request_metadata` specific event reason values For the `pull_request_metadata` event, the exact reason a metadata change was detected is passe through in `CI_PIPELINE_EVENT_REASON`. diff --git a/docs/versioned_docs/version-3.13/20-usage/51-plugins/20-creating-plugins.md b/docs/versioned_docs/version-3.16/20-usage/51-plugins/20-creating-plugins.md similarity index 100% rename from docs/versioned_docs/version-3.13/20-usage/51-plugins/20-creating-plugins.md rename to docs/versioned_docs/version-3.16/20-usage/51-plugins/20-creating-plugins.md diff --git a/docs/versioned_docs/version-3.13/20-usage/51-plugins/51-overview.md b/docs/versioned_docs/version-3.16/20-usage/51-plugins/51-overview.md similarity index 100% rename from docs/versioned_docs/version-3.13/20-usage/51-plugins/51-overview.md rename to docs/versioned_docs/version-3.16/20-usage/51-plugins/51-overview.md diff --git a/docs/versioned_docs/version-3.13/20-usage/51-plugins/_category_.yaml b/docs/versioned_docs/version-3.16/20-usage/51-plugins/_category_.yaml similarity index 100% rename from docs/versioned_docs/version-3.13/20-usage/51-plugins/_category_.yaml rename to docs/versioned_docs/version-3.16/20-usage/51-plugins/_category_.yaml diff --git a/docs/versioned_docs/version-3.13/20-usage/60-services.md b/docs/versioned_docs/version-3.16/20-usage/60-services.md similarity index 86% rename from docs/versioned_docs/version-3.13/20-usage/60-services.md rename to docs/versioned_docs/version-3.16/20-usage/60-services.md index 3697fef48..fa243f70f 100644 --- a/docs/versioned_docs/version-3.13/20-usage/60-services.md +++ b/docs/versioned_docs/version-3.16/20-usage/60-services.md @@ -37,6 +37,20 @@ services: - 51820/udp ``` +## Stopping + +Services that are no longer needed receive a **SIGTERM** signal. If they do not respond, they are forcibly terminated with **SIGKILL**. +If there are services that do not shut down properly and this doesn't matter, you can simply ignore the error: + +```diff + services: + - name: database + image: mysql ++ failure: ignore # we don't care how mysql exits + ports: + - 3306 +``` + ## Configuration Service containers generally expose environment variables to customize service startup such as default usernames, passwords and ports. Please see the official image documentation to learn more. diff --git a/docs/versioned_docs/version-3.13/20-usage/70-volumes.md b/docs/versioned_docs/version-3.16/20-usage/70-volumes.md similarity index 100% rename from docs/versioned_docs/version-3.13/20-usage/70-volumes.md rename to docs/versioned_docs/version-3.16/20-usage/70-volumes.md diff --git a/docs/versioned_docs/version-3.13/20-usage/72-extensions/40-configuration-extension.md b/docs/versioned_docs/version-3.16/20-usage/72-extensions/40-configuration-extension.md similarity index 77% rename from docs/versioned_docs/version-3.13/20-usage/72-extensions/40-configuration-extension.md rename to docs/versioned_docs/version-3.16/20-usage/72-extensions/40-configuration-extension.md index c8f41ea48..f263a0693 100644 --- a/docs/versioned_docs/version-3.13/20-usage/72-extensions/40-configuration-extension.md +++ b/docs/versioned_docs/version-3.16/20-usage/72-extensions/40-configuration-extension.md @@ -23,25 +23,36 @@ As Woodpecker will pass private information like tokens and will execute the ret In addition to the ability to configure the extension per repository, you can also configure a global endpoint in the Woodpecker server configuration. This can be useful if you want to use the extension for all repositories. Be careful if you share your Woodpecker server with others as they will also use your configuration extension. -The global configuration will be called before the repository specific configuration extension if both are configured. +The global configuration will be called before the repository specific configuration extension if both are configured and the repository has not enabled the exclusive setting. ```ini title="Server" -WOODPECKER_CONFIG_SERVICE_ENDPOINT=https://example.com/ciconfig +WOODPECKER_CONFIG_EXTENSION_ENDPOINT=https://example.com/ciconfig ``` ## How it works When a pipeline is triggered Woodpecker will fetch the pipeline configuration from the repository, then make a HTTP POST request to the configured extension with a JSON payload containing some data like the repository, pipeline information and the current config files retrieved from the repository. The extension can then send back modified or even new pipeline configurations following Woodpeckers official yaml format that should be used. +You can enable the exclusive setting (both globally and on a per-repo level). Then Woodpecker will only call your extension, but nothing else. This allows you to completely skip the forge. Requests sent to the extension will not have the configuration files added. + ### Request The extension receives an HTTP POST request with the following JSON payload: +:::info +The `netrc` field is only included in the request when the global `WOODPECKER_CONFIG_EXTENSION_NETRC` is set to `true` (default: `false`) or the per-repo "Send netrc credentials" is checked. +::: + ```ts class Request { repo: Repo; pipeline: Pipeline; - netrc: Netrc; + netrc?: Netrc; // only included when netrc sending is enabled (see above) + configuration?: { + // list of configurations. Not send if there was none. + name: string; // filename of the configuration file + data: string; // content of the configuration file + }[]; } ``` @@ -52,15 +63,12 @@ Checkout the following models for more information: - [netrc model](https://github.com/woodpecker-ci/woodpecker/blob/main/server/model/netrc.go) :::tip -The `netrc` data is pretty powerful as it contains credentials to access the repository. You can use this to fetch files or other information (like changed files, issues) from the repository using the forge api or even clone the repository. +The `netrc` data is pretty powerful as it contains credentials to access the repository. You can use this to clone the repository or even use the forge (Github or Gitlab, ...) API to get more information about the repository. ::: Example request: ```json -// Please check the latest structure in the models mentioned above. -// This example is likely outdated. - { "repo": { "id": 100, @@ -122,11 +130,16 @@ Example request: "updated_at": 0, "verified": false }, + "configuration": [ + { + "name": ".woodpecker.yaml", + "data": "steps:\n - name: backend\n image: alpine\n commands:\n - echo \"Hello there from Repo (.woodpecker.yaml)\"\n" + } + ], "netrc": { "machine": "myforge.com", "login": "myUser", - "password": "myPassword", - "type": "forge" + "password": "forge-access-token" } } ``` diff --git a/docs/versioned_docs/version-3.16/20-usage/72-extensions/50-registry-extension.md b/docs/versioned_docs/version-3.16/20-usage/72-extensions/50-registry-extension.md new file mode 100644 index 000000000..c4f460327 --- /dev/null +++ b/docs/versioned_docs/version-3.16/20-usage/72-extensions/50-registry-extension.md @@ -0,0 +1,160 @@ +# Registry extension + +Woodpecker uses the registry extension to get registry credentials. You can configure an HTTP endpoint in the repository settings in the extensions tab. + +Using such an extension can be useful if you want to: + +- Centralize registry credential management +- Use an external storage for credentials +- Dynamically manage which credentials Woodpecker should use + +## Security + +:::warning +As Woodpecker will pass private information like tokens and will execute the returned configuration, it is extremely important to secure the external extension. Therefore Woodpecker signs every request. Read more about it in the [security section](./index.md#security). +::: + +## Global configuration + +In addition to the ability to configure the extension per repository, you can also configure a global endpoint in the Woodpecker server configuration. This can be useful if you want to use the extension for all repositories. Be careful if +you share your Woodpecker server with others as they will also use your registry extension. + +If both the global and the repo-level extension return credentials for a registry, it will use the credentials from the repo extension. + +```ini title="Server" +WOODPECKER_REGISTRY_EXTENSION_ENDPOINT=https://example.com/ciconfig +``` + +## How it works + +When a pipeline is triggered, Woodpecker will fetch the credentials from your service. As fallback, it uses the credentials configured directly in Woodpecker. + +### Request + +The extension receives an HTTP POST request with the following JSON payload: + +:::info +The `netrc` field is only included in the request when the global `WOODPECKER_REGISTRY_EXTENSION_NETRC` is set to `true` (default: `false`) or the per-repo "Send netrc credentials" is checked. +::: + +```ts +class Request { + repo: Repo; + pipeline: Pipeline; + netrc?: Netrc; // only included when netrc sending is enabled (see above) +} +``` + +Checkout the following models for more information: + +- [repo model](https://github.com/woodpecker-ci/woodpecker/blob/main/server/model/repo.go) +- [pipeline model](https://github.com/woodpecker-ci/woodpecker/blob/main/server/model/pipeline.go) +- [netrc model](https://github.com/woodpecker-ci/woodpecker/blob/main/server/model/netrc.go) + +:::tip +The `netrc` data is pretty powerful as it contains credentials to access the repository. You can use this to clone the repository or even use the forge (Github or Gitlab, ...) API to get more information about the repository. +::: + +Example request: + +```json +// Please check the latest structure in the models mentioned above. +// This example is likely outdated. + +{ + "repo": { + "id": 100, + "uid": "", + "user_id": 0, + "namespace": "", + "name": "woodpecker-test-pipeline", + "slug": "", + "scm": "git", + "git_http_url": "", + "git_ssh_url": "", + "link": "", + "default_branch": "", + "private": true, + "visibility": "private", + "active": true, + "config": "", + "trusted": false, + "protected": false, + "ignore_forks": false, + "ignore_pulls": false, + "cancel_pulls": false, + "timeout": 60, + "counter": 0, + "synced": 0, + "created": 0, + "updated": 0, + "version": 0 + }, + "pipeline": { + "author": "myUser", + "author_avatar": "https://myforge.com/avatars/d6b3f7787a685fcdf2a44e2c685c7e03", + "author_email": "my@email.com", + "branch": "main", + "changed_files": ["some-filename.txt"], + "commit": "2fff90f8d288a4640e90f05049fe30e61a14fd50", + "created_at": 0, + "deploy_to": "", + "enqueued_at": 0, + "error": "", + "event": "push", + "finished_at": 0, + "id": 0, + "link_url": "https://myforge.com/myUser/woodpecker-testpipe/commit/2fff90f8d288a4640e90f05049fe30e61a14fd50", + "message": "test old config\n", + "number": 0, + "parent": 0, + "ref": "refs/heads/main", + "refspec": "", + "clone_url": "", + "reviewed_at": 0, + "reviewed_by": "", + "sender": "myUser", + "signed": false, + "started_at": 0, + "status": "", + "timestamp": 1645962783, + "title": "", + "updated_at": 0, + "verified": false + }, + "netrc": { + "machine": "myforge.com", + "login": "myUser", + "password": "forge-access-token" + } +} +``` + +### Response + +The extension should respond with a JSON payload containing the new configuration files in Woodpecker's official YAML format. +If the extension wants to keep the existing configuration files, it can respond with HTTP status `204 No Content`. + +```ts +class Response { + registries: { + address: string; // the docker registry address + username: string; // registry username + password: string; // registry password + }[]; +} +``` + +Example response: + +```json +{ + "registries": [ + { + "address": "docker.io", + "username": "woodpecker-bot", + "password": "your-pass-word-123" + } + ] +} +``` diff --git a/docs/versioned_docs/version-3.16/20-usage/72-extensions/55-secret-extension.md b/docs/versioned_docs/version-3.16/20-usage/72-extensions/55-secret-extension.md new file mode 100644 index 000000000..f1d9634e4 --- /dev/null +++ b/docs/versioned_docs/version-3.16/20-usage/72-extensions/55-secret-extension.md @@ -0,0 +1,174 @@ +# Secret extension + +Woodpecker uses the secret extension to get secrets from an external service. You can configure an HTTP endpoint in the repository settings in the extensions tab. + +Using such an extension can be useful if you want to: + +- Centralize secret management (e.g. HashiCorp Vault, AWS Secrets Manager) +- Dynamically generate secrets per pipeline + +## Security + +:::warning +As Woodpecker will pass private information like tokens and will execute the returned configuration, it is extremely important to secure the external extension. Therefore Woodpecker signs every request. Read more about it in the security section. +::: + +## Global configuration + +In addition to the ability to configure the extension per repository, you can also configure a global endpoint in the Woodpecker server configuration. This can be useful if you want to use the extension for all repositories. Be careful if +you share your Woodpecker server with others as they will also use your secret extension. + +If both the global and the repo-level extension return a secret with the same name, it will use the secret from the repo extension. + +```ini title="Server" +WOODPECKER_SECRET_EXTENSION_ENDPOINT=https://example.com/secrets +WOODPECKER_SECRET_EXTENSION_NETRC=false +``` + +## How it works + +When a pipeline is triggered, Woodpecker will fetch secrets from your service. The extension secrets are merged with the secrets configured directly in Woodpecker, with extension secrets taking priority by name. If the extension is unavailable, Woodpecker falls back to the locally configured secrets. + +### Request + +The extension receives an HTTP POST request with the following JSON payload: + +:::info +The `netrc` field is only included in the request when the global `WOODPECKER_SECRET_EXTENSION_NETRC` is set to `true` (default: `false`) or the per-repo "Send netrc credentials" is checked. +::: + +```ts +class Request { + repo: Repo; + pipeline: Pipeline; + netrc?: Netrc; // only included when netrc sending is enabled (see above) +} +``` + +Checkout the following models for more information: + +- [repo model](https://github.com/woodpecker-ci/woodpecker/blob/main/server/model/repo.go) +- [pipeline model](https://github.com/woodpecker-ci/woodpecker/blob/main/server/model/pipeline.go) +- [netrc model](https://github.com/woodpecker-ci/woodpecker/blob/main/server/model/netrc.go) + +:::tip +The `netrc` data is pretty powerful as it contains credentials to access the repository. You can use this to clone the repository or even use the forge (Github or Gitlab, ...) API to get more information about the repository. +::: + +Example request: + +```json +// Please check the latest structure in the models mentioned above. +// This example is likely outdated. + +{ + "repo": { + "id": 100, + "uid": "", + "user_id": 0, + "namespace": "", + "name": "woodpecker-test-pipeline", + "slug": "", + "scm": "git", + "git_http_url": "", + "git_ssh_url": "", + "link": "", + "default_branch": "", + "private": true, + "visibility": "private", + "active": true, + "config": "", + "trusted": false, + "protected": false, + "ignore_forks": false, + "ignore_pulls": false, + "cancel_pulls": false, + "timeout": 60, + "counter": 0, + "synced": 0, + "created": 0, + "updated": 0, + "version": 0 + }, + "pipeline": { + "author": "myUser", + "author_avatar": "https://myforge.com/avatars/d6b3f7787a685fcdf2a44e2c685c7e03", + "author_email": "my@email.com", + "branch": "main", + "changed_files": ["some-filename.txt"], + "commit": "2fff90f8d288a4640e90f05049fe30e61a14fd50", + "created_at": 0, + "deploy_to": "", + "enqueued_at": 0, + "error": "", + "event": "push", + "finished_at": 0, + "id": 0, + "link_url": "https://myforge.com/myUser/woodpecker-testpipe/commit/2fff90f8d288a4640e90f05049fe30e61a14fd50", + "message": "test old config\n", + "number": 0, + "parent": 0, + "ref": "refs/heads/main", + "refspec": "", + "clone_url": "", + "reviewed_at": 0, + "reviewed_by": "", + "sender": "myUser", + "signed": false, + "started_at": 0, + "status": "", + "timestamp": 1645962783, + "title": "", + "updated_at": 0, + "verified": false + }, + "netrc": { + "machine": "myforge.com", + "login": "myUser", + "password": "forge-access-token" + } +} +// Note: the "netrc" field is omitted when netrc sending is not enabled. +``` + +### Response + +The extension should respond with a JSON object containing a `secrets` array. +If the extension wants to keep the existing secrets without adding any, it can respond with HTTP status `204 No Content`. + +```ts +class Response { + secrets: { + name: string; // the secret name, matched by from_secret in pipeline config + value: string; // the secret value + images?: string[]; // optional: restrict to specific plugins + events?: string[]; // optional: restrict to specific pipeline events + }[]; +} +``` + +Example response: + +```json +{ + "secrets": [ + { + "name": "docker_password", + "value": "your-secret-password-123" + }, + { + "name": "deploy_token", + "value": "super-secret-token", + "events": ["push", "tag"] + } + ] +} +``` + +## 3rd Party Extensions + +:::danger +These extensions are neither developed nor verified by Woodpecker CI. Make sure you trust them before using. +::: + +_Add your extension here!_ diff --git a/docs/versioned_docs/version-3.13/20-usage/72-extensions/_category_.yaml b/docs/versioned_docs/version-3.16/20-usage/72-extensions/_category_.yaml similarity index 100% rename from docs/versioned_docs/version-3.13/20-usage/72-extensions/_category_.yaml rename to docs/versioned_docs/version-3.16/20-usage/72-extensions/_category_.yaml diff --git a/docs/versioned_docs/version-3.13/20-usage/72-extensions/index.md b/docs/versioned_docs/version-3.16/20-usage/72-extensions/index.md similarity index 79% rename from docs/versioned_docs/version-3.13/20-usage/72-extensions/index.md rename to docs/versioned_docs/version-3.16/20-usage/72-extensions/index.md index e30cb53f4..0c85187bf 100644 --- a/docs/versioned_docs/version-3.13/20-usage/72-extensions/index.md +++ b/docs/versioned_docs/version-3.16/20-usage/72-extensions/index.md @@ -5,6 +5,12 @@ Woodpecker allows you to replace internal logic with external extensions by usin There is currently one type of extension available: - [Configuration extension](./40-configuration-extension.md) to modify or generate pipeline configurations on the fly. +- [Registry extension](./50-registry-extension.md) to get registry credentials from the extension. +- [Secret extension](./55-secret-extension.md) to get secrets from an external service. + +:::note +Woodpecker's permission handling is linked to the forge. A user on your forge that has admin access to the repo will also get admin permissions for the repository in Woodpecker and can then change the configured extensions. This could be used to get credentials of the forge user. Make sure you trust the repo admins that can sign in to Woodpecker. +::: ## Security diff --git a/docs/versioned_docs/version-3.13/20-usage/72-linter.md b/docs/versioned_docs/version-3.16/20-usage/72-linter.md similarity index 100% rename from docs/versioned_docs/version-3.13/20-usage/72-linter.md rename to docs/versioned_docs/version-3.16/20-usage/72-linter.md diff --git a/docs/versioned_docs/version-3.16/20-usage/73-local-execution.md b/docs/versioned_docs/version-3.16/20-usage/73-local-execution.md new file mode 100644 index 000000000..004735ef5 --- /dev/null +++ b/docs/versioned_docs/version-3.16/20-usage/73-local-execution.md @@ -0,0 +1,87 @@ +# Local pipeline execution + +`woodpecker-cli exec` runs workflow files from your local checkout. Use it to test pipeline changes before pushing them, to debug a workflow without waiting for a server run, or to replay a server pipeline with downloaded metadata. + +## Requirements + +- Install `woodpecker-cli` from the [distribution packages](../30-administration/05-installation/30-packages.md) or a release archive. +- Run the command from the repository checkout, or pass `--repo-path` to point at it. +- Make sure the backend you want to use is available locally. The Docker backend needs access to a Docker daemon. The local backend runs commands directly on your host and does not reproduce the container image environment. + +## Run a workflow file + +Create or edit a workflow file, then run it directly: + +```shell +woodpecker-cli exec .woodpecker/my-first-workflow.yaml +``` + +You can also run every `.yaml` and `.yml` file in a workflow directory: + +```shell +woodpecker-cli exec .woodpecker/ +``` + +By default, Woodpecker auto-detects a backend. Select one explicitly when you want the local run to match a specific agent backend: + +```shell +woodpecker-cli exec --backend-engine docker .woodpecker/my-first-workflow.yaml +woodpecker-cli exec --backend-engine local .woodpecker/my-first-workflow.yaml +``` + +## Pass metadata + +Metadata values are set automatically, but you can override them to test conditions such as branches, pull requests, tags, and events: + +```shell +woodpecker-cli exec \ + --pipeline-event push \ + --commit-branch main \ + --commit-sha "$(git rev-parse HEAD)" \ + --repo octocat/hello-world \ + .woodpecker/my-first-workflow.yaml +``` + +If you downloaded pipeline metadata from the Woodpecker UI, pass it with `--metadata-file` and adjust individual values with other flags when needed: + +```shell +woodpecker-cli exec \ + --metadata-file pipeline-metadata.json \ + --pipeline-event pull_request \ + .woodpecker/my-first-workflow.yaml +``` + +## Pass environment variables and secrets + +Use `--env` for regular environment variables: + +```shell +woodpecker-cli exec \ + --env GOFLAGS=-mod=readonly \ + .woodpecker/test.yaml +``` + +Secrets are not downloaded from the server. Pass the values needed for local debugging explicitly: + +```shell +woodpecker-cli exec \ + --secrets deploy_token="$DEPLOY_TOKEN" \ + .woodpecker/deploy.yaml +``` + +For multiple secrets, keep them in a local YAML file that is ignored by Git: + +```yaml title=".woodpecker/local-secrets.yaml" +deploy_token: ghp_example +registry_password: example-password +``` + +```shell +woodpecker-cli exec \ + --secrets-file .woodpecker/local-secrets.yaml \ + .woodpecker/deploy.yaml +``` + +## More options + +See the generated [CLI reference](../40-cli.md#exec) for the full list of `exec` flags. diff --git a/docs/versioned_docs/version-3.13/20-usage/75-project-settings.md b/docs/versioned_docs/version-3.16/20-usage/75-project-settings.md similarity index 94% rename from docs/versioned_docs/version-3.13/20-usage/75-project-settings.md rename to docs/versioned_docs/version-3.16/20-usage/75-project-settings.md index 170997b30..0a73cb19c 100644 --- a/docs/versioned_docs/version-3.13/20-usage/75-project-settings.md +++ b/docs/versioned_docs/version-3.16/20-usage/75-project-settings.md @@ -4,6 +4,10 @@ As the owner of a project in Woodpecker you can change project related settings ![project settings](./project-settings.png) +:::note +Woodpecker's permission handling is linked to the forge. A user on your forge that has admin access to the repo will also get admin permissions for the repository in Woodpecker and can then change the settings here. +::: + ## Pipeline path The path to the pipeline config file or folder. By default it is left empty which will use the following configuration resolution `.woodpecker/*.{yaml,yml}` -> `.woodpecker.yaml` -> `.woodpecker.yml`. If you set a custom path Woodpecker tries to load your configuration or fails if no configuration could be found at the specified location. To use a [multiple workflows](./25-workflows.md) with a custom path you have to change it to a folder path ending with a `/` like `.woodpecker/`. diff --git a/docs/versioned_docs/version-3.13/20-usage/80-badges.md b/docs/versioned_docs/version-3.16/20-usage/80-badges.md similarity index 100% rename from docs/versioned_docs/version-3.13/20-usage/80-badges.md rename to docs/versioned_docs/version-3.16/20-usage/80-badges.md diff --git a/docs/versioned_docs/version-3.13/20-usage/90-advanced-usage.md b/docs/versioned_docs/version-3.16/20-usage/90-advanced-usage.md similarity index 100% rename from docs/versioned_docs/version-3.13/20-usage/90-advanced-usage.md rename to docs/versioned_docs/version-3.16/20-usage/90-advanced-usage.md diff --git a/docs/versioned_docs/version-3.13/20-usage/_category_.yaml b/docs/versioned_docs/version-3.16/20-usage/_category_.yaml similarity index 100% rename from docs/versioned_docs/version-3.13/20-usage/_category_.yaml rename to docs/versioned_docs/version-3.16/20-usage/_category_.yaml diff --git a/docs/versioned_docs/version-3.13/20-usage/cron-settings.png b/docs/versioned_docs/version-3.16/20-usage/cron-settings.png similarity index 100% rename from docs/versioned_docs/version-3.13/20-usage/cron-settings.png rename to docs/versioned_docs/version-3.16/20-usage/cron-settings.png diff --git a/docs/versioned_docs/version-3.13/20-usage/linter-warnings-errors.png b/docs/versioned_docs/version-3.16/20-usage/linter-warnings-errors.png similarity index 100% rename from docs/versioned_docs/version-3.13/20-usage/linter-warnings-errors.png rename to docs/versioned_docs/version-3.16/20-usage/linter-warnings-errors.png diff --git a/docs/versioned_docs/version-3.13/20-usage/pipeline.png b/docs/versioned_docs/version-3.16/20-usage/pipeline.png similarity index 100% rename from docs/versioned_docs/version-3.13/20-usage/pipeline.png rename to docs/versioned_docs/version-3.16/20-usage/pipeline.png diff --git a/docs/versioned_docs/version-3.13/20-usage/project-settings.png b/docs/versioned_docs/version-3.16/20-usage/project-settings.png similarity index 100% rename from docs/versioned_docs/version-3.13/20-usage/project-settings.png rename to docs/versioned_docs/version-3.16/20-usage/project-settings.png diff --git a/docs/versioned_docs/version-3.13/20-usage/repo-new.png b/docs/versioned_docs/version-3.16/20-usage/repo-new.png similarity index 100% rename from docs/versioned_docs/version-3.13/20-usage/repo-new.png rename to docs/versioned_docs/version-3.16/20-usage/repo-new.png diff --git a/docs/versioned_docs/version-3.13/20-usage/secrets-plugins-filter.png b/docs/versioned_docs/version-3.16/20-usage/secrets-plugins-filter.png similarity index 100% rename from docs/versioned_docs/version-3.13/20-usage/secrets-plugins-filter.png rename to docs/versioned_docs/version-3.16/20-usage/secrets-plugins-filter.png diff --git a/docs/versioned_docs/version-3.13/30-administration/00-general.md b/docs/versioned_docs/version-3.16/30-administration/00-general.md similarity index 92% rename from docs/versioned_docs/version-3.13/30-administration/00-general.md rename to docs/versioned_docs/version-3.16/30-administration/00-general.md index 0ce0ee088..9937ff08c 100644 --- a/docs/versioned_docs/version-3.13/30-administration/00-general.md +++ b/docs/versioned_docs/version-3.16/30-administration/00-general.md @@ -41,3 +41,5 @@ Images are pushed to DockerHub and Quay. - woodpecker-agent ([DockerHub](https://hub.docker.com/r/woodpeckerci/woodpecker-agent) or [Quay](https://quay.io/repository/woodpeckerci/woodpecker-agent)) - woodpecker-cli ([DockerHub](https://hub.docker.com/r/woodpeckerci/woodpecker-cli) or [Quay](https://quay.io/repository/woodpeckerci/woodpecker-cli)) - woodpecker-autoscaler ([DockerHub](https://hub.docker.com/r/woodpeckerci/autoscaler)) + +For the full list of operating systems and architectures each component is built for, along with which execution backends are supported on each platform, see [Supported platforms](./05-installation/05-supported-platforms.md). diff --git a/docs/versioned_docs/version-3.16/30-administration/05-installation/05-supported-platforms.md b/docs/versioned_docs/version-3.16/30-administration/05-installation/05-supported-platforms.md new file mode 100644 index 000000000..0a3eb5d2d --- /dev/null +++ b/docs/versioned_docs/version-3.16/30-administration/05-installation/05-supported-platforms.md @@ -0,0 +1,57 @@ +# Supported platforms + +Woodpecker is shipped as container images and as pre-built binaries on the [GitHub releases](https://github.com/woodpecker-ci/woodpecker/releases/latest) page. Not every component is available for every platform: the server and the Docker/Kubernetes backends are Linux-centric, while the agent and CLI run on a wider set of operating systems via the Local backend. + +## Components + +| Component | Purpose | +| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `woodpecker-server` | Web UI, API, webhook receiver, pipeline scheduler. | +| `woodpecker-agent` | Executes pipeline workflows via a backend ([Docker](../10-configuration/11-backends/10-docker.md), [Kubernetes](../10-configuration/11-backends/20-kubernetes.md), [Local](../10-configuration/11-backends/30-local.md)). | +| `woodpecker-cli` | Command-line utility for interacting with the server. | +| `plugin-git` | Default clone plugin, invoked automatically by the agent at the start of every workflow. Distributed as a container image; binaries are also published for use with the Local backend. | + +## Component / platform matrix + +The table lists what is officially built and published by the Woodpecker project. "Image" means a container image is pushed to DockerHub and Quay. "Binary" means a pre-built tarball or `.exe` is attached to the GitHub release. + +| OS / Architecture | server | agent | cli | plugin-git | +| ------------------------------------ | -------------- | -------------- | -------------- | -------------- | +| linux / amd64 | Image + Binary | Image + Binary | Image + Binary | Image + Binary | +| linux / arm64 (arm64/v8) | Image + Binary | Image + Binary | Image + Binary | Image + Binary | +| linux / arm/v7 | Image | Image + Binary | Image + Binary | Image + Binary | +| linux / arm/v6 | Image | Image | Image | Image | +| linux / 386 | Image | Image | Image | Image | +| linux / ppc64le | Image | Image | Image | Image | +| linux / riscv64 | Image + Binary | Image + Binary | Image + Binary | Image | +| linux / s390x | Image | Image | Image | Image | +| windows / amd64 | Binary | Binary | Binary | Binary | +| windows / arm64 | – | – | – | Binary | +| darwin / amd64 (macOS Intel) | – | Binary | Binary | Binary | +| darwin / arm64 (macOS Apple Silicon) | – | Binary | Binary | Binary | +| freebsd / amd64 | Image + Binary | Image + Binary | Image + Binary | Binary | +| freebsd / arm64 | – | Image + Binary | Image + Binary | Binary | +| openbsd / amd64 | – | Binary | Binary | Binary | +| openbsd / arm64 | – | Binary | Binary | Binary | + +DEB and RPM packages are produced for `linux/amd64` and `linux/arm64`; see the [Distribution packages](./30-packages.md) page for download links and systemd unit examples. + +## Backend support per platform + +The agent can run on any platform listed above, but the available execution backends depend on the host operating system. + +| Backend | Linux | Windows | macOS | FreeBSD | OpenBSD | +| -------------------------------------------------------------- | --------- | ---------------------- | --------- | ---------------------- | --------- | +| [Docker](../10-configuration/11-backends/10-docker.md) | Supported | Supported[^win-docker] | – | [WIP][^freebsd-docker] | – | +| [Kubernetes](../10-configuration/11-backends/20-kubernetes.md) | Supported | – | – | – | – | +| [Local](../10-configuration/11-backends/30-local.md) | Supported | Supported | Supported | Supported | Supported | + +[^win-docker]: Works through WSL2 with Docker Desktop, and with native Windows containers. + +[^freebsd-docker]: FreeBSD Docker backend support is a work in progress; see [woodpecker-ci/woodpecker#6655](https://github.com/woodpecker-ci/woodpecker/issues/6655). + +Notes: + +- The **Docker** and **Kubernetes** backends require a Linux host on the agent because they rely on Linux container runtimes. On Windows, Docker is available via WSL2 or Windows containers (see footnote above). Running the agent on macOS or OpenBSD restricts you to the Local backend. +- The **Local** backend runs pipeline commands directly on the agent host with no isolation. It is the only backend available on macOS and OpenBSD, and is intended for trusted, private setups only. See the [Local backend documentation](../10-configuration/11-backends/30-local.md) for the full security notes. +- `plugin-git` is invoked as a container by default. On hosts where the Docker and Kubernetes backends are unavailable, configure the Local backend to use the [`plugin-git` binary](https://github.com/woodpecker-ci/plugin-git/releases/latest) instead, or disable the clone step and clone manually in the pipeline. diff --git a/docs/versioned_docs/version-3.13/30-administration/05-installation/10-docker-compose.md b/docs/versioned_docs/version-3.16/30-administration/05-installation/10-docker-compose.md similarity index 91% rename from docs/versioned_docs/version-3.13/30-administration/05-installation/10-docker-compose.md rename to docs/versioned_docs/version-3.16/30-administration/05-installation/10-docker-compose.md index 3db1569b0..3567dd9d3 100644 --- a/docs/versioned_docs/version-3.13/30-administration/05-installation/10-docker-compose.md +++ b/docs/versioned_docs/version-3.16/30-administration/05-installation/10-docker-compose.md @@ -2,7 +2,7 @@ This example [docker-compose](https://docs.docker.com/compose/) setup shows the deployment of a Woodpecker instance connected to GitHub (`WOODPECKER_GITHUB=true`). If you are using another forge, please change this including the respective secret settings. -Before starting, you will need to register an OAuth App with your forge — see the [forge documentation](https://woodpecker-ci.org/docs/3.13/administration/configuration/forges/overview) for instructions. +Before starting, you will need to register an OAuth App with your forge — see the [forge documentation](https://woodpecker-ci.org/docs/administration/configuration/forges/overview) for instructions. It creates persistent volumes for the server and agent config directories. The bundled SQLite DB is stored in `/var/lib/woodpecker` and is the most important part to be persisted as it holds all users and repository information. @@ -142,3 +142,14 @@ To store values in a docker secret you can use the following command: ```bash echo "my_agent_secret_key" | docker secret create woodpecker-agent-secret - ``` + +## SELinux Considerations + +If you're running Woodpecker on a system with SELinux enabled (RHEL, CentOS, Fedora, etc.), you may need to add the `:z` or `:Z` option to volume mounts. For the Docker socket volume: + +```yaml +volumes: + - /var/run/docker.sock:/var/run/docker.sock:z +``` + +For more details and other SELinux-related solutions, see the [Troubleshooting](../../20-usage/100-troubleshooting.md#selinux-issues) page. diff --git a/docs/versioned_docs/version-3.13/30-administration/05-installation/20-helm-chart.md b/docs/versioned_docs/version-3.16/30-administration/05-installation/20-helm-chart.md similarity index 100% rename from docs/versioned_docs/version-3.13/30-administration/05-installation/20-helm-chart.md rename to docs/versioned_docs/version-3.16/30-administration/05-installation/20-helm-chart.md diff --git a/docs/versioned_docs/version-3.13/30-administration/05-installation/30-packages.md b/docs/versioned_docs/version-3.16/30-administration/05-installation/30-packages.md similarity index 92% rename from docs/versioned_docs/version-3.13/30-administration/05-installation/30-packages.md rename to docs/versioned_docs/version-3.16/30-administration/05-installation/30-packages.md index ca4c55e2f..58df3ebe2 100644 --- a/docs/versioned_docs/version-3.13/30-administration/05-installation/30-packages.md +++ b/docs/versioned_docs/version-3.16/30-administration/05-installation/30-packages.md @@ -1,10 +1,16 @@ # Distribution packages +:::tip +For a full list of operating systems and architectures that Woodpecker is built for, including which components are available on each, see [Supported platforms](./05-supported-platforms.md). +::: + ## Official packages - DEB - RPM +DEB and RPM packages are built for `linux/amd64` and `linux/arm64`. For other architectures, use the binary tarballs or container images linked from the [Supported platforms](./05-supported-platforms.md) page. + The pre-built packages are available on the [GitHub releases](https://github.com/woodpecker-ci/woodpecker/releases/latest) page. The packages can be installed using the package manager of your distribution. ```Shell @@ -92,6 +98,7 @@ Woodpecker itself is not responsible for creating these packages. Please reach o - [YunoHost](https://apps.yunohost.org/app/woodpecker) - [Cloudron](https://www.cloudron.io/store/org.woodpecker_ci.cloudronapp.html) - [Easypanel](https://easypanel.io/docs/templates/woodpeckerci) +- [Homebrew](https://formulae.brew.sh/formula/woodpecker-cli) (CLI only) ### NixOS diff --git a/docs/versioned_docs/version-3.13/30-administration/05-installation/_category_.yaml b/docs/versioned_docs/version-3.16/30-administration/05-installation/_category_.yaml similarity index 100% rename from docs/versioned_docs/version-3.13/30-administration/05-installation/_category_.yaml rename to docs/versioned_docs/version-3.16/30-administration/05-installation/_category_.yaml diff --git a/docs/versioned_docs/version-3.13/30-administration/10-configuration/10-server.md b/docs/versioned_docs/version-3.16/30-administration/10-configuration/10-server.md similarity index 83% rename from docs/versioned_docs/version-3.13/30-administration/10-configuration/10-server.md rename to docs/versioned_docs/version-3.16/30-administration/10-configuration/10-server.md index 61e67b109..d6bb942ba 100644 --- a/docs/versioned_docs/version-3.13/30-administration/10-configuration/10-server.md +++ b/docs/versioned_docs/version-3.16/30-administration/10-configuration/10-server.md @@ -400,98 +400,26 @@ woodpecker_waiting_steps 0 # HELP woodpecker_worker_count Total number of workers. # TYPE woodpecker_worker_count gauge woodpecker_worker_count 4 +# HELP woodpecker_step_failures_total Total number of pipeline step failures. +# TYPE woodpecker_step_failures_total counter +woodpecker_step_failures_total{repo="woodpecker-ci/woodpecker",step="deploy",workflow="woodpecker"} 1 +# HELP woodpecker_step_duration_seconds Step duration in seconds. +# TYPE woodpecker_step_duration_seconds histogram +woodpecker_step_duration_seconds_bucket{repo="woodpecker-ci/woodpecker",step="deploy",workflow="woodpecker",le="1"} 0 +woodpecker_step_duration_seconds_bucket{repo="woodpecker-ci/woodpecker",step="deploy",workflow="woodpecker",le="5"} 0 +woodpecker_step_duration_seconds_bucket{repo="woodpecker-ci/woodpecker",step="deploy",workflow="woodpecker",le="10"} 0 +woodpecker_step_duration_seconds_bucket{repo="woodpecker-ci/woodpecker",step="deploy",workflow="woodpecker",le="30"} 1 +woodpecker_step_duration_seconds_bucket{repo="woodpecker-ci/woodpecker",step="deploy",workflow="woodpecker",le="60"} 1 +woodpecker_step_duration_seconds_bucket{repo="woodpecker-ci/woodpecker",step="deploy",workflow="woodpecker",le="300"} 1 +woodpecker_step_duration_seconds_bucket{repo="woodpecker-ci/woodpecker",step="deploy",workflow="woodpecker",le="600"} 1 +woodpecker_step_duration_seconds_bucket{repo="woodpecker-ci/woodpecker",step="deploy",workflow="woodpecker",le="1800"} 1 +woodpecker_step_duration_seconds_bucket{repo="woodpecker-ci/woodpecker",step="deploy",workflow="woodpecker",le="3600"} 1 +woodpecker_step_duration_seconds_bucket{repo="woodpecker-ci/woodpecker",step="deploy",workflow="woodpecker",le="+Inf"} 1 +woodpecker_step_duration_seconds_sum{repo="woodpecker-ci/woodpecker",step="deploy",workflow="woodpecker"} 12 +woodpecker_step_duration_seconds_count{repo="woodpecker-ci/woodpecker",step="deploy",workflow="woodpecker"} 1 ``` -## External Configuration API - -To provide additional management and preprocessing capabilities for pipeline configurations Woodpecker supports an HTTP API which can be enabled to call an external config service. -Before the run or restart of any pipeline Woodpecker will make a POST request to an external HTTP API sending the current repository, build information and all current config files retrieved from the repository. The external API can then send back new pipeline configurations that will be used immediately or respond with `HTTP 204` to tell the system to use the existing configuration. - -Every request sent by Woodpecker is signed using a [http-signature](https://datatracker.ietf.org/doc/html/rfc9421) by a private key (ed25519) generated on the first start of the Woodpecker server. You can get the public key for the verification of the http-signature from `http(s)://your-woodpecker-server/api/signature/public-key`. - -A simplistic example configuration service can be found here: [https://github.com/woodpecker-ci/example-config-service](https://github.com/woodpecker-ci/example-config-service) - -:::warning -You need to trust the external config service as it is getting secret information about the repository and pipeline and has the ability to change pipeline configs that could run malicious tasks. -::: - -### Configuration - -```ini title="Server" -WOODPECKER_CONFIG_SERVICE_ENDPOINT=https://example.com/ciconfig -``` - -#### Example request made by Woodpecker - -```json -{ - "repo": { - "id": 100, - "uid": "", - "user_id": 0, - "namespace": "", - "name": "woodpecker-test-pipe", - "slug": "", - "scm": "git", - "git_http_url": "", - "git_ssh_url": "", - "link": "", - "default_branch": "", - "private": true, - "visibility": "private", - "active": true, - "config": "", - "trusted": false, - "protected": false, - "ignore_forks": false, - "ignore_pulls": false, - "cancel_pulls": false, - "timeout": 60, - "counter": 0, - "synced": 0, - "created": 0, - "updated": 0, - "version": 0 - }, - "pipeline": { - "author": "myUser", - "author_avatar": "https://myforge.com/avatars/d6b3f7787a685fcdf2a44e2c685c7e03", - "author_email": "my@email.com", - "branch": "main", - "changed_files": ["some-file-name.txt"], - "commit": "2fff90f8d288a4640e90f05049fe30e61a14fd50", - "created_at": 0, - "deploy_to": "", - "enqueued_at": 0, - "error": "", - "event": "push", - "finished_at": 0, - "id": 0, - "link_url": "https://myforge.com/myUser/woodpecker-testpipe/commit/2fff90f8d288a4640e90f05049fe30e61a14fd50", - "message": "test old config\n", - "number": 0, - "parent": 0, - "ref": "refs/heads/main", - "refspec": "", - "clone_url": "", - "reviewed_at": 0, - "reviewed_by": "", - "sender": "myUser", - "signed": false, - "started_at": 0, - "status": "", - "timestamp": 1645962783, - "title": "", - "updated_at": 0, - "verified": false - }, - "netrc": { - "machine": "https://example.com", - "login": "user", - "password": "password" - } -} -``` +Step-level metrics are exported as long as `WOODPECKER_STEP_LEVEL_METRICS` is not disabled. #### Example response structure @@ -649,7 +577,7 @@ Examples: - Name: `WOODPECKER_SERVER_ADDR` - Default: `:8000` -Configures the HTTP listener port. +Configures the HTTP listener, supports unix socket via unix:// prefix". --- @@ -715,7 +643,8 @@ Example: `WOODPECKER_CUSTOM_JS_FILE=/usr/local/www/woodpecker.js` - Name: `WOODPECKER_GRPC_ADDR` - Default: `:9000` -Configures the gRPC listener port. +Configures the gRPC listener. Use `localhost:9000` or any IP address to bind it to a specific interface. +If you want an unix socket use `unix://` prefix, for example `unix:///run/woodpecker-grcp.sock`. --- @@ -748,6 +677,15 @@ Example: `:9001` --- +### STEP_LEVEL_METRICS + +- Name: `WOODPECKER_STEP_LEVEL_METRICS` +- Default: `true` + +Enable step-level metrics, including failed step counters and step duration histograms. + +--- + ### ADMIN - Name: `WOODPECKER_ADMIN` @@ -799,6 +737,19 @@ Always use authentication to clone repositories even if they are public. Needed --- +### ASYNC_REPOSITORY_UPDATE + +- Name: `WOODPECKER_ASYNC_REPOSITORY_UPDATE` +- Default: `false` + +Enable asynchronous fetching user permissions for repositories. Will drastically improve login speed for user login if the organisation has many git repositories. + +When disabled (default) users will have to wait for all repository access information before being redirected to the Woodpecker homepage. Choose this for strong consistency. + +When enabled users will immediately be redirected to the Woodpecker homepage, but might see outdated information if repository access changed or new repositories were added. Choose this for eventual consistency. + +--- + ### DEFAULT_ALLOW_PULL_REQUESTS - Name: `WOODPECKER_DEFAULT_ALLOW_PULL_REQUESTS` @@ -1062,12 +1013,100 @@ Supported variables: --- -### CONFIG_SERVICE_ENDPOINT +### CONFIG_EXTENSION_ENDPOINT -- Name: `WOODPECKER_CONFIG_SERVICE_ENDPOINT` +- Name: `WOODPECKER_CONFIG_EXTENSION_ENDPOINT` - Default: none -Specify a configuration service endpoint, see [Configuration Extension](#external-configuration-api) +Specify a configuration extension endpoint, see [Configuration Extension](../../20-usage/72-extensions/40-configuration-extension.md) + +--- + +### DEFAULT_PIPELINE_CONFIGS + +- Name: `WOODPECKER_DEFAULT_PIPELINE_CONFIGS` +- Default: `.woodpecker/`, `.woodpecker.yaml`, `.woodpecker.yml` + +Specify the default pipeline config paths. + +--- + +### DEFAULT_PIPELINE_CONFIG_EXTENSIONS + +- Name: `WOODPECKER_DEFAULT_PIPELINE_CONFIG_EXTENSIONS` +- Default: `.yaml`, `.yml` + +Specify the default pipeline config extensions when scanning a pipeline config directory. + +--- + +### CONFIG_EXTENSION_EXCLUSIVE + +- Name: `CONFIG_EXTENSION_EXCLUSIVE` +- Default: false + +Whether the forge request should be skipped for the global configuration endpoint. + +:::warning +If you enable this, all repos will exclusively use the global config service endpoint. There is no possibility to directly define pipelines in the forge, except the extension handles this case itself as well. +::: + +--- + +### CONFIG_EXTENSION_NETRC + +- Name: `WOODPECKER_CONFIG_EXTENSION_NETRC` +- Default: false + +Send `netrc` to the config extension endpoint. + +:::warning +The `netrc` data is pretty powerful as it contains credentials to access the repository. You can use this to clone the repository or even use the forge API to get more information about the repository. +::: + +--- + +### SECRET_EXTENSION_ENDPOINT + +- Name: `WOODPECKER_SECRET_EXTENSION_ENDPOINT` +- Default: none + +Specify a secret extension endpoint, see [Secret Extension](../../20-usage/72-extensions/55-secret-extension.md) + +--- + +### SECRET_EXTENSION_NETRC + +- Name: `WOODPECKER_SECRET_EXTENSION_NETRC` +- Default: false + +Send `netrc` to the secret extension endpoint. + +:::warning +The `netrc` data is pretty powerful as it contains credentials to access the repository. You can use this to clone the repository or even use the forge API to get more information about the repository. +::: + +--- + +### REGISTRY_EXTENSION_ENDPOINT + +- Name: `WOODPECKER_REGISTRY_EXTENSION_ENDPOINT` +- Default: none + +Specify a registry extension endpoint, see [Registry Extension](../../20-usage/72-extensions/50-registry-extension.md) + +--- + +### REGISTRY_EXTENSION_NETRC + +- Name: `WOODPECKER_REGISTRY_EXTENSION_NETRC` +- Default: false + +Send `netrc` to the registry extension endpoint. + +:::warning +The `netrc` data is pretty powerful as it contains credentials to access the repository. You can use this to clone the repository or even use the forge API to get more information about the repository. +::: --- @@ -1167,6 +1206,19 @@ Fully qualified public forge URL, used if forge url is not a public URL. Format: --- +### FORCE_IGNORE_SERVICE_FAILURE + +- Name: `WOODPECKER_FORCE_IGNORE_SERVICE_FAILURE` +- Default: true + +:::warning +Since v3.14.0, Woodpecker can report the status of services and detached steps. +Because these can now fail, until v4.0.0 is released, service failures are ignored by default to preserve backward compatibility. +We encourage you to disable this option and update your pipeline configuration. +::: + +--- + ### GITHUB\_\* See [GitHub configuration](./12-forges/20-github.md#configuration) diff --git a/docs/versioned_docs/version-3.13/30-administration/10-configuration/100-addons.md b/docs/versioned_docs/version-3.16/30-administration/10-configuration/100-addons.md similarity index 100% rename from docs/versioned_docs/version-3.13/30-administration/10-configuration/100-addons.md rename to docs/versioned_docs/version-3.16/30-administration/10-configuration/100-addons.md diff --git a/docs/versioned_docs/version-3.13/30-administration/10-configuration/11-backends/10-docker.md b/docs/versioned_docs/version-3.16/30-administration/10-configuration/11-backends/10-docker.md similarity index 100% rename from docs/versioned_docs/version-3.13/30-administration/10-configuration/11-backends/10-docker.md rename to docs/versioned_docs/version-3.16/30-administration/10-configuration/11-backends/10-docker.md diff --git a/docs/versioned_docs/version-3.13/30-administration/10-configuration/11-backends/20-kubernetes.md b/docs/versioned_docs/version-3.16/30-administration/10-configuration/11-backends/20-kubernetes.md similarity index 86% rename from docs/versioned_docs/version-3.13/30-administration/10-configuration/11-backends/20-kubernetes.md rename to docs/versioned_docs/version-3.16/30-administration/10-configuration/11-backends/20-kubernetes.md index 7de7054da..f64db0f39 100644 --- a/docs/versioned_docs/version-3.13/30-administration/10-configuration/11-backends/20-kubernetes.md +++ b/docs/versioned_docs/version-3.16/30-administration/10-configuration/11-backends/20-kubernetes.md @@ -81,6 +81,51 @@ steps: To give steps access to the Kubernetes API via service account, take a look at [RBAC Authorization](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) +By default, setting `serviceAccountName` from a step's backend options is **not allowed** for security reasons, as it would let any user with push access run pipeline pods under an arbitrary service account and inherit its permissions. To enable it, set [`WOODPECKER_BACKEND_K8S_SERVICE_ACCOUNT_NAME_ALLOW_FROM_STEP`](#backend_k8s_service_account_name_allow_from_step) on the agent. + +:::warning +Enabling `WOODPECKER_BACKEND_K8S_SERVICE_ACCOUNT_NAME_ALLOW_FROM_STEP` in multi-tenant environments allows pipeline authors to run pods as any service account in the namespace, which may lead to privilege escalation. Only enable it if you trust everyone with push access. +::: + +### Workspace volume + +`workspaceVolume` controls whether the default workspace volume is mounted into a service Pod. It only affects service +containers and does not disable explicitly configured service volumes. + +If unset, the default workspace volume is mounted. + +```yaml +services: + postgres: + image: postgres:16 + backend_options: + kubernetes: + workspaceVolume: false +``` + +### User namespaces + +`hostUsers` controls whether the Pod uses the host's user namespace. When set to `false`, Kubernetes runs the Pod in a dedicated user namespace where UID 0 inside the container maps to a non-root UID on the host, providing an additional layer of isolation. + +See the [Kubernetes documentation](https://kubernetes.io/docs/concepts/workloads/pods/user-namespaces/) for more information on user namespaces. + +```yaml +steps: + - name: build + image: alpine + commands: + - whoami + backend_options: + kubernetes: + hostUsers: false + securityContext: + runAsUser: 0 +``` + +:::note +User namespaces require Kubernetes v1.25+ with the `UserNamespacesSupport` feature gate enabled, and a compatible container runtime (e.g. CRI-O, containerd v2.0+). +::: + ### Node selector `nodeSelector` specifies the labels which are used to select the node on which the step will be executed. @@ -359,6 +404,26 @@ backend_options: The feature requires Kubernetes v1.30 or above. ::: +You can set `allowPrivilegeEscalation` to `false` to prevent a container from gaining more privileges than its parent process. + +```yaml +backend_options: + kubernetes: + securityContext: + allowPrivilegeEscalation: false +``` + +You can also drop [Linux capabilities](https://man7.org/linux/man-pages/man7/capabilities.7.html) from a container. Adding capabilities is not allowed. + +```yaml +backend_options: + kubernetes: + securityContext: + capabilities: + drop: + - ALL +``` + ### Annotations and labels You can specify arbitrary [annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) and [labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/) to be set on the Pod definition for a given workflow step using the following configuration: @@ -566,3 +631,21 @@ Secret names to pull images from private repositories. See, how to [Pull an Imag - Default: none, which will use the default priority class configured in Kubernetes Which [Kubernetes PriorityClass](https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/priority-class-v1/) to assign to created job pods. + +--- + +### BACKEND_K8S_PERMISSION_INIT_IMAGE + +- Name: `WOODPECKER_BACKEND_K8S_PERMISSION_INIT_IMAGE` +- Default: 'busybox:stable-musl' + +Container image used for the workspace permission init container, which is used to create the workspace directory and ensure correct permissions when running steps as non-root users. + +--- + +### BACKEND_K8S_SERVICE_ACCOUNT_NAME_ALLOW_FROM_STEP + +- Name: `WOODPECKER_BACKEND_K8S_SERVICE_ACCOUNT_NAME_ALLOW_FROM_STEP` +- Default: `false` + +Determines if the Pod `serviceAccountName` can be defined from a step's backend options. Disabled by default, as it would otherwise allow any user with push access to run pods under an arbitrary service account and escalate privileges. diff --git a/docs/versioned_docs/version-3.13/30-administration/10-configuration/11-backends/30-local.md b/docs/versioned_docs/version-3.16/30-administration/10-configuration/11-backends/30-local.md similarity index 90% rename from docs/versioned_docs/version-3.13/30-administration/10-configuration/11-backends/30-local.md rename to docs/versioned_docs/version-3.16/30-administration/10-configuration/11-backends/30-local.md index fe5f66149..261351499 100644 --- a/docs/versioned_docs/version-3.13/30-administration/10-configuration/11-backends/30-local.md +++ b/docs/versioned_docs/version-3.16/30-administration/10-configuration/11-backends/30-local.md @@ -27,6 +27,8 @@ code and execute commands. In order to use this backend, you need to download (or build) the [agent](https://github.com/woodpecker-ci/woodpecker/releases/latest), configure it and run it on the host machine. +The local backend is supported on Windows, macOS, FreeBSD and OpenBSD; see [Supported platforms](../../05-installation/05-supported-platforms.md) for the full component and backend matrix. + ## Step specific configuration ### Shell diff --git a/docs/versioned_docs/version-3.13/30-administration/10-configuration/11-backends/50-custom.md b/docs/versioned_docs/version-3.16/30-administration/10-configuration/11-backends/50-custom.md similarity index 100% rename from docs/versioned_docs/version-3.13/30-administration/10-configuration/11-backends/50-custom.md rename to docs/versioned_docs/version-3.16/30-administration/10-configuration/11-backends/50-custom.md diff --git a/docs/versioned_docs/version-3.13/30-administration/10-configuration/11-backends/_category_.yaml b/docs/versioned_docs/version-3.16/30-administration/10-configuration/11-backends/_category_.yaml similarity index 100% rename from docs/versioned_docs/version-3.13/30-administration/10-configuration/11-backends/_category_.yaml rename to docs/versioned_docs/version-3.16/30-administration/10-configuration/11-backends/_category_.yaml diff --git a/docs/versioned_docs/version-3.13/30-administration/10-configuration/12-forges/11-overview.md b/docs/versioned_docs/version-3.16/30-administration/10-configuration/12-forges/11-overview.md similarity index 100% rename from docs/versioned_docs/version-3.13/30-administration/10-configuration/12-forges/11-overview.md rename to docs/versioned_docs/version-3.16/30-administration/10-configuration/12-forges/11-overview.md diff --git a/docs/versioned_docs/version-3.13/30-administration/10-configuration/12-forges/20-github.md b/docs/versioned_docs/version-3.16/30-administration/10-configuration/12-forges/20-github.md similarity index 100% rename from docs/versioned_docs/version-3.13/30-administration/10-configuration/12-forges/20-github.md rename to docs/versioned_docs/version-3.16/30-administration/10-configuration/12-forges/20-github.md diff --git a/docs/versioned_docs/version-3.13/30-administration/10-configuration/12-forges/30-gitea.md b/docs/versioned_docs/version-3.16/30-administration/10-configuration/12-forges/30-gitea.md similarity index 77% rename from docs/versioned_docs/version-3.13/30-administration/10-configuration/12-forges/30-gitea.md rename to docs/versioned_docs/version-3.16/30-administration/10-configuration/12-forges/30-gitea.md index ad2f92e62..7631101fa 100644 --- a/docs/versioned_docs/version-3.13/30-administration/10-configuration/12-forges/30-gitea.md +++ b/docs/versioned_docs/version-3.16/30-administration/10-configuration/12-forges/30-gitea.md @@ -33,8 +33,29 @@ To configure the Docker network if the network's name is `gitea`, configure it l ## Registration +### User OAuth Application + Register your application with Gitea to create your client id and secret. You can find the OAuth applications settings of Gitea at `https://gitea./user/settings/`. It is very important that authorization callback URL matches your http(s) scheme and hostname exactly with `https:///authorize` as the path. +### System-wide OAuth Application + +If you are the administrator of both Gitea and Woodpecker, you may prefer to use a system-wide OAuth application instead of a user-level application. System-wide applications are managed at the Gitea site administrator level and are visible to all users. + +To create a system-wide OAuth application in Gitea: + +1. Navigate to the site administration settings at `https://gitea./admin/settings/applications` +2. Create a new OAuth2 application under the "OAuth2 Applications" section +3. Configure the application with the same settings as above (callback URL, etc.) +4. Use the generated client id and secret for Woodpecker configuration + +System-wide applications are particularly useful for: + +- Shared CI/CD environments where multiple users need Woodpecker access +- Organizations that want centralized control over OAuth applications +- Preventing user-level application quotas from affecting CI/CD operations + +### Local Connections + If you run the Woodpecker CI server on the same host as the Gitea instance, you might also need to allow local connections in Gitea, since version `v1.16`. Otherwise webhooks will fail. Add the following lines to your Gitea configuration (usually at `/etc/gitea/conf/app.ini`). ```ini diff --git a/docs/versioned_docs/version-3.13/30-administration/10-configuration/12-forges/35-forgejo.md b/docs/versioned_docs/version-3.16/30-administration/10-configuration/12-forges/35-forgejo.md similarity index 77% rename from docs/versioned_docs/version-3.13/30-administration/10-configuration/12-forges/35-forgejo.md rename to docs/versioned_docs/version-3.16/30-administration/10-configuration/12-forges/35-forgejo.md index bd45c2aac..56d64f877 100644 --- a/docs/versioned_docs/version-3.13/30-administration/10-configuration/12-forges/35-forgejo.md +++ b/docs/versioned_docs/version-3.16/30-administration/10-configuration/12-forges/35-forgejo.md @@ -33,8 +33,29 @@ To configure the Docker network if the network's name is `forgejo`, configure it ## Registration +### User OAuth Application + Register your application with Forgejo to create your client id and secret. You can find the OAuth applications settings of Forgejo at `https://forgejo./user/settings/`. It is very important that authorization callback URL matches your http(s) scheme and hostname exactly with `https:///authorize` as the path. +### System-wide OAuth Application + +If you are the administrator of both Forgejo and Woodpecker, you may prefer to use a system-wide OAuth application instead of a user-level application. System-wide applications are managed at the Forgejo site administrator level and are visible to all users. + +To create a system-wide OAuth application in Forgejo: + +1. Navigate to the site administration settings at `https://forgejo./admin/settings/applications` +2. Create a new OAuth2 application under the "OAuth2 Applications" section +3. Configure the application with the same settings as above (callback URL, etc.) +4. Use the generated client id and secret for Woodpecker configuration + +System-wide applications are particularly useful for: + +- Shared CI/CD environments where multiple users need Woodpecker access +- Organizations that want centralized control over OAuth applications +- Preventing user-level application quotas from affecting CI/CD operations + +### Local Connections + If you run the Woodpecker CI server on the same host as the Forgejo instance, you might also need to allow local connections in Forgejo. Otherwise webhooks will fail. Add the following lines to your Forgejo configuration (usually at `/etc/forgejo/conf/app.ini`). ```ini diff --git a/docs/versioned_docs/version-3.13/30-administration/10-configuration/12-forges/40-gitlab.md b/docs/versioned_docs/version-3.16/30-administration/10-configuration/12-forges/40-gitlab.md similarity index 100% rename from docs/versioned_docs/version-3.13/30-administration/10-configuration/12-forges/40-gitlab.md rename to docs/versioned_docs/version-3.16/30-administration/10-configuration/12-forges/40-gitlab.md diff --git a/docs/versioned_docs/version-3.13/30-administration/10-configuration/12-forges/50-bitbucket.md b/docs/versioned_docs/version-3.16/30-administration/10-configuration/12-forges/50-bitbucket.md similarity index 100% rename from docs/versioned_docs/version-3.13/30-administration/10-configuration/12-forges/50-bitbucket.md rename to docs/versioned_docs/version-3.16/30-administration/10-configuration/12-forges/50-bitbucket.md diff --git a/docs/versioned_docs/version-3.13/30-administration/10-configuration/12-forges/60-bitbucket_datacenter.md b/docs/versioned_docs/version-3.16/30-administration/10-configuration/12-forges/60-bitbucket_datacenter.md similarity index 100% rename from docs/versioned_docs/version-3.13/30-administration/10-configuration/12-forges/60-bitbucket_datacenter.md rename to docs/versioned_docs/version-3.16/30-administration/10-configuration/12-forges/60-bitbucket_datacenter.md diff --git a/docs/versioned_docs/version-3.13/30-administration/10-configuration/12-forges/_category_.yaml b/docs/versioned_docs/version-3.16/30-administration/10-configuration/12-forges/_category_.yaml similarity index 100% rename from docs/versioned_docs/version-3.13/30-administration/10-configuration/12-forges/_category_.yaml rename to docs/versioned_docs/version-3.16/30-administration/10-configuration/12-forges/_category_.yaml diff --git a/docs/versioned_docs/version-3.13/30-administration/10-configuration/12-forges/bitbucket_oauth.png b/docs/versioned_docs/version-3.16/30-administration/10-configuration/12-forges/bitbucket_oauth.png similarity index 100% rename from docs/versioned_docs/version-3.13/30-administration/10-configuration/12-forges/bitbucket_oauth.png rename to docs/versioned_docs/version-3.16/30-administration/10-configuration/12-forges/bitbucket_oauth.png diff --git a/docs/versioned_docs/version-3.13/30-administration/10-configuration/12-forges/bitbucket_permissions.png b/docs/versioned_docs/version-3.16/30-administration/10-configuration/12-forges/bitbucket_permissions.png similarity index 100% rename from docs/versioned_docs/version-3.13/30-administration/10-configuration/12-forges/bitbucket_permissions.png rename to docs/versioned_docs/version-3.16/30-administration/10-configuration/12-forges/bitbucket_permissions.png diff --git a/docs/versioned_docs/version-3.13/30-administration/10-configuration/12-forges/gitea_oauth.gif b/docs/versioned_docs/version-3.16/30-administration/10-configuration/12-forges/gitea_oauth.gif similarity index 100% rename from docs/versioned_docs/version-3.13/30-administration/10-configuration/12-forges/gitea_oauth.gif rename to docs/versioned_docs/version-3.16/30-administration/10-configuration/12-forges/gitea_oauth.gif diff --git a/docs/versioned_docs/version-3.13/30-administration/10-configuration/12-forges/github_oauth.png b/docs/versioned_docs/version-3.16/30-administration/10-configuration/12-forges/github_oauth.png similarity index 100% rename from docs/versioned_docs/version-3.13/30-administration/10-configuration/12-forges/github_oauth.png rename to docs/versioned_docs/version-3.16/30-administration/10-configuration/12-forges/github_oauth.png diff --git a/docs/versioned_docs/version-3.13/30-administration/10-configuration/30-agent.md b/docs/versioned_docs/version-3.16/30-administration/10-configuration/30-agent.md similarity index 83% rename from docs/versioned_docs/version-3.13/30-administration/10-configuration/30-agent.md rename to docs/versioned_docs/version-3.16/30-administration/10-configuration/30-agent.md index 589c57b83..bb7dc3783 100644 --- a/docs/versioned_docs/version-3.13/30-administration/10-configuration/30-agent.md +++ b/docs/versioned_docs/version-3.16/30-administration/10-configuration/30-agent.md @@ -63,7 +63,7 @@ To get an _agent token_ you have to register the agent manually in the server us - Name: `WOODPECKER_SERVER` - Default: `localhost:9000` -Configures gRPC address of the server. +Configures gRPC address to the server. If you want to use an unix socket add `unix://` prefix and the path. --- @@ -148,6 +148,19 @@ Configures the number of parallel workflows. --- +### AGENT_SINGLE_WORKFLOW + +- Name: `WOODPECKER_AGENT_SINGLE_WORKFLOW` +- Default: `false` + +Configures the agent to exit (shutdown) after executing one workflow. When configured, +`WOODPECKER_MAX_WORKFLOWS` is forced to 1. + +This one-shot mode is useful in ephemeral environments that are provisioned on demand +by external automation — for example, when an autoscaler spins up a dedicated machine. In these setups, the agent starts, executes exactly one workflow, and exits, allowing the environment to be cleanly torn down afterward. + +--- + ### AGENT_LABELS - Name: `WOODPECKER_AGENT_LABELS` @@ -202,7 +215,7 @@ After pinging for a keepalive check, the agent waits for a duration of this time - Name: `WOODPECKER_GRPC_SECURE` - Default: `false` -Configures if the connection to `WOODPECKER_SERVER` should be made using a secure transport. +Configures if the connection to `WOODPECKER_SERVER` should be made using a secure transport (tls). --- @@ -215,6 +228,33 @@ Configures if the gRPC server certificate should be verified, only valid when `W --- +## RETRY_TIMEOUT + +- Name: `WOODPECKER_RETRY_TIMEOUT` +- Default: `2m` + +Set how long the agent keeps retrying to reconnect to the server after the gRPC connection is lost before giving up. + +:::warning +If set to 0 we retry forever. +::: + +--- + +## LOG_ENTRY_STREAM_BUFFER_SIZE + +- Name: `WOODPECKER_LOG_ENTRY_STREAM_BUFFER_SIZE` +- Default: `100` + +Set how many log lines an agent can buffer before it blocks io.Pipe, expect logentries to reach 1 MB in worst case. +If used with local backend, tis can increase your performance in special cases significantly. + +:::warning +If set to 0 we are always blocking. +::: + +--- + ### BACKEND - Name: `WOODPECKER_BACKEND` diff --git a/docs/versioned_docs/version-3.13/30-administration/10-configuration/40-autoscaler.md b/docs/versioned_docs/version-3.16/30-administration/10-configuration/40-autoscaler.md similarity index 100% rename from docs/versioned_docs/version-3.13/30-administration/10-configuration/40-autoscaler.md rename to docs/versioned_docs/version-3.16/30-administration/10-configuration/40-autoscaler.md diff --git a/docs/versioned_docs/version-3.13/30-administration/10-configuration/_category_.yaml b/docs/versioned_docs/version-3.16/30-administration/10-configuration/_category_.yaml similarity index 100% rename from docs/versioned_docs/version-3.13/30-administration/10-configuration/_category_.yaml rename to docs/versioned_docs/version-3.16/30-administration/10-configuration/_category_.yaml diff --git a/docs/versioned_docs/version-3.13/30-administration/10-configuration/new-agent-connected.png b/docs/versioned_docs/version-3.16/30-administration/10-configuration/new-agent-connected.png similarity index 100% rename from docs/versioned_docs/version-3.13/30-administration/10-configuration/new-agent-connected.png rename to docs/versioned_docs/version-3.16/30-administration/10-configuration/new-agent-connected.png diff --git a/docs/versioned_docs/version-3.13/30-administration/10-configuration/new-agent-created.png b/docs/versioned_docs/version-3.16/30-administration/10-configuration/new-agent-created.png similarity index 100% rename from docs/versioned_docs/version-3.13/30-administration/10-configuration/new-agent-created.png rename to docs/versioned_docs/version-3.16/30-administration/10-configuration/new-agent-created.png diff --git a/docs/versioned_docs/version-3.13/30-administration/10-configuration/new-agent-registration.png b/docs/versioned_docs/version-3.16/30-administration/10-configuration/new-agent-registration.png similarity index 100% rename from docs/versioned_docs/version-3.13/30-administration/10-configuration/new-agent-registration.png rename to docs/versioned_docs/version-3.16/30-administration/10-configuration/new-agent-registration.png diff --git a/docs/versioned_docs/version-3.13/30-administration/_category_.yaml b/docs/versioned_docs/version-3.16/30-administration/_category_.yaml similarity index 100% rename from docs/versioned_docs/version-3.13/30-administration/_category_.yaml rename to docs/versioned_docs/version-3.16/30-administration/_category_.yaml diff --git a/docs/versioned_docs/version-3.13/40-cli.md b/docs/versioned_docs/version-3.16/40-cli.md similarity index 95% rename from docs/versioned_docs/version-3.13/40-cli.md rename to docs/versioned_docs/version-3.16/40-cli.md index ccf8bc8b6..b2709d193 100644 --- a/docs/versioned_docs/version-3.13/40-cli.md +++ b/docs/versioned_docs/version-3.16/40-cli.md @@ -247,6 +247,8 @@ execute a local pipeline **--backend-docker-network**="": backend docker network +**--backend-docker-stop-timeout**="": seconds Woodpecker waits for a container to stop gracefully before forcefully killing it (default: 20) + **--backend-docker-tls-verify**: enable or disable TLS verification for connecting to docker server (default: true) **--backend-docker-volumes**="": backend docker volumes (comma separated) @@ -263,6 +265,8 @@ execute a local pipeline **--backend-k8s-namespace-per-org**: Whether to enable namespace segregation per organization feature. When enabled, Woodpecker will create the Kubernetes resources to separated Kubernetes namespaces per Woodpecker organization. (default: false) +**--backend-k8s-permission-init-image**="": image used by the workspace permission init container (default: busybox:stable-musl) + **--backend-k8s-pod-affinity**="": backend k8s Agent-wide worker pod affinity, in YAML format **--backend-k8s-pod-affinity-allow-from-step**: whether to allow using affinity from step's backend options (default: false) @@ -287,12 +291,18 @@ execute a local pipeline **--backend-k8s-secctx-nonroot**: `run as non root` Kubernetes security context option (default: false) +**--backend-k8s-service-account-name-allow-from-step**: whether to allow using service account name from step's backend options (default: false) + +**--backend-k8s-stop-timeout**="": seconds Woodpecker waits for pods to stop gracefully before forcefully killing them (default: 20) + **--backend-k8s-storage-class**="": backend k8s storage class **--backend-k8s-storage-rwx**: backend k8s storage access mode, should ReadWriteMany (RWX) instead of ReadWriteOnce (RWO) be used? (default: true) (default: true) **--backend-k8s-volume-size**="": backend k8s volume size (default 10G) (default: 10G) +**--backend-local-isolated-home**: set HOME, USERPROFILE and other variables to an isolated directory, if false we ignore netrc (default: true) + **--backend-local-temp-dir**="": set a different temp dir to clone workflows into (default: system temporary directory) **--backend-no-proxy**="": if set, pass the environment variable down as "NO_PROXY" to steps @@ -307,6 +317,8 @@ execute a local pipeline **--commit-message**="": Set the metadata environment variable "CI_COMMIT_MESSAGE". +**--commit-pull-draft**: Set the metadata environment variable "CI_COMMIT_PULL_REQUEST_DRAFT". (default: false) + **--commit-pull-labels**="": Set the metadata environment variable "CI_COMMIT_PULL_REQUEST_LABELS". **--commit-pull-milestone**="": Set the metadata environment variable "CI_COMMIT_PULL_REQUEST_MILESTONE". @@ -319,6 +331,8 @@ execute a local pipeline **--commit-sha**="": Set the metadata environment variable "CI_COMMIT_SHA". +**--commit-timestamp**="": Set the metadata environment variable "CI_COMMIT_TIMESTAMP". (default: 0) + **--env**="": Set the metadata environment variable "CI_ENV". **--forge-type**="": Set the metadata environment variable "CI_FORGE_TYPE". @@ -367,6 +381,8 @@ execute a local pipeline **--prev-commit-message**="": Set the metadata environment variable "CI_PREV_COMMIT_MESSAGE". +**--prev-commit-message**="": Set the metadata environment variable "CI_PREV_COMMIT_TIMESTAMP". (default: 0) + **--prev-commit-ref**="": Set the metadata environment variable "CI_PREV_COMMIT_REF". **--prev-commit-refspec**="": Set the metadata environment variable "CI_PREV_COMMIT_REFSPEC". @@ -449,7 +465,7 @@ lint a pipeline configuration file **--plugins-privileged**="": allow plugins to run in privileged mode, if set empty, there is no -**--plugins-trusted-clone**="": plugins that are trusted to handle Git credentials in cloning steps (default: "docker.io/woodpeckerci/plugin-git:2.8.0", "docker.io/woodpeckerci/plugin-git", "quay.io/woodpeckerci/plugin-git") +**--plugins-trusted-clone**="": plugins that are trusted to handle Git credentials in cloning steps (default: "docker.io/woodpeckerci/plugin-git:2.9.2", "docker.io/woodpeckerci/plugin-git", "quay.io/woodpeckerci/plugin-git") **--strict**: treat warnings as errors (default: false) @@ -713,6 +729,8 @@ add a cron job **--branch**="": cron branch +**--enabled**: whether cron is enabled (default: true) + **--format**="": format output (deprecated) (default: \x1b[33m{{ .Name }} \x1b[0m\nID: {{ .ID }}\nBranch: {{ .Branch }}\nSchedule: {{ .Schedule }}\nNextExec: {{ .NextExec }}\n) **--name**="": cron name @@ -753,6 +771,8 @@ update a cron job **--branch**="": cron branch +**--enabled**: whether cron is enabled (default: true) + **--format**="": format output (deprecated) (default: \x1b[33m{{ .Name }} \x1b[0m\nID: {{ .ID }}\nBranch: {{ .Branch }}\nSchedule: {{ .Schedule }}\nNextExec: {{ .NextExec }}\n) **--id**="": cron id diff --git a/docs/versioned_docs/version-3.13/92-development/01-getting-started.md b/docs/versioned_docs/version-3.16/92-development/01-getting-started.md similarity index 100% rename from docs/versioned_docs/version-3.13/92-development/01-getting-started.md rename to docs/versioned_docs/version-3.16/92-development/01-getting-started.md diff --git a/docs/versioned_docs/version-3.13/92-development/02-core-ideas.md b/docs/versioned_docs/version-3.16/92-development/02-core-ideas.md similarity index 92% rename from docs/versioned_docs/version-3.13/92-development/02-core-ideas.md rename to docs/versioned_docs/version-3.16/92-development/02-core-ideas.md index 2f9570685..49a276d25 100644 --- a/docs/versioned_docs/version-3.13/92-development/02-core-ideas.md +++ b/docs/versioned_docs/version-3.16/92-development/02-core-ideas.md @@ -8,7 +8,7 @@ ## Addons and extensions If you are wondering whether your contribution will be accepted to be merged in the Woodpecker core, or whether it's better to write an -[addon](../30-administration/10-configuration/100-addons.md), [extension](../30-administration/10-configuration/10-server.md#external-configuration-api) or an +[addon](../30-administration/10-configuration/100-addons.md), [extension](../20-usage/72-extensions/40-configuration-extension.md) or an [external custom backend](../30-administration/10-configuration/11-backends/50-custom.md), please check these points: - Is your change very specific to your setup and unlikely to be used by anyone else? diff --git a/docs/versioned_docs/version-3.13/92-development/03-ui.md b/docs/versioned_docs/version-3.16/92-development/03-ui.md similarity index 100% rename from docs/versioned_docs/version-3.13/92-development/03-ui.md rename to docs/versioned_docs/version-3.16/92-development/03-ui.md diff --git a/docs/versioned_docs/version-3.13/92-development/04-docs.md b/docs/versioned_docs/version-3.16/92-development/04-docs.md similarity index 100% rename from docs/versioned_docs/version-3.13/92-development/04-docs.md rename to docs/versioned_docs/version-3.16/92-development/04-docs.md diff --git a/docs/versioned_docs/version-3.16/92-development/05-architecture.md b/docs/versioned_docs/version-3.16/92-development/05-architecture.md new file mode 100644 index 000000000..51e218749 --- /dev/null +++ b/docs/versioned_docs/version-3.16/92-development/05-architecture.md @@ -0,0 +1,98 @@ +# Architecture + +## Module Interactions + +![Woodpecker architecture](./woodpecker-architecture.svg) + + + +## System architecture + +### main package hierarchy + +| package | meaning | imports | +| ------------------ | -------------------------------------------------------------- | ------------------------------------- | +| `cmd/**` | parse command-line args & environment to stat server/cli/agent | all other | +| `agent/**` | code only agent (remote worker) will need | `pipeline`, `rpc`, `shared` | +| `cli/**` | code only cli tool does need | `pipeline`, `shared`, `woodpecker-go` | +| `server/**` | code only server will need | `pipeline`, `rpc`, `shared` | +| `pipeline/**` | core ci/cd engine from parsing to execution | `shared` | +| `rpc/**` | RPC interface for agent-server communication | `pipeline` | +| `shared/**` | code shared for all three main tools (go help utils) | only std and external libs | +| `woodpecker-go/**` | go client for server rest api | std | + +### Server + +| package | meaning | imports | +| -------------------- | ------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `server/api/**` | handle web requests from `server/router` | `pipeline`, `rpc`, `../badges`, `../ccmenu`, `../logging`, `../model`, `../pubsub`, `../queue`, `../forge`, `../shared`, `../store`, `shared`, (TODO: mv `server/router/middleware/session`) | +| `server/badges/**` | generate svg badges for pipelines | `../model` | +| `server/ccmenu/**` | generate xml ccmenu for pipelines | `../model` | +| `server/rpc/**` | gRPC server agents can connect to | `rpc`, `../logging`, `../model`, `../pubsub`, `../queue`, `../forge`, `../pipeline`, `../store` | +| `server/logging/**` | logging lib for gPRC server to stream logs while running | std | +| `server/model/**` | structs for store (db) and api (json) | std | +| `server/pipeline/**` | orchestrate pipelines (TODO: parts of it should move into /pipeline) | `pipeline`, `../model`, `../pubsub`, `../queue`, `../forge`, `../store`, `../plugins` | +| `server/pubsub/**` | pubsub lib for server to push changes to the WebUI | std | +| `server/queue/**` | queue lib for server where agents pull new pipelines from via gRPC | `server/model` | +| `server/forge/**` | forge lib for server to connect and handle forge specific stuff | `shared`, `server/model` | +| `server/router/**` | handle requests to REST API (and all middleware) and serve UI and WebUI config | `shared`, `../api`, `../model`, `../forge`, `../store`, `../web` | +| `server/store/**` | handle database | `server/model` | +| `server/web/**` | server SPA | | + +- `../` = `server/` + +### Agent + +| package | meaning | imports | +| -------------- | ---------------------------------------------------- | ------------------------------------------------------ | +| `agent/**` | agent implementation that runs workflows | `pipeline`, `rpc`, `shared` | +| `agent/rpc/**` | gRPC client for agent-server communication | `rpc`, `pipeline/backend/types`, std and external libs | +| `cmd/agent/**` | CLI interface for starting and configuring the agent | `agent`, std and external libs | + +The agent is a remote worker that connects to the server via gRPC to receive pipeline execution instructions and report back execution state and logs. +The agent polls the server's queue for new work, executes pipeline steps using the pipeline engine, and streams results back to the server. + +TODO: Review cmd/agent/core to determine if any logic should be moved into the agent package for better separation of concerns. + +### CLI + +| package | meaning | imports | +| ------------------------ | ----------------------------------------------------------------------- | -------------------------------------------------------------------------------- | +| `cli/admin/**` | admin commands for server management (users, secrets, registries, etc.) | `../common`, `../internal`, `woodpecker-go` | +| `cli/common/**` | shared utilities and helpers used across all CLI subcommands | `../internal/config`, `../update`, `shared` | +| `cli/context/**` | manage multiple server contexts (connections to different servers) | `../common`, `../internal/config`, `../output` | +| `cli/exec/**` | execute pipelines locally without server orchestration | `pipeline`, `../common`, `../lint`, `shared` | +| `cli/info/**` | display information about the current user | `../common`, `../internal` | +| `cli/internal/**` | internal utilities for HTTP client, auth, and server communication | `../internal/config`, `woodpecker-go`, `shared` | +| `cli/internal/config/**` | configuration file management (load, store, credentials) | std and external libs | +| `cli/lint/**` | validate pipeline configuration files | `pipeline/frontend/yaml`, `pipeline/frontend/yaml/linter`, `../common`, `shared` | +| `cli/org/**` | manage organization-level resources (secrets, registries) | `../common`, `../internal`, `woodpecker-go` | +| `cli/output/**` | formatting utilities for CLI output (tables, etc.) | std and external libs | +| `cli/pipeline/**` | manage pipeline operations (start, stop, approve, logs, etc.) | `../common`, `../internal`, `../output`, `woodpecker-go`, `shared` | +| `cli/repo/**` | manage repository-level resources (repos, crons, secrets, registries) | `../common`, `../internal`, `../output`, `woodpecker-go` | +| `cli/setup/**` | interactive first-time setup wizard for CLI configuration | `../internal/config` | +| `cli/update/**` | self-updater for the CLI binary | std and external libs | +| `cmd/cli/**` | CLI entry point and command structure | `cli/**` | + +The CLI provides a command-line interface for interacting with Woodpecker servers. +Each subcommand is organized into its own package under `cli//`. + +The `cli/exec` subcommand allows local pipeline execution for testing and development by combining pipeline parsing and execution without requiring a running server or agent. + +- `../` = `cli/` + +### Engine + +The engine is the shared kernel that validates, parses frontend facing config files, enrich it by the provided forge metadata and produce config for the backends to execute on based on that. It also contains the default backend implementations. + +#### Runtime + +The runtime is the package controlling how a workflow is executed, and can be found at `pipeline/runtime`. + +Pipeline/runtime flow diagram diff --git a/docs/versioned_docs/version-3.13/92-development/06-conventions.md b/docs/versioned_docs/version-3.16/92-development/06-conventions.md similarity index 100% rename from docs/versioned_docs/version-3.13/92-development/06-conventions.md rename to docs/versioned_docs/version-3.16/92-development/06-conventions.md diff --git a/docs/versioned_docs/version-3.13/92-development/07-guides.md b/docs/versioned_docs/version-3.16/92-development/07-guides.md similarity index 100% rename from docs/versioned_docs/version-3.13/92-development/07-guides.md rename to docs/versioned_docs/version-3.16/92-development/07-guides.md diff --git a/docs/versioned_docs/version-3.13/92-development/08-translations.md b/docs/versioned_docs/version-3.16/92-development/08-translations.md similarity index 100% rename from docs/versioned_docs/version-3.13/92-development/08-translations.md rename to docs/versioned_docs/version-3.16/92-development/08-translations.md diff --git a/docs/versioned_docs/version-3.13/92-development/09-openapi.md b/docs/versioned_docs/version-3.16/92-development/09-openapi.md similarity index 100% rename from docs/versioned_docs/version-3.13/92-development/09-openapi.md rename to docs/versioned_docs/version-3.16/92-development/09-openapi.md diff --git a/docs/versioned_docs/version-3.13/92-development/09-testing.md b/docs/versioned_docs/version-3.16/92-development/09-testing.md similarity index 98% rename from docs/versioned_docs/version-3.13/92-development/09-testing.md rename to docs/versioned_docs/version-3.16/92-development/09-testing.md index f27f71130..dd2604cdf 100644 --- a/docs/versioned_docs/version-3.13/92-development/09-testing.md +++ b/docs/versioned_docs/version-3.16/92-development/09-testing.md @@ -5,7 +5,7 @@ ### Unit Tests [We use default golang unit tests](https://go.dev/doc/tutorial/add-a-test) -with [`"github.com/stretchr/testify/assert"`](https://pkg.go.dev/github.com/stretchr/testify@v1.9.0/assert) to simplify testing. +with [`"github.com/stretchr/testify/assert"`](https://pkg.go.dev/github.com/stretchr/testify/assert) to simplify testing. ### Integration Tests diff --git a/docs/versioned_docs/version-3.13/92-development/10-packaging.md b/docs/versioned_docs/version-3.16/92-development/10-packaging.md similarity index 100% rename from docs/versioned_docs/version-3.13/92-development/10-packaging.md rename to docs/versioned_docs/version-3.16/92-development/10-packaging.md diff --git a/docs/versioned_docs/version-3.13/92-development/100-addons.md b/docs/versioned_docs/version-3.16/92-development/100-addons.md similarity index 100% rename from docs/versioned_docs/version-3.13/92-development/100-addons.md rename to docs/versioned_docs/version-3.16/92-development/100-addons.md diff --git a/docs/versioned_docs/version-3.16/92-development/40-deprecations.md b/docs/versioned_docs/version-3.16/92-development/40-deprecations.md new file mode 100644 index 000000000..f19f17c8e --- /dev/null +++ b/docs/versioned_docs/version-3.16/92-development/40-deprecations.md @@ -0,0 +1,43 @@ +# Deprecation Policy + +## Pipeline Configuration Changes + +Pipeline configuration (YAML syntax) changes follow a strict deprecation process to ensure users have sufficient time to migrate. + +### Process Timeline + +1. **Minor Version N.x - Add Deprecation Warning** + - Linter shows a warning (not an error) + - Old syntax remains functional + - Documentation is updated to reflect the new syntax + - Warning message includes guidance on required changes + +2. **Major Version (N+1).0 - Warning Becomes Error** + - Linter issues an error (pipeline fails) + - Old syntax is no longer supported + - Breaking change is documented in the migration guide + - Users **must** update their configurations + +3. **Minor Version (N+1).x - Code Cleanup** + - Deprecated code paths are removed + - Implementation is simplified/refactored + - Parser no longer recognizes the old syntax + +### Example + +Old syntax: `secrets: [token]` +New syntax: `environment: { TOKEN: { from_secret: token } }` + +- **v2.5.0:** Deprecation warning added in linter; both syntaxes work +- **v2.6-2.9:** Warning persists; both syntaxes remain functional +- **v3.0.0:** Linter error; old syntax fails (breaking change) +- **v3.1.0:** Deprecated code paths removed; parser simplified + +### Implementation Checklist + +When deprecating pipeline configuration syntax, ensure the following: + +- [ ] Add linter warning in `/pipeline/frontend/yaml/linter/` +- [ ] Update JSON schema in `/pipeline/frontend/yaml/linter/schema` +- [ ] Add test cases for deprecated syntax +- [ ] Update documentation to reflect the new syntax diff --git a/docs/versioned_docs/version-3.13/92-development/_category_.yaml b/docs/versioned_docs/version-3.16/92-development/_category_.yaml similarity index 100% rename from docs/versioned_docs/version-3.13/92-development/_category_.yaml rename to docs/versioned_docs/version-3.16/92-development/_category_.yaml diff --git a/docs/versioned_docs/version-3.13/92-development/ui-proxy.svg b/docs/versioned_docs/version-3.16/92-development/ui-proxy.svg similarity index 100% rename from docs/versioned_docs/version-3.13/92-development/ui-proxy.svg rename to docs/versioned_docs/version-3.16/92-development/ui-proxy.svg diff --git a/docs/versioned_docs/version-3.13/92-development/vscode-debug.png b/docs/versioned_docs/version-3.16/92-development/vscode-debug.png similarity index 100% rename from docs/versioned_docs/version-3.13/92-development/vscode-debug.png rename to docs/versioned_docs/version-3.16/92-development/vscode-debug.png diff --git a/docs/versioned_docs/version-3.13/92-development/vscode-run-test.png b/docs/versioned_docs/version-3.16/92-development/vscode-run-test.png similarity index 100% rename from docs/versioned_docs/version-3.13/92-development/vscode-run-test.png rename to docs/versioned_docs/version-3.16/92-development/vscode-run-test.png diff --git a/docs/versioned_docs/version-3.16/92-development/woodpecker-architecture.dot b/docs/versioned_docs/version-3.16/92-development/woodpecker-architecture.dot new file mode 100644 index 000000000..4ef873a1d --- /dev/null +++ b/docs/versioned_docs/version-3.16/92-development/woodpecker-architecture.dot @@ -0,0 +1,159 @@ +digraph WoodpeckerArchitecture { + graph [ + rankdir=TB, + splines=ortho, + nodesep=0.5, + ranksep=0.8, + fontname="Helvetica" + ] + + node [ + shape=box, + style="rounded,filled", + fillcolor="#2b2b2b", + fontcolor="white", + fontname="Helvetica" + ] + + edge [ + color="#bdbdbd", + arrowsize=0.7 + ] + + /* ===================== UI ===================== */ + subgraph cluster_ui { + label="UI" + fillcolor="#c7efe9" + fontcolor="black" + style="rounded,filled" + + ui_web [label="web/"] + } + + /* ===================== SDK ===================== */ + subgraph cluster_sdk { + label="SDK (woodpecker-go)" + fillcolor="#e8f5e9" + fontcolor="black" + style="rounded,filled" + + sdk [label="woodpecker-go"] + } + + /* ===================== CLI ===================== */ + subgraph cluster_cli { + label="woodpecker-cli" + fillcolor="#bfe9e0" + fontcolor="black" + style="rounded,filled" + + cli_cmd [label="cmd/cli/"] + cli_core [label="cli/"] + } + + /* ===================== Agent ===================== */ + subgraph cluster_agent { + label="woodpecker-agent" + fillcolor="#ffe0c7" + fontcolor="black" + style="rounded,filled" + + agent_cmd [label="cmd/agent/"] + agent_core [label="agent/"] + } + + /* ===================== Pipelines ===================== */ + subgraph cluster_pipelines { + label="Pipelines" + fillcolor="#ffe8d6" + fontcolor="black" + style="rounded,filled" + + pipe_core [label="pipeline/"] + pipe_frontend [label="pipeline/frontend/\n(yaml)"] + pipe_backend [label="pipeline/backend/\n(exec engines)"] + } + + /* ===================== Server ===================== */ + subgraph cluster_server { + label="woodpecker-server" + fillcolor="#dbe9ff" + fontcolor="black" + style="rounded,filled" + + srv_cmd [label="cmd/server/"] + srv_router [label="server/router/"] + srv_api [label="server/api/"] + srv_grpc [label="server/rpc/"] + srv_scheduler [label="server/scheduler/"] + srv_queue [label="server/queue/"] + srv_pubsub [label="server/pubsub/"] + srv_store [label="server/store/"] + srv_model [label="server/model/"] + srv_forge [label="server/forge/"] + } + + /* ===================== Shared Libs ===================== */ + subgraph cluster_shared { + label="Shared Libs" + fillcolor="#eeeeee" + fontcolor="black" + style="rounded,filled" + + shared_util [label="shared/util/"] + shared_token [label="shared/token/"] + shared_http [label="shared/httputil/"] + shared_log [label="shared/logger/"] + } + + /* ===================== External ===================== */ + subgraph cluster_external { + label="External Systems" + style="rounded,dashed" + fontcolor="white" + + ext_scm [label="SCM Providers", shape=cloud] + ext_db [label="Database", shape=cylinder] + } + + /* ===================== Runtime Interactions ===================== */ + + /* UI */ + ui_web -> srv_router [xlabel="HTTP"] + ui_web -> srv_api [xlabel="REST API"] + + /* CLI */ + cli_cmd -> cli_core + cli_core -> sdk + sdk -> srv_api [xlabel="REST API"] + + /* Agent */ + agent_cmd -> agent_core + agent_core -> srv_grpc [xlabel="gRPC connect"] + agent_core -> srv_queue [xlabel="poll work"] + agent_core -> pipe_backend [xlabel="execute steps"] + + /* Pipelines */ + pipe_frontend -> pipe_core + pipe_core -> pipe_backend + + /* Server internal flow */ + srv_cmd -> srv_router + srv_router -> srv_api + srv_api -> srv_store + srv_api -> srv_scheduler + srv_grpc -> srv_scheduler + srv_scheduler -> srv_queue + srv_scheduler -> srv_pubsub + srv_store -> srv_model + + /* External integrations */ + srv_forge -> ext_scm [xlabel="SCM API"] + srv_store -> ext_db [xlabel="SQL"] + + /* Shared libs usage (consumer -> library) */ + srv_router -> shared_token + srv_api -> shared_http + srv_grpc -> shared_log + pipe_core -> shared_util +} diff --git a/docs/versioned_docs/version-3.16/92-development/woodpecker-architecture.svg b/docs/versioned_docs/version-3.16/92-development/woodpecker-architecture.svg new file mode 100644 index 000000000..e1ba00037 --- /dev/null +++ b/docs/versioned_docs/version-3.16/92-development/woodpecker-architecture.svg @@ -0,0 +1,364 @@ + + + + + + +WoodpeckerArchitecture + + +cluster_ui + +UI + + +cluster_sdk + +SDK (woodpecker-go) + + +cluster_cli + +woodpecker-cli + + +cluster_agent + +woodpecker-agent + + +cluster_pipelines + +Pipelines + + +cluster_server + +woodpecker-server + + +cluster_shared + +Shared Libs + + +cluster_external + +External Systems + + + +ui_web + +web/ + + + +srv_router + +server/router/ + + + +ui_web->srv_router + + +HTTP + + + +srv_api + +server/api/ + + + +ui_web->srv_api + + +REST API + + + +sdk + +woodpecker-go + + + +sdk->srv_api + + +REST API + + + +cli_cmd + +cmd/cli/ + + + +cli_core + +cli/ + + + +cli_cmd->cli_core + + + + + +cli_core->sdk + + + + + +agent_cmd + +cmd/agent/ + + + +agent_core + +agent/ + + + +agent_cmd->agent_core + + + + + +pipe_backend + +pipeline/backend/ +(exec engines) + + + +agent_core->pipe_backend + + +execute steps + + + +srv_grpc + +server/rpc/ + + + +agent_core->srv_grpc + + +gRPC connect + + + +srv_queue + +server/queue/ + + + +agent_core->srv_queue + + +poll work + + + +pipe_core + +pipeline/ + + + +pipe_core->pipe_backend + + + + + +shared_util + +shared/util/ + + + +pipe_core->shared_util + + + + + +pipe_frontend + +pipeline/frontend/ +(yaml) + + + +pipe_frontend->pipe_core + + + + + +srv_cmd + +cmd/server/ + + + +srv_cmd->srv_router + + + + + +srv_router->srv_api + + + + + +shared_token + +shared/token/ + + + +srv_router->shared_token + + + + + +srv_scheduler + +server/scheduler/ + + + +srv_api->srv_scheduler + + + + + +srv_store + +server/store/ + + + +srv_api->srv_store + + + + + +shared_http + +shared/httputil/ + + + +srv_api->shared_http + + + + + +srv_grpc->srv_scheduler + + + + + +shared_log + +shared/logger/ + + + +srv_grpc->shared_log + + + + + +srv_scheduler->srv_queue + + + + + +srv_pubsub + +server/pubsub/ + + + +srv_scheduler->srv_pubsub + + + + + +srv_model + +server/model/ + + + +srv_store->srv_model + + + + + +ext_db + + +Database + + + +srv_store->ext_db + + +SQL + + + +srv_forge + +server/forge/ + + + +ext_scm + +SCM Providers + + + +srv_forge->ext_scm + + +SCM API + + + diff --git a/docs/versioned_sidebars/version-3.13-sidebars.json b/docs/versioned_sidebars/version-3.16-sidebars.json similarity index 100% rename from docs/versioned_sidebars/version-3.13-sidebars.json rename to docs/versioned_sidebars/version-3.16-sidebars.json diff --git a/docs/versions.json b/docs/versions.json index 7737f8285..8988808df 100644 --- a/docs/versions.json +++ b/docs/versions.json @@ -1 +1 @@ -["3.15", "3.14", "3.13", "2.8"] +["3.16", "3.15", "3.14", "2.8"]