I've been debugging case where Kubernetes backend agents occasionally are OOMKilled. During the investigation noticed an increasing amount of goroutines on the agent processes which turned out to be multiple goroutines hanging on:
```
runtime.gopark(proc.go:461)
runtime.chanrecv(chan.go:667)
runtime.chanrecv1(chan.go:509)
go.woodpecker-ci.org/woodpecker/v3/pipeline/backend/kubernetes.(*kube).WaitStep(kubernetes.go:325)
go.woodpecker-ci.org/woodpecker/v3/pipeline/runtime.(*Runtime).exec(executor.go:261)
go.woodpecker-ci.org/woodpecker/v3/pipeline/runtime.(*Runtime).execAll.func1.1(executor.go:174)
go.woodpecker-ci.org/woodpecker/v3/pipeline/runtime.(*Runtime).execAll.func1.2(executor.go:200)
runtime.goexit(asm_arm64.s:1268)
go.woodpecker-ci.org/woodpecker/v3/pipeline/runtime.(*Runtime).execAll.func1(executor.go:199)
```
My analysis is that on cancel nothing fires the `finished` channel, as pod is deleted i.e. not updated (there's a registed handler for pod update events). There was already a comment about adding cancellation handler for ctx.Done, so I think this is the proper way (instead of adding event handler for pod deletion).
## Fix: kill docker container on pipeline cancellation
### What was happening
When a pipeline step was canceled (e.g. manual cancel or agent shutdown), the Docker backend would stop waiting for the container but **would not actively terminate it**.
This caused containers running long-lived commands (like `sleep`, `tail -f`, servers, etc.) to continue running in the background.
## Related Issue
Closes#6016
### What this PR changes
This PR ensures that **when the step context is canceled**, the running Docker container is **immediately killed**.
- Listens to `ctx.Done()` in `WaitStep`
- Sends a `ContainerKill` using a non-cancelable context
- Preserves existing behavior for normal container exit
- Prevents orphaned containers after pipeline cancellation
### Why this approach
Docker containers do not automatically stop when the caller context is canceled.
Explicitly killing the container on `ctx.Done()` guarantees correct cleanup and matches expected CI behavior.
## problem
if steps where started concurrent, the stdout pipeline reader war overwritten and you randomly got the wrong command stream
from a step.
## change
where we have possible race conditions, we now use thread save types
e.g. store the command struct and the output reader in sync.Map
also a lot of tests where added
currently if we don't know the shell we just assume posix.
this adds a small test, to ensure it is and fail gracefully before doing weird stuff.
## Test Conf
```yaml
skip_clone: true
steps:
build:
image: "true"
commands:
- echo "building..."
```
6543andGitHub
Uğur TafralıandGitHub
qwerty287andGitHub
Harri AvellanandGitHub
Martin SchmidtandGitHub
Henrik HuittiandGitHub
LUKIEYFandGitHub
GustedandGitHub
LillyandGitHub
