Compare commits

...
Author SHA1 Message Date
Noah Campbell 2a2b0e0f30 reverted regression tests to how they were before
This was turning into something worthy of its own PR so I put it back to how it was before to work on next/separately
2025-10-14 11:30:05 -05:00
Noah Campbell 1013f2a160 does not recalculate affected e2e tests 2025-10-14 10:24:27 -05:00
Noah Campbell e73e1375cc uses unique namespaces for different preflight regression tests 2025-10-14 10:14:33 -05:00
Noah Campbell 9278d50252 use kubeconfig to properly set up preflight regression tests 2025-10-14 10:00:35 -05:00
Noah Campbell e084c0a46a regression tests run in matrix 2025-10-14 09:53:58 -05:00
Noah Campbell f7a941c220 regression tests run separate from relevant test detection 2025-10-14 08:11:07 -05:00
Noah Campbell 8920a9d09a Update Makefile 2025-10-13 14:35:25 -05:00
Noah Campbell a677646b17 reduced redundant building 2025-10-13 14:31:14 -05:00
Noah Campbell c84ea20b88 e2e test make target build the binaries first 2025-10-13 14:19:27 -05:00
Noah Campbell 409a02a125 Update affected-tests.yml 2025-10-13 14:02:20 -05:00
Noah Campbell 78af9208cf runs e2e preflight and support bundle tests as matrix 2025-10-13 13:53:09 -05:00
Noah Campbell fb0983480c Update affected-tests.yml 2025-10-13 13:40:06 -05:00
Noah Campbell 1aa74db81d Update affected-tests.yml 2025-10-13 13:24:35 -05:00
Noah Campbell 8f7cf26cc9 Update affected-tests.yml 2025-10-13 13:13:19 -05:00
Noah Campbell a6d3e3e423 Update affected-tests.yml 2025-10-13 13:02:45 -05:00
Noah Campbell 6ea8cbcb09 creates clusters for tests via replicated actions 2025-10-13 13:00:59 -05:00
Noah Campbell bf1dfa0318 removed references to nonexistent tests 2025-10-13 12:45:59 -05:00
Noah Campbell 45ee61aa4d creates cluster for tests and removed non testable files from being flagged 2025-10-13 12:01:58 -05:00
Noah Campbell e3458ad297 can run unit tests via makefile 2025-10-13 11:29:58 -05:00
Noah Campbell cb32216df8 runs all tests when test change is detected 2025-10-13 10:54:59 -05:00
Noah Campbell 2b82454ef8 Update affected-tests.yml 2025-10-13 10:18:43 -05:00
Noah Campbell e05962bcf3 affected test workflow runs on PR 2025-10-13 10:11:46 -05:00
Noah Campbell 0f747f55dc removed main requirement on affected tests 2025-10-13 10:07:26 -05:00
Noah Campbell 1c234e432c shows affected test summary 2025-10-13 10:00:49 -05:00
Noah Campbell d325804359 readded build binary action for tests 2025-10-13 09:52:47 -05:00
Noah Campbell 2d664b8736 separated build and test actions for relevant tests 2025-10-13 09:44:39 -05:00
Noah Campbell 8762bd1515 changed tests always are flagged as affected tests 2025-10-11 11:31:43 -07:00
Noah Campbell a2a424b201 added test and makes new relevant tests run tests 2025-10-11 11:17:26 -07:00
Noah Campbell 3f069125a2 added flag to specify files changed for testing 2025-10-11 10:49:54 -07:00
Noah Campbell 37292e6ba1 finds and runs relevant code change tests 2025-10-10 07:50:07 -07:00
Benjamin Yang 6c5c310eb3 Fix ollama clean (#1885)
* fixing .json format

* feat: aggregate files by resource type in Ollama agent for accurate cluster-wide analysis

- Group pod/deployment/event/node files by type before analysis
- Create cluster-wide summaries instead of per-file analysis
- Add context about empty namespaces being normal in Kubernetes
- Fixes false positives where empty namespaces were flagged as errors
- Improves accuracy from ~60% to ~95%
- Reduces analyzers from 21 to 12 (more efficient)
- Speeds up analysis by ~30 seconds
- Add cmd/analyze/main.go for building standalone analyze binary

* feat: aggregate files by resource type in Ollama agent for accurate cluster-wide analysis

- Group pod/deployment/event/node files by type before analysis
- Create cluster-wide summaries instead of per-file analysis
- Add context about empty namespaces being normal in Kubernetes
- Fixes false positives where empty namespaces were flagged as errors
- Improves accuracy from ~60% to ~95%
- Reduces analyzers from 21 to 12 (more efficient)
- Speeds up analysis by ~30 seconds
- Fix event limiting condition to track included events separately
- Update test to handle both aggregated and single-file analyzers
- Add cmd/analyze/main.go for building standalone analyze binary

* fixing error

* fixing bugbot

* fix bugbot errors

* fix bugbot errors

* bugbot errors

* fixing more bugbot errors

* fix: initialize namespace stats only after validating resource type

- Move namespace initialization to after kind validation
- Initialize for valid PodList/DeploymentList when items array exists
- Initialize for valid single Pod/Deployment when kind matches
- Skip initialization entirely for malformed/invalid JSON
- Prevents reporting namespaces with invalid resource files

* refactor: use if-else structure for clearer control flow

- Restructure pod/deployment aggregation to use explicit if-else
- Makes it clear that lists are processed in if block, singles in else
- Functionally identical but clearer for static analysis
- Resolves bugbot false positives about unreachable code
2025-10-08 16:57:00 -05:00
12 changed files with 1795 additions and 177 deletions
+192
View File
@@ -0,0 +1,192 @@
name: Affected Go Tests
on:
pull_request:
types: [opened, reopened, synchronize, ready_for_review]
permissions:
contents: read
jobs:
test-affected:
if: github.event.pull_request.draft == false
runs-on: ubuntu-latest
timeout-minutes: 30
outputs:
unit_has_changes: ${{ steps.affected.outputs.has_changes }}
e2e_has_changes: ${{ steps.affected_e2e.outputs.has_changes }}
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version-file: go.mod
cache: true
- name: Go Mod Download
run: go mod download
- name: Compute base ref
id: pr-info
run: |
echo "BASE_REF=origin/${{ github.base_ref }}" >> "$GITHUB_OUTPUT"
echo "Base ref: origin/${{ github.base_ref }}"
# 2) Detect relevant unit packages and e2e tests
- name: Compute affected packages
id: affected
run: |
set -euo pipefail
echo "Base: ${{ steps.pr-info.outputs.BASE_REF }}"
# Generate affected package list to a file for reuse in subsequent steps
go run ./scripts/affected-packages.go -base "${{ steps.pr-info.outputs.BASE_REF }}" > /tmp/affected.txt
echo "Affected packages:" || true
if [ -s /tmp/affected.txt ]; then
cat /tmp/affected.txt
else
echo "(none)"
fi
# Expose whether we have any packages to test
if [ -s /tmp/affected.txt ]; then
echo "has_changes=true" >> "$GITHUB_OUTPUT"
else
echo "has_changes=false" >> "$GITHUB_OUTPUT"
fi
- name: Compute affected e2e tests
id: affected_e2e
run: |
set -euo pipefail
go run ./scripts/affected-packages.go -mode=suites -base "${{ steps.pr-info.outputs.BASE_REF }}" > /tmp/affected-e2e.txt
awk -F: '$1=="preflight"{print $2}' /tmp/affected-e2e.txt > /tmp/preflight-tests.txt
awk -F: '$1=="support-bundle"{print $2}' /tmp/affected-e2e.txt > /tmp/support-tests.txt
if [ -s /tmp/preflight-tests.txt ] || [ -s /tmp/support-tests.txt ]; then
echo "has_changes=true" >> "$GITHUB_OUTPUT"
else
echo "has_changes=false" >> "$GITHUB_OUTPUT"
fi
- name: Publish affected summary
if: always()
run: |
{
echo "### Affected unit packages";
if [ -s /tmp/affected.txt ]; then
sed 's/^/- /' /tmp/affected.txt;
else
echo "- (none)";
fi;
echo;
echo "### Affected e2e tests";
if [ -s /tmp/affected-e2e.txt ]; then
sed 's/^/- /' /tmp/affected-e2e.txt;
else
echo "- (none)";
fi;
} | tee -a "$GITHUB_STEP_SUMMARY"
- name: Upload affected unit packages
uses: actions/upload-artifact@v4
with:
name: affected-unit
path: /tmp/affected.txt
if-no-files-found: warn
- name: Upload affected e2e artifacts
uses: actions/upload-artifact@v4
with:
name: affected-e2e
path: |
/tmp/affected-e2e.txt
/tmp/preflight-tests.txt
/tmp/support-tests.txt
if-no-files-found: warn
- name: No affected packages — skip tests
if: steps.affected.outputs.has_changes != 'true'
run: echo "No Go packages affected by this PR; skipping tests."
e2e-affected:
needs: test-affected
if: github.event.pull_request.draft == false
runs-on: ubuntu-latest
timeout-minutes: 45
strategy:
fail-fast: false
matrix:
suite: [unit, preflight, support-bundle]
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version-file: go.mod
cache: true
- name: Go Mod Download
run: go mod download
- name: Download affected unit packages
uses: actions/download-artifact@v4
with:
name: affected-unit
path: /tmp
- name: Download affected e2e artifacts
uses: actions/download-artifact@v4
with:
name: affected-e2e
path: /tmp
- name: Run unit tests (filtered)
if: matrix.suite == 'unit' && needs.test-affected.outputs.unit_has_changes == 'true'
run: |
set -euo pipefail
if grep -qx "./..." /tmp/affected.txt; then
echo "Module files changed; running all unit tests"
make test
else
echo "Running unit tests for affected packages"
pkgs=$(tr '\n' ' ' < /tmp/affected.txt)
PACKAGES="$pkgs" make test-packages
fi
- name: Run e2e (filtered) - ${{ matrix.suite }}
if: matrix.suite != 'unit' && needs.test-affected.outputs.e2e_has_changes == 'true'
run: |
set -euo pipefail
docker rm -f kind-cluster-control-plane 2>/dev/null || true
if [ "${{ matrix.suite }}" = "preflight" ]; then
file=/tmp/preflight-tests.txt
path=./test/e2e/preflight
else
file=/tmp/support-tests.txt
path=./test/e2e/support-bundle
fi
if [ -s "$file" ]; then
regex="$(grep -v '^$' "$file" | tr '\n' '|' | sed 's/|$//')"
if [ -n "$regex" ]; then
if [ "${{ matrix.suite }}" = "preflight" ]; then
E2EPATHS="$path" RUN="^(${regex})$" make preflight-e2e-go-test
else
E2EPATHS="$path" RUN="^(${regex})$" make support-bundle-e2e-go-test
fi
else
echo "No valid ${{ matrix.suite }} tests matched after filtering"
fi
else
echo "No ${{ matrix.suite }} e2e changes"
fi
+9 -4
View File
@@ -37,6 +37,7 @@ jobs:
- run: make tidy-diff
test-integration:
if: github.event_name == 'push'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
@@ -65,6 +66,7 @@ jobs:
path: bin/preflight
validate-preflight-e2e:
if: github.event_name == 'push'
runs-on: ubuntu-latest
needs: compile-preflight
steps:
@@ -95,6 +97,7 @@ jobs:
path: bin/support-bundle
validate-supportbundle-e2e:
if: github.event_name == 'push'
runs-on: ubuntu-latest
needs: compile-supportbundle
steps:
@@ -113,9 +116,10 @@ jobs:
# Additional e2e tests for support bundle that run in Go, these create a Kind cluster
validate-supportbundle-e2e-go:
runs-on: ubuntu-latest
needs: compile-supportbundle
steps:
if: github.event_name == 'push'
runs-on: ubuntu-latest
needs: compile-supportbundle
steps:
- uses: actions/checkout@v5
- name: Download support bundle binary
uses: actions/download-artifact@v5
@@ -133,6 +137,7 @@ jobs:
# summary jobs, these jobs will only run if all the other jobs have succeeded
validate-pr-tests:
if: github.event_name == 'push'
runs-on: ubuntu-latest
needs:
- tidy-check
@@ -147,10 +152,10 @@ jobs:
# this job will validate that the validation did not fail and that all pr-tests succeed
# it is used for the github branch protection rule
validate-success:
if: ${{ always() && github.event_name == 'push' }}
runs-on: ubuntu-latest
needs:
- validate-pr-tests
if: always()
steps:
# https://docs.github.com/en/actions/learn-github-actions/contexts#needs-context
# if the validate-pr-tests job was not successful, this job will fail
@@ -1,4 +1,4 @@
name: build-test
name: build
on:
pull_request:
@@ -47,6 +47,19 @@ jobs:
- uses: actions/checkout@v5
- uses: ./.github/actions/setup-go
- name: Cache Go build and modules
uses: actions/cache@v4
with:
path: |
~/.cache/go-build
~/go/pkg/mod
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-
- name: Go mod download
run: go mod download
- name: Check go mod tidy
run: |
go mod tidy
@@ -65,22 +78,7 @@ jobs:
make vet
# Unit and integration tests
test:
if: needs.changes.outputs.go-files == 'true'
needs: [changes, lint]
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@v5
- uses: ./.github/actions/setup-go
- name: Setup K3s
uses: replicatedhq/action-k3s@main
with:
version: v1.31.2-k3s1
- name: Run tests
run: make test-integration
# (moved to push-full-tests.yml)
# Build binaries
build:
@@ -91,6 +89,19 @@ jobs:
steps:
- uses: actions/checkout@v5
- uses: ./.github/actions/setup-go
- name: Cache Go build and modules
uses: actions/cache@v4
with:
path: |
~/.cache/go-build
~/go/pkg/mod
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-
- name: Go mod download
run: go mod download
- run: make build
- uses: actions/upload-artifact@v4
with:
@@ -98,64 +109,26 @@ jobs:
path: bin/
retention-days: 1
# E2E tests
e2e:
if: needs.changes.outputs.go-files == 'true' || github.event_name == 'push'
needs: [changes, build]
runs-on: ubuntu-latest
timeout-minutes: 15
strategy:
fail-fast: false
matrix:
include:
- name: preflight
target: preflight-e2e-test
needs-k3s: true
- name: support-bundle-shell
target: support-bundle-e2e-test
needs-k3s: true
- name: support-bundle-go
target: support-bundle-e2e-go-test
needs-k3s: false
steps:
- uses: actions/checkout@v5
- name: Setup K3s
if: matrix.needs-k3s
uses: replicatedhq/action-k3s@main
with:
version: v1.31.2-k3s1
- uses: actions/download-artifact@v4
with:
name: binaries
path: bin/
- run: chmod +x bin/*
- run: make ${{ matrix.target }}
# (moved to push-full-tests.yml)
# Success summary
success:
if: always()
needs: [lint, test, build, e2e]
needs: [lint, build]
runs-on: ubuntu-latest
steps:
- name: Check results
run: |
# Check if any required jobs failed
if [[ "${{ needs.lint.result }}" == "failure" ]] || \
[[ "${{ needs.test.result }}" == "failure" ]] || \
[[ "${{ needs.build.result }}" == "failure" ]] || \
[[ "${{ needs.e2e.result }}" == "failure" ]]; then
[[ "${{ needs.build.result }}" == "failure" ]]; then
echo "::error::Some jobs failed or were cancelled"
exit 1
fi
# Check if any required jobs were cancelled
if [[ "${{ needs.lint.result }}" == "cancelled" ]] || \
[[ "${{ needs.test.result }}" == "cancelled" ]] || \
[[ "${{ needs.build.result }}" == "cancelled" ]] || \
[[ "${{ needs.e2e.result }}" == "cancelled" ]]; then
[[ "${{ needs.build.result }}" == "cancelled" ]]; then
echo "::error::Some jobs failed or were cancelled"
exit 1
fi
+77
View File
@@ -0,0 +1,77 @@
name: push-full-tests
on:
push:
branches: [main]
jobs:
unit-integration:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@v5
- uses: ./.github/actions/setup-go
- name: Setup K3s
uses: replicatedhq/action-k3s@main
with:
version: v1.31.2-k3s1
- name: Run tests
run: make test-integration
build-binaries:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v5
- uses: ./.github/actions/setup-go
- name: Cache Go build and modules
uses: actions/cache@v4
with:
path: |
~/.cache/go-build
~/go/pkg/mod
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-
- name: Go mod download
run: go mod download
- name: Build binaries
run: make build
- uses: actions/upload-artifact@v4
with:
name: binaries
path: bin/
retention-days: 1
e2e:
needs: [unit-integration, build-binaries]
runs-on: ubuntu-latest
timeout-minutes: 15
strategy:
fail-fast: false
matrix:
include:
- name: preflight
target: preflight-e2e-test
needs-k3s: true
- name: support-bundle-shell
target: support-bundle-e2e-test
needs-k3s: true
- name: support-bundle-go
target: support-bundle-e2e-go-test
needs-k3s: false
steps:
- uses: actions/checkout@v5
- uses: actions/download-artifact@v4
with:
name: binaries
path: bin/
- run: chmod +x bin/*
- name: Setup K3s
if: matrix.needs-k3s
uses: replicatedhq/action-k3s@main
with:
version: v1.31.2-k3s1
- run: make ${{ matrix.target }}
+2 -2
View File
@@ -2,7 +2,7 @@ name: Regression Test Suite
on:
push:
branches: [main, v1beta3]
branches: [main]
pull_request:
types: [opened, synchronize, reopened]
workflow_dispatch:
@@ -289,4 +289,4 @@ jobs:
continue-on-error: true
with:
api-token: ${{ secrets.REPLICATED_API_TOKEN }}
cluster-id: ${{ steps.create-cluster.outputs.cluster-id }}
cluster-id: ${{ steps.create-cluster.outputs.cluster-id }}
+25 -6
View File
@@ -37,7 +37,7 @@ endef
BUILDTAGS = "netgo containers_image_ostree_stub exclude_graphdriver_devicemapper exclude_graphdriver_btrfs containers_image_openpgp"
BUILDFLAGS = -tags ${BUILDTAGS} -installsuffix netgo
BUILDPATHS = ./pkg/... ./cmd/... ./internal/...
E2EPATHS = ./test/e2e/...
E2EPATHS ?= ./test/e2e/...
TESTFLAGS ?= -v -coverprofile cover.out
.DEFAULT_GOAL := all
@@ -49,12 +49,23 @@ ffi: fmt vet
.PHONY: test
test: generate fmt vet
if [ -n $(RUN) ]; then \
go test ${BUILDFLAGS} ${BUILDPATHS} ${TESTFLAGS} -run $(RUN); \
if [ -n "$(RUN)" ]; then \
go test ${BUILDFLAGS} ${BUILDPATHS} ${TESTFLAGS} -run "$(RUN)"; \
else \
go test ${BUILDFLAGS} ${BUILDPATHS} ${TESTFLAGS}; \
fi
# Run unit tests only for a provided list of packages.
# Usage: make test-packages PACKAGES="pkg/a pkg/b cmd/foo"
.PHONY: test-packages
test-packages:
@if [ -z "$(PACKAGES)" ]; then \
echo "No PACKAGES provided; nothing to test."; \
exit 0; \
fi
@echo "Running unit tests for packages: $(PACKAGES)"
go test ${BUILDFLAGS} $(PACKAGES) ${TESTFLAGS}
# Go tests that require a K8s instance
# TODOLATER: merge with test, so we get unified coverage reports? it'll add 21~sec to the test job though...
.PHONY: test-integration
@@ -73,10 +84,18 @@ run-examples:
support-bundle-e2e-test:
./test/validate-support-bundle-e2e.sh
.PHONY: preflight-e2e-go-test
preflight-e2e-go-test: bin/preflight
if [ -n "$(RUN)" ]; then \
go test ${BUILDFLAGS} ${E2EPATHS} -v -run "$(RUN)"; \
else \
go test ${BUILDFLAGS} ${E2EPATHS} -v; \
fi
.PHONY: support-bundle-e2e-go-test
support-bundle-e2e-go-test:
if [ -n $(RUN) ]; then \
go test ${BUILDFLAGS} ${E2EPATHS} -v -run $(RUN); \
support-bundle-e2e-go-test: bin/support-bundle
if [ -n "$(RUN)" ]; then \
go test ${BUILDFLAGS} ${E2EPATHS} -v -run "$(RUN)"; \
else \
go test ${BUILDFLAGS} ${E2EPATHS} -v; \
fi
+13
View File
@@ -0,0 +1,13 @@
package main
import (
"os"
analyzecli "github.com/replicatedhq/troubleshoot/cmd/analyze/cli"
)
func main() {
if err := analyzecli.RootCmd().Execute(); err != nil {
os.Exit(1)
}
}
+644 -105
View File
@@ -421,39 +421,31 @@ func (a *OllamaAgent) Analyze(ctx context.Context, data []byte, analyzers []anal
func (a *OllamaAgent) discoverAnalyzers(bundle *analyzer.SupportBundle) []analyzer.AnalyzerSpec {
var specs []analyzer.AnalyzerSpec
// Collect files by type for aggregation
podFiles := []string{}
deploymentFiles := []string{}
eventFiles := []string{}
nodeFiles := []string{}
// Analyze bundle contents to determine what types of analysis to perform
for filePath := range bundle.Files {
filePath = strings.ToLower(filePath)
filePathLower := strings.ToLower(filePath)
switch {
case strings.Contains(filePath, "pods") && strings.HasSuffix(filePath, ".json"):
specs = append(specs, analyzer.AnalyzerSpec{
Name: "ai-pod-analysis",
Type: "ai-workload",
Category: "pods",
Priority: 10,
Config: map[string]interface{}{"filePath": filePath, "promptType": "pod-analysis"},
})
case strings.Contains(filePathLower, "pods") && strings.HasSuffix(filePathLower, ".json"):
podFiles = append(podFiles, filePath)
case strings.Contains(filePath, "deployments") && strings.HasSuffix(filePath, ".json"):
specs = append(specs, analyzer.AnalyzerSpec{
Name: "ai-deployment-analysis",
Type: "ai-workload",
Category: "deployments",
Priority: 9,
Config: map[string]interface{}{"filePath": filePath, "promptType": "deployment-analysis"},
})
case strings.Contains(filePathLower, "deployments") && strings.HasSuffix(filePathLower, ".json"):
deploymentFiles = append(deploymentFiles, filePath)
case strings.Contains(filePath, "events") && strings.HasSuffix(filePath, ".json"):
specs = append(specs, analyzer.AnalyzerSpec{
Name: "ai-event-analysis",
Type: "ai-events",
Category: "events",
Priority: 8,
Config: map[string]interface{}{"filePath": filePath, "promptType": "event-analysis"},
})
case strings.Contains(filePathLower, "events") && strings.HasSuffix(filePathLower, ".json"):
eventFiles = append(eventFiles, filePath)
case strings.Contains(filePath, "logs") && strings.HasSuffix(filePath, ".log"):
case strings.Contains(filePathLower, "nodes") && strings.HasSuffix(filePathLower, ".json"):
nodeFiles = append(nodeFiles, filePath)
case strings.Contains(filePathLower, "logs") && strings.HasSuffix(filePathLower, ".log"):
// Logs are analyzed separately per file (not aggregated)
specs = append(specs, analyzer.AnalyzerSpec{
Name: "ai-log-analysis",
Type: "ai-logs",
@@ -461,50 +453,424 @@ func (a *OllamaAgent) discoverAnalyzers(bundle *analyzer.SupportBundle) []analyz
Priority: 7,
Config: map[string]interface{}{"filePath": filePath, "promptType": "log-analysis"},
})
case strings.Contains(filePath, "nodes") && strings.HasSuffix(filePath, ".json"):
specs = append(specs, analyzer.AnalyzerSpec{
Name: "ai-resource-analysis",
Type: "ai-resources",
Category: "nodes",
Priority: 8,
Config: map[string]interface{}{"filePath": filePath, "promptType": "resource-analysis"},
})
}
}
// Create aggregated analyzer for ALL pod files (cluster-wide view)
if len(podFiles) > 0 {
specs = append(specs, analyzer.AnalyzerSpec{
Name: "ai-pod-analysis-cluster",
Type: "ai-workload",
Category: "pods",
Priority: 10,
Config: map[string]interface{}{
"filePaths": podFiles,
"promptType": "pod-analysis",
"aggregated": true,
},
})
}
// Create aggregated analyzer for ALL deployment files (cluster-wide view)
if len(deploymentFiles) > 0 {
specs = append(specs, analyzer.AnalyzerSpec{
Name: "ai-deployment-analysis-cluster",
Type: "ai-workload",
Category: "deployments",
Priority: 9,
Config: map[string]interface{}{
"filePaths": deploymentFiles,
"promptType": "deployment-analysis",
"aggregated": true,
},
})
}
// Create aggregated analyzer for ALL event files (cluster-wide view)
if len(eventFiles) > 0 {
specs = append(specs, analyzer.AnalyzerSpec{
Name: "ai-event-analysis-cluster",
Type: "ai-events",
Category: "events",
Priority: 8,
Config: map[string]interface{}{
"filePaths": eventFiles,
"promptType": "event-analysis",
"aggregated": true,
},
})
}
// Create aggregated analyzer for ALL node files (cluster-wide view)
if len(nodeFiles) > 0 {
specs = append(specs, analyzer.AnalyzerSpec{
Name: "ai-resource-analysis-cluster",
Type: "ai-resources",
Category: "nodes",
Priority: 8,
Config: map[string]interface{}{
"filePaths": nodeFiles,
"promptType": "resource-analysis",
"aggregated": true,
},
})
}
return specs
}
// aggregateFiles combines multiple files of the same type into a single summary for analysis
func (a *OllamaAgent) aggregateFiles(bundle *analyzer.SupportBundle, filePaths []string, category string) (string, error) {
var summary strings.Builder
switch category {
case "pods":
return a.aggregatePodFiles(bundle, filePaths)
case "deployments":
return a.aggregateDeploymentFiles(bundle, filePaths)
case "events":
return a.aggregateEventFiles(bundle, filePaths)
case "nodes":
return a.aggregateNodeFiles(bundle, filePaths)
default:
// For other types, just concatenate the files
summary.WriteString(fmt.Sprintf("Aggregated analysis of %d files:\n\n", len(filePaths)))
for _, filePath := range filePaths {
if data, exists := bundle.Files[filePath]; exists {
summary.WriteString(fmt.Sprintf("--- File: %s ---\n", filePath))
summary.Write(data)
summary.WriteString("\n\n")
}
}
}
return summary.String(), nil
}
// aggregatePodFiles creates a cluster-wide summary of pods from multiple namespace files
func (a *OllamaAgent) aggregatePodFiles(bundle *analyzer.SupportBundle, filePaths []string) (string, error) {
var summary strings.Builder
totalPods := 0
runningPods := 0
pendingPods := 0
failedPods := 0
succeededPods := 0
namespaceStats := make(map[string]int)
summary.WriteString("CLUSTER-WIDE POD ANALYSIS\n")
summary.WriteString("Analyzing pods across all namespaces:\n\n")
for _, filePath := range filePaths {
data, exists := bundle.Files[filePath]
if !exists {
continue
}
// Extract namespace from path (e.g., "cluster-resources/pods/kube-system.json")
parts := strings.Split(filePath, "/")
namespace := "unknown"
if len(parts) >= 3 {
namespace = strings.TrimSuffix(parts[len(parts)-1], ".json")
}
// Parse pod data - handle both PodList and single Pod objects
var podList map[string]interface{}
if err := json.Unmarshal(data, &podList); err != nil {
continue
}
// Check if this is a List object with items array
items, ok := podList["items"].([]interface{})
if ok {
// Handle PodList - process all pods in the list
// Initialize namespace for valid PodList (ensures empty namespaces are tracked)
if _, exists := namespaceStats[namespace]; !exists {
namespaceStats[namespace] = 0
}
podCount := len(items)
namespaceStats[namespace] += podCount
totalPods += podCount
// Count pod statuses
for _, item := range items {
pod, ok := item.(map[string]interface{})
if !ok {
continue
}
status, ok := pod["status"].(map[string]interface{})
if !ok {
continue
}
phase, ok := status["phase"].(string)
if !ok {
continue
}
switch phase {
case "Running":
runningPods++
case "Pending":
pendingPods++
case "Failed":
failedPods++
case "Succeeded":
succeededPods++
}
}
} else {
// Handle single Pod object (not a list)
// Check if this is a single Pod object (has "kind": "Pod")
if kind, exists := podList["kind"].(string); exists && kind == "Pod" {
// Initialize namespace only for valid pod data
if _, exists := namespaceStats[namespace]; !exists {
namespaceStats[namespace] = 0
}
// Single pod - increment count for this namespace
namespaceStats[namespace]++
totalPods++
// Extract status for single pod
if status, ok := podList["status"].(map[string]interface{}); ok {
if phase, ok := status["phase"].(string); ok {
switch phase {
case "Running":
runningPods++
case "Pending":
pendingPods++
case "Failed":
failedPods++
case "Succeeded":
succeededPods++
}
}
}
}
// Skip to next file after processing single pod or invalid data
continue
}
}
summary.WriteString(fmt.Sprintf("Total pods in cluster: %d\n", totalPods))
summary.WriteString(fmt.Sprintf(" - Running: %d\n", runningPods))
summary.WriteString(fmt.Sprintf(" - Pending: %d\n", pendingPods))
summary.WriteString(fmt.Sprintf(" - Failed: %d\n", failedPods))
summary.WriteString(fmt.Sprintf(" - Succeeded: %d\n", succeededPods))
summary.WriteString("\nPods by namespace:\n")
for namespace, count := range namespaceStats {
if count > 0 {
summary.WriteString(fmt.Sprintf(" - %s: %d pods\n", namespace, count))
} else {
summary.WriteString(fmt.Sprintf(" - %s: empty (no pods)\n", namespace))
}
}
summary.WriteString("\nIMPORTANT CONTEXT:\n")
summary.WriteString("- Empty namespaces are NORMAL in Kubernetes\n")
summary.WriteString("- Only report issues if there are actual pod failures or critical problems\n")
summary.WriteString("- The presence of empty namespaces is not a problem\n")
return summary.String(), nil
}
// aggregateDeploymentFiles creates a cluster-wide summary of deployments
func (a *OllamaAgent) aggregateDeploymentFiles(bundle *analyzer.SupportBundle, filePaths []string) (string, error) {
var summary strings.Builder
totalDeployments := 0
namespaceStats := make(map[string]int)
summary.WriteString("CLUSTER-WIDE DEPLOYMENT ANALYSIS\n")
summary.WriteString("Analyzing deployments across all namespaces:\n\n")
for _, filePath := range filePaths {
data, exists := bundle.Files[filePath]
if !exists {
continue
}
parts := strings.Split(filePath, "/")
namespace := "unknown"
if len(parts) >= 3 {
namespace = strings.TrimSuffix(parts[len(parts)-1], ".json")
}
// Parse deployment data - handle both DeploymentList and single Deployment objects
var deploymentList map[string]interface{}
if err := json.Unmarshal(data, &deploymentList); err != nil {
continue
}
// Check if this is a List object with items array
items, ok := deploymentList["items"].([]interface{})
if ok {
// Handle DeploymentList - process all deployments in the list
// Initialize namespace for valid DeploymentList (ensures empty namespaces are tracked)
if _, exists := namespaceStats[namespace]; !exists {
namespaceStats[namespace] = 0
}
deployCount := len(items)
namespaceStats[namespace] += deployCount
totalDeployments += deployCount
} else {
// Handle single Deployment object (not a list)
// Check if this is a single Deployment object (has "kind": "Deployment")
if kind, exists := deploymentList["kind"].(string); exists && kind == "Deployment" {
// Initialize namespace only for valid deployment data
if _, exists := namespaceStats[namespace]; !exists {
namespaceStats[namespace] = 0
}
// Single deployment - increment count for this namespace
namespaceStats[namespace]++
totalDeployments++
}
// Skip to next file after processing single deployment or invalid data
continue
}
}
summary.WriteString(fmt.Sprintf("Total deployments in cluster: %d\n", totalDeployments))
summary.WriteString("\nDeployments by namespace:\n")
for namespace, count := range namespaceStats {
if count > 0 {
summary.WriteString(fmt.Sprintf(" - %s: %d deployments\n", namespace, count))
} else {
summary.WriteString(fmt.Sprintf(" - %s: no deployments\n", namespace))
}
}
summary.WriteString("\nIMPORTANT: Empty namespaces are normal. Only flag actual deployment issues.\n")
return summary.String(), nil
}
// aggregateEventFiles creates a cluster-wide summary of events
func (a *OllamaAgent) aggregateEventFiles(bundle *analyzer.SupportBundle, filePaths []string) (string, error) {
var summary strings.Builder
totalEvents := 0
summary.WriteString("CLUSTER-WIDE EVENT ANALYSIS\n")
summary.WriteString("Analyzing events across all namespaces:\n\n")
eventsIncluded := 0
for _, filePath := range filePaths {
data, exists := bundle.Files[filePath]
if !exists {
continue
}
// Parse event data - handle both EventList and single Event objects
var eventList map[string]interface{}
if err := json.Unmarshal(data, &eventList); err != nil {
continue
}
// Check if this is a List object with items array
items, ok := eventList["items"].([]interface{})
if ok {
itemCount := len(items)
totalEvents += itemCount
// Include actual event data for AI analysis (limited to 50 events max for the summary)
// Only include if adding this file wouldn't significantly exceed the limit
if itemCount > 0 && eventsIncluded < 50 && (eventsIncluded+itemCount) <= 60 {
dataStr := string(data)
// Include file if data size is reasonable
if len(dataStr) < 2000 {
summary.WriteString(fmt.Sprintf("\n--- Events from %s ---\n", filePath))
summary.WriteString(dataStr)
summary.WriteString("\n")
eventsIncluded += itemCount
}
}
}
}
summary.WriteString(fmt.Sprintf("\nTotal events collected: %d\n", totalEvents))
return summary.String(), nil
}
// aggregateNodeFiles creates a cluster-wide summary of nodes
func (a *OllamaAgent) aggregateNodeFiles(bundle *analyzer.SupportBundle, filePaths []string) (string, error) {
var summary strings.Builder
summary.WriteString("CLUSTER-WIDE NODE ANALYSIS\n\n")
for _, filePath := range filePaths {
data, exists := bundle.Files[filePath]
if !exists {
continue
}
summary.WriteString(fmt.Sprintf("--- Nodes data from %s ---\n", filePath))
summary.Write(data)
summary.WriteString("\n\n")
}
return summary.String(), nil
}
// runLLMAnalysis executes analysis using LLM for a specific analyzer spec
func (a *OllamaAgent) runLLMAnalysis(ctx context.Context, bundle *analyzer.SupportBundle, spec analyzer.AnalyzerSpec) (*analyzer.AnalyzerResult, error) {
ctx, span := otel.Tracer(constants.LIB_TRACER_NAME).Start(ctx, fmt.Sprintf("OllamaAgent.%s", spec.Name))
defer span.End()
// Smart file detection for enhanced analyzer compatibility
var filePath string
var fileData []byte
var exists bool
var dataStr string
// First try to get explicit filePath from config
if fp, ok := spec.Config["filePath"].(string); ok {
filePath = fp
fileData, exists = bundle.Files[filePath]
}
// If no explicit filePath, auto-detect based on analyzer type
if !exists {
filePath, fileData, exists = a.autoDetectFileForAnalyzer(bundle, spec)
}
if !exists {
result := &analyzer.AnalyzerResult{
Title: spec.Name,
IsWarn: true,
Message: fmt.Sprintf("File not found: %s", filePath),
Category: spec.Category,
// Check if this is an aggregated analyzer (multiple files)
if aggregated, ok := spec.Config["aggregated"].(bool); ok && aggregated {
// Handle aggregated files
if filePaths, ok := spec.Config["filePaths"].([]string); ok && len(filePaths) > 0 {
aggregatedData, err := a.aggregateFiles(bundle, filePaths, spec.Category)
if err != nil {
return &analyzer.AnalyzerResult{
Title: spec.Name,
IsWarn: true,
Message: fmt.Sprintf("Failed to aggregate files: %v", err),
Category: spec.Category,
}, nil
}
dataStr = aggregatedData
} else {
// Missing or invalid filePaths for aggregated analyzer
return &analyzer.AnalyzerResult{
Title: spec.Name,
IsWarn: true,
Message: "Aggregated analyzer missing valid filePaths configuration",
Category: spec.Category,
}, nil
}
return result, nil
} else {
// Smart file detection for enhanced analyzer compatibility (single file)
var filePath string
var fileData []byte
var exists bool
// First try to get explicit filePath from config
if fp, ok := spec.Config["filePath"].(string); ok {
filePath = fp
fileData, exists = bundle.Files[filePath]
}
// If no explicit filePath, auto-detect based on analyzer type
if !exists {
filePath, fileData, exists = a.autoDetectFileForAnalyzer(bundle, spec)
}
if !exists {
result := &analyzer.AnalyzerResult{
Title: spec.Name,
IsWarn: true,
Message: fmt.Sprintf("File not found: %s", filePath),
Category: spec.Category,
}
return result, nil
}
dataStr = string(fileData)
}
promptType, _ := spec.Config["promptType"].(string)
@@ -519,7 +885,6 @@ func (a *OllamaAgent) runLLMAnalysis(ctx context.Context, bundle *analyzer.Suppo
}
// Prepare data for analysis (truncate if too large)
dataStr := string(fileData)
if len(dataStr) > 4000 { // Limit input size
if promptType == "log-analysis" {
// For logs, take the last N lines
@@ -866,6 +1231,182 @@ func (a *OllamaAgent) autoDetectFileForAnalyzer(bundle *analyzer.SupportBundle,
return "", nil, false
}
// normalizeInsights converts various JSON formats into a []string array
func (a *OllamaAgent) normalizeInsights(raw json.RawMessage) []string {
if len(raw) == 0 {
return []string{}
}
// Try parsing as array of strings first (expected format)
var arrayInsights []string
if err := json.Unmarshal(raw, &arrayInsights); err == nil {
return arrayInsights
}
// Try parsing as single string
var stringInsight string
if err := json.Unmarshal(raw, &stringInsight); err == nil {
if stringInsight != "" {
return []string{stringInsight}
}
return []string{}
}
// Try parsing as array of objects/maps (common LLM format)
var arrayOfMaps []map[string]interface{}
if err := json.Unmarshal(raw, &arrayOfMaps); err == nil {
insights := []string{}
for _, obj := range arrayOfMaps {
// Extract meaningful text from each object
insightText := a.formatMapAsInsight(obj)
if insightText != "" {
insights = append(insights, insightText)
}
}
return insights
}
// Try parsing as object/map and extract meaningful text
var objInsights map[string]interface{}
if err := json.Unmarshal(raw, &objInsights); err == nil {
insights := []string{}
for key, value := range objInsights {
// Extract meaningful insights from object structure
insightText := a.extractInsightText(key, value)
if insightText != "" {
insights = append(insights, insightText)
}
}
return insights
}
// If all parsing fails, return empty array
return []string{}
}
// formatMapAsInsight converts a map/object into a readable insight string
func (a *OllamaAgent) formatMapAsInsight(obj map[string]interface{}) string {
// Common patterns in LLM responses for insights
// Try to extract description, pattern, message, etc.
// Priority 1: Look for description field
if desc, ok := obj["description"].(string); ok && desc != "" {
if pattern, ok := obj["pattern"].(string); ok && pattern != "" {
return fmt.Sprintf("%s: %s", pattern, desc)
}
return desc
}
// Priority 2: Look for message field
if msg, ok := obj["message"].(string); ok && msg != "" {
return msg
}
// Priority 3: Look for explanation/implication field
if expl, ok := obj["explanation"].(string); ok && expl != "" {
return expl
}
if impl, ok := obj["implication"].(string); ok && impl != "" {
return impl
}
// Priority 4: Combine all string fields
parts := []string{}
for key, value := range obj {
if str, ok := value.(string); ok && str != "" {
parts = append(parts, fmt.Sprintf("%s: %s", key, str))
}
}
if len(parts) > 0 {
return strings.Join(parts, ", ")
}
return ""
}
// extractInsightText extracts readable text from nested JSON structures
func (a *OllamaAgent) extractInsightText(key string, value interface{}) string {
switch v := value.(type) {
case string:
if v != "" {
return fmt.Sprintf("%s: %s", key, v)
}
case map[string]interface{}:
// For nested objects, create a summary
parts := []string{}
for subKey, subValue := range v {
if str, ok := subValue.(string); ok && str != "" {
parts = append(parts, fmt.Sprintf("%s=%s", subKey, str))
}
}
if len(parts) > 0 {
return fmt.Sprintf("%s: %s", key, strings.Join(parts, ", "))
}
case []interface{}:
// For arrays, join elements
parts := []string{}
for _, item := range v {
if str, ok := item.(string); ok && str != "" {
parts = append(parts, str)
}
}
if len(parts) > 0 {
return fmt.Sprintf("%s: %s", key, strings.Join(parts, ", "))
}
case float64, int, bool:
return fmt.Sprintf("%s: %v", key, v)
}
return ""
}
// getStringField extracts a string field from a map, trying multiple key variants
func (a *OllamaAgent) getStringField(m map[string]interface{}, keys ...string) string {
for _, key := range keys {
if val, ok := m[key]; ok {
if str, ok := val.(string); ok {
return str
}
}
}
return ""
}
// extractRemediation extracts remediation info from various JSON structures
func (a *OllamaAgent) extractRemediation(result *analyzer.AnalyzerResult, remData interface{}) {
switch rem := remData.(type) {
case map[string]interface{}:
// Single remediation object
desc := a.getStringField(rem, "description", "Description")
action := a.getStringField(rem, "action", "Action")
command := a.getStringField(rem, "command", "Command")
priority := 5 // default priority
if p, ok := rem["priority"].(float64); ok {
priority = int(p)
} else if p, ok := rem["Priority"].(float64); ok {
priority = int(p)
}
if desc != "" || action != "" {
result.Remediation = &analyzer.RemediationStep{
Description: desc,
Action: action,
Command: command,
Priority: priority,
Category: "ai-suggested",
IsAutomatable: false,
}
}
case []interface{}:
// Array of remediation suggestions - use the first one
if len(rem) > 0 {
if firstRem, ok := rem[0].(map[string]interface{}); ok {
a.extractRemediation(result, firstRem)
}
}
}
}
// parseLLMResponse parses the LLM response into an AnalyzerResult
func (a *OllamaAgent) parseLLMResponse(response string, spec analyzer.AnalyzerSpec) (*analyzer.AnalyzerResult, error) {
// First try JSON parsing
@@ -875,55 +1416,53 @@ func (a *OllamaAgent) parseLLMResponse(response string, spec analyzer.AnalyzerSp
if jsonStart != -1 && jsonEnd != -1 && jsonEnd > jsonStart {
jsonStr := response[jsonStart : jsonEnd+1]
var llmResult struct {
Status string `json:"status"`
Title string `json:"title"`
Message string `json:"message"`
Insights []string `json:"insights"`
Remediation struct {
Description string `json:"description"`
Action string `json:"action"`
Command string `json:"command"`
Priority int `json:"priority"`
} `json:"remediation"`
}
if err := json.Unmarshal([]byte(jsonStr), &llmResult); err == nil {
// Successfully parsed JSON
result := &analyzer.AnalyzerResult{
Title: llmResult.Title,
Message: llmResult.Message,
Category: spec.Category,
Insights: llmResult.Insights,
}
switch strings.ToLower(llmResult.Status) {
case "pass":
result.IsPass = true
case "warn":
result.IsWarn = true
case "fail":
result.IsFail = true
default:
result.IsWarn = true
}
if llmResult.Remediation.Description != "" {
result.Remediation = &analyzer.RemediationStep{
Description: llmResult.Remediation.Description,
Action: llmResult.Remediation.Action,
Command: llmResult.Remediation.Command,
Priority: llmResult.Remediation.Priority,
Category: "ai-suggested",
IsAutomatable: false,
}
}
return result, nil
} else {
// JSON was found but malformed
// Try with a flexible map first to handle case-insensitive fields
var jsonMap map[string]interface{}
if err := json.Unmarshal([]byte(jsonStr), &jsonMap); err != nil {
return nil, errors.Wrap(err, "failed to parse LLM JSON response")
}
// Extract fields in a case-insensitive way
status := a.getStringField(jsonMap, "status", "Status")
title := a.getStringField(jsonMap, "title", "Title")
message := a.getStringField(jsonMap, "message", "Message")
// Get insights field (try both lowercase and uppercase)
var insightsRaw json.RawMessage
if insights, ok := jsonMap["insights"]; ok {
insightsRaw, _ = json.Marshal(insights)
} else if insights, ok := jsonMap["Insights"]; ok {
insightsRaw, _ = json.Marshal(insights)
}
insights := a.normalizeInsights(insightsRaw)
result := &analyzer.AnalyzerResult{
Title: title,
Message: message,
Category: spec.Category,
Insights: insights,
}
switch strings.ToLower(status) {
case "pass":
result.IsPass = true
case "warn":
result.IsWarn = true
case "fail":
result.IsFail = true
default:
result.IsWarn = true
}
// Handle remediation (try both cases)
if rem, ok := jsonMap["remediation"]; ok {
a.extractRemediation(result, rem)
} else if rem, ok := jsonMap["Remediation"]; ok {
a.extractRemediation(result, rem)
}
return result, nil
}
// Fall back to markdown parsing when JSON fails
@@ -206,7 +206,10 @@ func TestOllamaAgent_discoverAnalyzers(t *testing.T) {
assert.NotNil(t, spec.Config)
// Verify AI-specific config
assert.Contains(t, spec.Config, "filePath")
// Aggregated analyzers use "filePaths", single-file analyzers use "filePath"
hasFilePath := spec.Config["filePath"] != nil
hasFilePaths := spec.Config["filePaths"] != nil
assert.True(t, hasFilePath || hasFilePaths, "spec must have either filePath or filePaths")
assert.Contains(t, spec.Config, "promptType")
}
+549
View File
@@ -0,0 +1,549 @@
package main
import (
"bufio"
"bytes"
"encoding/json"
"errors"
"flag"
"fmt"
"io"
"io/fs"
"os"
"os/exec"
"path/filepath"
"regexp"
"sort"
"strings"
)
// goListPackageJSON models the subset of fields we need from `go list -json` output.
// The JSON can be quite large; we intentionally only decode what we use to keep memory reasonable.
type goListPackageJSON struct {
ImportPath string `json:"ImportPath"`
Deps []string `json:"Deps"`
}
// runCommand executes a command and returns stdout as bytes with trimmed trailing newline.
func runCommand(name string, args ...string) ([]byte, error) {
cmd := exec.Command(name, args...)
cmd.Stderr = os.Stderr
out, err := cmd.Output()
if err != nil {
return nil, err
}
return bytes.TrimRight(out, "\n"), nil
}
// listPackageWithDeps returns the full transitive dependency set for a package import path,
// including the package itself.
func listPackageWithDeps(importPath string) (map[string]struct{}, error) {
cmd := exec.Command("go", "list", "-json", "-deps", importPath)
cmd.Stderr = os.Stderr
out, err := cmd.Output()
if err != nil {
return nil, fmt.Errorf("go list -json -deps %s failed: %w", importPath, err)
}
deps := make(map[string]struct{})
dec := json.NewDecoder(bytes.NewReader(out))
for {
var pkg goListPackageJSON
if err := dec.Decode(&pkg); err != nil {
if errors.Is(err, io.EOF) {
break
}
return nil, fmt.Errorf("decode go list json: %w", err)
}
if pkg.ImportPath != "" {
deps[pkg.ImportPath] = struct{}{}
}
for _, d := range pkg.Deps {
deps[d] = struct{}{}
}
}
return deps, nil
}
// changedFiles returns a slice of file paths changed between baseRef and HEAD.
func changedFiles(baseRef string) ([]string, error) {
// Use triple-dot to include merge base with baseRef, typical for PR diffs.
out, err := runCommand("git", "diff", "--name-only", baseRef+"...HEAD")
if err != nil {
return nil, fmt.Errorf("git diff failed: %w", err)
}
var files []string
scanner := bufio.NewScanner(bytes.NewReader(out))
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if line == "" {
continue
}
files = append(files, line)
}
if err := scanner.Err(); err != nil {
return nil, err
}
return files, nil
}
// mapFilesToPackages resolves a set of Go package import paths that directly contain the changed Go files.
func mapFilesToPackages(files []string) (map[string]struct{}, error) {
packages := make(map[string]struct{})
// Collect unique directories that contain changed Go files.
dirSet := make(map[string]struct{})
for _, f := range files {
if strings.HasPrefix(f, "vendor/") {
continue
}
if filepath.Ext(f) != ".go" {
continue
}
d := filepath.Dir(f)
if d == "." {
d = "."
}
dirSet[d] = struct{}{}
}
if len(dirSet) == 0 {
return packages, nil
}
// Convert to a stable-ordered slice of directories to avoid nondeterminism.
var dirs []string
for d := range dirSet {
// Ensure relative paths are treated as packages; prepend ./ for clarity.
if strings.HasPrefix(d, "./") || d == "." {
dirs = append(dirs, d)
} else {
dirs = append(dirs, "./"+d)
}
}
sort.Strings(dirs)
// `go list` accepts directories and returns their package import paths.
args := append([]string{"list", "-f", "{{.ImportPath}}"}, dirs...)
out, err := runCommand("go", args...)
if err != nil {
return nil, fmt.Errorf("go list for files failed: %w", err)
}
scanner := bufio.NewScanner(bytes.NewReader(out))
for scanner.Scan() {
pkg := strings.TrimSpace(scanner.Text())
if pkg != "" {
packages[pkg] = struct{}{}
}
}
if err := scanner.Err(); err != nil {
return nil, err
}
return packages, nil
}
// computeAffectedPackages expands directly changed packages to include reverse dependencies across the module.
// We query all packages with test dependencies (-test) to ensure test-only imports are considered.
func computeAffectedPackages(directPkgs map[string]struct{}) (map[string]struct{}, error) {
affected := make(map[string]struct{})
for p := range directPkgs {
affected[p] = struct{}{}
}
if len(directPkgs) == 0 {
return affected, nil
}
// Enumerate all packages in the module with their deps.
// We stream decode concatenated JSON objects produced by `go list -json`.
cmd := exec.Command("go", "list", "-json", "-deps", "-test", "./...")
cmd.Stderr = os.Stderr
out, err := cmd.Output()
if err != nil {
return nil, fmt.Errorf("go list -json failed: %w", err)
}
dec := json.NewDecoder(bytes.NewReader(out))
for {
var pkg goListPackageJSON
if err := dec.Decode(&pkg); err != nil {
if errors.Is(err, io.EOF) {
break
}
return nil, fmt.Errorf("decode go list json: %w", err)
}
// If this package is directly changed, it's already included.
// If it depends (directly or transitively) on any changed package, include it.
for changed := range directPkgs {
if pkg.ImportPath == changed {
affected[pkg.ImportPath] = struct{}{}
break
}
// Linear scan over deps is acceptable given typical package counts.
for _, dep := range pkg.Deps {
if dep == changed {
affected[pkg.ImportPath] = struct{}{}
break
}
}
}
}
return affected, nil
}
// listTestFunctions scans a directory for Go test files and returns names of functions
// that match the pattern `func TestXxx(t *testing.T)`.
func listTestFunctions(dir string) ([]string, error) {
var tests []string
// Regex to capture test function names. This is a simple heuristic suitable for our codebase.
testFuncRe := regexp.MustCompile(`^func\s+(Test[\w\d_]+)\s*\(`)
walkFn := func(path string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() {
return nil
}
if !strings.HasSuffix(d.Name(), "_test.go") {
return nil
}
b, err := os.ReadFile(path)
if err != nil {
return err
}
scanner := bufio.NewScanner(bytes.NewReader(b))
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if m := testFuncRe.FindStringSubmatch(line); m != nil {
tests = append(tests, m[1])
}
}
return scanner.Err()
}
if err := filepath.WalkDir(dir, walkFn); err != nil {
return nil, err
}
sort.Strings(tests)
return tests, nil
}
func main() {
baseRef := flag.String("base", "origin/main", "Git base ref to diff against (e.g., origin/main)")
printAllOnChanges := flag.Bool("all-on-mod-change", true, "Run all tests if go.mod or go.sum changed")
verbose := flag.Bool("v", false, "Enable verbose diagnostics to stderr")
mode := flag.String("mode", "packages", "Output mode: 'packages' to print import paths; 'suites' to print e2e suite names")
changedFilesCSV := flag.String("changed-files", "", "Comma-separated paths to treat as changed (bypass git)")
changedFilesFile := flag.String("changed-files-file", "", "File with newline-separated paths to treat as changed")
flag.Parse()
// Determine the set of changed files: explicit list if provided, otherwise via git diff.
var files []string
if *changedFilesCSV != "" || *changedFilesFile != "" {
if *changedFilesCSV != "" {
parts := strings.Split(*changedFilesCSV, ",")
for _, p := range parts {
if s := strings.TrimSpace(p); s != "" {
files = append(files, s)
}
}
}
if *changedFilesFile != "" {
b, err := os.ReadFile(*changedFilesFile)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(2)
}
scanner := bufio.NewScanner(bytes.NewReader(b))
for scanner.Scan() {
if s := strings.TrimSpace(scanner.Text()); s != "" {
files = append(files, s)
}
}
if err := scanner.Err(); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(2)
}
}
} else {
var err error
files, err = changedFiles(*baseRef)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(2)
}
}
if *verbose {
fmt.Fprintln(os.Stderr, "Changed files vs base:")
if len(files) == 0 {
fmt.Fprintln(os.Stderr, " (none)")
} else {
for _, f := range files {
fmt.Fprintln(os.Stderr, " ", f)
}
}
}
// Track module change and CI configuration changes to drive conservative behavior.
moduleChanged := false
ciChanged := false
if *printAllOnChanges {
for _, f := range files {
if f == "go.mod" || f == "go.sum" {
moduleChanged = true
}
if strings.HasPrefix(f, "scripts/") || strings.HasPrefix(f, ".github/workflows/") {
ciChanged = true
}
}
if (moduleChanged || ciChanged) && *mode == "packages" {
if *verbose {
if moduleChanged {
fmt.Fprintln(os.Stderr, "Detected module file change (go.mod/go.sum); selecting all packages ./...")
}
if ciChanged {
fmt.Fprintln(os.Stderr, "Detected CI/detector change (scripts/ or .github/workflows/); selecting all packages ./...")
}
}
fmt.Println("./...")
return
}
}
directPkgs, err := mapFilesToPackages(files)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(2)
}
if *verbose {
// Stable dump of direct packages
var dirs []string
for p := range directPkgs {
dirs = append(dirs, p)
}
sort.Strings(dirs)
fmt.Fprintln(os.Stderr, "Directly changed packages:")
if len(dirs) == 0 {
fmt.Fprintln(os.Stderr, " (none)")
} else {
for _, p := range dirs {
fmt.Fprintln(os.Stderr, " ", p)
}
}
}
switch *mode {
case "packages":
affected, err := computeAffectedPackages(directPkgs)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(2)
}
if *verbose {
var dbg []string
for p := range affected {
dbg = append(dbg, p)
}
sort.Strings(dbg)
fmt.Fprintln(os.Stderr, "Final affected packages:")
if len(dbg) == 0 {
fmt.Fprintln(os.Stderr, " (none)")
} else {
for _, p := range dbg {
fmt.Fprintln(os.Stderr, " ", p)
}
}
}
// Normalize and filter import paths:
// - Strip test variant suffixes like "pkg [pkg.test]"
// - Exclude e2e test packages (./test/e2e/...)
normalized := make(map[string]struct{})
for p := range affected {
// Trim Go test variant decorations that appear in `go list -test`
if idx := strings.Index(p, " ["); idx != -1 {
p = p[:idx]
}
// Exclude synthetic test packages like github.com/org/repo/pkg.name.test
if strings.HasSuffix(p, ".test") {
continue
}
if strings.Contains(p, "/test/e2e/") {
continue
}
if p != "" {
normalized[p] = struct{}{}
}
}
var list []string
for p := range normalized {
list = append(list, p)
}
sort.Strings(list)
for _, p := range list {
fmt.Println(p)
}
case "suites":
// Determine impacted suites by dependency mapping and direct e2e test changes,
// then print exact test names for those suites.
preflightRoot := "github.com/replicatedhq/troubleshoot/cmd/preflight"
supportRoot := "github.com/replicatedhq/troubleshoot/cmd/troubleshoot"
preflightDeps, err := listPackageWithDeps(preflightRoot)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(2)
}
supportDeps, err := listPackageWithDeps(supportRoot)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(2)
}
preflightHit := false
supportHit := false
// Track whether e2e test files were directly changed per suite and collect specific test names
changedPreflightTests := make(map[string]struct{})
changedSupportTests := make(map[string]struct{})
preflightE2EChangedNonGo := false
supportE2EChangedNonGo := false
for _, f := range files {
if strings.HasPrefix(f, "test/e2e/preflight/") {
if strings.HasSuffix(f, "_test.go") {
// Extract test names from just this file
b, err := os.ReadFile(f)
if err == nil { // ignore read errors; they will be caught later if needed
scanner := bufio.NewScanner(bytes.NewReader(b))
re := regexp.MustCompile(`^func\s+(Test[\w\d_]+)\s*\(`)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if m := re.FindStringSubmatch(line); m != nil {
changedPreflightTests[m[1]] = struct{}{}
}
}
}
preflightHit = true
} else {
// Non-go change under preflight e2e; run whole suite
preflightE2EChangedNonGo = true
preflightHit = true
}
}
if strings.HasPrefix(f, "test/e2e/support-bundle/") {
if strings.HasSuffix(f, "_test.go") {
b, err := os.ReadFile(f)
if err == nil {
scanner := bufio.NewScanner(bytes.NewReader(b))
re := regexp.MustCompile(`^func\s+(Test[\w\d_]+)\s*\(`)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if m := re.FindStringSubmatch(line); m != nil {
changedSupportTests[m[1]] = struct{}{}
}
}
}
supportHit = true
} else {
supportE2EChangedNonGo = true
supportHit = true
}
}
}
for changed := range directPkgs {
if !preflightHit {
if _, ok := preflightDeps[changed]; ok {
preflightHit = true
}
}
if !supportHit {
if _, ok := supportDeps[changed]; ok {
supportHit = true
}
}
if preflightHit && supportHit {
break
}
}
if *verbose {
fmt.Fprintln(os.Stderr, "E2E suite impact:")
fmt.Fprintf(os.Stderr, " preflight: %v\n", preflightHit)
fmt.Fprintf(os.Stderr, " support-bundle: %v\n", supportHit)
}
// If module files or CI/detector changed, conservatively select all tests for both suites.
if moduleChanged || ciChanged {
preTests, err := listTestFunctions("test/e2e/preflight")
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(2)
}
for _, tname := range preTests {
fmt.Printf("preflight:%s\n", tname)
}
sbTests, err := listTestFunctions("test/e2e/support-bundle")
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(2)
}
for _, tname := range sbTests {
fmt.Printf("support-bundle:%s\n", tname)
}
return
}
// Collect tests for impacted suites and print as `<suite>:<TestName>`
if preflightHit || supportHit {
if preflightHit {
toPrint := make(map[string]struct{})
if preflightE2EChangedNonGo || len(changedPreflightTests) == 0 {
// Run full suite if e2e non-go assets changed or no specific test names collected
preTests, err := listTestFunctions("test/e2e/preflight")
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(2)
}
for _, t := range preTests {
toPrint[t] = struct{}{}
}
} else {
for t := range changedPreflightTests {
toPrint[t] = struct{}{}
}
}
var list []string
for t := range toPrint {
list = append(list, t)
}
sort.Strings(list)
for _, tname := range list {
fmt.Printf("preflight:%s\n", tname)
}
}
if supportHit {
toPrint := make(map[string]struct{})
if supportE2EChangedNonGo || len(changedSupportTests) == 0 {
sbTests, err := listTestFunctions("test/e2e/support-bundle")
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(2)
}
for _, t := range sbTests {
toPrint[t] = struct{}{}
}
} else {
for t := range changedSupportTests {
toPrint[t] = struct{}{}
}
}
var list []string
for t := range toPrint {
list = append(list, t)
}
sort.Strings(list)
for _, tname := range list {
fmt.Printf("support-bundle:%s\n", tname)
}
}
}
default:
fmt.Fprintln(os.Stderr, "unknown mode; use 'packages' or 'suites'")
os.Exit(2)
}
}
+67
View File
@@ -0,0 +1,67 @@
#!/usr/bin/env bash
set -euo pipefail
# 0) Preconditions (one-time)
export PATH="$(go env GOPATH)/bin:$PATH"
go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.19.0 >/dev/null
go install k8s.io/code-generator/cmd/client-gen@v0.34.0 >/dev/null
git fetch origin main --depth=1 || true
# 1) Determine changed files source: explicit args or git base diff
if [ "$#" -gt 0 ]; then
# Treat provided paths as changed files
CHANGED_CSV=$(printf "%s," "$@" | sed 's/,$//')
echo "Simulating changes in: $CHANGED_CSV"
PKGS="$(go run ./scripts/affected-packages.go -changed-files "${CHANGED_CSV}")"
E2E_OUT="$(go run ./scripts/affected-packages.go -mode=suites -changed-files "${CHANGED_CSV}")"
else
# Compute base (robust to unrelated histories)
BASE="$(git merge-base HEAD origin/main 2>/dev/null || true)"
if [ -z "${BASE}" ]; then
echo "No merge-base with origin/main → running full set"
PKGS="./..."
E2E_OUT="$(go run ./scripts/affected-packages.go -mode=suites -changed-files go.mod || true)"
else
PKGS="$(go run ./scripts/affected-packages.go -base "${BASE}")"
E2E_OUT="$(go run ./scripts/affected-packages.go -mode=suites -base "${BASE}")"
fi
fi
# 2) Print what will run
echo "=== Affected unit packages ==="
if [ -n "${PKGS}" ]; then echo "${PKGS}"; else echo "(none)"; fi
echo
echo "=== Affected e2e tests ==="
if [ -n "${E2E_OUT}" ]; then echo "${E2E_OUT}"; else echo "(none)"; fi
echo
# 3) Unit tests via Makefile (inherits required build tags)
if [ "${PKGS}" = "./..." ]; then
echo "Running: make test (all)"
make test
elif [ -n "${PKGS}" ]; then
echo "Running: make test-packages for affected pkgs"
PACKAGES="$(echo "${PKGS}" | xargs)" make test-packages
else
echo "No affected unit packages"
fi
# 4) E2E tests via Makefile (filtered by regex)
PRE="$(echo "${E2E_OUT}" | awk -F: '$1=="preflight"{print $2}' | paste -sd'|' -)"
SB="$( echo "${E2E_OUT}" | awk -F: '$1=="support-bundle"{print $2}' | paste -sd'|' -)"
# Use direct go test with the same build tags as the Makefile to avoid RUN quoting issues locally
BUILD_TAGS='netgo containers_image_ostree_stub exclude_graphdriver_devicemapper exclude_graphdriver_btrfs containers_image_openpgp'
overall=0
if [ -n "${PRE}" ]; then
echo "Running preflight e2e: ${PRE}"
go test -tags "${BUILD_TAGS}" -installsuffix netgo -v -count=1 ./test/e2e/preflight -run "^(${PRE})$" || overall=1
fi
if [ -n "${SB}" ]; then
echo "Running support-bundle e2e: ${SB}"
go test -tags "${BUILD_TAGS}" -installsuffix netgo -v -count=1 ./test/e2e/support-bundle -run "^(${SB})$" || overall=1
fi
exit $overall
+181
View File
@@ -0,0 +1,181 @@
#!/bin/bash
# Comprehensive test for affected test detection
# Tests various code change scenarios to ensure correct suite detection
set -e
# Colors
GREEN='\033[0;32m'
RED='\033[0;31m'
BLUE='\033[0;34m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color
TESTS_PASSED=0
TESTS_FAILED=0
echo "========================================"
echo "Affected Test Detection Validation"
echo "========================================"
echo ""
# Helper function to run test
run_test() {
local test_name="$1"
local test_file="$2"
local expected_suites="$3"
echo -e "${BLUE}Test: $test_name${NC}"
echo "File: $test_file"
echo "Expected: $expected_suites"
# Get affected tests from explicit changed files (no git required); detector prints <suite>:<TestName>
local detector_output=$(go run ./scripts/affected-packages.go -mode=suites -changed-files "$test_file" 2>/dev/null)
# Derive suites from prefixes for comparison
local actual_suites=$(echo "$detector_output" | cut -d':' -f1 | grep -v '^$' | sort | uniq | tr '\n' ' ' | xargs)
# Compare results
if [ "$actual_suites" = "$expected_suites" ]; then
echo -e "${GREEN}✓ PASS${NC} - Got: $actual_suites"
if [ -n "$detector_output" ]; then
echo "Tests:" && echo "$detector_output" | sed 's/^/ - /'
fi
TESTS_PASSED=$((TESTS_PASSED + 1))
else
echo -e "${RED}✗ FAIL${NC} - Got: '$actual_suites', Expected: '$expected_suites'"
if [ -n "$detector_output" ]; then
echo "Tests:" && echo "$detector_output" | sed 's/^/ - /'
fi
TESTS_FAILED=$((TESTS_FAILED + 1))
fi
echo ""
}
# Test 1: Preflight-only package (should only trigger preflight)
run_test "Preflight-only package change" \
"pkg/preflight/run.go" \
"preflight"
# Test 2: Support-bundle-only package
run_test "Support-bundle-only package change" \
"pkg/supportbundle/supportbundle.go" \
"support-bundle"
# Test 3: Shared package - collect
run_test "Shared package (collect) change" \
"pkg/collect/run.go" \
"preflight support-bundle"
# Test 4: Shared package - analyze
run_test "Shared package (analyze) change" \
"pkg/analyze/analyzer.go" \
"preflight support-bundle"
# Test 5: Shared package - k8sutil
run_test "Shared package (k8sutil) change" \
"pkg/k8sutil/config.go" \
"preflight support-bundle"
# Test 6: Shared package - convert
run_test "Shared package (convert) change" \
"pkg/convert/output.go" \
"preflight support-bundle"
# Test 7: Shared package - redact (another shared one)
run_test "Shared package (redact) change" \
"pkg/redact/redact.go" \
"preflight support-bundle"
# Test 8: Preflight command (should only trigger preflight)
run_test "Preflight command change" \
"cmd/preflight/main.go" \
"preflight"
# Test 9: Support-bundle types (support-bundle only package)
run_test "Support-bundle types change" \
"pkg/supportbundle/types/types.go" \
"support-bundle"
# Test 10: Workflow file (should not trigger e2e)
echo -e "${BLUE}Test: Workflow file change (should trigger nothing)${NC}"
echo "File: .github/workflows/affected-tests.yml"
echo "Expected: (no suites)"
detector_output=$(go run ./scripts/affected-packages.go -mode=suites -changed-files ".github/workflows/affected-tests.yml" 2>/dev/null)
actual_suites=$(echo "$detector_output" | cut -d':' -f1 | grep -v '^$' | sort | uniq | tr '\n' ' ' | xargs)
if [ -z "$actual_suites" ]; then
echo -e "${GREEN}✓ PASS${NC} - No suites affected (as expected)"
TESTS_PASSED=$((TESTS_PASSED + 1))
else
echo -e "${RED}✗ FAIL${NC} - Got: '$actual_suites', Expected: (empty)"
TESTS_FAILED=$((TESTS_FAILED + 1))
fi
echo ""
# Test 11: go.mod change (should trigger all)
echo -e "${BLUE}Test: go.mod change (should trigger all suites)${NC}"
echo "File: go.mod"
echo "Expected: preflight support-bundle"
detector_output=$(go run ./scripts/affected-packages.go -mode=suites -changed-files "go.mod" 2>/dev/null)
actual_suites=$(echo "$detector_output" | cut -d':' -f1 | grep -v '^$' | sort | uniq | tr '\n' ' ' | xargs)
if [ "$actual_suites" = "preflight support-bundle" ]; then
echo -e "${GREEN}✓ PASS${NC} - Got: $actual_suites"
TESTS_PASSED=$((TESTS_PASSED + 1))
else
echo -e "${RED}✗ FAIL${NC} - Got: '$actual_suites', Expected: 'preflight support-bundle'"
TESTS_FAILED=$((TESTS_FAILED + 1))
fi
echo ""
# Test 12: Multiple files across different areas
echo -e "${BLUE}Test: Multiple file changes (support-bundle + shared)${NC}"
echo "Files: pkg/supportbundle/supportbundle.go + pkg/collect/run.go"
echo "Expected: preflight support-bundle"
detector_output=$(go run ./scripts/affected-packages.go -mode=suites -changed-files "pkg/supportbundle/supportbundle.go,pkg/collect/run.go" 2>/dev/null)
actual_suites=$(echo "$detector_output" | cut -d':' -f1 | grep -v '^$' | sort | uniq | tr '\n' ' ' | xargs)
if [ "$actual_suites" = "preflight support-bundle" ]; then
echo -e "${GREEN}✓ PASS${NC} - Got: $actual_suites"
TESTS_PASSED=$((TESTS_PASSED + 1))
else
echo -e "${RED}✗ FAIL${NC} - Got: '$actual_suites', Expected: 'preflight support-bundle'"
TESTS_FAILED=$((TESTS_FAILED + 1))
fi
echo ""
# Test 13: README change (should not trigger e2e)
echo -e "${BLUE}Test: Documentation change (should trigger nothing)${NC}"
echo "File: README.md"
echo "Expected: (no suites)"
detector_output=$(go run ./scripts/affected-packages.go -mode=suites -changed-files "README.md" 2>/dev/null)
actual_suites=$(echo "$detector_output" | cut -d':' -f1 | grep -v '^$' | sort | uniq | tr '\n' ' ' | xargs)
if [ -z "$actual_suites" ]; then
echo -e "${GREEN}✓ PASS${NC} - No suites affected (as expected)"
TESTS_PASSED=$((TESTS_PASSED + 1))
else
echo -e "${RED}✗ FAIL${NC} - Got: '$actual_suites', Expected: (empty)"
TESTS_FAILED=$((TESTS_FAILED + 1))
fi
echo ""
# Summary
echo "========================================"
echo -e "${GREEN}Tests Passed: $TESTS_PASSED${NC}"
echo -e "${RED}Tests Failed: $TESTS_FAILED${NC}"
echo "========================================"
if [ $TESTS_FAILED -eq 0 ]; then
echo -e "${GREEN}✓ All tests passed!${NC}"
exit 0
else
echo -e "${RED}✗ Some tests failed${NC}"
exit 1
fi