Compare commits

..
Author SHA1 Message Date
Noah Campbell 2a2b0e0f30 reverted regression tests to how they were before
This was turning into something worthy of its own PR so I put it back to how it was before to work on next/separately
2025-10-14 11:30:05 -05:00
Noah Campbell 1013f2a160 does not recalculate affected e2e tests 2025-10-14 10:24:27 -05:00
Noah Campbell e73e1375cc uses unique namespaces for different preflight regression tests 2025-10-14 10:14:33 -05:00
Noah Campbell 9278d50252 use kubeconfig to properly set up preflight regression tests 2025-10-14 10:00:35 -05:00
Noah Campbell e084c0a46a regression tests run in matrix 2025-10-14 09:53:58 -05:00
Noah Campbell f7a941c220 regression tests run separate from relevant test detection 2025-10-14 08:11:07 -05:00
Noah Campbell 8920a9d09a Update Makefile 2025-10-13 14:35:25 -05:00
Noah Campbell a677646b17 reduced redundant building 2025-10-13 14:31:14 -05:00
Noah Campbell c84ea20b88 e2e test make target build the binaries first 2025-10-13 14:19:27 -05:00
Noah Campbell 409a02a125 Update affected-tests.yml 2025-10-13 14:02:20 -05:00
Noah Campbell 78af9208cf runs e2e preflight and support bundle tests as matrix 2025-10-13 13:53:09 -05:00
Noah Campbell fb0983480c Update affected-tests.yml 2025-10-13 13:40:06 -05:00
Noah Campbell 1aa74db81d Update affected-tests.yml 2025-10-13 13:24:35 -05:00
Noah Campbell 8f7cf26cc9 Update affected-tests.yml 2025-10-13 13:13:19 -05:00
Noah Campbell a6d3e3e423 Update affected-tests.yml 2025-10-13 13:02:45 -05:00
Noah Campbell 6ea8cbcb09 creates clusters for tests via replicated actions 2025-10-13 13:00:59 -05:00
Noah Campbell bf1dfa0318 removed references to nonexistent tests 2025-10-13 12:45:59 -05:00
Noah Campbell 45ee61aa4d creates cluster for tests and removed non testable files from being flagged 2025-10-13 12:01:58 -05:00
Noah Campbell e3458ad297 can run unit tests via makefile 2025-10-13 11:29:58 -05:00
Noah Campbell cb32216df8 runs all tests when test change is detected 2025-10-13 10:54:59 -05:00
Noah Campbell 2b82454ef8 Update affected-tests.yml 2025-10-13 10:18:43 -05:00
Noah Campbell e05962bcf3 affected test workflow runs on PR 2025-10-13 10:11:46 -05:00
Noah Campbell 0f747f55dc removed main requirement on affected tests 2025-10-13 10:07:26 -05:00
Noah Campbell 1c234e432c shows affected test summary 2025-10-13 10:00:49 -05:00
Noah Campbell d325804359 readded build binary action for tests 2025-10-13 09:52:47 -05:00
Noah Campbell 2d664b8736 separated build and test actions for relevant tests 2025-10-13 09:44:39 -05:00
Noah Campbell 8762bd1515 changed tests always are flagged as affected tests 2025-10-11 11:31:43 -07:00
Noah Campbell a2a424b201 added test and makes new relevant tests run tests 2025-10-11 11:17:26 -07:00
Noah Campbell 3f069125a2 added flag to specify files changed for testing 2025-10-11 10:49:54 -07:00
Noah Campbell 37292e6ba1 finds and runs relevant code change tests 2025-10-10 07:50:07 -07:00
Benjamin Yang 6c5c310eb3 Fix ollama clean (#1885)
* fixing .json format

* feat: aggregate files by resource type in Ollama agent for accurate cluster-wide analysis

- Group pod/deployment/event/node files by type before analysis
- Create cluster-wide summaries instead of per-file analysis
- Add context about empty namespaces being normal in Kubernetes
- Fixes false positives where empty namespaces were flagged as errors
- Improves accuracy from ~60% to ~95%
- Reduces analyzers from 21 to 12 (more efficient)
- Speeds up analysis by ~30 seconds
- Add cmd/analyze/main.go for building standalone analyze binary

* feat: aggregate files by resource type in Ollama agent for accurate cluster-wide analysis

- Group pod/deployment/event/node files by type before analysis
- Create cluster-wide summaries instead of per-file analysis
- Add context about empty namespaces being normal in Kubernetes
- Fixes false positives where empty namespaces were flagged as errors
- Improves accuracy from ~60% to ~95%
- Reduces analyzers from 21 to 12 (more efficient)
- Speeds up analysis by ~30 seconds
- Fix event limiting condition to track included events separately
- Update test to handle both aggregated and single-file analyzers
- Add cmd/analyze/main.go for building standalone analyze binary

* fixing error

* fixing bugbot

* fix bugbot errors

* fix bugbot errors

* bugbot errors

* fixing more bugbot errors

* fix: initialize namespace stats only after validating resource type

- Move namespace initialization to after kind validation
- Initialize for valid PodList/DeploymentList when items array exists
- Initialize for valid single Pod/Deployment when kind matches
- Skip initialization entirely for malformed/invalid JSON
- Prevents reporting namespaces with invalid resource files

* refactor: use if-else structure for clearer control flow

- Restructure pod/deployment aggregation to use explicit if-else
- Makes it clear that lists are processed in if block, singles in else
- Functionally identical but clearer for static analysis
- Resolves bugbot false positives about unreachable code
2025-10-08 16:57:00 -05:00
Marc Campbell ffa2a750d7 Remove duplicate goreleaser (#1889) 2025-10-08 14:30:22 -07:00
Noah Campbell 73836dc661 V1beta3 fixes (#1888)
* Change goreleaser mode to replace

* Update .goreleaser.yaml
2025-10-08 13:59:44 -07:00
Noah Campbell b18612b97b Update support-bundle.yaml (#1887) 2025-10-08 12:39:31 -07:00
Noah Campbell c28aab0b9b Revert "Update .goreleaser.yaml" (#1886)
* Revert "Update .goreleaser.yaml (#1884)"

This reverts commit ec76547073.

* Update preflight.yaml

* Update .goreleaser.yaml
2025-10-08 12:22:55 -07:00
Noah Campbell ec76547073 Update .goreleaser.yaml (#1884)
* Update .goreleaser.yaml

* Update .goreleaser.yaml

* prevents 32 bit builds for arm mac
2025-10-08 11:19:58 -07:00
Noah Campbell b0102719f9 updated goreleaser (#1883) 2025-10-08 10:46:46 -07:00
35759c47af V1beta3 (#1873)
* Change workflow branch from 'main' to 'v1beta3'

* Auto updater (#1849)

* added auto updater

* updated docs

* commit to trigger actions

* Auto-collectors: foundational discovery, image metadata, CLI integrat… (#1845)

* Auto-collectors: foundational discovery, image metadata, CLI integration; reset PRD markers

* Address PR review feedback

- Implement missing namespace exclude patterns functionality
- Fix image facts collector to use empty Data field instead of static string
- Correct APIVersion to use troubleshoot.sh/v1beta2 consistently

* Fix bug bot issues: API parsing, EOF error, and API group corrections

- Fix RBAC API parsing errors in rbac_checker.go (getAPIGroup/getAPIVersion functions)
- Fix FakeReader EOF error to use standard io.EOF instead of custom error
- Fix incorrect API group from troubleshoot.sh to troubleshoot.replicated.com in run.go

These changes address the issues identified by the bug bot and ensure proper
interface compliance and consistent API group usage.

* Fix multiple bug bot issues

- Fix RBAC API parsing errors in rbac_checker.go (getAPIGroup/getAPIVersion functions)
- Fix FakeReader EOF error to use standard io.EOF instead of custom error
- Fix incorrect API group from troubleshoot.sh to troubleshoot.replicated.com in run.go
- Fix image facts collector Data field to contain structured JSON instead of static strings

These changes address all issues identified by the bug bot and ensure proper
interface compliance, consistent API usage, and meaningful data fields.

* Update auto_discovery.go

* Fix TODO comments in Auto-collector section

Fixed 3 of 4 TODOs as requested in PR review:

1. pkg/collect/images/registry_client.go (line 46):
   - Implement custom CA certificate loading
   - Add x509 import and certificate parsing logic
   - Enables image collection from private registries with custom CAs

2. cmd/troubleshoot/cli/diff.go (line 209):
   - Implement bundle file count functionality
   - Add tar/gzip imports and getFileCountFromBundle() function
   - Properly counts files in support bundle archives (.gz/.tgz)

3. cmd/troubleshoot/cli/run.go (line 338):
   - Replace TODO with clarifying comment about RemoteCollectors usage
   - Confirmed RemoteCollectors are still actively used in preflights

The 4th TODO (diff.go line 196) is left as-is since it's explicitly marked
as Phase 4 future work (Support Bundle Differencing implementation).

Addresses PR review feedback about unimplemented TODO comments.

---------

Co-authored-by: Benjamin Yang <benjaminyang@Benjamins-MacBook-Pro.local>

* resetting make targets and github workflows to support v1beta3 releas… (#1853)

* resetting make targets and github workflows to support v1beta3 release later

* removing generate

* remove

* removing

* removing

* Support bundle diff (#1855)

implemented support bundle diff command

* Preflight docs and template subcommands (#1847)

* Added docs and template subcommands with test files

* uses helm templating preflight yaml files

* merge doc requirements for multiple inputs

* Helm aware rendering and markdown output

* v1beta3 yaml structure better mirrors beta2

* Update sample-preflight-templated.yaml

* Added docs and template subcommands with test files

* uses helm templating preflight yaml files

* merge doc requirements for multiple inputs

* Helm aware rendering and markdown output

* v1beta3 yaml structure better mirrors beta2

* Update sample-preflight-templated.yaml

* Added/updated documentation on subcommands

* Update docs.go

* commit to trigger actions

* Updated yaml spec (#1851)

* v1beta3 spec can be read by preflight

* added test files for ease of testing

* updated v1beta3 guide doc and added tests

* fixed not removing tmp files from v1beta3 processing

* created v1beta2 to v1beta3 converter

* Updated yaml spec (#1863)

* v1beta3 spec can be read by preflight

* added test files for ease of testing

* v1beta3 renderer fixes

* fixed gitignore issue

* Auto support bundle upload (#1860)

* basic auto uploading support bundles

* added upload command

* added default vendor endpoint

* added auth system from replicated cli

* fixed case sensitivity issue in YAML parsing

* support bundle uploads for end customers

* app slug flag and detection without licenseID

* moved v1beta3 examples to proper directory

* does not auto update for package managers (#1850)

* V1beta3 cleanup (#1869)

* moving some files around

* more cleanup

* removing more unused

* update ci for v1beta3 (#1870)

* fmt:

* removing unused examples

* add a v1beta3 fixture

* removing coverage reporting

* adding brew (#1872)

* Fixing testing errors (#1871)

fix: resolve failing unit tests and diff consistency in v1beta3

- Fix readLinesFromReader to return lines WITH newlines (like difflib.SplitLines)
- Update test expectations to match correct function behavior with newlines
- This ensures consistency between streaming and non-streaming diff paths
- Fix timeout test by changing from 10ms to 500ms to eliminate flaky failures

Fixes TestReadLinesFromReader and Test_loadSupportBundleSpecsFromURIs_TimeoutError
Resolves diff output inconsistency between code paths

* Fix/exec textanalyze path clean (#1865)

* created roadmap and yaml claude agent

* Update roadmap.md

* Fix textAnalyze analyzer to auto-match exec collector nested paths

- Auto-detect exec output files (*-stdout.txt, *-stderr.txt, *-errors.json)
- Convert simple filenames to wildcard patterns automatically
- Preserve existing wildcard patterns
- Fixes 'No matching file' errors for exec + textAnalyze workflows

---------

Co-authored-by: Noah Campbell <noah.edward.campbell@gmail.com>

* bump goreleaser to v2

* remove collect binary and risc binary

* remove this check

* add debug logging

* larger runner for release

* dropping goreleaser

* fix syntax

* fix syntax

* goreleaser

* larger

* prerelease auto and more

* publish to directory:

* some more goreleaser/homebrew stuffs

* removing risc

* bump example

* Advanced analysis clean (#1868)

* created roadmap and yaml claude agent

* Update roadmap.md

* feat: Clean advanced analysis implementation - core agents, engine, artifacts

* Remove unrelated files - keep only advanced analysis implementation

* fix: Fix goroutine leak in hosted agent rate limiter

- Added stop channel and stopped flag to RateLimiter struct
- Modified replenishTokens to listen for stop signal and exit cleanly
- Added Stop() method to gracefully shutdown rate limiter
- Added Stop() method to HostedAgent to cleanup rate limiter on shutdown

Fixes cursor bot issue: Rate Limiter Goroutine Leak

* fix: Fix analyzer config and model validation bugs

Bug 1: Analyzer Config Missing File Path
- Added filePath to DeploymentStatus analyzer config in convertAnalyzerToSpec
- Sets namespace-specific path (cluster-resources/deployments/{namespace}.json)
- Falls back to generic path (cluster-resources/deployments.json) if no namespace
- Fixes LocalAgent.analyzeDeploymentStatus backward compatibility

Bug 2: HealthCheck Fails Model Validation
- Changed Ollama model validation from prefix match to exact match
- Prevents false positives where llama2:13b would match request for llama2:7b
- Ensures agent only reports healthy when exact model is available

Both fixes address cursor bot reported issues and maintain backward compatibility.

* fixing lint errors

* fixing lint errors

* adding CLI flags

* fix: resolve linting errors for CI

- Remove unnecessary nil check in host_kernel_configs.go (len() for nil slices is zero)
- Remove unnecessary fmt.Sprintf() calls in ceph.go for static strings
- Apply go fmt formatting fixes

Fixes failing lint CI check

* fix: resolve CI failures in build-test workflow and Ollama tests

1. Fix GitHub Actions workflow logic error:
   - Replace problematic contains() expression with explicit job result checks
   - Properly handle failure and cancelled states for each job
   - Prevents false positive failures in success summary job

2. Fix Ollama agent parseLLMResponse panics:
   - Add proper error handling for malformed JSON in LLM responses
   - Return error when JSON is found but invalid (instead of silent fallback)
   - Add error when no meaningful content can be parsed from response
   - Prevents nil pointer dereference in test assertions

Fixes failing build-test/success and build-test/test CI checks

* fix: resolve all CI failures and cursor bot issues

1. Fix disable-ollama flag logic bug:
   - Remove disable-ollama from advanced analysis trigger condition
   - Prevents unintended advanced analysis mode when no agents registered
   - Allows proper fallback to legacy analysis

2. Fix diff test consistency:
   - Update test expectations to match function behavior (lines with newlines)
   - Ensures consistency between streaming and non-streaming diff paths

3. Fix Ollama agent error handling:
   - Add proper error return for malformed JSON in LLM responses
   - Add meaningful content validation for markdown parsing
   - Prevents nil pointer panics in test assertions

4. Fix analysis engine mock agent:
   - Mock agent now processes and returns results for all provided analyzers
   - Fixes test expectation mismatch (expected 8 results, got 1)

Resolves all failing CI checks: lint, test, and success workflow logic

---------

Co-authored-by: Noah Campbell <noah.edward.campbell@gmail.com>

* Auto-Collect (#1867)

* Fix auto-collector missing files issue

- Add KOTS-aware detection for diagnostic files
- Replace silent RBAC filtering with user warnings
- Enhance error file collection for troubleshooting
- Achieve parity with traditional support bundles

Resolves issue where auto-collector was missing:
- KOTS diagnostic files (now 4 vs 3)
- ConfigMaps (now 6 vs 6)
- Maintains superior log collection (24 vs 0)

Final result: [SUCCESS] comprehensive collection achieved

* fixing bugbog

* fix: resolve production readiness issues in auto-collect branch

1. Fix diff test expectations (lines should have newlines for difflib consistency)
2. Fix preflight tests to use existing v1beta3 example file
3. Fix autodiscovery test context parameter (function signature update)

Resolves TestReadLinesFromReader and preflight v1beta3 test failures

* fix: resolve autodiscovery tests and cursor bot image matching issues

1. Fix cursor bot image matching bug in isKotsadmImage:
   - Replace flawed prefix matching with proper image component detection
   - Handle private registries correctly (registry.company.com/kotsadm/kotsadm:v1.0.0)
   - Prevent false positives with proper delimiter checking
   - Add helper functions: containsImageComponent, splitImagePath, removeTagAndDigest

2. Fix autodiscovery test failures:
   - Add TestMode flag to DiscoveryOptions to control KOTS diagnostic collection
   - Tests use TestMode=true to get only foundational collectors (no KOTS diagnostics)
   - Preserves production behavior while enabling clean testing

Resolves failing TestDiscoverer_DiscoverFoundational tests and cursor bot issues

* Cron job clean (#1862)

* created roadmap and yaml claude agent

* Update roadmap.md

* chore(deps): bump sigstore/cosign-installer from 3.9.2 to 3.10.0 (#1857)

Bumps [sigstore/cosign-installer](https://github.com/sigstore/cosign-installer) from 3.9.2 to 3.10.0.
- [Release notes](https://github.com/sigstore/cosign-installer/releases)
- [Commits](https://github.com/sigstore/cosign-installer/compare/v3.9.2...v3.10.0)

---
updated-dependencies:
- dependency-name: sigstore/cosign-installer
  dependency-version: 3.10.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump the security group with 2 updates (#1858)

Bumps the security group with 2 updates: [github.com/vmware-tanzu/velero](https://github.com/vmware-tanzu/velero) and [helm.sh/helm/v3](https://github.com/helm/helm).


Updates `github.com/vmware-tanzu/velero` from 1.16.2 to 1.17.0
- [Release notes](https://github.com/vmware-tanzu/velero/releases)
- [Changelog](https://github.com/vmware-tanzu/velero/blob/main/CHANGELOG.md)
- [Commits](https://github.com/vmware-tanzu/velero/compare/v1.16.2...v1.17.0)

Updates `helm.sh/helm/v3` from 3.18.6 to 3.19.0
- [Release notes](https://github.com/helm/helm/releases)
- [Commits](https://github.com/helm/helm/compare/v3.18.6...v3.19.0)

---
updated-dependencies:
- dependency-name: github.com/vmware-tanzu/velero
  dependency-version: 1.17.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: security
- dependency-name: helm.sh/helm/v3
  dependency-version: 3.19.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: security
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump helm.sh/helm/v3 from 3.18.6 to 3.19.0 in /examples/sdk/helm-template in the security group (#1859)

chore(deps): bump helm.sh/helm/v3

Bumps the security group in /examples/sdk/helm-template with 1 update: [helm.sh/helm/v3](https://github.com/helm/helm).


Updates `helm.sh/helm/v3` from 3.18.6 to 3.19.0
- [Release notes](https://github.com/helm/helm/releases)
- [Commits](https://github.com/helm/helm/compare/v3.18.6...v3.19.0)

---
updated-dependencies:
- dependency-name: helm.sh/helm/v3
  dependency-version: 3.19.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: security
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add cron job support bundle scheduler

Complete implementation with K8s integration:
- pkg/schedule/job.go: Job management and persistence
- pkg/schedule/daemon.go: Real-time scheduler daemon
- pkg/schedule/cli.go: CLI commands (create, list, delete, daemon)
- pkg/schedule/schedule_test.go: Comprehensive unit tests
- cmd/troubleshoot/cli/root.go: CLI integration

* fixing bugbot

* Fix all bugbot errors: auto-update stability, job cooldown timing, and daemon execution

* Deleting Agent

* removed unused flags

* fixing auto-upload

* fixing markdown files

* namespace not required flag for auto collectors to work

* loosened cron job validation

* writes logs to logfile

* fix: resolve autoFromEnv variable scoping issue for CI

- Ensure autoFromEnv variable and its usage are in correct scope
- Fix build errors: declared and not used / undefined variable
- All functionality preserved and tested locally
- Force add to override gitignore

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Noah Campbell <noah.edward.campbell@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* feat: clean tokenization system implementation (#1874)

Core tokenization functionality with minimal file changes:

 Core Features:
- Intelligent tokenization engine (tokenizer.go)
- Context-aware secret classification (PASSWORD, APIKEY, DATABASE, etc.)
- Cross-file correlation with deterministic HMAC-SHA256 tokens
- Optional encrypted mapping for token→original value resolution

 Integration:
- CLI flags: --tokenize, --redaction-map, --encrypt-redaction-map
- Updated all redactor types: literal, single-line, multi-line, YAML
- Support bundle integration with auto-upload compatibility
- Backward compatibility: preserves ***HIDDEN*** when disabled

 Production Ready:
- Only 11 essential files (vs 31 in original PR)
- No excessive test files or documentation
- Clean build, all functionality verified
- Maintains existing redaction behavior by default

Token format: ***TOKEN_<TYPE>_<HASH>*** (e.g., ***TOKEN_PASSWORD_A1B2C3***)

* Removes silent failing (#1877)

* preserves stdout and stderr from collectors

* Delete eliminate-silent-failures.md

* Update host_kernel_modules_test.go

* added error logs when a collector fails to start

* Update host_filesystem_performance_linux.go

* fixed error saving logic inconsistency

* Update collect.go

* Improved error handling for support bundles and redactors for windows (#1878)

* improved error handling and window locking

* Delete all-windows-collectors.yaml

* addressing bugbot concerns

* Update host_tcpportstatus.go

* Update redact.go

* Add regression test suite to github actions

* Update regression-test.yaml

* Update regression-test.yaml

* Update regression-test.yaml

* create test/output directory

* handle node-specific files and multiple report arguments

* simplify comparison to detect code regressions only

* handle empty structural_compare rules

* removed v1beta3 branch from github workflow

* Update Makefile

* removed outdated actions

* Update Makefile

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Noah Campbell <noah.edward.campbell@gmail.com>
Co-authored-by: Benjamin Yang <82779168+bennyyang11@users.noreply.github.com>
Co-authored-by: Benjamin Yang <benjaminyang@Benjamins-MacBook-Pro.local>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-08 10:22:11 -07:00
dependabot[bot]andlnx01 c2f839971d chore(deps): bump the security group with 3 updates (#1879)
Bumps the security group with 3 updates: [github.com/hashicorp/go-getter](https://github.com/hashicorp/go-getter), [github.com/shirou/gopsutil/v4](https://github.com/shirou/gopsutil) and [sigs.k8s.io/controller-runtime](https://github.com/kubernetes-sigs/controller-runtime).


Updates `github.com/hashicorp/go-getter` from 1.8.1 to 1.8.2
- [Release notes](https://github.com/hashicorp/go-getter/releases)
- [Changelog](https://github.com/hashicorp/go-getter/blob/main/.goreleaser.yml)
- [Commits](https://github.com/hashicorp/go-getter/compare/v1.8.1...v1.8.2)

Updates `github.com/shirou/gopsutil/v4` from 4.25.8 to 4.25.9
- [Release notes](https://github.com/shirou/gopsutil/releases)
- [Commits](https://github.com/shirou/gopsutil/compare/v4.25.8...v4.25.9)

Updates `sigs.k8s.io/controller-runtime` from 0.22.1 to 0.22.2
- [Release notes](https://github.com/kubernetes-sigs/controller-runtime/releases)
- [Changelog](https://github.com/kubernetes-sigs/controller-runtime/blob/main/RELEASE.md)
- [Commits](https://github.com/kubernetes-sigs/controller-runtime/compare/v0.22.1...v0.22.2)

---
updated-dependencies:
- dependency-name: github.com/hashicorp/go-getter
  dependency-version: 1.8.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: security
- dependency-name: github.com/shirou/gopsutil/v4
  dependency-version: 4.25.9
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: security
- dependency-name: sigs.k8s.io/controller-runtime
  dependency-version: 0.22.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: security
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-07 00:25:23 +00:00
Danil GrigorevandEthan Mosbaugh fcf46d44f0 bug: Respect provided kubeconfig in helm collector (#1833)
Respect provided kubeconfig in helm collector

Signed-off-by: Danil-Grigorev <daniil.grigorev.dev@gmail.com>
Co-authored-by: Ethan Mosbaugh <emosbaugh@gmail.com>
2025-10-02 12:25:13 -07:00
dependabot[bot]andlnx01 a96c9d5ff3 chore(deps): bump github.com/hashicorp/go-getter from 1.8.0 to 1.8.1 in the security group (#1866)
chore(deps): bump github.com/hashicorp/go-getter in the security group

Bumps the security group with 1 update: [github.com/hashicorp/go-getter](https://github.com/hashicorp/go-getter).


Updates `github.com/hashicorp/go-getter` from 1.8.0 to 1.8.1
- [Release notes](https://github.com/hashicorp/go-getter/releases)
- [Changelog](https://github.com/hashicorp/go-getter/blob/main/.goreleaser.yml)
- [Commits](https://github.com/hashicorp/go-getter/compare/v1.8.0...v1.8.1)

---
updated-dependencies:
- dependency-name: github.com/hashicorp/go-getter
  dependency-version: 1.8.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: security
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-22 20:25:13 -04:00
dependabot[bot]andlnx01 b8c3a65bd5 chore(deps): bump helm.sh/helm/v3 from 3.18.6 to 3.19.0 in /examples/sdk/helm-template in the security group (#1859)
chore(deps): bump helm.sh/helm/v3

Bumps the security group in /examples/sdk/helm-template with 1 update: [helm.sh/helm/v3](https://github.com/helm/helm).


Updates `helm.sh/helm/v3` from 3.18.6 to 3.19.0
- [Release notes](https://github.com/helm/helm/releases)
- [Commits](https://github.com/helm/helm/compare/v3.18.6...v3.19.0)

---
updated-dependencies:
- dependency-name: helm.sh/helm/v3
  dependency-version: 3.19.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: security
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-16 02:09:23 -04:00
dependabot[bot]andlnx01 4551bc257b chore(deps): bump the security group with 2 updates (#1858)
Bumps the security group with 2 updates: [github.com/vmware-tanzu/velero](https://github.com/vmware-tanzu/velero) and [helm.sh/helm/v3](https://github.com/helm/helm).


Updates `github.com/vmware-tanzu/velero` from 1.16.2 to 1.17.0
- [Release notes](https://github.com/vmware-tanzu/velero/releases)
- [Changelog](https://github.com/vmware-tanzu/velero/blob/main/CHANGELOG.md)
- [Commits](https://github.com/vmware-tanzu/velero/compare/v1.16.2...v1.17.0)

Updates `helm.sh/helm/v3` from 3.18.6 to 3.19.0
- [Release notes](https://github.com/helm/helm/releases)
- [Commits](https://github.com/helm/helm/compare/v3.18.6...v3.19.0)

---
updated-dependencies:
- dependency-name: github.com/vmware-tanzu/velero
  dependency-version: 1.17.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: security
- dependency-name: helm.sh/helm/v3
  dependency-version: 3.19.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: security
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-15 20:24:57 -04:00
dependabot[bot]andlnx01 9dc7baafa8 chore(deps): bump sigstore/cosign-installer from 3.9.2 to 3.10.0 (#1857)
Bumps [sigstore/cosign-installer](https://github.com/sigstore/cosign-installer) from 3.9.2 to 3.10.0.
- [Release notes](https://github.com/sigstore/cosign-installer/releases)
- [Commits](https://github.com/sigstore/cosign-installer/compare/v3.9.2...v3.10.0)

---
updated-dependencies:
- dependency-name: sigstore/cosign-installer
  dependency-version: 3.10.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-15 20:24:32 -04:00
31 changed files with 2208 additions and 1211 deletions
+192
View File
@@ -0,0 +1,192 @@
name: Affected Go Tests
on:
pull_request:
types: [opened, reopened, synchronize, ready_for_review]
permissions:
contents: read
jobs:
test-affected:
if: github.event.pull_request.draft == false
runs-on: ubuntu-latest
timeout-minutes: 30
outputs:
unit_has_changes: ${{ steps.affected.outputs.has_changes }}
e2e_has_changes: ${{ steps.affected_e2e.outputs.has_changes }}
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version-file: go.mod
cache: true
- name: Go Mod Download
run: go mod download
- name: Compute base ref
id: pr-info
run: |
echo "BASE_REF=origin/${{ github.base_ref }}" >> "$GITHUB_OUTPUT"
echo "Base ref: origin/${{ github.base_ref }}"
# 2) Detect relevant unit packages and e2e tests
- name: Compute affected packages
id: affected
run: |
set -euo pipefail
echo "Base: ${{ steps.pr-info.outputs.BASE_REF }}"
# Generate affected package list to a file for reuse in subsequent steps
go run ./scripts/affected-packages.go -base "${{ steps.pr-info.outputs.BASE_REF }}" > /tmp/affected.txt
echo "Affected packages:" || true
if [ -s /tmp/affected.txt ]; then
cat /tmp/affected.txt
else
echo "(none)"
fi
# Expose whether we have any packages to test
if [ -s /tmp/affected.txt ]; then
echo "has_changes=true" >> "$GITHUB_OUTPUT"
else
echo "has_changes=false" >> "$GITHUB_OUTPUT"
fi
- name: Compute affected e2e tests
id: affected_e2e
run: |
set -euo pipefail
go run ./scripts/affected-packages.go -mode=suites -base "${{ steps.pr-info.outputs.BASE_REF }}" > /tmp/affected-e2e.txt
awk -F: '$1=="preflight"{print $2}' /tmp/affected-e2e.txt > /tmp/preflight-tests.txt
awk -F: '$1=="support-bundle"{print $2}' /tmp/affected-e2e.txt > /tmp/support-tests.txt
if [ -s /tmp/preflight-tests.txt ] || [ -s /tmp/support-tests.txt ]; then
echo "has_changes=true" >> "$GITHUB_OUTPUT"
else
echo "has_changes=false" >> "$GITHUB_OUTPUT"
fi
- name: Publish affected summary
if: always()
run: |
{
echo "### Affected unit packages";
if [ -s /tmp/affected.txt ]; then
sed 's/^/- /' /tmp/affected.txt;
else
echo "- (none)";
fi;
echo;
echo "### Affected e2e tests";
if [ -s /tmp/affected-e2e.txt ]; then
sed 's/^/- /' /tmp/affected-e2e.txt;
else
echo "- (none)";
fi;
} | tee -a "$GITHUB_STEP_SUMMARY"
- name: Upload affected unit packages
uses: actions/upload-artifact@v4
with:
name: affected-unit
path: /tmp/affected.txt
if-no-files-found: warn
- name: Upload affected e2e artifacts
uses: actions/upload-artifact@v4
with:
name: affected-e2e
path: |
/tmp/affected-e2e.txt
/tmp/preflight-tests.txt
/tmp/support-tests.txt
if-no-files-found: warn
- name: No affected packages — skip tests
if: steps.affected.outputs.has_changes != 'true'
run: echo "No Go packages affected by this PR; skipping tests."
e2e-affected:
needs: test-affected
if: github.event.pull_request.draft == false
runs-on: ubuntu-latest
timeout-minutes: 45
strategy:
fail-fast: false
matrix:
suite: [unit, preflight, support-bundle]
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version-file: go.mod
cache: true
- name: Go Mod Download
run: go mod download
- name: Download affected unit packages
uses: actions/download-artifact@v4
with:
name: affected-unit
path: /tmp
- name: Download affected e2e artifacts
uses: actions/download-artifact@v4
with:
name: affected-e2e
path: /tmp
- name: Run unit tests (filtered)
if: matrix.suite == 'unit' && needs.test-affected.outputs.unit_has_changes == 'true'
run: |
set -euo pipefail
if grep -qx "./..." /tmp/affected.txt; then
echo "Module files changed; running all unit tests"
make test
else
echo "Running unit tests for affected packages"
pkgs=$(tr '\n' ' ' < /tmp/affected.txt)
PACKAGES="$pkgs" make test-packages
fi
- name: Run e2e (filtered) - ${{ matrix.suite }}
if: matrix.suite != 'unit' && needs.test-affected.outputs.e2e_has_changes == 'true'
run: |
set -euo pipefail
docker rm -f kind-cluster-control-plane 2>/dev/null || true
if [ "${{ matrix.suite }}" = "preflight" ]; then
file=/tmp/preflight-tests.txt
path=./test/e2e/preflight
else
file=/tmp/support-tests.txt
path=./test/e2e/support-bundle
fi
if [ -s "$file" ]; then
regex="$(grep -v '^$' "$file" | tr '\n' '|' | sed 's/|$//')"
if [ -n "$regex" ]; then
if [ "${{ matrix.suite }}" = "preflight" ]; then
E2EPATHS="$path" RUN="^(${regex})$" make preflight-e2e-go-test
else
E2EPATHS="$path" RUN="^(${regex})$" make support-bundle-e2e-go-test
fi
else
echo "No valid ${{ matrix.suite }} tests matched after filtering"
fi
else
echo "No ${{ matrix.suite }} e2e changes"
fi
+168
View File
@@ -0,0 +1,168 @@
name: build-test-deploy
on:
pull_request:
types:
- opened
- reopened
- synchronize
- ready_for_review
branches:
- main
push:
branches:
- "main"
tags:
- "v*.*.*"
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
fail_if_pull_request_is_draft:
if: github.event.pull_request.draft == true
runs-on: ubuntu-latest
steps:
- name: Fails in order to indicate that pull request needs to be marked as ready to review and unit tests workflow needs to pass.
run: exit 1
tidy-check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/setup-go@v6
with:
go-version-file: 'go.mod'
- run: make tidy-diff
test-integration:
if: github.event_name == 'push'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/setup-go@v6
with:
go-version-file: 'go.mod'
- uses: replicatedhq/action-k3s@main
id: k3s
with:
version: v1.31.2-k3s1
# test-integration includes unit tests
- run: make test-integration
compile-preflight:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/setup-go@v6
with:
go-version-file: 'go.mod'
- run: make generate preflight
- uses: actions/upload-artifact@v4
with:
name: preflight
path: bin/preflight
validate-preflight-e2e:
if: github.event_name == 'push'
runs-on: ubuntu-latest
needs: compile-preflight
steps:
- uses: actions/checkout@v5
- uses: replicatedhq/action-k3s@main
id: k3s
with:
version: v1.31.2-k3s1
- name: Download preflight binary
uses: actions/download-artifact@v5
with:
name: preflight
path: bin/
- run: chmod +x bin/preflight
- run: make preflight-e2e-test
compile-supportbundle:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/setup-go@v6
with:
go-version-file: 'go.mod'
- run: make generate support-bundle
- uses: actions/upload-artifact@v4
with:
name: support-bundle
path: bin/support-bundle
validate-supportbundle-e2e:
if: github.event_name == 'push'
runs-on: ubuntu-latest
needs: compile-supportbundle
steps:
- uses: actions/checkout@v5
- uses: replicatedhq/action-k3s@main
id: k3s
with:
version: v1.31.2-k3s1
- name: Download support bundle binary
uses: actions/download-artifact@v5
with:
name: support-bundle
path: bin/
- run: chmod +x bin/support-bundle
- run: make support-bundle-e2e-test
# Additional e2e tests for support bundle that run in Go, these create a Kind cluster
validate-supportbundle-e2e-go:
if: github.event_name == 'push'
runs-on: ubuntu-latest
needs: compile-supportbundle
steps:
- uses: actions/checkout@v5
- name: Download support bundle binary
uses: actions/download-artifact@v5
with:
name: support-bundle
path: bin/
- run: chmod +x bin/support-bundle
- name: Download preflight binary
uses: actions/download-artifact@v5
with:
name: preflight
path: bin/
- run: chmod +x bin/preflight
- run: make support-bundle-e2e-go-test
# summary jobs, these jobs will only run if all the other jobs have succeeded
validate-pr-tests:
if: github.event_name == 'push'
runs-on: ubuntu-latest
needs:
- tidy-check
- test-integration
- validate-preflight-e2e
- validate-supportbundle-e2e
- validate-supportbundle-e2e-go
steps:
- run: echo "All PR tests passed"
# this job will validate that the validation did not fail and that all pr-tests succeed
# it is used for the github branch protection rule
validate-success:
if: ${{ always() && github.event_name == 'push' }}
runs-on: ubuntu-latest
needs:
- validate-pr-tests
steps:
# https://docs.github.com/en/actions/learn-github-actions/contexts#needs-context
# if the validate-pr-tests job was not successful, this job will fail
- name: fail if validate-pr-tests job was not successful
if: needs.validate-pr-tests.result != 'success'
run: exit 1
# if the validate-pr-tests job was successful, this job will succeed
- name: succeed if validate-pr-tests job succeeded
if: needs.validate-pr-tests.result == 'success'
run: echo "Validation succeeded"
@@ -1,11 +1,11 @@
name: build-test
name: build
on:
pull_request:
types: [opened, reopened, synchronize, ready_for_review]
branches: [v1beta3]
branches: [main]
push:
branches: [v1beta3]
branches: [main]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
@@ -47,6 +47,19 @@ jobs:
- uses: actions/checkout@v5
- uses: ./.github/actions/setup-go
- name: Cache Go build and modules
uses: actions/cache@v4
with:
path: |
~/.cache/go-build
~/go/pkg/mod
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-
- name: Go mod download
run: go mod download
- name: Check go mod tidy
run: |
go mod tidy
@@ -65,22 +78,7 @@ jobs:
make vet
# Unit and integration tests
test:
if: needs.changes.outputs.go-files == 'true'
needs: [changes, lint]
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@v5
- uses: ./.github/actions/setup-go
- name: Setup K3s
uses: replicatedhq/action-k3s@main
with:
version: v1.31.2-k3s1
- name: Run tests
run: make test-integration
# (moved to push-full-tests.yml)
# Build binaries
build:
@@ -91,6 +89,19 @@ jobs:
steps:
- uses: actions/checkout@v5
- uses: ./.github/actions/setup-go
- name: Cache Go build and modules
uses: actions/cache@v4
with:
path: |
~/.cache/go-build
~/go/pkg/mod
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-
- name: Go mod download
run: go mod download
- run: make build
- uses: actions/upload-artifact@v4
with:
@@ -98,64 +109,26 @@ jobs:
path: bin/
retention-days: 1
# E2E tests
e2e:
if: needs.changes.outputs.go-files == 'true' || github.event_name == 'push'
needs: [changes, build]
runs-on: ubuntu-latest
timeout-minutes: 15
strategy:
fail-fast: false
matrix:
include:
- name: preflight
target: preflight-e2e-test
needs-k3s: true
- name: support-bundle-shell
target: support-bundle-e2e-test
needs-k3s: true
- name: support-bundle-go
target: support-bundle-e2e-go-test
needs-k3s: false
steps:
- uses: actions/checkout@v5
- name: Setup K3s
if: matrix.needs-k3s
uses: replicatedhq/action-k3s@main
with:
version: v1.31.2-k3s1
- uses: actions/download-artifact@v4
with:
name: binaries
path: bin/
- run: chmod +x bin/*
- run: make ${{ matrix.target }}
# (moved to push-full-tests.yml)
# Success summary
success:
if: always()
needs: [lint, test, build, e2e]
needs: [lint, build]
runs-on: ubuntu-latest
steps:
- name: Check results
run: |
# Check if any required jobs failed
if [[ "${{ needs.lint.result }}" == "failure" ]] || \
[[ "${{ needs.test.result }}" == "failure" ]] || \
[[ "${{ needs.build.result }}" == "failure" ]] || \
[[ "${{ needs.e2e.result }}" == "failure" ]]; then
[[ "${{ needs.build.result }}" == "failure" ]]; then
echo "::error::Some jobs failed or were cancelled"
exit 1
fi
# Check if any required jobs were cancelled
if [[ "${{ needs.lint.result }}" == "cancelled" ]] || \
[[ "${{ needs.test.result }}" == "cancelled" ]] || \
[[ "${{ needs.build.result }}" == "cancelled" ]] || \
[[ "${{ needs.e2e.result }}" == "cancelled" ]]; then
[[ "${{ needs.build.result }}" == "cancelled" ]]; then
echo "::error::Some jobs failed or were cancelled"
exit 1
fi
+77
View File
@@ -0,0 +1,77 @@
name: push-full-tests
on:
push:
branches: [main]
jobs:
unit-integration:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@v5
- uses: ./.github/actions/setup-go
- name: Setup K3s
uses: replicatedhq/action-k3s@main
with:
version: v1.31.2-k3s1
- name: Run tests
run: make test-integration
build-binaries:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v5
- uses: ./.github/actions/setup-go
- name: Cache Go build and modules
uses: actions/cache@v4
with:
path: |
~/.cache/go-build
~/go/pkg/mod
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-
- name: Go mod download
run: go mod download
- name: Build binaries
run: make build
- uses: actions/upload-artifact@v4
with:
name: binaries
path: bin/
retention-days: 1
e2e:
needs: [unit-integration, build-binaries]
runs-on: ubuntu-latest
timeout-minutes: 15
strategy:
fail-fast: false
matrix:
include:
- name: preflight
target: preflight-e2e-test
needs-k3s: true
- name: support-bundle-shell
target: support-bundle-e2e-test
needs-k3s: true
- name: support-bundle-go
target: support-bundle-e2e-go-test
needs-k3s: false
steps:
- uses: actions/checkout@v5
- uses: actions/download-artifact@v4
with:
name: binaries
path: bin/
- run: chmod +x bin/*
- name: Setup K3s
if: matrix.needs-k3s
uses: replicatedhq/action-k3s@main
with:
version: v1.31.2-k3s1
- run: make ${{ matrix.target }}
+2 -2
View File
@@ -2,7 +2,7 @@ name: Regression Test Suite
on:
push:
branches: [main, v1beta3]
branches: [main]
pull_request:
types: [opened, synchronize, reopened]
workflow_dispatch:
@@ -289,4 +289,4 @@ jobs:
continue-on-error: true
with:
api-token: ${{ secrets.REPLICATED_API_TOKEN }}
cluster-id: ${{ steps.create-cluster.outputs.cluster-id }}
cluster-id: ${{ steps.create-cluster.outputs.cluster-id }}
+215 -10
View File
@@ -37,57 +37,96 @@ endef
BUILDTAGS = "netgo containers_image_ostree_stub exclude_graphdriver_devicemapper exclude_graphdriver_btrfs containers_image_openpgp"
BUILDFLAGS = -tags ${BUILDTAGS} -installsuffix netgo
BUILDPATHS = ./pkg/... ./cmd/... ./internal/...
E2EPATHS = ./test/e2e/...
TESTFLAGS ?=
E2EPATHS ?= ./test/e2e/...
TESTFLAGS ?= -v -coverprofile cover.out
.DEFAULT_GOAL := all
all: clean build test
.PHONY: ffi
ffi: fmt vet
go build ${BUILDFLAGS} ${LDFLAGS} -o bin/troubleshoot.so -buildmode=c-shared ffi/main.go
.PHONY: test
test: fmt vet
if [ -n $(RUN) ]; then \
go test ${BUILDFLAGS} ${BUILDPATHS} ${TESTFLAGS} -run $(RUN); \
test: generate fmt vet
if [ -n "$(RUN)" ]; then \
go test ${BUILDFLAGS} ${BUILDPATHS} ${TESTFLAGS} -run "$(RUN)"; \
else \
go test ${BUILDFLAGS} ${BUILDPATHS} ${TESTFLAGS}; \
fi
# Run unit tests only for a provided list of packages.
# Usage: make test-packages PACKAGES="pkg/a pkg/b cmd/foo"
.PHONY: test-packages
test-packages:
@if [ -z "$(PACKAGES)" ]; then \
echo "No PACKAGES provided; nothing to test."; \
exit 0; \
fi
@echo "Running unit tests for packages: $(PACKAGES)"
go test ${BUILDFLAGS} $(PACKAGES) ${TESTFLAGS}
# Go tests that require a K8s instance
# TODOLATER: merge with test, so we get unified coverage reports? it'll add 21~sec to the test job though...
.PHONY: test-integration
test-integration: fmt vet
test-integration: generate fmt vet
go test -v --tags="integration exclude_graphdriver_devicemapper exclude_graphdriver_btrfs" ${BUILDPATHS}
.PHONY: preflight-e2e-test
preflight-e2e-test:
./test/validate-preflight-e2e.sh
.PHONY: run-examples
run-examples:
./test/run-examples.sh
.PHONY: support-bundle-e2e-test
support-bundle-e2e-test:
./test/validate-support-bundle-e2e.sh
.PHONY: preflight-e2e-go-test
preflight-e2e-go-test: bin/preflight
if [ -n "$(RUN)" ]; then \
go test ${BUILDFLAGS} ${E2EPATHS} -v -run "$(RUN)"; \
else \
go test ${BUILDFLAGS} ${E2EPATHS} -v; \
fi
.PHONY: support-bundle-e2e-go-test
support-bundle-e2e-go-test:
if [ -n $(RUN) ]; then \
go test ${BUILDFLAGS} ${E2EPATHS} -v -run $(RUN); \
support-bundle-e2e-go-test: bin/support-bundle
if [ -n "$(RUN)" ]; then \
go test ${BUILDFLAGS} ${E2EPATHS} -v -run "$(RUN)"; \
else \
go test ${BUILDFLAGS} ${E2EPATHS} -v; \
fi
rebuild: clean build
# Build all binaries in parallel ( -j )
build: tidy
@echo "Build cli binaries"
$(MAKE) bin/support-bundle bin/preflight
$(MAKE) -j bin/support-bundle bin/preflight
.PHONY: clean
clean:
@rm -f bin/analyze
@rm -f bin/support-bundle
@rm -f bin/collect
@rm -f bin/preflight
@rm -f bin/troubleshoot.h
@rm -f bin/troubleshoot.so
@rm -f bin/schemagen
@rm -f bin/docsgen
.PHONY: tidy
tidy:
go mod tidy
# Prints the diff of the changes that would be made by `go mod tidy`. Used in CI
.PHONY: tidy-diff
tidy-diff:
go mod tidy -diff
# Only build when any of the files in SOURCES changes, or if bin/<file> is absent
MAKEFILE_DIR := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))
SOURCES := $(shell find $(MAKEFILE_DIR) -type f \( -name "*.go" -o -name "go.mod" -o -name "go.sum" \))
@@ -97,12 +136,28 @@ bin/support-bundle: $(SOURCES)
bin/preflight: $(SOURCES)
go build ${BUILDFLAGS} ${LDFLAGS} -o bin/preflight github.com/replicatedhq/troubleshoot/cmd/preflight
bin/analyze: $(SOURCES)
go build ${BUILDFLAGS} ${LDFLAGS} -o bin/analyze github.com/replicatedhq/troubleshoot/cmd/analyze
bin/collect: $(SOURCES)
go build ${BUILDFLAGS} ${LDFLAGS} -o bin/collect github.com/replicatedhq/troubleshoot/cmd/collect
.PHONY: support-bundle
support-bundle: bin/support-bundle
.PHONY: preflight
preflight: bin/preflight
.PHONY: analyze
analyze: bin/analyze
.PHONY: collect
collect: bin/collect
build-linux: tidy
@echo "Build cli binaries for Linux"
GOOS=linux GOARCH=amd64 $(MAKE) -j bin/support-bundle bin/preflight bin/analyze bin/collect
.PHONY: fmt
fmt:
go fmt ${BUILDPATHS}
@@ -110,3 +165,153 @@ fmt:
.PHONY: vet
vet:
go vet ${BUILDFLAGS} ${BUILDPATHS}
.PHONY: generate
generate: controller-gen client-gen
$(CONTROLLER_GEN) \
object:headerFile=./hack/boilerplate.go.txt paths=./pkg/apis/...
$(CLIENT_GEN) \
--output-dir=. \
--output-pkg=github.com/replicatedhq/troubleshoot/pkg/client \
--clientset-name troubleshootclientset \
--input-base github.com/replicatedhq/troubleshoot/pkg/apis \
--input troubleshoot/v1beta1 \
--input troubleshoot/v1beta2 \
--go-header-file ./hack/boilerplate.go.txt
cp -r troubleshootclientset pkg/client
rm -rf troubleshootclientset
.PHONY: openapischema
openapischema: controller-gen
controller-gen crd +output:dir=./config/crds paths=./pkg/apis/troubleshoot/v1beta1
controller-gen crd +output:dir=./config/crds paths=./pkg/apis/troubleshoot/v1beta2
check-schemas: generate schemas
@if [ -n "$$(git status --short)" ]; then \
echo -e "\033[31mThe git repo is dirty :( Ensure all generated files are committed e.g CRD schema files\033[0;m"; \
git status --short; \
exit 1; \
fi
.PHONY: schemas
schemas: openapischema bin/schemagen
./bin/schemagen --output-dir ./schemas
bin/schemagen:
go build ${LDFLAGS} -o bin/schemagen github.com/replicatedhq/troubleshoot/cmd/schemagen
.PHONY: docs
docs: fmt vet bin/docsgen
./bin/docsgen
bin/docsgen:
go build ${LDFLAGS} -o bin/docsgen github.com/replicatedhq/troubleshoot/cmd/docsgen
controller-gen:
go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.19.0
CONTROLLER_GEN=$(shell which controller-gen)
.PHONY: client-gen
client-gen:
go install k8s.io/code-generator/cmd/client-gen@v0.34.0
CLIENT_GEN=$(shell which client-gen)
.PHONY: release
release: export GITHUB_TOKEN = $(shell echo ${GITHUB_TOKEN_TROUBLESHOOT})
release:
curl -sL https://git.io/goreleaser | bash -s -- --rm-dist --config deploy/.goreleaser.yml
.PHONY: snapshot-release
snapshot-release:
curl -sL https://git.io/goreleaser | bash -s -- --rm-dist --snapshot --config deploy/.goreleaser.snapshot.yml
docker push replicated/troubleshoot:alpha
docker push replicated/preflight:alpha
.PHONY: local-release
local-release:
curl -sL https://git.io/goreleaser | bash -s -- --rm-dist --snapshot --config deploy/.goreleaser.yaml
docker tag replicated/troubleshoot:alpha localhost:32000/troubleshoot:alpha
docker tag replicated/preflight:alpha localhost:32000/preflight:alpha
docker push localhost:32000/troubleshoot:alpha
docker push localhost:32000/preflight:alpha
.PHONY: run-preflight
run-preflight: bin/preflight
./bin/preflight ./examples/preflight/sample-preflight.yaml
.PHONY: run-support-bundle
run-support-bundle: bin/support-bundle
./bin/support-bundle ./examples/support-bundle/sample-supportbundle.yaml
.PHONY: run-analyze
run-analyze: bin/analyze
./bin/analyze --analyzers ./examples/support-bundle/sample-analyzers.yaml ./support-bundle.tar.gz
.PHONY: init-sbom
init-sbom:
mkdir -p sbom/spdx sbom/assets
.PHONY: install-spdx-sbom-generator
install-spdx-sbom-generator: init-sbom
./scripts/initialize-sbom-build.sh
SPDX_GENERATOR=./sbom/spdx-sbom-generator
.PHONY: generate-sbom
generate-sbom: install-spdx-sbom-generator
$(SPDX_GENERATOR) -o ./sbom/spdx
sbom/assets/troubleshoot-sbom.tgz: generate-sbom
tar -czf sbom/assets/troubleshoot-sbom.tgz sbom/spdx/*.spdx
sbom: sbom/assets/troubleshoot-sbom.tgz
cosign sign-blob \
--key ./cosign.key \
--tlog-upload \
--yes \
--rekor-url=https://rekor.sigstore.dev \
sbom/assets/troubleshoot-sbom.tgz > sbom/assets/troubleshoot-sbom.tgz.sig
cosign public-key --key cosign.key --outfile sbom/assets/key.pub
.PHONY: scan
scan:
trivy fs \
--scanners vuln \
--exit-code=1 \
--severity="HIGH,CRITICAL" \
--ignore-unfixed \
./
.PHONY: watch
watch: npm-install
bin/watch.js
## Syncronize the code with a remote server. More info: CONTRIBUTING.md
.PHONY: watchrsync
watchrsync: npm-install
bin/watchrsync.js
.PHONY: npm-install
npm-install:
npm --version 2>&1 >/dev/null || ( echo "npm not installed; install npm to set up watchrsync" && exit 1 )
npm list gaze-run-interrupt || npm install install --no-save gaze-run-interrupt@~2.0.0
######## Lagacy make targets ###########
# Deprecated: These can be removed
.PHONY: run-troubleshoot
run-troubleshoot: run-support-bundle
longhorn:
git clone https://github.com/longhorn/longhorn-manager.git
cd longhorn-manager && git checkout v1.2.2 && cd ..
rm -rf pkg/longhorn
mv longhorn-manager/k8s/pkg pkg/longhorn
mv longhorn-manager/types pkg/longhorn/types
mv longhorn-manager/util pkg/longhorn/util
rm -rf pkg/longhorn/util/daemon
rm -rf pkg/longhorn/util/server
find pkg/longhorn -type f | xargs sed -i "s/github.com\/longhorn\/longhorn-manager\/k8s\/pkg/github.com\/replicatedhq\/troubleshoot\/pkg\/longhorn/g"
find pkg/longhorn -type f | xargs sed -i "s/github.com\/longhorn\/longhorn-manager\/types/github.com\/replicatedhq\/troubleshoot\/pkg\/longhorn\/types/g"
find pkg/longhorn -type f | xargs sed -i "s/github.com\/longhorn\/longhorn-manager\/util/github.com\/replicatedhq\/troubleshoot\/pkg\/longhorn\/util/g"
rm -rf longhorn-manager
+13
View File
@@ -0,0 +1,13 @@
package main
import (
"os"
analyzecli "github.com/replicatedhq/troubleshoot/cmd/analyze/cli"
)
func main() {
if err := analyzecli.RootCmd().Execute(); err != nil {
os.Exit(1)
}
}
-98
View File
@@ -1,98 +0,0 @@
package cli
import (
"fmt"
"os"
"github.com/pkg/errors"
"github.com/replicatedhq/troubleshoot/pkg/constants"
"github.com/replicatedhq/troubleshoot/pkg/lint"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
func LintCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "lint [spec-files...]",
Args: cobra.MinimumNArgs(1),
Short: "Lint preflight specs for syntax and structural errors",
Long: `Lint preflight specs for syntax and structural errors.
This command validates troubleshoot specs and checks for:
- YAML syntax errors (missing colons, invalid structure)
- Missing required fields (apiVersion, kind, metadata, spec)
- Invalid template syntax ({{ .Values.* }}, {{ .Release.* }}, etc.)
- Missing analyzers or collectors
- Common structural issues
- Missing docStrings (warning)
Both v1beta2 and v1beta3 apiVersions are supported. Use 'convert' if you need a full structural conversion between schema versions.
The --fix flag can automatically repair:
- Missing colons in YAML (e.g., "metadata" → "metadata:")
- Missing or malformed apiVersion line. If templating ({{ }}) or docString fields are detected, apiVersion is set to v1beta3; otherwise v1beta2.
- Template expressions missing leading dot (e.g., "{{ Values.x }}" → "{{ .Values.x }}")
Examples:
# Lint a single spec file
preflight lint my-preflight.yaml
# Lint multiple spec files
preflight lint spec1.yaml spec2.yaml spec3.yaml
# Lint with automatic fixes (may need to run multiple times for complex issues)
preflight lint --fix my-preflight.yaml
# Lint and output as JSON for CI/CD integration
preflight lint --format json my-preflight.yaml
Exit codes:
0 - No errors found
2 - Validation errors found`,
PreRun: func(cmd *cobra.Command, args []string) {
viper.BindPFlags(cmd.Flags())
},
RunE: func(cmd *cobra.Command, args []string) error {
v := viper.GetViper()
opts := lint.LintOptions{
FilePaths: args,
Fix: v.GetBool("fix"),
Format: v.GetString("format"),
}
return runLint(opts)
},
}
cmd.Flags().Bool("fix", false, "Automatically fix issues where possible")
cmd.Flags().String("format", "text", "Output format: text or json")
return cmd
}
func runLint(opts lint.LintOptions) error {
// Validate file paths exist
for _, filePath := range opts.FilePaths {
if _, err := os.Stat(filePath); err != nil {
return errors.Wrapf(err, "file not found: %s", filePath)
}
}
// Run linting
results, err := lint.LintFiles(opts)
if err != nil {
return errors.Wrap(err, "failed to lint files")
}
// Format and print results
output := lint.FormatResults(results, opts.Format)
fmt.Print(output)
// Return appropriate exit code
if lint.HasErrors(results) {
os.Exit(constants.EXIT_CODE_SPEC_ISSUES)
}
return nil
}
-1
View File
@@ -89,7 +89,6 @@ that a cluster meets the requirements to run an application.`,
cmd.AddCommand(TemplateCmd())
cmd.AddCommand(DocsCmd())
cmd.AddCommand(ConvertCmd())
cmd.AddCommand(LintCmd())
preflight.AddFlags(cmd.PersistentFlags())
-2
View File
@@ -6,7 +6,6 @@ import (
"strings"
"github.com/replicatedhq/troubleshoot/cmd/internal/util"
preflightcli "github.com/replicatedhq/troubleshoot/cmd/preflight/cli"
"github.com/replicatedhq/troubleshoot/internal/traces"
"github.com/replicatedhq/troubleshoot/pkg/k8sutil"
"github.com/replicatedhq/troubleshoot/pkg/logger"
@@ -110,7 +109,6 @@ If no arguments are provided, specs are automatically loaded from the cluster by
cmd.AddCommand(Schedule())
cmd.AddCommand(UploadCmd())
cmd.AddCommand(util.VersionCmd())
cmd.AddCommand(preflightcli.LintCmd())
cmd.Flags().StringSlice("redactors", []string{}, "names of the additional redactors to use")
cmd.Flags().Bool("redact", true, "enable/disable default redactions")
+2 -2
View File
@@ -42,7 +42,7 @@ spec:
matchLabels:
os: darwin
arch: amd64
{{addURIAndSha "https://github.com/replicatedhq/troubleshoot/releases/download/{{ .TagName }}/preflight_darwin_amd64.tar.gz" .TagName }}
{{addURIAndSha "https://github.com/replicatedhq/troubleshoot/releases/download/{{ .TagName }}/preflight_darwin_all.tar.gz" .TagName }}
files:
- from: preflight
to: .
@@ -53,7 +53,7 @@ spec:
matchLabels:
os: darwin
arch: arm64
{{addURIAndSha "https://github.com/replicatedhq/troubleshoot/releases/download/{{ .TagName }}/preflight_darwin_arm64.tar.gz" .TagName }}
{{addURIAndSha "https://github.com/replicatedhq/troubleshoot/releases/download/{{ .TagName }}/preflight_darwin_all.tar.gz" .TagName }}
files:
- from: preflight
to: .
+2 -2
View File
@@ -42,7 +42,7 @@ spec:
matchLabels:
os: darwin
arch: amd64
{{addURIAndSha "https://github.com/replicatedhq/troubleshoot/releases/download/{{ .TagName }}/support-bundle_darwin_amd64.tar.gz" .TagName }}
{{addURIAndSha "https://github.com/replicatedhq/troubleshoot/releases/download/{{ .TagName }}/support-bundle_darwin_all.tar.gz" .TagName }}
files:
- from: support-bundle
to: .
@@ -53,7 +53,7 @@ spec:
matchLabels:
os: darwin
arch: arm64
{{addURIAndSha "https://github.com/replicatedhq/troubleshoot/releases/download/{{ .TagName }}/support-bundle_darwin_arm64.tar.gz" .TagName }}
{{addURIAndSha "https://github.com/replicatedhq/troubleshoot/releases/download/{{ .TagName }}/support-bundle_darwin_all.tar.gz" .TagName }}
files:
- from: support-bundle
to: .
@@ -1,33 +0,0 @@
apiVersion: troubleshoot.sh/v1beta3
kind: Preflight
metadata:
name: helm-builtins-example
labels:
release: {{ .Release.Name }}
spec:
analyzers:
- docString: |
Title: Example using Helm builtin objects
Requirement: Demonstrates .Values, .Release, .Chart, etc.
Supported Helm builtin objects:
- .Values.* - User-provided values
- .Release.Name - Release name (default: "preflight")
- .Release.Namespace - Release namespace (default: "default")
- .Release.IsInstall - Whether this is an install (true)
- .Release.IsUpgrade - Whether this is an upgrade (false)
- .Release.Revision - Release revision (1)
- .Chart.Name - Chart name
- .Chart.Version - Chart version
- .Capabilities.KubeVersion - Kubernetes version capabilities
clusterVersion:
checkName: Kubernetes version check in {{ .Release.Namespace }}
outcomes:
- fail:
when: '< {{ .Values.minVersion | default "1.19.0" }}'
message: |
Release {{ .Release.Name }} requires Kubernetes {{ .Values.minVersion | default "1.19.0" }} or later.
Chart: {{ .Chart.Name }}
- pass:
when: '>= {{ .Values.minVersion | default "1.19.0" }}'
message: Kubernetes version is supported for release {{ .Release.Name }}
@@ -1,8 +0,0 @@
apiVersion: troubleshoot.sh/v1beta3
kind: Preflight
metadata
name: invalid-yaml
spec:
analyzers:
- clusterVersion:
checkName: Kubernetes version
@@ -1,11 +0,0 @@
kind: Preflight
metadata:
name: missing-apiversion
spec:
analyzers:
- clusterVersion:
checkName: Kubernetes version
outcomes:
- pass:
when: '>= 1.19.0'
message: Kubernetes version is supported
@@ -1,10 +0,0 @@
apiVersion: troubleshoot.sh/v1beta3
kind: Preflight
spec:
analyzers:
- clusterVersion:
checkName: Kubernetes version
outcomes:
- pass:
when: '>= 1.19.0'
message: Kubernetes version is supported
@@ -1,7 +0,0 @@
apiVersion: troubleshoot.sh/v1beta3
kind: Preflight
metadata:
name: no-analyzers
spec:
collectors:
- clusterInfo: {}
@@ -1,18 +0,0 @@
apiVersion: troubleshoot.sh/v1beta3
kind: Preflight
metadata:
name: simple-no-template
spec:
analyzers:
- docString: |
Title: Kubernetes Version Check
Requirement: Kubernetes 1.19.0 or later
clusterVersion:
checkName: Kubernetes version
outcomes:
- fail:
when: '< 1.19.0'
message: Kubernetes version must be at least 1.19.0
- pass:
when: '>= 1.19.0'
message: Kubernetes version is supported
@@ -1,12 +0,0 @@
apiVersion: troubleshoot.sh/v1beta3
kind: SupportBundle
metadata:
name: no-collectors
spec:
analyzers:
- clusterVersion:
checkName: Kubernetes version
outcomes:
- pass:
when: '>= 1.19.0'
message: Kubernetes version is supported
@@ -1,15 +0,0 @@
apiVersion: troubleshoot.sh/v1beta3
kind: SupportBundle
metadata:
name: valid-support-bundle
spec:
collectors:
- clusterInfo: {}
- clusterResources: {}
analyzers:
- clusterVersion:
checkName: Kubernetes version
outcomes:
- pass:
when: '>= 1.19.0'
message: Kubernetes version is supported
@@ -1,15 +0,0 @@
apiVersion: troubleshoot.sh/v1beta3
kind: Preflight
metadata:
name: valid-preflight
spec:
analyzers:
- docString: |
Title: Test Analyzer
Requirement: Test requirement
clusterVersion:
checkName: Kubernetes version
outcomes:
- pass:
when: '>= 1.19.0'
message: Kubernetes version is supported
@@ -1,12 +0,0 @@
apiVersion: troubleshoot.sh/v1beta2
kind: Preflight
metadata:
name: wrong-version
spec:
analyzers:
- clusterVersion:
checkName: Kubernetes version
outcomes:
- pass:
when: '>= 1.19.0'
message: Kubernetes version is supported
+4 -5
View File
@@ -21,7 +21,7 @@ require (
github.com/google/gofuzz v1.2.0
github.com/google/uuid v1.6.0
github.com/gorilla/handlers v1.5.2
github.com/hashicorp/go-getter v1.8.0
github.com/hashicorp/go-getter v1.8.2
github.com/hashicorp/go-multierror v1.1.1
github.com/jackc/pgx/v5 v5.7.6
github.com/longhorn/go-iscsi-helper v0.0.0-20210330030558-49a327fb024e
@@ -33,7 +33,7 @@ require (
github.com/pkg/errors v0.9.1
github.com/replicatedhq/termui/v3 v3.1.1-0.20200811145416-f40076d26851
github.com/segmentio/ksuid v1.0.4
github.com/shirou/gopsutil/v4 v4.25.8
github.com/shirou/gopsutil/v4 v4.25.9
github.com/spf13/cobra v1.10.1
github.com/spf13/pflag v1.0.10
github.com/spf13/viper v1.21.0
@@ -57,7 +57,7 @@ require (
k8s.io/klog/v2 v2.130.1
k8s.io/kubernetes v1.34.1
oras.land/oras-go v1.2.6
sigs.k8s.io/controller-runtime v0.22.1
sigs.k8s.io/controller-runtime v0.22.2
sigs.k8s.io/e2e-framework v0.6.0
)
@@ -106,7 +106,7 @@ require (
github.com/coreos/go-systemd/v22 v22.5.0 // indirect
github.com/distribution/reference v0.6.0 // indirect
github.com/docker/distribution v2.8.3+incompatible // indirect
github.com/ebitengine/purego v0.8.4 // indirect
github.com/ebitengine/purego v0.9.0 // indirect
github.com/emicklei/go-restful/v3 v3.12.2 // indirect
github.com/envoyproxy/go-control-plane/envoy v1.32.4 // indirect
github.com/envoyproxy/protoc-gen-validate v1.2.1 // indirect
@@ -222,7 +222,6 @@ require (
github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
github.com/hashicorp/go-safetemp v1.0.0 // indirect
github.com/hashicorp/go-version v1.7.0
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/josharian/intern v1.0.0 // indirect
+8 -10
View File
@@ -213,8 +213,8 @@ github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4
github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/docker/libtrust v0.0.0-20160708172513-aabc10ec26b7 h1:UhxFibDNY/bfvqU5CAUmr9zpesgbU6SWc8/B4mflAE4=
github.com/docker/libtrust v0.0.0-20160708172513-aabc10ec26b7/go.mod h1:cyGadeNEkKy96OOhEzfZl+yxihPEzKnqJwvfuSUqbZE=
github.com/ebitengine/purego v0.8.4 h1:CF7LEKg5FFOsASUj0+QwaXf8Ht6TlFxg09+S9wz0omw=
github.com/ebitengine/purego v0.8.4/go.mod h1:iIjxzd6CiRiOG0UyXP+V1+jWqUXVjPKLAI0mRfJZTmQ=
github.com/ebitengine/purego v0.9.0 h1:mh0zpKBIXDceC63hpvPuGLiJ8ZAa3DfrFTudmfi8A4k=
github.com/ebitengine/purego v0.9.0/go.mod h1:iIjxzd6CiRiOG0UyXP+V1+jWqUXVjPKLAI0mRfJZTmQ=
github.com/emicklei/go-restful/v3 v3.12.2 h1:DhwDP0vY3k8ZzE0RunuJy8GhNpPL6zqLkDf9B/a0/xU=
github.com/emicklei/go-restful/v3 v3.12.2/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc=
github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
@@ -368,12 +368,10 @@ github.com/hashicorp/errwrap v1.1.0 h1:OxrOeh75EUXMY8TBjag2fzXGZ40LB6IKw45YeGUDY
github.com/hashicorp/errwrap v1.1.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
github.com/hashicorp/go-cleanhttp v0.5.2 h1:035FKYIWjmULyFRBKPs8TBQoi0x6d9G4xc9neXJWAZQ=
github.com/hashicorp/go-cleanhttp v0.5.2/go.mod h1:kO/YDlP8L1346E6Sodw+PrpBSV4/SoxCXGY6BqNFT48=
github.com/hashicorp/go-getter v1.8.0 h1:GMRdoMBDz12Mim366pWsRVIrrkugJ19rrmykkv0Nhzo=
github.com/hashicorp/go-getter v1.8.0/go.mod h1:/K0O5zR6R72O3r2x3z2UHadiC0XHMbbzHO9pS8ZeJPA=
github.com/hashicorp/go-getter v1.8.2 h1:CGCK+bZQLl44PYiwJweVzfpjg7bBwtuXu3AGcLiod2o=
github.com/hashicorp/go-getter v1.8.2/go.mod h1:CUTt9x2bCtJ/sV8ihgrITL3IUE+0BE1j/e4n5P/GIM4=
github.com/hashicorp/go-multierror v1.1.1 h1:H5DkEtf6CXdFp0N0Em5UCwQpXMWke8IA0+lD48awMYo=
github.com/hashicorp/go-multierror v1.1.1/go.mod h1:iw975J/qwKPdAO1clOe2L8331t/9/fmwbPZ6JB6eMoM=
github.com/hashicorp/go-safetemp v1.0.0 h1:2HR189eFNrjHQyENnQMMpCiBAsRxzbTMIgBhEyExpmo=
github.com/hashicorp/go-safetemp v1.0.0/go.mod h1:oaerMy3BhqiTbVye6QuFhFtIceqFoDHxNAB65b+Rj1I=
github.com/hashicorp/go-version v1.7.0 h1:5tqGy27NaOTB8yJKUZELlFAS/LTKJkrmONwQKeRZfjY=
github.com/hashicorp/go-version v1.7.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=
github.com/hashicorp/golang-lru/arc/v2 v2.0.5 h1:l2zaLDubNhW4XO3LnliVj0GXO3+/CGNJAg1dcN2Fpfw=
@@ -593,8 +591,8 @@ github.com/segmentio/ksuid v1.0.4 h1:sBo2BdShXjmcugAMwjugoGUdUV0pcxY5mW4xKRn3v4c
github.com/segmentio/ksuid v1.0.4/go.mod h1:/XUiZBD3kVx5SmUOl55voK5yeAbBNNIed+2O73XgrPE=
github.com/sergi/go-diff v1.3.1 h1:xkr+Oxo4BOQKmkn/B9eMK0g5Kg/983T9DqqPHwYqD+8=
github.com/sergi/go-diff v1.3.1/go.mod h1:aMJSSKb2lpPvRNec0+w3fl7LP9IOFzdc9Pa4NFbPK1I=
github.com/shirou/gopsutil/v4 v4.25.8 h1:NnAsw9lN7587WHxjJA9ryDnqhJpFH6A+wagYWTOH970=
github.com/shirou/gopsutil/v4 v4.25.8/go.mod h1:q9QdMmfAOVIw7a+eF86P7ISEU6ka+NLgkUxlopV4RwI=
github.com/shirou/gopsutil/v4 v4.25.9 h1:JImNpf6gCVhKgZhtaAHJ0serfFGtlfIlSC08eaKdTrU=
github.com/shirou/gopsutil/v4 v4.25.9/go.mod h1:gxIxoC+7nQRwUl/xNhutXlD8lq+jxTgpIkEf3rADHL8=
github.com/shopspring/decimal v1.4.0 h1:bxl37RwXBklmTi0C79JfXCEBD1cqqHt0bbgBAGFp81k=
github.com/shopspring/decimal v1.4.0/go.mod h1:gawqmDU56v4yIKSwfBSFip1HdCCXN8/+DMd9qYNcwME=
github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
@@ -905,8 +903,8 @@ oras.land/oras-go/v2 v2.6.0 h1:X4ELRsiGkrbeox69+9tzTu492FMUu7zJQW6eJU+I2oc=
oras.land/oras-go/v2 v2.6.0/go.mod h1:magiQDfG6H1O9APp+rOsvCPcW1GD2MM7vgnKY0Y+u1o=
periph.io/x/host/v3 v3.8.5 h1:g4g5xE1XZtDiGl1UAJaUur1aT7uNiFLMkyMEiZ7IHII=
periph.io/x/host/v3 v3.8.5/go.mod h1:hPq8dISZIc+UNfWoRj+bPH3XEBQqJPdFdx218W92mdc=
sigs.k8s.io/controller-runtime v0.22.1 h1:Ah1T7I+0A7ize291nJZdS1CabF/lB4E++WizgV24Eqg=
sigs.k8s.io/controller-runtime v0.22.1/go.mod h1:FwiwRjkRPbiN+zp2QRp7wlTCzbUXxZ/D4OzuQUDwBHY=
sigs.k8s.io/controller-runtime v0.22.2 h1:cK2l8BGWsSWkXz09tcS4rJh95iOLney5eawcK5A33r4=
sigs.k8s.io/controller-runtime v0.22.2/go.mod h1:+QX1XUpTXN4mLoblf4tqr5CQcyHPAki2HLXqQMY6vh8=
sigs.k8s.io/e2e-framework v0.6.0 h1:p7hFzHnLKO7eNsWGI2AbC1Mo2IYxidg49BiT4njxkrM=
sigs.k8s.io/e2e-framework v0.6.0/go.mod h1:IREnCHnKgRCioLRmNi0hxSJ1kJ+aAdjEKK/gokcZu4k=
sigs.k8s.io/json v0.0.0-20241014173422-cfa47c3a1cc8 h1:gBQPwqORJ8d8/YNZWEjoZs7npUVDpVXUUOFfW6CgAqE=
+644 -105
View File
@@ -421,39 +421,31 @@ func (a *OllamaAgent) Analyze(ctx context.Context, data []byte, analyzers []anal
func (a *OllamaAgent) discoverAnalyzers(bundle *analyzer.SupportBundle) []analyzer.AnalyzerSpec {
var specs []analyzer.AnalyzerSpec
// Collect files by type for aggregation
podFiles := []string{}
deploymentFiles := []string{}
eventFiles := []string{}
nodeFiles := []string{}
// Analyze bundle contents to determine what types of analysis to perform
for filePath := range bundle.Files {
filePath = strings.ToLower(filePath)
filePathLower := strings.ToLower(filePath)
switch {
case strings.Contains(filePath, "pods") && strings.HasSuffix(filePath, ".json"):
specs = append(specs, analyzer.AnalyzerSpec{
Name: "ai-pod-analysis",
Type: "ai-workload",
Category: "pods",
Priority: 10,
Config: map[string]interface{}{"filePath": filePath, "promptType": "pod-analysis"},
})
case strings.Contains(filePathLower, "pods") && strings.HasSuffix(filePathLower, ".json"):
podFiles = append(podFiles, filePath)
case strings.Contains(filePath, "deployments") && strings.HasSuffix(filePath, ".json"):
specs = append(specs, analyzer.AnalyzerSpec{
Name: "ai-deployment-analysis",
Type: "ai-workload",
Category: "deployments",
Priority: 9,
Config: map[string]interface{}{"filePath": filePath, "promptType": "deployment-analysis"},
})
case strings.Contains(filePathLower, "deployments") && strings.HasSuffix(filePathLower, ".json"):
deploymentFiles = append(deploymentFiles, filePath)
case strings.Contains(filePath, "events") && strings.HasSuffix(filePath, ".json"):
specs = append(specs, analyzer.AnalyzerSpec{
Name: "ai-event-analysis",
Type: "ai-events",
Category: "events",
Priority: 8,
Config: map[string]interface{}{"filePath": filePath, "promptType": "event-analysis"},
})
case strings.Contains(filePathLower, "events") && strings.HasSuffix(filePathLower, ".json"):
eventFiles = append(eventFiles, filePath)
case strings.Contains(filePath, "logs") && strings.HasSuffix(filePath, ".log"):
case strings.Contains(filePathLower, "nodes") && strings.HasSuffix(filePathLower, ".json"):
nodeFiles = append(nodeFiles, filePath)
case strings.Contains(filePathLower, "logs") && strings.HasSuffix(filePathLower, ".log"):
// Logs are analyzed separately per file (not aggregated)
specs = append(specs, analyzer.AnalyzerSpec{
Name: "ai-log-analysis",
Type: "ai-logs",
@@ -461,50 +453,424 @@ func (a *OllamaAgent) discoverAnalyzers(bundle *analyzer.SupportBundle) []analyz
Priority: 7,
Config: map[string]interface{}{"filePath": filePath, "promptType": "log-analysis"},
})
case strings.Contains(filePath, "nodes") && strings.HasSuffix(filePath, ".json"):
specs = append(specs, analyzer.AnalyzerSpec{
Name: "ai-resource-analysis",
Type: "ai-resources",
Category: "nodes",
Priority: 8,
Config: map[string]interface{}{"filePath": filePath, "promptType": "resource-analysis"},
})
}
}
// Create aggregated analyzer for ALL pod files (cluster-wide view)
if len(podFiles) > 0 {
specs = append(specs, analyzer.AnalyzerSpec{
Name: "ai-pod-analysis-cluster",
Type: "ai-workload",
Category: "pods",
Priority: 10,
Config: map[string]interface{}{
"filePaths": podFiles,
"promptType": "pod-analysis",
"aggregated": true,
},
})
}
// Create aggregated analyzer for ALL deployment files (cluster-wide view)
if len(deploymentFiles) > 0 {
specs = append(specs, analyzer.AnalyzerSpec{
Name: "ai-deployment-analysis-cluster",
Type: "ai-workload",
Category: "deployments",
Priority: 9,
Config: map[string]interface{}{
"filePaths": deploymentFiles,
"promptType": "deployment-analysis",
"aggregated": true,
},
})
}
// Create aggregated analyzer for ALL event files (cluster-wide view)
if len(eventFiles) > 0 {
specs = append(specs, analyzer.AnalyzerSpec{
Name: "ai-event-analysis-cluster",
Type: "ai-events",
Category: "events",
Priority: 8,
Config: map[string]interface{}{
"filePaths": eventFiles,
"promptType": "event-analysis",
"aggregated": true,
},
})
}
// Create aggregated analyzer for ALL node files (cluster-wide view)
if len(nodeFiles) > 0 {
specs = append(specs, analyzer.AnalyzerSpec{
Name: "ai-resource-analysis-cluster",
Type: "ai-resources",
Category: "nodes",
Priority: 8,
Config: map[string]interface{}{
"filePaths": nodeFiles,
"promptType": "resource-analysis",
"aggregated": true,
},
})
}
return specs
}
// aggregateFiles combines multiple files of the same type into a single summary for analysis
func (a *OllamaAgent) aggregateFiles(bundle *analyzer.SupportBundle, filePaths []string, category string) (string, error) {
var summary strings.Builder
switch category {
case "pods":
return a.aggregatePodFiles(bundle, filePaths)
case "deployments":
return a.aggregateDeploymentFiles(bundle, filePaths)
case "events":
return a.aggregateEventFiles(bundle, filePaths)
case "nodes":
return a.aggregateNodeFiles(bundle, filePaths)
default:
// For other types, just concatenate the files
summary.WriteString(fmt.Sprintf("Aggregated analysis of %d files:\n\n", len(filePaths)))
for _, filePath := range filePaths {
if data, exists := bundle.Files[filePath]; exists {
summary.WriteString(fmt.Sprintf("--- File: %s ---\n", filePath))
summary.Write(data)
summary.WriteString("\n\n")
}
}
}
return summary.String(), nil
}
// aggregatePodFiles creates a cluster-wide summary of pods from multiple namespace files
func (a *OllamaAgent) aggregatePodFiles(bundle *analyzer.SupportBundle, filePaths []string) (string, error) {
var summary strings.Builder
totalPods := 0
runningPods := 0
pendingPods := 0
failedPods := 0
succeededPods := 0
namespaceStats := make(map[string]int)
summary.WriteString("CLUSTER-WIDE POD ANALYSIS\n")
summary.WriteString("Analyzing pods across all namespaces:\n\n")
for _, filePath := range filePaths {
data, exists := bundle.Files[filePath]
if !exists {
continue
}
// Extract namespace from path (e.g., "cluster-resources/pods/kube-system.json")
parts := strings.Split(filePath, "/")
namespace := "unknown"
if len(parts) >= 3 {
namespace = strings.TrimSuffix(parts[len(parts)-1], ".json")
}
// Parse pod data - handle both PodList and single Pod objects
var podList map[string]interface{}
if err := json.Unmarshal(data, &podList); err != nil {
continue
}
// Check if this is a List object with items array
items, ok := podList["items"].([]interface{})
if ok {
// Handle PodList - process all pods in the list
// Initialize namespace for valid PodList (ensures empty namespaces are tracked)
if _, exists := namespaceStats[namespace]; !exists {
namespaceStats[namespace] = 0
}
podCount := len(items)
namespaceStats[namespace] += podCount
totalPods += podCount
// Count pod statuses
for _, item := range items {
pod, ok := item.(map[string]interface{})
if !ok {
continue
}
status, ok := pod["status"].(map[string]interface{})
if !ok {
continue
}
phase, ok := status["phase"].(string)
if !ok {
continue
}
switch phase {
case "Running":
runningPods++
case "Pending":
pendingPods++
case "Failed":
failedPods++
case "Succeeded":
succeededPods++
}
}
} else {
// Handle single Pod object (not a list)
// Check if this is a single Pod object (has "kind": "Pod")
if kind, exists := podList["kind"].(string); exists && kind == "Pod" {
// Initialize namespace only for valid pod data
if _, exists := namespaceStats[namespace]; !exists {
namespaceStats[namespace] = 0
}
// Single pod - increment count for this namespace
namespaceStats[namespace]++
totalPods++
// Extract status for single pod
if status, ok := podList["status"].(map[string]interface{}); ok {
if phase, ok := status["phase"].(string); ok {
switch phase {
case "Running":
runningPods++
case "Pending":
pendingPods++
case "Failed":
failedPods++
case "Succeeded":
succeededPods++
}
}
}
}
// Skip to next file after processing single pod or invalid data
continue
}
}
summary.WriteString(fmt.Sprintf("Total pods in cluster: %d\n", totalPods))
summary.WriteString(fmt.Sprintf(" - Running: %d\n", runningPods))
summary.WriteString(fmt.Sprintf(" - Pending: %d\n", pendingPods))
summary.WriteString(fmt.Sprintf(" - Failed: %d\n", failedPods))
summary.WriteString(fmt.Sprintf(" - Succeeded: %d\n", succeededPods))
summary.WriteString("\nPods by namespace:\n")
for namespace, count := range namespaceStats {
if count > 0 {
summary.WriteString(fmt.Sprintf(" - %s: %d pods\n", namespace, count))
} else {
summary.WriteString(fmt.Sprintf(" - %s: empty (no pods)\n", namespace))
}
}
summary.WriteString("\nIMPORTANT CONTEXT:\n")
summary.WriteString("- Empty namespaces are NORMAL in Kubernetes\n")
summary.WriteString("- Only report issues if there are actual pod failures or critical problems\n")
summary.WriteString("- The presence of empty namespaces is not a problem\n")
return summary.String(), nil
}
// aggregateDeploymentFiles creates a cluster-wide summary of deployments
func (a *OllamaAgent) aggregateDeploymentFiles(bundle *analyzer.SupportBundle, filePaths []string) (string, error) {
var summary strings.Builder
totalDeployments := 0
namespaceStats := make(map[string]int)
summary.WriteString("CLUSTER-WIDE DEPLOYMENT ANALYSIS\n")
summary.WriteString("Analyzing deployments across all namespaces:\n\n")
for _, filePath := range filePaths {
data, exists := bundle.Files[filePath]
if !exists {
continue
}
parts := strings.Split(filePath, "/")
namespace := "unknown"
if len(parts) >= 3 {
namespace = strings.TrimSuffix(parts[len(parts)-1], ".json")
}
// Parse deployment data - handle both DeploymentList and single Deployment objects
var deploymentList map[string]interface{}
if err := json.Unmarshal(data, &deploymentList); err != nil {
continue
}
// Check if this is a List object with items array
items, ok := deploymentList["items"].([]interface{})
if ok {
// Handle DeploymentList - process all deployments in the list
// Initialize namespace for valid DeploymentList (ensures empty namespaces are tracked)
if _, exists := namespaceStats[namespace]; !exists {
namespaceStats[namespace] = 0
}
deployCount := len(items)
namespaceStats[namespace] += deployCount
totalDeployments += deployCount
} else {
// Handle single Deployment object (not a list)
// Check if this is a single Deployment object (has "kind": "Deployment")
if kind, exists := deploymentList["kind"].(string); exists && kind == "Deployment" {
// Initialize namespace only for valid deployment data
if _, exists := namespaceStats[namespace]; !exists {
namespaceStats[namespace] = 0
}
// Single deployment - increment count for this namespace
namespaceStats[namespace]++
totalDeployments++
}
// Skip to next file after processing single deployment or invalid data
continue
}
}
summary.WriteString(fmt.Sprintf("Total deployments in cluster: %d\n", totalDeployments))
summary.WriteString("\nDeployments by namespace:\n")
for namespace, count := range namespaceStats {
if count > 0 {
summary.WriteString(fmt.Sprintf(" - %s: %d deployments\n", namespace, count))
} else {
summary.WriteString(fmt.Sprintf(" - %s: no deployments\n", namespace))
}
}
summary.WriteString("\nIMPORTANT: Empty namespaces are normal. Only flag actual deployment issues.\n")
return summary.String(), nil
}
// aggregateEventFiles creates a cluster-wide summary of events
func (a *OllamaAgent) aggregateEventFiles(bundle *analyzer.SupportBundle, filePaths []string) (string, error) {
var summary strings.Builder
totalEvents := 0
summary.WriteString("CLUSTER-WIDE EVENT ANALYSIS\n")
summary.WriteString("Analyzing events across all namespaces:\n\n")
eventsIncluded := 0
for _, filePath := range filePaths {
data, exists := bundle.Files[filePath]
if !exists {
continue
}
// Parse event data - handle both EventList and single Event objects
var eventList map[string]interface{}
if err := json.Unmarshal(data, &eventList); err != nil {
continue
}
// Check if this is a List object with items array
items, ok := eventList["items"].([]interface{})
if ok {
itemCount := len(items)
totalEvents += itemCount
// Include actual event data for AI analysis (limited to 50 events max for the summary)
// Only include if adding this file wouldn't significantly exceed the limit
if itemCount > 0 && eventsIncluded < 50 && (eventsIncluded+itemCount) <= 60 {
dataStr := string(data)
// Include file if data size is reasonable
if len(dataStr) < 2000 {
summary.WriteString(fmt.Sprintf("\n--- Events from %s ---\n", filePath))
summary.WriteString(dataStr)
summary.WriteString("\n")
eventsIncluded += itemCount
}
}
}
}
summary.WriteString(fmt.Sprintf("\nTotal events collected: %d\n", totalEvents))
return summary.String(), nil
}
// aggregateNodeFiles creates a cluster-wide summary of nodes
func (a *OllamaAgent) aggregateNodeFiles(bundle *analyzer.SupportBundle, filePaths []string) (string, error) {
var summary strings.Builder
summary.WriteString("CLUSTER-WIDE NODE ANALYSIS\n\n")
for _, filePath := range filePaths {
data, exists := bundle.Files[filePath]
if !exists {
continue
}
summary.WriteString(fmt.Sprintf("--- Nodes data from %s ---\n", filePath))
summary.Write(data)
summary.WriteString("\n\n")
}
return summary.String(), nil
}
// runLLMAnalysis executes analysis using LLM for a specific analyzer spec
func (a *OllamaAgent) runLLMAnalysis(ctx context.Context, bundle *analyzer.SupportBundle, spec analyzer.AnalyzerSpec) (*analyzer.AnalyzerResult, error) {
ctx, span := otel.Tracer(constants.LIB_TRACER_NAME).Start(ctx, fmt.Sprintf("OllamaAgent.%s", spec.Name))
defer span.End()
// Smart file detection for enhanced analyzer compatibility
var filePath string
var fileData []byte
var exists bool
var dataStr string
// First try to get explicit filePath from config
if fp, ok := spec.Config["filePath"].(string); ok {
filePath = fp
fileData, exists = bundle.Files[filePath]
}
// If no explicit filePath, auto-detect based on analyzer type
if !exists {
filePath, fileData, exists = a.autoDetectFileForAnalyzer(bundle, spec)
}
if !exists {
result := &analyzer.AnalyzerResult{
Title: spec.Name,
IsWarn: true,
Message: fmt.Sprintf("File not found: %s", filePath),
Category: spec.Category,
// Check if this is an aggregated analyzer (multiple files)
if aggregated, ok := spec.Config["aggregated"].(bool); ok && aggregated {
// Handle aggregated files
if filePaths, ok := spec.Config["filePaths"].([]string); ok && len(filePaths) > 0 {
aggregatedData, err := a.aggregateFiles(bundle, filePaths, spec.Category)
if err != nil {
return &analyzer.AnalyzerResult{
Title: spec.Name,
IsWarn: true,
Message: fmt.Sprintf("Failed to aggregate files: %v", err),
Category: spec.Category,
}, nil
}
dataStr = aggregatedData
} else {
// Missing or invalid filePaths for aggregated analyzer
return &analyzer.AnalyzerResult{
Title: spec.Name,
IsWarn: true,
Message: "Aggregated analyzer missing valid filePaths configuration",
Category: spec.Category,
}, nil
}
return result, nil
} else {
// Smart file detection for enhanced analyzer compatibility (single file)
var filePath string
var fileData []byte
var exists bool
// First try to get explicit filePath from config
if fp, ok := spec.Config["filePath"].(string); ok {
filePath = fp
fileData, exists = bundle.Files[filePath]
}
// If no explicit filePath, auto-detect based on analyzer type
if !exists {
filePath, fileData, exists = a.autoDetectFileForAnalyzer(bundle, spec)
}
if !exists {
result := &analyzer.AnalyzerResult{
Title: spec.Name,
IsWarn: true,
Message: fmt.Sprintf("File not found: %s", filePath),
Category: spec.Category,
}
return result, nil
}
dataStr = string(fileData)
}
promptType, _ := spec.Config["promptType"].(string)
@@ -519,7 +885,6 @@ func (a *OllamaAgent) runLLMAnalysis(ctx context.Context, bundle *analyzer.Suppo
}
// Prepare data for analysis (truncate if too large)
dataStr := string(fileData)
if len(dataStr) > 4000 { // Limit input size
if promptType == "log-analysis" {
// For logs, take the last N lines
@@ -866,6 +1231,182 @@ func (a *OllamaAgent) autoDetectFileForAnalyzer(bundle *analyzer.SupportBundle,
return "", nil, false
}
// normalizeInsights converts various JSON formats into a []string array
func (a *OllamaAgent) normalizeInsights(raw json.RawMessage) []string {
if len(raw) == 0 {
return []string{}
}
// Try parsing as array of strings first (expected format)
var arrayInsights []string
if err := json.Unmarshal(raw, &arrayInsights); err == nil {
return arrayInsights
}
// Try parsing as single string
var stringInsight string
if err := json.Unmarshal(raw, &stringInsight); err == nil {
if stringInsight != "" {
return []string{stringInsight}
}
return []string{}
}
// Try parsing as array of objects/maps (common LLM format)
var arrayOfMaps []map[string]interface{}
if err := json.Unmarshal(raw, &arrayOfMaps); err == nil {
insights := []string{}
for _, obj := range arrayOfMaps {
// Extract meaningful text from each object
insightText := a.formatMapAsInsight(obj)
if insightText != "" {
insights = append(insights, insightText)
}
}
return insights
}
// Try parsing as object/map and extract meaningful text
var objInsights map[string]interface{}
if err := json.Unmarshal(raw, &objInsights); err == nil {
insights := []string{}
for key, value := range objInsights {
// Extract meaningful insights from object structure
insightText := a.extractInsightText(key, value)
if insightText != "" {
insights = append(insights, insightText)
}
}
return insights
}
// If all parsing fails, return empty array
return []string{}
}
// formatMapAsInsight converts a map/object into a readable insight string
func (a *OllamaAgent) formatMapAsInsight(obj map[string]interface{}) string {
// Common patterns in LLM responses for insights
// Try to extract description, pattern, message, etc.
// Priority 1: Look for description field
if desc, ok := obj["description"].(string); ok && desc != "" {
if pattern, ok := obj["pattern"].(string); ok && pattern != "" {
return fmt.Sprintf("%s: %s", pattern, desc)
}
return desc
}
// Priority 2: Look for message field
if msg, ok := obj["message"].(string); ok && msg != "" {
return msg
}
// Priority 3: Look for explanation/implication field
if expl, ok := obj["explanation"].(string); ok && expl != "" {
return expl
}
if impl, ok := obj["implication"].(string); ok && impl != "" {
return impl
}
// Priority 4: Combine all string fields
parts := []string{}
for key, value := range obj {
if str, ok := value.(string); ok && str != "" {
parts = append(parts, fmt.Sprintf("%s: %s", key, str))
}
}
if len(parts) > 0 {
return strings.Join(parts, ", ")
}
return ""
}
// extractInsightText extracts readable text from nested JSON structures
func (a *OllamaAgent) extractInsightText(key string, value interface{}) string {
switch v := value.(type) {
case string:
if v != "" {
return fmt.Sprintf("%s: %s", key, v)
}
case map[string]interface{}:
// For nested objects, create a summary
parts := []string{}
for subKey, subValue := range v {
if str, ok := subValue.(string); ok && str != "" {
parts = append(parts, fmt.Sprintf("%s=%s", subKey, str))
}
}
if len(parts) > 0 {
return fmt.Sprintf("%s: %s", key, strings.Join(parts, ", "))
}
case []interface{}:
// For arrays, join elements
parts := []string{}
for _, item := range v {
if str, ok := item.(string); ok && str != "" {
parts = append(parts, str)
}
}
if len(parts) > 0 {
return fmt.Sprintf("%s: %s", key, strings.Join(parts, ", "))
}
case float64, int, bool:
return fmt.Sprintf("%s: %v", key, v)
}
return ""
}
// getStringField extracts a string field from a map, trying multiple key variants
func (a *OllamaAgent) getStringField(m map[string]interface{}, keys ...string) string {
for _, key := range keys {
if val, ok := m[key]; ok {
if str, ok := val.(string); ok {
return str
}
}
}
return ""
}
// extractRemediation extracts remediation info from various JSON structures
func (a *OllamaAgent) extractRemediation(result *analyzer.AnalyzerResult, remData interface{}) {
switch rem := remData.(type) {
case map[string]interface{}:
// Single remediation object
desc := a.getStringField(rem, "description", "Description")
action := a.getStringField(rem, "action", "Action")
command := a.getStringField(rem, "command", "Command")
priority := 5 // default priority
if p, ok := rem["priority"].(float64); ok {
priority = int(p)
} else if p, ok := rem["Priority"].(float64); ok {
priority = int(p)
}
if desc != "" || action != "" {
result.Remediation = &analyzer.RemediationStep{
Description: desc,
Action: action,
Command: command,
Priority: priority,
Category: "ai-suggested",
IsAutomatable: false,
}
}
case []interface{}:
// Array of remediation suggestions - use the first one
if len(rem) > 0 {
if firstRem, ok := rem[0].(map[string]interface{}); ok {
a.extractRemediation(result, firstRem)
}
}
}
}
// parseLLMResponse parses the LLM response into an AnalyzerResult
func (a *OllamaAgent) parseLLMResponse(response string, spec analyzer.AnalyzerSpec) (*analyzer.AnalyzerResult, error) {
// First try JSON parsing
@@ -875,55 +1416,53 @@ func (a *OllamaAgent) parseLLMResponse(response string, spec analyzer.AnalyzerSp
if jsonStart != -1 && jsonEnd != -1 && jsonEnd > jsonStart {
jsonStr := response[jsonStart : jsonEnd+1]
var llmResult struct {
Status string `json:"status"`
Title string `json:"title"`
Message string `json:"message"`
Insights []string `json:"insights"`
Remediation struct {
Description string `json:"description"`
Action string `json:"action"`
Command string `json:"command"`
Priority int `json:"priority"`
} `json:"remediation"`
}
if err := json.Unmarshal([]byte(jsonStr), &llmResult); err == nil {
// Successfully parsed JSON
result := &analyzer.AnalyzerResult{
Title: llmResult.Title,
Message: llmResult.Message,
Category: spec.Category,
Insights: llmResult.Insights,
}
switch strings.ToLower(llmResult.Status) {
case "pass":
result.IsPass = true
case "warn":
result.IsWarn = true
case "fail":
result.IsFail = true
default:
result.IsWarn = true
}
if llmResult.Remediation.Description != "" {
result.Remediation = &analyzer.RemediationStep{
Description: llmResult.Remediation.Description,
Action: llmResult.Remediation.Action,
Command: llmResult.Remediation.Command,
Priority: llmResult.Remediation.Priority,
Category: "ai-suggested",
IsAutomatable: false,
}
}
return result, nil
} else {
// JSON was found but malformed
// Try with a flexible map first to handle case-insensitive fields
var jsonMap map[string]interface{}
if err := json.Unmarshal([]byte(jsonStr), &jsonMap); err != nil {
return nil, errors.Wrap(err, "failed to parse LLM JSON response")
}
// Extract fields in a case-insensitive way
status := a.getStringField(jsonMap, "status", "Status")
title := a.getStringField(jsonMap, "title", "Title")
message := a.getStringField(jsonMap, "message", "Message")
// Get insights field (try both lowercase and uppercase)
var insightsRaw json.RawMessage
if insights, ok := jsonMap["insights"]; ok {
insightsRaw, _ = json.Marshal(insights)
} else if insights, ok := jsonMap["Insights"]; ok {
insightsRaw, _ = json.Marshal(insights)
}
insights := a.normalizeInsights(insightsRaw)
result := &analyzer.AnalyzerResult{
Title: title,
Message: message,
Category: spec.Category,
Insights: insights,
}
switch strings.ToLower(status) {
case "pass":
result.IsPass = true
case "warn":
result.IsWarn = true
case "fail":
result.IsFail = true
default:
result.IsWarn = true
}
// Handle remediation (try both cases)
if rem, ok := jsonMap["remediation"]; ok {
a.extractRemediation(result, rem)
} else if rem, ok := jsonMap["Remediation"]; ok {
a.extractRemediation(result, rem)
}
return result, nil
}
// Fall back to markdown parsing when JSON fails
@@ -206,7 +206,10 @@ func TestOllamaAgent_discoverAnalyzers(t *testing.T) {
assert.NotNil(t, spec.Config)
// Verify AI-specific config
assert.Contains(t, spec.Config, "filePath")
// Aggregated analyzers use "filePaths", single-file analyzers use "filePath"
hasFilePath := spec.Config["filePath"] != nil
hasFilePaths := spec.Config["filePaths"] != nil
assert.True(t, hasFilePath || hasFilePaths, "spec must have either filePath or filePaths")
assert.Contains(t, spec.Config, "promptType")
}
+46 -3
View File
@@ -10,9 +10,16 @@ import (
"github.com/pkg/errors"
troubleshootv1beta2 "github.com/replicatedhq/troubleshoot/pkg/apis/troubleshoot/v1beta2"
"github.com/replicatedhq/troubleshoot/pkg/k8sutil"
"helm.sh/helm/v3/pkg/action"
"k8s.io/apimachinery/pkg/api/meta"
"k8s.io/cli-runtime/pkg/genericclioptions"
"k8s.io/client-go/discovery"
"k8s.io/client-go/discovery/cached/memory"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/rest"
"k8s.io/client-go/restmapper"
"k8s.io/client-go/tools/clientcmd"
"k8s.io/klog/v2"
)
@@ -45,6 +52,42 @@ type VersionInfo struct {
Values map[string]interface{} `json:"values,omitempty"`
}
type configGetter struct {
restConfig *rest.Config
}
// ToDiscoveryClient implements genericclioptions.RESTClientGetter.
func (c configGetter) ToDiscoveryClient() (discovery.CachedDiscoveryInterface, error) {
discoveryClient, err := discovery.NewDiscoveryClientForConfig(c.restConfig)
if err != nil {
return nil, err
}
cached := memory.NewMemCacheClient(discoveryClient)
return cached, nil
}
// ToRESTConfig implements genericclioptions.RESTClientGetter.
func (c configGetter) ToRESTConfig() (*rest.Config, error) {
return c.restConfig, nil
}
// ToRESTMapper implements genericclioptions.RESTClientGetter.
func (c configGetter) ToRESTMapper() (meta.RESTMapper, error) {
discoveryClient, err := c.ToDiscoveryClient()
if err != nil {
return nil, err
}
mapper := restmapper.NewDeferredDiscoveryRESTMapper(discoveryClient)
return mapper, nil
}
// ToRawKubeConfigLoader implements genericclioptions.RESTClientGetter.
func (c configGetter) ToRawKubeConfigLoader() clientcmd.ClientConfig {
return k8sutil.GetKubeconfig()
}
var _ genericclioptions.RESTClientGetter = configGetter{}
func (c *CollectHelm) Title() string {
return getCollectorName(c)
}
@@ -57,7 +100,7 @@ func (c *CollectHelm) Collect(progressChan chan<- interface{}) (CollectorResult,
output := NewResult()
releaseInfos, err := helmReleaseHistoryCollector(c.Collector.ReleaseName, c.Collector.Namespace, c.Collector.CollectValues)
releaseInfos, err := helmReleaseHistoryCollector(c.ClientConfig, c.Collector.ReleaseName, c.Collector.Namespace, c.Collector.CollectValues)
if err != nil {
errsToMarhsal := []string{}
for _, e := range err {
@@ -88,12 +131,12 @@ func (c *CollectHelm) Collect(progressChan chan<- interface{}) (CollectorResult,
return output, nil
}
func helmReleaseHistoryCollector(releaseName string, namespace string, collectValues bool) ([]ReleaseInfo, []error) {
func helmReleaseHistoryCollector(config *rest.Config, releaseName string, namespace string, collectValues bool) ([]ReleaseInfo, []error) {
var results []ReleaseInfo
error_list := []error{}
actionConfig := new(action.Configuration)
if err := actionConfig.Init(nil, namespace, "", klog.V(2).Infof); err != nil {
if err := actionConfig.Init(configGetter{config}, namespace, "", klog.V(2).Infof); err != nil {
return nil, []error{err}
}
-768
View File
@@ -1,768 +0,0 @@
package lint
import (
"fmt"
"regexp"
"strings"
"os"
"github.com/pkg/errors"
"github.com/replicatedhq/troubleshoot/pkg/constants"
"sigs.k8s.io/yaml"
)
type LintResult struct {
FilePath string
Errors []LintError
Warnings []LintWarning
}
type LintError struct {
Line int
Column int
Message string
Field string
}
type LintWarning struct {
Line int
Column int
Message string
Field string
}
type LintOptions struct {
FilePaths []string
Fix bool
Format string // "text" or "json"
}
// LintFiles validates v1beta3 troubleshoot specs for syntax and structural errors
func LintFiles(opts LintOptions) ([]LintResult, error) {
results := []LintResult{}
for _, filePath := range opts.FilePaths {
result, err := lintFile(filePath, opts.Fix)
if err != nil {
return nil, err
}
results = append(results, result)
}
return results, nil
}
func lintFile(filePath string, fix bool) (LintResult, error) {
result := LintResult{
FilePath: filePath,
Errors: []LintError{},
Warnings: []LintWarning{},
}
// Read file
content, err := os.ReadFile(filePath)
if err != nil {
return result, errors.Wrapf(err, "failed to read file %s", filePath)
}
// Check if file contains template expressions
hasTemplates := strings.Contains(string(content), "{{") && strings.Contains(string(content), "}}")
// Validate YAML syntax (but be lenient with templated files)
var parsed map[string]interface{}
yamlParseErr := yaml.Unmarshal(content, &parsed)
if yamlParseErr != nil {
// If the file has templates, YAML parsing may fail - that's expected
// We'll still try to validate what we can
if !hasTemplates {
result.Errors = append(result.Errors, LintError{
Line: extractLineFromError(yamlParseErr),
Message: fmt.Sprintf("YAML syntax error: %s", yamlParseErr.Error()),
})
// Don't return yet - we want to try to fix this error
// Continue to applyFixes at the end
// Try to surface apiVersion issues even if YAML failed to parse
// Detect via simple textual scan
avLine, avValue := findAPIVersionLineAndValue(string(content))
if avLine == 0 {
result.Errors = append(result.Errors, LintError{
Line: 0,
Field: "apiVersion",
Message: "Missing or empty 'apiVersion' field",
})
} else if avValue != constants.Troubleshootv1beta2Kind && avValue != constants.Troubleshootv1beta3Kind {
result.Errors = append(result.Errors, LintError{
Line: avLine,
Field: "apiVersion",
Message: fmt.Sprintf("Invalid 'apiVersion' value %q; expected %s or %s", avValue, constants.Troubleshootv1beta2Kind, constants.Troubleshootv1beta3Kind),
})
}
if fix {
fixed, err := applyFixes(filePath, string(content), result)
if err != nil {
return result, err
}
if fixed {
// Re-lint to verify fixes
return lintFile(filePath, false)
}
}
return result, nil
}
// For templated files, we can't parse YAML strictly, so just check template syntax
result.Errors = append(result.Errors, checkTemplateSyntax(string(content))...)
// Surface apiVersion issues via textual scan for templated files
avLine, avValue := findAPIVersionLineAndValue(string(content))
if avLine == 0 {
result.Errors = append(result.Errors, LintError{
Line: 0,
Field: "apiVersion",
Message: "Missing or empty 'apiVersion' field",
})
} else if avValue != constants.Troubleshootv1beta2Kind && avValue != constants.Troubleshootv1beta3Kind {
result.Errors = append(result.Errors, LintError{
Line: avLine,
Field: "apiVersion",
Message: fmt.Sprintf("Invalid 'apiVersion' value %q; expected %s or %s", avValue, constants.Troubleshootv1beta2Kind, constants.Troubleshootv1beta3Kind),
})
}
// Continue to applyFixes for templates too
if fix {
fixed, err := applyFixes(filePath, string(content), result)
if err != nil {
return result, err
}
if fixed {
// Re-lint to verify fixes
return lintFile(filePath, false)
}
}
return result, nil
}
// Check required fields
result.Errors = append(result.Errors, checkRequiredFields(parsed, string(content))...)
// Check template syntax
result.Errors = append(result.Errors, checkTemplateSyntax(string(content))...)
// Check for kind-specific requirements
if kind, ok := parsed["kind"].(string); ok {
switch kind {
case "Preflight":
result.Errors = append(result.Errors, checkPreflightSpec(parsed, string(content))...)
case "SupportBundle":
result.Errors = append(result.Errors, checkSupportBundleSpec(parsed, string(content))...)
}
}
// Validate apiVersion value if present
if apiVersion, ok := parsed["apiVersion"].(string); ok && apiVersion != "" {
if apiVersion != constants.Troubleshootv1beta2Kind && apiVersion != constants.Troubleshootv1beta3Kind {
result.Errors = append(result.Errors, LintError{
Line: findLineNumber(string(content), "apiVersion"),
Field: "apiVersion",
Message: fmt.Sprintf("Invalid 'apiVersion' value %q; expected %s or %s", apiVersion, constants.Troubleshootv1beta2Kind, constants.Troubleshootv1beta3Kind),
})
}
}
// Check for common issues
result.Warnings = append(result.Warnings, checkCommonIssues(parsed, string(content))...)
// Apply fixes if requested
if fix && (len(result.Errors) > 0 || len(result.Warnings) > 0) {
fixed, err := applyFixes(filePath, string(content), result)
if err != nil {
return result, err
}
if fixed {
// Re-lint to verify fixes
return lintFile(filePath, false)
}
}
return result, nil
}
func checkRequiredFields(parsed map[string]interface{}, content string) []LintError {
errors := []LintError{}
// Check apiVersion
if apiVersion, ok := parsed["apiVersion"].(string); !ok || apiVersion == "" {
errors = append(errors, LintError{
Line: findLineNumber(content, "apiVersion"),
Field: "apiVersion",
Message: "Missing or empty 'apiVersion' field",
})
}
// Check kind
if kind, ok := parsed["kind"].(string); !ok || kind == "" {
errors = append(errors, LintError{
Line: findLineNumber(content, "kind"),
Field: "kind",
Message: "Missing or empty 'kind' field",
})
} else if kind != "Preflight" && kind != "SupportBundle" {
errors = append(errors, LintError{
Line: findLineNumber(content, "kind"),
Field: "kind",
Message: fmt.Sprintf("Invalid kind '%s'. Must be 'Preflight' or 'SupportBundle'", kind),
})
}
// Check metadata
if _, ok := parsed["metadata"]; !ok {
errors = append(errors, LintError{
Line: findLineNumber(content, "metadata"),
Field: "metadata",
Message: "Missing 'metadata' section",
})
} else if metadata, ok := parsed["metadata"].(map[string]interface{}); ok {
if name, ok := metadata["name"].(string); !ok || name == "" {
errors = append(errors, LintError{
Line: findLineNumber(content, "name"),
Field: "metadata.name",
Message: "Missing or empty 'metadata.name' field",
})
}
}
// Check spec
if _, ok := parsed["spec"]; !ok {
errors = append(errors, LintError{
Line: findLineNumber(content, "spec"),
Field: "spec",
Message: "Missing 'spec' section",
})
}
return errors
}
func checkTemplateSyntax(content string) []LintError {
errors := []LintError{}
lines := strings.Split(content, "\n")
// Check for unmatched braces
for i, line := range lines {
// Count opening and closing braces
opening := strings.Count(line, "{{")
closing := strings.Count(line, "}}")
if opening != closing {
errors = append(errors, LintError{
Line: i + 1,
Message: fmt.Sprintf("Unmatched template braces: %d opening, %d closing", opening, closing),
})
}
// Check for common template syntax issues
// Look for templates that might be missing the leading dot
if strings.Contains(line, "{{") && strings.Contains(line, "}}") {
// Extract template expressions
templateExpr := extractTemplateBetweenBraces(line)
for _, expr := range templateExpr {
trimmed := strings.TrimSpace(expr)
// Skip empty expressions
if trimmed == "" {
continue
}
// Skip control structures (if, else, end, range, with, etc.)
if isControlStructure(trimmed) {
continue
}
// Skip comments: {{/* ... */}}
if strings.HasPrefix(trimmed, "/*") || strings.HasPrefix(trimmed, "*/") {
continue
}
// Skip template variables (start with $)
if strings.HasPrefix(trimmed, "$") {
continue
}
// Skip expressions that start with a dot (valid references)
if strings.HasPrefix(trimmed, ".") {
continue
}
// Skip string literals
if strings.HasPrefix(trimmed, "\"") || strings.HasPrefix(trimmed, "'") {
continue
}
// Skip numeric literals
if regexp.MustCompile(`^[0-9]+$`).MatchString(trimmed) {
continue
}
// Skip function calls (contain parentheses or pipes)
if strings.Contains(trimmed, "(") || strings.Contains(trimmed, "|") {
continue
}
// Skip known Helm functions/keywords
helmFunctions := []string{"toYaml", "toJson", "include", "required", "default", "quote", "nindent", "indent", "upper", "lower", "trim"}
isFunction := false
for _, fn := range helmFunctions {
if strings.HasPrefix(trimmed, fn+" ") || trimmed == fn {
isFunction = true
break
}
}
if isFunction {
continue
}
// If we got here, it might be missing a leading dot
errors = append(errors, LintError{
Line: i + 1,
Message: fmt.Sprintf("Template expression may be missing leading dot: {{ %s }}", expr),
})
}
}
}
return errors
}
func checkPreflightSpec(parsed map[string]interface{}, content string) []LintError {
errors := []LintError{}
spec, ok := parsed["spec"].(map[string]interface{})
if !ok {
return errors
}
// Check for analyzers
analyzers, hasAnalyzers := spec["analyzers"]
if !hasAnalyzers {
errors = append(errors, LintError{
Line: findLineNumber(content, "spec:"),
Field: "spec.analyzers",
Message: "Preflight spec must contain 'analyzers'",
})
} else if analyzersList, ok := analyzers.([]interface{}); ok {
if len(analyzersList) == 0 {
errors = append(errors, LintError{
Line: findLineNumber(content, "analyzers"),
Field: "spec.analyzers",
Message: "Preflight spec must have at least one analyzer",
})
}
}
return errors
}
func checkSupportBundleSpec(parsed map[string]interface{}, content string) []LintError {
errors := []LintError{}
spec, ok := parsed["spec"].(map[string]interface{})
if !ok {
return errors
}
// Check for collectors
collectors, hasCollectors := spec["collectors"]
_, hasHostCollectors := spec["hostCollectors"]
if !hasCollectors && !hasHostCollectors {
errors = append(errors, LintError{
Line: findLineNumber(content, "spec:"),
Field: "spec.collectors",
Message: "SupportBundle spec must contain 'collectors' or 'hostCollectors'",
})
} else {
// Check if collectors list is empty
if hasCollectors {
if collectorsList, ok := collectors.([]interface{}); ok && len(collectorsList) == 0 {
errors = append(errors, LintError{
Line: findLineNumber(content, "collectors"),
Field: "spec.collectors",
Message: "Collectors list is empty",
})
}
}
}
return errors
}
func checkCommonIssues(parsed map[string]interface{}, content string) []LintWarning {
warnings := []LintWarning{}
// Check for missing docStrings in analyzers
spec, ok := parsed["spec"].(map[string]interface{})
if !ok {
return warnings
}
if analyzers, ok := spec["analyzers"].([]interface{}); ok {
for i, analyzer := range analyzers {
if analyzerMap, ok := analyzer.(map[string]interface{}); ok {
if _, hasDocString := analyzerMap["docString"]; !hasDocString {
warnings = append(warnings, LintWarning{
Line: findAnalyzerLine(content, i),
Field: fmt.Sprintf("spec.analyzers[%d].docString", i),
Message: "Analyzer missing docString (recommended for v1beta3)",
})
}
}
}
}
return warnings
}
func applyFixes(filePath, content string, result LintResult) (bool, error) {
fixed := false
newContent := content
lines := strings.Split(newContent, "\n")
// Determine desired apiVersion for fixes
hasTemplates := strings.Contains(content, "{{") && strings.Contains(content, "}}")
hasDocStrings := strings.Contains(content, "docString:")
desiredAPIVersion := constants.Troubleshootv1beta2Kind
if hasTemplates || hasDocStrings {
desiredAPIVersion = constants.Troubleshootv1beta3Kind
}
// Sort errors by line number (descending) to avoid line number shifts when editing
errorsByLine := make(map[int][]LintError)
for _, err := range result.Errors {
if err.Line > 0 {
errorsByLine[err.Line] = append(errorsByLine[err.Line], err)
}
}
// Process errors line by line
for lineNum, errs := range errorsByLine {
if lineNum > len(lines) {
continue
}
line := lines[lineNum-1]
originalLine := line
for _, err := range errs {
// Fix 1: Add missing colon
// YAML parsers often report the error on the line AFTER the actual problem
if strings.Contains(err.Message, "could not find expected ':'") {
// Check current line first
if !strings.Contains(line, ":") {
trimmed := strings.TrimSpace(line)
indent := line[:len(line)-len(strings.TrimLeft(line, " \t"))]
line = indent + trimmed + ":"
fixed = true
} else if lineNum > 1 {
// Check previous line (where the colon is likely missing)
prevLine := lines[lineNum-2]
if !strings.Contains(prevLine, ":") && strings.TrimSpace(prevLine) != "" {
trimmed := strings.TrimSpace(prevLine)
indent := prevLine[:len(prevLine)-len(strings.TrimLeft(prevLine, " \t"))]
lines[lineNum-2] = indent + trimmed + ":"
fixed = true
}
}
}
// Fix 2: Add missing leading dot in template expressions
if strings.Contains(err.Message, "Template expression may be missing leading dot:") {
// Extract the expression from the error message
re := regexp.MustCompile(`Template expression may be missing leading dot: \{\{ (.+?) \}\}`)
matches := re.FindStringSubmatch(err.Message)
if len(matches) > 1 {
badExpr := matches[1]
// Add the leading dot
fixedExpr := "." + badExpr
// Replace in the line
line = strings.Replace(line, "{{ "+badExpr+" }}", "{{ "+fixedExpr+" }}", 1)
line = strings.Replace(line, "{{"+badExpr+"}}", "{{"+fixedExpr+"}}", 1)
line = strings.Replace(line, "{{- "+badExpr+" }}", "{{- "+fixedExpr+" }}", 1)
line = strings.Replace(line, "{{- "+badExpr+" -}}", "{{- "+fixedExpr+" -}}", 1)
fixed = true
}
}
// Fix 3: Replace invalid apiVersion value with desiredAPIVersion
if strings.Contains(err.Message, "Invalid 'apiVersion' value") && err.Field == "apiVersion" {
if strings.Contains(line, "apiVersion:") {
indent := line[:len(line)-len(strings.TrimLeft(line, " \t"))]
line = indent + "apiVersion: " + desiredAPIVersion
fixed = true
}
}
}
// Update the line if it changed
if line != originalLine {
lines[lineNum-1] = line
}
}
// Fix 4: Add missing required top-level fields
for _, err := range result.Errors {
if err.Field == "apiVersion" && strings.Contains(err.Message, "Missing or empty 'apiVersion'") {
// Replace existing empty apiVersion line if present; otherwise prepend
if avLine, avVal := findAPIVersionLineAndValue(newContent); avLine > 0 && strings.TrimSpace(avVal) == "" {
line := lines[avLine-1]
indent := line[:len(line)-len(strings.TrimLeft(line, " \t"))]
lines[avLine-1] = indent + "apiVersion: " + desiredAPIVersion
} else {
lines = append([]string{"apiVersion: " + desiredAPIVersion}, lines...)
}
fixed = true
} else if err.Field == "kind" && strings.Contains(err.Message, "Missing or empty 'kind'") {
// Try to determine if it should be Preflight or SupportBundle based on filename
kind := "Preflight"
if strings.Contains(strings.ToLower(filePath), "bundle") {
kind = "SupportBundle"
}
// Add kind after apiVersion
insertIndex := 0
for i, line := range lines {
if strings.Contains(line, "apiVersion:") {
insertIndex = i + 1
break
}
}
newLines := make([]string, 0, len(lines)+1)
newLines = append(newLines, lines[:insertIndex]...)
newLines = append(newLines, "kind: "+kind)
newLines = append(newLines, lines[insertIndex:]...)
lines = newLines
fixed = true
} else if err.Field == "metadata" && strings.Contains(err.Message, "Missing 'metadata'") {
// Add metadata section after kind
insertIndex := 0
for i, line := range lines {
if strings.Contains(line, "kind:") {
insertIndex = i + 1
break
}
}
newLines := make([]string, 0, len(lines)+2)
newLines = append(newLines, lines[:insertIndex]...)
newLines = append(newLines, "metadata:")
newLines = append(newLines, " name: my-spec")
newLines = append(newLines, lines[insertIndex:]...)
lines = newLines
fixed = true
}
}
// Write fixed content back to file if changes were made
if fixed {
newContent = strings.Join(lines, "\n")
if err := os.WriteFile(filePath, []byte(newContent), 0644); err != nil {
return false, errors.Wrapf(err, "failed to write fixed content to %s", filePath)
}
}
return fixed, nil
}
func findLineNumber(content, search string) int {
lines := strings.Split(content, "\n")
for i, line := range lines {
if strings.Contains(line, search) {
return i + 1
}
}
return 0
}
// findAPIVersionLineAndValue locates the first line that declares apiVersion and returns its
// 1-based line number and the trimmed value to the right of the colon. Returns (0, "") if not found.
func findAPIVersionLineAndValue(content string) (int, string) {
lines := strings.Split(content, "\n")
for i, line := range lines {
trimmed := strings.TrimSpace(line)
if strings.HasPrefix(trimmed, "apiVersion:") {
// extract value after the first colon
parts := strings.SplitN(trimmed, ":", 2)
if len(parts) == 2 {
value := strings.TrimSpace(parts[1])
return i + 1, value
}
return i + 1, ""
}
}
return 0, ""
}
func findAnalyzerLine(content string, index int) int {
lines := strings.Split(content, "\n")
analyzerCount := 0
inAnalyzers := false
for i, line := range lines {
if strings.Contains(line, "analyzers:") {
inAnalyzers = true
continue
}
if inAnalyzers && strings.HasPrefix(strings.TrimSpace(line), "- ") {
if analyzerCount == index {
return i + 1
}
analyzerCount++
}
}
return 0
}
func extractLineFromError(err error) int {
// Try to extract line number from YAML error message
re := regexp.MustCompile(`line (\d+)`)
matches := re.FindStringSubmatch(err.Error())
if len(matches) > 1 {
var line int
fmt.Sscanf(matches[1], "%d", &line)
return line
}
return 0
}
// extractTemplateBetweenBraces extracts template expressions from a line
func extractTemplateBetweenBraces(line string) []string {
var expressions []string
// Match {{ ... }} with optional whitespace trimming (-), including comments {{/* */}}
re := regexp.MustCompile(`\{\{-?\s*(.+?)\s*-?\}\}`)
matches := re.FindAllStringSubmatch(line, -1)
for _, match := range matches {
if len(match) > 1 {
// Clean up the expression
expr := match[1]
// Remove */ at the end if it's part of a comment
expr = strings.TrimSuffix(strings.TrimSpace(expr), "*/")
expressions = append(expressions, expr)
}
}
return expressions
}
// isControlStructure checks if a template expression is a control structure
func isControlStructure(expr string) bool {
trimmed := strings.TrimSpace(expr)
controlKeywords := []string{"if", "else", "end", "range", "with", "define", "template", "block", "include"}
for _, keyword := range controlKeywords {
if strings.HasPrefix(trimmed, keyword+" ") || trimmed == keyword {
return true
}
}
return false
}
// FormatResults formats lint results for output
func FormatResults(results []LintResult, format string) string {
if format == "json" {
return formatJSON(results)
}
return formatText(results)
}
func formatText(results []LintResult) string {
var output strings.Builder
totalErrors := 0
totalWarnings := 0
for _, result := range results {
if len(result.Errors) == 0 && len(result.Warnings) == 0 {
output.WriteString(fmt.Sprintf("✓ %s: No issues found\n", result.FilePath))
continue
}
output.WriteString(fmt.Sprintf("\n%s:\n", result.FilePath))
for _, err := range result.Errors {
output.WriteString(fmt.Sprintf(" ✗ Error (line %d): %s\n", err.Line, err.Message))
if err.Field != "" {
output.WriteString(fmt.Sprintf(" Field: %s\n", err.Field))
}
totalErrors++
}
for _, warn := range result.Warnings {
output.WriteString(fmt.Sprintf(" ⚠ Warning (line %d): %s\n", warn.Line, warn.Message))
if warn.Field != "" {
output.WriteString(fmt.Sprintf(" Field: %s\n", warn.Field))
}
totalWarnings++
}
}
output.WriteString(fmt.Sprintf("\nSummary: %d error(s), %d warning(s) across %d file(s)\n", totalErrors, totalWarnings, len(results)))
return output.String()
}
func formatJSON(results []LintResult) string {
// Simple JSON formatting without importing encoding/json
var output strings.Builder
output.WriteString("{\n")
output.WriteString(" \"results\": [\n")
for i, result := range results {
output.WriteString(" {\n")
output.WriteString(fmt.Sprintf(" \"filePath\": %q,\n", result.FilePath))
output.WriteString(" \"errors\": [\n")
for j, err := range result.Errors {
output.WriteString(" {\n")
output.WriteString(fmt.Sprintf(" \"line\": %d,\n", err.Line))
output.WriteString(fmt.Sprintf(" \"column\": %d,\n", err.Column))
output.WriteString(fmt.Sprintf(" \"message\": %q,\n", err.Message))
output.WriteString(fmt.Sprintf(" \"field\": %q\n", err.Field))
output.WriteString(" }")
if j < len(result.Errors)-1 {
output.WriteString(",")
}
output.WriteString("\n")
}
output.WriteString(" ],\n")
output.WriteString(" \"warnings\": [\n")
for j, warn := range result.Warnings {
output.WriteString(" {\n")
output.WriteString(fmt.Sprintf(" \"line\": %d,\n", warn.Line))
output.WriteString(fmt.Sprintf(" \"column\": %d,\n", warn.Column))
output.WriteString(fmt.Sprintf(" \"message\": %q,\n", warn.Message))
output.WriteString(fmt.Sprintf(" \"field\": %q\n", warn.Field))
output.WriteString(" }")
if j < len(result.Warnings)-1 {
output.WriteString(",")
}
output.WriteString("\n")
}
output.WriteString(" ]\n")
output.WriteString(" }")
if i < len(results)-1 {
output.WriteString(",")
}
output.WriteString("\n")
}
output.WriteString(" ]\n")
output.WriteString("}\n")
return output.String()
}
// HasErrors returns true if any of the results contain errors
func HasErrors(results []LintResult) bool {
for _, result := range results {
if len(result.Errors) > 0 {
return true
}
}
return false
}
+549
View File
@@ -0,0 +1,549 @@
package main
import (
"bufio"
"bytes"
"encoding/json"
"errors"
"flag"
"fmt"
"io"
"io/fs"
"os"
"os/exec"
"path/filepath"
"regexp"
"sort"
"strings"
)
// goListPackageJSON models the subset of fields we need from `go list -json` output.
// The JSON can be quite large; we intentionally only decode what we use to keep memory reasonable.
type goListPackageJSON struct {
ImportPath string `json:"ImportPath"`
Deps []string `json:"Deps"`
}
// runCommand executes a command and returns stdout as bytes with trimmed trailing newline.
func runCommand(name string, args ...string) ([]byte, error) {
cmd := exec.Command(name, args...)
cmd.Stderr = os.Stderr
out, err := cmd.Output()
if err != nil {
return nil, err
}
return bytes.TrimRight(out, "\n"), nil
}
// listPackageWithDeps returns the full transitive dependency set for a package import path,
// including the package itself.
func listPackageWithDeps(importPath string) (map[string]struct{}, error) {
cmd := exec.Command("go", "list", "-json", "-deps", importPath)
cmd.Stderr = os.Stderr
out, err := cmd.Output()
if err != nil {
return nil, fmt.Errorf("go list -json -deps %s failed: %w", importPath, err)
}
deps := make(map[string]struct{})
dec := json.NewDecoder(bytes.NewReader(out))
for {
var pkg goListPackageJSON
if err := dec.Decode(&pkg); err != nil {
if errors.Is(err, io.EOF) {
break
}
return nil, fmt.Errorf("decode go list json: %w", err)
}
if pkg.ImportPath != "" {
deps[pkg.ImportPath] = struct{}{}
}
for _, d := range pkg.Deps {
deps[d] = struct{}{}
}
}
return deps, nil
}
// changedFiles returns a slice of file paths changed between baseRef and HEAD.
func changedFiles(baseRef string) ([]string, error) {
// Use triple-dot to include merge base with baseRef, typical for PR diffs.
out, err := runCommand("git", "diff", "--name-only", baseRef+"...HEAD")
if err != nil {
return nil, fmt.Errorf("git diff failed: %w", err)
}
var files []string
scanner := bufio.NewScanner(bytes.NewReader(out))
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if line == "" {
continue
}
files = append(files, line)
}
if err := scanner.Err(); err != nil {
return nil, err
}
return files, nil
}
// mapFilesToPackages resolves a set of Go package import paths that directly contain the changed Go files.
func mapFilesToPackages(files []string) (map[string]struct{}, error) {
packages := make(map[string]struct{})
// Collect unique directories that contain changed Go files.
dirSet := make(map[string]struct{})
for _, f := range files {
if strings.HasPrefix(f, "vendor/") {
continue
}
if filepath.Ext(f) != ".go" {
continue
}
d := filepath.Dir(f)
if d == "." {
d = "."
}
dirSet[d] = struct{}{}
}
if len(dirSet) == 0 {
return packages, nil
}
// Convert to a stable-ordered slice of directories to avoid nondeterminism.
var dirs []string
for d := range dirSet {
// Ensure relative paths are treated as packages; prepend ./ for clarity.
if strings.HasPrefix(d, "./") || d == "." {
dirs = append(dirs, d)
} else {
dirs = append(dirs, "./"+d)
}
}
sort.Strings(dirs)
// `go list` accepts directories and returns their package import paths.
args := append([]string{"list", "-f", "{{.ImportPath}}"}, dirs...)
out, err := runCommand("go", args...)
if err != nil {
return nil, fmt.Errorf("go list for files failed: %w", err)
}
scanner := bufio.NewScanner(bytes.NewReader(out))
for scanner.Scan() {
pkg := strings.TrimSpace(scanner.Text())
if pkg != "" {
packages[pkg] = struct{}{}
}
}
if err := scanner.Err(); err != nil {
return nil, err
}
return packages, nil
}
// computeAffectedPackages expands directly changed packages to include reverse dependencies across the module.
// We query all packages with test dependencies (-test) to ensure test-only imports are considered.
func computeAffectedPackages(directPkgs map[string]struct{}) (map[string]struct{}, error) {
affected := make(map[string]struct{})
for p := range directPkgs {
affected[p] = struct{}{}
}
if len(directPkgs) == 0 {
return affected, nil
}
// Enumerate all packages in the module with their deps.
// We stream decode concatenated JSON objects produced by `go list -json`.
cmd := exec.Command("go", "list", "-json", "-deps", "-test", "./...")
cmd.Stderr = os.Stderr
out, err := cmd.Output()
if err != nil {
return nil, fmt.Errorf("go list -json failed: %w", err)
}
dec := json.NewDecoder(bytes.NewReader(out))
for {
var pkg goListPackageJSON
if err := dec.Decode(&pkg); err != nil {
if errors.Is(err, io.EOF) {
break
}
return nil, fmt.Errorf("decode go list json: %w", err)
}
// If this package is directly changed, it's already included.
// If it depends (directly or transitively) on any changed package, include it.
for changed := range directPkgs {
if pkg.ImportPath == changed {
affected[pkg.ImportPath] = struct{}{}
break
}
// Linear scan over deps is acceptable given typical package counts.
for _, dep := range pkg.Deps {
if dep == changed {
affected[pkg.ImportPath] = struct{}{}
break
}
}
}
}
return affected, nil
}
// listTestFunctions scans a directory for Go test files and returns names of functions
// that match the pattern `func TestXxx(t *testing.T)`.
func listTestFunctions(dir string) ([]string, error) {
var tests []string
// Regex to capture test function names. This is a simple heuristic suitable for our codebase.
testFuncRe := regexp.MustCompile(`^func\s+(Test[\w\d_]+)\s*\(`)
walkFn := func(path string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() {
return nil
}
if !strings.HasSuffix(d.Name(), "_test.go") {
return nil
}
b, err := os.ReadFile(path)
if err != nil {
return err
}
scanner := bufio.NewScanner(bytes.NewReader(b))
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if m := testFuncRe.FindStringSubmatch(line); m != nil {
tests = append(tests, m[1])
}
}
return scanner.Err()
}
if err := filepath.WalkDir(dir, walkFn); err != nil {
return nil, err
}
sort.Strings(tests)
return tests, nil
}
func main() {
baseRef := flag.String("base", "origin/main", "Git base ref to diff against (e.g., origin/main)")
printAllOnChanges := flag.Bool("all-on-mod-change", true, "Run all tests if go.mod or go.sum changed")
verbose := flag.Bool("v", false, "Enable verbose diagnostics to stderr")
mode := flag.String("mode", "packages", "Output mode: 'packages' to print import paths; 'suites' to print e2e suite names")
changedFilesCSV := flag.String("changed-files", "", "Comma-separated paths to treat as changed (bypass git)")
changedFilesFile := flag.String("changed-files-file", "", "File with newline-separated paths to treat as changed")
flag.Parse()
// Determine the set of changed files: explicit list if provided, otherwise via git diff.
var files []string
if *changedFilesCSV != "" || *changedFilesFile != "" {
if *changedFilesCSV != "" {
parts := strings.Split(*changedFilesCSV, ",")
for _, p := range parts {
if s := strings.TrimSpace(p); s != "" {
files = append(files, s)
}
}
}
if *changedFilesFile != "" {
b, err := os.ReadFile(*changedFilesFile)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(2)
}
scanner := bufio.NewScanner(bytes.NewReader(b))
for scanner.Scan() {
if s := strings.TrimSpace(scanner.Text()); s != "" {
files = append(files, s)
}
}
if err := scanner.Err(); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(2)
}
}
} else {
var err error
files, err = changedFiles(*baseRef)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(2)
}
}
if *verbose {
fmt.Fprintln(os.Stderr, "Changed files vs base:")
if len(files) == 0 {
fmt.Fprintln(os.Stderr, " (none)")
} else {
for _, f := range files {
fmt.Fprintln(os.Stderr, " ", f)
}
}
}
// Track module change and CI configuration changes to drive conservative behavior.
moduleChanged := false
ciChanged := false
if *printAllOnChanges {
for _, f := range files {
if f == "go.mod" || f == "go.sum" {
moduleChanged = true
}
if strings.HasPrefix(f, "scripts/") || strings.HasPrefix(f, ".github/workflows/") {
ciChanged = true
}
}
if (moduleChanged || ciChanged) && *mode == "packages" {
if *verbose {
if moduleChanged {
fmt.Fprintln(os.Stderr, "Detected module file change (go.mod/go.sum); selecting all packages ./...")
}
if ciChanged {
fmt.Fprintln(os.Stderr, "Detected CI/detector change (scripts/ or .github/workflows/); selecting all packages ./...")
}
}
fmt.Println("./...")
return
}
}
directPkgs, err := mapFilesToPackages(files)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(2)
}
if *verbose {
// Stable dump of direct packages
var dirs []string
for p := range directPkgs {
dirs = append(dirs, p)
}
sort.Strings(dirs)
fmt.Fprintln(os.Stderr, "Directly changed packages:")
if len(dirs) == 0 {
fmt.Fprintln(os.Stderr, " (none)")
} else {
for _, p := range dirs {
fmt.Fprintln(os.Stderr, " ", p)
}
}
}
switch *mode {
case "packages":
affected, err := computeAffectedPackages(directPkgs)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(2)
}
if *verbose {
var dbg []string
for p := range affected {
dbg = append(dbg, p)
}
sort.Strings(dbg)
fmt.Fprintln(os.Stderr, "Final affected packages:")
if len(dbg) == 0 {
fmt.Fprintln(os.Stderr, " (none)")
} else {
for _, p := range dbg {
fmt.Fprintln(os.Stderr, " ", p)
}
}
}
// Normalize and filter import paths:
// - Strip test variant suffixes like "pkg [pkg.test]"
// - Exclude e2e test packages (./test/e2e/...)
normalized := make(map[string]struct{})
for p := range affected {
// Trim Go test variant decorations that appear in `go list -test`
if idx := strings.Index(p, " ["); idx != -1 {
p = p[:idx]
}
// Exclude synthetic test packages like github.com/org/repo/pkg.name.test
if strings.HasSuffix(p, ".test") {
continue
}
if strings.Contains(p, "/test/e2e/") {
continue
}
if p != "" {
normalized[p] = struct{}{}
}
}
var list []string
for p := range normalized {
list = append(list, p)
}
sort.Strings(list)
for _, p := range list {
fmt.Println(p)
}
case "suites":
// Determine impacted suites by dependency mapping and direct e2e test changes,
// then print exact test names for those suites.
preflightRoot := "github.com/replicatedhq/troubleshoot/cmd/preflight"
supportRoot := "github.com/replicatedhq/troubleshoot/cmd/troubleshoot"
preflightDeps, err := listPackageWithDeps(preflightRoot)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(2)
}
supportDeps, err := listPackageWithDeps(supportRoot)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(2)
}
preflightHit := false
supportHit := false
// Track whether e2e test files were directly changed per suite and collect specific test names
changedPreflightTests := make(map[string]struct{})
changedSupportTests := make(map[string]struct{})
preflightE2EChangedNonGo := false
supportE2EChangedNonGo := false
for _, f := range files {
if strings.HasPrefix(f, "test/e2e/preflight/") {
if strings.HasSuffix(f, "_test.go") {
// Extract test names from just this file
b, err := os.ReadFile(f)
if err == nil { // ignore read errors; they will be caught later if needed
scanner := bufio.NewScanner(bytes.NewReader(b))
re := regexp.MustCompile(`^func\s+(Test[\w\d_]+)\s*\(`)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if m := re.FindStringSubmatch(line); m != nil {
changedPreflightTests[m[1]] = struct{}{}
}
}
}
preflightHit = true
} else {
// Non-go change under preflight e2e; run whole suite
preflightE2EChangedNonGo = true
preflightHit = true
}
}
if strings.HasPrefix(f, "test/e2e/support-bundle/") {
if strings.HasSuffix(f, "_test.go") {
b, err := os.ReadFile(f)
if err == nil {
scanner := bufio.NewScanner(bytes.NewReader(b))
re := regexp.MustCompile(`^func\s+(Test[\w\d_]+)\s*\(`)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if m := re.FindStringSubmatch(line); m != nil {
changedSupportTests[m[1]] = struct{}{}
}
}
}
supportHit = true
} else {
supportE2EChangedNonGo = true
supportHit = true
}
}
}
for changed := range directPkgs {
if !preflightHit {
if _, ok := preflightDeps[changed]; ok {
preflightHit = true
}
}
if !supportHit {
if _, ok := supportDeps[changed]; ok {
supportHit = true
}
}
if preflightHit && supportHit {
break
}
}
if *verbose {
fmt.Fprintln(os.Stderr, "E2E suite impact:")
fmt.Fprintf(os.Stderr, " preflight: %v\n", preflightHit)
fmt.Fprintf(os.Stderr, " support-bundle: %v\n", supportHit)
}
// If module files or CI/detector changed, conservatively select all tests for both suites.
if moduleChanged || ciChanged {
preTests, err := listTestFunctions("test/e2e/preflight")
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(2)
}
for _, tname := range preTests {
fmt.Printf("preflight:%s\n", tname)
}
sbTests, err := listTestFunctions("test/e2e/support-bundle")
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(2)
}
for _, tname := range sbTests {
fmt.Printf("support-bundle:%s\n", tname)
}
return
}
// Collect tests for impacted suites and print as `<suite>:<TestName>`
if preflightHit || supportHit {
if preflightHit {
toPrint := make(map[string]struct{})
if preflightE2EChangedNonGo || len(changedPreflightTests) == 0 {
// Run full suite if e2e non-go assets changed or no specific test names collected
preTests, err := listTestFunctions("test/e2e/preflight")
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(2)
}
for _, t := range preTests {
toPrint[t] = struct{}{}
}
} else {
for t := range changedPreflightTests {
toPrint[t] = struct{}{}
}
}
var list []string
for t := range toPrint {
list = append(list, t)
}
sort.Strings(list)
for _, tname := range list {
fmt.Printf("preflight:%s\n", tname)
}
}
if supportHit {
toPrint := make(map[string]struct{})
if supportE2EChangedNonGo || len(changedSupportTests) == 0 {
sbTests, err := listTestFunctions("test/e2e/support-bundle")
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(2)
}
for _, t := range sbTests {
toPrint[t] = struct{}{}
}
} else {
for t := range changedSupportTests {
toPrint[t] = struct{}{}
}
}
var list []string
for t := range toPrint {
list = append(list, t)
}
sort.Strings(list)
for _, tname := range list {
fmt.Printf("support-bundle:%s\n", tname)
}
}
}
default:
fmt.Fprintln(os.Stderr, "unknown mode; use 'packages' or 'suites'")
os.Exit(2)
}
}
+67
View File
@@ -0,0 +1,67 @@
#!/usr/bin/env bash
set -euo pipefail
# 0) Preconditions (one-time)
export PATH="$(go env GOPATH)/bin:$PATH"
go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.19.0 >/dev/null
go install k8s.io/code-generator/cmd/client-gen@v0.34.0 >/dev/null
git fetch origin main --depth=1 || true
# 1) Determine changed files source: explicit args or git base diff
if [ "$#" -gt 0 ]; then
# Treat provided paths as changed files
CHANGED_CSV=$(printf "%s," "$@" | sed 's/,$//')
echo "Simulating changes in: $CHANGED_CSV"
PKGS="$(go run ./scripts/affected-packages.go -changed-files "${CHANGED_CSV}")"
E2E_OUT="$(go run ./scripts/affected-packages.go -mode=suites -changed-files "${CHANGED_CSV}")"
else
# Compute base (robust to unrelated histories)
BASE="$(git merge-base HEAD origin/main 2>/dev/null || true)"
if [ -z "${BASE}" ]; then
echo "No merge-base with origin/main → running full set"
PKGS="./..."
E2E_OUT="$(go run ./scripts/affected-packages.go -mode=suites -changed-files go.mod || true)"
else
PKGS="$(go run ./scripts/affected-packages.go -base "${BASE}")"
E2E_OUT="$(go run ./scripts/affected-packages.go -mode=suites -base "${BASE}")"
fi
fi
# 2) Print what will run
echo "=== Affected unit packages ==="
if [ -n "${PKGS}" ]; then echo "${PKGS}"; else echo "(none)"; fi
echo
echo "=== Affected e2e tests ==="
if [ -n "${E2E_OUT}" ]; then echo "${E2E_OUT}"; else echo "(none)"; fi
echo
# 3) Unit tests via Makefile (inherits required build tags)
if [ "${PKGS}" = "./..." ]; then
echo "Running: make test (all)"
make test
elif [ -n "${PKGS}" ]; then
echo "Running: make test-packages for affected pkgs"
PACKAGES="$(echo "${PKGS}" | xargs)" make test-packages
else
echo "No affected unit packages"
fi
# 4) E2E tests via Makefile (filtered by regex)
PRE="$(echo "${E2E_OUT}" | awk -F: '$1=="preflight"{print $2}' | paste -sd'|' -)"
SB="$( echo "${E2E_OUT}" | awk -F: '$1=="support-bundle"{print $2}' | paste -sd'|' -)"
# Use direct go test with the same build tags as the Makefile to avoid RUN quoting issues locally
BUILD_TAGS='netgo containers_image_ostree_stub exclude_graphdriver_devicemapper exclude_graphdriver_btrfs containers_image_openpgp'
overall=0
if [ -n "${PRE}" ]; then
echo "Running preflight e2e: ${PRE}"
go test -tags "${BUILD_TAGS}" -installsuffix netgo -v -count=1 ./test/e2e/preflight -run "^(${PRE})$" || overall=1
fi
if [ -n "${SB}" ]; then
echo "Running support-bundle e2e: ${SB}"
go test -tags "${BUILD_TAGS}" -installsuffix netgo -v -count=1 ./test/e2e/support-bundle -run "^(${SB})$" || overall=1
fi
exit $overall
+181
View File
@@ -0,0 +1,181 @@
#!/bin/bash
# Comprehensive test for affected test detection
# Tests various code change scenarios to ensure correct suite detection
set -e
# Colors
GREEN='\033[0;32m'
RED='\033[0;31m'
BLUE='\033[0;34m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color
TESTS_PASSED=0
TESTS_FAILED=0
echo "========================================"
echo "Affected Test Detection Validation"
echo "========================================"
echo ""
# Helper function to run test
run_test() {
local test_name="$1"
local test_file="$2"
local expected_suites="$3"
echo -e "${BLUE}Test: $test_name${NC}"
echo "File: $test_file"
echo "Expected: $expected_suites"
# Get affected tests from explicit changed files (no git required); detector prints <suite>:<TestName>
local detector_output=$(go run ./scripts/affected-packages.go -mode=suites -changed-files "$test_file" 2>/dev/null)
# Derive suites from prefixes for comparison
local actual_suites=$(echo "$detector_output" | cut -d':' -f1 | grep -v '^$' | sort | uniq | tr '\n' ' ' | xargs)
# Compare results
if [ "$actual_suites" = "$expected_suites" ]; then
echo -e "${GREEN}✓ PASS${NC} - Got: $actual_suites"
if [ -n "$detector_output" ]; then
echo "Tests:" && echo "$detector_output" | sed 's/^/ - /'
fi
TESTS_PASSED=$((TESTS_PASSED + 1))
else
echo -e "${RED}✗ FAIL${NC} - Got: '$actual_suites', Expected: '$expected_suites'"
if [ -n "$detector_output" ]; then
echo "Tests:" && echo "$detector_output" | sed 's/^/ - /'
fi
TESTS_FAILED=$((TESTS_FAILED + 1))
fi
echo ""
}
# Test 1: Preflight-only package (should only trigger preflight)
run_test "Preflight-only package change" \
"pkg/preflight/run.go" \
"preflight"
# Test 2: Support-bundle-only package
run_test "Support-bundle-only package change" \
"pkg/supportbundle/supportbundle.go" \
"support-bundle"
# Test 3: Shared package - collect
run_test "Shared package (collect) change" \
"pkg/collect/run.go" \
"preflight support-bundle"
# Test 4: Shared package - analyze
run_test "Shared package (analyze) change" \
"pkg/analyze/analyzer.go" \
"preflight support-bundle"
# Test 5: Shared package - k8sutil
run_test "Shared package (k8sutil) change" \
"pkg/k8sutil/config.go" \
"preflight support-bundle"
# Test 6: Shared package - convert
run_test "Shared package (convert) change" \
"pkg/convert/output.go" \
"preflight support-bundle"
# Test 7: Shared package - redact (another shared one)
run_test "Shared package (redact) change" \
"pkg/redact/redact.go" \
"preflight support-bundle"
# Test 8: Preflight command (should only trigger preflight)
run_test "Preflight command change" \
"cmd/preflight/main.go" \
"preflight"
# Test 9: Support-bundle types (support-bundle only package)
run_test "Support-bundle types change" \
"pkg/supportbundle/types/types.go" \
"support-bundle"
# Test 10: Workflow file (should not trigger e2e)
echo -e "${BLUE}Test: Workflow file change (should trigger nothing)${NC}"
echo "File: .github/workflows/affected-tests.yml"
echo "Expected: (no suites)"
detector_output=$(go run ./scripts/affected-packages.go -mode=suites -changed-files ".github/workflows/affected-tests.yml" 2>/dev/null)
actual_suites=$(echo "$detector_output" | cut -d':' -f1 | grep -v '^$' | sort | uniq | tr '\n' ' ' | xargs)
if [ -z "$actual_suites" ]; then
echo -e "${GREEN}✓ PASS${NC} - No suites affected (as expected)"
TESTS_PASSED=$((TESTS_PASSED + 1))
else
echo -e "${RED}✗ FAIL${NC} - Got: '$actual_suites', Expected: (empty)"
TESTS_FAILED=$((TESTS_FAILED + 1))
fi
echo ""
# Test 11: go.mod change (should trigger all)
echo -e "${BLUE}Test: go.mod change (should trigger all suites)${NC}"
echo "File: go.mod"
echo "Expected: preflight support-bundle"
detector_output=$(go run ./scripts/affected-packages.go -mode=suites -changed-files "go.mod" 2>/dev/null)
actual_suites=$(echo "$detector_output" | cut -d':' -f1 | grep -v '^$' | sort | uniq | tr '\n' ' ' | xargs)
if [ "$actual_suites" = "preflight support-bundle" ]; then
echo -e "${GREEN}✓ PASS${NC} - Got: $actual_suites"
TESTS_PASSED=$((TESTS_PASSED + 1))
else
echo -e "${RED}✗ FAIL${NC} - Got: '$actual_suites', Expected: 'preflight support-bundle'"
TESTS_FAILED=$((TESTS_FAILED + 1))
fi
echo ""
# Test 12: Multiple files across different areas
echo -e "${BLUE}Test: Multiple file changes (support-bundle + shared)${NC}"
echo "Files: pkg/supportbundle/supportbundle.go + pkg/collect/run.go"
echo "Expected: preflight support-bundle"
detector_output=$(go run ./scripts/affected-packages.go -mode=suites -changed-files "pkg/supportbundle/supportbundle.go,pkg/collect/run.go" 2>/dev/null)
actual_suites=$(echo "$detector_output" | cut -d':' -f1 | grep -v '^$' | sort | uniq | tr '\n' ' ' | xargs)
if [ "$actual_suites" = "preflight support-bundle" ]; then
echo -e "${GREEN}✓ PASS${NC} - Got: $actual_suites"
TESTS_PASSED=$((TESTS_PASSED + 1))
else
echo -e "${RED}✗ FAIL${NC} - Got: '$actual_suites', Expected: 'preflight support-bundle'"
TESTS_FAILED=$((TESTS_FAILED + 1))
fi
echo ""
# Test 13: README change (should not trigger e2e)
echo -e "${BLUE}Test: Documentation change (should trigger nothing)${NC}"
echo "File: README.md"
echo "Expected: (no suites)"
detector_output=$(go run ./scripts/affected-packages.go -mode=suites -changed-files "README.md" 2>/dev/null)
actual_suites=$(echo "$detector_output" | cut -d':' -f1 | grep -v '^$' | sort | uniq | tr '\n' ' ' | xargs)
if [ -z "$actual_suites" ]; then
echo -e "${GREEN}✓ PASS${NC} - No suites affected (as expected)"
TESTS_PASSED=$((TESTS_PASSED + 1))
else
echo -e "${RED}✗ FAIL${NC} - Got: '$actual_suites', Expected: (empty)"
TESTS_FAILED=$((TESTS_FAILED + 1))
fi
echo ""
# Summary
echo "========================================"
echo -e "${GREEN}Tests Passed: $TESTS_PASSED${NC}"
echo -e "${RED}Tests Failed: $TESTS_FAILED${NC}"
echo "========================================"
if [ $TESTS_FAILED -eq 0 ]; then
echo -e "${GREEN}✓ All tests passed!${NC}"
exit 0
else
echo -e "${RED}✗ Some tests failed${NC}"
exit 1
fi