Compare commits

..
53 Commits
Author SHA1 Message Date
Noah Campbell 3cd2b9a87b doesnt harcode apiVersion line when looking and figures out which apiVersion to give if none is there 2025-10-13 13:20:01 -05:00
Noah Campbell 898a59fe40 improved --fix capabilities 2025-10-07 16:58:18 -07:00
Noah Campbell e71bf87cc2 added lint subcommand 2025-10-07 16:19:01 -07:00
Noah Campbell 774fd8d935 handle empty structural_compare rules 2025-10-07 09:08:02 -07:00
Noah Campbell 1d4315ced6 simplify comparison to detect code regressions only 2025-10-07 08:58:03 -07:00
Noah Campbell b6b2bf8483 handle node-specific files and multiple report arguments 2025-10-07 08:47:05 -07:00
Noah Campbell ac9a4c9c5b create test/output directory 2025-10-07 08:34:35 -07:00
Noah Campbell c532928cd2 Update regression-test.yaml 2025-10-07 08:21:52 -07:00
Noah Campbell 4b04098689 Update regression-test.yaml 2025-10-07 08:16:38 -07:00
Noah Campbell b3369c2b3e Update regression-test.yaml 2025-10-07 08:13:18 -07:00
Noah Campbell fbfd6c99fe Add regression test suite to github actions 2025-10-07 08:10:32 -07:00
Noah Campbell 12f8bc3daa Improved error handling for support bundles and redactors for windows (#1878)
* improved error handling and window locking

* Delete all-windows-collectors.yaml

* addressing bugbot concerns

* Update host_tcpportstatus.go

* Update redact.go
2025-10-06 14:32:29 -07:00
Noah Campbell b4e3ed278a Removes silent failing (#1877)
* preserves stdout and stderr from collectors

* Delete eliminate-silent-failures.md

* Update host_kernel_modules_test.go

* added error logs when a collector fails to start

* Update host_filesystem_performance_linux.go

* fixed error saving logic inconsistency

* Update collect.go
2025-10-04 15:14:10 -07:00
Benjamin Yang c67775c1a0 feat: clean tokenization system implementation (#1874)
Core tokenization functionality with minimal file changes:

 Core Features:
- Intelligent tokenization engine (tokenizer.go)
- Context-aware secret classification (PASSWORD, APIKEY, DATABASE, etc.)
- Cross-file correlation with deterministic HMAC-SHA256 tokens
- Optional encrypted mapping for token→original value resolution

 Integration:
- CLI flags: --tokenize, --redaction-map, --encrypt-redaction-map
- Updated all redactor types: literal, single-line, multi-line, YAML
- Support bundle integration with auto-upload compatibility
- Backward compatibility: preserves ***HIDDEN*** when disabled

 Production Ready:
- Only 11 essential files (vs 31 in original PR)
- No excessive test files or documentation
- Clean build, all functionality verified
- Maintains existing redaction behavior by default

Token format: ***TOKEN_<TYPE>_<HASH>*** (e.g., ***TOKEN_PASSWORD_A1B2C3***)
2025-09-30 15:04:23 -05:00
83e6cffe5f Cron job clean (#1862)
* created roadmap and yaml claude agent

* Update roadmap.md

* chore(deps): bump sigstore/cosign-installer from 3.9.2 to 3.10.0 (#1857)

Bumps [sigstore/cosign-installer](https://github.com/sigstore/cosign-installer) from 3.9.2 to 3.10.0.
- [Release notes](https://github.com/sigstore/cosign-installer/releases)
- [Commits](https://github.com/sigstore/cosign-installer/compare/v3.9.2...v3.10.0)

---
updated-dependencies:
- dependency-name: sigstore/cosign-installer
  dependency-version: 3.10.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump the security group with 2 updates (#1858)

Bumps the security group with 2 updates: [github.com/vmware-tanzu/velero](https://github.com/vmware-tanzu/velero) and [helm.sh/helm/v3](https://github.com/helm/helm).


Updates `github.com/vmware-tanzu/velero` from 1.16.2 to 1.17.0
- [Release notes](https://github.com/vmware-tanzu/velero/releases)
- [Changelog](https://github.com/vmware-tanzu/velero/blob/main/CHANGELOG.md)
- [Commits](https://github.com/vmware-tanzu/velero/compare/v1.16.2...v1.17.0)

Updates `helm.sh/helm/v3` from 3.18.6 to 3.19.0
- [Release notes](https://github.com/helm/helm/releases)
- [Commits](https://github.com/helm/helm/compare/v3.18.6...v3.19.0)

---
updated-dependencies:
- dependency-name: github.com/vmware-tanzu/velero
  dependency-version: 1.17.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: security
- dependency-name: helm.sh/helm/v3
  dependency-version: 3.19.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: security
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump helm.sh/helm/v3 from 3.18.6 to 3.19.0 in /examples/sdk/helm-template in the security group (#1859)

chore(deps): bump helm.sh/helm/v3

Bumps the security group in /examples/sdk/helm-template with 1 update: [helm.sh/helm/v3](https://github.com/helm/helm).


Updates `helm.sh/helm/v3` from 3.18.6 to 3.19.0
- [Release notes](https://github.com/helm/helm/releases)
- [Commits](https://github.com/helm/helm/compare/v3.18.6...v3.19.0)

---
updated-dependencies:
- dependency-name: helm.sh/helm/v3
  dependency-version: 3.19.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: security
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add cron job support bundle scheduler

Complete implementation with K8s integration:
- pkg/schedule/job.go: Job management and persistence
- pkg/schedule/daemon.go: Real-time scheduler daemon
- pkg/schedule/cli.go: CLI commands (create, list, delete, daemon)
- pkg/schedule/schedule_test.go: Comprehensive unit tests
- cmd/troubleshoot/cli/root.go: CLI integration

* fixing bugbot

* Fix all bugbot errors: auto-update stability, job cooldown timing, and daemon execution

* Deleting Agent

* removed unused flags

* fixing auto-upload

* fixing markdown files

* namespace not required flag for auto collectors to work

* loosened cron job validation

* writes logs to logfile

* fix: resolve autoFromEnv variable scoping issue for CI

- Ensure autoFromEnv variable and its usage are in correct scope
- Fix build errors: declared and not used / undefined variable
- All functionality preserved and tested locally
- Force add to override gitignore

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Noah Campbell <noah.edward.campbell@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-30 14:11:15 -05:00
Benjamin Yang 84dc8157b9 Auto-Collect (#1867)
* Fix auto-collector missing files issue

- Add KOTS-aware detection for diagnostic files
- Replace silent RBAC filtering with user warnings
- Enhance error file collection for troubleshooting
- Achieve parity with traditional support bundles

Resolves issue where auto-collector was missing:
- KOTS diagnostic files (now 4 vs 3)
- ConfigMaps (now 6 vs 6)
- Maintains superior log collection (24 vs 0)

Final result: [SUCCESS] comprehensive collection achieved

* fixing bugbog

* fix: resolve production readiness issues in auto-collect branch

1. Fix diff test expectations (lines should have newlines for difflib consistency)
2. Fix preflight tests to use existing v1beta3 example file
3. Fix autodiscovery test context parameter (function signature update)

Resolves TestReadLinesFromReader and preflight v1beta3 test failures

* fix: resolve autodiscovery tests and cursor bot image matching issues

1. Fix cursor bot image matching bug in isKotsadmImage:
   - Replace flawed prefix matching with proper image component detection
   - Handle private registries correctly (registry.company.com/kotsadm/kotsadm:v1.0.0)
   - Prevent false positives with proper delimiter checking
   - Add helper functions: containsImageComponent, splitImagePath, removeTagAndDigest

2. Fix autodiscovery test failures:
   - Add TestMode flag to DiscoveryOptions to control KOTS diagnostic collection
   - Tests use TestMode=true to get only foundational collectors (no KOTS diagnostics)
   - Preserves production behavior while enabling clean testing

Resolves failing TestDiscoverer_DiscoverFoundational tests and cursor bot issues
2025-09-30 12:26:10 -05:00
Benjamin YangandNoah Campbell a8fb5210f4 Advanced analysis clean (#1868)
* created roadmap and yaml claude agent

* Update roadmap.md

* feat: Clean advanced analysis implementation - core agents, engine, artifacts

* Remove unrelated files - keep only advanced analysis implementation

* fix: Fix goroutine leak in hosted agent rate limiter

- Added stop channel and stopped flag to RateLimiter struct
- Modified replenishTokens to listen for stop signal and exit cleanly
- Added Stop() method to gracefully shutdown rate limiter
- Added Stop() method to HostedAgent to cleanup rate limiter on shutdown

Fixes cursor bot issue: Rate Limiter Goroutine Leak

* fix: Fix analyzer config and model validation bugs

Bug 1: Analyzer Config Missing File Path
- Added filePath to DeploymentStatus analyzer config in convertAnalyzerToSpec
- Sets namespace-specific path (cluster-resources/deployments/{namespace}.json)
- Falls back to generic path (cluster-resources/deployments.json) if no namespace
- Fixes LocalAgent.analyzeDeploymentStatus backward compatibility

Bug 2: HealthCheck Fails Model Validation
- Changed Ollama model validation from prefix match to exact match
- Prevents false positives where llama2:13b would match request for llama2:7b
- Ensures agent only reports healthy when exact model is available

Both fixes address cursor bot reported issues and maintain backward compatibility.

* fixing lint errors

* fixing lint errors

* adding CLI flags

* fix: resolve linting errors for CI

- Remove unnecessary nil check in host_kernel_configs.go (len() for nil slices is zero)
- Remove unnecessary fmt.Sprintf() calls in ceph.go for static strings
- Apply go fmt formatting fixes

Fixes failing lint CI check

* fix: resolve CI failures in build-test workflow and Ollama tests

1. Fix GitHub Actions workflow logic error:
   - Replace problematic contains() expression with explicit job result checks
   - Properly handle failure and cancelled states for each job
   - Prevents false positive failures in success summary job

2. Fix Ollama agent parseLLMResponse panics:
   - Add proper error handling for malformed JSON in LLM responses
   - Return error when JSON is found but invalid (instead of silent fallback)
   - Add error when no meaningful content can be parsed from response
   - Prevents nil pointer dereference in test assertions

Fixes failing build-test/success and build-test/test CI checks

* fix: resolve all CI failures and cursor bot issues

1. Fix disable-ollama flag logic bug:
   - Remove disable-ollama from advanced analysis trigger condition
   - Prevents unintended advanced analysis mode when no agents registered
   - Allows proper fallback to legacy analysis

2. Fix diff test consistency:
   - Update test expectations to match function behavior (lines with newlines)
   - Ensures consistency between streaming and non-streaming diff paths

3. Fix Ollama agent error handling:
   - Add proper error return for malformed JSON in LLM responses
   - Add meaningful content validation for markdown parsing
   - Prevents nil pointer panics in test assertions

4. Fix analysis engine mock agent:
   - Mock agent now processes and returns results for all provided analyzers
   - Fixes test expectation mismatch (expected 8 results, got 1)

Resolves all failing CI checks: lint, test, and success workflow logic

---------

Co-authored-by: Noah Campbell <noah.edward.campbell@gmail.com>
2025-09-30 11:44:46 -05:00
Marc Campbell d3af4edd31 bump example 2025-09-29 21:34:01 -07:00
Marc Campbell b9dc00cc6c removing risc 2025-09-29 21:19:11 -07:00
Marc Campbell 8990c4b06e some more goreleaser/homebrew stuffs 2025-09-29 21:11:03 -07:00
Marc Campbell e03fa0a046 publish to directory: 2025-09-29 21:00:21 -07:00
Marc Campbell 758d454434 prerelease auto and more 2025-09-29 20:48:28 -07:00
Marc Campbell f0d0eda060 larger 2025-09-29 19:57:57 -07:00
Marc Campbell 4eea31d916 goreleaser 2025-09-29 19:55:14 -07:00
Marc Campbell cfea35e7c0 fix syntax 2025-09-29 19:47:50 -07:00
Marc Campbell 630aa631ee fix syntax 2025-09-29 19:44:50 -07:00
Marc Campbell b61728a11f dropping goreleaser 2025-09-29 19:20:17 -07:00
Marc Campbell 252f9650c9 larger runner for release 2025-09-29 19:02:49 -07:00
Marc Campbell 857a028585 add debug logging 2025-09-29 18:54:17 -07:00
Marc Campbell 6e8704d6f1 remove this check 2025-09-29 18:37:28 -07:00
Marc Campbell c8c246a809 remove collect binary and risc binary 2025-09-29 18:36:36 -07:00
Marc Campbell a41958c173 bump goreleaser to v2 2025-09-29 18:13:11 -07:00
Benjamin YangandNoah Campbell 17518349ff Fix/exec textanalyze path clean (#1865)
* created roadmap and yaml claude agent

* Update roadmap.md

* Fix textAnalyze analyzer to auto-match exec collector nested paths

- Auto-detect exec output files (*-stdout.txt, *-stderr.txt, *-errors.json)
- Convert simple filenames to wildcard patterns automatically
- Preserve existing wildcard patterns
- Fixes 'No matching file' errors for exec + textAnalyze workflows

---------

Co-authored-by: Noah Campbell <noah.edward.campbell@gmail.com>
2025-09-29 17:49:54 -07:00
Benjamin Yang 266647ef2f Fixing testing errors (#1871)
fix: resolve failing unit tests and diff consistency in v1beta3

- Fix readLinesFromReader to return lines WITH newlines (like difflib.SplitLines)
- Update test expectations to match correct function behavior with newlines
- This ensures consistency between streaming and non-streaming diff paths
- Fix timeout test by changing from 10ms to 500ms to eliminate flaky failures

Fixes TestReadLinesFromReader and Test_loadSupportBundleSpecsFromURIs_TimeoutError
Resolves diff output inconsistency between code paths
2025-09-29 17:48:51 -07:00
Marc Campbell 3062dd8d5c adding brew (#1872) 2025-09-29 15:01:22 -07:00
Marc Campbell 91db72b484 removing coverage reporting 2025-09-29 14:36:51 -07:00
Marc Campbell 5f0835f7d0 add a v1beta3 fixture 2025-09-29 11:02:51 -07:00
Marc Campbell 9af9845fb9 removing unused examples 2025-09-29 10:50:00 -07:00
Marc Campbell 007ef32e77 fmt: 2025-09-29 10:43:51 -07:00
Marc Campbell d9904d9632 update ci for v1beta3 (#1870) 2025-09-29 10:40:04 -07:00
Marc Campbell 074fee81f7 V1beta3 cleanup (#1869)
* moving some files around

* more cleanup

* removing more unused
2025-09-29 10:10:41 -07:00
Noah Campbell 7bd7eca528 does not auto update for package managers (#1850) 2025-09-18 12:56:12 -05:00
Noah Campbell b1830ff515 moved v1beta3 examples to proper directory 2025-09-17 17:38:26 -05:00
Noah Campbell 885819ea9a Auto support bundle upload (#1860)
* basic auto uploading support bundles

* added upload command

* added default vendor endpoint

* added auth system from replicated cli

* fixed case sensitivity issue in YAML parsing

* support bundle uploads for end customers

* app slug flag and detection without licenseID
2025-09-17 12:59:41 -05:00
Noah Campbell 20b0c5ed06 fixed gitignore issue 2025-09-17 11:02:53 -05:00
Noah Campbell 71067edb41 Updated yaml spec (#1863)
* v1beta3 spec can be read by preflight

* added test files for ease of testing

* v1beta3 renderer fixes
2025-09-17 10:38:11 -05:00
Noah Campbell d0584a4d4d Updated yaml spec (#1851)
* v1beta3 spec can be read by preflight

* added test files for ease of testing

* updated v1beta3 guide doc and added tests

* fixed not removing tmp files from v1beta3 processing

* created v1beta2 to v1beta3 converter
2025-09-16 15:43:46 -05:00
Noah Campbell acc1aad843 Preflight docs and template subcommands (#1847)
* Added docs and template subcommands with test files

* uses helm templating preflight yaml files

* merge doc requirements for multiple inputs

* Helm aware rendering and markdown output

* v1beta3 yaml structure better mirrors beta2

* Update sample-preflight-templated.yaml

* Added docs and template subcommands with test files

* uses helm templating preflight yaml files

* merge doc requirements for multiple inputs

* Helm aware rendering and markdown output

* v1beta3 yaml structure better mirrors beta2

* Update sample-preflight-templated.yaml

* Added/updated documentation on subcommands

* Update docs.go

* commit to trigger actions
2025-09-16 14:12:09 -05:00
Noah Campbell 8027e273e4 Support bundle diff (#1855)
implemented support bundle diff command
2025-09-15 13:20:35 -05:00
Marc Campbell 3b9d280a95 resetting make targets and github workflows to support v1beta3 releas… (#1853)
* resetting make targets and github workflows to support v1beta3 release later

* removing generate

* remove

* removing

* removing
2025-09-15 09:15:53 -07:00
Benjamin YangandBenjamin Yang 74811c669b Auto-collectors: foundational discovery, image metadata, CLI integrat… (#1845)
* Auto-collectors: foundational discovery, image metadata, CLI integration; reset PRD markers

* Address PR review feedback

- Implement missing namespace exclude patterns functionality
- Fix image facts collector to use empty Data field instead of static string
- Correct APIVersion to use troubleshoot.sh/v1beta2 consistently

* Fix bug bot issues: API parsing, EOF error, and API group corrections

- Fix RBAC API parsing errors in rbac_checker.go (getAPIGroup/getAPIVersion functions)
- Fix FakeReader EOF error to use standard io.EOF instead of custom error
- Fix incorrect API group from troubleshoot.sh to troubleshoot.replicated.com in run.go

These changes address the issues identified by the bug bot and ensure proper
interface compliance and consistent API group usage.

* Fix multiple bug bot issues

- Fix RBAC API parsing errors in rbac_checker.go (getAPIGroup/getAPIVersion functions)
- Fix FakeReader EOF error to use standard io.EOF instead of custom error
- Fix incorrect API group from troubleshoot.sh to troubleshoot.replicated.com in run.go
- Fix image facts collector Data field to contain structured JSON instead of static strings

These changes address all issues identified by the bug bot and ensure proper
interface compliance, consistent API usage, and meaningful data fields.

* Update auto_discovery.go

* Fix TODO comments in Auto-collector section

Fixed 3 of 4 TODOs as requested in PR review:

1. pkg/collect/images/registry_client.go (line 46):
   - Implement custom CA certificate loading
   - Add x509 import and certificate parsing logic
   - Enables image collection from private registries with custom CAs

2. cmd/troubleshoot/cli/diff.go (line 209):
   - Implement bundle file count functionality
   - Add tar/gzip imports and getFileCountFromBundle() function
   - Properly counts files in support bundle archives (.gz/.tgz)

3. cmd/troubleshoot/cli/run.go (line 338):
   - Replace TODO with clarifying comment about RemoteCollectors usage
   - Confirmed RemoteCollectors are still actively used in preflights

The 4th TODO (diff.go line 196) is left as-is since it's explicitly marked
as Phase 4 future work (Support Bundle Differencing implementation).

Addresses PR review feedback about unimplemented TODO comments.

---------

Co-authored-by: Benjamin Yang <benjaminyang@Benjamins-MacBook-Pro.local>
2025-09-12 13:44:26 -06:00
Noah Campbell a1d1a12c1a Auto updater (#1849)
* added auto updater

* updated docs

* commit to trigger actions
2025-09-12 12:30:57 -06:00
Noah Campbell 050dc8499f Change workflow branch from 'main' to 'v1beta3' 2025-09-12 11:08:16 -05:00
55 changed files with 901 additions and 6003 deletions
-163
View File
@@ -1,163 +0,0 @@
name: build-test-deploy
on:
pull_request:
types:
- opened
- reopened
- synchronize
- ready_for_review
branches:
- main
push:
branches:
- "main"
tags:
- "v*.*.*"
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
fail_if_pull_request_is_draft:
if: github.event.pull_request.draft == true
runs-on: ubuntu-latest
steps:
- name: Fails in order to indicate that pull request needs to be marked as ready to review and unit tests workflow needs to pass.
run: exit 1
tidy-check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/setup-go@v6
with:
go-version-file: 'go.mod'
- run: make tidy-diff
test-integration:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/setup-go@v6
with:
go-version-file: 'go.mod'
- uses: replicatedhq/action-k3s@main
id: k3s
with:
version: v1.31.2-k3s1
# test-integration includes unit tests
- run: make test-integration
compile-preflight:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/setup-go@v6
with:
go-version-file: 'go.mod'
- run: make generate preflight
- uses: actions/upload-artifact@v4
with:
name: preflight
path: bin/preflight
validate-preflight-e2e:
runs-on: ubuntu-latest
needs: compile-preflight
steps:
- uses: actions/checkout@v5
- uses: replicatedhq/action-k3s@main
id: k3s
with:
version: v1.31.2-k3s1
- name: Download preflight binary
uses: actions/download-artifact@v5
with:
name: preflight
path: bin/
- run: chmod +x bin/preflight
- run: make preflight-e2e-test
compile-supportbundle:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/setup-go@v6
with:
go-version-file: 'go.mod'
- run: make generate support-bundle
- uses: actions/upload-artifact@v4
with:
name: support-bundle
path: bin/support-bundle
validate-supportbundle-e2e:
runs-on: ubuntu-latest
needs: compile-supportbundle
steps:
- uses: actions/checkout@v5
- uses: replicatedhq/action-k3s@main
id: k3s
with:
version: v1.31.2-k3s1
- name: Download support bundle binary
uses: actions/download-artifact@v5
with:
name: support-bundle
path: bin/
- run: chmod +x bin/support-bundle
- run: make support-bundle-e2e-test
# Additional e2e tests for support bundle that run in Go, these create a Kind cluster
validate-supportbundle-e2e-go:
runs-on: ubuntu-latest
needs: compile-supportbundle
steps:
- uses: actions/checkout@v5
- name: Download support bundle binary
uses: actions/download-artifact@v5
with:
name: support-bundle
path: bin/
- run: chmod +x bin/support-bundle
- name: Download preflight binary
uses: actions/download-artifact@v5
with:
name: preflight
path: bin/
- run: chmod +x bin/preflight
- run: make support-bundle-e2e-go-test
# summary jobs, these jobs will only run if all the other jobs have succeeded
validate-pr-tests:
runs-on: ubuntu-latest
needs:
- tidy-check
- test-integration
- validate-preflight-e2e
- validate-supportbundle-e2e
- validate-supportbundle-e2e-go
steps:
- run: echo "All PR tests passed"
# this job will validate that the validation did not fail and that all pr-tests succeed
# it is used for the github branch protection rule
validate-success:
runs-on: ubuntu-latest
needs:
- validate-pr-tests
if: always()
steps:
# https://docs.github.com/en/actions/learn-github-actions/contexts#needs-context
# if the validate-pr-tests job was not successful, this job will fail
- name: fail if validate-pr-tests job was not successful
if: needs.validate-pr-tests.result != 'success'
run: exit 1
# if the validate-pr-tests job was successful, this job will succeed
- name: succeed if validate-pr-tests job succeeded
if: needs.validate-pr-tests.result == 'success'
run: echo "Validation succeeded"
+2 -2
View File
@@ -3,9 +3,9 @@ name: build-test
on:
pull_request:
types: [opened, reopened, synchronize, ready_for_review]
branches: [main]
branches: [v1beta3]
push:
branches: [main]
branches: [v1beta3]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
+4 -4
View File
@@ -50,9 +50,9 @@ sbom/
!testdata/supportbundle/*.tar.gz
!test/baselines/**/baseline.tar.gz
# Ignore built binaries (use / prefix to avoid catching source files)
/troubleshoot
/troubleshoot-test
# Ignore built binaries
troubleshoot
troubleshoot-test
cmd/troubleshoot/troubleshoot
cmd/*/troubleshoot
/support-bundle
support-bundle
+4 -190
View File
@@ -38,17 +38,13 @@ BUILDTAGS = "netgo containers_image_ostree_stub exclude_graphdriver_devicemapper
BUILDFLAGS = -tags ${BUILDTAGS} -installsuffix netgo
BUILDPATHS = ./pkg/... ./cmd/... ./internal/...
E2EPATHS = ./test/e2e/...
TESTFLAGS ?= -v -coverprofile cover.out
TESTFLAGS ?=
.DEFAULT_GOAL := all
all: clean build test
.PHONY: ffi
ffi: fmt vet
go build ${BUILDFLAGS} ${LDFLAGS} -o bin/troubleshoot.so -buildmode=c-shared ffi/main.go
.PHONY: test
test: generate fmt vet
test: fmt vet
if [ -n $(RUN) ]; then \
go test ${BUILDFLAGS} ${BUILDPATHS} ${TESTFLAGS} -run $(RUN); \
else \
@@ -58,17 +54,13 @@ test: generate fmt vet
# Go tests that require a K8s instance
# TODOLATER: merge with test, so we get unified coverage reports? it'll add 21~sec to the test job though...
.PHONY: test-integration
test-integration: generate fmt vet
test-integration: fmt vet
go test -v --tags="integration exclude_graphdriver_devicemapper exclude_graphdriver_btrfs" ${BUILDPATHS}
.PHONY: preflight-e2e-test
preflight-e2e-test:
./test/validate-preflight-e2e.sh
.PHONY: run-examples
run-examples:
./test/run-examples.sh
.PHONY: support-bundle-e2e-test
support-bundle-e2e-test:
./test/validate-support-bundle-e2e.sh
@@ -83,31 +75,19 @@ support-bundle-e2e-go-test:
rebuild: clean build
# Build all binaries in parallel ( -j )
build: tidy
@echo "Build cli binaries"
$(MAKE) -j bin/support-bundle bin/preflight
$(MAKE) bin/support-bundle bin/preflight
.PHONY: clean
clean:
@rm -f bin/analyze
@rm -f bin/support-bundle
@rm -f bin/collect
@rm -f bin/preflight
@rm -f bin/troubleshoot.h
@rm -f bin/troubleshoot.so
@rm -f bin/schemagen
@rm -f bin/docsgen
.PHONY: tidy
tidy:
go mod tidy
# Prints the diff of the changes that would be made by `go mod tidy`. Used in CI
.PHONY: tidy-diff
tidy-diff:
go mod tidy -diff
# Only build when any of the files in SOURCES changes, or if bin/<file> is absent
MAKEFILE_DIR := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))
SOURCES := $(shell find $(MAKEFILE_DIR) -type f \( -name "*.go" -o -name "go.mod" -o -name "go.sum" \))
@@ -117,28 +97,12 @@ bin/support-bundle: $(SOURCES)
bin/preflight: $(SOURCES)
go build ${BUILDFLAGS} ${LDFLAGS} -o bin/preflight github.com/replicatedhq/troubleshoot/cmd/preflight
bin/analyze: $(SOURCES)
go build ${BUILDFLAGS} ${LDFLAGS} -o bin/analyze github.com/replicatedhq/troubleshoot/cmd/analyze
bin/collect: $(SOURCES)
go build ${BUILDFLAGS} ${LDFLAGS} -o bin/collect github.com/replicatedhq/troubleshoot/cmd/collect
.PHONY: support-bundle
support-bundle: bin/support-bundle
.PHONY: preflight
preflight: bin/preflight
.PHONY: analyze
analyze: bin/analyze
.PHONY: collect
collect: bin/collect
build-linux: tidy
@echo "Build cli binaries for Linux"
GOOS=linux GOARCH=amd64 $(MAKE) -j bin/support-bundle bin/preflight bin/analyze bin/collect
.PHONY: fmt
fmt:
go fmt ${BUILDPATHS}
@@ -146,153 +110,3 @@ fmt:
.PHONY: vet
vet:
go vet ${BUILDFLAGS} ${BUILDPATHS}
.PHONY: generate
generate: controller-gen client-gen
$(CONTROLLER_GEN) \
object:headerFile=./hack/boilerplate.go.txt paths=./pkg/apis/...
$(CLIENT_GEN) \
--output-dir=. \
--output-pkg=github.com/replicatedhq/troubleshoot/pkg/client \
--clientset-name troubleshootclientset \
--input-base github.com/replicatedhq/troubleshoot/pkg/apis \
--input troubleshoot/v1beta1 \
--input troubleshoot/v1beta2 \
--go-header-file ./hack/boilerplate.go.txt
cp -r troubleshootclientset pkg/client
rm -rf troubleshootclientset
.PHONY: openapischema
openapischema: controller-gen
controller-gen crd +output:dir=./config/crds paths=./pkg/apis/troubleshoot/v1beta1
controller-gen crd +output:dir=./config/crds paths=./pkg/apis/troubleshoot/v1beta2
check-schemas: generate schemas
@if [ -n "$$(git status --short)" ]; then \
echo -e "\033[31mThe git repo is dirty :( Ensure all generated files are committed e.g CRD schema files\033[0;m"; \
git status --short; \
exit 1; \
fi
.PHONY: schemas
schemas: openapischema bin/schemagen
./bin/schemagen --output-dir ./schemas
bin/schemagen:
go build ${LDFLAGS} -o bin/schemagen github.com/replicatedhq/troubleshoot/cmd/schemagen
.PHONY: docs
docs: fmt vet bin/docsgen
./bin/docsgen
bin/docsgen:
go build ${LDFLAGS} -o bin/docsgen github.com/replicatedhq/troubleshoot/cmd/docsgen
controller-gen:
go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.19.0
CONTROLLER_GEN=$(shell which controller-gen)
.PHONY: client-gen
client-gen:
go install k8s.io/code-generator/cmd/client-gen@v0.34.0
CLIENT_GEN=$(shell which client-gen)
.PHONY: release
release: export GITHUB_TOKEN = $(shell echo ${GITHUB_TOKEN_TROUBLESHOOT})
release:
curl -sL https://git.io/goreleaser | bash -s -- --rm-dist --config deploy/.goreleaser.yml
.PHONY: snapshot-release
snapshot-release:
curl -sL https://git.io/goreleaser | bash -s -- --rm-dist --snapshot --config deploy/.goreleaser.snapshot.yml
docker push replicated/troubleshoot:alpha
docker push replicated/preflight:alpha
.PHONY: local-release
local-release:
curl -sL https://git.io/goreleaser | bash -s -- --rm-dist --snapshot --config deploy/.goreleaser.yaml
docker tag replicated/troubleshoot:alpha localhost:32000/troubleshoot:alpha
docker tag replicated/preflight:alpha localhost:32000/preflight:alpha
docker push localhost:32000/troubleshoot:alpha
docker push localhost:32000/preflight:alpha
.PHONY: run-preflight
run-preflight: bin/preflight
./bin/preflight ./examples/preflight/sample-preflight.yaml
.PHONY: run-support-bundle
run-support-bundle: bin/support-bundle
./bin/support-bundle ./examples/support-bundle/sample-supportbundle.yaml
.PHONY: run-analyze
run-analyze: bin/analyze
./bin/analyze --analyzers ./examples/support-bundle/sample-analyzers.yaml ./support-bundle.tar.gz
.PHONY: init-sbom
init-sbom:
mkdir -p sbom/spdx sbom/assets
.PHONY: install-spdx-sbom-generator
install-spdx-sbom-generator: init-sbom
./scripts/initialize-sbom-build.sh
SPDX_GENERATOR=./sbom/spdx-sbom-generator
.PHONY: generate-sbom
generate-sbom: install-spdx-sbom-generator
$(SPDX_GENERATOR) -o ./sbom/spdx
sbom/assets/troubleshoot-sbom.tgz: generate-sbom
tar -czf sbom/assets/troubleshoot-sbom.tgz sbom/spdx/*.spdx
sbom: sbom/assets/troubleshoot-sbom.tgz
cosign sign-blob \
--key ./cosign.key \
--tlog-upload \
--yes \
--rekor-url=https://rekor.sigstore.dev \
sbom/assets/troubleshoot-sbom.tgz > sbom/assets/troubleshoot-sbom.tgz.sig
cosign public-key --key cosign.key --outfile sbom/assets/key.pub
.PHONY: scan
scan:
trivy fs \
--scanners vuln \
--exit-code=1 \
--severity="HIGH,CRITICAL" \
--ignore-unfixed \
./
.PHONY: watch
watch: npm-install
bin/watch.js
## Syncronize the code with a remote server. More info: CONTRIBUTING.md
.PHONY: watchrsync
watchrsync: npm-install
bin/watchrsync.js
.PHONY: npm-install
npm-install:
npm --version 2>&1 >/dev/null || ( echo "npm not installed; install npm to set up watchrsync" && exit 1 )
npm list gaze-run-interrupt || npm install install --no-save gaze-run-interrupt@~2.0.0
######## Lagacy make targets ###########
# Deprecated: These can be removed
.PHONY: run-troubleshoot
run-troubleshoot: run-support-bundle
longhorn:
git clone https://github.com/longhorn/longhorn-manager.git
cd longhorn-manager && git checkout v1.2.2 && cd ..
rm -rf pkg/longhorn
mv longhorn-manager/k8s/pkg pkg/longhorn
mv longhorn-manager/types pkg/longhorn/types
mv longhorn-manager/util pkg/longhorn/util
rm -rf pkg/longhorn/util/daemon
rm -rf pkg/longhorn/util/server
find pkg/longhorn -type f | xargs sed -i "s/github.com\/longhorn\/longhorn-manager\/k8s\/pkg/github.com\/replicatedhq\/troubleshoot\/pkg\/longhorn/g"
find pkg/longhorn -type f | xargs sed -i "s/github.com\/longhorn\/longhorn-manager\/types/github.com\/replicatedhq\/troubleshoot\/pkg\/longhorn\/types/g"
find pkg/longhorn -type f | xargs sed -i "s/github.com\/longhorn\/longhorn-manager\/util/github.com\/replicatedhq\/troubleshoot\/pkg\/longhorn\/util/g"
rm -rf longhorn-manager
-13
View File
@@ -1,13 +0,0 @@
package main
import (
"os"
analyzecli "github.com/replicatedhq/troubleshoot/cmd/analyze/cli"
)
func main() {
if err := analyzecli.RootCmd().Execute(); err != nil {
os.Exit(1)
}
}
+14 -12
View File
@@ -7,7 +7,6 @@ import (
"github.com/pkg/errors"
"github.com/replicatedhq/troubleshoot/pkg/constants"
"github.com/replicatedhq/troubleshoot/pkg/lint"
"github.com/replicatedhq/troubleshoot/pkg/types"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
@@ -16,17 +15,24 @@ func LintCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "lint [spec-files...]",
Args: cobra.MinimumNArgs(1),
Short: "Lint v1beta2/v1beta3 preflight specs for syntax and structural errors",
Long: `Lint v1beta2/v1beta3 preflight specs for syntax and structural errors.
Short: "Lint preflight specs for syntax and structural errors",
Long: `Lint preflight specs for syntax and structural errors.
This command validates v1beta2/v1beta3 preflight specs and checks for:
- YAML syntax errors
This command validates troubleshoot specs and checks for:
- YAML syntax errors (missing colons, invalid structure)
- Missing required fields (apiVersion, kind, metadata, spec)
- Invalid template syntax ({{ .Values.* }})
- Invalid template syntax ({{ .Values.* }}, {{ .Release.* }}, etc.)
- Missing analyzers or collectors
- Common structural issues
- Missing docStrings (warning)
Both v1beta2 and v1beta3 apiVersions are supported. Use 'convert' if you need a full structural conversion between schema versions.
The --fix flag can automatically repair:
- Missing colons in YAML (e.g., "metadata" → "metadata:")
- Missing or malformed apiVersion line. If templating ({{ }}) or docString fields are detected, apiVersion is set to v1beta3; otherwise v1beta2.
- Template expressions missing leading dot (e.g., "{{ Values.x }}" → "{{ .Values.x }}")
Examples:
# Lint a single spec file
preflight lint my-preflight.yaml
@@ -34,16 +40,12 @@ Examples:
# Lint multiple spec files
preflight lint spec1.yaml spec2.yaml spec3.yaml
# Lint with automatic fixes
# Lint with automatic fixes (may need to run multiple times for complex issues)
preflight lint --fix my-preflight.yaml
# Lint and output as JSON for CI/CD integration
preflight lint --format json my-preflight.yaml
Notes:
- v1beta2 does not support templating; template syntax in v1beta2 files will be flagged as errors.
- v1beta3 supports templating and is linted with template-awareness.
Exit codes:
0 - No errors found
2 - Validation errors found`,
@@ -89,7 +91,7 @@ func runLint(opts lint.LintOptions) error {
// Return appropriate exit code
if lint.HasErrors(results) {
return types.NewExitCodeError(constants.EXIT_CODE_SPEC_ISSUES, nil)
os.Exit(constants.EXIT_CODE_SPEC_ISSUES)
}
return nil
-96
View File
@@ -1,96 +0,0 @@
package cli
import (
"fmt"
"os"
"github.com/pkg/errors"
"github.com/replicatedhq/troubleshoot/pkg/constants"
"github.com/replicatedhq/troubleshoot/pkg/lint"
"github.com/replicatedhq/troubleshoot/pkg/types"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
func LintCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "lint [spec-files...]",
Args: cobra.MinimumNArgs(1),
Short: "Lint v1beta2/v1beta3 troubleshoot specs for syntax and structural errors",
Long: `Lint v1beta2/v1beta3 troubleshoot specs (both preflight and support-bundle) for syntax and structural errors.
This command validates v1beta2/v1beta3 troubleshoot specs and checks for:
- YAML syntax errors
- Missing required fields (apiVersion, kind, metadata, spec)
- Invalid template syntax ({{ .Values.* }})
- Missing collectors or hostCollectors
- Common structural issues
- Missing docStrings (warning)
Examples:
# Lint a single spec file
support-bundle lint my-spec.yaml
# Lint multiple spec files
support-bundle lint spec1.yaml spec2.yaml spec3.yaml
# Lint with automatic fixes
support-bundle lint --fix my-spec.yaml
# Lint and output as JSON for CI/CD integration
support-bundle lint --format json my-spec.yaml
Notes:
- v1beta2 does not support templating; template syntax in v1beta2 files will be flagged as errors.
- v1beta3 supports templating and is linted with template-awareness.
Exit codes:
0 - No errors found
2 - Validation errors found`,
PreRun: func(cmd *cobra.Command, args []string) {
viper.BindPFlags(cmd.Flags())
},
RunE: func(cmd *cobra.Command, args []string) error {
v := viper.GetViper()
opts := lint.LintOptions{
FilePaths: args,
Fix: v.GetBool("fix"),
Format: v.GetString("format"),
}
return runLint(opts)
},
}
cmd.Flags().Bool("fix", false, "Automatically fix issues where possible")
cmd.Flags().String("format", "text", "Output format: text or json")
return cmd
}
func runLint(opts lint.LintOptions) error {
// Validate file paths exist
for _, filePath := range opts.FilePaths {
if _, err := os.Stat(filePath); err != nil {
return errors.Wrapf(err, "file not found: %s", filePath)
}
}
// Run linting
results, err := lint.LintFiles(opts)
if err != nil {
return errors.Wrap(err, "failed to lint files")
}
// Format and print results
output := lint.FormatResults(results, opts.Format)
fmt.Print(output)
// Return appropriate exit code
if lint.HasErrors(results) {
return types.NewExitCodeError(constants.EXIT_CODE_SPEC_ISSUES, nil)
}
return nil
}
+3 -21
View File
@@ -5,14 +5,11 @@ import (
"os"
"strings"
"errors"
"github.com/replicatedhq/troubleshoot/cmd/internal/util"
preflightcli "github.com/replicatedhq/troubleshoot/cmd/preflight/cli"
"github.com/replicatedhq/troubleshoot/internal/traces"
"github.com/replicatedhq/troubleshoot/pkg/constants"
"github.com/replicatedhq/troubleshoot/pkg/k8sutil"
"github.com/replicatedhq/troubleshoot/pkg/logger"
"github.com/replicatedhq/troubleshoot/pkg/types"
"github.com/replicatedhq/troubleshoot/pkg/updater"
"github.com/spf13/cobra"
"github.com/spf13/viper"
@@ -112,8 +109,8 @@ If no arguments are provided, specs are automatically loaded from the cluster by
cmd.AddCommand(Diff())
cmd.AddCommand(Schedule())
cmd.AddCommand(UploadCmd())
cmd.AddCommand(LintCmd())
cmd.AddCommand(util.VersionCmd())
cmd.AddCommand(preflightcli.LintCmd())
cmd.Flags().StringSlice("redactors", []string{}, "names of the additional redactors to use")
cmd.Flags().Bool("redact", true, "enable/disable default redactions")
@@ -137,12 +134,6 @@ If no arguments are provided, specs are automatically loaded from the cluster by
cmd.Flags().Bool("dry-run", false, "print support bundle spec without collecting anything")
cmd.Flags().Bool("auto-update", true, "enable automatic binary self-update check and install")
// Upload flags
cmd.Flags().Bool("auto-upload", false, "automatically upload resulting bundle to replicated.app")
cmd.Flags().String("license-id", "", "license ID for authentication when uploading (auto-detected from bundle if not provided)")
cmd.Flags().String("app-slug", "", "application slug when uploading (auto-detected from bundle if not provided)")
cmd.Flags().String("upload-domain", "", "custom domain for upload (default: replicated.app)")
// Auto-discovery flags
cmd.Flags().Bool("auto", false, "enable auto-discovery of foundational collectors. When used with YAML specs, adds foundational collectors to YAML collectors. When used alone, collects only foundational data")
cmd.Flags().Bool("include-images", false, "include container image metadata collection when using auto-discovery")
@@ -172,16 +163,7 @@ If no arguments are provided, specs are automatically loaded from the cluster by
}
func InitAndExecute() {
cmd := RootCmd()
if err := cmd.Execute(); err != nil {
var exitErr types.ExitError
if errors.As(err, &exitErr) {
if exitErr.ExitStatus() != constants.EXIT_CODE_FAIL && exitErr.ExitStatus() != constants.EXIT_CODE_WARN {
cmd.PrintErrln("Error:", err.Error())
}
os.Exit(exitErr.ExitStatus())
}
cmd.PrintErrln("Error:", err.Error())
if err := RootCmd().Execute(); err != nil {
os.Exit(1)
}
}
+2 -23
View File
@@ -242,26 +242,6 @@ func runTroubleshoot(v *viper.Viper, args []string) error {
}
}
// Attempt auto-upload before any early returns
if v.GetBool("auto-upload") && !response.FileUploaded {
licenseID := v.GetString("license-id")
appSlug := v.GetString("app-slug")
uploadDomain := v.GetString("upload-domain")
targetDomain := uploadDomain
if targetDomain == "" {
targetDomain = "replicated.app"
}
fmt.Fprintf(os.Stderr, "Auto-uploading bundle to %s...\n", targetDomain)
if err := supportbundle.UploadBundleAutoDetect(response.ArchivePath, licenseID, appSlug, uploadDomain); err != nil {
fmt.Fprintf(os.Stderr, "Auto-upload failed: %v\n", err)
fmt.Fprintf(os.Stderr, "You can manually upload the bundle using: support-bundle upload %s\n", response.ArchivePath)
} else {
response.FileUploaded = true
}
}
if !response.FileUploaded {
if appName := mainBundle.Labels["applicationName"]; appName != "" {
f := `A support bundle for %s has been created in this directory
@@ -289,12 +269,11 @@ the %s Admin Console to begin analysis.`
fmt.Printf("\r%s\r", cursor.ClearEntireLine())
}
if response.FileUploaded {
fmt.Printf("A support bundle has been created and uploaded to replicated.app for analysis.\n")
fmt.Printf("A support bundle has been created and uploaded to your cluster for analysis. Please visit the Troubleshoot page to continue.\n")
fmt.Printf("A copy of this support bundle was written to the current directory, named %q\n", response.ArchivePath)
} else {
fmt.Printf("A support bundle has been created in the current directory named %q\n", response.ArchivePath)
}
return nil
}
@@ -518,7 +497,7 @@ func (a *analysisOutput) FormattedAnalysisOutput() (outputJson string, err error
formatted, err := json.MarshalIndent(o, "", " ")
if err != nil {
return "", fmt.Errorf("\r * Failed to format analysis: %v", err)
return "", fmt.Errorf("\r * Failed to format analysis: %v\n", err)
}
return string(formatted), nil
}
+2 -7
View File
@@ -26,10 +26,7 @@ Examples:
support-bundle upload bundle.tar.gz --license-id YOUR_LICENSE_ID
# Specify both license and app
support-bundle upload bundle.tar.gz --license-id YOUR_LICENSE_ID --app-slug my-app
# Upload to a custom domain (e.g., development environment)
support-bundle upload bundle.tar.gz --upload-domain replicated-app-dev.example.com`,
support-bundle upload bundle.tar.gz --license-id YOUR_LICENSE_ID --app-slug my-app`,
RunE: func(cmd *cobra.Command, args []string) error {
v := viper.GetViper()
bundlePath := args[0]
@@ -42,10 +39,9 @@ Examples:
// Get upload parameters
licenseID := v.GetString("license-id")
appSlug := v.GetString("app-slug")
uploadDomain := v.GetString("upload-domain")
// Use auto-detection for uploads
if err := supportbundle.UploadBundleAutoDetect(bundlePath, licenseID, appSlug, uploadDomain); err != nil {
if err := supportbundle.UploadBundleAutoDetect(bundlePath, licenseID, appSlug); err != nil {
return errors.Wrap(err, "upload failed")
}
@@ -55,7 +51,6 @@ Examples:
cmd.Flags().String("license-id", "", "license ID for authentication (auto-detected from bundle if not provided)")
cmd.Flags().String("app-slug", "", "application slug (auto-detected from bundle if not provided)")
cmd.Flags().String("upload-domain", "", "custom domain for upload (default: replicated.app)")
return cmd
}
+2 -2
View File
@@ -42,7 +42,7 @@ spec:
matchLabels:
os: darwin
arch: amd64
{{addURIAndSha "https://github.com/replicatedhq/troubleshoot/releases/download/{{ .TagName }}/preflight_darwin_all.tar.gz" .TagName }}
{{addURIAndSha "https://github.com/replicatedhq/troubleshoot/releases/download/{{ .TagName }}/preflight_darwin_amd64.tar.gz" .TagName }}
files:
- from: preflight
to: .
@@ -53,7 +53,7 @@ spec:
matchLabels:
os: darwin
arch: arm64
{{addURIAndSha "https://github.com/replicatedhq/troubleshoot/releases/download/{{ .TagName }}/preflight_darwin_all.tar.gz" .TagName }}
{{addURIAndSha "https://github.com/replicatedhq/troubleshoot/releases/download/{{ .TagName }}/preflight_darwin_arm64.tar.gz" .TagName }}
files:
- from: preflight
to: .
+2 -2
View File
@@ -42,7 +42,7 @@ spec:
matchLabels:
os: darwin
arch: amd64
{{addURIAndSha "https://github.com/replicatedhq/troubleshoot/releases/download/{{ .TagName }}/support-bundle_darwin_all.tar.gz" .TagName }}
{{addURIAndSha "https://github.com/replicatedhq/troubleshoot/releases/download/{{ .TagName }}/support-bundle_darwin_amd64.tar.gz" .TagName }}
files:
- from: support-bundle
to: .
@@ -53,7 +53,7 @@ spec:
matchLabels:
os: darwin
arch: arm64
{{addURIAndSha "https://github.com/replicatedhq/troubleshoot/releases/download/{{ .TagName }}/support-bundle_darwin_all.tar.gz" .TagName }}
{{addURIAndSha "https://github.com/replicatedhq/troubleshoot/releases/download/{{ .TagName }}/support-bundle_darwin_arm64.tar.gz" .TagName }}
files:
- from: support-bundle
to: .
@@ -1,59 +0,0 @@
apiVersion: troubleshoot.sh/v1beta3
kind: SupportBundle
metadata:
name: test-v1beta3-secretref
spec:
collectors:
# Test 1: PostgreSQL with URI from secret
- postgres:
collectorName: postgres-with-secret
uri:
valueFrom:
secretKeyRef:
name: test-database-credentials
key: postgres-uri
# This will fail to connect (fake server) but that's OK -
# we're testing secret resolution, not actual DB connectivity
# Test 2: PostgreSQL with TLS certs from secret
- postgres:
collectorName: postgres-with-tls
uri:
value: "postgresql://testuser:testpass@localhost:5432/testdb"
tls:
cacert:
valueFrom:
secretKeyRef:
name: test-database-credentials
key: ca.crt
clientCert:
valueFrom:
secretKeyRef:
name: test-database-credentials
key: client.crt
clientKey:
valueFrom:
secretKeyRef:
name: test-database-credentials
key: client.key
# Test 3: MySQL with URI from secret
- mysql:
collectorName: mysql-with-secret
uri:
valueFrom:
secretKeyRef:
name: test-database-credentials
key: mysql-uri
# Test 4: Redis with URI from secret
- redis:
collectorName: redis-with-secret
uri:
valueFrom:
secretKeyRef:
name: test-database-credentials
key: redis-uri
# Test 5: Literal value (no secret) for comparison
- clusterInfo: {}
-39
View File
@@ -1,39 +0,0 @@
---
# Secret containing database credentials
apiVersion: v1
kind: Secret
metadata:
name: test-database-credentials
namespace: default
type: Opaque
stringData:
# PostgreSQL connection URI
postgres-uri: "postgresql://testuser:supersecret@postgres.example.com:5432/testdb?sslmode=require"
# MySQL connection URI
mysql-uri: "mysql://testuser:supersecret@mysql.example.com:3306/testdb"
# Redis connection URI
redis-uri: "redis://:supersecret@redis.example.com:6379"
# TLS certificates (example data)
ca.crt: |
-----BEGIN CERTIFICATE-----
MIICpDCCAYwCCQDU+pQ3ZUD30jANBgkqhkiG9w0BAQsFADAUMRIwEAYDVQQDDAls
b2NhbGhvc3QwHhcNMjQwMTAxMDAwMDAwWhcNMjUwMTAxMDAwMDAwWjAUMRIwEAYD
VQQDDAlsb2NhbGhvc3QwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC7
VJTUt9Us8cKjMzEfYyjiWA4R4/M2bS1+fWIcPm15A8IgC0qC1J3xGhE=
-----END CERTIFICATE-----
client.crt: |
-----BEGIN CERTIFICATE-----
MIICpDCCAYwCCQDU+pQ3ZUD30jANBgkqhkiG9w0BAQsFADAUMRIwEAYDVQQDDAls
b2NhbGhvc3QwHhcNMjQwMTAxMDAwMDAwWhcNMjUwMTAxMDAwMDAwWjAUMRIwEAYD
VQQDDA5jbGllbnQtY2VydA==
-----END CERTIFICATE-----
client.key: |
-----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQC7VJTUt9Us8cKj
MzEfYyjiWA4R4/M2bS1+fWIcPm15A8IgC0qC1J3xGhE=
-----END PRIVATE KEY-----
-236
View File
@@ -1,236 +0,0 @@
# v1beta3 Support Bundle Examples
This directory contains example Support Bundle specs using the v1beta3 API, which introduces `StringOrValueFrom` support for securely referencing Kubernetes Secrets and ConfigMaps in collector fields.
## Features
### StringOrValueFrom Pattern
The v1beta3 API introduces a Kubernetes-native pattern for referencing sensitive values:
```yaml
uri:
valueFrom:
secretKeyRef:
name: my-secret
key: connection-uri
```
or
```yaml
uri: "postgresql://localhost:5432/db" # Literal value
```
### Supported Collectors
Currently, v1beta3 supports `StringOrValueFrom` for:
- **Database collectors**: `postgres`, `mysql`, `redis`, `mssql`
- `uri` field - Connection strings from secrets
- `tls` fields - CA cert, client cert, and client key from secrets
## Examples
### 1. postgres-with-secret.yaml
Basic PostgreSQL collector with connection URI from a secret.
**Use case**: Securely store database credentials without hardcoding them in the spec.
```bash
kubectl apply -f postgres-with-secret.yaml
```
### 2. postgres-with-tls.yaml
PostgreSQL with TLS configuration from secrets.
**Use case**: Secure database connections with mutual TLS, storing certificates in secrets.
```bash
kubectl apply -f postgres-with-tls.yaml
```
### 3. multiple-databases.yaml
Multiple database collectors (PostgreSQL, MySQL, Redis, MSSQL) with various configurations.
**Use case**: Collect diagnostics from multiple databases in your application stack.
```bash
kubectl apply -f multiple-databases.yaml
```
### 4. cross-namespace-secrets.yaml
Accessing secrets from different namespaces.
**Use case**: Centralized credential management in a shared namespace.
```bash
kubectl apply -f cross-namespace-secrets.yaml
```
**RBAC Requirements**: The support bundle service account needs `get` permission on secrets in the referenced namespaces.
### 5. optional-secrets.yaml
Using the `optional` field for graceful degradation.
**Use case**: Collect diagnostics even when some credentials are unavailable (e.g., optional secondary databases).
```bash
kubectl apply -f optional-secrets.yaml
```
### 6. configmap-example.yaml
Using ConfigMaps for non-sensitive configuration.
**Use case**: Store non-sensitive connection strings (e.g., development databases) in ConfigMaps.
```bash
kubectl apply -f configmap-example.yaml
```
## Key Concepts
### Secret vs ConfigMap
- **Secrets**: Use for sensitive data (passwords, tokens, certificates)
- **ConfigMaps**: Use for non-sensitive configuration (development endpoints, feature flags)
### Optional Field
```yaml
uri:
valueFrom:
secretKeyRef:
name: my-secret
key: uri
optional: true # Returns empty string if secret/key doesn't exist
```
- `optional: false` (default): Collection fails if secret is missing
- `optional: true`: Returns empty string if secret/key is missing
### Cross-Namespace Access
```yaml
uri:
valueFrom:
secretKeyRef:
name: shared-secret
key: uri
namespace: other-namespace # Access secrets in different namespaces
```
If `namespace` is not specified, uses the support bundle's namespace.
### Backward Compatibility
v1beta3 maintains backward compatibility with v1beta2 TLS configuration:
```yaml
tls:
secret: # v1beta2 style
name: tls-secret
namespace: default
```
## RBAC Configuration
Support bundles need appropriate RBAC permissions to read secrets:
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: troubleshoot-secret-reader
rules:
- apiGroups: [""]
resources: ["secrets"]
resourceNames: ["postgres-connection", "redis-creds"] # Restrict to specific secrets
verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: troubleshoot-secret-reader-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: troubleshoot-secret-reader
subjects:
- kind: ServiceAccount
name: troubleshoot
namespace: default
```
## Migration from v1beta2
### Before (v1beta2):
```yaml
apiVersion: troubleshoot.sh/v1beta2
kind: SupportBundle
spec:
collectors:
- postgres:
uri: "postgresql://user:password@host:5432/db" # Hardcoded
```
### After (v1beta3):
```yaml
apiVersion: troubleshoot.sh/v1beta3
kind: SupportBundle
spec:
collectors:
- postgres:
uri:
valueFrom:
secretKeyRef:
name: postgres-connection
key: connection-uri
```
## Limitations
1. **No value composition**: Cannot combine multiple secrets into a single value
```yaml
# NOT SUPPORTED
uri: "postgresql://$(USERNAME):$(PASSWORD)@host:5432/db"
```
Store the complete connection string in a single secret key.
2. **Collector scope**: Only database collectors support `StringOrValueFrom` initially
- Future versions will extend to HTTP, Data, and other collectors
3. **No templating**: The entire field value comes from one source
## Security Best Practices
1. **Use resourceNames in RBAC**: Restrict access to specific secrets
2. **Separate secrets**: Don't reuse secrets across applications
3. **Rotate credentials**: Update secrets regularly
4. **Audit access**: Monitor secret access logs
5. **Redact output**: Ensure connection strings are redacted in bundle output
## Troubleshooting
### Error: "failed to get secret default/my-secret"
- **Cause**: Secret doesn't exist or RBAC denied access
- **Solution**: Verify secret exists: `kubectl get secret my-secret`
- **Solution**: Check RBAC: `kubectl auth can-i get secret/my-secret`
### Error: "key 'uri' not found in secret"
- **Cause**: Secret exists but doesn't contain the specified key
- **Solution**: Check secret keys: `kubectl get secret my-secret -o jsonpath='{.data}'`
### Error: "cannot specify both 'value' and 'valueFrom'"
- **Cause**: Both literal value and secret reference provided
- **Solution**: Use only one: either `value: "string"` or `valueFrom: {...}`
## Additional Resources
- [Troubleshoot Documentation](https://troubleshoot.sh)
- [v1beta3 API Reference](https://troubleshoot.sh/docs/v1beta3/)
- [Kubernetes Secrets](https://kubernetes.io/docs/concepts/configuration/secret/)
- [RBAC Authorization](https://kubernetes.io/docs/reference/access-authn-authz/rbac/)
@@ -1,57 +0,0 @@
apiVersion: troubleshoot.sh/v1beta3
kind: SupportBundle
metadata:
name: configmap-example
spec:
collectors:
# Database URI from ConfigMap (non-sensitive connection string)
- postgres:
collectorName: dev-database
uri:
valueFrom:
configMapKeyRef:
name: database-config
key: dev-connection-uri
# Redis URI from ConfigMap
- redis:
collectorName: dev-redis
uri:
valueFrom:
configMapKeyRef:
name: cache-config
key: redis-uri
# Mixed: URI from ConfigMap, password from Secret
# Note: This shows the limitation - you can't compose values from multiple sources
# The full connection string must be in one place
- mysql:
collectorName: staging-mysql
uri:
valueFrom:
secretKeyRef:
name: mysql-secret
key: complete-connection-string
---
apiVersion: v1
kind: ConfigMap
metadata:
name: database-config
data:
dev-connection-uri: "postgresql://devuser@dev-postgres.default.svc:5432/devdb"
---
apiVersion: v1
kind: ConfigMap
metadata:
name: cache-config
data:
redis-uri: "redis://dev-redis.default.svc:6379/0"
---
apiVersion: v1
kind: Secret
metadata:
name: mysql-secret
type: Opaque
stringData:
# Complete connection string with password included
complete-connection-string: "mysql://staging:stagingpass@staging-mysql:3306/stagingdb"
@@ -1,45 +0,0 @@
apiVersion: troubleshoot.sh/v1beta3
kind: SupportBundle
metadata:
name: cross-namespace-example
spec:
collectors:
# Database in one namespace, secret in another
- postgres:
collectorName: shared-database
uri:
valueFrom:
secretKeyRef:
name: shared-postgres-connection
key: uri
namespace: shared-services # Secret is in a different namespace
# Redis accessing centralized credentials
- redis:
collectorName: shared-cache
uri:
valueFrom:
secretKeyRef:
name: shared-redis-creds
key: uri
namespace: platform-credentials
---
# This secret would be in the 'shared-services' namespace
apiVersion: v1
kind: Secret
metadata:
name: shared-postgres-connection
namespace: shared-services
type: Opaque
stringData:
uri: "postgresql://shared:password@shared-postgres.shared-services.svc:5432/shared_db"
---
# This secret would be in the 'platform-credentials' namespace
apiVersion: v1
kind: Secret
metadata:
name: shared-redis-creds
namespace: platform-credentials
type: Opaque
stringData:
uri: "redis://shared-redis.shared-services.svc:6379/0"
@@ -1,78 +0,0 @@
apiVersion: troubleshoot.sh/v1beta3
kind: SupportBundle
metadata:
name: multi-database-support-bundle
spec:
collectors:
# PostgreSQL with secret reference
- postgres:
collectorName: primary-db
uri:
valueFrom:
secretKeyRef:
name: postgres-primary
key: connection-uri
# PostgreSQL replica with secret reference
- postgres:
collectorName: replica-db
uri:
valueFrom:
secretKeyRef:
name: postgres-replica
key: connection-uri
# Redis cache with secret reference
- redis:
collectorName: cache
uri:
valueFrom:
secretKeyRef:
name: redis-creds
key: uri
# MySQL with literal value (for development/testing)
- mysql:
collectorName: local-mysql
uri: "mysql://root:password@localhost:3306/testdb"
# MSSQL with secret reference
- mssql:
collectorName: legacy-db
uri:
valueFrom:
secretKeyRef:
name: mssql-connection
key: dsn
---
apiVersion: v1
kind: Secret
metadata:
name: postgres-primary
type: Opaque
stringData:
connection-uri: "postgresql://app:secret123@postgres-primary.default.svc:5432/appdb"
---
apiVersion: v1
kind: Secret
metadata:
name: postgres-replica
type: Opaque
stringData:
connection-uri: "postgresql://app:secret123@postgres-replica.default.svc:5432/appdb"
---
apiVersion: v1
kind: Secret
metadata:
name: redis-creds
type: Opaque
stringData:
uri: "redis://:cachesecret@redis.default.svc:6379/0"
---
apiVersion: v1
kind: Secret
metadata:
name: mssql-connection
type: Opaque
stringData:
dsn: "sqlserver://sa:Str0ngP@ssw0rd@mssql.default.svc:1433?database=legacy"
@@ -1,58 +0,0 @@
apiVersion: troubleshoot.sh/v1beta3
kind: SupportBundle
metadata:
name: optional-secrets-example
spec:
collectors:
# Required database - collection will fail if secret doesn't exist
- postgres:
collectorName: required-db
uri:
valueFrom:
secretKeyRef:
name: required-postgres
key: uri
optional: false # Default behavior - secret must exist
# Optional database - collection continues if secret doesn't exist
- postgres:
collectorName: optional-db
uri:
valueFrom:
secretKeyRef:
name: optional-postgres
key: uri
optional: true # Gracefully degrades if secret is missing
# Mixed required and optional TLS
- postgres:
collectorName: partially-optional
uri: "postgresql://localhost:5432/db"
tls:
cacert:
valueFrom:
secretKeyRef:
name: tls-certs
key: ca.crt
optional: false # CA cert is required
clientCert:
valueFrom:
secretKeyRef:
name: tls-certs
key: client.crt
optional: true # Client cert is optional (cert-only TLS)
clientKey:
valueFrom:
secretKeyRef:
name: tls-certs
key: client.key
optional: true # Client key is optional
---
apiVersion: v1
kind: Secret
metadata:
name: required-postgres
type: Opaque
stringData:
uri: "postgresql://user:pass@required-postgres:5432/db"
# Note: optional-postgres secret intentionally not created
@@ -1,21 +0,0 @@
apiVersion: troubleshoot.sh/v1beta3
kind: SupportBundle
metadata:
name: postgres-support-bundle
spec:
collectors:
- postgres:
collectorName: main-database
uri:
valueFrom:
secretKeyRef:
name: postgres-connection
key: connection-uri
---
apiVersion: v1
kind: Secret
metadata:
name: postgres-connection
type: Opaque
stringData:
connection-uri: "postgresql://myuser:mypassword@postgres.default.svc:5432/mydb?sslmode=require"
@@ -1,47 +0,0 @@
apiVersion: troubleshoot.sh/v1beta3
kind: SupportBundle
metadata:
name: postgres-tls-support-bundle
spec:
collectors:
- postgres:
collectorName: secure-database
uri:
valueFrom:
secretKeyRef:
name: postgres-connection
key: connection-uri
tls:
cacert:
valueFrom:
secretKeyRef:
name: postgres-tls
key: ca.crt
clientCert:
valueFrom:
secretKeyRef:
name: postgres-tls
key: tls.crt
clientKey:
valueFrom:
secretKeyRef:
name: postgres-tls
key: tls.key
---
apiVersion: v1
kind: Secret
metadata:
name: postgres-connection
type: Opaque
stringData:
connection-uri: "postgresql://myuser:mypassword@postgres.default.svc:5432/mydb?sslmode=verify-full"
---
apiVersion: v1
kind: Secret
metadata:
name: postgres-tls
type: kubernetes.io/tls
data:
ca.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCi4uLgotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0t
tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCi4uLgotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0t
tls.key: LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCi4uLgotLS0tLUVORCBQUklWQVRFIEtFWS0tLS0t
@@ -1,19 +0,0 @@
apiVersion: troubleshoot.sh/v1beta3
kind: SupportBundle
metadata:
name: invalid-collectors
spec:
collectors:
# Unknown collector type
- notACollector: {}
# Known collector but missing required fields (e.g., ceph requires namespace)
- ceph: {}
# Field exists but wrong type (should be a list)
hostCollectors: "not-a-list"
analyzers:
# Unknown analyzer type
- notAnAnalyzer: {}
# Known analyzer missing required 'outcomes'
- cephStatus:
namespace: default
+5 -4
View File
@@ -21,7 +21,7 @@ require (
github.com/google/gofuzz v1.2.0
github.com/google/uuid v1.6.0
github.com/gorilla/handlers v1.5.2
github.com/hashicorp/go-getter v1.8.2
github.com/hashicorp/go-getter v1.8.0
github.com/hashicorp/go-multierror v1.1.1
github.com/jackc/pgx/v5 v5.7.6
github.com/longhorn/go-iscsi-helper v0.0.0-20210330030558-49a327fb024e
@@ -33,7 +33,7 @@ require (
github.com/pkg/errors v0.9.1
github.com/replicatedhq/termui/v3 v3.1.1-0.20200811145416-f40076d26851
github.com/segmentio/ksuid v1.0.4
github.com/shirou/gopsutil/v4 v4.25.9
github.com/shirou/gopsutil/v4 v4.25.8
github.com/spf13/cobra v1.10.1
github.com/spf13/pflag v1.0.10
github.com/spf13/viper v1.21.0
@@ -57,7 +57,7 @@ require (
k8s.io/klog/v2 v2.130.1
k8s.io/kubernetes v1.34.1
oras.land/oras-go v1.2.6
sigs.k8s.io/controller-runtime v0.22.2
sigs.k8s.io/controller-runtime v0.22.1
sigs.k8s.io/e2e-framework v0.6.0
)
@@ -106,7 +106,7 @@ require (
github.com/coreos/go-systemd/v22 v22.5.0 // indirect
github.com/distribution/reference v0.6.0 // indirect
github.com/docker/distribution v2.8.3+incompatible // indirect
github.com/ebitengine/purego v0.9.0 // indirect
github.com/ebitengine/purego v0.8.4 // indirect
github.com/emicklei/go-restful/v3 v3.12.2 // indirect
github.com/envoyproxy/go-control-plane/envoy v1.32.4 // indirect
github.com/envoyproxy/protoc-gen-validate v1.2.1 // indirect
@@ -222,6 +222,7 @@ require (
github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
github.com/hashicorp/go-safetemp v1.0.0 // indirect
github.com/hashicorp/go-version v1.7.0
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/josharian/intern v1.0.0 // indirect
+10 -8
View File
@@ -213,8 +213,8 @@ github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4
github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/docker/libtrust v0.0.0-20160708172513-aabc10ec26b7 h1:UhxFibDNY/bfvqU5CAUmr9zpesgbU6SWc8/B4mflAE4=
github.com/docker/libtrust v0.0.0-20160708172513-aabc10ec26b7/go.mod h1:cyGadeNEkKy96OOhEzfZl+yxihPEzKnqJwvfuSUqbZE=
github.com/ebitengine/purego v0.9.0 h1:mh0zpKBIXDceC63hpvPuGLiJ8ZAa3DfrFTudmfi8A4k=
github.com/ebitengine/purego v0.9.0/go.mod h1:iIjxzd6CiRiOG0UyXP+V1+jWqUXVjPKLAI0mRfJZTmQ=
github.com/ebitengine/purego v0.8.4 h1:CF7LEKg5FFOsASUj0+QwaXf8Ht6TlFxg09+S9wz0omw=
github.com/ebitengine/purego v0.8.4/go.mod h1:iIjxzd6CiRiOG0UyXP+V1+jWqUXVjPKLAI0mRfJZTmQ=
github.com/emicklei/go-restful/v3 v3.12.2 h1:DhwDP0vY3k8ZzE0RunuJy8GhNpPL6zqLkDf9B/a0/xU=
github.com/emicklei/go-restful/v3 v3.12.2/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc=
github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
@@ -368,10 +368,12 @@ github.com/hashicorp/errwrap v1.1.0 h1:OxrOeh75EUXMY8TBjag2fzXGZ40LB6IKw45YeGUDY
github.com/hashicorp/errwrap v1.1.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
github.com/hashicorp/go-cleanhttp v0.5.2 h1:035FKYIWjmULyFRBKPs8TBQoi0x6d9G4xc9neXJWAZQ=
github.com/hashicorp/go-cleanhttp v0.5.2/go.mod h1:kO/YDlP8L1346E6Sodw+PrpBSV4/SoxCXGY6BqNFT48=
github.com/hashicorp/go-getter v1.8.2 h1:CGCK+bZQLl44PYiwJweVzfpjg7bBwtuXu3AGcLiod2o=
github.com/hashicorp/go-getter v1.8.2/go.mod h1:CUTt9x2bCtJ/sV8ihgrITL3IUE+0BE1j/e4n5P/GIM4=
github.com/hashicorp/go-getter v1.8.0 h1:GMRdoMBDz12Mim366pWsRVIrrkugJ19rrmykkv0Nhzo=
github.com/hashicorp/go-getter v1.8.0/go.mod h1:/K0O5zR6R72O3r2x3z2UHadiC0XHMbbzHO9pS8ZeJPA=
github.com/hashicorp/go-multierror v1.1.1 h1:H5DkEtf6CXdFp0N0Em5UCwQpXMWke8IA0+lD48awMYo=
github.com/hashicorp/go-multierror v1.1.1/go.mod h1:iw975J/qwKPdAO1clOe2L8331t/9/fmwbPZ6JB6eMoM=
github.com/hashicorp/go-safetemp v1.0.0 h1:2HR189eFNrjHQyENnQMMpCiBAsRxzbTMIgBhEyExpmo=
github.com/hashicorp/go-safetemp v1.0.0/go.mod h1:oaerMy3BhqiTbVye6QuFhFtIceqFoDHxNAB65b+Rj1I=
github.com/hashicorp/go-version v1.7.0 h1:5tqGy27NaOTB8yJKUZELlFAS/LTKJkrmONwQKeRZfjY=
github.com/hashicorp/go-version v1.7.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=
github.com/hashicorp/golang-lru/arc/v2 v2.0.5 h1:l2zaLDubNhW4XO3LnliVj0GXO3+/CGNJAg1dcN2Fpfw=
@@ -591,8 +593,8 @@ github.com/segmentio/ksuid v1.0.4 h1:sBo2BdShXjmcugAMwjugoGUdUV0pcxY5mW4xKRn3v4c
github.com/segmentio/ksuid v1.0.4/go.mod h1:/XUiZBD3kVx5SmUOl55voK5yeAbBNNIed+2O73XgrPE=
github.com/sergi/go-diff v1.3.1 h1:xkr+Oxo4BOQKmkn/B9eMK0g5Kg/983T9DqqPHwYqD+8=
github.com/sergi/go-diff v1.3.1/go.mod h1:aMJSSKb2lpPvRNec0+w3fl7LP9IOFzdc9Pa4NFbPK1I=
github.com/shirou/gopsutil/v4 v4.25.9 h1:JImNpf6gCVhKgZhtaAHJ0serfFGtlfIlSC08eaKdTrU=
github.com/shirou/gopsutil/v4 v4.25.9/go.mod h1:gxIxoC+7nQRwUl/xNhutXlD8lq+jxTgpIkEf3rADHL8=
github.com/shirou/gopsutil/v4 v4.25.8 h1:NnAsw9lN7587WHxjJA9ryDnqhJpFH6A+wagYWTOH970=
github.com/shirou/gopsutil/v4 v4.25.8/go.mod h1:q9QdMmfAOVIw7a+eF86P7ISEU6ka+NLgkUxlopV4RwI=
github.com/shopspring/decimal v1.4.0 h1:bxl37RwXBklmTi0C79JfXCEBD1cqqHt0bbgBAGFp81k=
github.com/shopspring/decimal v1.4.0/go.mod h1:gawqmDU56v4yIKSwfBSFip1HdCCXN8/+DMd9qYNcwME=
github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
@@ -903,8 +905,8 @@ oras.land/oras-go/v2 v2.6.0 h1:X4ELRsiGkrbeox69+9tzTu492FMUu7zJQW6eJU+I2oc=
oras.land/oras-go/v2 v2.6.0/go.mod h1:magiQDfG6H1O9APp+rOsvCPcW1GD2MM7vgnKY0Y+u1o=
periph.io/x/host/v3 v3.8.5 h1:g4g5xE1XZtDiGl1UAJaUur1aT7uNiFLMkyMEiZ7IHII=
periph.io/x/host/v3 v3.8.5/go.mod h1:hPq8dISZIc+UNfWoRj+bPH3XEBQqJPdFdx218W92mdc=
sigs.k8s.io/controller-runtime v0.22.2 h1:cK2l8BGWsSWkXz09tcS4rJh95iOLney5eawcK5A33r4=
sigs.k8s.io/controller-runtime v0.22.2/go.mod h1:+QX1XUpTXN4mLoblf4tqr5CQcyHPAki2HLXqQMY6vh8=
sigs.k8s.io/controller-runtime v0.22.1 h1:Ah1T7I+0A7ize291nJZdS1CabF/lB4E++WizgV24Eqg=
sigs.k8s.io/controller-runtime v0.22.1/go.mod h1:FwiwRjkRPbiN+zp2QRp7wlTCzbUXxZ/D4OzuQUDwBHY=
sigs.k8s.io/e2e-framework v0.6.0 h1:p7hFzHnLKO7eNsWGI2AbC1Mo2IYxidg49BiT4njxkrM=
sigs.k8s.io/e2e-framework v0.6.0/go.mod h1:IREnCHnKgRCioLRmNi0hxSJ1kJ+aAdjEKK/gokcZu4k=
sigs.k8s.io/json v0.0.0-20241014173422-cfa47c3a1cc8 h1:gBQPwqORJ8d8/YNZWEjoZs7npUVDpVXUUOFfW6CgAqE=
+3 -9
View File
@@ -191,9 +191,7 @@ func LoadFromCLIArgs(ctx context.Context, client kubernetes.Interface, args []st
// load URL spec first to remove URI key from the spec
urlSpec, err := loader.LoadSpecs(ctx, loader.LoadOptions{
RawSpec: rawURLSpec,
Client: client,
Namespace: vp.GetString("namespace"),
RawSpec: rawURLSpec,
})
if err != nil {
fmt.Println(color.YellowString("failed to load spec from URI %q: %v\n", v, err))
@@ -211,9 +209,7 @@ func LoadFromCLIArgs(ctx context.Context, client kubernetes.Interface, args []st
}
kinds, err := loader.LoadSpecs(ctx, loader.LoadOptions{
RawSpecs: rawSpecs,
Client: client,
Namespace: vp.GetString("namespace"),
RawSpecs: rawSpecs,
})
if err != nil {
return nil, err
@@ -367,9 +363,7 @@ func LoadFromCluster(ctx context.Context, client kubernetes.Interface, selectors
// Load troubleshoot specs from the raw specs
return loader.LoadSpecs(ctx, loader.LoadOptions{
RawSpecs: rawSpecs,
Client: client,
Namespace: ns,
RawSpecs: rawSpecs,
})
}
+105 -644
View File
@@ -421,31 +421,39 @@ func (a *OllamaAgent) Analyze(ctx context.Context, data []byte, analyzers []anal
func (a *OllamaAgent) discoverAnalyzers(bundle *analyzer.SupportBundle) []analyzer.AnalyzerSpec {
var specs []analyzer.AnalyzerSpec
// Collect files by type for aggregation
podFiles := []string{}
deploymentFiles := []string{}
eventFiles := []string{}
nodeFiles := []string{}
// Analyze bundle contents to determine what types of analysis to perform
for filePath := range bundle.Files {
filePathLower := strings.ToLower(filePath)
filePath = strings.ToLower(filePath)
switch {
case strings.Contains(filePathLower, "pods") && strings.HasSuffix(filePathLower, ".json"):
podFiles = append(podFiles, filePath)
case strings.Contains(filePath, "pods") && strings.HasSuffix(filePath, ".json"):
specs = append(specs, analyzer.AnalyzerSpec{
Name: "ai-pod-analysis",
Type: "ai-workload",
Category: "pods",
Priority: 10,
Config: map[string]interface{}{"filePath": filePath, "promptType": "pod-analysis"},
})
case strings.Contains(filePathLower, "deployments") && strings.HasSuffix(filePathLower, ".json"):
deploymentFiles = append(deploymentFiles, filePath)
case strings.Contains(filePath, "deployments") && strings.HasSuffix(filePath, ".json"):
specs = append(specs, analyzer.AnalyzerSpec{
Name: "ai-deployment-analysis",
Type: "ai-workload",
Category: "deployments",
Priority: 9,
Config: map[string]interface{}{"filePath": filePath, "promptType": "deployment-analysis"},
})
case strings.Contains(filePathLower, "events") && strings.HasSuffix(filePathLower, ".json"):
eventFiles = append(eventFiles, filePath)
case strings.Contains(filePath, "events") && strings.HasSuffix(filePath, ".json"):
specs = append(specs, analyzer.AnalyzerSpec{
Name: "ai-event-analysis",
Type: "ai-events",
Category: "events",
Priority: 8,
Config: map[string]interface{}{"filePath": filePath, "promptType": "event-analysis"},
})
case strings.Contains(filePathLower, "nodes") && strings.HasSuffix(filePathLower, ".json"):
nodeFiles = append(nodeFiles, filePath)
case strings.Contains(filePathLower, "logs") && strings.HasSuffix(filePathLower, ".log"):
// Logs are analyzed separately per file (not aggregated)
case strings.Contains(filePath, "logs") && strings.HasSuffix(filePath, ".log"):
specs = append(specs, analyzer.AnalyzerSpec{
Name: "ai-log-analysis",
Type: "ai-logs",
@@ -453,424 +461,50 @@ func (a *OllamaAgent) discoverAnalyzers(bundle *analyzer.SupportBundle) []analyz
Priority: 7,
Config: map[string]interface{}{"filePath": filePath, "promptType": "log-analysis"},
})
case strings.Contains(filePath, "nodes") && strings.HasSuffix(filePath, ".json"):
specs = append(specs, analyzer.AnalyzerSpec{
Name: "ai-resource-analysis",
Type: "ai-resources",
Category: "nodes",
Priority: 8,
Config: map[string]interface{}{"filePath": filePath, "promptType": "resource-analysis"},
})
}
}
// Create aggregated analyzer for ALL pod files (cluster-wide view)
if len(podFiles) > 0 {
specs = append(specs, analyzer.AnalyzerSpec{
Name: "ai-pod-analysis-cluster",
Type: "ai-workload",
Category: "pods",
Priority: 10,
Config: map[string]interface{}{
"filePaths": podFiles,
"promptType": "pod-analysis",
"aggregated": true,
},
})
}
// Create aggregated analyzer for ALL deployment files (cluster-wide view)
if len(deploymentFiles) > 0 {
specs = append(specs, analyzer.AnalyzerSpec{
Name: "ai-deployment-analysis-cluster",
Type: "ai-workload",
Category: "deployments",
Priority: 9,
Config: map[string]interface{}{
"filePaths": deploymentFiles,
"promptType": "deployment-analysis",
"aggregated": true,
},
})
}
// Create aggregated analyzer for ALL event files (cluster-wide view)
if len(eventFiles) > 0 {
specs = append(specs, analyzer.AnalyzerSpec{
Name: "ai-event-analysis-cluster",
Type: "ai-events",
Category: "events",
Priority: 8,
Config: map[string]interface{}{
"filePaths": eventFiles,
"promptType": "event-analysis",
"aggregated": true,
},
})
}
// Create aggregated analyzer for ALL node files (cluster-wide view)
if len(nodeFiles) > 0 {
specs = append(specs, analyzer.AnalyzerSpec{
Name: "ai-resource-analysis-cluster",
Type: "ai-resources",
Category: "nodes",
Priority: 8,
Config: map[string]interface{}{
"filePaths": nodeFiles,
"promptType": "resource-analysis",
"aggregated": true,
},
})
}
return specs
}
// aggregateFiles combines multiple files of the same type into a single summary for analysis
func (a *OllamaAgent) aggregateFiles(bundle *analyzer.SupportBundle, filePaths []string, category string) (string, error) {
var summary strings.Builder
switch category {
case "pods":
return a.aggregatePodFiles(bundle, filePaths)
case "deployments":
return a.aggregateDeploymentFiles(bundle, filePaths)
case "events":
return a.aggregateEventFiles(bundle, filePaths)
case "nodes":
return a.aggregateNodeFiles(bundle, filePaths)
default:
// For other types, just concatenate the files
summary.WriteString(fmt.Sprintf("Aggregated analysis of %d files:\n\n", len(filePaths)))
for _, filePath := range filePaths {
if data, exists := bundle.Files[filePath]; exists {
summary.WriteString(fmt.Sprintf("--- File: %s ---\n", filePath))
summary.Write(data)
summary.WriteString("\n\n")
}
}
}
return summary.String(), nil
}
// aggregatePodFiles creates a cluster-wide summary of pods from multiple namespace files
func (a *OllamaAgent) aggregatePodFiles(bundle *analyzer.SupportBundle, filePaths []string) (string, error) {
var summary strings.Builder
totalPods := 0
runningPods := 0
pendingPods := 0
failedPods := 0
succeededPods := 0
namespaceStats := make(map[string]int)
summary.WriteString("CLUSTER-WIDE POD ANALYSIS\n")
summary.WriteString("Analyzing pods across all namespaces:\n\n")
for _, filePath := range filePaths {
data, exists := bundle.Files[filePath]
if !exists {
continue
}
// Extract namespace from path (e.g., "cluster-resources/pods/kube-system.json")
parts := strings.Split(filePath, "/")
namespace := "unknown"
if len(parts) >= 3 {
namespace = strings.TrimSuffix(parts[len(parts)-1], ".json")
}
// Parse pod data - handle both PodList and single Pod objects
var podList map[string]interface{}
if err := json.Unmarshal(data, &podList); err != nil {
continue
}
// Check if this is a List object with items array
items, ok := podList["items"].([]interface{})
if ok {
// Handle PodList - process all pods in the list
// Initialize namespace for valid PodList (ensures empty namespaces are tracked)
if _, exists := namespaceStats[namespace]; !exists {
namespaceStats[namespace] = 0
}
podCount := len(items)
namespaceStats[namespace] += podCount
totalPods += podCount
// Count pod statuses
for _, item := range items {
pod, ok := item.(map[string]interface{})
if !ok {
continue
}
status, ok := pod["status"].(map[string]interface{})
if !ok {
continue
}
phase, ok := status["phase"].(string)
if !ok {
continue
}
switch phase {
case "Running":
runningPods++
case "Pending":
pendingPods++
case "Failed":
failedPods++
case "Succeeded":
succeededPods++
}
}
} else {
// Handle single Pod object (not a list)
// Check if this is a single Pod object (has "kind": "Pod")
if kind, exists := podList["kind"].(string); exists && kind == "Pod" {
// Initialize namespace only for valid pod data
if _, exists := namespaceStats[namespace]; !exists {
namespaceStats[namespace] = 0
}
// Single pod - increment count for this namespace
namespaceStats[namespace]++
totalPods++
// Extract status for single pod
if status, ok := podList["status"].(map[string]interface{}); ok {
if phase, ok := status["phase"].(string); ok {
switch phase {
case "Running":
runningPods++
case "Pending":
pendingPods++
case "Failed":
failedPods++
case "Succeeded":
succeededPods++
}
}
}
}
// Skip to next file after processing single pod or invalid data
continue
}
}
summary.WriteString(fmt.Sprintf("Total pods in cluster: %d\n", totalPods))
summary.WriteString(fmt.Sprintf(" - Running: %d\n", runningPods))
summary.WriteString(fmt.Sprintf(" - Pending: %d\n", pendingPods))
summary.WriteString(fmt.Sprintf(" - Failed: %d\n", failedPods))
summary.WriteString(fmt.Sprintf(" - Succeeded: %d\n", succeededPods))
summary.WriteString("\nPods by namespace:\n")
for namespace, count := range namespaceStats {
if count > 0 {
summary.WriteString(fmt.Sprintf(" - %s: %d pods\n", namespace, count))
} else {
summary.WriteString(fmt.Sprintf(" - %s: empty (no pods)\n", namespace))
}
}
summary.WriteString("\nIMPORTANT CONTEXT:\n")
summary.WriteString("- Empty namespaces are NORMAL in Kubernetes\n")
summary.WriteString("- Only report issues if there are actual pod failures or critical problems\n")
summary.WriteString("- The presence of empty namespaces is not a problem\n")
return summary.String(), nil
}
// aggregateDeploymentFiles creates a cluster-wide summary of deployments
func (a *OllamaAgent) aggregateDeploymentFiles(bundle *analyzer.SupportBundle, filePaths []string) (string, error) {
var summary strings.Builder
totalDeployments := 0
namespaceStats := make(map[string]int)
summary.WriteString("CLUSTER-WIDE DEPLOYMENT ANALYSIS\n")
summary.WriteString("Analyzing deployments across all namespaces:\n\n")
for _, filePath := range filePaths {
data, exists := bundle.Files[filePath]
if !exists {
continue
}
parts := strings.Split(filePath, "/")
namespace := "unknown"
if len(parts) >= 3 {
namespace = strings.TrimSuffix(parts[len(parts)-1], ".json")
}
// Parse deployment data - handle both DeploymentList and single Deployment objects
var deploymentList map[string]interface{}
if err := json.Unmarshal(data, &deploymentList); err != nil {
continue
}
// Check if this is a List object with items array
items, ok := deploymentList["items"].([]interface{})
if ok {
// Handle DeploymentList - process all deployments in the list
// Initialize namespace for valid DeploymentList (ensures empty namespaces are tracked)
if _, exists := namespaceStats[namespace]; !exists {
namespaceStats[namespace] = 0
}
deployCount := len(items)
namespaceStats[namespace] += deployCount
totalDeployments += deployCount
} else {
// Handle single Deployment object (not a list)
// Check if this is a single Deployment object (has "kind": "Deployment")
if kind, exists := deploymentList["kind"].(string); exists && kind == "Deployment" {
// Initialize namespace only for valid deployment data
if _, exists := namespaceStats[namespace]; !exists {
namespaceStats[namespace] = 0
}
// Single deployment - increment count for this namespace
namespaceStats[namespace]++
totalDeployments++
}
// Skip to next file after processing single deployment or invalid data
continue
}
}
summary.WriteString(fmt.Sprintf("Total deployments in cluster: %d\n", totalDeployments))
summary.WriteString("\nDeployments by namespace:\n")
for namespace, count := range namespaceStats {
if count > 0 {
summary.WriteString(fmt.Sprintf(" - %s: %d deployments\n", namespace, count))
} else {
summary.WriteString(fmt.Sprintf(" - %s: no deployments\n", namespace))
}
}
summary.WriteString("\nIMPORTANT: Empty namespaces are normal. Only flag actual deployment issues.\n")
return summary.String(), nil
}
// aggregateEventFiles creates a cluster-wide summary of events
func (a *OllamaAgent) aggregateEventFiles(bundle *analyzer.SupportBundle, filePaths []string) (string, error) {
var summary strings.Builder
totalEvents := 0
summary.WriteString("CLUSTER-WIDE EVENT ANALYSIS\n")
summary.WriteString("Analyzing events across all namespaces:\n\n")
eventsIncluded := 0
for _, filePath := range filePaths {
data, exists := bundle.Files[filePath]
if !exists {
continue
}
// Parse event data - handle both EventList and single Event objects
var eventList map[string]interface{}
if err := json.Unmarshal(data, &eventList); err != nil {
continue
}
// Check if this is a List object with items array
items, ok := eventList["items"].([]interface{})
if ok {
itemCount := len(items)
totalEvents += itemCount
// Include actual event data for AI analysis (limited to 50 events max for the summary)
// Only include if adding this file wouldn't significantly exceed the limit
if itemCount > 0 && eventsIncluded < 50 && (eventsIncluded+itemCount) <= 60 {
dataStr := string(data)
// Include file if data size is reasonable
if len(dataStr) < 2000 {
summary.WriteString(fmt.Sprintf("\n--- Events from %s ---\n", filePath))
summary.WriteString(dataStr)
summary.WriteString("\n")
eventsIncluded += itemCount
}
}
}
}
summary.WriteString(fmt.Sprintf("\nTotal events collected: %d\n", totalEvents))
return summary.String(), nil
}
// aggregateNodeFiles creates a cluster-wide summary of nodes
func (a *OllamaAgent) aggregateNodeFiles(bundle *analyzer.SupportBundle, filePaths []string) (string, error) {
var summary strings.Builder
summary.WriteString("CLUSTER-WIDE NODE ANALYSIS\n\n")
for _, filePath := range filePaths {
data, exists := bundle.Files[filePath]
if !exists {
continue
}
summary.WriteString(fmt.Sprintf("--- Nodes data from %s ---\n", filePath))
summary.Write(data)
summary.WriteString("\n\n")
}
return summary.String(), nil
}
// runLLMAnalysis executes analysis using LLM for a specific analyzer spec
func (a *OllamaAgent) runLLMAnalysis(ctx context.Context, bundle *analyzer.SupportBundle, spec analyzer.AnalyzerSpec) (*analyzer.AnalyzerResult, error) {
ctx, span := otel.Tracer(constants.LIB_TRACER_NAME).Start(ctx, fmt.Sprintf("OllamaAgent.%s", spec.Name))
defer span.End()
var dataStr string
// Smart file detection for enhanced analyzer compatibility
var filePath string
var fileData []byte
var exists bool
// Check if this is an aggregated analyzer (multiple files)
if aggregated, ok := spec.Config["aggregated"].(bool); ok && aggregated {
// Handle aggregated files
if filePaths, ok := spec.Config["filePaths"].([]string); ok && len(filePaths) > 0 {
aggregatedData, err := a.aggregateFiles(bundle, filePaths, spec.Category)
if err != nil {
return &analyzer.AnalyzerResult{
Title: spec.Name,
IsWarn: true,
Message: fmt.Sprintf("Failed to aggregate files: %v", err),
Category: spec.Category,
}, nil
}
dataStr = aggregatedData
} else {
// Missing or invalid filePaths for aggregated analyzer
return &analyzer.AnalyzerResult{
Title: spec.Name,
IsWarn: true,
Message: "Aggregated analyzer missing valid filePaths configuration",
Category: spec.Category,
}, nil
// First try to get explicit filePath from config
if fp, ok := spec.Config["filePath"].(string); ok {
filePath = fp
fileData, exists = bundle.Files[filePath]
}
// If no explicit filePath, auto-detect based on analyzer type
if !exists {
filePath, fileData, exists = a.autoDetectFileForAnalyzer(bundle, spec)
}
if !exists {
result := &analyzer.AnalyzerResult{
Title: spec.Name,
IsWarn: true,
Message: fmt.Sprintf("File not found: %s", filePath),
Category: spec.Category,
}
} else {
// Smart file detection for enhanced analyzer compatibility (single file)
var filePath string
var fileData []byte
var exists bool
// First try to get explicit filePath from config
if fp, ok := spec.Config["filePath"].(string); ok {
filePath = fp
fileData, exists = bundle.Files[filePath]
}
// If no explicit filePath, auto-detect based on analyzer type
if !exists {
filePath, fileData, exists = a.autoDetectFileForAnalyzer(bundle, spec)
}
if !exists {
result := &analyzer.AnalyzerResult{
Title: spec.Name,
IsWarn: true,
Message: fmt.Sprintf("File not found: %s", filePath),
Category: spec.Category,
}
return result, nil
}
dataStr = string(fileData)
return result, nil
}
promptType, _ := spec.Config["promptType"].(string)
@@ -885,6 +519,7 @@ func (a *OllamaAgent) runLLMAnalysis(ctx context.Context, bundle *analyzer.Suppo
}
// Prepare data for analysis (truncate if too large)
dataStr := string(fileData)
if len(dataStr) > 4000 { // Limit input size
if promptType == "log-analysis" {
// For logs, take the last N lines
@@ -1231,182 +866,6 @@ func (a *OllamaAgent) autoDetectFileForAnalyzer(bundle *analyzer.SupportBundle,
return "", nil, false
}
// normalizeInsights converts various JSON formats into a []string array
func (a *OllamaAgent) normalizeInsights(raw json.RawMessage) []string {
if len(raw) == 0 {
return []string{}
}
// Try parsing as array of strings first (expected format)
var arrayInsights []string
if err := json.Unmarshal(raw, &arrayInsights); err == nil {
return arrayInsights
}
// Try parsing as single string
var stringInsight string
if err := json.Unmarshal(raw, &stringInsight); err == nil {
if stringInsight != "" {
return []string{stringInsight}
}
return []string{}
}
// Try parsing as array of objects/maps (common LLM format)
var arrayOfMaps []map[string]interface{}
if err := json.Unmarshal(raw, &arrayOfMaps); err == nil {
insights := []string{}
for _, obj := range arrayOfMaps {
// Extract meaningful text from each object
insightText := a.formatMapAsInsight(obj)
if insightText != "" {
insights = append(insights, insightText)
}
}
return insights
}
// Try parsing as object/map and extract meaningful text
var objInsights map[string]interface{}
if err := json.Unmarshal(raw, &objInsights); err == nil {
insights := []string{}
for key, value := range objInsights {
// Extract meaningful insights from object structure
insightText := a.extractInsightText(key, value)
if insightText != "" {
insights = append(insights, insightText)
}
}
return insights
}
// If all parsing fails, return empty array
return []string{}
}
// formatMapAsInsight converts a map/object into a readable insight string
func (a *OllamaAgent) formatMapAsInsight(obj map[string]interface{}) string {
// Common patterns in LLM responses for insights
// Try to extract description, pattern, message, etc.
// Priority 1: Look for description field
if desc, ok := obj["description"].(string); ok && desc != "" {
if pattern, ok := obj["pattern"].(string); ok && pattern != "" {
return fmt.Sprintf("%s: %s", pattern, desc)
}
return desc
}
// Priority 2: Look for message field
if msg, ok := obj["message"].(string); ok && msg != "" {
return msg
}
// Priority 3: Look for explanation/implication field
if expl, ok := obj["explanation"].(string); ok && expl != "" {
return expl
}
if impl, ok := obj["implication"].(string); ok && impl != "" {
return impl
}
// Priority 4: Combine all string fields
parts := []string{}
for key, value := range obj {
if str, ok := value.(string); ok && str != "" {
parts = append(parts, fmt.Sprintf("%s: %s", key, str))
}
}
if len(parts) > 0 {
return strings.Join(parts, ", ")
}
return ""
}
// extractInsightText extracts readable text from nested JSON structures
func (a *OllamaAgent) extractInsightText(key string, value interface{}) string {
switch v := value.(type) {
case string:
if v != "" {
return fmt.Sprintf("%s: %s", key, v)
}
case map[string]interface{}:
// For nested objects, create a summary
parts := []string{}
for subKey, subValue := range v {
if str, ok := subValue.(string); ok && str != "" {
parts = append(parts, fmt.Sprintf("%s=%s", subKey, str))
}
}
if len(parts) > 0 {
return fmt.Sprintf("%s: %s", key, strings.Join(parts, ", "))
}
case []interface{}:
// For arrays, join elements
parts := []string{}
for _, item := range v {
if str, ok := item.(string); ok && str != "" {
parts = append(parts, str)
}
}
if len(parts) > 0 {
return fmt.Sprintf("%s: %s", key, strings.Join(parts, ", "))
}
case float64, int, bool:
return fmt.Sprintf("%s: %v", key, v)
}
return ""
}
// getStringField extracts a string field from a map, trying multiple key variants
func (a *OllamaAgent) getStringField(m map[string]interface{}, keys ...string) string {
for _, key := range keys {
if val, ok := m[key]; ok {
if str, ok := val.(string); ok {
return str
}
}
}
return ""
}
// extractRemediation extracts remediation info from various JSON structures
func (a *OllamaAgent) extractRemediation(result *analyzer.AnalyzerResult, remData interface{}) {
switch rem := remData.(type) {
case map[string]interface{}:
// Single remediation object
desc := a.getStringField(rem, "description", "Description")
action := a.getStringField(rem, "action", "Action")
command := a.getStringField(rem, "command", "Command")
priority := 5 // default priority
if p, ok := rem["priority"].(float64); ok {
priority = int(p)
} else if p, ok := rem["Priority"].(float64); ok {
priority = int(p)
}
if desc != "" || action != "" {
result.Remediation = &analyzer.RemediationStep{
Description: desc,
Action: action,
Command: command,
Priority: priority,
Category: "ai-suggested",
IsAutomatable: false,
}
}
case []interface{}:
// Array of remediation suggestions - use the first one
if len(rem) > 0 {
if firstRem, ok := rem[0].(map[string]interface{}); ok {
a.extractRemediation(result, firstRem)
}
}
}
}
// parseLLMResponse parses the LLM response into an AnalyzerResult
func (a *OllamaAgent) parseLLMResponse(response string, spec analyzer.AnalyzerSpec) (*analyzer.AnalyzerResult, error) {
// First try JSON parsing
@@ -1416,53 +875,55 @@ func (a *OllamaAgent) parseLLMResponse(response string, spec analyzer.AnalyzerSp
if jsonStart != -1 && jsonEnd != -1 && jsonEnd > jsonStart {
jsonStr := response[jsonStart : jsonEnd+1]
// Try with a flexible map first to handle case-insensitive fields
var jsonMap map[string]interface{}
if err := json.Unmarshal([]byte(jsonStr), &jsonMap); err != nil {
var llmResult struct {
Status string `json:"status"`
Title string `json:"title"`
Message string `json:"message"`
Insights []string `json:"insights"`
Remediation struct {
Description string `json:"description"`
Action string `json:"action"`
Command string `json:"command"`
Priority int `json:"priority"`
} `json:"remediation"`
}
if err := json.Unmarshal([]byte(jsonStr), &llmResult); err == nil {
// Successfully parsed JSON
result := &analyzer.AnalyzerResult{
Title: llmResult.Title,
Message: llmResult.Message,
Category: spec.Category,
Insights: llmResult.Insights,
}
switch strings.ToLower(llmResult.Status) {
case "pass":
result.IsPass = true
case "warn":
result.IsWarn = true
case "fail":
result.IsFail = true
default:
result.IsWarn = true
}
if llmResult.Remediation.Description != "" {
result.Remediation = &analyzer.RemediationStep{
Description: llmResult.Remediation.Description,
Action: llmResult.Remediation.Action,
Command: llmResult.Remediation.Command,
Priority: llmResult.Remediation.Priority,
Category: "ai-suggested",
IsAutomatable: false,
}
}
return result, nil
} else {
// JSON was found but malformed
return nil, errors.Wrap(err, "failed to parse LLM JSON response")
}
// Extract fields in a case-insensitive way
status := a.getStringField(jsonMap, "status", "Status")
title := a.getStringField(jsonMap, "title", "Title")
message := a.getStringField(jsonMap, "message", "Message")
// Get insights field (try both lowercase and uppercase)
var insightsRaw json.RawMessage
if insights, ok := jsonMap["insights"]; ok {
insightsRaw, _ = json.Marshal(insights)
} else if insights, ok := jsonMap["Insights"]; ok {
insightsRaw, _ = json.Marshal(insights)
}
insights := a.normalizeInsights(insightsRaw)
result := &analyzer.AnalyzerResult{
Title: title,
Message: message,
Category: spec.Category,
Insights: insights,
}
switch strings.ToLower(status) {
case "pass":
result.IsPass = true
case "warn":
result.IsWarn = true
case "fail":
result.IsFail = true
default:
result.IsWarn = true
}
// Handle remediation (try both cases)
if rem, ok := jsonMap["remediation"]; ok {
a.extractRemediation(result, rem)
} else if rem, ok := jsonMap["Remediation"]; ok {
a.extractRemediation(result, rem)
}
return result, nil
}
// Fall back to markdown parsing when JSON fails
@@ -206,10 +206,7 @@ func TestOllamaAgent_discoverAnalyzers(t *testing.T) {
assert.NotNil(t, spec.Config)
// Verify AI-specific config
// Aggregated analyzers use "filePaths", single-file analyzers use "filePath"
hasFilePath := spec.Config["filePath"] != nil
hasFilePaths := spec.Config["filePaths"] != nil
assert.True(t, hasFilePath || hasFilePaths, "spec must have either filePath or filePaths")
assert.Contains(t, spec.Config, "filePath")
assert.Contains(t, spec.Config, "promptType")
}
+1 -8
View File
@@ -146,7 +146,7 @@ func (h *OllamaHelper) downloadAndInstallWindows() error {
return errors.Wrap(err, "failed to create temporary file")
}
defer os.Remove(tmpFile.Name())
defer tmpFile.Close() // Ensures file is closed in error paths
defer tmpFile.Close()
// Download installer
resp, err := http.Get(h.downloadURL)
@@ -165,13 +165,6 @@ func (h *OllamaHelper) downloadAndInstallWindows() error {
return errors.Wrap(err, "failed to write installer")
}
// Close the file before executing it (required on Windows)
// Note: This will be called twice (here and via defer), but that's safe
// The defer ensures cleanup on error paths, this ensures closure before execution
if err := tmpFile.Close(); err != nil {
return errors.Wrap(err, "failed to close installer file")
}
// Run installer
klog.Info("Running Ollama installer...")
cmd := exec.Command(tmpFile.Name())
@@ -1,78 +0,0 @@
package v1beta3
import (
"github.com/replicatedhq/troubleshoot/pkg/multitype"
)
// CollectorMeta contains metadata for collectors
type CollectorMeta struct {
CollectorName string `json:"collectorName,omitempty" yaml:"collectorName,omitempty"`
// +optional
Exclude *multitype.BoolOrString `json:"exclude,omitempty" yaml:"exclude,omitempty"`
}
// Database represents database collectors (PostgreSQL, MySQL, Redis, MSSQL)
// In v1beta3, URI and TLS fields support valueFrom references
type Database struct {
CollectorMeta `json:",inline" yaml:",inline"`
// URI can be a literal value or reference to a Secret/ConfigMap
URI StringOrValueFrom `json:"uri" yaml:"uri"`
// Parameters for the database connection
Parameters []string `json:"parameters,omitempty"`
// TLS configuration with support for valueFrom references
TLS *TLSParams `json:"tls,omitempty" yaml:"tls,omitempty"`
}
// TLSParams contains TLS configuration
// In v1beta3, certificate fields support valueFrom references
type TLSParams struct {
// SkipVerify disables TLS verification
SkipVerify bool `json:"skipVerify,omitempty" yaml:"skipVerify,omitempty"`
// Secret references a Kubernetes Secret containing TLS materials (v1beta2 compatibility)
Secret *TLSSecret `json:"secret,omitempty" yaml:"secret,omitempty"`
// CACert can be a literal value or reference to a Secret/ConfigMap
CACert StringOrValueFrom `json:"cacert,omitempty" yaml:"cacert,omitempty"`
// ClientCert can be a literal value or reference to a Secret/ConfigMap
ClientCert StringOrValueFrom `json:"clientCert,omitempty" yaml:"clientCert,omitempty"`
// ClientKey can be a literal value or reference to a Secret/ConfigMap
ClientKey StringOrValueFrom `json:"clientKey,omitempty" yaml:"clientKey,omitempty"`
}
// TLSSecret references a Kubernetes Secret containing TLS materials
// Maintained for backward compatibility
type TLSSecret struct {
Name string `json:"name" yaml:"name"`
Namespace string `json:"namespace" yaml:"namespace"`
}
// Temporary placeholder types for minimal v1beta3 implementation
// These will be properly defined as we expand v1beta3 support
type AfterCollection struct {
CollectorMeta `json:",inline" yaml:",inline"`
// TODO: Add fields as needed
}
type Analyze struct {
// TODO: Add fields as needed
}
type HostAnalyze struct {
// TODO: Add fields as needed
}
type HostCollect struct {
// TODO: Add fields as needed
}
// Collect contains all collector definitions
// For phase 1, we're focusing on Database collectors with StringOrValueFrom support
type Collect struct {
// Database collectors with v1beta3 StringOrValueFrom support
Postgres *Database `json:"postgres,omitempty" yaml:"postgres,omitempty"`
Mssql *Database `json:"mssql,omitempty" yaml:"mssql,omitempty"`
Mysql *Database `json:"mysql,omitempty" yaml:"mysql,omitempty"`
Redis *Database `json:"redis,omitempty" yaml:"redis,omitempty"`
// TODO: Add remaining collector types as we expand v1beta3 support
// For now, these are placeholders to make the types compile
}
-162
View File
@@ -1,162 +0,0 @@
package v1beta3
import (
"context"
"fmt"
troubleshootv1beta2 "github.com/replicatedhq/troubleshoot/pkg/apis/troubleshoot/v1beta2"
"k8s.io/client-go/kubernetes"
)
// ConvertToV1Beta2WithResolution converts a v1beta3 SupportBundleSpec to v1beta2
// by resolving all StringOrValueFrom fields to their actual values
func ConvertToV1Beta2WithResolution(
ctx context.Context,
v3spec *SupportBundleSpec,
client kubernetes.Interface,
defaultNamespace string,
) (*troubleshootv1beta2.SupportBundleSpec, error) {
v2spec := &troubleshootv1beta2.SupportBundleSpec{
Uri: v3spec.Uri,
RunHostCollectorsInPod: v3spec.RunHostCollectorsInPod,
}
// Convert collectors
if v3spec.Collectors != nil {
v2collectors := make([]*troubleshootv1beta2.Collect, 0, len(v3spec.Collectors))
for _, v3collector := range v3spec.Collectors {
v2collector, err := convertCollector(ctx, v3collector, client, defaultNamespace)
if err != nil {
return nil, fmt.Errorf("failed to convert collector: %w", err)
}
v2collectors = append(v2collectors, v2collector)
}
v2spec.Collectors = v2collectors
}
// TODO: Convert AfterCollection, HostCollectors, Analyzers, HostAnalyzers when v1beta3 support is expanded
return v2spec, nil
}
// convertCollector converts a v1beta3 Collect to v1beta2 Collect
func convertCollector(
ctx context.Context,
v3collector *Collect,
client kubernetes.Interface,
defaultNamespace string,
) (*troubleshootv1beta2.Collect, error) {
v2collector := &troubleshootv1beta2.Collect{}
// Convert database collectors
if v3collector.Postgres != nil {
db, err := convertDatabase(ctx, v3collector.Postgres, client, defaultNamespace)
if err != nil {
return nil, fmt.Errorf("failed to convert postgres collector: %w", err)
}
v2collector.Postgres = db
}
if v3collector.Mysql != nil {
db, err := convertDatabase(ctx, v3collector.Mysql, client, defaultNamespace)
if err != nil {
return nil, fmt.Errorf("failed to convert mysql collector: %w", err)
}
v2collector.Mysql = db
}
if v3collector.Mssql != nil {
db, err := convertDatabase(ctx, v3collector.Mssql, client, defaultNamespace)
if err != nil {
return nil, fmt.Errorf("failed to convert mssql collector: %w", err)
}
v2collector.Mssql = db
}
if v3collector.Redis != nil {
db, err := convertDatabase(ctx, v3collector.Redis, client, defaultNamespace)
if err != nil {
return nil, fmt.Errorf("failed to convert redis collector: %w", err)
}
v2collector.Redis = db
}
// TODO: Add conversion for other collector types as v1beta3 support expands
return v2collector, nil
}
// convertDatabase converts a v1beta3 Database to v1beta2 Database
func convertDatabase(
ctx context.Context,
v3db *Database,
client kubernetes.Interface,
defaultNamespace string,
) (*troubleshootv1beta2.Database, error) {
// Resolve URI
uri, err := ResolveStringOrValueFrom(ctx, v3db.URI, client, defaultNamespace)
if err != nil {
return nil, fmt.Errorf("failed to resolve database URI: %w", err)
}
v2db := &troubleshootv1beta2.Database{
CollectorMeta: troubleshootv1beta2.CollectorMeta{
CollectorName: v3db.CollectorName,
Exclude: v3db.Exclude,
},
URI: uri,
Parameters: v3db.Parameters,
}
// Convert TLS params if present
if v3db.TLS != nil {
tlsParams, err := convertTLSParams(ctx, v3db.TLS, client, defaultNamespace)
if err != nil {
return nil, fmt.Errorf("failed to convert TLS params: %w", err)
}
v2db.TLS = tlsParams
}
return v2db, nil
}
// convertTLSParams converts v1beta3 TLSParams to v1beta2 TLSParams
func convertTLSParams(
ctx context.Context,
v3tls *TLSParams,
client kubernetes.Interface,
defaultNamespace string,
) (*troubleshootv1beta2.TLSParams, error) {
v2tls := &troubleshootv1beta2.TLSParams{
SkipVerify: v3tls.SkipVerify,
}
// Preserve v1beta2 Secret reference if present (backward compatibility)
if v3tls.Secret != nil {
v2tls.Secret = &troubleshootv1beta2.TLSSecret{
Name: v3tls.Secret.Name,
Namespace: v3tls.Secret.Namespace,
}
}
// Resolve v1beta3 StringOrValueFrom fields
caCert, err := ResolveStringOrValueFrom(ctx, v3tls.CACert, client, defaultNamespace)
if err != nil {
return nil, fmt.Errorf("failed to resolve CA cert: %w", err)
}
v2tls.CACert = caCert
clientCert, err := ResolveStringOrValueFrom(ctx, v3tls.ClientCert, client, defaultNamespace)
if err != nil {
return nil, fmt.Errorf("failed to resolve client cert: %w", err)
}
v2tls.ClientCert = clientCert
clientKey, err := ResolveStringOrValueFrom(ctx, v3tls.ClientKey, client, defaultNamespace)
if err != nil {
return nil, fmt.Errorf("failed to resolve client key: %w", err)
}
v2tls.ClientKey = clientKey
return v2tls, nil
}
@@ -1,386 +0,0 @@
package v1beta3
import (
"context"
"testing"
troubleshootv1beta2 "github.com/replicatedhq/troubleshoot/pkg/apis/troubleshoot/v1beta2"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes/fake"
)
func TestConvertToV1Beta2WithResolution_PostgresWithLiteralValue(t *testing.T) {
client := fake.NewSimpleClientset()
uri := "postgresql://user:pass@localhost:5432/db"
v3spec := &SupportBundleSpec{
Collectors: []*Collect{
{
Postgres: &Database{
URI: StringOrValueFrom{
Value: &uri,
},
},
},
},
}
v2spec, err := ConvertToV1Beta2WithResolution(context.Background(), v3spec, client, "default")
require.NoError(t, err)
require.NotNil(t, v2spec)
require.Len(t, v2spec.Collectors, 1)
require.NotNil(t, v2spec.Collectors[0].Postgres)
assert.Equal(t, "postgresql://user:pass@localhost:5432/db", v2spec.Collectors[0].Postgres.URI)
}
func TestConvertToV1Beta2WithResolution_PostgresWithSecretRef(t *testing.T) {
secret := &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Name: "db-secret",
Namespace: "default",
},
Data: map[string][]byte{
"uri": []byte("postgresql://user:secret-pass@db.example.com:5432/mydb"),
},
}
client := fake.NewSimpleClientset(secret)
v3spec := &SupportBundleSpec{
Collectors: []*Collect{
{
Postgres: &Database{
URI: StringOrValueFrom{
ValueFrom: &ValueFromSource{
SecretKeyRef: &SecretKeyRef{
Name: "db-secret",
Key: "uri",
},
},
},
},
},
},
}
v2spec, err := ConvertToV1Beta2WithResolution(context.Background(), v3spec, client, "default")
require.NoError(t, err)
require.NotNil(t, v2spec)
require.Len(t, v2spec.Collectors, 1)
require.NotNil(t, v2spec.Collectors[0].Postgres)
assert.Equal(t, "postgresql://user:secret-pass@db.example.com:5432/mydb", v2spec.Collectors[0].Postgres.URI)
}
func TestConvertToV1Beta2WithResolution_PostgresWithTLS(t *testing.T) {
secret := &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Name: "tls-secret",
Namespace: "default",
},
Data: map[string][]byte{
"ca.crt": []byte("-----BEGIN CERTIFICATE-----\nCA_CERT_DATA\n-----END CERTIFICATE-----"),
"client.crt": []byte("-----BEGIN CERTIFICATE-----\nCLIENT_CERT_DATA\n-----END CERTIFICATE-----"),
"client.key": []byte("-----BEGIN PRIVATE KEY-----\nCLIENT_KEY_DATA\n-----END PRIVATE KEY-----"),
},
}
client := fake.NewSimpleClientset(secret)
uri := "postgresql://user:pass@localhost:5432/db"
v3spec := &SupportBundleSpec{
Collectors: []*Collect{
{
Postgres: &Database{
URI: StringOrValueFrom{
Value: &uri,
},
TLS: &TLSParams{
CACert: StringOrValueFrom{
ValueFrom: &ValueFromSource{
SecretKeyRef: &SecretKeyRef{
Name: "tls-secret",
Key: "ca.crt",
},
},
},
ClientCert: StringOrValueFrom{
ValueFrom: &ValueFromSource{
SecretKeyRef: &SecretKeyRef{
Name: "tls-secret",
Key: "client.crt",
},
},
},
ClientKey: StringOrValueFrom{
ValueFrom: &ValueFromSource{
SecretKeyRef: &SecretKeyRef{
Name: "tls-secret",
Key: "client.key",
},
},
},
},
},
},
},
}
v2spec, err := ConvertToV1Beta2WithResolution(context.Background(), v3spec, client, "default")
require.NoError(t, err)
require.NotNil(t, v2spec)
require.Len(t, v2spec.Collectors, 1)
require.NotNil(t, v2spec.Collectors[0].Postgres)
require.NotNil(t, v2spec.Collectors[0].Postgres.TLS)
assert.Equal(t, "-----BEGIN CERTIFICATE-----\nCA_CERT_DATA\n-----END CERTIFICATE-----", v2spec.Collectors[0].Postgres.TLS.CACert)
assert.Equal(t, "-----BEGIN CERTIFICATE-----\nCLIENT_CERT_DATA\n-----END CERTIFICATE-----", v2spec.Collectors[0].Postgres.TLS.ClientCert)
assert.Equal(t, "-----BEGIN PRIVATE KEY-----\nCLIENT_KEY_DATA\n-----END PRIVATE KEY-----", v2spec.Collectors[0].Postgres.TLS.ClientKey)
}
func TestConvertToV1Beta2WithResolution_MultipleDatabases(t *testing.T) {
pgSecret := &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Name: "postgres-secret",
Namespace: "default",
},
Data: map[string][]byte{
"uri": []byte("postgresql://user:pass@pg.example.com:5432/db"),
},
}
mysqlSecret := &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Name: "mysql-secret",
Namespace: "default",
},
Data: map[string][]byte{
"uri": []byte("mysql://user:pass@mysql.example.com:3306/db"),
},
}
redisSecret := &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Name: "redis-secret",
Namespace: "default",
},
Data: map[string][]byte{
"uri": []byte("redis://redis.example.com:6379"),
},
}
client := fake.NewSimpleClientset(pgSecret, mysqlSecret, redisSecret)
v3spec := &SupportBundleSpec{
Collectors: []*Collect{
{
Postgres: &Database{
URI: StringOrValueFrom{
ValueFrom: &ValueFromSource{
SecretKeyRef: &SecretKeyRef{
Name: "postgres-secret",
Key: "uri",
},
},
},
},
},
{
Mysql: &Database{
URI: StringOrValueFrom{
ValueFrom: &ValueFromSource{
SecretKeyRef: &SecretKeyRef{
Name: "mysql-secret",
Key: "uri",
},
},
},
},
},
{
Redis: &Database{
URI: StringOrValueFrom{
ValueFrom: &ValueFromSource{
SecretKeyRef: &SecretKeyRef{
Name: "redis-secret",
Key: "uri",
},
},
},
},
},
},
}
v2spec, err := ConvertToV1Beta2WithResolution(context.Background(), v3spec, client, "default")
require.NoError(t, err)
require.NotNil(t, v2spec)
require.Len(t, v2spec.Collectors, 3)
require.NotNil(t, v2spec.Collectors[0].Postgres)
assert.Equal(t, "postgresql://user:pass@pg.example.com:5432/db", v2spec.Collectors[0].Postgres.URI)
require.NotNil(t, v2spec.Collectors[1].Mysql)
assert.Equal(t, "mysql://user:pass@mysql.example.com:3306/db", v2spec.Collectors[1].Mysql.URI)
require.NotNil(t, v2spec.Collectors[2].Redis)
assert.Equal(t, "redis://redis.example.com:6379", v2spec.Collectors[2].Redis.URI)
}
func TestConvertToV1Beta2WithResolution_SecretNotFound(t *testing.T) {
client := fake.NewSimpleClientset()
v3spec := &SupportBundleSpec{
Collectors: []*Collect{
{
Postgres: &Database{
URI: StringOrValueFrom{
ValueFrom: &ValueFromSource{
SecretKeyRef: &SecretKeyRef{
Name: "nonexistent-secret",
Key: "uri",
},
},
},
},
},
},
}
_, err := ConvertToV1Beta2WithResolution(context.Background(), v3spec, client, "default")
require.Error(t, err)
assert.Contains(t, err.Error(), "failed to convert collector")
assert.Contains(t, err.Error(), "failed to resolve database URI")
}
func TestConvertToV1Beta2WithResolution_PreservesCollectorMeta(t *testing.T) {
client := fake.NewSimpleClientset()
uri := "postgresql://user:pass@localhost:5432/db"
v3spec := &SupportBundleSpec{
Collectors: []*Collect{
{
Postgres: &Database{
CollectorMeta: CollectorMeta{
CollectorName: "my-postgres-collector",
},
URI: StringOrValueFrom{
Value: &uri,
},
Parameters: []string{"sslmode=require"},
},
},
},
}
v2spec, err := ConvertToV1Beta2WithResolution(context.Background(), v3spec, client, "default")
require.NoError(t, err)
require.NotNil(t, v2spec)
require.Len(t, v2spec.Collectors, 1)
require.NotNil(t, v2spec.Collectors[0].Postgres)
assert.Equal(t, "my-postgres-collector", v2spec.Collectors[0].Postgres.CollectorName)
assert.Equal(t, []string{"sslmode=require"}, v2spec.Collectors[0].Postgres.Parameters)
}
func TestConvertToV1Beta2WithResolution_TLSBackwardCompatibility(t *testing.T) {
client := fake.NewSimpleClientset()
uri := "postgresql://user:pass@localhost:5432/db"
v3spec := &SupportBundleSpec{
Collectors: []*Collect{
{
Postgres: &Database{
URI: StringOrValueFrom{
Value: &uri,
},
TLS: &TLSParams{
SkipVerify: true,
Secret: &TLSSecret{
Name: "old-tls-secret",
Namespace: "default",
},
},
},
},
},
}
v2spec, err := ConvertToV1Beta2WithResolution(context.Background(), v3spec, client, "default")
require.NoError(t, err)
require.NotNil(t, v2spec)
require.Len(t, v2spec.Collectors, 1)
require.NotNil(t, v2spec.Collectors[0].Postgres)
require.NotNil(t, v2spec.Collectors[0].Postgres.TLS)
assert.True(t, v2spec.Collectors[0].Postgres.TLS.SkipVerify)
require.NotNil(t, v2spec.Collectors[0].Postgres.TLS.Secret)
assert.Equal(t, "old-tls-secret", v2spec.Collectors[0].Postgres.TLS.Secret.Name)
assert.Equal(t, "default", v2spec.Collectors[0].Postgres.TLS.Secret.Namespace)
}
func TestConvertToV1Beta2WithResolution_EmptySpec(t *testing.T) {
client := fake.NewSimpleClientset()
v3spec := &SupportBundleSpec{}
v2spec, err := ConvertToV1Beta2WithResolution(context.Background(), v3spec, client, "default")
require.NoError(t, err)
require.NotNil(t, v2spec)
assert.Nil(t, v2spec.Collectors)
}
func TestConvertDatabase_AllDatabaseTypes(t *testing.T) {
secret := &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Name: "db-secret",
Namespace: "default",
},
Data: map[string][]byte{
"uri": []byte("test-uri"),
},
}
client := fake.NewSimpleClientset(secret)
v3db := &Database{
URI: StringOrValueFrom{
ValueFrom: &ValueFromSource{
SecretKeyRef: &SecretKeyRef{
Name: "db-secret",
Key: "uri",
},
},
},
}
// Test that the same database struct works for all DB types
ctx := context.Background()
pgDB, err := convertDatabase(ctx, v3db, client, "default")
require.NoError(t, err)
assert.Equal(t, "test-uri", pgDB.URI)
mysqlDB, err := convertDatabase(ctx, v3db, client, "default")
require.NoError(t, err)
assert.Equal(t, "test-uri", mysqlDB.URI)
mssqlDB, err := convertDatabase(ctx, v3db, client, "default")
require.NoError(t, err)
assert.Equal(t, "test-uri", mssqlDB.URI)
redisDB, err := convertDatabase(ctx, v3db, client, "default")
require.NoError(t, err)
assert.Equal(t, "test-uri", redisDB.URI)
}
// Helper function to convert v2spec back to ensure type compatibility
func ensureV2SpecCompatibility(v2spec *troubleshootv1beta2.SupportBundleSpec) {
// This function just exists to ensure the types are compatible
// If this compiles, we know the conversion produces valid v1beta2 types
_ = v2spec.Uri
_ = v2spec.Collectors
_ = v2spec.Analyzers
}
-6
View File
@@ -1,6 +0,0 @@
// +k8s:deepcopy-gen=package
// +k8s:defaulter-gen=TypeMeta
// +groupName=troubleshoot.sh
// Package v1beta3 is the v1beta3 version of the API.
package v1beta3
-46
View File
@@ -1,46 +0,0 @@
/*
Copyright 2019 Replicated, Inc..
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
// NOTE: Boilerplate only. Ignore this file.
// Package v1beta3 contains API Schema definitions for the troubleshoot v1beta3 API group
// +k8s:openapi-gen=true
// +k8s:deepcopy-gen=package,register
// +k8s:conversion-gen=github.com/replicatedhq/troubleshoot/pkg/apis/troubleshoot
// +k8s:defaulter-gen=TypeMeta
// +groupName=troubleshoot.sh
package v1beta3
import (
"k8s.io/apimachinery/pkg/runtime/schema"
"sigs.k8s.io/controller-runtime/pkg/scheme"
)
var (
// SchemeGroupVersion is group version used to register these objects
SchemeGroupVersion = schema.GroupVersion{Group: "troubleshoot.sh", Version: "v1beta3"}
// SchemeBuilder is used to add go types to the GroupVersionKind scheme
SchemeBuilder = &scheme.Builder{GroupVersion: SchemeGroupVersion}
// AddToScheme is required by pkg/client/...
AddToScheme = SchemeBuilder.AddToScheme
)
// Resource is required by pkg/client/listers/...
func Resource(resource string) schema.GroupResource {
return SchemeGroupVersion.WithResource(resource).GroupResource()
}
-117
View File
@@ -1,117 +0,0 @@
package v1beta3
import (
"context"
"fmt"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
)
// ResolveStringOrValueFrom resolves a StringOrValueFrom to its actual string value
// by fetching from Secrets or ConfigMaps as needed.
//
// Parameters:
// - ctx: Context for the resolution operation
// - sov: The StringOrValueFrom to resolve
// - client: Kubernetes client for fetching Secrets/ConfigMaps
// - defaultNamespace: Namespace to use when not specified in the reference
//
// Returns:
// - The resolved string value
// - An error if resolution fails (unless Optional is true)
func ResolveStringOrValueFrom(
ctx context.Context,
sov StringOrValueFrom,
client kubernetes.Interface,
defaultNamespace string,
) (string, error) {
// If Value is directly specified, use it
if sov.Value != nil {
return *sov.Value, nil
}
// If ValueFrom is not specified, return empty string
if sov.ValueFrom == nil {
return "", nil
}
// Resolve from SecretKeyRef
if sov.ValueFrom.SecretKeyRef != nil {
return resolveSecretKeyRef(ctx, sov.ValueFrom.SecretKeyRef, client, defaultNamespace)
}
// Resolve from ConfigMapKeyRef
if sov.ValueFrom.ConfigMapKeyRef != nil {
return resolveConfigMapKeyRef(ctx, sov.ValueFrom.ConfigMapKeyRef, client, defaultNamespace)
}
return "", nil
}
// resolveSecretKeyRef fetches a value from a Kubernetes Secret
func resolveSecretKeyRef(
ctx context.Context,
ref *SecretKeyRef,
client kubernetes.Interface,
defaultNamespace string,
) (string, error) {
namespace := ref.Namespace
if namespace == "" {
namespace = defaultNamespace
}
secret, err := client.CoreV1().Secrets(namespace).Get(ctx, ref.Name, metav1.GetOptions{})
if err != nil {
if isOptional(ref.Optional) {
return "", nil
}
return "", fmt.Errorf("failed to get secret %s/%s: %w", namespace, ref.Name, err)
}
value, ok := secret.Data[ref.Key]
if !ok {
if isOptional(ref.Optional) {
return "", nil
}
return "", fmt.Errorf("key %q not found in secret %s/%s", ref.Key, namespace, ref.Name)
}
return string(value), nil
}
// resolveConfigMapKeyRef fetches a value from a Kubernetes ConfigMap
func resolveConfigMapKeyRef(
ctx context.Context,
ref *ConfigMapKeyRef,
client kubernetes.Interface,
defaultNamespace string,
) (string, error) {
namespace := ref.Namespace
if namespace == "" {
namespace = defaultNamespace
}
configMap, err := client.CoreV1().ConfigMaps(namespace).Get(ctx, ref.Name, metav1.GetOptions{})
if err != nil {
if isOptional(ref.Optional) {
return "", nil
}
return "", fmt.Errorf("failed to get configmap %s/%s: %w", namespace, ref.Name, err)
}
value, ok := configMap.Data[ref.Key]
if !ok {
if isOptional(ref.Optional) {
return "", nil
}
return "", fmt.Errorf("key %q not found in configmap %s/%s", ref.Key, namespace, ref.Name)
}
return value, nil
}
// isOptional checks if the optional flag is set to true
func isOptional(optional *bool) bool {
return optional != nil && *optional
}
@@ -1,333 +0,0 @@
package v1beta3
import (
"context"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes/fake"
)
func TestResolveStringOrValueFrom_LiteralValue(t *testing.T) {
client := fake.NewSimpleClientset()
value := "literal-value"
sov := StringOrValueFrom{
Value: &value,
}
result, err := ResolveStringOrValueFrom(context.Background(), sov, client, "default")
require.NoError(t, err)
assert.Equal(t, "literal-value", result)
}
func TestResolveStringOrValueFrom_EmptyValue(t *testing.T) {
client := fake.NewSimpleClientset()
sov := StringOrValueFrom{}
result, err := ResolveStringOrValueFrom(context.Background(), sov, client, "default")
require.NoError(t, err)
assert.Equal(t, "", result)
}
func TestResolveStringOrValueFrom_SecretKeyRef(t *testing.T) {
secret := &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Name: "test-secret",
Namespace: "default",
},
Data: map[string][]byte{
"password": []byte("super-secret-password"),
},
}
client := fake.NewSimpleClientset(secret)
sov := StringOrValueFrom{
ValueFrom: &ValueFromSource{
SecretKeyRef: &SecretKeyRef{
Name: "test-secret",
Key: "password",
},
},
}
result, err := ResolveStringOrValueFrom(context.Background(), sov, client, "default")
require.NoError(t, err)
assert.Equal(t, "super-secret-password", result)
}
func TestResolveStringOrValueFrom_SecretKeyRef_WithNamespace(t *testing.T) {
secret := &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Name: "test-secret",
Namespace: "custom-namespace",
},
Data: map[string][]byte{
"password": []byte("secret-from-custom-ns"),
},
}
client := fake.NewSimpleClientset(secret)
sov := StringOrValueFrom{
ValueFrom: &ValueFromSource{
SecretKeyRef: &SecretKeyRef{
Name: "test-secret",
Key: "password",
Namespace: "custom-namespace",
},
},
}
result, err := ResolveStringOrValueFrom(context.Background(), sov, client, "default")
require.NoError(t, err)
assert.Equal(t, "secret-from-custom-ns", result)
}
func TestResolveStringOrValueFrom_SecretKeyRef_NotFound(t *testing.T) {
client := fake.NewSimpleClientset()
sov := StringOrValueFrom{
ValueFrom: &ValueFromSource{
SecretKeyRef: &SecretKeyRef{
Name: "nonexistent-secret",
Key: "password",
},
},
}
result, err := ResolveStringOrValueFrom(context.Background(), sov, client, "default")
require.Error(t, err)
assert.Contains(t, err.Error(), "failed to get secret")
assert.Equal(t, "", result)
}
func TestResolveStringOrValueFrom_SecretKeyRef_KeyNotFound(t *testing.T) {
secret := &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Name: "test-secret",
Namespace: "default",
},
Data: map[string][]byte{
"password": []byte("secret-value"),
},
}
client := fake.NewSimpleClientset(secret)
sov := StringOrValueFrom{
ValueFrom: &ValueFromSource{
SecretKeyRef: &SecretKeyRef{
Name: "test-secret",
Key: "nonexistent-key",
},
},
}
result, err := ResolveStringOrValueFrom(context.Background(), sov, client, "default")
require.Error(t, err)
assert.Contains(t, err.Error(), "key \"nonexistent-key\" not found")
assert.Equal(t, "", result)
}
func TestResolveStringOrValueFrom_SecretKeyRef_Optional(t *testing.T) {
client := fake.NewSimpleClientset()
optional := true
sov := StringOrValueFrom{
ValueFrom: &ValueFromSource{
SecretKeyRef: &SecretKeyRef{
Name: "nonexistent-secret",
Key: "password",
Optional: &optional,
},
},
}
result, err := ResolveStringOrValueFrom(context.Background(), sov, client, "default")
require.NoError(t, err)
assert.Equal(t, "", result)
}
func TestResolveStringOrValueFrom_SecretKeyRef_OptionalKeyNotFound(t *testing.T) {
secret := &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Name: "test-secret",
Namespace: "default",
},
Data: map[string][]byte{
"password": []byte("secret-value"),
},
}
client := fake.NewSimpleClientset(secret)
optional := true
sov := StringOrValueFrom{
ValueFrom: &ValueFromSource{
SecretKeyRef: &SecretKeyRef{
Name: "test-secret",
Key: "nonexistent-key",
Optional: &optional,
},
},
}
result, err := ResolveStringOrValueFrom(context.Background(), sov, client, "default")
require.NoError(t, err)
assert.Equal(t, "", result)
}
func TestResolveStringOrValueFrom_ConfigMapKeyRef(t *testing.T) {
configMap := &corev1.ConfigMap{
ObjectMeta: metav1.ObjectMeta{
Name: "test-configmap",
Namespace: "default",
},
Data: map[string]string{
"config-key": "config-value",
},
}
client := fake.NewSimpleClientset(configMap)
sov := StringOrValueFrom{
ValueFrom: &ValueFromSource{
ConfigMapKeyRef: &ConfigMapKeyRef{
Name: "test-configmap",
Key: "config-key",
},
},
}
result, err := ResolveStringOrValueFrom(context.Background(), sov, client, "default")
require.NoError(t, err)
assert.Equal(t, "config-value", result)
}
func TestResolveStringOrValueFrom_ConfigMapKeyRef_WithNamespace(t *testing.T) {
configMap := &corev1.ConfigMap{
ObjectMeta: metav1.ObjectMeta{
Name: "test-configmap",
Namespace: "custom-namespace",
},
Data: map[string]string{
"config-key": "config-from-custom-ns",
},
}
client := fake.NewSimpleClientset(configMap)
sov := StringOrValueFrom{
ValueFrom: &ValueFromSource{
ConfigMapKeyRef: &ConfigMapKeyRef{
Name: "test-configmap",
Key: "config-key",
Namespace: "custom-namespace",
},
},
}
result, err := ResolveStringOrValueFrom(context.Background(), sov, client, "default")
require.NoError(t, err)
assert.Equal(t, "config-from-custom-ns", result)
}
func TestResolveStringOrValueFrom_ConfigMapKeyRef_NotFound(t *testing.T) {
client := fake.NewSimpleClientset()
sov := StringOrValueFrom{
ValueFrom: &ValueFromSource{
ConfigMapKeyRef: &ConfigMapKeyRef{
Name: "nonexistent-configmap",
Key: "config-key",
},
},
}
result, err := ResolveStringOrValueFrom(context.Background(), sov, client, "default")
require.Error(t, err)
assert.Contains(t, err.Error(), "failed to get configmap")
assert.Equal(t, "", result)
}
func TestResolveStringOrValueFrom_ConfigMapKeyRef_KeyNotFound(t *testing.T) {
configMap := &corev1.ConfigMap{
ObjectMeta: metav1.ObjectMeta{
Name: "test-configmap",
Namespace: "default",
},
Data: map[string]string{
"config-key": "config-value",
},
}
client := fake.NewSimpleClientset(configMap)
sov := StringOrValueFrom{
ValueFrom: &ValueFromSource{
ConfigMapKeyRef: &ConfigMapKeyRef{
Name: "test-configmap",
Key: "nonexistent-key",
},
},
}
result, err := ResolveStringOrValueFrom(context.Background(), sov, client, "default")
require.Error(t, err)
assert.Contains(t, err.Error(), "key \"nonexistent-key\" not found")
assert.Equal(t, "", result)
}
func TestResolveStringOrValueFrom_ConfigMapKeyRef_Optional(t *testing.T) {
client := fake.NewSimpleClientset()
optional := true
sov := StringOrValueFrom{
ValueFrom: &ValueFromSource{
ConfigMapKeyRef: &ConfigMapKeyRef{
Name: "nonexistent-configmap",
Key: "config-key",
Optional: &optional,
},
},
}
result, err := ResolveStringOrValueFrom(context.Background(), sov, client, "default")
require.NoError(t, err)
assert.Equal(t, "", result)
}
func TestResolveStringOrValueFrom_ConfigMapKeyRef_OptionalKeyNotFound(t *testing.T) {
configMap := &corev1.ConfigMap{
ObjectMeta: metav1.ObjectMeta{
Name: "test-configmap",
Namespace: "default",
},
Data: map[string]string{
"config-key": "config-value",
},
}
client := fake.NewSimpleClientset(configMap)
optional := true
sov := StringOrValueFrom{
ValueFrom: &ValueFromSource{
ConfigMapKeyRef: &ConfigMapKeyRef{
Name: "test-configmap",
Key: "nonexistent-key",
Optional: &optional,
},
},
}
result, err := ResolveStringOrValueFrom(context.Background(), sov, client, "default")
require.NoError(t, err)
assert.Equal(t, "", result)
}
@@ -1,64 +0,0 @@
package v1beta3
// StringOrValueFrom represents a string value that can either be specified
// directly or sourced from a Kubernetes Secret or ConfigMap
type StringOrValueFrom struct {
// Value is a literal string value
// +optional
Value *string `json:"value,omitempty" yaml:"value,omitempty"`
// ValueFrom is a reference to a value in a Secret or ConfigMap
// +optional
ValueFrom *ValueFromSource `json:"valueFrom,omitempty" yaml:"valueFrom,omitempty"`
}
// ValueFromSource represents the source of a value from a Secret or ConfigMap
type ValueFromSource struct {
// SecretKeyRef references a key in a Secret
// +optional
SecretKeyRef *SecretKeyRef `json:"secretKeyRef,omitempty" yaml:"secretKeyRef,omitempty"`
// ConfigMapKeyRef references a key in a ConfigMap
// +optional
ConfigMapKeyRef *ConfigMapKeyRef `json:"configMapKeyRef,omitempty" yaml:"configMapKeyRef,omitempty"`
}
// SecretKeyRef references a specific key in a Kubernetes Secret
type SecretKeyRef struct {
// Name is the name of the Secret
Name string `json:"name" yaml:"name"`
// Key is the key within the Secret to read
Key string `json:"key" yaml:"key"`
// Namespace is the namespace of the Secret
// If not specified, defaults to the namespace where the SupportBundle is running
// +optional
Namespace string `json:"namespace,omitempty" yaml:"namespace,omitempty"`
// Optional specifies whether the Secret must exist
// If true and the Secret or key doesn't exist, resolves to empty string
// If false (default) and the Secret or key doesn't exist, resolution fails
// +optional
Optional *bool `json:"optional,omitempty" yaml:"optional,omitempty"`
}
// ConfigMapKeyRef references a specific key in a Kubernetes ConfigMap
type ConfigMapKeyRef struct {
// Name is the name of the ConfigMap
Name string `json:"name" yaml:"name"`
// Key is the key within the ConfigMap to read
Key string `json:"key" yaml:"key"`
// Namespace is the namespace of the ConfigMap
// If not specified, defaults to the namespace where the SupportBundle is running
// +optional
Namespace string `json:"namespace,omitempty" yaml:"namespace,omitempty"`
// Optional specifies whether the ConfigMap must exist
// If true and the ConfigMap or key doesn't exist, resolves to empty string
// If false (default) and the ConfigMap or key doesn't exist, resolution fails
// +optional
Optional *bool `json:"optional,omitempty" yaml:"optional,omitempty"`
}
@@ -1,65 +0,0 @@
/*
Copyright 2019 Replicated, Inc..
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package v1beta3
import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
// SupportBundleSpec defines the desired state of SupportBundle
type SupportBundleSpec struct {
AfterCollection []*AfterCollection `json:"afterCollection,omitempty" yaml:"afterCollection,omitempty"`
Collectors []*Collect `json:"collectors,omitempty" yaml:"collectors,omitempty"`
HostCollectors []*HostCollect `json:"hostCollectors,omitempty" yaml:"hostCollectors,omitempty"`
Analyzers []*Analyze `json:"analyzers,omitempty" yaml:"analyzers,omitempty"`
HostAnalyzers []*HostAnalyze `json:"hostAnalyzers,omitempty" yaml:"hostAnalyzers,omitempty"`
// URI optionally defines a location which is the source of this spec to allow updating of the spec at runtime
Uri string `json:"uri,omitempty" yaml:"uri,omitempty"`
RunHostCollectorsInPod bool `json:"runHostCollectorsInPod,omitempty" yaml:"runHostCollectorsInPod,omitempty"`
}
// SupportBundleStatus defines the observed state of SupportBundle
type SupportBundleStatus struct {
// INSERT ADDITIONAL STATUS FIELD - define observed state of cluster
// Important: Run "make" to regenerate code after modifying this file
}
// +genclient
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
// SupportBundle is the Schema for the SupportBundles API
// +k8s:openapi-gen=true
type SupportBundle struct {
metav1.TypeMeta `json:",inline" yaml:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty" yaml:"metadata,omitempty"`
Spec SupportBundleSpec `json:"spec,omitempty" yaml:"spec,omitempty"`
Status SupportBundleStatus `json:"status,omitempty"`
}
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
// SupportBundleList contains a list of SupportBundle
type SupportBundleList struct {
metav1.TypeMeta `json:",inline"`
metav1.ListMeta `json:"metadata,omitempty"`
Items []SupportBundle `json:"items"`
}
func init() {
SchemeBuilder.Register(&SupportBundle{}, &SupportBundleList{})
}
@@ -1,441 +0,0 @@
//go:build !ignore_autogenerated
/*
Copyright 2019 Replicated, Inc..
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
// Code generated by controller-gen. DO NOT EDIT.
package v1beta3
import (
"github.com/replicatedhq/troubleshoot/pkg/multitype"
runtime "k8s.io/apimachinery/pkg/runtime"
)
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *AfterCollection) DeepCopyInto(out *AfterCollection) {
*out = *in
in.CollectorMeta.DeepCopyInto(&out.CollectorMeta)
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AfterCollection.
func (in *AfterCollection) DeepCopy() *AfterCollection {
if in == nil {
return nil
}
out := new(AfterCollection)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *Analyze) DeepCopyInto(out *Analyze) {
*out = *in
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Analyze.
func (in *Analyze) DeepCopy() *Analyze {
if in == nil {
return nil
}
out := new(Analyze)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *Collect) DeepCopyInto(out *Collect) {
*out = *in
if in.Postgres != nil {
in, out := &in.Postgres, &out.Postgres
*out = new(Database)
(*in).DeepCopyInto(*out)
}
if in.Mssql != nil {
in, out := &in.Mssql, &out.Mssql
*out = new(Database)
(*in).DeepCopyInto(*out)
}
if in.Mysql != nil {
in, out := &in.Mysql, &out.Mysql
*out = new(Database)
(*in).DeepCopyInto(*out)
}
if in.Redis != nil {
in, out := &in.Redis, &out.Redis
*out = new(Database)
(*in).DeepCopyInto(*out)
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Collect.
func (in *Collect) DeepCopy() *Collect {
if in == nil {
return nil
}
out := new(Collect)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *CollectorMeta) DeepCopyInto(out *CollectorMeta) {
*out = *in
if in.Exclude != nil {
in, out := &in.Exclude, &out.Exclude
*out = new(multitype.BoolOrString)
**out = **in
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CollectorMeta.
func (in *CollectorMeta) DeepCopy() *CollectorMeta {
if in == nil {
return nil
}
out := new(CollectorMeta)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *ConfigMapKeyRef) DeepCopyInto(out *ConfigMapKeyRef) {
*out = *in
if in.Optional != nil {
in, out := &in.Optional, &out.Optional
*out = new(bool)
**out = **in
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ConfigMapKeyRef.
func (in *ConfigMapKeyRef) DeepCopy() *ConfigMapKeyRef {
if in == nil {
return nil
}
out := new(ConfigMapKeyRef)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *Database) DeepCopyInto(out *Database) {
*out = *in
in.CollectorMeta.DeepCopyInto(&out.CollectorMeta)
in.URI.DeepCopyInto(&out.URI)
if in.Parameters != nil {
in, out := &in.Parameters, &out.Parameters
*out = make([]string, len(*in))
copy(*out, *in)
}
if in.TLS != nil {
in, out := &in.TLS, &out.TLS
*out = new(TLSParams)
(*in).DeepCopyInto(*out)
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Database.
func (in *Database) DeepCopy() *Database {
if in == nil {
return nil
}
out := new(Database)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *HostAnalyze) DeepCopyInto(out *HostAnalyze) {
*out = *in
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new HostAnalyze.
func (in *HostAnalyze) DeepCopy() *HostAnalyze {
if in == nil {
return nil
}
out := new(HostAnalyze)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *HostCollect) DeepCopyInto(out *HostCollect) {
*out = *in
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new HostCollect.
func (in *HostCollect) DeepCopy() *HostCollect {
if in == nil {
return nil
}
out := new(HostCollect)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *SecretKeyRef) DeepCopyInto(out *SecretKeyRef) {
*out = *in
if in.Optional != nil {
in, out := &in.Optional, &out.Optional
*out = new(bool)
**out = **in
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SecretKeyRef.
func (in *SecretKeyRef) DeepCopy() *SecretKeyRef {
if in == nil {
return nil
}
out := new(SecretKeyRef)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *StringOrValueFrom) DeepCopyInto(out *StringOrValueFrom) {
*out = *in
if in.Value != nil {
in, out := &in.Value, &out.Value
*out = new(string)
**out = **in
}
if in.ValueFrom != nil {
in, out := &in.ValueFrom, &out.ValueFrom
*out = new(ValueFromSource)
(*in).DeepCopyInto(*out)
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new StringOrValueFrom.
func (in *StringOrValueFrom) DeepCopy() *StringOrValueFrom {
if in == nil {
return nil
}
out := new(StringOrValueFrom)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *SupportBundle) DeepCopyInto(out *SupportBundle) {
*out = *in
out.TypeMeta = in.TypeMeta
in.ObjectMeta.DeepCopyInto(&out.ObjectMeta)
in.Spec.DeepCopyInto(&out.Spec)
out.Status = in.Status
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SupportBundle.
func (in *SupportBundle) DeepCopy() *SupportBundle {
if in == nil {
return nil
}
out := new(SupportBundle)
in.DeepCopyInto(out)
return out
}
// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.
func (in *SupportBundle) DeepCopyObject() runtime.Object {
if c := in.DeepCopy(); c != nil {
return c
}
return nil
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *SupportBundleList) DeepCopyInto(out *SupportBundleList) {
*out = *in
out.TypeMeta = in.TypeMeta
in.ListMeta.DeepCopyInto(&out.ListMeta)
if in.Items != nil {
in, out := &in.Items, &out.Items
*out = make([]SupportBundle, len(*in))
for i := range *in {
(*in)[i].DeepCopyInto(&(*out)[i])
}
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SupportBundleList.
func (in *SupportBundleList) DeepCopy() *SupportBundleList {
if in == nil {
return nil
}
out := new(SupportBundleList)
in.DeepCopyInto(out)
return out
}
// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.
func (in *SupportBundleList) DeepCopyObject() runtime.Object {
if c := in.DeepCopy(); c != nil {
return c
}
return nil
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *SupportBundleSpec) DeepCopyInto(out *SupportBundleSpec) {
*out = *in
if in.AfterCollection != nil {
in, out := &in.AfterCollection, &out.AfterCollection
*out = make([]*AfterCollection, len(*in))
for i := range *in {
if (*in)[i] != nil {
in, out := &(*in)[i], &(*out)[i]
*out = new(AfterCollection)
(*in).DeepCopyInto(*out)
}
}
}
if in.Collectors != nil {
in, out := &in.Collectors, &out.Collectors
*out = make([]*Collect, len(*in))
for i := range *in {
if (*in)[i] != nil {
in, out := &(*in)[i], &(*out)[i]
*out = new(Collect)
(*in).DeepCopyInto(*out)
}
}
}
if in.HostCollectors != nil {
in, out := &in.HostCollectors, &out.HostCollectors
*out = make([]*HostCollect, len(*in))
for i := range *in {
if (*in)[i] != nil {
in, out := &(*in)[i], &(*out)[i]
*out = new(HostCollect)
**out = **in
}
}
}
if in.Analyzers != nil {
in, out := &in.Analyzers, &out.Analyzers
*out = make([]*Analyze, len(*in))
for i := range *in {
if (*in)[i] != nil {
in, out := &(*in)[i], &(*out)[i]
*out = new(Analyze)
**out = **in
}
}
}
if in.HostAnalyzers != nil {
in, out := &in.HostAnalyzers, &out.HostAnalyzers
*out = make([]*HostAnalyze, len(*in))
for i := range *in {
if (*in)[i] != nil {
in, out := &(*in)[i], &(*out)[i]
*out = new(HostAnalyze)
**out = **in
}
}
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SupportBundleSpec.
func (in *SupportBundleSpec) DeepCopy() *SupportBundleSpec {
if in == nil {
return nil
}
out := new(SupportBundleSpec)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *SupportBundleStatus) DeepCopyInto(out *SupportBundleStatus) {
*out = *in
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SupportBundleStatus.
func (in *SupportBundleStatus) DeepCopy() *SupportBundleStatus {
if in == nil {
return nil
}
out := new(SupportBundleStatus)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *TLSParams) DeepCopyInto(out *TLSParams) {
*out = *in
if in.Secret != nil {
in, out := &in.Secret, &out.Secret
*out = new(TLSSecret)
**out = **in
}
in.CACert.DeepCopyInto(&out.CACert)
in.ClientCert.DeepCopyInto(&out.ClientCert)
in.ClientKey.DeepCopyInto(&out.ClientKey)
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new TLSParams.
func (in *TLSParams) DeepCopy() *TLSParams {
if in == nil {
return nil
}
out := new(TLSParams)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *TLSSecret) DeepCopyInto(out *TLSSecret) {
*out = *in
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new TLSSecret.
func (in *TLSSecret) DeepCopy() *TLSSecret {
if in == nil {
return nil
}
out := new(TLSSecret)
in.DeepCopyInto(out)
return out
}
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *ValueFromSource) DeepCopyInto(out *ValueFromSource) {
*out = *in
if in.SecretKeyRef != nil {
in, out := &in.SecretKeyRef, &out.SecretKeyRef
*out = new(SecretKeyRef)
(*in).DeepCopyInto(*out)
}
if in.ConfigMapKeyRef != nil {
in, out := &in.ConfigMapKeyRef, &out.ConfigMapKeyRef
*out = new(ConfigMapKeyRef)
(*in).DeepCopyInto(*out)
}
}
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ValueFromSource.
func (in *ValueFromSource) DeepCopy() *ValueFromSource {
if in == nil {
return nil
}
out := new(ValueFromSource)
in.DeepCopyInto(out)
return out
}
@@ -20,7 +20,6 @@ package scheme
import (
troubleshootv1beta1 "github.com/replicatedhq/troubleshoot/pkg/apis/troubleshoot/v1beta1"
troubleshootv1beta2 "github.com/replicatedhq/troubleshoot/pkg/apis/troubleshoot/v1beta2"
troubleshootv1beta3 "github.com/replicatedhq/troubleshoot/pkg/apis/troubleshoot/v1beta3"
v1 "k8s.io/apimachinery/pkg/apis/meta/v1"
runtime "k8s.io/apimachinery/pkg/runtime"
schema "k8s.io/apimachinery/pkg/runtime/schema"
@@ -34,7 +33,6 @@ var ParameterCodec = runtime.NewParameterCodec(Scheme)
var localSchemeBuilder = runtime.SchemeBuilder{
troubleshootv1beta1.AddToScheme,
troubleshootv1beta2.AddToScheme,
troubleshootv1beta3.AddToScheme,
}
// AddToScheme adds all types of this clientset into the given scheme. This allows composition
-72
View File
@@ -400,13 +400,6 @@ func (c *CollectClusterResources) Collect(progressChan chan<- interface{}) (Coll
}
output.SaveResult(c.BundlePath, path.Join(constants.CLUSTER_RESOURCES_DIR, fmt.Sprintf("%s-errors.json", constants.CLUSTER_RESOURCES_CONFIGMAPS)), marshalErrors(configMapsErrors))
// Replicated License
licenseData, licenseErr := replicatedLicense(ctx, client, namespaceNames)
if licenseErr == nil {
output.SaveResult(c.BundlePath, path.Join(constants.CLUSTER_RESOURCES_DIR, constants.CLUSTER_RESOURCES_REPLICATED_LICENSE), bytes.NewBuffer(licenseData))
}
return output, nil
}
@@ -2183,68 +2176,3 @@ func storeCustomResource(name string, objects any, m map[string][]byte) error {
m[fmt.Sprintf("%s.yaml", name)] = y
return nil
}
// replicatedLicense searches for the replicated secret across namespaces,
// extracts the config.yaml field, and extracts the licenseID and appSlug.
// Note: secret.Data already contains decoded bytes; no base64 decoding is required.
func replicatedLicense(ctx context.Context, client *kubernetes.Clientset, namespaces []string) ([]byte, error) {
// Structure to parse the config.yaml content
type ConfigYAML struct {
License string `yaml:"license"` // This is a YAML string containing the License object
}
type LicenseSpec struct {
LicenseID string `yaml:"licenseID"`
AppSlug string `yaml:"appSlug"`
}
type License struct {
Spec LicenseSpec `yaml:"spec"`
}
// Search through all namespaces for the replicated secret
for _, namespace := range namespaces {
secret, err := client.CoreV1().Secrets(namespace).Get(ctx, "replicated", metav1.GetOptions{})
if err != nil {
// Secret not found in this namespace, continue to next
continue
}
// Extract the config.yaml field from the secret data
configYAMLBase64, exists := secret.Data["config.yaml"]
if !exists {
continue
}
configYAMLBytes := configYAMLBase64
// Parse the YAML to extract the license field
var config ConfigYAML
if err := yaml.Unmarshal(configYAMLBytes, &config); err != nil {
// Malformed config in this namespace; try the next namespace
continue
}
// Parse the license field (which is a YAML string) to extract licenseID and appSlug
var license License
if err := yaml.Unmarshal([]byte(config.License), &license); err != nil {
// Malformed license in this namespace; try the next namespace
continue
}
// Return both licenseID and appSlug as JSON
licenseData := map[string]string{
"licenseID": license.Spec.LicenseID,
"appSlug": license.Spec.AppSlug,
}
licenseJSON, err := json.Marshal(licenseData)
if err != nil {
return nil, fmt.Errorf("failed to marshal license data: %w", err)
}
return licenseJSON, nil
}
// No replicated secret with a parsable license found in any namespace
return nil, fmt.Errorf("replicated secret with parsable license not found in any namespace")
}
+3 -46
View File
@@ -10,16 +10,9 @@ import (
"github.com/pkg/errors"
troubleshootv1beta2 "github.com/replicatedhq/troubleshoot/pkg/apis/troubleshoot/v1beta2"
"github.com/replicatedhq/troubleshoot/pkg/k8sutil"
"helm.sh/helm/v3/pkg/action"
"k8s.io/apimachinery/pkg/api/meta"
"k8s.io/cli-runtime/pkg/genericclioptions"
"k8s.io/client-go/discovery"
"k8s.io/client-go/discovery/cached/memory"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/rest"
"k8s.io/client-go/restmapper"
"k8s.io/client-go/tools/clientcmd"
"k8s.io/klog/v2"
)
@@ -52,42 +45,6 @@ type VersionInfo struct {
Values map[string]interface{} `json:"values,omitempty"`
}
type configGetter struct {
restConfig *rest.Config
}
// ToDiscoveryClient implements genericclioptions.RESTClientGetter.
func (c configGetter) ToDiscoveryClient() (discovery.CachedDiscoveryInterface, error) {
discoveryClient, err := discovery.NewDiscoveryClientForConfig(c.restConfig)
if err != nil {
return nil, err
}
cached := memory.NewMemCacheClient(discoveryClient)
return cached, nil
}
// ToRESTConfig implements genericclioptions.RESTClientGetter.
func (c configGetter) ToRESTConfig() (*rest.Config, error) {
return c.restConfig, nil
}
// ToRESTMapper implements genericclioptions.RESTClientGetter.
func (c configGetter) ToRESTMapper() (meta.RESTMapper, error) {
discoveryClient, err := c.ToDiscoveryClient()
if err != nil {
return nil, err
}
mapper := restmapper.NewDeferredDiscoveryRESTMapper(discoveryClient)
return mapper, nil
}
// ToRawKubeConfigLoader implements genericclioptions.RESTClientGetter.
func (c configGetter) ToRawKubeConfigLoader() clientcmd.ClientConfig {
return k8sutil.GetKubeconfig()
}
var _ genericclioptions.RESTClientGetter = configGetter{}
func (c *CollectHelm) Title() string {
return getCollectorName(c)
}
@@ -100,7 +57,7 @@ func (c *CollectHelm) Collect(progressChan chan<- interface{}) (CollectorResult,
output := NewResult()
releaseInfos, err := helmReleaseHistoryCollector(c.ClientConfig, c.Collector.ReleaseName, c.Collector.Namespace, c.Collector.CollectValues)
releaseInfos, err := helmReleaseHistoryCollector(c.Collector.ReleaseName, c.Collector.Namespace, c.Collector.CollectValues)
if err != nil {
errsToMarhsal := []string{}
for _, e := range err {
@@ -131,12 +88,12 @@ func (c *CollectHelm) Collect(progressChan chan<- interface{}) (CollectorResult,
return output, nil
}
func helmReleaseHistoryCollector(config *rest.Config, releaseName string, namespace string, collectValues bool) ([]ReleaseInfo, []error) {
func helmReleaseHistoryCollector(releaseName string, namespace string, collectValues bool) ([]ReleaseInfo, []error) {
var results []ReleaseInfo
error_list := []error{}
actionConfig := new(action.Configuration)
if err := actionConfig.Init(configGetter{config}, namespace, "", klog.V(2).Infof); err != nil {
if err := actionConfig.Init(nil, namespace, "", klog.V(2).Infof); err != nil {
return nil, []error{err}
}
+1 -13
View File
@@ -80,19 +80,7 @@ func DeterministicIDForCollector(collector *troubleshootv1beta2.Collect) string
}
func selectorToString(selector []string) string {
result := strings.Replace(strings.Join(selector, "-"), "=", "-", -1)
// Sanitize characters that are invalid in Windows filenames: < > : " / \ | ? *
// Replace them with underscores to ensure cross-platform compatibility
result = strings.ReplaceAll(result, "*", "all")
result = strings.ReplaceAll(result, "?", "_")
result = strings.ReplaceAll(result, ":", "_")
result = strings.ReplaceAll(result, "<", "_")
result = strings.ReplaceAll(result, ">", "_")
result = strings.ReplaceAll(result, "|", "_")
result = strings.ReplaceAll(result, "\"", "_")
result = strings.ReplaceAll(result, "/", "_")
result = strings.ReplaceAll(result, "\\", "_")
return result
return strings.Replace(strings.Join(selector, "-"), "=", "-", -1)
}
func pathToString(path string) string {
-1
View File
@@ -61,7 +61,6 @@ const (
CLUSTER_RESOURCES_LEASES = "leases"
CLUSTER_RESOURCES_VOLUME_ATTACHMENTS = "volumeattachments"
CLUSTER_RESOURCES_CONFIGMAPS = "configmaps"
CLUSTER_RESOURCES_REPLICATED_LICENSE = "license.json"
// SelfSubjectRulesReview evaluation responses
SELFSUBJECTRULESREVIEW_ERROR_AUTHORIZATION_WEBHOOK_UNSUPPORTED = "webhook authorizer does not support user rule resolution"
-345
View File
@@ -1,345 +0,0 @@
package lint
import (
"regexp"
"strings"
"github.com/replicatedhq/troubleshoot/pkg/constants"
)
func applyFixesInMemory(content string, result LintResult) (string, bool, error) {
fixed := false
newContent := content
lines := strings.Split(newContent, "\n")
// Fix A: If templating errors exist in a v1beta2 file, upgrade apiVersion to v1beta3 (minimal, deterministic)
hasTemplateInV1beta2 := false
for _, e := range result.Errors {
if e.Field == "template" && strings.Contains(e.Message, "not supported in v1beta2") {
hasTemplateInV1beta2 = true
break
}
}
if hasTemplateInV1beta2 {
for i, line := range lines {
if strings.HasPrefix(strings.TrimSpace(line), "apiVersion:") && strings.Contains(line, constants.Troubleshootv1beta2Kind) {
indent := line[:len(line)-len(strings.TrimLeft(line, " \t"))]
lines[i] = indent + "apiVersion: " + constants.Troubleshootv1beta3Kind
fixed = true
break
}
}
}
// Sort errors by line number (descending) to avoid line number shifts when editing
errorsByLine := make(map[int][]LintError)
for _, err := range result.Errors {
if err.Line > 0 {
errorsByLine[err.Line] = append(errorsByLine[err.Line], err)
}
}
// Process errors line by line
for lineNum, errs := range errorsByLine {
if lineNum > len(lines) {
continue
}
line := lines[lineNum-1]
originalLine := line
for _, err := range errs {
// Fix 1: Add missing colon
if strings.Contains(err.Message, "could not find expected ':'") {
if !strings.Contains(line, ":") {
trimmed := strings.TrimSpace(line)
indent := line[:len(line)-len(strings.TrimLeft(line, " \t"))]
line = indent + trimmed + ":"
fixed = true
}
}
// Fix 2: Add missing leading dot in template expressions
if strings.Contains(err.Message, "Template expression may be missing leading dot:") {
// Extract the expression from the error message
re := regexp.MustCompile(`Template expression may be missing leading dot: \{\{ (.+?) \}\}`)
matches := re.FindStringSubmatch(err.Message)
if len(matches) > 1 {
badExpr := matches[1]
// Add the leading dot
fixedExpr := "." + badExpr
// Replace in the line
line = strings.Replace(line, "{{ "+badExpr+" }}", "{{ "+fixedExpr+" }}", 1)
line = strings.Replace(line, "{{"+badExpr+"}}", "{{"+fixedExpr+"}}", 1)
line = strings.Replace(line, "{{- "+badExpr+" }}", "{{- "+fixedExpr+" }}", 1)
line = strings.Replace(line, "{{- "+badExpr+" -}}", "{{- "+fixedExpr+" -}}", 1)
fixed = true
}
}
// Fix 3: Fix wrong apiVersion
if strings.Contains(err.Message, "File must contain apiVersion:") && err.Field == "apiVersion" {
if strings.Contains(line, "apiVersion:") && !strings.Contains(line, constants.Troubleshootv1beta3Kind) {
// Replace existing apiVersion with correct one
indent := line[:len(line)-len(strings.TrimLeft(line, " \t"))]
line = indent + "apiVersion: " + constants.Troubleshootv1beta3Kind
fixed = true
}
}
}
// Update the line if it changed
if line != originalLine {
lines[lineNum-1] = line
}
}
// Fix B: Wrap mapping under required list fields (collectors, hostCollectors, analyzers)
for _, err := range result.Errors {
if strings.HasPrefix(err.Message, "Expected 'collectors' to be a list") {
if wrapFirstChildAsList(&lines, "collectors:") {
fixed = true
}
}
if strings.HasPrefix(err.Message, "Expected 'hostCollectors' to be a list") {
if wrapFirstChildAsList(&lines, "hostCollectors:") || convertScalarToEmptyList(&lines, "hostCollectors:") {
fixed = true
}
}
if strings.HasPrefix(err.Message, "Expected 'analyzers' to be a list") {
if wrapFirstChildAsList(&lines, "analyzers:") {
fixed = true
}
}
}
// Fix C: Add missing required fields with empty placeholders (non-assumptive)
// Collectors
for _, err := range result.Errors {
if strings.HasPrefix(err.Message, "Missing required field '") && strings.Contains(err.Message, " for collector '") {
// Parse field and collector type
// e.g., Missing required field 'namespace' for collector 'ceph'
fieldName := between(err.Message, "Missing required field '", "'")
collectorType := betweenAfter(err.Message, "collector '", "'")
if fieldName == "" || collectorType == "" {
continue
}
// Only handle simple case where the list item is in {} form: "- type: {}"
// Find the list item line from current content
cur := strings.Join(lines, "\n")
lineNum := findCollectorLine(cur, "collectors", indexFromField(err.Field))
if lineNum > 0 {
li := lineNum - 1
if strings.Contains(lines[li], "- "+collectorType+": {}") {
indent := lines[li][:len(lines[li])-len(strings.TrimLeft(lines[li], " \t"))]
childIndent := indent + " "
// choose placeholder: outcomes -> [] ; others -> ""
placeholder := "\"\""
if fieldName == "outcomes" {
placeholder = "[]"
}
lines[li] = strings.Replace(lines[li], ": {}", ":\n"+childIndent+fieldName+": "+placeholder, 1)
fixed = true
} else if strings.Contains(lines[li], "- "+collectorType+":") {
// Multi-line mapping; insert missing field under this item
if insertMissingFieldUnderListItem(&lines, li, fieldName) {
fixed = true
}
}
}
}
}
// Analyzers
for _, err := range result.Errors {
if strings.HasPrefix(err.Message, "Missing required field '") && strings.Contains(err.Message, " for analyzer '") {
fieldName := between(err.Message, "Missing required field '", "'")
analyzerType := betweenAfter(err.Message, "analyzer '", "'")
if fieldName == "" || analyzerType == "" {
continue
}
cur := strings.Join(lines, "\n")
lineNum := findAnalyzerLine(cur, indexFromField(err.Field))
if lineNum > 0 {
li := lineNum - 1
if strings.Contains(lines[li], "- "+analyzerType+": {}") {
indent := lines[li][:len(lines[li])-len(strings.TrimLeft(lines[li], " \t"))]
childIndent := indent + " "
placeholder := "\"\""
if fieldName == "outcomes" {
placeholder = "[]"
}
lines[li] = strings.Replace(lines[li], ": {}", ":\n"+childIndent+fieldName+": "+placeholder, 1)
fixed = true
} else if strings.Contains(lines[li], "- "+analyzerType+":") {
if insertMissingFieldUnderListItem(&lines, li, fieldName) {
fixed = true
}
}
}
}
}
// Return fixed content if changes were made
if fixed {
newContent = strings.Join(lines, "\n")
return newContent, true, nil
}
return content, false, nil
}
// wrapFirstChildAsList prefixes the first child mapping line under the given key with '- '
func wrapFirstChildAsList(lines *[]string, key string) bool {
arr := *lines
// find key line index
baseIdx := -1
for i, l := range arr {
if strings.Contains(l, key) {
baseIdx = i
break
}
}
if baseIdx == -1 {
return false
}
baseIndent := arr[baseIdx][:len(arr[baseIdx])-len(strings.TrimLeft(arr[baseIdx], " \t"))]
// find first child line with greater indent
for j := baseIdx + 1; j < len(arr); j++ {
line := arr[j]
if strings.TrimSpace(line) == "" {
continue
}
// stop when indentation goes back to or less than base
if !strings.HasPrefix(line, baseIndent+" ") && !strings.HasPrefix(line, baseIndent+"\t") {
break
}
trimmed := strings.TrimSpace(line)
if strings.HasPrefix(trimmed, "- ") {
// already a list
return false
}
// prefix '- '
childIndent := line[:len(line)-len(strings.TrimLeft(line, " \t"))]
arr[j] = childIndent + "- " + strings.TrimSpace(line)
*lines = arr
return true
}
return false
}
// convertScalarToEmptyList changes `key: <scalar>` to `key: []` on the same line
func convertScalarToEmptyList(lines *[]string, key string) bool {
arr := *lines
for i, l := range arr {
trimmed := strings.TrimSpace(l)
if strings.HasPrefix(trimmed, key) {
// If already ends with ':' leave for wrapper; else replace value with []
if strings.HasSuffix(trimmed, ":") {
return false
}
// Replace everything after the first ':' with [] preserving indentation/key
parts := strings.SplitN(l, ":", 2)
if len(parts) == 2 {
arr[i] = parts[0] + ": []"
*lines = arr
return true
}
}
}
return false
}
// indexFromField extracts the numeric index from a path like spec.collectors[1] or spec.analyzers[0]
func indexFromField(field string) int {
// find [number]
start := strings.Index(field, "[")
end := strings.Index(field, "]")
if start == -1 || end == -1 || end <= start+1 {
return 0
}
numStr := field[start+1 : end]
// naive parse
n := 0
for _, ch := range numStr {
if ch < '0' || ch > '9' {
return 0
}
n = n*10 + int(ch-'0')
}
return n
}
// insertMissingFieldUnderListItem inserts "fieldName: <placeholder>" under list item at startIdx
// Placeholder is [] for outcomes, "" otherwise. Preserves indentation by using the next child indentation if available
func insertMissingFieldUnderListItem(lines *[]string, startIdx int, fieldName string) bool {
arr := *lines
baseLine := arr[startIdx]
baseIndent := baseLine[:len(baseLine)-len(strings.TrimLeft(baseLine, " \t"))]
// Determine child indentation: prefer next non-empty line's indent if deeper than base
childIndent := baseIndent + " "
insertPos := startIdx + 1
for j := startIdx + 1; j < len(arr); j++ {
if strings.TrimSpace(arr[j]) == "" {
insertPos = j + 1
continue
}
lineIndent := arr[j][:len(arr[j])-len(strings.TrimLeft(arr[j], " \t"))]
if len(lineIndent) > len(baseIndent) {
childIndent = lineIndent
insertPos = j
}
break
}
// Choose placeholder
placeholder := "\"\""
if fieldName == "outcomes" {
placeholder = "[]"
}
// Insert new line
newLine := childIndent + fieldName + ": " + placeholder
// Avoid duplicate insert if the field already exists within this block
for k := startIdx + 1; k < len(arr); k++ {
if strings.TrimSpace(arr[k]) == "" {
continue
}
// Stop when block ends (indentation returns to base or less)
kIndent := arr[k][:len(arr[k])-len(strings.TrimLeft(arr[k], " \t"))]
if len(kIndent) <= len(baseIndent) {
break
}
if strings.HasPrefix(strings.TrimSpace(arr[k]), fieldName+":") {
return false
}
}
arr = append(arr[:insertPos], append([]string{newLine}, arr[insertPos:]...)...)
*lines = arr
return true
}
// between extracts substring between prefix and suffix (first occurrences)
func between(s, prefix, suffix string) string {
i := strings.Index(s, prefix)
if i == -1 {
return ""
}
s2 := s[i+len(prefix):]
j := strings.Index(s2, suffix)
if j == -1 {
return ""
}
return s2[:j]
}
// betweenAfter extracts substring between prefix and suffix starting search after prefix
func betweenAfter(s, prefix, suffix string) string {
i := strings.Index(s, prefix)
if i == -1 {
return ""
}
s2 := s[i+len(prefix):]
j := strings.Index(s2, suffix)
if j == -1 {
return ""
}
return s2[:j]
}
-103
View File
@@ -1,103 +0,0 @@
package lint
import (
"fmt"
"strings"
)
// FormatResults formats lint results for output
func FormatResults(results []LintResult, format string) string {
if format == "json" {
return formatJSON(results)
}
return formatText(results)
}
func formatText(results []LintResult) string {
var output strings.Builder
totalErrors := 0
totalWarnings := 0
for _, result := range results {
if len(result.Errors) == 0 && len(result.Warnings) == 0 {
output.WriteString(fmt.Sprintf("✓ %s: No issues found\n", result.FilePath))
continue
}
output.WriteString(fmt.Sprintf("\n%s:\n", result.FilePath))
for _, err := range result.Errors {
output.WriteString(fmt.Sprintf(" ✗ Error (line %d): %s\n", err.Line, err.Message))
if err.Field != "" {
output.WriteString(fmt.Sprintf(" Field: %s\n", err.Field))
}
totalErrors++
}
for _, warn := range result.Warnings {
output.WriteString(fmt.Sprintf(" ⚠ Warning (line %d): %s\n", warn.Line, warn.Message))
if warn.Field != "" {
output.WriteString(fmt.Sprintf(" Field: %s\n", warn.Field))
}
totalWarnings++
}
}
output.WriteString(fmt.Sprintf("\nSummary: %d error(s), %d warning(s) across %d file(s)\n", totalErrors, totalWarnings, len(results)))
return output.String()
}
func formatJSON(results []LintResult) string {
// Simple JSON formatting without importing encoding/json
var output strings.Builder
output.WriteString("{\n")
output.WriteString(" \"results\": [\n")
for i, result := range results {
output.WriteString(" {\n")
output.WriteString(fmt.Sprintf(" \"filePath\": %q,\n", result.FilePath))
output.WriteString(" \"errors\": [\n")
for j, err := range result.Errors {
output.WriteString(" {\n")
output.WriteString(fmt.Sprintf(" \"line\": %d,\n", err.Line))
output.WriteString(fmt.Sprintf(" \"column\": %d,\n", err.Column))
output.WriteString(fmt.Sprintf(" \"message\": %q,\n", err.Message))
output.WriteString(fmt.Sprintf(" \"field\": %q\n", err.Field))
output.WriteString(" }")
if j < len(result.Errors)-1 {
output.WriteString(",")
}
output.WriteString("\n")
}
output.WriteString(" ],\n")
output.WriteString(" \"warnings\": [\n")
for j, warn := range result.Warnings {
output.WriteString(" {\n")
output.WriteString(fmt.Sprintf(" \"line\": %d,\n", warn.Line))
output.WriteString(fmt.Sprintf(" \"column\": %d,\n", warn.Column))
output.WriteString(fmt.Sprintf(" \"message\": %q,\n", warn.Message))
output.WriteString(fmt.Sprintf(" \"field\": %q\n", warn.Field))
output.WriteString(" }")
if j < len(result.Warnings)-1 {
output.WriteString(",")
}
output.WriteString("\n")
}
output.WriteString(" ]\n")
output.WriteString(" }")
if i < len(results)-1 {
output.WriteString(",")
}
output.WriteString("\n")
}
output.WriteString(" ]\n")
output.WriteString("}\n")
return output.String()
}
-63
View File
@@ -1,63 +0,0 @@
package lint
import (
"fmt"
"regexp"
"strings"
)
// findLineNumber returns the first 1-based line number containing the search string
func findLineNumber(content, search string) int {
lines := strings.Split(content, "\n")
for i, line := range lines {
if strings.Contains(line, search) {
return i + 1
}
}
return 0
}
func findAnalyzerLine(content string, index int) int {
return findListItemLine(content, "analyzers", index)
}
// findCollectorLine locates the starting line of the Nth entry in a collectors list
func findCollectorLine(content string, field string, index int) int {
return findListItemLine(content, field, index)
}
// findListItemLine locates the starting line of the Nth entry in a list under listKey
func findListItemLine(content, listKey string, index int) int {
lines := strings.Split(content, "\n")
count := 0
inList := false
for i, line := range lines {
if strings.Contains(line, listKey+":") {
inList = true
continue
}
if inList && strings.HasPrefix(strings.TrimSpace(line), "- ") {
if count == index {
return i + 1
}
count++
}
if inList && !strings.HasPrefix(line, " ") && !strings.HasPrefix(line, "\t") && strings.TrimSpace(line) != "" {
break
}
}
return 0
}
// extractLineFromError tries to parse a YAML error message for a line number
func extractLineFromError(err error) int {
// Try to extract line number from YAML error message
re := regexp.MustCompile(`line (\d+)`)
matches := re.FindStringSubmatch(err.Error())
if len(matches) > 1 {
var line int
fmt.Sscanf(matches[1], "%d", &line)
return line
}
return 0
}
+727 -205
View File
@@ -2,245 +2,767 @@ package lint
import (
"fmt"
"os"
"regexp"
"strings"
"sync"
"os"
"github.com/pkg/errors"
"github.com/replicatedhq/troubleshoot/internal/util"
"github.com/replicatedhq/troubleshoot/pkg/constants"
"sigs.k8s.io/yaml"
)
// LintFiles validates troubleshoot specs for syntax and structural errors
type LintResult struct {
FilePath string
Errors []LintError
Warnings []LintWarning
}
type LintError struct {
Line int
Column int
Message string
Field string
}
type LintWarning struct {
Line int
Column int
Message string
Field string
}
type LintOptions struct {
FilePaths []string
Fix bool
Format string // "text" or "json"
}
// LintFiles validates v1beta3 troubleshoot specs for syntax and structural errors
func LintFiles(opts LintOptions) ([]LintResult, error) {
results := []LintResult{}
// Load known analyzer/collector types from schemas (best effort)
ensureKnownTypesLoaded()
for _, filePath := range opts.FilePaths {
// Read entire file once
fileBytes, readErr := os.ReadFile(filePath)
if readErr != nil {
return nil, errors.Wrapf(readErr, "failed to read file %s", filePath)
result, err := lintFile(filePath, opts.Fix)
if err != nil {
return nil, err
}
fileContent := string(fileBytes)
// Split into YAML documents
docs := util.SplitYAML(fileContent)
// Pre-compute starting line number for each doc within the file (1-based)
docStarts := make([]int, len(docs))
runningStart := 1
for i, d := range docs {
docStarts[i] = runningStart
// Count lines in this doc
runningStart += util.EstimateNumberOfLines(d)
// Account for the '---' separator line between documents
if i < len(docs)-1 {
runningStart += 1
}
}
// Lint each document, in parallel
type docOutcome struct {
errs []LintError
warns []LintWarning
newDoc string
changed bool
}
outcomes := make([]docOutcome, len(docs))
var wg sync.WaitGroup
wg.Add(len(docs))
for i := range docs {
i := i
go func() {
defer wg.Done()
// Compute lint result for this doc, optionally applying fixes in-memory
res, finalDoc, _ /*changed*/, _ := lintContentInMemory(docs[i], opts.Fix)
// Adjust line numbers to file coordinates
lineOffset := docStarts[i] - 1
for idx := range res.Errors {
if res.Errors[idx].Line > 0 {
res.Errors[idx].Line += lineOffset
}
}
for idx := range res.Warnings {
if res.Warnings[idx].Line > 0 {
res.Warnings[idx].Line += lineOffset
}
}
changed := finalDoc != docs[i]
outcomes[i] = docOutcome{
errs: res.Errors,
warns: res.Warnings,
newDoc: finalDoc,
changed: changed,
}
}()
}
wg.Wait()
// Assemble per-file result
fileResult := LintResult{FilePath: filePath}
writeNeeded := false
newDocs := make([]string, len(docs))
for i, oc := range outcomes {
fileResult.Errors = append(fileResult.Errors, oc.errs...)
fileResult.Warnings = append(fileResult.Warnings, oc.warns...)
if oc.changed {
writeNeeded = true
}
if oc.newDoc == "" {
newDocs[i] = docs[i]
} else {
newDocs[i] = oc.newDoc
}
}
if writeNeeded {
// Reassemble with the same delimiter used by util.SplitYAML
updated := strings.Join(newDocs, "\n---\n")
if writeErr := os.WriteFile(filePath, []byte(updated), 0644); writeErr != nil {
return nil, errors.Wrapf(writeErr, "failed to write fixed content to %s", filePath)
}
}
results = append(results, fileResult)
results = append(results, result)
}
return results, nil
}
func lintContentInMemory(content string, fix bool) (LintResult, string, bool, error) {
// Compute result for the provided content
compute := func(body string) LintResult {
res := LintResult{Errors: []LintError{}, Warnings: []LintWarning{}}
func lintFile(filePath string, fix bool) (LintResult, error) {
result := LintResult{
FilePath: filePath,
Errors: []LintError{},
Warnings: []LintWarning{},
}
// Check if content contains template expressions
hasTemplates := strings.Contains(body, "{{") && strings.Contains(body, "}}")
// Read file
content, err := os.ReadFile(filePath)
if err != nil {
return result, errors.Wrapf(err, "failed to read file %s", filePath)
}
// Validate YAML syntax (but be lenient with templated files)
var parsed map[string]interface{}
if err := yaml.Unmarshal([]byte(body), &parsed); err != nil {
// If the content has templates, YAML parsing may fail - that's expected for v1beta3 only
if !hasTemplates {
res.Errors = append(res.Errors, LintError{
Line: extractLineFromError(err),
Message: fmt.Sprintf("YAML syntax error: %s", err.Error()),
})
return res
}
// Check if file contains template expressions
hasTemplates := strings.Contains(string(content), "{{") && strings.Contains(string(content), "}}")
// Attempt to detect apiVersion from raw content
detectedAPIVersion := detectAPIVersionFromContent(body)
if detectedAPIVersion == "" {
res.Errors = append(res.Errors, LintError{
Line: findLineNumber(body, "apiVersion"),
// Validate YAML syntax (but be lenient with templated files)
var parsed map[string]interface{}
yamlParseErr := yaml.Unmarshal(content, &parsed)
if yamlParseErr != nil {
// If the file has templates, YAML parsing may fail - that's expected
// We'll still try to validate what we can
if !hasTemplates {
result.Errors = append(result.Errors, LintError{
Line: extractLineFromError(yamlParseErr),
Message: fmt.Sprintf("YAML syntax error: %s", yamlParseErr.Error()),
})
// Don't return yet - we want to try to fix this error
// Continue to applyFixes at the end
// Try to surface apiVersion issues even if YAML failed to parse
// Detect via simple textual scan
avLine, avValue := findAPIVersionLineAndValue(string(content))
if avLine == 0 {
result.Errors = append(result.Errors, LintError{
Line: 0,
Field: "apiVersion",
Message: "Missing or unreadable 'apiVersion' field",
Message: "Missing or empty 'apiVersion' field",
})
return res
}
if detectedAPIVersion == constants.Troubleshootv1beta2Kind {
// v1beta2 does not support templating
addTemplatingErrorsForAllLines(&res, body)
return res
}
// For v1beta3 with templates, we can't parse YAML strictly, so just check template syntax
templateErrors, templateValueRefs := checkTemplateSyntax(body)
res.Errors = append(res.Errors, templateErrors...)
// Add warning about template values for v1beta3
if detectedAPIVersion == constants.Troubleshootv1beta3Kind && len(templateValueRefs) > 0 {
res.Warnings = append(res.Warnings, LintWarning{
Line: 1,
Field: "template-values",
Message: fmt.Sprintf("Template values that must be provided at runtime: %s", strings.Join(templateValueRefs, ", ")),
} else if avValue != constants.Troubleshootv1beta2Kind && avValue != constants.Troubleshootv1beta3Kind {
result.Errors = append(result.Errors, LintError{
Line: avLine,
Field: "apiVersion",
Message: fmt.Sprintf("Invalid 'apiVersion' value %q; expected %s or %s", avValue, constants.Troubleshootv1beta2Kind, constants.Troubleshootv1beta3Kind),
})
}
return res
if fix {
fixed, err := applyFixes(filePath, string(content), result)
if err != nil {
return result, err
}
if fixed {
// Re-lint to verify fixes
return lintFile(filePath, false)
}
}
return result, nil
}
// Determine apiVersion from parsed YAML
apiVersion := ""
if v, ok := parsed["apiVersion"].(string); ok {
apiVersion = v
}
if apiVersion == "" {
res.Errors = append(res.Errors, LintError{
Line: findLineNumber(body, "apiVersion"),
// For templated files, we can't parse YAML strictly, so just check template syntax
result.Errors = append(result.Errors, checkTemplateSyntax(string(content))...)
// Surface apiVersion issues via textual scan for templated files
avLine, avValue := findAPIVersionLineAndValue(string(content))
if avLine == 0 {
result.Errors = append(result.Errors, LintError{
Line: 0,
Field: "apiVersion",
Message: "Missing or empty 'apiVersion' field",
})
return res
} else if avValue != constants.Troubleshootv1beta2Kind && avValue != constants.Troubleshootv1beta3Kind {
result.Errors = append(result.Errors, LintError{
Line: avLine,
Field: "apiVersion",
Message: fmt.Sprintf("Invalid 'apiVersion' value %q; expected %s or %s", avValue, constants.Troubleshootv1beta2Kind, constants.Troubleshootv1beta3Kind),
})
}
// Templating policy: only v1beta3 supports templating
if apiVersion == constants.Troubleshootv1beta2Kind && hasTemplates {
addTemplatingErrorsForAllLines(&res, body)
}
// Check required fields
res.Errors = append(res.Errors, checkRequiredFields(parsed, body)...)
// Check template syntax and collect template value references
templateErrors, templateValueRefs := checkTemplateSyntax(body)
res.Errors = append(res.Errors, templateErrors...)
// Check for kind-specific requirements
if kind, ok := parsed["kind"].(string); ok {
switch kind {
case "Preflight":
res.Errors = append(res.Errors, checkPreflightSpec(parsed, body)...)
// Validate analyzer entries
res.Errors = append(res.Errors, validateAnalyzers(parsed, body)...)
case "SupportBundle":
res.Errors = append(res.Errors, checkSupportBundleSpec(parsed, body)...)
// Validate analyzers if present in SupportBundle specs as well
res.Errors = append(res.Errors, validateAnalyzers(parsed, body)...)
// Validate collector entries (collectors and hostCollectors)
res.Errors = append(res.Errors, validateCollectors(parsed, body, "collectors")...)
res.Errors = append(res.Errors, validateCollectors(parsed, body, "hostCollectors")...)
}
}
// Check for common issues
res.Warnings = append(res.Warnings, checkCommonIssues(parsed, body, apiVersion, templateValueRefs)...)
return res
}
// Initial lint
result := compute(content)
// Apply fixes if requested (multi-pass within a single invocation), in-memory
changed := false
if fix && (len(result.Errors) > 0 || len(result.Warnings) > 0) {
const maxFixPasses = 3
for pass := 0; pass < maxFixPasses; pass++ {
updatedContent, fixed, err := applyFixesInMemory(content, result)
// Continue to applyFixes for templates too
if fix {
fixed, err := applyFixes(filePath, string(content), result)
if err != nil {
return result, content, changed, err
return result, err
}
if !fixed {
break
if fixed {
// Re-lint to verify fixes
return lintFile(filePath, false)
}
changed = true
content = updatedContent
// Recompute without applying fixes in this cycle
result = compute(content)
if len(result.Errors) == 0 && len(result.Warnings) == 0 {
break
}
return result, nil
}
// Check required fields
result.Errors = append(result.Errors, checkRequiredFields(parsed, string(content))...)
// Check template syntax
result.Errors = append(result.Errors, checkTemplateSyntax(string(content))...)
// Check for kind-specific requirements
if kind, ok := parsed["kind"].(string); ok {
switch kind {
case "Preflight":
result.Errors = append(result.Errors, checkPreflightSpec(parsed, string(content))...)
case "SupportBundle":
result.Errors = append(result.Errors, checkSupportBundleSpec(parsed, string(content))...)
}
}
// Validate apiVersion value if present
if apiVersion, ok := parsed["apiVersion"].(string); ok && apiVersion != "" {
if apiVersion != constants.Troubleshootv1beta2Kind && apiVersion != constants.Troubleshootv1beta3Kind {
result.Errors = append(result.Errors, LintError{
Line: findLineNumber(string(content), "apiVersion"),
Field: "apiVersion",
Message: fmt.Sprintf("Invalid 'apiVersion' value %q; expected %s or %s", apiVersion, constants.Troubleshootv1beta2Kind, constants.Troubleshootv1beta3Kind),
})
}
}
// Check for common issues
result.Warnings = append(result.Warnings, checkCommonIssues(parsed, string(content))...)
// Apply fixes if requested
if fix && (len(result.Errors) > 0 || len(result.Warnings) > 0) {
fixed, err := applyFixes(filePath, string(content), result)
if err != nil {
return result, err
}
if fixed {
// Re-lint to verify fixes
return lintFile(filePath, false)
}
}
return result, nil
}
func checkRequiredFields(parsed map[string]interface{}, content string) []LintError {
errors := []LintError{}
// Check apiVersion
if apiVersion, ok := parsed["apiVersion"].(string); !ok || apiVersion == "" {
errors = append(errors, LintError{
Line: findLineNumber(content, "apiVersion"),
Field: "apiVersion",
Message: "Missing or empty 'apiVersion' field",
})
}
// Check kind
if kind, ok := parsed["kind"].(string); !ok || kind == "" {
errors = append(errors, LintError{
Line: findLineNumber(content, "kind"),
Field: "kind",
Message: "Missing or empty 'kind' field",
})
} else if kind != "Preflight" && kind != "SupportBundle" {
errors = append(errors, LintError{
Line: findLineNumber(content, "kind"),
Field: "kind",
Message: fmt.Sprintf("Invalid kind '%s'. Must be 'Preflight' or 'SupportBundle'", kind),
})
}
// Check metadata
if _, ok := parsed["metadata"]; !ok {
errors = append(errors, LintError{
Line: findLineNumber(content, "metadata"),
Field: "metadata",
Message: "Missing 'metadata' section",
})
} else if metadata, ok := parsed["metadata"].(map[string]interface{}); ok {
if name, ok := metadata["name"].(string); !ok || name == "" {
errors = append(errors, LintError{
Line: findLineNumber(content, "name"),
Field: "metadata.name",
Message: "Missing or empty 'metadata.name' field",
})
}
}
// Check spec
if _, ok := parsed["spec"]; !ok {
errors = append(errors, LintError{
Line: findLineNumber(content, "spec"),
Field: "spec",
Message: "Missing 'spec' section",
})
}
return errors
}
func checkTemplateSyntax(content string) []LintError {
errors := []LintError{}
lines := strings.Split(content, "\n")
// Check for unmatched braces
for i, line := range lines {
// Count opening and closing braces
opening := strings.Count(line, "{{")
closing := strings.Count(line, "}}")
if opening != closing {
errors = append(errors, LintError{
Line: i + 1,
Message: fmt.Sprintf("Unmatched template braces: %d opening, %d closing", opening, closing),
})
}
// Check for common template syntax issues
// Look for templates that might be missing the leading dot
if strings.Contains(line, "{{") && strings.Contains(line, "}}") {
// Extract template expressions
templateExpr := extractTemplateBetweenBraces(line)
for _, expr := range templateExpr {
trimmed := strings.TrimSpace(expr)
// Skip empty expressions
if trimmed == "" {
continue
}
// Skip control structures (if, else, end, range, with, etc.)
if isControlStructure(trimmed) {
continue
}
// Skip comments: {{/* ... */}}
if strings.HasPrefix(trimmed, "/*") || strings.HasPrefix(trimmed, "*/") {
continue
}
// Skip template variables (start with $)
if strings.HasPrefix(trimmed, "$") {
continue
}
// Skip expressions that start with a dot (valid references)
if strings.HasPrefix(trimmed, ".") {
continue
}
// Skip string literals
if strings.HasPrefix(trimmed, "\"") || strings.HasPrefix(trimmed, "'") {
continue
}
// Skip numeric literals
if regexp.MustCompile(`^[0-9]+$`).MatchString(trimmed) {
continue
}
// Skip function calls (contain parentheses or pipes)
if strings.Contains(trimmed, "(") || strings.Contains(trimmed, "|") {
continue
}
// Skip known Helm functions/keywords
helmFunctions := []string{"toYaml", "toJson", "include", "required", "default", "quote", "nindent", "indent", "upper", "lower", "trim"}
isFunction := false
for _, fn := range helmFunctions {
if strings.HasPrefix(trimmed, fn+" ") || trimmed == fn {
isFunction = true
break
}
}
if isFunction {
continue
}
// If we got here, it might be missing a leading dot
errors = append(errors, LintError{
Line: i + 1,
Message: fmt.Sprintf("Template expression may be missing leading dot: {{ %s }}", expr),
})
}
}
}
return result, content, changed, nil
return errors
}
func checkPreflightSpec(parsed map[string]interface{}, content string) []LintError {
errors := []LintError{}
spec, ok := parsed["spec"].(map[string]interface{})
if !ok {
return errors
}
// Check for analyzers
analyzers, hasAnalyzers := spec["analyzers"]
if !hasAnalyzers {
errors = append(errors, LintError{
Line: findLineNumber(content, "spec:"),
Field: "spec.analyzers",
Message: "Preflight spec must contain 'analyzers'",
})
} else if analyzersList, ok := analyzers.([]interface{}); ok {
if len(analyzersList) == 0 {
errors = append(errors, LintError{
Line: findLineNumber(content, "analyzers"),
Field: "spec.analyzers",
Message: "Preflight spec must have at least one analyzer",
})
}
}
return errors
}
func checkSupportBundleSpec(parsed map[string]interface{}, content string) []LintError {
errors := []LintError{}
spec, ok := parsed["spec"].(map[string]interface{})
if !ok {
return errors
}
// Check for collectors
collectors, hasCollectors := spec["collectors"]
_, hasHostCollectors := spec["hostCollectors"]
if !hasCollectors && !hasHostCollectors {
errors = append(errors, LintError{
Line: findLineNumber(content, "spec:"),
Field: "spec.collectors",
Message: "SupportBundle spec must contain 'collectors' or 'hostCollectors'",
})
} else {
// Check if collectors list is empty
if hasCollectors {
if collectorsList, ok := collectors.([]interface{}); ok && len(collectorsList) == 0 {
errors = append(errors, LintError{
Line: findLineNumber(content, "collectors"),
Field: "spec.collectors",
Message: "Collectors list is empty",
})
}
}
}
return errors
}
func checkCommonIssues(parsed map[string]interface{}, content string) []LintWarning {
warnings := []LintWarning{}
// Check for missing docStrings in analyzers
spec, ok := parsed["spec"].(map[string]interface{})
if !ok {
return warnings
}
if analyzers, ok := spec["analyzers"].([]interface{}); ok {
for i, analyzer := range analyzers {
if analyzerMap, ok := analyzer.(map[string]interface{}); ok {
if _, hasDocString := analyzerMap["docString"]; !hasDocString {
warnings = append(warnings, LintWarning{
Line: findAnalyzerLine(content, i),
Field: fmt.Sprintf("spec.analyzers[%d].docString", i),
Message: "Analyzer missing docString (recommended for v1beta3)",
})
}
}
}
}
return warnings
}
func applyFixes(filePath, content string, result LintResult) (bool, error) {
fixed := false
newContent := content
lines := strings.Split(newContent, "\n")
// Determine desired apiVersion for fixes
hasTemplates := strings.Contains(content, "{{") && strings.Contains(content, "}}")
hasDocStrings := strings.Contains(content, "docString:")
desiredAPIVersion := constants.Troubleshootv1beta2Kind
if hasTemplates || hasDocStrings {
desiredAPIVersion = constants.Troubleshootv1beta3Kind
}
// Sort errors by line number (descending) to avoid line number shifts when editing
errorsByLine := make(map[int][]LintError)
for _, err := range result.Errors {
if err.Line > 0 {
errorsByLine[err.Line] = append(errorsByLine[err.Line], err)
}
}
// Process errors line by line
for lineNum, errs := range errorsByLine {
if lineNum > len(lines) {
continue
}
line := lines[lineNum-1]
originalLine := line
for _, err := range errs {
// Fix 1: Add missing colon
// YAML parsers often report the error on the line AFTER the actual problem
if strings.Contains(err.Message, "could not find expected ':'") {
// Check current line first
if !strings.Contains(line, ":") {
trimmed := strings.TrimSpace(line)
indent := line[:len(line)-len(strings.TrimLeft(line, " \t"))]
line = indent + trimmed + ":"
fixed = true
} else if lineNum > 1 {
// Check previous line (where the colon is likely missing)
prevLine := lines[lineNum-2]
if !strings.Contains(prevLine, ":") && strings.TrimSpace(prevLine) != "" {
trimmed := strings.TrimSpace(prevLine)
indent := prevLine[:len(prevLine)-len(strings.TrimLeft(prevLine, " \t"))]
lines[lineNum-2] = indent + trimmed + ":"
fixed = true
}
}
}
// Fix 2: Add missing leading dot in template expressions
if strings.Contains(err.Message, "Template expression may be missing leading dot:") {
// Extract the expression from the error message
re := regexp.MustCompile(`Template expression may be missing leading dot: \{\{ (.+?) \}\}`)
matches := re.FindStringSubmatch(err.Message)
if len(matches) > 1 {
badExpr := matches[1]
// Add the leading dot
fixedExpr := "." + badExpr
// Replace in the line
line = strings.Replace(line, "{{ "+badExpr+" }}", "{{ "+fixedExpr+" }}", 1)
line = strings.Replace(line, "{{"+badExpr+"}}", "{{"+fixedExpr+"}}", 1)
line = strings.Replace(line, "{{- "+badExpr+" }}", "{{- "+fixedExpr+" }}", 1)
line = strings.Replace(line, "{{- "+badExpr+" -}}", "{{- "+fixedExpr+" -}}", 1)
fixed = true
}
}
// Fix 3: Replace invalid apiVersion value with desiredAPIVersion
if strings.Contains(err.Message, "Invalid 'apiVersion' value") && err.Field == "apiVersion" {
if strings.Contains(line, "apiVersion:") {
indent := line[:len(line)-len(strings.TrimLeft(line, " \t"))]
line = indent + "apiVersion: " + desiredAPIVersion
fixed = true
}
}
}
// Update the line if it changed
if line != originalLine {
lines[lineNum-1] = line
}
}
// Fix 4: Add missing required top-level fields
for _, err := range result.Errors {
if err.Field == "apiVersion" && strings.Contains(err.Message, "Missing or empty 'apiVersion'") {
// Replace existing empty apiVersion line if present; otherwise prepend
if avLine, avVal := findAPIVersionLineAndValue(newContent); avLine > 0 && strings.TrimSpace(avVal) == "" {
line := lines[avLine-1]
indent := line[:len(line)-len(strings.TrimLeft(line, " \t"))]
lines[avLine-1] = indent + "apiVersion: " + desiredAPIVersion
} else {
lines = append([]string{"apiVersion: " + desiredAPIVersion}, lines...)
}
fixed = true
} else if err.Field == "kind" && strings.Contains(err.Message, "Missing or empty 'kind'") {
// Try to determine if it should be Preflight or SupportBundle based on filename
kind := "Preflight"
if strings.Contains(strings.ToLower(filePath), "bundle") {
kind = "SupportBundle"
}
// Add kind after apiVersion
insertIndex := 0
for i, line := range lines {
if strings.Contains(line, "apiVersion:") {
insertIndex = i + 1
break
}
}
newLines := make([]string, 0, len(lines)+1)
newLines = append(newLines, lines[:insertIndex]...)
newLines = append(newLines, "kind: "+kind)
newLines = append(newLines, lines[insertIndex:]...)
lines = newLines
fixed = true
} else if err.Field == "metadata" && strings.Contains(err.Message, "Missing 'metadata'") {
// Add metadata section after kind
insertIndex := 0
for i, line := range lines {
if strings.Contains(line, "kind:") {
insertIndex = i + 1
break
}
}
newLines := make([]string, 0, len(lines)+2)
newLines = append(newLines, lines[:insertIndex]...)
newLines = append(newLines, "metadata:")
newLines = append(newLines, " name: my-spec")
newLines = append(newLines, lines[insertIndex:]...)
lines = newLines
fixed = true
}
}
// Write fixed content back to file if changes were made
if fixed {
newContent = strings.Join(lines, "\n")
if err := os.WriteFile(filePath, []byte(newContent), 0644); err != nil {
return false, errors.Wrapf(err, "failed to write fixed content to %s", filePath)
}
}
return fixed, nil
}
func findLineNumber(content, search string) int {
lines := strings.Split(content, "\n")
for i, line := range lines {
if strings.Contains(line, search) {
return i + 1
}
}
return 0
}
// findAPIVersionLineAndValue locates the first line that declares apiVersion and returns its
// 1-based line number and the trimmed value to the right of the colon. Returns (0, "") if not found.
func findAPIVersionLineAndValue(content string) (int, string) {
lines := strings.Split(content, "\n")
for i, line := range lines {
trimmed := strings.TrimSpace(line)
if strings.HasPrefix(trimmed, "apiVersion:") {
// extract value after the first colon
parts := strings.SplitN(trimmed, ":", 2)
if len(parts) == 2 {
value := strings.TrimSpace(parts[1])
return i + 1, value
}
return i + 1, ""
}
}
return 0, ""
}
func findAnalyzerLine(content string, index int) int {
lines := strings.Split(content, "\n")
analyzerCount := 0
inAnalyzers := false
for i, line := range lines {
if strings.Contains(line, "analyzers:") {
inAnalyzers = true
continue
}
if inAnalyzers && strings.HasPrefix(strings.TrimSpace(line), "- ") {
if analyzerCount == index {
return i + 1
}
analyzerCount++
}
}
return 0
}
func extractLineFromError(err error) int {
// Try to extract line number from YAML error message
re := regexp.MustCompile(`line (\d+)`)
matches := re.FindStringSubmatch(err.Error())
if len(matches) > 1 {
var line int
fmt.Sscanf(matches[1], "%d", &line)
return line
}
return 0
}
// extractTemplateBetweenBraces extracts template expressions from a line
func extractTemplateBetweenBraces(line string) []string {
var expressions []string
// Match {{ ... }} with optional whitespace trimming (-), including comments {{/* */}}
re := regexp.MustCompile(`\{\{-?\s*(.+?)\s*-?\}\}`)
matches := re.FindAllStringSubmatch(line, -1)
for _, match := range matches {
if len(match) > 1 {
// Clean up the expression
expr := match[1]
// Remove */ at the end if it's part of a comment
expr = strings.TrimSuffix(strings.TrimSpace(expr), "*/")
expressions = append(expressions, expr)
}
}
return expressions
}
// isControlStructure checks if a template expression is a control structure
func isControlStructure(expr string) bool {
trimmed := strings.TrimSpace(expr)
controlKeywords := []string{"if", "else", "end", "range", "with", "define", "template", "block", "include"}
for _, keyword := range controlKeywords {
if strings.HasPrefix(trimmed, keyword+" ") || trimmed == keyword {
return true
}
}
return false
}
// FormatResults formats lint results for output
func FormatResults(results []LintResult, format string) string {
if format == "json" {
return formatJSON(results)
}
return formatText(results)
}
func formatText(results []LintResult) string {
var output strings.Builder
totalErrors := 0
totalWarnings := 0
for _, result := range results {
if len(result.Errors) == 0 && len(result.Warnings) == 0 {
output.WriteString(fmt.Sprintf("✓ %s: No issues found\n", result.FilePath))
continue
}
output.WriteString(fmt.Sprintf("\n%s:\n", result.FilePath))
for _, err := range result.Errors {
output.WriteString(fmt.Sprintf(" ✗ Error (line %d): %s\n", err.Line, err.Message))
if err.Field != "" {
output.WriteString(fmt.Sprintf(" Field: %s\n", err.Field))
}
totalErrors++
}
for _, warn := range result.Warnings {
output.WriteString(fmt.Sprintf(" ⚠ Warning (line %d): %s\n", warn.Line, warn.Message))
if warn.Field != "" {
output.WriteString(fmt.Sprintf(" Field: %s\n", warn.Field))
}
totalWarnings++
}
}
output.WriteString(fmt.Sprintf("\nSummary: %d error(s), %d warning(s) across %d file(s)\n", totalErrors, totalWarnings, len(results)))
return output.String()
}
func formatJSON(results []LintResult) string {
// Simple JSON formatting without importing encoding/json
var output strings.Builder
output.WriteString("{\n")
output.WriteString(" \"results\": [\n")
for i, result := range results {
output.WriteString(" {\n")
output.WriteString(fmt.Sprintf(" \"filePath\": %q,\n", result.FilePath))
output.WriteString(" \"errors\": [\n")
for j, err := range result.Errors {
output.WriteString(" {\n")
output.WriteString(fmt.Sprintf(" \"line\": %d,\n", err.Line))
output.WriteString(fmt.Sprintf(" \"column\": %d,\n", err.Column))
output.WriteString(fmt.Sprintf(" \"message\": %q,\n", err.Message))
output.WriteString(fmt.Sprintf(" \"field\": %q\n", err.Field))
output.WriteString(" }")
if j < len(result.Errors)-1 {
output.WriteString(",")
}
output.WriteString("\n")
}
output.WriteString(" ],\n")
output.WriteString(" \"warnings\": [\n")
for j, warn := range result.Warnings {
output.WriteString(" {\n")
output.WriteString(fmt.Sprintf(" \"line\": %d,\n", warn.Line))
output.WriteString(fmt.Sprintf(" \"column\": %d,\n", warn.Column))
output.WriteString(fmt.Sprintf(" \"message\": %q,\n", warn.Message))
output.WriteString(fmt.Sprintf(" \"field\": %q\n", warn.Field))
output.WriteString(" }")
if j < len(result.Warnings)-1 {
output.WriteString(",")
}
output.WriteString("\n")
}
output.WriteString(" ]\n")
output.WriteString(" }")
if i < len(results)-1 {
output.WriteString(",")
}
output.WriteString("\n")
}
output.WriteString(" ]\n")
output.WriteString("}\n")
return output.String()
}
// HasErrors returns true if any of the results contain errors
func HasErrors(results []LintResult) bool {
for _, result := range results {
if len(result.Errors) > 0 {
return true
}
}
return false
}
-425
View File
@@ -1,425 +0,0 @@
package lint
import (
"os"
"path/filepath"
"strings"
"testing"
)
func TestLintMultipleFiles(t *testing.T) {
// Get the project root by going up from pkg/lint
projectRoot := filepath.Join("..", "..")
testDir := filepath.Join(projectRoot, "examples", "test-error-messages")
tests := []struct {
name string
files []string
expectErrors map[string][]string // filename -> expected error substrings
expectWarnings map[string][]string // filename -> expected warning substrings
expectPass map[string]bool // filename -> should pass without errors
}{
{
name: "valid v1beta3 with templates",
files: []string{
"helm-builtins-v1beta3.yaml",
},
expectErrors: map[string][]string{},
expectWarnings: map[string][]string{
"helm-builtins-v1beta3.yaml": {
"Template values that must be provided at runtime: minVersion",
},
},
expectPass: map[string]bool{
"helm-builtins-v1beta3.yaml": false, // has warnings
},
},
{
name: "invalid collectors and analyzers",
files: []string{
"invalid-collectors-analyzers.yaml",
},
expectErrors: map[string][]string{
"invalid-collectors-analyzers.yaml": {
// The linter may stop early due to structural issues
// At minimum, it should catch the hostCollectors type error
"Expected 'hostCollectors' to be a list",
},
},
expectPass: map[string]bool{
"invalid-collectors-analyzers.yaml": false,
},
},
{
name: "missing required fields",
files: []string{
"missing-apiversion-v1beta3.yaml",
"missing-metadata-v1beta3.yaml",
"no-analyzers-v1beta3.yaml",
},
expectErrors: map[string][]string{
"missing-apiversion-v1beta3.yaml": {
"Missing or empty 'apiVersion' field",
},
"missing-metadata-v1beta3.yaml": {
"Missing 'metadata' section",
},
"no-analyzers-v1beta3.yaml": {
"Preflight spec must contain 'analyzers'",
},
},
expectPass: map[string]bool{
"missing-apiversion-v1beta3.yaml": false,
"missing-metadata-v1beta3.yaml": false,
"no-analyzers-v1beta3.yaml": false,
},
},
{
name: "v1beta2 file (valid but with docString warning)",
files: []string{
"wrong-apiversion-v1beta3.yaml", // Actually has v1beta2 which is valid
},
expectErrors: map[string][]string{},
expectWarnings: map[string][]string{
"wrong-apiversion-v1beta3.yaml": {
"Some analyzers are missing docString",
},
},
expectPass: map[string]bool{
"wrong-apiversion-v1beta3.yaml": true, // No errors, just warnings
},
},
{
name: "support bundle specs",
files: []string{
"support-bundle-no-collectors-v1beta3.yaml",
"support-bundle-valid-v1beta3.yaml",
},
expectErrors: map[string][]string{
"support-bundle-no-collectors-v1beta3.yaml": {
"SupportBundle spec must contain 'collectors' or 'hostCollectors'",
},
},
expectPass: map[string]bool{
"support-bundle-no-collectors-v1beta3.yaml": false,
"support-bundle-valid-v1beta3.yaml": true,
},
},
{
name: "multiple files with mixed validity",
files: []string{
"support-bundle-valid-v1beta3.yaml",
"missing-metadata-v1beta3.yaml",
"wrong-apiversion-v1beta3.yaml",
},
expectErrors: map[string][]string{
"missing-metadata-v1beta3.yaml": {
"Missing 'metadata' section",
},
},
expectWarnings: map[string][]string{
"wrong-apiversion-v1beta3.yaml": {
"Some analyzers are missing docString",
},
},
expectPass: map[string]bool{
"support-bundle-valid-v1beta3.yaml": true,
"missing-metadata-v1beta3.yaml": false,
"wrong-apiversion-v1beta3.yaml": true, // No errors, just warnings
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Build full file paths
filePaths := make([]string, len(tt.files))
for i, f := range tt.files {
filePaths[i] = filepath.Join(testDir, f)
// Check file exists
if _, err := os.Stat(filePaths[i]); os.IsNotExist(err) {
t.Skipf("Test file %s does not exist, skipping", filePaths[i])
}
}
// Run linter
opts := LintOptions{
FilePaths: filePaths,
Fix: false,
Format: "text",
}
results, err := LintFiles(opts)
if err != nil {
t.Fatalf("LintFiles failed: %v", err)
}
// Verify we got results for all files
if len(results) != len(filePaths) {
t.Errorf("Expected %d results, got %d", len(filePaths), len(results))
}
// Check each result
for _, result := range results {
filename := filepath.Base(result.FilePath)
// Check expected errors
if expectedErrors, ok := tt.expectErrors[filename]; ok {
if len(expectedErrors) > 0 && len(result.Errors) == 0 {
t.Errorf("File %s: expected errors but got none", filename)
}
for _, expectedErr := range expectedErrors {
found := false
for _, err := range result.Errors {
if strings.Contains(err.Message, expectedErr) {
found = true
break
}
}
if !found {
t.Errorf("File %s: expected error containing '%s' but not found in errors: %v",
filename, expectedErr, getErrorMessages(result.Errors))
}
}
}
// Check expected warnings
if expectedWarnings, ok := tt.expectWarnings[filename]; ok {
for _, expectedWarn := range expectedWarnings {
found := false
for _, warn := range result.Warnings {
if strings.Contains(warn.Message, expectedWarn) {
found = true
break
}
}
if !found {
t.Errorf("File %s: expected warning containing '%s' but not found in warnings: %v",
filename, expectedWarn, getWarningMessages(result.Warnings))
}
}
}
// Check if should pass
if shouldPass, ok := tt.expectPass[filename]; ok {
hasNoErrors := len(result.Errors) == 0
if shouldPass && !hasNoErrors {
t.Errorf("File %s: expected to pass but has errors: %v",
filename, getErrorMessages(result.Errors))
} else if !shouldPass && hasNoErrors && len(tt.expectErrors[filename]) > 0 {
t.Errorf("File %s: expected to fail but passed", filename)
}
}
}
})
}
}
func TestLintWithFix(t *testing.T) {
// Create a temporary directory for test files
tmpDir, err := os.MkdirTemp("", "lint-test-*")
if err != nil {
t.Fatalf("Failed to create temp dir: %v", err)
}
defer os.RemoveAll(tmpDir)
tests := []struct {
name string
content string
expectFix bool
fixedContent string // substring that should appear after fix
}{
{
name: "fix v1beta2 with templates to v1beta3",
content: `apiVersion: troubleshoot.sh/v1beta2
kind: Preflight
metadata:
name: test-{{ .Values.name }}
spec:
analyzers:
- clusterVersion:
outcomes:
- pass:
when: '>= 1.19.0'
message: OK`,
expectFix: true,
fixedContent: "apiVersion: troubleshoot.sh/v1beta3",
},
{
name: "fix missing leading dot in template",
content: `apiVersion: troubleshoot.sh/v1beta3
kind: Preflight
metadata:
name: test-{{ Values.name }}
spec:
analyzers:
- clusterVersion:
outcomes:
- pass:
when: '>= 1.19.0'
message: OK`,
expectFix: true,
fixedContent: "{{ .Values.name }}",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Write test content to temp file
testFile := filepath.Join(tmpDir, tt.name+".yaml")
if err := os.WriteFile(testFile, []byte(tt.content), 0644); err != nil {
t.Fatalf("Failed to write test file: %v", err)
}
// Run linter with fix enabled
opts := LintOptions{
FilePaths: []string{testFile},
Fix: true,
Format: "text",
}
results, err := LintFiles(opts)
if err != nil {
t.Fatalf("LintFiles failed: %v", err)
}
if len(results) != 1 {
t.Fatalf("Expected 1 result, got %d", len(results))
}
// Read the potentially fixed content
fixedBytes, err := os.ReadFile(testFile)
if err != nil {
t.Fatalf("Failed to read fixed file: %v", err)
}
fixedContent := string(fixedBytes)
// Check if fix was applied
if tt.expectFix {
if !strings.Contains(fixedContent, tt.fixedContent) {
t.Errorf("Expected fixed content to contain '%s', but got:\n%s",
tt.fixedContent, fixedContent)
}
}
})
}
}
func TestHasErrors(t *testing.T) {
tests := []struct {
name string
results []LintResult
expected bool
}{
{
name: "no errors",
results: []LintResult{
{
FilePath: "test1.yaml",
Errors: []LintError{},
Warnings: []LintWarning{{Message: "warning"}},
},
{
FilePath: "test2.yaml",
Errors: []LintError{},
},
},
expected: false,
},
{
name: "has errors",
results: []LintResult{
{
FilePath: "test1.yaml",
Errors: []LintError{{Message: "error"}},
},
{
FilePath: "test2.yaml",
Errors: []LintError{},
},
},
expected: true,
},
{
name: "empty results",
results: []LintResult{},
expected: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result := HasErrors(tt.results)
if result != tt.expected {
t.Errorf("HasErrors() = %v, want %v", result, tt.expected)
}
})
}
}
func TestFormatResults(t *testing.T) {
results := []LintResult{
{
FilePath: "test.yaml",
Errors: []LintError{
{Line: 5, Message: "Missing field", Field: "spec.analyzers"},
},
Warnings: []LintWarning{
{Line: 10, Message: "Consider adding docString", Field: "spec.analyzers[0]"},
},
},
}
t.Run("text format", func(t *testing.T) {
output := FormatResults(results, "text")
// Check for key components in text output
if !strings.Contains(output, "test.yaml") {
t.Error("Text output missing file path")
}
if !strings.Contains(output, "Error (line 5)") {
t.Error("Text output missing error with line number")
}
if !strings.Contains(output, "Warning (line 10)") {
t.Error("Text output missing warning with line number")
}
if !strings.Contains(output, "Summary:") {
t.Error("Text output missing summary")
}
})
t.Run("json format", func(t *testing.T) {
output := FormatResults(results, "json")
// Check for key JSON components
if !strings.Contains(output, `"filePath"`) {
t.Error("JSON output missing filePath field")
}
if !strings.Contains(output, `"errors"`) {
t.Error("JSON output missing errors field")
}
if !strings.Contains(output, `"warnings"`) {
t.Error("JSON output missing warnings field")
}
if !strings.Contains(output, `"line": 5`) {
t.Error("JSON output missing line number")
}
})
}
// Helper functions
func getErrorMessages(errors []LintError) []string {
messages := make([]string, len(errors))
for i, err := range errors {
messages[i] = err.Message
}
return messages
}
func getWarningMessages(warnings []LintWarning) []string {
messages := make([]string, len(warnings))
for i, warn := range warnings {
messages[i] = warn.Message
}
return messages
}
-181
View File
@@ -1,181 +0,0 @@
package lint
import (
"fmt"
"regexp"
"sort"
"strings"
)
// detectAPIVersionFromContent tries to extract apiVersion from raw YAML text
func detectAPIVersionFromContent(content string) string {
lines := strings.Split(content, "\n")
for _, line := range lines {
trimmed := strings.TrimSpace(line)
if strings.HasPrefix(trimmed, "apiVersion:") {
parts := strings.SplitN(trimmed, ":", 2)
if len(parts) == 2 {
val := strings.TrimSpace(parts[1])
// strip quotes if present
val = strings.Trim(val, "'\"")
return val
}
}
}
return ""
}
// addTemplatingErrorsForAllLines records an error for each line containing template braces in versions that do not support templating
func addTemplatingErrorsForAllLines(result *LintResult, content string) {
lines := strings.Split(content, "\n")
for i, line := range lines {
if strings.Contains(line, "{{") && strings.Contains(line, "}}") {
result.Errors = append(result.Errors, LintError{
Line: i + 1,
Message: "Templating is not supported in v1beta2 specs",
Field: "template",
})
}
}
}
func checkTemplateSyntax(content string) ([]LintError, []string) {
errors := []LintError{}
lines := strings.Split(content, "\n")
templateValueRefs := map[string]bool{}
// Check for unmatched braces
for i, line := range lines {
// Count opening and closing braces
opening := strings.Count(line, "{{")
closing := strings.Count(line, "}}")
if opening != closing {
errors = append(errors, LintError{
Line: i + 1,
Message: fmt.Sprintf("Unmatched template braces: %d opening, %d closing", opening, closing),
})
}
// Check for common template syntax issues
// Look for templates that might be missing the leading dot
if strings.Contains(line, "{{") && strings.Contains(line, "}}") {
// Extract template expressions
templateExpr := extractTemplateBetweenBraces(line)
for _, expr := range templateExpr {
trimmed := strings.TrimSpace(expr)
// Skip empty expressions
if trimmed == "" {
continue
}
// Skip comments: {{/* ... */}}
if strings.HasPrefix(trimmed, "/*") || strings.HasPrefix(trimmed, "*/") {
continue
}
// Track template value references for warning (check this before skipping control structures)
if strings.Contains(trimmed, ".Values.") {
// Extract the value path
valuePattern := regexp.MustCompile(`\.Values\.(\w+(?:\.\w+)*)`)
matches := valuePattern.FindAllStringSubmatch(trimmed, -1)
for _, match := range matches {
if len(match) > 1 {
templateValueRefs[match[1]] = true
}
}
}
// Skip control structures (if, else, end, range, with, etc.)
if isControlStructure(trimmed) {
continue
}
// Skip template variables (start with $)
if strings.HasPrefix(trimmed, "$") {
continue
}
// Skip expressions that start with a dot (valid references)
if strings.HasPrefix(trimmed, ".") {
continue
}
// Skip string literals
if strings.HasPrefix(trimmed, "\"") || strings.HasPrefix(trimmed, "'") {
continue
}
// Skip numeric literals
if regexp.MustCompile(`^[0-9]+$`).MatchString(trimmed) {
continue
}
// Skip function calls (contain parentheses or pipes)
if strings.Contains(trimmed, "(") || strings.Contains(trimmed, "|") {
continue
}
// Skip known Helm functions/keywords
helmFunctions := []string{"toYaml", "toJson", "include", "required", "default", "quote", "nindent", "indent", "upper", "lower", "trim"}
isFunction := false
for _, fn := range helmFunctions {
if strings.HasPrefix(trimmed, fn+" ") || trimmed == fn {
isFunction = true
break
}
}
if isFunction {
continue
}
// If we got here, it might be missing a leading dot
errors = append(errors, LintError{
Line: i + 1,
Message: fmt.Sprintf("Template expression may be missing leading dot: {{ %s }}", expr),
})
}
}
}
// Collect template values that need to be provided at runtime
var valueList []string
for val := range templateValueRefs {
valueList = append(valueList, val)
}
// Sort for consistent output
sort.Strings(valueList)
return errors, valueList
}
// extractTemplateBetweenBraces extracts template expressions from a line
func extractTemplateBetweenBraces(line string) []string {
var expressions []string
// Match {{ ... }} with optional whitespace trimming (-), including comments {{/* */}}
re := regexp.MustCompile(`\{\{-?\s*(.+?)\s*-?\}\}`)
matches := re.FindAllStringSubmatch(line, -1)
for _, match := range matches {
if len(match) > 1 {
// Clean up the expression
expr := match[1]
// Remove */ at the end if it's part of a comment
expr = strings.TrimSuffix(strings.TrimSpace(expr), "*/")
expressions = append(expressions, expr)
}
}
return expressions
}
// isControlStructure checks if a template expression is a control structure
func isControlStructure(expr string) bool {
trimmed := strings.TrimSpace(expr)
controlKeywords := []string{"if", "else", "end", "range", "with", "define", "template", "block", "include"}
for _, keyword := range controlKeywords {
if strings.HasPrefix(trimmed, keyword+" ") || trimmed == keyword {
return true
}
}
return false
}
-39
View File
@@ -1,39 +0,0 @@
package lint
// Core types used by the lint package
type LintResult struct {
FilePath string
Errors []LintError
Warnings []LintWarning
}
type LintError struct {
Line int
Column int
Message string
Field string
}
type LintWarning struct {
Line int
Column int
Message string
Field string
}
type LintOptions struct {
FilePaths []string
Fix bool
Format string // "text" or "json"
}
// HasErrors returns true if any of the results contain errors
func HasErrors(results []LintResult) bool {
for _, result := range results {
if len(result.Errors) > 0 {
return true
}
}
return false
}
-452
View File
@@ -1,452 +0,0 @@
package lint
import (
"fmt"
"os"
"path/filepath"
"strings"
"encoding/json"
)
func checkRequiredFields(parsed map[string]interface{}, content string) []LintError {
errors := []LintError{}
// Check apiVersion
if apiVersion, ok := parsed["apiVersion"].(string); !ok || apiVersion == "" {
errors = append(errors, LintError{
Line: findLineNumber(content, "apiVersion"),
Field: "apiVersion",
Message: "Missing 'apiVersion'",
})
}
// Check kind
if kind, ok := parsed["kind"].(string); !ok || kind == "" {
errors = append(errors, LintError{
Line: findLineNumber(content, "kind"),
Field: "kind",
Message: "Missing or empty 'kind' field",
})
} else if kind != "Preflight" && kind != "SupportBundle" {
errors = append(errors, LintError{
Line: findLineNumber(content, "kind"),
Field: "kind",
Message: fmt.Sprintf("Expected kind 'Preflight' or 'SupportBundle' (found '%s')", kind),
})
}
// Check metadata
if _, ok := parsed["metadata"]; !ok {
errors = append(errors, LintError{
Line: findLineNumber(content, "metadata"),
Field: "metadata",
Message: "Missing 'metadata' section",
})
} else if metadata, ok := parsed["metadata"].(map[string]interface{}); ok {
if name, ok := metadata["name"].(string); !ok || name == "" {
errors = append(errors, LintError{
Line: findLineNumber(content, "name"),
Field: "metadata.name",
Message: "Missing 'metadata.name'",
})
}
}
// Check spec
if _, ok := parsed["spec"]; !ok {
errors = append(errors, LintError{
Line: findLineNumber(content, "spec"),
Field: "spec",
Message: "Missing 'spec' section",
})
}
return errors
}
func checkPreflightSpec(parsed map[string]interface{}, content string) []LintError {
errors := []LintError{}
spec, ok := parsed["spec"].(map[string]interface{})
if !ok {
return errors
}
// Check for analyzers
analyzers, hasAnalyzers := spec["analyzers"]
if !hasAnalyzers {
errors = append(errors, LintError{
Line: findLineNumber(content, "spec:"),
Field: "spec.analyzers",
Message: "Preflight spec must contain 'analyzers'",
})
} else if analyzersList, ok := analyzers.([]interface{}); ok {
if len(analyzersList) == 0 {
errors = append(errors, LintError{
Line: findLineNumber(content, "analyzers"),
Field: "spec.analyzers",
Message: "Preflight spec must have at least one analyzer",
})
}
}
return errors
}
func checkSupportBundleSpec(parsed map[string]interface{}, content string) []LintError {
errors := []LintError{}
spec, ok := parsed["spec"].(map[string]interface{})
if !ok {
return errors
}
// Check for collectors
collectors, hasCollectors := spec["collectors"]
_, hasHostCollectors := spec["hostCollectors"]
if !hasCollectors && !hasHostCollectors {
errors = append(errors, LintError{
Line: findLineNumber(content, "spec:"),
Field: "spec.collectors",
Message: "SupportBundle spec must contain 'collectors' or 'hostCollectors'",
})
} else {
// Check if collectors list is empty
if hasCollectors {
if collectorsList, ok := collectors.([]interface{}); ok && len(collectorsList) == 0 {
errors = append(errors, LintError{
Line: findLineNumber(content, "collectors"),
Field: "spec.collectors",
Message: "Collectors list is empty",
})
}
}
}
return errors
}
// checkCommonIssues aggregates advisory warnings based on best practices
func checkCommonIssues(parsed map[string]interface{}, content string, apiVersion string, templateValueRefs []string) []LintWarning {
warnings := []LintWarning{}
// Check for missing docStrings in analyzers and collectors
spec, ok := parsed["spec"].(map[string]interface{})
if !ok {
return warnings
}
// Check if any analyzers are missing docString
analyzersMissingDocString := false
if analyzers, ok := spec["analyzers"].([]interface{}); ok {
for _, analyzer := range analyzers {
if analyzerMap, ok := analyzer.(map[string]interface{}); ok {
// Check if docString exists at the analyzer level (v1beta3)
if _, hasDocString := analyzerMap["docString"]; !hasDocString {
analyzersMissingDocString = true
break
}
}
}
}
// Check if any collectors are missing docString
collectorsMissingDocString := false
if collectors, ok := spec["collectors"].([]interface{}); ok {
for _, collector := range collectors {
if collectorMap, ok := collector.(map[string]interface{}); ok {
// Get the actual collector type (first key-value pair)
for _, collectorSpec := range collectorMap {
if specMap, ok := collectorSpec.(map[string]interface{}); ok {
if _, hasDocString := specMap["docString"]; !hasDocString {
collectorsMissingDocString = true
break
}
}
// Only check the first key since collectors have single type
break
}
}
if collectorsMissingDocString {
break
}
}
}
// Add consolidated warnings if any items are missing docString
if analyzersMissingDocString && collectorsMissingDocString {
warnings = append(warnings, LintWarning{
Line: findLineNumber(content, "spec:"),
Field: "spec",
Message: "Some analyzers and collectors are missing docString (recommended for v1beta3)",
})
} else if analyzersMissingDocString {
warnings = append(warnings, LintWarning{
Line: findLineNumber(content, "analyzers:"),
Field: "spec.analyzers",
Message: "Some analyzers are missing docString (recommended for v1beta3)",
})
} else if collectorsMissingDocString {
warnings = append(warnings, LintWarning{
Line: findLineNumber(content, "collectors:"),
Field: "spec.collectors",
Message: "Some collectors are missing docString (recommended for v1beta3)",
})
}
// Add warning about template values that need to be provided at runtime (v1beta3 only)
if apiVersion == "troubleshoot.sh/v1beta3" && len(templateValueRefs) > 0 {
warnings = append(warnings, LintWarning{
Line: 1,
Field: "template-values",
Message: fmt.Sprintf("Template values that must be provided at runtime: %s", strings.Join(templateValueRefs, ", ")),
})
}
return warnings
}
// --- Schema-backed quick validation (best-effort) ---
type schemaTypeInfo struct {
required map[string]struct{}
properties map[string]struct{}
}
var (
knownAnalyzerTypes map[string]struct{}
knownCollectorTypes map[string]struct{}
analyzerTypeInfo map[string]schemaTypeInfo
collectorTypeInfo map[string]schemaTypeInfo
knownTypesLoaded bool
)
func ensureKnownTypesLoaded() {
if knownTypesLoaded {
return
}
knownAnalyzerTypes = map[string]struct{}{}
knownCollectorTypes = map[string]struct{}{}
analyzerTypeInfo = map[string]schemaTypeInfo{}
collectorTypeInfo = map[string]schemaTypeInfo{}
// Analyzer schema (v1beta2)
loadKeysFromSchema(
filepath.Join("schemas", "analyzer-troubleshoot-v1beta2.json"),
[]string{"properties", "spec", "properties", "analyzers", "items", "properties"},
knownAnalyzerTypes,
)
loadTypeInfoFromSchema(
filepath.Join("schemas", "analyzer-troubleshoot-v1beta2.json"),
[]string{"properties", "spec", "properties", "analyzers", "items", "properties"},
analyzerTypeInfo,
)
// Collector schema (v1beta2)
loadKeysFromSchema(
filepath.Join("schemas", "collector-troubleshoot-v1beta2.json"),
[]string{"properties", "spec", "properties", "collectors", "items", "properties"},
knownCollectorTypes,
)
loadTypeInfoFromSchema(
filepath.Join("schemas", "collector-troubleshoot-v1beta2.json"),
[]string{"properties", "spec", "properties", "collectors", "items", "properties"},
collectorTypeInfo,
)
knownTypesLoaded = true
}
// loadKeysFromSchema walks a JSON object by keysPath and adds map keys at that node into dest
func loadKeysFromSchema(schemaPath string, keysPath []string, dest map[string]struct{}) {
data, err := os.ReadFile(schemaPath)
if err != nil {
return
}
var obj map[string]interface{}
if err := json.Unmarshal(data, &obj); err != nil {
return
}
node := interface{}(obj)
for _, key := range keysPath {
m, ok := node.(map[string]interface{})
if !ok {
return
}
node, ok = m[key]
if !ok {
return
}
}
props, ok := node.(map[string]interface{})
if !ok {
return
}
for k := range props {
dest[k] = struct{}{}
}
}
// loadTypeInfoFromSchema records required/properties for each type under the node
func loadTypeInfoFromSchema(schemaPath string, keysPath []string, dest map[string]schemaTypeInfo) {
data, err := os.ReadFile(schemaPath)
if err != nil {
return
}
var obj map[string]interface{}
if err := json.Unmarshal(data, &obj); err != nil {
return
}
node := interface{}(obj)
for _, key := range keysPath {
m, ok := node.(map[string]interface{})
if !ok {
return
}
node, ok = m[key]
if !ok {
return
}
}
typesNode, ok := node.(map[string]interface{})
if !ok {
return
}
for typeName, raw := range typesNode {
m, ok := raw.(map[string]interface{})
if !ok {
continue
}
info := schemaTypeInfo{required: map[string]struct{}{}, properties: map[string]struct{}{}}
if req, ok := m["required"].([]interface{}); ok {
for _, r := range req {
if s, ok := r.(string); ok {
info.required[s] = struct{}{}
}
}
}
if props, ok := m["properties"].(map[string]interface{}); ok {
for prop := range props {
info.properties[prop] = struct{}{}
}
}
dest[typeName] = info
}
}
// validateAnalyzers delegates to the generic typed-list validator
func validateAnalyzers(parsed map[string]interface{}, content string) []LintError {
return validateTypedList(parsed, content, "analyzers", "analyzer", knownAnalyzerTypes, analyzerTypeInfo)
}
func validateCollectors(parsed map[string]interface{}, content string, field string) []LintError {
return validateTypedList(parsed, content, field, "collector", knownCollectorTypes, collectorTypeInfo)
}
// validateTypedList provides generic validation for lists of typed single-key objects
func validateTypedList(
parsed map[string]interface{},
content string,
listKey string,
subject string,
knownTypes map[string]struct{},
typeInfo map[string]schemaTypeInfo,
) []LintError {
var errs []LintError
spec, ok := parsed["spec"].(map[string]interface{})
if !ok {
return errs
}
raw, exists := spec[listKey]
if !exists {
return errs
}
list, ok := raw.([]interface{})
if !ok {
errs = append(errs, LintError{
Line: findLineNumber(content, listKey+":"),
Field: "spec." + listKey,
Message: fmt.Sprintf("Expected '%s' to be a list", listKey),
})
return errs
}
for i, item := range list {
m, ok := item.(map[string]interface{})
if !ok {
errs = append(errs, LintError{
Line: findListItemLine(content, listKey, i),
Field: fmt.Sprintf("spec.%s[%d]", listKey, i),
Message: fmt.Sprintf("Expected %s entry to be a mapping", subject),
})
continue
}
// Count non-docString keys (docString is metadata in v1beta3, not a type)
typeCount := 0
var typ string
var body interface{}
for k, v := range m {
if k == "docString" {
// docString is metadata, not a type - skip it
continue
} else {
typeCount++
typ, body = k, v
}
}
// Check that we have exactly one type (excluding docString)
if typeCount != 1 {
errs = append(errs, LintError{
Line: findListItemLine(content, listKey, i),
Field: fmt.Sprintf("spec.%s[%d]", listKey, i),
Message: fmt.Sprintf("%s entry must specify exactly one %s type", strings.Title(subject), subject),
})
continue
}
// If no actual type was found (only docString), skip further validation
if typ == "" {
continue
}
if len(knownTypes) > 0 {
if _, ok := knownTypes[typ]; !ok {
errs = append(errs, LintError{
Line: findListItemLine(content, listKey, i),
Field: fmt.Sprintf("spec.%s[%d]", listKey, i),
Message: fmt.Sprintf("Unknown %s type '%s'", subject, typ),
})
}
}
bodyMap, ok := body.(map[string]interface{})
if !ok {
errs = append(errs, LintError{
Line: findListItemLine(content, listKey, i),
Field: fmt.Sprintf("spec.%s[%d].%s", listKey, i, typ),
Message: fmt.Sprintf("Expected %s definition to be a mapping", subject),
})
continue
}
if ti, ok := typeInfo[typ]; ok {
for req := range ti.required {
if _, ok := bodyMap[req]; !ok {
errs = append(errs, LintError{
Line: findListItemLine(content, listKey, i),
Field: fmt.Sprintf("spec.%s[%d].%s.%s", listKey, i, typ, req),
Message: fmt.Sprintf("Missing required field '%s' for %s '%s'", req, subject, typ),
})
}
}
for k := range bodyMap {
if _, ok := ti.properties[k]; !ok {
errs = append(errs, LintError{
Line: findListItemLine(content, listKey, i),
Field: fmt.Sprintf("spec.%s[%d].%s.%s", listKey, i, typ, k),
Message: fmt.Sprintf("Unknown field '%s' for %s '%s'", k, subject, typ),
})
}
}
}
}
return errs
}
+2 -157
View File
@@ -8,14 +8,12 @@ import (
"github.com/pkg/errors"
"github.com/replicatedhq/troubleshoot/internal/util"
troubleshootv1beta2 "github.com/replicatedhq/troubleshoot/pkg/apis/troubleshoot/v1beta2"
troubleshootv1beta3 "github.com/replicatedhq/troubleshoot/pkg/apis/troubleshoot/v1beta3"
"github.com/replicatedhq/troubleshoot/pkg/client/troubleshootclientset/scheme"
"github.com/replicatedhq/troubleshoot/pkg/constants"
"github.com/replicatedhq/troubleshoot/pkg/docrewrite"
"github.com/replicatedhq/troubleshoot/pkg/types"
v1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/client-go/kubernetes"
"k8s.io/klog/v2"
"sigs.k8s.io/yaml"
)
@@ -25,8 +23,6 @@ var decoder runtime.Decoder
func init() {
// Allow serializing Secrets and ConfigMaps
_ = v1.AddToScheme(scheme.Scheme)
// Ensure v1beta3 Troubleshoot types are registered for decoding
_ = troubleshootv1beta3.AddToScheme(scheme.Scheme)
decoder = scheme.Codecs.UniversalDeserializer()
}
@@ -44,14 +40,6 @@ type LoadOptions struct {
// If true, the loader will return an error if any of the specs are not valid
// else the invalid specs will be ignored
Strict bool
// Client is the kubernetes client used for resolving v1beta3 StringOrValueFrom fields
// If not provided, v1beta3 specs with secretKeyRef will fail to load
Client kubernetes.Interface
// Namespace is the default namespace for resolving v1beta3 secret references
// Defaults to "default" if not provided
Namespace string
}
// TODO: Additional requirements needed in this package
@@ -76,18 +64,8 @@ type LoadOptions struct {
// the documents are not valid, else the invalid documents will be ignored.
func LoadSpecs(ctx context.Context, opt LoadOptions) (*TroubleshootKinds, error) {
opt.RawSpecs = append(opt.RawSpecs, opt.RawSpec)
// Default namespace to "default" if not provided
namespace := opt.Namespace
if namespace == "" {
namespace = "default"
}
l := specLoader{
strict: opt.Strict,
client: opt.Client,
namespace: namespace,
ctx: ctx,
strict: opt.Strict,
}
return l.loadFromStrings(opt.RawSpecs...)
@@ -162,10 +140,7 @@ func NewTroubleshootKinds() *TroubleshootKinds {
}
type specLoader struct {
strict bool
client kubernetes.Interface
namespace string
ctx context.Context
strict bool
}
// loadFromStrings accepts a list of strings (exploded) which should be yaml documents
@@ -241,23 +216,6 @@ func (l *specLoader) loadFromSplitDocs(splitdocs []string) (*TroubleshootKinds,
kinds := NewTroubleshootKinds()
for _, doc := range splitdocs {
// Check if this is a v1beta3 spec
var parsed parsedDoc
if err := yaml.Unmarshal([]byte(doc), &parsed); err == nil && parsed.APIVersion == constants.Troubleshootv1beta3Kind {
// Only handle v1beta3 SupportBundle specially (to resolve valueFrom and convert)
if parsed.Kind == "SupportBundle" {
if err := l.loadV1Beta3Spec(doc, kinds); err != nil {
// Always surface v1beta3 SupportBundle errors so users get actionable guidance
return nil, err
}
// handled as support bundle; move to next doc
continue
}
// For other v1beta3 kinds (e.g., Preflight), fall through to the generic
// v1beta3->v1beta2 conversion path below to preserve prior behavior.
}
// Handle v1beta2 and v1beta1 specs
converted, err := docrewrite.ConvertToV1Beta2([]byte(doc))
if err != nil {
if !l.strict {
@@ -305,119 +263,6 @@ func (l *specLoader) loadFromSplitDocs(splitdocs []string) (*TroubleshootKinds,
return kinds, nil
}
// loadV1Beta3Spec handles loading and resolving v1beta3 specs
func (l *specLoader) loadV1Beta3Spec(doc string, kinds *TroubleshootKinds) error {
// Unmarshal to v1beta3 types
obj, _, err := decoder.Decode([]byte(doc), nil, nil)
if err != nil {
return types.NewExitCodeError(constants.EXIT_CODE_SPEC_ISSUES,
errors.Wrapf(err, "failed to decode v1beta3 spec: '%s'", doc),
)
}
switch v3spec := obj.(type) {
case *troubleshootv1beta3.SupportBundle:
// Resolve secrets and convert to v1beta2
requiresClient := v1beta3SpecRequiresClient(&v3spec.Spec)
if requiresClient && l.client == nil {
return types.NewExitCodeError(
constants.EXIT_CODE_SPEC_ISSUES,
errors.New("kubernetes client required"),
)
}
v2spec, err := troubleshootv1beta3.ConvertToV1Beta2WithResolution(l.ctx, &v3spec.Spec, l.client, l.namespace)
if err != nil {
// When secret/configmap references are present, show a clear guidance message
// instead of leaking underlying RBAC or lookup errors.
if requiresClient {
return types.NewExitCodeError(
constants.EXIT_CODE_SPEC_ISSUES,
errors.New("this v1beta3 SupportBundle uses secret/configmap references and must be run in a cluster"),
)
}
return types.NewExitCodeError(constants.EXIT_CODE_SPEC_ISSUES,
errors.Wrap(err, "failed to resolve and convert v1beta3 support bundle spec"),
)
}
// Create v1beta2 support bundle
v2bundle := troubleshootv1beta2.SupportBundle{
TypeMeta: v3spec.TypeMeta,
ObjectMeta: v3spec.ObjectMeta,
Spec: *v2spec,
}
// Update apiVersion to v1beta2
v2bundle.APIVersion = constants.Troubleshootv1beta2Kind
kinds.SupportBundlesV1Beta2 = append(kinds.SupportBundlesV1Beta2, v2bundle)
// TODO: Add other v1beta3 types as they are implemented
default:
return types.NewExitCodeError(constants.EXIT_CODE_SPEC_ISSUES,
errors.Errorf("unsupported v1beta3 kind: %T", v3spec),
)
}
return nil
}
// v1beta3SpecRequiresClient returns true if the v1beta3 spec contains any
// StringOrValueFrom references that require fetching from the cluster.
func v1beta3SpecRequiresClient(spec *troubleshootv1beta3.SupportBundleSpec) bool {
if spec == nil || spec.Collectors == nil {
return false
}
for _, c := range spec.Collectors {
if c == nil {
continue
}
// Database collectors
if c.Postgres != nil && databaseRequiresClient(c.Postgres) {
return true
}
if c.Mysql != nil && databaseRequiresClient(c.Mysql) {
return true
}
if c.Mssql != nil && databaseRequiresClient(c.Mssql) {
return true
}
if c.Redis != nil && databaseRequiresClient(c.Redis) {
return true
}
}
return false
}
func databaseRequiresClient(db *troubleshootv1beta3.Database) bool {
if db == nil {
return false
}
if stringOrValueFromHasRef(db.URI) {
return true
}
if db.TLS != nil {
if stringOrValueFromHasRef(db.TLS.CACert) ||
stringOrValueFromHasRef(db.TLS.ClientCert) ||
stringOrValueFromHasRef(db.TLS.ClientKey) {
return true
}
}
return false
}
func stringOrValueFromHasRef(s troubleshootv1beta3.StringOrValueFrom) bool {
if s.ValueFrom == nil {
return false
}
return s.ValueFrom.SecretKeyRef != nil || s.ValueFrom.ConfigMapKeyRef != nil
}
func isSecret(parsedDocHead parsedDoc) bool {
if parsedDocHead.Kind == "Secret" && parsedDocHead.APIVersion == "v1" {
return true
-289
View File
@@ -1,289 +0,0 @@
package loader
import (
"context"
"testing"
troubleshootv1beta2 "github.com/replicatedhq/troubleshoot/pkg/apis/troubleshoot/v1beta2"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes/fake"
)
func TestLoadSpecs_V1Beta3WithSecretRef(t *testing.T) {
// Create test secret
secret := &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Name: "postgres-secret",
Namespace: "default",
},
Data: map[string][]byte{
"uri": []byte("postgresql://user:password@localhost:5432/mydb"),
},
}
client := fake.NewSimpleClientset(secret)
spec := `
apiVersion: troubleshoot.sh/v1beta3
kind: SupportBundle
metadata:
name: test-bundle
spec:
collectors:
- postgres:
collectorName: main-db
uri:
valueFrom:
secretKeyRef:
name: postgres-secret
key: uri
`
kinds, err := LoadSpecs(context.Background(), LoadOptions{
RawSpec: spec,
Client: client,
Namespace: "default",
})
require.NoError(t, err)
require.NotNil(t, kinds)
require.Len(t, kinds.SupportBundlesV1Beta2, 1)
bundle := kinds.SupportBundlesV1Beta2[0]
assert.Equal(t, "test-bundle", bundle.Name)
assert.Equal(t, "troubleshoot.sh/v1beta2", bundle.APIVersion)
require.Len(t, bundle.Spec.Collectors, 1)
require.NotNil(t, bundle.Spec.Collectors[0].Postgres)
assert.Equal(t, "main-db", bundle.Spec.Collectors[0].Postgres.CollectorName)
assert.Equal(t, "postgresql://user:password@localhost:5432/mydb", bundle.Spec.Collectors[0].Postgres.URI)
}
func TestLoadSpecs_V1Beta3WithTLSSecrets(t *testing.T) {
// Create test secret with TLS certs
secret := &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Name: "tls-secret",
Namespace: "default",
},
Data: map[string][]byte{
"ca.crt": []byte("CA_CERT_DATA"),
"client.crt": []byte("CLIENT_CERT_DATA"),
"client.key": []byte("CLIENT_KEY_DATA"),
},
}
client := fake.NewSimpleClientset(secret)
spec := `
apiVersion: troubleshoot.sh/v1beta3
kind: SupportBundle
metadata:
name: test-bundle
spec:
collectors:
- postgres:
uri:
value: "postgresql://localhost:5432/db"
tls:
cacert:
valueFrom:
secretKeyRef:
name: tls-secret
key: ca.crt
clientCert:
valueFrom:
secretKeyRef:
name: tls-secret
key: client.crt
clientKey:
valueFrom:
secretKeyRef:
name: tls-secret
key: client.key
`
kinds, err := LoadSpecs(context.Background(), LoadOptions{
RawSpec: spec,
Client: client,
Namespace: "default",
})
require.NoError(t, err)
require.NotNil(t, kinds)
require.Len(t, kinds.SupportBundlesV1Beta2, 1)
bundle := kinds.SupportBundlesV1Beta2[0]
require.Len(t, bundle.Spec.Collectors, 1)
require.NotNil(t, bundle.Spec.Collectors[0].Postgres)
require.NotNil(t, bundle.Spec.Collectors[0].Postgres.TLS)
assert.Equal(t, "CA_CERT_DATA", bundle.Spec.Collectors[0].Postgres.TLS.CACert)
assert.Equal(t, "CLIENT_CERT_DATA", bundle.Spec.Collectors[0].Postgres.TLS.ClientCert)
assert.Equal(t, "CLIENT_KEY_DATA", bundle.Spec.Collectors[0].Postgres.TLS.ClientKey)
}
func TestLoadSpecs_V1Beta3MultipleCollectors(t *testing.T) {
pgSecret := &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Name: "postgres-secret",
Namespace: "default",
},
Data: map[string][]byte{
"uri": []byte("postgresql://localhost:5432/db"),
},
}
mysqlSecret := &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Name: "mysql-secret",
Namespace: "default",
},
Data: map[string][]byte{
"uri": []byte("mysql://localhost:3306/db"),
},
}
client := fake.NewSimpleClientset(pgSecret, mysqlSecret)
spec := `
apiVersion: troubleshoot.sh/v1beta3
kind: SupportBundle
metadata:
name: test-bundle
spec:
collectors:
- postgres:
uri:
valueFrom:
secretKeyRef:
name: postgres-secret
key: uri
- mysql:
uri:
valueFrom:
secretKeyRef:
name: mysql-secret
key: uri
`
kinds, err := LoadSpecs(context.Background(), LoadOptions{
RawSpec: spec,
Client: client,
Namespace: "default",
})
require.NoError(t, err)
require.NotNil(t, kinds)
require.Len(t, kinds.SupportBundlesV1Beta2, 1)
bundle := kinds.SupportBundlesV1Beta2[0]
require.Len(t, bundle.Spec.Collectors, 2)
require.NotNil(t, bundle.Spec.Collectors[0].Postgres)
assert.Equal(t, "postgresql://localhost:5432/db", bundle.Spec.Collectors[0].Postgres.URI)
require.NotNil(t, bundle.Spec.Collectors[1].Mysql)
assert.Equal(t, "mysql://localhost:3306/db", bundle.Spec.Collectors[1].Mysql.URI)
}
func TestLoadSpecs_V1Beta3WithoutClient(t *testing.T) {
spec := `
apiVersion: troubleshoot.sh/v1beta3
kind: SupportBundle
metadata:
name: test-bundle
spec:
collectors:
- postgres:
uri:
valueFrom:
secretKeyRef:
name: postgres-secret
key: uri
`
_, err := LoadSpecs(context.Background(), LoadOptions{
RawSpec: spec,
Strict: true, // Enable strict mode to get error instead of warning
// No client provided
})
require.Error(t, err)
assert.Contains(t, err.Error(), "kubernetes client required")
}
func TestLoadSpecs_V1Beta3MixedWithV1Beta2(t *testing.T) {
secret := &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Name: "postgres-secret",
Namespace: "default",
},
Data: map[string][]byte{
"uri": []byte("postgresql://localhost:5432/db"),
},
}
client := fake.NewSimpleClientset(secret)
specs := `
---
apiVersion: troubleshoot.sh/v1beta3
kind: SupportBundle
metadata:
name: v1beta3-bundle
spec:
collectors:
- postgres:
uri:
valueFrom:
secretKeyRef:
name: postgres-secret
key: uri
---
apiVersion: troubleshoot.sh/v1beta2
kind: SupportBundle
metadata:
name: v1beta2-bundle
spec:
collectors:
- clusterInfo: {}
`
kinds, err := LoadSpecs(context.Background(), LoadOptions{
RawSpec: specs,
Client: client,
Namespace: "default",
})
require.NoError(t, err)
require.NotNil(t, kinds)
require.Len(t, kinds.SupportBundlesV1Beta2, 2)
// Find the v1beta3-converted bundle
var v3Bundle *troubleshootv1beta2.SupportBundle
var v2Bundle *troubleshootv1beta2.SupportBundle
for i := range kinds.SupportBundlesV1Beta2 {
if kinds.SupportBundlesV1Beta2[i].Name == "v1beta3-bundle" {
v3Bundle = &kinds.SupportBundlesV1Beta2[i]
}
if kinds.SupportBundlesV1Beta2[i].Name == "v1beta2-bundle" {
v2Bundle = &kinds.SupportBundlesV1Beta2[i]
}
}
require.NotNil(t, v3Bundle, "v1beta3 bundle should be converted and loaded")
require.NotNil(t, v2Bundle, "v1beta2 bundle should be loaded")
// Verify v1beta3 bundle was resolved correctly
require.Len(t, v3Bundle.Spec.Collectors, 1)
require.NotNil(t, v3Bundle.Spec.Collectors[0].Postgres)
assert.Equal(t, "postgresql://localhost:5432/db", v3Bundle.Spec.Collectors[0].Postgres.URI)
// Verify v1beta2 bundle was loaded correctly
require.Len(t, v2Bundle.Spec.Collectors, 1)
require.NotNil(t, v2Bundle.Spec.Collectors[0].ClusterInfo)
}
+3 -24
View File
@@ -16,9 +16,8 @@ import (
)
// ExtractLicenseFromBundle extracts the license ID from a support bundle
// It first looks for cluster-resources/license.json, then falls back to searching
// cluster-resources/configmaps/* for a license field
// Returns both the license ID and the app slug
// It looks in cluster-resources/configmaps/* for a license field
// Returns both the license ID and the app slug (from the filename where license was found)
func ExtractLicenseFromBundle(bundlePath string) (string, string, error) {
file, err := os.Open(bundlePath)
if err != nil {
@@ -43,27 +42,7 @@ func ExtractLicenseFromBundle(bundlePath string) (string, string, error) {
return "", "", errors.Wrap(err, "failed to read tar header")
}
// First priority: check for the new license.json file
if strings.Contains(header.Name, "cluster-resources/license.json") && header.Typeflag == tar.TypeReg {
content := make([]byte, header.Size)
if _, err := io.ReadFull(tarReader, content); err != nil {
continue
}
// Parse the license.json file
var licenseData struct {
LicenseID string `json:"licenseID"`
AppSlug string `json:"appSlug"`
}
if err := json.Unmarshal(content, &licenseData); err == nil {
if licenseData.LicenseID != "" && licenseData.AppSlug != "" {
return licenseData.LicenseID, licenseData.AppSlug, nil
}
}
continue
}
// Fallback: process files in cluster-resources/configmaps/
// Only process files in cluster-resources/configmaps/ (may be nested under bundle directory)
if !strings.Contains(header.Name, "cluster-resources/configmaps/") {
continue
}
+5 -17
View File
@@ -10,7 +10,7 @@ import (
// UploadToReplicatedApp uploads a support bundle directly to replicated.app
// using the app slug as the upload path
func UploadToReplicatedApp(bundlePath, licenseID, appSlug, uploadDomain string) error {
func UploadToReplicatedApp(bundlePath, licenseID, appSlug string) error {
// Open the bundle file
file, err := os.Open(bundlePath)
if err != nil {
@@ -23,14 +23,8 @@ func UploadToReplicatedApp(bundlePath, licenseID, appSlug, uploadDomain string)
return errors.Wrap(err, "failed to stat file")
}
// Use custom domain if provided, otherwise default to replicated.app
domain := uploadDomain
if domain == "" {
domain = "replicated.app"
}
// Build the upload URL using the app slug
uploadURL := fmt.Sprintf("https://%s/supportbundle/upload/%s", domain, appSlug)
uploadURL := fmt.Sprintf("https://replicated.app/supportbundle/upload/%s", appSlug)
// Create the request
req, err := http.NewRequest("POST", uploadURL, file)
@@ -59,7 +53,7 @@ func UploadToReplicatedApp(bundlePath, licenseID, appSlug, uploadDomain string)
}
// UploadBundleAutoDetect uploads a support bundle with automatic license and app slug detection
func UploadBundleAutoDetect(bundlePath string, providedLicenseID, providedAppSlug, uploadDomain string) error {
func UploadBundleAutoDetect(bundlePath string, providedLicenseID, providedAppSlug string) error {
licenseID := providedLicenseID
// Always extract from bundle to get app slug (and license if not provided)
@@ -85,15 +79,9 @@ func UploadBundleAutoDetect(bundlePath string, providedLicenseID, providedAppSlu
appSlug = extractedAppSlug
}
// Determine target domain for upload message
targetDomain := uploadDomain
if targetDomain == "" {
targetDomain = "replicated.app"
}
// Upload the bundle
fmt.Printf("Uploading support bundle to %s...\n", targetDomain)
if err := UploadToReplicatedApp(bundlePath, licenseID, appSlug, uploadDomain); err != nil {
fmt.Printf("Uploading support bundle to replicated.app...\n")
if err := UploadToReplicatedApp(bundlePath, licenseID, appSlug); err != nil {
return errors.Wrap(err, "failed to upload bundle")
}