69 Commits
Author SHA1 Message Date
Noah CampbellandGitHub b4e3ed278a Removes silent failing (#1877)
* preserves stdout and stderr from collectors

* Delete eliminate-silent-failures.md

* Update host_kernel_modules_test.go

* added error logs when a collector fails to start

* Update host_filesystem_performance_linux.go

* fixed error saving logic inconsistency

* Update collect.go
2025-10-04 15:14:10 -07:00
Benjamin YangandGitHub c67775c1a0 feat: clean tokenization system implementation (#1874)
Core tokenization functionality with minimal file changes:

 Core Features:
- Intelligent tokenization engine (tokenizer.go)
- Context-aware secret classification (PASSWORD, APIKEY, DATABASE, etc.)
- Cross-file correlation with deterministic HMAC-SHA256 tokens
- Optional encrypted mapping for token→original value resolution

 Integration:
- CLI flags: --tokenize, --redaction-map, --encrypt-redaction-map
- Updated all redactor types: literal, single-line, multi-line, YAML
- Support bundle integration with auto-upload compatibility
- Backward compatibility: preserves ***HIDDEN*** when disabled

 Production Ready:
- Only 11 essential files (vs 31 in original PR)
- No excessive test files or documentation
- Clean build, all functionality verified
- Maintains existing redaction behavior by default

Token format: ***TOKEN_<TYPE>_<HASH>*** (e.g., ***TOKEN_PASSWORD_A1B2C3***)
2025-09-30 15:04:23 -05:00
Marc Campbell 007ef32e77 fmt: 2025-09-29 10:43:51 -07:00
Noah CampbellandGitHub 885819ea9a Auto support bundle upload (#1860)
* basic auto uploading support bundles

* added upload command

* added default vendor endpoint

* added auth system from replicated cli

* fixed case sensitivity issue in YAML parsing

* support bundle uploads for end customers

* app slug flag and detection without licenseID
2025-09-17 12:59:41 -05:00
Diamon WigginsandGitHub 7a6bffeff5 chore: fix noisy info logs (#1808)
* refine logging

* keep progress message at level 0
2025-07-09 20:58:47 -04:00
Ethan MosbaughandGitHub a4a387eb0e chore: CVE-2024-0406 remove github.com/mholt/archiver/v3 dependency (#1793) 2025-06-06 11:35:56 -07:00
9bca9c5245 chore(deps): bump github.com/distribution/distribution/v3 from 3.0.0-rc.3 to 3.0.0 (#1771)
* chore(deps): bump github.com/distribution/distribution/v3

Bumps [github.com/distribution/distribution/v3](https://github.com/distribution/distribution) from 3.0.0-rc.3 to 3.0.0.
- [Release notes](https://github.com/distribution/distribution/releases)
- [Commits](https://github.com/distribution/distribution/compare/v3.0.0-rc.3...v3.0.0)

---
updated-dependencies:
- dependency-name: github.com/distribution/distribution/v3
  dependency-version: 3.0.0
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* update go

* use constant format strings

* f

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrew Lavery <laverya@umich.edu>
2025-04-21 16:45:47 +00:00
Xav PaiceandGitHub 86b7e54466 Revert "feat: save YAML spec used to generate support bundle/preflight" (#1715)
Revert "feat: save YAML spec used to generate support bundle/preflight (#1713)"

This reverts commit f6f51acbd5.
2025-01-06 09:42:58 +11:00
Gerard NguyenandGitHub f6f51acbd5 feat: save YAML spec used to generate support bundle/preflight (#1713)
* save YAML spec of support bundle

* save YAML spec of preflight

* add unit test

* redact TLS private key by default in output spec

* update YAML path for HTTP TLS redactor
2025-01-04 11:35:43 +11:00
AshandGitHub ecc92b1e3e [bug] Quick fix for handling non 200 status codes when loading specs from URI (#1695)
* Quick fix for handling non 200 status codes when loading specs from URI

Go http client already handles 3xx responses for us

* note
2024-11-25 15:04:38 +00:00
Gerard NguyenandGitHub 7bb88e6b83 feat: ensure Copy collector run last (#1688)
* ensure Copy collector run last

* * add unit test
* reorder in Preflight as well
2024-11-15 10:59:38 +11:00
deeeea7cec exec remote host collectors in a daemonset (#1671)
Co-authored-by: Gerard Nguyen <gerard@replicated.com>
Co-authored-by: Dexter Yan <yanshaocong@gmail.com>
2024-11-12 08:47:24 +13:00
d25aa7d0ea fix: Do not fail analysis if node list does not exist (#1678)
* fix: Do not error if node list does not exist

Signed-off-by: Evans Mungai <evans@replicated.com>

* fix test fail

---------

Signed-off-by: Evans Mungai <evans@replicated.com>
Co-authored-by: Dexter Yan <yanshaocong@gmail.com>
2024-11-08 09:53:03 +13:00
Diamon WigginsandGitHub 06506ed95d Fix remote host collection RBAC checks (#1672)
* fix remote host collection rbac checks

* move saveNodeList into collectRemoteHost function

* fix resource attribute list and retrieve namespace from kubeconfig

* revert change to set a default namespace from kubeconfig

* remove duplicate code
2024-11-07 10:07:27 -05:00
Dexter YanandGitHub 350418c6e9 feat(host-collector): add progress for host collector (#1659) 2024-10-25 15:34:09 +13:00
Dexter YanandGitHub 0d21eed5f8 fix(support): add missing host collectors for ParseSupportBundle (#1656)
* fix(support): add missing host collectors for ParseSupportBundle

* update

* add host ananlyers
2024-10-22 13:07:44 +13:00
b88bc8ddf7 Refactor Multi Node Analyzers (#1646)
* initial refactor of host os analyzer

* refactor remote collect analysis

---------

Signed-off-by: Evans Mungai <evans@replicated.com>
Co-authored-by: Gerard Nguyen <gerard@replicated.com>
Co-authored-by: Evans Mungai <evans@replicated.com>
2024-10-22 10:45:50 +13:00
Evans MungaiandGitHub 9c24ab6067 chore: Remove preempted deprecation warnings (#1655)
Signed-off-by: Evans Mungai <evans@replicated.com>
2024-10-22 08:35:36 +11:00
Gerard NguyenandGitHub 289102f16d bug: fix nil check in host collector filter (#1653)
* add nil check in filter host collector
2024-10-18 15:58:32 +11:00
8105fa00e9 Refactor Remote Host Collection (#1633)
* refactor remote collectors

* add remotecollect params struct

* remove commented checkrbac function

* removed unused function

* add temp comments

* refactor to not require RemoteCollect method per collector

* removed unneeded param

* removed unneeded param

* more refactor

* more refactor

* remove unneeded function

* remove debug print

* fix analyzer results

* move rbac to separate file

* be more specific with rbac function name

* fix imports

* fix node list file

* make k8s rest client config consistent with in cluster collection

* add ctx and otel tracing

* add test for allCollectedData

* move runHostCollectorsInPod to spec instead of metadata

* make generate

* fix broken references to supportbundle metadata

* add e2e tests

* update loader tests

* fix tests

* fix hostos remote collector spec

* update remoteHostCollectrs.yaml

---------

Co-authored-by: Dexter Yan <yanshaocong@gmail.com>
2024-10-09 18:38:49 +13:00
Gerard NguyenandGitHub c1c4b612a4 feat: [sc-113128] Create node list file before running remote host collector (#1632)
* create node list
2024-10-01 14:43:24 +10:00
Dexter YanandGitHub e97b9613a5 feat(support-bundle): add runHostCollectorsInPod in spec (#1608) 2024-09-20 11:57:58 -05:00
Evans MungaiandGitHub aea4f7c87c feat: Optionally save preflight bundles to disk (#1612)
* feat: Optionally save preflight bundles to disk

Signed-off-by: Evans Mungai <evans@replicated.com>

* Add e2e test of saving preflight bundle

Signed-off-by: Evans Mungai <evans@replicated.com>

* Update cli docs

Signed-off-by: Evans Mungai <evans@replicated.com>

* Expose GetVersionFile function publicly

Signed-off-by: Evans Mungai <evans@replicated.com>

* Store analysis.json file in preflight bundle

Signed-off-by: Evans Mungai <evans@replicated.com>

* Run go fmt when running lint fixers

Signed-off-by: Evans Mungai <evans@replicated.com>

* Always generate a preflight bundle in CLI

Signed-off-by: Evans Mungai <evans@replicated.com>

* Print saving bundle message to stderr

Signed-off-by: Evans Mungai <evans@replicated.com>

* Revert changes in docs directory

Signed-off-by: Evans Mungai <evans@replicated.com>

* Use NewResult constructor

Signed-off-by: Evans Mungai <evans@replicated.com>

* Log always when preflight bundle is saved to disk

Signed-off-by: Evans Mungai <evans@replicated.com>

---------

Signed-off-by: Evans Mungai <evans@replicated.com>
2024-09-16 23:36:52 +01:00
Evans MungaiandGitHub e4363a1e50 fix (supportbundle): Add default collectors when expected (#1418)
* fix (supportbundle): Add default collectors when expected

* Remove unnecessary change

* Add default collectors to a empty spec

* Add more tests
2024-01-08 11:32:04 +00:00
Evans MungaiandGitHub e5e26eea14 fix(support-bundle): default in-cluster collectors in host support bundle (#1394)
* fix(support-bundle): default in-cluster collectors in host support bundle

Ensure cluster-resources and cluster-info collectors are present only
when a support bundle spec contains in-cluster collectors.

* Various improvements

* Improve error messages
* Util function appending elements to a nil slice that allows adding
  specs to an empty slice of collectors/analysers/redactors

* Fix failing test
2023-11-27 18:33:02 +00:00
Archit SharmaandGitHub 7038da85b1 Velero analyzer (#1366)
* feat: add velero analyzer (#806)

  * updated schema
  * analyzer without collector
  * tests
  * covers deprecated Restic repository type
  * velero version from deployment image to check deprecated type
  * read for both velero pod kinds (velero*, node-agent*)

---------

Signed-off-by: Archit Sharma <archit@pm.me>
2023-11-03 18:41:17 +05:30
Evans MungaiandGitHub 15a4802cd2 feat: Add dry run flag to print support bundle specs to std out (#1337)
* Add dry-run flag

* No traces on dry run

* More refactoring

* More updates to support bundle binary

* More refactoring changes

* Different approach of loading specs from URIs

* Self review

* More changes after review and testing

* fix how we parse oci image uri

* Remove unnecessary comment

* Add missing file

* Fix failing tests

* Better error check for no collectors

* Add default collectors when parsing support bundle specs

* Add missed test fixture

* Download specs with correct headers

* Fix typo
2023-10-10 18:43:32 +01:00
Pavan Sokke NagarajandGitHub 39314ef200 chore: export error ErrInsufficientPermissionsToRun and func ShowTextResultsStructured (#1297)
* chore: move ErrInsufficientPermissions to collect

* chore: export func ShowTextResultsStructured

* chore: rename to ErrInsufficientPermissionsToRun
2023-08-08 14:17:15 -04:00
Evans MungaiandGitHub 401dfe2c57 feat: add loader APIs to load specs from raw troubleshoot spec (#1202)
* feat: add loader APIs to load specs from a list of yaml docs

The change introduces a loader package that will contain loader
public APIs. The aim of these APIs will be to, given any source of
troubleshoot specs, the loaders will fetch the specs and parse out
all troubleshoot objects that can be extracted.

* Some refactoring

* Some more changes

* More changes caught when testing vendor portal

* Add tests and rename Troubleshoot kinds struct

* Additional test

* Handle ConfigMap and Secrets with multiple specs in them

* Fix failing test

* Revert multidoc split implementation

* Fix merge conflict

* Change LoadFromXXX functions to a single LoadSpecs function
2023-06-06 16:48:29 -04:00
Dexter YanandGitHub 8fe5bffae4 feat(analyzer): remove duplicated analyzers (#1056)
feat(analyzer): add DedupAnalyzers
2023-03-22 11:45:41 -04:00
Evans MungaiandGitHub 546ffde14b feat: use klog as the default logging library (#1008) 2023-02-24 18:24:51 +00:00
Evans MungaiandGitHub 100f9a13b6 feat: Record summary of execution times of support bundle operations (collect/redact/analyse) (#935)
When running a support bundle, we want to know how long each operation
(collect, redact, analyze) takes. This commit adds a new trace exporter
that records the start and end times of each operation, and then prints
a summary of the execution. The summary is also stored in the support
bundle.

Related to #923
2023-02-07 09:50:21 +00:00
yunju.llyandGitHub 1114902707 refactor: keep support bundle concat logic to be consistent with Preflight concat (#1002)
* refactor: keep support bundle concat logic to be consistent with Preflight
* test: add tests for support bundle spec concat function
2023-02-03 11:05:11 +00:00
4fca6aff98 Deduplication for In-Cluster Collectors (#972)
* adding dedup for in cluster collectors

* add tests

* return collector as is whenever marshalling to json fails

---------

Co-authored-by: Evans Mungai <evans@replicated.com>
2023-02-01 14:14:43 -05:00
Diamon WigginsandGitHub 2fcdc77cd3 Standardize Cluster Resources Collector File Paths (#971)
* using const for cluster resources k8s objects to standardize directories and files
2023-01-25 13:34:15 -05:00
Evans MungaiandGitHub a523551da9 feat(redactors): Run redactors on an existing support bundle (#887)
* feat(redactors): Run redactors on an existing support bundle

Add redact subcommand to support-bundle to allow running redactors on an
existing bundle to creating a new redacted bundle.

The command will be launched like so

support-bundle redact <redactor urls> --bundle support-bundle.tar.gz

Fixes: #705
2023-01-03 18:05:15 +00:00
Dexter YanandGitHub be26462c19 feat(cluster_resources): increase default client burst and qps (#920)
* feat(collect): add client burst and qps
2022-12-22 09:49:42 +13:00
Diamon WigginsandGitHub a4c4b24056 Deduplication for Cluster Resources Collector (#832)
* add dedup for cluster resources collector
* restructure both collect.go in both pkg/supportbundle and pkg/preflight to be more similar for eventual refactor
2022-12-07 15:10:31 -04:00
Evans MungaiandGitHub fbbcf87405 feat(collectors): Store all pod logs in cluster-resources directory (#821)
* feat(collectors): Store all pod logs in cluster-resources directory

All pod logs collected by the logs collector will now be stored in
/cluster-resources/pods/logs/[namespace]/[pod]/[container].log. This
will provide consistency and allow sbctl to find the logs when we run
`kubectl logs <pod>`. To allow backwards compatibility, symlinks of the
log files will be created in the current expected locations.

Closes: #744
2022-11-22 07:10:34 +13:00
Diamon WigginsandGitHub c34d80c300 Discover Redactors in Cluster (#827)
Adds the ability to search for support bundle specs and redactors, in both configmaps and secrets
2022-11-10 17:36:51 +13:00
Xav PaiceandGitHub 3513eeca19 Ensure clusterResources is added prior to other collectors (#768)
This change ensures that the clusterResources collector runs prior to any others
in order to not collect info on pods that collectors run during collection.

Additionally centralizes functions that are common to all collection to make future
maintenance simpler.

Fixes: #767
2022-11-01 12:16:01 +13:00
Diamon WigginsandGitHub 04c7a18da3 Fix Progress Callback for Support Bundle Collection (#781)
fix progress callback for support bundle and revert collector title changes
2022-10-14 12:29:59 -04:00
ada manciniandGitHub eb40b9422f implement uri: field (#702) 2022-10-05 15:35:55 +13:00
Diamon WigginsandGitHub c7b84ad1e5 Refactor in-clusters collectors to use struct per collector (#670)
refactor in-clusters collectors to use struct per collector
2022-10-03 13:53:05 -04:00
Diamon Wiggins e53871b4dc adding tests 2022-09-13 23:00:57 -04:00
Diamon Wiggins ed34afc51b adding back default collectors 2022-09-11 18:33:35 -04:00
Diamon Wiggins d4e5c0dba4 discovery support bundle specs from secrets matching label selector 2022-09-11 18:28:24 -04:00
Dan Jones d70ecf7e5f Add a rough test for bundle concatenation
Signed-off-by: Dan Jones <danj@replicated.com>
2022-09-08 17:16:38 +01:00
Dan Jones b5a56e5239 Remove redundant loops
Signed-off-by: Dan Jones <danj@replicated.com>
2022-09-07 09:14:09 +01:00
Dan Jones b6f8ac211e Format
Signed-off-by: Dan Jones <danj@replicated.com>
2022-09-05 15:54:54 +01:00