32 Commits
Author SHA1 Message Date
Noah Campbell b4e3ed278a Removes silent failing (#1877)
* preserves stdout and stderr from collectors

* Delete eliminate-silent-failures.md

* Update host_kernel_modules_test.go

* added error logs when a collector fails to start

* Update host_filesystem_performance_linux.go

* fixed error saving logic inconsistency

* Update collect.go
2025-10-04 15:14:10 -07:00
Benjamin Yang c67775c1a0 feat: clean tokenization system implementation (#1874)
Core tokenization functionality with minimal file changes:

 Core Features:
- Intelligent tokenization engine (tokenizer.go)
- Context-aware secret classification (PASSWORD, APIKEY, DATABASE, etc.)
- Cross-file correlation with deterministic HMAC-SHA256 tokens
- Optional encrypted mapping for token→original value resolution

 Integration:
- CLI flags: --tokenize, --redaction-map, --encrypt-redaction-map
- Updated all redactor types: literal, single-line, multi-line, YAML
- Support bundle integration with auto-upload compatibility
- Backward compatibility: preserves ***HIDDEN*** when disabled

 Production Ready:
- Only 11 essential files (vs 31 in original PR)
- No excessive test files or documentation
- Clean build, all functionality verified
- Maintains existing redaction behavior by default

Token format: ***TOKEN_<TYPE>_<HASH>*** (e.g., ***TOKEN_PASSWORD_A1B2C3***)
2025-09-30 15:04:23 -05:00
Gerard Nguyen 7bb88e6b83 feat: ensure Copy collector run last (#1688)
* ensure Copy collector run last

* * add unit test
* reorder in Preflight as well
2024-11-15 10:59:38 +11:00
deeeea7cec exec remote host collectors in a daemonset (#1671)
Co-authored-by: Gerard Nguyen <gerard@replicated.com>
Co-authored-by: Dexter Yan <yanshaocong@gmail.com>
2024-11-12 08:47:24 +13:00
Diamon Wiggins 06506ed95d Fix remote host collection RBAC checks (#1672)
* fix remote host collection rbac checks

* move saveNodeList into collectRemoteHost function

* fix resource attribute list and retrieve namespace from kubeconfig

* revert change to set a default namespace from kubeconfig

* remove duplicate code
2024-11-07 10:07:27 -05:00
Dexter Yan 350418c6e9 feat(host-collector): add progress for host collector (#1659) 2024-10-25 15:34:09 +13:00
b88bc8ddf7 Refactor Multi Node Analyzers (#1646)
* initial refactor of host os analyzer

* refactor remote collect analysis

---------

Signed-off-by: Evans Mungai <evans@replicated.com>
Co-authored-by: Gerard Nguyen <gerard@replicated.com>
Co-authored-by: Evans Mungai <evans@replicated.com>
2024-10-22 10:45:50 +13:00
Gerard Nguyen 289102f16d bug: fix nil check in host collector filter (#1653)
* add nil check in filter host collector
2024-10-18 15:58:32 +11:00
Diamon WigginsandDexter Yan 8105fa00e9 Refactor Remote Host Collection (#1633)
* refactor remote collectors

* add remotecollect params struct

* remove commented checkrbac function

* removed unused function

* add temp comments

* refactor to not require RemoteCollect method per collector

* removed unneeded param

* removed unneeded param

* more refactor

* more refactor

* remove unneeded function

* remove debug print

* fix analyzer results

* move rbac to separate file

* be more specific with rbac function name

* fix imports

* fix node list file

* make k8s rest client config consistent with in cluster collection

* add ctx and otel tracing

* add test for allCollectedData

* move runHostCollectorsInPod to spec instead of metadata

* make generate

* fix broken references to supportbundle metadata

* add e2e tests

* update loader tests

* fix tests

* fix hostos remote collector spec

* update remoteHostCollectrs.yaml

---------

Co-authored-by: Dexter Yan <yanshaocong@gmail.com>
2024-10-09 18:38:49 +13:00
Dexter Yan e97b9613a5 feat(support-bundle): add runHostCollectorsInPod in spec (#1608) 2024-09-20 11:57:58 -05:00
Evans Mungai aea4f7c87c feat: Optionally save preflight bundles to disk (#1612)
* feat: Optionally save preflight bundles to disk

Signed-off-by: Evans Mungai <evans@replicated.com>

* Add e2e test of saving preflight bundle

Signed-off-by: Evans Mungai <evans@replicated.com>

* Update cli docs

Signed-off-by: Evans Mungai <evans@replicated.com>

* Expose GetVersionFile function publicly

Signed-off-by: Evans Mungai <evans@replicated.com>

* Store analysis.json file in preflight bundle

Signed-off-by: Evans Mungai <evans@replicated.com>

* Run go fmt when running lint fixers

Signed-off-by: Evans Mungai <evans@replicated.com>

* Always generate a preflight bundle in CLI

Signed-off-by: Evans Mungai <evans@replicated.com>

* Print saving bundle message to stderr

Signed-off-by: Evans Mungai <evans@replicated.com>

* Revert changes in docs directory

Signed-off-by: Evans Mungai <evans@replicated.com>

* Use NewResult constructor

Signed-off-by: Evans Mungai <evans@replicated.com>

* Log always when preflight bundle is saved to disk

Signed-off-by: Evans Mungai <evans@replicated.com>

---------

Signed-off-by: Evans Mungai <evans@replicated.com>
2024-09-16 23:36:52 +01:00
Pavan Sokke Nagaraj 39314ef200 chore: export error ErrInsufficientPermissionsToRun and func ShowTextResultsStructured (#1297)
* chore: move ErrInsufficientPermissions to collect

* chore: export func ShowTextResultsStructured

* chore: rename to ErrInsufficientPermissionsToRun
2023-08-08 14:17:15 -04:00
Evans Mungai 546ffde14b feat: use klog as the default logging library (#1008) 2023-02-24 18:24:51 +00:00
Evans Mungai 100f9a13b6 feat: Record summary of execution times of support bundle operations (collect/redact/analyse) (#935)
When running a support bundle, we want to know how long each operation
(collect, redact, analyze) takes. This commit adds a new trace exporter
that records the start and end times of each operation, and then prints
a summary of the execution. The summary is also stored in the support
bundle.

Related to #923
2023-02-07 09:50:21 +00:00
Diamon WigginsandEvans Mungai 4fca6aff98 Deduplication for In-Cluster Collectors (#972)
* adding dedup for in cluster collectors

* add tests

* return collector as is whenever marshalling to json fails

---------

Co-authored-by: Evans Mungai <evans@replicated.com>
2023-02-01 14:14:43 -05:00
Evans Mungai a523551da9 feat(redactors): Run redactors on an existing support bundle (#887)
* feat(redactors): Run redactors on an existing support bundle

Add redact subcommand to support-bundle to allow running redactors on an
existing bundle to creating a new redacted bundle.

The command will be launched like so

support-bundle redact <redactor urls> --bundle support-bundle.tar.gz

Fixes: #705
2023-01-03 18:05:15 +00:00
Dexter Yan be26462c19 feat(cluster_resources): increase default client burst and qps (#920)
* feat(collect): add client burst and qps
2022-12-22 09:49:42 +13:00
Diamon Wiggins a4c4b24056 Deduplication for Cluster Resources Collector (#832)
* add dedup for cluster resources collector
* restructure both collect.go in both pkg/supportbundle and pkg/preflight to be more similar for eventual refactor
2022-12-07 15:10:31 -04:00
Evans Mungai fbbcf87405 feat(collectors): Store all pod logs in cluster-resources directory (#821)
* feat(collectors): Store all pod logs in cluster-resources directory

All pod logs collected by the logs collector will now be stored in
/cluster-resources/pods/logs/[namespace]/[pod]/[container].log. This
will provide consistency and allow sbctl to find the logs when we run
`kubectl logs <pod>`. To allow backwards compatibility, symlinks of the
log files will be created in the current expected locations.

Closes: #744
2022-11-22 07:10:34 +13:00
Xav Paice 3513eeca19 Ensure clusterResources is added prior to other collectors (#768)
This change ensures that the clusterResources collector runs prior to any others
in order to not collect info on pods that collectors run during collection.

Additionally centralizes functions that are common to all collection to make future
maintenance simpler.

Fixes: #767
2022-11-01 12:16:01 +13:00
Diamon Wiggins 04c7a18da3 Fix Progress Callback for Support Bundle Collection (#781)
fix progress callback for support bundle and revert collector title changes
2022-10-14 12:29:59 -04:00
Diamon Wiggins c7b84ad1e5 Refactor in-clusters collectors to use struct per collector (#670)
refactor in-clusters collectors to use struct per collector
2022-10-03 13:53:05 -04:00
Xav Paice f06201e050 Small typo fix in collect.go 2022-08-02 14:36:34 +12:00
Diamon Wiggins 114749d5b7 always attempt to generate a support bundle when host collectors exists 2022-07-30 18:57:24 -04:00
Diamon Wiggins cab3fc7f4e Redact Host Collectors in Support Bundles (#614)
Add redactors for Host Collectors
2022-07-01 18:57:58 -04:00
diamonwiggins 17fe3db79f adding host collectors to support bundles 2022-05-11 22:50:03 +00:00
Dimitri KoshkinandSalah Aldeen Al Saleh 111396eb39 fix: pass redact flag when running support-bundle (#406)
Co-authored-by: Salah Aldeen Al Saleh <sg.alsaleh@gmail.com>
2021-10-20 10:47:27 -07:00
divolgin 0e8bedc281 Save collector data to disk directly 2021-09-29 00:15:02 +00:00
Salah Aldeen Al Saleh 465a533640 store analysis in the support bundle (#417)
* store analysis in the support bundle
2021-09-10 11:58:16 -07:00
emosbaugh 8dcfa9886d Copy from host collector (#391)
* Copy from host collector

* namespace improvements

* better support for multiple nodes
2021-07-22 12:25:59 -07:00
emosbaugh 39350b5722 ConfigMap collector and secrets can be collected by selectors (#384)
* ConfigMap collector and secrets can be collected by selectors

* follow docs

* Pass context and kubernetes client to collectors

* collect tests

* analyze tests

* fix tests

* improvements
2021-07-08 16:30:26 -07:00
Dan Stough b8622a65ad chore(support-bundle): refactor out support bundle pkg 2021-05-04 14:41:54 +00:00