Noah Campbell
b4e3ed278a
Removes silent failing ( #1877 )
...
* preserves stdout and stderr from collectors
* Delete eliminate-silent-failures.md
* Update host_kernel_modules_test.go
* added error logs when a collector fails to start
* Update host_filesystem_performance_linux.go
* fixed error saving logic inconsistency
* Update collect.go
2025-10-04 15:14:10 -07:00
Benjamin Yang
c67775c1a0
feat: clean tokenization system implementation ( #1874 )
...
Core tokenization functionality with minimal file changes:
✅ Core Features:
- Intelligent tokenization engine (tokenizer.go)
- Context-aware secret classification (PASSWORD, APIKEY, DATABASE, etc.)
- Cross-file correlation with deterministic HMAC-SHA256 tokens
- Optional encrypted mapping for token→original value resolution
✅ Integration:
- CLI flags: --tokenize, --redaction-map, --encrypt-redaction-map
- Updated all redactor types: literal, single-line, multi-line, YAML
- Support bundle integration with auto-upload compatibility
- Backward compatibility: preserves ***HIDDEN*** when disabled
✅ Production Ready:
- Only 11 essential files (vs 31 in original PR)
- No excessive test files or documentation
- Clean build, all functionality verified
- Maintains existing redaction behavior by default
Token format: ***TOKEN_<TYPE>_<HASH>*** (e.g., ***TOKEN_PASSWORD_A1B2C3***)
2025-09-30 15:04:23 -05:00
Gerard Nguyen
7bb88e6b83
feat: ensure Copy collector run last ( #1688 )
...
* ensure Copy collector run last
* * add unit test
* reorder in Preflight as well
2024-11-15 10:59:38 +11:00
deeeea7cec
exec remote host collectors in a daemonset ( #1671 )
...
Co-authored-by: Gerard Nguyen <gerard@replicated.com >
Co-authored-by: Dexter Yan <yanshaocong@gmail.com >
2024-11-12 08:47:24 +13:00
Diamon Wiggins
06506ed95d
Fix remote host collection RBAC checks ( #1672 )
...
* fix remote host collection rbac checks
* move saveNodeList into collectRemoteHost function
* fix resource attribute list and retrieve namespace from kubeconfig
* revert change to set a default namespace from kubeconfig
* remove duplicate code
2024-11-07 10:07:27 -05:00
Dexter Yan
350418c6e9
feat(host-collector): add progress for host collector ( #1659 )
2024-10-25 15:34:09 +13:00
b88bc8ddf7
Refactor Multi Node Analyzers ( #1646 )
...
* initial refactor of host os analyzer
* refactor remote collect analysis
---------
Signed-off-by: Evans Mungai <evans@replicated.com >
Co-authored-by: Gerard Nguyen <gerard@replicated.com >
Co-authored-by: Evans Mungai <evans@replicated.com >
2024-10-22 10:45:50 +13:00
Gerard Nguyen
289102f16d
bug: fix nil check in host collector filter ( #1653 )
...
* add nil check in filter host collector
2024-10-18 15:58:32 +11:00
Diamon Wiggins and Dexter Yan
8105fa00e9
Refactor Remote Host Collection ( #1633 )
...
* refactor remote collectors
* add remotecollect params struct
* remove commented checkrbac function
* removed unused function
* add temp comments
* refactor to not require RemoteCollect method per collector
* removed unneeded param
* removed unneeded param
* more refactor
* more refactor
* remove unneeded function
* remove debug print
* fix analyzer results
* move rbac to separate file
* be more specific with rbac function name
* fix imports
* fix node list file
* make k8s rest client config consistent with in cluster collection
* add ctx and otel tracing
* add test for allCollectedData
* move runHostCollectorsInPod to spec instead of metadata
* make generate
* fix broken references to supportbundle metadata
* add e2e tests
* update loader tests
* fix tests
* fix hostos remote collector spec
* update remoteHostCollectrs.yaml
---------
Co-authored-by: Dexter Yan <yanshaocong@gmail.com >
2024-10-09 18:38:49 +13:00
Dexter Yan
e97b9613a5
feat(support-bundle): add runHostCollectorsInPod in spec ( #1608 )
2024-09-20 11:57:58 -05:00
Evans Mungai
aea4f7c87c
feat: Optionally save preflight bundles to disk ( #1612 )
...
* feat: Optionally save preflight bundles to disk
Signed-off-by: Evans Mungai <evans@replicated.com >
* Add e2e test of saving preflight bundle
Signed-off-by: Evans Mungai <evans@replicated.com >
* Update cli docs
Signed-off-by: Evans Mungai <evans@replicated.com >
* Expose GetVersionFile function publicly
Signed-off-by: Evans Mungai <evans@replicated.com >
* Store analysis.json file in preflight bundle
Signed-off-by: Evans Mungai <evans@replicated.com >
* Run go fmt when running lint fixers
Signed-off-by: Evans Mungai <evans@replicated.com >
* Always generate a preflight bundle in CLI
Signed-off-by: Evans Mungai <evans@replicated.com >
* Print saving bundle message to stderr
Signed-off-by: Evans Mungai <evans@replicated.com >
* Revert changes in docs directory
Signed-off-by: Evans Mungai <evans@replicated.com >
* Use NewResult constructor
Signed-off-by: Evans Mungai <evans@replicated.com >
* Log always when preflight bundle is saved to disk
Signed-off-by: Evans Mungai <evans@replicated.com >
---------
Signed-off-by: Evans Mungai <evans@replicated.com >
2024-09-16 23:36:52 +01:00
Pavan Sokke Nagaraj
39314ef200
chore: export error ErrInsufficientPermissionsToRun and func ShowTextResultsStructured ( #1297 )
...
* chore: move ErrInsufficientPermissions to collect
* chore: export func ShowTextResultsStructured
* chore: rename to ErrInsufficientPermissionsToRun
2023-08-08 14:17:15 -04:00
Evans Mungai
546ffde14b
feat: use klog as the default logging library ( #1008 )
2023-02-24 18:24:51 +00:00
Evans Mungai
100f9a13b6
feat: Record summary of execution times of support bundle operations (collect/redact/analyse) ( #935 )
...
When running a support bundle, we want to know how long each operation
(collect, redact, analyze) takes. This commit adds a new trace exporter
that records the start and end times of each operation, and then prints
a summary of the execution. The summary is also stored in the support
bundle.
Related to #923
2023-02-07 09:50:21 +00:00
Diamon Wiggins and Evans Mungai
4fca6aff98
Deduplication for In-Cluster Collectors ( #972 )
...
* adding dedup for in cluster collectors
* add tests
* return collector as is whenever marshalling to json fails
---------
Co-authored-by: Evans Mungai <evans@replicated.com >
2023-02-01 14:14:43 -05:00
Evans Mungai
a523551da9
feat(redactors): Run redactors on an existing support bundle ( #887 )
...
* feat(redactors): Run redactors on an existing support bundle
Add redact subcommand to support-bundle to allow running redactors on an
existing bundle to creating a new redacted bundle.
The command will be launched like so
support-bundle redact <redactor urls> --bundle support-bundle.tar.gz
Fixes : #705
2023-01-03 18:05:15 +00:00
Dexter Yan
be26462c19
feat(cluster_resources): increase default client burst and qps ( #920 )
...
* feat(collect): add client burst and qps
2022-12-22 09:49:42 +13:00
Diamon Wiggins
a4c4b24056
Deduplication for Cluster Resources Collector ( #832 )
...
* add dedup for cluster resources collector
* restructure both collect.go in both pkg/supportbundle and pkg/preflight to be more similar for eventual refactor
2022-12-07 15:10:31 -04:00
Evans Mungai
fbbcf87405
feat(collectors): Store all pod logs in cluster-resources directory ( #821 )
...
* feat(collectors): Store all pod logs in cluster-resources directory
All pod logs collected by the logs collector will now be stored in
/cluster-resources/pods/logs/[namespace]/[pod]/[container].log. This
will provide consistency and allow sbctl to find the logs when we run
`kubectl logs <pod>`. To allow backwards compatibility, symlinks of the
log files will be created in the current expected locations.
Closes : #744
2022-11-22 07:10:34 +13:00
Xav Paice
3513eeca19
Ensure clusterResources is added prior to other collectors ( #768 )
...
This change ensures that the clusterResources collector runs prior to any others
in order to not collect info on pods that collectors run during collection.
Additionally centralizes functions that are common to all collection to make future
maintenance simpler.
Fixes : #767
2022-11-01 12:16:01 +13:00
Diamon Wiggins
04c7a18da3
Fix Progress Callback for Support Bundle Collection ( #781 )
...
fix progress callback for support bundle and revert collector title changes
2022-10-14 12:29:59 -04:00
Diamon Wiggins
c7b84ad1e5
Refactor in-clusters collectors to use struct per collector ( #670 )
...
refactor in-clusters collectors to use struct per collector
2022-10-03 13:53:05 -04:00
Xav Paice
f06201e050
Small typo fix in collect.go
2022-08-02 14:36:34 +12:00
Diamon Wiggins
114749d5b7
always attempt to generate a support bundle when host collectors exists
2022-07-30 18:57:24 -04:00
Diamon Wiggins
cab3fc7f4e
Redact Host Collectors in Support Bundles ( #614 )
...
Add redactors for Host Collectors
2022-07-01 18:57:58 -04:00
diamonwiggins
17fe3db79f
adding host collectors to support bundles
2022-05-11 22:50:03 +00:00
Dimitri Koshkin and Salah Aldeen Al Saleh
111396eb39
fix: pass redact flag when running support-bundle ( #406 )
...
Co-authored-by: Salah Aldeen Al Saleh <sg.alsaleh@gmail.com >
2021-10-20 10:47:27 -07:00
divolgin
0e8bedc281
Save collector data to disk directly
2021-09-29 00:15:02 +00:00
Salah Aldeen Al Saleh
465a533640
store analysis in the support bundle ( #417 )
...
* store analysis in the support bundle
2021-09-10 11:58:16 -07:00
emosbaugh
8dcfa9886d
Copy from host collector ( #391 )
...
* Copy from host collector
* namespace improvements
* better support for multiple nodes
2021-07-22 12:25:59 -07:00
emosbaugh
39350b5722
ConfigMap collector and secrets can be collected by selectors ( #384 )
...
* ConfigMap collector and secrets can be collected by selectors
* follow docs
* Pass context and kubernetes client to collectors
* collect tests
* analyze tests
* fix tests
* improvements
2021-07-08 16:30:26 -07:00
Dan Stough
b8622a65ad
chore(support-bundle): refactor out support bundle pkg
2021-05-04 14:41:54 +00:00