Benjamin Yang and GitHub
c67775c1a0
feat: clean tokenization system implementation ( #1874 )
...
Core tokenization functionality with minimal file changes:
✅ Core Features:
- Intelligent tokenization engine (tokenizer.go)
- Context-aware secret classification (PASSWORD, APIKEY, DATABASE, etc.)
- Cross-file correlation with deterministic HMAC-SHA256 tokens
- Optional encrypted mapping for token→original value resolution
✅ Integration:
- CLI flags: --tokenize, --redaction-map, --encrypt-redaction-map
- Updated all redactor types: literal, single-line, multi-line, YAML
- Support bundle integration with auto-upload compatibility
- Backward compatibility: preserves ***HIDDEN*** when disabled
✅ Production Ready:
- Only 11 essential files (vs 31 in original PR)
- No excessive test files or documentation
- Clean build, all functionality verified
- Maintains existing redaction behavior by default
Token format: ***TOKEN_<TYPE>_<HASH>*** (e.g., ***TOKEN_PASSWORD_A1B2C3***)
2025-09-30 15:04:23 -05:00
Ethan Mosbaugh and GitHub
b80f38a9a0
fix(redact): multi-line redactors strip empty lines ( #1742 )
2025-02-20 21:55:05 -05:00
Xav Paice and GitHub
86b7e54466
Revert "feat: save YAML spec used to generate support bundle/preflight" ( #1715 )
...
Revert "feat: save YAML spec used to generate support bundle/preflight (#1713 )"
This reverts commit f6f51acbd5 .
2025-01-06 09:42:58 +11:00
Gerard Nguyen and GitHub
f6f51acbd5
feat: save YAML spec used to generate support bundle/preflight ( #1713 )
...
* save YAML spec of support bundle
* save YAML spec of preflight
* add unit test
* redact TLS private key by default in output spec
* update YAML path for HTTP TLS redactor
2025-01-04 11:35:43 +11:00
Evans Mungai and GitHub
e2adfa3774
fix: Correct selector used to find installer specs when redacting ( #1456 )
2024-02-07 10:36:15 +00:00
Evans Mungai and GitHub
86279b4ec4
chore(redactors): memory consumption improvements ( #1332 )
...
* Document additional go tool profiling flags
* Add a regex cache to avoid compiling regular expressions all the time
* Reduce max buffer capacity
* Prefer bytes to strings
Strings are immutable and hence we need to create a new one
all the time when operation on them
* Some more changes
* More bytes
* Use writer.Write instead of fmt.FPrintf
* Clear regex cache when resetting redactors
* Logs errors when redactors error since they get swallowed
* Add an improvement comment
* Limit the number of goroutines spawned when redacting
* Minor improvement
* Write byte slices one at a time instead of concatenating them first
* Add a test for writeBytes
* Additional tests
2023-09-15 13:09:21 -04:00
Dexter Yan and GitHub
d730be0fae
feat(redact): use a scan regex for default redact rule of lines to improve cpu usage and reduce time cost ( #1291 )
2023-08-25 15:11:04 +12:00
Evans Mungai and GitHub
546ffde14b
feat: use klog as the default logging library ( #1008 )
2023-02-24 18:24:51 +00:00
Diamon Wiggins and GitHub
2fcdc77cd3
Standardize Cluster Resources Collector File Paths ( #971 )
...
* using const for cluster resources k8s objects to standardize directories and files
2023-01-25 13:34:15 -05:00
Xav Paice and GitHub
c85bf9a9a6
BREAKING: remove IP address redaction ( #734 )
...
This change removes the IPv4 address redaction which previously ran by default on all
support bundle collections.
Folks that want to redact IPv4 addresses will need to add that redactor manually to their redactor specs.
2022-11-30 08:42:42 +13:00
Xav Paice
4acd7e4951
Add Ceph auth redactors
...
Add automatic redactors for Ceph auth_dump json which includes auth keys.
2022-06-27 17:50:38 +12:00
Salah Aldeen Al Saleh and GitHub
c7c21e88fb
fix custom resources redaction file path ( #480 )
...
* fix custom resources redaction file path
2021-11-02 12:36:15 -07:00
cf03503216
feat: Collect custom resources ( #447 )
...
* feat: Collect custom resources
Co-authored-by: Martin Hrabovcin<mhrabovcin@users.noreply.github.com >
Co-authored-by: Andrew Reed <andrew@replicated.com >
2021-10-21 16:49:59 -05:00
divolgin
0e8bedc281
Save collector data to disk directly
2021-09-29 00:15:02 +00:00
divolgin
53e8ee5495
Add a default redactor for postgres db connection string
2021-06-29 22:38:32 +00:00
Marc Campbell
c7fdec0291
Removing Scopeagent
2021-01-28 18:22:48 +00:00
divolgin
a0ce85ae1e
Adding troubleshoot.sh/v1beta2
2020-09-01 19:57:11 +00:00
Andrew Lavery
6e874483b6
update yaml key name, fixup example troubleshoot
2020-06-15 13:55:07 -04:00
Andrew Lavery
e66d12dab2
combine multiline and single line regex
2020-06-15 11:15:50 -04:00
Andrew Lavery
a561254756
break apart redactor type
2020-06-09 18:43:44 -04:00
Marc Campbell
661d7fcfc4
Update names
2020-05-27 13:10:46 -07:00
Andrew Lavery
b9c3ad2635
better default redactor names
2020-05-27 14:36:40 -04:00
Andrew Lavery
f03916cfd2
include whether a redaction is from a default redactor in reports
2020-05-27 13:46:39 -04:00
Andrew Lavery
553718eada
fully spell out default redactor names
2020-05-26 19:33:40 -04:00
Andrew Lavery
171c440f27
improve generated names for custom redactors
2020-05-26 18:13:48 -04:00
Andrew Lavery
869a9eafd3
actually increment withinRedactNum
2020-05-26 18:07:23 -04:00
Andrew Lavery
6c61040f74
default redactor name improvements
...
for instance, defaultRegex.SECRET_ACCESS_KEY instead of the raw regex
2020-05-26 17:15:30 -04:00
Andrew Lavery
0d84c62c78
add yaml/json annotations for redaction reports
2020-05-20 14:58:37 -04:00
Andrew Lavery
5a2c153026
add tests for redaction reports
2020-05-20 14:36:07 -04:00
Andrew Lavery
fb2f028fb5
add methods to get and clear redactions
2020-05-20 10:28:46 -04:00
Andrew Lavery
d37ace6e51
better naming scheme that does not leak secrets
2020-05-19 17:21:55 -04:00
Andrew Lavery
b6e92e1bdf
record yaml redactions
2020-05-14 18:49:30 -04:00
Andrew Lavery
27369acbf4
plumbing for literal/regex/multiline reports
2020-05-14 16:00:58 -04:00
Andrew Lavery
15b100e0ee
support * when redacting yaml maps
2020-04-27 22:01:04 -04:00
Andrew Lavery
56150c3345
add support for redacting items within yaml documents
2020-04-27 20:46:52 -04:00
Andrew Lavery
8f594e8764
add support for custom multiline redactors, add yaml redactors to spec
2020-04-27 16:50:25 -04:00
Andrew Lavery
37f75c1a3f
use gobwas/glob, not filepath, file matching
2020-04-23 13:13:46 -04:00
Andrew Lavery
d016e3269c
add global and per-collector redactors
...
add redact type, and begin wiring global redactors
use per-collector redactors
add a test of the 'data' collector and redaction
handle literal string replacements
remove redundant types and redact calls
add proper redactor type, foundations of global redactors
accept global redactors from the CLI, include sample redaction spec
2020-04-16 14:03:00 -04:00
Frank Shotwell
f8870e5fc0
move scope instrumentation to handle table-driven tests
2020-01-31 06:25:07 +00:00
Frank Shotwell
10b2941c57
Add scope test instrumentation
2020-01-30 21:01:06 +00:00
divolgin
9942f4bf65
Adding basic redaction functionality
2019-07-17 22:57:19 +00:00