github/weave-scope
1
0
Fork 0
mirror of https://github.com/weaveworks/scope.git synced 2026-07-16 12:00:11 +00:00
Code Issues Packages Projects Releases Wiki Activity
5,251 Commits 85 Branches 64 Tags
scan-data
Commit Graph

31 Commits

Author SHA1 Message Date
Bryan Boreham
f60c2380fb Update job definition 2019-07-02 08:39:38 +00:00
Bryan Boreham
a9e9e476b4 Don't tag for quay.io 2019-07-02 08:39:28 +00:00
Bryan Boreham
8b31c7e0fc Add 'keep' option to delete everything but some orgs 2019-06-13 17:48:44 +00:00
Bryan Boreham
1c11aefe03 Read compressed records file 2019-06-10 07:19:58 +00:00
Bryan Boreham
3456c844ec Don't run delete batches in parallel 
Now we are handling multiple org-hour records in parallel, it's best
no to have additional parallelism at the delete level where they may
collide on hash key.
 2019-06-09 09:27:04 +00:00
Bryan Boreham
04668590f6 Metric for DynamoDB errors 2019-06-09 09:26:20 +00:00
Bryan Boreham
89121cb509 Wait using the correct batch size 
Otherwise waits tend to bust the rate-limiter batch size and it
doesn't wait at all.
 2019-06-08 14:22:46 +00:00
Bryan Boreham
0368edf612 Metric for records done 2019-06-08 13:50:58 +00:00
Bryan Boreham
f9f28f6d78 Do multiple org-hour records in parallel 2019-06-08 13:50:03 +00:00
Bryan Boreham
dd71191bb6 Check earlier for blank records file 2019-06-08 13:10:19 +00:00
Bryan Boreham
b729c5c3bb Reduce rate limit defaults 2019-06-08 13:07:51 +00:00
Bryan Boreham
424a18cf2b Fix batching again 2019-06-08 13:07:33 +00:00
Bryan Boreham
e46514c24a Advance s3_items_deleted counter more steadily 2019-06-08 11:25:51 +00:00
Bryan Boreham
15546dc690 If no delete orgs file supplied, delete all records 2019-06-08 11:24:35 +00:00
Bryan Boreham
654c26442c Spacing of debug message 2019-06-08 11:24:11 +00:00
Bryan Boreham
ae9441bb53 Fix batching to dynamodb 2019-06-08 11:23:48 +00:00
Bryan Boreham
2d588a57a7 Implement pagination on DynamoDB queries 2019-06-06 17:08:22 +00:00
Bryan Boreham
4ce900e466 Fix delete record parsing 2019-06-06 16:48:07 +00:00
Bryan Boreham
e60e25da4a Remove duplicate 'segments' flag 2019-06-06 16:41:45 +00:00
Bryan Boreham
cfe7ca7cea Read a file of org-hour records 2019-06-04 17:07:58 +00:00
Bryan Boreham
18cd43d2bf Refactor: delete from DynamoDB in same goroutine as S3 2019-06-04 16:07:31 +00:00
Bryan Boreham
bc59b03f67 Fix errorf format bug 2019-06-04 15:14:10 +00:00
Bryan Boreham
cdf28708e8 Refactor: extract deleteOneOrgHour() method 2019-06-04 15:13:48 +00:00
Bryan Boreham
388d273e5f Add 'just scan' option 2019-04-09 09:46:19 +00:00
Bryan Boreham
b23f7a7b0d Default stop hour to now 2018-11-20 11:56:50 +00:00
Bryan Boreham
2fe3f46cf7 Handle retries ourselves 2018-11-20 07:53:20 +00:00
Bryan Boreham
a81bd74a4a Use rate-limiters on DynamoDB operations 2018-11-18 22:53:28 +00:00
Bryan Boreham
1ce9b1e53c Refactor to make bigger S3 writes 2018-11-17 13:47:35 +00:00
Bryan Boreham
c6f16436a9 Sample Kubernetes job 2018-11-15 18:19:17 +00:00
Bryan Boreham
ab796c7c6f Fix up logging 2018-11-15 18:18:07 +00:00
Bryan Boreham
7143356fd5 Move to extras directory, add Makefile, Dockerfile 2018-11-15 18:17:27 +00:00