github/weave-scope
1
0
Fork 0
mirror of https://github.com/weaveworks/scope.git synced 2026-09-06 10:17:19 +00:00
Code Issues Packages Projects Releases Wiki Activity
5,251 Commits 85 Branches 64 Tags
scan-data
Commit Graph
31 Commits
Author SHA1 Message Date
Bryan Boreham f60c2380fb Update job definition 2019-07-02 08:39:38 +00:00
Bryan Boreham a9e9e476b4 Don't tag for quay.io 2019-07-02 08:39:28 +00:00
Bryan Boreham 8b31c7e0fc Add 'keep' option to delete everything but some orgs 2019-06-13 17:48:44 +00:00
Bryan Boreham 1c11aefe03 Read compressed records file 2019-06-10 07:19:58 +00:00
Bryan Boreham 3456c844ec Don't run delete batches in parallel 
Now we are handling multiple org-hour records in parallel, it's best
no to have additional parallelism at the delete level where they may
collide on hash key.
 2019-06-09 09:27:04 +00:00
Bryan Boreham 04668590f6 Metric for DynamoDB errors 2019-06-09 09:26:20 +00:00
Bryan Boreham 89121cb509 Wait using the correct batch size 
Otherwise waits tend to bust the rate-limiter batch size and it
doesn't wait at all.
 2019-06-08 14:22:46 +00:00
Bryan Boreham 0368edf612 Metric for records done 2019-06-08 13:50:58 +00:00
Bryan Boreham f9f28f6d78 Do multiple org-hour records in parallel 2019-06-08 13:50:03 +00:00
Bryan Boreham dd71191bb6 Check earlier for blank records file 2019-06-08 13:10:19 +00:00
Bryan Boreham b729c5c3bb Reduce rate limit defaults 2019-06-08 13:07:51 +00:00
Bryan Boreham 424a18cf2b Fix batching again 2019-06-08 13:07:33 +00:00
Bryan Boreham e46514c24a Advance s3_items_deleted counter more steadily 2019-06-08 11:25:51 +00:00
Bryan Boreham 15546dc690 If no delete orgs file supplied, delete all records 2019-06-08 11:24:35 +00:00
Bryan Boreham 654c26442c Spacing of debug message 2019-06-08 11:24:11 +00:00
Bryan Boreham ae9441bb53 Fix batching to dynamodb 2019-06-08 11:23:48 +00:00
Bryan Boreham 2d588a57a7 Implement pagination on DynamoDB queries 2019-06-06 17:08:22 +00:00
Bryan Boreham 4ce900e466 Fix delete record parsing 2019-06-06 16:48:07 +00:00
Bryan Boreham e60e25da4a Remove duplicate 'segments' flag 2019-06-06 16:41:45 +00:00
Bryan Boreham cfe7ca7cea Read a file of org-hour records 2019-06-04 17:07:58 +00:00
Bryan Boreham 18cd43d2bf Refactor: delete from DynamoDB in same goroutine as S3 2019-06-04 16:07:31 +00:00
Bryan Boreham bc59b03f67 Fix errorf format bug 2019-06-04 15:14:10 +00:00
Bryan Boreham cdf28708e8 Refactor: extract deleteOneOrgHour() method 2019-06-04 15:13:48 +00:00
Bryan Boreham 388d273e5f Add 'just scan' option 2019-04-09 09:46:19 +00:00
Bryan Boreham b23f7a7b0d Default stop hour to now 2018-11-20 11:56:50 +00:00
Bryan Boreham 2fe3f46cf7 Handle retries ourselves 2018-11-20 07:53:20 +00:00
Bryan Boreham a81bd74a4a Use rate-limiters on DynamoDB operations 2018-11-18 22:53:28 +00:00
Bryan Boreham 1ce9b1e53c Refactor to make bigger S3 writes 2018-11-17 13:47:35 +00:00
Bryan Boreham c6f16436a9 Sample Kubernetes job 2018-11-15 18:19:17 +00:00
Bryan Boreham ab796c7c6f Fix up logging 2018-11-15 18:18:07 +00:00
Bryan Boreham 7143356fd5 Move to extras directory, add Makefile, Dockerfile 2018-11-15 18:17:27 +00:00