Commit Graph

5251 Commits

Author SHA1 Message Date
Bryan Boreham
f60c2380fb Update job definition 2019-07-02 08:39:38 +00:00
Bryan Boreham
a9e9e476b4 Don't tag for quay.io 2019-07-02 08:39:28 +00:00
Bryan Boreham
8b31c7e0fc Add 'keep' option to delete everything but some orgs 2019-06-13 17:48:44 +00:00
Bryan Boreham
1c11aefe03 Read compressed records file 2019-06-10 07:19:58 +00:00
Bryan Boreham
3456c844ec Don't run delete batches in parallel
Now we are handling multiple org-hour records in parallel, it's best
no to have additional parallelism at the delete level where they may
collide on hash key.
2019-06-09 09:27:04 +00:00
Bryan Boreham
04668590f6 Metric for DynamoDB errors 2019-06-09 09:26:20 +00:00
Bryan Boreham
89121cb509 Wait using the correct batch size
Otherwise waits tend to bust the rate-limiter batch size and it
doesn't wait at all.
2019-06-08 14:22:46 +00:00
Bryan Boreham
0368edf612 Metric for records done 2019-06-08 13:50:58 +00:00
Bryan Boreham
f9f28f6d78 Do multiple org-hour records in parallel 2019-06-08 13:50:03 +00:00
Bryan Boreham
dd71191bb6 Check earlier for blank records file 2019-06-08 13:10:19 +00:00
Bryan Boreham
b729c5c3bb Reduce rate limit defaults 2019-06-08 13:07:51 +00:00
Bryan Boreham
424a18cf2b Fix batching again 2019-06-08 13:07:33 +00:00
Bryan Boreham
e46514c24a Advance s3_items_deleted counter more steadily 2019-06-08 11:25:51 +00:00
Bryan Boreham
15546dc690 If no delete orgs file supplied, delete all records 2019-06-08 11:24:35 +00:00
Bryan Boreham
654c26442c Spacing of debug message 2019-06-08 11:24:11 +00:00
Bryan Boreham
ae9441bb53 Fix batching to dynamodb 2019-06-08 11:23:48 +00:00
Bryan Boreham
2d588a57a7 Implement pagination on DynamoDB queries 2019-06-06 17:08:22 +00:00
Bryan Boreham
4ce900e466 Fix delete record parsing 2019-06-06 16:48:07 +00:00
Bryan Boreham
e60e25da4a Remove duplicate 'segments' flag 2019-06-06 16:41:45 +00:00
Bryan Boreham
cfe7ca7cea Read a file of org-hour records 2019-06-04 17:07:58 +00:00
Bryan Boreham
18cd43d2bf Refactor: delete from DynamoDB in same goroutine as S3 2019-06-04 16:07:31 +00:00
Bryan Boreham
bc59b03f67 Fix errorf format bug 2019-06-04 15:14:10 +00:00
Bryan Boreham
cdf28708e8 Refactor: extract deleteOneOrgHour() method 2019-06-04 15:13:48 +00:00
Bryan Boreham
388d273e5f Add 'just scan' option 2019-04-09 09:46:19 +00:00
Bryan Boreham
b23f7a7b0d Default stop hour to now 2018-11-20 11:56:50 +00:00
Bryan Boreham
2fe3f46cf7 Handle retries ourselves 2018-11-20 07:53:20 +00:00
Bryan Boreham
a81bd74a4a Use rate-limiters on DynamoDB operations 2018-11-18 22:53:28 +00:00
Bryan Boreham
1ce9b1e53c Refactor to make bigger S3 writes 2018-11-17 13:47:35 +00:00
Bryan Boreham
c6f16436a9 Sample Kubernetes job 2018-11-15 18:19:17 +00:00
Bryan Boreham
ab796c7c6f Fix up logging 2018-11-15 18:18:07 +00:00
Bryan Boreham
7143356fd5 Move to extras directory, add Makefile, Dockerfile 2018-11-15 18:17:27 +00:00
Bryan Boreham
2ffb4bc32b Refactor; delete from DynamoDB too 2018-11-15 16:48:51 +00:00
Bryan Boreham
1bc5a23922 WIP program to delete Scope data 2018-11-15 14:22:23 +00:00
Bryan Boreham
e49c2dcb84 Update aws-sdk-go library to latest release
From version 1.0.5 to version 1.15.74.
Simply because the old version was really old.
2018-11-13 15:08:30 +00:00
Simon
730d5e3b62 Merge pull request #3425 from weaveworks/revert-3422-upgrade-font-awesome-5
Revert "Upgrade to Font Awesome 5"
2018-11-12 11:36:24 +01:00
Simon
e78f8a8de4 Revert "Upgrade to Font Awesome 5" 2018-11-12 11:22:19 +01:00
Bryan Boreham
73de9c3f03 Merge pull request #3422 from weaveworks/upgrade-font-awesome-5
Upgrade to Font Awesome 5
2018-11-12 10:02:47 +00:00
Bryan Boreham
6356f5c118 Merge pull request #3392 from gotjosh/add-tests-to-probe-cri
Fix intention of support for fallback protocol and add tests for probe/cri/registry
2018-11-12 10:01:30 +00:00
Bryan Boreham
fb0359ccd9 Merge pull request #3424 from ScottBrenner/patch-1
Update introducing.md
2018-11-12 10:00:11 +00:00
Scott Brenner
5d71a9ff5b Update introducing.md
Grammar fix.
2018-11-09 16:15:48 -08:00
Scott Brenner
a9e2d37691 Update introducing.md
Typo fix.
2018-11-09 16:09:49 -08:00
Simon Howe
2cc58f5b00 Switch to fa5 icons and tweak font-sizes a bit 2018-11-09 14:59:53 +01:00
Bryan Boreham
aac7356e31 Merge pull request #3421 from weaveworks/eks-pause
Add EKS to paused container detection
2018-11-09 12:48:42 +00:00
Simon Howe
52fc460f8e Load up fa5 instead of 4 2018-11-09 13:41:37 +01:00
Stefan Prodan
717cb13a84 Add EKS to paused container detection
fix #3259
2018-11-09 13:48:11 +02:00
Bryan Boreham
62d55596c6 Merge pull request #3393 from ycao56/basic-auth
Add http Basic Auth
2018-11-07 14:32:03 +00:00
Bryan Boreham
a5d29a9a58 Merge pull request #3411 from gruebel/master
Clean up Dockerfiles
2018-11-06 08:44:03 +00:00
Bryan Boreham
5798d049df Merge pull request #3419 from weaveworks/disable-reporting-k8s-cluster
Disable host and endpoint reporting in Kubernetes cluster mode
2018-11-06 08:39:49 +00:00
Bryan Boreham
71972c302a Disable host and endpoint reporting in Kubernetes cluster mode
When the probe is run with --probe.kubernetes.role=cluster, disable
host reporting and endpoint reporting since it duplicates what is
being done by the per-host probe.
2018-11-05 21:59:29 +00:00
Yu Cao
308f923e48 use github.com/goji/httpauth instead 2018-11-01 17:51:24 -04:00