Commit Graph

35 Commits

Author SHA1 Message Date
Bryan Boreham
ae98ab26c1 Add dump() for report stats 2018-10-11 15:23:58 +00:00
Bryan Boreham
56c6685e73 Drop topologies which are way too big 2018-10-11 15:23:58 +00:00
Bryan Boreham
8693f0ab08 Add tracing spans to app
Also log things like number of nodes in Map, total number of reports.
2018-10-11 15:22:38 +00:00
Marc Carré
2ba50b8b3d Update golang.org/x/net/context to latest
```
$ gvt delete golang.org/x/net/context
$ gvt fetch golang.org/x/net/context
2018/07/23 18:03:49 Fetching: golang.org/x/net/context
$ git grep -l "golang.org/x/net/context" | grep -v vendor | xargs sed -i '' 's:golang.org/x/net/context:context:g'
$ git grep -l "context/ctxhttp" | grep -v vendor | xargs sed -i '' 's:context/ctxhttp:golang.org/x/net/context/ctxhttp:g'
$ gofmt -s -w app
$ gofmt -s -w common
$ gofmt -s -w probe
$ gofmt -s -w prog
$ gofmt -s -w tools
```
fixed a bunch of:
```
cannot use func literal (type func("github.com/weaveworks/scope/vendor/golang.org/x/net/context".Context) error) as type func("context".Context) error
```
2018-07-23 20:10:18 +02:00
Marc Carré
d46c2266ce Change Sirupsen/logrus to sirupsen/logrus
```
$ git grep -l Sirupsen | grep -v vendor | xargs sed -i '' 's:github.com/Sirupsen/logrus:github.com/sirupsen/logrus:g'
$ gofmt -s -w app
$ gofmt -s -w common
$ gofmt -s -w probe
$ gofmt -s -w prog
$ gofmt -s -w tools
```
2018-07-23 20:10:14 +02:00
Bryan Boreham
126a171f62 Make 'fast' merger the default 2018-06-22 11:59:43 +00:00
Matthias Radestock
72b9e9c6b9 add Reporter.HasReports() for cheap report availability checking
This requires no report reading / merging.

We plan to expose this in the HTTP API, so the UI gets a cheap way of
checking whether the app is currently receiving data from probes.
2017-12-14 00:13:45 +00:00
Matthias Radestock
54fe1e37da cosmetic 2017-12-13 23:52:48 +00:00
Matthias Radestock
6c4bf58fe2 refactor: extract awsCollector.getReportKeys helper
This makes Report() more readable. We also intend to use this function
elsewhere.
2017-12-13 23:50:05 +00:00
Matthias Radestock
3a122aeed5 refactor: rename awsCollector.getReportKeys to reportKeysInRange
...so we can recycle the former name.
2017-12-13 23:40:04 +00:00
Roberto Bruggemann
e58f865d3f Upgrade reports before caching
This change will reduces upgrades, since `report.Upgrade` only performs an upgrade if necessary.
2017-12-12 11:25:04 +00:00
Matthias Radestock
1f95e3efd4 obtain historic_reports capability from Reporter 2017-07-31 09:58:31 +01:00
Alfonso Acosta
7b4b410f4a Review Feedback 2017-07-20 11:01:44 +00:00
Alfonso Acosta
c6d5a6a646 Make linter happy 2017-07-20 10:05:48 +00:00
Alfonso Acosta
f66d79d427 Backoff when writing to Dynamo and S3 2017-07-18 15:28:08 +00:00
Filip Barl
b6dfe25499 Time travel control (#2524)
* Hacky working prototype.

* Operate with time.Duration offset instead of fixed timestamp.

* Polished the backend code.

* Made a nicer UI component.

* Small refactorings of the websockets code.

* Fixed the backend tests.

* Better websocketing and smoother transitions

* Small styling refactoring.

* Detecting empty topologies.

* Improved error messaging.

* Addressed some of David's comments.

* Moved nodesDeltaBuffer to a global state to fix the paused status rendering bug.

* Small styling changes

* Changed the websocket global state variables a bit.

* Polishing & refactoring.

* More polishing.

* Final refactoring.

* Addressed a couple of bugs.

* Hidden the timeline control behind Cloud context and a feature flag.

* Addressed most of @davkal's comments.

* Added mixpanel tracking.
2017-06-12 11:22:17 +02:00
Adam Harrison
83c073fad0 Correct dynamoDB metric label
Discovered whilst attempting to resolve github.com/weaveworks/service/issues/1157
2017-03-17 12:09:40 +00:00
Jonathan Lange
e8085b01b6 Use 'common' library 2016-12-07 11:22:38 +00:00
Tom Wilkie
21ee72a384 Add OpenTracing support to TimeRequestHistogram (#2023)
* Vendor open tracing

* Extend TimeRequestHistorgram to also emit opentracing spans.

* Update some of the labels
2016-11-22 14:43:15 +00:00
Tom Wilkie
32edfc9112 Don't reencode reports in the collector (#1819)
* Don't reencode reports in the collector

* Review feedback

* Fix comment

* Update alpine URLs so it will build

* Fix tests
2016-08-22 17:37:41 +01:00
Matthias Radestock
6ba812e82f restore compatibility of old probes with new m/t app
This got broken in #1682
2016-08-16 23:53:07 +01:00
Matthias Radestock
e6a474ead7 fix report store query chaining
We fell victim to variable shadowing here. Each store would be fed the
original list of report keys, instead of only the ones that weren't
found in the previous store. So if a single report was missing from the
in-process cache, we would then fetch all reports from memcache. And if
that in turn was missing a single report we would fetch all reports from
S3.

We chain report stores for a reason - to reduce latency and, in case of
the in-process cache, eliminate decoding costs. So this bug has a huge
impact on query service performance.

To make matters worse, we return *all* the reports - now possibly in
triplicate. Fortunately, the SmartMerger filters these out, so at least
we were not incurring extra merge costs.
2016-08-16 00:55:39 +01:00
Jonathan Lange
1fd8a5fb88 Use StoreReport in main AWS routine 2016-07-15 11:24:37 +01:00
Jonathan Lange
46dfeb627d Call it reportKey 2016-07-15 11:24:37 +01:00
Jonathan Lange
0058229687 Extract functions for calculating keys
Not so much for re-use as to help jml understand what's going on
2016-07-15 11:24:36 +01:00
Jonathan Lange
2bfd6d7eb7 Parametrize compression level 2016-07-15 11:24:36 +01:00
Jonathan Lange
d83d7318d0 Remove the old metric 2016-07-12 18:16:35 +01:00
Jonathan Lange
d2298aa8f3 Store a histogram of report sizes 2016-07-12 16:37:29 +01:00
Tom Wilkie
3173f6ad75 Use histograms over summaries 2016-07-12 11:15:57 +01:00
Paul Bellamy
8cb1ecdf2c Merge pull request #1642 from weaveworks/refactoring-timing
refactor some timing helpers into a common lib
2016-07-05 13:01:28 +01:00
Paul Bellamy
7736564337 refactor some timing helpers into a common lib 2016-07-05 12:29:00 +01:00
Jonathan Lange
31c88fd62b Instrumentation that we might like to keep 2016-07-04 16:03:50 +01:00
Jonathan Lange
7dd2c6371e Parametrize window rather than assuming default 2016-07-04 13:50:54 +01:00
Jonathan Lange
9e0f0c51b9 Configuration type for AWS collector 2016-06-30 17:01:58 +01:00
Jonathan Lange
baacaa8cc5 Rename dynamoCollector to awsCollector 2016-06-30 16:44:43 +01:00