Commit Graph

438 Commits

Author SHA1 Message Date
Matthias Radestock
a9e5b99a74 add benchmarks for rendering & summarising processes-by-name 2017-12-26 11:59:50 +00:00
Matthias Radestock
f4668ca692 add benchmarks for report summarization
This is the last phase before result marshalling in the
/api/topology/{name} handler.
2017-12-24 09:48:41 +00:00
Matthias Radestock
06e0bed015 refactor: extract helper for rendering a topology 2017-12-24 09:44:48 +00:00
Matthias Radestock
6784d95ac8 refactor: extract getting report for benchmarking 2017-12-24 09:18:20 +00:00
Matthias Radestock
d4d7764311 cosmetic(ish) 2017-12-24 09:13:22 +00:00
Matthias Radestock
bf14b3dafd upgrade reports we merge / render
This better reflects what the app is doing.
2017-12-24 08:55:45 +00:00
Matthias Radestock
615cb6af59 simplify error handling 2017-12-24 08:43:24 +00:00
Matthias Radestock
794c345aee cosmetic: more logical function order 2017-12-24 08:33:13 +00:00
Matthias Radestock
49e4c7b0b6 make report upgrade benchmark reflect new phasing
Since #2979 upgrading happens prior to merging.
2017-12-24 08:29:37 +00:00
Matthias Radestock
43fadceb85 cosmetic: better function grouping 2017-12-24 08:18:12 +00:00
Matthias Radestock
ad72a2cf2f refactor: rename topology rendering benchmarks 2017-12-24 08:16:57 +00:00
Matthias Radestock
70d84170d6 introduce a cheap /api/probes?sparse variant
In many cases we only need to know whether there are _any_ connected
probes, and not the probe details. Obtaining that info is cheaper
since it requires no reading or merging or reports.
2017-12-14 01:07:40 +00:00
Matthias Radestock
72b9e9c6b9 add Reporter.HasReports() for cheap report availability checking
This requires no report reading / merging.

We plan to expose this in the HTTP API, so the UI gets a cheap way of
checking whether the app is currently receiving data from probes.
2017-12-14 00:13:45 +00:00
Matthias Radestock
54fe1e37da cosmetic 2017-12-13 23:52:48 +00:00
Matthias Radestock
6c4bf58fe2 refactor: extract awsCollector.getReportKeys helper
This makes Report() more readable. We also intend to use this function
elsewhere.
2017-12-13 23:50:05 +00:00
Matthias Radestock
3a122aeed5 refactor: rename awsCollector.getReportKeys to reportKeysInRange
...so we can recycle the former name.
2017-12-13 23:40:04 +00:00
Roberto Bruggemann
10f3e4c0dc In-memory collector: merge reports after upgrade
Upgrading reports individually reduces the amount of upgrades, since some reports might have been upgraded in previously.
2017-12-12 14:16:48 +00:00
Roberto Bruggemann
e58f865d3f Upgrade reports before caching
This change will reduces upgrades, since `report.Upgrade` only performs an upgrade if necessary.
2017-12-12 11:25:04 +00:00
Matthias Radestock
ea20de58be Merge pull request #2964 from weaveworks/do-not-reportallocs
do not report allocations in benchmarks
2017-12-08 14:11:18 +00:00
Matthias Radestock
d528292cd4 make report upgrading fast when it's a no-op
The vast majority of the cost is memory allocation, so doing a first
pass to see whether any upgrading is necessary at all, and thus
avoiding allocation when it isn't, is a massive saving.
2017-12-07 21:01:34 +00:00
Matthias Radestock
d3e23e18fc do not start/stop benchmark timer for no-ops 2017-12-07 20:31:11 +00:00
Matthias Radestock
f305c35bfd do not report allocations in benchmarks
That's what -benchmem is for.
2017-12-07 20:27:31 +00:00
Matthias Radestock
acde0ea294 add some more topology rendering benchmarks 2017-12-02 13:04:57 +00:00
Matthias Radestock
bc0c8324be add report merging benchmark 2017-12-02 11:51:16 +00:00
Matthias Radestock
c953313b01 move main report processing benchmarks in one place
so they can share code and are easier to run in combination.

We take advantage of the code sharing by generalising the report
rendering benchmarks to read & merge reports from a dir.
2017-12-02 11:29:16 +00:00
Matthias Radestock
f2fed067d5 simplify render benchmark code
by using report.MakeFromFile instead of NewFileConnector
2017-11-29 11:37:39 +00:00
Matthias Radestock
9dca7627b6 filter unconnected nodes after applying user-specified filters
...rather than before. That way, nodes which become unconnected during
filtering are removed, which is what we want. ATM we are depending on
some 'unconnected' filtering inside every filter, which is expensive
and largely redundant. We should soon be able to remove that.

downside: 'unconnected' filtering is no longer memoised.
2017-11-28 06:54:15 +00:00
Matthias Radestock
2d964b669a refactor: separate filtering from rendering in topology description
This is a step towards filtering unconnected nodes after all custom
filters have been applied.
2017-11-28 06:54:15 +00:00
Matthias Radestock
7bbded9e84 refactor(ish): introduce Tranformers
so we can generalise the filter step in render.Render et al. That will
allow us to apply whole-topology filters in that step.
2017-11-28 06:54:15 +00:00
Matthias Radestock
8c4ae0577d Merge pull request #2947 from weaveworks/simplify-decoration
Decorators, begone!
2017-11-21 20:49:45 +00:00
Matthias Radestock
a12d707d9b refactor: rename decorateWithStats to computeStats
to de-decorate
2017-11-21 20:17:00 +00:00
Matthias Radestock
0c43526465 refactor: use new FilterFunc.Apply
instead of constructing temporary Filter renderers.

This also makes clearer what is going on.
2017-11-21 20:16:59 +00:00
Matthias Radestock
88e8b52d66 Decorators, begone!
Decorators were just a complicated way of constructing filters.
2017-11-21 20:16:30 +00:00
Matthias Radestock
ae153e57f5 refactor: remove Decorator from render.Renderer.Render() signature 2017-11-18 18:04:52 +00:00
Matthias Radestock
a82d245e93 simplify render decoration
Decoration is in fact quite a simple process that is applied on entry
to rendering: we take a base renderer, transform it with a decorator,
and then render a report with it. The new render.Decorate() function
does exactly that.

There is one exception. When rendering an individual node, e.g. for
showing its details panel in the UI, we must not lose the node during
decoration. That requires some special logic, which previously resided
in the PreciousNodeRenderer, and now lives in handleNode.
2017-11-18 18:04:52 +00:00
Matthias Radestock
4e4c3b25c4 Expand the app's k8s namespace calculation
to include recently added k8s types.

This is all rather inefficient. See #2945.
2017-11-18 17:35:57 +00:00
Matthias Radestock
8f7e00f46a Stats are easily produced as part of Rendering
...so there is no need for a separate Stats method.

step 1: return stats from Render
2017-11-08 07:15:28 +00:00
Bryan Boreham
8a11697957 Add BenchmarkTopologyHosts() and BenchmarkTopologyContainers()
To time a single topology, for more focused optimisation
2017-11-06 20:48:25 +00:00
Matthias Radestock
0b9bab255c memoise top-level rendering
This deals with browsers requesting the same rendering for the same
timespan when no new reports have been received for that timespan.
2017-11-04 22:24:46 +00:00
Damien Lespiau
466927ce68 lint: Fix 2 sites failing a recently introduced golint check
golint has gained a new check and we don't freeze the golint version so CI was
failing on unrelated PRs:

app/multitenant/consul_client.go:69:2: redundant if ...; err != nil check, just return error instead.
report/marshal.go:188:2: redundant if ...; err != nil check, just return error instead.

Fix those!
2017-09-26 12:15:09 +01:00
Jonathan Lange
f9e37a21f5 Continue processing reports if billing fails 2017-09-22 12:20:10 +01:00
Roland Schilter
0d381a34d6 Link scope-ui graphs clickable to prometheus queries (#2664)
scope-app:
* Adds `-app.metrics-graph` cli flag for configuring the base url to
use for graph links; supports :orgID and :query placeholders
* Assigns query URLs to existing metrics and appends empty metrics if missing

scope-ui:
* Extends <CloudFeature /> with option alwaysShow
* Adds <CloudLink /> to simplify routing when in cloud vs not in cloud
* Links metric graphs in the ui's node details view for all k8s
toplogies and containers so far
* Tracks metric graph click in mixpanel `scope.node.metric.click`
* Uses percentages and MB for CPU/Memory urls
* Passes timetravel timestamp to cortex in deeplink
2017-08-15 18:56:23 +01:00
Matthias Radestock
c312579bfe log sqs messages at 'debug' rather than 'info' level
Fixes #2790.
2017-08-07 10:17:05 +01:00
Matthias Radestock
1f95e3efd4 obtain historic_reports capability from Reporter 2017-07-31 09:58:31 +01:00
Alfonso Acosta
7b4b410f4a Review Feedback 2017-07-20 11:01:44 +00:00
Alfonso Acosta
c6d5a6a646 Make linter happy 2017-07-20 10:05:48 +00:00
Alfonso Acosta
f66d79d427 Backoff when writing to Dynamo and S3 2017-07-18 15:28:08 +00:00
Alfonso Acosta
34bfc22b4f Fix tests 2017-07-03 20:20:28 +00:00
Mike Lang
c521b50bc6 Remove type filter in controllers view
This was always temporary until we could search for type via the search box instead,
since it resulted in too much clutter / didn't seem a useful use-case.

As it turns out, searching like this was already possible.
2017-06-29 16:03:53 -07:00
Mike Lang
ef2b6f6c55 Remove replica sets
Use a special kind of selector renderer to elide replica sets from pod nodes
and directly reference deployment parents instead.
Do the inverse (replace replica sets with pods) during the mapping from pod to deployment.

Note we can no longer use renderParents since we're using a non-standard Selector
2017-06-29 10:13:15 -07:00