Commit Graph

427 Commits

Author SHA1 Message Date
Matthias Radestock
70d84170d6 introduce a cheap /api/probes?sparse variant
In many cases we only need to know whether there are _any_ connected
probes, and not the probe details. Obtaining that info is cheaper
since it requires no reading or merging or reports.
2017-12-14 01:07:40 +00:00
Matthias Radestock
72b9e9c6b9 add Reporter.HasReports() for cheap report availability checking
This requires no report reading / merging.

We plan to expose this in the HTTP API, so the UI gets a cheap way of
checking whether the app is currently receiving data from probes.
2017-12-14 00:13:45 +00:00
Matthias Radestock
54fe1e37da cosmetic 2017-12-13 23:52:48 +00:00
Matthias Radestock
6c4bf58fe2 refactor: extract awsCollector.getReportKeys helper
This makes Report() more readable. We also intend to use this function
elsewhere.
2017-12-13 23:50:05 +00:00
Matthias Radestock
3a122aeed5 refactor: rename awsCollector.getReportKeys to reportKeysInRange
...so we can recycle the former name.
2017-12-13 23:40:04 +00:00
Roberto Bruggemann
10f3e4c0dc In-memory collector: merge reports after upgrade
Upgrading reports individually reduces the amount of upgrades, since some reports might have been upgraded in previously.
2017-12-12 14:16:48 +00:00
Roberto Bruggemann
e58f865d3f Upgrade reports before caching
This change will reduces upgrades, since `report.Upgrade` only performs an upgrade if necessary.
2017-12-12 11:25:04 +00:00
Matthias Radestock
ea20de58be Merge pull request #2964 from weaveworks/do-not-reportallocs
do not report allocations in benchmarks
2017-12-08 14:11:18 +00:00
Matthias Radestock
d528292cd4 make report upgrading fast when it's a no-op
The vast majority of the cost is memory allocation, so doing a first
pass to see whether any upgrading is necessary at all, and thus
avoiding allocation when it isn't, is a massive saving.
2017-12-07 21:01:34 +00:00
Matthias Radestock
d3e23e18fc do not start/stop benchmark timer for no-ops 2017-12-07 20:31:11 +00:00
Matthias Radestock
f305c35bfd do not report allocations in benchmarks
That's what -benchmem is for.
2017-12-07 20:27:31 +00:00
Matthias Radestock
acde0ea294 add some more topology rendering benchmarks 2017-12-02 13:04:57 +00:00
Matthias Radestock
bc0c8324be add report merging benchmark 2017-12-02 11:51:16 +00:00
Matthias Radestock
c953313b01 move main report processing benchmarks in one place
so they can share code and are easier to run in combination.

We take advantage of the code sharing by generalising the report
rendering benchmarks to read & merge reports from a dir.
2017-12-02 11:29:16 +00:00
Matthias Radestock
f2fed067d5 simplify render benchmark code
by using report.MakeFromFile instead of NewFileConnector
2017-11-29 11:37:39 +00:00
Matthias Radestock
9dca7627b6 filter unconnected nodes after applying user-specified filters
...rather than before. That way, nodes which become unconnected during
filtering are removed, which is what we want. ATM we are depending on
some 'unconnected' filtering inside every filter, which is expensive
and largely redundant. We should soon be able to remove that.

downside: 'unconnected' filtering is no longer memoised.
2017-11-28 06:54:15 +00:00
Matthias Radestock
2d964b669a refactor: separate filtering from rendering in topology description
This is a step towards filtering unconnected nodes after all custom
filters have been applied.
2017-11-28 06:54:15 +00:00
Matthias Radestock
7bbded9e84 refactor(ish): introduce Tranformers
so we can generalise the filter step in render.Render et al. That will
allow us to apply whole-topology filters in that step.
2017-11-28 06:54:15 +00:00
Matthias Radestock
8c4ae0577d Merge pull request #2947 from weaveworks/simplify-decoration
Decorators, begone!
2017-11-21 20:49:45 +00:00
Matthias Radestock
a12d707d9b refactor: rename decorateWithStats to computeStats
to de-decorate
2017-11-21 20:17:00 +00:00
Matthias Radestock
0c43526465 refactor: use new FilterFunc.Apply
instead of constructing temporary Filter renderers.

This also makes clearer what is going on.
2017-11-21 20:16:59 +00:00
Matthias Radestock
88e8b52d66 Decorators, begone!
Decorators were just a complicated way of constructing filters.
2017-11-21 20:16:30 +00:00
Matthias Radestock
ae153e57f5 refactor: remove Decorator from render.Renderer.Render() signature 2017-11-18 18:04:52 +00:00
Matthias Radestock
a82d245e93 simplify render decoration
Decoration is in fact quite a simple process that is applied on entry
to rendering: we take a base renderer, transform it with a decorator,
and then render a report with it. The new render.Decorate() function
does exactly that.

There is one exception. When rendering an individual node, e.g. for
showing its details panel in the UI, we must not lose the node during
decoration. That requires some special logic, which previously resided
in the PreciousNodeRenderer, and now lives in handleNode.
2017-11-18 18:04:52 +00:00
Matthias Radestock
4e4c3b25c4 Expand the app's k8s namespace calculation
to include recently added k8s types.

This is all rather inefficient. See #2945.
2017-11-18 17:35:57 +00:00
Matthias Radestock
8f7e00f46a Stats are easily produced as part of Rendering
...so there is no need for a separate Stats method.

step 1: return stats from Render
2017-11-08 07:15:28 +00:00
Bryan Boreham
8a11697957 Add BenchmarkTopologyHosts() and BenchmarkTopologyContainers()
To time a single topology, for more focused optimisation
2017-11-06 20:48:25 +00:00
Matthias Radestock
0b9bab255c memoise top-level rendering
This deals with browsers requesting the same rendering for the same
timespan when no new reports have been received for that timespan.
2017-11-04 22:24:46 +00:00
Damien Lespiau
466927ce68 lint: Fix 2 sites failing a recently introduced golint check
golint has gained a new check and we don't freeze the golint version so CI was
failing on unrelated PRs:

app/multitenant/consul_client.go:69:2: redundant if ...; err != nil check, just return error instead.
report/marshal.go:188:2: redundant if ...; err != nil check, just return error instead.

Fix those!
2017-09-26 12:15:09 +01:00
Jonathan Lange
f9e37a21f5 Continue processing reports if billing fails 2017-09-22 12:20:10 +01:00
Roland Schilter
0d381a34d6 Link scope-ui graphs clickable to prometheus queries (#2664)
scope-app:
* Adds `-app.metrics-graph` cli flag for configuring the base url to
use for graph links; supports :orgID and :query placeholders
* Assigns query URLs to existing metrics and appends empty metrics if missing

scope-ui:
* Extends <CloudFeature /> with option alwaysShow
* Adds <CloudLink /> to simplify routing when in cloud vs not in cloud
* Links metric graphs in the ui's node details view for all k8s
toplogies and containers so far
* Tracks metric graph click in mixpanel `scope.node.metric.click`
* Uses percentages and MB for CPU/Memory urls
* Passes timetravel timestamp to cortex in deeplink
2017-08-15 18:56:23 +01:00
Matthias Radestock
c312579bfe log sqs messages at 'debug' rather than 'info' level
Fixes #2790.
2017-08-07 10:17:05 +01:00
Matthias Radestock
1f95e3efd4 obtain historic_reports capability from Reporter 2017-07-31 09:58:31 +01:00
Alfonso Acosta
7b4b410f4a Review Feedback 2017-07-20 11:01:44 +00:00
Alfonso Acosta
c6d5a6a646 Make linter happy 2017-07-20 10:05:48 +00:00
Alfonso Acosta
f66d79d427 Backoff when writing to Dynamo and S3 2017-07-18 15:28:08 +00:00
Alfonso Acosta
34bfc22b4f Fix tests 2017-07-03 20:20:28 +00:00
Mike Lang
c521b50bc6 Remove type filter in controllers view
This was always temporary until we could search for type via the search box instead,
since it resulted in too much clutter / didn't seem a useful use-case.

As it turns out, searching like this was already possible.
2017-06-29 16:03:53 -07:00
Mike Lang
ef2b6f6c55 Remove replica sets
Use a special kind of selector renderer to elide replica sets from pod nodes
and directly reference deployment parents instead.
Do the inverse (replace replica sets with pods) during the mapping from pod to deployment.

Note we can no longer use renderParents since we're using a non-standard Selector
2017-06-29 10:13:15 -07:00
Matthias Radestock
5317b224ec Merge pull request #2652 from weaveworks/do-not-filter-endpoint
do not filter endpoints by procspied/ebpf in renderers

Fixes #2551.
Fixes #2558.
2017-06-28 08:54:38 +01:00
Roland Schilter
e6bc1d6ec2 Merge pull request #2649 from weaveworks/1975-honor-docker-host-in-probe
Honor DOCKER_* env variables in probe and app
2017-06-28 09:07:52 +02:00
Matthias Radestock
3209d65d52 filter out unconnected pseudo nodes on the outside
The change is necessitated by the removal of procspied/ebpf endpoint
filtering in the renderers, as a result of which the odd
conntracked-only, unconnected pseudo node can sneak through.

This new way of doing things also makes renderers more composable and
robust, and more directly reflects the objective:
- in the process topologies, filter out all unconnected nodes
- in all other topologies, filter out unconnected pseudo nodes
2017-06-27 20:28:30 +01:00
Mike Lang
8eb72cf276 Remove deployment and daemonset views, change combined to controllers and remove bare pods and replica sets
Since we still need to map through replica sets to find matching deployments, we simply filter them out as a post-step.
2017-06-27 10:19:04 -07:00
Mike Lang
d0cbf47c1c Add an option group to Combined view to select group type (daemonset or deployment)
This is a union set, so it will be suitable even as we continue to add more node types to this view.
2017-06-27 10:19:04 -07:00
Mike Lang
6dae014352 Add new view for 'combined' daemonset/deployment view
The idea is that this view shows all 'pods or groups of pods' at 'the highest level of abstraction'.
For now, we just show daemonsets and deployments.
2017-06-27 10:19:04 -07:00
Roland Schilter
651e52b5a5 Honor DOCKER_* env variables in probe and app
Changed default for flag `-app.docker` to use the DOCKER_* env variables
instead of hardcoded /var/run/docker.sock; uses docker's default if
no DOCKER_HOST defined, for both probe and app.

Fixes #1975
2017-06-27 17:14:49 +02:00
Mike Lang
0e54b27e81 Don't omitempty for option group default value
The value "" is meaningful, and the UI does not behave correctly if it's not present.
2017-06-21 15:17:07 -07:00
Mike Lang
383ace4d7e Replace 'none' id for the none option in union types with ''
In union option groups, the list of selected ids is encoded as a comma-delimited string.
For example, if 'foo' and 'bar' are selected, the value 'foo,bar' is sent, ie ["foo", "bar"] -> "foo,bar"
Under this scheme, with nothing selected, the empty string should be sent, ie. [] -> ""

Before this change, the frontend code called the "none option" by id 'none'.
There were several issues with this:
* The frontend would send 'none' when nothing was selected, not ''. The backend ignored this as it ignores
  junk values in the options, treating them as though they hadn't been given.
* The backend would attempt to set the default value of an option to "nothing selected", ie. [], by sending ''.
  The frontend would interpret this as nothing selected, *not even the 'none' option*, which caused a visual bug.
* Everything would break if one of the legitimate options had the id 'none', which could easily happen eg.
  if a user had a 'none' k8s namespace. This is perhaps an unusual name, but our code shouldn't break when
  a particular arbitary string is used as an input.

With this change, the none option is called '', which fixes all the above problems:
* The frontend encodes [''] as ''
* The frontend decodes '' as [''], and therefore shows the '' option as selected
* The string '' is not a valid k8s namespace name and is a reasonable "prohibited value" for all other use cases.
  The backend already couldn't handle a value with this id correctly prior to this change anyway.
2017-06-21 15:03:18 -07:00
Filip Barl
eb64d3f09b Make API calls with time travel timestamp (#2600)
* Node details fetching reports at proper timestamp.

* Corrected all the relevant timestamps in the UI.

* Renamed some state variables.

* Time travel works for topologies list.

* Added a whole screen overlay for time travel.

* Polished the backend.

* Make time travel work also with the Resource View.

* Fixed the jest tests.

* Fixed the empty view message for resource view.

* Some naming polishing.

* Addressed the comments.
2017-06-20 12:31:22 +02:00
Filip Barl
b6dfe25499 Time travel control (#2524)
* Hacky working prototype.

* Operate with time.Duration offset instead of fixed timestamp.

* Polished the backend code.

* Made a nicer UI component.

* Small refactorings of the websockets code.

* Fixed the backend tests.

* Better websocketing and smoother transitions

* Small styling refactoring.

* Detecting empty topologies.

* Improved error messaging.

* Addressed some of David's comments.

* Moved nodesDeltaBuffer to a global state to fix the paused status rendering bug.

* Small styling changes

* Changed the websocket global state variables a bit.

* Polishing & refactoring.

* More polishing.

* Final refactoring.

* Addressed a couple of bugs.

* Hidden the timeline control behind Cloud context and a feature flag.

* Addressed most of @davkal's comments.

* Added mixpanel tracking.
2017-06-12 11:22:17 +02:00