Commit Graph
378 Commits
Author SHA1 Message Date
Matthias Radestock eaafe4db6f copy Shortcut property in Report.Copy()
This was causing shortcut reports to be delivered as normal reports.

This got broken in 0ecb908c22, when Report.BackwardCompatible(),
which does a Copy(), was introduced in the publishing path.

Fixes #3113.
2018-03-25 09:02:36 +01:00
Matthias Radestock c9d9068f68 Merge pull request #2399 from weaveworks/big-roundtrip
Add a test that checks if reports with data round-trip
2018-02-26 13:44:22 +00:00
Matthias Radestock 306a744189 Merge pull request #3095 from weaveworks/blank-latestmap-insert
Blank out value on LatestMap decode insert
2018-02-26 13:43:40 +00:00
Bryan Boreham f44f8806cd Add a test that checks if reports with data round-trip
Previously the only roundtrip test was for an empty report.
This test has fake data similar to that found in real reports.
'Metrics' does not round-trip exactly, so a DeepEqual workaround is
added for that.
2018-02-26 10:52:50 +00:00
Bryan Boreham f17640646e Blank out value on LatestMap decode insert
Fixing a rare case that came up in a test. In order for this to cause
a problem, the data being decoded has to have entries out of order,
and have a value that is nil or omitted.
2018-02-26 10:34:53 +00:00
Bryan Boreham 64570f1311 Add Kubernetes service type and ports 2018-02-24 14:58:13 +00:00
Bryan Boreham b5cdcb9a42 Move DNS name mapping from endpoint to report 2018-02-20 16:14:21 +00:00
Bryan Boreham b742846835 Optimise processTopology() (#3074)
* Add a benchmark for processprocessTopology()

* Shortcut merging with an empty set

* Use more efficient apis to create process node
2018-02-19 10:13:58 +00:00
Bryan Boreham f72ced3380 Add topology.ReplaceNode() for efficiency (#3073)
* Add topology.ReplaceNode() for efficiency

In some places AddNode() was called after adding to an existing node,
in which case the Merge() is just a waste of time.
2018-02-19 10:13:31 +00:00
Matthias Radestock 5b30b668ae refactor: don't return receiver in Topology.AddNode()
This had little use and was obscuring the mutating nature of
AddNode().
2018-02-19 05:10:04 +00:00
Bryan Boreham cb6edfeec1 Set 'omitempty' on Node Adjacency
Looks like an omission
2018-02-07 13:39:54 +00:00
Roberto Bruggemann 4233b75528 report.Upgrade: reconstruct namespaces
Old probes do not report namespace topologies.
`report.upgradeNamespaces()` recontructs namespace topologies using the data available from other kubernetes resources.

Also, add a test.
2018-01-03 13:55:27 +00:00
Roberto Bruggemann ccfcc61042 The probe reports namespaces 2018-01-03 13:55:27 +00:00
Roberto Bruggemann f9f0487fdc Fix erroneous comments 2018-01-02 18:02:08 +00:00
Matthias Radestock 286453b9a9 refactor: move RenderContext where it belongs
It shouldn't be in 'report' because that is shared between the probe
and the app, and RenderContext, as the name suggests, is only needed
in the app.
2017-12-27 16:22:52 +00:00
Matthias Radestock 0cb34b53cc introduce ParseProcessNodeID
unused for now
2017-12-26 02:27:53 +00:00
Matthias Radestock 590fa55cf5 refactor: banish TheInternet
TheInternet hasn't existed as a single node for a long time.

We also move & export the IsInternetNode predicate so it can be used
in more places.
2017-12-25 18:29:04 +00:00
Matthias Radestock ceb3539d35 ensure hostNodeIDs actually are what they claim
It's a dynamic type check of sorts.

In the process we stop abusing ParseNodeID for extracting the host,
which in turn allows us to clarify its purpose.
2017-12-25 18:14:34 +00:00
Matthias Radestock 38e389e519 optimisation of Copy()ing/Merge()ing some report data structures
We consistently

- pre-allocate in Copy()
- merge the smaller into a copy of the larger in Merge()

This doesn't make much of a difference overall, since there are
comparatively few instances of these structures. But it costs little
in the code, and the consistency alone is worth it.
2017-12-25 14:16:20 +00:00
Matthias Radestock 3de04685f8 optimisation: pre-allocate, and fewer slices during summarisation
Pre-allocating slices really pays dividends when we create loads of
them and they don't grow very large.

We take special care to return nil rather than 0-length slices. This
a) saves further on allocation, and b) is required for some crude
tests to pass that match on nil rather than length.

Also, a bunch of code in templates/tables used slices as Maybe's,
i.e. returning nil or a single-element slice, which is horrendously
inefficient. Eliminating these saves a lot of allocations.
2017-12-25 12:33:43 +00:00
Matthias Radestock dcda88471a get rid of report.Topologies()
The three report.Walk* functions are quite enough.
2017-12-24 22:30:20 +00:00
Matthias Radestock 9419c3ef5c refactor(ish): reduce number of topology lists
Having 6 lists of topolgies in the same file is a bit much:

1. consts for topology names
2. Report type definition
3. MakeReport() Report initialisation
4. Report.Topology(name) lookup
5. Report.TopologyMap() mapping of names to topology references
6. Report.WalkPairedTopologies() iterator over topology references

We get rid of 5 and 6 by introducing a topologyNames slice. So we
are down to 5.

We replace Report.TopologyMap() with a new function,
WalkNamedTopologies, that uses topologyNames. WalkPairedTopologies()
is updated to operate in a similar fashion. Likewise for
WalkTopologies() and Topologies() - these were previously calling
Walk[Paired]Topologies, but it is clearer to simply implement them
directly.
2017-12-24 22:26:57 +00:00
Matthias Radestock 583e81d733 make Report.Toplogy(name) fast
Previously this was buidling a fresh map of all topologies, just so it
could look up the one given as an argument.

Node summarisation (via detailed.Summaries) in particular was badly
affected by that.
2017-12-24 19:11:24 +00:00
Matthias Radestock 315b05062f remove vestiges of edge metadata from comments 2017-12-17 18:38:36 +00:00
Matthias Radestock b540a1639b fix comment: overlay edges *are* present 2017-12-17 18:37:14 +00:00
Matthias Radestock e93b69cf10 remove Node.Edges
It is unused and none of the adjacency mapping code in the renderer
takes any notice of it. Removing this shrinks the report size.

Edges were introduced in #838. At the time we had an experimental
packet sniffer under experimental/sniff/sniffer.go. That got removed
in #1646.

We can resurrect this if we ever decide to add meta data to edges.
2017-12-17 13:28:22 +00:00
Matthias Radestock e24d3e93fe Add static map of common keys to reduce allocations 2017-12-11 20:26:47 +00:00
Matthias Radestock 1f2247a8c4 move node metadata keys into report package
Both the probe and the app (for rendering) need to know about them.
2017-12-11 20:26:08 +00:00
Roberto Bruggemann c08d39f8bd report.Upgrade() add deployments to pods as parent
This was previously done selectPodsWithDeployments.Render().
2017-12-11 14:50:41 +00:00
Matthias Radestock ea20de58be Merge pull request #2964 from weaveworks/do-not-reportallocs
do not report allocations in benchmarks
2017-12-08 14:11:18 +00:00
Matthias Radestock d528292cd4 make report upgrading fast when it's a no-op
The vast majority of the cost is memory allocation, so doing a first
pass to see whether any upgrading is necessary at all, and thus
avoiding allocation when it isn't, is a massive saving.
2017-12-07 21:01:34 +00:00
Matthias Radestock f305c35bfd do not report allocations in benchmarks
That's what -benchmem is for.
2017-12-07 20:27:31 +00:00
Bryan Boreham d1bf98dfde Remove struct wrapping LatestMap
It isn't necessary, and it prevents the codec from seeing maps as empty
2017-12-06 15:46:31 +00:00
Matthias Radestock 4162b5d734 allocate less memory in LatestMap merging
Most maps we merge have the same keys, or at least one set of keys is
the subset of the other. Therefore, allocate a result slice capable of
holding only the max of number keys, rather than the sum.
2017-12-02 13:13:08 +00:00
Matthias Radestock c953313b01 move main report processing benchmarks in one place
so they can share code and are easier to run in combination.

We take advantage of the code sharing by generalising the report
rendering benchmarks to read & merge reports from a dir.
2017-12-02 11:29:16 +00:00
Bryan Boreham 05d5d339c4 Decode reports from a byte buffer
Reading and uncompressing the entire message into memory first allows
the decoder to avoid memory allocations in field names, etc.
2017-11-29 21:56:55 +00:00
Matthias Radestock cdbc01ecf9 simplify report.MakeFromFile
this new version is less efficient... but not for much longer
2017-11-29 21:48:52 +00:00
Matthias Radestock ecf3ae90ff add a benchmark for report unmarshalling 2017-11-29 21:48:52 +00:00
Bryan Boreham b9890064c2 Copy net.ParseIP and modify to save memory allocations
Pass in a slice on the stack instead of allocating one on the heap:
reduces garbage, hence makes the program run faster

Also apply knowledge that critbitgo will do an append() with one extra
byte, so we do that allocation up-front too.  This is innocuous should
we stop using critbitgo or should its internals change.
2017-11-15 23:15:55 +00:00
Bryan Boreham e41332a775 In id-parsing functions, Split strings using Index to save allocations
These routines are called a lot, so this change reduces garbage collection
2017-11-15 23:15:10 +00:00
Filip Barl e233e64279 Improved the duration parsing on frontend. 2017-11-03 10:43:41 +01:00
Filip Barl 320b9e240f Abstracted the report data types. 2017-11-03 10:43:41 +01:00
Bryan Boreham 5acece6e58 Refactor: loop replaced with append() 2017-10-11 09:50:06 +00:00
Bryan Boreham a52c9df48c Added benchmarks for encode and decode of StringLatestMap 2017-10-01 16:19:00 +00:00
Bryan Boreham 736ae5e7c8 Refactor: extract fn to make map for benchmarks 2017-10-01 16:19:00 +00:00
Bryan Boreham 29f139d424 Re-implement LatestMap as a sorted slice for better performance
both the Python generator and the Go generated code are checked in
2017-10-01 16:18:59 +00:00
Bryan Boreham 9322b54b68 Stop causing errors by parsing empty strings as time.Time
We encode the zero time as an empty string, so we should do the
reverse when decoding.

Otherwise time.Parse() returns an error, which is created on the heap,
causing extra garbage-collection load.
2017-09-30 14:33:20 +00:00
Damien Lespiau 466927ce68 lint: Fix 2 sites failing a recently introduced golint check
golint has gained a new check and we don't freeze the golint version so CI was
failing on unrelated PRs:

app/multitenant/consul_client.go:69:2: redundant if ...; err != nil check, just return error instead.
report/marshal.go:188:2: redundant if ...; err != nil check, just return error instead.

Fix those!
2017-09-26 12:15:09 +01:00
Roland Schilter 0d381a34d6 Link scope-ui graphs clickable to prometheus queries (#2664)
scope-app:
* Adds `-app.metrics-graph` cli flag for configuring the base url to
use for graph links; supports :orgID and :query placeholders
* Assigns query URLs to existing metrics and appends empty metrics if missing

scope-ui:
* Extends <CloudFeature /> with option alwaysShow
* Adds <CloudLink /> to simplify routing when in cloud vs not in cloud
* Links metric graphs in the ui's node details view for all k8s
toplogies and containers so far
* Tracks metric graph click in mixpanel `scope.node.metric.click`
* Uses percentages and MB for CPU/Memory urls
* Passes timetravel timestamp to cortex in deeplink
2017-08-15 18:56:23 +01:00
Matthias Radestock 7db84aba21 make network calculation commutative 2017-08-10 11:08:42 +01:00