Commit Graph

570 Commits

Author SHA1 Message Date
Matthias Radestock
bd214227dc introduce ParsePseudoNodeID
unused for now

A convenient place to document and deal with the ugliness.
2017-12-26 02:27:54 +00:00
Matthias Radestock
56a1efd38a optimisation: pre-allocate some maps
This doesn't make much of a difference, since we don't create many of
these. But it's easy enough to do, and every little helps.
2017-12-26 00:35:02 +00:00
Matthias Radestock
590fa55cf5 refactor: banish TheInternet
TheInternet hasn't existed as a single node for a long time.

We also move & export the IsInternetNode predicate so it can be used
in more places.
2017-12-25 18:29:04 +00:00
Matthias Radestock
ceb3539d35 ensure hostNodeIDs actually are what they claim
It's a dynamic type check of sorts.

In the process we stop abusing ParseNodeID for extracting the host,
which in turn allows us to clarify its purpose.
2017-12-25 18:14:34 +00:00
Matthias Radestock
a8e3d04d21 cosmetic: fix some comments 2017-12-25 16:21:46 +00:00
Matthias Radestock
3de04685f8 optimisation: pre-allocate, and fewer slices during summarisation
Pre-allocating slices really pays dividends when we create loads of
them and they don't grow very large.

We take special care to return nil rather than 0-length slices. This
a) saves further on allocation, and b) is required for some crude
tests to pass that match on nil rather than length.

Also, a bunch of code in templates/tables used slices as Maybe's,
i.e. returning nil or a single-element slice, which is horrendously
inefficient. Eliminating these saves a lot of allocations.
2017-12-25 12:33:43 +00:00
Matthias Radestock
fc66827af7 refactor: don't set shape based on unitialised topology
This doesn't make any difference to the outcome - we were simply
setting the shape in the NodeSummary to "", which is what it starts
out as - but looks less weird in the code.
2017-12-23 22:39:08 +00:00
Matthias Radestock
d861b41837 refactor: inline summarisation of metadata, metrics, tables
This removes a bunch of duplication and scattering of little pieces of
code.
2017-12-23 22:34:45 +00:00
Matthias Radestock
651e42e54e fix accidental report fixture modification
Report.Copy() shallow-copies the nodes in Report.Nodes. Hence
Node.Metrics is shared between the original and the copy. Hence bad
things happen when modifying it.

This bug has laid dormant because by luck other tests in detailed_test
involving metrics are executed first.
2017-12-23 22:21:24 +00:00
Matthias Radestock
724ea0c230 refactor: extract common code of joinResults.addChild variants 2017-12-21 10:05:56 +00:00
Matthias Radestock
9713a156c7 refactor: extract helper 2017-12-21 10:05:56 +00:00
Matthias Radestock
7d261d0ca0 don't map image adjacencies to hosts
it's just wrong
2017-12-21 10:05:56 +00:00
Matthias Radestock
ee4a56e8ad optimse common one->one mapping case
one->many is much rarer
2017-12-19 19:34:52 +00:00
Matthias Radestock
25eeec0227 enhance joinResult to handle mapping one node to multiple nodes 2017-12-18 21:13:15 +00:00
Matthias Radestock
f920f1b9d2 refactor: extract helper to add joinResult mapping 2017-12-18 18:41:52 +00:00
Matthias Radestock
ba7af78cfa ensure result adjacencies start empty
Any existing list would be mutated by result(), which is bad.

Note that all the existing newJoinResults() call sites pass in nodes
with no adjacencies, so this is purely a safety measure.
2017-12-17 18:24:32 +00:00
Matthias Radestock
a6f24fc151 remove superfluous lookups
Since we seed the joinResult with the nodes from the topology we are
mapping to, we know the 'create' function is only called when there is
no node with the specified id.

This neatly makes the 'create' function only do what it says,
i.e. return _new_ nodes.
2017-12-17 18:24:32 +00:00
Matthias Radestock
ac87c2b6e8 optimise & simplify node propagation in joinResult
Instead of copying unmatched nodes at the end, and matched nodes when
we encounter them, copy *all* nodes at the beginning.
2017-12-17 18:14:48 +00:00
Matthias Radestock
4dde1ce715 optimise and align endpoints2Hosts
Avoid a reduce step by joining the host topology in endpoints2Hosts.

This is similar to what we do in connectionJoin and
endpoints2Processes.
2017-12-17 18:14:47 +00:00
Matthias Radestock
f16908aea9 refactor: fix up adjacencies as part of joinResult.result() 2017-12-17 18:14:47 +00:00
Matthias Radestock
e5117a652f remove superfluous fixupAdjacencies calls
fixupAdjacencies fixes up adjancencies of mapped nodes. The nodes at
the call sites we are removing aren't mapped.
2017-12-17 18:14:30 +00:00
Matthias Radestock
cc859926ef tiny simplifying refactor 2017-12-17 15:36:28 +00:00
Matthias Radestock
0c894e9446 refactor: drop networks from render.MapFunc
All the MapFuncs that needed networks have been elevated to Renderers.
2017-12-17 00:18:12 +00:00
Matthias Radestock
20138b9218 don't exclude NATed connections in mapping to processes
We used to ignore source endpoints that are associated with multiple
destination endpoints, which is a partial workaround for our inability
to correctly represent two connections from the same source ip/port
but different processes, or the same destination ip/port but different
processes. See #2665.

However, that condition is too coarse. In particular, we end up
ignoring endpoints that are connected to NATed destinations, since the
latter are represented by two (or more) endpoints.

The change here corrects that.
2017-12-15 11:40:43 +00:00
Roberto Bruggemann
c08d39f8bd report.Upgrade() add deployments to pods as parent
This was previously done selectPodsWithDeployments.Render().
2017-12-11 14:50:41 +00:00
Matthias Radestock
1865c46368 refactor: introduce a constant for "copy_of"
since it's shared between the probe and renderer
2017-12-09 10:45:59 +00:00
Matthias Radestock
f305c35bfd do not report allocations in benchmarks
That's what -benchmem is for.
2017-12-07 20:27:31 +00:00
Matthias Radestock
9fbc388f82 filter internet adjacencies
Edges between the incoming and outgoing internet nodes are typically
artifacts of imperfect connection tracking, e.g. when VIPs and NAT
traversal are in use.

So lets filter them out.
2017-11-28 13:23:17 +00:00
Matthias Radestock
956303694a always apply ColorConnected in process renderers
This eliminates the awkward distinction between ProcessRenderer and
ColorConnectedProcessRenderer.

It also ensures that processes resulting from direct rendering of the
process topology (/api/topology/processes is invoking
ProcessWithContainerNameRenderer and /api/topology/processes-by-name
is invoking ProcessNameRenderer) are colored and hence summarising
them correctly sets the 'linkable' property. This was the behaviour
prior to the revamping of the rendering pipeline. However, it doesn't
actually make a practical difference since process detail panels only
show other processes as connection endpoints, and these are always
marked linkable anyway.
2017-11-28 06:54:15 +00:00
Matthias Radestock
f193a2101c refactor: remove duplication of filtering logic 2017-11-28 06:54:15 +00:00
Matthias Radestock
1416fe928f remove filtering of unconnected pseudo nodes from ordinary filters
It's now done via a special filter, once, after all other filters have
been applied.

Some tests need updating since they were relying on ordinary filters
doing that filtering.
2017-11-28 06:54:15 +00:00
Matthias Radestock
9dca7627b6 filter unconnected nodes after applying user-specified filters
...rather than before. That way, nodes which become unconnected during
filtering are removed, which is what we want. ATM we are depending on
some 'unconnected' filtering inside every filter, which is expensive
and largely redundant. We should soon be able to remove that.

downside: 'unconnected' filtering is no longer memoised.
2017-11-28 06:54:15 +00:00
Matthias Radestock
7bbded9e84 refactor(ish): introduce Tranformers
so we can generalise the filter step in render.Render et al. That will
allow us to apply whole-topology filters in that step.
2017-11-28 06:54:15 +00:00
Matthias Radestock
97fadd6d9f optimising refactor: FilterUnconnected{Pseudo} w/o LatestMap
Instead of three passes
1. building a 'connected' node set
2. marking nodes from that set with a LatestMap entry
3. removing unmarked nodes

we just do two
1. building a 'connected' node set
2. removing nodes not in that set

This does entail duplication of the adjecency list pruning code from
FilterFunc.Apply. We will be able to eliminate that eventually, but
not just yet.

Also, we cannot get rid of ColorConnected completely;
ColorConnectedProcessRenderer uses it to mark nodes, and that
information is required by detailed.processNodeSummary to determine
whether a process in the details panel can be rendered as a link.
2017-11-28 06:54:15 +00:00
Matthias Radestock
be8801ab92 optimisation: faster knownServiceCache
We use an LRU cache implementation that is keyed on strings, which
avoids conversion to/from interface and the memory allocation that
entails.
2017-11-25 19:33:49 +00:00
Matthias Radestock
5bdf46956d optimisation: make Memoise(Memoise(...)) only memoise once
All outputs of Memoise() are fixpoints to the function, i.e. feeding
them as inputs to the function just returns them.

We don't hit this optimisation in current code but have had instances
in the past.
2017-11-23 17:03:33 +00:00
Matthias Radestock
1c39d14c74 make render.ResetCache() reset all caches
so that benchmarks yield more consistent results.
2017-11-22 13:00:39 +00:00
Matthias Radestock
1c206fb1c9 cosmetic: bring comment up to date 2017-11-22 10:40:34 +00:00
Bryan Boreham
5697d53034 Rewrite MapX2Host as a RenderFunc using joinResults
It's more efficient as many input Nodes map onto a few hosts.
2017-11-22 10:39:37 +00:00
Bryan Boreham
3e62f7b754 Rewrite MapProcess2Name to process all nodes in one pass
This is more efficient as there are typically many fewer names than
processes.
2017-11-22 10:33:00 +00:00
Matthias Radestock
346a0360ef Refactor: split addToResults into two functions
This is preparatory to future refactorings: all existing calls are to
Endpoints which have no children and where we don't want a Counter.

We make addChildAndChildren an obvious extension of addChild even
though it adds a dead code path (we never call addChildAndChildren
with an endpoint).
2017-11-22 10:30:18 +00:00
Matthias Radestock
f56a44e67d cosmetic: move a function 2017-11-22 09:46:33 +00:00
Matthias Radestock
334dba20e7 cosmetic: move some functions 2017-11-22 09:19:10 +00:00
Matthias Radestock
b24334c246 fix small bug in Filtered calculation
Filtering nodes *adds* to the Filtered count.
2017-11-22 09:13:23 +00:00
Matthias Radestock
0c43526465 refactor: use new FilterFunc.Apply
instead of constructing temporary Filter renderers.

This also makes clearer what is going on.
2017-11-21 20:16:59 +00:00
Matthias Radestock
e643ec56be refactor: move filter logic to FilterFunc
so it becomes accessible w/o having to construct a Filter renderer.
2017-11-21 20:16:59 +00:00
Matthias Radestock
88e8b52d66 Decorators, begone!
Decorators were just a complicated way of constructing filters.
2017-11-21 20:16:30 +00:00
Matthias Radestock
093857f37f refactor: extract a little test helper 2017-11-21 20:13:14 +00:00
Matthias Radestock
67950985ef refactor: introduce IsConnected FilterFunc
and rename existing IsConnected const to IsConnectedMark
2017-11-20 09:28:56 +00:00
Matthias Radestock
ae153e57f5 refactor: remove Decorator from render.Renderer.Render() signature 2017-11-18 18:04:52 +00:00