Commit Graph

67 Commits

Author SHA1 Message Date
Matthias Radestock
c8ea7ba49e refactor: simplify joinResults.add*
After dropping extra metadata in the rest of this PR, our usage of
joinResults.add* only ever ends creating minimal nodes, from just an
id and topology. Hence joinResults.add* can be invoked with simply an
id and topology instead of a generic node creation function.
2017-12-29 14:40:18 +00:00
Matthias Radestock
a7b6bb09a1 refactor: lift closures and extract constants 2017-12-29 14:40:18 +00:00
Matthias Radestock
2d6badba79 cope with processes that have no HostNodeID metadata
Some process nodes may not have a HostNodeID metadata, e.g. when an
endpoint references a pid that we know nothing about. When mapping
processes to containers, we therefore shouldn't rely on
HostNodeID. Instead we can obtain the hostID from the process node ID.

This has been broken for a while, possibly forever.
2017-12-29 14:40:16 +00:00
Matthias Radestock
4c1d5640ae don't propagate IsConnectedMark in MapProcess2Container
...when creating Uncontained pseudo nodes. IsConnectedMark only
affects summarisation of genuine process nodes.
2017-12-28 12:56:52 +00:00
Matthias Radestock
5d06f90f10 don't propagate HostNodeID in MapProcess2Container
...when creating Uncontained pseudo nodes. Summarisation of
Uncontained/Umanaged only looks at the ID, which includes the
HostNodeID.

We adjust the promotion of Uncontained to Unmanaged, to operate on the
ID instead of (re)extracting the hostID from the HostNodeID
metadata. With that, nothing looks at the HostNodeID metadata of
Uncontained/Unmanaged nodes.
2017-12-28 12:54:21 +00:00
Matthias Radestock
7ce160d492 don't propagate ImageID in MapContainer2ContainerImage
The ImageID is already the id of the node we are creation, and that's
what summarisation renders in the event we fail to join this node with
a node from the ContainerImage topology that has more metadata.
Nothing is looking at the ImageID metadata field.
2017-12-28 12:50:11 +00:00
Matthias Radestock
9034296209 don't propagate ImageID in MapContainerImage2Name
This was building a set of all the image ids represented by the same
unversioned image. Well, it was doing that until I broke it with a
silly mistake in #1739 - instead of extracting the imageID from the
original node ID, it's extracting it from the updated ID, which is the
unversioned image. Even if it was working though, it's pointless
since nothing is looking at that info.
2017-12-28 12:50:11 +00:00
Matthias Radestock
a46b9c9c24 don't propagate ContainerHostname in MapContainer2Hostname
The container hostname is already the id of the node, and that's what
summarisation renders. Nothing looks at the docker.ContainerHostname
metadata of nodes in the ContainerHostname group topology.
2017-12-28 12:50:11 +00:00
Matthias Radestock
3d1a44aae0 extract label from id when summarising un[contained,managed]
I missed this in #2998.
2017-12-28 12:48:12 +00:00
Matthias Radestock
56a1efd38a optimisation: pre-allocate some maps
This doesn't make much of a difference, since we don't create many of
these. But it's easy enough to do, and every little helps.
2017-12-26 00:35:02 +00:00
Matthias Radestock
a8e3d04d21 cosmetic: fix some comments 2017-12-25 16:21:46 +00:00
Matthias Radestock
a6f24fc151 remove superfluous lookups
Since we seed the joinResult with the nodes from the topology we are
mapping to, we know the 'create' function is only called when there is
no node with the specified id.

This neatly makes the 'create' function only do what it says,
i.e. return _new_ nodes.
2017-12-17 18:24:32 +00:00
Matthias Radestock
ac87c2b6e8 optimise & simplify node propagation in joinResult
Instead of copying unmatched nodes at the end, and matched nodes when
we encounter them, copy *all* nodes at the beginning.
2017-12-17 18:14:48 +00:00
Matthias Radestock
f16908aea9 refactor: fix up adjacencies as part of joinResult.result() 2017-12-17 18:14:47 +00:00
Matthias Radestock
e5117a652f remove superfluous fixupAdjacencies calls
fixupAdjacencies fixes up adjancencies of mapped nodes. The nodes at
the call sites we are removing aren't mapped.
2017-12-17 18:14:30 +00:00
Matthias Radestock
0c894e9446 refactor: drop networks from render.MapFunc
All the MapFuncs that needed networks have been elevated to Renderers.
2017-12-17 00:18:12 +00:00
Matthias Radestock
956303694a always apply ColorConnected in process renderers
This eliminates the awkward distinction between ProcessRenderer and
ColorConnectedProcessRenderer.

It also ensures that processes resulting from direct rendering of the
process topology (/api/topology/processes is invoking
ProcessWithContainerNameRenderer and /api/topology/processes-by-name
is invoking ProcessNameRenderer) are colored and hence summarising
them correctly sets the 'linkable' property. This was the behaviour
prior to the revamping of the rendering pipeline. However, it doesn't
actually make a practical difference since process detail panels only
show other processes as connection endpoints, and these are always
marked linkable anyway.
2017-11-28 06:54:15 +00:00
Matthias Radestock
346a0360ef Refactor: split addToResults into two functions
This is preparatory to future refactorings: all existing calls are to
Endpoints which have no children and where we don't want a Counter.

We make addChildAndChildren an obvious extension of addChild even
though it adds a dead code path (we never call addChildAndChildren
with an endpoint).
2017-11-22 10:30:18 +00:00
Matthias Radestock
67950985ef refactor: introduce IsConnected FilterFunc
and rename existing IsConnected const to IsConnectedMark
2017-11-20 09:28:56 +00:00
Matthias Radestock
ae153e57f5 refactor: remove Decorator from render.Renderer.Render() signature 2017-11-18 18:04:52 +00:00
Matthias Radestock
9bd8bd825b remove now superfluous Renderer.Stats method
step 2 (and final step) in producing stats as part of Rendering
2017-11-08 07:15:28 +00:00
Matthias Radestock
8f7e00f46a Stats are easily produced as part of Rendering
...so there is no need for a separate Stats method.

step 1: return stats from Render
2017-11-08 07:15:28 +00:00
Bryan Boreham
4feb451760 Refactor join-Renderer helper functions as methods
New type joinResult is created to hold the nodes and ID mapping.
The implementation move from host.go to render.go.
2017-11-06 22:12:13 +00:00
Bryan Boreham
cbba3c0fd3 Clarify use of 'id' in addToResults
Pass 'id' through to the create function and expect that the result Node has that ID.
Extract a function newPseudoNode for common calls.
2017-11-06 22:12:13 +00:00
Bryan Boreham
e16aaf6c43 Merge nodeToIP, endpoints2Nodes and ipToNode into one Renderer
This is much more efficient as we skip creating then merging all intermediate Nodes
2017-11-06 22:05:05 +00:00
Matthias Radestock
5f4f9da4df memoise a few more shared renderers 2017-11-05 00:21:45 +00:00
Matthias Radestock
b793a9efa8 memoise shared top-level renderers
and add a comment indicating non-memoisation of other, not shared
top-level renderers.

This memoisation is effective when the browser requests multiple
topologies for the same report.
2017-11-04 23:05:52 +00:00
Matthias Radestock
ac9f7dfad1 simplify node propagation in ipToNode 2017-07-13 17:10:43 +01:00
Matthias Radestock
bd9fafff42 don't create *derived* nodes in nodeToIP
IP nodes are children of the "proper" topology nodes, not the other
way round. The former relationship is established in ipToNode.
2017-07-13 17:00:14 +01:00
Matthias Radestock
4a77e9bfac don't bother tracking the orignal node topology
We always join with the original renderer and hence all the nodes
produced by ipToNode will get their other data, including topology,
via that route.
2017-07-13 16:35:52 +01:00
Matthias Radestock
c7f94d8a74 join with original renderer in ConnectionJoin
This guarantees that the output won't contain bare nodes containing
just an id and topology, as produced by ipToNode. Previously this was
ensured by calling convention, now it's ensured by construction.
2017-07-13 16:28:02 +01:00
Matthias Radestock
387a68a3c7 tiny refactor: swap ConnectionJoin arg order
to match the style of other render HoFs, which take their inputs last.
2017-07-13 14:23:46 +01:00
Matthias Radestock
f0ae2bd98c refactor: use inline StringSet constructor 2017-07-04 06:29:19 +01:00
Matthias Radestock
9dc50b5202 refactor: hide "empty set" constants
They are an implementation detail.
2017-07-03 01:26:22 +01:00
Matthias Radestock
3209d65d52 filter out unconnected pseudo nodes on the outside
The change is necessitated by the removal of procspied/ebpf endpoint
filtering in the renderers, as a result of which the odd
conntracked-only, unconnected pseudo node can sneak through.

This new way of doing things also makes renderers more composable and
robust, and more directly reflects the objective:
- in the process topologies, filter out all unconnected nodes
- in all other topologies, filter out unconnected pseudo nodes
2017-06-27 20:28:30 +01:00
Matthias Radestock
e5a04ec5a4 do not filter endpoints by procspied/ebpf in renderers
The filtering of endpoints causes some connections to get missed for
non-eBPF-tracked connections. Furthermore, the filtering of endpoints
is entirely pointless when the probes run eBPF since the filters just
pass through eBPF-tracked endpoints (for good reason too; because
otherwise some connections would be missed). So in that case it is
just costing CPU and removing it actually improves performance.

Note that removing the filtering does not result in over-counting
connections since that is done by source ip:port pairs.

Fixes #2551.
Fixes #2558.
2017-06-27 20:28:13 +01:00
Matthias Radestock
d66b28de2a performance: only color connected once
ProcessRenderer was coloring connected nodes because we need that info
for rendering details panels. However, the main process topology view
renderers depending on ProcessRenderer were also doing coloring
themselves. For the 'processes' topology that was literally
duplicating work. For the 'processes-by-name' topology that was
throwing away the process coloring, and then coloring at the name
level.

Solution: remove the coloring from the ProcessRenderer, thus
eliminating the duplicate/thrown-away work, and introduce a
ColorConnectedProcessRenderer which is only used in places that
populate details panels.
2017-06-22 10:28:39 +01:00
Matthias Radestock
8c6a171553 refactor: extract a couple of heavily used constants 2017-06-21 21:12:55 +01:00
Matthias Radestock
6eaffb44e0 fix bug: handle short-lived ebpf-tracked connections again
This got broken in #2559.

The problem here is similar to #2551.
2017-06-04 18:42:54 +01:00
Matthias Radestock
30c38a958f remove blatant falsehoods from comments 2017-06-04 16:23:03 +01:00
Matthias Radestock
ebcf9dcf10 refactor: rename ShortLivedConnectionJoin to ConnectionJoin
since it's dealing with *all* connections, not just short-lived ones.
2017-06-04 16:10:21 +01:00
Matthias Radestock
9bc7b30f0f extract and expand endpoint procspied filter
The filter needs to exclude both procspied and eBPF-tracked endpoints,
since both will be picked up by the process topology.
2017-06-04 16:10:21 +01:00
Matthias Radestock
ee0736df69 refactor: extract constant mapEndpoint2IP 2017-06-04 16:10:21 +01:00
Matthias Radestock
6697f4a897 refactor: declosure ShortLivedConnectionJoin 2017-06-04 16:10:21 +01:00
Filip Barl
2e9255b190 Addressed the comments and fixed the tests. 2017-02-20 11:40:40 +01:00
Filip Barl
f1904a626f Fix filtering issue for uncontained nodes in DNS name view (#2170). 2017-02-20 11:38:21 +01:00
Alfonso Acosta
0a135e6330 Check for known services before external IPs
Known services can be internal (e.g. same VPC in AWS)
2017-01-31 15:37:57 +00:00
Alfonso Acosta
216cc0d605 Add image table to container nodes
Also, extend metadata of images with sizes
2016-11-02 13:16:11 +00:00
Alfonso Acosta
c5ac315b38 Review feedback 2016-09-20 16:50:29 +00:00
Alfonso Acosta
5c080ec062 Add pseudo-nodes for known services
AWS S3 and Dynamo to start with
2016-09-19 14:38:58 +00:00