Use a special kind of selector renderer to elide replica sets from pod nodes
and directly reference deployment parents instead.
Do the inverse (replace replica sets with pods) during the mapping from pod to deployment.
Note we can no longer use renderParents since we're using a non-standard Selector
The change is necessitated by the removal of procspied/ebpf endpoint
filtering in the renderers, as a result of which the odd
conntracked-only, unconnected pseudo node can sneak through.
This new way of doing things also makes renderers more composable and
robust, and more directly reflects the objective:
- in the process topologies, filter out all unconnected nodes
- in all other topologies, filter out unconnected pseudo nodes
The filtering of endpoints causes some connections to get missed for
non-eBPF-tracked connections. Furthermore, the filtering of endpoints
is entirely pointless when the probes run eBPF since the filters just
pass through eBPF-tracked endpoints (for good reason too; because
otherwise some connections would be missed). So in that case it is
just costing CPU and removing it actually improves performance.
Note that removing the filtering does not result in over-counting
connections since that is done by source ip:port pairs.
Fixes#2551.
Fixes#2558.
* Maps metrics if there is a single pod in the controller, as per all other views.
* Also added heavy commenting on the increasingly-complex render chain
Since there are multiple types in the same topology, displaying the type is important.
We do this in multiple places:
* Add node type to minor label
* Add node type as metadata and include in metadata template.
Even though this will always be the same for every node of that topology, this was
the easiest way to add it so it displays in the table view.
Note we can't control ordering of columns in table view, it's always alphabetical.
The existing technique of "reducing" the two rendered graphs for daemonsets and deployments
had a glaring issue that no connections would ever be made between nodes of different types,
since that information would've been discarded earlier in the process.
It also makes it hard to identify "parentless" pods.
This commit extends the Map2Parent function, teaching it:
* To check multiple topologies for parents
* To pass through nodes with no parents found without modification
Since we already had two 'modes' for what to do with nodes without parents,
and it would've been clunky to try to encode the third option into the existing PseudoNodeID
arg in some way, we instead split it into two args, with the first being an enum specifying
either the old pseudo node behaviour, the old drop behaviour, or the new keep behaviour.
We then use the new Map2Parent to map pods to:
* A replica set, if it has one
* A daemonset, if it has one
* Itself, if neither of the above
and then map again from the results to any deployment, leaving as-is any nodes that
don't map to a deployment. Hence we are left with:
* Deployments
* Daemonsets
* Replica sets, but only if they map to no deployment
* Pods, but only if they map to none of the above
and connections between all these will be calculated correctly.
ProcessRenderer was coloring connected nodes because we need that info
for rendering details panels. However, the main process topology view
renderers depending on ProcessRenderer were also doing coloring
themselves. For the 'processes' topology that was literally
duplicating work. For the 'processes-by-name' topology that was
throwing away the process coloring, and then coloring at the name
level.
Solution: remove the coloring from the ProcessRenderer, thus
eliminating the duplicate/thrown-away work, and introduce a
ColorConnectedProcessRenderer which is only used in places that
populate details panels.
The rendering code checks whether endpoint IPs are part of
cluster-local networks. Due to the prevalence of endpoints - medium
sized reports can contain many thousands of endpoints - this is
performance critical. Alas the existing code performs the check via a
linear scan of a list of networks. That is slow when there are more
than a few, which will be the case in the context of k8s, since there
the probes register service IPs as local /32 networks.
Here we change representation of the set of networks to a prefix
tree (aka trie), which is well-suited for IP network membership checks
since networks are in fact a bitstring prefixes.
The specific representation is a crit-bit tree, but that choice was
purely based on implementation convenience - the chosen library is the
only one I could find that directly supports IP networks.
The rendering code checks whether endpoint IPs are part of
cluster-local networks. Due to the prevalence of endpoints - medium
sized reports can contain many thousands of endpoints - this is
performance critical. Alas the existing code performs the check via a
linear scan of a list of networks. That is slow when there are more
than a few. Unfortunately in some common k8s network setups, e.g. on
AWS, a cluster can contain hundreds of networks, due to /32 networks
derived from interfaces with multiple IPs.
Here we change representation of the set of networks to a prefix
tree (aka trie), which is well-suited for IP network membership checks
since networks are in fact a bitstring prefixes.
The specific representation is a crit-bit tree, but that choice was
purely based on implementation convenience - the chosen library is the
only one I could find that directly supports IP networks.
the information is constant and already present in the id, so we can
extract it from there.
That reduces the report size and improves report encoding/decoding
performance. It should reduce memory usage too and improve report
merging performance too.
NB: Probes with this change are incompatible with old apps.
We have to introduce the kinda hacky concept of a 'No Stack' stack
to reconcile it with the idea of a 'default' k8s namespace. This is important
because swarm services without a stack don't have the same docker labels as ones that do.
Curiously, they still have what appears to be a stack name 'prefix' on their names,
but I can't isolate that name anywhere easily so they'll just have to make do.
I basically copy-pasted updateFilters to make this work, todo go back and refactor
to not duplicate 90% of the code.
We replace the existing data structure with a simpler one that
only specifies how to get the parent label, which is the only
part of the Parent struct that can't be generated from the node info alone.
Future work: Standardize this concept of a label and put it in the topology instead.
Though that already exists...so just use it?
The default sets the node label to the node ID.
This is likely to not look very good, but the intent is that it creates an obvious problem,
ie. that the node ID is being used as the label, rather than a silent omission or more subtle problem.
Possible future work:
* For single-component IDs, extract the component automatically and use that instead.
* Instead of functions, in simple cases just have a LUT by topology with common behaviours like
'stack = true or false', 'label = this key in node.Latest'
The latter opens up to eventually moving this info inside the report itself ala topology templates,
or at least centralizing it in the source.
Currently, if a topology does not have any specific info in nodeSummariesByID,
any children of the node that belong to that topology will be silently omitted.
This change adds a default behaviour for such topologies, with no special columns
but at least it is displayed at all.
Unlisted topologies are displayed after all listed ones, in arbitrary order.
Note that completely bogus or other special cases (eg. topology = Pseudo) still will not
be displayed as report.Topology() will fail.
