Edges between the incoming and outgoing internet nodes are typically
artifacts of imperfect connection tracking, e.g. when VIPs and NAT
traversal are in use.
So lets filter them out.
It's now done via a special filter, once, after all other filters have
been applied.
Some tests need updating since they were relying on ordinary filters
doing that filtering.
...rather than before. That way, nodes which become unconnected during
filtering are removed, which is what we want. ATM we are depending on
some 'unconnected' filtering inside every filter, which is expensive
and largely redundant. We should soon be able to remove that.
downside: 'unconnected' filtering is no longer memoised.
Instead of three passes
1. building a 'connected' node set
2. marking nodes from that set with a LatestMap entry
3. removing unmarked nodes
we just do two
1. building a 'connected' node set
2. removing nodes not in that set
This does entail duplication of the adjecency list pruning code from
FilterFunc.Apply. We will be able to eliminate that eventually, but
not just yet.
Also, we cannot get rid of ColorConnected completely;
ColorConnectedProcessRenderer uses it to mark nodes, and that
information is required by detailed.processNodeSummary to determine
whether a process in the details panel can be rendered as a link.
Decoration is in fact quite a simple process that is applied on entry
to rendering: we take a base renderer, transform it with a decorator,
and then render a report with it. The new render.Decorate() function
does exactly that.
There is one exception. When rendering an individual node, e.g. for
showing its details panel in the UI, we must not lose the node during
decoration. That requires some special logic, which previously resided
in the PreciousNodeRenderer, and now lives in handleNode.
The change is necessitated by the removal of procspied/ebpf endpoint
filtering in the renderers, as a result of which the odd
conntracked-only, unconnected pseudo node can sneak through.
This new way of doing things also makes renderers more composable and
robust, and more directly reflects the objective:
- in the process topologies, filter out all unconnected nodes
- in all other topologies, filter out unconnected pseudo nodes
The filtering of endpoints causes some connections to get missed for
non-eBPF-tracked connections. Furthermore, the filtering of endpoints
is entirely pointless when the probes run eBPF since the filters just
pass through eBPF-tracked endpoints (for good reason too; because
otherwise some connections would be missed). So in that case it is
just costing CPU and removing it actually improves performance.
Note that removing the filtering does not result in over-counting
connections since that is done by source ip:port pairs.
Fixes#2551.
Fixes#2558.
We have to introduce the kinda hacky concept of a 'No Stack' stack
to reconcile it with the idea of a 'default' k8s namespace. This is important
because swarm services without a stack don't have the same docker labels as ones that do.
Curiously, they still have what appears to be a stack name 'prefix' on their names,
but I can't isolate that name anywhere easily so they'll just have to make do.
I basically copy-pasted updateFilters to make this work, todo go back and refactor
to not duplicate 90% of the code.
This field changes the option group behaviour depending on its value.
Currently only supports two values:
"one" (default): Old behaviour, one option can be selected
"union": Any number of options can be selected, and the filters are OR-ed togther
It is written in such a way as to easily enable a future "intersection" option,
as per union but AND-ing the filters. But this is not done here. YAGNI.
This reverts commit 76ddc75fb8, reversing
changes made to 3ade2933eb.
We are rolling this back for now because it's causing a bug where sub-topologies
would have ~3000 repeated cases of the k8s filters, causing performance issues clientside.
To facilitate this, we replace the existing functionality of updateFilters which
sets k8s topologies to have the filters [namespace, managed], to instead append the namespace filter
to any existing. This lets it apply to both k8s and container topologies without overwriting existing
container filters. We instead set the managed filter in the static definition.
This however has the side effect that the ordering of the namespace filter and the managed filter
in k8s topologies has been reversed, so it reads:
Show Unmanaged | Hide Unmanaged
foo | bar | default | baz | All Namespaces
instead of:
foo | bar | default | baz | All Namespaces
Show Unmanaged | Hide Unmanaged
Based on work from Lorenzo, updated by Iago, Alban, Alessandro and
Michael.
This PR adds connection tracking using eBPF. This feature is not enabled by default.
For now, you can enable it by launching scope with the following command:
```
sudo ./scope launch --probe.ebpf.connections=true
```
This patch allows scope to get notified of every connection event,
without relying on the parsing of /proc/$pid/net/tcp{,6} and
/proc/$pid/fd/*, and therefore improve performance.
We vendor https://github.com/iovisor/gobpf in Scope to load the
pre-compiled ebpf program and https://github.com/weaveworks/tcptracer-bpf
to guess the offsets of the structures we need in the kernel. In this
way we don't need a different pre-compiled ebpf object file per kernel.
The pre-compiled ebpf program is included in the vendoring of
tcptracer-bpf.
The ebpf program uses kprobes/kretprobes on the following kernel functions:
- tcp_v4_connect
- tcp_v6_connect
- tcp_set_state
- inet_csk_accept
- tcp_close
It generates "connect", "accept" and "close" events containing the
connection tuple but also pid and netns.
Note: the IPv6 events are not supported in Scope and thus not passed on.
probe/endpoint/ebpf.go maintains the list of connections. Similarly to
conntrack, it also keeps the dead connections for one iteration in order
to report short-lived connections.
The code for parsing /proc/$pid/net/tcp{,6} and /proc/$pid/fd/* is still
there and still used at start-up because eBPF only brings us the events
and not the initial state. However, the /proc parsing for the initial
state is now done in foreground instead of background, via
newForegroundReader().
NAT resolution on connections from eBPF works in the same way as it did
on connections from /proc: by using conntrack. One of the two conntrack
instances is only started to get the initial state and then it is
stopped since eBPF detects short-lived connections.
The Scope Docker image size comparison:
- weaveworks/scope in current master: 22 MB (compressed), 68 MB
(uncompressed)
- weaveworks/scope with this patchset: 23 MB (compressed), 69 MB
(uncompressed)
Fixes#1168 (walking /proc to obtain connections is very expensive)
Fixes#1260 (Short-lived connections not tracked for containers in
shared networking namespaces)
Fixes#1962 (Port ebpf tracker to Go)
Fixes#1961 (Remove runtime kernel header dependency from ebpf tracker)
gofmt load_container_filters.go
removed the environment variable for container label filters
Added the --app.container-label-filter command line argument, and load_container_filters.go now uses the results from that
Changed init() to InitializeTopologies()
Changed init() to InitializeTopologies() so that it can be called after the container filters are loaded from the command line argument. init() executes before main() in prog/main.go, so the flag parsing isn't finished before init() is called
Applied lint fixes
fixed lint issues
brought back the init function for api_topologies.go
Addressed many of the PR comments, except escaping colons
Renamed IsDesired to HasLabel in render/filters.go
Allows for the user to escape colons
added registry function for modifying the container filters
created a separate function that parses the container filter flags
simplified registry.addContainerFilters()
addressed review comments
switched API Topology Description IDs to constants
addressed review comments
joined constants
added test functions
addressed most of the review comments
Changed containerLabelFilters to an array of APItopologyOptions, placing the parsing in the Set() function. Removed parsing from HasLabel in render/filters.go
refactored code
added test that applies to the container filtering by labels
applied golint
made Registry items private and added a MakeRegistry() function
fixed usage of topologyRegistry.RendererForTopology
Added container label filters by exclusion
minor update to report_fixture
Modified container labels test to use existing report
I added labels to the existing containers in the fixed report for testing.
refactored code
refactored code
further code refactoring
addressed @ijsnellf's review comments
unexported Registry, and reduced duplicate code
addressed @ijsnellf's review comments
Addressed review comments
Addressed final review comments
* Add filters for pseudo nodes.
- Don't filter the internet node as a pseudo node.
- Rename pseudo filter to unmanaged/uncontained.
- Review feedback
- Move the FilterFoo funcs into the tests
- Drop the 'nodes' from filter labels.
* Fix experimental
Squash of:
* We have to keep all the container hostnames until the end so we can
count how many we've filtered
* Adding tests for ContainerHostnameRenderer and PodServiceRenderer with
filters
* Because we filter on image name we need the image name before
filtering
* Alternative approach to passing decorators.
* Refactor out some of the decorator capture
* Don't memoise decorated calls to Render
* Fixing filtered counts on containers topology
Tricky, because we need the filters to be silent sometimes (when they're
in the middle), but not when they're at the top, so we take the "top"
filter's stats. However, this means we have to compose all
user-specified filters into a single Filter layer, so we can get all
stats.
There are no more Silent filters, as all filters are silent (unless they
are at the top).
Additionally, I clarified some of the filters as their usage/terminology
was inconsistent and confused. Now Filter(IsFoo, ...) *keeps* only nodes
where IsFoo is true.
