To facilitate this, we replace the existing functionality of updateFilters which
sets k8s topologies to have the filters [namespace, managed], to instead append the namespace filter
to any existing. This lets it apply to both k8s and container topologies without overwriting existing
container filters. We instead set the managed filter in the static definition.
This however has the side effect that the ordering of the namespace filter and the managed filter
in k8s topologies has been reversed, so it reads:
Show Unmanaged | Hide Unmanaged
foo | bar | default | baz | All Namespaces
instead of:
foo | bar | default | baz | All Namespaces
Show Unmanaged | Hide Unmanaged
`launch` first does a `scope --dry-run` to parse and validate Scope
cmdline arguments before starting the main Scope container and
processes. Use the same Docker args for the dry run to make sure the
container has the same privileges and mount points. Otherwise valid
`$WEAVESCOPE_DOCKER_ARGS` might fail here.
Based on work from Lorenzo, updated by Iago, Alban, Alessandro and
Michael.
This PR adds connection tracking using eBPF. This feature is not enabled by default.
For now, you can enable it by launching scope with the following command:
```
sudo ./scope launch --probe.ebpf.connections=true
```
This patch allows scope to get notified of every connection event,
without relying on the parsing of /proc/$pid/net/tcp{,6} and
/proc/$pid/fd/*, and therefore improve performance.
We vendor https://github.com/iovisor/gobpf in Scope to load the
pre-compiled ebpf program and https://github.com/weaveworks/tcptracer-bpf
to guess the offsets of the structures we need in the kernel. In this
way we don't need a different pre-compiled ebpf object file per kernel.
The pre-compiled ebpf program is included in the vendoring of
tcptracer-bpf.
The ebpf program uses kprobes/kretprobes on the following kernel functions:
- tcp_v4_connect
- tcp_v6_connect
- tcp_set_state
- inet_csk_accept
- tcp_close
It generates "connect", "accept" and "close" events containing the
connection tuple but also pid and netns.
Note: the IPv6 events are not supported in Scope and thus not passed on.
probe/endpoint/ebpf.go maintains the list of connections. Similarly to
conntrack, it also keeps the dead connections for one iteration in order
to report short-lived connections.
The code for parsing /proc/$pid/net/tcp{,6} and /proc/$pid/fd/* is still
there and still used at start-up because eBPF only brings us the events
and not the initial state. However, the /proc parsing for the initial
state is now done in foreground instead of background, via
newForegroundReader().
NAT resolution on connections from eBPF works in the same way as it did
on connections from /proc: by using conntrack. One of the two conntrack
instances is only started to get the initial state and then it is
stopped since eBPF detects short-lived connections.
The Scope Docker image size comparison:
- weaveworks/scope in current master: 22 MB (compressed), 68 MB
(uncompressed)
- weaveworks/scope with this patchset: 23 MB (compressed), 69 MB
(uncompressed)
Fixes#1168 (walking /proc to obtain connections is very expensive)
Fixes#1260 (Short-lived connections not tracked for containers in
shared networking namespaces)
Fixes#1962 (Port ebpf tracker to Go)
Fixes#1961 (Remove runtime kernel header dependency from ebpf tracker)
9e32194 Secure GCP servers for Scope: open port 80.
a22536a Secure GCP servers for Scope.
89c3a29 Merge pull request #78 from weaveworks/lint-merge-rebase-issue-in-docs
73ad56d Add linter function to avoid bad merge/rebase artefact
git-subtree-dir: tools
git-subtree-split: 9e32194cf16f8f6eee7acb5dbf311c9003f0fedb
* Add options to hide args and env vars
To allow for use of weave-scope in an unauthenticated environment,
add options to the probe to hide comand line arguments and
environment variables, which might contain secret data.
Fixes#2222
* Change docker.NewRegistry arguments to be a struct
* Remove redundant declarations of default values
* Move registry options outside to improve readability
Merge all reports received within a specified interval, discarding the
originals. This improves performance of Report() on repeated
invocation since it ends up merging fewer reports.
For example, if reports are received every second (e.g. 3 probes
reporting at the default probe.publishInterval of 3s), and the
app.windows is 15s (the default) and the report generation interval is
5s (e.g. one UI accessing every TOPOLOGY_INTERVAL; the default), then
the original code would have to merge 15 reports per UI access,
whereas this code will only need to merge 8 to 9 reports (3-4 merged
reports from previous invocation plus 5 recently received reports).
