This is a small program used to watch app and probe running inside
the container. We need to go round a few houses to be able to build
it from vendor directory.
Stop baking the toolchain and dependencies into the build image.
Instead, run the install step each time, but use volume mounts or
CircleCI caching to keep the happy path fast.
Previously, some parts of the client (UI) directory were baked into the build
image, and some parts were mounted or copied into the build environment.
As a result, files baked into the build image require a two-step
update for changes to take effect in CI.
Now, for dockerised builds, we pre-install very little into the build
image and mount the whole directory into the build environment.
However, we do overlay a volume on the node_modules folder to allow the
standard build toolchain to be separate from the host build toolchain.
Non-dockerised builds (e.g. CI) are now more similar to the dockerised
versions.
We needed it because some of our invocations of Weave Net's `weave`
script depended on it, but that is no longer the case as of Weave Net
2.1.
Fixes#2974.
When you run the client locally with `yarn start` and modify the
app/probe code you don't want to compile the client on `make prog/scope`.
This allows you to do `SCOPE_SKIP_UI_ASSETS=true make prog/scope` to
only build the scope binary.
It seems that on my OS the passed param to find gets expanded early and
thus the command
$(shell find ./ -path ./vendor -prune -o -type f -name *.go)
results in
./test.go ./vendor
instead of including all the go files from subdirs. Quoting helps.
'codecgen' embeds a random integer in each identifier; this means code
coverage across different CircleCI lanes may not match.
Here we force the integer to 23 on every CircleCI build so they always match.
Based on work from Lorenzo, updated by Iago, Alban, Alessandro and
Michael.
This PR adds connection tracking using eBPF. This feature is not enabled by default.
For now, you can enable it by launching scope with the following command:
```
sudo ./scope launch --probe.ebpf.connections=true
```
This patch allows scope to get notified of every connection event,
without relying on the parsing of /proc/$pid/net/tcp{,6} and
/proc/$pid/fd/*, and therefore improve performance.
We vendor https://github.com/iovisor/gobpf in Scope to load the
pre-compiled ebpf program and https://github.com/weaveworks/tcptracer-bpf
to guess the offsets of the structures we need in the kernel. In this
way we don't need a different pre-compiled ebpf object file per kernel.
The pre-compiled ebpf program is included in the vendoring of
tcptracer-bpf.
The ebpf program uses kprobes/kretprobes on the following kernel functions:
- tcp_v4_connect
- tcp_v6_connect
- tcp_set_state
- inet_csk_accept
- tcp_close
It generates "connect", "accept" and "close" events containing the
connection tuple but also pid and netns.
Note: the IPv6 events are not supported in Scope and thus not passed on.
probe/endpoint/ebpf.go maintains the list of connections. Similarly to
conntrack, it also keeps the dead connections for one iteration in order
to report short-lived connections.
The code for parsing /proc/$pid/net/tcp{,6} and /proc/$pid/fd/* is still
there and still used at start-up because eBPF only brings us the events
and not the initial state. However, the /proc parsing for the initial
state is now done in foreground instead of background, via
newForegroundReader().
NAT resolution on connections from eBPF works in the same way as it did
on connections from /proc: by using conntrack. One of the two conntrack
instances is only started to get the initial state and then it is
stopped since eBPF detects short-lived connections.
The Scope Docker image size comparison:
- weaveworks/scope in current master: 22 MB (compressed), 68 MB
(uncompressed)
- weaveworks/scope with this patchset: 23 MB (compressed), 69 MB
(uncompressed)
Fixes#1168 (walking /proc to obtain connections is very expensive)
Fixes#1260 (Short-lived connections not tracked for containers in
shared networking namespaces)
Fixes#1962 (Port ebpf tracker to Go)
Fixes#1961 (Remove runtime kernel header dependency from ebpf tracker)
It's just not worth the extra complication in the Makefile, and the
code for it was actually broken.
Also, outdent comment so it doesn't get splattered onto the console.
When the directory is created during the build process, it is as root.
This prevents us deleting root-owned files inside the directory during make clean.
In client/build/, we get around this because the directory contains a tracked file,
so it's created during git checkout.
We make this the case for build-external by adding a hidden, empty file to track.
This won't fix existing checkouts though, so we also add a fix-up step to make clean.
The use of .gitignore as the empty file to track is taken from
https://git.wiki.kernel.org/index.php/GitFaq#Can_I_add_empty_directories.3F
Specifically:
"If you really need a directory to exist in checkouts you should create a file in it.
.gitignore works well for this purpose"
