But it leads to another issues - problems with importing https://github.com/hashicorp/hcl/hcl/printers. HCL was released in version 2 so that the file structure was changes.
I do not see any way to install it in changed Makefile. Go installs does not support pining to given tags so that always the newest version is installed.
For now, the impact on build process should be minimal. it is used only in ./tools/lint/lint() method. I was able to execute it locally without executing hclfmt.
1)backend/Dockerfile 2) probe/endpoint/dns_snooper.go
3) client/Dockerfile 4) docker/Dockerfile.cloud-agent
5) probe/process/walker_linux_test.go & 6) tools/lint
1)'backend/Dockerfile' : Conditional added so that the cross-compiling should
be done on amd64. Also removed support for sh-lint for ppc64le for now.
As the version for shfmt mentioned in the dockerfile is not available for
ppc64le and the later version does't work fine with existing application.
2)'probe/endpoint/dns_snooper.go' : Renamed this file so as to reuse for ppc64le
and added a build-constraint. Now this file will be build for amd64 on linux
and ppc64le on linux.
3)'client/Dockerfile' : Modified the version of the base image for node from
8.4.0 to 8.11, as this version supports multiarch.
4)'docker/Dockerfile.cloud-agent' : Modified the version of the base image for
golang from 1.10.2-strech to 1.10.2, which supports multiarch.
5) 'probe/process/walker_linux_test.go' : Test fixed to run for ppc64le,
modified the code to accept RSSBytes based on pageSize value per
architecture, instead of hard-coded values.
6)'tools/lint' : Modified the file to skip the sh-lint implementation for ppc64le.
PR #3231
Based on work from Lorenzo, updated by Iago, Alban, Alessandro and
Michael.
This PR adds connection tracking using eBPF. This feature is not enabled by default.
For now, you can enable it by launching scope with the following command:
```
sudo ./scope launch --probe.ebpf.connections=true
```
This patch allows scope to get notified of every connection event,
without relying on the parsing of /proc/$pid/net/tcp{,6} and
/proc/$pid/fd/*, and therefore improve performance.
We vendor https://github.com/iovisor/gobpf in Scope to load the
pre-compiled ebpf program and https://github.com/weaveworks/tcptracer-bpf
to guess the offsets of the structures we need in the kernel. In this
way we don't need a different pre-compiled ebpf object file per kernel.
The pre-compiled ebpf program is included in the vendoring of
tcptracer-bpf.
The ebpf program uses kprobes/kretprobes on the following kernel functions:
- tcp_v4_connect
- tcp_v6_connect
- tcp_set_state
- inet_csk_accept
- tcp_close
It generates "connect", "accept" and "close" events containing the
connection tuple but also pid and netns.
Note: the IPv6 events are not supported in Scope and thus not passed on.
probe/endpoint/ebpf.go maintains the list of connections. Similarly to
conntrack, it also keeps the dead connections for one iteration in order
to report short-lived connections.
The code for parsing /proc/$pid/net/tcp{,6} and /proc/$pid/fd/* is still
there and still used at start-up because eBPF only brings us the events
and not the initial state. However, the /proc parsing for the initial
state is now done in foreground instead of background, via
newForegroundReader().
NAT resolution on connections from eBPF works in the same way as it did
on connections from /proc: by using conntrack. One of the two conntrack
instances is only started to get the initial state and then it is
stopped since eBPF detects short-lived connections.
The Scope Docker image size comparison:
- weaveworks/scope in current master: 22 MB (compressed), 68 MB
(uncompressed)
- weaveworks/scope with this patchset: 23 MB (compressed), 69 MB
(uncompressed)
Fixes#1168 (walking /proc to obtain connections is very expensive)
Fixes#1260 (Short-lived connections not tracked for containers in
shared networking namespaces)
Fixes#1962 (Port ebpf tracker to Go)
Fixes#1961 (Remove runtime kernel header dependency from ebpf tracker)
After updating build-tools, I got new errors:
> ./tools/lint: line 134: hclfmt: command not found
> go install net: open /usr/local/go/pkg/linux_amd64_race/net.a:
> permission denied
This patch works around that by doing the same thing as:
https://github.com/weaveworks/build-tools/pull/44/files
- Ensure we use up to date build container when running tests.
- Parallelise host setup for integration tests.
- Pass some more circle env vars through to ./tools/tests so the scheduler works.
- Gather coverage from multiple shards.
- Create files as the right owner inside the container.
- Parallelise the circle.yaml
- Parallelise the intergration tests using existing machinery
- Build experimental in parallel.
- Use custom scheduler prefix for integration tests, so schedules don't clash with weave.
