9920c4ea48
Based on work from Lorenzo, updated by Iago, Alban, Alessandro and
Michael.
This PR adds connection tracking using eBPF. This feature is not enabled by default.
For now, you can enable it by launching scope with the following command:
```
sudo ./scope launch --probe.ebpf.connections=true
```
This patch allows scope to get notified of every connection event,
without relying on the parsing of /proc/$pid/net/tcp{,6} and
/proc/$pid/fd/*, and therefore improve performance.
We vendor https://github.com/iovisor/gobpf in Scope to load the
pre-compiled ebpf program and https://github.com/weaveworks/tcptracer-bpf
to guess the offsets of the structures we need in the kernel. In this
way we don't need a different pre-compiled ebpf object file per kernel.
The pre-compiled ebpf program is included in the vendoring of
tcptracer-bpf.
The ebpf program uses kprobes/kretprobes on the following kernel functions:
- tcp_v4_connect
- tcp_v6_connect
- tcp_set_state
- inet_csk_accept
- tcp_close
It generates "connect", "accept" and "close" events containing the
connection tuple but also pid and netns.
Note: the IPv6 events are not supported in Scope and thus not passed on.
probe/endpoint/ebpf.go maintains the list of connections. Similarly to
conntrack, it also keeps the dead connections for one iteration in order
to report short-lived connections.
The code for parsing /proc/$pid/net/tcp{,6} and /proc/$pid/fd/* is still
there and still used at start-up because eBPF only brings us the events
and not the initial state. However, the /proc parsing for the initial
state is now done in foreground instead of background, via
newForegroundReader().
NAT resolution on connections from eBPF works in the same way as it did
on connections from /proc: by using conntrack. One of the two conntrack
instances is only started to get the initial state and then it is
stopped since eBPF detects short-lived connections.
The Scope Docker image size comparison:
- weaveworks/scope in current master: 22 MB (compressed), 68 MB
(uncompressed)
- weaveworks/scope with this patchset: 23 MB (compressed), 69 MB
(uncompressed)
Fixes #1168 (walking /proc to obtain connections is very expensive)
Fixes #1260 (Short-lived connections not tracked for containers in
shared networking namespaces)
Fixes #1962 (Port ebpf tracker to Go)
Fixes #1961 (Remove runtime kernel header dependency from ebpf tracker)
Weave Scope - Troubleshooting & Monitoring for Docker & Kubernetes
Weave Scope automatically generates a map of your application, enabling you to intuitively understand, monitor, and control your containerized, microservices based application.
Understand your Docker containers in real-time
Choose an overview of your container infrastructure, or focus on a specific microservice. Easily identify and correct issues to ensure the stability and performance of your containerized applications.
Contextual details and deep linking
View contextual metrics, tags and metadata for your containers. Effortlessly navigate between processes inside your container to hosts your containers run on, arranged in expandable, sortable tables. Easily to find the container using the most CPU or memory for a given host or service.
Interact with and manage containers
Interact with your containers directly: pause, restart and stop containers. Launch a command line. All without leaving the scope browser window.
Getting started
sudo curl -L git.io/scope -o /usr/local/bin/scope
sudo chmod a+x /usr/local/bin/scope
scope launch
This script downloads and runs a recent Scope image from Docker Hub.
Now, open your web browser to http://localhost:4040. (If you're using
boot2docker, replace localhost with the output of boot2docker ip.)
For instructions on installing Scope on Kubernetes, DCOS or ECS, see the docs.
Getting help
If you have any questions about, feedback for or problems with Scope:
- Read the Weave Scope docs.
- Invite yourself to the #weave-community slack channel.
- Ask a question on the #weave-community slack channel.
- Join the Weave User Group and get invited to online talks, hands-on training and meetups in your area.
- Send an email to weave-users@weave.works
- File an issue.
Your feedback is always welcome!