Commit Graph

709 Commits

Author SHA1 Message Date
Mike Lang
c149e5792a k8s probe: Fix a panic (nil pointer deref) when a cronjob has never been scheduled
in which case cj.Status.LastScheduled is nil.
New behaviour is to omit it from the map (and therefore the display) if it has never been scheduled.
2017-08-01 14:14:44 -07:00
Matthias Radestock
1e38e78518 Merge pull request #2779 from weaveworks/2748-synthesise-service-network
synthesise k8s service network from service IPs

Fixes #2748.
2017-08-01 13:17:27 +01:00
Matthias Radestock
4dae7edc9c synthesise k8s service network from service IPs
This prevents cluttering host.LocalNetworks with lots of /32
addresses. These were unsightly and rather distracting in the UI. They
also bloated the report and slowed down server-side rendering.

Fixes #2748.
2017-08-01 12:17:50 +01:00
Matthias Radestock
e77d40fc16 refactor: inline connectionTracker.performFlowWalk 2017-07-30 09:23:41 +01:00
Matthias Radestock
b93b19a7c7 refactor: simplify connection polarity reversal 2017-07-30 08:48:13 +01:00
Matthias Radestock
65cebed6c4 get rid of endpoint type indicators
The app stopped paying attention to these some time ago.

Removing them shrinks reports by 3-10%.
2017-07-30 08:38:56 +01:00
Alfonso Acosta
3e4b3cbbf5 Add pod restart count to details pane 2017-07-27 13:15:53 +00:00
Mike Lang
486bdcc796 k8s: Use 'DaemonSet', 'StatefulSet' etc instead of 'Daemon Set', 'Stateful Set'
We can't search for terms with spaces.
2017-07-26 13:49:54 -07:00
Matthias Radestock
3ab48bcdbe Merge pull request #2756 from weaveworks/2464-maximize-timeout
maximize report publishing timeout

Fixes #2464.
2017-07-26 11:41:48 +01:00
Matthias Radestock
d8c747ef20 downgrade "Dropping report" log message to warning
It indicates degraded functionality, not catastrophe.
2017-07-25 21:29:18 +01:00
Matthias Radestock
935f6e6c20 maximize report publishing timeout
If the app really does take a long time to process reports, it is
better not to time out and send it more reports. However, we do want
to send at least one report per app.window, otherwise the scope UI
will go blank.

Fixes #2464 (as much as is practically possible)
2017-07-25 21:20:39 +01:00
Matthias Radestock
a5a9180605 do not back off on timeouts when sending reports
...since doing so unnecessarily throttles report sending, to the point
where the app is receiving reports so infrequently that often it has
no data to show.

The timeout period itself is sufficient to prevent thrashing.

Fixes #2745.
2017-07-25 21:16:35 +01:00
Matthias Radestock
3c6ae972ab new full reports are more important than old and shortcut reports
so when there is backpressure in publishing reports, drop shortcut
reports in preference to full reports, and drop old full reports in
preference to new full reports.

Fixes #2738.
2017-07-24 22:19:27 +01:00
Matthias Radestock
9e6ecee37d optimisation: don't copy report stream unnecessarily
We don't need to copy from the reader when publishing to just one
destination.
2017-07-21 17:56:08 +01:00
Matthias Radestock
d368854b90 defend against nils
Fixes #2508. Hopefully.
2017-07-20 15:59:59 +01:00
Mike Lang
fe3bdbfcdc probe/kubernetes: Speed up lookups of large lists of cronjobs and jobs
Currently joining the two lists is O(mn), by putting into a hashmap first it's O(m+n)
2017-07-19 11:31:43 -07:00
Mike Lang
38814d54a6 deployment: Fix usage of Spec.Replicas, which is a pointer
Spec.Replicas is a *int32, with a value of nil occurring when the user doesn't set it.
In this case k8s defaults to 1, so we mimic this to show the effective value.
2017-07-18 11:35:50 -07:00
Mike Lang
481258d8fc kubernetes probe: Collect info on cronjobs and statefulsets
Most of the time you only care about cronjobs, not the jobs that make them up,
so we only collect full cronjob data. We associate pods of jobs with the parent cronjob
2017-07-18 11:35:50 -07:00
Alfonso Acosta
b7d292e161 Gather Weave Net plugin and proxy info from report
Instead of using Docker, because after Weave Net 2.0 there are no proxy nor
plugin containers.

This has the drawback of not detecting the plugin/proxy in systems running
Weave Net < 2.0 , but I think we can live with it.
2017-07-17 13:23:37 +00:00
Bryan Boreham
88ca9812b2 Fix up tests for change to NewReporter() 2017-07-13 16:24:17 +00:00
Bryan Boreham
3e9eb83d12 Use Kubernetes node name to filter pods if possible 2017-07-13 16:24:17 +00:00
Matthias Radestock
e603a28ca4 Merge pull request #2704 from weaveworks/2689-2700-ebpf-init
don't miss, or fail to forget, initial connections

Fixes #2689.
Fixes #2700.
2017-07-13 11:39:31 +01:00
Matthias Radestock
b087e95711 bump tcptracer-bpf version 2017-07-12 07:27:35 +01:00
Matthias Radestock
ebc3cddf01 don't miss, or fail to forget, initial connections
...when initialising eBPF-based connection tracking.

Previously we were ignoring all eBPF events until we had gathered the
existing connections. That means we could a) miss connections created
during the gathering, and b) fail to forget connections that got
closed during the gathering.

The fix comprises the following changes:

1. pay attention to eBPF events immediately. That way we do not
miss anything.

2. remember connections for which we received a Close event during the
initalisation phase, and subsequently drop gathered existing
connections that match these. That way we do not erroneously consider
a gathered connection as open when it got closed since the gathering.

3. drop gathered existing connections which match connections detected
through eBPF events. The latter typically have more / current
metadata. In particular, PIDs can be missing from the former.

Fixes #2689.
Fixes #2700.
2017-07-11 22:50:47 +01:00
Matthias Radestock
d568c50ec4 make EbpfTracker.dead go-routine-safe and .stop() idempotent
Without synchronisation, the isDead() call might return a stale value,
delaying deadness detection potentially indefinitely.

Without the guards / idempotence in .stop(), invoking stop() more than
once could cause a panic, since tracer.Stop() closes a channel (which
panics on a closed channel). Multiple stop() invocations are rare, but
not impossible.
2017-07-11 19:38:07 +01:00
Matthias Radestock
cf6353327a eliminate race in ebpf initialization
We were enabling event processing before feeding in the initial
connections, which results in a non-deterministic outcome.
2017-07-11 19:38:07 +01:00
Matthias Radestock
15215d0c2c prevent concurrent map access in ebpf fd install event handler
which presumably could cause havoc
2017-07-11 19:38:07 +01:00
Matthias Radestock
3883d8f1af fix a minor leak in ebfp fdinstall_pids table
when we got an fd install event but the pid was dead by time we
processed it, we would fail to remove the watcher for that pid from
the fdinstall_pids table.

This is a minor, and bounded, leak, since the table only contains pids
that were alive when we initialized ebpf. And this change only plugs
that leak very partially, since we will never remove pids that die
while sitting in accept().
2017-07-11 19:38:07 +01:00
Matthias Radestock
e2cbe7ac26 refactor: a bit of inlining 2017-07-11 19:38:06 +01:00
Matthias Radestock
3baeb3d238 refactor: use fourTuple as map key instead of string 2017-07-11 19:38:06 +01:00
Matthias Radestock
ad7b5cdc19 refactor: remove pointless interface
premature abstraction
2017-07-11 19:38:06 +01:00
Matthias Radestock
8a56540648 refactor: eliminate global var 2017-07-11 19:38:06 +01:00
Matthias Radestock
8bd0188537 respect UseConntrack setting in ebpf initialisation 2017-07-11 19:37:11 +01:00
Matthias Radestock
7ea0800f8b refactor: extract helper to get initial flows 2017-07-10 07:34:20 +01:00
Matthias Radestock
07e7adbd63 refactor: make performFlowWalk data flow more obvious 2017-07-10 07:22:12 +01:00
Matthias Radestock
19e45ec248 refactor: eliminate global var 2017-07-07 10:18:43 +01:00
Matthias Radestock
8cf79b2e4a bump tcptracer-bpf version and use it to fix race
We defer starting the ebpf tracer until we've set the global var which
is referenced by the callback functions. Previously the var could be
unset when the callbacks are invoked, resulting in a segfault.

Fixes #2687.
2017-07-07 06:56:28 +01:00
Matthias Radestock
f0ae2bd98c refactor: use inline StringSet constructor 2017-07-04 06:29:19 +01:00
Alfonso Acosta
6c03540b1f Merge pull request #2659 from weaveworks/use-new-k8s-go-client
Use new k8s go client
2017-07-03 23:23:41 +02:00
Alfonso Acosta
84afe9fe70 Fix typo 2017-07-03 20:20:28 +00:00
Alfonso Acosta
34bfc22b4f Fix tests 2017-07-03 20:20:28 +00:00
Alfonso Acosta
7d59936d8c HostNetwork is now inlined in the pod spec 2017-07-03 20:20:28 +00:00
Alfonso Acosta
8bbbf25809 Migrate probe to new new kubernetes go-client
This namely involved importing new libraries and using the new Clientset.

Changes worth mentioning:

* The new kubernetes library doesn't provide StoreToLister wrappers, so now I am going the casting directly.
* Deleting the pods and getting their logs is done in a cleaner way (using the
  Clientset instead of the lower-level RESTclient).
2017-07-03 20:20:27 +00:00
Matthias Radestock
d12603c516 tiny refactor: use inline string set constructor in test 2017-07-03 07:55:17 +01:00
Matthias Radestock
430e74a80a refactor: remove report latest map Delete()
It wasn't used, and is problematic in any case since it introduces
non-monotonicity.
2017-07-03 02:06:21 +01:00
Matthias Radestock
9dc50b5202 refactor: hide "empty set" constants
They are an implementation detail.
2017-07-03 01:26:22 +01:00
Roland Schilter
e6bc1d6ec2 Merge pull request #2649 from weaveworks/1975-honor-docker-host-in-probe
Honor DOCKER_* env variables in probe and app
2017-06-28 09:07:52 +02:00
Mike Lang
889972c48a Display node type on k8s controller nodes
Since there are multiple types in the same topology, displaying the type is important.
We do this in multiple places:

* Add node type to minor label

* Add node type as metadata and include in metadata template.
  Even though this will always be the same for every node of that topology, this was
  the easiest way to add it so it displays in the table view.
  Note we can't control ordering of columns in table view, it's always alphabetical.
2017-06-27 10:19:04 -07:00
Roland Schilter
651e52b5a5 Honor DOCKER_* env variables in probe and app
Changed default for flag `-app.docker` to use the DOCKER_* env variables
instead of hardcoded /var/run/docker.sock; uses docker's default if
no DOCKER_HOST defined, for both probe and app.

Fixes #1975
2017-06-27 17:14:49 +02:00
Matthias Radestock
286e481771 Merge pull request #2645 from weaveworks/2644-initial-ebpf-polarity
correct polarity of initial connections

Fixes #2644.
2017-06-26 09:10:46 +01:00