154 Commits

Author SHA1 Message Date
Peter Bourgon
36b743fe1f probe, app: reliable shutdown
- Restructure main funcs for clean defer-stack-unwinds
- Fix Docker container to handle signals properly
- Introduce runsvinit for container init process
- Integration test
2015-09-29 11:45:12 +02:00
Peter Bourgon
8602132ab6 Move concurrency from resolve to HTTP GET
- Process DNS resolution serially
- Process up to 10 HTTP GET (for app ID) concurrently

More than 10 concurrent GET requests will block on the semaphore. This
will cause the staticResolver.resolve method to block, which is probably
fine: it will just delay the next resolve loop, currently at 1m
intervals.

To make this a little bit more robust, I've also added a fastClient for
app ID resolution, with a timeout (total, including connect, request,
and response) of 5s.
2015-09-24 16:56:37 +02:00
Peter Bourgon
eccc74aafe Make the resolver concurrent 2015-09-24 16:11:55 +02:00
Peter Bourgon
c818f08c06 Refactor MultiPublisher
- Set instead of Add, to allow replacement of endpoints
- Break out individual Publishers to their own files and tests
2015-09-24 16:11:55 +02:00
Peter Bourgon
64fdf6a780 common/sanitize + relevant updates 2015-09-24 16:11:54 +02:00
Tom Wilkie
92b5b639dd Extend docker module testing. 2015-09-24 05:30:40 +00:00
Tom Wilkie
487a97e9f2 Merge pull request #516 from weaveworks/extract-capture
probe/sniff: move to experimental
2015-09-24 12:25:38 +08:00
Tom Wilkie
e2dfcb1def Review feedback 2015-09-24 03:07:20 +00:00
Peter Bourgon
c607ee00d8 probe/sniff: move to experimental
Resolves #466
2015-09-23 14:38:41 +02:00
Tom Wilkie
7da599194d Test the nat mapper. 2015-09-23 11:01:29 +00:00
Tom Wilkie
f9815afad3 Remove some duplicate functionality, add some basic tests. 2015-09-23 09:46:11 +00:00
Tom Wilkie
233b29d822 Merge pull request #513 from weaveworks/416-audit-add-node-usage
Audit uses to Topology.Nodes, use AddNode where appropriate.
2015-09-23 16:02:10 +08:00
Tom Wilkie
792e274745 Audit uses to Topology.Nodes, use AddNode where appropriate. 2015-09-23 07:36:51 +00:00
Tom Wilkie
b9ffca42e5 Don't fail if there are no existing nat mappings. 2015-09-23 03:35:53 +00:00
Tom Wilkie
fcb7c47bd9 Always include endpoint nodes brought in by procspy, even if they get merged with conntracked endpoints. 2015-09-16 03:46:15 +00:00
Tom Wilkie
54299dc0fd Review feedback. 2015-09-15 10:03:06 +00:00
Tom Wilkie
b7598c376e Don't show shortlived connections in the host view. 2015-09-15 10:03:04 +00:00
Tom Wilkie
89b5a8d432 We don't know which end is local for shortlived connections, so don't tag any with the hostid. 2015-09-15 10:03:03 +00:00
Tom Wilkie
766d8772d7 Do a conntrack -L before -E to capture existing connections and NAT mappings. 2015-09-15 10:02:59 +00:00
Tom Wilkie
925851169e Use AddNode when adding nodes to the topology, such that the get merged with any existing node that might be there. 2015-09-15 10:02:58 +00:00
Tom Wilkie
7d6b5a540e Revert "Merge pull request #450 from weaveworks/scope-284"
This reverts commit 46345e3bdb, reversing
changes made to bf3e9a1601.

Conflicts:
	probe/process/walker_darwin.go
2015-09-15 03:14:39 +00:00
Peter Bourgon
1001f550c0 Fix bad merge on Darwin 2015-09-14 15:55:38 +02:00
Alvaro Saurin
a7dc0ac206 Use a mutex in the /proc files cache
Some minor renames and cleanups.
2015-09-14 11:14:25 +02:00
Alvaro Saurin
026d8e9065 Minor: refactored readerLinux/readIntoBuffer() for removing a parameter 2015-09-14 10:59:23 +02:00
Alvaro Saurin
2f9f54688f Keep a cache of open files, reducing the number of open/close cycles in the /proc dir 2015-09-14 10:59:23 +02:00
Alvaro Saurin
14dc6a5391 Cache processes and connections when reading from the /proc
Code cleanups
2015-09-14 10:53:24 +02:00
Alvaro Saurin
4331fb1c79 Drop all the dependencies we had on procspy by moving some code to probe. 2015-09-14 10:44:53 +02:00
Tom Wilkie
2fdfa1cbf5 Fix flaky docker tests. 2015-09-14 05:09:33 +00:00
Tom Wilkie
c9f39d4129 Address one intermittent failure in registry_test.go 2015-09-14 04:57:56 +00:00
Peter Bourgon
65b78206ee xfer: move Buffer to own file; update comment
overlay: mutex for Weave status
2015-09-11 10:12:14 +02:00
Tom Wilkie
a8c163b5d7 Remove O(n^2) behaviour in weave tagger. 2015-09-09 15:09:27 +00:00
Tom Wilkie
bb20a81338 Only serialise and compress reports once per publish. 2015-09-09 15:08:55 +00:00
Tom Wilkie
d5570f27a7 Only conntrack tcp connections to reduce cpu load from xml marshalling. 2015-09-09 15:08:54 +00:00
Tom Wilkie
b7c22b7a8f Only fetch weave status report once per tick. 2015-09-09 15:08:43 +00:00
Tom Wilkie
5bd324db3f Generate reports in parallel (NB this doesn't actually seem to be worth it.) 2015-09-09 15:05:05 +00:00
Tom Wilkie
d411afd916 WithNode is a CPU hog. 2015-09-09 15:01:57 +00:00
Tom Wilkie
953106f80f Warn when generating the report takes too long. 2015-09-09 15:01:48 +00:00
Tom Wilkie
b2c271c511 Push reports to the app in the background and in parallel. 2015-09-09 09:39:28 +00:00
Tom Wilkie
9ab98baafc Merge pull request #451 from weaveworks/reverse-resolver-fix
Small improvements to reverse resolver
2015-09-07 11:43:47 +01:00
Tom Wilkie
4da819b18c Merge pull request #449 from weaveworks/447-short-lived-in-apps
Prevent short lived connections appearing as pseudo nodes in the applications view.
2015-09-07 11:32:38 +01:00
Tom Wilkie
f5d6872c51 Don't rate limit requests that have already been resolved, and cache negative responses. 2015-09-07 10:30:16 +00:00
Peter Bourgon
7513b4e396 Wrap comments at 80col throughout the fileset 2015-09-07 10:36:35 +02:00
Tom Wilkie
85bfcddb3f Prevent short lived connections appearing as pseudo nodes in the applications view. 2015-09-06 16:26:57 +00:00
Tom Wilkie
ad6702a196 Some review feedback. 2015-09-05 19:39:22 +00:00
Alvaro Saurin
15e25edc40 New asynchronous, caching DNS resolver for reverse resolutions
Add nodes for the remote side of connections iff we have a DNS reverse resolution for the IP.
Unit test for the resolver
2015-09-04 17:21:45 +02:00
Tom Wilkie
57f7b4f2e6 Rename NodeMetadata -> Node 2015-09-03 16:18:54 +00:00
Tom Wilkie
9ab22bd8dc Wait longer for the goroutine to stop. 2015-09-03 08:33:15 +00:00
Tom Wilkie
144a798602 Review Feedback 2015-09-02 14:32:47 +00:00
Tom Wilkie
afc2788777 Move EdgeMetadata into the NodeMetadata struct. 2015-09-02 14:32:44 +00:00
Tom Wilkie
899e2d7a5c Review feedback 2015-09-01 11:01:25 +00:00