65 Commits

Author SHA1 Message Date
Paul Bellamy
1b39659d10 Merge pull request #742 from weaveworks/284-profiling
Various CPU usage gains from profiling
2015-12-11 12:05:24 +00:00
Tom Wilkie
5050bacd72 Correctly wait for all background goroutines to finish before returning from AppClient.Close() 2015-12-11 11:20:53 +00:00
Tom Wilkie
9cff5699cb Don't decode the certificates everytime we create an AppClient. 2015-12-11 10:27:26 +00:00
Tom Wilkie
0b1a2ef5c7 Wait for pipes to close in Close(). 2015-12-10 15:44:50 +00:00
Tom Wilkie
ff3748866b Fix infinite loop when closing AppClient. 2015-12-10 12:53:31 +00:00
Tom Wilkie
ac9c011475 Pipe plumbing
- Add store of pipes in the app
- Add pipe type, handling impedance mismatch, used in app and probe.
- App <-> Probe pipes have their own websockets.
- Add pipe websocket endpoint in app.
- Pipe IDs are strings, lose the request/response IDs, and give the json encoder lowercase field names.
- Add simple golang ws client, for testing.
- Pipe lifecycle plumbing.
- Ref count and timeout both ends of pipes in the app
- Deal with POST /api/pipe/:pid?_method=delete
- Add end-to-end unit test for pipes.
- Add test for timing out pipes.
- Update go-docker client to tomwilkie/go-dockerclient
- Backend work for non-raw ttys
- Close pipes when they close themselves in the probe
- Ensure all http connections are done before returning from client.Stop()
2015-12-10 12:51:17 +00:00
Tom Wilkie
b85746704b Merge pull request #726 from weaveworks/630-merge-appclient
Merge http publisher and app client.
2015-12-07 11:38:11 +00:00
Tom Wilkie
cb8ae536de Review feedback 2015-12-07 11:38:00 +00:00
Tom Wilkie
f1d225c836 Add hostname to /api 2015-12-04 15:33:12 +01:00
Tom Wilkie
563a6e06ba Extend the testing of the AppClient. 2015-12-04 13:03:48 +00:00
Tom Wilkie
4f4d986571 Merge http publish and app client. 2015-12-04 12:17:21 +00:00
Paul Bellamy
92ec7d9397 Move probe main.go to prog/probe/, break out a probe struct with appropriate responsibilities.
Also adds test for probe 'engine'
2015-11-09 16:25:28 +00:00
Tom Wilkie
76d34330a9 Add Control plumbing. 2015-11-06 17:39:48 +00:00
Tom Wilkie
1429d749d9 Rename semaphore v/p to release/acquire. 2015-11-05 15:43:42 +00:00
Paul Bellamy
9ea156cfef Moving xfer.Collector into App (the only place its used) 2015-11-05 11:31:48 +00:00
Tom Wilkie
920250e414 Close idle http connections when we stop a Http Publisher. 2015-10-29 12:14:17 +00:00
Paul Bellamy
650f015421 Review feedback 2015-10-27 11:36:14 +00:00
Paul Bellamy
e41352a220 pass the hostname through to xfer.HttpPublisher, so we can verify ssl certs 2015-10-27 10:51:41 +00:00
Tom Wilkie
0288b3151b Use gocertifi instead of random ca-certs. 2015-10-23 16:35:41 +00:00
Paul Bellamy
a1466cb3fc Review Feedback 2015-10-20 14:26:27 +01:00
Paul Bellamy
986abd24cd Add ssl support for the probe
Hosts ending in :443 are treated as SSL. There is also a
--probe.insecure flag, for using it in development environments where
SSL cannot be verified.
2015-10-19 17:32:32 +01:00
Paul Bellamy
637cb23ba8 Send the service credentials to initial /api request 2015-10-19 10:29:05 +01:00
Tom Wilkie
af4a35c67d Don't leak goroutines in multi publisher; fixes #531 2015-09-30 11:55:55 +00:00
Peter Bourgon
b78be6a71b xfer: fix TestBackgroundPublisher 2015-09-29 13:32:52 +02:00
Tom Wilkie
8265abad8b Test for BackgroundPublisher 2015-09-25 09:09:31 +00:00
Peter Bourgon
8602132ab6 Move concurrency from resolve to HTTP GET
- Process DNS resolution serially
- Process up to 10 HTTP GET (for app ID) concurrently

More than 10 concurrent GET requests will block on the semaphore. This
will cause the staticResolver.resolve method to block, which is probably
fine: it will just delay the next resolve loop, currently at 1m
intervals.

To make this a little bit more robust, I've also added a fastClient for
app ID resolution, with a timeout (total, including connect, request,
and response) of 5s.
2015-09-24 16:56:37 +02:00
Peter Bourgon
790da39f04 Publish an io.Reader, not a bytes.Buffer 2015-09-24 16:11:55 +02:00
Peter Bourgon
eca45ca9d5 Need to copy the buffer in the MultiPublisher 2015-09-24 16:11:55 +02:00
Peter Bourgon
c818f08c06 Refactor MultiPublisher
- Set instead of Add, to allow replacement of endpoints
- Break out individual Publishers to their own files and tests
2015-09-24 16:11:55 +02:00
Peter Bourgon
64fdf6a780 common/sanitize + relevant updates 2015-09-24 16:11:54 +02:00
Tom Wilkie
792e274745 Audit uses to Topology.Nodes, use AddNode where appropriate. 2015-09-23 07:36:51 +00:00
Tom Wilkie
5e0014e344 Remove reference counted buffers, rely on GC. 2015-09-16 04:45:10 +00:00
Peter Bourgon
65b78206ee xfer: move Buffer to own file; update comment
overlay: mutex for Weave status
2015-09-11 10:12:14 +02:00
Tom Wilkie
bb20a81338 Only serialise and compress reports once per publish. 2015-09-09 15:08:55 +00:00
Tom Wilkie
d8ac330589 Backoff on failed report pushes. 2015-09-09 09:56:39 +00:00
Tom Wilkie
b2c271c511 Push reports to the app in the background and in parallel. 2015-09-09 09:39:28 +00:00
Tom Wilkie
57f7b4f2e6 Rename NodeMetadata -> Node 2015-09-03 16:18:54 +00:00
Peter Bourgon
44e6fd2adb Merge pull request #399 from weaveworks/topology-copy
Moar functional: Topology Merge methods don't mutate the receiver
2015-08-27 17:05:12 +02:00
Peter Bourgon
8f55c74b27 xfer: fixes for new merge semantics 2015-08-26 12:02:11 +01:00
Alvaro Saurin
b60c39b7de GZip compression for app HTTP endpoints and probe -> app communication. 2015-08-26 09:56:14 +00:00
Peter Bourgon
43bfecbba4 Add X-Scope-Probe-ID header to POSTs 2015-08-13 13:01:58 +02:00
Peter Bourgon
edf0ceb212 Authorization: Scope-Probe token=<token> 2015-08-07 18:52:16 +02:00
Peter Bourgon
7d51f173ea Probes POST to apps
- App takes POST report on /api/report
- Probe publishes to configured target(s)
- Name resolution happens on probe-side
- There's no longer an xfer.ProbePort
- xfer.Collector responsibility is reduced
- Fixes to remaining experimental components.
- rm experimental/bridge: it's not being used, and by changing the
  app/probe comm model, it would require a complete refactor anyway. We
  can easily rebuild it when we need to. It will even be much simpler.
- rm experimental/graphviz: it's broken for some time anyway, and we
  don't really need to play around with it as a rendering option
  anymore.
- rm experimental/oneshot: we never use this anymore.
2015-08-07 15:45:15 +02:00
Peter Bourgon
3dd59c8b9b Fixes to NodeMetadata
NewNodeMetadata -> MakeNodeMetadata. It doesn't return a pointer, so
Make is more idiomatic.

Invoke MakeNodeMetadata when necessary. The zero value for a
NodeMetadata is no longer valid.

Split MakeNodeMetadata to two constructors. MakeNodeMetadata when you
don't have anything to prepopulate; MakeNodeMetadataWith when you do.

Also, a fix to the tests in app. We unmarshal a RenderableNode struct,
which has a JSON-ignored NodeMetadata field. The zero value is invalid,
so we need to fix that before performing comparisons.
2015-07-30 17:20:44 +02:00
Tom Wilkie
f85195308c report.Metadata -> map[string]string 2015-07-16 12:06:05 +00:00
Tom Wilkie
c2065836e2 Make NodeMetadata a struct, move existing map[string]string to NodeMetadata.Metadata 2015-07-15 10:06:36 +00:00
Tom Wilkie
9fc02d941e Fix flaky tests
- Make poll take interfaces, do diff on error
- Use poll in TestRegistryEvents
- Improve the locking to prevent deadlocks and data races in registry_test.go
2015-07-08 13:54:09 +00:00
Tom Wilkie
d2d73c3cd4 Add -cpu 4 to tests. 2015-07-06 15:07:33 +00:00
Peter Bourgon
90a0286909 Refactor tests to have appropriate packages
By default, tests should be in package pkg_test. If they need to test
package internals, they can be in package pkg, but then should carry a
suffix of foo_internal_test.go.

This changeset enforces that idiom across the codebase, and adds a check
to the linter to make sure it remains.

Also, some fixes to comments.
2015-06-29 18:06:44 +02:00
Tom Wilkie
175c488d74 Increase timeouts to prevent flakiness. 2015-06-24 10:03:26 +00:00