Commit Graph

54 Commits

Author SHA1 Message Date
Paul Bellamy
92ec7d9397 Move probe main.go to prog/probe/, break out a probe struct with appropriate responsibilities.
Also adds test for probe 'engine'
2015-11-09 16:25:28 +00:00
Tom Wilkie
76d34330a9 Add Control plumbing. 2015-11-06 17:39:48 +00:00
Tom Wilkie
1429d749d9 Rename semaphore v/p to release/acquire. 2015-11-05 15:43:42 +00:00
Paul Bellamy
9ea156cfef Moving xfer.Collector into App (the only place its used) 2015-11-05 11:31:48 +00:00
Tom Wilkie
920250e414 Close idle http connections when we stop a Http Publisher. 2015-10-29 12:14:17 +00:00
Paul Bellamy
650f015421 Review feedback 2015-10-27 11:36:14 +00:00
Paul Bellamy
e41352a220 pass the hostname through to xfer.HttpPublisher, so we can verify ssl certs 2015-10-27 10:51:41 +00:00
Tom Wilkie
0288b3151b Use gocertifi instead of random ca-certs. 2015-10-23 16:35:41 +00:00
Paul Bellamy
a1466cb3fc Review Feedback 2015-10-20 14:26:27 +01:00
Paul Bellamy
986abd24cd Add ssl support for the probe
Hosts ending in :443 are treated as SSL. There is also a
--probe.insecure flag, for using it in development environments where
SSL cannot be verified.
2015-10-19 17:32:32 +01:00
Paul Bellamy
637cb23ba8 Send the service credentials to initial /api request 2015-10-19 10:29:05 +01:00
Tom Wilkie
af4a35c67d Don't leak goroutines in multi publisher; fixes #531 2015-09-30 11:55:55 +00:00
Peter Bourgon
b78be6a71b xfer: fix TestBackgroundPublisher 2015-09-29 13:32:52 +02:00
Tom Wilkie
8265abad8b Test for BackgroundPublisher 2015-09-25 09:09:31 +00:00
Peter Bourgon
8602132ab6 Move concurrency from resolve to HTTP GET
- Process DNS resolution serially
- Process up to 10 HTTP GET (for app ID) concurrently

More than 10 concurrent GET requests will block on the semaphore. This
will cause the staticResolver.resolve method to block, which is probably
fine: it will just delay the next resolve loop, currently at 1m
intervals.

To make this a little bit more robust, I've also added a fastClient for
app ID resolution, with a timeout (total, including connect, request,
and response) of 5s.
2015-09-24 16:56:37 +02:00
Peter Bourgon
790da39f04 Publish an io.Reader, not a bytes.Buffer 2015-09-24 16:11:55 +02:00
Peter Bourgon
eca45ca9d5 Need to copy the buffer in the MultiPublisher 2015-09-24 16:11:55 +02:00
Peter Bourgon
c818f08c06 Refactor MultiPublisher
- Set instead of Add, to allow replacement of endpoints
- Break out individual Publishers to their own files and tests
2015-09-24 16:11:55 +02:00
Peter Bourgon
64fdf6a780 common/sanitize + relevant updates 2015-09-24 16:11:54 +02:00
Tom Wilkie
792e274745 Audit uses to Topology.Nodes, use AddNode where appropriate. 2015-09-23 07:36:51 +00:00
Tom Wilkie
5e0014e344 Remove reference counted buffers, rely on GC. 2015-09-16 04:45:10 +00:00
Peter Bourgon
65b78206ee xfer: move Buffer to own file; update comment
overlay: mutex for Weave status
2015-09-11 10:12:14 +02:00
Tom Wilkie
bb20a81338 Only serialise and compress reports once per publish. 2015-09-09 15:08:55 +00:00
Tom Wilkie
d8ac330589 Backoff on failed report pushes. 2015-09-09 09:56:39 +00:00
Tom Wilkie
b2c271c511 Push reports to the app in the background and in parallel. 2015-09-09 09:39:28 +00:00
Tom Wilkie
57f7b4f2e6 Rename NodeMetadata -> Node 2015-09-03 16:18:54 +00:00
Peter Bourgon
44e6fd2adb Merge pull request #399 from weaveworks/topology-copy
Moar functional: Topology Merge methods don't mutate the receiver
2015-08-27 17:05:12 +02:00
Peter Bourgon
8f55c74b27 xfer: fixes for new merge semantics 2015-08-26 12:02:11 +01:00
Alvaro Saurin
b60c39b7de GZip compression for app HTTP endpoints and probe -> app communication. 2015-08-26 09:56:14 +00:00
Peter Bourgon
43bfecbba4 Add X-Scope-Probe-ID header to POSTs 2015-08-13 13:01:58 +02:00
Peter Bourgon
edf0ceb212 Authorization: Scope-Probe token=<token> 2015-08-07 18:52:16 +02:00
Peter Bourgon
7d51f173ea Probes POST to apps
- App takes POST report on /api/report
- Probe publishes to configured target(s)
- Name resolution happens on probe-side
- There's no longer an xfer.ProbePort
- xfer.Collector responsibility is reduced
- Fixes to remaining experimental components.
- rm experimental/bridge: it's not being used, and by changing the
  app/probe comm model, it would require a complete refactor anyway. We
  can easily rebuild it when we need to. It will even be much simpler.
- rm experimental/graphviz: it's broken for some time anyway, and we
  don't really need to play around with it as a rendering option
  anymore.
- rm experimental/oneshot: we never use this anymore.
2015-08-07 15:45:15 +02:00
Peter Bourgon
3dd59c8b9b Fixes to NodeMetadata
NewNodeMetadata -> MakeNodeMetadata. It doesn't return a pointer, so
Make is more idiomatic.

Invoke MakeNodeMetadata when necessary. The zero value for a
NodeMetadata is no longer valid.

Split MakeNodeMetadata to two constructors. MakeNodeMetadata when you
don't have anything to prepopulate; MakeNodeMetadataWith when you do.

Also, a fix to the tests in app. We unmarshal a RenderableNode struct,
which has a JSON-ignored NodeMetadata field. The zero value is invalid,
so we need to fix that before performing comparisons.
2015-07-30 17:20:44 +02:00
Tom Wilkie
f85195308c report.Metadata -> map[string]string 2015-07-16 12:06:05 +00:00
Tom Wilkie
c2065836e2 Make NodeMetadata a struct, move existing map[string]string to NodeMetadata.Metadata 2015-07-15 10:06:36 +00:00
Tom Wilkie
9fc02d941e Fix flaky tests
- Make poll take interfaces, do diff on error
- Use poll in TestRegistryEvents
- Improve the locking to prevent deadlocks and data races in registry_test.go
2015-07-08 13:54:09 +00:00
Tom Wilkie
d2d73c3cd4 Add -cpu 4 to tests. 2015-07-06 15:07:33 +00:00
Peter Bourgon
90a0286909 Refactor tests to have appropriate packages
By default, tests should be in package pkg_test. If they need to test
package internals, they can be in package pkg, but then should carry a
suffix of foo_internal_test.go.

This changeset enforces that idiom across the codebase, and adds a check
to the linter to make sure it remains.

Also, some fixes to comments.
2015-06-29 18:06:44 +02:00
Tom Wilkie
175c488d74 Increase timeouts to prevent flakiness. 2015-06-24 10:03:26 +00:00
Peter Bourgon
49ec565a04 Improve error messages during report validation 2015-06-16 10:42:40 +02:00
Tom Wilkie
c5d10867c6 Add Validate method to Report & Topology. 2015-06-11 13:27:49 +00:00
Tom Wilkie
894439a449 Prevent multiple connections for the same app. 2015-06-11 10:02:54 +00:00
Tom Wilkie
16e1d3526b Too verbose 2015-06-10 10:15:43 +00:00
Peter Bourgon
d438261742 HostMetadata becomes Host Topology
- All HostMetadata information becomes NodeMetadata
- Significant change to mechanics, but same net effect
- LocalNets becomes "local_networks", space-separated list of CIDRs
- Load becomes simple single string
- Use MakeHostNodeID for indexing into Host topology
- (That changes the app /origin/{id} handler; will be removed later)
2015-06-09 15:38:08 +02:00
Peter Bourgon
1e0b83eb6d Merge branch report-fields (#190) 2015-06-09 14:25:54 +02:00
Tom Wilkie
20618a16c7 Start testing with the race detector. 2015-06-08 15:01:01 +00:00
Peter Bourgon
754eac3152 Rename NewReport -> MakeReport. 2015-06-05 13:55:40 +00:00
Peter Bourgon
06d805db3c Refactoring of xfer/collector.go
- Add interface to xfer/collector.go
- Add some more tests
- quit field changed to make it consistent with other Stoppable components in Scope
- Add peek, only used for tests
2015-06-04 10:27:30 +00:00
Peter Bourgon
36d04da82e type connEncoder can be function-scoped 2015-05-19 14:48:16 +02:00
Peter Bourgon
08f5905742 Use host -> (Conn, Encoder) mapping for active conns 2015-05-19 12:45:48 +02:00