Commit Graph
343 Commits
Author SHA1 Message Date
Tom Wilkie 7377945302 Use smart merger in the dynamodb collector. (#1543) 2016-05-27 08:57:07 +01:00
Tom Wilkie c8828826ae Allow user to specify table name and queue prefix. (#1538)
* Allow user to specify table name and queue prefix.

* Trim leading slash, catch missed queue prefix

* Comment out publish step until devwww is fixed.
2016-05-25 10:09:32 +01:00
Tom Wilkie 861605a5ee Instrument SQS calls 2016-05-23 16:48:31 +01:00
Tom Wilkie 5a9aebbcb4 lint 2016-05-23 16:19:23 +01:00
Tom Wilkie f36bb4e2fb Gather dyanmodb latency 2016-05-23 14:02:34 +01:00
Tom Wilkie 334701f92e Add a missing return. 2016-05-20 19:17:15 +01:00
Tom Wilkie 24062be6c9 Increase test replicas (#1529)
* Increase number of test VMs to 5 per shard.

* Make pipe router test shorter.
2016-05-19 11:00:51 +01:00
Tom Wilkie 8f772a696d Add flag to disable reporting of processes (and procspied endpoints) 2016-05-17 17:29:09 +01:00
Alfonso Acosta 3cf3c713ae Meassure report sizes 2016-05-10 09:45:29 +00:00
Paul Bellamy 2d10a6a9a6 Merge pull request #1447 from weaveworks/1441-cache-size
Remove cache from SmartMerger.
2016-05-09 10:51:23 +01:00
Tom Wilkie 2dae03501e Remove the caching 2016-05-09 10:08:14 +01:00
Paul Bellamy 541699d193 Review Feedback 2016-05-09 09:19:11 +01:00
Paul Bellamy 16a5c738d9 Deployment and ReplicaSet views for k8s 2016-05-09 09:03:57 +01:00
Paul BellamyandTom Wilkie bb284edee8 set 'default' as the default namespace filter instead of 'all' (#1445) 2016-05-06 18:17:32 +01:00
Tom Wilkie 71d3126c82 Limit merge cache to 200 entries and expire entries old than merge window. 2016-05-06 17:54:57 +01:00
Tom Wilkie 54a760a56d Log(n) complexity report merger. 2016-05-04 17:53:09 +01:00
Paul Bellamy 0e70f70ffd Review feedback 2016-05-03 12:49:02 +01:00
Paul Bellamy 02a0e752e3 fix up stats on sub-topologies 2016-05-03 12:47:26 +01:00
Paul Bellamy fe853e3f0f filter out deleted pods when calculating available namespaces 2016-05-03 12:47:26 +01:00
Paul Bellamy 8758921215 pass nil for Noop a few other places 2016-05-03 12:47:26 +01:00
Paul Bellamy 2af2b1f15a Filter by Kubernetes Namespaces 2016-05-03 12:47:24 +01:00
Tom WilkieandTom Wilkie cf879b268e Aggressively pass nil for the decorator in the rendering pipeline to improve performance. 2016-04-29 11:42:33 +01:00
Paul Bellamy 64450a4830 Merge pull request #1371 from weaveworks/1219-grouped-node-counts-2
Fixing grouped node count for filtered children nodes
2016-04-28 13:30:15 +01:00
Paul Bellamy 3d3aed2bb3 Fixing grouped node count for filtered children nodes
Squash of:

* We have to keep all the container hostnames until the end so we can
  count how many we've filtered

* Adding tests for ContainerHostnameRenderer and PodServiceRenderer with
  filters

* Because we filter on image name we need the image name before
  filtering

* Alternative approach to passing decorators.

* Refactor out some of the decorator capture

* Don't memoise decorated calls to Render

* Fixing filtered counts on containers topology

  Tricky, because we need the filters to be silent sometimes (when they're
  in the middle), but not when they're at the top, so we take the "top"
  filter's stats. However, this means we have to compose all
  user-specified filters into a single Filter layer, so we can get all
  stats.

  There are no more Silent filters, as all filters are silent (unless they
  are at the top).

  Additionally, I clarified some of the filters as their usage/terminology
  was inconsistent and confused. Now Filter(IsFoo, ...) *keeps* only nodes
  where IsFoo is true.
2016-04-28 12:23:43 +01:00
Tom Wilkie b05ef74552 Report hostname and version in probe struct, and version in host node. 2016-04-26 09:25:15 +01:00
Tom Wilkie 901f46c5fc Report if newer version are availible in /api (#1366)
* Report if newer version are availible in /api

* Render version update hint in UI, next to version

* Fix lint
2016-04-22 10:25:00 +01:00
Tom Wilkie 99204e1ff7 Add k8s pod log control (#1298)
* Remove individually vendored k8s.io/kubernetes/pkg/<foo>

* Vendor the whole of vendor/k8s.io/kubernetes/pkg

* Add k8s pod log control

* Tag pods with host id and include them in the host topology as children.

* adding a basic test for kubernetes.Reporter.GetLogs
2016-04-21 13:48:50 +01:00
Paul BellamyandTom Wilkie 1edeb8d190 Removing report.Node.WithID (#1315)
* removing usage of report.Node.WithID

* report.Topology.AddNode can use the node's ID field
2016-04-19 16:48:03 +01:00
Tom Wilkie 0396a79d7f Don't show non-internet pseudo nodes. (#1326) 2016-04-18 14:18:19 +01:00
Tom Wilkie e2cb836272 Correctly expire the cache in the collector 2016-04-13 11:29:53 +01:00
Paul Bellamy f211d48cda Merge pull request #1126 from weaveworks/plugins
Plugins
2016-04-12 18:03:26 +01:00
Paul Bellamy 7632e0b3c5 Adding support for plugins, with basic example of iowait, and ebpf
Squash of:
* Include plugins in the report
* show plugin list in the UI
* moving metric and metadata templates into the probe reports
* update js for prime -> priority
* added retry to plugin handshake
* added iowait plugin
* review feedback
* plugin documentation
2016-04-12 17:22:14 +01:00
Tom Wilkie 281ba58845 Add /api/probes endpoint 2016-04-12 17:17:18 +01:00
Tom Wilkie ad338109f3 Remove unused function 2016-04-12 16:24:50 +01:00
Tom Wilkie da53aac25d Tests & vartious fixes for consul pipe router.
Fix a few bugs in the consul pipe router:
- Don't share a pointer
- Write nil to pipe when closing a bridge connection to ensure the connection shutdown.
- Ensure we shutdown bridge connections correctly
2016-04-12 14:28:10 +01:00
Paul Bellamy 945029c890 Merge pull request #1234 from weaveworks/middleware
Add various middleware to app.
2016-04-06 16:22:31 +01:00
Tom Wilkie 734a01d603 Review feedback, lint & fix tests. 2016-04-06 15:59:02 +01:00
Tom Wilkie fd75e45715 Multitenant DynamoDB collector should use a string for the hash key.
This is because the key is of the form "<userid>-<hour bucket>", but as I was testing without a userid, I didn't notice that "-<hour bucket>" was a valid number.
2016-04-06 14:59:53 +01:00
Tom Wilkie fd368e5e5f Unhack dealing with slashed in node names. 2016-04-06 14:47:14 +01:00
Paul Bellamy 2c6b6e6707 Refactoring rendering to remove RenderableNode
Squash of:
- use detailed.Summaries to render topology nodes
- ban merging nodes of different topologies (they should be mapped)
- need to prune parents when mapping node types
- render container images by id if they have no name
- remove separate render ids and prune parents in NewDerived*
- don't render metrics/metadata for groups of nodes
- fixing up tests
- removing pending unit tests (for mapping.go, for now)
- updating experimental dir for RenderableNode removal
2016-03-29 14:13:03 +01:00
Paul BellamyandDavid Kaltschmidt 430130c03a backend changes for new option rendering 2016-03-29 13:26:49 +02:00
Tom Wilkie 89139fed81 Merge pull request #1200 from weaveworks/1181-hide-in-ui
Stop hiding kubernetes topologies in the backend.
2016-03-24 13:35:28 +00:00
Tom Wilkie f0f03d367c Spelling 2016-03-23 15:49:50 +00:00
Tom Wilkie 9610e95ed9 Break out factories for app backends, and encode everything in urls. 2016-03-23 15:41:37 +00:00
Tom Wilkie ac638d17dd Review feedback 2016-03-23 15:41:37 +00:00
Tom Wilkie 0a8fd8a3a6 Actor model for pipe router 2016-03-23 15:41:37 +00:00
Tom Wilkie 9885706402 Multi-tenant backends for app.
Add DynamoDB based collector
    - Store compressed reports in dynamodb

    Add SQS based control router.
    - Uses a queue per probe and a queue per UI for control requests & responses.

    Add Consul-based, horizontally-scalable, multi-tenant pipe router.
    - Uses consul to coordinate each end of pipe connections replicas of a pipe service.
2016-03-23 15:41:37 +00:00
Tom Wilkie 4f6066f0ff Stop hiding kubernetes topologies in the backend. 2016-03-23 12:43:47 +00:00
Tom Wilkie 581b711eb4 Websocket and pipe fixes. 2016-03-15 17:13:49 +00:00
Tom Wilkie b4739af279 Refactoring for multitenancy 2016-03-09 13:48:28 +00:00