* Add in-process caching to dynamodb collector
* Add metrics for dynamodb consumed capacity and report size
* Log and return errors during report collection
* Increase compression to the max
* Put reports in S3 and just use DynamoDB as an index.
* Review feedback
Squash of:
* We have to keep all the container hostnames until the end so we can
count how many we've filtered
* Adding tests for ContainerHostnameRenderer and PodServiceRenderer with
filters
* Because we filter on image name we need the image name before
filtering
* Alternative approach to passing decorators.
* Refactor out some of the decorator capture
* Don't memoise decorated calls to Render
* Fixing filtered counts on containers topology
Tricky, because we need the filters to be silent sometimes (when they're
in the middle), but not when they're at the top, so we take the "top"
filter's stats. However, this means we have to compose all
user-specified filters into a single Filter layer, so we can get all
stats.
There are no more Silent filters, as all filters are silent (unless they
are at the top).
Additionally, I clarified some of the filters as their usage/terminology
was inconsistent and confused. Now Filter(IsFoo, ...) *keeps* only nodes
where IsFoo is true.
Squash of:
* Include plugins in the report
* show plugin list in the UI
* moving metric and metadata templates into the probe reports
* update js for prime -> priority
* added retry to plugin handshake
* added iowait plugin
* review feedback
* plugin documentation
- Add interfaces to allow for alternative implementations for Collector, ControlRouter
and PipeRouter.
- Pass contexts on http handlers to these interfaces. Although not used by the current
(local, in-memory) implementations, the idea is this will be used to pass headers to
implementations which support multitenancy (by, for instance, putting an authenticating
reverse proxy in form of the app, and then inspecting the headers of the request for
a used id).
* New encoding format:
* Msgpack reports between probe<->app (smaller representation, faster to
encode/decode).
* Still use JSON between app<->UI (try to avoid making javascript deal with
mspack).
The app still suports publishing reports in both gob and JSON, not braking
backwards compatibility.
* Use compile-time generated marshallers/unmarshallers for higher performance. In
order to be able to skip code-generation for certain types, I included
https://github.com/2opremio/go-1/tree/master/codec/codecgen instead of
upstream until https://github.com/ugorji/go/pull/139 is merged.
* Encode/decode intermediate types using github.com/ugorji/go/codec.Selfer
for higher performance and reducing garbage collection (no temporary buffers).
* Added helper for installing scope on gcloud
* Added topologies Pods and Pods-by-Service
* Uses k8s.io/kubernetes/pkg/client/cache for the client
* Filter kube-system nodes by default
* Only show the k8s topologies if we've received a non-empty k8s report
- App takes POST report on /api/report
- Probe publishes to configured target(s)
- Name resolution happens on probe-side
- There's no longer an xfer.ProbePort
- xfer.Collector responsibility is reduced
- Fixes to remaining experimental components.
- rm experimental/bridge: it's not being used, and by changing the
app/probe comm model, it would require a complete refactor anyway. We
can easily rebuild it when we need to. It will even be much simpler.
- rm experimental/graphviz: it's broken for some time anyway, and we
don't really need to play around with it as a rendering option
anymore.
- rm experimental/oneshot: we never use this anymore.
