For when we want to collect reports in memory, but not save them to store.
Extract this functionality out of awsCollector to create new
liveCollector object.
Collectors hold recent reports in memory.
When querier needs 'live' data, fetch it from collectors instead
of from the long-term store.
Send reports from collector to querier in msgpack; disable compression
on REST call, otherwise Go silently decompresses, which takes longer.
Where a report has been merged from several probes, give it a higher
limit before dropping topologies.
We will already have applied the limit on each single-probe report as
it came in, except for historical ones.
Doesn't do anything at present, but will be used later.
Change the signature on BillingEmitter.Close() to match. Note we didn't use the error returned.
This reverts commit 6b72246fe6.
The app merges reports within a 15-second window of its own time, so
if one or more probes have a time that is several seconds different
they will get excluded from the window.
Previously the code called `time.Now()` in two different places so the
timestamps didn't match. Now we use the timestamp of the report itself.
Add the collector's local time to the report if it didn't have one.
Shortcut reports are sent to update the UI quickly, on events like a
container starting. We don't need to persist them in the time-travel
data since the same information will be covered by a regular report a
few seconds later.
Previously we would merge all reports in a 15-second window.
Now we use a 'quantum' of 3 seconds, similar to the single-user app.
E.g. a 30-node cluster will have 150 individual reports over 15
seconds, but the new code will merge 5 pre-merged reports plus 20-ish
very recent individual ones.
This limits the max heap size used for deserialising, since we only do
3 seconds at once per instance.
Individual reports are still put into the cache, but should get
displaced by the pre-merged ones under LRU.
In a multitenant system it is useful to be able to drill into which
tenants have the most or biggest reports.
Signed-off-by: Bryan Boreham <bryan@weave.works>
* Pass Go context down to Renderers
This is useful for cancellation or tracing.
* Add tracing spans to app
Also log things like number of nodes in Map, total number of reports.
This requires no report reading / merging.
We plan to expose this in the HTTP API, so the UI gets a cheap way of
checking whether the app is currently receiving data from probes.
