fix report store query chaining

We fell victim to variable shadowing here. Each store would be fed the
original list of report keys, instead of only the ones that weren't
found in the previous store. So if a single report was missing from the
in-process cache, we would then fetch all reports from memcache. And if
that in turn was missing a single report we would fetch all reports from
S3.

We chain report stores for a reason - to reduce latency and, in case of
the in-process cache, eliminate decoding costs. So this bug has a huge
impact on query service performance.

To make matters worse, we return *all* the reports - now possibly in
triplicate. Fortunately, the SmartMerger filters these out, so at least
we were not incurring extra merge costs.
This commit is contained in:
Matthias Radestock
2016-08-15 23:20:34 +01:00
parent ed5463facd
commit e6a474ead7

View File

@@ -277,7 +277,8 @@ func (c *awsCollector) getReports(reportKeys []string) ([]report.Report, error)
if store == nil {
continue
}
found, missing, err := store.FetchReports(missing)
found, newMissing, err := store.FetchReports(missing)
missing = newMissing
if err != nil {
log.Warningf("Error fetching from cache: %v", err)
}