* Add in-process caching to dynamodb collector
* Add metrics for dynamodb consumed capacity and report size
* Log and return errors during report collection
* Increase compression to the max
* Put reports in S3 and just use DynamoDB as an index.
* Review feedback
Fix a few bugs in the consul pipe router:
- Don't share a pointer
- Write nil to pipe when closing a bridge connection to ensure the connection shutdown.
- Ensure we shutdown bridge connections correctly
This is because the key is of the form "<userid>-<hour bucket>", but as I was testing without a userid, I didn't notice that "-<hour bucket>" was a valid number.
Add DynamoDB based collector
- Store compressed reports in dynamodb
Add SQS based control router.
- Uses a queue per probe and a queue per UI for control requests & responses.
Add Consul-based, horizontally-scalable, multi-tenant pipe router.
- Uses consul to coordinate each end of pipe connections replicas of a pipe service.
