18 Commits

Author SHA1 Message Date
Bryan Boreham
b6d5594f9f perf(probe): publish delta reports to reduce data size
Similar to video compression which uses key-frames and differences
between them: every N publishes we send a full report, but inbetween
we only send what has changed.

Fairly simple approach in the probe - hold on to the last full report,
and for the deltas remove anything that would be merged in from the
full report.

On the receiving side in the app it already merges a set of reports
together to produce the final output for rendering, so provided N is
smaller than that set we don't need to do anything different.

Deltas don't need to represent nodes that have disappeared - an
earlier full node will have that node so it would be merged into the
final output anyway.
2019-09-18 08:00:28 +00:00
Bryan Boreham
8e305f1ce9 Don't copy StringSet if merging a subset
Make StringSet.Merge() work like StringLatestMap.Merge()
2018-07-12 19:54:01 +00:00
Matthias Radestock
4dbb913ca2 refactor: remove dead Copy() code 2017-07-04 07:35:30 +01:00
Matthias Radestock
5b131b1ea6 refactor: remove a couple of report set Remove() functions
They were unused and their naming was inconsistent with similar
operations (that are called Delete()).
2017-07-03 01:26:22 +01:00
Matthias Radestock
b5f3aa68ae refactor: optimise "empty" case in report set constructors 2017-07-03 01:26:22 +01:00
Matthias Radestock
9dc50b5202 refactor: hide "empty set" constants
They are an implementation detail.
2017-07-03 01:26:22 +01:00
Tom Wilkie
f154e7a483 Add connection tables to details panel 2016-03-03 16:17:55 +00:00
Tom Wilkie
bc3f946810 Make NodeMetadatas immutable. 2016-01-23 13:02:16 -08:00
Paul Bellamy
56122dd0cc Details panel backend redesign
Megasquish:
  [app] remove unused edge endpoint
  [WIP] refactoring node details api endpoint
  [WIP] plumbing the children through the rendering process
  adding IDList.Remove and StringSet.Remove
  [WIP] working on adding parents to detailed node renderings
  WIP UI components with mock backend data for new details
  grouping children by type
  UI components for node details health and info
  metric formatters for details panel
  Column headers and links for details table
  [WIP] started on rendering node metadata and metrics in the detail view
  DetailedNode.LabelMajor -> DetailedNode.Label
  rendering decent labels for parents of detailed nodes
  render metrics onto the top-level detailed node
  removing dead code
  Links to relatives
  metrics have a Format not Unit
  Show more/less actions for tables and relatives
  adjusted metric formatter
  TopologyTagger should tag k8s topology nodes
  make renderablenode ids more consistent, e.g. container:abcd1234
  working on rendering correct summaries for each node
  adding report.Node.Rank, so that merging is independent of order
  rendering children and parents correctly
  output child renderableNode ids, so we can link to them
  add group field to metrics, so they can be grouped
  Refactored details health items to prepare for grouping
  add metrics to processNodeSummaries
  hide summary section if there is no data for it
  fixing up tests
  moving detailed node rendering into a separate package
  Node ID/Topology are fields not metadata
    - This way I think we don't have to care about Metadata being non-commutative.
    - ID and topology are still non-commutative, as I'm not sure how to sanely
  merge them, but it's possible we don't care.
  host memory usage is a filesize, not a percent
  working on fixing some tests
  adding children to hosts detail panel
    - Had to redo how parents are calculated, so that children wouldn't interfere with it
    - have to have the host at the end because it is non-commutative
  only render links for linkable children (i.e. not unconnected processes)
  resolving TODOs
  fixing up lint errors
  make nil a valid value for render.Children so tests are cleaner
  working on backend tests
  make client handle missing metrics property
  Stop rendering container image nodes with process summaries/parents
  fix parent link to container images
  Calculate parents as a set on report.Node (except k8s)
  refactoring detailed.NodeSummary stuff
  removing RenderableNode.Summary*, we already track it on report.Node
  working on tests
  add Columns field to NodeSummaryGroup
  fixing up render/topologies_test
  fix children links to container images
  get children of hosts rendering right
  working on host renderer tests
  Change container report.Node.ID to a1b2c3;<container>
  The id should be globally unique, so we don't need the host id.
    This lets the kubernetes probe return a container node with the pod id,
    which will get merged into the real containers with other reports. The
    catch is that the kubernetes api doesn't tell us which hostname the
    container is running on, so we can't populate the old-style node ids.
  change terminology of system pods and services
  Fix kubernetes services with no selector
  Fixes handling of kubernetes service, which has no pods
  fix parent links for pods/services
  refactor detailed metadata to include sets and latest data
  fixing up host rendering tests
  fleshing out tests for node metadata and metrics
  don't render container pseudo-nodes as processes
  Update test for id format change.
2016-01-19 16:39:37 +01:00
Tom Wilkie
6913431138 Make IDList an alias for StringSet 2015-10-27 14:12:34 +00:00
Tom Wilkie
1f6b3d4a45 Move Adjacency info inside the NodeMetadata struct; move pseudo node generation to the probes. 2015-08-28 15:20:27 +00:00
Peter Bourgon
42be036428 report: always copy 2015-08-25 17:52:20 +01:00
Bryan Boreham
1fa853befe Neater implementation of de-duplication in MakeIDList() 2015-07-08 13:38:39 +01:00
Bryan Boreham
35f9dc622e More efficient slice-insert - less copying, less garbage 2015-07-07 22:36:22 +01:00
Bryan Boreham
baf0d94af9 Implement a Merge function on IDLists which is more efficient than repeated Add 2015-07-07 22:36:22 +01:00
Bryan Boreham
76d658b193 Make MakeIDList() enforce invariant 2015-07-07 22:16:50 +01:00
Tom Wilkie
c5d10867c6 Add Validate method to Report & Topology. 2015-06-11 13:27:49 +00:00
Peter Bourgon
7c56939e22 Update IDList
- MakeIDList rather than NewIDList
- Iterate on test coverage
- Drop Copy method, for now
2015-06-08 19:52:26 +02:00