Commit Graph

4457 Commits

Author SHA1 Message Date
Matthias Radestock
f5b38a0019 Merge pull request #2956 from weaveworks/sizing-merge-slice
optimisation: allocate less memory in LatestMap merging
2017-12-05 06:22:55 +00:00
Matthias Radestock
4162b5d734 allocate less memory in LatestMap merging
Most maps we merge have the same keys, or at least one set of keys is
the subset of the other. Therefore, allocate a result slice capable of
holding only the max of number keys, rather than the sum.
2017-12-02 13:13:08 +00:00
Matthias Radestock
acde0ea294 add some more topology rendering benchmarks 2017-12-02 13:04:57 +00:00
Matthias Radestock
bc0c8324be add report merging benchmark 2017-12-02 11:51:16 +00:00
Matthias Radestock
c953313b01 move main report processing benchmarks in one place
so they can share code and are easier to run in combination.

We take advantage of the code sharing by generalising the report
rendering benchmarks to read & merge reports from a dir.
2017-12-02 11:29:16 +00:00
Matthias Radestock
2f4c6507e5 Merge pull request #2955 from weaveworks/fix-replicaset-desiredreplicas
fix incorrect reporting of replicaset DesiredReplicas
2017-12-01 15:04:12 +00:00
Matthias Radestock
914acf6e3d fix incorrect reporting of replicaset DesiredReplicas
ReplicaSetSpec.Replicas is an *int32. Just like in DeploymentSpec,
where we deal with that in the same way.
2017-11-30 18:40:00 +00:00
Bryan Boreham
d6e47c0f5d Merge pull request #2386 from weaveworks/decode-bytes
Decode reports from a byte buffer
2017-11-29 18:41:24 -08:00
Bryan Boreham
05d5d339c4 Decode reports from a byte buffer
Reading and uncompressing the entire message into memory first allows
the decoder to avoid memory allocations in field names, etc.
2017-11-29 21:56:55 +00:00
Matthias Radestock
cdbc01ecf9 simplify report.MakeFromFile
this new version is less efficient... but not for much longer
2017-11-29 21:48:52 +00:00
Matthias Radestock
ecf3ae90ff add a benchmark for report unmarshalling 2017-11-29 21:48:52 +00:00
Matthias Radestock
d5eab89e97 Merge pull request #2954 from weaveworks/filter-internet-adjacencies
filter internet adjacencies

Fixes #1110.
2017-11-29 19:04:45 +00:00
Matthias Radestock
f2fed067d5 simplify render benchmark code
by using report.MakeFromFile instead of NewFileConnector
2017-11-29 11:37:39 +00:00
Matthias Radestock
9fbc388f82 filter internet adjacencies
Edges between the incoming and outgoing internet nodes are typically
artifacts of imperfect connection tracking, e.g. when VIPs and NAT
traversal are in use.

So lets filter them out.
2017-11-28 13:23:17 +00:00
Matthias Radestock
e73ba58740 Merge pull request #2951 from weaveworks/one-pass-unconnected
filter out unconnected pseudo nodes just once, at the end
2017-11-28 08:06:12 +00:00
Matthias Radestock
956303694a always apply ColorConnected in process renderers
This eliminates the awkward distinction between ProcessRenderer and
ColorConnectedProcessRenderer.

It also ensures that processes resulting from direct rendering of the
process topology (/api/topology/processes is invoking
ProcessWithContainerNameRenderer and /api/topology/processes-by-name
is invoking ProcessNameRenderer) are colored and hence summarising
them correctly sets the 'linkable' property. This was the behaviour
prior to the revamping of the rendering pipeline. However, it doesn't
actually make a practical difference since process detail panels only
show other processes as connection endpoints, and these are always
marked linkable anyway.
2017-11-28 06:54:15 +00:00
Matthias Radestock
f193a2101c refactor: remove duplication of filtering logic 2017-11-28 06:54:15 +00:00
Matthias Radestock
1416fe928f remove filtering of unconnected pseudo nodes from ordinary filters
It's now done via a special filter, once, after all other filters have
been applied.

Some tests need updating since they were relying on ordinary filters
doing that filtering.
2017-11-28 06:54:15 +00:00
Matthias Radestock
9dca7627b6 filter unconnected nodes after applying user-specified filters
...rather than before. That way, nodes which become unconnected during
filtering are removed, which is what we want. ATM we are depending on
some 'unconnected' filtering inside every filter, which is expensive
and largely redundant. We should soon be able to remove that.

downside: 'unconnected' filtering is no longer memoised.
2017-11-28 06:54:15 +00:00
Matthias Radestock
2d964b669a refactor: separate filtering from rendering in topology description
This is a step towards filtering unconnected nodes after all custom
filters have been applied.
2017-11-28 06:54:15 +00:00
Matthias Radestock
7bbded9e84 refactor(ish): introduce Tranformers
so we can generalise the filter step in render.Render et al. That will
allow us to apply whole-topology filters in that step.
2017-11-28 06:54:15 +00:00
Matthias Radestock
97fadd6d9f optimising refactor: FilterUnconnected{Pseudo} w/o LatestMap
Instead of three passes
1. building a 'connected' node set
2. marking nodes from that set with a LatestMap entry
3. removing unmarked nodes

we just do two
1. building a 'connected' node set
2. removing nodes not in that set

This does entail duplication of the adjecency list pruning code from
FilterFunc.Apply. We will be able to eliminate that eventually, but
not just yet.

Also, we cannot get rid of ColorConnected completely;
ColorConnectedProcessRenderer uses it to mark nodes, and that
information is required by detailed.processNodeSummary to determine
whether a process in the details panel can be rendered as a link.
2017-11-28 06:54:15 +00:00
Matthias Radestock
91fd411c37 Merge pull request #2952 from weaveworks/string-keyed-known-service-cache
optimisation: faster knownServiceCache
2017-11-26 15:11:51 +00:00
Matthias Radestock
be8801ab92 optimisation: faster knownServiceCache
We use an LRU cache implementation that is keyed on strings, which
avoids conversion to/from interface and the memory allocation that
entails.
2017-11-25 19:33:49 +00:00
Matthias Radestock
b10f7178ce Merge pull request #2950 from weaveworks/memoise-fixpoint
optimisation: make Memoise(Memoise(...)) only memoise once
2017-11-24 17:35:28 +00:00
Matthias Radestock
5bdf46956d optimisation: make Memoise(Memoise(...)) only memoise once
All outputs of Memoise() are fixpoints to the function, i.e. feeding
them as inputs to the function just returns them.

We don't hit this optimisation in current code but have had instances
in the past.
2017-11-23 17:03:33 +00:00
Matthias Radestock
cfac26757f Merge pull request #2949 from weaveworks/reset-all-caches
make render.ResetCache() reset all caches
2017-11-22 23:32:39 +00:00
Simon
d9815beff0 Merge pull request #2948 from weaveworks/fixes-flux-image-status
Correctly show whether there are new images or not.
2017-11-22 16:44:54 +01:00
Matthias Radestock
1c39d14c74 make render.ResetCache() reset all caches
so that benchmarks yield more consistent results.
2017-11-22 13:00:39 +00:00
Bryan Boreham
3bc8f22c11 Merge pull request #2938 from weaveworks/more-renderers
Rewrite more Map-Reduces as Renderers to save garbage
2017-11-22 11:21:06 +00:00
Matthias Radestock
1c206fb1c9 cosmetic: bring comment up to date 2017-11-22 10:40:34 +00:00
Bryan Boreham
5697d53034 Rewrite MapX2Host as a RenderFunc using joinResults
It's more efficient as many input Nodes map onto a few hosts.
2017-11-22 10:39:37 +00:00
Bryan Boreham
3e62f7b754 Rewrite MapProcess2Name to process all nodes in one pass
This is more efficient as there are typically many fewer names than
processes.
2017-11-22 10:33:00 +00:00
Matthias Radestock
346a0360ef Refactor: split addToResults into two functions
This is preparatory to future refactorings: all existing calls are to
Endpoints which have no children and where we don't want a Counter.

We make addChildAndChildren an obvious extension of addChild even
though it adds a dead code path (we never call addChildAndChildren
with an endpoint).
2017-11-22 10:30:18 +00:00
Matthias Radestock
f56a44e67d cosmetic: move a function 2017-11-22 09:46:33 +00:00
Matthias Radestock
334dba20e7 cosmetic: move some functions 2017-11-22 09:19:10 +00:00
Matthias Radestock
b24334c246 fix small bug in Filtered calculation
Filtering nodes *adds* to the Filtered count.
2017-11-22 09:13:23 +00:00
Matthias Radestock
8c4ae0577d Merge pull request #2947 from weaveworks/simplify-decoration
Decorators, begone!
2017-11-21 20:49:45 +00:00
Matthias Radestock
a12d707d9b refactor: rename decorateWithStats to computeStats
to de-decorate
2017-11-21 20:17:00 +00:00
Matthias Radestock
0c43526465 refactor: use new FilterFunc.Apply
instead of constructing temporary Filter renderers.

This also makes clearer what is going on.
2017-11-21 20:16:59 +00:00
Matthias Radestock
e643ec56be refactor: move filter logic to FilterFunc
so it becomes accessible w/o having to construct a Filter renderer.
2017-11-21 20:16:59 +00:00
Matthias Radestock
88e8b52d66 Decorators, begone!
Decorators were just a complicated way of constructing filters.
2017-11-21 20:16:30 +00:00
Matthias Radestock
093857f37f refactor: extract a little test helper 2017-11-21 20:13:14 +00:00
Matthias Radestock
e93362e70a Merge pull request #2946 from weaveworks/expand-k8s-namespace-calculation
Expand the app's k8s namespace calculation
2017-11-21 07:55:42 +00:00
Simon Howe
ef99e672c9 Correctly show whether there are new images or not.
find returns 'undefined' not 'null'!
2017-11-20 14:37:33 +01:00
Matthias Radestock
67950985ef refactor: introduce IsConnected FilterFunc
and rename existing IsConnected const to IsConnectedMark
2017-11-20 09:28:56 +00:00
Matthias Radestock
ae153e57f5 refactor: remove Decorator from render.Renderer.Render() signature 2017-11-18 18:04:52 +00:00
Matthias Radestock
a82d245e93 simplify render decoration
Decoration is in fact quite a simple process that is applied on entry
to rendering: we take a base renderer, transform it with a decorator,
and then render a report with it. The new render.Decorate() function
does exactly that.

There is one exception. When rendering an individual node, e.g. for
showing its details panel in the UI, we must not lose the node during
decoration. That requires some special logic, which previously resided
in the PreciousNodeRenderer, and now lives in handleNode.
2017-11-18 18:04:52 +00:00
Matthias Radestock
830cfc964d Merge pull request #2944 from weaveworks/fewer-ref
pass render filters and maps by value rather than reference
2017-11-18 17:55:34 +00:00
Matthias Radestock
4e4c3b25c4 Expand the app's k8s namespace calculation
to include recently added k8s types.

This is all rather inefficient. See #2945.
2017-11-18 17:35:57 +00:00