Commit Graph

905 Commits

Author SHA1 Message Date
Bryan Boreham
8be6ca5e1e fixup: remove timestamp argument from node.WithLatest() 2019-10-19 15:00:55 +00:00
Bryan Boreham
07287c6ac0 Move timestamp up from 'Latest' string map to Report level
We never merge multiple string values in a single node from different
sources, so it's sufficient to take the value from the latest report.
2019-10-19 15:00:55 +00:00
Bryan Boreham
e1bfa01cae performance: Update plugins to new-style controls data 2019-10-14 18:53:28 +00:00
Bryan Boreham
0b641b1848 performance: Send active controls as a single string per node
Instead of a whole extra data structure which is quite expensive to
marshal and unmarshal, just send the information in a string.  No
clever merging strategy is required - the states are all set in one
place per node type.
2019-10-14 18:52:57 +00:00
Bryan Boreham
b24917993e fix: report http error if /api call fails
Previously it would try to run the JSON decoder on a string like "404
not found" and report that failing.
2019-10-06 17:27:49 +00:00
Bryan Boreham
bd9f88b985 Merge pull request #3696 from weaveworks/ebpf-non-strings
handle IP addresses in binary rather than strings
2019-10-04 14:45:15 +01:00
Bryan Boreham
23d8a418e1 performance: network namespace ID is a 32-bit quantity
This shrinks some data-structures slightly.

Citation: https://github.com/torvalds/linux/blob/6f0d349d922b/include/linux/ns_common.h#L10
2019-10-04 13:11:30 +00:00
Bryan Boreham
c6ce47f87d diagnostics: make fourTuple.String() human-readable 2019-10-04 13:09:50 +00:00
Bryan Boreham
cbbb2ff24c performance: in Docker reporter, reduce IP type conversions.
The code was converting IP addresses to strings and back again.
2019-09-25 20:15:34 +00:00
Bryan Boreham
2941850a75 performance: in connection tracker, hold IP addresses in binary rather than strings
This is more compact, and saves effort converting to and from the string format.
2019-09-25 20:15:05 +00:00
Bryan Boreham
8d9e337a75 chore: fix typos in debugging format strings 2019-09-25 20:08:29 +00:00
Satyam Zode
e7e9e97943 Merge pull request #3606 from weaveworks/update-gopacket
Update google/gopacket library
2019-09-23 16:11:08 +05:30
Bryan Boreham
17c1aaa131 chore(probe): for Kubernetes 1.16 move to 'v1' APIs
Scope will no longer work with Kubernetes 1.8 and below.

For CronJob there is no 'v1' as yet, but we can remove the alpha
version.
2019-09-21 15:52:34 +00:00
Bryan Boreham
9208e08bf3 Change dns snooper timeout to avoid spinning in pcap
The previous code seems to be relying on a 64-bit to 32-bit conversion
working in a certain way; when gopacket was changed to cast the value
explicitly it starts returning immeditely from pcap.
2019-09-20 14:32:53 +00:00
Akash Srivastava
4b6b12d2c8 Merge pull request #3688 from weaveworks/fix-testregdelete-flake
fix(test-flake): poll for result in TestRegistryDelete() to avoid race
2019-09-19 16:39:50 +05:30
Bryan Boreham
49dfd98c94 fix(test-flake): poll for result in TestRegistryDelete() to avoid race
Remove the `runtime.GoSched()` that doesn't guarantee anything.
2019-09-18 21:32:44 +00:00
Bryan Boreham
71a359e1d7 Merge pull request #3679 from weaveworks/probe-unsafe-merge
perf(probe): reduce copying of nodes
2019-09-18 15:56:06 +01:00
Bryan Boreham
b6d5594f9f perf(probe): publish delta reports to reduce data size
Similar to video compression which uses key-frames and differences
between them: every N publishes we send a full report, but inbetween
we only send what has changed.

Fairly simple approach in the probe - hold on to the last full report,
and for the deltas remove anything that would be merged in from the
full report.

On the receiving side in the app it already merges a set of reports
together to produce the final output for rendering, so provided N is
smaller than that set we don't need to do anything different.

Deltas don't need to represent nodes that have disappeared - an
earlier full node will have that node so it would be merged into the
final output anyway.
2019-09-18 08:00:28 +00:00
Bryan Boreham
eff5a1f9f7 Refactor: pull Publish() call up to publishLoop() 2019-09-18 08:00:28 +00:00
Bryan Boreham
a811afdba1 Merge pull request #3678 from weaveworks/nodes-omitempty
perf(probe): add 'omitempty' tag to Topology.Nodes
2019-09-17 16:25:52 +01:00
Akash Srivastava
0203757cf5 Merge pull request #3675 from weaveworks/reduce-probe-dependency
Stop render package depending on probe
2019-09-16 12:56:56 +05:30
Bryan Boreham
871751873b Stop render package depending on probe
This dependency makes it harder to see the structure of the program,
and sometimes complicates compilation.

Mostly just changing the source of strings that are already exported
from the report package.  A few new strings have to be moved there,
plus the function `IsPauseImageName()`.
2019-09-15 17:03:04 +00:00
Bryan Boreham
4c52889316 Add 'omitempty' tag to Topology.Nodes
So we save space writing out empty topologies.

Need to fix up `app_client_internal_test.go` to use Scope's
`test/reflect` package that understands empty==nil, so now it doesn't
need a previous workaround.

Remove a similar workaround in `probe_internal_test.go` that isn't
necessary since it's already using that package.
2019-09-15 15:50:08 +00:00
Bryan Boreham
2f9c9913c4 perf(probe): reduce copying of nodes
Where we know we are merging several reports into the one, we can call
UnsafeMerge() and skip the copy that Merge() will do.
2019-09-15 15:40:28 +00:00
Bryan Boreham
48aad1a20d Remove unused string constants 2019-09-13 11:42:21 +00:00
Bryan Boreham
15467d7310 Move host-related names out of probe code
Reduce the dependency on low-level libraries
2019-09-13 11:41:09 +00:00
Bryan Boreham
5cba126c12 Merge pull request #3600 from weaveworks/expose-probe-metrics
Expose probe metrics to Prometheus
2019-08-20 14:35:06 +01:00
Bryan Boreham
eba9f31f3f fix(probe): restart conntrack handler periodically to clear out data
We observe a slow increase in connections reported, and are unable to
find the root cause, so clear down the data every six hours and start
from a clean sheet.
2019-08-13 16:30:56 +00:00
Bryan Boreham
6e715d2697 fix(probe): Loosen ebpf parameters to reduce restarts
Delay kernel events by up to 0.2ms, to reduce the chance the ebpf
reporter sends them out-of-order, and allow out-of-order events to
happen up to once a minute without giving up on the ebpf reporter.
2019-08-13 16:17:23 +00:00
Bryan Boreham
609d9a7506 Merge pull request #3363 from princerachit/format-error-log
refactor(logs): Add reporter name to error logs.
2019-07-31 14:51:30 +01:00
Bryan Boreham
5e57b0dbcf Add metrics for conntrack and ebpf errors 2019-07-09 13:01:47 +00:00
Bryan Boreham
20e7d5d476 Add metric counting report publish attempts and errors 2019-07-09 13:01:26 +00:00
Bryan Boreham
ab7a7dcb16 Expose probe metrics to Prometheus
We are already timing all report, tag and tick operations.
If Prometheus is in use, expose those metrics that way.

Adjust metrics naming to fit with Prometheus norms.

The previous way these metrics were exposed was via SIGUSR1, and we
can only have one "sink", so make it either-or.

Signed-off-by: Bryan Boreham <bryan@weave.works>
2019-07-09 12:34:05 +00:00
Bryan Boreham
161014857d Merge pull request #3646 from weaveworks/remove-useless-metric
fix: remove unused metric SpyDuration
2019-07-04 17:37:16 +01:00
Bryan Boreham
c1e110ac7b fix: remove unused metric SpyDuration
The call to register this metric was removed in #633 over three years ago.
If it isn't registered then nobody can see the values.
The measurement is duplicated by metrics added in #658.
2019-07-04 13:53:00 +00:00
Bryan Boreham
1e6a6a7bb4 fix: handle errors reported by the conntrack package
In particular, ENOBUFS which indicates data has been dropped.
With this change the collector will restart, thus resynchronising with
the OS.
2019-07-04 13:30:46 +00:00
Bryan Boreham
380dd8357f fix(probe): Use a buffered chan to reduce the chance of losing events
Event notifications from Docker will be dropped if not collected
quickly enough; using a buffered chan reduces the chance of this
happening.
2019-06-05 10:36:38 +00:00
Bryan Boreham
7a93c73343 Always delete container info when we get a 'destroy' event
Previously it would only delete if Docker said the container didn't
exist, which is a race between Docker sending the event and Docker
removing the info from its own records.

Extract a couple of functions to make the action clearer.
2019-05-24 16:31:42 +00:00
Satyam Zode
36361d6b7d Add Kubernetes Job support to Weave Scope
This covers:
- Implementation of the job resource in probe
- Reporter changes for the job
- Add Describe Control to the job
- Pass job object to the kube controller renderer

ToDo:
- Need to decide the shape for the Job.

Signed-off-by: Satyam Zode <satyamzode@gmail.com>
2019-05-19 02:36:45 +05:30
Filip Barl
cdc33855d4 Merge pull request #3589 from openebs/feature/kubectl-describe
Add describe control to all k8s resources
2019-05-17 13:37:15 +02:00
Akash Srivastava
5a53ca350e Update rank of DeletePod control from 1 to 3
Signed-off-by: Akash Srivastava <akashsrivastava4927@gmail.com>
2019-05-17 15:59:08 +05:30
Bryan Boreham
72882835d5 Merge pull request #3566 from tiriplicamihai/fix-dns-snooper-multiple-cnames
Fix dnssnooper probe for multiple CNAMEs.
2019-05-07 16:04:18 +01:00
tiriplicamihai
0b4e26ed77 Fix formatting. 2019-04-30 19:24:23 +03:00
tiriplicamihai
92d3c1d5e9 Fix test data. 2019-04-30 19:09:02 +03:00
tiriplicamihai
1fbe648e82 Add newline. 2019-04-30 18:41:16 +03:00
tiriplicamihai
364a7423a5 Add tests for dns snooper. 2019-04-30 18:38:07 +03:00
Akash Srivastava
029bbbcec8 Add describe control to all k8s resources
This commit will add a new control i.e. Describe
which will describe a k8s resource similar to kubectl describe.

Signed-off-by: Akash Srivastava <akashsrivastava4927@gmail.com>
2019-04-01 11:22:12 +05:30
Akash Srivastava
8d784b7102 Upgrade client-go version to 10.0.0
This will
Upgrade:
        - k8s.io/client-go v10.0.0
	- k8s.io/api version to kubernetes-1.13.0
	- k8s.io/apimachinery version to kubernetes-1.13.0
        - github.com/imdario/mergo to v0.3.5

Add:
        - k8s.io/klog v0.1.0
        - sigs.k8s.io/yaml v1.1.0

Changes:
        - modified:   probe/kubernetes/client.go
        - modified:   probe/kubernetes/controls.go

To ScaleUp and ScaleDown a Deployment, Scale method was used,
which is removed in client-go-10.0.0
Instead of Scale use UpdateScale method of Deployment to
ScaleUp and ScaleDown the Deployment.

Signed-off-by: Akash Srivastava <akashsrivastava4927@gmail.com>
2019-03-29 17:04:14 +05:30
Filip Barl
e66a3e08dc Add a confirmation dialog for deleting a pod. 2019-02-26 11:57:00 +01:00
Filip Barl
97fdcdc525 Option to censor raw reports by command line args and env vars. 2019-02-15 17:31:08 +01:00