Commit Graph
822 Commits
Author SHA1 Message Date
gotjosh 3faf109b8f Fix intention of support for endpoints without protocols
When we receive an endpoint address without a protocol, our code states we
don't support them and that the format is deprecated.

In reality it was not the case, e.g:

When we received an address in the form of `127.0.0.1`, we'd attempt to
parse the scheme from it, we'd realise is does have one (would be
equivalent to "") and our function `parseEndpoint` would return `"", "", fmt.Error`.

Then, our `parseEndpointWithFallbackProtocol` would use the
fallback protocol (unix) and attempt to connect to `unix://127.0.0.1`.

This meant two things:

1. The error returned from `parseEndpoint` would never be thrown
2. We would connect anyways since the address is valid

This commit changes the assertion logic to match the intention of using
a fallback protocol when one is not supplied.
2018-11-01 20:21:54 +00:00
gotjosh beeb27810c Add tests for probe/cri/registry
Unhappy path tests try to cover three scenarios:

- When the endpoint URL scheme is not explicitly supported e.g. HTTP
- When the endpoint URL scheme is TCP which is also not supported
- When the fail to parse the given URL (to extract the scheme)

The happy path covers two scenarios:

- When we specify the supported scheme in the URL which is an unix
socket e.g. unix///var/run/dockershim.sock
- When we pass a socket address but fail to specify the scheme but our registry attempts
to use the fallback protocol e.g. var/run/dockershim.sock
2018-11-01 20:17:03 +00:00
Bryan Boreham 8cccbb695b Merge pull request #3274 from weaveworks/kubernetes-tagger
Enable Kubernetes objects to be reported just once in a cluster
2018-10-19 16:48:37 +01:00
Akash Srivastava 5b8a8c9f56 Merge branch 'master' into volume-snapshot 2018-10-17 23:36:09 +05:30
Bryan Boreham 2a5f871d74 Merge pull request #3379 from weaveworks/dont-report-dead-processes
Probe: Don't report dead or defunct processes
2018-10-16 16:01:16 +01:00
Akash Srivastava 4701c61969 Add clone support for volume snapshot
- Add control on volume snapshot to take clone
- Add control on volume snapshot to delete it

Signed-off-by: Akash Srivastava <akash.srivastava@openebs.io>
2018-10-16 01:04:44 +05:30
Akash Srivastava 8692a4d57d Add control on PVC to take volume snapshot
Signed-off-by: Akash Srivastava <akash.srivastava@openebs.io>
2018-10-16 01:04:44 +05:30
Akash Srivastava 81a682893f Add kubernetes volume snapshot support
- Add a new client in probe/kubernetes/client.go
  which will be used to fetch details of Snapshot CRD
- Visualize VolumeSnapshot and VolumeSnapshotData
- Add adjacency between PV and VolumeSnapshot
- Add adjacency between VolumeSnapshot and VolumeSnapshotData
- Add snapshot filter to hide and show snapshots

Signed-off-by: Akash Srivastava <akash.srivastava@openebs.io>
2018-10-16 01:04:44 +05:30
Bryan Boreham bff0302259 Merge pull request #3391 from weaveworks/only-local-pods
Discard pod updates for other nodes
2018-10-15 14:16:18 +01:00
Bryan Boreham b82aff1210 Discard pod updates for other nodes
We were filtering pods to those on the local node already, for regular
reports, so we should also filter events that come in asynchronously.
2018-10-14 15:24:34 +00:00
Bryan Boreham 195c4e030e Raise report publishing rate to 30ms default 2018-10-14 12:10:27 +00:00
Bryan Boreham 64197d15be Rate limit report publishing
So that, if many shortcut reports are produced in quick succession,
they will tend to get merged together.

Expand the queue size for shortcut reports, to avoid holding up the
producer so much.
2018-10-13 14:18:52 +00:00
Bryan Boreham 1279a02b7d Stop tagging pods with host ID
So they can be reported centrally, find the pod host ID from the child
containers.
2018-10-11 17:54:29 +00:00
Bryan Boreham 98d52bd480 Allow kubelet port to be disabled 2018-10-11 17:54:29 +00:00
Bryan Boreham dd21a55a1e Refactor: implement kubernetes tagger in separate struct 2018-10-11 17:54:28 +00:00
Bryan Boreham 2ec58da1cf Don't report dead processes
If the state is reported as dead or zombie, return an error so the
process will be skipped.
2018-10-09 14:32:02 +00:00
Ajay Kelkar 68bce4f57c Using assert for equality checks 2018-10-03 14:19:54 +05:30
Ajay Kelkar 27a0b2460d Absolute basic UT 2018-10-01 21:26:26 +05:30
Bryan Boreham 65bdb145f9 Merge pull request #3334 from weaveworks/registry-test-sleep
Sleep to stop TestRegistryDelete() failing
2018-09-01 13:47:30 +01:00
Bryan Boreham ef832eb128 Simplify fetch of all IP addresses
Code was unnecessarily fetching all links then fetching all addresses
filtered by link, when we can just get the addresses without any
filter.
2018-08-27 17:10:43 +00:00
Bryan Boreham ec71876879 Sleep to stop TestRegistryDelete() failing 2018-08-26 21:33:26 +00:00
meghalidhoble 625998b91e Change made to the listed files, to enable weaveworks-scope on Power(ppc64le)
1)backend/Dockerfile 2) probe/endpoint/dns_snooper.go
3) client/Dockerfile 4) docker/Dockerfile.cloud-agent
5) probe/process/walker_linux_test.go & 6) tools/lint

1)'backend/Dockerfile' : Conditional added so that the cross-compiling should
   be done on amd64. Also removed support for sh-lint for ppc64le for now.
   As the version for shfmt mentioned in the dockerfile is not available for
   ppc64le and the later version does't work fine with existing application.
2)'probe/endpoint/dns_snooper.go' : Renamed this file so as to reuse for ppc64le
   and added a build-constraint. Now this file will be build for amd64 on linux
   and ppc64le on linux.
3)'client/Dockerfile' : Modified the version of the base image for node from
   8.4.0 to 8.11, as this version supports multiarch.
4)'docker/Dockerfile.cloud-agent' : Modified the version of the base image for
   golang from 1.10.2-strech to 1.10.2, which supports multiarch.
5) 'probe/process/walker_linux_test.go' : Test fixed to run for ppc64le,
    modified the code to accept RSSBytes based on pageSize value per
    architecture, instead of hard-coded values.
6)'tools/lint' : Modified the file to skip the sh-lint implementation for ppc64le.

PR #3231
2018-08-13 12:45:25 +05:30
Bryan Boreham 50a83b060f Merge pull request #3279 from weaveworks/check-pid
Check container is running before trying to open its namespace
2018-08-07 09:32:21 +01:00
Bryan Boreham aa40f944d3 Merge pull request #3260 from openebs/storage-driver
Add storage driver name to Persistent Volume
2018-07-31 17:58:38 +01:00
Lili Cosic e6d9bcc1cb Add CRI probe
When the probe.cri is enabled the CRI probe will be used to gather
the container information via the CRI API. For now only the basic
information is included in the generated report, those that we can get
via the CRI ListContainersRequest.
2018-07-26 10:51:59 +01:00
Bryan Boreham 53c406f7e0 Check container is running before trying to open its namespace
Otherwise we get pointless errors.
2018-07-23 21:37:22 +00:00
Marc Carré bcbc462842 Fix probe & app's shutdown to please signals.SignalHandlerLoop
Fixes:
```
not enough arguments in call to signals.SignalHandlerLoop
	have ()
	want (logging.Interface, ...signals.SignalReceiver)
```
2018-07-23 20:10:19 +02:00
Marc Carré 2ba50b8b3d Update golang.org/x/net/context to latest
```
$ gvt delete golang.org/x/net/context
$ gvt fetch golang.org/x/net/context
2018/07/23 18:03:49 Fetching: golang.org/x/net/context
$ git grep -l "golang.org/x/net/context" | grep -v vendor | xargs sed -i '' 's:golang.org/x/net/context:context:g'
$ git grep -l "context/ctxhttp" | grep -v vendor | xargs sed -i '' 's:context/ctxhttp:golang.org/x/net/context/ctxhttp:g'
$ gofmt -s -w app
$ gofmt -s -w common
$ gofmt -s -w probe
$ gofmt -s -w prog
$ gofmt -s -w tools
```
fixed a bunch of:
```
cannot use func literal (type func("github.com/weaveworks/scope/vendor/golang.org/x/net/context".Context) error) as type func("context".Context) error
```
2018-07-23 20:10:18 +02:00
Marc Carré d46c2266ce Change Sirupsen/logrus to sirupsen/logrus
```
$ git grep -l Sirupsen | grep -v vendor | xargs sed -i '' 's:github.com/Sirupsen/logrus:github.com/sirupsen/logrus:g'
$ gofmt -s -w app
$ gofmt -s -w common
$ gofmt -s -w probe
$ gofmt -s -w prog
$ gofmt -s -w tools
```
2018-07-23 20:10:14 +02:00
Akash Srivastava 715747bda4 Add storage driver name to Persistent Volume
Every PV is backed by storage driver. This will show
storage driver in the PV metadata.

Signed-off-by: Akash Srivastava <akash.srivastava@openebs.io>
2018-07-20 11:41:06 +05:30
Bryan Boreham 5edc85822b AddParent() function to optimise the case where there is only one parent
Avoids creating and discarding a StringSet just to carry the
parameters, and makes the code more readable too.
2018-07-17 13:51:46 +00:00
Bryan Boreham 0d2409c72c Add method to add a single string into Sets
This avoids creating and discarding a StringSet just to pass the parameter
2018-07-17 13:51:46 +00:00
Bryan Boreham 761fafe61b Add benchmark for Kubernetes Report() 2018-07-16 19:41:49 +00:00
Bryan Boreham 80dbd3443c probe: Add -probe.publish.stdout option for debugging
This option gives a crude way to view the raw probe data as json in
the container logs, so that you can check exactly what it would have
sent.

We stub out the PipeClient interface with a dummy implementation in
this mode.
2018-06-19 22:59:01 +00:00
Bryan Boreham 06c895267c Reports: streamline report serialization
Move the creation of the buffer and the choice of compression level
(which never changes) into WriteBinary(), to simplify the code.
2018-06-19 22:59:01 +00:00
Bryan Boreham 56137211b5 probe: Eliminate appclient.reportPublisher
Simplification: move the 'noControls' functionality into the probe, as
we don't need a whole struct to do that.

The ReportPublisher interface also moves into probe where it belongs:
"the consumer should define the interface" - Dave Cheney
2018-06-19 22:59:01 +00:00
Bryan Boreham 96f51c47af probe: Eliminate Publisher interface from app_client
Simplification: everything now implements Publish(Report), and we do
away with writing/reading/writing in the MultiAppClient.
2018-06-19 22:51:25 +00:00
Bryan Boreham 8c18a785e7 Merge pull request #3132 from openebs/add-pv-pvc-support
Add Kubernetes Storage (pv and pvc) support in Weave Scope
2018-06-08 12:41:21 +01:00
Satyam Zode d9bd17af8a Refactor edge frontend code and reporter
- Refactor dotted edge logic.
- Change Storage view to show storage components as well as all
  the pods.
- Sentence case storage related variables.

Signed-off-by: Satyam Zode <satyam.zode@openebs.io>
2018-06-08 16:37:48 +05:30
Satyam Zode 24741bec2a Simplify volume rendering code in render
- Remove the complex volume rendering code.
- Remove volume specific code from Transform.
- Rename filtering methods
- Remove extra params from PV, PVC nodes to make table view look good.

Signed-off-by: Satyam Zode <satyam.zode@openebs.io>
2018-06-08 16:37:42 +05:30
Satyam Zode d26b2c3805 Add Kubernetes storage class resource to weave scope
This will:

- Add StorageClass resource. Storage classes are mentioned
in the PVC spec. We're using storage class name from PVC spec to
add adjacency to the PVC node.
- Add square sheet shape for StorageClass.
- Add storage filter in the PODS topology.
Storage Filter will allow user to see distinct view of
stateful applications.
- Add visually distinct edge to show storage adjacency.

Signed-off-by: Satyam Zode <satyam.zode@openebs.io>
2018-06-08 16:36:29 +05:30
Satyam Zode 2f69973de6 Add adjacencies for kubernetes storage components
- Kubernetes storage components such as PV and PVC are connected based on two
parameters Persistent volume claim name and Persistent Volume name.
- PVC contains the volume name which is, PV name itself. Hence, we can
show edge for PVC and PV.
- This will bring higher level visibility for kubernetes storage components.

Signed-off-by: Satyam Zode <satyam.zode@openebs.io>
2018-06-07 17:04:56 +05:30
Satyam Zode 23210a6a77 Add Kubernetes volumes support in Weave Scope
This will

- Add Kubernetes volume resources such as PV, PVC.
- Add shapes for Kubernetes PV and PVC
- Add `Cylinder` shape for PV and `Dotted Cylinder` shape for PVC.

Signed-off-by: Satyam Zode <satyam.zode@openebs.io>
2018-06-07 17:01:29 +05:30
Bryan Boreham 009af5be0c Probe: remove backwards-compatibility code when publishing reports
Removed to reduce CPU and memory usage in probes.

This code was added in August 2016 so that newer probes could be used
with older apps. Since then we have adopted the stance that new apps
will accept reports from old probes but not vice-versa, on a version
change.
2018-06-05 16:47:23 +00:00
Bryan Boreham 20ce708db9 Don't bother deduplicating IPs; they end up in a set 2018-06-04 16:41:24 +00:00
Bryan Boreham c6c51f36f7 Limit network namespace code to compile on Linux only 2018-06-04 10:54:02 +00:00
Bryan Boreham ade54ba84e probe: stop calling 'weave ps'
Now that we enter the container namespace to fetch IPs for every
container, there is no need to have 'weave ps' do it.

This does mean we lose Weave MAC addresses, but that is a rather
idiosyncratic feature anyway.
2018-06-02 22:22:08 +00:00
Bryan Boreham ff5b2affe0 probe: fetch container IP addresses from inside its namespace
So that we can pick up addresses added via CNI or other mechanisms
that Docker is not aware of.
2018-06-02 21:49:30 +00:00
Michael Schubert 7bb1e38de3 ebpf: update check for known faulty Ubuntu kernels
With c75700fe04 we added code to detect
Ubuntu Xenial kernels with a regression in the eBPF subsystem in order
to gently fallback to procfs scanning on such systems (and not crash the
host system by running eBPF code).

With the latest kernel update for Ubuntu Xenial, the bug was fixed:

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1763454

Therefore we can update the added check with an upper limit and make
sure that eBPF connection tracking only is disabled on kernels within
the range having the bug.

xref: https://github.com/weaveworks/scope/issues/3131
2018-05-23 11:38:04 +02:00
Stefan Prodan 439b67880e Fix pause image detection for Kubernetes 1.10 2018-05-19 14:03:17 +03:00