Paul Bellamy 56122dd0cc Details panel backend redesign
Megasquish:
  [app] remove unused edge endpoint
  [WIP] refactoring node details api endpoint
  [WIP] plumbing the children through the rendering process
  adding IDList.Remove and StringSet.Remove
  [WIP] working on adding parents to detailed node renderings
  WIP UI components with mock backend data for new details
  grouping children by type
  UI components for node details health and info
  metric formatters for details panel
  Column headers and links for details table
  [WIP] started on rendering node metadata and metrics in the detail view
  DetailedNode.LabelMajor -> DetailedNode.Label
  rendering decent labels for parents of detailed nodes
  render metrics onto the top-level detailed node
  removing dead code
  Links to relatives
  metrics have a Format not Unit
  Show more/less actions for tables and relatives
  adjusted metric formatter
  TopologyTagger should tag k8s topology nodes
  make renderablenode ids more consistent, e.g. container:abcd1234
  working on rendering correct summaries for each node
  adding report.Node.Rank, so that merging is independent of order
  rendering children and parents correctly
  output child renderableNode ids, so we can link to them
  add group field to metrics, so they can be grouped
  Refactored details health items to prepare for grouping
  add metrics to processNodeSummaries
  hide summary section if there is no data for it
  fixing up tests
  moving detailed node rendering into a separate package
  Node ID/Topology are fields not metadata
    - This way I think we don't have to care about Metadata being non-commutative.
    - ID and topology are still non-commutative, as I'm not sure how to sanely
  merge them, but it's possible we don't care.
  host memory usage is a filesize, not a percent
  working on fixing some tests
  adding children to hosts detail panel
    - Had to redo how parents are calculated, so that children wouldn't interfere with it
    - have to have the host at the end because it is non-commutative
  only render links for linkable children (i.e. not unconnected processes)
  resolving TODOs
  fixing up lint errors
  make nil a valid value for render.Children so tests are cleaner
  working on backend tests
  make client handle missing metrics property
  Stop rendering container image nodes with process summaries/parents
  fix parent link to container images
  Calculate parents as a set on report.Node (except k8s)
  refactoring detailed.NodeSummary stuff
  removing RenderableNode.Summary*, we already track it on report.Node
  working on tests
  add Columns field to NodeSummaryGroup
  fixing up render/topologies_test
  fix children links to container images
  get children of hosts rendering right
  working on host renderer tests
  Change container report.Node.ID to a1b2c3;<container>
  The id should be globally unique, so we don't need the host id.
    This lets the kubernetes probe return a container node with the pod id,
    which will get merged into the real containers with other reports. The
    catch is that the kubernetes api doesn't tell us which hostname the
    container is running on, so we can't populate the old-style node ids.
  change terminology of system pods and services
  Fix kubernetes services with no selector
  Fixes handling of kubernetes service, which has no pods
  fix parent links for pods/services
  refactor detailed metadata to include sets and latest data
  fixing up host rendering tests
  fleshing out tests for node metadata and metrics
  don't render container pseudo-nodes as processes
  Update test for id format change.
2016-01-19 16:39:37 +01:00
2016-01-19 16:39:37 +01:00
2016-01-06 14:01:39 +00:00
2016-01-19 16:39:37 +01:00
2016-01-19 16:39:37 +01:00
2016-01-19 16:39:37 +01:00
2016-01-19 16:39:37 +01:00
2016-01-19 16:39:37 +01:00
2016-01-04 16:48:19 +00:00
2016-01-06 11:48:29 +00:00
2015-12-15 16:48:23 +00:00
2015-05-19 10:02:02 +00:00
2016-01-16 11:56:34 -08:00
2015-05-19 10:02:55 +00:00
2015-12-11 12:23:01 +00:00

Scope

Circle CI Coverage Status Sourcegraph

Weave Scope Screenshot

Overview

Weave Scope automatically generates a map of your containers, enabling you to intuitively understand, monitor, and control your applications.

Getting started

sudo wget -O /usr/local/bin/scope \
  https://github.com/weaveworks/scope/releases/download/latest_release/scope
sudo chmod a+x /usr/local/bin/scope
sudo scope launch

This script will download and run a recent Scope image from the Docker Hub. Now, open your web browser to http://localhost:4040. (If you're using boot2docker, replace localhost with the output of boot2docker ip.)

Getting help

If you have any questions about, feedback for or problem with Scope we invite you to:

Your feedback is always welcome!

Requirements

Scope does not need any configuration and does not require the Weave Network. Scope does need to be running on every machine you want to monitor.

Scope allows anyone with access to the UI control over your containers: as such, the Scope app endpoint (port 4040) should not be made accessible on the Internet. Additionally traffic between the app and the probe is currently insecure and should not traverse the internet.

Architecture

Weave Scope consists of two components: the app and the probe. These two components are deployed as a single Docker container using the scope script.

The probe is responsible for gathering information about the host is it running on. This information is sent to the app in the form of a report. The app is responsible for processing reports from the probe into usable topologies, serving the UI, and pushing these topologies to the UI.

+--Docker host----------+
|  +--Container------+  |    .---------------.
|  |                 |  |    | Browser       |
|  |  +-----------+  |  |    |---------------|
|  |  | scope-app |<---------|               |
|  |  +-----------+  |  |    |               |
|  |        ^        |  |    |               |
|  |        |        |  |    '---------------'
|  | +-------------+ |  |
|  | | scope-probe | |  |
|  | +-------------+ |  |
|  |                 |  |
|  +-----------------+  |
+-----------------------+

Using Weave Scope in Standalone Mode

When running Scope in a cluster, each probe sends reports to each app. The App merges the reports from each probe into a more complete report. You need to run Scope on every machine you want to monitor.

+--Docker host----------+      +--Docker host----------+
|  +--Container------+  |      |  +--Container------+  |
|  |                 |  |      |  |                 |  |
|  |  +-----------+  |  |      |  |  +-----------+  |  |
|  |  | scope-app |<-----.    .----->| scope-app |  |  |
|  |  +-----------+  |  | \  / |  |  +-----------+  |  |
|  |        ^        |  |  \/  |  |        ^        |  |
|  |        |        |  |  /\  |  |        |        |  |
|  | +-------------+ |  | /  \ |  | +-------------+ |  |
|  | | scope-probe |-----'    '-----| scope-probe | |  |
|  | +-------------+ |  |      |  | +-------------+ |  |
|  |                 |  |      |  |                 |  |
|  +-----------------+  |      |  +-----------------+  |
+-----------------------+      +-----------------------+

If you run Scope on the same machine as the Weave Network, the probe will use weaveDNS to automatically discover other apps on your network. Scope acheives this by registering itself under the address scope.weave.local. Each probe will send reports to every app registered under this address. Therefore, if you have a running weaveDNS setup, you do not need to take any further steps.

If you do not wish to use weaveDNS, you can instruct Scope to cluster with other Scope instances on the command line. Hostnames and IP addresses are acceptable, both with and without ports:

# scope launch scope1:4030 192.168.0.12 192.168.0.11:4030

Hostnames will be regularly resolved as A records, and each answer used as a target.

Using Weave Scope in Cloud Service Mode

Scope can also be used to feed reports to the Scope Service. The Scope Service allows you centrally manage and share access to your Scope UI. In this configuration, you only run the probe locally; the apps are hosted for you.

To get an account on the Scope Service, sign up at scope.weave.works. You need to run a probe on every machine you want to monitor with Scope. To launch a probe and send reports to the service, run the following command:

sudo scope launch --service-token=<token>
                       .-~~~-.
                 .- ~'`       )_   ___
                /               `-'   )_
               |    scope.weave.works   \
                \                      .'
                  ~-______________..--'
                           ^^
                           ||
                           ||
+--Docker host----------+  ||  +--Docker host----------+
|  +--Container------+  |  ||  |  +--Container------+  |
|  |                 |  |  ||  |  |                 |  |
|  | +-------------+ |  | /  \ |  | +-------------+ |  |
|  | | scope-probe |-----'    '-----| scope-probe | |  |
|  | +-------------+ |  |      |  | +-------------+ |  |
|  |                 |  |      |  |                 |  |
|  +-----------------+  |      |  +-----------------+  |
+-----------------------+      +-----------------------+

Using Weave Scope with Kubernetes

Scope comes with built-in Kubernetes support. We recommend to run Scope natively in your Kubernetes cluster using this resource definitions.

  1. If you are running a Kubernetes version lower than 1.1, make sure your cluster allows running pods in privileged mode (required by the Scope probes). To allow privileged pods, your API Server and all your Kubelets must be provided with flag --allow_privileged at launch time.

  2. Make sure your cluster supports DaemonSets in your cluster. DaemonSets are needed to ensure that each Kubernetes node runs a Scope Probe:

    • To enable them in an existing cluster, make sure to add a --runtime-config=extensions/v1beta1/daemonsets=true argument to the apiserver's configuration (normally found at /etc/kubernetes/manifest/kube-apiserver.manifest) followed by a restart of the apiserver and controller manager.

    • If you are creating a new cluster, set KUBE_ENABLE_DAEMONSETS=true in your cluster configuration.

  3. Download the resource definitions:

for I in app-rc app-svc probe-ds; do curl -s -L https://raw.githubusercontent.com/TheNewNormal/kube-charts/master/weavescope/manifests/scope-$I.yaml -o scope-$I.yaml; done


4. Tweak the Scope probe configuration at `scope-probe-ds.yaml`, namely:
   * If you have an account at http://scope.weave.works and want to use Scope in
     Cloud Service Mode, uncomment the `--probe.token=foo` argument, substitute `foo`
     by the token found in your account page, and comment out the
     `$(WEAVE_SCOPE_APP_SERVICE_HOST):$(WEAVE_SCOPE_APP_SERVICE_PORT)` argument.

5. Install Scope in your cluster (order is important):
   

kubectl create -f scope-app-rc.yaml # Only if you want to run Scope in Standalone Mode kubectl create -f scope-app-svc.yaml # Only if you want to run Scope in Standalone Mode kubectl create -f scope-probe-ds.yaml



## <a name="developing"></a>Developing

The build is in two stages. `make deps` installs some tools we use later in
the build. `make` builds the UI build container, builds the UI in said
container, builds the backend build container, builds the app and probe in a
said container, and finally pushes the lot into a Docker image called
**weaveworks/scope**.

make deps make


Then, run the local build via

./scope launch


## <a name="developing"></a>Debugging

Scope has a collection of built in debugging tools to aid Scope delevopers.

- To have the app or probe dump their goroutine stacks, run:

pkill -SIGQUIT scope-(app|probe) docker logs weavescope


- The probe is instrumented with various counters and timers. To have it dump
  those values, run:

pkill -SIGUSR1 scope-probe docker logs weavescope


- The app and probe both include golang's pprof integration for gathering CPU
  and memory profiles.  To use these with the probe, you must launch Scope with
  the following arguments `scope launch --probe.http.listen :4041`.  You can
  then collect profiles in the usual way:

go tool pprof http://localhost:(4040|4041)/debug/pprof/profile

Description
Monitoring, visualisation & management for Docker & Kubernetes
Readme Pixar 103 MiB
Languages
Go 60.9%
JavaScript 28.3%
Shell 6.7%
Python 1.6%
HCL 1%
Other 1.5%