From 3f1d71c09e49b333014f987cc1282ee425913559 Mon Sep 17 00:00:00 2001 From: Alessandro Puccetti Date: Wed, 2 Nov 2016 16:31:07 +0100 Subject: [PATCH 1/8] site/plugins: update the plugins list The plugins now are hosted in a new organization [Weaveworks Plugins] (https://github.com/weaveworks-plugins). This patch updates the list of the available plugins and their links to the new repositories. --- site/plugins.md | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/site/plugins.md b/site/plugins.md index e04be905a..cbfb83388 100644 --- a/site/plugins.md +++ b/site/plugins.md @@ -5,6 +5,7 @@ menu_order: 80 The following topics are discussed: + * [Official Plugins](#official-plugins) * [Listening Protocol](#listening-protocol) * [Reporting](#reporting) * [Interfaces](#interfaces) @@ -13,17 +14,21 @@ With a Scope probe plugin, you can insert custom metrics into Scope and have the ![Custom Metrics With Plugins](images/plugin-features.png) -You can find some examples in [the example plugins](https://github.com/weaveworks/scope/tree/master/examples/plugins) directory. +## Official Plugins -There are currently two different examples: +You can find all the official plugins at [Weaveworks Plugins](https://github.com/weaveworks-plugins). -* A [Python plugin](https://github.com/weaveworks/scope/tree/master/examples/plugins/http-requests) using [bcc](http://iovisor.github.io/bcc/) to extract incoming HTTP request rates per process, without any application-level instrumentation requirements and negligible performance toll (metrics are obtained in-kernel without any packet copying to userspace). +* [IOWait](https://github.com/weaveworks-plugins/scope-iowait): a Go plugin using [iostat](https://en.wikipedia.org/wiki/Iostat) to provide host-level CPU IO wait or idle metrics. ->**Note:** This plugin needs a [recent kernel version with ebpf support](https://github.com/iovisor/bcc/blob/master/INSTALL.md#kernel-configuration). It will not compile on current [dlite](https://github.com/nlf/dlite) and boot2docker hosts. +* [HTTP Statistics](https://github.com/weaveworks-plugins/scope-http-statistics): A Python plugin using [bcc](http://iovisor.github.io/bcc/) to track multiple metrics about HTTP per process, without any application-level instrumentation requirements and negligible performance toll. This plugin is a work in progress, as of now it implements two metrics (for more information read the [plugin documentation](https://github.com/weaveworks-plugins/scope-http-statistics)): + * Number of HTTP requests per seconds. + * Number of HTTP responses code per second (per code). - * A [Go plugin](https://github.com/weaveworks/scope/tree/master/examples/plugins/iovisor), using [iostat](https://en.wikipedia.org/wiki/Iostat) to provide host-level CPU IO wait metrics. +> **Note:** This plugin needs a [recent kernel version with ebpf support](https://github.com/iovisor/bcc/blob/master/INSTALL.md#kernel-configuration). It will not compile on current [dlite](https://github.com/nlf/dlite) and boot2docker hosts. -The example plugins are run by calling `make` in their directory. This builds the plugin, and immediately runs it in the foreground. To run the plugin in the background, see the `Makefile` for examples of the `docker run ...` command. +* [Traffic Control](https://github.com/weaveworks-plugins/scope-traffic-control): This plugin allows the user to modify latency and packet loss for a specific container via buttons in the UI's container detailed view. + +* [Volume Count](https://github.com/weaveworks-plugins/scope-volume-count): This plugin is a Python application that asks docker for the the number of mounted volumes for each container, providing container-level count. If the running plugin was picked up by Scope, you will see it in the list of `PLUGINS` in the bottom right of the UI. From 8894a32302bb061d4fa3b9f511a6527368f5a151 Mon Sep 17 00:00:00 2001 From: Alessandro Puccetti Date: Wed, 2 Nov 2016 17:16:55 +0100 Subject: [PATCH 2/8] site/plugins: update plugins internals This patch updates the plugins description on registration and reporting. --- site/plugins.md | 41 ++++++++++++++++++++++++++++------------- 1 file changed, 28 insertions(+), 13 deletions(-) diff --git a/site/plugins.md b/site/plugins.md index cbfb83388..ef8da18a2 100644 --- a/site/plugins.md +++ b/site/plugins.md @@ -6,9 +6,11 @@ menu_order: 80 The following topics are discussed: * [Official Plugins](#official-plugins) - * [Listening Protocol](#listening-protocol) - * [Reporting](#reporting) - * [Interfaces](#interfaces) + * [Plugins Internals](#plugins-internals) + * [Plugin ID](#plugin-id) + * [Plugin Registration](#plugin-registration) + * [Reporting](#interfaces-interface) + * [Other Interfaces](#other-interfaces) With a Scope probe plugin, you can insert custom metrics into Scope and have them display in the user interface together with the Scope's standard set of metrics. @@ -32,15 +34,30 @@ You can find all the official plugins at [Weaveworks Plugins](https://github.com If the running plugin was picked up by Scope, you will see it in the list of `PLUGINS` in the bottom right of the UI. -## Listening Protocol +## Plugins Internals + +This section explains the fundamental parts of the plugins structure necessary to understand how a plugin communicates with Scope. +You can find more practical examples in [Weaveworks Plugins](https://github.com/weaveworks-plugins) repositories. + +### Plugin ID + +Each plugin should have an unique ID. It is forbidden to change it +during the plugin's lifetime. The scope probe will get the plugin's ID +from the plugin's socket filename. For example, the socket named +`my-plugin.sock`, the scope probe will deduce the ID as +`my-plugin`. IDs can only contain alphanumeric sequences, optionally +separated with a dash. + +### Plugin registration All plugins must listen for HTTP connections on a Unix socket in the `/var/run/scope/plugins` directory. The Scope probe recursively scans that directory every 5 seconds, to look for any sockets being added (or removed). It is also valid to put the plugin Unix socket into a sub-directory, in case you want to apply some permissions, or store any other information with the socket. -When a new plugin is detected, the scope probe begins requesting reports from it via `GET /report`. +When a new plugin is detected, the scope probe begins requesting reports from it via `GET /report`. So every plugins **must** implement the report interface. +Implementing an interface means handling specific requests. All plugin endpoints are expected to respond within 500ms, and respond in the JSON format. -### Reporting +### Reporting Interface When the Scope probe discovers a new plugin Unix socket, it begins to periodically make a `GET` request to the `/report` endpoint. The report data structure returned from this will be merged into the probe's report and sent to the app. An example of the report structure can be viewed at the `/api/report` endpoint of any Scope app. @@ -63,17 +80,15 @@ For example: } ``` -> **Note:** The `Plugins` section includes exactly one plugin description. The plugin description fields are: `interfaces` including `reporter`. - -The fields are: +> **Note:** The `Plugins` section includes exactly one plugin description. The plugin description fields are: * `id` is used to check for duplicate plugins. It is required. * `label` is a human readable plugin label displayed in the UI. It is required. -* `description` is displayed in the UI -* `interfaces` is a list of interfaces which this plugin supports. It is required, and must equal `["reporter"]`. -* `api_version` is used to ensure both the plugin and the scope probe can speak to each other. It is required, and must match the probe. +* `description` is displayed in the UI. It is required. +* `interfaces` is a list of interfaces which this plugin supports. It is required, and must contain at least `["reporter"]`. +* `api_version` is used to ensure both the plugin and the scope probe can speak to each other. It is required, and must match the probe's value. -### Interfaces +### Other Interfaces Currently the only interface a plugin can fulfill is `reporter`. From d7e523e19e1c15f552af4e6391dd09d221cafe04 Mon Sep 17 00:00:00 2001 From: Alessandro Puccetti Date: Wed, 2 Nov 2016 17:33:03 +0100 Subject: [PATCH 3/8] site/plugins: add controls documentation --- site/plugins.md | 128 ++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 119 insertions(+), 9 deletions(-) diff --git a/site/plugins.md b/site/plugins.md index ef8da18a2..8af19e579 100644 --- a/site/plugins.md +++ b/site/plugins.md @@ -9,8 +9,8 @@ The following topics are discussed: * [Plugins Internals](#plugins-internals) * [Plugin ID](#plugin-id) * [Plugin Registration](#plugin-registration) - * [Reporting](#interfaces-interface) - * [Other Interfaces](#other-interfaces) + * [Reporter Interface](#reporter-interface) + * [Controller Interface](#controller-interface) With a Scope probe plugin, you can insert custom metrics into Scope and have them display in the user interface together with the Scope's standard set of metrics. @@ -57,7 +57,7 @@ Implementing an interface means handling specific requests. All plugin endpoints are expected to respond within 500ms, and respond in the JSON format. -### Reporting Interface +### Reporter Interface When the Scope probe discovers a new plugin Unix socket, it begins to periodically make a `GET` request to the `/report` endpoint. The report data structure returned from this will be merged into the probe's report and sent to the app. An example of the report structure can be viewed at the `/api/report` endpoint of any Scope app. @@ -67,12 +67,12 @@ For example: ```json { - "Processes": {}, + ..., "Plugins": [ { - "id": "iowait", - "label": "IOWait", - "description": "Adds a graph of CPU IO Wait to hosts", + "id": "plugin-id", + "label": "Human Friendly Name", + "description": "Plugin's brief description", "interfaces": ["reporter"], "api_version": "1", } @@ -88,9 +88,119 @@ For example: * `interfaces` is a list of interfaces which this plugin supports. It is required, and must contain at least `["reporter"]`. * `api_version` is used to ensure both the plugin and the scope probe can speak to each other. It is required, and must match the probe's value. -### Other Interfaces +### Controller Interface + +Plugins _may_ implement the controller interface. Implementing the +controller interface means that the plugin can react to HTTP `POST` +control requests sent by the app. The plugin will receive them only +for controls it exposed in its reports. The requests will come to the +`/control` endpoint. + +Add the "controller" string to the interfaces field in the plugin specification. + +#### Control + +The `POST` requests will have a JSON-encoded body with the following contents: + +```json +{ + "AppID": "some ID of an app", + "NodeID": "an ID of the node that had the control activated", + "Control": "the name of the activated control" +} +``` + +The body of the response should also be a JSON-encoded data. Usually +the body would be an empty JSON object (so, "{}" after +serialization). If some error happens during handling the control, +then the plugin can send a response with an `error` field set, for +example: + +```json +{ + "error": "An error message here" +} +``` + +Sometimes the control activation can make the control obsolete, so the +plugin may want to hide it (for example, control for stopping the +container should be hidden after the container is stopped). For this +to work, the plugin can send a shortcut report by filling the +`ShortcutReport` field in the response, like for example: + +```json +{ + "ShortcutReport": { body of the report here } +} +``` + +#### How to expose controls + +Each topology in the report (be it host, pod, endpoint and so on) has +a set of available controls a node in the topology may want to +show. The following (rather artificial) example shows a topology with +two controls (`ctrl-one` and `ctrl-two`) and two nodes, each having a +different control from the two: + +```json +{ + "Host": { + "controls": { + "ctrl-one": { + "id": "ctrl-one", + "human": "Ctrl One", + "icon": "fa-futbol-o", + "rank": 1 + }, + "ctrl-two": { + "id": "ctrl-two", + "human": "Ctrl Two", + "icon": "fa-beer", + "rank": 2 + } + }, + "nodes": { + "host1": { + "latestControls": { + "ctrl-one": { + "timestamp": "2016-07-20T15:51:05Z01:00", + "value": { + "dead": false + } + } + } + }, + "host2": { + "latestControls": { + "ctrl-two": { + "timestamp": "2016-07-20T15:51:05Z01:00", + "value": { + "dead": false + } + } + } + } + } + } +} +``` + +When control "ctrl-one" is activated, the plugin will receive a +request like: + +```json +{ + "AppID": "some ID of an app", + "NodeID": "host1", + "Control": "ctrl-one" +} +``` + +A short note about the "icon" field of the topology control - the +value for it can be taken from [Font Awesome +Cheatsheet](http://fontawesome.io/cheatsheet/) + -Currently the only interface a plugin can fulfill is `reporter`. **See Also** From 0ef871134377add13a2da5b4042b870040688303 Mon Sep 17 00:00:00 2001 From: Alessandro Puccetti Date: Wed, 2 Nov 2016 17:34:45 +0100 Subject: [PATCH 4/8] site/plugins: add node naming documentation This patch adds the documentation on the node naming conventions and rules. --- site/plugins.md | 47 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) diff --git a/site/plugins.md b/site/plugins.md index 8af19e579..5938e881d 100644 --- a/site/plugins.md +++ b/site/plugins.md @@ -200,7 +200,54 @@ A short note about the "icon" field of the topology control - the value for it can be taken from [Font Awesome Cheatsheet](http://fontawesome.io/cheatsheet/) +#### Node naming +Very often the controller plugin wants to add some controls to already +existing nodes (like controls for network traffic management to nodes +representing the running Docker container). To achieve that, it is +important to make sure that the node ID in the plugin's report matches +the ID of the node created by the probe. The ID is a +semicolon-separated list of strings. + +For containers, images, hosts and others the ID is usually formatted +as `${name};<${tag}>`. The `${name}` variable is usually a name of a +thing the node represents, like an ID of the Docker container or the +hostname. The `${tag}` denotes the type of the node. There is a fixed +set of tags used by the probe: + +- host +- container +- container_image +- pod +- service +- deployment +- replica_set + +The examples of "tagged" node names: + +- The Docker container with full ID + 2299a2ca59dfd821f367e689d5869c4e568272c2305701761888e1d79d7a6f51: + `2299a2ca59dfd821f367e689d5869c4e568272c2305701761888e1d79d7a6f51;` +- The Docker image with name `docker.io/alpine`: + `docker.io/alpine;` +- The host with name `example.com`: `example.com;` + +The fixed set of tags listed above is not a complete set of names a +node can have though. For example, nodes representing processes +have ID formatted as `${host};${pid}`. Probably the easiest ways to +discover how the nodes are named are: + +- Read the code in + [report/id.go](https://github.com/weaveworks/scope/blob/master/report/id.go). +- Browse the Weave Scope GUI, select some node and search for an `id` + key in the `nodeDetails` array in the address bar. + - For example in the + `http://localhost:4040/#!/state/{"controlPipe":null,"nodeDetails":[{"id":"example.com;","label":"example.com","topologyId":"hosts"}],…` + URL, you can find the `example.com;` which is an ID of the node + representing the host. + - Mentally substitute the `` with `/`. This can appear in + Docker image names, so `docker.io/alpine` in the address bar will + be `docker.ioalpine`. **See Also** From ef0cee946a1ec61f9ecc0a35c3e838d33324b69f Mon Sep 17 00:00:00 2001 From: Alessandro Puccetti Date: Thu, 3 Nov 2016 11:30:14 +0100 Subject: [PATCH 5/8] site/plugins: add developing guide This path adds a guide with code examples based on the Scope IOWait plugin on how to develop a plugin for scope. --- site/plugins.md | 136 +++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 135 insertions(+), 1 deletion(-) diff --git a/site/plugins.md b/site/plugins.md index 5938e881d..dc9d3ae0c 100644 --- a/site/plugins.md +++ b/site/plugins.md @@ -11,6 +11,7 @@ The following topics are discussed: * [Plugin Registration](#plugin-registration) * [Reporter Interface](#reporter-interface) * [Controller Interface](#controller-interface) + * [Plugins Developing Guide](#plugins-developing-guide) With a Scope probe plugin, you can insert custom metrics into Scope and have them display in the user interface together with the Scope's standard set of metrics. @@ -249,7 +250,140 @@ discover how the nodes are named are: Docker image names, so `docker.io/alpine` in the address bar will be `docker.ioalpine`. - **See Also** + +## Plugins Developing Guide + +This section shows how to develop a simple plugin in Go. The following code is a simplified version of the [Scope IOWait](https://github.com/weaveworks-plugins/scope-iowait) plugin. + +### Base Structure + +As stated in the previous section, a plugins need to put a socket in the `/var/run/scope/plugins` to be able to communicate with Scope. +The best practice is to put the socket in a sub-directory with named with the plugin ID (e.g `/var/run/scope/plugins/plugins-id/plugins-id.sock`). +This is useful because the plugin can set more restrictive permissions to avoid unauthorized access and store other information along with the socket if needed. +Example of helper function for setting up the socket: + +```go +func setupSocket(socketPath string) (net.Listener, error) { + os.RemoveAll(filepath.Dir(socketPath)) + if err := os.MkdirAll(filepath.Dir(socketPath), 0700); err != nil { + return nil, fmt.Errorf("failed to create directory %q: %v", filepath.Dir(socketPath), err) + } + listener, err := net.Listen("unix", socketPath) + if err != nil { + return nil, fmt.Errorf("failed to listen on %q: %v", socketPath, err) + } + + log.Printf("Listening on: unix://%s", socketPath) + return listener, nil +} +``` + +Because Scope detects running plugins by looking into the `/var/run/scope/plugins` directory, plugins should remove their socket and the directory (if created) when they exit. +The side effect of not doing that is that the Scope UI will show that a plugin is running but is not reachable. +To do that you can use the following helper function: + +```go +func setupSignals(socketPath string) { + interrupt := make(chan os.Signal, 1) + signal.Notify(interrupt, os.Interrupt, syscall.SIGTERM) + go func() { + <-interrupt + os.RemoveAll(filepath.Dir(socketPath)) + os.Exit(0) + }() +} +``` + +Also you should add in the main function the following: + +```golang +defer func() { + os.RemoveAll(filepath.Dir(socketPath)) + }() +``` + +This will ensure that when the plugin terminates because of an error or an interrupt command, the `/var/run/scope/plugins/plugins-id` directory will be removed. +A bare minimum boilerplate can be the following: + +```go +package main + +import ( + syscall +) + +func main() { + const socketPath = "/var/run/scope/plugins/my-plugin/my-plugin.sock" + setupSignals(socketPath) + + listener, err := setupSocket(socketPath) + + plugin := &Plugin{} + http.HandleFunc("/report", plugin.Report) + + defer func() { + listener.Close() + os.RemoveAll(filepath.Dir(socketPath)) + }() +} + +``` + +### Report + +As stated in the [Plugins Internals](#plugins-internals) section, the reporter interface is mandatory. +Implementing the reporter interface means to handle `GET /report` requests so the following code is sufficient to implement it: + +```go +// Plugin groups the methods a plugin needs +type Plugin struct { + lock sync.Mutex +} + +type report struct { + Plugins []pluginSpec +} + +func (p *Plugin) makeReport() (*report, error) { + rpt := &report{ + Plugins: []pluginSpec{ + { + ID: "plugin-id", + Label: "Plugin Name", + Description: "Plugin short description", + Interfaces: []string{"reporter"}, + APIVersion: "1", + }, + }, + } + return rpt, nil +} + +// Report is called by scope when a new report is needed. It is part of the +// "reporter" interface, which all plugins must implement. +func (p *Plugin) Report(w http.ResponseWriter, r *http.Request) { + p.lock.Lock() + defer p.lock.Unlock() + log.Println(r.URL.String()) + rpt, err := p.makeReport() + if err != nil { + log.Printf("error: %v", err) + http.Error(w, err.Error(), http.StatusInternalServerError) + return + } + raw, err := json.Marshal(*rpt) + if err != nil { + log.Printf("error: %v", err) + http.Error(w, err.Error(), http.StatusInternalServerError) + return + } + w.WriteHeader(http.StatusOK) + w.Write(raw) +} + +``` + +**See Also** * [Building Scope](/site/building.md) From 8ca299e4f242978868f8cca7d0dc2ab1b784bbdf Mon Sep 17 00:00:00 2001 From: Anita Buehrle Date: Sun, 13 Nov 2016 15:19:49 -0600 Subject: [PATCH 6/8] Some edits to the plugin docs --- site/plugins.md | 165 +++++++++++++++++++++++++----------------------- 1 file changed, 86 insertions(+), 79 deletions(-) diff --git a/site/plugins.md b/site/plugins.md index dc9d3ae0c..01c915ab2 100644 --- a/site/plugins.md +++ b/site/plugins.md @@ -6,61 +6,66 @@ menu_order: 80 The following topics are discussed: * [Official Plugins](#official-plugins) - * [Plugins Internals](#plugins-internals) - * [Plugin ID](#plugin-id) - * [Plugin Registration](#plugin-registration) + * [How Plugins Communicate with Scope](#plugins-internals) + * [Plugin IDs](#plugin-id) + * [Registering Plugins](#plugin-registration) * [Reporter Interface](#reporter-interface) * [Controller Interface](#controller-interface) - * [Plugins Developing Guide](#plugins-developing-guide) + * [Control](#control) + * [How to Expose Controls](#expose-controls) + * [Naming Nodes](#naming-nodes) + * [A Guide to Developing Plugins](#plugins-developing-guide) + * [Setting up the Structure](#structure) + * [Defining the Reporter Interface](#reporter-interface) -With a Scope probe plugin, you can insert custom metrics into Scope and have them display in the user interface together with the Scope's standard set of metrics. +Any kind of metrics can be generated and inserted into Scope using custom plugins. Metrics generated through your plugin are displayed in the user interface alongside the standard set of metrics that are found in Weave Scope. ![Custom Metrics With Plugins](images/plugin-features.png) ## Official Plugins -You can find all the official plugins at [Weaveworks Plugins](https://github.com/weaveworks-plugins). +Official Weave Scope plugins can be found at [Weaveworks Plugins](https://github.com/weaveworks-plugins). -* [IOWait](https://github.com/weaveworks-plugins/scope-iowait): a Go plugin using [iostat](https://en.wikipedia.org/wiki/Iostat) to provide host-level CPU IO wait or idle metrics. +* [IOWait](https://github.com/weaveworks-plugins/scope-iowait): is a Go plugin that uses [iostat](https://en.wikipedia.org/wiki/Iostat) to provide host-level CPU IO wait or idle metrics. -* [HTTP Statistics](https://github.com/weaveworks-plugins/scope-http-statistics): A Python plugin using [bcc](http://iovisor.github.io/bcc/) to track multiple metrics about HTTP per process, without any application-level instrumentation requirements and negligible performance toll. This plugin is a work in progress, as of now it implements two metrics (for more information read the [plugin documentation](https://github.com/weaveworks-plugins/scope-http-statistics)): +* [HTTP Statistics](https://github.com/weaveworks-plugins/scope-http-statistics): is a Python plugin that uses [bcc](http://iovisor.github.io/bcc/) to track multiple metrics about HTTP per process. It does this without any application-level instrumentation requirements and without a negligible performance toll. This plugin is a work in progress, and currently implements the following (for more information read the [plugin documentation](https://github.com/weaveworks-plugins/scope-http-statistics)): * Number of HTTP requests per seconds. * Number of HTTP responses code per second (per code). -> **Note:** This plugin needs a [recent kernel version with ebpf support](https://github.com/iovisor/bcc/blob/master/INSTALL.md#kernel-configuration). It will not compile on current [dlite](https://github.com/nlf/dlite) and boot2docker hosts. +> **Note:** The HTTP Statistics plugin requires a [recent kernel version with ebpf support](https://github.com/iovisor/bcc/blob/master/INSTALL.md#kernel-configuration) and it will not compile on [dlite](https://github.com/nlf/dlite) or on boot2docker hosts. -* [Traffic Control](https://github.com/weaveworks-plugins/scope-traffic-control): This plugin allows the user to modify latency and packet loss for a specific container via buttons in the UI's container detailed view. +* [Traffic Control](https://github.com/weaveworks-plugins/scope-traffic-control): This plugin allows you to modify latency and packet loss for a specific container via controls from the container's detailed view in the Scope user interface. -* [Volume Count](https://github.com/weaveworks-plugins/scope-volume-count): This plugin is a Python application that asks docker for the the number of mounted volumes for each container, providing container-level count. +* [Volume Count](https://github.com/weaveworks-plugins/scope-volume-count): This plugin (written in Python) requests the number of mounted volumes for each container, and provides a container-level count. -If the running plugin was picked up by Scope, you will see it in the list of `PLUGINS` in the bottom right of the UI. +>**Note:**Installed and running plugins are shown in the list of `PLUGINS` in the bottom right of the Scope UI. -## Plugins Internals +## How Plugins Communicate with Scope -This section explains the fundamental parts of the plugins structure necessary to understand how a plugin communicates with Scope. -You can find more practical examples in [Weaveworks Plugins](https://github.com/weaveworks-plugins) repositories. +In this section how the different components of a plugin communicate with Scope are described. You can also find practical examples of how plugins work in the [Weaveworks Plugins](https://github.com/weaveworks-plugins) repositories. -### Plugin ID +### Plugin IDs -Each plugin should have an unique ID. It is forbidden to change it -during the plugin's lifetime. The scope probe will get the plugin's ID -from the plugin's socket filename. For example, the socket named -`my-plugin.sock`, the scope probe will deduce the ID as -`my-plugin`. IDs can only contain alphanumeric sequences, optionally -separated with a dash. +Each plugin must have a unique ID and this ID must not change +during the plugin's lifetime. Scope probes retrieve the plugin's ID +from the plugin's socket filename. For example, if a socket is named +`my-plugin.sock`, the scope probe deduces the ID as +`my-plugin`. IDs may contain only alphanumeric sequences that are optionally +separated by a dash. -### Plugin registration +### Registering Plugins -All plugins must listen for HTTP connections on a Unix socket in the `/var/run/scope/plugins` directory. The Scope probe recursively scans that directory every 5 seconds, to look for any sockets being added (or removed). It is also valid to put the plugin Unix socket into a sub-directory, in case you want to apply some permissions, or store any other information with the socket. +All plugins listen for HTTP connections on a UNIX socket in the `/var/run/scope/plugins` directory. The Scope probe recursively scans that directory every 5 seconds and looks for any added or removed sockets. -When a new plugin is detected, the scope probe begins requesting reports from it via `GET /report`. So every plugins **must** implement the report interface. -Implementing an interface means handling specific requests. +If you want to run permissions or store any other information with the socket, you can also put the plugin UNIX socket into a sub-directory. -All plugin endpoints are expected to respond within 500ms, and respond in the JSON format. +When a new plugin is detected, the Scope probe begins requesting reports from it via `GET /report`. It is therefore important that **every plugin implements the report interface**. Implementing the report interface also means handling specific requests. + +All plugin endpoints are expected to respond within 500ms, and **must** respond using the JSON format. ### Reporter Interface -When the Scope probe discovers a new plugin Unix socket, it begins to periodically make a `GET` request to the `/report` endpoint. The report data structure returned from this will be merged into the probe's report and sent to the app. An example of the report structure can be viewed at the `/api/report` endpoint of any Scope app. +When a Scope probe discovers a new plugin UNIX socket, it begins to periodically make a `GET` request to the `/report` endpoint. The report data structure returned from this is merged into the probe's report and sent to the app. An example of the report structure can be viewed at the `/api/report` endpoint of any Scope app. In addition to any data about the topology nodes, the report returned from the plugin must include some metadata about the plugin itself. @@ -81,27 +86,27 @@ For example: } ``` -> **Note:** The `Plugins` section includes exactly one plugin description. The plugin description fields are: +> **Note:** The `Plugins` section includes exactly one plugin description that displays in the UI. The other plugin fields are: -* `id` is used to check for duplicate plugins. It is required. -* `label` is a human readable plugin label displayed in the UI. It is required. -* `description` is displayed in the UI. It is required. -* `interfaces` is a list of interfaces which this plugin supports. It is required, and must contain at least `["reporter"]`. -* `api_version` is used to ensure both the plugin and the scope probe can speak to each other. It is required, and must match the probe's value. +* `id` - checks for duplicate plugins. It is required. +* `label` - a human readable label displayed in the UI. It is required. +* `description` - displayed in the UI. It is required. +* `interfaces` - a list of interfaces that the plugin supports. It is required, and must contain at least `["reporter"]`. +* `api_version` - ensure both the plugin and the scope probe can speak to each other. It is required, and must match the probe's value. ### Controller Interface -Plugins _may_ implement the controller interface. Implementing the +Plugins _may_ also implement the controller interface. Implementing the controller interface means that the plugin can react to HTTP `POST` -control requests sent by the app. The plugin will receive them only -for controls it exposed in its reports. The requests will come to the +control requests sent by the app. The plugin receives them only +for the controls it exposed in its reports. All such requests come to the `/control` endpoint. Add the "controller" string to the interfaces field in the plugin specification. -#### Control +#### Control -The `POST` requests will have a JSON-encoded body with the following contents: +The `POST` requests contain a JSON-encoded body with the following: ```json { @@ -111,10 +116,10 @@ The `POST` requests will have a JSON-encoded body with the following contents: } ``` -The body of the response should also be a JSON-encoded data. Usually -the body would be an empty JSON object (so, "{}" after -serialization). If some error happens during handling the control, -then the plugin can send a response with an `error` field set, for +The body of the response should also be JSON-encoded data. In most cases, +the body is an empty JSON object (so, "{}" after +serialization). If an error occurs when handling the control, +then the plugin sends a response with an `error` field set, for example: ```json @@ -123,11 +128,11 @@ example: } ``` -Sometimes the control activation can make the control obsolete, so the +Sometimes the control activation can make the control obsolete, and so the plugin may want to hide it (for example, control for stopping the container should be hidden after the container is stopped). For this -to work, the plugin can send a shortcut report by filling the -`ShortcutReport` field in the response, like for example: +to work, the plugin sends a shortcut report by filling the +`ShortcutReport` field in the response, like so: ```json { @@ -135,13 +140,12 @@ to work, the plugin can send a shortcut report by filling the } ``` -#### How to expose controls +#### How to Expose Controls -Each topology in the report (be it host, pod, endpoint and so on) has -a set of available controls a node in the topology may want to +Each topology in the report (be it host, pod, endpoint and so on) contains +a set of available controls that a node in the topology may want to show. The following (rather artificial) example shows a topology with -two controls (`ctrl-one` and `ctrl-two`) and two nodes, each having a -different control from the two: +two controls (`ctrl-one` and `ctrl-two`) and two nodes, each with a different control defined: ```json { @@ -186,8 +190,8 @@ different control from the two: } ``` -When control "ctrl-one" is activated, the plugin will receive a -request like: +When control "ctrl-one" is activated, the plugin receives a +request as follows: ```json { @@ -201,20 +205,21 @@ A short note about the "icon" field of the topology control - the value for it can be taken from [Font Awesome Cheatsheet](http://fontawesome.io/cheatsheet/) -#### Node naming +#### Naming Nodes -Very often the controller plugin wants to add some controls to already -existing nodes (like controls for network traffic management to nodes +Often the controller plugin may want to add controls to already +existing nodes (for example add controls for network traffic management to nodes representing the running Docker container). To achieve that, it is important to make sure that the node ID in the plugin's report matches the ID of the node created by the probe. The ID is a semicolon-separated list of strings. -For containers, images, hosts and others the ID is usually formatted +For containers, images, hosts and others, the ID is usually formatted as `${name};<${tag}>`. The `${name}` variable is usually a name of a thing the node represents, like an ID of the Docker container or the -hostname. The `${tag}` denotes the type of the node. There is a fixed -set of tags used by the probe: +hostname. The `${tag}` denotes the type of the node. + +There is a fixed set of tags used by the probe: - host - container @@ -224,7 +229,7 @@ set of tags used by the probe: - deployment - replica_set -The examples of "tagged" node names: +These are examples of "tagged" node names: - The Docker container with full ID 2299a2ca59dfd821f367e689d5869c4e568272c2305701761888e1d79d7a6f51: @@ -235,32 +240,31 @@ The examples of "tagged" node names: The fixed set of tags listed above is not a complete set of names a node can have though. For example, nodes representing processes -have ID formatted as `${host};${pid}`. Probably the easiest ways to +have IDs formatted as `${host};${pid}`. The easiest way to discover how the nodes are named are: -- Read the code in + 1. Read the code in [report/id.go](https://github.com/weaveworks/scope/blob/master/report/id.go). -- Browse the Weave Scope GUI, select some node and search for an `id` + 2. Browse the Weave Scope GUI, select some node and search for an `id` key in the `nodeDetails` array in the address bar. - For example in the `http://localhost:4040/#!/state/{"controlPipe":null,"nodeDetails":[{"id":"example.com;","label":"example.com","topologyId":"hosts"}],…` URL, you can find the `example.com;` which is an ID of the node representing the host. - - Mentally substitute the `` with `/`. This can appear in + 3. Mentally substitute the `` with `/`. This can appear in Docker image names, so `docker.io/alpine` in the address bar will be `docker.ioalpine`. -## Plugins Developing Guide +## A Guide to Developing Plugins -This section shows how to develop a simple plugin in Go. The following code is a simplified version of the [Scope IOWait](https://github.com/weaveworks-plugins/scope-iowait) plugin. +This section explains how to develop a simple plugin in Go. The code used here is a simplified version of the [Scope IOWait](https://github.com/weaveworks-plugins/scope-iowait) plugin. -### Base Structure +### Setting up the Structure -As stated in the previous section, a plugins need to put a socket in the `/var/run/scope/plugins` to be able to communicate with Scope. -The best practice is to put the socket in a sub-directory with named with the plugin ID (e.g `/var/run/scope/plugins/plugins-id/plugins-id.sock`). -This is useful because the plugin can set more restrictive permissions to avoid unauthorized access and store other information along with the socket if needed. -Example of helper function for setting up the socket: +As stated in the previous section, plugins need to be put into the `/var/run/scope/plugins` socket directory to be able to communicate with Scope. The best practice is to put the socket into a sub-directory and name it with the plugin ID (for example, `/var/run/scope/plugins/plugins-id/plugins-id.sock`). This is useful because the plugin can set more restrictive permissions to avoid unauthorized access as well as store other information along with the socket if needed. + +Example of a helper function for setting up the socket: ```go func setupSocket(socketPath string) (net.Listener, error) { @@ -278,9 +282,9 @@ func setupSocket(socketPath string) (net.Listener, error) { } ``` -Because Scope detects running plugins by looking into the `/var/run/scope/plugins` directory, plugins should remove their socket and the directory (if created) when they exit. -The side effect of not doing that is that the Scope UI will show that a plugin is running but is not reachable. -To do that you can use the following helper function: +Because Scope detects running plugins by looking into the `/var/run/scope/plugins` directory, plugins should remove their socket and the directory (if created) when they exit. The side effect of not doing that is that the Scope UI will show that a plugin is running but that it is not reachable. + +To remove the socket, and the directory, you can use the following helper function: ```go func setupSignals(socketPath string) { @@ -294,7 +298,7 @@ func setupSignals(socketPath string) { } ``` -Also you should add in the main function the following: +Also add the following to the main function the following: ```golang defer func() { @@ -302,7 +306,8 @@ defer func() { }() ``` -This will ensure that when the plugin terminates because of an error or an interrupt command, the `/var/run/scope/plugins/plugins-id` directory will be removed. +This ensures that when the plugin terminates because of an error or an interrupt command, the `/var/run/scope/plugins/plugins-id` directory will be removed. + A bare minimum boilerplate can be the following: ```go @@ -329,10 +334,12 @@ func main() { ``` -### Report +### Defining the Reporter Interface -As stated in the [Plugins Internals](#plugins-internals) section, the reporter interface is mandatory. -Implementing the reporter interface means to handle `GET /report` requests so the following code is sufficient to implement it: +As stated in the [How Plugins Communicate with Scope](#plugins-internals) section, the reporter interface is mandatory. +Implementing the reporter interface means handling `GET /report` requests. + +The following code snippet is sufficient to implement it: ```go // Plugin groups the methods a plugin needs From 30bd628852b4bb64a02d4ce6f8f7e99da2b2d5be Mon Sep 17 00:00:00 2001 From: Anita Buehrle Date: Sun, 13 Nov 2016 15:32:27 -0600 Subject: [PATCH 7/8] put tags in code format --- site/plugins.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/site/plugins.md b/site/plugins.md index 01c915ab2..425f3a38e 100644 --- a/site/plugins.md +++ b/site/plugins.md @@ -221,13 +221,13 @@ hostname. The `${tag}` denotes the type of the node. There is a fixed set of tags used by the probe: -- host -- container -- container_image -- pod -- service -- deployment -- replica_set +- `host` +- `container` +- `container_image` +- `pod` +- `service` +- `deployment` +- `replica_set` These are examples of "tagged" node names: From d26a320ebecce13c6af7ebef593bb8a26cdc1147 Mon Sep 17 00:00:00 2001 From: Alessandro Puccetti Date: Mon, 14 Nov 2016 14:25:21 +0100 Subject: [PATCH 8/8] site/plugins: remove trailing whitespaces. --- site/plugins.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/site/plugins.md b/site/plugins.md index 425f3a38e..126ffcd89 100644 --- a/site/plugins.md +++ b/site/plugins.md @@ -13,7 +13,7 @@ The following topics are discussed: * [Controller Interface](#controller-interface) * [Control](#control) * [How to Expose Controls](#expose-controls) - * [Naming Nodes](#naming-nodes) + * [Naming Nodes](#naming-nodes) * [A Guide to Developing Plugins](#plugins-developing-guide) * [Setting up the Structure](#structure) * [Defining the Reporter Interface](#reporter-interface) @@ -55,7 +55,7 @@ separated by a dash. ### Registering Plugins -All plugins listen for HTTP connections on a UNIX socket in the `/var/run/scope/plugins` directory. The Scope probe recursively scans that directory every 5 seconds and looks for any added or removed sockets. +All plugins listen for HTTP connections on a UNIX socket in the `/var/run/scope/plugins` directory. The Scope probe recursively scans that directory every 5 seconds and looks for any added or removed sockets. If you want to run permissions or store any other information with the socket, you can also put the plugin UNIX socket into a sub-directory. @@ -217,7 +217,7 @@ semicolon-separated list of strings. For containers, images, hosts and others, the ID is usually formatted as `${name};<${tag}>`. The `${name}` variable is usually a name of a thing the node represents, like an ID of the Docker container or the -hostname. The `${tag}` denotes the type of the node. +hostname. The `${tag}` denotes the type of the node. There is a fixed set of tags used by the probe: @@ -337,7 +337,7 @@ func main() { ### Defining the Reporter Interface As stated in the [How Plugins Communicate with Scope](#plugins-internals) section, the reporter interface is mandatory. -Implementing the reporter interface means handling `GET /report` requests. +Implementing the reporter interface means handling `GET /report` requests. The following code snippet is sufficient to implement it: