From 84723244ccd1336354e58d10d8a12fe1021b4f69 Mon Sep 17 00:00:00 2001 From: Alessandro Puccetti Date: Wed, 16 Nov 2016 16:43:06 +0100 Subject: [PATCH 01/12] site/plugins: add report data structures section This patch adds the report's data structures outline and specifications for the topologies, the nodes, the metadata templates, the table templates, the metric templates, the controls, the metadata, the metrics and the time window. --- site/plugins.md | 246 +++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 242 insertions(+), 4 deletions(-) diff --git a/site/plugins.md b/site/plugins.md index 126ffcd89..537093d69 100644 --- a/site/plugins.md +++ b/site/plugins.md @@ -16,7 +16,8 @@ The following topics are discussed: * [Naming Nodes](#naming-nodes) * [A Guide to Developing Plugins](#plugins-developing-guide) * [Setting up the Structure](#structure) - * [Defining the Reporter Interface](#reporter-interface) + * [Defining the Reporter Interface](#defining-reporter-interface) + * [Report Data Structures](#report-data-structures) Any kind of metrics can be generated and inserted into Scope using custom plugins. Metrics generated through your plugin are displayed in the user interface alongside the standard set of metrics that are found in Weave Scope. @@ -334,7 +335,7 @@ func main() { ``` -### Defining the Reporter Interface +### Defining the Reporter Interface As stated in the [How Plugins Communicate with Scope](#plugins-internals) section, the reporter interface is mandatory. Implementing the reporter interface means handling `GET /report` requests. @@ -390,8 +391,245 @@ func (p *Plugin) Report(w http.ResponseWriter, r *http.Request) { ``` +### Report Data structures +A report can contain many types of information. +If you go back to the [Reporter Interface](#reporter-interface) section, you see the top-level `Plugins` attribute. +Along with that, a report may contain multiple topologies. +An example of a report containing few topologies is the following: + +```json +{ + "Host": {}, + "Container": {}, + "Process": {}, + ..., + "Plugins": [...,] +} +``` + +### Topologies +Topologies can be of various types. +Each topology consists of a list of nodes and other information about the topology. +The available topologies are: + +- `Endpoint` nodes are (address, port) tuples on each host. +- `Process` nodes are processes on each host. +- `Container` nodes represent all Docker containers on hosts running probes. +- `Pods` nodes represent all Kubernetes pods running on hosts running probes. +- `Service` nodes represent all Kubernetes services running on hosts running probes. +- `Deployment` nodes represent all Kubernetes deployments running on hosts running probes. +- `ReplicaSet` nodes represent all Kubernetes ReplicaSets running on hosts running probes. +- `ContainerImage` nodes represent all Docker container images on hosts running probes. +- `Host` nodes are physical hosts that run probes. +- `Overlay` nodes are active peers in any software-defined network that's overlaid on the infrastructure. + +The topology structure consists of the following attributes: + +- `nodes` - is the list of the nodes that compose the topology. +- `controls` - contains the list of IDs of the active controls at a particular time. +- `metadata_templates` - contains the templates used to render data into the Scope UI. +- `table_templates` - contains the templates used to render tables into the Scope UI. +- `metric_templates` - contains the templates used to render metrics into the Scope UI. + +**Note**: These attribute are not required. But a topology with no `nodes` does not have any information to render, `metadata_templates`, as well as `table_templates`, are needed to know how to render the information carried by `nodes` in the Scope UI. + +### Nodes +A Node contains information about a specific element of a topology. +For example the Host topology will contain nodes describing all the hosts in it. +The same is for containers and Container topology, pods and Pod topology and so on. +Nodes are represented as follows: + +```json +{ + "Host": { + "nodes" : { + "hostID;": {...} + } + }, + "Container": { + "nodes" : { + "containerID;": {...} + } + }, + ... +} +``` + +Nodes are stored in dictionary. The ID of nodes representing hosts or containers have the format `ID;`, with type equal to the literal string `host` or `container` accordingly. The `ID` is the alphanumeric identifier of the nodes. +A node contains all the information about the represented object (e.g. host, container, pod, etc.). +In particular a node may contain: + +- `latest` - an id-value map containing the latest values. These values are a single piece of information. +- `latestControls` - the latest available controls. +- `metrics` - the collection of metrics to display in the UI. + +### Controls +Controls describe interfaces that expose actions that the user can perform on different objects (e.g. host, container, etc.). +Controls are an element of nodes. In this way, each control in a node is attached to it and performs an action on the object described by the node itself. Below is an example of how controls are represented in the JSON report. +In the report the attribute `latest_controls` contains all the controls exposed by scope and/or plugins, but only the alive ones will be listed in the attribute `controls`. + +```json +"controls": { + "timestamp": "2016-11-17T08:53:04.567890059Z", + "controls": [ + "switchToIOWait" + ] +}, +"latestControls": { + "switchToIOWait": { + "timestamp": "2016-11-17T08:53:03.171438309Z", + "value": { + "dead": false + } + }, + "switchToIdle": { + "timestamp": "2016-11-17T08:53:03.171438309Z", + "value": { + "dead": true + } + } +} +``` + +- `timestamp` is the timestamp of when the control was exposed. +- `value` is an object containing the control value. At the moment, only the state is available. + - `dead` is a boolean to know the state (active, dead) of a control. It is useful to show controls only when they are in a usable state. + +### Metadata +All metadata entries are placed within nodes in the section named `latest`. +This section contains the latest values to display and consists of `timestamp` and `value`. +Scope uses `metadata_templates` to know how to display such data. +To pair metadata with its template, it is necessary to use the `metadata-template-id` as a key to identify that particular piece of data. Example: + +```json +"metadata-templates": { + "metadata-templates-id": { + "id": "metadata-templates-id", + "label": "Human-readable description", + "priority": 1.6, + "from": "latest" + } +} + +"latest": { + "metadata-templates-id": { + "timestamp": "2016-11-17T08:53:02.189193735Z", + "value": "-" + } +} +``` + +### Metadata Templates +Metadata Templates describe a particular metadata item. +This description is used to extract metadata from a node and display it on Scope UI. + +```json +"metadata_templates": { + "metadata-template-id": { + "id": "metadata-template-id", + "label": "Human-readable description", + "dataType": "number", + "priority": 13.5, + "from": "latest" + }, + "another-metadata-template-id": {...} +} +``` + +- `metadata-template-identifier` and `id` identify a particular metadata template. +- `label` contains the label that will be used by Scope UI. +- `dataType` specifies the type of data, this will determine how the value is displayed. Possible values for this attribute are: number, ip, and datetime. +- `priority` is a floating point value used to decide the display ordering (lower values are displayed before higher ones). If it is omitted the UI display the value as last. +- `from` indicates where to look for the metadata. The possible values are: + - `latest` + - `sets` + - `counters` + +### Table Templates +Table Templates describe a table and how to identify which metadata templates belong to the table. + +```json +"table_templates": { + "table-template-id": { + "id": "table-template-id", + "label": "Human-readable description", + "prefix": "table-id-" + }, + "another-table-template-id": {...} +} +``` + +- `table-template-identifier` and `id` identify a particular table template. +- `label` contains the label that will be used by Scope UI. +- `prefix` is used to identify which metadata templates belong to the table. + +If you want to display data in a table, you have to define a table template and prepend the table prefix to all the metadata templates that identifies the data you want to put in such table. + +### Metrics +Metrics are a particular kind of data that can be plotted on the UI as a graph. +Scope uses the `metric_templates` to know how to display such data. To pair a metric with its template, it is necessary to use the `metric-template-id` as key to identify that particular metric. +Metrics are suitable for information such as CPU and memory usage, HTTP requests rate, I/O operations, etc. +The following is an example of a report with a metric and a its metric template: + +```json +"metric_templates": { + "metric-template-id": { + "id": "metric-id", + "label": "Human-readable description", + "format": "percent", + "priority": 1.6 + }, +} + +"metrics": { + "metadata-template-id": { + "samples": [ + { + "date": "2016-11-17T08:53:03.171424664Z", + "value": 98.24 + } + ], + "min": 0, + "max": 100 + } +} +``` + +- `samples` is the list of the samples for this report. +- `min` is the minimum value possible. +- `max` is the maximum value possible. + +### Metric Templates +Metric Templates describe a particular metric. +The following is an example of metric template: + +```json +"metric_templates": { + "metric-template-id": { + "id": "metric-id", + "label": "Human-readable description", + "format": "percent", + "priority": 1.6 + }, + "another-metric-template-id": {...} +} +``` + +- `metric-template-id` and `id` identify a particular metric template. +- `label` contains the label that will be used by Scope UI. +- `format` describes how the metrics is formatted. + - `percent` the metric value is a percentage. + - `filesize` the metric value is a file size (e.g. memory usage), it is displayed with the suffix KB, MB, GB, etc. + - `integer` the metric value is an integer. +- `priority` is a floating point value used to decide the display ordering (lower values are displayed before higher ones). + +### Time Window +A report may have an attribute called `Window`. +This is the time window, expressed as duration, within which the data contained in the report are considered valid. +The default window is 15 seconds. +You may change the window value using the option `-app.window ` when launching scope. +However, using values less than 15 seconds increases the change of information not be correctly displayed. + **See Also** * [Building Scope](/site/building.md) - - From d2fc48fa4ec4cdf9c8078e6fb9792c2ac691f934 Mon Sep 17 00:00:00 2001 From: Alban Crequy Date: Tue, 4 Apr 2017 15:28:25 +0200 Subject: [PATCH 02/12] site/plugins: data structures: fixes --- site/plugins.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/site/plugins.md b/site/plugins.md index 537093d69..75853508d 100644 --- a/site/plugins.md +++ b/site/plugins.md @@ -498,11 +498,12 @@ In the report the attribute `latest_controls` contains all the controls exposed ### Metadata All metadata entries are placed within nodes in the section named `latest`. This section contains the latest values to display and consists of `timestamp` and `value`. +Both should be written as strings in the json (with the double quotes). Scope uses `metadata_templates` to know how to display such data. To pair metadata with its template, it is necessary to use the `metadata-template-id` as a key to identify that particular piece of data. Example: ```json -"metadata-templates": { +"metadata_templates": { "metadata-templates-id": { "id": "metadata-templates-id", "label": "Human-readable description", @@ -514,7 +515,7 @@ To pair metadata with its template, it is necessary to use the `metadata-templat "latest": { "metadata-templates-id": { "timestamp": "2016-11-17T08:53:02.189193735Z", - "value": "-" + "value": "42" } } ``` @@ -538,7 +539,7 @@ This description is used to extract metadata from a node and display it on Scope - `metadata-template-identifier` and `id` identify a particular metadata template. - `label` contains the label that will be used by Scope UI. -- `dataType` specifies the type of data, this will determine how the value is displayed. Possible values for this attribute are: number, ip, and datetime. +- `dataType` specifies the type of data, this will determine how the value is displayed. Possible values for this attribute are: "number", "ip", "datetime" and "" for strings. - `priority` is a floating point value used to decide the display ordering (lower values are displayed before higher ones). If it is omitted the UI display the value as last. - `from` indicates where to look for the metadata. The possible values are: - `latest` From 176ffcfef40c79ac14b90a9640f921873f6b1276 Mon Sep 17 00:00:00 2001 From: Alban Crequy Date: Tue, 4 Apr 2017 15:50:57 +0200 Subject: [PATCH 03/12] site/plugins.md: fixes after Iago's review --- site/plugins.md | 34 +++++++++++++++++++--------------- 1 file changed, 19 insertions(+), 15 deletions(-) diff --git a/site/plugins.md b/site/plugins.md index 75853508d..21ffa9029 100644 --- a/site/plugins.md +++ b/site/plugins.md @@ -395,7 +395,7 @@ func (p *Plugin) Report(w http.ResponseWriter, r *http.Request) { A report can contain many types of information. If you go back to the [Reporter Interface](#reporter-interface) section, you see the top-level `Plugins` attribute. Along with that, a report may contain multiple topologies. -An example of a report containing few topologies is the following: +An example of a report containing a few topologies is the following: ```json { @@ -409,7 +409,7 @@ An example of a report containing few topologies is the following: ### Topologies Topologies can be of various types. -Each topology consists of a list of nodes and other information about the topology. +Each topology consists of some general information and a list of nodes. The available topologies are: - `Endpoint` nodes are (address, port) tuples on each host. @@ -431,12 +431,13 @@ The topology structure consists of the following attributes: - `table_templates` - contains the templates used to render tables into the Scope UI. - `metric_templates` - contains the templates used to render metrics into the Scope UI. -**Note**: These attribute are not required. But a topology with no `nodes` does not have any information to render, `metadata_templates`, as well as `table_templates`, are needed to know how to render the information carried by `nodes` in the Scope UI. +**Note**: These attribute are not required. But a topology with no `nodes` does not have any information to render. +`metadata_templates`, as well as `table_templates`, are needed to know how to render the information carried by `nodes` in the Scope UI. ### Nodes A Node contains information about a specific element of a topology. -For example the Host topology will contain nodes describing all the hosts in it. -The same is for containers and Container topology, pods and Pod topology and so on. +For example, the Host topology will contain nodes describing all the hosts in it. +The same applies for containers and the Container topology, pods and the Pod topology and so on. Nodes are represented as follows: ```json @@ -455,18 +456,21 @@ Nodes are represented as follows: } ``` -Nodes are stored in dictionary. The ID of nodes representing hosts or containers have the format `ID;`, with type equal to the literal string `host` or `container` accordingly. The `ID` is the alphanumeric identifier of the nodes. -A node contains all the information about the represented object (e.g. host, container, pod, etc.). -In particular a node may contain: +Nodes are stored in a dictionary. +The ID of nodes representing hosts or containers has the format `ID;`, with type equal to the literal string `host` or `container` respectively. +`ID` is the alphanumeric identifier of the nodes. -- `latest` - an id-value map containing the latest values. These values are a single piece of information. +A node contains all the information about the represented object (e.g. host, container, pod, etc.). +In particular, a node may contain: + +- `latest` - an id-value map containing the latest values. As opposed to the values in metrics, these values will not normally change. - `latestControls` - the latest available controls. - `metrics` - the collection of metrics to display in the UI. ### Controls Controls describe interfaces that expose actions that the user can perform on different objects (e.g. host, container, etc.). Controls are an element of nodes. In this way, each control in a node is attached to it and performs an action on the object described by the node itself. Below is an example of how controls are represented in the JSON report. -In the report the attribute `latest_controls` contains all the controls exposed by scope and/or plugins, but only the alive ones will be listed in the attribute `controls`. +In the report the attribute `latest_controls` contains all the controls exposed by scope and/or plugins, but only those alive will be listed in the attribute `controls`. ```json "controls": { @@ -491,7 +495,7 @@ In the report the attribute `latest_controls` contains all the controls exposed } ``` -- `timestamp` is the timestamp of when the control was exposed. +- `timestamp` specifies when the control was exposed. - `value` is an object containing the control value. At the moment, only the state is available. - `dead` is a boolean to know the state (active, dead) of a control. It is useful to show controls only when they are in a usable state. @@ -540,7 +544,7 @@ This description is used to extract metadata from a node and display it on Scope - `metadata-template-identifier` and `id` identify a particular metadata template. - `label` contains the label that will be used by Scope UI. - `dataType` specifies the type of data, this will determine how the value is displayed. Possible values for this attribute are: "number", "ip", "datetime" and "" for strings. -- `priority` is a floating point value used to decide the display ordering (lower values are displayed before higher ones). If it is omitted the UI display the value as last. +- `priority` is a floating point value used to decide the display ordering (lower values are displayed before higher ones). If omitted, the UI will display it last. - `from` indicates where to look for the metadata. The possible values are: - `latest` - `sets` @@ -564,13 +568,13 @@ Table Templates describe a table and how to identify which metadata templates be - `label` contains the label that will be used by Scope UI. - `prefix` is used to identify which metadata templates belong to the table. -If you want to display data in a table, you have to define a table template and prepend the table prefix to all the metadata templates that identifies the data you want to put in such table. +If you want to display data in a table, you have to define a table template and prepend the table prefix to all the metadata templates that identify the data you want to put in such table. ### Metrics Metrics are a particular kind of data that can be plotted on the UI as a graph. Scope uses the `metric_templates` to know how to display such data. To pair a metric with its template, it is necessary to use the `metric-template-id` as key to identify that particular metric. Metrics are suitable for information such as CPU and memory usage, HTTP requests rate, I/O operations, etc. -The following is an example of a report with a metric and a its metric template: +The following is an example of a report with a metric preceded by its metric template: ```json "metric_templates": { @@ -629,7 +633,7 @@ A report may have an attribute called `Window`. This is the time window, expressed as duration, within which the data contained in the report are considered valid. The default window is 15 seconds. You may change the window value using the option `-app.window ` when launching scope. -However, using values less than 15 seconds increases the change of information not be correctly displayed. +However, using values less than 15 seconds increases the chance of information not being correctly displayed. **See Also** From bcae01c580ea666a45fea0f9d03fac07c2ad7648 Mon Sep 17 00:00:00 2001 From: Alban Crequy Date: Tue, 4 Apr 2017 15:56:59 +0200 Subject: [PATCH 04/12] site/plugins.md: clarify the difference between latest and metrics --- site/plugins.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/site/plugins.md b/site/plugins.md index 21ffa9029..a7cf14334 100644 --- a/site/plugins.md +++ b/site/plugins.md @@ -463,9 +463,9 @@ The ID of nodes representing hosts or containers has the format `ID;`, wit A node contains all the information about the represented object (e.g. host, container, pod, etc.). In particular, a node may contain: -- `latest` - an id-value map containing the latest values. As opposed to the values in metrics, these values will not normally change. +- `latest` - an id-value map containing the latest values. Each id has only one value. - `latestControls` - the latest available controls. -- `metrics` - the collection of metrics to display in the UI. +- `metrics` - the collection of metrics to display in the UI. Each metric has multiple timestamped values. ### Controls Controls describe interfaces that expose actions that the user can perform on different objects (e.g. host, container, etc.). @@ -592,6 +592,10 @@ The following is an example of a report with a metric preceded by its metric tem { "date": "2016-11-17T08:53:03.171424664Z", "value": 98.24 + }, + { + "date": "2016-11-17T08:53:04.171789887Z", + "value": 80.11 } ], "min": 0, From 6966a8c04eedbc621db1a5f651f41a6bd0b9d550 Mon Sep 17 00:00:00 2001 From: Alban Crequy Date: Tue, 4 Apr 2017 16:03:02 +0200 Subject: [PATCH 05/12] site/plugins.md: english wording --- site/plugins.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/site/plugins.md b/site/plugins.md index a7cf14334..56226a438 100644 --- a/site/plugins.md +++ b/site/plugins.md @@ -637,7 +637,7 @@ A report may have an attribute called `Window`. This is the time window, expressed as duration, within which the data contained in the report are considered valid. The default window is 15 seconds. You may change the window value using the option `-app.window ` when launching scope. -However, using values less than 15 seconds increases the chance of information not being correctly displayed. +However, using values smaller than 15 seconds increases the chance of information not being correctly displayed. **See Also** From 73a2eb0b3b8fe349043c888fdec55fd604f3c6c5 Mon Sep 17 00:00:00 2001 From: Alban Crequy Date: Tue, 4 Apr 2017 16:27:22 +0200 Subject: [PATCH 06/12] site/plugins.md: more fixes thanks to Iago --- site/plugins.md | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/site/plugins.md b/site/plugins.md index 56226a438..5967b1c7d 100644 --- a/site/plugins.md +++ b/site/plugins.md @@ -393,7 +393,7 @@ func (p *Plugin) Report(w http.ResponseWriter, r *http.Request) { ### Report Data structures A report can contain many types of information. -If you go back to the [Reporter Interface](#reporter-interface) section, you see the top-level `Plugins` attribute. +If you go back to the [Reporter Interface](#reporter-interface) section, you will see the top-level `Plugins` attribute. Along with that, a report may contain multiple topologies. An example of a report containing a few topologies is the following: @@ -410,9 +410,9 @@ An example of a report containing a few topologies is the following: ### Topologies Topologies can be of various types. Each topology consists of some general information and a list of nodes. -The available topologies are: +These are the available topologies: -- `Endpoint` nodes are (address, port) tuples on each host. +- `Endpoint` nodes are `(address, port)` tuples on each host. - `Process` nodes are processes on each host. - `Container` nodes represent all Docker containers on hosts running probes. - `Pods` nodes represent all Kubernetes pods running on hosts running probes. @@ -470,7 +470,7 @@ In particular, a node may contain: ### Controls Controls describe interfaces that expose actions that the user can perform on different objects (e.g. host, container, etc.). Controls are an element of nodes. In this way, each control in a node is attached to it and performs an action on the object described by the node itself. Below is an example of how controls are represented in the JSON report. -In the report the attribute `latest_controls` contains all the controls exposed by scope and/or plugins, but only those alive will be listed in the attribute `controls`. +In the report, the attribute `latest_controls` contains all the controls exposed by scope and/or plugins, but only those alive will be listed in the attribute `controls`. ```json "controls": { @@ -502,7 +502,7 @@ In the report the attribute `latest_controls` contains all the controls exposed ### Metadata All metadata entries are placed within nodes in the section named `latest`. This section contains the latest values to display and consists of `timestamp` and `value`. -Both should be written as strings in the json (with the double quotes). +Both should be written as json strings (with double quotes). Scope uses `metadata_templates` to know how to display such data. To pair metadata with its template, it is necessary to use the `metadata-template-id` as a key to identify that particular piece of data. Example: @@ -546,9 +546,9 @@ This description is used to extract metadata from a node and display it on Scope - `dataType` specifies the type of data, this will determine how the value is displayed. Possible values for this attribute are: "number", "ip", "datetime" and "" for strings. - `priority` is a floating point value used to decide the display ordering (lower values are displayed before higher ones). If omitted, the UI will display it last. - `from` indicates where to look for the metadata. The possible values are: - - `latest` - - `sets` - - `counters` + - `latest` + - `sets` + - `counters` ### Table Templates Table Templates describe a table and how to identify which metadata templates belong to the table. @@ -627,9 +627,9 @@ The following is an example of metric template: - `metric-template-id` and `id` identify a particular metric template. - `label` contains the label that will be used by Scope UI. - `format` describes how the metrics is formatted. - - `percent` the metric value is a percentage. - - `filesize` the metric value is a file size (e.g. memory usage), it is displayed with the suffix KB, MB, GB, etc. - - `integer` the metric value is an integer. + - `percent` the metric value is a percentage. + - `filesize` the metric value is a file size (e.g. memory usage), it is displayed with the suffix KB, MB, GB, etc. + - `integer` the metric value is an integer. - `priority` is a floating point value used to decide the display ordering (lower values are displayed before higher ones). ### Time Window From 525beba83ce75642c8aca1999d202404327c7691 Mon Sep 17 00:00:00 2001 From: Michael Schubert Date: Tue, 18 Apr 2017 09:31:03 +0200 Subject: [PATCH 07/12] Mention new `ECSTask` and `ECSService` --- site/plugins.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/site/plugins.md b/site/plugins.md index 5967b1c7d..61958ea03 100644 --- a/site/plugins.md +++ b/site/plugins.md @@ -421,6 +421,8 @@ These are the available topologies: - `ReplicaSet` nodes represent all Kubernetes ReplicaSets running on hosts running probes. - `ContainerImage` nodes represent all Docker container images on hosts running probes. - `Host` nodes are physical hosts that run probes. +- `ECSTask` nodes represent [AWS ECS](https://aws.amazon.com/ecs/) [tasks](http://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definitions.html). +- `ECSService` nodes represent [AWS ECS services](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html). - `Overlay` nodes are active peers in any software-defined network that's overlaid on the infrastructure. The topology structure consists of the following attributes: From 3fb13e885b8e6ba7be22e129a1e1dd3b5aa2be4f Mon Sep 17 00:00:00 2001 From: Michael Schubert Date: Tue, 18 Apr 2017 09:33:09 +0200 Subject: [PATCH 08/12] site/plugins.md: fix identifier typo --- site/plugins.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/site/plugins.md b/site/plugins.md index 61958ea03..cfe73719b 100644 --- a/site/plugins.md +++ b/site/plugins.md @@ -532,18 +532,18 @@ This description is used to extract metadata from a node and display it on Scope ```json "metadata_templates": { - "metadata-template-id": { - "id": "metadata-template-id", + "traffic-control-pktloss": { + "id": "traffic-control-pktloss", "label": "Human-readable description", "dataType": "number", "priority": 13.5, "from": "latest" }, - "another-metadata-template-id": {...} + "another-plugins-id": {...} } ``` -- `metadata-template-identifier` and `id` identify a particular metadata template. +- `id` is a string identifying the particular metadata template (here `traffic-control-pktloss`) and is also used as a key to the template value. - `label` contains the label that will be used by Scope UI. - `dataType` specifies the type of data, this will determine how the value is displayed. Possible values for this attribute are: "number", "ip", "datetime" and "" for strings. - `priority` is a floating point value used to decide the display ordering (lower values are displayed before higher ones). If omitted, the UI will display it last. From d9b6e81ef6547b868455172ab7742f98f66a213a Mon Sep 17 00:00:00 2001 From: Michael Schubert Date: Tue, 18 Apr 2017 09:52:14 +0200 Subject: [PATCH 09/12] site/plugins.md: clarify that node ids are different --- site/plugins.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/site/plugins.md b/site/plugins.md index cfe73719b..c6822fb39 100644 --- a/site/plugins.md +++ b/site/plugins.md @@ -459,8 +459,7 @@ Nodes are represented as follows: ``` Nodes are stored in a dictionary. -The ID of nodes representing hosts or containers has the format `ID;`, with type equal to the literal string `host` or `container` respectively. -`ID` is the alphanumeric identifier of the nodes. +The ID of nodes is different depending on the node topology. For instance, nodes representing hosts or containers have the format `ID;`, where `ID` is the alphanumeric identifier of the nodes and type is the literal string `host` or `container` respectively. A node representing an endpoint could look like `pc-4026531969;127.0.0.1;36238`. A node contains all the information about the represented object (e.g. host, container, pod, etc.). In particular, a node may contain: From beefec17f23c068af50e49a3544504da86946c28 Mon Sep 17 00:00:00 2001 From: Michael Schubert Date: Tue, 18 Apr 2017 10:16:37 +0200 Subject: [PATCH 10/12] site/plugins.md: document `sets` and `counters` --- site/plugins.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/site/plugins.md b/site/plugins.md index c6822fb39..a8b6b685f 100644 --- a/site/plugins.md +++ b/site/plugins.md @@ -467,6 +467,8 @@ In particular, a node may contain: - `latest` - an id-value map containing the latest values. Each id has only one value. - `latestControls` - the latest available controls. - `metrics` - the collection of metrics to display in the UI. Each metric has multiple timestamped values. +- `sets` - a string->set-of-strings map, for example a list of local networks. +- `counters` - a string->int map. ### Controls Controls describe interfaces that expose actions that the user can perform on different objects (e.g. host, container, etc.). From 1bc0bca2ada138620dd0a66a8ae6ccc05392d2f7 Mon Sep 17 00:00:00 2001 From: Alban Crequy Date: Wed, 26 Apr 2017 14:25:32 +0200 Subject: [PATCH 11/12] site/plugins.md: English wording From Anita Buehrle's review --- site/plugins.md | 38 ++++++++++++++++++-------------------- 1 file changed, 18 insertions(+), 20 deletions(-) diff --git a/site/plugins.md b/site/plugins.md index a8b6b685f..daa868da0 100644 --- a/site/plugins.md +++ b/site/plugins.md @@ -19,7 +19,7 @@ The following topics are discussed: * [Defining the Reporter Interface](#defining-reporter-interface) * [Report Data Structures](#report-data-structures) -Any kind of metrics can be generated and inserted into Scope using custom plugins. Metrics generated through your plugin are displayed in the user interface alongside the standard set of metrics that are found in Weave Scope. +Any kind of metric can be generated and inserted in Scope with custom plugins. Metrics generated through a plugin are displayed in the user interface alongside the standard metrics found in Weave Scope. ![Custom Metrics With Plugins](images/plugin-features.png) @@ -408,8 +408,7 @@ An example of a report containing a few topologies is the following: ``` ### Topologies -Topologies can be of various types. -Each topology consists of some general information and a list of nodes. +A topology consists of a list of nodes along with controls and templates described below. These are the available topologies: - `Endpoint` nodes are `(address, port)` tuples on each host. @@ -505,8 +504,8 @@ In the report, the attribute `latest_controls` contains all the controls exposed ### Metadata All metadata entries are placed within nodes in the section named `latest`. This section contains the latest values to display and consists of `timestamp` and `value`. -Both should be written as json strings (with double quotes). -Scope uses `metadata_templates` to know how to display such data. +Both should be written as JSON strings (with double quotes). +Scope uses `metadata_templates` to display this data. To pair metadata with its template, it is necessary to use the `metadata-template-id` as a key to identify that particular piece of data. Example: ```json @@ -528,8 +527,8 @@ To pair metadata with its template, it is necessary to use the `metadata-templat ``` ### Metadata Templates -Metadata Templates describe a particular metadata item. -This description is used to extract metadata from a node and display it on Scope UI. +A metadata template describes a kind of metadata that is present in zero, one or several nodes and it specifies how to display such metadata in Scope. +Metadata templates are not placed within nodes but in the `metadata_templates` section so that the same information do not need to be repeated in all nodes that use it. ```json "metadata_templates": { @@ -545,16 +544,16 @@ This description is used to extract metadata from a node and display it on Scope ``` - `id` is a string identifying the particular metadata template (here `traffic-control-pktloss`) and is also used as a key to the template value. -- `label` contains the label that will be used by Scope UI. -- `dataType` specifies the type of data, this will determine how the value is displayed. Possible values for this attribute are: "number", "ip", "datetime" and "" for strings. -- `priority` is a floating point value used to decide the display ordering (lower values are displayed before higher ones). If omitted, the UI will display it last. +- `label` contains the label used by the Scope UI. +- `dataType` specifies the type of data, and determines how the value is displayed. Possible values for this attribute are: "number", "ip", "datetime" and "" for strings. +- `priority` is a floating point value used to decide the display ordering (lower values are displayed before higher ones). If omitted, the UI displays it last. - `from` indicates where to look for the metadata. The possible values are: - `latest` - `sets` - `counters` ### Table Templates -Table Templates describe a table and how to identify which metadata templates belong to the table. +Table Templates describe a table and also how to identify the metadata templates that belong to the table. ```json "table_templates": { @@ -568,15 +567,15 @@ Table Templates describe a table and how to identify which metadata templates be ``` - `table-template-identifier` and `id` identify a particular table template. -- `label` contains the label that will be used by Scope UI. +- `label` contains the label used by the Scope UI. - `prefix` is used to identify which metadata templates belong to the table. -If you want to display data in a table, you have to define a table template and prepend the table prefix to all the metadata templates that identify the data you want to put in such table. +To display data in a table, define a table template and prepend the table prefix to all of the metadata templates that identify the data you want to put into the table. ### Metrics Metrics are a particular kind of data that can be plotted on the UI as a graph. -Scope uses the `metric_templates` to know how to display such data. To pair a metric with its template, it is necessary to use the `metric-template-id` as key to identify that particular metric. -Metrics are suitable for information such as CPU and memory usage, HTTP requests rate, I/O operations, etc. +Scope uses `metric_templates` to display graph data in Scope. To pair a metric with its template, use the `metric-template-id` as the key for identifying a particular metric. +Metrics can be used to display CPU and memory usage, HTTP requests rate, I/O operations, etc. The following is an example of a report with a metric preceded by its metric template: ```json @@ -627,17 +626,16 @@ The following is an example of metric template: } ``` -- `metric-template-id` and `id` identify a particular metric template. -- `label` contains the label that will be used by Scope UI. -- `format` describes how the metrics is formatted. +- `metric-template-id` and `id` identify a specific metric template. +- `label` contains the label used by Scope UI. +- `format` describes how the metrics are formatted and can be: - `percent` the metric value is a percentage. - `filesize` the metric value is a file size (e.g. memory usage), it is displayed with the suffix KB, MB, GB, etc. - `integer` the metric value is an integer. - `priority` is a floating point value used to decide the display ordering (lower values are displayed before higher ones). ### Time Window -A report may have an attribute called `Window`. -This is the time window, expressed as duration, within which the data contained in the report are considered valid. +The `Window` attribute is a time window, expressed as a duration and it defines the period from which data in the report is considered valid. The default window is 15 seconds. You may change the window value using the option `-app.window ` when launching scope. However, using values smaller than 15 seconds increases the chance of information not being correctly displayed. From a3d262f79fbcd013629731c0a7fb5efb16dc60ad Mon Sep 17 00:00:00 2001 From: Alban Crequy Date: Wed, 26 Apr 2017 16:56:28 +0200 Subject: [PATCH 12/12] site/plugins.md: Metadata Templates: change English phrasing --- site/plugins.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/site/plugins.md b/site/plugins.md index daa868da0..56c2ef8b5 100644 --- a/site/plugins.md +++ b/site/plugins.md @@ -527,8 +527,8 @@ To pair metadata with its template, it is necessary to use the `metadata-templat ``` ### Metadata Templates -A metadata template describes a kind of metadata that is present in zero, one or several nodes and it specifies how to display such metadata in Scope. -Metadata templates are not placed within nodes but in the `metadata_templates` section so that the same information do not need to be repeated in all nodes that use it. +A metadata template describes a kind of metadata that is present in zero, one or several nodes of the topology and specifies how to display such metadata in Scope. +Metadata templates are not placed within nodes but in the `metadata_templates` section of the JSON file. ```json "metadata_templates": {