48 Commits

Author SHA1 Message Date
Bryan Boreham
4f75600931 Pin SHOUT library at working version
The version we did have has a bug of not importing 'time'.

Remove other vendor'd copy of SHOUT library which may or may not be used.
2020-12-30 18:30:34 +00:00
Bryan Boreham
b438707a92 Clean up lint errors in shell scripts 2020-12-30 18:30:34 +00:00
Ben Sooraj
23270fecc3 k8s example: updated elasticsearch's image tag from the default 'latest' to '7.4.0'
To fix the ImagePullBackOff error

Fixes #3401
2019-10-10 21:53:37 +05:30
Bryan Boreham
d375f3c07d chore(example): update example Kubernetes app to apps/v1 2019-09-21 16:00:58 +00:00
Bryan Boreham
b6d5594f9f perf(probe): publish delta reports to reduce data size
Similar to video compression which uses key-frames and differences
between them: every N publishes we send a full report, but inbetween
we only send what has changed.

Fairly simple approach in the probe - hold on to the last full report,
and for the deltas remove anything that would be merged in from the
full report.

On the receiving side in the app it already merges a set of reports
together to produce the final output for rendering, so provided N is
smaller than that set we don't need to do anything different.

Deltas don't need to represent nodes that have disappeared - an
earlier full node will have that node so it would be merged into the
final output anyway.
2019-09-18 08:00:28 +00:00
Bryan Boreham
711aa66bd5 Add OpenTracing span for report.ReadBinary()
So we can see the timing and size in Jaeger.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2019-05-10 15:34:53 +00:00
Bryan Boreham
dad34df575 Merge pull request #3267 from weaveworks/pool-gzwriters
Re-use gzip writers in a pool
2018-07-13 16:09:24 +01:00
Bryan Boreham
dac008b9c4 Early return from LatestMap merge loop
If we run out of things to look at in the other map, return quickly.
Also move the equal-key case above the less-than case, since maps with
equal keys are the common case when merging.
2018-07-12 19:54:01 +00:00
Bryan Boreham
c91fd6c67b Optimise merge of LatestMaps with same keys, different timestamps
Swap the two maps so we are merging older into later.
2018-07-12 19:54:01 +00:00
Bryan Boreham
43a0a7e5d3 Don't copy LatestMap if merging a subset 2018-07-12 19:50:02 +00:00
Bryan Boreham
c96611b13d Less verbose string dump of LatestMap
The default time.Time.String() includes nanoseconds, wall-clock adjustment, etc.
Also don't include the key twice when stringizing the map itself.
2018-07-08 16:32:51 +00:00
Bryan Boreham
70cd79307e Re-use gzip writers in a pool
Since they allocate sizeable chunks of memory, this can save on garbage-collection.
2018-07-08 16:30:07 +00:00
Bryan Boreham
06c895267c Reports: streamline report serialization
Move the creation of the buffer and the choice of compression level
(which never changes) into WriteBinary(), to simplify the code.
2018-06-19 22:59:01 +00:00
Bryan Boreham
96f51c47af probe: Eliminate Publisher interface from app_client
Simplification: everything now implements Publish(Report), and we do
away with writing/reading/writing in the MultiAppClient.
2018-06-19 22:51:25 +00:00
Marc Carré
4e58b83024 Add org.opencontainers.image.* labels to Dockerfiles
- This should ultimately help for image-to-code back references.
- `org.label-schema.*` labels are now deprecated, in favour of `org.opencontainers.image.*` labels.
  See also: https://github.com/opencontainers/image-spec/blob/master/annotations.md#back-compatibility-with-label-schema
- Git revision (`git rev-parse HEAD`) is now injected at `docker build` time.
2018-05-14 11:54:33 +01:00
Bryan Boreham
27cf8f3b06 Fix format string to work for all types 2018-04-05 08:18:52 +00:00
Bryan Boreham
f17640646e Blank out value on LatestMap decode insert
Fixing a rare case that came up in a test. In order for this to cause
a problem, the data being decoded has to have entries out of order,
and have a value that is nil or omitted.
2018-02-26 10:34:53 +00:00
Matthias Radestock
e24d3e93fe Add static map of common keys to reduce allocations 2017-12-11 20:26:47 +00:00
Bryan Boreham
d1bf98dfde Remove struct wrapping LatestMap
It isn't necessary, and it prevents the codec from seeing maps as empty
2017-12-06 15:46:31 +00:00
Matthias Radestock
4162b5d734 allocate less memory in LatestMap merging
Most maps we merge have the same keys, or at least one set of keys is
the subset of the other. Therefore, allocate a result slice capable of
holding only the max of number keys, rather than the sum.
2017-12-02 13:13:08 +00:00
cglewis
bbde07d42f MAINTAINER is deprecated, now using LABEL 2017-10-31 10:32:09 -07:00
Bryan Boreham
5acece6e58 Refactor: loop replaced with append() 2017-10-11 09:50:06 +00:00
Bryan Boreham
29f139d424 Re-implement LatestMap as a sorted slice for better performance
both the Python generator and the Go generated code are checked in
2017-10-01 16:18:59 +00:00
Bryan Boreham
ae9d708dbc Reformat Python examples to placate linter 2017-07-13 16:11:59 +00:00
Matthias Radestock
4dbb913ca2 refactor: remove dead Copy() code 2017-07-04 07:35:30 +01:00
Matthias Radestock
430e74a80a refactor: remove report latest map Delete()
It wasn't used, and is problematic in any case since it introduces
non-monotonicity.
2017-07-03 02:06:21 +01:00
Matthias Radestock
9dc50b5202 refactor: hide "empty set" constants
They are an implementation detail.
2017-07-03 01:26:22 +01:00
Bryan Boreham
1898b67e1f Use 127.0.0.1 instead of localhost in case that name resolves to something else 2017-06-05 10:31:27 +00:00
Matthias Radestock
fa0f4a4d59 add copyreport utility
useful for codec and report structure experiments
2017-05-26 15:33:15 +01:00
Alban Crequy
0096688415 shfmt: fix shell formatting
Symptoms:
```
extras/dialer/time-scope-probe-cgroup:31:18: arithmetic expressions must consist of names and numbers
extras/dialer/time-scope-probe-cgroup: run shfmt -i 4 -w extras/dialer/time-scope-probe-cgroup
extras/dialer/time-scope-probe-proc:28:18: arithmetic expressions must consist of names and numbers
extras/dialer/time-scope-probe-proc: run shfmt -i 4 -w extras/dialer/time-scope-probe-proc
extras/in_parallel.sh:29:16: arithmetic expressions must consist of names and numbers
extras/in_parallel.sh: run shfmt -i 4 -w extras/in_parallel.sh
```

Visible on https://circleci.com/gh/kinvolk/scope/980
2017-05-22 17:27:35 +02:00
Bryan Boreham
36ff5ab71a Write psMap out directly
rather than going via intermediate data structure
2017-04-25 10:38:56 +00:00
Alban Crequy
ddf654fdf2 extras/dialer: updates and fixes
* remove randomness as far as possible
* make listener to close connections
* report time in a csv-friendly format
* remove tools from README
* rename time-scope-probe -> time-scope-probe-proc
* add time-scope-probe-cgroup
2017-03-30 07:30:28 +02:00
Bryan Boreham
b3f53a7a81 Call CodecDecodeSelf() instead of Decode()
This avoids a runtime type lookup, so goes a little faster.
Also having less recursion makes it easier to interpret profiles.
2017-03-23 12:25:12 +00:00
Bryan Boreham
de97a72661 Remove indirection via LatestMap
Structs like StringLatestMap now use ps.Map directly, which saves
a memory allocation for LatestEntry.Value to point to.
The values in the ps.Map are now pointers, which saves a memory
allocation indirecting a value type to an interface{}
2017-03-20 11:51:04 +00:00
Alessandro Puccetti
6b2b483cbc extras/in_parallel.sh: remove erroneous quotes
$COMMAND can contain "make RM="

This patch fixes a regression caused by
https://github.com/weaveworks/scope/pull/2068/files#diff-42138a4802cce915f347919ccd707876R28
2017-02-22 11:16:51 +01:00
Alessandro Puccetti
e81f674a02 extra/in_parallel.sh: call the right python version
We want to use the python installed as dependency to use an updated
version of the requests module (>=v2.4.2).

This regression is caused by
https://github.com/weaveworks/build-tools/pull/73
2017-02-21 17:25:39 +01:00
Alessandro Puccetti
340438b3c3 extras/build_on_circle.sh: fix find error
The find command fails on circleci running ubuntu 14.04 with the
following message:

Unix filenames usually don't contain slashes (though pathnames do).
That means that '-name `./*.go'' will probably evaluate to false all the
time on this system.  You might find the '-wholename' test more useful,
or perhaps '-samefile'.  Alternatively, if you are using GNU grep, you
could use 'find ... -print0 | grep -FzZ `./*.go''.'

This fix is compatible with ubuntu 12.04

This regression is caused by
https://github.com/weaveworks/scope/pull/2068/files#diff-3072c24bd91fea6eb71aecc590951d7e
2017-02-21 17:19:19 +01:00
Michael Schubert
c349c165e5 extras/dialer: allow to seed random
Helpful if you want to get the same random numbers in subsequent runs
(e.g. to compare results between branches).
2017-02-08 12:42:32 +01:00
Iago López Galeiras
5cb5c7d42e Fix shfmt issues 2017-01-18 11:51:41 +01:00
Iago López Galeiras
43d9f38c5e Fix various linter issues
Found via shellcheck.
2017-01-18 11:51:41 +01:00
Iago López Galeiras
643827d9a7 Re-add generate_latest_map script
It was removed from the build-tools repository. Add it to extra/ and fix
a couple of linter errors.
2017-01-09 16:40:16 +01:00
Michael Schubert
e8f9bc11e7 extras/dialer: move dialer.go to sub directory
`extras/dialer` contains both a dialer script and dialer.go, the go
sources for the dialer server and client, to allow users to build the
dialer container themselves. Move dialer.go to a sub directory `src`
since `make clean` (running `go clean ./...`) deletes the dialer script
otherwise.

Fixup for 4d20a91a97

Signed-off-by: Michael Schubert <michael@kinvolk.io>
2016-12-22 10:32:07 +01:00
Michael Schubert
4d20a91a97 Add extras/dialer
Based on the dialer.sh script by @alban:
https://gist.github.com/alban/7abe7f6307184c58be7429e63a2d9053

Signed-off-by: Michael Schubert <michael@kinvolk.io>
2016-12-13 09:31:57 +01:00
Tom Wilkie
2a00fd2d78 Allow user to specify URLs on the command line, and use that to allow per-target tokens. (#1901)
Also:
- Parse targets on startup and catch badly formed ones before Scope can start.
- If no port is specified, use default port for scheme; if no scheme is specificed, use 4040.
- Use username as probe token
2016-10-05 10:59:56 -07:00
Alban Crequy
652913084a circle.yml: deploy master with non-upstream hub accounts (#1655)
* circle.yml: deploy master with non-upstream hub accounts

Since b4abe582f4, images are tagged on
Docker Hub. But this was not working for non-upstream Docker Hub
accounts. This patch should fix that.

Symptoms: https://circleci.com/gh/kinvolk/scope/23

> The push refers to a repository [docker.io/kinvolk/scope] (len: 1)
> error getting tags for kinvolk/scope: Tag does not exist for
> master-b89e9ec

Also, ensure that the Docker tag name does not include any '/'.

Symptoms: https://circleci.com/gh/kinvolk/scope/24

> docker tag weaveworks/scope weaveworks/scope:alban/hub1-ef739d9
> repository name component must match "[a-z0-9]+(?:[._-][a-z0-9]+)*"

* extras/in_parallel.sh: ensure unique schedule name

Use CircleCI environment variables as documented on:
https://circleci.com/docs/environment-variables/

Symptoms:
> $ cd $SRCDIR/extras; ./build_on_circle.sh
> Doing graphviz/graphviz
> make: *** No rule to make target `graphviz/graphviz'.  Stop.

See: https://github.com/weaveworks/scope/pull/1655#issuecomment-232696357

* circle.yml: separate wcloud deployment

Do not run wcloud for non-weaveworks builds, as suggested by
https://github.com/weaveworks/scope/pull/1655#discussion_r71179373
2016-07-21 10:26:23 +01:00
Paul Bellamy
985912f964 Replace wget in instructions with curl, as it's more widely avail. on macs 2016-07-13 14:25:13 +01:00
Jonathan Lange
7cc829e463 What's tracer? 2016-07-07 15:02:26 +01:00
Jonathan Lange
1c440486f0 Delete experimental, replace with 'extras'
extras contains fixprobe and example, and ought to build properly
2016-07-07 14:30:08 +01:00