Add github.com/prometheus/procfs library

This commit is contained in:
Xuewei Zhang
2020-01-31 16:01:33 -08:00
parent 7f9437cba0
commit 9ade82734d
30 changed files with 2434 additions and 300 deletions

View File

@@ -1,6 +1,4 @@
# Run only staticcheck for now. Additional linters will be enabled one-by-one.
linters:
enable:
- staticcheck
- govet
disable-all: true

View File

@@ -2,17 +2,120 @@
Prometheus uses GitHub to manage reviews of pull requests.
* If you are a new contributor see: [Steps to Contribute](#steps-to-contribute)
* If you have a trivial fix or improvement, go ahead and create a pull request,
addressing (with `@...`) the maintainer of this repository (see
addressing (with `@...`) a suitable maintainer of this repository (see
[MAINTAINERS.md](MAINTAINERS.md)) in the description of the pull request.
* If you plan to do something more involved, first discuss your ideas
on our [mailing list](https://groups.google.com/forum/?fromgroups#!forum/prometheus-developers).
This will avoid unnecessary work and surely give you and us a good deal
of inspiration.
of inspiration. Also please see our [non-goals issue](https://github.com/prometheus/docs/issues/149) on areas that the Prometheus community doesn't plan to work on.
* Relevant coding style guidelines are the [Go Code Review
Comments](https://code.google.com/p/go-wiki/wiki/CodeReviewComments)
and the _Formatting and style_ section of Peter Bourgon's [Go: Best
Practices for Production
Environments](http://peter.bourgon.org/go-in-production/#formatting-and-style).
Environments](https://peter.bourgon.org/go-in-production/#formatting-and-style).
* Be sure to sign off on the [DCO](https://github.com/probot/dco#how-it-works)
## Steps to Contribute
Should you wish to work on an issue, please claim it first by commenting on the GitHub issue that you want to work on it. This is to prevent duplicated efforts from contributors on the same issue.
Please check the [`help-wanted`](https://github.com/prometheus/procfs/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22) label to find issues that are good for getting started. If you have questions about one of the issues, with or without the tag, please comment on them and one of the maintainers will clarify it. For a quicker response, contact us over [IRC](https://prometheus.io/community).
For quickly compiling and testing your changes do:
```
make test # Make sure all the tests pass before you commit and push :)
```
We use [`golangci-lint`](https://github.com/golangci/golangci-lint) for linting the code. If it reports an issue and you think that the warning needs to be disregarded or is a false-positive, you can add a special comment `//nolint:linter1[,linter2,...]` before the offending line. Use this sparingly though, fixing the code to comply with the linter's recommendation is in general the preferred course of action.
## Pull Request Checklist
* Branch from the master branch and, if needed, rebase to the current master branch before submitting your pull request. If it doesn't merge cleanly with master you may be asked to rebase your changes.
* Commits should be as small as possible, while ensuring that each commit is correct independently (i.e., each commit should compile and pass tests).
* If your patch is not getting reviewed or you need a specific person to review it, you can @-reply a reviewer asking for a review in the pull request or a comment, or you can ask for a review on IRC channel [#prometheus](https://webchat.freenode.net/?channels=#prometheus) on irc.freenode.net (for the easiest start, [join via Riot](https://riot.im/app/#/room/#prometheus:matrix.org)).
* Add tests relevant to the fixed bug or new feature.
## Dependency management
The Prometheus project uses [Go modules](https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more) to manage dependencies on external packages. This requires a working Go environment with version 1.12 or greater installed.
All dependencies are vendored in the `vendor/` directory.
To add or update a new dependency, use the `go get` command:
```bash
# Pick the latest tagged release.
go get example.com/some/module/pkg
# Pick a specific version.
go get example.com/some/module/pkg@vX.Y.Z
```
Tidy up the `go.mod` and `go.sum` files and copy the new/updated dependency to the `vendor/` directory:
```bash
# The GO111MODULE variable can be omitted when the code isn't located in GOPATH.
GO111MODULE=on go mod tidy
GO111MODULE=on go mod vendor
```
You have to commit the changes to `go.mod`, `go.sum` and the `vendor/` directory before submitting the pull request.
## API Implementation Guidelines
### Naming and Documentation
Public functions and structs should normally be named according to the file(s) being read and parsed. For example,
the `fs.BuddyInfo()` function reads the file `/proc/buddyinfo`. In addition, the godoc for each public function
should contain the path to the file(s) being read and a URL of the linux kernel documentation describing the file(s).
### Reading vs. Parsing
Most functionality in this library consists of reading files and then parsing the text into structured data. In most
cases reading and parsing should be separated into different functions/methods with a public `fs.Thing()` method and
a private `parseThing(r Reader)` function. This provides a logical separation and allows parsing to be tested
directly without the need to read from the filesystem. Using a `Reader` argument is preferred over other data types
such as `string` or `*File` because it provides the most flexibility regarding the data source. When a set of files
in a directory needs to be parsed, then a `path` string parameter to the parse function can be used instead.
### /proc and /sys filesystem I/O
The `proc` and `sys` filesystems are pseudo file systems and work a bit differently from standard disk I/O.
Many of the files are changing continuously and the data being read can in some cases change between subsequent
reads in the same file. Also, most of the files are relatively small (less than a few KBs), and system calls
to the `stat` function will often return the wrong size. Therefore, for most files it's recommended to read the
full file in a single operation using an internal utility function called `util.ReadFileNoStat`.
This function is similar to `ioutil.ReadFile`, but it avoids the system call to `stat` to get the current size of
the file.
Note that parsing the file's contents can still be performed one line at a time. This is done by first reading
the full file, and then using a scanner on the `[]byte` or `string` containing the data.
```
data, err := util.ReadFileNoStat("/proc/cpuinfo")
if err != nil {
return err
}
reader := bytes.NewReader(data)
scanner := bufio.NewScanner(reader)
```
The `/sys` filesystem contains many very small files which contain only a single numeric or text value. These files
can be read using an internal function called `util.SysReadFile` which is similar to `ioutil.ReadFile` but does
not bother to check the size of the file before reading.
```
data, err := util.SysReadFile("/sys/class/power_supply/BAT0/capacity")
```

View File

@@ -1,6 +1,6 @@
# procfs
This procfs package provides functions to retrieve system, kernel and process
This package provides functions to retrieve system, kernel, and process
metrics from the pseudo-filesystems /proc and /sys.
*WARNING*: This package is a work in progress. Its API may still break in
@@ -13,7 +13,8 @@ backwards-incompatible ways without warnings. Use it at your own risk.
## Usage
The procfs library is organized by packages based on whether the gathered data is coming from
/proc, /sys, or both. Each package contains an `FS` type which represents the path to either /proc, /sys, or both. For example, current cpu statistics are gathered from
/proc, /sys, or both. Each package contains an `FS` type which represents the path to either /proc,
/sys, or both. For example, cpu statistics are gathered from
`/proc/stat` and are available via the root procfs package. First, the proc filesystem mount
point is initialized, and then the stat information is read.
@@ -29,10 +30,17 @@ Some sub-packages such as `blockdevice`, require access to both the proc and sys
stats, err := fs.ProcDiskstats()
```
## Package Organization
The packages in this project are organized according to (1) whether the data comes from the `/proc` or
`/sys` filesystem and (2) the type of information being retrieved. For example, most process information
can be gathered from the functions in the root `procfs` package. Information about block devices such as disk drives
is available in the `blockdevices` sub-package.
## Building and Testing
The procfs library is normally built as part of another application. However, when making
changes to the library, the `make test` command can be used to run the API test suite.
The procfs library is intended to be built as part of another application, so there are no distributable binaries.
However, most of the API includes unit tests which can be run with `make test`.
### Updating Test Fixtures

View File

@@ -31,7 +31,7 @@ type BuddyInfo struct {
Sizes []float64
}
// NewBuddyInfo reads the buddyinfo statistics from the specified `proc` filesystem.
// BuddyInfo reads the buddyinfo statistics from the specified `proc` filesystem.
func (fs FS) BuddyInfo() ([]BuddyInfo, error) {
file, err := os.Open(fs.proc.Path("buddyinfo"))
if err != nil {

167
vendor/github.com/prometheus/procfs/cpuinfo.go generated vendored Normal file
View File

@@ -0,0 +1,167 @@
// Copyright 2019 The Prometheus Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package procfs
import (
"bufio"
"bytes"
"strconv"
"strings"
"github.com/prometheus/procfs/internal/util"
)
// CPUInfo contains general information about a system CPU found in /proc/cpuinfo
type CPUInfo struct {
Processor uint
VendorID string
CPUFamily string
Model string
ModelName string
Stepping string
Microcode string
CPUMHz float64
CacheSize string
PhysicalID string
Siblings uint
CoreID string
CPUCores uint
APICID string
InitialAPICID string
FPU string
FPUException string
CPUIDLevel uint
WP string
Flags []string
Bugs []string
BogoMips float64
CLFlushSize uint
CacheAlignment uint
AddressSizes string
PowerManagement string
}
// CPUInfo returns information about current system CPUs.
// See https://www.kernel.org/doc/Documentation/filesystems/proc.txt
func (fs FS) CPUInfo() ([]CPUInfo, error) {
data, err := util.ReadFileNoStat(fs.proc.Path("cpuinfo"))
if err != nil {
return nil, err
}
return parseCPUInfo(data)
}
// parseCPUInfo parses data from /proc/cpuinfo
func parseCPUInfo(info []byte) ([]CPUInfo, error) {
cpuinfo := []CPUInfo{}
i := -1
scanner := bufio.NewScanner(bytes.NewReader(info))
for scanner.Scan() {
line := scanner.Text()
if strings.TrimSpace(line) == "" {
continue
}
field := strings.SplitN(line, ": ", 2)
switch strings.TrimSpace(field[0]) {
case "processor":
cpuinfo = append(cpuinfo, CPUInfo{}) // start of the next processor
i++
v, err := strconv.ParseUint(field[1], 0, 32)
if err != nil {
return nil, err
}
cpuinfo[i].Processor = uint(v)
case "vendor_id":
cpuinfo[i].VendorID = field[1]
case "cpu family":
cpuinfo[i].CPUFamily = field[1]
case "model":
cpuinfo[i].Model = field[1]
case "model name":
cpuinfo[i].ModelName = field[1]
case "stepping":
cpuinfo[i].Stepping = field[1]
case "microcode":
cpuinfo[i].Microcode = field[1]
case "cpu MHz":
v, err := strconv.ParseFloat(field[1], 64)
if err != nil {
return nil, err
}
cpuinfo[i].CPUMHz = v
case "cache size":
cpuinfo[i].CacheSize = field[1]
case "physical id":
cpuinfo[i].PhysicalID = field[1]
case "siblings":
v, err := strconv.ParseUint(field[1], 0, 32)
if err != nil {
return nil, err
}
cpuinfo[i].Siblings = uint(v)
case "core id":
cpuinfo[i].CoreID = field[1]
case "cpu cores":
v, err := strconv.ParseUint(field[1], 0, 32)
if err != nil {
return nil, err
}
cpuinfo[i].CPUCores = uint(v)
case "apicid":
cpuinfo[i].APICID = field[1]
case "initial apicid":
cpuinfo[i].InitialAPICID = field[1]
case "fpu":
cpuinfo[i].FPU = field[1]
case "fpu_exception":
cpuinfo[i].FPUException = field[1]
case "cpuid level":
v, err := strconv.ParseUint(field[1], 0, 32)
if err != nil {
return nil, err
}
cpuinfo[i].CPUIDLevel = uint(v)
case "wp":
cpuinfo[i].WP = field[1]
case "flags":
cpuinfo[i].Flags = strings.Fields(field[1])
case "bugs":
cpuinfo[i].Bugs = strings.Fields(field[1])
case "bogomips":
v, err := strconv.ParseFloat(field[1], 64)
if err != nil {
return nil, err
}
cpuinfo[i].BogoMips = v
case "clflush size":
v, err := strconv.ParseUint(field[1], 0, 32)
if err != nil {
return nil, err
}
cpuinfo[i].CLFlushSize = uint(v)
case "cache_alignment":
v, err := strconv.ParseUint(field[1], 0, 32)
if err != nil {
return nil, err
}
cpuinfo[i].CacheAlignment = uint(v)
case "address sizes":
cpuinfo[i].AddressSizes = field[1]
case "power management":
cpuinfo[i].PowerManagement = field[1]
}
}
return cpuinfo, nil
}

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,8 @@
module github.com/prometheus/procfs
go 1.12
require (
github.com/google/go-cmp v0.3.0
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4
github.com/google/go-cmp v0.3.1
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e
)

View File

@@ -1,4 +1,4 @@
github.com/google/go-cmp v0.3.0 h1:crn/baboCvb5fXaQ0IJ1SGTsTVrWpDsCWC8EGETZijY=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4 h1:YUO/7uOKsKeq9UokNS62b8FYywz3ker1l1vDZRCRefw=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
github.com/google/go-cmp v0.3.1 h1:Xye71clBPdm5HgqGwUkwhbynsUJZhDbS20FvLhQ2izg=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e h1:vcxGaoTs7kV8m5Np9uUNQin4BrLOthgV7252N8V+FwY=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=

View File

@@ -26,7 +26,7 @@ const (
// DefaultSysMountPoint is the common mount point of the sys filesystem.
DefaultSysMountPoint = "/sys"
// DefaultConfigfsMountPoint is the commont mount point of the configfs
// DefaultConfigfsMountPoint is the common mount point of the configfs
DefaultConfigfsMountPoint = "/sys/kernel/config"
)

View File

@@ -0,0 +1,38 @@
// Copyright 2019 The Prometheus Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package util
import (
"io"
"io/ioutil"
"os"
)
// ReadFileNoStat uses ioutil.ReadAll to read contents of entire file.
// This is similar to ioutil.ReadFile but without the call to os.Stat, because
// many files in /proc and /sys report incorrect file sizes (either 0 or 4096).
// Reads a max file size of 512kB. For files larger than this, a scanner
// should be used.
func ReadFileNoStat(filename string) ([]byte, error) {
const maxBufferSize = 1024 * 512
f, err := os.Open(filename)
if err != nil {
return nil, err
}
defer f.Close()
reader := io.LimitReader(f, maxBufferSize)
return ioutil.ReadAll(reader)
}

View File

@@ -23,6 +23,8 @@ import (
// SysReadFile is a simplified ioutil.ReadFile that invokes syscall.Read directly.
// https://github.com/prometheus/node_exporter/pull/728/files
//
// Note that this function will not read files larger than 128 bytes.
func SysReadFile(file string) (string, error) {
f, err := os.Open(file)
if err != nil {
@@ -35,7 +37,8 @@ func SysReadFile(file string) (string, error) {
//
// Since we either want to read data or bail immediately, do the simplest
// possible read using syscall directly.
b := make([]byte, 128)
const sysFileBufferSize = 128
b := make([]byte, sysFileBufferSize)
n, err := syscall.Read(int(f.Fd()), b)
if err != nil {
return "", err

View File

@@ -33,6 +33,9 @@ func NewValueParser(v string) *ValueParser {
return &ValueParser{v: v}
}
// Int interprets the underlying value as an int and returns that value.
func (vp *ValueParser) Int() int { return int(vp.int64()) }
// PInt64 interprets the underlying value as an int64 and returns a pointer to
// that value.
func (vp *ValueParser) PInt64() *int64 {
@@ -40,16 +43,27 @@ func (vp *ValueParser) PInt64() *int64 {
return nil
}
v := vp.int64()
return &v
}
// int64 interprets the underlying value as an int64 and returns that value.
// TODO: export if/when necessary.
func (vp *ValueParser) int64() int64 {
if vp.err != nil {
return 0
}
// A base value of zero makes ParseInt infer the correct base using the
// string's prefix, if any.
const base = 0
v, err := strconv.ParseInt(vp.v, base, 64)
if err != nil {
vp.err = err
return nil
return 0
}
return &v
return v
}
// PUInt64 interprets the underlying value as an uint64 and returns a pointer to

View File

@@ -15,6 +15,7 @@ package procfs
import (
"bufio"
"bytes"
"encoding/hex"
"errors"
"fmt"
@@ -24,6 +25,8 @@ import (
"os"
"strconv"
"strings"
"github.com/prometheus/procfs/internal/util"
)
// IPVSStats holds IPVS statistics, as exposed by the kernel in `/proc/net/ip_vs_stats`.
@@ -64,17 +67,16 @@ type IPVSBackendStatus struct {
// IPVSStats reads the IPVS statistics from the specified `proc` filesystem.
func (fs FS) IPVSStats() (IPVSStats, error) {
file, err := os.Open(fs.proc.Path("net/ip_vs_stats"))
data, err := util.ReadFileNoStat(fs.proc.Path("net/ip_vs_stats"))
if err != nil {
return IPVSStats{}, err
}
defer file.Close()
return parseIPVSStats(file)
return parseIPVSStats(bytes.NewReader(data))
}
// parseIPVSStats performs the actual parsing of `ip_vs_stats`.
func parseIPVSStats(file io.Reader) (IPVSStats, error) {
func parseIPVSStats(r io.Reader) (IPVSStats, error) {
var (
statContent []byte
statLines []string
@@ -82,7 +84,7 @@ func parseIPVSStats(file io.Reader) (IPVSStats, error) {
stats IPVSStats
)
statContent, err := ioutil.ReadAll(file)
statContent, err := ioutil.ReadAll(r)
if err != nil {
return IPVSStats{}, err
}

277
vendor/github.com/prometheus/procfs/meminfo.go generated vendored Normal file
View File

@@ -0,0 +1,277 @@
// Copyright 2019 The Prometheus Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package procfs
import (
"bufio"
"bytes"
"fmt"
"io"
"strconv"
"strings"
"github.com/prometheus/procfs/internal/util"
)
// Meminfo represents memory statistics.
type Meminfo struct {
// Total usable ram (i.e. physical ram minus a few reserved
// bits and the kernel binary code)
MemTotal uint64
// The sum of LowFree+HighFree
MemFree uint64
// An estimate of how much memory is available for starting
// new applications, without swapping. Calculated from
// MemFree, SReclaimable, the size of the file LRU lists, and
// the low watermarks in each zone. The estimate takes into
// account that the system needs some page cache to function
// well, and that not all reclaimable slab will be
// reclaimable, due to items being in use. The impact of those
// factors will vary from system to system.
MemAvailable uint64
// Relatively temporary storage for raw disk blocks shouldn't
// get tremendously large (20MB or so)
Buffers uint64
Cached uint64
// Memory that once was swapped out, is swapped back in but
// still also is in the swapfile (if memory is needed it
// doesn't need to be swapped out AGAIN because it is already
// in the swapfile. This saves I/O)
SwapCached uint64
// Memory that has been used more recently and usually not
// reclaimed unless absolutely necessary.
Active uint64
// Memory which has been less recently used. It is more
// eligible to be reclaimed for other purposes
Inactive uint64
ActiveAnon uint64
InactiveAnon uint64
ActiveFile uint64
InactiveFile uint64
Unevictable uint64
Mlocked uint64
// total amount of swap space available
SwapTotal uint64
// Memory which has been evicted from RAM, and is temporarily
// on the disk
SwapFree uint64
// Memory which is waiting to get written back to the disk
Dirty uint64
// Memory which is actively being written back to the disk
Writeback uint64
// Non-file backed pages mapped into userspace page tables
AnonPages uint64
// files which have been mapped, such as libraries
Mapped uint64
Shmem uint64
// in-kernel data structures cache
Slab uint64
// Part of Slab, that might be reclaimed, such as caches
SReclaimable uint64
// Part of Slab, that cannot be reclaimed on memory pressure
SUnreclaim uint64
KernelStack uint64
// amount of memory dedicated to the lowest level of page
// tables.
PageTables uint64
// NFS pages sent to the server, but not yet committed to
// stable storage
NFSUnstable uint64
// Memory used for block device "bounce buffers"
Bounce uint64
// Memory used by FUSE for temporary writeback buffers
WritebackTmp uint64
// Based on the overcommit ratio ('vm.overcommit_ratio'),
// this is the total amount of memory currently available to
// be allocated on the system. This limit is only adhered to
// if strict overcommit accounting is enabled (mode 2 in
// 'vm.overcommit_memory').
// The CommitLimit is calculated with the following formula:
// CommitLimit = ([total RAM pages] - [total huge TLB pages]) *
// overcommit_ratio / 100 + [total swap pages]
// For example, on a system with 1G of physical RAM and 7G
// of swap with a `vm.overcommit_ratio` of 30 it would
// yield a CommitLimit of 7.3G.
// For more details, see the memory overcommit documentation
// in vm/overcommit-accounting.
CommitLimit uint64
// The amount of memory presently allocated on the system.
// The committed memory is a sum of all of the memory which
// has been allocated by processes, even if it has not been
// "used" by them as of yet. A process which malloc()'s 1G
// of memory, but only touches 300M of it will show up as
// using 1G. This 1G is memory which has been "committed" to
// by the VM and can be used at any time by the allocating
// application. With strict overcommit enabled on the system
// (mode 2 in 'vm.overcommit_memory'),allocations which would
// exceed the CommitLimit (detailed above) will not be permitted.
// This is useful if one needs to guarantee that processes will
// not fail due to lack of memory once that memory has been
// successfully allocated.
CommittedAS uint64
// total size of vmalloc memory area
VmallocTotal uint64
// amount of vmalloc area which is used
VmallocUsed uint64
// largest contiguous block of vmalloc area which is free
VmallocChunk uint64
HardwareCorrupted uint64
AnonHugePages uint64
ShmemHugePages uint64
ShmemPmdMapped uint64
CmaTotal uint64
CmaFree uint64
HugePagesTotal uint64
HugePagesFree uint64
HugePagesRsvd uint64
HugePagesSurp uint64
Hugepagesize uint64
DirectMap4k uint64
DirectMap2M uint64
DirectMap1G uint64
}
// Meminfo returns an information about current kernel/system memory statistics.
// See https://www.kernel.org/doc/Documentation/filesystems/proc.txt
func (fs FS) Meminfo() (Meminfo, error) {
b, err := util.ReadFileNoStat(fs.proc.Path("meminfo"))
if err != nil {
return Meminfo{}, err
}
m, err := parseMemInfo(bytes.NewReader(b))
if err != nil {
return Meminfo{}, fmt.Errorf("failed to parse meminfo: %v", err)
}
return *m, nil
}
func parseMemInfo(r io.Reader) (*Meminfo, error) {
var m Meminfo
s := bufio.NewScanner(r)
for s.Scan() {
// Each line has at least a name and value; we ignore the unit.
fields := strings.Fields(s.Text())
if len(fields) < 2 {
return nil, fmt.Errorf("malformed meminfo line: %q", s.Text())
}
v, err := strconv.ParseUint(fields[1], 0, 64)
if err != nil {
return nil, err
}
switch fields[0] {
case "MemTotal:":
m.MemTotal = v
case "MemFree:":
m.MemFree = v
case "MemAvailable:":
m.MemAvailable = v
case "Buffers:":
m.Buffers = v
case "Cached:":
m.Cached = v
case "SwapCached:":
m.SwapCached = v
case "Active:":
m.Active = v
case "Inactive:":
m.Inactive = v
case "Active(anon):":
m.ActiveAnon = v
case "Inactive(anon):":
m.InactiveAnon = v
case "Active(file):":
m.ActiveFile = v
case "Inactive(file):":
m.InactiveFile = v
case "Unevictable:":
m.Unevictable = v
case "Mlocked:":
m.Mlocked = v
case "SwapTotal:":
m.SwapTotal = v
case "SwapFree:":
m.SwapFree = v
case "Dirty:":
m.Dirty = v
case "Writeback:":
m.Writeback = v
case "AnonPages:":
m.AnonPages = v
case "Mapped:":
m.Mapped = v
case "Shmem:":
m.Shmem = v
case "Slab:":
m.Slab = v
case "SReclaimable:":
m.SReclaimable = v
case "SUnreclaim:":
m.SUnreclaim = v
case "KernelStack:":
m.KernelStack = v
case "PageTables:":
m.PageTables = v
case "NFS_Unstable:":
m.NFSUnstable = v
case "Bounce:":
m.Bounce = v
case "WritebackTmp:":
m.WritebackTmp = v
case "CommitLimit:":
m.CommitLimit = v
case "Committed_AS:":
m.CommittedAS = v
case "VmallocTotal:":
m.VmallocTotal = v
case "VmallocUsed:":
m.VmallocUsed = v
case "VmallocChunk:":
m.VmallocChunk = v
case "HardwareCorrupted:":
m.HardwareCorrupted = v
case "AnonHugePages:":
m.AnonHugePages = v
case "ShmemHugePages:":
m.ShmemHugePages = v
case "ShmemPmdMapped:":
m.ShmemPmdMapped = v
case "CmaTotal:":
m.CmaTotal = v
case "CmaFree:":
m.CmaFree = v
case "HugePages_Total:":
m.HugePagesTotal = v
case "HugePages_Free:":
m.HugePagesFree = v
case "HugePages_Rsvd:":
m.HugePagesRsvd = v
case "HugePages_Surp:":
m.HugePagesSurp = v
case "Hugepagesize:":
m.Hugepagesize = v
case "DirectMap4k:":
m.DirectMap4k = v
case "DirectMap2M:":
m.DirectMap2M = v
case "DirectMap1G:":
m.DirectMap1G = v
}
}
return &m, nil
}

View File

@@ -15,19 +15,13 @@ package procfs
import (
"bufio"
"bytes"
"fmt"
"io"
"os"
"strconv"
"strings"
)
var validOptionalFields = map[string]bool{
"shared": true,
"master": true,
"propagate_from": true,
"unbindable": true,
}
"github.com/prometheus/procfs/internal/util"
)
// A MountInfo is a type that describes the details, options
// for each mount, parsed from /proc/self/mountinfo.
@@ -58,18 +52,10 @@ type MountInfo struct {
SuperOptions map[string]string
}
// Returns part of the mountinfo line, if it exists, else an empty string.
func getStringSliceElement(parts []string, idx int, defaultValue string) string {
if idx >= len(parts) {
return defaultValue
}
return parts[idx]
}
// Reads each line of the mountinfo file, and returns a list of formatted MountInfo structs.
func parseMountInfo(r io.Reader) ([]*MountInfo, error) {
func parseMountInfo(info []byte) ([]*MountInfo, error) {
mounts := []*MountInfo{}
scanner := bufio.NewScanner(r)
scanner := bufio.NewScanner(bytes.NewReader(info))
for scanner.Scan() {
mountString := scanner.Text()
parsedMounts, err := parseMountInfoString(mountString)
@@ -89,57 +75,75 @@ func parseMountInfo(r io.Reader) ([]*MountInfo, error) {
func parseMountInfoString(mountString string) (*MountInfo, error) {
var err error
// OptionalFields can be zero, hence these checks to ensure we do not populate the wrong values in the wrong spots
separatorIndex := strings.Index(mountString, "-")
if separatorIndex == -1 {
return nil, fmt.Errorf("no separator found in mountinfo string: %s", mountString)
mountInfo := strings.Split(mountString, " ")
mountInfoLength := len(mountInfo)
if mountInfoLength < 11 {
return nil, fmt.Errorf("couldn't find enough fields in mount string: %s", mountString)
}
beforeFields := strings.Fields(mountString[:separatorIndex])
afterFields := strings.Fields(mountString[separatorIndex+1:])
if (len(beforeFields) + len(afterFields)) < 7 {
return nil, fmt.Errorf("too few fields")
if mountInfo[mountInfoLength-4] != "-" {
return nil, fmt.Errorf("couldn't find separator in expected field: %s", mountInfo[mountInfoLength-4])
}
mount := &MountInfo{
MajorMinorVer: getStringSliceElement(beforeFields, 2, ""),
Root: getStringSliceElement(beforeFields, 3, ""),
MountPoint: getStringSliceElement(beforeFields, 4, ""),
Options: mountOptionsParser(getStringSliceElement(beforeFields, 5, "")),
MajorMinorVer: mountInfo[2],
Root: mountInfo[3],
MountPoint: mountInfo[4],
Options: mountOptionsParser(mountInfo[5]),
OptionalFields: nil,
FSType: getStringSliceElement(afterFields, 0, ""),
Source: getStringSliceElement(afterFields, 1, ""),
SuperOptions: mountOptionsParser(getStringSliceElement(afterFields, 2, "")),
FSType: mountInfo[mountInfoLength-3],
Source: mountInfo[mountInfoLength-2],
SuperOptions: mountOptionsParser(mountInfo[mountInfoLength-1]),
}
mount.MountId, err = strconv.Atoi(getStringSliceElement(beforeFields, 0, ""))
mount.MountId, err = strconv.Atoi(mountInfo[0])
if err != nil {
return nil, fmt.Errorf("failed to parse mount ID")
}
mount.ParentId, err = strconv.Atoi(getStringSliceElement(beforeFields, 1, ""))
mount.ParentId, err = strconv.Atoi(mountInfo[1])
if err != nil {
return nil, fmt.Errorf("failed to parse parent ID")
}
// Has optional fields, which is a space separated list of values.
// Example: shared:2 master:7
if len(beforeFields) > 6 {
mount.OptionalFields = make(map[string]string)
optionalFields := beforeFields[6:]
for _, field := range optionalFields {
optionSplit := strings.Split(field, ":")
target, value := optionSplit[0], ""
if len(optionSplit) == 2 {
value = optionSplit[1]
}
// Checks if the 'keys' in the optional fields in the mountinfo line are acceptable.
// Allowed 'keys' are shared, master, propagate_from, unbindable.
if _, ok := validOptionalFields[target]; ok {
mount.OptionalFields[target] = value
}
if mountInfo[6] != "" {
mount.OptionalFields, err = mountOptionsParseOptionalFields(mountInfo[6 : mountInfoLength-4])
if err != nil {
return nil, err
}
}
return mount, nil
}
// mountOptionsIsValidField checks a string against a valid list of optional fields keys.
func mountOptionsIsValidField(s string) bool {
switch s {
case
"shared",
"master",
"propagate_from",
"unbindable":
return true
}
return false
}
// mountOptionsParseOptionalFields parses a list of optional fields strings into a double map of strings.
func mountOptionsParseOptionalFields(o []string) (map[string]string, error) {
optionalFields := make(map[string]string)
for _, field := range o {
optionSplit := strings.SplitN(field, ":", 2)
value := ""
if len(optionSplit) == 2 {
value = optionSplit[1]
}
if mountOptionsIsValidField(optionSplit[0]) {
optionalFields[optionSplit[0]] = value
}
}
return optionalFields, nil
}
// Parses the mount options, superblock options.
func mountOptionsParser(mountOptions string) map[string]string {
opts := make(map[string]string)
@@ -159,20 +163,18 @@ func mountOptionsParser(mountOptions string) map[string]string {
// Retrieves mountinfo information from `/proc/self/mountinfo`.
func GetMounts() ([]*MountInfo, error) {
f, err := os.Open("/proc/self/mountinfo")
data, err := util.ReadFileNoStat("/proc/self/mountinfo")
if err != nil {
return nil, err
}
defer f.Close()
return parseMountInfo(f)
return parseMountInfo(data)
}
// Retrieves mountinfo information from a processes' `/proc/<pid>/mountinfo`.
func GetProcMounts(pid int) ([]*MountInfo, error) {
f, err := os.Open(fmt.Sprintf("/proc/%d/mountinfo", pid))
data, err := util.ReadFileNoStat(fmt.Sprintf("/proc/%d/mountinfo", pid))
if err != nil {
return nil, err
}
defer f.Close()
return parseMountInfo(f)
return parseMountInfo(data)
}

View File

@@ -183,7 +183,6 @@ func (netDev NetDev) Total() NetDevLine {
names = append(names, ifc.Name)
total.RxBytes += ifc.RxBytes
total.RxPackets += ifc.RxPackets
total.RxPackets += ifc.RxPackets
total.RxErrors += ifc.RxErrors
total.RxDropped += ifc.RxDropped
total.RxFIFO += ifc.RxFIFO

163
vendor/github.com/prometheus/procfs/net_sockstat.go generated vendored Normal file
View File

@@ -0,0 +1,163 @@
// Copyright 2019 The Prometheus Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package procfs
import (
"bufio"
"bytes"
"errors"
"fmt"
"io"
"strings"
"github.com/prometheus/procfs/internal/util"
)
// A NetSockstat contains the output of /proc/net/sockstat{,6} for IPv4 or IPv6,
// respectively.
type NetSockstat struct {
// Used is non-nil for IPv4 sockstat results, but nil for IPv6.
Used *int
Protocols []NetSockstatProtocol
}
// A NetSockstatProtocol contains statistics about a given socket protocol.
// Pointer fields indicate that the value may or may not be present on any
// given protocol.
type NetSockstatProtocol struct {
Protocol string
InUse int
Orphan *int
TW *int
Alloc *int
Mem *int
Memory *int
}
// NetSockstat retrieves IPv4 socket statistics.
func (fs FS) NetSockstat() (*NetSockstat, error) {
return readSockstat(fs.proc.Path("net", "sockstat"))
}
// NetSockstat6 retrieves IPv6 socket statistics.
//
// If IPv6 is disabled on this kernel, the returned error can be checked with
// os.IsNotExist.
func (fs FS) NetSockstat6() (*NetSockstat, error) {
return readSockstat(fs.proc.Path("net", "sockstat6"))
}
// readSockstat opens and parses a NetSockstat from the input file.
func readSockstat(name string) (*NetSockstat, error) {
// This file is small and can be read with one syscall.
b, err := util.ReadFileNoStat(name)
if err != nil {
// Do not wrap this error so the caller can detect os.IsNotExist and
// similar conditions.
return nil, err
}
stat, err := parseSockstat(bytes.NewReader(b))
if err != nil {
return nil, fmt.Errorf("failed to read sockstats from %q: %v", name, err)
}
return stat, nil
}
// parseSockstat reads the contents of a sockstat file and parses a NetSockstat.
func parseSockstat(r io.Reader) (*NetSockstat, error) {
var stat NetSockstat
s := bufio.NewScanner(r)
for s.Scan() {
// Expect a minimum of a protocol and one key/value pair.
fields := strings.Split(s.Text(), " ")
if len(fields) < 3 {
return nil, fmt.Errorf("malformed sockstat line: %q", s.Text())
}
// The remaining fields are key/value pairs.
kvs, err := parseSockstatKVs(fields[1:])
if err != nil {
return nil, fmt.Errorf("error parsing sockstat key/value pairs from %q: %v", s.Text(), err)
}
// The first field is the protocol. We must trim its colon suffix.
proto := strings.TrimSuffix(fields[0], ":")
switch proto {
case "sockets":
// Special case: IPv4 has a sockets "used" key/value pair that we
// embed at the top level of the structure.
used := kvs["used"]
stat.Used = &used
default:
// Parse all other lines as individual protocols.
nsp := parseSockstatProtocol(kvs)
nsp.Protocol = proto
stat.Protocols = append(stat.Protocols, nsp)
}
}
if err := s.Err(); err != nil {
return nil, err
}
return &stat, nil
}
// parseSockstatKVs parses a string slice into a map of key/value pairs.
func parseSockstatKVs(kvs []string) (map[string]int, error) {
if len(kvs)%2 != 0 {
return nil, errors.New("odd number of fields in key/value pairs")
}
// Iterate two values at a time to gather key/value pairs.
out := make(map[string]int, len(kvs)/2)
for i := 0; i < len(kvs); i += 2 {
vp := util.NewValueParser(kvs[i+1])
out[kvs[i]] = vp.Int()
if err := vp.Err(); err != nil {
return nil, err
}
}
return out, nil
}
// parseSockstatProtocol parses a NetSockstatProtocol from the input kvs map.
func parseSockstatProtocol(kvs map[string]int) NetSockstatProtocol {
var nsp NetSockstatProtocol
for k, v := range kvs {
// Capture the range variable to ensure we get unique pointers for
// each of the optional fields.
v := v
switch k {
case "inuse":
nsp.InUse = v
case "orphan":
nsp.Orphan = &v
case "tw":
nsp.TW = &v
case "alloc":
nsp.Alloc = &v
case "mem":
nsp.Mem = &v
case "memory":
nsp.Memory = &v
}
}
return nsp
}

View File

@@ -207,10 +207,6 @@ func (u NetUnix) parseUsers(hexStr string) (uint64, error) {
return strconv.ParseUint(hexStr, 16, 32)
}
func (u NetUnix) parseProtocol(hexStr string) (uint64, error) {
return strconv.ParseUint(hexStr, 16, 32)
}
func (u NetUnix) parseType(hexStr string) (NetUnixType, error) {
typ, err := strconv.ParseUint(hexStr, 16, 16)
if err != nil {

View File

@@ -22,6 +22,7 @@ import (
"strings"
"github.com/prometheus/procfs/internal/fs"
"github.com/prometheus/procfs/internal/util"
)
// Proc provides information about a running process.
@@ -121,13 +122,7 @@ func (fs FS) AllProcs() (Procs, error) {
// CmdLine returns the command line of a process.
func (p Proc) CmdLine() ([]string, error) {
f, err := os.Open(p.path("cmdline"))
if err != nil {
return nil, err
}
defer f.Close()
data, err := ioutil.ReadAll(f)
data, err := util.ReadFileNoStat(p.path("cmdline"))
if err != nil {
return nil, err
}
@@ -141,13 +136,7 @@ func (p Proc) CmdLine() ([]string, error) {
// Comm returns the command name of a process.
func (p Proc) Comm() (string, error) {
f, err := os.Open(p.path("comm"))
if err != nil {
return "", err
}
defer f.Close()
data, err := ioutil.ReadAll(f)
data, err := util.ReadFileNoStat(p.path("comm"))
if err != nil {
return "", err
}
@@ -252,13 +241,11 @@ func (p Proc) MountStats() ([]*Mount, error) {
// It supplies information missing in `/proc/self/mounts` and
// fixes various other problems with that file too.
func (p Proc) MountInfo() ([]*MountInfo, error) {
f, err := os.Open(p.path("mountinfo"))
data, err := util.ReadFileNoStat(p.path("mountinfo"))
if err != nil {
return nil, err
}
defer f.Close()
return parseMountInfo(f)
return parseMountInfo(data)
}
func (p Proc) fileDescriptors() ([]string, error) {

View File

@@ -14,22 +14,16 @@
package procfs
import (
"io/ioutil"
"os"
"strings"
"github.com/prometheus/procfs/internal/util"
)
// Environ reads process environments from /proc/<pid>/environ
func (p Proc) Environ() ([]string, error) {
environments := make([]string, 0)
f, err := os.Open(p.path("environ"))
if err != nil {
return environments, err
}
defer f.Close()
data, err := ioutil.ReadAll(f)
data, err := util.ReadFileNoStat(p.path("environ"))
if err != nil {
return environments, err
}

View File

@@ -15,11 +15,10 @@ package procfs
import (
"bufio"
"fmt"
"io/ioutil"
"os"
"bytes"
"regexp"
"strings"
"github.com/prometheus/procfs/internal/util"
)
// Regexp variables
@@ -46,21 +45,15 @@ type ProcFDInfo struct {
// FDInfo constructor. On kernels older than 3.8, InotifyInfos will always be empty.
func (p Proc) FDInfo(fd string) (*ProcFDInfo, error) {
f, err := os.Open(p.path("fdinfo", fd))
data, err := util.ReadFileNoStat(p.path("fdinfo", fd))
if err != nil {
return nil, err
}
defer f.Close()
fdinfo, err := ioutil.ReadAll(f)
if err != nil {
return nil, fmt.Errorf("could not read %s: %s", f.Name(), err)
}
var text, pos, flags, mntid string
var inotify []InotifyInfo
scanner := bufio.NewScanner(strings.NewReader(string(fdinfo)))
scanner := bufio.NewScanner(bytes.NewReader(data))
for scanner.Scan() {
text = scanner.Text()
if rPos.MatchString(text) {

View File

@@ -15,8 +15,8 @@ package procfs
import (
"fmt"
"io/ioutil"
"os"
"github.com/prometheus/procfs/internal/util"
)
// ProcIO models the content of /proc/<pid>/io.
@@ -43,13 +43,7 @@ type ProcIO struct {
func (p Proc) IO() (ProcIO, error) {
pio := ProcIO{}
f, err := os.Open(p.path("io"))
if err != nil {
return pio, err
}
defer f.Close()
data, err := ioutil.ReadAll(f)
data, err := util.ReadFileNoStat(p.path("io"))
if err != nil {
return pio, err
}

View File

@@ -24,11 +24,13 @@ package procfs
// > full avg10=0.00 avg60=0.13 avg300=0.96 total=8183134
import (
"bufio"
"bytes"
"fmt"
"io"
"io/ioutil"
"os"
"strings"
"github.com/prometheus/procfs/internal/util"
)
const lineFormat = "avg10=%f avg60=%f avg300=%f total=%d"
@@ -55,24 +57,21 @@ type PSIStats struct {
// resource from /proc/pressure/<resource>. At time of writing this can be
// either "cpu", "memory" or "io".
func (fs FS) PSIStatsForResource(resource string) (PSIStats, error) {
file, err := os.Open(fs.proc.Path(fmt.Sprintf("%s/%s", "pressure", resource)))
data, err := util.ReadFileNoStat(fs.proc.Path(fmt.Sprintf("%s/%s", "pressure", resource)))
if err != nil {
return PSIStats{}, fmt.Errorf("psi_stats: unavailable for %s", resource)
}
defer file.Close()
return parsePSIStats(resource, file)
return parsePSIStats(resource, bytes.NewReader(data))
}
// parsePSIStats parses the specified file for pressure stall information
func parsePSIStats(resource string, file io.Reader) (PSIStats, error) {
func parsePSIStats(resource string, r io.Reader) (PSIStats, error) {
psiStats := PSIStats{}
stats, err := ioutil.ReadAll(file)
if err != nil {
return psiStats, fmt.Errorf("psi_stats: unable to read data for %s", resource)
}
for _, l := range strings.Split(string(stats), "\n") {
scanner := bufio.NewScanner(r)
for scanner.Scan() {
l := scanner.Text()
prefix := strings.Split(l, " ")[0]
switch prefix {
case "some":

View File

@@ -16,10 +16,10 @@ package procfs
import (
"bytes"
"fmt"
"io/ioutil"
"os"
"github.com/prometheus/procfs/internal/fs"
"github.com/prometheus/procfs/internal/util"
)
// Originally, this USER_HZ value was dynamically retrieved via a sysconf call
@@ -113,13 +113,7 @@ func (p Proc) NewStat() (ProcStat, error) {
// Stat returns the current status information of the process.
func (p Proc) Stat() (ProcStat, error) {
f, err := os.Open(p.path("stat"))
if err != nil {
return ProcStat{}, err
}
defer f.Close()
data, err := ioutil.ReadAll(f)
data, err := util.ReadFileNoStat(p.path("stat"))
if err != nil {
return ProcStat{}, err
}

View File

@@ -15,13 +15,13 @@ package procfs
import (
"bytes"
"io/ioutil"
"os"
"strconv"
"strings"
"github.com/prometheus/procfs/internal/util"
)
// ProcStat provides status information about the process,
// ProcStatus provides status information about the process,
// read from /proc/[pid]/stat.
type ProcStatus struct {
// The process ID.
@@ -29,6 +29,9 @@ type ProcStatus struct {
// The process name.
Name string
// Thread group ID.
TGID int
// Peak virtual memory size.
VmPeak uint64
// Virtual memory size.
@@ -72,13 +75,7 @@ type ProcStatus struct {
// NewStatus returns the current status information of the process.
func (p Proc) NewStatus() (ProcStatus, error) {
f, err := os.Open(p.path("status"))
if err != nil {
return ProcStatus{}, err
}
defer f.Close()
data, err := ioutil.ReadAll(f)
data, err := util.ReadFileNoStat(p.path("status"))
if err != nil {
return ProcStatus{}, err
}
@@ -113,6 +110,8 @@ func (p Proc) NewStatus() (ProcStatus, error) {
func (s *ProcStatus) fillStatus(k string, vString string, vUint uint64, vUintBytes uint64) {
switch k {
case "Tgid":
s.TGID = int(vUint)
case "Name":
s.Name = vString
case "VmPeak":

View File

@@ -26,7 +26,7 @@ var (
procLineRE = regexp.MustCompile(`(\d+) (\d+) (\d+)`)
)
// Schedstat contains scheduler statistics from /proc/schedstats
// Schedstat contains scheduler statistics from /proc/schedstat
//
// See
// https://www.kernel.org/doc/Documentation/scheduler/sched-stats.txt
@@ -36,7 +36,6 @@ var (
// jiffies when they are actually in nanoseconds since 2.6.23 with the
// introduction of CFS. A fix to the documentation is pending. See
// https://lore.kernel.org/patchwork/project/lkml/list/?series=403473
type Schedstat struct {
CPUs []*SchedstatCPU
}
@@ -57,6 +56,7 @@ type ProcSchedstat struct {
RunTimeslices uint64
}
// Schedstat reads data from /proc/schedstat
func (fs FS) Schedstat() (*Schedstat, error) {
file, err := os.Open(fs.proc.Path("schedstat"))
if err != nil {

View File

@@ -15,13 +15,14 @@ package procfs
import (
"bufio"
"bytes"
"fmt"
"io"
"os"
"strconv"
"strings"
"github.com/prometheus/procfs/internal/fs"
"github.com/prometheus/procfs/internal/util"
)
// CPUStat shows how much time the cpu spend in various stages.
@@ -164,16 +165,15 @@ func (fs FS) NewStat() (Stat, error) {
// Stat returns information about current cpu/process statistics.
// See https://www.kernel.org/doc/Documentation/filesystems/proc.txt
func (fs FS) Stat() (Stat, error) {
f, err := os.Open(fs.proc.Path("stat"))
fileName := fs.proc.Path("stat")
data, err := util.ReadFileNoStat(fileName)
if err != nil {
return Stat{}, err
}
defer f.Close()
stat := Stat{}
scanner := bufio.NewScanner(f)
scanner := bufio.NewScanner(bytes.NewReader(data))
for scanner.Scan() {
line := scanner.Text()
parts := strings.Fields(scanner.Text())
@@ -237,7 +237,7 @@ func (fs FS) Stat() (Stat, error) {
}
if err := scanner.Err(); err != nil {
return Stat{}, fmt.Errorf("couldn't parse %s: %s", f.Name(), err)
return Stat{}, fmt.Errorf("couldn't parse %s: %s", fileName, err)
}
return stat, nil