Commit Graph
40 Commits
Author SHA1 Message Date
Archit Bansal 44dc4aa6c1 Add health-check-monitor 2020-05-27 14:08:42 -07:00
Xuewei Zhang 83b09277f0 Collect more cpu/disk/memory metrics 2020-02-03 15:29:45 -08:00
Xuewei Zhang b3f811d171 Add detection for ext4 errors 2019-12-06 14:49:17 -08:00
Kubernetes Prow RobotandGitHub 3a41fc2fc3 Merge pull request #392 from arekkusu/origin/patch-2
Improve systemctl check, style + cleanup
2019-11-29 01:33:03 -08:00
Alexandre 4df720c2a0 Improve systemctl check, style + cleanup
- Use `systemctl is-active` to check if service is running
  - Cleaner that `grep` on `systemctl status` output
  - Return success means service is running/active
  - Return failure means not running which could be due to
    stopped/failed service or that service does not exist

- Use `command -v` instead of `which`
  Ref: https://github.com/koalaman/shellcheck/wiki/SC2230

- Follow Google "Shell Style Guide": indent, use "readonly"

- Minor: Rephrase comment, avoid all caps
2019-11-29 14:14:19 +09:00
Alexandre a91b568149 Support "nf_conntrack", check 90% full, style
- Script was checking for "ip_conntrack_..." which was replaced by "nf_conntrack_..." on newer system. Now support both.

-  Return failure ("not ok") when table is more than 90% full.
  - Not sure what value is best here but I think that is better than when the table is full.
    Otherwise we might end up with a value close to the max or bouncing around.

- Replaced cat by "$(< file )" to avoid calling external command
- Follow Google "Shell Style Guide": 2 space indent, use preferred "[[ test ]]", add "readonly"
- Include current connection usage in output message
2019-11-29 13:20:37 +09:00
Kubernetes Prow RobotandGitHub 5345185ec2 Merge pull request #341 from iranzo/patch-1
Update network_problem.sh
2019-09-15 01:00:37 -07:00
Xuewei Zhang 0f0e5eff0f Adding stackdriver exporter 2019-09-12 18:30:00 -07:00
Pablo Iranzo Gómez fa94b42849 Use bashate recommendations on network_problem script 2019-09-05 15:46:45 +02:00
Xuewei Zhang f9b5e60a43 Add e2e test for NPD
The first test is a very simple test. It installs NPD on a VM, and then
verifies that NPD reports metric host_uptime in Prometheus format.
2019-08-16 01:33:29 -07:00
Zhen Wang a8527712f6 Update the detection method for docker overlay2 issue 2019-08-01 22:16:44 -07:00
Zhen Wang 570ae0cb20 Make systemd monitor look back for 5m 2019-07-30 11:17:02 -07:00
Xuewei Zhang 94af7de97b Report metrics from custom-plugin-monitor 2019-07-25 11:28:38 -07:00
Xuewei Zhang fbebcf311b Report metrics from system-log-monitor 2019-07-12 14:38:21 -07:00
Xuewei Zhang 4944ac3e48 Implement host collector as part of system-stats-monitor
Host collector report three things today:
1. Host OS uptime (in seconds)
2. Host kernel version (as a metric label)
3. Host OS version (as a metric label)
2019-06-27 16:40:11 -07:00
Zhen Wang b94a555dfc Add systemd monitor for kubelet, docker, and containerd restart events 2019-06-18 10:26:53 -07:00
Xuewei Zhang 7ad5dec712 Add disk metrics support. 2019-06-13 00:51:17 -07:00
Andy Xie 33dffe0761 enable codnition updaet when message change for custom plugin 2018-12-11 13:14:49 +08:00
Zhen Wang 6b983a9ea3 Detect corrupt docker overlay2 2018-11-27 00:35:42 -08:00
Zhen Wang 1f636381b8 Detect kubelet and container runtime frequent crashes 2018-11-26 22:41:06 -08:00
Zhen Wang ecaa61e7d3 Detect readonly filesystem 2018-11-20 11:20:48 -08:00
Jan HeidbrinkandJan Heidbrink 659f31c0f2 Adapt OOMKilling pattern to current kernels 2018-07-31 15:15:45 +02:00
David Ashpole bf730e9c63 add log-counter go plugin 2018-06-20 15:55:19 -07:00
Jasmine HegmanandGitHub 76ce35cddc Possibly enhanced network_problem custom plugin
My comment was eaten by github in !152 and wanted to raise attention incase this was meant to be an exit instead of an echo, otherwise feel free to close!
2018-01-05 11:26:15 -07:00
Rohit Ramkumar 69b6b58ee3 Addressed comments 2017-12-19 08:32:27 -08:00
Rohit Ramkumar cd472c7765 Add empty conditions list 2017-11-27 11:35:48 -08:00
Rohit Ramkumar fb12f3b70e Add network monitor script as plugin 2017-11-27 11:33:38 -08:00
Andy Xie 10dbfef1a8 add custom problem detector plugin 2017-11-22 10:14:09 +08:00
Ajit Kumar d2de52f090 Add rule for docker image pull error 2017-06-21 13:48:58 -07:00
Julius Milan b579984f0a Fix abrt-adaptor config for cpp problems
This modifies pattern for catching cpp problem messages produced by
ABRT. Found that not all mentioned messages fit into former pattern.
For example following is valid cpp problem message produced by ABRT:

Process xxx (bad_binary) crashed in Will::Fail::a() [clone .isra.2]()

but doesn't fit former pattern, since it's last part contains
whitespaces.
2017-05-11 15:40:25 +02:00
Julius Milan abcf6a4f4b Add ABRT adaptor config 2017-03-23 16:15:56 +01:00
Random-Liu 10fc831409 Change kernel specific name in code base and change syslog to filelog. 2017-02-15 13:07:01 -08:00
Random-Liu 27cc831408 Add arbitrary daemon log support 2017-02-10 11:32:35 -08:00
Random-Liu d281cb8a15 Fix kernel monitor issues:
* Change `unregister_netdevice` to be an event to fix #47.
* Change `KernelPanic` to `KernelOops` because we can't handle kernel
panic currently.
* Use system boot time instead of "StartPattern" to fix #48.
2017-02-09 16:09:27 -08:00
Random-Liu 2ef2af99eb Update Readme.md 2017-01-19 01:59:09 -08:00
Random-Liu c15d463ad5 Finish the journald support 2017-01-19 01:59:09 -08:00
Lantao Liu 532f933bd8 This PR:
1) Add lookback support in kernel monitor. After started, Kernel monitor
will check some old logs to detect problems which happened before last
node reboot.
2) Add `lookback` and `startPattern` in kernel monitor configuration.
  * `lookback` specifies how long time kernel monitor should look back.
  * `startPattern` specifies which log indicates the node is started.
  kernel monitor will clear all current node conditions once it finds
  a node start log. This makes sure that old problems won't change the
  node condition.
3) Add support for kernel panic monitoring, the null pointer and divide
0 kernel panic will be surfaced as event. Usually kernel monitor will
report these events during looking back phase.
2016-08-20 19:11:26 -07:00
Lantao Liu 5b07afd325 1. Make source and conditions configurable.
2. Add multiple events and conditions support in problem interface.
2016-06-02 15:32:02 -07:00
Lantao Liu 8759e4d610 Use Patch instead of UpdateStatus. 2016-05-30 19:22:32 -07:00
Lantao Liu f0312655bd Add first version of node-problem-detector 2016-05-17 15:55:33 -07:00