Commit Graph
4 Commits
Author SHA1 Message Date
Naga Ravi Chaitanya Elluri de0567b067 Tweak the etcd alert severity 2023-06-16 09:19:17 -04:00
Naga Ravi Chaitanya Elluri 0eb8d38596 Expand SLOs profile to cover monitoring for more alerts
This commit:
- Also sets appropriate severity to avoid false failures for the
  test cases especially given that theses are monitored during the chaos
  vs post chaos. Critical alerts are all monitored post chaos with few
  monitored during the chaos that represent overall health and performance
  of the service.
- Renames Alerts to SLOs validation

Metrics reference: https://github.com/cloud-bulldozer/kube-burner/blob/f09a492b1343079d5988290761145a0fff3fd23e/cmd/kube-burner/ocp-config/alerts.yml
2023-06-14 16:58:36 -04:00
Naga Ravi Chaitanya Elluri 9858f96c78 Change the severity of the etcd leader election check to warning
This is the first step towards the goal to only have metrics tracking
the overall health and performance of the component/cluster. For instance,
for etcd disruption scenarios, leader elections are expected, we should instead
track etcd leader availability and fsync latency under critical catergory vs leader
elections.
2023-05-31 11:50:20 -04:00
Naga Ravi Chaitanya Elluri e30a4243f6 Add support to alerting on metrics evaluation
This commit enables alerting in Kraken based on the Prometheus queries defined
by the user and modifies the return code of the run to determine pass/fail for
the run.
2021-06-22 15:22:37 -04:00