Compile fixed patterns during monitor and log counter initialization, pass compiled expressions to the log buffer, and reject invalid log counter patterns before watching logs.
Recorded events refer to the node with an ObjectReference that has no
UID. The client already reads the Node object at startup to check that
kube-apiserver is ready, and this change stores the UID from that read.
The node status patch returns the Node object, so the client stores
that UID too.
This change removes the TODO in problem_client.go.
/livez still checks process and dataplane sync health; it only excludes
the node-eligibility check used by /healthz for scale-down draining.
Signed-off-by: dpacgdm <dpac.gdm@gmail.com>
config/disk-log-message-filelog.json contained trailing commas and no
trailing newline, so it could not be parsed by encoding/json and NPD
would fail to start when configured with it.
- Fix the invalid JSON
- Add TestShippedConfigs, which verifies that every shipped JSON config
under config/ (top level and one directory down) parses (catching
trailing commas, UTF-8 BOMs and other syntax errors) and is in the
canonical format (2-space indent, trailing newline), showing a diff on
failure
- Add `make fmt-configs` to rewrite nonconforming files in place
(UPDATE_EXPECTED=y)
- Normalize the existing config files to the canonical format
(whitespace-only changes)
This change keeps each compiled regular expression in a cache in the
log buffer. The cache size is not more than the number of patterns
given to Match.
This change removes the TODO in log_buffer.go.
Group rules by their effective invoke interval.
Use the global interval when a rule omits its interval.
Reject non-positive invoke intervals before scheduler startup.
Test grouped cadence, concurrency, and shutdown behavior.
The restart path closes the failed parser before retrying. If stopping
is signaled during the retry wait, watchLoop's deferred cleanup closed
the same parser again, logging a spurious 'file already closed' error
at shutdown. Clear the reference after closing and nil-check the defer.
The log monitor stops draining logCh before calling watcher.Stop(), so
with a full channel (e.g. a kmsg burst at shutdown) watchLoop blocked
on the send forever, never called tomb.Done(), and Stop() hung.
Select on tomb.Stopping() alongside the send.
If the restarted parser's channel closes again right away (reads keep
failing on the reopened /dev/kmsg), the watcher restarts in a tight
loop with no delay, spinning a CPU core and flooding the logs
(measured 37k restarts in 200ms). Delay the first attempt when the
previous restart was less than retryDelay ago.
Cluster Autoscaler marks nodes with ToBeDeletedByClusterAutoscaler during
scale-down. kube-proxy /healthz intentionally fails in that case for LB
connection draining (KEP-3836), which made NPD report kube-proxy as
unhealthy. Switch the kube-proxy probe to /livez, which reflects process
health only.
Signed-off-by: dpacgdm <dpac.gdm@gmail.com>
Closing the parser in Stop() triggers kmsgparser's read goroutine to
close its output channel, which made watchLoop take the restart path
(added in #1192) during intentional shutdown. Leave parser cleanup to
watchLoop's defer, which is the single owner of the parser lifecycle.
Usage of `localhost` is family agnostic and will work regardless if
cluster is IPv4 or IPv6. The current value of `127.0.0.1` only works
for IPv4 clusters.
BREAKING CHANGE: It may break in rare cases where `localhost` does not
resolve as `127.0.0.1` (if OS config does not follow RFC5735 and
RFC6761).
Both outputs are currently hardcoded to being enabled, this allows disabling one or the other. Defaults to both enabled to retain current behavior.
Larger clusters can save some etcd I/O by skipping one of these outputs if they aren't being consumed. In our case we aren't consuming the Events so writing them just creates more churn.
kubernetes-prow[bot]andGitHub
Ciprian Hacman
Veer Singh
dpacgdm
Kubernetes Prow RobotandGitHub
Arjun Raman
Arjun RamanandGitHub
novahe
Sergey Kanzhelev
marinoborgesandGitHub
Johanan Liebermann
Marek Chodor
Chris Henzie
Nick Parker
googs1025
tashen
Sudheer Nedlumane
Zhen Wang
Henry Beberman