From 93c8242b894125f6c64d5bb16bf08892e2f9f472 Mon Sep 17 00:00:00 2001 From: Jack Francis Date: Mon, 11 Jan 2021 13:35:25 -0800 Subject: [PATCH] always drain before reboot This changes the pre-reboot drain functionality so that it always runs, regardless of the value of the Unschedulable node property. Because kubectl drain is idempotent, we shouldn't have to worry about whether the node has already been set to Unschedulable (perhaps due to a prior, unsuccessful loop of the kured reboot cycle): we can run it over and over again. And because this drain func actually does a cordon + drain (and it only performs the drain if a cordon is successful), we can be sure that we aren't going to be thrashing this node w/ respect to scheduled pods. This also fixes an edge case: if the node has been marked Unschedulable out-of-band, but workloads remain Running on this node, kured will no longer reboot the node's underlying VM/machine while it is actively running pods. --- cmd/kured/main.go | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/cmd/kured/main.go b/cmd/kured/main.go index d2626fc..e1a57e4 100644 --- a/cmd/kured/main.go +++ b/cmd/kured/main.go @@ -377,9 +377,7 @@ func rebootAsRequired(nodeID string, window *timewindow.TimeWindow, TTL time.Dur continue } - if !nodeMeta.Unschedulable { - drain(client, node) - } + drain(client, node) commandReboot(nodeID) for { log.Infof("Waiting for reboot")