Unattended OS upgrades on a DietPi k3s cluster
My cluster mostly updates itself: ArgoCD syncs the manifests and Renovate keeps the image tags current. The one thing I still SSH’d in to do was patch the nodes’ own OS. Now I handed that off too, with the system-upgrade-controller running a monthly apt full-upgrade Plan and unattended-upgrades applying security fixes in between.
The setup
K3s itself tracks the stable channel via system-upgrade-controller (SUC). Security packages are handled continuously by unattended-upgrades. Once a month a second SUC Plan drains one node at a time, runs apt-get full-upgrade in a chroot of the host, and reboots the node into the new kernel. A CronJob bumps the Plan’s version on the first Sunday of the month so it re-runs.
The node didn’t reboot
DietPi disables systemd-logind by default. This makes systemctl reboot fail with login1.service failed to load. So the drain succeeds, the upgrade runs, and then the node just sits there, cordoned, never coming back.
The fix is systemctl --force reboot, which bypasses logind. I changed the Plan’s self-reboot and kured’s rebootCommand to it.