OVH used Sydney region to pilot mass reboots for KVM Januscape fix

OVH used its Sydney datacenter as the first production test for a fleetwide emergency patch of Januscape, a critical KVM guest-to-host escape tracked as CVE-2026-53359. The French cloud operator said the incident forced it to reboot tens of thousands of hosts supporting about 1 million virtual machines, a disruptive choice that shows how little room cloud providers have when a hypervisor isolation bug is exploitable.

Julien Levrard, OVH’s chief information security officer, described the response in a Monday post that gave an unusually specific account of the operational trade-offs behind a major cloud security patch. Januscape allowed an attacker with root access inside a guest virtual machine to run code as root on the underlying host, crash that host, or compromise other guest VMs on the same machine, according to OVH.

That class of bug goes directly at the multi-tenant model used by large KVM-based clouds. Providers sell the assumption that one customer’s workload cannot interfere with another’s. A working guest-host escape breaks that boundary.

Why OVH chose reboots

OVH considered several mitigations and rejected them. Disabling nested virtualization through a small host configuration change was not workable, Levrard said, because OVH does not know which tenants depend on the feature and uses it internally to move VMs between physical hosts. Live patching was considered too risky because it could add instability. Live migration to already patched hosts was also rejected because OVH believed moving the full VM fleet that way could take months.

The company instead backported a fix into the Debian distribution it runs in production and scheduled host reboots, with advance notice to customers but no opt-out. OVH’s executive committee approved the plan on the grounds that the fleet needed to be patched before attacks began, handling customers one by one would leave the cloud exposed longer, and the approach protected the largest share of customers while accepting impact on a smaller group.

OVH also limited what it told customers while the work was under way. Levrard wrote that more detailed communication during execution, while infrastructure remained unpatched, could have raised risk by encouraging people to test the public exploit.

Sydney went first

OVH chose Sydney as the pilot region because it is one of the company’s smaller regions and Australia’s east coast time zone let European teams run the operation during their workday while hitting a quieter period in Sydney. The rollout then proceeded in waves across other regions.

The company did not reboot hosts by rack alone. OVH said its orchestrators calculated which hosts ran instances from the same customer project, then prevented two hosts serving the same project from being rebooted in the same window. A host had to return before the next one in the same anti-affinity class could be taken down.

OVH set shutdown thresholds for the operation: 15 simultaneous host failures in high-density regions, or five in other regions. The process still produced problems. Some VMs did not restart after hypervisor reboots, some saw data corruption during forced shutdowns, and OpenStack APIs returned HTTP 503 errors for hours, causing one patching wave to be delayed. In Canada, Levrard said API traffic reached 10 times the usual peak and overloaded management and support teams.

Hardware failures also surfaced. Levrard said about 20 to 30 hosts out of 6,000 failed to recover by themselves on the first night, citing faulty memory, BIOS configuration issues and inactive network interfaces. Some systems required new CMOS batteries.

Levrard called the operation successful relative to its scale, while saying OVH needs to improve before the next kernel vulnerability response. The company is conducting a post-mortem focused on reducing restart impact and improving customer notice and support.

This story draws on original reporting from The Register.