Google Cloud London outage caused by PLC misconfiguration

Google Cloud said an October 20 outage at its London europe-west-2-b availability zone happened because a misconfiguration prevented emergency generators from starting after a power failure. The outage affected services at Lloyds, HSBC, Royal Mail, and Revolut. Google later reprogrammed the affected controller and changed the site's electrical setup.

What failed

The facility uses a 2N utility power architecture with complete duplicate power paths for every critical load, backed by a central high-voltage generator bank managed by a dual-redundant Programmable Logic Controller (PLC). It receives power from two independent utility feeders, called A and B, plus a standby utility connection configured for manual campus-wide transfer during maintenance or emergency events.

Google said the root cause was a failure in an external underground cable that brings power to the site. That cable connects to primary feeder A and broke while feeder B was not working, because circuit breaker feeder B had been physically opened for corrective repairs after issues identified during a quarterly generator load test on 14 Oct 2026. As a result, primary utility feed A was supporting the total critical load while the feed B breaker stayed open.

Why generators did not start

The site uses uninterruptible power supplies (UPS) that Google said can keep systems running for five minutes if the availability zone loses its grid connection. The UPSes worked, but a voltage-sensing design flaw and PLC misconfiguration prevented the generator plant from receiving an automatic start signal. PLC logic indicated that a healthy utility source was available even though the primary and secondary distribution feeds were de-energized, so it did not issue a start command. The UPSes then ran out of power while the generators remained off.

Changes after the incident

After reconnecting to its power utility, Google reprogrammed the PLC so it would not repeat the mistake. The company has since made many changes to the electrical setup at europe-west-2-b and says it tested them and that they all work.

Google also listed two action items: a communications enhancement to bolster notification speed by developing an automated alert-to-communication mechanism for location-impacting incidents, and an effort to expand platform-wide testing outcomes for infrastructure-level events.

Sources: The Register