Skip to content

The Cloud Runs on Generators

On November 12, 2026, Microsoft 365 users started losing access to their Exchange Online mailboxes. The problem began in the Americas and spread to every region. Microsoft’s own incident record puts the outage at just under 24 hours, and some tenants were still working through delayed mail on November 14. The next afternoon, the FAA called a nationwide ground stop that held flights for more than four hours, the first nationwide stop since January 2023.

Neither outage started with a power failure. Microsoft traced its problem to an authentication and routing component and found no sign of an attack. Transportation Secretary Sean Duffy said the FAA outage came from legacy telecom equipment that got overloaded.

Both outages still carry lessons for anyone responsible for data center backup power. Microsoft’s recovery followed the same pattern we see after a site loses power without a clean shutdown. Industry data also shows power problems cause more serious data center outages than anything else, and that’s the risk a facility owner has the most control over.

What a Retry Storm Looks Like

During the Microsoft outage, a large number of Exchange Online servers failed health checks and dropped out of service. Engineers brought servers back, and those restored servers went unresponsive again as soon as traffic hit them. Microsoft had to scale up virtual machines to absorb the surge as regions came back online. Once mail was moving again, traffic to Exchange Online ran about 19% above normal while queued messages and mailbox syncs caught up.

IT teams call this a retry storm, or a thundering herd. Every app and connected device that lost its connection starts asking to reconnect at the same moment. The first servers back online take the whole backlog at once, overload, and drop again.

Software engineers build in exponential backoff and circuit breakers to spread those requests out. Those tools assume the hardware underneath stayed up. When a server loses power without a clean shutdown, any data being written at that moment gets cut off. Databases have to be checked or restored before the application can come back, and only after that can the IT team start easing traffic back in. A power drop that lasts a few seconds can turn into hours of recovery work.

Power Is the Leading Cause of Serious Data Center Outages

Uptime Institute tracks data center outages every year. In its 2025 analysis, power problems caused 54% of the impactful outages operators reported, more than any other cause. UPS failures showed up in 42% of those power outages and transfer switch failures in 36%. Generator failures accounted for 28%, and some outages involved more than one failure. Uptime’s 2026 report found power is still the leading cause.

The load on these buildings keeps going up. Lawrence Berkeley National Laboratory found U.S. data centers used about 76 terawatt-hours of electricity in 2018 and 176 terawatt-hours in 2023. That took them from 1.9% to 4.4% of all U.S. electricity as AI servers came online. The lab’s 2024 report projected 6.7% to 12% by 2028, and its June 2026 update estimates about 11.8% by 2030.

Every rack added to a data center also adds cooling load, and both transfer to the generator when the grid drops. A generator sized for the building five years ago may come up short once GPU racks and the chillers that cool them are added.

How Data Center Backup Power Takes Over When the Grid Fails

Most power-related IT failures happen in the first seconds to minutes after the utility drops, while the facility is switching over. Here’s the sequence on a typical standby system:

  1. Utility power drops. The facility loses its primary feed.
  2. The UPS carries the IT load. UPS batteries pick up the servers instantly. Most systems are sized to carry that load for a few minutes, which gives the generator time to get going.
  3. The automatic transfer switch (ATS) signals the generator to start. The ATS senses the voltage loss and sends a start signal. The engine cranks and comes up to rated voltage and frequency. Emergency systems serving life safety loads have to restore power within 10 seconds under NFPA 110 and NEC Article 700.
  4. The ATS transfers the load. Once the generator is stable, the ATS disconnects the building from the dead utility feed and connects it to the generator. Sites running multiple generators need their paralleling switchgear to sync the units before they share the load.
  5. The generator takes the whole building in one step. The full facility load, IT and cooling included, hits the generator as a block load.
  6. Utility returns. The ATS transfers back to the grid, and the generator runs a cooldown cycle before shutting off.

If the UPS batteries run out before the generator is ready, the servers lose power. The same thing happens when the generator fails to start or stalls under the block load. Even if the generator picks up a few minutes later, the IT team is now looking at database recovery and a retry storm of its own.

Where the Handoff Breaks

These are the problems we run into most often on emergency calls:

  • Fuel. A tank nobody refilled is one of the most common no-start causes we see. Diesel that sits for a long time also breaks down, which is why we offer fuel sampling and fuel polishing.
  • Starting batteries. Weak batteries won’t crank the engine fast enough to make the transfer window.
  • UPS batteries. Batteries lose capacity as they age, so a UPS rated for 10 minutes when new can come up well short a few years later.
  • The transfer switch. An ATS that has never been tested under real load hasn’t proven it will transfer.
  • Block load acceptance. A generator can pass its weekly no-load exercise for years and still stall when the full building hits it at once.
  • Wiring after changes to the site. New equipment and rewiring can introduce problems that only show up during a transfer.

That last one bit a landfill customer of ours. Their previous service company had called it an engine problem, because every time load switched over, the engine died and the generator shut down. Our technician ran his own transfer test and found the phase rotation was wrong. The gas extraction pump was trying to turn backward every time it kicked on. Left alone, it would have kept pushing landfill gas back into the mound.

Load Bank Testing Finds Problems Before the Grid Does

Our technicians put it plainly. If your generator is going to fail, you want it to fail while we’re load bank testing it.

A load bank test is a dyno test for a generator. We connect a load bank and bring the unit from no load up to 80% or 90% of its rated capacity, often all the way to 100%. At high altitude we typically stop at 80%, because pushing to 100% there can drop frequency and fuel pressure. We also hit the unit with a block load of 60% to 80% all at once, since that’s what happens when the ATS transfers a building.

During the test we watch oil pressure and coolant temperature and check that amperage stays balanced across all phases. We also confirm the unit holds steady voltage and frequency at each load step. A typical test runs one to four hours depending on what the site requires.

A weekly no-load exercise confirms the engine starts and runs. Carrying a building takes a test under real load. Diesel units that only run lightly can also build up unburned fuel in the exhaust, called wet stacking, and a load bank run burns that off. See how our load bank testing service works.

A Data Center Backup Power Checklist

  • Check your generator size against today’s load. Add up every load that transfers to the generator, including the cooling for any new racks. Our generator size calculator and generator sizing guide are good places to start.
  • Test UPS and starting batteries. Batteries are part of every preventive maintenance visit we do.
  • Run a transfer test under real load. It’s the test that catches transfer switch faults and wiring mistakes like reversed phase rotation.
  • Load bank test at least once a year. Include a block load step.
  • Stay on a preventive maintenance schedule. Our technicians run a 31-point inspection twice a year that covers batteries, oil, coolant, and the fuel system.
  • Have a fuel plan. Know how many hours your tank covers at full load and who delivers when it runs low.

Backup Power Equipment for Your Facility

Whether you run a regional colocation center or a manufacturing plant that runs its ERP in the cloud, your IT uptime depends on the generator in your yard. A software failover doesn’t help if the servers running the backup copies lose power too.

Generator Source buys, sells, rents, and services commercial and industrial generators from 20kW to over 4MW. That covers a 500kW natural gas unit for a smaller site up to a 2,000kW diesel system for a large data center. We own our inventory outright, so units in our yards are ready to ship. We also offer load bank testing and load bank rentals out of our branches in Brighton, Colorado, Jacksonville and Pensacola, Florida, and Austin, Texas. Our service technicians are EGSA certified.

Frequently Asked Questions

Were the November 2026 Microsoft and FAA outages caused by power failures? Neither one was. Microsoft traced its November 12 to 13 Exchange Online outage to an authentication and routing component and found no evidence of malicious activity. The FAA’s November 13 ground stop was blamed on overloaded legacy telecom equipment. Both outages show how hard recovery gets once a large system drops, and that recovery gets harder when servers lose power without a clean shutdown.

What causes most data center outages? Power problems are the leading cause of serious data center outages. Uptime Institute’s 2025 outage analysis found they caused 54% of the impactful outages operators reported. UPS failures were the most common power-related cause, followed by transfer switch failures. The same research found outages tied to IT and network issues are increasing.

How long does a UPS keep servers running when the grid goes down? Most UPS systems are sized to carry the IT load for a few minutes, long enough for the generator to start and the transfer switch to move the load. Runtime depends on how the system was sized and how old the batteries are. Battery capacity drops with age, which is why we test UPS and starting batteries at every preventive maintenance visit.

How often should a commercial generator get a load bank test? We recommend at least once a year for any critical facility. NFPA 110 requires emergency generators to be exercised monthly under load, and calls for an annual load bank test when those monthly runs don’t reach the required load. Data centers and healthcare facilities often test more often based on their own uptime requirements.

What is a retry storm? A retry storm, also called a thundering herd, happens when a large number of apps and devices try to reconnect to a system at the same time after an outage. The first servers back online get overloaded and can fail again. Microsoft saw this during its November 2026 outage, when restored servers went unresponsive once traffic returned.

What size generator does a data center need? It depends on the total load that transfers to the generator, which includes IT equipment, cooling, lighting, and building systems. Sites adding AI or high-density racks should re-check their sizing, because cooling load goes up with the IT load. Our generator size calculator gives you a starting point, and our team can walk through the numbers with you.

Get Your Data Center Backup Power Checked

Don’t wait for the next grid outage to find out whether your transfer works. Contact Generator Source to schedule a load bank test or get your standby power sizing reviewed. You can also browse our in-stock commercial and industrial generators.

Sources