What happens during an Amazon EC2 availability zone outage?

When an Amazon EC2 availability zone experiences a power or networking outage, the instances, storage volumes, and networking components physically located within that zone become unreachable or shut down. Because Amazon Web Services designs each availability zone as an isolated failure domain with independent power, cooling, and physical security, the disruption remains contained to the affected zone while sibling zones in the same region continue operating normally. Automated cloud services such as Elastic Load Balancing, Auto Scaling, and Multi-AZ databases detect the health degradation and shift traffic and workloads to healthy availability zones, though single-zone deployments require manual intervention or restoration of the underlying infrastructure to recover.

How Availability Zone Isolation Works

An AWS Region consists of multiple Availability Zones (AZs) connected through low-latency, high-throughput, and redundant fiber-optic networking. Each AZ comprises one or more discrete data centers housed in separate facilities. To prevent a single physical incident from cascading across an entire region, AZs do not share utility power grids, backup generators, uninterruptible power supplies (UPS), or core networking switches with other zones.

When a power failure occurs, facility-level UPS systems and backup diesel generators immediately engage to maintain operations. An AZ-wide power outage only impacts EC2 workloads if both primary utility power and backup generation systems fail, or if power distribution units (PDUs) experience a fault. Similarly, in a networking outage—such as fiber cuts, router failures, or control-plane congestion—redundant networking paths attempt to reroute traffic automatically before isolation occurs.

Impact on EC2 Instances and Storage

When power or network connectivity is lost within an AZ, the immediate impact on EC2 resources depends on the nature of the failure and the storage types attached to the instances:

  • Instance State and Reachability: During a network outage, EC2 instances may continue running on their physical hosts, but they fail system reachability checks and cannot send or receive traffic. During a power outage, the underlying physical hypervisors power down abruptly, placing the instances into an unreachable or stopped state.
  • Instance Store Volumes: Ephemeral instance store volumes reside physically on the host server. If the host loses power, any data stored on volatile NVMe or SSD instance store volumes is permanently lost, whereas non-volatile volumes may persist only if the instance reboots on the same healthy hardware once power returns.
  • Amazon EBS Volumes: Amazon Elastic Block Store (EBS) volumes are replicated within the same Availability Zone across multiple storage servers. During an AZ outage, EBS volumes in that zone become unavailable and cannot be attached to instances in other AZs directly. Once power and network connectivity return, EBS volumes undergo consistency checks before becoming available for I/O operations again.

Control Plane and API Behavior

During a severe AZ outage, AWS prioritizes the stability of healthy Availability Zones through a design principle known as static stability. The EC2 control plane—responsible for launching, stopping, and modifying instances—isolates the impaired zone to prevent control-plane exhaustion across the region.

As a result, API calls targeting resources in the impaired AZ (such as RunInstances, StopInstances, or CreateSnapshot) may throttle or fail. However, if applications are architected to rely on pre-provisioned capacity or independent control plane endpoints in healthy AZs, they continue running without needing to mutate state in the affected zone.

Automated Recovery by AWS Services

Workloads architected across multiple Availability Zones trigger automated failover mechanisms when an AZ experiences a power or networking disruption:

  1. Elastic Load Balancing (ELB): Application Load Balancers (ALB) and Network Load Balancers (NLB) continuously perform health checks on registered EC2 targets. When targets in the affected AZ fail consecutive health checks, the load balancer stops routing traffic to that zone and distributes incoming requests across healthy AZs. If DNS-level zonal shift or Route 53 health checks are configured, DNS resolution also removes the unhealthy AZ IP addresses.
  2. EC2 Auto Scaling: When an Auto Scaling group spans multiple AZs, it detects that instances in the impaired zone have become unhealthy. Rather than continuously attempting to launch replacement instances in the broken AZ, Auto Scaling rebalances capacity by launching new EC2 instances in the remaining healthy Availability Zones to maintain the desired instance count.
  3. Managed Databases and Stateful Services: Services that utilize EC2 under the hood, such as Amazon RDS configured in a Multi-AZ deployment, detect the loss of the primary instance in the affected zone. RDS automatically promotes the synchronous standby replica in a healthy AZ to become the new primary and updates the database DNS endpoint, typically completing failover within a few minutes.

Calculation of Multi-AZ Availability

Deploying EC2 instances across multiple Availability Zones dramatically reduces the probability of total application downtime. If the availability of a single Availability Zone is represented as \(A_{z}\), then the failure probability of that zone is \(1 - A_{z}\). Assuming independent failure domains across \(n\) Availability Zones, the overall availability \(A_{total}\) of a system capable of running as long as at least one zone remains operational is expressed as:

\[A_{total} = 1 - (1 - A_{z})^{n}\]

For example, if a single zone achieves an availability of \(A_{z} = 0.999\) (99.9%), deploying redundant EC2 capacity across \(n = 3\) Availability Zones yields a theoretical availability of:

\[A_{total} = 1 - (1 - 0.999)^{3} = 1 - (0.001)^{3} = 0.999999999\]

What Happens When the Outage Resolves

Once AWS engineers restore utility power, stabilize backup generators, or repair networking equipment, recovery follows a staged sequence to prevent power surges and network congestion (often called a thundering herd problem):

  • Hardware and Storage Restoration: Physical racks are powered back on in phases, and the internal EBS storage fabric re-establishes quorum and verifies volume replication integrity before allowing compute nodes to mount volumes.
  • Instance Recovery: Instances that remained in a running state on hosts that only lost network connectivity immediately resume network communication. Instances whose physical hosts lost power may either reboot automatically via EC2 Auto Recovery or remain in an impaired/stopped state requiring a manual Stop and Start API call to migrate them to healthy hardware.
  • Auto Scaling Rebalancing: Once the recovered AZ begins passing health checks and accepting RunInstances API calls again, Multi-AZ Auto Scaling groups gradually launch instances in the restored zone and terminate excess instances in other zones to restore an even distribution across all configured Availability Zones.