How Does DynamoDB Manage Automatic Partition Scaling?

Amazon DynamoDB manages automatic partition scaling through a process called partition splitting, where it dynamically divides physical storage units to accommodate growing data volumes and throughput demands without disrupting application performance. As a fully managed NoSQL database, DynamoDB abstracts away underlying server infrastructure by allocating data across physical partitions, which are SSD-backed storage allocations managed behind the scenes.

Understanding DynamoDB Partitions

A partition is an isolated unit of compute and storage that holds a subset of a table's data. DynamoDB uses horizontal partitioning, distributing items across multiple storage nodes based on an internal hash algorithm applied to each item's partition key.

Each individual partition in DynamoDB operates within specific hardware and architectural boundaries:

  • Storage Limit: A single partition can store a maximum of 10 GB of data.
  • Throughput Limits: A single partition can support up to 1,000 Write Capacity Units (WCUs) or 3,000 Read Capacity Units (RCUs) per second.

When a table is first created, DynamoDB provisions an initial set of partitions based on the initial provisioned throughput or the chosen capacity mode.

The Partition Splitting Process

When a partition reaches its maximum threshold for either storage capacity or throughput, DynamoDB automatically initiates a partition split.

The partition splitting process occurs seamlessly through these steps:

  1. Trigger Identification: DynamoDB continuously monitors partition health and resource utilization. A split is triggered when data volume approaches 10 GB or when total read/write traffic exceeds single-partition throughput limits.
  2. Range Division: DynamoDB identifies the hash key range assigned to the original partition and divides that hash range into two equal, non-overlapping segments.
  3. Data Reallocation: The existing partition's data is allocated into two new child partitions based on the updated hash key boundaries.
  4. Metadata Updates: The internal request router updates its global routing table so incoming requests targeting specific partition keys are directed to the correct new child partition.

During a partition split, the original data remains fully accessible, ensuring zero downtime and maintaining sub-10 millisecond latency for ongoing application operations.

Handling Throughput and Heat

Beyond storage-driven partition splits, DynamoDB manages variable workloads using adaptive capacity and heat-based splitting:

  • Adaptive Capacity: If traffic patterns become imbalanced and certain partition keys experience elevated request volume, DynamoDB automatically shifts unused throughput capacity from quiet partitions to high-traffic partitions.
  • Splitting for Heat: If a single partition continues to receive disproportionate traffic despite adaptive capacity adjustments, DynamoDB splits the hot partition to isolate high-frequency items onto separate physical nodes.

By continuously evaluating data growth and access metrics, DynamoDB handles partition expansion entirely behind the scenes, enabling tables to scale horizontally to accommodate virtually unlimited dataset sizes.