How to Compare EC2 Graviton2, Graviton3, and Graviton4?

Evaluating performance gains across AWS Graviton2, Graviton3, and Graviton4 instances in Amazon EC2 requires a structured benchmarking methodology that combines micro-architectural metrics, synthetic workloads, and production-grade application testing. As AWS evolves its custom Arm-based silicon from the Neoverse N1 in Graviton2 to the Neoverse V1 in Graviton3 and the Neoverse V2 in Graviton4, organizations must look beyond raw clock speeds and measure real-world throughput, tail latency, memory bandwidth, and price-performance efficiency to justify migration efforts.

Architectural Evolution Across Graviton Generations

Before executing benchmarks, it is essential to understand the hardware changes driving performance differences across each generation. Unlike traditional x86 processors that often utilize Simultaneous Multithreading (SMT), every vCPU on a Graviton instance is a physical core. This design eliminates thread contention between sibling hyperthreads and yields more predictable scaling.

The jump from Graviton2 to Graviton3 introduced DDR5 memory, offering 50% more memory bandwidth, alongside Scalable Vector Extensions (SVE) and BFloat16 support for machine learning inference. Moving from Graviton3 to Graviton4 doubles the L2 cache per core to 2 MB, transitions to the Armv9 architecture with SVE2, increases memory speed to DDR5-5600, and expands maximum instance sizes up to 192 vCPUs in dual-socket configurations.

Step 1: Standardize the Software and Compiler Environment

Hardware improvements in Graviton3 and Graviton4 rely heavily on modern Linux kernels, updated system libraries, and compiler flags that exploit newer instruction sets. Testing an older binary compiled for generic Arm64 (armv8-a) will fail to utilize features like Large System Extensions (LSE), SVE, or SVE2, skewing your benchmark results.

To ensure a fair evaluation across m6g (Graviton2), m7g (Graviton3), and m8g (Graviton4) instances:

• Operating System: Use Linux kernel 6.1 or later (such as Amazon Linux 2023 or Ubuntu 24.04 LTS) to ensure optimal scheduler awareness, page sizing, and hardware perf counter support.
• C/C++ and Rust Compilers: Use GCC 12+ or Clang/LLVM 15+ and compile with -mcpu=native on each target instance, or target specific architectures explicitly (-mcpu=neoverse-n1, -mcpu=neoverse-v1, -mcpu=neoverse-v2).
• Runtime Versions: Use OpenJDK 21+ for Java workloads, Go 1.21+ for Go services, Python 3.11+ for interpreted workloads, and Node.js 20+ for JavaScript applications. Newer runtimes contain intrinsic optimizations for Armv8.4 and Armv9 instructions.

Step 2: Measure Micro-Architectural Subsystem Gains

Before testing full applications, run targeted synthetic benchmarks to isolate how CPU compute, memory bandwidth, and cache hierarchies behave across the three generations.

Memory Bandwidth and Latency

Database engines, in-memory caches (like Redis or Valkey), and analytics workloads are frequently bottlenecked by memory bandwidth rather than CPU cycles. Because Graviton3 moved to DDR5 and Graviton4 doubled the L2 cache while increasing DDR5 clock rates, you should quantify memory throughput using the STREAM benchmark or sysbench:

sysbench memory --memory-block-size=1M --memory-total-size=100G --threads=$(nproc) run

Compare the transferred megabytes per second across identically sized instances (for example, r6g.2xlarge, r7g.2xlarge, and r8g.2xlarge). Workloads with large working sets will exhibit a steep reduction in LLC (Last Level Cache) misses on Graviton4 due to the 2 MB per-core L2 cache.

Compute, Cryptography, and Compression

Graviton3 and Graviton4 feature enhanced cryptographic accelerators and wider vector units. You can evaluate raw cryptographic throughput using OpenSSL speed tests:

openssl speed -elapsed -evp aes-256-gcm
openssl speed -elapsed rsa2048

For compression-heavy workloads (such as Kafka brokers, log ingestion pipelines, or object storage gateways), benchmark zstd or zlib-ng compression and decompression speeds across all three instance families.

Step 3: Conduct Application-Level Load Testing

Synthetic benchmarks establish theoretical ceilings, but production decisions require testing with representative application traffic. Set up an isolated load generation environment where the load generator runs on a separate EC2 instance within the same Availability Zone and Placement Group to minimize network jitter.

Key Application Metrics to Capture

1. Maximum Sustainable Throughput: Measure Requests Per Second (RPS) or Transactions Per Second (TPS) before error rates exceed acceptable thresholds.
2. Tail Latency under Load: Compare \(P_{50}\), \(P_{95}\), and \(P_{99}\) latencies at fixed load intervals (for example, at 50% and 75% CPU utilization). Because Graviton cores do not share execution pipelines via hyperthreading, latency degradation follows a smoother curve up to high CPU utilization.
3. Throughput at Target SLA: Determine the highest RPS an instance can sustain while keeping \(P_{99}\) latency below a strict service-level objective (e.g., \(P_{99} < 50\text{ ms}\)).

Calculate the relative performance gain \(G_{\text{perf}}\) between generations using the following formula:

\[G_{\text{perf}} = \left( \frac{\text{Throughput}_{\text{target}} - \text{Throughput}_{\text{baseline}}}{\text{Throughput}_{\text{baseline}}} \right) \times 100\%\]

Step 4: Profile Hardware Counters with Linux Perf

To understand why a workload runs faster on Graviton3 or Graviton4, use the Linux perf tool to inspect hardware performance monitoring units (PMUs). Ensure your test runs on enough vCPUs (or dedicated metal instances) where hardware counters are exposed, and capture Instructions Per Cycle (IPC), branch mispredictions, and cache misses:

perf stat -e cycles,instructions,cache-references,cache-misses,L1-dcache-load-misses,L2D_CACHE_REFILL -p <PID> -- sleep 30

When comparing output across generations, calculate the Instructions Per Cycle:

\[\text{IPC} = \frac{\text{Instructions}}{\text{Cycles}}\]

A higher IPC on Graviton4 compared to Graviton3 and Graviton2 indicates that the wider instruction decode front-end, improved branch predictor, and 2 MB L2 cache are keeping the execution units fed without stalling on main memory fetches.

Step 5: Evaluate Price-Performance Efficiency

Newer Graviton generations carry a slight on-demand hourly price premium over their predecessors, but their higher throughput per vCPU typically results in a lower total cost to serve a given workload.

To evaluate the true economic benefit, normalize your performance metrics against the hourly instance cost. Calculate the Cost Per Million Requests (\(C_{\text{req}}\)) for each instance family:

\[C_{\text{req}} = \frac{\text{Hourly Instance Price (USD)}}{\text{RPS}_{\text{SLA}} \times 3600} \times 10^6\]

Alternatively, compute the Price-Performance Ratio (\(R_{\text{pp}}\)) relative to Graviton2:

\[R_{\text{pp}} = \left( \frac{\text{Throughput}_{\text{New}}}{\text{Cost}_{\text{New}}} \right) \div \left( \frac{\text{Throughput}_{\text{Baseline}}}{\text{Cost}_{\text{Baseline}}} \right)\]

If \(R_{\text{pp}} > 1.0\), migrating to the newer Graviton generation reduces your overall EC2 spend because you can either scale down the instance size (for example, moving from an m6g.4xlarge to an m8g.2xlarge) or reduce the total number of instances in your Auto Scaling group while maintaining the same cluster capacity.