How to Benchmark EC2 C-Series CPU Performance?

Benchmarking single-threaded versus multi-threaded performance across Amazon EC2 C-series generations requires isolating processor architectures, standardizing operating system configurations, and executing synthetic and application-level workloads under controlled conditions. By testing instances such as C5, C6i, C6g, C7i, and C7g with tools like Sysbench, Stress-ng, and CoreMark, engineers can quantify IPC improvements, evaluate Simultaneous Multithreading scaling, and determine the optimal price-performance ratio for compute-intensive applications.

Understanding Amazon EC2 C-Series Architecture Differences

Amazon EC2 compute-optimized instances span multiple processor generations and instruction set architectures. Before running benchmarks, you must account for how each family maps virtual CPUs to physical cores, how turbo boost behaves under single-core versus all-core loads, and how cache hierarchies differ.

Understanding vCPU topology is critical when comparing single-threaded and multi-threaded metrics. On Intel-based C5, C6i, and C7i instances, a single vCPU is a hardware thread sharing execution resources with a sibling thread on the same physical core. On AWS Graviton (C6g, C7g) and AMD Genoa (C7a) instances, every vCPU corresponds to an independent physical core.

Standardizing the Benchmarking Environment

To produce repeatable results across generations and architectures, the software stack and kernel parameters must be identical across all test instances.

Operating System and Compiler Parity

Provision all instances with the same Linux distribution and kernel version, such as Amazon Linux 2023 or Ubuntu 24.04 LTS. When compiling benchmarks from source, use the same GCC or Clang version across both x86_64 and aarch64 builds. Use architecture-specific optimization flags (-march=native -O3) so that each C-series generation can utilize its supported vector instructions (such as AVX-512 on C6i/C7i and SVE on C7g).

Instance Sizing and Noisy Neighbor Mitigation

Avoid benchmarking on large or xlarge instance sizes when evaluating generational scaling, as smaller instances share the underlying physical host and last-level cache with other tenants. Use 8xlarge, 12xlarge, 16xlarge, or .metal sizes to minimize noisy neighbor interference and observe NUMA topology effects.

CPU Governor and Topology Verification

Verify the CPU topology and ensure the operating system scaling governor is set to performance where supported:

# Inspect physical cores, sockets, and threads per core
lscpu

# Set CPU frequency governor to performance if cpupower is available
sudo cpupower frequency-set -g performance

Measuring Single-Threaded vs. Multi-Threaded Scaling

To analyze how an instance transitions from single-threaded execution to full multi-threaded saturation, run tests across two distinct thread counts: \(N = 1\) and \(N = \text{Total vCPUs}\). Additionally, on x86 instances with Simultaneous Multithreading, testing at \(N = \frac{\text{Total vCPUs}}{2}\) isolates physical core scaling before hyper-threading contention begins.

The theoretical speedup \(S(N)\) according to Amdahl's Law is defined as:

\[S(N) = \frac{1}{(1 - P) + \frac{P}{N}}\]

Where \(P\) is the parallelizable fraction of the workload and \(N\) is the number of threads. In real-world EC2 benchmarking, we measure the empirical Parallel Scaling Efficiency \(E(N)\) and the Multiprocessing Multiplier \(M\):

\[M = \frac{\text{Score}_{\text{Multi-Threaded}}}{\text{Score}_{\text{Single-Threaded}}}\]

\[E(N) = \frac{M}{N} \times 100\%\]

On Graviton instances (C6g, C7g) and C7a, where each vCPU is a physical core and clock frequencies remain flat under load, \(E(N)\) for compute-bound tasks typically approaches \(95\%\) to \(99\%\). On SMT-enabled instances (C5, C6i, C7i), running a single thread utilizes the full physical core and achieves peak single-core turbo frequency, whereas saturating all vCPUs splits physical core resources between sibling threads and reduces the clock multiplier, often resulting in \(E(N)\) values between \(55\%\) and \(70\%\).

Executing Synthetic CPU Benchmarks

Use a combination of integer, floating-point, cryptographic, and memory-latency benchmarks to capture a complete performance profile.

1. Sysbench CPU Benchmark

Sysbench calculates prime numbers up to a specified value, testing 64-bit integer division and branching performance.

# Install sysbench and numactl
sudo dnf install -y sysbench numactl

# Single-threaded execution pinned to physical core 0
numactl --physcpubind=0 sysbench cpu --cpu-max-prime=50000 --threads=1 --time=60 run

# Half-vCPU execution (Physical core count on x86 SMT instances)
 sysbench cpu --cpu-max-prime=50000 --threads=$(($(nproc) / 2)) --time=60 run

# Full multi-threaded execution across all vCPUs
sysbench cpu --cpu-max-prime=50000 --threads=$(nproc) --time=60 run

Record the events per second metric and the 95th percentile latency from each run.

2. Stress-ng Matrix and CPU Methods

stress-ng provides hundreds of specific CPU stressor methods (bogo ops/s) to test floating-point, vector, and integer units independently.

# Install stress-ng
sudo dnf install -y stress-ng

# Single-threaded 64-bit floating-point matrix product test
taskset -c 0 stress-ng --matrix 1 --matrix-size 256 --timeout 60s --metrics-brief

# Multi-threaded 64-bit floating-point matrix product test across all vCPUs
stress-ng --matrix $(nproc) --matrix-size 256 --timeout 60s --metrics-brief

3. OpenSSL Speed Benchmark

Cryptographic hashing and TLS throughput highlight generational instruction set enhancements (such as VAES on Ice Lake/Sapphire Rapids and crypto extensions on Graviton3).

# Single-threaded SHA256 and AES-256-GCM throughput
taskset -c 0 openssl speed -elapsed -evp aes-256-gcm

# Multi-threaded throughput across all available vCPUs
openssl speed -elapsed -multi $(nproc) -evp aes-256-gcm

Controlling Thread Affinity and Testing Without SMT

When comparing an 8-vCPU C7i (c7i.2xlarge, 4 physical cores with 2 threads each) against an 8-vCPU C7g (c7g.2xlarge, 8 physical cores with 1 thread each), you must test how the x86 instance performs when threads are pinned strictly to physical cores without sibling thread contention.

You can disable SMT dynamically inside the Linux guest OS to benchmark physical core parity:

# Disable SMT dynamically to test physical cores only
echo off | sudo tee /sys/devices/system/cpu/smt/control

# Verify active vCPU count
nproc

# Re-enable SMT after testing
echo on | sudo tee /sys/devices/system/cpu/smt/control

Alternatively, inspect which vCPUs share the same physical core so you can bind multi-threaded workloads cleanly:

# Display vCPU to Core mapping
lscpu -e=CPU,CORE,SOCKET,NODE

On EC2 Intel instances, the first half of the vCPU IDs (\(0\) to \(\frac{N}{2}-1\)) typically map to the primary threads on physical cores, while the second half (\(\frac{N}{2}\) to \(N-1\)) map to the sibling hyper-threads.

Automating Cross-Generation Sweeps

To eliminate cold-start variance and collect statistically significant data across thread counts, use an automated bash harness that steps through thread counts from \(1\) up to $(nproc):

#!/usr/bin/env bash
set -euo pipefail

INSTANCE_TYPE=$(curl -s http://169.254.169.254/latest/meta-data/instance-type || echo "unknown")
MAX_THREADS=$(nproc)
OUTPUT_FILE="benchmark_${INSTANCE_TYPE}.csv"

echo "instance_type,threads,events_per_sec,avg_latency_ms,p95_latency_ms" > "$OUTPUT_FILE"

for THREADS in 1 2 4 $((MAX_THREADS / 2)) $MAX_THREADS; do
    # Deduplicate thread counts if instance has few vCPUs
    if [ "$THREADS" -gt "$MAX_THREADS" ] || [ "$THREADS" -lt 1 ]; then
        continue
    fi

if [ "$THREADS" -eq 1 ]; then
        CMD_PREFIX="taskset -c 0"
    else
        CMD_PREFIX=""
    fi

RESULT=$($CMD_PREFIX sysbench cpu --cpu-max-prime=50000 --threads="$THREADS" --time=30 run)

EPS=$(echo "$RESULT" | awk '/events per second:/ {print $4}')
    AVG_LAT=$(echo "$RESULT" | awk '/avg:/ {print $2}')
    P95_LAT=$(echo "$RESULT" | awk '/95th percentile:/ {print $3}')

echo "${INSTANCE_TYPE},${THREADS},${EPS},${AVG_LAT},${P95_LAT}" >> "$OUTPUT_FILE"
done

Calculating Price-Performance Normalization

Raw performance numbers across C-series generations must be normalized against instance hourly on-demand or savings plan pricing to guide architectural decisions.

Compute two key metrics for each instance type:

\[\text{Single-Thread Cost Efficiency} = \frac{\text{Score}_{\text{1 Thread}}}{\text{Instance Hourly Price (\$)}}\]

\[\text{Multi-Thread Throughput per Dollar} = \frac{\text{Score}_{\text{All vCPUs}}}{\text{Instance Hourly Price (\$)}}\]

When evaluating workloads bound by single-thread latency—such as Redis, single-threaded Node.js processes, or sequential build steps—compare the single-thread throughput along with the per-vCPU hourly rate. When evaluating horizontally or vertically scalable workloads—such as video encoding, high-concurrency web servers, or batch analytics—use the Multi-Thread Throughput per Dollar metric at 80% to 100% CPU utilization to identify the most cost-effective C-series generation.