Does Amazon EC2 Support Fractional GPU and MIG Partitioning?

Yes, Amazon EC2 supports both hardware-based Multi-Instance GPU partitioning and software-based fractional GPU sharing across select accelerated computing instances. Through NVIDIA Multi-Instance GPU technology on A100, H100, and H200 Tensor Core GPUs, users can partition a single physical GPU into up to seven isolated instances with dedicated compute and memory resources. Additionally, AWS provides native fractional GPU allocation through Amazon ECS, Amazon EKS, and AWS Batch using time-slicing, Multi-Process Service, and G6f fractional GPU instance types.

Hardware-Based Partitioning with NVIDIA MIG on EC2

NVIDIA Multi-Instance GPU (MIG) expands the utilization of accelerated EC2 instances by dividing a single physical GPU into multiple independent GPU instances. Unlike software-level sharing, MIG partitions the hardware at the silicon level. Each MIG instance receives its own dedicated Streaming Multiprocessors (SMs), high-bandwidth memory (HBM), memory controllers, and cache crossbars. This hardware isolation guarantees predictable latency, consistent throughput, and fault isolation between concurrent workloads.

On Amazon EC2, MIG is supported on the following instance families:

• P4d and P4de instances: Equipped with NVIDIA A100 Tensor Core GPUs (40 GB or 80 GB HBM2/HBM2e).
• P5 instances: Equipped with NVIDIA H100 Tensor Core GPUs (80 GB HBM3).
• P5e and P5en instances: Equipped with NVIDIA H200 Tensor Core GPUs (141 GB HBM3e).

Mathematical Resource Allocation in MIG

A physical NVIDIA A100, H100, or H200 GPU is divided into seven compute slices and eight memory slices. When partitioning a GPU into \(N\) isolated instances (where \(1 \le N \le 7\)), the memory allocated to a specific slice \(M_i\) and the compute capacity \(C_i\) can be approximated by:

\[M_i \approx \frac{k}{8} \times M_{\text{total}}, \quad C_i = \frac{c}{7} \times C_{\text{total}}\]

where \(k \in \{1, 2, 4, 8\}\) represents the number of memory slices assigned to the profile, and \(c \in \{1, 2, 3, 4, 7\}\) represents the number of compute slices.

For example, on an EC2 p4d.24xlarge instance with 40 GB A100 GPUs, common MIG profiles include:

• 1g.5gb: 1/7th of the compute SMs and 5 GB of memory (up to 7 instances per GPU).
• 2g.10gb: 2/7ths of the compute SMs and 10 GB of memory (up to 3 instances per GPU).
• 3g.20gb: 3/7ths of the compute SMs and 20 GB of memory (up to 2 instances per GPU).
• 7g.40gb: The entire compute and memory capacity of the GPU (1 instance per GPU).

Configuring MIG on an EC2 Instance

To enable and configure MIG on a supported Amazon EC2 Deep Learning AMI or Ubuntu/Amazon Linux instance with the NVIDIA driver installed, use the nvidia-smi utility:

# Enable MIG mode on GPU 0 (requires root privileges)
sudo nvidia-smi -i 0 -mig 1

# Reset the GPU or reboot the instance if active processes block the state change
sudo nvidia-smi --gpu-reset -i 0

# List available GPU Instance (GI) profiles
nvidia-smi mig -lgip

# Create three 2g.10gb profiles (Profile ID 14 on A100 40GB) and immediately create Compute Instances (CI)
sudo nvidia-smi mig -i 0 -cgi 14,14,14 -C

# Verify the created MIG devices
nvidia-smi -L

Native Fractional GPU Instances (EC2 G6f)

For workloads that do not require an entire GPU and do not run on A100 or H100 architectures, Amazon EC2 offers G6f instances powered by NVIDIA L4 Tensor Core GPUs. Unlike standard instances that attach one or more full physical GPUs, G6f instances expose a hypervisor-managed fraction of a physical NVIDIA L4 GPU (1/8, 1/4, or 1/2 of the 24 GB GPU memory and compute capacity).

• g6f.large: 1/8th of an L4 GPU (3 GB GPU memory)
• g6f.xlarge: 1/4th of an L4 GPU (6 GB GPU memory)
• g6f.2xlarge: 1/2 of an L4 GPU (12 GB GPU memory)

These instances allow graphics-intensive applications, small-scale machine learning inference, and remote workstations to consume fractional GPU resources directly at the EC2 instance level without manual driver partitioning.

Software-Based Fractional GPU Sharing on EC2

On EC2 instances featuring GPUs that do not support hardware MIG—such as G4dn (NVIDIA T4), G5 (NVIDIA A10G), and standard G6 (NVIDIA L4)—fractional GPU sharing is achieved through software concurrency mechanisms.

1. GPU Time-Slicing

Time-slicing allows multiple containers or processes to share a single GPU by rapidly switching execution contexts. When configured via the NVIDIA Container Toolkit and the Kubernetes Device Plugin on Amazon EKS, a single physical GPU can be advertised as \(R\) virtual GPU replicas. If \(R = 4\), the Kubernetes scheduler treats one physical GPU on a g5.xlarge node as four allocatable nvidia.com/gpu resources.

Time-slicing does not enforce memory isolation or fault isolation. If one container allocates more VRAM than available or triggers an out-of-memory (OOM) exception, co-located processes on the same GPU can be impacted.

2. NVIDIA Multi-Process Service (MPS)

NVIDIA MPS provides spatial sharing of compute resources on a single GPU. Instead of switching between processes sequentially in time slices, MPS funnels CUDA kernels from multiple processes into a single hardware context. This allows concurrent execution of kernels on separate Streaming Multiprocessors when a single workload cannot saturate the entire GPU. MPS also supports setting thread execution limits and pinned device memory limits per client process.

Container Orchestration Support on AWS

When running containerized workloads on EC2 instances, AWS container orchestration services provide direct integration for fractional GPUs and MIG:

• Amazon EKS: Supports both MIG (using single or mixed strategies) and time-slicing via the NVIDIA GPU Operator or the NVIDIA k8s-device-plugin. Administrators define a ConfigMap specifying the number of replicas per GPU or the desired MIG profile layout, allowing pods to request fractional slices via standard resources.limits["nvidia.com/gpu"] or profile-specific keys like nvidia.com/mig-1g.5gb.
• Amazon ECS: Supports fractional GPU sharing when configuring custom MIG instances or pinning multiple containers to shared GPU UUIDs within task definitions.

Comparison of GPU Sharing Methods on EC2