How do you group EC2 Capacity Reservations for clusters?

Grouping multiple Amazon EC2 Capacity Reservations using AWS Resource Groups allows you to logically manage and target reserved compute capacity across different instance types and Availability Zones for cluster deployments. By combining individual Capacity Reservations into a single resource group, you can configure Auto Scaling groups, EC2 Fleet, or standalone instance launches to consume reserved capacity from the entire group automatically. This article covers the prerequisites, step-by-step AWS CLI commands to create and populate a Capacity Reservation group, and methods for launching cluster instances into the grouped capacity.

Prerequisites and Limitations

Before creating a Capacity Reservation resource group for cluster workloads, ensure your reservations and target deployments meet the following requirements:

  • AWS Resource Groups Integration: Capacity Reservation groups are built using the AWS Resource Groups service with a specific configuration type (AWS::EC2::CapacityReservation).
  • Eligibility Rules: You can add both open and targeted Capacity Reservations to a group. However, instances launched into the group must match the attributes (instance type, platform, tenancy, and Availability Zone) of at least one active reservation with available capacity in that group.
  • Cluster Placement Groups: If your cluster requires low-latency networking via a Cluster Placement Group, the individual Capacity Reservations can be associated with placement groups, or you can specify the placement group at launch time alongside the Capacity Reservation resource group ARN.
  • IAM Permissions: You need permissions for both EC2 (ec2:CreateCapacityReservation, ec2:RunInstances) and AWS Resource Groups (resource-groups:CreateGroup, resource-groups:PutGroupConfiguration, resource-groups:GroupResources).

Step 1: Create Individual EC2 Capacity Reservations

First, create the individual Capacity Reservations that will make up your cluster's compute pool. For cluster deployments that only allow specific workloads to consume the reserved capacity, set the --instance-match-criteria to targeted.

aws ec2 create-capacity-reservation \
    --instance-type c6i.4xlarge \
    --instance-platform Linux/UNIX \
    --availability-zone us-east-1a \
    --instance-count 10 \
    --instance-match-criteria targeted \
    --tag-specifications 'ResourceType=capacity-reservation,Tags=[{Key=Cluster,Value=HPC-Production}]'

Create any additional Capacity Reservations needed for your cluster (such as different instance types or secondary Availability Zones) and note their Amazon Resource Names (ARNs), which follow the format arn:aws:ec2:region:account-id:capacity-reservation/cr-xxxxxxxxxxxxxxxxx.

Step 2: Create a Capacity Reservation Resource Group

Use the AWS Resource Groups API (aws resource-groups create-group) to create a logical group dedicated to EC2 Capacity Reservations. You must include the --configuration parameter specifying AWS::EC2::CapacityReservation as the group configuration type and AWS::EC2::CapacityReservation in the AllowedResourceTypes parameter of AWS::ResourceGroups::Generic.

aws resource-groups create-group \
    --name MyClusterCapacityGroup \
    --description "Capacity Reservations for HPC cluster deployment" \
    --configuration '[
        {
            "Type": "AWS::EC2::CapacityReservation"
        },
        {
            "Type": "AWS::ResourceGroups::Generic",
            "Parameters": [
                {
                    "Name": "allowed-resource-types",
                    "Values": ["AWS::EC2::CapacityReservation"]
                }
            ]
        }
    ]'

Save the GroupArn returned in the command output, which looks like arn:aws:resource-groups:us-east-1:123456789012:group/MyClusterCapacityGroup.

Step 3: Add Capacity Reservations to the Resource Group

Once the group is created, register your individual Capacity Reservation ARNs into the resource group using the group-resources command:

aws resource-groups group-resources \
    --group MyClusterCapacityGroup \
    --resource-arns \
        arn:aws:ec2:us-east-1:123456789012:capacity-reservation/cr-0123456789abcdef0 \
        arn:aws:ec2:us-east-1:123456789012:capacity-reservation/cr-0fedcba9876543210

If the total number of reservations is \(N\) and each reservation \(i\) provides \(c_i\) instances, the total reserved cluster capacity \(C_{\text{total}}\) managed by the resource group is:

\[C_{\text{total}} = \sum_{i=1}^{N} c_i\]

To verify which Capacity Reservations are currently active inside your group, run:

aws resource-groups list-group-resources \
    --group MyClusterCapacityGroup

Step 4: Target the Resource Group in Cluster Deployments

With the Capacity Reservations grouped, you can direct cluster nodes to consume capacity from the group by referencing the resource group ARN in your launch configurations.

Launching Standalone Cluster Instances

When launching instances directly with run-instances, pass the CapacityReservationResourceGroupArn inside the --capacity-reservation-specification parameter. You can also combine this with a Cluster Placement Group for tightly coupled node-to-node communication:

aws ec2 run-instances \
    --image-id ami-0abcdef1234567890 \
    --instance-type c6i.4xlarge \
    --count 10 \
    --placement GroupName=MyHPCClusterPlacementGroup \
    --capacity-reservation-specification \
        CapacityReservationTarget={CapacityReservationResourceGroupArn=arn:aws:resource-groups:us-east-1:123456789012:group/MyClusterCapacityGroup}

Using Launch Templates with EC2 Fleet or Auto Scaling Groups

For automated cluster scaling across multiple instance types or Availability Zones, define the Capacity Reservation resource group target inside an EC2 Launch Template:

aws ec2 create-launch-template \
    --launch-template-name ClusterNodeTemplate \
    --version-description "Targets MyClusterCapacityGroup" \
    --launch-template-data '{
        "ImageId": "ami-0abcdef1234567890",
        "CapacityReservationSpecification": {
            "CapacityReservationTarget": {
                "CapacityReservationResourceGroupArn": "arn:aws:resource-groups:us-east-1:123456789012:group/MyClusterCapacityGroup"
            }
        }
    }'

When an Auto Scaling group or EC2 Fleet uses this Launch Template, EC2 automatically matches the requested instance attributes against the reservations in MyClusterCapacityGroup and fulfills the launch from the appropriate reservation in the group.

Removing Reservations from a Group

As your cluster requirements evolve, you can remove individual Capacity Reservations from the group without modifying your Launch Templates or cluster orchestration scripts:

aws resource-groups ungroup-resources \
    --group MyClusterCapacityGroup \
    --resource-arns arn:aws:ec2:us-east-1:123456789012:capacity-reservation/cr-0fedcba9876543210