Can EC2 Spot and On-Demand Run in One Auto Scaling Group?

Yes, Amazon EC2 Spot Instances can be combined with On-Demand Instances inside a single Auto Scaling group using a Mixed Instances Policy. This article explores how AWS Auto Scaling manages both purchase options simultaneously, the configuration steps required via EC2 Launch Templates, how allocation strategies optimize cost and availability, and the mathematical models behind mixed fleet pricing.

Understanding Mixed Instances Policies in Amazon EC2

Historically, running both On-Demand and Spot capacity required maintaining multiple Auto Scaling groups and synchronizing their scaling policies. Amazon EC2 Auto Scaling simplifies this architecture by allowing a single Auto Scaling group (ASG) to provision instances across multiple purchase options and instance types. This capability is driven by a Mixed Instances Policy paired with an EC2 Launch Template.

By combining On-Demand and Spot Instances in the same group, you can establish a reliable baseline of constant capacity using On-Demand Instances (which may also be covered by Savings Plans or Reserved Instances) while scaling bursts or fault-tolerant workloads on cost-effective Spot capacity.

Key Configuration Components

When configuring a mixed-instance Auto Scaling group, three primary components govern how instances are launched and maintained:

1. EC2 Launch Template

Unlike legacy Launch Configurations, an EC2 Launch Template is required to use a Mixed Instances Policy. The Launch Template defines the baseline instance configuration, including the Amazon Machine Image (AMI), security groups, IAM instance profile, user data, and default instance type.

2. Instance Type Overrides

Within the Auto Scaling group configuration, you can specify multiple instance types that override the default type defined in the Launch Template. Providing a diverse list of compatible instance types (for example, m6i.large, m5.large, c6i.large, and r6i.large) increases the number of Spot capacity pools available to the Auto Scaling group, reducing the likelihood of Spot capacity shortages.

For workloads that require varied instance sizes, you can assign weighted capacities. For example, if your group scales based on vCPU count, an m6i.large (2 vCPUs) can be assigned a weight of \(2\), while an m6i.xlarge (4 vCPUs) can be assigned a weight of \(4\).

3. Purchase Option Split (Instances Distribution)

The InstancesDistribution block defines how capacity is divided between On-Demand and Spot Instances using two critical parameters:

  • On-Demand Base Capacity (OnDemandBaseCapacity): The minimum amount of the Auto Scaling group's capacity that must be fulfilled by On-Demand Instances. As the group launches instances, it provisions On-Demand capacity first until this base threshold is met.
  • On-Demand Percentage Above Base Capacity (OnDemandPercentageAboveBaseCapacity): For any additional capacity required beyond the base capacity, this percentage dictates the split between On-Demand and Spot Instances. Setting this to \(0\) means all capacity above the base will be Spot Instances; setting it to \(20\) means \(20\%\) of the scaling capacity above the base will be On-Demand and \(80\%\) will be Spot.

Mathematical Breakdown of Capacity Distribution

To calculate the exact number of On-Demand and Spot units launched for a given total desired capacity \(C_{\text{total}}\), let \(B\) represent the On-Demand base capacity and \(p\) represent the On-Demand percentage above base expressed as a decimal (\(0 \le p \le 1\)).

First, the capacity allocated to the base is:

\[C_{\text{base}} = \min(C_{\text{total}}, B)\]

Next, the remaining capacity above the base is:

\[C_{\text{above}} = \max(0, C_{\text{total}} - B)\]

The total On-Demand capacity \(C_{\text{OD}}\) and total Spot capacity \(C_{\text{Spot}}\) are then determined by:

\[C_{\text{OD}} = C_{\text{base}} + \lceil p \cdot C_{\text{above}} \rceil\]

\[C_{\text{Spot}} = C_{\text{total}} - C_{\text{OD}}\]

Example Calculation

Suppose an Auto Scaling group has a desired capacity of \(C_{\text{total}} = 24\) instances, an On-Demand base capacity of \(B = 4\), and an On-Demand percentage above base of \(25\%\) (\(p = 0.25\)):

\[C_{\text{base}} = \min(24, 4) = 4\]

\[C_{\text{above}} = 24 - 4 = 20\]

\[C_{\text{OD}} = 4 + (0.25 \times 20) = 4 + 5 = 9 \text{ On-Demand Instances}\]

\[C_{\text{Spot}} = 24 - 9 = 15 \text{ Spot Instances}\]

If the average hourly On-Demand price is \(P_{\text{OD}}\) and the average Spot discount is \(d\) (where \(0 < d < 1\)), the effective hourly cost \(E_{\text{cost}}\) of the fleet is:

\[E_{\text{cost}} = C_{\text{OD}} \cdot P_{\text{OD}} + C_{\text{Spot}} \cdot P_{\text{OD}} \cdot (1 - d)\]

Allocation Strategies for On-Demand and Spot

Auto Scaling uses separate allocation strategies to decide which instance types to launch from your overrides list for each purchase option.

Spot Allocation Strategies

  • Price Capacity Optimized (price-capacity-optimized): Recommended for most workloads. AWS identifies the Spot capacity pools with the highest availability and then selects from the lowest-priced pools within that subset. This balances low interruption rates with deep cost savings.
  • Capacity Optimized (capacity-optimized): Launches instances from the most available Spot capacity pools based on real-time capacity data, prioritizing low interruption risk over raw price.
  • Lowest Price (lowest-price): Diversifies Spot Instances across the N lowest-priced pools per Availability Zone. While it targets the lowest immediate cost, it can result in higher interruption rates if the cheapest pools have limited capacity.

On-Demand Allocation Strategies

  • Prioritized (prioritized): Uses the exact order of instance types listed in your overrides, starting with the first type and moving down the list only if capacity is unavailable in an Availability Zone.
  • Lowest Price (lowest-price): Automatically selects the lowest-priced On-Demand instance type available in each Availability Zone.

Example AWS CLI Configuration

Below is an example JSON structure used with the aws autoscaling create-auto-scaling-group command to configure a Mixed Instances Policy:

{
  "AutoScalingGroupName": "production-mixed-asg",
  "MixedInstancesPolicy": {
    "LaunchTemplate": {
      "LaunchTemplateSpecification": {
        "LaunchTemplateName": "web-server-template",
        "Version": "$Latest"
      },
      "Overrides": [
        { "InstanceType": "c6i.large" },
        { "InstanceType": "c5.large" },
        { "InstanceType": "m6i.large" },
        { "InstanceType": "m5.large" }
      ]
    },
    "InstancesDistribution": {
      "OnDemandAllocationStrategy": "prioritized",
      "OnDemandBaseCapacity": 4,
      "OnDemandPercentageAboveBaseCapacity": 20,
      "SpotAllocationStrategy": "price-capacity-optimized"
    }
  },
  "MinSize": 4,
  "MaxSize": 50,
  "DesiredCapacity": 14,
  "VPCZoneIdentifier": "subnet-0123456789abcdef0,subnet-0fedcba9876543210"
}

Handling Spot Interruptions with Capacity Rebalancing

When running Spot Instances inside an Auto Scaling group, EC2 emits two types of notifications before reclaiming capacity:

  1. EC2 Instance Rebalance Recommendation: A proactive signal sent when a Spot Instance is at an elevated risk of interruption.
  2. Two-Minute Spot Interruption Notice: The final warning before the instance is terminated or stopped.

By enabling Capacity Rebalancing on the Auto Scaling group, AWS automatically responds to Rebalance Recommendations by launching a replacement Spot Instance from an available pool before terminating the at-risk instance. Once the new instance passes health checks (or after termination lifecycle hooks complete), the old instance is gracefully drained and terminated.

Best Practices for Mixed Auto Scaling Groups

  • Select Multiple Instance Families and Generations: Include at least 4 to 10 compatible instance types across different families (such as m6i, m5, c6i, c6a) and Availability Zones to maximize Spot pool diversity.
  • Use Attribute-Based Instance Type Selection: Instead of manually listing dozens of instance types, specify vCPU and memory ranges in the Mixed Instances Policy so EC2 automatically selects matching instance types, including newer generations as they are released.
  • Align Base Capacity with Commitments: Set OnDemandBaseCapacity to match your active Reserved Instances or Compute Savings Plans to ensure you never waste pre-committed spend.
  • Configure Elastic Load Balancing Connection Draining: Combine Capacity Rebalancing with Deregistration Delay (Connection Draining) on your Application or Network Load Balancer so in-flight requests complete before an interrupted Spot Instance terminates.