Curl Retry Options and Handling Transient Errors

When transferring data over a network using curl, temporary network glitches, rate limits, or brief server outages can cause requests to fail. This article explains how to use curl’s built-in retry options to automatically reattempt failed requests, details how curl defines and handles transient errors, and provides practical examples of the command-line flags you can use to make your scripts more resilient.

Understanding Curl’s Retry Mechanism

The primary option for retrying failed requests in curl is the --retry <num> flag. When you specify this option followed by a number, curl will attempt to re-send the request up to that many times if it encounters what it considers a “transient” error.

By default, curl does not retry indefinitely or on every type of failure. It uses an exponential backoff algorithm between attempts. It starts by waiting one second before the first retry, then doubles the wait time for each subsequent attempt (2 seconds, 4 seconds, 8 seconds, and so on) up to a maximum limit of 10 seconds per interval.

What Curl Considers a Transient Error

Curl is selective about which failures warrant a retry. It will only trigger a retry if it detects a transient (temporary) error, which includes:

By default, curl will not retry on client-side errors (like HTTP 404 Not Found or 403 Forbidden) because these are permanent errors that a retry will not resolve. It also does not retry on a “Connection Refused” error by default, as this usually indicates the target server is down or the port is closed.

Key Curl Retry Flags

To customize how curl handles retries, you can combine the standard retry flag with several modifier options:

1. --retry <number>

This defines the maximum number of retry attempts.

curl --retry 5 https://example.com/api

2. --retry-delay <seconds>

If you want to disable the default exponential backoff and enforce a strict, consistent wait time between attempts, use this flag.

curl --retry 3 --retry-delay 5 https://example.com/api

This command waits exactly 5 seconds between each of the 3 retry attempts.

3. --retry-max-time <seconds>

This limits the total time allowed for all retry attempts combined. If the cumulative retry time exceeds this limit, curl will stop retrying, even if it has not reached the maximum retry count.

curl --retry 10 --retry-max-time 30 https://example.com/api

4. --retry-connrefused

By default, curl considers a refused connection to be a hard failure. Adding this flag tells curl to treat a connection refusal as a transient error, making it highly useful when waiting for a local service or container to boot up.

curl --retry 5 --retry-connrefused https://localhost:8080

5. Handling HTTP 429 (Too Many Requests)

While curl does not automatically retry standard 4xx client errors, some servers send a Retry-After header with a 429 (Too Many Requests) response. To respect this header and retry accordingly, you must combine --retry with the --fail (or -f) flag. The --fail flag forces curl to treat HTTP errors as document failures, allowing the retry logic to take effect.