How to Set the Referer Header in Curl

This article explains how the curl command-line tool handles the HTTP Referer header. It covers curl’s default behavior, demonstrates how to manually set a custom referrer using specific command-line flags, and explains how to configure curl to automatically update the referrer when following redirects.

Default Curl Behavior

By default, curl does not send a Referer header with its HTTP requests. If the destination server requires a referrer for security, tracking, or access control, the request may be blocked or rejected unless you explicitly configure curl to include one.

Setting the Referer Header Manually

There are two primary ways to manually set the Referer header in a curl request.

Method 1: Using the -e or --referer Option

The most direct way to specify a referrer is by using the -e (or --referer) option followed by the URL you want to send.

curl -e "https://example.com/source-page" https://httpbin.org/headers

Method 2: Using the -H or --header Option

Alternatively, you can manually construct the HTTP header using the -H (or --header) option. This is useful if you are already passing multiple headers and want to keep your command structure consistent.

curl -H "Referer: https://example.com/source-page" https://httpbin.org/headers

Both methods achieve the exact same result on the receiving server.

Automatically Handling Redirects with Referers

When scraping websites or testing APIs, you often need to follow redirects using curl’s -L (or --location) flag. If you want curl to automatically update the Referer header to match the URL of the redirecting page, you can use the ;auto suffix.

Setting a Start URL and Enabling Auto-Referer

You can provide an initial referrer and tell curl to automatically update it for subsequent redirects by appending ;auto to the end of your referrer string:

curl -L -e "https://example.com/start-page;auto" https://example.com/redirect-target

Pure Auto-Referer

If you do not want to define an initial referrer but want curl to automatically handle the referrer header during redirects, use ;auto on its own:

curl -L -e ";auto" https://example.com/redirect-target

In this scenario, the first request will contain no Referer header, but any subsequent redirected requests will automatically include the URL of the previous page as the referrer.