How to Send Custom HTTP Headers With Wget?

You can easily send a custom HTTP header with a wget request by using the --header option followed by your specific header name and value enclosed in quotes. This capability is essential for simulating specific browser behaviors, passing authentication tokens, testing API endpoints, or bypassing basic server-side restrictions. This article will guide you through the exact syntax for adding single or multiple headers, practical real-world examples, and common use cases for web scraping and development.

The Basic Syntax for Custom Headers

The standard syntax for including a custom header in your wget command requires the --header flag. The format looks like this:

wget --header="Header-Name: Header-Value" URL

For example, if you want to send a custom tracking header called X-My-Header with a value of 12345 to a test server, you would run the following command in your terminal:

wget --header="X-My-Header: 12345" http://example.com

Sending Multiple HTTP Headers

In many scenarios, a single header is not enough. You might need to send an authorization token while simultaneously specifying the content type. To send multiple custom headers, you simply repeat the --header option for each individual header you want to include.

wget --header="Authorization: Bearer xyz123" --header="Accept: application/json" http://example.com/api

Common Real-World Use Cases

Modifying headers allows you to interact with web servers more effectively. Here are three of the most frequent scenarios where custom headers are required:

• Mimicking a Specific Browser (User-Agent): Some websites block default wget requests to prevent automated scraping. You can spoof a standard web browser by modifying the User-Agent header.

wget --header="User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64)" http://example.com

• Setting a Cookie Header: If you need to access a page that requires a session cookie without managing a cookie file, you can pass it directly via the headers.

wget --header="Cookie: session_id=abcde98765" http://example.com/dashboard

• Specifying Host Headers: When testing a website before DNS changes have propagated, you can point wget to an IP address but specify the intended domain using the Host header.

wget --header="Host: www.yourdomain.com" http://192.0.2.1/index.html

Using the Shorthand Alternative

If you prefer shorter commands, wget also supports a single-dash lowercase alias for the header flag, which is -e combined with the http_proxy or custom header definitions, but the official single-character shorthand specifically for headers is -O for output and --header for headers. However, if you are looking for the shortest native flag, wget relies primarily on --header. If you find yourself typing long strings frequently, you can also add permanent headers to your local ~/.wgetrc configuration file using the header = syntax.