How to Save Cookies with Wget?

This article provides a quick overview and practical guide on how to instruct the wget command-line utility to capture and store cookies sent by a server. You will learn the specific command-line flags required to write cookies to a local file, how to load them for subsequent requests, and see real-world examples of handling session data during web scraping or automation tasks.

The --save-cookies Flag

To instruct wget to save cookies received from a server, you need to use the --save-cookies option followed by the path to the file where you want to store them. By default, wget does not save cookies that are meant to expire at the end of the session (session cookies). To force wget to save these session cookies as well, you must pair it with the --keep-session-cookies flag.

Here is the standard command structure:

wget --save-cookies cookies.txt --keep-session-cookies https://example.com

Key Components Explained

• --save-cookies <filename>: This tells wget to look at the Set-Cookie headers sent by the server and write them into the specified file using the standard Netscape cookie format.
• --keep-session-cookies: This ensures that temporary session cookies—which are vital for staying logged into most modern websites—are not discarded when the command finishes execution.

Reusing Saved Cookies

Once you have saved the cookies to a file, you often need to send them back to the server in a follow-up request (for example, accessing a protected page after hitting a login form). To do this, use the --load-cookies flag:

wget --load-cookies cookies.txt https://example.com/dashboard

Combining Saving and Loading

For advanced automation, you can combine these flags into a single command. This allows wget to read existing cookies from a file, update them if the server sends new ones, and write the updated set back to the same file:

wget --load-cookies cookies.txt --save-cookies cookies.txt --keep-session-cookies https://example.com/profile