Stream File Uploads with Axios in Node.js

Streaming file uploads in Node.js using Axios allows applications to transfer large files efficiently without exhausting system memory. Instead of buffering an entire file into RAM before sending, you can stream the data directly from the filesystem to the destination server. This guide covers the procedure for implementing streaming uploads using both raw binary streams and multipart form-data streams.

1. Direct Binary Stream Upload

When sending a raw file stream (e.g., application/octet-stream), you pass a readable stream directly to the data property in Axios and provide the file's size using the Content-Length header.

const axios = require('axios');
const fs = require('fs');
const path = require('path');

async function uploadRawStream(filePath, targetUrl) {
  const fileStats = fs.statSync(filePath);
  const fileStream = fs.createReadStream(filePath);

  try {
    const response = await axios.put(targetUrl, fileStream, {
      headers: {
        'Content-Type': 'application/octet-stream',
        'Content-Length': fileStats.size,
      },
      maxBodyLength: Infinity,
      maxContentLength: Infinity,
    });

    return response.data;
  } catch (error) {
    fileStream.destroy();
    throw error;
  }
}

2. Multipart Form-Data Stream Upload

For standard file uploads (equivalent to an HTML file input), use the form-data library to append the file stream.

Step 1: Install Dependencies

npm install axios form-data

Step 2: Implement the Multipart Stream

const axios = require('axios');
const FormData = require('form-data');
const fs = require('fs');

async function uploadMultipartStream(filePath, targetUrl) {
  const form = new FormData();
  const fileStream = fs.createReadStream(filePath);

  form.append('file', fileStream);

  try {
    const response = await axios.post(targetUrl, form, {
      headers: {
        ...form.getHeaders(),
      },
      maxBodyLength: Infinity,
      maxContentLength: Infinity,
    });

    return response.data;
  } catch (error) {
    fileStream.destroy();
    throw error;
  }
}

Essential Configuration and Best Practices

• Disable Axios Size Limits: By default, Axios limits request body sizes. Always set maxBodyLength: Infinity and maxContentLength: Infinity in the request configuration when uploading large files.
• Set Proper Headers: When using raw streams, always calculate and provide the Content-Length header via fs.statSync(filePath).size. For multipart uploads, use form.getHeaders() to ensure the correct multipart/form-data boundary is set.
• Handle Stream Errors: Always clean up and destroy the readable stream (fileStream.destroy()) inside a catch block to prevent file descriptor leaks if the network request fails prematurely.