Stream File Uploads with Axios in Node.js

Streaming file uploads in Node.js using Axios allows applications to transfer large files efficiently without exhausting system memory. Instead of buffering an entire file into RAM before sending, you can stream the data directly from the filesystem to the destination server. This guide covers the procedure for implementing streaming uploads using both raw binary streams and multipart form-data streams.

1. Direct Binary Stream Upload

When sending a raw file stream (e.g., application/octet-stream), you pass a readable stream directly to the data property in Axios and provide the file's size using the Content-Length header.

const axios = require('axios');
const fs = require('fs');
const path = require('path');

async function uploadRawStream(filePath, targetUrl) {
  const fileStats = fs.statSync(filePath);
  const fileStream = fs.createReadStream(filePath);

  try {
    const response = await axios.put(targetUrl, fileStream, {
      headers: {
        'Content-Type': 'application/octet-stream',
        'Content-Length': fileStats.size,
      },
      maxBodyLength: Infinity,
      maxContentLength: Infinity,
    });

    return response.data;
  } catch (error) {
    fileStream.destroy();
    throw error;
  }
}

2. Multipart Form-Data Stream Upload

For standard file uploads (equivalent to an HTML file input), use the form-data library to append the file stream.

Step 1: Install Dependencies

npm install axios form-data

Step 2: Implement the Multipart Stream

const axios = require('axios');
const FormData = require('form-data');
const fs = require('fs');

async function uploadMultipartStream(filePath, targetUrl) {
  const form = new FormData();
  const fileStream = fs.createReadStream(filePath);

  form.append('file', fileStream);

  try {
    const response = await axios.post(targetUrl, form, {
      headers: {
        ...form.getHeaders(),
      },
      maxBodyLength: Infinity,
      maxContentLength: Infinity,
    });

    return response.data;
  } catch (error) {
    fileStream.destroy();
    throw error;
  }
}

Essential Configuration and Best Practices