Handling Large JSON in Axios Without Memory Leaks

When fetching massive JSON datasets using Axios in Node.js or browser environments, the default behavior of buffering and parsing the entire response in memory can lead to severe memory bloat or Out-Of-Memory (OOM) crashes. This article outlines the primary methods to mitigate memory retention in Axios, including streaming responses, using incremental JSON parsers, bypassing Axios's default transform mechanisms, piping directly to disk, and managing object lifecycle references.

Set Response Type to Stream

By default, Axios loads the entire HTTP response body into memory as a string or parsed object. In Node.js environments, you can configure Axios to treat the response as a Node.js ReadableStream by setting responseType: 'stream'. This ensures that incoming chunks are consumed piece by piece rather than accumulated in the V8 heap.

const response = await axios({
  method: 'get',
  url: 'https://api.example.com/large-dataset.json',
  responseType: 'stream'
});

Use Incremental Streaming JSON Parsers

Receiving a stream prevents network buffering, but buffering chunks back into a single string for standard JSON.parse() negates the memory benefits. Instead, pipeline the response stream into an incremental JSON parsing library, such as stream-json or JSONStream. These parsers emit events for individual objects or array items, allowing each item to be processed and immediately garbage collected.

const { parser } = require('stream-json');
const { streamArray } = require('stream-json/streamers/StreamArray');

const response = await axios({
  method: 'get',
  url: 'https://api.example.com/large-array.json',
  responseType: 'stream'
});

const pipeline = response.data
  .pipe(parser())
  .pipe(streamArray());

pipeline.on('data', ({ key, value }) => {
  // Process single object; memory is reclaimed after this iteration
  handleItem(value);
});

Bypass Default Axios Transforms

Axios applies built-in transformation functions to parse JSON strings automatically. When dealing with large payloads, this can cause duplicate memory allocation—once for the raw string and once for the parsed object tree. You can disable automatic parsing by overriding transformResponse.

const response = await axios.get('https://api.example.com/large-data', {
  transformResponse: [(data) => data] // Skips JSON.parse on the full buffer
});

Pipe Directly to Persistent Storage

If the payload does not need immediate in-memory processing, pipe the Axios response stream directly to a local file system write stream or a database pipeline. Writing the data directly to disk keeps memory usage constant, regardless of payload size.

const fs = require('fs');

const response = await axios({
  method: 'get',
  url: 'https://api.example.com/export.json',
  responseType: 'stream'
});

const writer = fs.createWriteStream('./output.json');
response.data.pipe(writer);

await new Promise((resolve, reject) => {
  writer.on('finish', resolve);
  writer.on('error', reject);
});

Release References and Handle Stream Destruction

Memory leaks often occur when streams fail to close or when long-lived closures retain references to previous chunks. Always ensure that streams are properly destroyed on error or completion, and avoid accumulating parsed items into arrays located outside the stream's event loop scope. If using AbortController to cancel requests, verify that attached stream listeners are properly unregistered to permit timely garbage collection.