JavaScript Streams: Readable, Writable, Duplex, Transform

Streams in JavaScript are fundamental objects used to handle reading and writing data sequentially chunk by chunk, preventing high memory usage when processing large datasets. This guide explains the core differences between the four primary stream types in JavaScript—Readable, Writable, Duplex, and Transform streams—covering their mechanisms, standard implementations, and specific use cases.

1. Readable Streams

A Readable stream represents a source from which data can be consumed. It produces data sequentially and passes it into a buffer until a consumer reads it.

• Data Flow: One-way (Source \(\rightarrow\) Consumer).
• Key Events/Methods: Emits 'data', 'end', and 'error' events; uses methods like stream.read(), stream.pipe(), and stream.pause().
• Common Examples:
  • fs.createReadStream() for reading files from disk.
  • http.IncomingMessage representing the incoming request body on a server.
  • process.stdin for reading terminal input.

2. Writable Streams

A Writable stream represents a destination where data can be written. It consumes chunks of data and writes them to a specific target, handling backpressure automatically when the destination cannot keep up with the incoming rate.

• Data Flow: One-way (Producer \(\rightarrow\) Destination).
• Key Events/Methods: Emits 'drain', 'finish', and 'error' events; uses methods like stream.write() and stream.end().
• Common Examples:
  • fs.createWriteStream() for writing data to disk.
  • http.ServerResponse for sending data back to a client.
  • process.stdout for printing output to the console.

3. Duplex Streams

A Duplex stream is a stream that implements both the Readable and Writable interfaces. It can both produce and consume data independently. The read and write channels operate separately, meaning the data being read does not necessarily depend on or modify the data being written.

• Data Flow: Bidirectional and independent (Read channel \(\leftrightarrow\) Write channel).
• Key Feature: Maintains two separate internal buffers—one for reading and one for writing.
• Common Examples:
  • net.Socket in TCP connections, where you can send outgoing network packets while simultaneously receiving incoming packets.

4. Transform Streams

A Transform stream is a specialized type of Duplex stream where the output is causally linked to the input. Instead of maintaining independent read and write channels, a Transform stream accepts input data, applies a transformation algorithm to it, and outputs the modified data to be read.

• Data Flow: Pass-through modification (Input \(\rightarrow\) Transformation logic \(\rightarrow\) Output).
• Key Feature: Implements an internal _transform() method to compute readable chunks directly from writable chunks.
• Common Examples:
  • zlib.createGzip() for compressing data chunks on the fly.
  • crypto.createCipheriv() for encrypting data streams.
  • Custom parsers (e.g., CSV to JSON line-by-line converters).

Key Differences Summary