Streams process data piece by piece instead of loading everything into memory. Buffers handle binary data.

Why Streams?

Reading a 1 GB file into memory crashes most apps. Streams read chunks sequentially:

  Source → [chunk] → Transform → [chunk] → Destination

Stream Types

Type Example
Readable fs.createReadStream, HTTP request body
Writable fs.createWriteStream, HTTP response
Duplex TCP sockets
Transform gzip compression, encryption

Reading a File with Streams 

  import fs from 'fs';

const readStream = fs.createReadStream('large-file.txt', { encoding: 'utf-8' });

readStream.on('data', (chunk) => {
    console.log('Chunk:', chunk.length, 'bytes');
});

readStream.on('end', () => console.log('Done reading'));
readStream.on('error', (err) => console.error(err));

Piping Streams 

  import fs from 'fs';

fs.createReadStream('input.txt')
  .pipe(fs.createWriteStream('output.txt'));

Transform Stream 

  import { Transform } from 'stream';
import fs from 'fs';

const upperCase = new Transform({
    transform(chunk, encoding, callback) {
        callback(null, chunk.toString().toUpperCase());
    }
});

fs.createReadStream('input.txt')
  .pipe(upperCase)
  .pipe(fs.createWriteStream('output.txt'));

Buffers

Binary data container:

  const buf = Buffer.from('Hello', 'utf-8');
console.log(buf);           // <Buffer 48 65 6c 6c 6f>
console.log(buf.toString()); // 'Hello'
console.log(buf.length);    // 5 bytes

// Concatenate buffers
const buf1 = Buffer.from('Hello ');
const buf2 = Buffer.from('World');
const combined = Buffer.concat([buf1, buf2]);

Async Iteration with Streams 

  import { createReadStream } from 'fs';
import { createInterface } from 'readline';

const rl = createInterface({
    input: createReadStream('data.txt'),
    crlfDelay: Infinity
});

for await (const line of rl) {
    console.log('Line:', line);
}

Practical Example: HTTP File Download 

  import http from 'http';
import fs from 'fs';

http.get('http://example.com/large-file.zip', (response) => {
    const file = fs.createWriteStream('download.zip');
    response.pipe(file);
    file.on('finish', () => {
        file.close();
        console.log('Download complete');
    });
});

Use streams for files, HTTP bodies, compression, and any large data processing in Node.js.

Backpressure

When a writable stream cannot keep up, readable streams pause automatically when piped:

  const readable = fs.createReadStream('large-input.txt');
const writable = fs.createWriteStream('output.txt');

readable.pipe(writable);

writable.on('drain', () => readable.resume());
readable.on('pause', () => console.log('Readable paused — backpressure'));

Object Mode Streams

Streams can carry JavaScript objects instead of buffers:

  const { Transform } = require('stream');

const jsonParser = new Transform({
    objectMode: true,
    transform(chunk, enc, cb) {
        cb(null, JSON.parse(chunk.toString()));
    }
});

pipeline handles cleanup and errors better than manual .pipe() chains:

  import { pipeline } from 'stream/promises';
import { createGzip } from 'zlib';

await pipeline(
    fs.createReadStream('input.txt'),
    createGzip(),
    fs.createWriteStream('input.txt.gz')
);

Memory Comparison

Approach 1 GB file memory usage
readFileSync ~1 GB
Stream (64 KB chunks) ~64 KB

Common Pitfalls

  • Forgetting to handle 'error' events — unhandled errors crash the process.
  • Not closing write streams — call file.close() or use pipeline.
  • Mixing sync and async stream handlers inconsistently.