nodejs logo

Streams

Solving I/O Bound Problems

Created by Evan Oxfeld / @evanoxfeld

Overview

  • Explore I/O bound problems in Node
  • Explore Streams API

Streams Example - Unzip


fs.createReadStream('path/to/archive.zip')
  .pipe(unzip.Parse())
  .pipe(fstream.Writer('output/path'));
				

Code!

Behind the curtain

## util.pump(readable, writable) * Sets up event handlers for 'data', 'end' * Ignores other events e.g. 'error', 'close' * API limits customization, not chainable

Streams in 0.8


	readable.pipe(duplex).pipe(writable)
						
## Streams in 0.8 * Readable * Emit data events * Optionally implement pause() and resume() * Writable * Implement write() and end()
## Awesome! Let's demo!

Issues - Backpressure



Readable Writable

Issues - Backpressure



Readable Writable

Issues - Backpressure



Readable Writable

Issues - Backpressure

Flood

More Issues

  • Buffering and backpressure
  • No on.('pipe') method
  • pause() isn't a guarantee

Streams of Tomorrow, Finally Here

Streams in 0.10

  • Readable streams are now suck streams
  • read([size]) is equivalent to write(data)
    • read() returns null if less data is buffered than size
    • write() return false or pass a callback
  • Object mode
  • Same composable pipe API
  • Shared base classes for backpressure and buffering

Streams 0.10 Base Classes

  • Readable, Writable, Duplex
  • Transform: implement _transform(chunk, encoding, cb)
  • PassThrough: no method to implement
# Example stream.Transform Code!

Unzip Demo

Conclusions

  • Stream API is great for solving I/O bound problems
  • Streams2 developed in the open with a parallel user-land module
@substack
Streams make programming in node simple, elegant, and composable.

References

Questions?