Everything You Need to Know About Streams in Node.js

Streams in Node.js are infamous for being hard to work with and even harder to understand. This article will make it easier for you to understand streams, latest trends in Node JS and how to work with it.

What are Streams?

Streams are a collection of data just like arrays or strings. However, streams might not be available all at once. What makes them really powerful when working with large amounts of data is that they process the content piece by piece without keeping it all in the memory.

One of the best real-life examples would be streaming services like Netflix and YouTube. These websites never ask you to download the video at once, instead, they send the video in chunks, allowing you to watch the video continuously.

Streams are used to read or write input into output sequentially. They can also handle data coming from an external source one chunk at a time. Besides, they give you the power of composability in your code.

In a nutshell, streams can be used to handle reading/writing files, network communications, or any kind of end-to-end information exchange efficiently.

Why Use Streams?

There are two major advantages of using Streams compared to other data handling methods.

• First, you don’t need to load large amounts of data to process it.
• Second, it takes less time to start processing the data as you won’t have to wait until the entire payload has been transmitted.

Types of Streams

• Writable: These are the streams that allow us to write data.
• Readable: These are the streams from which data can be read.
• Duplex: These are both readable and writable.
• Transform: These are the streams that can modify or transform the data as it is read and written.

The request moduleis a readable stream while the response module is writable steam. The fs module allows you to work with both readable and writable streams.

If you use Express you are more or less using streams to interact with the client. Since TCP sockets, TLS stack, and other connections are based on Node.js streams, they are also used in database connection drivers.

Creating a Readable Stream

The first step to creating a readable stream is initializing it.

const Stream = require(‘stream’)

constreadableStream = new Stream.Readable()

The next step would be sending data to it.

readableStream.push(‘ping!’) 

readableStream.push(‘pong!’)

async iterator

When working with streams, it is recommended to use an asynchronous iterator. The asynchronous iterator is a protocol for retrieving the contents of a data asynchronously. In simple terms, the current task is paused before retrieving an item.

Ensure that you don’t mix async functions with EventEmitterbecause as of now, there’s no way to catch a rejection when it is emitted within an event handler. This, in turn, makes it hard to track bugs and memory leaks.

Therefore, the best way is to always wrap the content in a try/catch block and handle errors. The pull request aims to solve this issue once it lands on the Node core.

Types of Reading Modes

The 2 modes of readable streams are 1) Flowing 2) Paused

Regardless of whether a stream is in flowing mode or paused mode, it can be in an object mode.

Flowing Mode: In this mode, the data is read from the underlying system automatically using events via the EventEmitter interface.

Paused Mode: In this mode, the stream.read() method must be called explicitly to read chunks of data from the stream.

Creating a Writable Stream

In order to create a writable stream, you need to call write() on the stream instance. It would look like this.

In the end, call writable.end() to indicate that no more data will be written to the Writable.

pipeline

Piping is a tool in which you provide the output of one stream as the input to another. Piping is generally used to process streamed data in multiple steps. Also, ensure that you use pipeline and not pipe. 

The Stream Module

The Node.js stream module provides the groundwork upon which all the streaming APIs are made. The stream is an example of the EventEmitter class which manages events asynchronously in Node.js.

After all, Streams are not that hard (pun intended). Streams and pipeline are the most robust features in Node.js Streams.

Author Bio

Bharat Patel, who heads the digital marketing team at BrainvireInfotech, is armed with over 12 years of experience in the fields of online marketing and project management. He is extremely proactive in implementing the latest technological innovations in his projects. Bharat’s core expertise lies in search engine optimization (SEO), social media marketing, and conversion rate optimization, among other things. His immense flare of writing encourages him to consistently pen down words revolving around current trends and innovations that relate to his fields of interest.