Streams and Buffer in Node.Js

Introduction 

We require streams in Node Js to handle and manipulate the streaming data like a video, a large file, etc. The streams module in Node.js is used to manage all the streams. A stream is an abstract interface used to work along with the streaming data in Node.js. There are many stream objects,Node.js offers us. 

For instance, both are considered stream instances if we request an HTTP server and process.stdout. Streams can be readable, writable, or both. All streams are instances of EventEmitter. 

To access the stream module, the syntax to be used is:

 

const stream = require('stream'); 

 

The stream module is used to create new types of stream instances. 

 

Types of Streams in Node.Js

In NodeJs, there are four different types of streams available for us to use. We have four main types of streams available. Let’s see each of them with one-liner explanations.

 

  1. Readable streams → This stream is used to create a stream of data for reading, for example: to read a large chunk of files.

    Example:

     

const fs = require('fs');

const readableStream = fs.createReadStream('./article.md', {
    highWaterMark: 10
});

readableStream.on('readable'() => {
    process.stdout.write(`[${readableStream.read()}]`);
});

readableStream.on('end'() => {
    console.log('DONE');
});

 

 

 

  1. Writable streams → This stream is used to create a stream of data to write. For example: to write a large amount of data in a file.

    Example: 

 

const fs = require('fs'); 
const file = fs.createWriteStream('file.txt'); 
for (let i = 0; i < 10000; i++) 

file.write('Hello world ' + i); 
}
file.end();



 

  1. Duplex streams → This stream is used to create a stream that is both readable and writable at the same time.

    Example:
const server = http.createServer((req, res) => {
  

    let body = '';

    req.setEncoding('utf8');
    req.on('data'(chunk) => {
        body += chunk;
    });

    req.on('end'() => {
        consol.log(body);

        try {
            // Send 'Hello World' to the user
            res.write('Hello World');
            res.end();
        } catch (er) {
            res.statusCode = 400;
            return res.end(`error: ${er.message}`);
        }
    });
}); 



 

  1. Transform streams → This stream is used to create a readable and writable stream, but the data in the stream can be modified while reading and writing to the stream.

    Example: Creating a transform stream for calculating a value.
     

var crypto = require('crypto');
var stream = require('stream');
var util = require('util');
var Transform = stream.Transform ||
  require('readable-stream').Transform;

function codingninjas(options) {
    if (!(this instanceof codingninjas)) {
    return new codingninjas(options);
  }


  Transform.call(this, options);

  this.digester = crypto.createHash('val1');
}
util.inherits(codingninjas, Transform);

codingninjas.prototype._transform = function (chunk, enc, cb) {
 
  var buffer = (Buffer.isBuffer(chunk)) ?
    chunk :
    new Buffer(chunk, enc);
  this.digester.update(buffer); 

  cb();
};

codingninjas.prototype._flush = function (cb) {
  this.push(this.digester.digest('hex'));
  cb();
};


// try it out
var codingninjas = new ShaSum();
codingninjas.pipe(process.stdout); // output to stdout
codingninjas.write('hello world\n'); // input line 1
codingninjas.write('another line');  // input line 2
codingninjas.end();  // finish

 

Buffers in Streams

In Node Js, buffers are used to store raw binary data. A buffer represents a chunk of memory that is allocated on our computer. The size of the buffer, once set, cannot be changed. A buffer is used to store bytes. Now, the next question is, what a byte is correct? A byte is a sequence of eight bits. Bits are a basic storage unit for your computer; these bits are used to a  value that can be either 0 or 1. 

An example for buffer in Node.js looks like this: 
 

<Buffer 81 271 365 231 261 2e> 


In the above example, we can see ten pairs of letters and numbers. Each pair is used to represent a byte that is stored in the buffer. The total size of the buffer is 10.

Stream and Event Emitters

EventEmitter is a class that helps us to create a publisher-subscriber pattern in NodeJS. Using the event emitter, we can simply raise a new event from a different part of an application, and a listener will listen to the raised event and have some action performed for the event.
Let’s consider an example for Event Emitters:

var emitter = require('events').EventEmitter; 
var em = new emitter(); 
//Subscribe FirstEvent 
em.addListener('FirstEvent'function (data
{
 console.log('First subscriber: ' + data); 
});
 //Subscribe SecondEvent 
em.on('SecondEvent'function (data
console.log('First subscriber: ' + data); 
}); 
// Raising FirstEvent 
em.emit('FirstEvent''This is my first Node.js event emitter example.');
 // Raising SecondEvent 
em.emit('SecondEvent''This is my second Node.js event emitter example.');

Flowing and Non-flowing 

There are two types of readable streams in NodeJs: 

  • Flowing stream — A stream used to pass data from the system and provide this data to your programs. 
     
  • Non-flowing stream — The Non-flowing stream that does not push data automatically. Instead, the non-flowing stream stores the data in the buffer explicitly calls the read() method of the stream to read it. 

 

Buffer Management

Buffers are developed for developers who use binary data. Buffers were introduced where conventionally allocated with the strings rather than binaries. Buffers are connected deeply with the streams, and this means that when a stream processor accepts data faster than it can digest, it sends the data to a buffer.

→ How to create a buffer?

There are three different ways to allocate a buffer using the Buffer API, which is used to create a new buffer, methods like Buffer.from()Buffer.alloc(), and Buffer.allocUnsafe() are used. 

 

We can create a void buffer with a size of 12 bytes using the given command.

const buf1 = Buffer.alloc(12);


We can also create a buffer using the UTF-8 encoded strings:

 

const buf2 = Buffer.from('Coding Ninjas!');

 

There are the encodings accepted while creating a buffer. We have listed a few of them :

  • ASCII
  • UTF-8
  • Base64
  • Latin1
  • Binary
  • Hex

The last method that is used to create a buffer is given below:
 

const buf3 = Buffer.allocUnsafe(22);

 

Frequently Asked Questions

  1. What are Stream and Buffers?

    Ans: A stream is an abstract interface used to work along with the streaming data in Node.js. There are many stream objects,Node.js offers us.  A buffer represents a chunk of memory that is allocated on our computer. The size of the buffer, once set, cannot be changed. A buffer is used to store bytes.
     
  2. What does the method Buffer.from() do?

    Ans: The Buffer.form() method is used for creating a new buffer that contains a specified string, array, or buffer itself.
     
  3. What are the different types of streams?

    Ans: In NodeJs, there are four different types of streams available. 
    → Readable Streams
    → Writable Streams 
    → Duplex Streams
    → Transform Streams

Key Takeaways

Hey everyone, so let’s brief out the article. Let’s discuss in brief whatever we have discussed here. 

  •  In this article, we have briefed about what Streams and Buffers are. We require streams in Node Js to handle and manipulate the streaming data like a video, a large file, etc. The streams module in Node.js is used to manage all the streams. A stream is an abstract interface used to work along with the streaming data in Node.js. There are many stream objects,Node.js offers us.
     
  • We have discussed different types of the four available streams available for developers. Along with these types of streams, we have covered what buffers are and how to manage buffers in the NodeJs.In Node Js, buffers are used to store raw binary data. A buffer represents a chunk of memory that is allocated on our computer. The size of the buffer, once set, cannot be changed. A buffer is used to store bytes.
     

Isn’t Web Development engaging? Building new websites and using amazing animations and different APIs, don’t be scared if you cannot grasp the hold of this vast development. We have the perfect web development course for you to make you stand out from your fellow developers. 

Happy Learning Ninjas!
 

Was this article helpful ?
0 upvotes

Comments

No comments yet

Be the first to share what you think