The Internals of Node.js

Part 5. Libuv Thread Pool

“Libuv Thread Pool”

In the previous section, we found that ‘pbkdf2’ function call does not behave as single-threaded. What’s going on ‘pbkdf2’ function behind the scene?

Recall, ‘pbkdf2’ function has both the javascript and c++ implementation but it actually delegated all the work to the c++ side, and that’s where the actual hashing process took place.

Inside ofc++file, there are references to a libuv library which gives Node.js better access to the operating system. The Node’s c++ side and libuv library decides to do expensive calculations outside of the event loop entirely.

By default libuv creates 4 threads as shown in the diagram so that it can execute intensive tasks such as the ‘pbkdf2’ function.

In short, in addition to the single-thread used for the event loop, there are four other threads created bylibuv library that can be used to process expensive calculations. Many of the functions included in the Node.js standard library will automatically make use of these newly created thread pools.

Note that our CPU runs all the instructions inside of a thread one by one. So, if our event loop was responsible for doing all the intensive tasks without the help of libuv, then Node.js can do nothing else while running the ‘pbkdf2’ function.

So with the help of additional threads created by Node.js library, we don’t have to wait until the expensive process is completed so that we can do other things inside of our event loop.

“Thread pool and Multi-threading”

From now on, I am going to demonstrate what we have discussed through coding. My laptop is MacPro 2019 with hexacore.

At my code editor, I am going to call ‘pbkdf2’ function and duplicate it down a dozen times. So in total, we will have 13 separate ‘pbkdf2’ function calls as shown below.

Now I’ll go back over to the terminal and run this file and let’s see what happens.

If you run the codes, you will get the first 12 results and after some period of time, the 13th result suddenly appears. By the way, depending on your machine CPU you might see some different results.

Let me explain to you what’s going on with my computer with a diagram.

So again the first 12 calls took 1.1 seconds to complete and then the 13th call took about 0.5 additional seconds.

There are several interesting things that I want to point out.

1. First, the first group of 12 function calls took the exact same amount of time which was 1.1 seconds.
2. If you recall the previous section, executed time for one function call took about 0.5 seconds.
3. Note that the 13th calls took only one additional 0.5 seconds

Let’s try to figure out why we saw the change in timing with the first group that ran and then took only half seconds to complete the 13th call. Right now I am using 2019 macbook Pro with hexacore processer, and I will set my thread number equal to 12.

So I know the figure down below may not be accurate, but I will assume it will describe the hardware that’s inside of my computer.

So the first function call was taken and we can kind of imagine that maybe it got assigned to the first thread and then, this first thread got assigned to core number one. Similarly, the second call to PBK to maybe got assigned to the second thread and then the second thread got assigned to the core number one as well.

Remember, since our CPU has access to multi-threading, it allows us to process more than one thread at once, and so cores are able to process these two threads at the same time. However, each core still had to do twice the amount of work.

Which is why it took roughly twice as long to calculate the hashing value for these first 12 values. It doesn’t actually speed up the processing. It just allows for some amount of concurrency inside of the work. That is why it took roughly double the amount of time.

Once those first 12 tasks were done, Node.js was now able to call our last 13th function call.

Since core number 6 was only processing 1 function call, it took only half seconds instead of one.

Ok, we have covered many materials so for. Let’s recap what we have discussed.

Recap

• The Node’s c++ side and libuv library decides to do expensive calculations outside of the event loop entirely.
• By default libuv creates 4 threads as shown in the diagram so that it can execute intensive tasks such as the ‘pbkdf2’ function.
• CPU allows us to process more than one thread at once, and so cores are able to process these two threads at the same time. However, each core still had to do twice the amount of work.

Reference

Level Up with Node.js: Advanced Node.js Concepts