At this meetup, our colleague Lehel Papacek talked about the internals of JavaScript runtimes and the Event Loop, working in the heart of the runtime. With such a structure, we can easily write our code in asynchronous manner - but we still must know our tool!
There were several reasons for holding this meetup. First of all, there is a surging interest this year (and also a couple of years before) in JavaScript, which became the world’s most popular programming language. That being said, it’s not hard to imagine that this programming language is currently used across the whole industry, from browsers, through servers to IoT and robotics.
Stackoverflow’s 2019 Developer Survey
Knowing the environment which allows your code to run is not to be undertaken lightly - especially when you just started your JavaScript journey, but also if you are a veteran who wants to refresh their memory or to give a chance to learn something new. This is important, as JavaScript has a different approach to concurrency than other popular languages and it’s essential to understand correctly the way it works under the hood.
Asynchronous JavaScript
In order to understand this strange beast of a language, we must start from the very beginning. Everything starts with the JavaScript Engine, the foundation of every runtime. In a nutshell, the engine allows your code to run, by compiling and executing it.
The most famous one is Google’s V8, running inside the Chrome browser. It is (as every other JavaScript engine) single-threaded, executing one statement at time, consisting of a Memory Heap and a Call Stack. From bird’s view, it seems pretty simple - but trust us, there is a lot going on inside it!
“So, if the engine is single-threaded, how can we achieve asynchronous behavior?”, you may ask. Well, to figure that one out, we need to dig a bit deeper.
There are things built on JS engines and they are called Runtimes. The browser you are using currently is a JavaScript Runtime itself. They add additional features on top of the engine’s possibilities, e.g.: DOM, Websockets, etc. in browser’s WebAPIs and http, Buffer, fs and other APIs provided by Node.js on servers. But the most important feature for us is the possibility of writing and executing asynchronous functions.
One of the most important reasons why we have this feature provided is that I/O is expensive. We can write synchronous, blocking code (that’s how the engine alone works), but that will choke our execution, as we will process one task, or request at a time.
But fear not and welcome the asynchronous, non-blocking way of programming, powered by your JS runtime!
Node.js & The Event Loop - Inside a runtime
We will continue our explorations inside Node.js JavaScript runtime, used for server-side development, but the vast majority we mentioned at this meetup also applies to other runtimes like Google Chrome - if you are more frontend oriented. There are three main components of a runtime that we need to distinguish:
V8 engineNode.js API (Node.js Bindings)Threading library - libuv
The V8 engine and the Node.js APIs being already mentioned, we arrived to the star of this meetup - the Event Loop. It’s in the heart of the threading library called libuv, that has been developed for and used by Node.js. You can easily find the correct library inside your browser or any other JS runtime - the mechanisms are the same.
As you guessed - this library allows us to have asynchronous execution under the hood, while our code still runs on a single thread.
We can imagine the Event Loop as an infinite while loop, with several responsibilities:
Offloads asynchronous tasks to libuv’s Worker Pool or Node.js’ own implementationsWaits for Call Stack to emptyChecks for pending tasks in Task Queue (you can find names like Event Queue, Callback Queue, etc. in literature - it means the same)Executes registered callbacks once the task is finished
Enough of explanations, let’s see the Event Loop in action!
We will start off with a simple code snippet, which uses a timer to represent asynchronous execution. With setTimeout, we are delaying code execution until a certain period, given in milliseconds.
This little animation is essential to understand, as it shows the main characteristics of the Event Loop - and thus the asynchronous execution itself.It can be obvious, but let’s break down what happened, so we can more easily understand the forthcoming snippets.
Script execution started. The console.log statement is synchronous, so it was executed right awaysetTimeout is an asynchronous function, so that task has been deferred to internal mechanisms to take care of - run the timer and signal if it’s finishedGo on with synchronous execution and run the second console.log statementThe timer is invalidated and the registered callback is enqueued to Task Queue ready for executionThe call stack is empty and the Event Loop can do it’s duties: check the Task Queue and run the handlers, dequeuing them in FIFO manner. The callback function is executed as regular code on call stack and the program finishes, as nothing else is left of Task Queue.
Macrotasks & Microtasks
We only met setTimeout until now, but we could used setImmediate, fs.readFile, crypto.generateKeyPair or any other async API really, without changing the behaviour of our program. These tasks are called macrotasks.
In order to understand them, we need to know our Event Loop a bit better. In it’s single iteration (tick, cycle), the Event Loop goes through several phases. Some of them are not relevant for us, as they are mostly related to internal processes, but the ones that we are interested in are pretty exciting - each of them is responsible for a certain type of task and has its own queue.
Timers (setTimeout / setInterval)Pending callbacks (nearly every I/O callback)Check (setImmediate)Close callbacks (socket.on(‘close’)...)
These phases are visited in the same order every cycle. That means that our setImmediate callback will always be executed after our I/O handlers, which will be executed after invalidated timer callbacks, and so on.
The funny thing is, that there is more: meet the microtasks! They are behaving a bit different than the others, as they have their own queue(s) and those queues are visited and emptied after every macrotask. This is important, because microtasks can bite us back if we don’t know their background.One of the microtasks is process.nextTick. Let’s see an example which ties together the logic between macro- and microtasks.
After we saw every step of the Event Loop for this program, we can guess the outcome too. One small remark, though: we called setTimeout with 0 milliseconds, which effectively means that the timer invalidates immediately. Of course, this was only for the sake of this example.
Until now we only used old-fashioned callbacks in examples, so how Promises fit in all this?
Promise API
With Promise API in our bag, we finally got rid of callback hell and we can control asynchronous functions more efficiently. This feature was introduced as a part of ECMAScript 2015 and async functions return a Promise object as a future result (in contrast to calling callback functions).
We will not go into the internals of this API, considering only the aspect that is related to our topic: resolved Promises are microtasks and they have their own queue. The same rules apply to them as to process.nextTicks, with one simple constraint: nextTicks are top priority, every other pending task is executed after them. We can see this behavior in the example below.
From ECMAScript 2017, we can improve the readability of our code even further, using the brand new async / await feature, which is nothing else than syntactic sugar over the Promise API. Thus, every rule that applies to Promises, also applies to functions written with async / await construct.
The JavaScript code written with async / await keywords is similar to synchronous code, try / catch blocks can be used, but without losing the asynchronous behavior - so much win!
How not to screw things up
We cannot forget that our concurrency model is different from other common ones and our code still lives in single thread. With that in mind, it makes sense that Node.js (or any other JavaScript runtime) doesn’t like CPU-intensive tasks. With such code on call stack, we could block the Event Loop and our asynchronous model to work.
So first and foremost, avoid running heavy computation on main thread. This applies to CPU-intensive tasks like image processing, long running loops, etc. The solution? Use asynchronous code whenever it’s possible.
Did you ever read a file from the file system using Node.js? You can use the fs.readFile async function, which is fine, until you acknowledge that it will take out one thread from libuv’s Worker Pool for hard work, meaning that other clients will have one thread less. By design, the thread number in Worker Pool is fairly limited, defaulting to 4 and extendible to 128.
This problem can be avoided using a technique called task partitioning - let’s demonstrate it on example using read streams.
By using streams, we achieve much better thread management under the hood, as every thread from the Worker Pool will only do a portion of a bigger task.
Let’s not forget the rule we mentioned earlier, that every microtask queue is visited and emptied after every macrotask. That means that we could spam nextTicks inside nextTicks and Promises inside Promises in same Event Loop cycle, unintentionally delaying the execution of the next macrotask. Please don’t do that.
Worker Threads - hello multi-threading!
Of course, Node.js is not a silver bullet for all our problems and until now we would reconsider our choice of tech stack if we knew that it will use CPU-intensive tasks.
Released recently in Node.js version 10.5.0, we have a new feature that can come handy in server-side programming: Worker Threads. Although it’s still in experimental stage, we can try it out.
Basically, we can create threads on our own, having a separate Event Loop and the whole threading mechanism in each of them. This can sound familiar to frontend developers, as Worker Threads are the Node.js way of handling Web Workers. The idea is the same.
From the example above, we see that a CPU-intensive task is started in a separate thread, not blocking our parent Event Loop, which is demonstrated with setImmediate’s callback execution.
Worker Threads are a new, powerful feature, especially in server-side world, so we are looking forward to see it working in the future!
Thanks for reading through this overview of our recent Meetup held in Novi Sad by Lehel Papacek. See you soon!
