Software Development

How Did Node.js Come About?

To help you better understand what Node.js is and how some of its development decisions came about, let's explore the history of the platform.

 

Origins

Node.js was originally developed by Ryan Dahl, a PhD student in mathematics who thought better of it, abandoned his efforts, and instead preferred to travel to South America with a one-way ticket and very little money in his pocket. There, he kept his head above water by teaching English. During this time, he got in touch with PHP as well as Ruby and discovered his affection for web development. The problem with working with the Ruby framework, called Rails, was that it couldn’t deal with concurrent requests without any workaround. The applications were too slow and utilized the CPU entirely. Dahl found a solution to his problems with Mongrel, a web server for applications based on Ruby.

 

Unlike traditional web servers, Mongrel responds to user requests and generates responses dynamically, where otherwise only static HTML pages are delivered.

 

The task that actually led to the creation of Node.js is quite trivial from today’s point of view. In 2005, Dahl was looking for an elegant way to implement a progress bar for file uploads. However, the technologies available at the time only allowed unsatisfactory solutions. Regarding file transfers, HTTP was used for relatively small files, and File Transfer Protocol (FTP) was used for larger files. The status of the upload was queried using long polling, which is a technique where the client sends long-lived requests to the server, and the server uses the open channel for replies. Dahl’s first attempt to implement a progress bar took place in Mongrel. After sending the file to the server, it checked the status of the upload using a large number of Asynchronous JavaScript and XML (AJAX) requests and displayed it graphically in a progress bar. However, the downside of this implementation was Ruby’s single-threaded approach and the large number of requests that were required.

 

Another promising approach involved an implementation in C. Here, Dahl’s options weren’t limited to one thread. However, C as a programming language for the web has a decisive disadvantage: only a small number of developers are enthusiastic about this field of application. Dahl was also confronted with this problem and discarded this approach after a short time.

 

The search for a suitable programming language to solve his problem continued and led him to functional programming languages such as Haskell. Haskell’s approach is built on nonblocking input/output (I/O), which means that all read and write operations are asynchronous and don’t block the execution of a program. This allows the language to remain single-threaded at its core and doesn’t introduce the problems that arise from parallel programming. Among other things, no resources have to be synchronized, and no problems are caused by the runtime of parallel threads. However, Dahl still wasn’t fully satisfied with this solution and was looking for other options.

 

Birth of Node.js

Dahl then found the solution he was finally satisfied with: JavaScript. He realized that this scripting language could meet all his requirements. JavaScript had already been established on the web for years, so there were powerful engines and a large number of developers. In January 2009, he began working on his implementation for server-side JavaScript, which can be regarded as the birth of Node.js. Another reason for implementing the solution in JavaScript, according to Dahl, was the fact that the developers of JavaScript didn’t envision this area of use. At that time, no native web server existed in JavaScript, it couldn’t handle files in a file system, and there was no implementation of sockets to communicate with other applications or systems. All these points spoke in favor of JavaScript as the basis for a platform for interactive web applications because no determinations had yet been made in this area, and, consequently, no mistakes had yet been made either. The architecture of JavaScript also argued for such an implementation. The approach of top-level functions (i.e., functions that aren’t linked to any object, are freely available, and can be assigned to variables) offers a high degree of flexibility in development and enables functional approaches to solutions.

 

Thus, Dahl selected other libraries in addition to the JavaScript engine, which is responsible for interpreting the JavaScript source code, and put them together in one platform. In September 2009, Isaac Schlueter started working on a package manager for Node.js, the Node Package Manager (npm).

 

Breakthrough of Node.js

After Dahl integrated all the components and created the first executable examples on the new Node.js platform, he needed a way to introduce Node.js to the public. This also became necessary because his financial resources shrank considerably due to the development of Node.js, and he would have had to stop working on Node.js if he didn’t find any sponsors. He chose the JavaScript conference JSConf EU in November 2009 in Berlin as his presentation platform. Dahl put all his eggs in one basket. If the presentation was a success and he found sponsors to support his work on Node.js, he could continue his involvement; if not, almost a year’s work would have been in vain. In a rousing talk, he introduced Node.js to the audience and showed how to create a fully functional web server with just a few lines of JavaScript code. As another example, he introduced an implementation of an Internet Relay Chat (IRC) chat server. The source code for this demonstration comprised about 400 lines. Using this example, he demonstrated the architecture and thus the strengths of Node.js while making it tangible for the audience. The presentation didn’t miss its mark and led to Joyent stepping in as a sponsor for Node.js. Joyent is a San Francisco-based software and services provider offering hosting solutions and cloud infrastructure. With its commitment, Joyent included the open-source software Node.js in its product portfolio and made Node.js available to its customers as part of its hosting offerings. Dahl was hired by Joyent and became a full-time maintainer for Node.js from that point on.

 

Node.js Conquers Windows

The developers made a significant step toward the spread of Node.js by introducing native support for Windows in version 0.6 in November 2011. Up to that point, Node.js could only be installed awkwardly on Windows via Cygwin.

 

Since version 0.6.3 in November 2011, npm has been an integral part of the Node.js packages and is thus automatically delivered when Node.js is installed.

 

Surprisingly, at the start of 2012, Dahl announced that he would finally retire from active development after three years of working on Node.js. He handed over the reins of development to Schlueter. The latter, like Dahl, was an employee at Joyent and actively involved in the development of the Node.js core. The change unsettled the community, as it wasn’t clear whether the platform would continue to develop without Dahl. A signal that the Node.js community considered as being strong enough for solid further development came with the release of version 0.8 in June 2012, which was primarily intended to significantly improve the performance and stability of Node.js.

 

With version 0.10 in March 2013, one of the central interfaces of Node.js changed: the Stream application programming interface (API). With this change, it became possible to actively pull data from a stream. Because the previous API was already widely used, both interfaces continued to be supported.

 

io.js: The Fork of Node.js

In January 2014, there was another change in the project management of Node.js. Schlueter, who left Node.js maintenance in favor of his own company (called npmjs), the host of the npm repository, was succeeded by TJ Fontaine. Under his direction, version 0.12 was released in February 2014. A common criticism of Node.js at the time was that the framework had still not reached the supposedly stable version 1.0, which prevented numerous companies from using Node.js for critical applications.

 

Many developers were unhappy with Joyent, which had provided maintainers for Node.js since Dahl, and so the community fractured in December 2014. The result was io.js, a fork of Node.js that was developed separately from the original platform. As a result, the independent Node.js Foundation was founded in February 2015, which was responsible for the further development of io.js. At the same time, version 0.12 of the Node.js project was released.

 

Node.js Reunited

In June 2015, the two projects io.js and Node.js were merged into the Node.js Foundation. With version 4 of the project, the merger was completed. Further development of the Node.js platform is now coordinated by a committee within the Node.js Foundation rather than by individuals. As a result, we see more frequent releases and a stable version with long-term support (LTS).

 

Deno: A New Star in the JavaScript Sky

Since the merger of io.js and Node.js, things have become quieter around Node.js. The regular releases, the stability, and also the integration of new features, such as worker threads, HTTP/2 or performance hooks, keep up the good mood within the community. And just when things were starting to get almost too quiet around Node.js, an old acquaintance, Dahl, took the stage again in 2018 to introduce a new JavaScript platform called Deno during his talk, “10 Things I Regret about Node.js.”

 

The idea behind Deno is to create a better Node.js, untethered from the backwards compatibility constraints that prevent revolutionary leaps in development. For example, Deno is based on TypeScript by default and adds a fundamentally different module system. Deno’s core is also quite different from Node.js, as it’s written almost entirely in Rust.

 

Nevertheless, there are also some common features. For example, Deno is based on the tried and tested V8 engine, which also forms the heart of Node.js. And you don’t have to do without the huge number of npm packages either. For this purpose, Deno provides a compatibility layer.

 

OpenJS Foundation 1

In 2015, the Node.js Foundation was established to coordinate the development of the platform. The foundation was a subordinate project of the Linux Foundation. In 2019, the JS Foundation and the Node.js Foundation then merged to form the OpenJS Foundation. In addition to Node.js, it includes a number of other popular projects such as webpack, ESLint, and Electron.

 

Editor’s note: This post has been adapted from a section of the book Node.js: The Comprehensive Guide by Sebastian Springer.

Recommendation

Node.js: The Comprehensive Guide
Node.js: The Comprehensive Guide

If you’re developing server-side JavaScript applications, you need Node.js! Start with the basics of the Node.js environment: installation, application structure, and modules. Then follow detailed code examples to learn about web development using frameworks like Express and Nest. Learn about different approaches to asynchronous programming, including RxJS and data streams. Details on peripheral topics such as testing, security, performance, and more make this your all-in-one daily reference for Node.js!

Learn More
Rheinwerk Computing
by Rheinwerk Computing

Rheinwerk Computing is an imprint of Rheinwerk Publishing and publishes books by leading experts in the fields of programming, administration, security, analytics, and more.

Comments