The basics of Node.js in 30 minutes

Published in

Tech Tonic

27 min readJan 9, 2023

Audience

Node.js is an extremely popular server-side JavaScript runtime suited for running web apps. If you already know or have used Node.js, you may want to skip this article as this covers the very basics. If you are new to Node.js, this article will take you on a quick crash course on the absolute basics of Node.js. You should be able to read the article in about 30 minutes. The purpose of this article is to give you enough information to get you started on the Node.js journey. If you’re reading further, let’s get started!

Introduction

You would have heard about Node.js from somewhere, personal or at work, which led you to finding the resources that’ll get you started on it. But what exactly is Node.js?

The official description of Node.js is:

As an asynchronous event-driven JavaScript runtime, Node.js is designed to build scalable network applications. Using asynchronous programming, many connections can be handled concurrently. Upon each connection, the callback is fired, but if there is no work to be done, Node.js will sleep.

Node.js is basically a server-side JavaScript runtime that can run JavaScript applications in the backend. Let’s take a brief look at the history of JavaScript. Since its inception is 1995 by Brendon Eich, the very popular JavaScript ran in the browser for advanced client-side programming. JavaScript is a scripting or programming language that enables implementation of complex features on web pages. Powered by JavaScript, a web page can do more than just sit there and display static information to be looked at. For example — displaying timely content updates, interactive maps, animated 2D/3D graphics, scrolling video jukeboxes, etc. — it’s easy to bet that JavaScript is probably involved. It is the third leg of the standard web technologies stool. The other two legs are: HTML and CSS. In other words, JavaScript is a scripting language that enables creation of dynamically updating content, control multimedia, animate images, and pretty much everything else.

From 1995 to 2009, JavaScript, though being very popular, was limited to the browser. There were two teams in most of the places: The frontend team which knew about HTML/CSS/JavaScript, and the backend team which knew about Java/PHP/Ruby/Python/etc. The inventor of Node.js, Ryan Dahl, got the idea of running JavaScript on the server-side so that the same person who does the frontend can also do the backend. Node.js lets developers use JavaScript to write command line tools and for server-side scripting. The functionality of running scripts server-side produces dynamic web page content before the page is sent to the user’s web browser. Consequently, Node.js represents a “JavaScript everywhere” paradigm, unifying web-application development around a single programming language, rather than different languages for server-side and client-side scripts. Node.js has an event-driven architecture capable of asynchronous I/O. These design choices aim to optimize throughput and scalability in web applications with many input/output operations, as well as for real-time Web applications (e.g., real-time communication programs and browser games).

Node.js brought the concept of full-stack development to life. With Node.js, the same person, who knows JavaScript, can write both the frontend and the backend. On the technical side, with Node.js, it was also possible to utilize the concept of async programming to handle numerous concurrent requests. There was no need to allocate a dedicated server threadpool, like the norm was at that time. This was a total shift from the way things were at that time. The Node.js got extremely popular over the next 13 years. In the year 2023, Node.js is still going very strong. There is hardly any competition to it.

Though Node.js is very popular, it is not suited for all the use cases. Node.js uses a single threaded architecture with async programming to carry out multiple tasks at once. Node.js is very good in I/O bound operations. Therefore, Node.js is ideally suited for web apps. Being single threaded, Node.js is very bad in CPU intensive operations. Any CPU intensive operation will end up blocking others.

Before moving ahead, there are three important points to make:

Node.js has gone through numerous releases in the last 13 years, from v0.0.1 to v19. This article is written for the most recent Node.js available at the time of writing: v19.4.0.
This article is being written in 2023. Wherever possible, the examples uses async programming through promises. Callbacks are avoided as much as possible by wrapping them into promises.
Knowledge of JavaScript is required for writing Node.js programs.

Installation

Node.js is available for the common platforms like Linux, Mac, and Windows. The official site to get Node.js is: https://nodejs.org/en/.

Node.js can be installed in the following ways:

Get the Node.js installer from the official site (installer is specific to the platform)
Get the Node.js from a version manager (NVM). A version manager is useful when you need to maintain multiple Node.js versions. If you’re using a single Node.js version, a version manager is not really required.

Node.js always has two releases available: A stable release which is in long term support (LTS), and the latest release which contains all the new features. At the time of writing, v18.13 is in LTS, while v19.4 is the latest.

We’ll take a look at installation of Node.js on Mac and Windows platforms.

Installation on Mac

For Mac, Node.js offers a .pkg installer:

The installation can be verified by checking the version:

> /usr/local/bin/node -v
v19.4.0

On Mac, Node.js can also be installed using the popular HomeBrew package manager:

> brew install node
==> Fetching node
==> Downloading https://ghcr.io/v2/homebrew/core/node/manifests/19.4.0
Already downloaded: /Users/mayankc/Library/Caches/Homebrew/downloads/4382f8cd536c10de77f616e04b0f3d9288b0d73bf6426e4f438dc43a541faaec--node-19.4.0.bottle_manifest.json
==> Downloading https://ghcr.io/v2/homebrew/core/node/blobs/sha256:9cbe7ed5378e54632f88095
Already downloaded: /Users/mayankc/Library/Caches/Homebrew/downloads/a5e6cd38f845085c52161cff07037a1759aec3f1fcdf8c53edc7420fdf10ef3f--node--19.4.0.arm64_ventura.bottle.tar.gz
==> Pouring node--19.4.0.arm64_ventura.bottle.tar.gz
==> Caveats
Bash completion has been installed to:
  /opt/homebrew/etc/bash_completion.d
==> Summary
🍺  /opt/homebrew/Cellar/node/19.4.0: 2,157 files, 54.1MB

The installation from Homebrew happens at the path: /opt/homebrew/bin. The Node version can be verified using -v option:

> which node
/opt/homebrew/bin/node
> node -v
v19.4.0

Installation on Windows

Just like Mac, Node.js offers an installer for Windows. The installation on Windows is the same as installing any other application.

After downloading the installer MSI, run it to install Node.js:

The installed node can be checked using the version command:

Getting Started

The best and easiest way to get started with Node.js is through Node REPL: Read Evaluate Print Loop. The REPL is a node shell where code can be written. Each line of code gets executed immediately. The context and history is maintained. Any variables created in previous line can be used later. The REPL (or node shell) can be started by running node without any arguments:

~: node
Welcome to Node.js v19.4.0.
Type ".help" for more information.
>

The node shell or REPL waits for the next instructions. The instructions could be any JavaScript code. The REPL shell also has suggestions/completions, making it very easy to write Node.js code.

We can start with simple console printing:

> console.log("Hello world!");
Hello world!
undefined

Next, let’s try fetching some data using fetch (also to show that variables can be created and used anytime):

> const rsp = await fetch("https://nodejs.org");
undefined
> const rspBody = await rsp.text();
undefined
> console.log("Response body size=", rspBody.length);
Response body size= 14692
undefined
> console.log("Response status code=", rsp.status);
Response status code= 200
undefined

Next, let’s read a file (this is achieved using Node’s core readFile API):

> const { readFile } = await import("node:fs/promises");
undefined
> const fileData = await readFile("/var/tmp/testdata/sample.txt", "utf-8");
undefined
> fileData
'Learning JavaScript & Typescript Is Fun!\n'

Finally, new web APIs can be also be tried. The following example shows TextEncoder in REPL:

> new TextEncoder().encode("Hello world!");
Uint8Array(12) [
   72, 101, 108, 108,
  111,  32, 119, 111,
  114, 108, 100,  33
]

As mentioned earlier, REPL also offers suggestions/completions. The following GIF file demonstrates it:

Event loop

The event loop is at the heart of Node.js. The structure of Node.js is based on a non-blocking model, and the event loop is the fundamental concept used by Node.js to facilitate the non-blocking model (aka async programming). Modern kernels can handle multiple operations as they are multithreaded. The kernel notifies Node.js when one task is completed and allows it to add the necessary call back to the event queue. The event loop runs through many phases:

Check timers
Check pending callbacks
Idle and preparation phase
Polling phase
Checking phase
Closing callbacks

The event loop has been explained very well in Node.js’s official documentation: https://nodejs.org/en/docs/guides/event-loop-timers-and-nexttick/.

A very simplified event loop is shown in the diagram below:

All new requests come to the event loop. If they are I/O bound, they are given to the event demux & the event loop continues with the next request. The event demux uses low-level OS calls to carry out the request. Once the operation completes, the result is sent to an event queue. The event loop takes data out of event queue (if there is any) and processes the event handler associated with that request (like a callback).

NPM

Node.js and NPM goes hand-in-hand. Although Node.js can be used without NPM, it is not practical because NPM provides access to more than 1.4M ready to use packages. There is almost always a package for anything we’re looking for. NPM CLI (runs through the npm command) comes installed with Node.js (no need to install it separately).

NPM consists of two main parts:

a CLI (command-line interface) tool for publishing and downloading packages
an online repository that hosts JavaScript packages

NPM provides a versioned repository of more than 1.4M packages. Any Node.js app using NPM packages needs to maintain a dependency list along with the package versions. This config file is called package.json. The package.json file is equivalent of pom.xml from Java, Gemfile in Ruby, etc.

To start a project, npm init command can be used:

/var/tmp/nodeProj: npm init
This utility will walk you through creating a package.json file.
It only covers the most common items, and tries to guess sensible defaults.

See `npm help init` for definitive documentation on these fields
and exactly what they do.

Use `npm install <pkg>` afterwards to install a package and
save it as a dependency in the package.json file.

Press ^C at any time to quit.
package name: (nodeproj) 
version: (1.0.0) 
description: First Node.js project
entry point: (index.js) 
test command: 
git repository: 
keywords: 
author: 
license: (ISC) 
About to write to /private/var/tmp/nodeProj/package.json:

{
  "name": "nodeproj",
  "version": "1.0.0",
  "description": "First Node.js project",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC"
}


Is this OK? (yes) y

The output of npm init is a package.json file which, as of now, contains no dependencies:

/var/tmp/nodeProj: cat package.json 
{
  "name": "nodeproj",
  "version": "1.0.0",
  "description": "First Node.js project",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC"
}

The npm install (or npm i) can be used to install all the dependencies present in the package.json file. This produces the famous node_modules folder in the current directory, where all the dependencies are stored.

To add a dependency to package.json, npm add command can be used. As we’ll be using express package in subsequent sections, let’s install express as the dependency:

/var/tmp/nodeProj: npm add express

added 57 packages, and audited 58 packages in 2s

7 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

The npm add command downloads the package from NPM, places it in node_modules, recursively install all the dependencies, and updates the package.json file. The updated package.json file is:

/var/tmp/nodeProj: cat package.json 
{
  "name": "nodeproj",
  "version": "1.0.0",
  "description": "First Node.js project",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "express": "^4.18.2"
  }
}

The express package and all its dependencies are placed in node_modules folder. Though it looks like a single package from package.json, the recursive dependency installer turns it into a number of packages:

/var/tmp/nodeProj: ls node_modules/
accepts   fresh   parseurl
array-flatten  function-bind  path-to-regexp
body-parser  get-intrinsic  proxy-addr
bytes   has   qs
call-bind  has-symbols  range-parser
content-disposition http-errors  raw-body
content-type  iconv-lite  safe-buffer
cookie   inherits  safer-buffer
cookie-signature ipaddr.js  send
debug   media-typer  serve-static
depd   merge-descriptors setprototypeof
destroy   methods   side-channel
ee-first  mime   statuses
encodeurl  mime-db   toidentifier
escape-html  mime-types  type-is
etag   ms   unpipe
express   negotiator  utils-merge
finalhandler  object-inspect  vary
forwarded  on-finished

NPM is documented very well at their official site: https://www.npmjs.com/.

Command-line arguments

The command-line arguments passed to the node application are present in the a basic array of strings: process.argv.

> process.argv;

:- node app.mjs -a -b 1 -c def -g
process.argv= [
  '/opt/homebrew/Cellar/node/19.4.0/bin/node',
  '/Users/mayankc/Work/source/app.mjs',
  '-a',
  '-b',
  '1',
  '-c',
  'def',
  '-g'
]

All the supplied args are there, but they are quite tough to use. If we want the value for option c, we need to parse the array. For parsing command-line args, there is a very useful NPM module: yargs-parser.

:- npm i yargs-parser

added 1 package, and audited 59 packages in 528ms

The yargs-parser provides a parse API that takes the args array as input and returns the parsed options. An example will help understand it better:

import parse from "yargs-parser";
console.log("Args=", parse(process.argv));

// ---

:- node app.mjs -a -b 1 -c def -g
Args= {
  _: [
    '/opt/homebrew/Cellar/node/19.4.0/bin/node',
    '/Users/mayankc/Work/source/app.mjs'
  ],
  a: true,
  b: 1,
  c: 'def',
  g: true
}

Now it is very easy to check the presence and get the values of the args.

HTTP server

As Node.js is ideally suited for I/O bound web apps, the first thing to is to write a simple web server. Node.js provides a very basic native HTTP server. The native server that comes with Node.js, but is a bit tough to use. The other extremely popular option is the express framework. We’ll look at both in this section.

The use-case here is to write an HTTP server that’ll extract the name attribute out of the provided JSON in the request body. The extracted name will be returned to the HTTP response in JSON format: { “name”: “<extracted-name”}.

First, let’s write an HTTP server using native APIs only. For this, we need to import http from the core HTTP module. Two HTTP APIs are required: createServer and listen. The request body needs to be collected and then parsed using JSON.parse. The response body needs to be stringified before sending to the caller. The application file app.mjs needs .mjs extension as we’ll be using imports in the code. If we were using commonJS style require, then the application file will have .cjs extension.

app.mjs

import http from "node:http";

http.createServer(
  (req, resp) => {
    if (req.method !== "POST") {
      return resp.writeHead(405).end();
    }
    if (req.url !== "/") {
      return resp.writeHead(404).end();
    }
    let data = "";
    req.on("data", (c) => data += c);
    req.on("end", () => {
      resp.writeHead(200, {
        "Content-Type": "application/json",
      });
      let reqBody = {};
      try {
        reqBody = JSON.parse(data);
      } catch (e) {}
      const ret = { name: reqBody.name ?? "Anonymous" };
      resp.end(JSON.stringify(ret));
    });
  },
).listen(3000);

As we can see, there is a lot of work required when using the native HTTP server. The advantage is that the native HTTP servers will be the fastest. As we’re just starting to learn Node.js, there is no need to use the native server. Frameworks are much better & easier to get started. Before going to express, let’s run a couple of tests using curl:

~: node app.mjs

~: curl http://localhost:3000 -X POST
{"name":"Anonymous"}~: 
~: curl http://localhost:3000 -d '{"name": "Mayank C"}'
{"name":"Mayank C"}

Now, let’s rewrite the same HTTP server using the very popular express framework.

import express from "express";

const app = express();
app.use(express.json());

app.post("/", (req, resp) => {
  resp.json({
    name: req.body?.name ?? "Anonymous",
  });
});

app.listen(3000);

The basic HTTP server for the same use-case is much cleaner and shorter than the native code. The request and response body gets processed by the express framework. There is no need to run JSON.parse & JSON.stringifiy APIs. Using the express middleware express.json(), the request body gets parsed by the express framework. For the response, any JavaScript object can be given.

Let’s run a couple of tests using curl:

~: curl http://localhost:3000 -X POST
{"name":"Anonymous"}

~: curl http://localhost:3000 -d '{"name": "Mayank C"}' -H 'content-type: application/json'
{"name":"Mayank C"}

HTTPS server

An HTTPS server can be started using the same APIs, but with two additional options: Keyfile, and Certfile. The cert and key files represent both parts of a certificate, key being the private key to the certificate and cert being the signed certificate. The creation of HTTPS server is a joint venture of the core https module and the express package.

import { readFile } from "node:fs/promises";
import https from "node:https";
import express from "express";
const key = await readFile("./certs/localhost.key", "utf8");
const cert = await readFile("./certs/localhost.crt", "utf8");

const app = express();
app.get("/", (req, resp) => resp.send("Hello from HTTPS!"));
https.createServer({ key, cert }, app).listen(3000);

The HTTPS server can be tested using curl with -k option (this is required as the certificate is self-signed):

:- curl https://localhost:3000 -k
Hello from HTTPS!

HTTP/2 server

HTTP/2 is a major revision of the HTTP network protocol used by the World Wide Web. It was derived from the earlier experimental SPDY protocol, originally developed by Google. Most major browsers had added HTTP/2 support by the end of 2015.[13] About 97% of web browsers used have the capability.[14] As of December 2022, 41% (after topping out at just over 50%) of the top 10 million websites supported HTTP/2. HTTP/2 is not a new protocol, but is quite popular.

The HTTP/2 server on Node.js is always secure. Also, HTTP/2 works with streams, therefore the request body gets presented as a stream event.

import http2 from "node:http2";
import { readFile } from "node:fs/promises";

const server = http2.createSecureServer({
  key: await readFile("./certs/localhost.key"),
  cert: await readFile("./certs/localhost.crt"),
});

server.on("error", (err) => console.error(err));
server.on("stream", (stream, headers) => {
  stream.respond({
    "content-type": "text/plain; charset=utf-8",
    ":status": 200,
  });
  stream.end("Hello World from HTTP/2!");
});
server.listen(3000);

The HTTP/2 server can be tested using curl:

:- curl https://localhost:3000 -kv
* ALPN: offers h2
* ALPN: offers http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN: server accepted h2
* Server certificate:
*  subject: C=US; ST=YourState; L=YourCity; O=Example-Certificates; CN=localhost.local
*  start date: Oct 21 16:28:58 2019 GMT
*  expire date: Sep 27 16:28:58 2118 GMT
*  issuer: C=US; CN=Example-Root-CA
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* Using HTTP2, server supports multiplexing
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* h2h3 [:method: GET]
* h2h3 [:path: /]
* h2h3 [:scheme: https]
* h2h3 [:authority: localhost:3000]
* h2h3 [user-agent: curl/7.86.0]
* h2h3 [accept: */*]
* Using Stream ID: 1 (easy handle 0x148814a00)
> GET / HTTP/2
> Host: localhost:3000
> user-agent: curl/7.86.0
> accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
< HTTP/2 200 
< content-type: text/plain; charset=utf-8
< date: Sun, 08 Jan 2023 22:48:25 GMT
< 
Hello World from HTTP/2

File I/O

Reading & writing files is a basic work for any programming language or runtime like Node.js. There is a core module ‘fs’, which contains exhaustive APIs to work with files and the file system. In this section, we’ll look at reading, writing, and appending files.

The required APIs from core ‘fs’ module can be imported directly:

import { appendFile, readFile, writeFile, stat } from "node:fs/promises";

The core fs module traditionally had only callback based APIs. However, Node.js understood the need to offer promise based APIs for better and cleaner code. The core fs module has both callback and promise based APIs. The promise APIs can be imported using fs/promises.

Reading file

To read a file as binary data, the readFile API can be used with just the file path:

const file = await readFile("/var/tmp/testdata/sample.txt");

The file variable is a Node.js Buffer containing the raw file data:

file = <Buffer 4c 65 61 72 6e 69 6e 67 20 4a 61 76 61 53 63 72 69 70 74 20 26 20 54 79 70 65 73 63 72 69 70 74 20 49 73 20 46 75 6e 21 0a>

To read a file as text, the same readFile API can be used with file path and an additional argument to specify the encoding of returned data (utf-8 is used to get file as a string).

const file = await readFile("/var/tmp/testdata/sample.txt", "utf-8");

The file variable is a JavaScript string containing the entire file contents:

file = Learning JavaScript & Typescript Is Fun!

Writing file

To write binary data into file, the writeFile API can be used with file path and then data to be written. Node.js understands the type of data and write it appropriately. The writeFile API doesn’t return anything. It does raise exception if writing fails for any reason.

const rawData = Buffer.from("Hello world!");
await writeFile("./out.txt", rawData);

//--
:- cat out.txt 
Hello world!

To write textual data (JavaScript string) into file, the same writeFile API can be used.

await writeFile("./out.txt", "Hello world from Node.js");

//--
:- cat out.txt 
Hello world from Node.js

Appending to file

Just like the writeFile API, any kind of data can be appended to a file using appendFile API. Node.js detects the type of data and writes it appropriately.

await appendFile("./out.txt", "\nThis is more data....");

//--
:- cat out.txt 
Hello world from Node.js
This is more data....

Exists

The stat API can be used to find if a file exists. The stat API throws an exception if file doesn’t exists or is inaccessible to the current user.

import { stat } from "node:fs/promises";

try {
  const fileInfo = await stat("/var/tmp/testdata/someRandomFile");
  console.log("File info=", fileInfo);
} catch (e) {
  console.log(e.message);
}

//--
ENOENT: no such file or directory, stat '/var/tmp/testdata/someRandomFile'

Child process

A child process is useful in running task outside the main application. The child process could be used for running a Node.js subprocess, any other language subprocess like Python, Deno, etc., run a shell command, or run any arbitrary shell script. The common use-cases involves spawning a child and waiting for it finish. Once the child finishes, the output and error data is collected back by the parent process.

The child process capabilities are provided by Node’s core child_process module (yes, there is an underscore in the name). There are a number of APIs, but we’ll be covering the two commonly used APIs:

Spawn

import { spawn } from "node:child_process";

The spawn API is still a callback/event based API. The first input is the executable path or name of the shell command. The second input is an array of strings for the arguments to the child process. Once the child process is spawned, events are emitted: data (to receive data from child), close (when the child process finishes). Node.js doesn’t provide any promise based equivalent of this API. This is one of the cases where we’ll have to use the callback based API. The following code runs a child process for a shell command ls -l /var/tmp.

import { spawn } from "node:child_process";

const ls = spawn("ls", ["-lh", "/var/tmp"]);

ls.stdout.on("data", (data) => {
  console.log(`stdout: ${data}`);
});

ls.stderr.on("data", (data) => {
  console.error(`stderr: ${data}`);
});

ls.on("close", (code) => {
  console.log(`child process exited with code ${code}`);
});

The output from a quick test is:

:- node app.mjs 
stdout: total 248
-rw-r--r--    1 mayankc        wheel   5.5K Dec  5 15:21 a.sh
-rw-r--r--    1 _windowserver  wheel    80K Dec  5 13:29 cbrgbc_1.sqlite
-rw-r--r--    1 _windowserver  wheel    32K Dec  5 13:29 cbrgbc_1.sqlite-shm
-rw-r--r--    1 _windowserver  wheel     0B Dec  5 13:29 cbrgbc_1.sqlite-wal
drwxr-xr-x    2 root           wheel    64B Dec  5 13:29 kernel_panics
drwxr-xr-x    5 mayankc        wheel   160B Jan  7 13:58 nodeProj
drwxr-xr-x  281 mayankc        wheel   8.8K Jan  2 12:54 perfResults
-rwxr-xr-x    1 mayankc        wheel    54B Dec 14 22:24 someScript.sh
lrwxr-xr-x    1 mayankc        wheel    48B Jan  7 12:12 testdata -> /Users/mayankc/Work/testdata

child process exited with code 0

It might be cumbersome to use this API. There is an NPM package: https://www.npmjs.com/package/@npmcli/promise-spawn which wraps the event based API into a promise API.

Exec

The other popular API for creating a child process is exec. Unlike spawn, the exec API takes a simple string (containing the entire command-line) as input, and needs a single callback for the returned data. Node.js doesn’t provide a promise equivalent of this API, but this can be promisified easily using core util module’s promisify API. The util.promisify API can convert any callback based API to its promise equivalent.

import { exec } from "node:child_process";
import { promisify } from "node:util";
const execP = promisify(exec);

const { stdout, stderr } = await execP("ls -lh /var/tmp");
console.log(`stdout: ${stdout}`);
console.error(`stderr: ${stderr}`);

The code using exec API is a clean one-liner. It’s equivalent to: run a child process and get the output. The output from a quick run is:

:- node app.mjs 
stdout: total 248
-rw-r--r--    1 mayankc        wheel   5.5K Dec  5 15:21 a.sh
-rw-r--r--    1 _windowserver  wheel    80K Dec  5 13:29 cbrgbc_1.sqlite
-rw-r--r--    1 _windowserver  wheel    32K Dec  5 13:29 cbrgbc_1.sqlite-shm
-rw-r--r--    1 _windowserver  wheel     0B Dec  5 13:29 cbrgbc_1.sqlite-wal
drwxr-xr-x    2 root           wheel    64B Dec  5 13:29 kernel_panics
drwxr-xr-x    5 mayankc        wheel   160B Jan  7 13:58 nodeProj
drwxr-xr-x  281 mayankc        wheel   8.8K Jan  2 12:54 perfResults
-rwxr-xr-x    1 mayankc        wheel    54B Dec 14 22:24 someScript.sh
lrwxr-xr-x    1 mayankc        wheel    48B Jan  7 12:12 testdata -> /Users/mayankc/Work/source/denoExamples/testdata

stderr:

HTTP Client

The modern Node.js comes with web fetch API that can be used to fetch remote resources. The web fetch API is a promise based API which provides web standard Request and Response interfaces.

The first example fetches Node.js’s home page:

const resp = await fetch("https://nodejs.org");
const respBody = await resp.text();
console.log("Resp code=", resp.status, ", length=", respBody.length);

// Resp code= 200 , length= 14692

The next example sends a JSON body in the POST request to postman’s echo API which will echo it back in the response:

const resp = await fetch("https://postman-echo.com/post", {
  method: "POST",
  body: JSON.stringify({
    name: "Mayank C",
  }),
  headers: {
    "content-type": "application/json",
  },
});
const respBody = await resp.json();
console.log("Resp code=", resp.status, ", rcvd JSON=", respBody);

The output of a quick run is:

Resp code= 200 , rcvd JSON= {
  args: {},
  data: { name: 'Mayank C' },
  files: {},
  form: {},
  headers: {
    'x-forwarded-proto': 'https',
    'x-forwarded-port': '443',
    host: 'postman-echo.com',
    'x-amzn-trace-id': 'Root=1-63bb18f4-4bc2c8371c55f32826775fee',
    'content-length': '19',
    'content-type': 'application/json',
    accept: '*/*',
    'accept-language': '*',
    'sec-fetch-mode': 'cors',
    'user-agent': 'undici',
    'accept-encoding': 'br, gzip, deflate'
  },
  json: { name: 'Mayank C' },
  url: 'https://postman-echo.com/post'
}

The last example sends a multipart-form data request to the postman echo API. The web standard FormData API is used. For files, a web standard Blob object populated from the file can be passed as a file field.

import { readFile } from "node:fs/promises";

const fd = new FormData();
fd.set("field1", "value1");
fd.set("field2", "value2");
fd.set(
  "file1",
  new Blob([await readFile("/var/tmp/testdata/sample.txt")]),
  "description.txt",
);
const resp = await fetch("https://myapi.free.beeceptor.com", {
  method: "POST",
  body: fd,
});
const respBody = await resp.json();
console.log("Resp code=", resp.status, ", rcvd JSON=", respBody);

The log from the beeceptor is as follows:

------formdata-undici-0.9225471752965178
Content-Disposition: form-data; name="field1"

value1
------formdata-undici-0.9225471752965178
Content-Disposition: form-data; name="field2"

value2
------formdata-undici-0.9225471752965178
Content-Disposition: form-data; name="file1"; filename="description.txt"
Content-Type: application/octet-stream

Learning JavaScript & Typescript Is Fun!

------formdata-undici-0.9225471752965178--

WebSockets

The WebSocket is an advanced technology that makes it possible to open a two-way interactive communication session between the user’s browser and a server. With this API, you can send messages to a server and receive event-driven responses without having to poll the server for a reply. WebSocket runs as an upgrade over existing HTTP protocol.

Node.js doesn’t come with built-in support for either WebSocket server or client. There are two popular options: 1) Use the ws NPM module 2) Use socket.io for real time communications. We’ll be using ws module in this article.

:- npm i ws
added 1 package, and audited 60 packages in 1s

The ws module exposes very similar interfaces for the client and the server. There are three interesting events in the lifecycle of a websocket:

Connection with remote (open)
Exchange of data (message)
Closing connection (close)

The following is the code of a websocket echo server which simply echoes back all the messages received from a client. The client establishes connection with server, sends a single message, closes connection on receiving a response from the server:

client.mjs

import WebSocket from "ws";

const ws = new WebSocket("ws://localhost:3000");

ws.on("open", () => {
  console.log("Connection established with server");
  const msg = Date.now() + " some message";
  console.log("sending: %s", msg);
  ws.send(msg);
});

ws.on("message", (data) => {
  console.log("received: %s", data);
  ws.close();
});

ws.on("close", () => {
  console.log("Connection closed with server");
});

server.mjs

import { WebSocketServer } from "ws";

const wss = new WebSocketServer({ port: 3000 });
wss.on("connection", (ws) => {
  console.log("Client has connected");
  ws.on("message", (data) => {
    console.log("Echoing: %s", data);
    ws.send(data);
  });
  ws.on("close", () => console.log("Client has disconnected"));
});

The following is the log of a run between client and server:

:- node client.mjs 
Connection established with server
sending: 1673239534485 some message
received: 1673239534485 some message
Connection closed with server

:- node server.mjs 
Client has connected
Echoing: 1673239534485 some message
Client has disconnected

OS & Process

Node.js exposes process level details and APIs through the core process module. Also, the process module is available in the global space. Node.js exposes OS level details through the core os module. The OS module needs to be imported.

import process from "node:process";
import os from "node:os"

On exit

A callback can be registered to be called whenever the Node process exists.

console.log("Process is up", new Date());
setTimeout(() => {}, 5000);
process.on("exit", (code) => {
  console.log("Process is exiting", new Date());
});

// --
:- node app.mjs 
Process is up 2023-01-08T22:55:40.411Z
Process is exiting 2023-01-08T22:55:45.422Z

Unhandled rejection

A similar callback can be registered whenever there is a rejected promise without a handler.

console.log("Process is up, at", new Date());
setTimeout(() => {
  Promise.reject("Rejecting for demonstration");
}, 5000);
process.on("unhandledRejection", (_, p) => {
  console.log("Process caught exception:", p, ", at", new Date());
});

//--
:- node app.mjs 
Process is up, at 2023-01-08T22:59:41.834Z
Process caught exception: Promise { <rejected> 'Rejecting for demonstration' } , at 2023-01-08T22:59:46.844Z

Arch & platform

The processor architecture & platform is available in process.arch & process.platform:

> process.arch
'arm64'

> process.platform
'darwin'

CPU usage

The CPU usage is available in process.cpuUsage:

> process.cpuUsage()
{ user: 253248, system: 67359 }

Memory usage

The memory usage is available in process.memoryUsage:

> process.memoryUsage()
{
  rss: 47202304,
  heapTotal: 6815744,
  heapUsed: 5377904,
  external: 1100614,
  arrayBuffers: 10327
}

Environment variables

The environment variables are available as a plain JavaScript object in process.env:

> process.env
{
  NVM_INC: '/Users/mayankc/.nvm/versions/node/v10.23.3/include/node',
  MANPATH: '/Users/mayankc/.nvm/versions/node/v10.23.3/share/man:/opt/homebrew/share/man:',
  ...
  _: '/opt/homebrew/bin/node',
  __CF_USER_TEXT_ENCODING: '0x1F5:0x0:0x0'
}

Own pid

The process.pid variable contains own process id:

> process.pid
75044

Parent pid

The parent process id is available in process.ppid:

> process.ppid
59578

Uptime

The time in seconds since process has started is available in process.uptime:

> process.uptime()
544.146383166

CPUs

The CPU information can be obtained from OS module’s cpu API. The return from API is a pretty detailed object. The number of CPUs is the number of keys in the detailed object.

import os from "node:os";
console.log("Number of CPUS=", Object.keys(os.cpus()).length);

//--
Number of CPUS= 8

Temporary directory

The os.tmpDir API returns the temporary directory that can be used to temporarily store files.

console.log("Path to temporary directory=", os.tmpdir());

//Path to temporary directory= /var/folders/hx/0f6tcxq52f396vfq0c8xfszc0000gn/T

Hostname

The hostname of the server is provided by the os.hostname API:

console.log("Hostname=", os.hostname());

//Hostname= MacBook-Pro-2.local

Web APIs

Deno started the trend of supporting web APIs in their runtime. This made it pretty easy for full stack developers to write server-side code. The same APIs that they were always using in the browser are now available on the server-side. Recently, Node.js has also started offering functionality through web APIs. It’s a work in progress.

The web APIs are available in the global scope. There is no need to import anything to use them.

Base 64

To convert a string to base64, web standard btoa API can be used:

> btoa("Hello world!");
'SGVsbG8gd29ybGQh'

To convert a base64 string to string, web standard atob can be used:

> atob("SGVsbG8gTm9kZS5qcyE=")
'Hello Node.js!'

Random ID

To generate a v4 random ID, web standard crypto.randomUUID API can be used:

> crypto.randomUUID()
'02f00439-1b21-4231-808a-7603902cca74'

Hashing

The web standard crypto.subtle.digest API can be used to create any kind of secure hash (SHA-1 to SHA-512). There are two inputs: 1) The hashing algorithm 2) The data to hash. The output is an ArrayBuffer which needs a view like Uint8Array to work with it.

> await crypto.subtle.digest("SHA-1", "Hello world!");
ArrayBuffer {
  [Uint8Contents]: <d3 48 6a e9 13 6e 78 56 bc 42 21 23 85 ea 79 70 94 47 58 02>,
  byteLength: 20
}

> await crypto.subtle.digest("SHA-512", "Hello world!");
ArrayBuffer {
  [Uint8Contents]: <f6 cd e2 a0 f8 19 31 4c dd e5 5f c2 27 d8 d7 da e3 d2 8c c5 56 22 2a 0a 8a d6 6d 91 cc ad 4a ad 60 94 f5 17 a2 18 23 60 c9 aa cf 6a 3d c3 23 16 2c b6 fd 8c df fe db 0f e0 38 f5 5e 85 ff b5 b6>,
  byteLength: 64
}

It is very common to convert the hash output to hex string so that it can be sent to a remote party. The following code generates a hex string for the hash of a given data:

const input = "Testing the hashing";
const rawSha1 = await crypto.subtle.digest("SHA-1", input);
const rawSha5 = await crypto.subtle.digest("SHA-512", input);

const bufToHex = (buf) =>
  [...new Uint8Array(buf)].map((x) => x.toString(16).padStart(2, "0")).join("");

console.log("SHA-1 hex:", bufToHex(rawSha1));
console.log("SHA-512 hex:", bufToHex(rawSha5));

//--
SHA-1 hex: 1c133b693a7903fb59b9ba4a04caaef4ad7f4604
SHA-512 hex: 602b4fc7614538e8338309c7ea94c86f90be9c9ffa56a31caf97d3289d8c163a5c619d6e9ce24826d4698e5546b494d08668db8bcbaa2704d76664df687c56e6

String to binary (and vice-versa)

The web standard TextEncoder and TextDecoder APIs can be used to convert a string to bytes and vice-versa.

const td = new TextDecoder(), te = new TextEncoder();
const data1 = "Testing the encoder";
const data2 = te.encode("Testing the decoder");

console.log("data1 bytes=", te.encode(data1));
console.log("data2 string=", td.decode(data2));

//--
data1 bytes= Uint8Array(19) [
   84, 101, 115, 116, 105, 110,
  103,  32, 116, 104, 101,  32,
  101, 110,  99, 111, 100, 101,
  114
]
data2 string= Testing the decoder

Auto-reloading

While developing a Node.js application, it is cumbersome to restart the application everytime there is a change (which is very often during development phase). To make it easy for developers, Node.js comes with a built-in watcher that reloads the application as soon there is a change in the provided watch path or any of the dependent modules.

app.mjs

import express from "express";

const app = express();

app.get("/", (req, resp) => {
  resp.send("Hello world!");
});

app.listen(3000);

The application is started with — watch option:

:- node --watch app.mjs
(node:82044) ExperimentalWarning: Watch mode is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)

Now as soon a change is made in app.mjs file, the watcher will reload the application:

Debugging in VSCode

Visual Studio Code was ranked the most popular IDE by StackOverflow, the world’s biggest programming Q&A site. Chances are very high that VS Code would be used for developing Node.js applications. To speed up development & debugging, VS Code comes with a built-in excellent Node.js debugger.

First, we write a simple hello world express app in VS Code editor:

To run the application, go to Run -> Start Debugging. If it’s first time debugging, VS Code could ask for run configuration, choose Node.js in the list.

When the debugging session starts, the VS Code makes a copy of the code and runs it inside the VS Code debugger:

The orange bar at the bottom indicates that the server is running. Now, to make an HTTP request, open a terminal using Terminal -> New terminal. This starts a new terminal at the bottom:

The server is responding to the requests. Now, to start debugging, we’ll add a breakpoint at the line where HTTP response is prepared.

Now, we can run the curl command again. This time the server pauses at line number 5.

On the left hand panel, we can check the variables: req & res. Whenever we’re done debugging, the play button can be used to continue the application.

Performance

The last section of this article covers Node.js’s performance for a simple hello world HTTP server use-case. This is not a comparison with competitors. This is about how fast Node.js can process requests when there are:

10 concurrent connections
50 concurrent connections
100 concurrent connections
200 concurrent connections

The test is executed for Node’s native hello world server (we’ve seen it earlier) on MacBook Pro M1 with 16G RAM. We cover a variety of measurements ranging from requests per second, mean, median, minimum, maximum, average CPU, and average memory. A total of 1M requests are executed for each test.