Smelly Code

NodeJs with(out) Express ~ Part 1

May 27, 2019 ∙ 👩🏻‍💻 7 min

Learning a new tech stack is always challenging. It requires patience and a right set of materials. Material which not only scratches the surface but also helps people with relevant references to level up their skills. A few weeks ago, I started exploring NodeJs to learn server-side programming. I wanted to understand the underlying mechanics of basic stuff like authentication, session management, HTTP transactions, etc. So I googled keywords like “How to implement authentication in NodeJs”, “Session Management in NodeJs”, “Handle post request in NodeJs”, etc. A large number of articles/links appeared respective to each of these. They all were using one framework or another. Most of them were using Express. I was looking for stuff with plain NodeJs. So I asked people if there’s any source available which implements such stuff without any framework. I didn’t get any response. So I decided to peel the layers of abstraction by myself and write one. This series of articles is all about how one can implement things without using any framework. It will parallelly demonstrate certain implementations in ExpressJs. Let’s get started with a basic HTTP server and a few HTTP transactions.

HTTP Server

NodeJs has a native http module to create an HTTP server. It also provides modules for other protocols like UDP(dgram). For brevity, throughout this series we’ll stick to HTTP. Here’s a simple HTTP node server.

// Importing http module to create a server.
const http = require('http');

// Host for our server.
const HOST = 'localhost';

// Port at which our server will run/listen.
const PORT = 3000;

// Creating a server with handler.
const server = http.createServer((req, res) => {
  res.statusCode = 200;
  res.setHeader('Content-Type', 'text/plain');
  res.end('Hello Stranger!!');
});

// Setup server at PORT.
server.listen(PORT, HOST, () => {
  console.log(`Server is running at http://${HOST}:${PORT}`);
});

To create a server, we have used the http.createServer api with a request handler. After creating a server instance, we set it up to port 3000 with hostname localhost. All the request made to localhost:3000 will be handled by our server. The job of request handler is to receive and process the incoming requests and respond with appropriate data or message(s). In the snippet above, the request handler sets the status code to 200 and sends Hello Stranger!! greeting as plain text.

Express version of the above snippet:

// Importing express module.
const express = require('express');

// Port at which our server will listen.
const PORT = 3001;

// Creating an express app.
const app = express();

// Setting up request handler also known as middleware
app.use((req, res) => {
  res.statusCode = 200;
  res.setHeader('Conent-Type', 'text/plain');
  res.send('Hello Stranger!!');
});

// Setup server at PORT.
app.listen(PORT, () => {
  console.log(`Server is running at ${PORT}`);
});

Express version more or less looks same as the native version. But a lot is happening under the hood. We’ll keep peeking as we move forward. At this point, express has abstracted the server creation for us.

Snapshot of the source code of app.listen taken from the repo:

app.listen = function listen() {
  var server = http.createServer(this);
  return server.listen.apply(server, arguments);
};

It has also introduces a new way of registering handlers/middlewares using app.use. Middleware is basically a function which gets invoked with request, response and next arguments in request-response cycle. You can read more on middlewares here.

Here’s the link for the repo, if you want to play with the code.

HTTP Transaction

Any call made by a client to a server with HTTP protocol is considered as HTTP transaction. HTTP has various transaction types—widely known as HTTP Methods.

When a server receives a request, it calls the request handler with request and response parameters.

  • request is an instance of IncomingMessage which is a ReadableStream. It has all the neccessary inforamtion like method, url, path, parameter, etc.

  • response is an instance of ServerResponse which is a WritableStream. It helps us to send data back to the client.

Let’s quickly introduce a /greet endpoint which will take a query param name and return a greeting. eg. hitting /greet?name=Sheldon will return Hello Sheldon!!.

//... other require statements.
const url = require('url');

// ...necessary variable decalrations
// ............

// Request handler for /greet end point.
const handleGreetRequest = (req, res) => {
  const { method, url: reqUrl } = req;
  const urlParts = url.parse(reqUrl, true);
  if (method === 'GET') {
    const { query: queryParams } = urlParts;
    res.end(`Hello ${queryParams.name || 'Stranger'}!`);
  } else {
    res.statusCode = 404;
    res.end('Not found.');
  }
};

// Generic request handler.
const onRequest = (req, res) => {
  if (req.url.startsWith('/greet')) {
    handleGreetRequest(req, res);
  } else {
    res.statusCode = 404;
    res.end('Not found.');
  }
};

// Creating a server with handler.
const server = http.createServer(onRequest);

// ... server setup code
// ...............

We have defined the request handler as onRequest function. It invokes the respective request handler depending on the URL. If the URL doesn’t match to any handler then 404 is sent to the client. onRequest function hands over requests made with /greet endpoint to handleGreetRequest function. handleGreetRequest parses the url—using url module—to get query params, creates a greeting with the value of name param in query and sends it to the client. The full code for the above snippet can be found here.

Let’s introduce another endpoint called /echo. It will be POST call which will send the request body as is back to the client .

// .... require statements and some other declaration statements
// ........

const handleEchoRequest = (req, res) => {
  if (req.method === 'POST') {
    let body = [];
    req
      .on('data', chunk => {
        body.push(chunk);
      })
      .on('end', () => {
        body = Buffer.concat(body);
        res.end(body);
      });
  } else {
    res.statusCode = 404;
    res.end('Not found.');
  }
};

const onRequest = (req, res) => {
  if (req.url.startsWith('/echo')) {
    handleEchoRequest(req, res);
  } else {
    res.statusCode = 404;
    res.end('Not found.');
  }
};

// Creating a server with handler.
const server = http.createServer(onRequest);

// ... server setup code
// ...............

As we know that data are transferred as packages or chunks over the network, so we must set up an accumulator which collects the data as soon as they arrive. That’s why we have set up a data event listener which pushes chunks to body array. We’ve also added an end event listener which gets triggered when all chunks are received. Here we collate chunks and send back to the client.

If you look at the above snippets you might realize that we are doing so many other things instead of focusing on the core logic of endpoints. Also, the retrieval of data from the request is a tad tedious. That’s why ExpressJs and many other frameworks/modules have got a lot of traction because they remove the clutter and let developers focus on core business logics.

Let’s write both /greet and /echo endpoint using Express.

const express = require('express');
const bodyParser = require('body-parser');

const PORT = 3001;

const app = express();

app.use(bodyParser.urlencoded({ extended: false }));
app.use(bodyParser.json());

app.get('/greet', (req, res) => {
  const {
    query: { name }
  } = req;
  res.send(`Hello ${name || 'Stranger'}!`);
});

app.post('/echo', (req, res) => {
  res.send(req.body);
});

// Setup server at PORT.
app.listen(PORT, () => {
  console.log(`Server is running at ${PORT}`);
});

Express version looks much cleaner because has abstracted the nitty gritty of HTTP transactions and exposed apis relevant to HTTP methods. We have also used a body-parser module which acts as a middleware to accumulates data from the request and make them available as request.body to subsequent middlewares.

Conclusion

Creating applications with plain NodeJs requires a little extra effort cause we directly deal with native apis. Sometimes, it can be a bit overwhelming and tedious. Frameworks like ExpressJs remove these pain points and help us write applications in a more elegant way with seamless development experience. But it is also important to be familiar with things lying under the hood. So that frameworks can be leveraged!

Hi, I am Hitesh.

|