By Srinivasan Subramanian
Web application development is not what it used to be, even a couple of years back. Today, there are so many options, and the uninitiated are often confused about what’s good for them. Not just the broad stack (the various tiers or technologies used), but even for tools that aid in development, there are many choices. This book stakes a claim that the MERN stack is great for developing a complete web application, and takes the reader through all that is necessary to get that done.
In this article, I’ll give a broad overview of the technologies that the MERN stack consists of. This article will focus on how these concepts affect an evaluation of whether MERN is a good choice for your next web application project.
Any web application is made using multiple technologies. The combinations of these technologies is called a “stack”, popularized by the LAMP stack, which is an acronym for Linux, Apache, MySQL, PHP, all open-source components. As web development matured and their interactivity came to the fore, Single Page Applications (SPAs) became more popular. An SPA is a web application paradigm that avoids refreshing a web page to display new contents instead uses lightweight calls to the server to get some data or snippets and updates the web page. The result looks quite nifty as compared to the old way of reloading the page entirely.This brought about a rise in front-end frameworks, since much of the work was done on the client side. At approximately the same time, though completely unrelated, NoSQL databases also started gaining popularity.
Although we pick a few defining technologies to define a stack, these are not enough to build a complete web application. Other tools are required to help the process of development, and other libraries are needed to complement React. This book is about all of this – how to build a complete web application based on the MERN stack, using other complementary tools that make it easy for us to do it.
I'll give a quick introduction to the main components that form the MERN stack, and a few other libraries and tools that we'll be using to build our web application. I'll just touch upon the salient features, and leave the details to chapters in the book where they are more appropriate.
React anchors the MERN stack. In some sense, this is the defining component of the MERN stack.
I'll discuss a few things about React that makes it stand out.
Why Facebook Invented React
Facebook folks built react for their own use, and later they open-sourced it. Now, why did they have to build a new library when there are tons of them out there?
React was born not in the Facebook application that we all see, rather in Facebook's Ads organization. Originally, they used a typical client-side MVC model to start with, which had all the regular two-way data binding and templates. Views would listen to changes on models, and they would respond to those changes by updating themselves.
Soon, this got pretty hairy as the application became more and more complex. What would happen was that a change would cause an update, that would cause another update (because something changed due to that update), which would cause yet another and so on. Such cascading updates became difficult to maintain, because there would be subtle difference in the code to update the view, depending on the root cause of the update.
Then they thought, why do we need to deal with all this, when all the code to depict the model in a view is already there? Aren't we replicating the code by adding smaller and smaller snippets to manage transitions? Why can't we use the templates (that is, the views) themselves to manage state changes?
That's when they started thinking of building something that's declarative rather than imperative.
React views are declarative. What this really means is that you, as a programmer, don't have to worry about managing the effect of changes in the view's state or the data. In other words, you don't worry about transitions or mutations in the DOM caused by changes to the view's state. How does this work?
A React component declares how the view looks like, given the data. When the data changes, if you are used to the jQuery way of doing things, you'd typically do some DOM manipulation. Not in React. You just don't do anything! The React library figures out how the new view looks like, and just applies the changes between the old view and the new view. This makes the views consistent, predictable and easier to maintain and simpler to understand.
Won't this be too slow? Will it not cause the entire screen to be refreshed on every data change? Well, React takes care of this using its Virtual DOM technology. You declare how the view looks like not in the form of HTML or a DOM, but in the form of a virtual representation, an in-memory data structure. React can compute the differences in the Virtual DOM very efficiently, and can apply only these changes to the actual DOM. Compared to manual updates which does only the required DOM changes, this adds very little overhead because the algorithm to compute the differences in the Virtual DOM has been optimized to the hilt.
The fundamental building block of React is a Component that maintains its own state and renders itself.
In React, all you do is build components. Then, you put components together to make another component that depicts a completeview or page. A component encapsulates the state of data and the view, or how it is rendered. This makes writing and reasoning about the entire application easier, by splitting it into components and focusing on one thing at a time.
Components talk to each other by sharing state information in the form of read-only properties to their child components and by call backs to their parent components.
Manyweb application frameworks rely on templates to automate the task of creating repetitive HTML or DOM elements. The templating language in these frameworks is something that the developer will have to learn and practice. Not in React.
React can be run on the server too. That’s what isomorphic means: the same code can run on both server and the browser.
Node.js ships with a bunch of core modules compiled into the binary. These modules provide access to the operating system elements such as the file system, networking, input/output, etc. They also provide some utility functions that are commonly required by most programs.
Apart from your own files and the core modules, you can also find a great amount of third party open-source libraries available for easy installation. That brings us to npm.
Node.js and npm
npm is the default package manager for Node.js. You can use npm to install third party libraries (packages) and also manage dependencies between them. The npm registry (www.npmjs.com) is a public repository of all modules published by people for the purpose of sharing.
As of writing this book, npm tops the list of module or package repository, having more than 250,000 packages (source: http://www.modulecounts.com). Maven, which used to be the biggest two years back, has just half the number now. This shows that npm is not just the largest, but also the fastest growing repository. It is often touted that the success of Node.js is largely owed to npm and the module ecosystem that has sprung around it.
npm is not just easy to use both for creating and using modules, but it has a unique conflict resolution technique that allows multiple conflicting versions of a module to exist side-by-side to satisfy dependencies. Thus, in most cases, npm just works.
Node.js is Event Driven
Node.js has an asynchronous, event driven, non-blocking input/output (I/O) model, as opposed to using threads to achieve multi-tasking.
Most other languages depend on threads to do things simultaneously. But in fact, there is no such thing as simultaneous when it comes to a single processor running your code. Threads give the feeling of simultaneous by letting other pieces of code run while one piece waits (blocks) for some event to complete. Typically, these are I/O events such as reading from a file or communicating over the network. For a programmer, this means that you write your code sequentially. For example, on one line, you make a call to open a file, and on the next line, you have your file handle ready. What really happens is that your code is blocked while the file is being opened. If you have another thread running, the operating system or the language canswitch out this code and start running some other code during the blocked period.
Node.js, on the other hand, has no threads. It relies on callbacks to let you know that a pending task is completed. So, if you write a line of code to open a file, you supply it with a callback function to receive the results. On the next line, you continue to do other things that don't require the file handle. If you are used to asynchronous Ajax calls, you will immediately know what I mean. Event driven programming is natural to Node.js due to the underlying language constructs such as closures.
Node.js achieves multi-tasking using an event loop. This is nothing but a queue of events that need to be processed, and callbacks to be run on those events. In the above example, the file being ready to be read will be an event which will trigger the callback you supplied while opening it. If you don’t understand this completely, don’t worry. The examples in the rest of this book should make you comfortable about how it really works.
On the one hand, an event based approach makes Node.js applications fast and lets the programmer be blissfully oblivious of semaphores and locks that are utilized to synchronize multi-threaded events. But on the other hand, getting used to this model takes some learning and practice.
The Express framework lets you define routes, specifications of what to do when a HTTP request matching a certain pattern arrives. The matching specification is regular-expression (regex) based and is very flexible, like most other web application frameworks. The what-to-do part is just a function that is given the parsed HTTP request.
Express parses request URL, headers and parameters for you. On the response side, it has, as expected, all functionality required by web applications. This includes setting response codes, setting cookies, sending custom headers etc. Further, you can write Express middleware, custom pieces of code that can be inserted in any request / response processing path to achieve common functionality such as logging, authentication etc.
Express does not have a template engine built in, but it supports any template engine of your choice such as pug, mustache etc. But, for an SPA, you will not need to use a server side template engine. This is because all dynamic content generation is done on the client, and the web server only serves static files and data via API calls. Especially with MERN stack, page generation is handled by React itself on the server side.
In summary, Express is a web server framework meant for Node.js, not very different from many other server-side frameworks in terms of what you can achieve with it.
MongoDB is the database used in the MERN stack. It is a NoSQL document-oriented database, with a flexible schema and a JSON based query language. I'll discuss a few things that MongoDB is (and is not) here.
NoSQL stands for “non-relational”, no matter what the acronym expands to. It's essentially not a conventional database where you have tables and columns, called relational databases. I find that there are two attributes of NoSQL that differentiate them from the conventional.
The first is their ability to horizontally scale by distributing the load over multiple servers. They do this by sacrificing an important (for some) aspect of the traditional databases: strong consistency, that is, the data is not necessarily consistent for very brief amounts of time across replicas. For more information, look up and read up on the “CAP theorem” (https://en.wikipedia.org/wiki/CAP_theorem). But in reality, very few applications require web-scale, and this aspect of NoSQL databases comes into play very rarely.
The second, according to me more important, aspect is that NoSQL databases are not necessarily relational databases. You don't have to think of your data in terms of rows and columns of tables. The difference in the representation in the application and on disk, is sometimes called Impedance Mismatch. This is a term borrowed from Electrical Engineering,which means, roughly, that we’re not talking the same language. In MongoDB, instead, you can think of the persisted data just as you see them in your application code, that is, as objects or documents. This helps a programmer avoid a translation layer, whereby one has to convert or map the objects that the code deals with to relational tables. Such translations are called Object Relational Mapping (ORM) layers.
Compared to relational databases where data is stored in the form of relations, or tables, MongoDB is a document-oriented database. The unit of storage (comparable to a row) is a document, or an object, and multiple documents are stored in collections (comparable to a table). Every document in a collection has a unique identifier using which it can be accessed. The identifier is indexed automatically.
Imagine the storage structure of an invoice, with the customer name, address etc. and a list of items (lines) in the invoice. If you had to store this in a relational database, you would use two tables, say, invoice and invoice_lines, with the lines or items referring to the invoice via a foreign-key relation. Not so in MongoDB. You would store the entire invoice as a single document, fetch it and update it in an atomic operation. This applies not just to line items in an invoice. The document can be any kind of deeply nested object.
Modern relational databases have started supporting one level of nesting by allowing array fields and JSON fields, but it is not the same as a true document database. MongoDB has the ability to index on deeply nested fields, which relational databases cannot.
The downside is that the data is stored de-normalized. This means that data is sometimes duplicated, requiring more storage space. Also, things like renaming a master (catalog) entry name would mean sweeping through the database. But then, storage has become relatively cheap these days, and renaming master entries are rare operations.
Storing an object in a MongoDB database does not have to follow a prescribed schema. All documents in a collection need not have the same set of fields.
This means that, especially during early stages of development, you don't need to add/rename columns in the schema. You can quickly add fields in your application code without having to worry about database migration scripts. At first this may seem a boon, but in effect all it does is that it transfers the responsibility of data sanity from the database to your application code. I find that in larger teams and more stable products, it is better to have a strict or semi-strict schema. Using Object Document Mapping libraries such as mongoose (not covered in this book) alleviates this problem.
For relational databases, we had a query language called SQL. For MongoDB, the query language is based on JSON: you create, search for, make changes, delete documents by specifying the operation in a JSON object. The query language is not English-like (you don't SELECT or say WHERE), and therefore much easier to construct programmatically.
Data is also interchanged in JSON format. In fact, the data is natively stored in a variation of JSON called BSON (where B stands for Binary) in order to efficiently utilize space. When you retrieve a document from a collection, it is returned as a JSON object.
Tools and Libraries
It's hard to build any web application without using tools to help you on your way. Here's a brief introduction to the other tools apart from the MERN stack components that we will be using to develop our sample application in this book.
React gives us only the View rendering capability and helps manage interactions in a single component. When it comes to transitioning between different views of the component and keeping the browser URL in sync with the current state of the view, we need something more.
This capability of managing URLs and history is called routing. This is similar to server side routing that Express does: a URL is parsed and based on its components a piece of code is associated with the URL. React-Router not only does this, but also manages the browser’s Back button functionality so that we can transition between what seem as pages without loading the entire page from the server. We could have built these ourselves, but React-Router is a very easy-to-use library that manages this for us.
Bootstrap, the most popular CSS framework, has been adapted to React and the project is called React-Bootstrap. This library not only gives us most of the Bootstrap functionality, but the components and widgets provided by this library also give us a wealth of information on how to design our own widgets and components.
There are other component/CSS libraries built for React (such as Material-UI, MUI, Elemental UI etc.) and also individual components (such as react-select, react-treeview and react-date-picker). All these are good choices too, depending on what you are trying to achieve. But I found that React-Bootstrap is the most comprehensive single library with the familiarity of Bootstrap (which I presume most of you will already be familiar with).
This tool is indispensable when it comes to modularizing code. There are other competing tools such as Bower and Browserify which also serve the purpose of modularizing and bundling all the client code, but I found that webpack is easier to use and does not require another tool (like gulp or grunt) to manage the build process.
Very often, we'll feel the need for a library to address a seemingly common problem that all of us would face. In this book, we'll use body-parser (to parse POST data in the form of JSON, or form data), eslint (for ensuring our code follows conventions) and express-session, all on the server side, and some more like react-select on the client side.
So, we have a fair idea of the MERN stack and what it is based on. But is it really far superior to any other stack, say, LAMP, MEAN, J2EE etc.? By all means, all these stacks are good enough for most modern web applications. All said and done, familiarity is the crux of productivity in software, so I wouldn't advise a MERN beginner to blindly start their new project on MERN, especially if they have an aggressive deadline. I'd advise them to choose the stack that they are already familiar with.
But MERN does have its special place. It is ideally suited for web applications that have a large amount of interactivity built into the front-end. Go back and re-read the section on “Why Facebook built React”, it will give you some insights. You could perhaps achieve the same with other stacks, but you'll find that it is most convenient to do so with MERN. So, if you do have a choice of stacks, and the luxury of a little time to get familiar, you may find that MERN is a good choice. I'll talk about a few things that I like about MERN, and these may help you decide.
Apart from the obvious advantage of not having to switch contexts while writing client side and server side code, having a single language across tiers also lets you share code between these. I can think of functions that execute business logic, do validation etc. that can be shared. They need to be run on the client side so that user experience is better by being more responsive to user inputs. They also need to be run on the server side to protect the data model.
I have found that this often saves me a lot of hassle in terms of transformations. No Object Relational Mapping (ORM), not having to force fit an object model into rows and columns, no special serializing and de-serializing code. An Object Document Mapper(ODM) such as mongoose may help enforce a schema and make things even simpler, but the bottom line is that you save a lot of data transformation code.
Further, it just lets me think in terms of native objects, and see them as their natural selves even when inspecting the database directly using a shell.
Due to its event driven architecture and non-blocking I/O, the claim is that Node.js is very fast and a resilient web server.
Although it takes a little getting used to, I have no doubt that when your application starts scaling and receiving a lot of traffic, this will play an important role in cutting costs and savings in terms of time spent in trouble-shooting server CPU and I/O problems.
The npm Ecosystem
I've already discussed about the huge number of npm packages available freely for everyone to use. Any problem that you face that you think others should have faced too, you'll find that there is an npm package for that. Even if it doesn't fit your needs exactly, you can fork it and make your own npm package.
SPAs used to have the problem that they were not SEO friendly. One had to use workarounds like running PhantomJS on the server to pseudo-generate HTML pages, or use Prerender.io services that did the same for you. These introduce an additional complexity.
With the MERN stack, serving pages out of the server is natural and doesn't require tools which are after-thoughts. This is made possible due to the Virtual DOM technique used by React. Once you have a virtual DOM, the layer that translates it to a renderable page can be abstracted. For the browser, it is the real DOM. For the server-side, it is HTML. In fact, React Native has taken it to another extreme: it can even be a mobile app!
I don't cover React Native in this book, but this should give you a feel of what Virtual DOM can do for you in future.
It's not a Framework!
Not many people like or appreciate this, but I really like the fact that React is a library, not a framework.
A framework is opinionated, has a set way of doing things. The framework asks you to fill in variations of what it thinks all of us want to get done. A library, on the other hand, gives you tools using which you construct your application. In the short term, a framework helps a lot by getting most of the standard stuff out of the way. But over time, vagaries of the framework, its assumptions about what we want to get done, and the learning curve will make you wish you had some control on what's happening under the hood, especially when you have some special requirements.
With a library, an experienced architect can design his or her application with complete freedom to pick and choose from the library's functions, and build their own framework that fits their application's unique needs and vagaries. So, for an experienced architect or very unique application needs, a library is better, even though a framework can get you started quickly.
Pro MERN Stack lets you experience what it takes, and what it is like, to develop an application using the MERN stack. The work that we will do as part of this book encourages thinking and experimenting rather than reading. That’s why I have a lot of examples, at the same time, there are exercises that make you think. Finally, it uses the least common denominator to get this done: the CRUD app.
If you are game, read on. Code ahoy!
About the Author
Srinivasan (Vasan) Subramanian grew up in various cities in India and decided to settle in Bengaluru where he started his career. He joined Accel in 2013.
Vasan is a technologist who has had a long career building software products and leading engineering teams. He has worked as the engineering head in Barracuda Networks and Insta Health Solutions before Accel. He is passionate about best practices in Web technologies, software processes and security. He likes to build Android apps in his spare time.
He works closely with early stage startups in the Accel portfolio, consulting, mentoring and helping them on all things tech.
Vasan studied electronics and communication at IIT Madras and holds an MBA from IIM Bangalore.