A fundamental part of Node is the event loop, a concept underlying the behavior of JavaScript as well as most other interactive systems. In many languages, event models are bolted onto the side, but JavaScript events have always been a core part of the language. This is because JavaScript has always dealt with user interaction. Anyone who has used a modern Web browser is used to Web pages that do things "onclick," "onmouseover," etc. These events are so common that we hardly think about them when writing Web page interaction, but having this event support in the language is incredibly powerful. On the server, instead of the limited set of events based on the user-driven interaction with the Web page's DOM, we have an infinite variety of events based on what's happening in the server software we use. For example, the HTTP server module provides an event called "request," emitted when a user sends the Web server a request.

The event loop is the system that JavaScript uses to deal with these incoming request from various parts of the system in a sane manner. There are a number of ways people deal with "real-time" or "parallel" issues in computing. Most of them are fairly complex and frankly make our brains hurt. JavaScript takes a simple approach that makes the process much more understandable but does introduce a few constraints. By having a grasp of how the event loop works, you'll be able to use it to it's full advantage and avoid the pitfalls of this approach.

Node takes the approach that all I/O activities should be non-blocking (for reasons we'll explain more later). This means that HTTP requests, database queries, file I/O, and other things that require the program to wait do not halt execution until they return data. Instead, they run independently, and then emit an event when their data is available. This means that programming in Node.js has lots of callbacks dealing with all kinds of I/O. Callbacks often initiate other callbacks in a cascading fashion. This is a very different from browser programming. There is still a certain amount of linear setup, but the bulk of the code involves dealing with callbacks.

Because of this somewhat unfamiliar programming style, we need to look for patterns to help us effectively program on the server. That starts with the event loop. I think that most people intuitively understand event driven programming because it is like everyday life. Imagine you are cooking. You are chopping a bell pepper and a pot starts to boil over (Figure 3.1, “Event Driven People”). You finish the slice you are doing, and then turn down the stove. Rather than trying to chop and turn down the stove at the same time, you achieve the same result in a much safer manner by rapdily switching contexts. Event driven programming does the same thing. By allowing the programmer to write code that only ever works on one callback at a time, the program is both understandable but also able to quickly perform many tasks efficiently.

Figure 3.1. Event Driven People

In every day life we are used to having all sorts of internal callbacks for dealing with events, and yet, like JavaScript, we always do just one thing at once. Yes, yes, I can see you are rubbing your tummy and patting your head at the same time, well done. But, if you try to do any serious activities at the same time, it goes wrong pretty quick. This is like JavaScript. It's great at letting events drive the action, but it's "single-threaded" so that only one thing happens at once.

This single-threaded concept is really important. One of the criticisms leveled at Node.js fairly often is its lack of "concurrency." That is, it doesn't use all of the CPUs on a machine to run the JavaScript. The problem with running code on multiple CPUs at once is that it requires co-ordination between multiple "threads" of execution. In order for multiple CPUs to effectively split up work, they would have to be able to talk to each other about the current state of the program, what work they'd each done, etc. While this is possible, it's a more complex model that requires more effort from both the programmer and the system. JavaScript's approach is simple: there is only one thing happening at once. Since everything that Node does is non-blocking, the time between an event being emitted and Node being able to act on that event is very small, because it's not waiting on things like disk I/O.

Another way to think about the event loop is to compare it to a postman (mailman). To our event loop postman, each letter is an event. He has a stack of events to deliver in order. For each letter (event) the postman gets, he walks to the route to deliver the letter (Figure 3.2, “The Event Loop Postman”). The route is the callback function assigned to that event (sometimes more than one). However, critically, since our postman only has a single set of legs, he can walk only a single code path at once.

Figure 3.2. The Event Loop Postman

Sometimes, while the postman is walking a code route, someone will give him another letter. This is the callback function he is visiting at the moment. In this case, the postman delivers the new message immediately (after all, someone gave it to him directly instead of going via the post office, so it must be urgent). The postman will diverge from his current code path and walk the proper code path to deliver the new event. He then carries on walking the original event that emitted the event he just walked.

Let's look at the behaviour of our postman in a typical program by picking something really simple. Suppose we have a Web (HTTP) server that get requests, retrieves some data from a database, and returns it to the user. In this scenario we have a few events to deal with. First (as in most cases) comes the request event from the user asking the Web server for a Web page. The callback that deals with the initial request (let's call it callback A) looks at the request object and figures out what data it needs from the database. It then makes a request to the database for that data, passing another function, callback B, to be called on the response event. Having handled the request, callback A returns. When the database has found the data, it issues the responseevent. The event loop then calls callback B, which sends the data back to the user.

This seems fairly straight forward. The obvious things to note here are the "break" in the code, which you wouldn't get in a procedural system. Since Node.js is non-blocking system, when we get to the database call that would make us wait, we instead issue a callback. This means that different functions must start handling the request and finish handling it when the data is ready to return. So we need to make sure that we pass any state we need to the callback, or make it available in some other way. JavaScript programming typically does it through closures. We'll discuss that in more detail later.

Why does this make Node more efficient? Imagine ordering food at a fast food resturant. When you get in line at the counter the server taking your order can behave in two ways. One of them is event driven and one of them isn't. Let's start with the typical approach taken by PHP and many other web platforms. When you ask the server for your order, he take it but won't serve any other customers until he has completed your order. There are a few things he can do after he's typed in your order: process your payment, pour your drink and so on. However the server is still going to have to wait an unknown amount of time for the kitchen to make your burger (to one of the authors, who is vegetarian, orders always seems to takes ages). If, as in the tradtional approach of web application frameworks, each server (thread) is allocated to just one request at a time, the only way to scale up is to add more threads. However, it's also very obvious that our server isn't being very efficient. He's spending a lot of time waiting for the kitchen to cook the food.

Obviously, in real life resturants use a much more efficient model. When a server has finished taking your order, you recieve a number which they can use to call you back. You could say a call-back number. This is how Node works. When slow things like I/O start, Node simply gives them a callback reference and then gets on with other work that is ready now, like the next customer (or event in Node's case). It's important to note that as we saw in the example of the postman at no time does a resturant server ever deal with two customers at the same time. When they are calling someone back to collect an order they are not taking a new one and vice versa. By acting in an event-driven way, the servers are able to maximize their throughput.

Another interesting thing this analogy illustrates is cases where Node fits well and where it doesn't fit. In a small resturant where the kitchen staff and the wait staff are the same people, no tradeoff can be made by becoming event-driven. Since all the work is being done by the same people event-driven architectures don't add anything. If all (or most of) the work your server does is computation, Node might not be the ideal model.

However, we can also see when the architecture fits. Imagine there are two servers and new four customers in a resturant (Figure 3.3, “Fast-food, fast-code”). If the servers serve only one customer at a time, the first two customers will get the fastest possible order, but the third and forth customers will get a terrible experience. The first two customers will get their food as soon as it is ready because the servers have dedicated their whole attention to fullfilling their order. That comes at the cost of the other two customers. In an event driven model, the first two customers might have to wait a short amount of time for the servers to finish taking the orders of the third and forth customers before they got their food, but the average wait time (latency) of the system will be much much lower.

Figure 3.3. Fast-food, fast-code

Let's look at another example. We've given the event loop postman a letter to deliver that requires a gate to be opened. He gets there and the gate is closed, so he simply waits and tries again, and again. He's trapped in an endless loop waiting for the gate to open (Figure 3.4, “Blocking the event loop”). Perhaps there is a letter on the stack that will ask someone to open the gate so the postman can get through. Surely that will solve things, right? Unfortunately it won't unless the postman gets to deliver the letter, and currently he's stuck waiting endlessly for the gate to open. This is because the event that opens the gate is external to the current event callback. If we emit the event from within a callback, we already know our postman will go and deliver that letter before carrying on, but when events are emitted outside the currently executing piece of code, they will not be called until that piece of code has been fully evaluated to its conclusion.

Figure 3.4. Blocking the event loop

As an example, the following code creates a loop that Node.js (or a browser) will never break out of:

EE = require('events').EventEmitter;
ee = new EE();

die = false;

ee.on('die', function() {
    die = true;
});

setTimeout(function() {
    ee.emit('die');
}, 100);

while(!die) {
}

console.log('done');

            

In this example, console.log will never be called because the while loop stops Node from ever getting a chance to call back the timeout and emit the die event. Although it's unlikely we'd program a loop like this that relies on an external condition to exit, it illustrates how Node.js can only do one thing at once, and getting a fly in the ointment can really screw up the whole server. This is why non-blocking I/O is an essential part of event driven programming.

Let's consider some numbers. When we run an operation in the CPU (not a line of JavaScript but a single machine code operation), it takes about 1/3 of a nano second. A 3ghz processor runs 3x10⁹ instructions a second, so each instruction takes 10^-9/3 seconds each. There are typically two types of memory in a CPU, L1 and L2 cache, each of which takes approximately 2-5ns to access. If we get data from memory (RAM), it takes about 80ns, which is about 2 orders of magnitude slower than running an instruction. However, all of these things are in the same ballpark. Getting things from slower forms of I/O is not quite so good. Imagine that getting data from RAM is equivalent to the weight of a cat. Retrieving data from the hard drive, then, could be considered to be the weight of a whale. Getting things from the network is like 100 whales. Think about how running var foo = "bar" versus a database query is a single cat versus 100 blue whales. Blocking I/O doesn't put an actual gate in front of the event loop postman, but it does send him via Timbuktu when he is delivering his events.

Given a basic understanding of the event loop, let's look at the standard Node.js code for creating an HTTP server:

var http = require('http');
http.createServer(function (req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  res.end('Hello World\n');
}).listen(8124, "127.0.0.1");
console.log('Server running at http://127.0.0.1:8124/'

            

This code is the most basic example from the Node.js Web site (but as we'll see soon, it's not the ideal way to code). The example creates an HTTP server using a factory method in the http library. The factory method creates a new HTTP server and attaches a callback to the request event. The callback is specified as the argument to the createServer method. What's interesting here is what happens when this code is run. The first thing Node.js does is run the code above from top to bottom. This can be considered the 'setup' phase of Node programming. Since we attached some event listeners, Node.js doesn't exit, but waits for an event to be fired. If we didn't attach any events, Node.js would exit as soon as it had run the code.

So what happens when the server gets an HTTP request? Node.js emits the request event, which causes the callbacks attached to that event to be run in order. In this case, there is only one callback, the anonymous function we passed as an argument to createServer. Let's assume it's the first request the server has had since setup. Since there is no other code running, the request event is handled immediately and the callback is run. It's a very simple callback and it runs pretty fast.

Let's assume that our site gets really popular and we get lots of requests. If, for the sake of argument, our callback takes 1 second and we get a second request shortly after the first one, the second request isn't going to be acted on for another second or so. Obviously, a second is a really long time, and as we look at the requirements of real world applications, the problem of blocking the event loop becomes more damaging to the user experience. The operating system kernel actually handles the TCP connections to clients for the HTTP server, so there isn't a risk of rejecting new connections, but there is a real danger of not acting on them. The upshot of this is that we want to keep Node.js as event-driven and non-blocking as possible. In the same way that a slow I/O event should use callbacks to indicate the presence of data Node.js can act on, the Node.js program itself should be written in such a way that no single callback ties up the event loop for extended pieces of time.

This means that you should follow two strategies when writing a Node.js server:

Taking the event-driven approach works effectively with the event loop (the name is a hint it would), but it's also important to write event-driven code in a way that is easy to read and understand. In the previous example, we used an anonymous function as the event callback, which makes things hard in a couple of ways. First, we have no control over where the code is used. An anonymous function's callstack starts from when it is used. Rather than when the callback is attached to an event. This affects debugging. If everything is an anonymous event, it can sometimes be hard to distinguish similar callbacks when an exception occurs.

Event-driven programming is different from procedural programming. The easiest way to learn it is to practice routine patterns that have been discovered by previous generations of programmers. That is the purpose of this chapter.

Before we launch into patterns, we'll take a look at what is really happening behind various programming styles to give the patterns some context. Most of this chapter will focus on I/O, because, as discussed in the previous chapter, event-driven programming is focused on solving problems with I/O. When it is working with data in memory that doesn't require I/O, Node can be completely procedural.

fs.readFile('foo.txt', 'utf8', function(err, data) {
  console.log(data);
};
fs.readFile('bar.txt', 'utf8', function(err, data) {
  console.log(data);
};

          

server.on('request', function(req, res) {
  //get session information from memcached
  memcached.getSession(req, function(session) {
    //get information from db
    db.get(session.user, function(userData) {
      //some other web service call
      ws.get(req, function(wsData) {
        //render page
        page = pageRender(req, session, userData, wsData);
        //output the response
        res.write(page);
      });
    });
  });
});
      

          

server.on('request', getMemCached(req, res) {
  memcached.getSession(req, getDbInfo(session) {
    db.get(session.user, getWsInfo(userData) {
      ws.get(req, render(wsData) {
        //render page
        page = pageRender(req, session, userData, wsData);
        //output the response
        res.write(page);
      });
    });
  });
});
      

          

var render = function(wsData) {
  page = pageRender(req, session, userData, wsData);
}; 

var getWsInfo = function(userData) {
  ws.get(req, render);
};

var getDbInfo = function(session) {
  db.get(session.user, getWsInfo);
};

var getMemCached = function(req, res) {
  memcached.getSession(req, getDbInfo);
};

          

       server.on('request', function(req, res) {

  var render = function(wsData) {
    page = pageRender(req, session, userData, wsData);
  };

  var getWsInfo = function(userData) {
    ws.get(req, render);
  };

  var getDbInfo = function(session) {
    db.get(session.user, getWsInfo);
  };

  var getMemCached = function(req, res) {
    memcached.getSession(req, getDbInfo);
  };

}

         

       var AwesomeClass = function() {
  this.awesomeProp = 'awesome!'
  this.awesomeFunc = function(text) {
    console.log(text + ' is awesome!')
  }
}

var awesomeObject = new AwesomeClass()

function middleware(func) {
  oldFunc = func.awesomeFunc
  func.awesomeFunc = function(text) {
    text = text + ' really'
    oldFunc(text)
  }
}

function anotherMiddleware(func) {
  func.anotherProp = 'super duper' 
}

function caller(input) {
  input.awesomeFunc(input.anotherProp)
}

middleware(awesomeObject)
anotherMiddleware(awesomeObject)
caller(awesomeObject)

         

One of the challenges of writing a book is how to explain things in the simplest way possible. That runs counter to showing techniques and functional code that you'd want to deploy. While we should always strive to have the simplest, most understandable code possible, sometimes you need to do things that make code more robust, or faster at the cost of making it less simple. This section provides guidance about how to harden the applications you deploy, which you can take with you as you explore upcoming chapters. This section is about writing code with maturity that will keep your application running long into the future. It's not exhaustive but if you write robust code you won't have to deal with so many maintenance issues. One of the trade-offs of Node's single threaded approach is a tendancy to be brittle. These techniques help mitigate this risk.

Deploying a production application is not the same as running test programs on your laptop. Servers can have a wide variety of resource constraints but they tend to have a lot more resources than the typical machine you would develop on. Typically front-end servers have many more cores (CPUs) than laptop or desktop machine, but less hard drive space. They also have a lot of ram. Currently itself Node has some contraints, such as a maxmimum JavaScript heap size. This affects the way you deploy because you want to maximize the use of the CPUs and memory on the machine while using Node's easy to program single threaded approach.

var http = require('http')

var opts = {
  host: 'sfnsdkfjdsnk.com',
  port: 80,
  path: '/'
}

try {
  http.get(opts, function(res) {
    console.log('Will this get called?')
  })
}
catch (e) {
  console.log('Will we catch an error?')
}

        

var http = require('http')

var opts = {
  host: 'dskjvnfskcsjsdkcds.net',
  port: 80,
  path: '/'
}

var req = http.get(opts, function(res) {
  console.log('This will never get called')
})

req.on('error', function(e) {
  console.log('Got that pesky error trapped')
})

        

var cluster = require('cluster');
var http = require('http');
var numCPUs = require('os').cpus().length;

if (cluster.isMaster) {
  // Fork workers.
  for (var i = 0; i < numCPUs; i++) {
    cluster.fork();
  }

  cluster.on('death', function(worker) {
    console.log('worker ' + worker.pid + ' died');
  });
} else {
  // Worker processes have a http server.
  http.Server(function(req, res) {
    res.writeHead(200);
    res.end("hello world\n");
  }).listen(8000);
}

        

      if (cluster.isMaster) {
  //Fork workers.
  for (var i=0; i<numCPUs; i++) {
    cluster.fork();
  }

  cluster.on('death', function(worker) {
    console.log('worker ' + worker.pid + ' died');
    cluster.fork();
  });
}

        

var cluster = require('cluster');
var http = require('http');
var numCPUs = require('os').cpus().length;

var rssWarn = (12 * 1024 * 1024)
  , heapWarn = (10 * 1024 * 1024)

if(cluster.isMaster) {
  for(var i=0; i<numCPUs; i++) {
    var worker = cluster.fork();
    worker.on('message', function(m) {
      if (m.memory) {
        if(m.memory.rss > rssWarn) {
          console.log('Worker ' + m.process + ' using too much memory.')
        }
      }
    })
  }
} else {
  //Server
  http.Server(function(req,res) {
    res.writeHead(200);
    res.end('hello world\n')
  }).listen(8000)
  //Report stats once a second
  setInterval(function report(){
    process.send({memory: process.memoryUsage(), process: process.pid});
  }, 1000)
}

        

var cluster = require('cluster');
var http = require('http');
var numCPUs = require('os').cpus().length;

var rssWarn = (50 * 1024 * 1024)
  , heapWarn = (50 * 1024 * 1024)

var workers = {}

if(cluster.isMaster) {
  for(var i=0; i<numCPUs; i++) {
    createWorker()
  }
  
  setInterval(function() {
    var time = new Date().getTime()
    for(pid in workers) {
      if(workers.hasOwnProperty(pid) &&
         workers[pid].lastCb + 5000 < time) {

        console.log('Long running worker ' + pid + ' killed')
        workers[pid].worker.kill()
        delete workers[pid]
        createWorker()
      }
    }
  }, 1000)
} else {
  //Server
  http.Server(function(req,res) {
    //mess up 1 in 200 reqs
    if (Math.floor(Math.random() * 200) === 4) {
      console.log('Stopped ' + process.pid + ' from ever finishing')
      while(true) { continue }
    }
    res.writeHead(200);
    res.end('hello world from '  + process.pid + '\n')
  }).listen(8000)
  //Report stats once a second
  setInterval(function report(){
    process.send({cmd: "reportMem", memory: process.memoryUsage(), process: process.pid})
  }, 1000)
}

function createWorker() {
  var worker = cluster.fork()
  console.log('Created worker: ' + worker.pid)
  workers[worker.pid] = {worker:worker, lastCb: new Date().getTime() - 1000} //allow boot time
  worker.on('message', function(m) {
    if(m.cmd === "reportMem") {
      workers[m.process].lastCb = new Date().getTime()
      if(m.memory.rss > rssWarn) {
        console.log('Worker ' + m.process + ' using too much memory.')
      }
    }
  })
}