Unravelling Nested Callbacks with FF

If you’re a javascript developer, then callback functions are your bread-and-butter. You know all about the so-called “pyramid of doom”, and you know all kinds of ways to defeat it. If you’re a javascript developer, you might regularly use a promise library like Q, or perhaps something more sequential like node async. Maybe you even get excited or angry when someone mentions co-routines. But if you’re a javascript developer, the last thing you ever wanted in life was another asynchronous flow control library. Well, it might be time to take another look.

The Problem

Let’s start with what we all know, callback hell:

The problem with this is twofold. First, unless you have a really wide screen it is hard to read. But more importantly, as the logic grows more and more complex, code like this becomes difficult to reason about. For instance, what do we do when an important method like addFriend returns an error (can we still cache the userData)? What happens if an uncaught exception is thrown inside/outside the callback chain? How do we schedule multiple asynchronous operations at once and wait for them all to complete before continuing?

The Workarounds

There are a variety of ways to deal with the callback problem. One technique is to use a promise library. In a nutshell, promises ditch the callback paradigm in favor of an event based system. Using a method like then, you can register onSuccess and onError (and sometimes onProgress) handlers for your asynchronous operations. What makes this especially useful in terms of the “pyramid of doom” is that promises can be chained together. Let’s rewrite the nested callbacks above using the Q library to see promises in action:

In the above example, we used the Q.nfcall method to turn our asynchronous operations into promises. Then, in the onSuccess handlers, we return new promises representing the next async operation. In this way we are able to “schedule” a series of handlers for our asynchronous operations rather than simply nesting the callback functions. It is also important to note that in order to use userData across these handlers, we must declare the user variable in the enclosing scope, and set it in the authenticate success handler. Even still, the result is noticeably cleaner code than nested callbacks.

Another method for unravelling the pyramid is to use a generalized flow control library like async. Let’s rewrite the above example using this library instead:

Here we used the async.waterfall method to specify an array of functions that we want to run in order, and one at a time. Each function is given a next parameter, which we use to forward the results of the asynchronous operations to the proceeding function. Once again, we have to declare a user variable in the enclosing scope in order to use it throughout the handlers. In any case, a nice feature of async.waterfall is that all errors are forwarded to the final callback so you can deal with them all in one place. And again, the resulting code is much cleaner than using nested callbacks.

While both of these options are great, life-savers even, what happens when I want to intermingle series AND parallel operations? The logic using the aforementioned libraries can again become complex and hard to read.

Luckily, there are other libraries out there that are specifically designed for this situation. Let’s have a look at Step:

Like async.waterfall, we are specifying a list of functions to be executed in serial order. But instead of invoking next to move on the proceeding function, we use the this.parallel method to tell the Step library to wait for the results from the current operations before moving on. When we want to pass data to the next function immediately, we simply return the variable and it will get passed as a parameter to the proceeding function. This way we do not have to declare a user variable in the enclosing scope, we can directly pass it to the following functions.

FF

Unfortunately, there are some drawbacks to the Step library. First, the "this" keyword is reserved by Step, so you cannot bind your functions to any specific context. This makes code hard to read and write when following OOP practices. Second, if you want to forward local data from one function to the next you need to return it, which means you are limited to passing one object at a time. Third, you have to manually handle errors in each function rather then being able to handle errors in one place (like we did with async).

Tim Caswell, the original creator of Step, realized these limitations and created a vision of what Step would be like if he could rewrite it from scratch. The ff library is our extrapolation of this vision. Let’s see the above example rewritten in ff:

You’ll notice that ff does not rely on the this object, so you are free to run your code using any context you want. Also, using the ff.pass method we can forward as many variables as we want to the next function. This is more dynamic and flexible than relying on the return statement. Finally, if there are any errors returned in the callbacks (or even thrown during execution), we can deal with them all in one place by attaching an .onError handler.

Another interesting feature of ff is that it is entirely Promises/A+ compliant. This means if you have an existing framework that relies on the promise api, or if you just love the promise spec itself, you can still write your code that way:

In the above code, we used the then method to specify how we wanted to handle our ff operation when it finished running of all its functions (or produced an error). For more information about promises, check out Nathan Stott‘s excellent HowToNode Article.

In any case, FF was designed to be a poweful, concise, and easy to read way of handling your asynchronous operations. If you’re interested in learning more about how FF works, check out the documentation. Also, be sure to fork us on github, and come find us on twitter.

Happy coding!

3 thoughts on “Unravelling Nested Callbacks with FF

  1. Robert Schultz

    I have been using Step since I first started using Node back in 2010. I use it in almost every single Node project and module I create and couldn’t imagine using vanilla Node with Step. I too also recently wrote my own module (https://github.com/Sembiance/node-tiptoe) that is based on Step but adds a few more features. Your FF module has a lot of attractive features and I’m strongly considering using it. I took a similar approach that FF did, in that errors are no longer passed as the first argument of each function, but rather handled at the end (I pass them to the final function, but your idea of an onError is much cleaner). However one thing I didn’t see in the FF lib was a way to handle/recover from errors in the chain. For example, if you know a particular function sometimes throw errors that you can gracefully recover from, having a way for those errors to be passed to the next function in the chain rather than stopping the chain all the way to onError may be nice. I came up with this.capture() where for the next chain only errors will be passed. Anyways, great work on the FF lib, it’s quite nice :)

    Reply
    1. Michael Henretty Post author

      Hi Robert,

      Node-tiptoe looks pretty cool. It’s no coincidence that we both liked Step, but wanted better error handling and a way to finish things immediately with something like “finish()” or “succeed()”. Non-linear flows are extremely handy in many circumstances.

      As far as a way to “capture” the error in the next function, I think what you are looking for is “f.slotPlain()”. This will forward the error to the next function as a parameter rather than to the onError/onComplete handler. That said, I like your terminology of “capture” rather than “plain” because it’s more self-explanatory. Any chance you’d want to submit a pull request to FF?

      In any case, nice work!

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>