Adam Terlson: I $.Promise - Part 2 - Chained and Parallel Async Calls

We established in I $.Promise - Part 1 - Why do jQuery Promises exist? that promises solve some conceptual problems created by using a traditional callback function.

Okay, now what?

In this post, I am going to show how using a traditional callback-style approach manifests into real unsolvable scenarios (or at the least ugly/unmaintainable code). And, of course, how easy it is with Promises instead.

Fetching a model from a server before updating the UI

You've got this model which knows how to fetch its data from the server and populate itself. You want to fire that off, get some data back, and when that's all done update your UI with that new information. Pretty simple stuff, but the flow across all these examples looks like this:

Make a new model.
Fetch new data from the server.
Set local properties on the model from inside the model itself.
Update the UI with new information after all properties are set.

First, with callbacks:

var model = function () {
    var self = this;
    self.setProperties = function () {
        // Do some sets with data
    };
    self.fetch = function (callback) {
        $.get('...', function (args) {
            self.setProperties(args);
            callback();
        });
    };
}

function refreshTheUI() {
    // Refresh UI with new data
}

var myModel = new model();
myModel.fetch(refreshTheUI);

This will work all well and good! Not too bad! But what if it takes more than one async operation to fully populate this model. Let's say it's a composite model and needs to make calls to two different end points. Okay, this will be interesting. Let's start by following a model of chaining callbacks.

var model = function () {
    var self = this;
    self.setPropertiesFromFetch1 = function (args) {
        // Do some sets
    };
    self.setPropertiesFromFetch2 = function (args) {
        // Do some more sets
    };
    self.fetch = function (callback) {
        // Fire off the request for the first bit of information
        $.get('...', function (args) {
            self.setPropertiesFromFetch1(args);

            // Get the second bit of information
            $.get('...'), function (moreArgs) {
                self.setPropertiesFromFetch2(moreArgs);

                callback();
            }
        });
    };
}

function refreshTheUI() {
    // Refresh UI with new data
}

var myModel = new model();
myModel.fetch(refreshTheUI);

This is made easier by callback() being within our closure. If we wanted to extract those anonymous functions out into their own functions on the model, we'd need to be passing callback() along the way.

Beyond that, this is slower than it needs to be as you are chaining rather than running these $.get() calls in parallel. Okay, let's speed it up, using the simplest way I know how (using callbacks) to ensure that we don't execute our passed-in callback function until it's all complete.

var model = function () {
    var self = this;

    self.hasFetched1 = false;
    self.hasFetched2 = false;

    self.setPropertiesFromFetch1 = function (args) {
        // Do some sets

        self.hasFetched1 = true;
    };
    self.setPropertiesFromFetch2 = function (args) {
        // Do some more sets

        self.hasFetched2 = true;
    };

    self.callIfDone = function (callback) {
        if (self.hasFetched1 && self.hasFetched2) {
            callback();
        }
    }

    self.fetch = function (callback) {
        // Fire off the request for the first bit of information
        $.get('...', function (args) {
            self.setPropertiesFromFetch1(args);

            self.callIfDone(callback);
        });

        // Get the second bit of information (in parallel now!)
        $.get('...'), function (moreArgs) {
            self.setPropertiesFromFetch2(moreArgs);

            self.callIfDone(callback);
        }
    };
}

function refreshTheUI() {
    // Refresh UI with new data
}

var myModel = new model();
myModel.fetch(refreshTheUI);

Well we've sped things up, but some downfalls of this approach are becoming clear:

Having "hasFetched" properties is ridiculous but necessary under a callback-based approach if you want to block the execution of callback() until the completion of async calls run in parallel.
There's redundancy in the dual execution of callIfDone().
Look at how big our code got! Imagine if there were 10 child models; you'd have to write some special handler and call it 10 times or do a lot of copy-paste, being sure that callback() gets executed at the right time all the while.
It's up to the model itself to marshal when to actually execute the callback function given to it.

The complexities of chaining and running asynchronous operations in parallel is quickly overwhelming, not to mention verbose. I was going to make one more example of how you'd run one operation in series, followed by two in parallel with callbacks only, but I'm not that masochistic. It's a pain to say the least.

The good news is that this is super simple using a Promise-based approach. Let's blow this mess up and rewrite it.

A Promise-based Approach

First, let's go back to the simple case of just one call to $.get() and attaching the success handler.

var model = function () {
    var self = this;

    self.setProperties = function (args) {
        // Do some sets
    };

    self.fetch = function () {
        // Fire off the request for the first bit of information
        // Because we attach this done() handler first, we can
        // be assured that it'll get executed before whomever 
        // may add to this function chain later.
        var fetching = $.get('...').done(self.setProperties);
        return fetching;
    };
}

function refreshTheUI() {
    // Refresh UI with new data
}

var myModel = new model();
myModel.fetch().done(refreshTheUi);

Way simpler.

Note that the model no longer cares about the consumer's callback at all! The model's concerns are that of populating itself and nothing more--no callback function marshaling! All we need to do is return a promise that's resolved when the model wishes and let whomever gets it attach to it as they will.

In this scenario, our model says that fetch() is complete when $.get() is complete, so we can just return the promise that $.get() gives us straight away. But things aren't always quite so simple.

Running async operations in parallel using $.when()

Let's show how to make this a composite model with multiple $.get() calls just as before, using $.when().

var model = function () {
    var self = this;

    self.setPropertiesFromFetch1 = function (args) {
        // Do some sets
    };
    self.setPropertiesFromFetch2 = function (args) {
        // Do some more sets
    };

    self.fetch = function () {
        var fetching1 = $.get('...').done(self.setPropertiesFromFetch1);
        var fetching2 = $.get('...').done(self.setPropertiesFromFetch2);
        
        // $.when returns a **new** promise
        var fetchingBoth = $.when(fetching1, fetching2);
        return fetchingBoth;
    };
}

function refreshTheUI() {
    // Refresh UI with new data
}

var myModel = new model();
myModel.fetch().done(refreshTheUi);

$.when() is used when you want a new promise that is only done when both of its children are done (and is rejected if any child gets rejected). Here are some notes about $.when():

$.when() returns a new promise that wraps its inner promises.
The promise that it returns is only resolved after all promises given to it are resolved and their done() callbacks have completed.
The promise that it returns is rejected if any child is rejected. Its fail() callbacks are executed only after the inner promise's fail() callbacks are complete. Other promises are unaffected.
You cannot pass an array to $.when(). If you want to do this, you'll need to use $.when.apply($, promiseArr).
If you give something that's null, undefined, a plain object or otherwise not a promise, $.when() will treat it as a Promise that's already resolved rather than throwing any kind of error. So $.when(promise1, null, promise2) equates to $.when(promise1, promise2). Keep this in mind, as it's burned me when trying to figure out why $.when() was calling its attached done() handlers too early.

Because the promise returned by $.when() doesn't have resolve() called on it until each of its inner promises have had their done() chains complete, we can be assured that when refreshTheUi() gets called all properties have been set via the setProperties() functions. Awesome.

Note that as of jQuery 1.8 Deferred.Pipe is now an alias for Deferred.Then. I am leaving the below as-written but in your code replace .pipe for .then if possible. Recognizing than Deferred.then now serves a dual purpose (as shorthand and as pipe) is important to advanced promise implementations.

Going from parallel calls to chained calls is easy with Deferred.pipe()

$.when() is to "parallel" as Deferred.pipe() is to "serial/chained". Until recently, the documentation for Deferred.pipe() was exceptionally unclear on its usage, though I'm happy to say it's been made a bit clearer as of this writing. It talked about filtering (something I find to be a rare use case) and really glossed over the very important fact that Deferred.pipe() returns a new promise, just like $.when().

It's easiest for now to just think of Deferred.pipe()* as a chained alternative to $.when(), but its use slightly differs.

* Note how I called it "Deferred.pipe()"? This is because it's called on the Promise you want to chain off of and doesn't exist on its own like $.when() does.

Let's put it to use.

var model = function () {
    var self = this;

    self.setPropertiesFromFetch1 = function (args) {
        // Do some sets
    };
    self.setPropertiesFromFetch2 = function (args) {
        // Do some more sets
    };

    self.fetch = function () {
        var fetching1 = $.get('...').done(self.setPropertiesFromFetch1);
        
        // Deferred.pipe also returns a **new** promise
        var fetchingBoth = fetching1.pipe(function () {
            // This part here is only executed once fetching1 returns successful
            // since the first parameter of pipe is the done callback
            var fetching2 = $.get('...').done(self.setPropertiesFromFetch2);

            // Returning fetching2 here will 'pipe' its results into fetchingBoth
            // Therefore, fetchingBoth is only successful when both fetching1
            // and fetching2 are successful.
            return fetching2;
        });
        return fetchingBoth;
    };
}

function refreshTheUI() {
    // Refresh UI with new data
}

var myModel = new model();
myModel.fetch().done(refreshTheUi);

Here are some notes to keep in mind when using Deferred.pipe():

It's called on the promise you want to chain from.
It returns a new promise.
The promise that pipe() returns' result (whether it's rejected() or resolved()) depends on that of the inner returned promise, if given.
Its method signature matches that of Deferred.then(), and ~~although the documentation makes no mention of it as of this writing~~, as of jQuery 1.8, Deferred.then() is an alias for Deferred.pipe(). They are exactly the same, even though their stated purposes are quite different.

Getting even more complex is simple with Promises

Creating something with exceptionally complex async logic is super simple using Promises. Remember that "serial then parallel" challenge I mentioned using callbacks? Well it's as simple as this:

    self.fetch = function () {
        var fetchMeFirst = $.get('...');
        
        // fetchingEverything is a new promise that's successful
        // if the three $.get() calls complete successfully.
        var fetchingEverything = fetchMeFirst.pipe(function () {
            var fetching1 = $.get('...').done(self.setPropertiesFromFetch1);
            var fetching2 = $.get('...').done(self.setPropertiesFromFetch2);

            return $.when(fetching1, fetching2);
        });
        return fetchingEverything;
    };

Promises are freaking awesome.

Adam Terlson

Friday, September 14, 2012

I $.Promise - Part 2 - Chained and Parallel Async Calls