Closures are not complicated

Note December 2010: The terminology I attribute to the ECMAScript spec below was from the then-current 3rd edition. The current 5th edition spec (there was no 4th edition) uses different terminology. At some point I'll get around to updating...)

Closures seem to frighten people a bit. Partially I suspect this is down to the academic nature of the term "closure". It sounds like something Californians look to achieve, not something to help you write software. (Disclosure: I mostly grew up in San Francisco, so I'm allowed to poke fun at my fellow Californians. Don't Try This At Home.) I suspect it's also because if you don't know the rules for them, they seem mysterious, and the mysterious is often frightening.

First off, what is a closure? I'll leave a thorough definition to my academic betters, but let's put it this way for my fellow plodders and I: A closure is a function with data intrinsically bound to it.

Consider this code:

function updateDisplay(panel, contentId)
{
    var url;

    url = 'getcontent?id=' + contentId;
    jsonRequest(
        url,
        function(resp)
        {
            if (resp.err)
            {
                panel.addClassName('error');
                panel.update(
                    'Error retrieving content ID '
                    + contentId
                    + ' from "'
                    + url
                    + '", error: '
                    + resp.err);
            }
            else
            {
                panel.removeClassName('error');
                panel.update(resp.json.content);
            }
        }
    );
}

This updates a panel element based on the results of a call to retrieve some JSON data. The call to the jsonRequest function accepts two parameters: The URL that will return the JSON data, and a callback function to trigger when the request completes (one way or the other). If the data was returned successfully, the callback sets the content of the panel from the JSON data and makes sure that the "error" CSS class is not set on the element; if there's an error, it shows details about the error and sets the "error" CSS class. (In this example, we happen to be using Prototype to extend our panel element with the nifty class name and update methods, but that's as much to keep the example simple as anything else.)

The callback above is an example of a closure: A function with data bound to it, in this case the panel and contentId arguments we passed into the updateDisplay function, and also updateDisplay's url local variable. It's useful to have this information bound to the callback, because the jsonRequest function doesn't know anything about panel or contentId, it just knows about triggering the callback — but the callback has the information it needs to do its job.

Magic? Nah. I can't speak for closures in other languages, but there's nothing complicated about closures in JavaScript. Seriously. They're dead easy. You need to know, say, three things and you're good to go. Well, four things, but that's only because someone has been misinforming you. Here's a quick list of those four things, after which we'll delve in a bit deeper, and then I'll tell you something at the end that will surprise you for three seconds before you go "Oh, of course":

1. In JavaScript, everything is a data structure, even functions, and — critically — even the context in which functions run. One aspect of that context is a "call object" (that's Flanagan's term, the ECMA specification uses "activation object" — I'll use "call object" because I'm lazy and it's less typing I find it clearer).
2. JavaScript variable names are resolved on the basis of a "scope chain", which includes (among other things) the global object and all of the current call objects in scope.
3. Functions in JavaScript are lexically scoped — for the plodders like me out there, that means that the things a function can access (the things "in scope" for it) are determined by the context in which the function is defined, not the context in which it's executed.
4. Closures do not create memory leaks. Someone probably told you that they did at some point (perhaps they thought that's what Microsoft was trying to say here).

Okay, let's see how those things come together to make closures easy.

#1: Everything is a object

When we call a JavaScript function, the context of that call to the function — the parameters we've used, the variables within the function itself — are part of an object called the call object (or, again, "activation object" in ECMA parlance). The interpreter creates a call object for this particular execution of the function and sets some properties on that call object before passing it into the function's code. The properties are:

1. A property called arguments which is an array (of sorts, in most implementations it's not actually an Array object) of the actual arguments we called the function with, plus a callee property which is a reference to the function being called.
2. A property for each of the arguments defined by the function declaration. The values of these properties are set to the values passed into the function. If any arguments weren't specified (because JavaScript lets you call functions with however many arguments you want, regardless of the definition), properties for any missing arguments are still created on the call object — with the value undefined.
   
3. A property for every declared variable within the function (e.g., every "var" statement). (These start out with the value undefined.)

Let's look again at selected bits of the updateDisplay function from earlier:

function updateDisplay(panel, contentId)
{
    var url;

    // ...
}

And let's assume we call it with a reference to a 'leftPanel' div and a contentId of 123:

updateDisplay(leftPanel, 123);

That creates a call object for this execution of updateDisplay that essentially looks like this:

call.arguments = [leftPanel, 123]
call.arguments.callee = updateDisplay
call.panel = leftPanel
call.contentId = 123
call.url = undefined

This object is then used within the body of the function when the code uses the arguments panel or contentId, or the local variable url, etc.

"But wait," you say. "I don't refer to an object when I use the arguments or variables in my function, I just use their names." Indeed — the use of the call object is implicit. How do we end up using the call object when we just write "contentId" (for instance)? Because once the call object is created, it's put at the top of the scope chain for the duration of the function call. Which takes us nicely to...

#2: Variable names are resolved using a "scope chain"

If you've written JavaScript in a browser environment, you've probably used the global document object, and perhaps the global navigator object as well. Here's the thing: Those aren't global objects. Those are properties of a global object — in fact, of the global object. The document and navigator properties are set up for you by the browser, but they're just properties of an object.

So how do you get away with just giving the property name, rather than giving the object name as well? Because of the scope chain, an ordered series (indeed, a chain) of objects. When the JavaScript interpreter sees an unqualified variable reference, it checks the top object on the scope chain to see if it has a property by that name: If so, it gets used; if not, the interpreter checks the next object down the chain. The global object is the bottom of the chain, so if you just type document.writeln("Blah blah blah"), eventually the document property is found on the global object and used.

(The global object is an abstract entity in the generic JavaScript definition, but in the specific case of a web browser you know it by another name: window. In browsers, the window object is the global object; it just also has a property, "window", that refers to itself.)

So quick: Within a function, how do variable names get resolved? Right! When the function is being executed, the call object with all those nifty properties for the function's arguments and variables is put at the top of the scope chain. So during our call to updateDisplay from earlier, the call object for the call is at the top of the scope chain, followed by the global object, like this:

When the interpreter sees a variable reference, say contentId, it looks on the call object: If there's a contentId property on the call object, it gets used; otherwise, the interpreter looks at the next object in the scope chain, which in this case is the global object.

(There's more to know about the scope chain than I've described here; the astute reader will be wondering, for instance, how objects and their instance members come into play, or what the with statement does to things. Alas, we can't tackle everything all at once...)

But how can I know when I'm writing updateDisplay that the scope chain will look like that? I mean, doesn't it depend on who's calling the function? Nope. And that brings us to our next point:

#3: Functions in JavaScript are lexically scoped

Okay, so we get the concept of the call object, which is created when we call a function; and we get the global object, which sits at the bottom of the scope chain to handle globals. But what's this "lexically scoped" thing? It's just a fancy way of saying that a function's scope is determined by where it's defined (e.g., the text defining it; léxis is Greek for "word" or "speech"), not where it's called.

Let's put that another way: When a function is defined, the code defining it is in some kind of context — a function is running, or the page itself is running if you've done the code at the top level. That context has an active scope chain when the function is defined. So when creating the function object, the interpreter creates a property on it (called [[Scope]] in the ECMA spec, but you can't access it directly) with a reference to the active execution context's scope chain. Even when the context goes away (e.g., the function returns), because the function object created by the definition has a reference to the scope chain, the scope chain isn't garbage collected — it's kept alive by the active reference to it from our function object. (Assuming that we've kept a reference to the function object, as we did in our example by passing it into the jsonRequest method; otherwise, the function and the scope chain are both garbage-collected since nothing references them.)

Remember our closure at the beginning of this post? It gets a [[Scope]] property pointing to the scope chain in effect when updateDisplay was called with panel = leftPanel, etc.:

So when we call it, first its scope chain is put in place, then the call object for this particular call to the function is put on top of the scope chain, and the function is executed with this chain:

And there we are, the closure can access panel and all of the other properties of the call object for the call to updateDisplay because they're on the scope chain. They're on the scope chain because that was stuck onto the closure's function object when it was created. No magic. Just objects, the scope chain, and lexical scoping all working together.

#4: Closures don't cause memory leaks

So why do people think closures cause memory leaks? A couple of reasons, I suspect, but chief among them being a bug in an issue with Internet Explorer. IE's DOM objects are not garbage-collected in the same way that JavaScript objects are; instead, IE shows its COM roots by using reference counting, a mechanism that is, unfortunately, prey to issues with circular references. Event handlers can easily end up being circular references, and so in IE it's easy to "leak" the memory associated with a DOM element and its event handler because they're referring to each other. (JavaScript's garbage collector doesn't have an issue with circular references; it works on the "reachability" principle rather than reference counting.) This is an issue with event handlers, rather than closures, but since many event handlers are written as closures, people associate it with closures. Fortunately, it's not that hard to kick IE into shape (in this particular way); Crockford talks about the problem and its solution here and many toolkits [like Prototype] will do this for you if you ask them nicely.

Lest I be accused of Microsoft-bashing, I should point out that IE is not the only browser that sometimes loses track of things; it's just by far the worst. Firefox 2 has a bad habit of leaking a bit of memory on XMLHttpRequest calls, which also frequently involve closures. The good news there is that Firefox 3 beta 3 is looking awfully good indeed on this front.

Separate from browser issues, though, if you're not really clear on how closures work, particularly with regard to the scope chain, you'll miss the fact that a closure keeps a reference to all of the variables and arguments in scope where it's created, not just the ones it uses; and so if you have (say) a massive temporary array allocated in that scope that the closure doesn't use, you might be tempted to say that the closure is causing the array's memory to leak. (The answer there is simple: Clear the array's variable when you're done with it.)

In Conclusion

There are lots of nifty things you can do with closures. I'll post a follow-up in a couple of days, "Closures By Example", demonstrating a few of them and just generally walking through some seemingly-complicated examples. But until then, I'll just reiterate my theme: Closures in JavaScript are not complicated. They're powerful, but if you understand that 1. Everything is a data structure, 2. Variables — I should probably call them unqualified references — are resolved according to a scope chain, and 3. The scope chain for a function is defined by where the function is defined lexically, you'll be golden.

Oh! I said I'd tell you something at the end that would surprise you for about three seconds before you said "Oh, of course." Here it is:

All JavaScript functions are closures.

Happy coding!

JavaScript's Curiously-Powerful OR Operator (||)

The "or" operator (||). It seems innocuous enough: A binary logical operator that returns true if either of its params is true, usually used in the context of a conditional statement:

if (a || b)

Perhaps if you've been programming in any of several other high-level languages for a while, you expect that it won't evaluate the second param if the first one is true, something called short-circuiting. And you'd be right. But there's a lot more to the JavaScript || operator than that!

Consider this statement:

x = a || b;

If the || operator were purely a logical operator, then x would receive one of two values: true or false. But that's not what happens. Instead, x gets the value of a if a is true or can be converted to being "true", otherwise it gets the value of b. And that's another kettle of fish entirely.

First off, there's this question of "...or can be converted to being "true"..." What is truth? (And whither beauty? Never mind, that's a different blog...) I won't go into the ins and outs of JavaScript's type coercion here (refer to Flanagan's book), but for our purposes today, a can be converted to "true" if it's not zero, an empty string, null, undefined, NaN, or false (or document.all on browsers, for historic reasons). So 1 is considered true, as is the string "Fred", as is an object reference.

But lots of languages do at least some level of type coercion; what marks the JavaScript operator out as special is the result of the expression: What you get back isn't a true or false, but rather the value of one of the operator's parameters. Now that's interesting, and it's something you may not be expecting if you're relatively new to the language. It can make it difficult for a newcomer to read some of the more dense JavaScript found in sample code, toolkits, etc.

Here's an example I ran across a few weeks back (the names have been changed for simplicity):

function sort(comparator)
{
    comparator = comparator || this.defaultComparator;
    // ...
}

Here, the author provides a sort method which takes an optional comparator function reference; if you don't supply one, the class's default comparator is used instead. Let's break that down a bit in each scenario.

If you call the method without supplying a comparator function reference:

1. comparator is undefined.
2. The expression evaluates comparator, sees the undefined value, and converts it to a boolean: false.
3. Because the first parameter is false, the operator returns the second parameter's value.

And now let's look at it if you do supply a comparator function reference:

1. comparator is a reference to a function.
2. The expression evaluates comparator, sees the function reference, and converts it to a boolean: true.
3. Because the first parameter is true, the operator returns the first parameter's value (it never evaluates the second parameter at all).

So the author's line

comparator = comparator || this.defaultComparator;

sets up the comparator variable with the default if it didn't already have a value.

You might be asking, why not just do this:

if (!comparator)
{
    comparator = this.defaultComparator;
}

Well, for one thing, it doesn't demonstrate l33t skilz, which is just as important to the current set of twentysomething coders as it was to the previous set, and the set before that (as it is now, has it ever been). But they would probably also point out that even after being minified, the result is in the region of six characters longer -- or nearly 12% of the total. (And they both beat out the version using the ternary operator.) Size counts with downloaded JavaScript. Some would also probably argue that it's more "expressive". Plodders like myself might raise counter-arguments about clarity, but that's the thing, to a day-in, day-out JavaScript coder, the statement isn't unclear.

Let's take our example one step further: Suppose our hypothetical class with the sort method only sets its defaultComparator on the instance if it's explicitly set, and relies on a class-wide default otherwise? The || operator lets us write the choice of the comparator function with a single line (split into three lines here for word-wrap):

comparator = comparator
          || this.defaultComparator
          || NiftyClass.classComparator;

The first || tests comparator and, finding it false-ish, takes the value of this.defaultComparator; then the second || evaluates this.defaultComparator and, finding it false-ish, takes the value of NiftyClass.classComparator. Putting it more simply: It selects the first one that's defined. As a good friend of mine said "...it is kind of a selection operator more than a logical one..." Indeed, it's almost a specialized version of the ternary operator where the test is for "truth" or "existence" (and I figure that's got to account for 80% or more of the use of the conditional operator).

So, do we all want to see the || idiom for selection rather than if statements? There we enter the realm of style, I tend to doubt that either way is markedly better than the other in terms of execution speed (which probably varies from engine to engine anyway), the size difference is minimal, and it's personal choice which is more expressive. But if you're going to be reading any significant amount of modern JavaScript, you'll see this x = a || b || c; idiom a lot, so be prepared for it. And if you're an old plodder like me, well...it does kinda grow on you. ;-)

[I should point out that in early versions of JavaScript, the || operator did evaluate to true or false, rather than providing this nifty return-the-value-of-the-param behavior. But the JavaScript engines in all major browsers (including IE6) use the behavior described above.]

Nifty Snippets

Welcome to Nifty Snippets, my new blog for capturing little snippets of code and techniques (as well as engineering, business, and teamwork "lessons learned"). Initially the focus will be on Ajax stuff, and that means lots of JavaScript.

Now, JavaScript is about 18 times more interesting than most people think. This is not a toy language, this is an incredibly rich, powerful, expressive language that until recently has been dramatically under-used. Partially that's because of how it was introduced to the world (scripting events in Netscape Navigator), and partially I suspect it's down to a bias many of us had toward hierarchical, class-based languages like Java and C++. JavaScript is neither hierarchical nor class-based; it's a prototype language.

This first post is just meant to say "hi". (Hi!) But I'll also throw out a few handy links for those with an interest in JavaScript, Ajax, and rich web applications:

• If you're doing any serious work in JavaScript, read Crockford. If there's something powerful and interesting about the JavaScript language, the odds are Douglas Crockford has written about it in detail. Don't let the depth of the language daunt you, you can ease your way into it.
  
• Similarly, if you're doing Ajax apps, you need JavaScript: The Definitive Guide by David Flanagan [O'Reilly, Amazon].
• Lots of people are really getting a lot out of Prototype, which is basically a bunch of nifty stuff for JavaScript wrapped up into a toolkit.
• Many of those same people are enjoying the various effects and other goodness available via script.aculo.us.
• Alternately, perhaps you'd like to peruse one of the dozens of other JavaScript toolkits out there.

Those'll get you started, anyway. Back soon with our first snippet!