Wednesday 15 May 2013

Private properties in ES6 -- and ES3, and ES5

Please note: ES6 changed markedly after this article was written in 2013; what was finalized and released as ES2015 in 2015 does not have private properties after all. The concept of the "private Name" object morphed into Symbol which was similar, but different, and in particular the idea that they would be non-discoverable was dropped. (Symbols are entirely discoverable, for instance with Object.getOwnPropertySymbols.) You can have truly private per-instance information accessible even to prototype functions using WeakMap. At some point I'll rewrite the article, but the technique described herein is still valid if you don't like to just use an underscore convention or similar.

JavaScript famously lacks private properties on objects. The next version will have them (more on that below), but we can have most of the benefits of the upcoming improvement right now (without resorting to the usual hidden variables in the constructor), even on older engines. In this post, I look at what's coming, and what we can do now.

"But wait," I hear you say, "Don't we already know how to do private data, à la Crockford's Private Members in JavaScript?" Yes, but it has some downsides, not least that those members aren't actually properties, and methods shared via the prototype can't use them. A few months back, looking at the way private properties are being added to JavaScript, I was struck by how easily we can have very nearly private properties right now, today. Real properties. And of course those properties can be data properties, or methods (since methods are just data properties referring to functions). So now I'm finally getting around to writing it up.

First, a couple of terms:

  • ES3 - The version of ECMAScript (standard JavaScript) defined by the third edition specification in 1999. All major browsers support ES3.
  • ES5 - The version of ECMAScript defined by the [fifth edition specification][1] in 2011. Most modern browsers support at least parts of ES5, and many support nearly all of it.
  • ES6 - The upcoming version of ECMAScript being defined for the next specification. Early access drafts and such are available here.

The Typical Pattern For Private Information

Okay, let's start with the typical pattern for private data in classes of objects, as popularized by Crockford and others. Let's assume we have a Foo constructor and we want the objects it creates to have a hidden nifty piece of data:

function Foo() {
    var nifty = 42;

    this.method1 = function() {
        // Has access to `nifty` because this function
        // closes over it
        console.log("Truly private nifty info: " + nifty);
    };
}
Foo.prototype.method2 = function() {
    // Does not have access to `nifty`, because this
    // function does not close over it. So the following
    // line would be an error, there is no `nifty` in scope:
    // console.log("Truly private nifty info: " + nifty);
};

var f = new Foo();
f.method1(); // Can use `nifty`
f.method2(); // Cannot :-(

nifty is truly private. Nothing has access to it but the method1 function of the object created by that specific call to new Foo. So that's great...

...but there are a few issues with this pattern:

  1. Each and every Foo object gets its own method1 function. We don't get the reuse we get with method2, where there's only one of them shared by all Foo objects via the prototype. In terms of memory impact and so on, this isn't really a big deal with modern engines unless you're creating thousands and thousands of these, since modern engines are able to reuse the code of method1 even though a new method1 object is created for each Foo object. But it's unsatisfying, and there are some development patterns that involve dynamically changing the prototype, which are obviously unable to act on method1 above, as it's not on the prototype.
  2. Different Foo objects can only see their own private data, not the private data in other Foo objects. This is markedly different from languages with true private members, and makes certain operations quite difficult.
  3. There's no way to have protected properties: If we have a Bar that inherits from Foo, there's no way for Foo objects to define things that Bar objects can see but other code cannot. (Some would argue this is a good thing, because protected data members create serious coupling between the base objects and the derived objects. But they're still quite popular in Java, C#, etc.)

...and further, just subjectively, it's clunky from a style perspective.

Truly Private Properties in ES6

With ES6, we can get truly private properties, because ES6 will have private name objects. Private name objects let you use a special kind of object as a property name, rather than a string. If you don't have access to that specific name object, you can't retrieve the property from the object.

Here's what that looks like:

// ES6 private properties (not yet available in the wild
import Name from "@name"; // (Remember that this syntax is still in flux)
var Foo = (function() {
    // Create a private name object as our private property key
    var nifty = new Name();

    // Our constructor    
    function Foo() {
        // We can just assign here as normal
        this[nifty] = 42;
    }

    // Methods shared by all Foo instances
    Foo.prototype.method1 = function() {
        // This method has access to `nifty`, because it
        // closes over the private key
        console.log("Truly private nifty info: " + this[nifty]);
    };
    Foo.prototype.method2 = function() {
        // Also has access, for the same reason
        console.log("Truly private nifty info: " + this[nifty]);
    };

    return Foo;
})();

var f = new Foo();
f.method1(); // Can use `nifty`
f.method2(); // Can too! :-)
// Both `method1` and `method2` are *reused* by all `Foo` objects

It's just that simple. Properties created using private name objects are automatically non-enumerable, so they don't show up in for-in loops or calls to Object.keys, and code that doesn't have the specific Name object we created (nifty) cannot access that property. It is a property on the instance, but it's truly private.

So, how does this look against those issues with the typical pattern?

  1. Prototype methods on Foo objects can access the private data. Foo objects don't each have to have their own method1 as in the typical pattern shown at the top.
  2. Since all Foo code has access to the key, different Foo objects can see the private data in other Foo objects, as in Java, C#, etc.
  3. We could define Bar, deriving from Foo, in the same scoping function, which would mean it had access to the nifty name object and therefore to the nifty information in Foo objects.

Voilá! Truly private properties.

Which is great, but we don't have ES6 yet. Is there a way we can get there, or get close? Yes! We can get really close right now in ES5, and it's nearly as good in ES3.

Near-Private Properties in ES5 (and even ES3)

As of ES5, we can create non-enumerable properties (ones that are not included in for-in and don't show up in Object.keys). So those are fairly well hidden, but if whoever you're trying to keep this private from glances at the object once in the debugger, they can learn the name and use the property (and you end up with the consequent issues when you change something you considered private but which got used by the guy down the corridor anyway).

So what's the answer? Make the name different every time. Suppose we define our own Name constructor until we have ES6, and make it generate a random string of a reasonable length (and never the same string twice):

var Name = function() {
    var used = {};

    function Name() {
        var length, str;

        do {
            length = 5 + Math.floor(Math.random() * 10);
            str = "_";
            while (length--) {
                str += String.fromCharCode(32 + Math.floor(95 * Math.random()));
            }
        }
        while (used[str]);
        used[str] = true;
        return new String(str); // Since this is called via `new`, we have to return an object to override the default
    }

    return Name;
}();

Now we can use the ES6 code above (minus the import statement, of course) and get really obscure properties (even in ES3), in that they have names that change every time the code runs. The guy down the corridor can look at the name in the debugger, but he can't write code relying on it, because it's always changing. Instead, he has to come down the corridor and ask you to make an API change so he can do what he needs to do, which is what he should have done in the first place.

In ES5, we can take it a step further and make the property non-enumerable so that in addition to having a random name, it doesn't show up in for-in or Object.keys. We can even do that in the same codebase by only using the ES5 feature if it's present.

Here's a complete ES3 and ES5 example, using our Name constructor from above, changes called out with *** markers:

// Nearly-private properties
// ***No `import` here (once the final form is determined, we'll probably be able to feature test for it)
var Foo = (function() {
    // Create a random string as our private property key
    var nifty = new Name();

    // Our constructor    
    function Foo() {
        // We can just assign here as normal
        this[nifty] = 42;
    }

    // ***On ES5, make the property non-enumerable
    // (that's the default for properties created with
    // Object.defineProperty)
    if (Object.defineProperty) { // Only needed for ES3-compatibility
        Object.defineProperty(Foo.prototype, nifty, {
            writable: true
        });
    }
    // ***End change

    // Methods shared by all Foo instances
    Foo.prototype.method1 = function() {
        // This method has access to `nifty`, because it
        // closes over the private key
        console.log("Truly private nifty info: " + this[nifty]);
    };
    Foo.prototype.method2 = function() {
        // Also has access, for the same reason
        console.log("Truly private nifty info: " + this[nifty]);
    };

    return Foo;
})();

var f = new Foo();
f.method1(); // Can use nifty!
f.method2(); // Can too! :-)
// Both `method1` and `method2` are *reused* by all `Foo` objects

That's it! Virtually identical to the ES6 code, and it provides nearly as good encapsulation, certainly on ES5. The property we create is not truly private, but it's really obscure (on ES5) and pretty obscure even on ES3. In ES5 it doesn't show up in for-in loops (because the property we created is non-enumerable), and even on ES3 its name changes every time the code runs. So any code attempting to use the private data must first figure out the property name, which is a non-trivial exercise (probably impossible purely in code in ES5, as that code can't get a list of the non-enumerable property names of the object). Naturally, one glance at the object in a debugger shows you the property and its value, but nothing is private from debuggers, and the name will change next time. The guy down the corridor will be forced to get up and ask you to make the information available in the API, rather than using your private data!

Hey, What About Methods?

The great thing is that there's absolutely nothing special about methods using this pattern. You want a private method? Just define it, just like you define a private data property. Because of course, JavaScript doesn't really have methods, just properties that refer to functions and a bit of syntactic sugar.

Here's an example for ES3 and ES5 that uses both a private data property, and a private method. And of course, making it ES6 instead just requires adding the import and then (optionally) removing the Object.defineProperties call:

var Greeter = (function() {
    var normalize = new Name();     // Private worker method key
    var personName = new Name();    // Private data key

    // Our constructor    
    function Greeter(n) {
        this[personName] = n;
    }

    // Private properties
    if (Object.definePropertes) { // Only needed for ES3-compatibility
        Object.definePropertes(Greeter.prototype, {
            normalize:  {writable: true},
            personName: {writable: true}
        });
    }

    // Methods shared by all Greeter instances
    Greeter.prototype[normalize] = function(arg) {
        // Okay, so this is a really boring thing for the private worker method to do
        var s = this[personName];
        return s.substring(0, 1).toUpperCase() + s.substring(1).toLowerCase();
    };
    Greeter.prototype.sayHey = function() {
        console.log("Hey " + this[normalize]());
    };
    Greeter.prototype.sayBye = function() {
        console.log("Bye " + this[normalize]());
    };

    return Greeter;
})();

var g = new Greeter("JACK"); // Note the all caps
g.sayHey(); // "Hey Jack"
g.sayBye(); // "Bye Jack"

Happy Coding!

5 comments:

Anonymous said...

Great post, but I wonder if we are missing the point with private properties? They are an attempt to improve maintainability by controlling the "api" with which you can modify the object. There are clear advantages with this approach but what about the disadvantages? The java convention of making everything private/protected until there is a clear need to do otherwise results in inflexible code. Fine if it's your own code and can change it but what if it's someone else's? In short, I think the javascript philosophy of flexibility and expressiveness (dynamically-typed, prototypal) is in danger of being subverted in the moves to make it more like other languages. Encouraging classical style inheritance, private members and so on encourages a coding style that cannot take full advantage of javascript's strengths

T.J. Crowder said...

@Rob: Thanks! Yes, I love the flexibility of JavaScript as well.

Having privacy and using it by default are two different topics. I think the ability to make things private when appropriate is an unmitigated Good Thing. Different people quite reasonably have different views on when "appropriate" is, though, and it's a good point to raise. :-) I myself am probably a bit wishy-washy on the subject.

On the greater topic of the direction of JavaScript, I think if you study the various proposals with traction, on the whole you'll find that the expressiveness, flexibility, and prototypical nature of JavaScript are largely being enhanced, not subverted. Lots of them are specifically around making interaction with prototypes easier and clearer. For instance, the set prototype operator allows us to directly set the prototype of an object, much more directly and clearly (once you get used to it being backward) than Object.create (and broader in scope; you can set the prototype of more kinds of objects, including arrays and regular expressions). Another example is making it easy to use your prototype's version of something (e.g., "super" calls and properties), which was essentially impossible to do reliably in ES3 (and still ugly in ES5) unless you used the class pattern for inheritance and a robust helper library.

Even the classes proposal is still about prototypical inheritance, it's just about making the language more expressive and succinct for that common pattern. There's nothing in that proposal that can't be done today (or arguably even with ES3) with enough effort. (My Lineage library, for instance, combined with the private pattern above, probably covers it.) What the classes proposal does is dramatically simplify the hard parts by making use of the individual proposals I mentioned above (using the prototype's version of something, setting the prototype of objects, private properties, etc.). It improves the expressiveness of the language for that pattern. But the clever thing is that you don't have to use the class pattern for inheritance if you prefer other patterns (and many do); the individual pieces are all available without classes.

So I don't think the flexibility and expressiveness are in danger. I have concerns about aspects of the direction and some proposals (I think they're going overboard on symbols, for instance, at the expense of readability), but not about the general focus, which is (it seems to me) about flexibility, expressiveness, and retaining the essential nature of JavaScript.

Anyway, didn't mean to ramble. :-)

Happy coding!

-- T.J.

Esailija said...

Underscore prefixing is great and works most like Java's private. You get static verification that you are not breaking into API but
there is nothing that prevents you if you actually wanted to. You can for example use reflection to access them. So what you
would do in Javascript is just use underscore prefix and enforce it in your build step so that the guy down the corridor
will not accidentally checkout code dependent on internals.

The closure constructor antipattern breeds incredibly inflexible and unreusable code. You cannot access the data without rewriting the source code even for temporary reasons. The data is not owned by the returned object. It is
incompatible with all ES5 object APIs. The kind of security it offers is like java with security manager on (With the runtime performance penalties too of course) except it is of course *completely useless in all Javascript applications.

This new feature seems just as inflexible and misguided like the closure constructor. Even if you are not invoking the performance penalty from the huge GC pressure you are now using the objects like they were hash maps
and force all methods of the object to do context look ups.

all out alien said...

Not using es6 and needing access to a private variable you could consider assigning the prototype method within the constructor (i.e. within the Constructor scope). So within Foo:

if (!Foo.prototype.method2) {
Foo.prototype.method2 = function() {
return "Truly private nifty info: " + nifty;
};
}

T.J. Crowder said...

@Renzo: No, that would make all instances use the `nifty` value of the first instance.