Preserve all current syntax, but add support for dynamicity

[EthanRutherford]

(Branched from #75, to avoid having multiple conflicting conversations in one thread any longer)

The following syntax:

class Foo {
    x = 0;
    #y = 5;
    method() {
        this.#z = 10;
    }
    #privateMethod() {
        return this;
    }
    #arrow = () => this;
    addNewMethod() {
        const name = Math.random().toString();
        this.#[name] = function() {return this.x;};
    }
}

Transformed into current-js (i.e. via babel):

// method which wraps methods such that, if "this" is the private container,
// swap it out with the instance of the public class
// (this could probably be improved, but interpret as a POC)
function setValue(privateThis, trueThis, name, value) {
    if (value instanceof Function) {
        return privateThis[name] = function(...args) {
            if (this === privateThis) {
                return func.apply(trueThis, args);
            }

            return func.apply(this, args);
        }
    }

    return privateThis[name] = value;
}

const Foo = (() => {
    const privates = new WeakMap();

    return class {
        constructor() {
            //if there had been a constructor with super, everything up until the super call would go here
            //create the private container
            const fooPrivates = {
                y: 5,
                arrow: () => this,
            };
            setValue(fooPrivates, this, "privateMethod", function() {
                return this;
            });
            privates.set(this, fooPrivates);
            //property initialization
            this.x = 0;
            //if there had been a constructor, the rest of the body would go here
        }
        method() {
            // we lexically have access to privates, so if whatever "this" is is in there,
            // accessing and even writing new values is trivial
            if (!privates.has(this)) {
                throw new IllegalAccessException();
            }

            setValue(privates.get(this), this, "z", 10);
        }
        addNewMethod() {
            // adding a new method is also simple
            const name = Math.random().toString();
            if (!privates.has(this)) {
                throw new IllegalAccessException();
            }

            setValue(privates.get(this), this, name, function() {return this.x;});
        }
    }
})();

For the implementation in JSNext, an exotic type of container, which natively has "this" refer to the instance of the class (as opposed to itself), is installed in a private slot in the class instance.

The rest of the behavior falls out naturally from parallels to the babel implementation. It is also fairly trivial (syntax up for debate) to allow means to destructure, iterate, or even dynamically access private properties.

Personally, I would have them provided thusly:

function foo() {
    // destructure
    const {x, y} = this.#;
    // iterate
    for (let z of this.#) {}
    //dynamic access
    this.#["foo"];
}

As far as I can tell, the sole difference between this and the proposal as-is in terms of behavior is the addition of dynamic read/write, which some users will want. There should be no reductions in the safety of this when compared to the proposal as-is, but we have the benefit of providing a class author the option to work with private fields dynamically, instead of restricting them to a static interface.

[bakkot]

Here's a thing the current proposal allows:

class Outer {
  #brand;
  static Inner = class {
    #owner;
    constructor(owner) {
      try {
        owner.#brand;
      } catch {
        throw new TypeError('owner must be an Outer');
      }
      this.#owner = owner;
    }
  };
}

Keeping in mind that JS does not have types, what would this look like under your proposal?

[EthanRutherford]

This appears to be comparable to a "friend" class in other languages.

I concede, this may not be possible with exactly the same syntax, but perhaps a slightly altered syntax could work?

class Outer {
    #brand;
    static Inner = class {
        #owner;
        constructor(owner) {
            try {
                Outer#(owner).brand; // I'm not 100% on this syntax, but this would be the idea
                // owner.Outer#brand? 
                // owner.##brand? (explicitly call out the nested-ness?)
            } catch {
                throw new TypeError('owner must be an Outer');
            }
            this.#owner = owner;
        }
    };
}

Transformed, it would look like the following:

// assuming the same helper method from above..
const Outer = (() => {
    const privates1 = new WeakMap();

    const theClass = class {
        constructor() {
            privates1.set(this, {brand: undefined});
        }
    };

    theClass.Inner = (() => {
        const privates2 = new WeakMap();

        class theClass = class {
            constructor(owner) {
                privates2.set(this, {owner: undefined});

                try {
                    if (!privates1.has(owner)) {
                        throw new IllegalAccessException();
                    }

                    privates1.get(owner).brand;
                } catch {
                    throw new TypeError('owner must be an Outer');
                }
                privates2.get(this).owner = owner;
            }
        }
    });

    return theClass;
})();

I agree, the nested class is a potentially very useful and powerful feature, and is worth preserving. However, it is likely not going to be an incredibly common case, so I think that having a slightly altered syntax may not be so bad a tradeoff. Besides this, directly calling out that the private field comes from an outer class as opposed to the inner class may be worthwhile for readability's sake.

As a follow up question, in the current design, how would this work?

class Outer {
    #brand;
    static Inner = class {
        #brand;
        constructor(owner) {
            // which "brand" does this utilize?
            // is the name reused here, or do we get two copies of the same name,
            // which are now conflicting?
            // assuming this doesn't crash, our brand is now no longer a valid way
            // to identify that owner is indeed of type Outer
            owner.#brand;
        }
    };
}

A potential benefit of having an explicit syntax for accessing the privates of instances of other (outer nested) classes is that this potential naming collision/confusion is avoided.

(forgive the less-than-ideal names, assume that a tool which implemented this transformation would generate more appropriate names)

[bakkot]

I concede, this may not be possible with exactly the same syntax, but perhaps a slightly altered syntax could work?

The problem with this syntax, or anything like it, is that Outer is just a binding in the current scope. It can be rebound; it doesn't have any static meaning. (Also, classes aren't even required to have names.)

As a follow up question, in the current design, how would this work?

The inner #brand would shadow the outer one. I'm not super worried about this, since the conflict is only possible when textually within the class body. Just avoid names which shadow when you need to refer to the outer name, like you'd avoid shadowing an outer variable to which you need to refer. There's no real problem having to use a different name, since private names aren't observable outside the class.

[EthanRutherford]

Outer is just a binding in the current scope

This is true, but other syntaxes are possible. While it's not necessarily my favorite, this.##foo is a possible syntax which does not depend on changeable bindings.

I would agree that shadowing is not so terrible a problem, but still, the ability to have shadowed names but still explicitly refer to both the inner and outer instance has some worth.

[EthanRutherford]

I've put some time into organizing my thoughts on why I think dynamic access is important. To begin, it's helpful to keep in mind that Javascript is a dynamic scripting language, and people using it are used to being able to do dynamic things like add properties to an object at will. My primary concern can be boiled down to gears.

Public properties are dynamic. They are like driving an automatic transmission. If I need a new property, no worries. Just assign one, and keep on your merry way. Just like an automatic: if you need more speed, simply press the pedal down further, and everything just works.

Private properties under the current design are static. They are like driving a manual. If you need a new property, stop what you're doing. Go back to the class declaration. Add a private property to the class body. Now you can return to where you were, and keep going. Like driving a stick: you have to let off the gas, press in the clutch, shift the gear stick, let off the clutch, and reengage the gas.

The metaphor also includes the other kinds of dynamic access, like iteration and computed names and destructuring, but the forward declaration of private properties fit the comparison best.

The thing is, this proposal may be easy to explain. It's even pretty easy to understand. But that doesn't change the fact that private properties and public properties are different in several key ways, not strictly related to them being private. Users who are used to writing in javascript are going to have a similar experience to using private methods as drivers switching from automatic to manual: there's going to be a lot of grinding gears. An author realizes he needs a new property, and tries to simply assign to a new property. Program spits out errors (perhaps a linter catches the error, but still, no one likes having an error thrown at them). Users used to dynamic writing of properties are likely to forget this restriction sometimes, and even when they remember it it's a bit of a pain to have to break your concentration, jump over to the class body to declare a new private field, and then return to where you were and try to resume your previous train of thought. And, I'd argue this isn't something that people will necessarily simply eventually stop having a problem with. Public and private field access are frequently going to used in close proximity, and each transition from using a public to a private to another public etc. is a context switch. To continue to use our vehicle analogy, this is kind of like switching back and forth between an automatic and a manual. Even a trained professional is not immune to mode errors. I think that this could lead to a potentially frustrating experience, for the same reason that having a program where on some screens ctrl+v means paste, and on others means something else entirely would be frustrating.

There are also definitely some valid use cases for dynamic access of private fields. One example use case which depends on dynamic access and iterability is a cloning method.

clone() {
    const other = new Foo();
    //clone the properties
    for (const [name, value] of Object.entries(this))
        other[name] = value;
    for (const [name, value] of this.#) // or whatever the syntax would end up being
        other.#[name] = value;
    return other;
}

One could see how, in a class with even three or four private fields, writing out each private field individually could become cumbersome.There are other similar use cases, like flushing state to disk or sending bytes across a network which would similarly desire a concise way to iterate through all fields.

It's rather unfortunate that such a method cannot be this concise under the current design. Not to mention, if one were to add a new private field later, the method would also have to be updated to reflect it, which could lead to errors caused by failing to update both locations.

[bakkot]

@EthanRutherford Thanks for the clear summary of your position. I'm sure other people will have thoughts, but let me give mine:

In many ways, JavaScript has been moving away from its dynamic, imperative, scripting language roots towards a more static, declarative, general purpose language. This started with strict mode, which banned dynamic scope (outside the global object, which we couldn't touch) and made it an error to assign to an undeclared variable. ES2015 went further in this direction, with TDZ for lexical declarations, static exports, and other features. Classes especially exemplify this: to a large extent they're declarative sugar over old imperative patterns. Most of the reason public fields are a necessary feature instead of just more noise is that they make fields declarative, rather than requiring them to be created imperatively in the constructor. And classes enforce strict mode in their bodies unconditionally.

I would argue that it's actually really nice to have your language make certain static guarantees for you. In any strict code, I can skim the top level of a block scope and know exactly which variable bindings it introduces. As long as no one is still using var, I can even know that no nested code will create a binding which escapes it, unless it's writing directly to the global object. Of course, we've never had anything like this for properties - objects can be written in a declarative style as ({foo: bar}), but unless you seal or freeze them anyone can add more fields later - but I don't think that implies we shouldn't. For a long time we didn't have anything like that for scopes, either: and in fact scopes were quite literally objects; the distinction I've drawn here between variables and properties is not one which used to exist. It was invented alongside the static scoping guarantees.

We can't provide any real guarantees for public properties, of course, but for private properties we are already providing much stronger guarantees than we get for public properties: in particular, the guarantee that no code outside the class can observe its private fields without directly inspecting the source of the class, unless the class chooses to reveal them. The only place we have a similar guarantee currently is with closed-over variables, and for that reason many people teach a pattern of using closed-over variables for private fields. In modern JavaScript - in strict code - closed-over variables provide another extremely nice guarantee, namely, that they must be declared explicitly. That's a really nice part of that pattern for private state, which it would be a shame if private fields as a language feature lacked.

I think the benefits that the guarantees private fields afford in this proposal outweigh the costs of not being able to dynamically create them. Just as I like that I get an error if I write foo = 0 in code which can't see a foo, I like that I get an error if I write this.#foo = 0 in code which can't see #foo. In fact I get an early error, which is nicer still. Yes, it means I might have to go out a few levels of indentation and declare a field if I want a new one (or, ideally, hit a hotkey in my editor to have it do that for me), but at least I'll never have to switch files. In exchange, I get to know statically, just by glancing at the top level of the class, exactly which private fields every single instance of the class will have.

Program spits out errors (perhaps a linter catches the error, but still, no one likes having an error thrown at them).

I do! If I have an error in my code, I want to get an error!

And, I'd argue this isn't something that people will necessarily simply eventually stop having a problem with.

I really think it is, in most cases. If I expected people to continue dynamically creating public fields within class bodies for the common case, I wouldn't have pushed for class fields to go in the language. Classes are meant to be declarative. If you're using them as such, and because the syntax for private fields so closely matches that for public fields, there's no mode switch necessary.

---

Now, that's mostly about dynamically creating private fields. Iterating over them isn't nearly so bad, in my mind; it means I lose the ability to safely add or rename fields (which is not that small a thing to lose!), but at least I still get static shape.

But if fields are all statically declared, it's much less necessary to be able to iterate over them. It's just a convenience - and even then only a convenience in the fairly rare case that all your private fields are of a similar kind. You can write a class decorator which will give you this power, if you really need it.

That said, we could add this convenience eventually in a follow-on proposal, if after class fields went out into the world it really proved to be something people needed. But given that other languages with records rarely if ever provide such a mechanism, and that we have never felt the need to add the ability to iterate over all the variables in a scope, I am skeptical that it will prove that necessary.

[littledan]

@EthanRutherford Could you achieve the necessary dynamicness by using a single object as your private field which is then dynamically accessed?

We've discussed several times making private fields more dynamic. They are already a good bit more dynamic than originally proposed. However, as @bakkot explains above, staticness has been a design goal.

[EthanRutherford]

Given that one would have to stick to the single object (i.e. once you add a new #foo, the clean copy function goes out the window), it would. However, I still would advocate for the ability to iterate and access fields as in my clone method above. I still believe it will be worth being able to easily make such cloning, equality checking, serializing, etc. methods. It allows you to not have to continually update these such methods as you change the shape of the class during development.

I'm curious what @bakkot means by "it means I lose the ability to safely add or rename fields". I can see how dynamic access may mean somewhere in my class I'm keeping a string reference to a private field name, but I don't think that's going to be a case that ever actually shows up in the wild. I'm really not sure how adding new fields is affected though.

In my imagined use cases, dynamic access does not necessarily mean string based access, just a means by which if I iterate over the private fields, I can access the private keys (perhaps behaving something more akin to Symbols).

All that being said, I'm fine with that being explored in a follow-on proposal. I'm also curious about what you said about being "more dynamic" than they used to be. What was the proposal like initially?

[littledan]

For one, the original proposal added private fields to the instance based on the original prototype chain as the class was set up, rather than respecting prototype chain mutations as it does now.

If you're OK with this being explored as a follow-on proposal, I'll close this issue and the discussion can continue in another repository.

[bakkot]

I'm curious what @bakkot means by "it means I lose the ability to safely add or rename fields". I can see how dynamic access may mean somewhere in my class I'm keeping a string reference to a private field name, but I don't think that's going to be a case that ever actually shows up in the wild.

I can say with absolute confidence that it will show up in the wild. People will do anything they have the power to do.

You also don't need dynamic access here: maybe they're just inspecting the name. For example, say I have a method which iterates over all the private fields in the class and zeros those whose name starts with 'dynamic'. If a minifier (for example) renames those fields, that method breaks.

I'm really not sure how adding new fields is affected though.

Say I have a method in my class which iterates over all the fields in my class and recalculates them in a particular way, expecting them to all be of the same kind. If I add a new field of a different kind (or, worse, if I have tooling that adds a new field), that method breaks.