Frozen intrinsics experimental flag

[guybedford]

This PR implements a node --frozen-intrinsics flag which applies the deep freeze method from SES (https://github.com/Agoric/SES/blob/15ecf1520c8314f7ec29165548eccff1118e294d/src/bundle/deepFreeze.js) to all known intrinsics.

There is a more detailed write up by those involved in these security initiatives at https://docs.google.com/document/d/1h__FmXsEWRuNrzAV_l3Iw9i_z8fCXSokGfBiW8-nDNg/edit?ts=5c1adaed#heading=h.q1x6on1y6aju, I just put this together as it seems an easy low-hanging fruit to improve security in Node.js.

Checklist

• make -j4 test (UNIX), or vcbuild test (Windows) passes
• tests and/or benchmarks are included
• documentation is changed or added
• commit message follows commit guidelines

[nodejs-github-bot]

@guybedford build started: https://ci.nodejs.org/blue/organizations/jenkins/node-test-pull-request-lite-pipeline/detail/node-test-pull-request-lite-pipeline/2373/pipeline

[vsemozhetbyt]

https://github.com/nodejs/node/blob/master/doc/node.1 may also need to be updated)

[devsnek]

@guybedford

I'd be more interested in seeing a solution where userland can still polyfill and such. There have been other proposals where node uses the original methods without needing to freeze everything.

also fwiw:

IteratorPrototype = Object.getPrototypeOf(Object.getPrototypeOf([][Symbol.iterator]()))

TypedArray = Object.getPrototypeOf(Uint8Array)

ThrowTypeError = Object.getOwnPropertyDescriptor(Function.prototype, 'caller').get

[addaleax]

• What’s the idea behind an --experimental-... flag? Does this mean the goal is to eventually enable this by default? That seems … maybe not feasible?
• Should this be applied in a per-Context way, i.e. also affect code inside vm.* environments? I think that would require making lib/internal/per_context.js more powerful, but it might be worth it? (/cc @joyeecheung)

[guybedford]

What’s the idea behind an --experimental-... flag? Does this mean the goal is to eventually enable this by default? That seems … maybe not feasible?

The goal would be to make it a non-experimental flag, but there are still a few things to ensure before such a stage:

1. We probably want this spec bug fixed Normative: Make non-writable prototype properties not prevent assigning to instance tc39/ecma262#1320 as it stops classes extending any of these builtins from overridding properties otherwise.
2. It could be worth considering including in this ensuring that global.Array is itself unwritable / unconfigurable, but that seems like a next step after this PR to me
3. The per_context handling also definitely sounds preferable. I just tried implementing this, but hit (1) causing a bug on the buffer code - class FastBuffer extends Uint8Array {}; FastBuffer.prototype.toString = .... Also per_context is too early to catch the console and timers freezing as well. We should definitely come back around on this though.

So the --experimental aspect allows as to move forward despite 1 - 3 above. Although I'm certainly open to discussion about this approach.

@devsnek thanks for filling in the remaining anonymous intrinsics there, that's a huge help.

[vdeturckheim]

Do we have numbers regarding a potential perf impact here?

Also, if I'm not mistaken this would prevent adding methods to global classes but since Global is not frozen, one could still replace the value of say String.

Also, could it be possible to make this list expendable or to expose the deep freeze method for users who might want to use it?

[addaleax]

The goal would be to make it a non-experimental flag, but there are still a few things to ensure before such a stage:

In that case I’d prefer to give the flag its desired non-experimental name and document it as experimental.

[guybedford]

In that case I’d prefer to give the flag its desired non-experimental name and document it as experimental.

@addaleax sounds good, I've renamed to --frozen-intrinsics and made it clear it is experimental. If I should note this in any other way too let me know.

Do we have numbers regarding a potential perf impact here?

In most benchmarks these days for v8, frozen objects behave identically to non-frozen ones. The time to freeze on my machine is about 20ms which is no insignificant (against a 80ms startup). But we should be able to integrate the frozen objects into the snapshot in future to avoid this.

Also, if I'm not mistaken this would prevent adding methods to global classes but since Global is not frozen, one could still replace the value of say String.

Yes exactly, I've explained this clearer in the notes. We could use a getter / setter approach to fix this, or we could look at some context approaches too.

Also, could it be possible to make this list expendable or to expose the deep freeze method for users who might want to use it?

Exposing the freeze API is something for userland, this internal API may even be optimized out in future so this is a non-goal.

[jdalton]

I noticed --frozen-intrinsics breaks console creation which could be related to this note:

> new console.Console({ stdout: process.stdout, stderr: process.stderr });
Thrown:
TypeError: Cannot assign to read only property 'log' of object '#<Console>'
    at new Console (internal/console/constructor.js:112:13)

[joyeecheung]

I'd say prepareUserCodeExecution is a better place for this, as this does not make a lot of sense in use cases like node --help

[guybedford]

Thanks. This is now in pre_execution.js. Let me know if that seems right.

[joyeecheung]

Should this be applied in a per-Context way, i.e. also affect code inside vm.* environments? I think that would require making lib/internal/per_context.js more powerful, but it might be worth it?

@addaleax How would this option get passed to NewContext? Maybe a boolean? Currently at the point per_context.js is executed the JS land cannot access the CLI options yet, also note that we need to override things like global.console by ourselves during bootstrap that is done after per_context.js

In most benchmarks these days for v8, frozen objects behave identically to non-frozen ones. The time to freeze on my machine is about 20ms which is no insignificant (against a 80ms startup). But we should be able to integrate the frozen objects into the snapshot in future to avoid this.

I recall there are still some perf issues regarding frozen objects in V8, cc @caitp who's been working on that

[joyeecheung]

FWIW, this looks more like a v8 flag than a Node flag to me, or this looks less overreaching if V8 provides an API for us to do this.

[caitp]

FWIW, this looks more like a v8 flag than a Node flag to me, or this looks less overreaching if V8 provides an API for us to do this.

It's not a v8 flag, and v8 does not currently provide an API which does this.

I recall there are still some perf issues regarding frozen objects in V8

As discussed offline, there are some well known issues regarding indexed property access to frozen objects --- however, these would generally not affect the things on the white list, currently. OTOH, the main cost would be some slight increased startup cost, since these frozen objects aren't frozen in any snapshot.

I'll be working on speeding up other cases, so please ping me on perf bugs regarding frozen object access.

[joyeecheung]

It's not a v8 flag, and v8 does not currently provide an API which does this.

Yes, I was talking hypothetically

But we should be able to integrate the frozen objects into the snapshot in future to avoid this.

@guybedford The freezing depends on a CLI flag, so we can't include it in a snapshot for the default startup, the best workaround I can think of is to build two snapshots and include them both by default, but I am not sure how much the size overhead would be if we do this for different combinations of CLI flags.

[devsnek]

i agree that this would make sense as a feature for v8, rather than node. even if they don't come up with a magical optimization for the snapshot, it would be a lot faster than this, since they can just directly flip some flags. (and we wouldn't have to worry about stuff like new globals exposed by v8 flags, which this misses (for example, WeakRef))

[sam-github]

I've been trying to figure out how to move this forward and make it usable.

I can't find a response to #25685 (comment), and the issue still exists in 12.1.0, any idea what is happening there?

Is there a plan to get this out of experimental somewhere I've missed?

I expected monkey patching of JS primordials to be relatively uncommon in node.js code, at least in well-behaved node code, so was hoping this flag would actually be usable, but it turns out it isn't. I ran on a couple pretty vanilla projects I had around, and they failed for reasons like these:

var o = {}; o.toString = () => {}; // toString is frozen in the Object prototype
function F() {}; F.bind = () -> {}; // bind is frozen in the Function prototype (express does this in router)

Those both look to me like reasonable bits of code mutating user-created objects, not attempts to monkey-patch intrinsics. I don't think we can get this out of experimental without addressing these issues, but they are part of the definition of Object.freeze(), and having read through tc39/ecma262#1320 and tc39/ecma262#1307, the definition of Object.freeze() does not seem likely to change. @bmeck, do you still hold out hopes of getting TC39 to budge on this? It looks like the conclusion was that changing Object.freeze would be web-breaking, so can't go on. Would it be crazy to propose a variant, like Object.freeze(Boolean sane), or is there just no way forward for this at the language spec level?

Assuming no change in the language spec, how can we move this forward?

Some possibilities:

1. V8 can implement an option to freeze intrinsics. This would not get setTimeout() and other things defined by nodejs (probably), but its still of value.
2. V8 could expose an API in either js or C++ that does a more useful "freeze()", akin to @bmeck 's TC39 proposals I linked to above.

Are there any other options? @nodejs/v8 @ofrobots, do you have any opinions? Or any suggestions - is it possible for Node.js to use existing V8 APIs to achieve a kind of "freeze" that is more useful, i.e., does not break prototypical inheritance?

If we have no options to freeze intrinsics in a useful way, disappointing as it is, maybe we should consider removing the experimental feature.

[bmeck]

It is extremely unlikely that a language change will occur, initial tests show that ~ 15% of all websites hit usage counters showing we would alter their behavior.

[hashseed]

The getOriginals proposal sounds like an alternative to this?

[bmeck]

@hashseed to an extent, but it is even harder to use ergonomically. also if getOriginals is mutable it wouldn't be a replacement for this.

[sam-github]

AFAICT, it allows writing code that defends against mutation of "originals", but frozen internals was intended to protect pre-existing/all code from that mutation ocurring in the first place.

[hashseed]

... which would be a fairly severe departure from the ECMA spec. I'm not sure splitting off into a "safe" JS dialect is desirable.

[bmeck]

@hashseed can you explain that, this topic of freezing globals is brought up in quite a large % of TC39 meetings and is not thought of as splitting the language usually in committee discussion.

[sam-github]

A number of people have expressed the desire the ability to make it impossible, at least after app initialization, to prevent mutation of internals. I don't think anybody is suggesting it be the default (which would make it a dialect), or that everyone would desire this feature.

@hashseed Am I to take it that you/V8 don't think adding such a run mode is desireable? Or are you saying that if it isn't a TC39 feature, V8 is unlikely to implement it?

@bmeck was there discussion ever in TC39 that you recall of a "make-the-world-immutable" API, that could be called by user-code, perhaps after some initial setup, to indicate no futher mutation of intrinsics would be allowed? Note how I'm careful to avoid the word "freeze", since freezing has other side-effects that are not so useful.

[hashseed]

This is more the domain of @ajklein, so I'd like to defer to him.

[bmeck]

@sam-github there have been several ideas on what to call such an API, historically this has been called the "deep freeze" operation but more recent talks in the Realms WG calls seem to lead towards the term "purify". There has certainly been investigations into polyfills and mutations but people from @Agoric or Bloomberg could state more than myself in terms of actual deployments of those, however these often are done via "endowments". However, allowing such an operation is a bit tricky since a trusted Realm generally needs to exist to prevent unwelcome mutations / to whitelist the mutations (this exists for polyfill mutations in Realms for properties as well as if import maps replace the "originals" module specifiers).

[ajklein]

This thread is new to me; I think I've mostly understood what's under discussion, but forgive me if I'm covering something that's already addressed above.

The thing I want to make sure is clear (in reference to #25685 (comment)) is that it's not the definition of Object.freeze that's problematic (for tc39/ecma262#1320), but the treatment of assigning to properties on objects. So providing a V8 API for "make everything immutable" wouldn't help, unless that "make everything immutable" API did something outside the scope of the standard ECMAScript object model.

[bmeck]

@ajklein not sure what you mean, but tc39/ecma262#1320 is problematic for anything trying to use things with non-writable data props on their prototypes. There are alternative workarounds as discussed in TC39 around getters/setters (which do not solve this in userland), proposing a new extension to the object model (I find that unlikely), or somehow finding a workaround for tc39/ecma262#1320. In reality I find most of those unlikely, but do find the results of freezing appealing, and quite a bit of code does work, in particular things like classes and object literals which use non-delegating operations via define work fine. Ideally we would have a way to freeze things such that frozen objects remain usable for the cases like above, but right now JS does not provide such a path. Freezing is quite unergonomic/requires non-intuitive code to maintain its usefulness without such workarounds, but does work with some workarounds quite well as discussed via the deployments at Bloomberg and the like.

[ajklein]

@bmeck My comment was in response to @sam-github's suggestion that V8 could somehow provide an API that would avoid the kind of breakage listed in this thread (and that you agree is a problem).

[guybedford]

@sam-github thanks for trying this out and sharing your thoughts. It is great to see discussion on these cases stirred up from its use - this was the hope from the start!

One goal of making this flag public was to see if these breaks can be fixed and PR'd to common libraries as they happen.

That Console subclassing bug is actually a Node.js core bug for frozen intrinsics - I've posted a simple fix for this in #27663, a direct console fix could also be provided as well here.

Apart from any potential Node.js core bugs like that, there are basically two main issues that come up for these workflows - one for objects and one for classes.

For the object case - var o = {}; o.toString = () => {} can be resolved with var o = { toString: () => {} }.

And for the case of classes:

class X {
  constructor () {
    this.toString = 'asdf';
  }
}

which can be resolved by declaring the overridden builtin properties as class instance properties at the time of definition of the class:

class X {
  toString = undefined;
  constructor () {
    this.toString = 'asdf';
  }
}

Since these are such straightforward fixes for both of these problems, they should also be easily acceptable PRs for any library.

We're very much seeing how far we get here by testing this out. If it turns out there is far too much ecosystem friction then yes we can revert and come up with new primitives for frozen intrinsics.

But if it is just a couple of PRs to libraries with the above fixes then we can already get those benefits today.

I am still hoping for a better primitive though I must admit, although I have limited time myself to champion anything there. By making these issues explicit in Node.js hopefully we can continue this conversation.

[sam-github]

it's not the definition of Object.freeze that's problematiThe thing I want to make sure is clear is that it's not the definition of Object.freeze that's problematic , but the treatment of assigning to properties on objects

@ajklein If I understand you, you are saying that since Object.freeze works by setting writable: false for the properties, that the problem is what that "false" means for assigment of prototypically derived objects, not the definition of Object.freeze

If so, I don't think the distinction is meaningful. If Object.freeze was defined in terms os a primitive (writable: false,) that doesn't work as wanted, that's a problem with the definition of Object.freeze, IMO.

I don't know the purpose/history of Object.freeze(), just how its currently defined, and that if we want to prevent mutation of global objects, Object.freeze() doesn't work well for that purpose. It quite possibly works absolutely perfect for some other use-case.

Changing the definition of Object.feeze sounds like its web-breaking, so are there other options?

I can see V8 not being keen to create non-standard language facilities.

I'm curious to know whether exposing a "make all of these .... objects immutable" C++ API would be anathema to V8 as a non-standard js dialect, or perhaps could be considered a useful V8 embedder specific API.

[sam-github]

@guybedford You showed how to fix one of the two issues I ran into before stopping, what about the other? Function is not a class, and the object literal syntax can't be used to define a function, so I don't see how to adopt the approach to the express router example:

function F() {}; F.bind = () -> {}; // bind is frozen in the Function prototype (express does this in router)

[guybedford]

@sam-github it would help to understand what fixes might be possible if you could point to the exact piece of code in Express that is doing this?

[sam-github]

% node --frozen-intrinsics server.js
/home/sam/s/express-example-app/node_modules/express/lib/router/index.js:508
  proto[method] = function(path){
                ^

TypeError: Cannot assign to read only property 'bind' of function 'function(options) {
  var opts = options || {};

  function router(req, res, next) {
    router.handle(req, res...<omitted>...
}'
    at /home/sam/s/express-example-app/node_modules/express/lib/router/index.js:508:17
    at Array.forEach (<anonymous>)
    at Object.<anonymous> (/home/sam/s/express-example-app/node_modules/express/lib/router/index.js:507:23)                                                                  
    at Module._compile (internal/modules/cjs/loader.js:759:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:770:10)
    at Module.load (internal/modules/cjs/loader.js:628:32)
    at Function.Module._load (internal/modules/cjs/loader.js:555:12)
    at Module.require (internal/modules/cjs/loader.js:666:19)
    at require (internal/modules/cjs/helpers.js:16:16)
    at Object.<anonymous> (/home/sam/s/express-example-app/node_modules/express/lib/application.js:17:14)                                                                    
s/express-example-app (master *% u=) % npm ls express
express-example-app@1.0.4 /home/sam/s/express-example-app
└── express@4.16.4 

@guybedford I believe this would be reproducible with any express app, but I used github.com/strongloop/express-example-app, because I had it lying around, and it has no deps other than express.