lib: hide navigator behind a flag

[aduh95]

The relevance of Navigator API in Node.js has not been established, and exposing it early has caused breakage in the ecosystem. IMO we should not expose it by default, not until there's consensus around it.

[nodejs-github-bot]

Review requested:

• @nodejs/startup
• @nodejs/web-standards

[anonrig]

I'm -1 on this. It was a change included within a semver major release. Reverting this at this late, would not help with adoption.

[GeoffreyBooth]

I’m also -1. It arguably is complete now that we have navigator.userAgent, as that’s the only property listed in the WinterCG minimum: https://common-min-api.proposal.wintercg.org/#requirements-for-navigatoruseragent. We don’t need to and might not add any more properties, which makes “far from complete” a bit of a strange thing to say.

Comparing us to Bun and Deno, we have the two properties in common with both other runtimes: #50269 (comment).

Also this flag is unnecessary. I already posted instructions on how to remove the global if desired: #50310. Since the API is already potentially complete, it’s better that it’s enabled by default and users can disable it rather than the reverse.

Please tag @nodejs/web-standards on issues/PRs related to Web APIs.

[aduh95]

Reverting this at this late, would not help with adoption.

My goal is not to help adoption, but limit ecosystem breakage. I think having this enabled by default does more harm than good – i.e. my impression is that the subset of users who needs/benefits from the presence of this API is much smaller than the subset that's being disrupted by it. Putting behind a flag lets folks who benefit from it keep using it, without disrupting the others.

[ljharb]

I don't think telling people to do a multi-decade bad practice - modify objects they don't own, which includes deleting globals - is something that's appropriate to recommend under any circumstances.

[KhafraDev]

The Navigator API is still very far from complete

What does "complete" mean here? Node will never be compliant with the spec (ie. navigator plugins or cookies), nor will node match what other WinterCG compliant environments do because afaict they all support different things.

[ljharb]

@GeoffreyBooth unfortunately @nodejs/web-standards wasn't pinged on #50310, which seems like a problem.

[tniessen]

LGTM

[benjamingr]

It's experimental and if people believe it's not ready or has consensus I think hiding it is a good step while discussion is had

[aduh95]

It arguably is complete

What does "complete" mean here?

That's fair, "complete" is not well defined, and not exactly relevant, I should not have used it. I've updated the OP to deal with the more concrete problem some people have with this API: is it relevant for Node.js and its users? The answer may very well be yes, but from what I read it's not obvious to everyone. And indeed, some folks would still prefer to not have this API no matter how "complete" its implementation would be.

[benjamingr]

(And note, this is only to mitigate breakage, people are running into this in v21 even-though most people are probably on v20 or v18 - I believe our users that this is real breakage and we should discuss it once we've resolved it for them)

[GeoffreyBooth]

As I said in yesterday’s TSC meeting, we made two mistakes:

1. We shipped navigator without navigator.userAgent, which is by far the most commonly used property of navigator and the one that’s in the WinterCG spec. Now that userAgent shipped in 21.1.0, many fewer public libraries are broken now.

2. We failed to catch the above via CITGM. We need to add some libraries in CITGM that use web standards, like packages involved in server-side rendering. I started this process in feat: add pnpm as optional package manager citgm#1020.

I think if navigator.userAgent had shipped in 21.0.0, we wouldn’t be having this discussion now. Inevitably there are some packages that check for other properties that might still be broken, but comparatively far fewer; and that’s always the cost of adding new globals, which is why they’re semver-major.

My biggest objection to this PR is that there’s no end date where the flag ever gets removed. Now that we have userAgent, there’s really nothing else we need to add. There’s no list of properties where it’s like, “okay, after these 10 are added, we can enable navigator by default.” This isn’t really an experimental API at all; it’s either added to Node or it’s not. There will always be libraries out there that make assumptions about what APIs are available or not in the environments they run in, and perhaps fail to write proper guards for when their assumptions are wrong. If we live in fear of ever breaking them, then we’ll never be able to add web APIs.

And there is value in navigator.userAgent. There’s a lot to be said for having a cross-platform way to know what runtime code is running in; there’s a reason that every other runtime, both server-side and browsers, implements it. (That’s also why it was so bad of us to ship navigator without it, since it wasn’t an unreasonable assumption on libraries’ parts that if navigator existed, then so would userAgent.) Especially since the fix is so easy, both on the libraries’ side and on the application developer’s side, I think there’s a lot more good gained by keeping navigator than there is harm avoided by removing it.

[ljharb]

The breakage is caused by exposing it as a global; could perhaps instead of requiring a flag, it move to a requireable/importable module? That way node can get feedback, and it'll be exposed by default, with no risk of breakage while the kinks are worked out.

[benjamingr]

My biggest objection to this PR is that there’s no end date where the flag ever gets removed.

Presumably, when we've assessed ecosystem breakage, made a reasonable attempt to contact authors of big'ish libraries that break because of this change and let them know, communicated with our users and tried to restore their trust after we've caused them breakage.

None of these steps are particularly hard and I'm optimistic navigator can be part of Node.js in a way that leaves our users happy.

[GeoffreyBooth]

Presumably, when we’ve assessed ecosystem breakage, made a reasonable attempt to contact authors of big’ish libraries that break because of this change and let them know, communicated with our users and tried to restore their trust after we’ve caused them breakage.

In a way, shipping the break was the outreach; we heard from the affected libraries in #50269. And all of the ones we heard from are fixed by 21.1.0, because we added userAgent. Yes it would’ve been better to do the outreach before shipping the breaking change, but oh well.

But also, 21.0.0 was a semver major release of Node. It’s allowed to and supposed to include breaking changes. And this one wasn’t hidden: it’s one of the notable changes in the release notes. I don’t think it’s “breaking trust” of users to ship a breaking change in a semver-major release. And better now, in a non-LTS release, than in 22.

[GeoffreyBooth]

Rather than one-off flags like --no-experimental-fetch and so on, what if there was a flag --no-global-[variable] (so like --no-global-navigator, --no-global-fetch etc.) so that any global that people want to disable can be removed? Within reason, like maybe we would throw on --no-global-process and some others.

The downside would be that some code would be written expecting the global to exist and might break when this flag was used, but at least this way any potentially problematic global is under the user’s control and the user can be responsible for using a mix of dependencies that work with whatever settings the user runs Node under.

[KhafraDev]

I don't think that's the issue with navigator; the issue is that it's a bad api and discussing if it even has a place in node.

[GeoffreyBooth]

(a2) In 21.x:

• navigator is global by default
• Expose util.navigator or process.navigator
• Add --no-global-navigator that disables the navigator global

I think the advantage of this option is that we keep most of the disruption in the 21.x line, which would reduce the impact of 22.x. There are presumably lots of people who will upgrade straight from 20 to 22, so the more that any breakage is discovered and fixed during 21.x, the less that will break when people start using 22.

[aduh95]

I would like this change to be directed specifically at the 21.x branch, with 22.x having navigator exposed as global by default with the flag as a no-op.

I don't think this is a good idea, it's going to create git conflicts (at least in the history section of the docs). Why not land this and then do a semver-major follow up that switches the flag?

[Uzlopak]

Seems like apache echarts has more issues. But i would rather provide a PR to that project and make it compatible.

apache/echarts#19233

[Ethan-Arrowood]

In my opinion this API was landed too early. I still don't know if I agree we should have this at all. Node.js and WinterCG compliance has not been agreed upon afaik, so simply for that sake we should be more hesitant to be adding WinterCG related APIs without further consideration.

[GeoffreyBooth]

Seems like apache echarts has more issues.

Tracing the error cited on that issue, it seems like it boils down to this:

https://github.com/ecomfe/zrender/blob/97f6e72123a40dbc22963fa14810d1c512e5e23b/src/core/env.ts#L40-L42

else if (typeof navigator === 'undefined') {
    // In node
    env.node = true;

Which . . . isn’t a bug in Node. There isn’t some future version of the navigator API that will unbreak users who write inherently flawed code such as this.

[ljharb]

True, but it is a reasonable assumption for code to have made that a non-browser will never have navigator because they don't (and never will) navigate - that's the kind of ecosystem breakage that adding it as a global risks.

[GeoffreyBooth]

Why not land this and then do a semver-major follow up that switches the flag?

Or let’s go with option a2 and replace this PR with one that adds a --no-global-navigator flag. That could land on main and it would be permanent. Why not go with that?

[tniessen]

Seems like apache echarts has more issues.

@GeoffreyBooth That is precisely the kind of breakage I predicted in #50412 (comment). These checks exist and they are reasonable based on standards and conventions from far more than a decade.

This already seems to have inspired undesirable workarounds, such as emulating a DOM because Node.js pretends to be a browser (see apache/echarts#19233 (comment)).

Which . . . isn’t a bug in Node. There isn’t some future version of the navigator API that will unbreak users who write inherently flawed code such as this.

It may be your personal opinion that such code is "flawed." However, as multiple people have explained here and elsewhere, a navigator was almost universally understood as a web browser (and referred to as such in the relevant standards) until, very recently, a few server runtimes decided to break that convention. I don't think the Node.js project should blame the ecosystem for any breakage here.

[GeoffreyBooth]

That is precisely the kind of breakage I predicted in #50412 (comment).

Yes, and it’s what @jasnell was saying about “inherently brittle” code and what I was describing as “anticipated” breakage. Quibbling over whether I’m justified in using the word “flawed” to describe such code is a distraction. Want to call it “based upon a completely reasonable assumption”? Sure, fine, I don’t care. It doesn’t matter what we think about it.

What matters is where we go from here. There is no future version of the Node navigator API that will unbreak users who have written code like if (navigator) { // browser. So the options are really:

1. Accept that such code is broken and will need to be updated to work in latest Node. (We should also provide a better opt-out than doc: explain how to disable navigator #50310, such as @jasnell’s proposed --no-global-navigator.)

2. Remove the navigator API, never to return.

The other options don’t really make sense in my view:

3. Flag the navigator API, and keep it flagged forever.
   a. This API isn’t useful at all behind a flag, because its usefulness is for library authors. Consumers who want a global navigator can always just create one, either directly or via something like jsdom. The benefit of including this within Node core is so that library authors can rely on its existence (or inversely, so that consumers can import libraries that were never intended to run in Node but could, if we supply all the web APIs that the libraries depend on).

4. Provide the navigator API but not as a global.
   a. Same problems as the previous option. Any library author who is intending to write code that runs in a browser or Node can already do so. The benefit to having navigator as a global is for consumers to import libraries that weren’t intended to be run in Node, but could be if we supply all the web APIs that the library depends on.

5. Flag the navigator API, and unflag it at some future date when some conditions are met.
   a. What date? What conditions? I’ve asked this several times now but no one who’s approved this PR has given an answer. @jasnell has at least proposed something concrete. I don’t think we should flag this API, and hide our contributors’ work, without a clear path forward for them.
   b. If one of the conditions for unflagging is “when it’s not a breaking change” then we might as well just remove it completely, because it will never not be a breaking change because of if (navigator) { // browser.
   c. If the future date is “22” and the conditions are “after we’ve done sufficient outreach”, then what does that look like? What could we do that’s more impactful than what’s already happened, shipping the break in 21.0.0?
   d. If the future conditions are “when we’ve added the properties that we think the API needs” then as I’ve written above I think we’ve already done that when we added userAgent. We have PRs that look like they’re landing soon for navigator.platform and navigator.language/languages. That would bring us into parity with the union of properties supported by Bun and Deno. I’m not sure what, if any, properties we’re likely to add beyond that; anything that we choose not to support will be a similar unresolvable break along the lines of if (navigator) itself.

I think we need to give our users a bit more credit here. They understand that semver-major releases of Node have breaking changes, and that some work on the user’s side might be required as a result, but that the benefit is new functionality that wouldn’t have been possible otherwise. Adding navigator increases the universe of dependencies that can run in Node without some kind of transformation at build time, where a library written for a browser can just run as written. That’s a big benefit! That’s why we add web standards in general. And we have a lot of users who write frontend or full-stack apps who appreciate that benefit, and are willing to make updates as needed to help us get closer to that goal. Indeed, those are the same users who are the ones who might have been broken by navigator‘s addition, and I haven’t seen a drumbeat from them that we should flag or remove it.

[ljharb]

There is of course a 5th option, expose it unflagged, just not as a global. That way, anyone who wants to write universal code can do so explicitly with a small fallback, and no code risks being broken.

[mcollina]

I really @ljharb option.

[Uzlopak]

I fixed the issue in echarts.

[legendecas]

True, but it is a reasonable assumption for code to have made that a non-browser will never have navigator because they don't (and never will) navigate - that's the kind of ecosystem breakage that adding it as a global risks.

Web workers have a navigator as well and it doesn't navigate and doesn't have access to window or document. Environment detection for window or document access based on the presence of a global navigator is broken even in browser environments.

Given that the issues (#50412 (comment)) are fixed by the addition of navigator.userAgent, I believe the path forward to achieve Web API compatibility would be @jasnell's proposal to add a flag --no-global-navigator to hide the globally installed navigator for those who still need the option to avoid breakage.

[ljharb]

It’s only broken there for the extreme minority of code that’s run in a web worker. worker_threads would be analogous, but regular node is where the risk of breakage is.

[mcollina]

I changed my mind on this.
Unless other major breakages shows up, I think we can keep it.

[joyeecheung]

I think in hindsight we probably should've done a runtime deprecation of first access to globalThis.natvigator to give this breakage a grace period, not sure if it's still timely enough to put our implementation behind a flag for now and do the deprecation.

[aduh95]

FYI I have opened an alternative in #50562 which adds a flag but keep the global by default (I.e. with the other PR it’s opt-out, while this one was opt-in). I figured opening a new PR was more logical as those who reviewed this PR may have very different opinions on the other one, happy to get your reviews, hopefully we can get consensus on either PR (and it doesn’t look it’s going to happen on that one).

[they-cloned-me]

Sorry to ping you @mcollina, but this thing is still fuckin’ up Sass/Dart

[GeoffreyBooth]

this thing is still fuckin’ up Sass/Dart

How so? The issue you linked to was closed as completed three months ago and hasn’t had any activity since.

[they-cloned-me]

this thing is still fuckin’ up Sass/Dart

How so? The issue you linked to was closed as completed three months ago and hasn’t had any activity since.

Hey, unfortunately it still exists.
I just upgraded a project to use node 21, alongside updating sass to the latest version and all hell broke lose.

Error

Pre-transform error: Preprocessor dependency "sass" failed to load: 
Cannot read properties of undefined (reading 'indexOf')

Pre-transform error: Preprocessor dependency "sass" failed to load:  
Cannot read properties of undefined (reading 'pop') 

Environment and package details

------------------------------
- Operating System: Darwin (Sonoma 14.3 Beta)
- Node Version:     v21.0.0
- Nuxt Version:     3.9.0
- Package Manager:  npm@10.2.0
- Sass:             ^1.69.6
------------------------------

Almost exactly the same thing that was described in the original issue.

Prior to this update, I had no issue with sass.
What sucks is that if I run it on Node 20, it works perfectly ☹️

[GeoffreyBooth]

• Node Version: v21.0.0

The current latest version of the 21.x line is 21.5.0. It might very well fix your issue. A comment on a related issue on the sass repo says that Node 21.1.0 resolves it for them.

[they-cloned-me]

• Node Version: v21.0.0

The current latest version of the 21.x line is 21.5.0. It might very well fix your issue. A comment on a related issue on the sass repo says that Node 21.1.0 resolves it for them.

Thanks for your help and patience.
Indeed, Node 21.5 fixes it!

Super thank you 🙏🏿