JSON modules

[MylesBorins]

Hey All,

I'd like to explore support for importing json, similar to how Node.js currently support require('./some.json')

Expected behavior:

imported json would export an object or array with the content from the provided json file.

Why:

Currently the was to get JSON is via fetch + import.meta.url... and requires a bit of back and forth... the eventual result is a promise to resolve to the json object. Being able to statically import and parse json would allow us to import specific symbols, export symbols, and asynchronously get resources during the fetch phase.

What are the next steps?:

I'm assuming a spec change in the HTML spec, but unsure what else we would need to do. Thoughts?

[domenic]

So concretely, for those not familiar with Node.js's support here, this would mean that if you import something with a JSON mime type, you would get back the result as if JSON.parse were applied to the files contents (decoded as UTF-8). If it doesn't parse as JSON, then the module ends up in the error state, similar to JavaScript parse errors.

As for next steps, the main one is implementer interest. /cc @whatwg/modules; does this seem like a reasonable addition?

[bmeck]

I think it is reasonable, but would note that this is for static data, whereas fetch() would get new data each time it is used; import() will not get new modules for each use. As long as this is well understood it seems good for a variety of usages that we see in bundler usages.

[tabatkins]

I'm happy with certain well-known and commonly-used content-types getting good module support, even if it's just a "trivial" wrapper around a constructor or function call. JSON, CSS, etc are all good.

[annevk]

This came up at the last W3C TPAC too, I raised WICG/webcomponents#770 at the time. If the complexity is indeed branching on a JSON MIME type and then using parse JSON from bytes I think Mozilla would be supportive. If we also want to advertise support to servers in some manner the discussion might become a little more involved. (It seems apart from an object or array this could also return a primitive, by the way.)

[thw0rted]

I don't know if it matters to the standards process, but there is some discussion of the topic already. If you look at the answers on that SO question (and at linked/related questions), you'll see there is some confusion as well. Because many (most?) people writing in ES6 or later end up transpiling via Babel -- which means using a module loader provided by the transpiler, rather than the browser-native one -- they can write import { x } from "foo.json" today, and as far as they are concerned, it works. Presumably this is because the Babel runtime loader is actually more advanced than the browser native implementation.

[bmeck]

There are some complexities if we are wishing to exactly match tooling as it exists today.

To note, tools supporting JSON as an importable module type do allow both named and default exports. The JSON provided always assigns the result value to the default and then "picks" the relevant Object properties to expose as named exports e.g.

{"x": {}}

~=

export default {x: {}}
// export x = default.x //

This kind of aliasing where updating default.x updates x isn't available to Source Text Module Records, but that is probably fine.

Additionally, for non-Identifier properties, the only way to access values would be through the default export due to being unable to import non-Identifier bindings.

import locales from './locales.json';
locales[navigator.language];

I think the simplest solution of not supporting named exports would be fine for now and encourage a more uniform usage.

[ljharb]

Conceptually a json file is a single thing; that babel allows named imports from it is merely a product of it conflating destructuring with named imports. I definitely agree json should be confined to default import.

[MylesBorins]

What are the next steps for getting this off the ground? Spec Text? Implementor interest?

I'm happy to help drive this but need some mentorship on the process

[domenic]

Implementer interest would be ideal, otherwise spec text might be wasted effort. But, you could start spec and test work ahead of implementer interest, if you're OK taking the risk.

I'm happy to mentor on spec/test work when you're ready; let's discuss in a more high-bandwidth medium like IRC or offline.

[MylesBorins]

I will be at tc39 next week and can work on getting some implementer interest.

[jhnns]

Conceptually a json file is a single thing; that babel allows named imports from it is merely a product of it conflating destructuring with named imports. I definitely agree json should be confined to default import.

As background information why this complexity was introduced: aliasing the top-level JSON keys to named exports makes the JSON file tree-shakable on the first level. So if only x is used in the application, the rest can be removed. This can make a notable difference if the JSON file is big enough.

[guybedford]

Tree shaking isn't really a good argument for named exports anymore. Build tools can check statically analyzable member expressions and determine if an object binding escapes the static analysis.

[robpalme]

Let's consider multiple modules that depend on the same remote source of static JSON. With today's approach of having each user fetch & JSON.parse the result, they get their own object which is pristine.

If the module loader is now holding a shared mutable copy of the data, how can a single user/callsite confidently access the original data?

Will there be a special way to request the untainted copy? Or should we consider JSON modules being deeply frozen by default?

[thw0rted]

If anybody can come up with a reasonable use case for mutable JSON imports, I'd like to hear it, but otherwise freezing seems like a simple solution.

[jhnns]

Tree shaking isn't really a good argument for named exports anymore. Build tools can check statically analyzable member expressions and determine if an object binding escapes the static analysis.

These checks are very difficult and I think there are a lot of situations where the object binding would escape. Using named imports is certainly a lot easier to statically analyze. But I also see that allowing named imports of first-level JSON keys kind of abuses named imports for the sake of static analysis, so I'm not a strong advocate of this feature. Just wanted to give some background information.

Will there be a special way to request the untainted copy? Or should we consider JSON modules being deeply frozen by default?

I also thought about this. I would definitely prefer freezing it but I also don't see a strong reason why the host environment should enforce this. Maybe security reasons?

[guybedford]

Freezing sounds very sensible. Note that freezing would also help static analysis because then even in the case of a reference escaping the analysis, member expressions can still be inlined.

[ljharb]

an alternative to recursively freezing might be if the import was a thunk - a function that returns a new mutable object.

[domenic]

I don't think this is the place to introduce any kind of freezing or recursive freezing into the web platform. The platform is full of shared mutable objects, e.g. window, or JS modules' namespace objects. There's no reason to treat JSON modules' namespace objects specially.

As always, if you want to create a frozen copy of one of these shared mutable objects, your code needs to run first, and do the freezing itself.

[guybedford]

@domenic your argument seems to be nothing more than to state the status quo as canon. I'd have hoped for a more convincing point here.

[domenic]

Consistency is valuable. JSON modules should not depart from JS modules.

[guybedford]

Perhaps this pushes the argument back towards being for named exports then, since that would ensure the consistency guarantees of JS modules? Although it does mean introducing a valid identifier filter unfortunately.

[domenic]

No, JS modules that represent a single thing do use default exports, so the consistency is perfect there.

[Jessidhia]

FWIW, a new feature of webpack@4 was support for JSON-as-modules.

When the JSON document is an object, properties are plucked (when they are valid identifiers) and made available as independent exports, with the default export being guaranteed to be the entire unmodified JSON.parse result (which may not even be an object). If you do actually use only named imports (other than default), it is capable of deleting the provably-unused parts of the object from the bundle.

[domenic]

I realize this feature never seems to have gotten concrete signs of implementer interest. Blink is interested in implementing; any thoughts from Gecko or WebKit? @annevk @jonco3 / @rniwa @Constellation.

Concretely, #4407 has the pull request.

[annevk]

Mozilla is interested.

[dandclark]

Microsoft is on board for implementing this in Blink.

[TimvdLippe]

Should this issue be reopened since #4943 reverted #4407 which closed this issue?

[annevk]

Once someone takes the time to resolve WICG/webcomponents#839 I'm sure the conversation will find its way back to this repository somehow. Not sure we need a new tracking issue or reopen this one.

[trusktr]

Seems like treating JSON file content as a default export is the only way to have consistency across all JSON features.

With JSON files containing content other than objects, for example

true

null

"foo"

123

[1, "2", true]

named imports don't work (obviously).

The only way to keep usage consistent with JSON features is a default export, then in their code people can choose to detect whether the import is a boolean, an object, or something else, or what properties they want to pluck from the default import in case it is an object.

import pkg from './package.json'
import child from 'child_process'

child.exec(pkg.scripts['build:dev'], ...)

If we don't mind being less consistent across JSON features, then the next best option is for default to work as above, but with named imports also available from JSON files that have object values (the same as @Jessidhia last mentioned).

But the question is, is consistency across the JSON features important? Most JSON values are objects, and most keys in those objects are valid identifiers.