Please provide a json basic type

[NoelAbrahams]

Hi,

JSON is an important data exchange format. At present there is no explicit way to annotate an object as pure JSON. We were expecting union types to solve this problem via the following:

    interface Json {
        [x: string]: string|number|boolean|Date|Json|JsonArray;
    }
    interface JsonArray extends Array<string|number|boolean|Date|Json|JsonArray> { }

There are currently a number of problems with this.

• Contextual typing is absent in parenthetic expression Consider contextually typing parenthesized expressions #920

    interface Id extends Json {
        id: number;
    }
  
    var z = (): Id => ({id: 'foo'}); // Error: Missing index signature

• Contextual typing does not flow from base to subclasses Consider contextually typing class member functions by their base class/interface members #1373

class Bass {

    f(): Id{ return undefined;}
}

 // Error: missing index signature
class Foo extends Bass {
    f() { return { id: 10 }} 
}

• Object literals require explicit index signatures Recursive type with index signature issues spurious error #1889:

    interface Foo extends Json {

        foo: { val: string }; // Error: Not assignable

    }

• Other problems:

  // Error: Missing index signature
  var result: Id[] = 'a|b'.split('|').map(item => {
        return { id: 0 };
  });

The first two problems look likely to be resolved, but not the last two.

This motivates the introduction of a natively supported json type that conforms to the JSON spec.

It should be possible to derive a custom JSON object from the json type with non of the problems above. Furthermore, since JSON is a data exchange format it should be possible to mark fields as nullable on types deriving from json (which would trigger a type guard).

I am of course aware that this is all rather vague!

[RyanCavanaugh]

What kind help could the compiler provide with this?

In the case where you're receiving JSON, the fact that you know everything is a string, number, boolean, array, null, or object isn't terribly useful? You could prevent yourself from passing a JSON object into an function expecting a Function, but in all other cases it's within spitting distance of any.

In the case where you're emitting JSON, the type system doesn't really encapsulate all the aspects of JSON that are important. JSON.stringify skips properties from the prototype and properties of type Function, both of which are going to be common when doing class-based programs. I think people wouldn't like it if JSON.stringify(myClass) were an error, but now we're really back in the case where it's practically any (minus calling JSON.stringify(someFunc) which is hopefully rare and a quick error to spot). And obviously we have no way to warn you about circular data structures that will cause an error.

[DanielRosenwasser]

For the record:

• Not only is Consider contextually typing parenthesized expressions #920 likely to be resolved, it currently is in master.
• Consider contextually typing class member functions by their base class/interface members #1373 needs a proposal first.
• I don't think it's unfeasible for Recursive type with index signature issues spurious error #1889 to be resolved, but it is a difficult problem.

[NoelAbrahams]

The discussion is with respect to the case where both the client and server are written in JavaScript. For this case the JSON is shared and can be quite large and complex.

There are use-cases even where Ajax is not involved where data transfer only works when the payload is string. An example of this is the HTML5 dataTransfer object. For this case having the json type ensures the payload is correct on both ends and that JSON.parse is not going to fail.

In the case where you're receiving JSON, the fact that you know everything is a string, number, boolean, array, null, or object isn't terribly useful

True if we were simply looking at how the received JSON is subsequently used. As I mentioned above, the json annotation helps to ensure invalid data structures are not created in the first place, e.g.

interface Foo extends json {
 image: File; // error
}

In the case where you're emitting JSON, the type system doesn't really encapsulate all the aspects of JSON that are important. JSON.stringify skips properties from the prototype and properties of type Function, both of which are going to be common when doing class-based programs.

The point is classes encapsulate behaviour. JSON represents data. People who .stringify classes are a special breed - rather like the dodo 😄.

Edit: Ignore the following in the light of strict-null-checks
I am also very interested in exploring what (if anything) can be done to describe nullability in JSON types. Pretty much all the null reference problems in my experience occur around the use of JSON. Since null is explicitly a JSON value, perhaps it should be an error to access a property on a json-derived type without a type guard:

interface Foo extends json {
  bar: string;
}

var x: Foo;
x.bar.toString(); // Error
x.bar && x.bar.toString(); // okay

[NoelAbrahams]

@jbondc, I'm not sure that I understand. How does that help to provide compile-time safety for JSON?

[NoelAbrahams]

Seems like not-null types

Yes, but shouldn't break existing code ;) I'm trying to explore the implications of defining a json type, same as any or int. As I noted above, this type would be an ideal candidate to have default nullable properties, firstly because that's part of the JSON spec and secondly, since we use JSON to transfer data, that's where a lot of the null reference problems occur.

JSON is a structured object, so an interface should be able to describe with 100% fidelity the data it contains.

Yes, but with int for example the following is an error:

var x: int = "10";

But not so for interfaces:

interface Foo extends json {
   html: HtmlElement; // We would like this to be an error
}

[AlicanC]

Since when JSONs are objects?

type Json = string;

[NoelAbrahams]

@AlicanC,

JSON is string over the wire, but when one calls JSON.parse it becomes an object. This object is a subset of regular JavaScript and that is what we'd like to model.

[AlicanC]

Yes, the JSON becomes an object and it's not JSON anymore. JSON is what you give to JSON.parse(), not what you get from it. What you get is a value. Maybe you guys should rename your types to JsonValue.

[NoelAbrahams]

Maybe you guys should rename your types to JsonValue.

Can you elaborate? Which guys and what types?

[Elephant-Vessel]

Seems like the problems that you described above are resolved now? Trying out this approach and it looks like it's working just fine.

Also: I think this has deeper utility that just representing some arbitrary format like JSON. It enables us to more clearly distinguish data from operations on data. That's a quite fundamental piece of our domain as software developers, I don't think that would be a totally useless concept to have available and tangible in code, even if OOP still might be the norm.

[electricessence]

https://github.com/electricessence/TypeScript.NET/blob/master/source/JSON.d.ts

This is what I have, but I'm not sure if it's adding any value really. I still have to constantly add <type> constraints everywhere to make sure it's correct.

I'm wondering if it's simply more effective to do run-time validation than be concerned with compile time constraints.

I think part of the challenge is that a JSON blob can consist of either a map or an array and in those cases the indexers are different.

So for example, I've had to do use <T extends JsonMap | JsonArray> for the expected output parameter.
https://github.com/electricessence/TypeScript.NET/blob/master/_utility/file-promise.ts#L59-L80

And therefore still have to pass one or the other in order for it to work as show in the first link.

So again, I wish there were more examples of where typing JSON helps.

[JasonKleban]

This type would be useful for data that must be able to roundtrip through JSON serialization and deserialization, such as data to be stored or otherwise faithfully reproduce across a remoting boundary.

It seems to me that this simple definition of Json works to provide some guidance for some library that works with arbitrary user data in that way, but there are a few problems. I don't know if there is already a solution to them, or if not, what the solution ought to be. I'm just saying it would be useful. Not sure why the type system needs an explicit extend JsonMap here to recognize the compatibility. I don't think there's any way to disallow classes from extending Json.

export interface JsonMap { [member: string]: string | number | boolean | null | JsonArray | JsonMap };
export interface JsonArray extends Array<string | number | boolean | null | JsonArray | JsonMap> {}
export type Json = JsonMap | JsonArray | string | number | boolean | null;

interface Document extends JsonMap {
    one: string;
    two: boolean;
    3.141592: "pi" | boolean;
}

function clone<T extends Json>(data : T) { return <T>(JSON.parse(JSON.stringify(data))); }

var a : Json = "Hello"; clone(a).toLowerCase();     // 👍
var b : Json = 42; clone(b).toExponential();        // 👍
var c : Json = true; { let t = clone(c); }          // 👍
var d : Json = null; { let t = clone(d); }          // 👍
var e : Json = [1, 2, ""]; clone(e).length;         // 👍
var f : Json = {}; {let t = clone(f); }             // 👍 f : JsonMap

var g : Json = { a: "Hello" }; clone(g).a;          // 👍 g .. l : JsonMap, not especially useful as written
var h : Json = { b: 42 };
var i : Json = { c: true };
var j : Json = { d: null };
var k : Json = { e: [1, 2, ""] };
var l : Json = { e: { 5.4: "foo", mixed: "key types" } };

var m : Json = () => "baz";                         // ERROR 👍
var n : JsonMap = { a : "bar", fn: () => "baz" };   // ERROR 👍
var o : JsonMap = [{ a : "bar", fn: () => "baz" }]; // ERROR 👍

{
    let p : Document = { one : "foo", two : false, 3.141592: "pi" };
    let t = clone(p); // t : Document 👍
    t.one; // : string 👍
    t.two; // : boolean 👍
    let tt = t[3.141592]; // tt : "pi" | boolean 👍
    let tu = t[3.1415]; // tu : string | number | boolean | null | JsonArray | JsonMap
}

var q = () => "baz"; clone(q);                      // ERROR 👍
var r = { a : "bar", fn: () => "baz" }; clone(r);   // ERROR 👍
var s = [{ a : "bar", fn: () => "baz" }]; clone(s); // ERROR 👍

class MyClass implements JsonMap { [key : string] : string }  
clone(new MyClass()) instanceof MyClass; // 😱 false

// Structural typing isn't enough for otherwise-compatible interfaces
interface OtherLibDocument {
    one: string;
    two: boolean;
    3.141592: "pi" | boolean;
}

{
    let p2 : OtherLibDocument = { one : "foo", two : false, 3.141592: "pi" };
    let t = clone(p2); // 😱 Property 'includes' is missing in type 'OtherLibDocument'
}

interface OtherLibDocument2 {
    one: string;
    two: boolean;
    3.141592: "pi" | boolean;
}

// Definition-Merge other libraries' compatible interfaces
interface OtherLibDocument2 extends JsonMap { }

{
    let p3 : OtherLibDocument2 = { one : "foo", two : false, 3.141592: "pi" };
    let t = clone(p3); // t : Document 👍
    t.one; // : string 👍
    t.two; // : boolean 👍
    let tt = t[3.141592]; // tt : "pi" | boolean 👍
    let tu = t[3.1415]; // tu : string | number | boolean | null | JsonArray | JsonMap
}

[aluanhaddad]

@NoelAbrahams

The point is classes encapsulate behaviour. JSON represents data. People who .stringify classes are a special breed - rather like the dodo 😄.

I think you are on to something here 😆 but, judging by the number of Angular users on Stack Overflow who expect the simple use of TypeScript to automagically transform JSON.parse into Newtonsoft.Json.JsonConvert.DeserializeObject, they have no idea they are destined for extinction 😝.

[niedzielski]

Another "me too." Here's what we ended up with:

export type JSONPrimitive = string | number | boolean | null;
export type JSONValue = JSONPrimitive | JSONObject | JSONArray;
export type JSONObject = { [member: string]: JSONValue };
export interface JSONArray extends Array<JSONValue> {}

Unfortunately, usage on arbitrary types requires a type assertion like unmarshal(jsonType as {}).

[ghost]

Looks like not too late to the party. Google search didn't take long to hit on this issue & what others suggest. What I came up with is basically same as @niedzielski. Despite caveats, it's simple enough to be useful and sits a comfortable distance from settling for just any.

In difference to @niedzielski's version, preferring Json to all-caps JSON prefix and thinking JsonMap instead of JsonObject (as others also have above) as feel "Map" is less overloaded than "Object".

Also going with AnyJson rather than JsonValue.

End result is

type AnyJson =  boolean | number | string | null | JsonArray | JsonMap;
interface JsonMap {  [key: string]: AnyJson; }
interface JsonArray extends Array<AnyJson> {}

Ambivalent about whether or not to separate out JsonPrimitive.

[bitjson]

For those finding this issue after the release of TypeScript 2.9, TypeScript now has Support for well-typed JSON imports.

TypeScript is now able to import JSON files as input files when using the node strategy for moduleResolution. This means you can use json files as part of their project, and they’ll be well-typed!

These JSON files will also carry over to your output directory so that things “just work” at runtime.

[nevir]

This would still be quite useful. For example, I would like to describe a RPC interface via typescript, but assert that inputs/outputs are JSON-serializable. Unfortunately, even with the JSON* interfaces described above, that doesn't seem feasible w/ current typescript:

export type JSONPrimitive = string | number | boolean | null;
export type JSONValue = JSONPrimitive | JSONObject | JSONArray;
export type JSONObject = { [member: string]: JSONValue };
export interface JSONArray extends Array<JSONValue> {}

export interface ServiceDeclaration {
  [key: string]: (params?: JSONObject) => Promise<JSONValue>;
}

// Expected: No errors.
interface MyService extends ServiceDeclaration {
  // Error: Property 'doThing' of type '(params?: { id: string; } | undefined) => Promise<string>' is not assignable to string index type '(params?: JSONObject | undefined) => Promise<JSONValue>'.
  doThing(params?: { id: string }): Promise<string>;
}

---

One trick that gets me closer is to have template types that ask for keys explicitly, to drop the index type (e.g. the same as typescript’s built in Record type):

export type IsJSONObject<TKeys extends string = string> = { [Key in TKeys]: JSONValue };

interface Foo {
  id: string;
}

function doThing(params: JSONObject) {}
function doThing2<T>(params: IsJSONObject<keyof T>) {}

const foo: Foo = null as any;
// Error: Argument of type 'Foo' is not assignable to parameter of type 'JSONObject'.
//   Index signature is missing in type 'Foo'.
doThing(foo);
// No error!
doThing2(foo)

I'm not sure how to express that in terms of ServiceDeclaration, though…

[toolness]

Just wanted to mention that this type would also be very useful for me in light of the TS 3.1 breaking change "narrowing functions now intersects {}, Object, and unconstrained generic type parameters". My existing code essentially has a type that could be either something JSON-serializable or a function; before TS 3.1, the former was blissfully expressed as an unconstrained generic, but now I need to figure out how to narrow its type, and doing that would be much easier if a JSON type existed. (On the other hand, I might be going about solving this problem in entirely the wrong way.)

[felixfbecker]

My use case is strictly typing cloneable values that can be passed to WebWorkers through postMessage(). If an object contains a method for example, it will through a runtime error when trying to send it to the Weorker. This should be possible to catch at compile time.

[soanvig]

Just bumping this up, providing my case for support for this.

I can have a function, that receives something which is serialized with JSON.stringify under the hood. I don't want function's user to KNOW that out of thin air.

It's really easy to make a mistake for example providing data that will be unexpectedly stringified (objects like Date), and leading to incorrect JSON.parse behavior.

Given in this discussion JsonValue type works OK, but fails for { [x: string]: JsonValue }. It doesn't work if the object is strict interface. Interface doesn't have index type.

[isoroka-plana]

I wanted to create a function that would allow downloading some data as JSON for debugging purposes. For now, I am forced to use unknown, any or put together my own type, but really what I need is any JSON-serializable data just as some of the comments have suggested.

An example of how I would like types on my function to behave:

openDebugInfoDownloadDialog('message', { key: 'value' }); // valid

openDebugInfoDownloadDialod('message', {date: new Date(), func: () => {} }) // invalid

[DanielRosenwasser]

TypeScript has come a long way since this issue was filed. I think it is sufficiently expressive enough to describe a json-able type, and there's a bunch of examples above of how one might do that. The one I defined in #57034 was this:

export type Json =
    | string
    | number
    | boolean
    | null
    | Json[]
    | { [key: string]: Json }
    ;

I don't think it warrants adding to lib.d.ts. Generally we've avoided adding new global types unless they are used somehow by the type-checker itself (or by some built-in functions), and this would not be. So defining it locally or in a library is fine for now. So I'm going to close this issue.

[nabe1653]

It's sad ending. Still I think built-in functions JSON.parse should return Json at least.

[kachkaev]

I use JsonArray / JsonObject / JsonValue from type-fest. These types do job perfectly!

Using some Json type from TS core would be a bit more convenient for end users in short term, but I understand adding them today means more legacy to manage in the future.

[kachkaev]

One thing that’d be great to see in the core is unknown instead of any in JSON.parse / stringify:

TypeScript/src/lib/es5.d.ts

Lines 1137 to 1159 in b75261d

	interface JSON {
	/**
	* Converts a JavaScript Object Notation (JSON) string into an object.
	* @param text A valid JSON string.
	* @param reviver A function that transforms the results. This function is called for each member of the object.
	* If a member contains nested objects, the nested objects are transformed before the parent object is.
	*/
	parse(text: string, reviver?: (this: any, key: string, value: any) => any): any;
	/**
	* Converts a JavaScript value to a JavaScript Object Notation (JSON) string.
	* @param value A JavaScript value, usually an object or array, to be converted.
	* @param replacer A function that transforms the results.
	* @param space Adds indentation, white space, and line break characters to the return-value JSON text to make it easier to read.
	*/
	stringify(value: any, replacer?: (this: any, key: string, value: any) => any, space?: string | number): string;
	/**
	* Converts a JavaScript value to a JavaScript Object Notation (JSON) string.
	* @param value A JavaScript value, usually an object or array, to be converted.
	* @param replacer An array of strings and numbers that acts as an approved list for selecting the object properties that will be stringified.
	* @param space Adds indentation, white space, and line break characters to the return-value JSON text to make it easier to read.
	*/
	stringify(value: any, replacer?: (number | string)[] | null, space?: string | number): string;
	}

I was caught up by any a couple of times before.

Curious to know if a switch to unknown is possible in some next major version. Or are ESLint rules like @typescript-eslint/no-unsafe-argument or @typescript-eslint/no-unsafe-assignment the right thing to use here?

[yawaramin]

Wait, why would adding a JSON type for the return type of JSON.parse be a maintenance burden? JSON is not exactly a fast-changing standard, right? 🤔

[milesj]

JSON.parse should always be any/unknown because JSON data is just that, unknown. Even if you are parsing a JSON file that you know the structure of, or receiving an HTTP response you know the structure of, the JS/TS interpreter does not, so having a type that is string | null | array | object | etc is entirely useless because you can't actually access any of the data in the result without type narrowing.

You'll still need to do typeof checks, Array.isArray checks, and every other kind of check for it to actually be accurate and type-safe. You're better of using an explicit type, like interface UserResponse {...} and casting it, or a robust solution like zod, joi, etc. But again, a JSON type is kind of useless.

At minimum, I agree with @kachkaev that the type should be unknown over any because any is just bad.

[Peeja]

It's not useless if you're going to pass it to something that takes JSON objects.

For instance, as described above, a toJSON() function should return a JSON object. It would be valuable to type it as such, to ask the implementation to prove that it returns something JSON-y. The result of a JSON.parse() should be a valid return value for a toJSON() method.

[reverofevil]

But again, a JSON type is kind of useless

Those typeof checks you mentioned are harder to do on any or unknown. For a Json you know for a fact there would be no symbol, function, undefined, or bigint.

Also try JSON.stringify(undefined) and see what the resulting string is. How would you use any or unknown argument type to avoid that?

[milesj]

Those typeof checks you mentioned are harder to do on any or unknown. For a Json you know for a fact there would be no symbol, function, undefined, or bigint.

In almost all cases you would be checking for number/string/object/null/etc explicitly, so those other branches can be ignored. And again, the fact that this would require so much type narrowing anyways just isn't worth the effort. Either trust the source outright, or use zod/etc.

Also try JSON.stringify(undefined) and see what the resulting string is. How would you use any or unknown argument type to avoid that?

Pretty sure this could just be solved with {} or {} | null.

[reverofevil]

In almost all cases you would be checking for

What about exhaustive checks?

or use zod/etc

How would you implement zod? What if you cannot tolerate extra 57KiB in your bundle?

Pretty sure this could just be solved with {} or {} | null.

JSON.stringify(() => {})