Providing glob pattern to input flag in cli does not produce TS .d.ts files in the same directory of the respective schema file

[neophyt3]

How to reproduce

I copied the below snippet and pasted in package json

{
  "scripts": {
    "compile-schemas": "json2ts -i src/**/*.schema.json"
  }
}

My project folder structure is like

|-src
  |-controllers
  |-plugins
    |-env
      |-some_schema.schema.json
  |-otherDIRs...

I ran npm run compile-schemas and I get the compiled types in console output

Expected Output

As per the help menu of CLI, it is mentioned
With no OUT_FILE and when IN_FILE is specified, create .d.ts file in the same directory.
so I was expecting types definitions to be produced in the env folder but it did not create any files and instead it printed in STDOUT
Is this intentional? coz I could raise PR but I got confused whether this is intentional or bug when I looked into the source code.

[bcherny]

Hey there! This is the expected behavior. See

json-schema-to-typescript/test/testCLI.ts

Line 30 in 335c425

test('file in (-i), pipe out', t => {

.

To output to a directory, use -o or -o src:

json-schema-to-typescript/test/testCLI.ts

Line 104 in 335c425

"node dist/src/cli.js -i './test/resources/MultiSchema/**/*.json' -o ./test/resources/MultiSchema/foo/bar/out"

.

I'll leave this issue open to collect feedback. We can change this behavior if someone wants to make a good case for it :)

[ajotaos]

This is happening to me as well, based on the docs, I understood that without -o it would create the files in the same directory.

[Blfrg]

As noted by the others on this thread (thumbs up included)
This help declaration line:

With no OUT_FILE and when IN_FILE is specified, create .d.ts file in the same directory.

leads one to believe that given the following command:

json2ts -i 'src/**/*.json'

One should be able to expect these input files:

/src/foo/bar/one.json
/src/foo/bar/two.json
/src/baz/other.json

to output:

/src/foo/bar/one.d.ts
/src/foo/bar/two.d.ts
/src/baz/other.d.ts

If this were the case, one could maintain schemas and definitions in the same folder with the source.ts which is relying on them.
Doing so keeps the files grouped, scoped, modular and reusable across projects.

The current flags and resulting output only allow for a single merged > output.d.ts
With sufficiently large definition sets and file counts this would prove undesirable and may group together many unrelated things.

Also bear in mind the command as it is documented - does not "create .d.ts file in the same directory" (or create a file anywhere).
One still has to pipe it somewhere to have a file created, which doesn't match the documented behavior and example.
Easy enough to ascertain once the output is seen, but unexpected to say the least.
Honestly I wondered if the STDOUT I saw was confirming what was parsed and written into the files
until I checked the expected output locations to find the files weren't there.

In fact, we're experiencing the behavior of the documentation line following the one we're attempting to utilize:

With no OUT_FILE nor IN_FILE, write to standard output.

This may constitute this as a bug instead of intended behavior.

---

An alternative developer flow is to recreate the /src folder structure under /schemas
and anticipate the same output structure under /types

This structure forces one to browse 3 similar structures (src/**/, /schemas/**/, /types/**/)
to bring all the pieces together for review.
Though this structure has it's place for more general defaults or broadly used schemas/types.

---

An even less desirable devloper flow would be multiple calls to json2ts
with varying -i + -o for every possible /src/**/ folder set:

json2ts -i src/foo/bar/ -o /src/foo/bar/
json2ts -i src/baz/ -o /src/baz/
json2ts -i src/other/ -o /src/other/

This would naturally be better suited as a script
and thus becomes additional overhead to manage and modify throughout the projects growth.
Depending on your project size, structure, and change rate, this could be very burdensome.

---

-i /schemas -o /types + -i /src/**/*.json is capped at 2 common i/o variations and is easy to maintain.

---

Note: each source file has the 'auto generated - do not edit' banner in the piped output making for a lot of redundant comments in the merged output file.

/* tslint:disable */
/**
 * This file was automatically generated by json-schema-to-typescript.
 * DO NOT MODIFY IT BY HAND. Instead, modify the source JSONSchema file,
 * and run json-schema-to-typescript to regenerate this file.
 */

export interface OneSchema {
  example: string;
}
/* tslint:disable */
/**
 * This file was automatically generated by json-schema-to-typescript.
 * DO NOT MODIFY IT BY HAND. Instead, modify the source JSONSchema file,
 * and run json-schema-to-typescript to regenerate this file.
 */

export interface TwoSchema {
  example: string
}
/* tslint:disable */
/**
 * This file was automatically generated by json-schema-to-typescript.
 * DO NOT MODIFY IT BY HAND. Instead, modify the source JSONSchema file,
 * and run json-schema-to-typescript to regenerate this file.
 */

export interface OtherSchema {
  example: string
}

This also suggests to me that individual output files is the real intention here.

---

@bcherny Please consider the following;

Since modifying the existing behavior would be considered a breaking change, and likely preferable to be avoided;
I propose implementing a 'feature flag' to introduce the desired behavior.
I'm not sure what the most indicative flag(s) would be, but it could be something like:

-m, --multi, --multipleOutputFiles (cannot be used with -o)

Example usage:

json2ts -i 'src/**/*.json' -m

This feature flag should keep track of all globbed inputs found
and create an output file in the same absolute [resolved] path
named exactly the same, except with the extension replaced by .d.ts

[Buckwich]

For example NRWL/NX (a monorepo build system) often uses schema.json and schema.d.ts in its executors and generators. So I would like to run json2ts -i './**/schema.json' -m to make it simpler to keep them in sync.

I made a possible implementation as suggested by @Blfrg in #501

[kennethnym]

I wrote a simple script that generates *.schema.d.ts files next to their JSON schema files in src/**/*.schema.json:

const fs = require("fs")
const path = require("path")
const glob = require("glob")
const {
  compileFromFile
} = require("json-schema-to-typescript")

glob(
  // assuming this script is placed at the root of your project
  path.join(__dirname, "src", "**/*.schema.json"), {},
  (err, paths) => {
    if (err) {
      console.error("Cannot scan for JSON schema files.")
      return
    }

    if (paths.length === 0) {
      console.log("No JSON schema files found.")
      return
    }

    Promise.all(
      paths.map((schema) =>
        compileFromFile(schema).then((ts) =>
          fs.writeFileSync(
            path.join(
              path.dirname(schema),
              path.basename(schema).replace("json", "d.ts")
            ),
            ts
          )
        )
      )
    ).then(() => {
      console.log("Generated types for JSON schemas.")
    })
  }
)

[bcherny]

Circling back after reading through the discussion, here's what I'd suggest:

1/ Currently, when your input -i is a glob and you specify and output -o directory, we flatten the input structure.

For example, imagine the following structure:

• schemas/
  • a.json
  • b.json
  • c/
    • d.json

json2ts -i "./schemas/**/*.json" -o ./schemas

This results in:

• schemas/
  • a.json
  • a.d.ts
  • b.json
  • b.d.ts
  • c/
    • d.json
  • d.d.ts

2/ The right default behavior, in hindsight, is probably to preserve directory structure (rather than automatically flatten it). To preserve backwards compatibility, let's introduce a new flag --preserve-directory-structure to control this behavior. The flag should default to false. The CLI should throw if --preserve-directory-structure is specified but --output is not.

Example:

a/ Current behavior

json2ts -i "./schemas/**/*.json" -o ./schemas --preserve-directory-structure=false

Result:

• schemas/
  • a.json
  • a.d.ts
  • b.json
  • b.d.ts
  • c/
    • d.json
  • d.d.ts

b/ New behavior with --preserve-directory-structure enabled

json2ts -i "./schemas/**/*.json" -o ./schemas --preserve-directory-structure

Result:

• schemas/
  • a.json
  • a.d.ts
  • b.json
  • b.d.ts
  • c/
    • d.json
    • d.d.ts

c/ Invalid flags:

json2ts -i "./schemas/**/*.json" --preserve-directory-structure

Throws, telling the user that --preserve-directory-structure can only be used when --output is specified.

I'd welcome feedback on this, and implementations with tests.

[spthiel]

Regarding your idea for implementation details.
Pretend we call
json2ts -i "./src/schema/**/*.json" -o ./src --preserve-directory-structure
with this structure:

src/
└── schema/
    ├── a.json
    ├── b.json
    └── c/
        └── d.json

is the expected output

src/
├── schema/
│   ├── a.json
│   ├── b.json
│   └── c/
│       └── d.json
├── a.d.ts
├── b.d.ts
└── c/
    └── d.d.ts

(aka: root folder of schema is interpreted as ./src/schema - the path segment up to the first glob pattern of the input)
or

src/
└── schema/
    ├── a.json
    ├── a.d.ts
    ├── b.json
    ├── b.d.ts
    └── c/
        ├── d.json
        └── d.d.ts

(aka: root folder of schema is interpreted as ./src/ - the path segment of the glob pattern matching the output directory)

Conversely using
json2ts -i "./schema/**/*.json" -o ./src --preserve-directory-structure
with

src/
schema/
├── a.json
├── b.json
└── c/
    └── d.json

do we expect

src/
├── a.d.ts
├── b.d.ts
└── c/
    └── d.d.ts
schema/
├── a.json
├── b.json
└── c/
    └── d.json

(aka: root folder of schema is interpreted as ./schemas - the path segment up to the first glob pattern of the input)
or

src/
└── schema/
    ├── a.d.ts
    ├── b.d.ts
    └── c/
        └── d.d.ts
schema/
├── a.json
├── b.json
└── c/
    └── d.json

(aka: root folder of schema is interpreted as . - the path segment of the glob pattern matching the output directory)
?

Imho there is quite a bit of uncertainty (and implementation difficulty) in how your idea would work in practice and I would favour only allowing --preserve-directory-structure to work without an output given and only support creating the .d.ts files next to the .json file