Specify (absolute) path for library projects

[andrewvarga]

I'm moving this issue here from codeplex:
https://typescript.codeplex.com/workitem/1414

This feature would help to use "library projects" in a way that you only have to specify the actual path once (eg. with a compiler flag).

I'd love to see this implemented in the official compiler. I think it's small work and adds huge value.
I can have a try at adding this but I'd value some guidelines on how to edit the compiler and how it would make it's way to be part of the official compiler.

[danquirk]

Seems reasonable. If someone has a precise proposal (https://github.com/Microsoft/TypeScript/wiki/Writing-Good-Design-Proposals) that would be great.

[andrewvarga]

Without being too formal (yet) I think the compiler flagged version could work (as proposed in the codeplex issue).
Maybe it would be more convenient to be able to specify a config json file, similarly to requirejs, something like this:

tsc ... --var pathConfig=config.json

config.json contains the absolute paths of the libraries

{
"projectA": "/libs/projectA/src",
"projectB": "/libs/projectB/trunk/src"
}

[bestander]

+1 the current compiler does not provide enough configuration.
Have a look at gulp and grunt tsc wrappers, there are several and they are quite big. This is an indication that current tsc needs more thought.

[andrewvarga]

I'm thinking if a config.json will be used, should we have it as an individual file just for the path config or would there be other uses for a common config.json that could be fed to the compiler. In which case it would have a content similar to this:

{
"path": {
   "projectA": "/libs/projectA/src",
   "projectB": "/libs/projectB/src"
},
otherConfigOption: {...}
}

Regarding implementation and usage, I think it'd be pretty straightforward, just substitute the project's path so this:

/// <reference path="{projectA}/Foo.ts" />

becomes this:

/// <reference path="/libs/projectA/src/Foo.ts" />

It only makes sense to use {projectA} as the first characters of the path string although it wouldn't hurt to allow to place it anywhere.

What do you think?

[andrewvarga]

To me this is the only missing peace from TypeScript to be able use it in production. So please do comment on the above! :)

[ifeltsweet]

This is a big one for me. There absolutely must be a way to import references relative to your project's root.

[basarat]

There absolutely must be a way to import references relative to your project's root

As a current workaround you can use grunt-ts transforms : https://github.com/grunt-ts/grunt-ts#transforms

[andrewvarga]

Actually there's absolute path available for references but it's relative to the root of the hard drive which I think is quite unuseful - couldn't it be changed to be relative to the project's root folder ie. the folder you run the compiler in. Eg. if you compile your files like this:

Main.ts --out $ProjectFileDir$/build/app.js (projectfiledir is a webstorm variable)

then project root is the folder Main.ts is located in.

Nevertheless, even if it did work like this I think there would be a need to use {projectA} like substitutions, because that can work like a symbolic link to a library project.
If you just used the path of the project instead and that path changed you'd have to change it everywhere you use that library project from.

[andrewvarga]

Is there anything I can do to help this move forward?

[ecp3]

Is there a reason why the same lookup strategy as the 'require' statement can't be used? A fully qualified path from the root would then work as expected.

[paztis]

Wow.
It's been over a year since I proposed this solution and it is still not implemented (https://typescript.codeplex.com/discussions/451757).
It is too difficult to develop?

[andrewvarga]

@RyanCavanaugh sent me an answer regarding my questions. This includes tips for implementing if anyone would like to - I didn't have the time to get into it yet.

Here's his reply:

Additional things we’d like to see in a proposal like this would be:

• Detailed motivating examples, with brief commentary on how the current compiler doesn’t meet your needs
• Several example config files, and a description of their schema
• If possible, an implementation (in a fork)

Tips on implementing, since I think this should be fairly straightforward:

• In types.ts:
  • Add a field to CompilerOptions to store the library paths => file paths mappings
• In tsc.ts:
  • Add code somewhere to read the specified config file (since commandLineParser doesn’t do file I/O itself)
  • Modify getSourceFile to apply the path transforms to the provided filename
• In commandLineParser.ts:
  • Add a new commandline option for specifying the config file

Caveats: Our design backlog is very full right now (lots of ES6 features to figure out), so it will be a while before we have a chance to look at anything lower-level. We also might end up folding this suggestion in to the external module resolution proposal if it makes sense.

[eschwartz]

If anyone's working on this, see #1567 for a "motivating example".

TL;DR: RequireJS does not play nicely when you try to use relative module paths from various project locations. Currently the only way to get around this is to define every AMD module in a separate definition file, with the aliased AMD paths.

For an AMD project of any real size, this is a maintenance nightmare.

---

Caveats: Our design backlog is very full right now (lots of ES6 features to figure out), so it will be a while before we have a chance to look at anything lower-level. We also might end up folding this suggestion in to the external module resolution proposal if it makes sense.

I understand the push for more and more features, but I would urge the TypeScript team to take a step back and make sure that existing features are well thought out and documented. I just spent the last week trying to convert a large RequireJS project over to TypeScript. I was disappointed with how many "gotchyas" I ran into when trying to use the TypeScript module system, and how little documentation there was for the problems I experienced.

[basarat]

RequireJS does not play nicely when you try to use relative module paths from various project locations

@eschwartz I've dropped all desire to use AMD for new projects due to all the config nightmares. Even for non-TypeScript projects its painful. Consider browserify or webpack.

[NoelAbrahams]

I've dropped all desire to use AMD for new projects due to all the config nightmares

I've always had a bad feeling about AMD. Internal modules + Uglify is another option.

[eschwartz]

@basarat Where were you 2 years ago to tell me that....?

Here's the dilemma I see: on the one hand, TypeScript has promoted itself by saying that it's really easy to "make the switch". Building in configuration or whatever to support ReqJS quirks would really help make that case. On the other hand, do we really want to design a language spec around the idiosyncrasies of some non-standard vendor library made in 2010?

As much as I might complain about how hard the switch was, as a javascript developer I should know by now that my codebase is going to be obsolete every two years, anyways :)

In the end, I think the biggest hurdles I saw as a TS convert were:

1. The module system is funky and fragile (I get the feeling the TS community is already aware of this)
2. Documentation is lacking in some areas, especially the module system, and compiler errors.

Maybe if I have a some time yet over the holidays, I can put together a little walkthrough for converting from ReqJS -> AMD.

[NoelAbrahams]

do we really want to design a language spec around the idiosyncrasies of some non-standard vendor library made in 2010

No 😃

I often worry about statements coming out of TypeScript along the lines of "we need to support real-world libraries that follow such and such a pattern" - when most of them are going to be obsolete when ES6 and typed JavaScript takes hold.

@eschwartz, you may find some useful information on modules in these old codeplex threads:

• Is module confusion holding back TypeScript?
• We're searching a good workflow for our typescript project. Which approaches do you use.

[mconner]

Coming from a Java background, this doesn't look much different than the java compiler dealing with classes and packages. In Java, you define classes in a packages (e.g.: in com.foo.bar.SomeClass) and the compiler finds classes referenced by Java code relative to the classpath: javac -classpath c:\someproject\classes c:\anotherproject\MyClass1.java ....
So if MyClass1 imports com.foo.bar.SomeClass, the compiler will look for that class in file c:\someproject\classes\com\foo\bar\Person.class and anywhere else you put on the classpath.

Similarily, a tsc compiler option could be added: --referencePath, and the syntax for a reference could be updated to recognize a path relative to this referencePath using some special character that only has meaning if you use the -referencePath option, maybe '@', or something:
/// <reference path="@/common/util/Foo.ts" />

So, with: tsc -referencePath C:\someproject\main\ts C:\someproject\test\ts\SomeTypeScript.ts
the typescript compiler would look for Foo.ts in C:\someproject\main\ts\common\util\Foo.ts

No special configuration files are needed for this. You would just have to tell the compiler the root(s) of your file structure to search for references, and indicate that the path is relative in your tag in your .ts files.

[TomBecker-BD]

It is an anti-pattern to put relative paths in source code. When you move or copy code to a different level in the directory hierarchy, it breaks. When you move a library it breaks all source files that reference it. Also, the library path has to be specified somewhere, in order to load the code, so duplicating the paths in the sources violates the DRY principle.

I agree with @mconner, this problem has already been solved in C and other compilers where you can specify include paths. However, it generally doesn't require the sources to specify a variable (such as "{projectA}" or character (such as "@"). The source file specifies a partial path, or just the file name, and the compiler finds it.

As a workaround I am using a "_references.ts" file and avoiding putting reference paths in other sources.

Proposal

I would like to propose two properties to the "tsconfig.json" file.

    "libraries": [
        "/libs/projectA/src",
        "/libs/projectB/src"
    ],
    "references": [
        "/libs/angularjs/angular.d.ts"
    ]

The "libraries" property is similar to @andrewvarga's proposal, but it only defines search paths, not variables for substitution. A source file can reference "/libs/projectA/src/Foo.ts" using:

    /// <reference path="Foo.ts" />

It should be an error if the reference path does not resolve to a single file.

The "references" property works the same as "_references.ts". Having the property in "tsconfig.json" puts all the configuration information together in one place so it is easier to maintain.

What the Compiler Has to Do

1. Automatically include the reference paths specified in the "references" property, the same as if they had been specified in "_references.ts".
2. When a reference path is encountered in the source, search relative to the source file location (for backward compatibility), and also relative to the paths in the "libraries" property.

There is a performance impact because the reference path search needs to be exhaustive, in order to catch ambiguous reference errors. However, the exhaustive search only needs to be done if the developer opts in by specifying a "libraries" property. Also, there are many ways the searching could be optimized, and the results can be cached.

There is no impact on generated code, compatibility, or any other features.

[mconner]

The reason I suggested a special character (e.g.: @) is that without it, the path is ambiguous. Is it from the root (i.e.: relative to one of the specified paths) , or is it relative to the current file. In Java, it is always from the root (or roots, as defined by the classpath), and so no special character is necessary. But to support both existing relative paths and root-level paths, it would be better to be explicit. This also reduces the amount of searching that would be necessary, and eliminates an accidental hit (however unlikely) where the author meant one, but got another.

[basarat]

FWIW I made a proof of concept of the fact that tooling can make it easier : https://github.com/TypeStrong/atom-typescript#relative-paths it can do even more over time.

[RyanCavanaugh]

We consider this thing to be outside the scope of tsc, which we generally think of a compiler that does what it needs to and nothing extra. External build systems that understand TypeScript can provide this functionality themselves and provide better flexibility and configurability.

If those external tools settle on a good common pattern that everyone likes, or don't come up with anything at all, we can revisit this post-2.0.

[eschwartz]

We consider this thing to be outside the scope of tsc, which we generally think of a compiler that does what it needs to and nothing extra.

@RyanCavanaugh - I would really like this to be the case, but the problem is that tsc has already taken on the task of converting ES6 style module imports to AMD style imports. Now that that decision is made, we're kind of stuck with supporting the quirks of AMD module loading, one of which is the lack of support for relative import paths.

I think the best solution would be for TypeScript to split out the *TypeScript to Javascript compiler" from the the "ES6 module to CommonJS/AMD" compiler. Then we could certainly close this issue as out-of-scope. But if that's not he direction the TS team wants to go, I would like to see better support for AMD.

[mhegazy]

@eschwartz We are looking into solving some of these issues in #2338, we expect this to be in TypeScript 1.6; can you take a look at the suggestion and let us know if that solves the issues you are running into with AMD?

[SonofNun15]

Banging my head against this issue currently. My scenario:

Code lives in a 'source' folder that is a sibling of the 'typings' folder (from tsd). So all of the reference tags use relative paths to step out of 'source' and into 'typings'. This code is then being deployed via bower, which, by default, dumps the files into a 'bower_components' folder. So now the type files (*.d.ts) are within 'bower_components\source' and need one more relative step up to get to the 'typings' folder at the root level.

I'll take a look at the suggested transform suggestions above.

[SonofNun15]

Turns out if I leave out the /// <reference> tags altogether the code still compiles as long as the ambient declaration files are given to the compiler or added to tsd.d.ts

[basarat]

That is by design. See http://blog.icanmakethiswork.io/2015/02/hey-tsconfigjson-where-have-you-been.html?m=1 for more about not needing reference tags

[jmp909]

it'd be good to have a ~ like .NET .. then at least we could move around files in the directories themselves whilst not having to worry about breaking relative references

eg
/// <reference path="~/typings/jQuery.d.ts" />