Skip to content

Leave files from node_modules out when calculating getCommonSourceDirectory #11993

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
5 commits merged into from
Nov 11, 2016
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions src/compiler/program.ts
Original file line number Diff line number Diff line change
Expand Up @@ -431,13 +431,14 @@ namespace ts {
return program;

function getCommonSourceDirectory() {
if (typeof commonSourceDirectory === "undefined") {
if (options.rootDir && checkSourceFilesBelongToPath(files, options.rootDir)) {
if (commonSourceDirectory === undefined) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is also used by maproot and sourceroot.. So not doing this for bundle emit has effect on how the paths would be represented... Because --mapRoot --sourceRoot --outFile is a valid option

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you write a test case?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when the files exist in node_modules and src folder and --sourceroot is specified?

Copy link
Author

@ghost ghost Nov 3, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If --sourceRoot is specified, then the "sourceRoot" property in the source map is whatever you specified.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes but the paths are relative to common source directory and with this change the commonsourcedirectory would be different from what it used to be if say eg your sources are in ./src/ and ./node_modules/ which means the sources will be just relative to ./src/ instead or ./

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wrote a test case. @sheetalkamat could you look at the baselines and tell me if they're what you would expect?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think test case is ok, and realized that scenario I was talking about sourceroot and maproot works because --out is only allowed with amd or system and hence there is no way to get bundled emit that includes modules from node_modules folder.

const emittedFiles = filterSourceFilesInDirectory(files, isSourceFileFromExternalLibrary);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is ignoring .d.ts files in the own compilation as well ? (not from external library)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. For the --outDir case that makes sense since we are just emitting .ts files. Can you provide another case where we would want the .d.ts files included?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But right now we didnt ignore .d.ts files from our own program to compute the common source directory? So this is intended change?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added a .d.ts file to the test.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you need a new test case:

//@filename: app/src/index.ts

//@filename: app/lib/bar.d.ts

//@filename: app/tsconfig.json

To make sure common source directory is still app

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I added a test commonSourceDirectory_dts.ts. This appears to already be the behavior of the current compiler -- the referenced file does not become part of the output.

if (options.rootDir && checkSourceFilesBelongToPath(emittedFiles, options.rootDir)) {
// If a rootDir is specified and is valid use it as the commonSourceDirectory
commonSourceDirectory = getNormalizedAbsolutePath(options.rootDir, currentDirectory);
}
else {
commonSourceDirectory = computeCommonSourceDirectory(files);
commonSourceDirectory = computeCommonSourceDirectory(emittedFiles);
}
if (commonSourceDirectory && commonSourceDirectory[commonSourceDirectory.length - 1] !== directorySeparator) {
// Make sure directory path ends with directory separator so this string can directly
Expand Down
20 changes: 16 additions & 4 deletions src/compiler/utilities.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2576,20 +2576,33 @@ namespace ts {
}
else {
const sourceFiles = targetSourceFile === undefined ? getAllEmittableSourceFiles() : [targetSourceFile];
return filter(sourceFiles, isNonDeclarationFile);
return filterSourceFilesInDirectory(sourceFiles, file => host.isSourceFileFromExternalLibrary(file));
}

function getAllEmittableSourceFiles() {
return options.noEmitForJsFiles ? filter(host.getSourceFiles(), sourceFile => !isSourceFileJavaScript(sourceFile)) : host.getSourceFiles();
}
}

/** Don't call this for `--outFile`, just for `--outDir` or plain emit. */
export function filterSourceFilesInDirectory(sourceFiles: SourceFile[], isSourceFileFromExternalLibrary: (file: SourceFile) => boolean): SourceFile[] {
return filter(sourceFiles, file => shouldEmitInDirectory(file, isSourceFileFromExternalLibrary));
}

function isNonDeclarationFile(sourceFile: SourceFile) {
return !isDeclarationFile(sourceFile);
}

/**
* Whether a file should be emitted in a non-`--outFile` case.
* Don't emit if source file is a declaration file, or was located under node_modules
*/
function shouldEmitInDirectory(sourceFile: SourceFile, isSourceFileFromExternalLibrary: (file: SourceFile) => boolean): boolean {
return isNonDeclarationFile(sourceFile) && !isSourceFileFromExternalLibrary(sourceFile);
}

function isBundleEmitNonExternalModule(sourceFile: SourceFile) {
return !isDeclarationFile(sourceFile) && !isExternalModule(sourceFile);
return isNonDeclarationFile(sourceFile) && !isExternalModule(sourceFile);
}

/**
Expand Down Expand Up @@ -2677,8 +2690,7 @@ namespace ts {
else {
const sourceFiles = targetSourceFile === undefined ? getSourceFilesToEmit(host) : [targetSourceFile];
for (const sourceFile of sourceFiles) {
// Don't emit if source file is a declaration file, or was located under node_modules
if (!isDeclarationFile(sourceFile) && !host.isSourceFileFromExternalLibrary(sourceFile)) {
if (shouldEmitInDirectory(sourceFile, file => host.isSourceFileFromExternalLibrary(file))) {
onSingleFileEmit(host, sourceFile);
}
}
Expand Down
2 changes: 1 addition & 1 deletion src/harness/compilerRunner.ts
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,7 @@ class CompilerBaselineRunner extends RunnerBase {
});

it("Correct Sourcemap output for " + fileName, () => {
Harness.Compiler.doSourcemapBaseline(justName, options, result);
Harness.Compiler.doSourcemapBaseline(justName, options, result, harnessSettings);
});

it("Correct type/symbol baselines for " + fileName, () => {
Expand Down
48 changes: 18 additions & 30 deletions src/harness/harness.ts
Original file line number Diff line number Diff line change
Expand Up @@ -470,19 +470,6 @@ namespace Utils {
}
}

namespace Harness.Path {
export function getFileName(fullPath: string) {
return fullPath.replace(/^.*[\\\/]/, "");
}

export function filePath(fullPath: string) {
fullPath = ts.normalizeSlashes(fullPath);
const components = fullPath.split("/");
const path: string[] = components.slice(0, components.length - 1);
return path.join("/") + "/";
}
}

namespace Harness {
export interface IO {
newLine(): string;
Expand Down Expand Up @@ -1090,7 +1077,9 @@ namespace Harness {
{ name: "suppressOutputPathCheck", type: "boolean" },
{ name: "noImplicitReferences", type: "boolean" },
{ name: "currentDirectory", type: "string" },
{ name: "symlink", type: "string" }
{ name: "symlink", type: "string" },
// Emitted js baseline will print full paths for every output file
{ name: "fullEmitPaths", type: "boolean" }
];

let optionsIndex: ts.Map<ts.CommandLineOption>;
Expand Down Expand Up @@ -1565,7 +1554,7 @@ namespace Harness {
return file.writeByteOrderMark ? "\u00EF\u00BB\u00BF" : "";
}

export function doSourcemapBaseline(baselinePath: string, options: ts.CompilerOptions, result: CompilerResult) {
export function doSourcemapBaseline(baselinePath: string, options: ts.CompilerOptions, result: CompilerResult, harnessSettings: Harness.TestCaseParser.CompilerSettings) {
if (options.inlineSourceMap) {
if (result.sourceMaps.length > 0) {
throw new Error("No sourcemap files should be generated if inlineSourceMaps was set.");
Expand All @@ -1587,10 +1576,8 @@ namespace Harness {
}

let sourceMapCode = "";
for (let i = 0; i < result.sourceMaps.length; i++) {
sourceMapCode += "//// [" + Harness.Path.getFileName(result.sourceMaps[i].fileName) + "]\r\n";
sourceMapCode += getByteOrderMarkText(result.sourceMaps[i]);
sourceMapCode += result.sourceMaps[i].code;
for (const sourceMap of result.sourceMaps) {
sourceMapCode += fileOutput(sourceMap, harnessSettings);
}

return sourceMapCode;
Expand All @@ -1611,23 +1598,19 @@ namespace Harness {
tsCode += "//// [" + header + "] ////\r\n\r\n";
}
for (let i = 0; i < tsSources.length; i++) {
tsCode += "//// [" + Harness.Path.getFileName(tsSources[i].unitName) + "]\r\n";
tsCode += "//// [" + ts.getBaseFileName(tsSources[i].unitName) + "]\r\n";
tsCode += tsSources[i].content + (i < (tsSources.length - 1) ? "\r\n" : "");
}

let jsCode = "";
for (let i = 0; i < result.files.length; i++) {
jsCode += "//// [" + Harness.Path.getFileName(result.files[i].fileName) + "]\r\n";
jsCode += getByteOrderMarkText(result.files[i]);
jsCode += result.files[i].code;
for (const file of result.files) {
jsCode += fileOutput(file, harnessSettings);
}

if (result.declFilesCode.length > 0) {
jsCode += "\r\n\r\n";
for (let i = 0; i < result.declFilesCode.length; i++) {
jsCode += "//// [" + Harness.Path.getFileName(result.declFilesCode[i].fileName) + "]\r\n";
jsCode += getByteOrderMarkText(result.declFilesCode[i]);
jsCode += result.declFilesCode[i].code;
for (const declFile of result.declFilesCode) {
jsCode += fileOutput(declFile, harnessSettings);
}
}

Expand All @@ -1652,6 +1635,11 @@ namespace Harness {
});
}

function fileOutput(file: GeneratedFile, harnessSettings: Harness.TestCaseParser.CompilerSettings): string {
const fileName = harnessSettings["fullEmitPaths"] ? file.fileName : ts.getBaseFileName(file.fileName);
return "//// [" + fileName + "]\r\n" + getByteOrderMarkText(file) + file.code;
}

export function collateOutputs(outputFiles: Harness.Compiler.GeneratedFile[]): string {
// Collect, test, and sort the fileNames
outputFiles.sort((a, b) => ts.compareStrings(cleanName(a.fileName), cleanName(b.fileName)));
Expand Down Expand Up @@ -1848,7 +1836,7 @@ namespace Harness {
}

// normalize the fileName for the single file case
currentFileName = testUnitData.length > 0 || currentFileName ? currentFileName : Path.getFileName(fileName);
currentFileName = testUnitData.length > 0 || currentFileName ? currentFileName : ts.getBaseFileName(fileName);

// EOF, push whatever remains
const newTestFile2 = {
Expand Down Expand Up @@ -2012,7 +2000,7 @@ namespace Harness {

export function isDefaultLibraryFile(filePath: string): boolean {
// We need to make sure that the filePath is prefixed with "lib." not just containing "lib." and end with ".d.ts"
const fileName = Path.getFileName(filePath);
const fileName = ts.getBaseFileName(filePath);
return ts.startsWith(fileName, "lib.") && ts.endsWith(fileName, ".d.ts");
}

Expand Down
2 changes: 1 addition & 1 deletion src/harness/runnerbase.ts
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ abstract class RunnerBase {
/** Replaces instances of full paths with fileNames only */
static removeFullPaths(path: string) {
// If its a full path (starts with "C:" or "/") replace with just the filename
let fixedPath = /^(\w:|\/)/.test(path) ? Harness.Path.getFileName(path) : path;
let fixedPath = /^(\w:|\/)/.test(path) ? ts.getBaseFileName(path) : path;

// when running in the browser the 'full path' is the host name, shows up in error baselines
const localHost = /http:\/localhost:\d+/g;
Expand Down
29 changes: 29 additions & 0 deletions tests/baselines/reference/commonSourceDirectory.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
//// [tests/cases/compiler/commonSourceDirectory.ts] ////

//// [index.ts]
// Test that importing a file from `node_modules` does not affect calculation of the common source directory.

export const x = 0;

//// [bar.d.ts]
declare module "bar" {
export const y = 0;
}

//// [index.ts]
/// <reference path="../types/bar.d.ts"/>
import { x } from "foo";
import { y } from "bar";
x + y;


//// [/app/bin/index.js]
"use strict";
/// <reference path="../types/bar.d.ts"/>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is my point... without this change it would have been bin/app/index.js -- so is this really what we want.. and this would be breaking change.. @mhegazy to take a look and confirm this is indeed expected change...

var foo_1 = require("foo");
var bar_1 = require("bar");
foo_1.x + bar_1.y;
//# sourceMappingURL=/app/myMapRoot/index.js.map

//// [/app/bin/index.d.ts]
/// <reference path="../../types/bar.d.ts" />
2 changes: 2 additions & 0 deletions tests/baselines/reference/commonSourceDirectory.js.map

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

58 changes: 58 additions & 0 deletions tests/baselines/reference/commonSourceDirectory.sourcemap.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
===================================================================
JsFile: index.js
mapUrl: /app/myMapRoot/index.js.map
sourceRoot: /app/mySourceRoot/
sources: index.ts
===================================================================
-------------------------------------------------------------------
emittedFile:/app/bin/index.js
sourceFile:index.ts
-------------------------------------------------------------------
>>>"use strict";
>>>/// <reference path="../types/bar.d.ts"/>
1 >
2 >^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1 >
2 >/// <reference path="../types/bar.d.ts"/>
1 >Emitted(2, 1) Source(1, 1) + SourceIndex(0)
2 >Emitted(2, 42) Source(1, 42) + SourceIndex(0)
---
>>>var foo_1 = require("foo");
1 >
2 >^^^^^^^^^^^^^^^^^^^^^^^^^^^
3 > ^->
1 >
>
2 >import { x } from "foo";
1 >Emitted(3, 1) Source(2, 1) + SourceIndex(0)
2 >Emitted(3, 28) Source(2, 25) + SourceIndex(0)
---
>>>var bar_1 = require("bar");
1->
2 >^^^^^^^^^^^^^^^^^^^^^^^^^^^
1->
>
2 >import { y } from "bar";
1->Emitted(4, 1) Source(3, 1) + SourceIndex(0)
2 >Emitted(4, 28) Source(3, 25) + SourceIndex(0)
---
>>>foo_1.x + bar_1.y;
1 >
2 >^^^^^^^
3 > ^^^
4 > ^^^^^^^
5 > ^
6 > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^->
1 >
>
2 >x
3 > +
4 > y
5 > ;
1 >Emitted(5, 1) Source(4, 1) + SourceIndex(0)
2 >Emitted(5, 8) Source(4, 2) + SourceIndex(0)
3 >Emitted(5, 11) Source(4, 5) + SourceIndex(0)
4 >Emitted(5, 18) Source(4, 6) + SourceIndex(0)
5 >Emitted(5, 19) Source(4, 7) + SourceIndex(0)
---
>>>//# sourceMappingURL=/app/myMapRoot/index.js.map
24 changes: 24 additions & 0 deletions tests/baselines/reference/commonSourceDirectory.symbols
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
=== /app/index.ts ===
/// <reference path="../types/bar.d.ts"/>
import { x } from "foo";
>x : Symbol(x, Decl(index.ts, 1, 8))

import { y } from "bar";
>y : Symbol(y, Decl(index.ts, 2, 8))

x + y;
>x : Symbol(x, Decl(index.ts, 1, 8))
>y : Symbol(y, Decl(index.ts, 2, 8))

=== /node_modules/foo/index.ts ===
// Test that importing a file from `node_modules` does not affect calculation of the common source directory.

export const x = 0;
>x : Symbol(x, Decl(index.ts, 2, 12))

=== /types/bar.d.ts ===
declare module "bar" {
export const y = 0;
>y : Symbol(y, Decl(bar.d.ts, 1, 16))
}

27 changes: 27 additions & 0 deletions tests/baselines/reference/commonSourceDirectory.types
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
=== /app/index.ts ===
/// <reference path="../types/bar.d.ts"/>
import { x } from "foo";
>x : 0

import { y } from "bar";
>y : 0

x + y;
>x + y : number
>x : 0
>y : 0

=== /node_modules/foo/index.ts ===
// Test that importing a file from `node_modules` does not affect calculation of the common source directory.

export const x = 0;
>x : 0
>0 : 0

=== /types/bar.d.ts ===
declare module "bar" {
export const y = 0;
>y : 0
>0 : 0
}

21 changes: 21 additions & 0 deletions tests/baselines/reference/commonSourceDirectory_dts.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
//// [tests/cases/compiler/commonSourceDirectory_dts.ts] ////

//// [bar.d.ts]
// Test that importing a file from `node_modules` does not affect calculation of the common source directory.

declare const y: number;

//// [index.ts]
/// <reference path="../lib/bar.d.ts" />
export const x = y;


//// [/app/bin/index.js]
"use strict";
/// <reference path="../lib/bar.d.ts" />
exports.x = y;
//# sourceMappingURL=/app/myMapRoot/index.js.map

//// [/app/bin/index.d.ts]
/// <reference path="../lib/bar.d.ts" />
export declare const x: number;
2 changes: 2 additions & 0 deletions tests/baselines/reference/commonSourceDirectory_dts.js.map

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading