Compilation time 4x slower after upgrading Typescript from 2.5.3 to 2.6.1

[pamtbaau]

After upgrading Typescript from 2.5.3 to 2.61 compilation time went up from 1min to 4min. Downgrading TS and upgrading again confirmed the difference in compilation time.

As a side note: ts-loader does not seem to be impacted by Typescript upgrade.

os: Windows 10
webpack: 3.8.1
awesome-typescript-loader: 3.2.3

[cmargulhano]

You could use TS-loader instead awesome-typescript-loader

[pamtbaau]

@cmargulhano After reading about thread-loader I do not see how thread-loader can be used instead of awesome-typescript-loader. Did you perhaps mean 'add in front of'?

Something is going wrong somewhere that leads to a 4-times increase of compilation time. My first impression is that adding worker pools is not the right solution for fixing the cause of the issue. That feels like adding a tow-truck to improve my cars speed when it runs out of gas...

As a side note: According to issue #497, thread-loader only shaves approx 20% extra of compilation time (in that use case) after switching from awesome-typescript-loader to ts-loader.

[cmargulhano]

Sorry, I meant TS-loader as mentioned in issue #497

[pamtbaau]

As mentioned in the issue, I already compared awesome-typescript-loader with ts-loader which seems not to be impacted by the upgrade of typescript...

[cmargulhano]

In my case, this solves the performance issues. I also update the webpack versions. My build time decreased from 4 minutes to 40 seconds.

[denisnikolayev]

We have the same issue and the problem is really funny. it's intersections of three bugs :)
We are using rxjs and noticed that they have three empty d.ts files (e.g. "node_modules\rxjs\add\observable\throw.d.ts"). So awesome loader notices that the files are empty and return undefined.

function ensureFile(fileName: string) {
        if (!files[fileName]) {
            const text = compiler.sys.readFile(fileName);
            if (text) {
                files[fileName] = {
                    text,
                    version: 0,
                    snapshot: compiler.ScriptSnapshot.fromString(text)
                };
            }
        }
    }

So typescript doesn't get file text and thing that the file are missing. After building it trys to find all missing files (file added to cache but without parsing)

missingFilePaths = arrayFrom(filesByName.keys(), p => <Path>p).filter(p => !filesByName.get(p));

https://github.com/Microsoft/TypeScript/blob/592ee00906557ed313f7692fe6662251e65f915d/src/compiler/program.ts#L607

therefore it can not reuse cache because (it finds this files on file system)

            // and hence cant reuse the structure.
            // This is same as how we dont reuse the structure if one of the file from old program is now missing
            if (oldProgram.getMissingFilePaths().some(missingFilePath => host.fileExists(missingFilePath))) {
                return oldProgram.structureIsReused = StructureIsReused.Not;
            }

https://github.com/Microsoft/TypeScript/blob/592ee00906557ed313f7692fe6662251e65f915d/src/compiler/program.ts#L885

So @s-panferov is it possible to remove check on empty file?

[s-panferov]

@denisnikolayev awesome! Thanks for the investigation. I'll make a fix.

"they have three empty d.ts files" 🤦‍♂️

[pamtbaau]

Uhhh... so what has upgrading from Typescript 2.5.3 to 2.6.1 got to do with this? Does Typescript handle empty d.ts files differently as of Typescript 2.6.1?

[denisnikolayev]

@pamtbaau have no idea. maybe they improved validation of project structure. if file didn't exist before that means structure was changed and we have to rebuild everything

@s-panferov np.
btw I noticed that the hottest place in compilation process is resolving imports (finding node_module and files). For some imports which are in deep folder structure we have 90 checking of existing files. And it checks for each import in each file (files * imports * deep folder structure).
But as I can see typescript support some cache (fifth arg in compiler.resolveModuleName https://github.com/Microsoft/TypeScript/blob/d05443bb1d3c346fbef4c2468ff06b00041b688d/src/compiler/moduleNameResolver.ts#L437), but you don't use it

resolveModuleNames(moduleNames: string[], containingFile: string) {
            const resolved =  moduleNames.map(module =>
                compiler.resolveModuleName(module, containingFile, compilerOptions, compiler.sys).resolvedModule);

it seems like it can increase speed a lot.

[s-panferov]

Please try awesome-typescript-loader@3.4.0-0 and say how it goes. I don't have any Angular/Rx specific applications, so it's hard to test.

[pamtbaau]

Compilation time from ~4 minutes down to ~50 seconds (app uses Angular/rxjs).
(ts-loader takes ~120 seconds)

I'm happy with awesome-typescript-loader again...