Skip to content

Cryptic "JavaScript heap out of memory" error when Transport Stream (.ts) video file is accidentally compiled #21453

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
shirakaba opened this issue Jan 29, 2018 · 3 comments
Labels
Duplicate An existing issue was already created

Comments

@shirakaba
Copy link

TypeScript version: 2.8.0-dev.20180127 (also applies to v2.4.2 and v2.6.2).
WebStorm version: 2017.3.3 (also applies to 2017.3.1).
VS Code version: 1.19.3
Search terms: "tsserver heap"

Background:

  • Accidentally adding MPEG-TS files (which also use the .ts extension) to the project tree causes the TypeScript service in the WebStorm and VS Code IDEs to hang and provide no useful error message.
  • Directly running tsc v2.8.0-dev.20180127 from the command-line works just fine on the .ts files, as does the webpack v3.10.0 build process (via ts-loader v3.2.0), because they only consume whatever files are explicitly imported, however, IDE TypeScript services hang because they analyse every file in the project tree.

MWE

Simpy add a Transport Stream (.ts) video file under your project tree (eg. Big Buck Bunny – 61.7Mb), ie. into a folder at the project root called vids/. Now open a TypeScript source file in the project in an IDE such as VS Code.

const x: number = 0;
x = "hello"; // Should throw error.

Config

Sample tsconfig.json (exact values unimportant; tsc --init likely suffices):

{
 "compilerOptions": {
 "watch": true,
 "sourceMap": true,
 "target": "es5",
 "baseUrl": ".",
 "outDir": "./built",
 "allowJs": true,
 "module": "commonjs",
 "removeComments": true,
 "declaration": false,
 "noImplicitAny": true,
 "noImplicitThis": true,
 "strictNullChecks": true,
 "noLib": false
 }
}

Expected behaviour:

Errors and warnings in TypeScript source should be indicated by syntax highlighting or red underlines in the latest WebStorm IDE and VS Code.

Actual behaviour:

After 1-2 minutes of hanging on startup, each IDE's TypeScript service throws a heap error, and so doesn't get as far as communicating the errors/warnings to the IDE for appropriate syntax highlighting.

WEBSTORM

Log:

Process: TypeScript service version: 2.8.0-dev.20180127
Process: Default service options: {"watch":true,"sourceMap":true,"target":1,"baseUrl":"/Users/me/Sites/proj","outDir":"/Users/me/Sites/proj/built","allowJs":true,"typeRoots":["/Users/me/Sites/proj/node_modules/@types","/Users/me/Documents/git/types"],"module":1,"removeComments":true,"declaration":false,"noImplicitAny":true,"noImplicitThis":true,"strictNullChecks":true,"noLib":false,"configFilePath":"/Users/me/Sites/proj/tsconfig.json"}
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [/usr/local/bin/node]
 2: node::FatalException(v8::Isolate*, v8::Local<v8::Value>, v8::Local<v8::Message>) [/usr/local/bin/node]
 3: v8::Utils::ReportOOMFailure(char const*, bool) [/usr/local/bin/node]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/usr/local/bin/node]
 5: v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [/usr/local/bin/node]
 6: v8::internal::Runtime_AllocateInTargetSpace(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/local/bin/node]
 7: 0x2f298220463d
 8: 0x2f298237d4ac

VS CODE

Symptoms:

  • VS Code gets stuck on [TypeScript Importer]: initializing for about 1-2 minutes.
  • Upon failing initialisation, I receive two errors: "Extension host terminated unexpectedly", and "Canceled".
  • My 'output', and 'debug console' tabs are all blank, while my 'problems' tab says only "No problems have been detected in the workspace so far."

Log:

console.ts:123 [Extension Host] processWorkspaceFiles Array(13)0: Object$mid: 1external: "file:///Users/me/Sites/proj/lib/css/common.css.d.ts"fsPath: "/Users/me/Sites/proj/lib/css/common.css.d.ts"path: "/Users/me/Sites/proj/lib/css/common.css.d.ts"scheme: "file"__proto__: Object1: Object$mid: 1external: "file:///Users/me/Sites/proj/lib/css/fonts.css.d.ts"fsPath: "/Users/me/Sites/proj/lib/css/fonts.css.d.ts"path: "/Users/me/Sites/proj/lib/css/fonts.css.d.ts"scheme: "file"__proto__: Object2: Object$mid: 1external: "file:///Users/me/Sites/proj/lib/css/Mosaic.css.d.ts"fsPath: "/Users/me/Sites/proj/lib/css/Mosaic.css.d.ts"path: "/Users/me/Sites/proj/lib/css/Mosaic.css.d.ts"scheme: "file"__proto__: Object3: Object$mid: 1external: "file:///Users/me/Sites/proj/lib/css/Prompt.css.d.ts"fsPath: "/Users/me/Sites/proj/lib/css/Prompt.css.d.ts"path: "/Users/me/Sites/proj/lib/css/Prompt.css.d.ts"scheme: "file"__proto__: Object4: Object$mid: 1external: "file:///Users/me/Sites/proj/r/vids/proj-dummy.ts"fsPath: "/Users/me/Sites/proj/r/vids/proj-dummy.ts"path: "/Users/me/Sites/proj/r/vids/proj-dummy.ts"scheme: "file"__proto__: Object5: Object$mid: 1external: "file:///Users/me/Sites/proj/src/startup/Entry.ts"fsPath: "/Users/me/Sites/proj/src/startup/Entry.ts"path: "/Users/me/Sites/proj/src/startup/Entry.ts"scheme: "file"__proto__: Object6: Object7: Object8: Object$mid: 1external: "file:///Users/me/Sites/proj/src/templates/Mosaic.ts"fsPath: "/Users/me/Sites/proj/src/templates/Mosaic.ts"path: "/Users/me/Sites/proj/src/templates/Mosaic.ts"scheme: "file"__proto__: Object9: Object10: Object$mid: 1external: "file:///Users/me/Sites/proj/src/templates/VideoSelect.ts"fsPath: "/Users/me/Sites/proj/src/templates/VideoSelect.ts"path: "/Users/me/Sites/proj/src/templates/VideoSelect.ts"scheme: "file"__proto__: Object11: Object$mid: 1external: "file:///Users/me/Sites/proj/src/ui/MosaicView.ts"fsPath: "/Users/me/Sites/proj/src/ui/MosaicView.ts"path: "/Users/me/Sites/proj/src/ui/MosaicView.ts"scheme: "file"__proto__: Object12: Object$mid: 1external: "file:///Users/me/Sites/proj/src/ui/Splashscreen.ts"fsPath: "/Users/me/Sites/proj/src/ui/Splashscreen.ts"path: "/Users/me/Sites/proj/src/ui/Splashscreen.ts"scheme: "file"__proto__: Objectlength: 13__proto__: Array(0) true false
index.js:38 [uncaught exception]: Error: write EPIPE
onError @ index.js:38
index.js:41 Error: write EPIPE
    at exports._errnoException (util.js:1050)
    at WriteWrap.afterWrite [as oncomplete] (net.js:813)
onError @ index.js:41
events.js:163 Uncaught Error: write EPIPE
    at exports._errnoException (util.js:1050)
    at WriteWrap.afterWrite [as oncomplete] (net.js:813)
extensionService.ts:215 Extension host terminated unexpectedly. Code:  null  Signal:  SIGABRT
t._onExtensionHostCrashed @ extensionService.ts:215
messageService.ts:126 Canceled
e.doShow @ messageService.ts:126
messageService.ts:126 Canceled
e.doShow @ messageService.ts:126
messageService.ts:126 Canceled
e.doShow @ messageService.ts:126
messageService.ts:126 Canceled
e.doShow @ messageService.ts:126
messageService.ts:126 Extension host terminated unexpectedly.
e.doShow @ messageService.ts:126
extensionHost.ts:200 [Extension Host] debugger inspector at %cFATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: %cnode::Abort() [/Applications/Visual Studio Code.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib]
 2: node::DLOpen(v8::FunctionCallbackInfo<v8::Value> const&) [/Applications/Visual Studio Code.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib]
 3: %cv8::internal::FatalProcessOutOfMemory(char const*) [/Applications/Visual Studio Code.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib]
 4: v8::internal::FatalProcessOutOfMemory(char const*) [/Applications/Visual Studio Code.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib]
 5: %cv8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [/Applications/Visual Studio Code.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib]
 6: %cv8::internal::RegisterConfiguration::AreAliases(v8::internal::MachineRepresentation, int, v8::internal::MachineRepresentation, int) const [/Applications/Visual Studio Code.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib]
 7: 0x1fec78285bbc
%c
<--- Last few GCs --->

[4387:0x7f94e4013600]   363933 ms: Mark-sweep 2054.8 (2118.3) -> 2054.8 (2114.8) MB, 1969.6 / 0.0 ms  (+ 0.0 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 1970 ms) last resort 
[4387:0x7f94e4013600]   365854 ms: Mark-sweep 2054.8 (2114.8) -> 2054.8 (2114.8) MB, 1919.4 / 0.0 ms  last resort 


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x34bdacd2b959 <JS Object>
    1: parseDecorators [/Users/me/.vscode/extensions/rbbit.typescript-hero-0.12.0/node_modules/typescript/lib/typescript.js:~18486] [pc=0x1fec78a9e285](this=0x2d7044613e61 <JS Global Object>)
    2: parseDeclaration [/Users/me/.vscode/extensions/rbbit.typescript-hero-0.12.0/node_modules/typescript/lib/typescript.js:~18167] [pc=0x1fec78a97cd0](this=0x2d7044613e61 <JS Global Object>)
    3:...

Related issues:

I have encountered this error two or three times over the last year (it was present in TypeScript v2.1.5 too), and every time it catches me out, killing a lot of work hours in tracking down the problem again.

#12804 (comment)

Workaround:

MPEG-TS files can alternatively be named .tsv or .tsa, which stops them being consumed as TypeScript files. However, the onus is rather on TypeScript to handle them properly, as MPEG-TS has been using the .ts file extension since at least its release in 1995, well before the TypeScript project.

Proposal:

  • Determine whether a .ts file is of MPEG-TS type by examining the first few bytes of it. The Wikipedia page indicates that an MPEG-TS file is merely a "sequence of packets, without any global header". Thus, a valid MPEG-TS file will start with a header packet (the format is again described on the Wikipedia page), beginning with a sync byte, invariably with the value 0x47 (The character 'G'). If there is a chance that this could clash with a valid TypeScript file, then further bytes should be read in.
  • Implement a default, yet overridable, file size limit for .ts files to prevent the TypeScript compiler spending too long hanging on them (the existing --disableSizeLimit is for JS only as far as I understand). This would handle the case of invalid MPEG-TS files (not really a big concern, though) or other unforeseen file extension clashes with .ts.
  • Provide a better error message.
@j-oliveras
Copy link
Contributor

Duplicated of #21136.

@DanielRosenwasser
Copy link
Member

In the meantime you can add the file to your tsconfig.json's top-level exclude list

{
    "compilerOptions": {
        // ...
    },
    "exclude": [
        "your-file-here.ts",
        "your-media-directory-here.ts"
    ]
}

@typescript-bot
Copy link
Collaborator

Automatically closing this issue for housekeeping purposes. The issue labels indicate that it is unactionable at the moment or has already been addressed.

@microsoft microsoft locked and limited conversation to collaborators Jul 3, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Duplicate An existing issue was already created
Projects
None yet
Development

No branches or pull requests

4 participants