TypeScript missing critical class definition exports

[kaby76]

I was trying to write a template for TypeScript for testing the new target on grammars-v4. There is (are) important classes that need to be exported. Otherwise, I am blocked, and there will be no testing of the target in grammars-v4 because the trgen templates require a consistent runtime API in order to provide consistent functionality across targets.

• Recognizer missing in export list.

  antlr4/runtime/JavaScript/src/antlr4/index.d.ts

  Line 10 in c2bfa34

  export * from './Token';

• List not sorted alphabetically for quick lookup for missing exports.

  antlr4/runtime/JavaScript/src/antlr4/index.d.ts

  Line 1 in c2bfa34

  export * from "./InputStream";

• TokenSource missing export.
• CommonToken missing TypeScript wrapper declaration and export.

I don't know why certain classes do not have corresponding TypeScript export class definition files. It should be identical to the list of exported symbols in Java.

[ericvergnaud]

As a principle, I only export what is strictly required, excluding internal classes.
That can be hard to define until you bump into issues.
Can you provide an example of failing generated trgen code ?

[kaby76]

As a principle, I only export what is strictly required, excluding internal classes. That can be hard to define until you bump into issues. Can you provide an example of failing generated trgen code ?

I'm still going through with the creation of the template for TypeScript. So far, it's just Recognizer, which is easy to fix. But, I'll let you know what else may be missing, probably not much because the JavaScript target was working fine.

[kaby76]

Also missing is the CharStreams class. I use it in JavaScript (here, here, here), but there's no TypeScript decl for the class.

[ericvergnaud]

CharStreams : There is and it is exported. Please check beta 5 ?

[kaby76]

CharStreams : There is and it is exported. Please check beta 5 ?

Thanks so much, again!!

[kaby76]

• Version 4.12.0-beta.5 (and dev branch) doesn't have Recognizer exported.
• CharStreams is not available even in 4.12.0-beta.5. In the JavaScript code, I could import and reference the static methods of CharStreams.js. I can't import CharStreams in typescript (i.e., import { CharStreams } from 'antlr4'; => compilation error), nor can I import the functions of CharStreams.js on the import statement, as I can do for CharStream and CommonTokenStream (i.e., import { fromPath } from 'antlr4'; => compilation error). CharStreams is not a class. How can I import the functions in CharStreams.d.ts in a TypeScript program? TypeScript has classes and static methods. I don't understand why these functions are wrapped in a class called "CharStreams" as in all the other targets.

[ericvergnaud]

Let me check that

[ericvergnaud]

In TypeScript, CharStreams is a class, and the methods are static methods.
It's a factory for actual streams, so you should write:

import { CharStreams } from 'antlr4';
const stream = CharStreams.fromPath(path);

I have added Recognizer, and fixed another issue in beta 6.

[ericvergnaud]

If you're loading content from files you might want to upgrade to beta 8, I just added the encoding parameter to the FileStream API

[kaby76]

Excellent. Thank you.

Two other things that would be nice to have are: Lexer.reset() and ParserATNSimulator.trace_atn_sim. The later would be really nice since it allows for the parse to be compared across targets, which @parrt added recently. ParserATNSimulator.debug is available, but isn't useful because it outputs too much, so output can't be compared across targets as easily.

Otherwise, this works great. Already, the TypeScript port has shown a few problems with the JavaScript driver code I wrote. This is a great port. It's JavaScript finally done right.

[ericvergnaud]

Done, check beta 9

[ericvergnaud]

and thanks for the compliment

[kaby76]

and thanks for the compliment, I struggle with some contributors wanting me to reproduce the mistakes accumulated in the js API over 10 years...

You bet! This is a wonderful port. And we already have our first "customer" in grammars-v4.

[kaby76]

Looks like CommonToken is used in a number of grammar base classes, including the python/python3 grammar. That's somewhat easy even for me to fix.

import { Token } from "./Token";

export declare class CommonToken extends Token {
    constructor(source: number, type: number, channel: number, start: number, stop: number);
    clone(): CommonToken;
    cloneWithType(type: number): CommonToken;
    toString(): string;
}

The python/python3 grammar base class tries to set _token, but it can't because emitToken() is not available in the TypeScript code. We're also going to need skip(), more(), mode(d) declared in Lexer.d.ts.

[ericvergnaud]

Can you submit a PR ? That would avoid me adding stuff in the wrong place...

[ericvergnaud]

@kaby76 I think we can close this ?

[RobEin]

Excuse me.
How can I get the Token.DEFAULT_CHANNEL constant with the TypeScript target?

[ericvergnaud]

Aargh... it's missing. Can you propose a PR to fix that ?
Until it gets released, you can safely assume its value is 0.

[RobEin]

I would need additional features that are missing from the TypeScript runtime:

• Lexer class:
  _modeStack property (I implemented here)
  mode method (I implemented here)

• Token class:
  source property (I implemented here)

• Recognizer class:
  getSymbolicNames method (I implemented here)
  getErrorListenerDispatch method (I implemented here)

[ericvergnaud]

Hi. Sure. Can you submit a PR ?