Top-level await support in REPL #245

aseemk · 2016-12-02T22:59:26Z

Hey there,

Great work on ts-node! I just looked through the code a bit, and you all have clearly put a lot of thought and time into it. Thank you.

One request: I love using the REPL for quick exploration/experimentation/learning. And that often involves async steps, e.g. DB queries or API requests.

Before TypeScript, we used to use Streamline.js, and I loved that its REPL supported making async calls and awaiting their responses:

$ _node
_node> fs.stat('/dev/null', _)
{ dev: 1011986504, ... }
_node> r.get('https://httpbin.org/ip', _).body
'{\n  "origin": "1.2.3.4"\n}\n'
_node> ...

I understand that TypeScript and ES6/7 as languages don't allow top-level await in modules, but would you be open to supporting that in the REPL?

The way the Streamline REPL does it is simply by wrapping the code in an async IIFE, and only calling the REPL eval callback once the async IIFE finishes:

https://github.com/Sage/streamlinejs/blob/v2.0.13/lib/repl.js#L23-L42

Thank you in advance for the consideration! Cheers.

The text was updated successfully, but these errors were encountered:

blakeembrey · 2016-12-02T23:02:19Z

Off then top of my head, I don't think it's possible to support without more hacking of the TypeScript compiler and/or into the future with ES6 modules. You need an async function for await to compile, but you can't import from inside a function (it must be at the top-level). Happy to keep the issue open though.

aseemk · 2016-12-02T23:04:39Z

Thanks for the quick reply!

Could ts-node wrap in async IIFE only if the input had no imports? (And anything else problematic?)

It could still be really valuable to support top-level await for simple cases, even if it wasn't supported for more complex ones.

simonbuchan · 2017-01-11T22:15:23Z

Maybe just .then() before printing?

> fetch('https://google.com').then(r => r.text())
... awaiting promise id: 2 (Ctrl-C to cancel)
'<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">\n<TITLE>302 Moved</TITLE></HEAD><BODY>\n<H1>302 Moved</H1>\nThe document has moved\n<A HREF="http://www.google.co.nz/?gfe_rd=cr&amp;ei=uK12WNbvMMTN8gfyir2ADw">here</A>.\n</BODY></HTML>'

Ideally do it more interactively, eg. show the prompt below the currently awaiting promises, print them as they arrive or something, but it's a start.

unional · 2017-03-22T18:17:54Z

related topic:
https://gist.github.com/Rich-Harris/0b6f317657f5167663b493c722647221

jessedvrs · 2018-03-28T12:34:17Z

Temporary workaround which blocks the REPL while resolving:

npm install deasync --save-dev
# or
yarn add deasync --dev

const awaiter = promise => { const nothing = Symbol(); let ret = nothing; promise.then(response => { ret = response }); while(ret === nothing) { require('deasync').runLoopOnce(); } return ret; }

const value = awaiter(somePromise)

webmaster128 · 2018-07-20T06:58:22Z

There is top level await in the node 10+ repl and it could theoretically be utilized running node --experimental-repl-await --eval 'require("./node_modules/ts-node/dist/bin.js")'. However, this is stopped by the TypeScript compiler complaining

> const a: string = await Promise.resolve("a")
[eval].ts(1,19): error TS1308: 'await' expression is only allowed within an async function.

which is totally right, since top level await is not (yet?) valid JavaScript but a hack for repl convenience.

I don't see a way to get the await keyword untouched through the TypeScript compiler.

webmaster128 · 2018-07-20T13:57:01Z

Updated version of @jessedvrs's workaround that works in a type safe fashion:

npm install deasync2 --save-dev
# or
yarn add deasync2 --dev

function wait<T>(promise: Promise<T>): T { return require('deasync2').await(promise); }

const value = wait(somePromise);

blakeembrey · 2018-07-21T16:46:55Z

@webmaster128 If I understand it, that flag would not do anything for ts-node since it would only apply to the default REPL instance. We'd have to go ahead and implement something like nodejs/node@eeab7bc ourselves here. It should be a bit easier to do for us, but we'd need to make sure we're handling all the same things.

webmaster128 · 2018-07-21T16:59:17Z

@blakeembrey if you run node --experimental-repl-await --eval 'require("./node_modules/ts-node/dist/bin.js")', then you set process.binding('config').experimentalREPLAwait = true via the node command line (see nodejs/node#19604). This runs our bin.ts, which then uses lib/repl.js I guess via import { start, Recoverable } from 'repl'. Thus I expect it to work automatically. I think the only one that stops this from working is the TypeScript compiler, that says top level await is invalid.

blakeembrey · 2018-07-21T18:13:22Z

@webmaster128 I think there's a misunderstanding. All the code to handle top-level await is within defaultEval, which we don't use because we aren't evaling as JavaScript directly. Therefore we need to replicate all the logic to handle this into ts-node - the flag won't do anything since it only applies to the node.js REPL and you aren't opening the node.js REPL. The TypeScript compiler fix is the trivial part, we just ignore that diagnostic in the REPL.

webmaster128 · 2018-07-21T19:48:24Z

I think there's a misunderstanding. All the code to handle top-level await is within defaultEval, which we don't use because we aren't evaling as JavaScript directly.

You are probably right. I was a bit too optimistic here and did not follow the core carefully enough.

Would it be possible to implement the await keyword in the ts-node REPL using the wait<T>(promise: Promise<T>): T function from above?

webmaster128 · 2018-08-09T19:14:14Z

The TypeScript compiler fix is the trivial part, we just ignore that diagnostic in the REPL.

@blakeembrey could you elaborate on that part? I think I have a solution for running top level await in JavaScript utilizing the JavaScript AST (very similar to https://github.com/ef4/async-repl/blob/master/stubber.js and working in the ts-node REPL setup). However, I don't know how to teach typescript to pass the await untouched from TS code to JavaScript.

blakeembrey · 2018-08-10T16:32:54Z

There’s a feature in the README that allows you to ignore TypeScript diagnostic codes, it’d just be reusing that from the REPL.

webmaster128 · 2018-08-14T21:19:05Z

Thanks @blakeembrey! To make top level await available available in JavaScript, you need to do two things:

Ignore diagnostics

    myTypeScriptService = register({
      project: tsconfigPath,
      ignoreDiagnostics: [
        "1308", // TS1308: 'await' expression is only allowed within an async function.
      ],
    });

Set target to "es2017" or higher. Otherwise an awaiter is generated by TypeScript and await is rewritten to yield

This works for simple cases like for the cases await 1 and await 1 + await 2.

In other scenarios, TypeScript treats await as a variable name, e.g. await (1):

TSError: ⨯ Unable to compile TypeScript:
[eval].ts(1,1): error TS2552: Cannot find name 'await'. Did you mean 'waits'?

blakeembrey · 2018-08-15T03:10:59Z

I don’t think we can rely on the target though, can we instead parse and wrap the file before giving to to TypeScript instead? That way, even if it generates the awaiter, the code output will be functional?

Pokute · 2018-08-23T10:44:02Z

Top-level await seems to work at least to some extent with esm package after disabling both 1308 and 2304 diagnostics in source files. TS_NODE_IGNORE_DIAGNOSTICS=1308,2304 TS_NODE_COMPILER_OPTIONS='{"target":"es6"}' node -r esm -r ts-node/register topAwait.ts

Ignoring TS1308 is pretty simple and side-effect free since it seems to be related only to top-level await. Unfortunately TS2304 is required mainly due to error TS2304: Cannot find name 'await' and this suppresses all other 'cannot find name' errors.

It doesn't seem to work in command-line REPL though if started without a source file defined.

webmaster128 · 2018-08-23T11:10:34Z

I don’t think we can rely on the target though, can we instead parse and wrap the file before giving to to TypeScript instead?

That would require this AST manipulation in TypeScript and not in JavaScript. I think the mayor challenge is how to expose local variables in TypeScript when rewriting:

let a = await 1;

a // 1

as

(async () {
  let a = await 1;
})();

a // not available

what happens in the JavaScript version is that local declarations are converted to global properties (which only works in non-strict mode), e.g.

let a = await 1;

a // 1

becomes

(async () {
  a = await 1;
})();

a // 1

blakeembrey · 2018-08-26T23:49:46Z

@webmaster128 That's a very good point, I didn't consider how the previous JS approach got around this issue either. Why couldn't you do AST manipulation with TypeScript though? I'm not sure myself how mature that API is, but transforms do exist as a feature.

Another question, but how would const variables be possible here? I guess we can rely on TypeScript type checking to workaround. It looks like in Chrome (and maybe node.js, haven't tested yet), it's rewritten const so it's no longer a const when using await.

webmaster128 · 2018-08-27T06:28:30Z

Why couldn't you do AST manipulation with TypeScript though?

Didn't try it.

It looks like in Chrome (and maybe node.js, haven't tested yet), it's rewritten const so it's no longer a const when using await.

Oh, interesting. I thought they had a more sophisticated approach allowing const. The method I use does not support const either on the JS side. But TypeScript takes care of that. But I don't think that converting top level let/const to var semantics in a REPL is a big deal.

Do you have an idea for valid TypeScript that exposes variables declared in an async function body to the global scope?

feliperyan · 2020-05-28T23:35:29Z

Hello everyone, just stumbled on this one. Wondering if there are simple instructions on how to start a TS repl with ts-node and simply use async 100 without being hit with:

[eval].ts:1:1 - error TS1378: Top-level 'await' expressions are only allowed when the 'module' option is set to 'esnext' or 'system', and the 'target' option is set to 'es2017' or higher.

ilyakatz · 2020-07-01T04:37:49Z

Would love to know if there has been any updates to this one. Is it possible to do?

cspotcode · 2020-07-01T14:14:04Z

@feliperyan have you tried setting options they way the error message tells you to?

guifel · 2020-10-05T11:25:39Z

Solutions mentioned here (ignore TS error + --experimental-repl-await) work with scripts but not in REPL mode.

cspotcode · 2020-10-05T14:10:15Z

Does anyone know how node's REPL handles top-level await? Can we use their top-level await mechanism? Is it exposed as an API? Can we copy their source code?

Someone will need to propose how we can implement this.

dansebcar · 2020-10-06T18:12:36Z

In case it helps anyone else, I've had good results in REPL mode with node --require ts-node/register/transpile-only --experimental-repl-await (using node 14)

vamsiampolu · 2021-02-25T02:10:39Z

I was playing around with the repl today and managed to get it working. Looking at the issue about native ESM support with ts-node helped

Environment

Node.js Version: lts/fermium (v14.16.0)
Typescript Version: 4.2.2

Configuration

package.json (just the important bit to allow treating .js as ES6 modules):

{
  "type": "module",
  "main": "./src/index.js"
}

Typescript config (just the important bit):

{
  "compilerOptions": {
     "allowSyntheticDefaultImports": true,
      "target": "es2017",
      "module" "ESNext"
  },
  "include": ["src/**/*"]
}

Usage

If you are loading a typescript module that has a top level await:

node --experimental-repl-await --loader ts-node/esm ./src/clients.ts

node --experimental-repl-await --loader ts-node/esm

To import a module within the repl, use dynamic imports.

> const clients = await import('./src/clients.ts')

ejose19 · 2021-07-03T14:59:38Z

In case it helps anyone else, I've had good results in REPL mode with node --require ts-node/register/transpile-only --experimental-repl-await (using node 14)

It's worth noting this only helps for importing typescript files from the REPL, but you can't use any TS syntax in node REPL, ie:

const foo: string = 'bar';

const foo: string = 'bar';
      ^^^

Uncaught SyntaxError: Missing initializer in const declaration

ejose19 · 2021-07-03T15:06:29Z

Does anyone know how node's REPL handles top-level await? Can we use their top-level await mechanism? Is it exposed as an API? Can we copy their source code?

Someone will need to propose how we can implement this.

I've inspected the source and it seems it's handled here:

https://github.com/nodejs/node/blob/975bbbc443c47df777d460fadaada3c6d265b321/lib/repl.js#L424-L440

https://github.com/nodejs/node/blob/975bbbc443c47df777d460fadaada3c6d265b321/lib/internal/repl/await.js#L91-L176

blakeembrey added you can do this Good candidate for a pull request. enhancement labels Dec 9, 2016

filipesilva mentioned this issue Mar 14, 2017

Typeroots not inferred with TS2.1+ on Windows #283

Closed

blakeembrey mentioned this issue Oct 5, 2017

Add top level await in repl #440

Closed

webmaster128 mentioned this issue Jul 20, 2018

[iov-cli] Add wait() function to repl iov-one/iov-core#181

Merged

kitsonk mentioned this issue Jul 26, 2018

Top Level Await microsoft/TypeScript#25988

Closed

4 tasks

jdalton mentioned this issue Aug 22, 2018

Top-level await doesn't work with ts-node typescript files. standard-things/esm#580

Closed

sindresorhus mentioned this issue Sep 3, 2018

Allow top-level await tsdjs/tsd#1

Closed

ejose19 mentioned this issue Jul 3, 2021

feat: add REPL top level await support #1383

Merged

cspotcode added this to the next milestone Jul 8, 2021

cspotcode closed this as completed in #1383 Aug 8, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Top-level await support in REPL #245

Top-level await support in REPL #245

aseemk commented Dec 2, 2016

blakeembrey commented Dec 2, 2016 •

edited

Loading

aseemk commented Dec 2, 2016

simonbuchan commented Jan 11, 2017

unional commented Mar 22, 2017

jessedvrs commented Mar 28, 2018 •

edited

Loading

webmaster128 commented Jul 20, 2018

webmaster128 commented Jul 20, 2018

blakeembrey commented Jul 21, 2018

webmaster128 commented Jul 21, 2018 •

edited

Loading

blakeembrey commented Jul 21, 2018

webmaster128 commented Jul 21, 2018

webmaster128 commented Aug 9, 2018 •

edited

Loading

blakeembrey commented Aug 10, 2018

webmaster128 commented Aug 14, 2018

blakeembrey commented Aug 15, 2018

Pokute commented Aug 23, 2018 •

edited

Loading

webmaster128 commented Aug 23, 2018

blakeembrey commented Aug 26, 2018

webmaster128 commented Aug 27, 2018

feliperyan commented May 28, 2020

ilyakatz commented Jul 1, 2020

cspotcode commented Jul 1, 2020

guifel commented Oct 5, 2020 •

edited

Loading

cspotcode commented Oct 5, 2020

dansebcar commented Oct 6, 2020

vamsiampolu commented Feb 25, 2021 •

edited

Loading

ejose19 commented Jul 3, 2021

ejose19 commented Jul 3, 2021

Top-level await support in REPL #245

Top-level await support in REPL #245

Comments

aseemk commented Dec 2, 2016

blakeembrey commented Dec 2, 2016 • edited Loading

aseemk commented Dec 2, 2016

simonbuchan commented Jan 11, 2017

unional commented Mar 22, 2017

jessedvrs commented Mar 28, 2018 • edited Loading

webmaster128 commented Jul 20, 2018

webmaster128 commented Jul 20, 2018

blakeembrey commented Jul 21, 2018

webmaster128 commented Jul 21, 2018 • edited Loading

blakeembrey commented Jul 21, 2018

webmaster128 commented Jul 21, 2018

webmaster128 commented Aug 9, 2018 • edited Loading

blakeembrey commented Aug 10, 2018

webmaster128 commented Aug 14, 2018

blakeembrey commented Aug 15, 2018

Pokute commented Aug 23, 2018 • edited Loading

webmaster128 commented Aug 23, 2018

blakeembrey commented Aug 26, 2018

webmaster128 commented Aug 27, 2018

feliperyan commented May 28, 2020

ilyakatz commented Jul 1, 2020

cspotcode commented Jul 1, 2020

guifel commented Oct 5, 2020 • edited Loading

cspotcode commented Oct 5, 2020

dansebcar commented Oct 6, 2020

vamsiampolu commented Feb 25, 2021 • edited Loading

ejose19 commented Jul 3, 2021

ejose19 commented Jul 3, 2021

blakeembrey commented Dec 2, 2016 •

edited

Loading

jessedvrs commented Mar 28, 2018 •

edited

Loading

webmaster128 commented Jul 21, 2018 •

edited

Loading

webmaster128 commented Aug 9, 2018 •

edited

Loading

Pokute commented Aug 23, 2018 •

edited

Loading

guifel commented Oct 5, 2020 •

edited

Loading

vamsiampolu commented Feb 25, 2021 •

edited

Loading