Adding import declarations using the Compiler API

[matthewmueller]

Right now it doesn't seem possible to add an ImportDeclaration programmatically via the Compiler API without updating the entire AST. My use-case is to be able to create https://github.com/zeit/styled-jsx with the TS Compiler over babel. When it detects a <style> JSXElement, it will append import JSXStyle from 'styled-jsx to the top of the code. I'd like to be able to do this too.

I've looked at how Typescript adds the tslib and it seems like it uses an internal imports key on the SourceFile node. Would it be possible to give us access to that?

TypeScript Version: Version 2.4.1

Code

Sorry this isn't entirely self-contained, but the original is about 100 LOC. This should give you the gist, and it's mostly from the tslib code in the compiler.

const node = (node as ts.Node) as ts.SourceFile
const externalHelpersModuleReference = ts.createLiteral('styled')
const importDecl = ts.createImportDeclaration(
  /*decorators*/ undefined,
  /*modifiers*/ undefined,
  /*importClause*/ undefined
)
externalHelpersModuleReference.parent = importDecl
importDecl.parent = node

Expected behavior:

Get the result text back with an import statement at the top of the source file

Actual behavior:

No changes

[weswigham]

Are you doing this in a before or an after transform?

[matthewmueller]

Ah good question. Currently traversing the AST in a before transform.

…

On Tue, Sep 12, 2017 at 1:17 AM Wesley Wigham ***@***.***> wrote: Are you doing this in a before or an after transform? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#18369 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAKZO4bWC5ANzqGYXT1vLEFbfPpdsuyQks5shXWpgaJpZM4PSU87> .

[weswigham]

I think the changes made in #18017 should have got you covered, then - are you still broken on typescript@next?

[matthewmueller]

Awesome! I'll give it a shot. At first glance though, it seems like this PR addresses updating an existing import, it doesn't test adding an import that wasn't previously there.

Is adding an import addressed in this PR? I think I basically need a way to attach a new import onto the statements block of the SourceFile node.

P.S. Thanks a lot for exposing the compiler API. This is the last hurdle to having only 1 compiler step (no babel!).

[weswigham]

@MatthewMuller You should just be able to use updateSourceFileNode in your transformation and replace the statements list with one with your extra import added.

[matthewmueller]

@weswigham, thanks for pointing that out. I wasn't sure if that would cause the whole file to be re-parsed or not.

I just tried implementing your suggestion but I got a bit stuck:

const src = (node as ts.Node) as ts.SourceFile
const imp = "var style = require('Style');\n"
const range = ts.createTextChangeRange(
  ts.createTextSpanFromBounds(0, imp.length),
  imp.length + src.text.length
)
const updated = ts.updateSourceFile(src, imp + src.text, range)
return (updated as ts.Node) as T

Results in a compiler error:

Error: Debug Failure. False expression.
at checkChangeRange (node_modules/typescript/lib/typescript.js:18346:26)
at Object.updateSourceFile (node_modules/typescript/lib/typescript.js:17980:13)
at Object.updateSourceFile (node_modules/typescript/lib/typescript.js:12348:47)

I'm sure I'm doing something wrong here with the range but it's not clear to me what that could be.

Any pointers would be greatly appreciated!

---

EDIT: Okay! I was able to get this working with:

const src = (node as ts.Node) as ts.SourceFile
const imp = "var style = require('styles');\n"
const updated = ts.createSourceFile(
  src.fileName,
  imp + src.text,
  src.languageVersion
)
return (updated as ts.Node) as T

I also found a way to get around the issue above. However, with both create and update the src.text, blows away all the prior changes I made to the AST, so it seems like I need a way to modify the AST, not overwrite it.

[weswigham]

@matthewmueller If you want to write it as a transformer (and not string manipulations), you should write it like so to add import style = require("styles"):

    const file = node as SourceFile;
    updateSourceFileNode(file, [createImportEqualsDeclaration(
        /*decorators*/ undefined,
        /*modifiers*/ undefined,
        "style",
        createExternalModuleReference(createLiteral("styles"))
    ), ...file.statements]);

or like so to add var style = require("styles"):

    const file = node as SourceFile;
    updateSourceFileNode(file, [createVariableStatement(
        /*modifiers*/ undefined,
        createVariableDeclarationList([createVariableDeclaration("style", /*type*/ undefined, createCall(createIdentifier("require"), [], [createLiteral("styles")]))])
    ), ...file.statements]);

Reparsing an entire file is incredibly wasteful (and will likely cause you to lose accurate sourcemaps and other things, like prior AST modifications 😉 ).

[matthewmueller]

Soooooo good! I had no idea about the [ ...file.statements ].

Interestingly, it doesn't seem possible to do:

const update: ts.NodeArray<ts.Statement> = [].concat(file.statements)

Thanks for all your help! I'll drop a link here once the repo is public in case anyone else wants another example of modifying typescript's AST.

[matthewmueller]

Ahh one more thing. How can we indicate that it should be using the ["default"]?

If the original file is:

import sandwich from "sandwich";
console.log(sandwich);
console.log(salami)

And I want to add import salami from 'salami' at build time. The compiled output will look like this:

"use strict";
exports.__esModule = true;
var salami = require("salami");
var sandwich_1 = require("sandwich");
console.log(sandwich_1["default"]);
console.log(salami);

You'll notice that sandwich gets handled properly, but the newly added import does not.

I've put together a self-contained example runnable via ts-node file.ts:

import * as ts from 'typescript'

const transform = {
  [ts.SyntaxKind.SourceFile]: (node: any) => {
    const file = (node as ts.Node) as ts.SourceFile
    const update = ts.updateSourceFileNode(file, [
      ts.createImportDeclaration(
        undefined,
        undefined,
        ts.createImportClause(
          ts.createIdentifier('salami'),
          undefined
          // ts.createNamespaceImport(ts.createIdentifier('salami'))
        ),
        ts.createLiteral('salami')
      ),
      ...file.statements
    ])

    return update as ts.Node
  }
}

console.log(
  ts.transpileModule(
    'import sandwich from "sandwich"; console.log(sandwich); console.log(salami)',
    {
      reportDiagnostics: true,
      transformers: {
        before: [visitor(transform)]
      },
      compilerOptions: {
        noEmitOnError: true,
        jsx: ts.JsxEmit.React,
        module: ts.ModuleKind.CommonJS,
        // module: ts.ModuleKind.ESNext,
        importHelpers: true,
        jsxFactory: 'h'
      }
    }
  )
)

function visitor(transform: any) {
  return (context: ts.TransformationContext) => {
    return (sourceFile: ts.SourceFile): ts.SourceFile => {
      return visit(sourceFile)
      function visit<T extends ts.Node>(node: T): T {
        const t = transform[node.kind]
        if (!t) {
          return ts.visitEachChild(node, visit, context)
        }

        // enter a node
        if (typeof t === 'object' && t.enter) {
          const tr = t.enter(node, context)
          if (tr) node = tr as T
        }

        // if it's a function, treat it
        // the same way as t.enter
        if (typeof t === 'function') {
          const tr = t(node, context)
          if (tr) node = tr as T
        }

        // visit each child
        let exitNode = ts.visitEachChild(node, visit, context)

        // if we have an exit call function
        if (typeof t === 'object' && t.exit) {
          const tr = t.exit(exitNode, context)
          if (tr) exitNode = tr as T
        }

        return exitNode
      }
    }
  }
}

Any assistance would be greatly appreciated! I think this is the last hurdle :-)

[matthewmueller]

@weswigham sorry to be a bother, but do you mind taking a look at this if you have a moment? This is the last piece of the puzzle. I'm just a bit stuck with why my newly created import doesn't get compiled to:

myimport["default"] when the target changes to commonjs

[weswigham]

Hm. I'll take a look; but this should also give you the effective output you want, as a workaround in the meantime:

const file = (node as ts.Node) as ts.SourceFile;
const update = ts.updateSourceFileNode(file, [
  ts.createImportDeclaration(
    undefined,
    undefined,
    ts.createImportClause(
      undefined,
      ts.createNamedImports([ts.createImportSpecifier(ts.createIdentifier("default"), ts.createIdentifier("salami"))])
    ),
    ts.createLiteral('salami')
  ),
  ...file.statements
]);

[weswigham]

OH, wait, you're asking why the references don't get updated! That's on you and your transform, sorry! Since your new nodes aren't bound to the same reference in the binder phase (as transforms happen after binding), TS has no idea that the salami you're adding is the same salami referenced elsewhere in the file; so it's up to you and your transform to track and rename everything appropriately! Sorry!

If this is too cumbersome for your use-case; I'd suggest opening an issue asking for a prebind transform phase, where you could add new nodes which can be bound/typechecked (with the downside being that there is no symbol/type/reference information ready).

[matthewmueller]

TS has no idea that the salami you're adding is the same salami referenced elsewhere in the file; so it's up to you and your transform to track and rename everything appropriately! Sorry!

Ahh that makes sense now. I thought that would be happening in between the before and after transforms and there was some sort of option/property I was missing on my import declaration node. Okay so that would also mean that my transform needs to know what we're targeting (es6 vs commonjs) and act accordingly.

I'll give it a go manually and open another issue if it's quite difficult. I'd imagine it's not so bad if I change the import to a require statement when targeting commonjs instead of adding ["default"] to each reference.

I really appreciate you taking the time to help me out, thanks!

[matthewmueller]

Alrighty, in case this helps anyone else out. Here's how you'd support different module options.

{
    [ts.SyntaxKind.SourceFile]: (node, context) => {
      if (!context) return
      const opts = context.getCompilerOptions()
      const { module } = opts

      // const StyledTSX = require('styled-tsx')
      if (module === ts.ModuleKind.CommonJS) {
        const file = (node as ts.Node) as ts.SourceFile
        const update = ts.updateSourceFileNode(file, [
          ts.createVariableStatement(
            /*modifiers*/ undefined,
            ts.createVariableDeclarationList([
              ts.createVariableDeclaration(
                importName,
                /*type*/ undefined,
                ts.createPropertyAccess(
                  ts.createCall(
                    ts.createIdentifier('require'),
                    [],
                    [ts.createLiteral(importFrom)]
                  ),
                  ts.createIdentifier('default')
                )
              )
            ])
          ),
          ...file.statements
        ])
        return (update as ts.Node) as T
      }

      // import StyledTSX from 'styled-tsx'
      if (module === ts.ModuleKind.ES2015 || module === ts.ModuleKind.ESNext) {
        const file = (node as ts.Node) as ts.SourceFile
        const update = ts.updateSourceFileNode(file, [
          ts.createImportDeclaration(
            /* decorators */ undefined,
            /* modifiers */ undefined,
            ts.createImportClause(ts.createIdentifier(importName), undefined),
            ts.createLiteral(importFrom)
          ),
          ...file.statements
        ])
        return (update as ts.Node) as T
      }

      return node
    }
  }

Doesn't yet cover AMD/UMD modules though.