Unexpected token errors coming from import-in-the-middle

[SerenModz21]

Is there an existing issue for this?

• I have checked for existing issues https://github.com/getsentry/sentry-javascript/issues
• I have reviewed the documentation https://docs.sentry.io/
• I am using the latest SDK release https://github.com/getsentry/sentry-javascript/releases

How do you use Sentry?

Sentry Saas (sentry.io)

Which SDK are you using?

@sentry/node

SDK Version

8.8.0

Framework Version

Node v20.14.0

Link to Sentry event

No response

SDK Setup

Sentry.init({
    dsn: envParseString("SENTRY_DSN"),
    integrations: [
        Sentry.nativeNodeFetchIntegration(),
        Sentry.consoleIntegration(),
        Sentry.onUncaughtExceptionIntegration(),
        Sentry.onUnhandledRejectionIntegration(),
        Sentry.functionToStringIntegration(),
        Sentry.modulesIntegration(),
        Sentry.linkedErrorsIntegration(),
        Sentry.httpIntegration(),
    ],
    tracesSampleRate: 1.0,
    environment: nodeEnvironment,
    release: `${name}@${version}`,
});

Steps to Reproduce

I simply had migrated from v7 to v8.

I was using 8.7.0 with globalThis._sentryEsmLoaderHookRegistered = true; until the "no such file or directory" got issue, which was the case with 8.8.0. Since installing the update, i commented out that line of code to check if everything had been fixed but i get spammed with the mentioned unexpected token errors.

Just like the "no such file or directory" error, i'm just gonna have to keep using globalThis._sentryEsmLoaderHookRegistered = true; or just fallback to v7 for the time being.

There is #12357 which is quite similar, the difference being that I do not use TSX but the standard TypeScript compiler. Where my startup is simly tsc && node --enable-source-maps .. In addition, my project involves discord.js, @sapphire/framework, and other sapphire plugins/utilities (eg. decorators)

Expected Result

Errors should not appear.

Actual Result

SyntaxError [Error]: Unexpected token (3:40)
    at pp$4.raise (PATH_REDACTED\node_modules\acorn\dist\acorn.js:3573:15)
    at pp$9.unexpected (PATH_REDACTED\node_modules\acorn\dist\acorn.js:772:10)
    at pp$9.semicolon (PATH_REDACTED\node_modules\acorn\dist\acorn.js:749:68)
    at Parser.parseImport (PATH_REDACTED\node_modules\acorn-import-attributes\lib\index.js:242:12)
    at pp$8.parseStatement (PATH_REDACTED\node_modules\acorn\dist\acorn.js:948:51)
    at pp$8.parseTopLevel (PATH_REDACTED\node_modules\acorn\dist\acorn.js:829:23)
    at Parser.parse (PATH_REDACTED\node_modules\acorn\dist\acorn.js:601:17)
    at Function.parse (PATH_REDACTED\node_modules\acorn\dist\acorn.js:651:37)
    at getEsmExports (PATH_REDACTED\node_modules\@opentelemetry\instrumentation\node_modules\import-in-the-middle\lib\get-esm-exports.js:37:23)     
    at getExports (PATH_REDACTED\node_modules\@opentelemetry\instrumentation\node_modules\import-in-the-middle\lib\get-exports.js:73:12) {    
  pos: 166,
  loc: { line: 3, column: 40 },
  raisedAt: 170
}

[Lms24]

Hi @SerenModz21 thanks for writing in and sorry for the troubles! I'm not yet sure how or why this error is thrown, especially because just transpiling with TSC shouldn't have an impact here. To debug this further:

• Just to confirm: Your're transpiling your code to ESM ("import" syntax), correct?
• which installation method of the Node SDK are you using?
  • related, are you passing the --import when you're starting your application as documented in the ESM-specific guides?

Let's first check if the SDK is set up correctly and go from there.

[SerenModz21]

I do indeed transpile to ESM. As for installation, that would be the late initialization method. Unfortunately however, I made the mistake of not trying with --import @sentry/node/preload; therefore, I will give such a try and provide an update on any further news. I am just a little confused how auto-instrumentation would make a difference in this situation.

[Lms24]

The --import @sentry/node/preload option ensures that import-in-the-middle and OpenTelemetry instrumentation is added early enough. I'm not sure if it makes a difference with your issue but IITM has been known for interfering with import syntax. Let's see if it changes something!

[timfish]

It could be the --enable-source-maps option which I have a suspicion could be implemented via loader hooks or at least hits the resolution hook.

I guess import-in-the-middle should be passing non-js files straight through to the next loader. Maybe if there are any exceptions iitm should console.error them but fall back to the next loader rather than fail at this point. It would make it less of a blocker when it fails on things like this.

[SerenModz21]

As an update to my previous comment, I have now tried the following three commands:

node --import @sentry/node/preload --enable-source-maps .
node --import @sentry/node/preload .
node --import @sentry/node/preload dist/index.js

However, they all show the "unexpected token" error, followed by the node version (20.14.0) and then it finishes executing, as in it just finishes and does not start the actual node app.

❯ node --import @sentry/node/preload .

node:internal/modules/run_main:129
    triggerUncaughtException(
    ^
SyntaxError [Error]: Unexpected token (3:40)
    at pp$4.raise (PATH_REDACTED\node_modules\acorn\dist\acorn.js:3573:15)     
    at pp$9.unexpected (PATH_REDACTED\node_modules\acorn\dist\acorn.js:772:10) 
    at pp$9.semicolon (PATH_REDACTED\node_modules\acorn\dist\acorn.js:749:68)  
    at Parser.parseImport (PATH_REDACTED\node_modules\acorn-import-attributes\lib\index.js:242:12)
    at pp$8.parseStatement (PATH_REDACTED\node_modules\acorn\dist\acorn.js:948:51)
    at pp$8.parseTopLevel (PATH_REDACTED\node_modules\acorn\dist\acorn.js:829:23)
    at Parser.parse (PATH_REDACTED\node_modules\acorn\dist\acorn.js:601:17)    
    at Function.parse (PATH_REDACTED\node_modules\acorn\dist\acorn.js:651:37)  
    at getEsmExports (PATH_REDACTED\node_modules\@opentelemetry\instrumentation\node_modules\import-in-the-middle\lib\get-esm-exports.js:37:23)
    at getExports (PATH_REDACTED\node_modules\@opentelemetry\instrumentation\node_modules\import-in-the-middle\lib\get-exports.js:73:12) {
  pos: 166,
  loc: { line: 3, column: 40 },
  raisedAt: 170
}

Node.js v20.14.0

My apologies for the delay in mentioning this update.

[SerenModz21]

In addition, here is the output with debug logs:

❯ SENTRY_DEBUG=1 node --import @sentry/node/preload .
Sentry Logger [debug]: @opentelemetry/api: Registered a global for diag v1.9.0.
Sentry Logger [log]: [Sentry] Preloaded Http instrumentation
Sentry Logger [log]: [Sentry] Preloaded Express instrumentation
Sentry Logger [log]: [Sentry] Preloaded Connect instrumentation
Sentry Logger [log]: [Sentry] Preloaded Fastify instrumentation
Sentry Logger [log]: [Sentry] Preloaded Hapi instrumentation
Sentry Logger [log]: [Sentry] Preloaded Koa instrumentation
Sentry Logger [log]: [Sentry] Preloaded Nest instrumentation
Sentry Logger [log]: [Sentry] Preloaded Mongo instrumentation
Sentry Logger [log]: [Sentry] Preloaded Mongoose instrumentation
Sentry Logger [log]: [Sentry] Preloaded Mysql instrumentation
Sentry Logger [log]: [Sentry] Preloaded Mysql2 instrumentation
Sentry Logger [log]: [Sentry] Preloaded Postgres instrumentation
Sentry Logger [log]: [Sentry] Preloaded Hapi instrumentation
Sentry Logger [log]: [Sentry] Preloaded Graphql instrumentation
Sentry Logger [log]: [Sentry] Preloaded Redis instrumentation

node:internal/modules/run_main:129
    triggerUncaughtException(
    ^
SyntaxError [Error]: Unexpected token (3:40)

[AnthonyDugarte]

any update?

SENTRY_DEBUG=1 node -r dotenv/config --import @sentry/node/preload .
Sentry Logger [debug]: @opentelemetry/api: Registered a global for diag v1.9.0.
Sentry Logger [log]: [Sentry] Preloaded Http instrumentation
Sentry Logger [log]: [Sentry] Preloaded Express instrumentation
Sentry Logger [log]: [Sentry] Preloaded Connect instrumentation
Sentry Logger [log]: [Sentry] Preloaded Fastify instrumentation
Sentry Logger [log]: [Sentry] Preloaded Hapi instrumentation
Sentry Logger [log]: [Sentry] Preloaded Koa instrumentation
Sentry Logger [log]: [Sentry] Preloaded Nest instrumentation
Sentry Logger [log]: [Sentry] Preloaded Mongo instrumentation
Sentry Logger [log]: [Sentry] Preloaded Mongoose instrumentation
Sentry Logger [log]: [Sentry] Preloaded Mysql instrumentation
Sentry Logger [log]: [Sentry] Preloaded Mysql2 instrumentation
Sentry Logger [log]: [Sentry] Preloaded Postgres instrumentation
Sentry Logger [log]: [Sentry] Preloaded Hapi instrumentation
Sentry Logger [log]: [Sentry] Preloaded Graphql instrumentation
Sentry Logger [log]: [Sentry] Preloaded Redis instrumentation

node:internal/process/esm_loader:34
      internalBinding('errors').triggerUncaughtException(
                                ^
SyntaxError [Error]: Unexpected token (13:76)
    at pp$4.raise (/path-to-project/node_modules/.pnpm/acorn@8.10.0/node_modules/acorn/dist/acorn.js:3560:15)
    at pp$9.unexpected (/path-to-project/node_modules/.pnpm/acorn@8.10.0/node_modules/acorn/dist/acorn.js:768:10)
    at pp$9.semicolon (/path-to-project/node_modules/.pnpm/acorn@8.10.0/node_modules/acorn/dist/acorn.js:745:68)
    at Parser.parseImport (/path-to-project/node_modules/.pnpm/acorn-import-attributes@1.9.5_acorn@8.10.0/node_modules/acorn-import-attributes/lib/index.js:242:12)
    at pp$8.parseStatement (/path-to-project/node_modules/.pnpm/acorn@8.10.0/node_modules/acorn/dist/acorn.js:944:51)
    at pp$8.parseTopLevel (/path-to-project/node_modules/.pnpm/acorn@8.10.0/node_modules/acorn/dist/acorn.js:825:23)
    at Parser.parse (/path-to-project/node_modules/.pnpm/acorn@8.10.0/node_modules/acorn/dist/acorn.js:597:17)
    at Function.parse (/path-to-project/node_modules/.pnpm/acorn@8.10.0/node_modules/acorn/dist/acorn.js:647:37)
    at getEsmExports (/path-to-project/node_modules/.pnpm/import-in-the-middle@1.8.0/node_modules/import-in-the-middle/lib/get-esm-exports.js:37:23)
    at getExports (/path-to-project/node_modules/.pnpm/import-in-the-middle@1.8.0/node_modules/import-in-the-middle/lib/get-exports.js:73:12) {
  pos: 978,
  loc: { line: 13, column: 76 },
  raisedAt: 982
}

Node.js v20.12.1

[timfish]

I suspect this is caused by a combination of your dependencies and import-in-the-middle but I've not yet worked out the dependency that's causing this.

[Lms24]

@AnthonyDugarte have you tried removing -r dotenv/config? Not sure if this changes something but if dotenv.config requires a JSON file, it might be problematic for IITM 🤔

[AnthonyDugarte]

@AnthonyDugarte have you tried removing -r dotenv/config? Not sure if this changes something but if dotenv.config requires a JSON file, it might be problematic for IITM 🤔

@Lms24 that gave me an idea, I have a internal dependency that imports json files:

import json from './file.json' with { type: 'json' };

Commented those out and the previous error went away so prob will make them js objects for now.

Now I got a different issue, checking that one

[SerenModz21]

I tried removing the modules integration and also tried commenting out all integrations that I use but unfortuantely, the Unexpected import statement in CJS module error remains.

Here is a small repo that I have setup that reproduces the error: https://github.com/SerenModz21/sentry-error

[timfish]

Thanks for the repro, super helpful. I managed to track this down and create PR to fix this. I'll update the import-in-the-middle patch at a more sensible time tomorrow!

It's worth noting that this doesn't appear to work:

import { name, version } from "../../package.json" with { type: "json" };

The JSON is exported as the default export so you'll need to do:

import pkg from "../../package.json" with { type: "json" };

[timfish]

I've updated the patch here to include the above linked PR.

[SerenModz21]

I have just tested the new patch and my application is now able to execute and be used, which is great news! The errors still get logged but I'm assuming that is to be expected right now, until the PRs finish getting merged and released.

So that you have some logs of everything: https://gist.github.com/SerenModz21/81b8a593d1935a3de45eaded861f4043
I have uploaded it to a gist as I don't want to make the history of this issue get a little too long.

[timfish]

The errors still get logged but I'm assuming that is to be expected right now, until the PRs finish getting merged and released

import-in-the-middle will continue to log warnings for these errors even after all PRs are merged.

[SerenModz21]

The errors still get logged but I'm assuming that is to be expected right now, until the PRs finish getting merged and released

import-in-the-middle will continue to log warnings for these errors even after all PRs are merged.

Ah, that is fine. Im guess it's just a matter of time until the Acorn issue that you made gets addressed? That being the import attribute one.

In addition, I would just like to say that I have appreciated your time greatly in helping me resolve the issue I was having - thank you!

[FozzieHi]

@timfish I'm experiencing a similar issue, do you think this is related?

Error: 'import-in-the-middle' failed to wrap 'file:///app/node_modules/zlib-sync/index.js'
    at getSource (/app/node_modules/import-in-the-middle/hook.js:304:21)
    at async load (/app/node_modules/import-in-the-middle/hook.js:319:26)
    at async nextLoad (node:internal/modules/esm/hooks:866:22)
    at async Hooks.load (node:internal/modules/esm/hooks:449:20)
    at async handleMessage (node:internal/modules/esm/worker:196:18) {
  cause: Error: Unexpected closing brace.
    at @:4:142
      at parseSource (/app/node_modules/cjs-module-lexer/lexer.js:185:17)
      at parseCJS (/app/node_modules/cjs-module-lexer/lexer.js:43:5)
      at getCjsExports (/app/node_modules/import-in-the-middle/lib/get-exports.js:37:20)
      at getExports (/app/node_modules/import-in-the-middle/lib/get-exports.js:100:12)
      at async /app/node_modules/import-in-the-middle/lib/get-exports.js:48:28
      at async Promise.all (index 0)
      at async getCjsExports (/app/node_modules/import-in-the-middle/lib/get-exports.js:40:5)
      at async processModule (/app/node_modules/import-in-the-middle/hook.js:131:23)
      at async getSource (/app/node_modules/import-in-the-middle/hook.js:269:25)
      at async load (/app/node_modules/import-in-the-middle/hook.js:319:26) {
    loc: 973
  }
}

Is this an issue within import-in-the-middle or Sentry's implementation of it? I'm happy to open an issue wherever.

[timfish]

@FozzieHi I think this is just a warning being logged. Does this affect your application otherwise?

[FozzieHi]

@FozzieHi I think this is just a warning being logged. Does this affect your application otherwise?

It does seem to yes, zlib-sync is not used by me directly but discord.js uses it for compression, and it logs out this below the import-in-the-middle error:

WebSocketShard: Compression is enabled but zlib-sync is not installed, falling back to identify compress

[AbhiPrasad]

@FozzieHi mind opening a new issue? We can dive deeper there.

For everyone else:

With the newest release of import-in-the-middle v1.9.0 this should be fixed.

If you upgrade to a fresh install of the latest version of the Node SDK it should use import-in-the-middle@1.9.0 by default.

[FozzieHi]

@AbhiPrasad Will do, thanks for your work on this though!