[js/web] ESM: use the bundled target as default export

[fs-eire]

Description

ESM: use the bundled target as default export

In this change, the default import of the following entries:

import from 'onnxruntime-web';
import from 'onnxruntime-web/all';
import from 'onnxruntime-web/webgpu';

will use the "bundled" version, which has no dynamic import.

This change should only apply to ESM on web.

[lucasgelfond]

🔥

[qdrk]

This is somehow broken again in the latest dev releases. For example in 1.21.0-dev.20250117-db8e10b0b9

[fs-eire]

This is somehow broken again in the latest dev releases. For example in 1.21.0-dev.20250117-db8e10b0b9

Could you please create a new issue and describe the reproduce steps? I will take a look if I can reproduce it.

[qdrk]

This is somehow broken again in the latest dev releases. For example in 1.21.0-dev.20250117-db8e10b0b9

Could you please create a new issue and describe the reproduce steps? I will take a look if I can reproduce it.

Thank you @fs-eire! I'm using transformer.js which depends on onnxruntime-web. Trying to initialize the model in a Chrome extension's background js, but got this error:

Uncaught (in promise) Error: no available backend found. ERR: [webgpu] TypeError: import() is disallowed on ServiceWorkerGlobalScope by the HTML specification. See https://github.com/w3c/ServiceWorker/issues/1356.
    at Ji (http://localhost:9999/node_modules/.vite/deps/onnxruntime-web.js?v=d13d4b24:83:13)
    at async e.create (http://localhost:9999/node_modules/.vite/deps/onnxruntime-web.js?v=d13d4b24:631:20)

Project is is a little messy to extract out, but if you clone the https://github.com/huggingface/transformers.js/tree/main/examples/extension project and upgrade it to 3.3.0+, you'll see this issue. Anything below 3.2.4 works totally fine.

@xenova in case this is something you have noticed too.

[fs-eire]

Ah, I see. so this is a service worker use case.

There are some limitation in service worker, for example, it cannot use dynamic loading. I am tracking the status of supporting service worker for transformer.js in #23257

[qdrk]

Thanks! Just wondering, is dynamic loading only used in the newer dev releases like 1.21.0-dev.20250117-db8e10b0b9? Since the older one works totally fine, like 1.21.0-dev.20241205-d27fecd3d3

[qdrk]

@fs-eire could it be this change? 0627a6c

I'm trying to revert this and build locally but got a different error after that:

[fs-eire]

@fs-eire could it be this change? 0627a6c

I'm trying to revert this and build locally but got a different error after that:

Yes, it is because of this change. But simply reverting it will not fix. Since it is a use case of transformersjs, a workaround for current version is to set env.backends.onnx.wasm.wasmPaths to undefined.

I am working on figuring out the best solution for problems of exports for transformerjs. Some details are here: huggingface/transformers.js#1161

[qdrk]

@fs-eire could it be this change? 0627a6c
I'm trying to revert this and build locally but got a different error after that:

Yes, it is because of this change. But simply reverting it will not fix. Since it is a use case of transformersjs, a workaround for current version is to set env.backends.onnx.wasm.wasmPaths to undefined.

I am working on figuring out the best solution for problems of exports for transformerjs. Some details are here: huggingface/transformers.js#1161

Somehow still got the error. Is this the right way to set the wasmPaths?

import {
  AutoTokenizer,
  PreTrainedTokenizer,
  AutoModelForCausalLM,
  TextStreamer,
  InterruptableStoppingCriteria,
  env,
} from "@huggingface/transformers";

env.backends.onnx.wasm!.wasmPaths = undefined;

[fs-eire]

@fs-eire could it be this change? 0627a6c
I'm trying to revert this and build locally but got a different error after that:

Yes, it is because of this change. But simply reverting it will not fix. Since it is a use case of transformersjs, a workaround for current version is to set env.backends.onnx.wasm.wasmPaths to undefined.
I am working on figuring out the best solution for problems of exports for transformerjs. Some details are here: huggingface/transformers.js#1161

Somehow still got the error. Is this the right way to set the wasmPaths?

import {
  AutoTokenizer,
  PreTrainedTokenizer,
  AutoModelForCausalLM,
  TextStreamer,
  InterruptableStoppingCriteria,
  env,
} from "@huggingface/transformers";

env.backends.onnx.wasm!.wasmPaths = undefined;

This should work in my understanding. Could you share a full example for me to debug?

[qdrk]

Sure, prepared one here: https://github.com/qdrk/vite-web-extension

To reproduce

1. clone the repo
2. cd to the directory, run npm install (I'm using node v18)
3. run npm run dev
4. go to chrome, load unpacked extension, navigate to this project, you should see a directory called "dist_chrome", load that
5. open the dev tool of the service worker, in the console you should see the error.

Thanks in advance for looking into it!

[fs-eire]

Sure, prepared one here: https://github.com/qdrk/vite-web-extension

To reproduce

1. clone the repo
2. cd to the directory, run npm install (I'm using node v18)
3. run npm run dev
4. go to chrome, load unpacked extension, navigate to this project, you should see a directory called "dist_chrome", load that
5. open the dev tool of the service worker, in the console you should see the error.

Thanks in advance for looking into it!

I can reproduce the issue locally.

The solution is:

1. Use latest dev version of onnxruntime-web
2. Apply fix package export huggingface/transformers.js#1161 to transformers.js ( This PR included latest dev version of onnxruntime-web )
   • This will allow the JavaScript code to find the correct URL for the WebAssembly file.
3. Add script-src 'wasm-unsafe-eval' to content_security_policy to allow using WebAssembly. See No way to use WebAssembly on Chrome without 'unsafe-eval' WebAssembly/content-security-policy#7 for more info.

    {
      "manifest_version": 3,
      "name": "name in manifest.json",
      ...
   +  "content_security_policy": {
   +    "extension_pages": "script-src 'self' 'wasm-unsafe-eval'; object-src 'self'"
   +  },

   • This will allow the JS to load and run WebAssembly.

[qdrk]

I can confirm the workaround works for the repro repo! Even though the issue still exists for my other project, but it's probably because it's setup slightly differently. Thank you so much @fs-eire for the prompt resolution!

[qdrk]

Multi-threading seems to have some issue too, got "Worker is not defined" when creating the inferenceSession.

[fs-eire]

Multi-threading seems to have some issue too, got "Worker is not defined" when creating the inferenceSession.

(Please correct me if I’m wrong)

in my understanding, creating a Worker in MV3 is not allowed, so you just have to use single thread.

see https://issues.chromium.org/issues/40772041