[Bug]: Issues importing chromadb UnhandledSchemeError: Reading from "https://unpkg.com/@xenova/transformers@2.13.2"

[ashbuilds]

What happened?

Hi, Im trying to import chromadb into my nextjs server but getting the issue UnhandledSchemeError: Reading from "https://unpkg.com/@xenova/transformers@2.13.2", I tried to mark this as external modules but it doesn't work. Is there any workaround? Thank you!

Versions

chromadb: 1.9.2, node v22.10.0

Relevant log output

⨯ https://unpkg.com/@xenova/transformers@2.13.2
Module build failed: UnhandledSchemeError: Reading from "https://unpkg.com/@xenova/transformers@2.13.2" is not handled by plugins (Unhandled scheme).
Webpack supports "data:" and "file:" URIs by default.
You may need an additional plugin to handle "https:" URIs.
Import trace for requested module:
https://unpkg.com/@xenova/transformers@2.13.2
./node_modules/.pnpm/chromadb@1.9.1_openai@4.68.1_zod@3.23.8_/node_modules/chromadb/dist/chromadb.mjs

[CarmineXmX]

same problem

[BrianMwangi21]

Any new feedback on this ? Getting the same issue.

[fortysevenlabs]

If we are using an alternative embedding like OpenAI, why does @xenova/transformers / chromadb-default-embeddings even need to be installed? Can we make this optional as well?

[ashbuilds]

This is my workaround for now, but would be great if this can be optional.

  webpack: (config, { isServer }) => {
    config.externals = config.externals || [];
    
    // TODO: Workaround for issue : https://github.com/chroma-core/chroma/issues/2988
    config.externals.push({
      'https://unpkg.com/@xenova/transformers@2.13.2': 'transformers',
    });
    return config;
  }

[ChrisCates]

The workaround mentioned by @ashbuilds didn't work by just doing that alone.
However, it worked when I simply just installed the dependencies.

cohere-ai
@xenova/transformers
chromadb-default-embed

I'd like to avoid having to install dependencies... and then still have it direct to a CDN, I'll throw a PR at Chroma tomorrow to make sure that you dont need chroma's dependencies in your own package.json.

[ashbuilds]

The workaround mentioned by @ashbuilds didn't work by just doing that alone.
However, it worked when I simply just installed the dependencies.

cohere-ai
@xenova/transformers
chromadb-default-embed

I'd like to avoid having to install dependencies... and then still have it direct to a CDN, I'll throw a PR at Chroma tomorrow to make sure that you dont need chroma's dependencies in your own package.json.

Hi, yes you need @xenova/transformers package to be installed