How should import() work in service workers?

[jakearchibald]

We blocked import() in service workers because we weren't sure how they should work, but it wasn't intended to be a forever-fix. Now that modules are becoming more popular, and browsers now support module service workers, how should import() work?

Some thoughts:

• It should work the same in classic service workers as it does with module service workers.
• Ideally it should promote patterns that work offline.
• Loading of a module in one service worker cannot be impacted by another service worker.

Some ideas:

Other ideas considered

Option 1

Before the service worker is "installed", calls to import() are fetched, bypassing all service workers. These module resources are cached along with the service worker script.

After the service worker is installed, calls to import() will only use resources that are cached along with the service worker script, otherwise they will fail.

Eg:

addEventListener('install', (event) => {
  event.waitUntil((async () => {
    const cache = await caches.open('static-v1');
    return Promise.all([
      cache.addAll(urls),
      import('./big-script.js'),
      import('./another-big-script.js'),
    ]);
  })());
});

addEventListener('fetch', (event) => {
  event.respondWith((async () => {
    if (whatever) {
      const { complicatedThing } = await import('./big-script.js');
      // …etc…
    }
  })());
});

• ✅ Compatible with classic and module service workers
• ✅ Promotes working offline
• ✅ Similar to how importScripts works
• ✅ Don't need to know the full import tree, except for dynamic imports
• ⚠️ Requires parsing (and maybe executing) stuff in the install phase that isn't actually used

Another idea:

Option 2

Before the service worker is "activated", calls to import() just go to the network, bypassing all service workers.

After the service worker is "activated", calls to import() will go via that service worker's "fetch" event.

addEventListener('install', (event) => {
  event.waitUntil((async () => {
    const cache = await caches.open('static-v1');
    return cache.addAll([
      ...urls,
      './big-script.js',
      './big-script-dependency.js',
      './another-big-script.js',
      './another-big-script-dependency.js',
    ]);
  })());
});

addEventListener('fetch', (event) => {
  event.respondWith((async () => {
    if (whatever) {
      const { complicatedThing } = await import('./big-script.js');
    }
    return caches.match(event.request);
  })());
});

• ✅ Compatible with classic and module service workers
• ✅ Scripts can be cached without parsing them
• ⚠️ Need to list out all the module's dependencies
• ⚠️ It can work offline, via the same mechanism as pages, but it can also become network dependent. It's versatile, but folks might not test for it?
• ⚠️ Implementation complexities? I think it's the first time a request from a service worker will go through its own fetch event.

We could overcome the dependencies issue with something like cache.addModule('./big-script.js'), which would crawl the tree similar to modulepreload. That means we're having to parse the modules, but it doesn't need to execute them.

I preferred "option 2" when it was just in my head, but now I've written it down and thought it through, I don't think I like it. So, what about:

Option 1.5

Before the service worker is "installed", calls to import() fail.

Before the service worker is "installed", calls to installEvent.installModule(url) crawl a module tree similar to modulepreload, and cache the script along with the service worker script. This returns a promise that indicates success. It isn't necessary to pass this to waitUntil, but it doesn't hurt to do so.

After the service worker is "installed", calls to import() will only use resources that are cached along with the service worker script, otherwise they will fail.

After the service worker is "installed", calls to installEvent.installModule(url) reject.

Eg:

addEventListener('install', (event) => {
  event.installModule('./big-script.js');
  event.installModule('./another-big-script.js');
  event.waitUntil(
    caches.open('static-v1').then(c => c.addAll(urls)),
  );
});

addEventListener('fetch', (event) => {
  event.respondWith((async () => {
    if (whatever) {
      const { complicatedThing } = await import('./big-script.js');
      // …etc…
    }
  })());
});

• ✅ Compatible with classic and module service workers
• ✅ Promotes working offline
• ✅ Don't need to know the full import tree, except for dynamic imports
• ✅ Requires parsing the scripts at install time, but avoids executing them

Thoughts @wanderview @asutherland @jeffposnick @mfalken @youennf?

[asutherland]

xref #1356 which is the previous discussion in this space I was thinking of.

Option 1.5 seems like a great approach. I like the strong policy decision about the modulePreload/import install boundary and think this makes sense in a module world and could make for a more sane mental model for developers, but am not deeply invested in that. Whereas I am deeply invested in avoiding option 2 letting ServiceWorkers process their own fetch events!

As always, thank you for your truly fantastic write-ups.

[jeffposnick]

Option 1.5 seems pretty reasonable to me!

Something I'm curious about is if/how modules loaded via import() should contribute to the service worker's overall byte-for-byte update check. In options 1 & 1.5, given that the imported modules would be cached via the same mechanism used for the top-level script and static imports, would they also obey the updateViaCache registration setting, and would a change in an import()-ed module trigger a service worker update?

[wanderview]

Small bikeshed: I think modulepreload would confuse people with preload semantics in the main thread where something is just preloaded for the current load. This is more caching than preloading, right? Maybe cacheModule(foo) or "install"?

Also, should we make this work for importScripts as well? I mean, why should importScripts have to parse/compile/execute for offline caching?

Another strawperson:

self.addEventListener('install', evt => {
  evt.waitUntil(async function() {
    await evt.installScript('foo.js', 'classic');
    await evt.installScript('bar.js', 'module');
  }());
});

Note, I'd prefer not to hang this method off of caches if its not using the cache API normally. The install event seems a reasonable place?

[jakearchibald]

Note, I'd prefer not to hang this method off of caches if its not using the cache API normally. The install event seems a reasonable place?

This is why I went with modulePreload rather than cacheModule.

Putting this on the install event is obviously a better place for this! installModule? Worklets use addModule, but that includes immediate execution. Anyway, there's more behaviour stuff to figure out before we need to settle on names (like the stuff @jeffposnick mentions).

Edit: I've updated the example to use installEvent.installModule(url)

[ghazale-hosseinabadi]

@jakearchibald Is there a timeline on when this will be done? Thanks!

[jakearchibald]

I don't have a specific timeline for this, sorry

[ghazale-hosseinabadi]

I can work on this in Q3, with some guidance from the content layer. Do you have an estimate on how many engineering hours is needed for this? Thanks!

[jakearchibald]

I can try and get the spec and tests ready for then. @mfalken any idea on the eng hours?

[mfalken]

I would estimate about 1 month of development.

[jakearchibald]

Whatever exceptions we make for 'local' URLs in #1595 should be made here too.

[ggaabe]

Has any progress been made on this front recently? Would be great to be able to properly import a module.

[jimmywarting]

Guess I will have to still resort to using a custom version of shimport + a custom loader and eval() still with this kind of restrictions.

i wish that i could just import() something from OPFS
still wishes that things could just depend on network related things to progressively install plugins as needed as you explore the site and our plugin section more to allow for progressive enhancement.

[Danscap]

please allow imports for service workers soon!

[larry-xu]

@jakearchibald are there any updates on supporting dynamic import() for service workers? With the adoption of service workers for manifest v3 browser extensions, there is increasing interest from extension developers and browser vendors to support this functionality for extension service workers. However I think this would be useful to support for all service workers in general.

[jakearchibald]

It isn't something I personally have time for right now. If someone else wants to step in and do the work, I'm happy to review.

[mikemaccana]

So is it possible to load third party modules in service workers right now? I looked at Shimport per @jimmywarting 's suggestion but shimport creates <script> tags if import doesn't work - Service Workers don't have a DOM, so I don't think adding <script> tags would work. Is it possible to load third party modules at all?

[jimmywarting]

@mikemaccana Shimport fallbacks to eval if it's inside a worker that do not have DOM. so shimport still works in service workers

[mikemaccana]

Wait I think I've confused import() (a function I haven't heard of before) with ES6 import. ES6 imports do indeed work in service workers. Nevermind.

[serapath]

does firefox not support imports in service workers?
I'm using firefox 118

here is how i do it:

• https://github.com/serapath/ff-sw-import/

It works in google chrome, but not in mozilla firefox:

• https://serapath.github.io/ff-sw-import/

MDN says it should work:

• https://developer.mozilla.org/en-US/docs/Web/API/ServiceWorkerContainer/register#type
• https://developer.mozilla.org/en-US/docs/Web/API/ServiceWorkerContainer/register#browser_compatibility

caniuse says it should work:

• https://caniuse.com/mdn-javascript_statements_import_service_worker_support