bug: service worker server.url not injecting capacitor

[dylanvdmerwe]

Bug Report

Capacitor Version

Latest Dependencies:

  @capacitor/cli: 3.3.2
  @capacitor/core: 3.3.2
  @capacitor/android: 3.3.2
  @capacitor/ios: 3.3.2

Installed Dependencies:

  @capacitor/ios: not installed
  @capacitor/cli: 3.3.2
  @capacitor/android: 3.3.2
  @capacitor/core: 3.3.2

Platform(s)

Current Behavior

Calling any native plugins when an app is served from an Angular service worker causes the plugins to fail. Using server.url to load the site on Android.

Platform ready does not help for this.

Expected Behavior

Capacitor needs to be injected when a site is served through a service worker.

Code Reproduction

app.module.ts

export function init_app(appLoadService: AppLoadService) {
  return () => appLoadService.initApp();
}

@NgModule({
  declarations: [AppComponent],
  entryComponents: [],
  imports: [
    BrowserModule,
    IonicModule.forRoot(),
    AppRoutingModule,
    ServiceWorkerModule.register('ngsw-worker.js', {
      enabled: environment.production,
      // Register the ServiceWorker as soon as the app is stable
      // or after 30 seconds (whichever comes first).
      registrationStrategy: 'registerWhenStable:30000',
    }),
  ],
  providers: [
    { provide: RouteReuseStrategy, useClass: IonicRouteStrategy },
    {
      provide: APP_INITIALIZER,
      useFactory: init_app,
      deps: [AppLoadService],
      multi: true,
    },
  ],
  bootstrap: [AppComponent],
})
export class AppModule {}

app-load-service.ts

@Injectable({
  providedIn: 'root',
})
export class AppLoadService {
  constructor(
    private device: DeviceInfo,
    private platform: Platform
  ) {}

  async initApp(): Promise<any> {
    await this.platform.ready();

    SplashScreen.hide(); // breaks
  }
}

[dylanvdmerwe]

It actually seems that the capacitor script is not being injected after the site is served by a service worker.

I have included this on Android but still getting plugin errors:

I don't believe that this works any more.

The site is picking up that its running on Android through a service worker:
console.log('Platforms: ' + Capacitor.getPlatform());
console.log('Is Native: ' + Capacitor.isNativePlatform());
Platforms: android
Is Native: true

[webuniverseio]

Silly question, but for a workaround, it is possible to inject capacitor script manually?

[goforu]

The issue, anyone have a workaround?

[dylanvdmerwe]

Please see a repo here: https://github.com/happendev/TestCapacitor

The PWA site is hosted on https://testapp.happen.zone.

Android:

1. When site loaded through the Android app without a service worker, native plugins work, capacitor works.
2. When site is loaded through a service worker after the site has been cached, native plugins and capacitor do not work.

Maybe I have done something wrong?

[dylanvdmerwe]

When the Android app is first installed you can see Capacitor is working:

The site files are downloaded and served directly on first run:

When the app is launched the second time, Capacitor is not injected:

The files are now served from the service worker:

[dylanvdmerwe]

I have yet to do some additional testing, but I hope this issue is not the same on iOS.

[jgoux]

@dylanvdmerwe I started looking into what should be done in order to support this use case (I'll need it as well).

This is what we talked about with @thomasvidas on Discord :

me :
I think we need to introduce the ServiceWorkerClient in order to intercept requests from the service worker : https://stackoverflow.com/a/59606081 and https://stackoverflow.com/questions/62354423/how-to-use-serviceworkercontroller-on-android/63491172#63491172
The shouldInterceptRequest is the same as the WebviewClient's one so it should be a matter of calling them both

@thomasvidas :
That seems pretty doable. I'd assume that WebViewLocalServer.shouldInterceptRequest() could be modified to check if it should serve from the service worker or from the normal local capacitor server.
Maybe it would be better to change bridge.getLocalServer()? But I'm more wary of that since this doesn't work on API 21-23 and I try to avoid touching the bridge if I can since it touches so much of the codebase.

@dylanvdmerwe Did you test on iOS a well?

[dylanvdmerwe]

@jgoux I have tried to get a service worker working so I can test this on iOS - but I did not come right with whatever needs to be enabled for this. Does anyone know what I missed in the repo? I would love to check out how the service worker is affected on iOS as well as part of this. Or is there another repo where this works that I can use for testing?

[xEsk]

I'm facing this "problem" too....

As a temporary "workaround" (in order to solve this), I force a site reload when the ServiceWorker is Ready. I know and I also don't like the solution, but I needed to implement this 💩 while this "issue/bug/feature" is addressed.

if (Capacitor.isNativePlatform() && Capacitor.getPlatform() === 'android') {
  const reboot = localStorage.getItem('reboot')
  if (reboot !== '1') {
    localStorage.setItem('reboot', '1')
    setTimeout(() => location.reload(), 1)
  } else {
    localStorage.setItem('reboot', '0')
  }
}

This code is working for me, at the time as I write this... (is working on production).

Forgive me... 😂

[dylanvdmerwe]

Using this kind of functionality is primarily not for the app stores in our case, we really want to use this for apps that are rolled out (enterprise) to operators and employees at our clients. This would drastically reduce the building required to deploy updates to them, as all we would need to do is update the PWA.

Please advise if there is anything that I can do to assist or test to get server.url working for Android and iOS through a service worker on Capacitor.

[dylanvdmerwe]

@jgoux Do you have any more feedback on the way forward on this?

Has anyone tried anything further as we would think this kind of thing would be incredible to get working through Capacitor apps.

[jgoux]

@dylanvdmerwe Hello Dylan,

I didn't have time to investigate yet.

Did you already try this solution : https://stackoverflow.com/a/55916297/1728627 ?

I believe @mhjam made a contribution to make it works here : #1465, maybe he knows more about it? 😄

[goforu]

Nope, It's not working anymore.

[chrisharris77]

If the path your service worker is loading contains a ".html" file extension, the the following update in WebViewLocalServer.handleProxyRequest() resolves the issue (if you also hook up a service worker client delegating to WebViewLocalServer like in https://stackoverflow.com/questions/62354423/how-to-use-serviceworkercontroller-on-android/63491172#63491172). Note that this is exactly the same hack that is in WebViewLocalServer.handleLocalRequest(), so if it's good enough for local requests it should be good enough for proxy ones too. I've opened a pull request for it: #5428

                if (url.contains(".html")) {
                    isHtmlText = true;
                } else {
                    for (Map.Entry<String, String> header : headers.entrySet()) {
                        if (header.getKey().equalsIgnoreCase("Accept") && header.getValue().toLowerCase().contains("text/html")) {
                            isHtmlText = true;
                            break;
                        }
                    }
                }

[dylanvdmerwe]

@chrisharris77 No this does not appear to work, at least not when using a normal ionic app as per my repo.

https://github.com/happendev/TestCapacitor

https://testapp.happen.zone/

1. Install app for the first time. The app works. It loads the files from the site and appears to install a service worker for offline. Plugins work.
2. Restart the app, the app loads the files from the service worker's offline cache. Capacitor does not seem to be initialized and as such plugins fail and the app breaks when they are used.

[dylanvdmerwe]

if (Capacitor.isNativePlatform() && Capacitor.getPlatform() === 'android') {
const reboot = localStorage.getItem('reboot')
if (reboot !== '1') {
localStorage.setItem('reboot', '1')
setTimeout(() => location.reload(), 1)
} else {
localStorage.setItem('reboot', '0')
}
}

Hi @xEsk, can you provide some more context, or a repo for this? I have tried, but even with this Capacitor is not initialized which means plugins don't work when loaded from the SW offline version. Any use of the plugins breaks.

[dylanvdmerwe]

The first run (when not served by the service worker):

• Platform: Android
• Native: true.

This is the service worker cached PWA version of the app running on Android device.

As you can see, it still detects that the platform is Android and isNativePlatform is true. But it has not initialized any android plugin stuff which is why the plugins are breaking.

Is there maybe a way to force Capacitor to run in 'web' mode here (as a true PWA), as then the plugins should work? Ideally it should setup itself correctly and run in proper Android mode.

[Aarbel]

@dylanvdmerwe @webuniverseio @goforu @xEsk @chrisharris77

Any updates about this issue ?

[Aarbel]

if (Capacitor.isNativePlatform() && Capacitor.getPlatform() === 'android') {
  const reboot = localStorage.getItem('reboot')
  if (reboot !== '1') {
    localStorage.setItem('reboot', '1')
    setTimeout(() => location.reload(), 1)
  } else {
    localStorage.setItem('reboot', '0')
  }
}

Is this part still working ?
Do service workers also work with this fix ? (managing dynamic modules download)

Thanks a lot !

[dharmeshds]

Is there any permanent resolution to this issue to make service workers work with capacitor plugins on Android? It is very frustrating that my Ionic6/Angular13 app works nicely with the service worker on browser but fails to register the same on Android. When I implement the workaround suggested, service worker loads but capacitor plugins fail to load on the device. Please update when there is any resolution or workaround for this.

[BraulioMonroy]

ffs

[Aarbel]

Still nobody found a solution here ?

[yoyo930021]

I using this answer and work for me on android with capacitor 2.
https://stackoverflow.com/questions/55894716/how-to-package-a-hosted-web-app-with-ionic-capacitor/55916297#55916297

but I replace ServiceWorkerController to ServiceWorkerControllerCompat for my case.

[Aarbel]

Thanks a lot @yoyo930021

I'm ready to pay someone as freelance to develop the supported module / fix for capacitor.
Who wants here ?

[perpetuallight]

Update: blocking network loads like so [ServiceWorkerController].getServiceWorkerWebSettings().setBlockNetworkLoads(true); seems to consistently prevent the service worker from intercepting network requests. But looking at its state in devtools shows that it gets stuck in the "trying to install" state and the network tab shows that the service worker failed to load due to CACHE_MISS. I wonder if this is because of the Service-Worker: script in the request headers.

If there is anyone else using the 3rd party service worker for something other than caching I would be interested to know if it will load the other features without the request header present while "blocking network loads".

It is not super clear how some of these settings are supposed to behave from the documentation. So I am trying to infer what I can.

[perpetuallight]

Update: I was assuming that the Service-Worker: script header was coming from the server but it was in the request. So I removed it and it did still run just without the conventional semantic understanding from the browser. Possibly observed another side effect that appears to cause the caching logic to infinitely loop on itself, at least in my particular 3rd party worker.

Then I decided to modify the Accept header to match the initial requests' value which happened to have "text/html" in it while the worker was using */*. This was apparently what was preventing the Capacitor proxy from processing it.

In my case forcing the proxy to process it fixes the whole cache.

So this will work for me. And I imagine it will help someone else too. Check your request headers for Accept: text/html if it's not there try modifying it to match.

final String path = "" // Your path condition here.
ServiceWorkerController swc = ServiceWorkerController.getInstance();
swc.setServiceWorkerClient(new ServiceWorkerClient() {
  @Override
  public WebResourceResponse shouldInterceptRequest(WebResourceRequest req) {
    if (req.getUrl().toString().endsWith(path))
      req.getRequestHeaders().put("Accept", "text/html");
    return bridge.getLocalServer().shouldInterceptRequest(req);
  }
});

[cyprian-nziim]

When the Android app is first installed you can see Capacitor is working:

The site files are downloaded and served directly on first run:

When the app is launched the second time, Capacitor is not injected:

The files are now served from the service worker:

Had this sam issue,

This worked for me. I just have to check if platform is Android, then I reloadForCapacitorInjection.
Hope this helps.
Thanks for all your inputs above.

`reloadForCapacitorInjection() {

const scripts = document.querySelectorAll("head script[type='text/javascript']:not([src])");
console.log('Looking for Script with Capacitor')
let matches = [];
if(scripts.length) {
  scripts.forEach((script, index) => {
    if (script && script.textContent?.includes("Capacitor")) { 
      matches.push(script);
      console.log('found Script with Capacitor')
    } else if(index+1 == scripts.length && matches.length < 1) {
      console.log('reloading no Capacitor')
      location.reload();
    }
  })
} else {
  console.log('reloading no Capacitor')
  location.reload();
}

}`

[kyllerss]

I am also impacted by this. Any updates on whether or not this is on the roadmap? Thanks!

[shahrukh1990]

I am also facing same issue. I have removed the service worker from my code. Still same.
Any idea?

[pspaulding]

I've tried several of the above suggestions, including various changes to MainActivity.java, and none of them worked. I am using Webpack 4 + sw-precache-webpack-plugin. The basic window.Capacitor functions work fine when using a service worker, but all the plugins (including @capacitor/app) fail to work. In my case, the stripe-terminal plugin that I am using only makes sense to use when the application is online, and only a subset of our users need the functionality provided by the plugin.

I've also tried isolating all use of the plugin to a separate entry point that is not managed by the service worker, and loading it as an ES module ahead of the application. Still no luck (probably because the index.html is managed by a service worker). I imagine those who are more knowledgeable about Capacitor are laughing at me right now.

My hack which others may be able to use is to toggle the service worker on/off by anticipating the user's needs (and reloading). This is far from ideal, as it means the users that depend on the plugin do not benefit from a service worker for the 95% of the time that they are not using the plugin's functionality.

Overall, using Capacitor has been a positive experience. I just feel a bit uneasy as my primary reason for switching from Cordova (which worked fine with service workers) to Capacitor was for this particular plugin.

[yoyo930021]

I've tried several of the above suggestions, including various changes to MainActivity.java, and none of them worked. I am using Webpack 4 + sw-precache-webpack-plugin. The basic window.Capacitor functions work fine when using a service worker, but all the plugins (including @capacitor/app) fail to work. In my case, the stripe-terminal plugin that I am using only makes sense to use when the application is online, and only a subset of our users need the functionality provided by the plugin.

I've also tried isolating all use of the plugin to a separate entry point that is not managed by the service worker, and loading it as an ES module ahead of the application. Still no luck (probably because the index.html is managed by a service worker). I imagine those who are more knowledgeable about Capacitor are laughing at me right now.

My hack which others may be able to use is to toggle the service worker on/off by anticipating the user's needs (and reloading). This is far from ideal, as it means the users that depend on the plugin do not benefit from a service worker for the 95% of the time that they are not using the plugin's functionality.

Overall, using Capacitor has been a positive experience. I just feel a bit uneasy as my primary reason for switching from Cordova (which worked fine with service workers) to Capacitor was for this particular plugin.

Do you try to this idea?
#5278 (comment)

The request.getUrl().toString().contains("index.html") replace to your html filename or detect html logic.

[pspaulding]

Thanks @yoyo930021 , I just tried it again and it did not work. Java is not my forte, but when I do the following:

public class MainActivity extends BridgeActivity {
    private static final String TAG = "myapp";
    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
Log.v(TAG, "SDK_INT:" + Build.VERSION.SDK_INT);
        if (Build.VERSION.SDK_INT >= 24) {
            ServiceWorkerController swController = ServiceWorkerController.getInstance();

            swController.setServiceWorkerClient(new ServiceWorkerClient() {
                @Override
                public WebResourceResponse shouldInterceptRequest(WebResourceRequest request) {
                    Log.v(TAG, request.getUrl().toString());
                    if (request.getUrl().toString().contains("index.html")) {
                        request.getRequestHeaders().put("Accept", "text/html");
                    }
                    var result = bridge.getLocalServer().shouldInterceptRequest(request);
                    return result;
                }
            });
        }
    }
}

the first Log.v logs "SDK_INT:30" in Logcat, but the second Log.v(TAG, request.getUrl().toString()); is never called.

[pspaulding]

Oops, just realized it wasn't logging because I had just disabled the service worker. Too many moving parts, LOL. When I reenable the service worker, the first URL logged is "https://MY_DOMAIN/service-worker.js"...I never see an "index.html", but since I configure the Capacitor URL for just "https://MY_DOMAIN" and that server responds with index.html, I'm not entirely surprised. The line immediately before mine is from the "Capacitor/Console" tag, "I", "File: https://MY_DOMAIN/ - Line 228 - Msg: undefined".

4/20/23 UPDATE: Now it is working. Despite the fact that "index.html" is never logged! I am going to do some more testing.

4/21/23 UPDATE: "index.html" is logged, just had trouble finding it in the sea of other data. So, in my case, the request.getUrl().toString().contains("index.html") logic is key to making remote URLs function with service workers. Thanks for your help @yoyo930021

[henry-alakazhang]

Hey everyone. I also ran into this issue a while ago, but successfully worked around it by adding the Accept: text/html header. (I also had to work around the Angular service worker hash checks by rewriting the ngsw.json, but that's a separate story). In testing and in most cases, the app works fine now. First load downloads the app normally; the service worker registers itself; second load is from service worker cache and the Capacitor runtime is injected as it should be.

However, in the wild, I've got a very high volume Sentry error reporting that "XYZ" Plugin is not implemented on android. These are definitely coming from Android users on the latest version of the app and website, but it doesn't happen consistently. Other telemetry tells me the same users are able to use the app at least some of the time.

Has anyone else run into this? The same issue where the runtime isn't injected, but flaky instead of all the time. If not, I guess there must be something else wrong inside our service worker wrapper.

EDIT: It was in fact something else wrong inside our service worker wrapper. We had some additional logic (🙃) to avoid weird interactions with in-app browsers, which was sometimes causing the index.html to not be injected.

[Aarbel]

Accept: text/html

Thanks a lot Henry, can give us more details about where you add the Accept: text/html header ?

[yoyo930021]

I suggest not attempting to enable service workers in capacitor android. Although there are various hacks available to make it function to some extent, in reality, it performs poorly.

Known issues:

1. Service worker requests do not use a custom user agent.
2. The service worker is re-established every time the app is launched and is only useful during the current app lifecycle.

[toriphes]

After working several hours on this I ended up with this solution working on capacitor 5.
Hope this can help someone.

public class MainActivity extends BridgeActivity {

  @Override
  public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);

    if (android.os.Build.VERSION.SDK_INT >= android.os.Build.VERSION_CODES.N) {
      ServiceWorkerController swController = null;
      swController = ServiceWorkerController.getInstance();

      swController.setServiceWorkerClient(new ServiceWorkerClient() {
        @Override
        public WebResourceResponse shouldInterceptRequest(WebResourceRequest request) {
          if (request.getUrl().toString().contains("index.html")) {
            request.getRequestHeaders().put("Accept", "text/html");
          }
          return bridge.getLocalServer().shouldInterceptRequest(request);
        }
      });
    }
  }
}

[leo-buneev]

2. The service worker is re-established every time the app is launched and is only useful during the current app lifecycle.

Just for info - I can't confirm that. We're using SW successfully (with text/html workaround) on capacitor(v4) android with remote URL to store application files, and it works offline after app restart. So it's useful not only during app lifecycle.

[COBRASoft5]

What about iOS? Does the service worker work the same as on Android within Capacitor ?

[perpetuallight]

I thought service workers were a staple of PWA's? The user-agent could be part of the problem. I remember considering that when I was working on this. Perhaps it can be addressed at the specification level. On second thought, probably not. I remember what the user-agent looked like now. It was very intentional. Definitely for the server to use. But in this case the client may need to be aware as well at a higher level. If I remember correctly I couldn't use it because it was inconsistently available between requests. That could be addressed at the spec level.

[perpetuallight]

I've tried several of the above suggestions, including various changes to MainActivity.java, and none of them worked. I am using Webpack 4 + sw-precache-webpack-plugin. The basic window.Capacitor functions work fine when using a service worker, but all the plugins (including @capacitor/app) fail to work. In my case, the stripe-terminal plugin that I am using only makes sense to use when the application is online, and only a subset of our users need the functionality provided by the plugin.

I've also tried isolating all use of the plugin to a separate entry point that is not managed by the service worker, and loading it as an ES module ahead of the application. Still no luck (probably because the index.html is managed by a service worker). I imagine those who are more knowledgeable about Capacitor are laughing at me right now.

My hack which others may be able to use is to toggle the service worker on/off by anticipating the user's needs (and reloading). This is far from ideal, as it means the users that depend on the plugin do not benefit from a service worker for the 95% of the time that they are not using the plugin's functionality.

Overall, using Capacitor has been a positive experience. I just feel a bit uneasy as my primary reason for switching from Cordova (which worked fine with service workers) to Capacitor was for this particular plugin.

In your case, I think stripe uses service workers anyway. I could be wrong, but I seem to remember that being the case. So you may want to try the same proxy bypass but load something else instead when offline? The proxy bypass I am referring to is the same one everyone seems to be using but modified for your case.

[perpetuallight]

It seems like these edge cases are going to be quite difficult to work with in Capacitor in general due to how the proxy is done. Maybe the better fix is to simply add something to the documentation about when/why/how to bypass the proxy to handle a given edge case?

[Erudition]

If it's not supported that's fine, but the Capacitor docs literally suggest using a Service Worker to make the webapp progressive when it's run in-browser, with no hint that it will break plugin functionality when run on native mobile.

Perhaps we can add certain Capacitor-injection paths to a service worker's interception "exception list"?

Also: disabling the service worker was not enough, to fix the "not implemented on android" errors. I also had to rm -rf android/app/src/main/assets/* before cap sync.

[Erudition]

2. The service worker is re-established every time the app is launched and is only useful during the current app lifecycle.

Actually, the service worker stays established and persists between sessions so long as the webview's storage is not cleared - in fact it's only on the second+ load of the app (with SW serving interceptions) that all my Capacitor plugins crash.

If there are specific Capacitor injection scripts that we need to blocklist, that would be a good solution for now!

[bosung90]

Would be nice to have this included as part of Capacitor by default, many people will be able to save time.
#5278 (comment)

[ZeroHackeR]

After working several hours on this I ended up with this solution working on capacitor 5. Hope this can help someone.

public class MainActivity extends BridgeActivity {

  @Override
  public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);

    if (android.os.Build.VERSION.SDK_INT >= android.os.Build.VERSION_CODES.N) {
      ServiceWorkerController swController = null;
      swController = ServiceWorkerController.getInstance();

      swController.setServiceWorkerClient(new ServiceWorkerClient() {
        @Override
        public WebResourceResponse shouldInterceptRequest(WebResourceRequest request) {
          if (request.getUrl().toString().contains("index.html")) {
            request.getRequestHeaders().put("Accept", "text/html");
          }
          return bridge.getLocalServer().shouldInterceptRequest(request);
        }
      });
    }
  }
}

Hello everyone! :)

I am having a similar issue.

1. I downloaded an html file to app's data folder.
2. Then, I used Capacitor.convertFileSrc to load the html file.
3. The HTML was loaded. However, Capacitor scripts were not being injected.

How do I fix the issue? Please help! :(

PS. already tried above codes and it didn't help.

[Erudition]

@mlynch

You would probably be well served with Live Updates which would let you do all of that and keep a fast local experience.

Unfortunately I'm not able to include proprietary software or services in my app. I just want to update as soon as my CI builds a new static app on github pages, saving the app store updates for plugin changes - but also have offline support which Service Workers are perfect for (and is already working well in browsers).

Since none of the workarounds here are working (unless you do SSR), in the mean time I will try to have a local startup file that grabs Capacitor and then hands it off to the app somehow

[ionitron-bot]

Thanks for the issue! This issue is being locked to prevent comments that are not relevant to the original issue. If this is still an issue with the latest version of Capacitor, please create a new issue and ensure the template is fully filled out.