Get clicked registerProtocolHandler (protocol_handlers) links in background

[hanguokai]

registerProtocolHandler can't work in extensions

Standard Web has navigator.registerProtocolHandler method. But in my test, it can't work in extensions. On Chrome, in fact, registerProtocolHandler() itself registers successfully, but the browser can't navigate the page to an extension link.

Extension version of registerProtocolHandler

Firefox supports protocol_handlers.

Chrome is implementing this feature (since years ago).

I created this issue here hoping to move it forward. It is useful if extensions can registerProtocolHandler(well known protocol or custom protocol). Without this feature, developers need to register content scripts to handle it that need host permissions, or context menu(let users right click links).

[hanguokai]

Update: After adding web_accessible_resources, clicking the protocol links can navigate to the extension page.

It seems that registerProtocolHandler and protocol_handlers are very similar. And registerProtocolHandler already works in extensions.

Next, I want to discuss:

What is the difference between them.

The permission of registerProtocolHandler is triggered in extension pages. The permission of protocol_handlers is triggered when installing the extension. Are there any other important differences?

Functionality and user experience

On the other hand, the user experience of simply redirecting a web page to an extension page is not very good. For example, I'd like to keep the current page open and handle it with a background event, or open the extension page in a new tab or new window to handle it (maybe close it soon).

[xeenon]

I am in support for doing this like Firefox with protocol_handlers for Safari. This position was later changed to opposed in a more recent comment after talking with more membered of the Safari and WebKit team.

[hanguokai]

Expanding my previous comment, I added an issue in both service work spec #1665 and registerProtocolHandler spec #8596.

[hanguokai]

Sync some information, from whatwg/html#8596 and w3c/ServiceWorker#1665

Current Native Apps Behavior

When click a registered protocol link, the native app is woken up and the link is passed to the app for processing. Note: the current page are not closed or navigate to other page, you can continue to click other links in the page that active the native app again.

Current Web Behavior

At present, when click a registered protocol link, Chrome navigate current page to the registered URL. In other words, current page is unload, you can't continue to click other links in this page. In some scenarios, this behavior is not expected, instead users expect the behavior like native apps. Of course, if the link has target="_blank" attribute, browser open the target URL in a new tab, but it depends on the websites, the developers who call registerProtocolHandler() can't control them.

At present, the spec of registerProtocolHandler doesn't describe the detailed behavior what happen when click the link. The behavior depends on how the browser implements it.

Proposal

I hope registerProtocolHandler() can support native apps' behavior. When users click the link, fire an event with the link information in service worker, then the website(web app) can handle it in service worker, and the current page stay there.

This approach is more flexible for developers, and the current page does not navigate to other page. For example, in service worker, developers can open a new tab or popup window, or active(focus) an already opened page or popup window to handle it.

Below is a sample code for demonstration purpose:

// in a web page of example.com
// here add a new parameter, which tell the browser fire event in service worker and don't navigate current.
navigator.registerProtocolHandler(
    "magnet", // protocal
    "https://example.com/add?uri=%s", // url
    {type: 'service-worker', navigation: 'none'} // new option object
);

// in service worker of example.com
self.addEventListener("protocol_handler", e => {
  let protocol = e.protocol; // "magnet"
  let link = e.originalLink; // the %s part
}); 

1. PWA "protocol_handlers"

https://developer.chrome.com/articles/url-protocol-handler/
This is a declarative way. It is easy to use, but only for installed PWA. Non-PWA web and browser extensions can't use it.

2. Firefox extension's "protocol_handlers"

https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/manifest.json/protocol_handlers
Similar to PWA, it is also a declarative way, but just for extensions. There is no background event for clicking a register link.

So both 1) and 2) are declarative ways, and are not general(universal) solution.

My proposal

My proposal expands registerProtocolHandler and let service worker to handle the click event.

1. This is a programmatic solution. This approach is more flexible (than declarative way) and leaves it up to the developer to manage how it is handled. It is very similar to handle notification event in service worker.
2. It should work for 1) installed PWA, 2) non PWA(general web), and 3) browser extensions (Chrome MV3 is service worker based, and Firefox now support event page).

[hanguokai]

If this proposal can't move forward in Web standards side, I think we can try the following improvements in browser extensions side.

1. Declare protocol_handlers in extension manifest
   Use current Firefox's way. It tells the browser which protocols this extension want to handle.

2. Add a listener in background.js or service_worker.js

browser.runtime.onRegisterProtocalLinkClicked.addListener(e => {
  let protocol = e.protocol; // "magnet"
  let link = e.originalLink; // the %s part
});

If developers add this listener, then when user click a registered link, the browser don't navigate current page to uriTemplate(it can be ignore in this case), instead trigger an event to that listener.

[fred-wang]

The title of the Chromium issue was confusing, so to clarify: https://bugs.chromium.org/p/chromium/issues/detail?id=64100#c59 ; basically the owner of the extension code had concerns about the approach with protocol_handler in the manifest and what ended up being implemented in Chrome is giving special power to registerProtocolHandler in web extensions.

[hanguokai]

@fred-wang Thanks for your explanation for the current status of Chrome.

For me, both dynamic way(registerProtocolHandler()) and static way(declare it in manifest) are not a problem. I can guide users to complete registration. My point is that the behavior of navigating(redirect) to the handler page is an unacceptable user experience for users in some cases.

For example, there are many download(magnet) links on a typical BitTorrent website page, when the user clicks a link, the BitTorrent client downloads one, clicks another link and downloads another, and the current page does not change. But current behavior of Web protocol handler is that click a magnet link, goto handler page, then the user needs to go back to the previous page; then click next magnet link, goto handler page, then go back; click next link …… This behavior makes it impossible to use it in this scenario.

So I thought of another alternative behavior. When the user clicks a link, instead of redirecting to the handler page, trigger a background event to tell me the link information. In browser extensions, background events are widely used to obtain information and process it. I think this is more reasonable. So I proposal a new behavior for standard Web and browser extensions (here).

[javifernandez]

I'm trying to reactivate the discussion about the declarative approach implemented by Firefox to manage the custom handlers registration from extensions. It seems that Safari expressed support and it was suggested in the 2022-12-08 meeting to gather additional feedback form Chrome.

As it was commented there were some concerns raised by the owners of the extension code during the discussion in the bug 64100, but 2 years passed since then. I tried to reactivate the discussion in the bug without much success so far, so perhaps we could revisit the topic again in the next meeting ?

[hanguokai]

By the way, currently developers use content script as a workaround to protocol handler. For example:

// in content script
let links = document.querySelectorAll('...');
for (let link of link) {
  link.addEventListener('click', function(e) {
    e.preventDefault(); // prevent default protocol handler
    e.stopImmediatePropagation();
    let link = e.currentTarget;
    let url = link.href;
    browser.runtime.sendMessage({url}); // send link to background
  });
}

// in background script
browser.runtime.onMessage.addListener(function(request) {
  let url = request.url
  // handle the url when user click a link
});

Note: when user click the link, the page doesn't navigate to other page.

The disadvantage of content scripts is that it needs host permissions. This is why I propose support for protocol handlers in service worker. See comment-1 and comment-2.

[javifernandez]

In latest versions of Chrome (not sure from which version, but I can try to figure it out) extensions become privileged actors when it comes to use the registerProtocolHandler API. Hence, it's possible to register a custom handler just using this simple script run as service worker:

  "background": {
       "service_worker": "background.js"
   },
   "web_accessible_resources": [
       {
           "resources": [ "register.html" ],
           "extension_ids": ["*"]
       }

The service worker could be as simple as this:

 chrome.runtime.onInstalled.addListener(details => {
  if (details.reason === chrome.runtime.OnInstalledReason.INSTALL) {
      chrome.tabs.create({"url": "register.html"});
  }
});

And the register.html script just need to trigger the user gesture, mandatory to invoke the registerProtocolHandler API

function register() {
    console.log("Registering ipfs...");
    navigator.registerProtocolHandler("ipfs", "https://dweb.link/ipfs/?uri=%s", "title_ipfs");
}

window.addEventListener("DOMContentLoaded", function() {
    document.getElementsByTagName("button")[0].addEventListener("click", register); // user gesture
});

[javifernandez]

The problem of using the extension's tab is that it has some usability drawbacks; the user would have to trigger the registration with an user gesture and give content twice to, basically, the same thing.

[hanguokai]

The problem of using the extension's tab is that it has some usability drawbacks; the user would have to trigger the registration with an user gesture and give content twice to, basically, the same thing.

This is not a drawback. Users only click a button and confirm it. You can add some description for this button. There is one more benefit: you can provide a button for unregisterProtocolHandler() when users want to cancel it.

[javifernandez]

The problem of using the extension's tab is that it has some usability drawbacks; the user would have to trigger the registration with an user gesture and give content twice to, basically, the same thing.

This is not a drawback. Users only click a button and confirm it. You can add some description for this button. There is one more benefit: you can provide a button for unregisterProtocolHandler() when users want to cancel it.

Well, first users click to "install an extension" that they know what its for. Then they have to click a button to register the handler, and then grant the registration, again.

[hanguokai]

Well, first users click to "install an extension" that they know what its for. Then they have to click a button to register the handler, and then grant the registration, again.

This is a best practice. Other optional permissions do the same process.

[javifernandez]

Well, first users click to "install an extension" that they know what its for. Then they have to click a button to register the handler, and then grant the registration, again.

This is a best practice. Other optional permissions do the same process.

Except for the additional step of granting the handler registration.

I mean, the usual workflow is to install + granting optional permissions, but in this case it seems there is one additional, and perhaps redundant, step.

This is why I think the declarative approach provided by the Manifest's protocol_handler gives us some advantages. Users would be able to avoid the HTML api and grants the registration as an optional permission.

[oliverdunk]

Adding supportive: chrome Supportive from Chrome for the protocol_handlers field in the manifest. We didn't have a chance to discuss it in the meeting, but I had some discussion about it internally and since this is the direction PWAs are taking it makes sense to do similar in extensions. I think this is slightly different to what we originally said in the Chromium bug but we're on board.

It seems like the other part of this issue is about adding new events/ways to handle protocols without opening a page. I haven't discussed that.

[hanguokai]

Sometimes I feel like the people in the video conference don't think deeply about what I'm saying. For example, in the first video conference about this issue, they thought they were talking about the manifest. But I'm talking about behavior when clicking on links.

[hanguokai]

@oliverdunk Chrome team also support my idea (get clicked link info in service worker). In WICG/web-app-launch#86 , Chrome member would like to support "launch_handler": { "client_mode": "service-worker" } in PWA manifest launch_handler in the future.

[hanguokai]

"protocol_handlers": [
  {
    "protocol": "ircs",
    "name": "IRC Mozilla Extension",
    "uriTemplate": "https://irccloud.mozilla.com/#!/%s",
    "background": true // add a new field for new behavior (demo purpose)
  }
]

So in the extension manifest, except uriTemplate, it should add a new field that tell the browser the new click behavior.

[javifernandez]

Adding supportive: chrome Supportive from Chrome for the protocol_handlers field in the manifest. We didn't have a chance to discuss it in the meeting, but I had some discussion about it internally and since this is the direction PWAs are taking it makes sense to do similar in extensions. I think this is slightly different to what we originally said in the Chromium bug but we're on board.

Cool ! Thank you @oliverdunk for the effort to unblock this old issue. Would you mind then add some feedback on the Chrome bug 64100 that we have precisely for this ?

We also have API Overview Doc that we could get back to live.

It seems like the other part of this issue is about adding new events/ways to handle protocols without opening a page. I haven't discussed that.
s like the other part of this issue is about adding new events/ways to handle protocols without opening a page. I haven't discussed that.

Yes, I think that's a different topic that deserves its own issue. Perhaps the HTML spec, where the Custom Handlers API is defined would be more appropriated. We would need first to ensure there is enough interest from major browser vendors.

[javifernandez]

Sometimes I feel like the people in the video conference don't think deeply about what I'm saying. For example, in the first video conference about this issue, they thought they were talking about the manifest. But I'm talking about behavior when clicking on links.

Yes, I think there was a confusion regarding this. As I said, this behavior is specified in the Custom Handlers section of the HTML spec:

9. Navigate an appropriate navigable to resultURL.

So if we want to change that behavior, that spec is where we would need to put our efforts on.

[hanguokai]

So if we want to change that behavior, that spec is where we would need to put our efforts on.

This is why I created whatwg/html#8596 and w3c/ServiceWorker#1665 I commented at #317 (comment)

I also commented at #317 (comment) (copy below again)

If this proposal can't move forward in Web standards side, I think we can try the following improvements in browser extensions side.

1. Declare protocol_handlers in extension manifest
   Use current Firefox's way. It tells the browser which protocols this extension want to handle.

2. Add a listener in background.js or service_worker.js

browser.runtime.onRegisterProtocalLinkClicked.addListener(e => {
  let protocol = e.protocol; // "magnet"
  let link = e.originalLink; // the %s part
});

If developers add this listener, then when user click a registered link, the browser don't navigate current page to uriTemplate(it can be ignore in this case), instead trigger an event to that listener.

[hanguokai]

As commented before, declare protocol_handlers in manifest is not a blocker for developers, it can be workaround by registerProtocolHandler. But click behavior is a real blocker for web and extension developers, it can be workaround only by content script as I commented at #317 (comment)

[hanguokai]

My second comment in this issue already started talking about the click behavior (navigation behavior). It should not be confused with the manifest issue. Copy it below again:

Functionality and user experience

On the other hand, the user experience of simply redirecting a web page to an extension page is not very good. For example, I'd like to keep the current page open and handle it with a background event, or open the extension page in a new tab or new window to handle it (maybe close it soon).

[hanguokai]

To @javifernandez @oliverdunk @fred-wang and other people

In order to distinguish these two different topics, I created a new issue #365 for "protocol_handlers" in manifest. Please discuss manifest issue in #365 . I will modify current issue's title and labels.

[javifernandez]

My second comment in this issue already started talking about the click behavior (navigation behavior). It should not be confused with the manifest issue. Copy it below again:

I understand, however, we can change the HTML API from the Web Extensions spec; even though this issue may have had other purpose, it got a promising agreement between 3 major browser vendors about the implementation of the Web protocol_handlers property of the Web Extensions Manifest as the primary way to register custom handlers from extensions.

IMHO, we should keep focus of this issue on that specific goal, and even close it with an agreement conclusions. We can discuss the changes you are proposing in the HTML API in the other issues you've mentioned. Hence, I'm not sure the change in the title helps to solve the confusion, as I don't think Web Extensions is the right place to change the behavior of the HTML API.

[javifernandez]

To @javifernandez @oliverdunk @fred-wang and other people

In order to distinguish these two different topics, I created a new issue #365 for "protocol_handlers" in manifest. Please discuss manifest issue in #365 . I will modify current issue's title and labels.

I´d rather use a new issue for the change in the registerProtocolHandler, and as I commented, I don't think Extensions is the right venue for that.

This issue has the explicit comments of the different browser vendors supporting the implementation of Manifest's protocol_handler, so it'd be better to get back the original title and even close the issue with an agreement.

[hanguokai]

This issue doesn't change the HTML or Web Service Worker Spec, I reported my proposal in their repositories. Instead, this issue discusses how to resolve related issue independently in extensions. And I proposed further proposals(ideas) for extensions.

How to register protocol handler is not the point here. I support both declarative and programmatic registration. I researched some history to understand the current state of protocol handler. Browser vendors haven't really discussed this issue in the past, they even didn't understand my main point. Don't worry, their comments here didn't go away, I move their position labels to #365 .

[javifernandez]

How to register protocol handler is not the point here. I support both declarative and programmatic registration. I researched some history to understand the current state of protocol handler. Browser vendors haven't really discussed this issue in the past, they even didn't understand my main point. Don't worry, their comments here didn't go away, I move their position labels to #365 .

The Custom Handlers section, where the registerProtocolHandler HTML API is defined, states also the browser's behavior should use the registered protocol, not only the registration process. It states:

9. Navigate an appropriate navigable to resultURL.

Where the Navigate actions is also precisely specified.

Isn't that the behavior you are proposing to change ?

[hanguokai]

Isn't that the behavior you are proposing to change ?

I'm not sure if it's necessary to modify HTML's API, which is a meeting point for several related areas.

[me]: The spec doesn't describe the detailed behavior what happen when click the link.
[domenic]: Right. The spec does not mandate browser UI behavior. Each browser gets to choose that.
[me]: To support different behaviors, the api may need to add a new parameter.
[domenic]: I don't think so; I think browser vendors are able to implement the behavior you want with the current spec.
source: whatwg/html#8596 (comment)

It may also be resolved by "Web App Launch Handling" in PWA. So, extensions may also be supported in the same way.

[me]: is this proposal related to web-app-launch?
[alancutter]: Yes it is, you most likely want "launch_handler": { "client_mode": "service-worker" }. The reason for doing it at this level is so it can affect all launches, not just protocol handler launches. I recommend not altering the existing registerProtocolHandler() API and instead pursuing this API.
[source]: WICG/web-app-launch#86

Possible extensions to "Web App Launch Handling":
Add "service-worker" as a client_mode option. This would invoke a service worker launch event.
source: https://github.com/WICG/web-app-launch/blob/main/launch_handler.md#possible-extensions-to-this-proposal

Like PWA Launch Handling behavior, it can bring some information (e.g. a URL or a FileHandle) to developers' callback. So extensions may also be supported in the same way.

By the way, why Chrome would like to support "protocol_handlers" in manifest now? [oliverdunk] said since this is the direction PWAs are taking it makes sense to do similar in extensions.

PS: registerProtocolHandler is a very old API that has not been significantly improved over the years, and some of its own problems have prevented it from being widely used by Web developers. So, in addition to extensions, I also want to push for improvements in Web standards.

[oliverdunk]

Sometimes I feel like the people in the video conference don't think deeply about what I'm saying. For example, in the first video conference about this issue, they thought they were talking about the manifest. But I'm talking about behavior when clicking on links.

I'm sorry you feel that way, and I really appreciate you bringing it up. It's definitely not the intention.

In this case, I think we realised during the meetings that there was a small change which we could all agree on and make in the short term. Consequently, discussion moved on to that. You did the right thing by pointing out you'd still like to see the larger change to how protocols are opened discussed.

I think personally I would've preferred to keep this issue as is and open a new one for the new event, but given things have already been changed around, I think we should probably leave them as they are.

Cool ! Thank you @oliverdunk for the effort to unblock this old issue. Would you mind then add some feedback on the Chrome bug 64100 that we have precisely for this ?

Done!

[oliverdunk]

@hanguokai To summarise from today's meeting (notes), it'd be interesting to write down a few cases (including your BitTorrent one) where this would be useful. Also - do you have thoughts on if just immediately closing the extension page would work? Clearly that's a bit of a workaround, but it seems like something that may help a little.

[hanguokai]

do you have thoughts on if just immediately closing the extension page would work?

This is not a feasible solution. For details, see my previous comments whatwg/html#8596 or #317 (comment) and whatwg/html#8596 (comment) . I have given a workaround in my previous comment, see #317 (comment) .