Selenium IDE broken in Chrome extensions using Manifest V3

[ferbs]

The upcoming Manifest v3 changes to Chromium browser extensions will break the Selenium IDE. (As well as Tampermonkey and many other useful extensions.) It will block any code not bundled/shipped within the extension itself.
See Google's announced, still-draft plans.

See also Tampermonkey discussion.

On a forum they may or may not monitor I've suggested using tab-scoped permissions for executing remotely hosted scripts as an alternative to completely blocking them. Perhaps the Selenium team has contacts on the Chromium team and can talk to them about this as well?

[corevo]

Hey, thanks for putting this to our attention, for the time being I believe that it is still a draft, and we are still in the "wait-and-see" stage.
When manifest v3 rolls out, extensions will have to adapt, and there will be more than a few years until old v2 extensions will get removed, I believe that by this point Google will do its thinking and maybe they'll soften their views, or have a whitelist, only time will tell.

[ferbs]

It's possible that the Chromium team hasn't considered Selenium IDE as a use case while working on their v3 plans. I'd suggest a polite mention of it now. Maybe ask one of your Google contributors/committers if they can find out more? (And suggest more finely grained permissions for external code.)

The team may have decided that angering Tampermonkey users is worth the increased security but I suspect they haven't taken into account the harm it would also do to QA and web automation.

[tourdedave]

I passed this along to some of the Google committers on the Selenium project. We'll wait and see if they have anything to say.

[bobcode99]

I have a question to ask, does selenium-ide start planning update manifest to v3 version?
Because of Manifest V2 support timeline shows that at January 2023:

Chrome Web Store stops accepting updates to existing Manifest V2 extensions
Chrome stops running Manifest V2 extensions

[toddtarsi]

@bobcode99 - That is a great question. Honestly, we're planning to move to being an electron app, so we're probably not planning to support manifest-v3 in this product's lifetime. However, we intend to make migrating this as easy as possible, and to make the code largely reusable, with some slight changes to move away from webextension specific antipatterns and gotchas

[atigm]

Any update ?

[toddtarsi]

Heck yeah! We're trying to get v4 out using non-code-signed releases within the next week or two, but it doesn't have code-export yet. Once our committee that grants budget actually gets us some signing certificates, we can deploy code-signed binaries for all platforms, but we're not quite there yet

[atigm]

Thank you

[vguntupalli-mn]

Hi

Can you please help me understand the below, in above context?

1.) Do you have plans to support Manifest-3 version of Chrome extension? If so, do you have any timelines?
2.) If not, is the Electron version the "go-forward" one?

Thanks in advance,
Vijay

[toddtarsi]

@vguntupalli-mn -

1. I was hoping this would be a 0-work transition, but if it's not, then no I won't support it. I don't have any interest in working on v3 anymore.
2. After looking at the work to migrate that I don't care to do, I guess so.

[vguntupalli-mn]

@toddtarsi Thank you for the quick response. If I assume as a summary of all of it, to be that there is no plan to support Manifest Version-3 extension of Selenium IDE for chrome, is that accurate (in any release of the IDE)?

[toddtarsi]

@vguntupalli-mn - I don't personally. Just not interested in spending my evenings on it. I spent seven months making v4 good enough for my use case and don't care enough about v3 to spend that time and stress on it. PRs are welcome if you feel otherwise.

[vguntupalli-mn]

@toddtarsi thank you very much, for confirming and also for the quick responses. Have a good day!!!

[whomobile]

Just in case anyone is still interested in v3 support, I want to share what I found. I tried to update it and it looks like there are many efforts needed to update dependencies, I am blocked while updating react-dnd components and unsafe-eval errors on packages.

1. changing the manifest itself was not complex, though not 1:1 changes, mainly CSP and permission are issue.
2. background.js Some window reference has to change with self.window.
3. some old packages have to be upgraded which fixes security issues.

I don't exactly why unsafe-eval is not allowed even it's configured in the manifest, if that works, though not secure, it looks possible to upgrade v3 manifest.

just FYI only. (v4 looks more promising than v3, but it still has some limitations. )

[toddtarsi]

@whomobile - You hit the nail on the head. I am not sure what to do about the unsafe-eval stuff. They say use static files for your eval stuff now and just don't eval, but 'eval' is basically the only way a web-based R&P tool can work. I looked for a while and just couldn't muster the effort. Especially because v2 to v3 was due to permission model changes. WebEx is a shit platform to target.

[toddtarsi]

@whomobile - Is there anything I could add to v4 to make it better for your use case? I know documentation and code signing haha, those I'll get back into. Our six month plan (probably four month plan now :P) is to remove the electron specific bindings to the playback windows, so that we could drive whatever browser you want in local playback. For my two year plan, I'm basically tripling down on code export and looking at paring down the IDE to just being a UI on a REPL console basically. Recent developments like Bidi means we should be able to go language and framework agnostic and deprecate a large portion of the codebase (side-runtime will instead derive from language bindings + a bit of extra sugar, side-runner will be replaced with whatever your chosen test runner is)

[whomobile]

@toddtarsi, thank you for putting in the effort overall. I like the idea of IDE becoming a REPL console. a few thoughts here, (just a dump of thoughts.)

1. IDE can have many target use cases. one of them records and play. The current v4 approach works well for 'single page' test cases. across pages, there are some troubles. (e.g selecting windows/tab for action, back/forward contents etcs).

2. IDE can be a tool to make a solid test script. (Just for example, as it has multiple targets supported, I guess, between steps, if not able to find targets, it could automatically move to the next possible targets, but typically it stuck. certainly, manually change them but maybe a little loop between targets may help. )

[toddtarsi]

@whomobile - Thanks for letting me know your thoughts. When going forward, I'll keep these problems in mind. I really believe the repl console is the path to enhancing both of those aspects where the tool struggles, but I don't want to waste too much time blowing air out about it when I need to just get to work on changing the thing over.

[seyedmaysamlavasani]

@whomobile and @toddtarsi - we have an old project on selenium ide v.3 but cannot migrate it to manifest v.3 because of the reasons u discussed above. I am just checking if there is any new finding / insight you guys have. Or you still think it is not possible to do this because of the unsafe Vals?

[toddtarsi]

@seyedmaysamlavasani - I would recommend migrating to v4. It's supposedly in alpha, but I'm up to v90 and the worst bug at this point is an intermittent hang when running suites at high concurrency.

https://github.com/SeleniumHQ/selenium-ide/releases/tag/v4.0.1-alpha.90

If you want to migrate from v3 of the IDE to v4, here's an npm module that should do it for you in one line (unless you have custom commands via plugins):

https://www.npmjs.com/package/@seleniumhq/side-migrate

[maheshsundar98]

@toddtarsi @whomobile @seyedmaysamlavasani we also have a project using selenium ide. we need to convert manifest 3 (we are in manifest 2) as per recent chrome timeline announcements. is there any update or any workaround to overcome the unsave vals ?

[toddtarsi]

@maheshsundar98 - No, and to do it would be a fool's errand imo.

Arbitrary script execution is no longer allowed, so you do a bunch of work to create a messaging channel to pass through the arbitrary scripts and then eval them in the receiving client and then reply with the response. That's great except that's exactly what v3 is about preventing, so now you're likely against the terms of the store itself. So then Google has excuse to arbitrarily take you down whenever they want to do a big puppeteer push.

And you know v4 will come in a few years with whatever hole you squeezed through plugged up. Arbitrary communication will be disabled, maybe eval even. No thanks. I consider my time too valuable to play that game. I'd recommend moving to v4 which is on Electron and not confined to a permission model that also conveniently works as a business lever.