Problems with tab_restart_browser

[turban1988]

Hi

I found another problem with the tab_restart_browser function. The problem seems to be that spawning a new window fails due to the same origin policy. Sadly the problem does not occur deterministically. But identified websites on which the problem sometimes occurs (1) https://jalopnik.com/tag/ford and (2) https://jalopnik.com/volkswagen-thieves-almost-pull-off-the-perfect-train-he-1833936849#replies

This is the error I get

2019-04-10 16:42:09,575 - BrowserManager       - INFO     - BROWSER 1: Crash in driver, restarting browser manager 
 Traceback (most recent call last):
  File "/home/openwpm/new_try_ext_neu/OpenWPM/automation/BrowserManager.py", line 407, in BrowserManager
    browser_params, manager_params, extension_socket)
  File "/home/openwpm/new_try_ext_neu/OpenWPM/automation/Commands/command_executor.py", line 21, in execute_command
    extension_socket=extension_socket)
  File "/home/openwpm/new_try_ext_neu/OpenWPM/automation/Commands/browser_commands.py", line 156, in get_website
    tab_restart_browser(webdriver)
  File "/home/openwpm/new_try_ext_neu/OpenWPM/automation/Commands/browser_commands.py", line 138, in tab_restart_browser
    webdriver.execute_script("window.open('')")
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 543, in execute_script
    'args': converted_args})['value']
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 308, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
    raise exception_class(message, screen, stacktrace)
WebDriverException: Message: Error: Permission denied to access property "_wrapped"

[turban1988]

A try-catch clause does solve the problem for me.

    try:
        webdriver.execute_script("window.open('')")
    except WebDriverException as e:
        pass

This also resolves #279 .

However, the code migth fail at the assert statment. But that did not happen in my minimalistic tests.
https://github.com/mozilla/OpenWPM/blob/e095344a1343cecd31be126941278efd0dae618c/automation/Commands/browser_commands.py#L106
Also, I am not quite aware of other consequenes that might occure if no new window/tab is spawned but the old one is used.

[englehardt]

Thanks! I bet this website is overwriting or wrapping window.open, which can lead to a permission error.

I know we initially changed to this approach because of issues sending keyboard events to Firefox chrome after the upgrade to Selenium 3. I think it's worth revisiting those events to see if we're able to more reliably create and close windows.

[valkyr13]

@englehardt can i work on this?

[englehardt]

@valkyr13 Yes. I suggest first trying what we used to do before upgrading to Selenium 3. I wonder whether some of the unreliability has since been fixed.

[vringar]

@valkyr13 are you still on this?

[ankushduacodes]

@vringar May I work on this?

[vringar]

Go ahead!

[ankushduacodes]

@vringar I am sorry for the delays in the response to this issue. I was quite busy for the past two weeks and could not look into it much. I will be actively working on this issue and #758 from now on.

Thank you for your patience! :)

[vringar]

Once again, no worries! :)
Welcome back, I look forward to hearing from you again.

[ankushduacodes]

@vringar As outlined in this comment by Steven, I tried to check if key combinations would work.
Here's what I have tried so far to debug this issue(macOS specifics with geckodriver):

Serial no.	Key combinations	Status
1	CMD + c	--Works
2	CMD + v	--Works
3	CMD + t	--Doesn't Work
4	CMD + w	--Doesn't Work
5	CMD + SHFT + j	--Doesn't Work
6	CMD + h	--Doesn't Work
7	CMD + SHFT + h	--Doesn't Work

---

Here's the code I was trying (using conda activate openwpm):

import time
from selenium import webdriver
from selenium.webdriver import ActionChains
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.firefox.webdriver import WebDriver


def main():
    driver: WebDriver = webdriver.Firefox(executable_path='/Users/ankushdua/Downloads/geckodriver')
    driver.get('http://www.mozilla.org')
    action = ActionChains(driver)
    action.key_down(Keys.COMMAND).send_keys('t').key_up(Keys.COMMAND).perform()
    time.sleep(4)
    driver.close()


if __name__ == '__main__':
    main()

---

What I understood:

The shortcuts/key combinations that try to manipulate the tab or whole browser window do not seem to work at all (I could not get any of the combinations from 3 to 7 did not work at all).
I tried the same in Linux(Ubuntu) and got the same results.

Please let me know what could be the next step.

---

Update

As outlined in this StackOverflow answer, Selenium4 provides a new method to SwitchTo class:

https://github.com/SeleniumHQ/selenium/blob/941dc9c6b2e2aa4f701c1b72be8de03d4b7e996a/py/selenium/webdriver/remote/switch_to.py#L99-L111

Here is the usage to that method(from new docs):

# Opens a new tab and switches to new tab
driver.switch_to.new_window('tab')

# Opens a new window and switches to new window
driver.switch_to.new_window('window')

Now, Selenium4 is still in the development stage and has not released the new_window feature yet (Confirmed this twice by downloading the latest release source code). Do you want to wait for Selenium4 to be release before we tackle this issue or Do you have an alternative strategy for now?

[vringar]

Thanks for the detailed writeup on this!
Can you think of another way to open an about:blank tab and then closing all other windows and tabs?

[ankushduacodes]

I am looking into it. I will update you soon.

[ankushduacodes]

@vringar I was looking into emulating keypresses using javascript but could not make it happen :(
Unfortunately, At this point, I am not aware of any other ways to open a new tab/window. I think for now window.open() is our best bet unless you have something in mind.

[vringar]

Then we'll just leave this open until selenium releases a version with driver.switch_to._new_window()
Thanks again for all of your research!

[ankushduacodes]

@vringar While going through some extension APIs over at Bugzilla, I realized that it may be possible to load an extra extension that will solely be responsible for spawning a new browser using the windows API for extensions.

What do you think?

[vringar]

I think we can wait this out and not introduce any more code on our side. Thanks for the suggestion though!