Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testing: Resolve remaining failures of Selenium tests #682

Closed
bschmalhofer opened this issue Dec 27, 2020 · 25 comments
Closed

Testing: Resolve remaining failures of Selenium tests #682

bschmalhofer opened this issue Dec 27, 2020 · 25 comments
Assignees
Labels
bug Something isn't working as intended selenium Related to fixing or improving testing with Selenium
Milestone

Comments

@bschmalhofer
Copy link
Contributor

bschmalhofer commented Dec 27, 2020

Currently there are 103 failures from running the Selenium tests under Docker. These should be resolved. When fixing the test is too much effort, then skipping the test is fine too,

See also #175, #681, #678, #907, #909, #921, #929

@bschmalhofer bschmalhofer added the bug Something isn't working as intended label Dec 27, 2020
bschmalhofer added a commit that referenced this issue Dec 28, 2020
These scripts were failing because Kernel/System/OTOBOCommunity does not exist
bschmalhofer added a commit that referenced this issue Dec 28, 2020
Issue #682: remove OTOBOCommunity specific checks
bschmalhofer added a commit to RotherOSS/otobo-docker that referenced this issue Dec 28, 2020
Because the sample files are needed where the browser runs.
bschmalhofer added a commit to RotherOSS/otobo-docker that referenced this issue Dec 28, 2020
@bschmalhofer
Copy link
Contributor Author

Some fixes:

  • mount /opt/otobo for the sample files
  • Don't try to load Kernel::System::OTOBOCommunity

Current status: 89 test scripts failing

@wollmers
Copy link
Contributor

wollmers commented Dec 28, 2020

Selenium tests of installed packages, e. g. FAQ, are not skipped and need adaption.

root@ubuntu:/opt/otobo-docker# grep -i 'selenium' prove_10.0.x_2020-12-28-101043.out
[...]

Can't call method "Get" on an undefined value at /opt/otobo/scripts/test/Selenium/Agent/AgentFAQAdd.t line 25.
[11:03:15] /opt/otobo/scripts/test/Selenium/Agent/AgentFAQAdd.t ........................................................ 
Can't call method "Get" on an undefined value at /opt/otobo/scripts/test/Selenium/Agent/AgentFAQCategory.t line 24.
[11:03:15] /opt/otobo/scripts/test/Selenium/Agent/AgentFAQCategory.t ................................................... 
Can't call method "Get" on an undefined value at /opt/otobo/scripts/test/Selenium/Agent/AgentFAQDelete.t line 24.

[...]

@bschmalhofer
Copy link
Contributor Author

@bschmalhofer
Copy link
Contributor Author

bschmalhofer commented Dec 30, 2020

There is still a problem with the mounted volume /opt/otobo in the container otobo_selenium-chrome_1 . The sample files are in /opt/otobo/scripts/test/sample , but they are not readable because of file permissions.

seluser@709d1d3578b8:/$ ls -l /opt/otobo/scripts/test/sample/NotificationEvent/Export_Notification_Appointment_reminder_notification.yml
-rw-rw---- 1 1000 1000 8418 Sep 25 15:15 /opt/otobo/scripts/test/sample/NotificationEvent/Export_Notification_Appointment_reminder_notification.yml
seluser@709d1d3578b8:/$ id
uid=1200(seluser) gid=1201(seluser) groups=1201(seluser),27(sudo)

The following assumes that all required sample files are in scripts/test/sample and that the browser needs no other files from /opt/otobo . Then a workaround could be to:

  • create a dir /opt/otobo/scripts/test in the running container
  • 'docker cp' the dir sample into the created dir

These actions would need to be redone when restarting the container.

An alternative would be to create yet another OTOBO specific Docker image that already includes /opt/otobo/scripts/test/sample.

bschmalhofer added a commit to RotherOSS/otobo-docker that referenced this issue Dec 31, 2020
bschmalhofer added a commit to RotherOSS/otobo-docker that referenced this issue Dec 31, 2020
…r_cp

Issue RotherOSS/otobo#682: copy only scripts/test/sample into the sel…
bschmalhofer added a commit that referenced this issue Dec 31, 2020
bschmalhofer added a commit that referenced this issue Jan 1, 2021
@bschmalhofer
Copy link
Contributor Author

Current status: 64 Selenium test scripts failing

@bschmalhofer
Copy link
Contributor Author

Fixed login to the customer interface: 56 scripts still failing

bschmalhofer added a commit that referenced this issue Apr 26, 2021
just because the following command ran into a time out
bschmalhofer added a commit that referenced this issue Apr 26, 2021
bschmalhofer added a commit that referenced this issue Apr 26, 2021
Forgot to add the call to todo().
bschmalhofer added a commit that referenced this issue Apr 26, 2021
Login() should pass when one of the 5 login attempts succeed.
This means that the tests in the look must be declared as TODO.
bschmalhofer added a commit that referenced this issue Apr 26, 2021
bschmalhofer added a commit that referenced this issue Apr 27, 2021
bschmalhofer added a commit that referenced this issue Apr 27, 2021
reason of sporadic failures is not obvious
bschmalhofer added a commit that referenced this issue Apr 27, 2021
As otherwise the whole subtest fails, and an exception is thrown.
bschmalhofer added a commit that referenced this issue Apr 27, 2021
bschmalhofer added a commit that referenced this issue Apr 28, 2021
Because it has sporadic failures.
bschmalhofer added a commit that referenced this issue Apr 28, 2021
bschmalhofer added a commit that referenced this issue Apr 29, 2021
Because of sporadic failures.
bschmalhofer added a commit that referenced this issue Apr 29, 2021
bschmalhofer added a commit that referenced this issue Apr 29, 2021
Because of sporadic failures.
bschmalhofer added a commit that referenced this issue Apr 29, 2021
bschmalhofer added a commit that referenced this issue Apr 30, 2021
Because of sporadic failures.
Give up on sleeping.
bschmalhofer added a commit that referenced this issue Apr 30, 2021
@bschmalhofer
Copy link
Contributor Author

Let's close this epic issue. Three consecutive runs reported no failures. This does not mean that the Selenium test are all passing, just that the, maybe sporadically, failing tests are marked as TODO, and have an issue assigned.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working as intended selenium Related to fixing or improving testing with Selenium
Projects
None yet
Development

No branches or pull requests

2 participants