Fix editor tests on Windows #15623

calixteman · 2022-10-25T13:36:02Z

In [Editor] Use the global clipboard for the copy/paste/cut operations #15373, we implemented copy/paste actions in using the system clipboard.
For any reasons, on Windows, the clipboard doesn't contain the expected data when the tests are ran in parallel, hence the tests which are using the clipboard need to be ran sequentially.
Make sure that we can paste after having copied.

Snuffleupagus

r=me, with one suggestion and with passing tests; thank you!

test/integration/test_utils.js

Snuffleupagus · 2022-10-25T14:11:59Z

/botio integrationtest

pdfjsbot · 2022-10-25T14:12:00Z

From: Bot.io (Linux m4)

Received

Command cmd_integrationtest from @Snuffleupagus received. Current queue size: 0

Live output at: http://54.241.84.105:8877/37c73cedab0c89f/output.txt

pdfjsbot · 2022-10-25T14:12:01Z

From: Bot.io (Windows)

Received

Command cmd_integrationtest from @Snuffleupagus received. Current queue size: 0

Live output at: http://54.193.163.58:8877/507f222e846f35e/output.txt

pdfjsbot · 2022-10-25T14:16:10Z

From: Bot.io (Linux m4)

Success

Full output at http://54.241.84.105:8877/37c73cedab0c89f/output.txt

Total script time: 4.15 mins

Integration Tests: Passed

pdfjsbot · 2022-10-25T14:22:21Z

From: Bot.io (Windows)

Failed

Full output at http://54.193.163.58:8877/507f222e846f35e/output.txt

Total script time: 10.32 mins

Integration Tests: FAILED

Snuffleupagus · 2022-10-25T16:18:23Z

/botio integrationtest

pdfjsbot · 2022-10-25T16:18:25Z

From: Bot.io (Linux m4)

Received

Command cmd_integrationtest from @Snuffleupagus received. Current queue size: 0

Live output at: http://54.241.84.105:8877/ac681056814812b/output.txt

pdfjsbot · 2022-10-25T16:18:25Z

From: Bot.io (Windows)

Received

Command cmd_integrationtest from @Snuffleupagus received. Current queue size: 0

Live output at: http://54.193.163.58:8877/f001990448daa2b/output.txt

pdfjsbot · 2022-10-25T16:22:31Z

From: Bot.io (Linux m4)

Success

Full output at http://54.241.84.105:8877/ac681056814812b/output.txt

Total script time: 4.08 mins

Integration Tests: Passed

pdfjsbot · 2022-10-25T16:28:45Z

From: Bot.io (Windows)

Failed

Full output at http://54.193.163.58:8877/f001990448daa2b/output.txt

Total script time: 10.31 mins

Integration Tests: FAILED

Snuffleupagus · 2022-10-25T16:32:05Z

/botio integrationtest

pdfjsbot · 2022-10-25T16:32:07Z

From: Bot.io (Linux m4)

Received

Command cmd_integrationtest from @Snuffleupagus received. Current queue size: 0

Live output at: http://54.241.84.105:8877/c97d98a4c72ba7c/output.txt

pdfjsbot · 2022-10-25T16:32:07Z

From: Bot.io (Windows)

Received

Command cmd_integrationtest from @Snuffleupagus received. Current queue size: 0

Live output at: http://54.193.163.58:8877/f8a3f1525870720/output.txt

pdfjsbot · 2022-10-25T16:36:19Z

From: Bot.io (Linux m4)

Success

Full output at http://54.241.84.105:8877/c97d98a4c72ba7c/output.txt

Total script time: 4.18 mins

Integration Tests: Passed

pdfjsbot · 2022-10-25T16:41:50Z

From: Bot.io (Windows)

Failed

Full output at http://54.193.163.58:8877/f8a3f1525870720/output.txt

Total script time: 9.70 mins

Integration Tests: FAILED

calixteman · 2022-10-25T20:02:12Z

/botio integrationtest

pdfjsbot · 2022-10-25T20:02:14Z

From: Bot.io (Linux m4)

Received

Command cmd_integrationtest from @calixteman received. Current queue size: 0

Live output at: http://54.241.84.105:8877/ee5d0f1d8c23f0e/output.txt

pdfjsbot · 2022-10-25T20:02:14Z

From: Bot.io (Windows)

Received

Command cmd_integrationtest from @calixteman received. Current queue size: 0

Live output at: http://54.193.163.58:8877/0d23122b5ae2a94/output.txt

pdfjsbot · 2022-10-25T20:06:51Z

From: Bot.io (Linux m4)

Failed

Full output at http://54.241.84.105:8877/ee5d0f1d8c23f0e/output.txt

Total script time: 4.60 mins

Integration Tests: FAILED

pdfjsbot · 2022-10-25T20:11:26Z

From: Bot.io (Windows)

Failed

Full output at http://54.193.163.58:8877/0d23122b5ae2a94/output.txt

Total script time: 9.17 mins

Integration Tests: FAILED

calixteman · 2022-10-25T20:13:37Z

/botio integrationtest

pdfjsbot · 2022-10-25T20:13:39Z

From: Bot.io (Linux m4)

Received

Command cmd_integrationtest from @calixteman received. Current queue size: 0

Live output at: http://54.241.84.105:8877/b04e870ef4ec9d2/output.txt

pdfjsbot · 2022-10-25T20:13:39Z

From: Bot.io (Windows)

Received

Command cmd_integrationtest from @calixteman received. Current queue size: 0

Live output at: http://54.193.163.58:8877/d801c0b146ada4c/output.txt

pdfjsbot · 2022-10-25T20:17:51Z

From: Bot.io (Linux m4)

Success

Full output at http://54.241.84.105:8877/b04e870ef4ec9d2/output.txt

Total script time: 4.18 mins

Integration Tests: Passed

pdfjsbot · 2022-10-25T20:23:27Z

From: Bot.io (Windows)

Failed

Full output at http://54.193.163.58:8877/d801c0b146ada4c/output.txt

Total script time: 9.78 mins

Integration Tests: FAILED

- In mozilla#15373, we implemented copy/paste actions in using the system clipboard. For any reasons, on Windows, the clipboard doesn't contain the expected data when the tests are ran in parallel, hence the tests which are using the clipboard need to be ran sequentially. - Make sure that we can paste after having copied.

calixteman · 2022-10-25T20:48:21Z

/botio integrationtest

pdfjsbot · 2022-10-25T20:48:23Z

From: Bot.io (Windows)

Received

Command cmd_integrationtest from @calixteman received. Current queue size: 0

Live output at: http://54.193.163.58:8877/39d95697a0f7dd2/output.txt

pdfjsbot · 2022-10-25T20:48:23Z

From: Bot.io (Linux m4)

Received

Command cmd_integrationtest from @calixteman received. Current queue size: 0

Live output at: http://54.241.84.105:8877/42c4d12faaeba01/output.txt

pdfjsbot · 2022-10-25T20:52:31Z

From: Bot.io (Linux m4)

Success

Full output at http://54.241.84.105:8877/42c4d12faaeba01/output.txt

Total script time: 4.13 mins

Integration Tests: Passed

pdfjsbot · 2022-10-25T20:58:19Z

From: Bot.io (Windows)

Failed

Full output at http://54.193.163.58:8877/39d95697a0f7dd2/output.txt

Total script time: 9.92 mins

Integration Tests: FAILED

calixteman · 2022-10-26T09:08:17Z

/botio integrationtest

pdfjsbot · 2022-10-26T09:08:18Z

From: Bot.io (Windows)

Received

Command cmd_integrationtest from @calixteman received. Current queue size: 0

Live output at: http://54.193.163.58:8877/51a7843d890ef84/output.txt

pdfjsbot · 2022-10-26T09:08:18Z

From: Bot.io (Linux m4)

Received

Command cmd_integrationtest from @calixteman received. Current queue size: 0

Live output at: http://54.241.84.105:8877/54883c709b242e1/output.txt

pdfjsbot · 2022-10-26T09:12:24Z

From: Bot.io (Linux m4)

Success

Full output at http://54.241.84.105:8877/54883c709b242e1/output.txt

Total script time: 4.09 mins

Integration Tests: Passed

pdfjsbot · 2022-10-26T09:18:10Z

From: Bot.io (Windows)

Failed

Full output at http://54.193.163.58:8877/51a7843d890ef84/output.txt

Total script time: 9.85 mins

Integration Tests: FAILED

calixteman · 2022-10-26T09:21:59Z

I think windows tests are in a better shape with this patch.
We still have this issue with PageOpen and an intermittent with zip codes, but at least it should fix the editing ones.
I'm not super happy to add these waitForTimeout but I don't see any other way to do.
@Snuffleupagus, are you still ok with this patch (I made few changes after your approval) ?

Snuffleupagus · 2022-10-26T09:29:49Z

I think windows tests are in a better shape with this patch.

Agreed, it seems like it helped on the bots. (It's weird that these problems don't show up locally...)

We still have this issue with PageOpen

Do you have time to continue investigating this in #15308?

I'm not super happy to add these waitForTimeout but I don't see any other way to do.

Agreed, but since this is only in tests I suppose it's not that bad overall.

are you still ok with this patch (I made few changes after your approval) ?

Sure (I looked at the changes and it seems fine).

calixteman · 2022-10-26T09:34:06Z

We still have this issue with PageOpen

Do you have time to continue investigating this in #15308?

I'll try to find some time this week or the week after.

calixteman added the editor label Oct 25, 2022

calixteman requested a review from Snuffleupagus October 25, 2022 13:36

Snuffleupagus added the test label Oct 25, 2022

Snuffleupagus approved these changes Oct 25, 2022

View reviewed changes

test/integration/test_utils.js Outdated Show resolved Hide resolved

calixteman force-pushed the editor_int_test branch from 24e266b to 8d7aaa7 Compare October 25, 2022 13:52

calixteman force-pushed the editor_int_test branch from 8d7aaa7 to e8c2165 Compare October 25, 2022 20:01

calixteman force-pushed the editor_int_test branch from e8c2165 to 2384fbc Compare October 25, 2022 20:48

calixteman merged commit 9a33a0f into mozilla:master Oct 26, 2022

calixteman deleted the editor_int_test branch October 26, 2022 09:31

Fix editor tests on Windows #15623

Fix editor tests on Windows #15623

Conversation

calixteman commented Oct 25, 2022

Snuffleupagus left a comment

Choose a reason for hiding this comment

Snuffleupagus commented Oct 25, 2022

pdfjsbot commented Oct 25, 2022

From: Bot.io (Linux m4)

Received

pdfjsbot commented Oct 25, 2022

From: Bot.io (Windows)

Received

pdfjsbot commented Oct 25, 2022

From: Bot.io (Linux m4)

Success

pdfjsbot commented Oct 25, 2022

From: Bot.io (Windows)

Failed

Snuffleupagus commented Oct 25, 2022

pdfjsbot commented Oct 25, 2022

From: Bot.io (Linux m4)

Received

pdfjsbot commented Oct 25, 2022

From: Bot.io (Windows)

Received

pdfjsbot commented Oct 25, 2022

From: Bot.io (Linux m4)

Success

pdfjsbot commented Oct 25, 2022

From: Bot.io (Windows)

Failed

Snuffleupagus commented Oct 25, 2022

pdfjsbot commented Oct 25, 2022

From: Bot.io (Linux m4)

Received

pdfjsbot commented Oct 25, 2022

From: Bot.io (Windows)

Received

pdfjsbot commented Oct 25, 2022

From: Bot.io (Linux m4)

Success

pdfjsbot commented Oct 25, 2022

From: Bot.io (Windows)

Failed

calixteman commented Oct 25, 2022

pdfjsbot commented Oct 25, 2022

From: Bot.io (Linux m4)

Received

pdfjsbot commented Oct 25, 2022

From: Bot.io (Windows)

Received

pdfjsbot commented Oct 25, 2022

From: Bot.io (Linux m4)

Failed

pdfjsbot commented Oct 25, 2022

From: Bot.io (Windows)

Failed

calixteman commented Oct 25, 2022

pdfjsbot commented Oct 25, 2022

From: Bot.io (Linux m4)

Received

pdfjsbot commented Oct 25, 2022

From: Bot.io (Windows)

Received

pdfjsbot commented Oct 25, 2022

From: Bot.io (Linux m4)

Success

pdfjsbot commented Oct 25, 2022

From: Bot.io (Windows)

Failed

calixteman commented Oct 25, 2022

pdfjsbot commented Oct 25, 2022

From: Bot.io (Windows)

Received

pdfjsbot commented Oct 25, 2022

From: Bot.io (Linux m4)

Received

pdfjsbot commented Oct 25, 2022

From: Bot.io (Linux m4)