Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

flaky: wasi.test-wasi #51822

Closed
marco-ippolito opened this issue Feb 21, 2024 · 4 comments · Fixed by #52538
Closed

flaky: wasi.test-wasi #51822

marco-ippolito opened this issue Feb 21, 2024 · 4 comments · Fixed by #52538
Labels
flaky-test Issues and PRs related to the tests with unstable failures on the CI. windows Issues and PRs related to the Windows platform.

Comments

@marco-ippolito
Copy link
Member

marco-ippolito commented Feb 21, 2024

Test

wasi/test-wasi

Platform

Windows

Console output

06:42:50     node:assert:126
06:42:50       throw new AssertionError(obj);
06:42:50       ^
06:42:50     
06:42:50     AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
06:42:50     
06:42:50     3221225477 !== 0
06:42:50     
06:42:50         at innerRunWASI (d:\workspace\node-test-binary-windows-js-suites\node\test\wasi\test-wasi.js:80:12)
06:42:50         at runWASI (d:\workspace\node-test-binary-windows-js-suites\node\test\wasi\test-wasi.js:86:5)
06:42:50         at Object.<anonymous> (d:\workspace\node-test-binary-windows-js-suites\node\test\wasi\test-wasi.js:111:3)
06:42:50         at Module._compile (node:internal/modules/cjs/loader:1368:14)
06:42:50         at Module._extensions..js (node:internal/modules/cjs/loader:1426:10)
06:42:50         at Module.load (node:internal/modules/cjs/loader:1205:32)
06:42:50         at Module._load (node:internal/modules/cjs/loader:1021:12)
06:42:50         at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:142:12)
06:42:50         at node:internal/main/run_main_module:28:49 {
06:42:50       generatedMessage: true,
06:42:50       code: 'ERR_ASSERTION',
06:42:50       actual: 3221225477,
06:42:50       expected: 0,
06:42:50       operator: 'strictEqual'
06:42:50     }
06:42:50     
06:42:50     Node.js v22.0.0-pre

Build links

Additional information

@nodejs/wasi

@marco-ippolito marco-ippolito added the flaky-test Issues and PRs related to the tests with unstable failures on the CI. label Feb 21, 2024
@github-actions github-actions bot added the windows Issues and PRs related to the Windows platform. label Feb 21, 2024
@marco-ippolito marco-ippolito changed the title falky: wasi/test-wasi is flaky falky: wasi.test-wasi Feb 21, 2024
@marco-ippolito marco-ippolito changed the title falky: wasi.test-wasi flaky: wasi.test-wasi Feb 21, 2024
@targos
Copy link
Member

targos commented Feb 21, 2024

3221225477 is 0xC000005 (Access Violation). I think this is equivalent to a segmentation fault so there's a real bug somewhere.

@nodejs/platform-windows.

@joyeecheung
Copy link
Member

joyeecheung commented Feb 22, 2024

I tried resuming Windows a few times lately and all of them failed because of this error on Windows + ARM - and it's almost constantly failing as far as I can tell from CI history. I think we should mark it as flaky for Windows + ARM for now. On a side note it would be easier to debug the test if it's split instead of putting all the test cases together in one file.

@joyeecheung
Copy link
Member

Maybe someone from @nodejs/wasi is up for splitting that test? It seems that could be rewritten with custom configuration in python instead, which would also help making the test run faster by running the test cases in parallel (currently it takes 3.7s to run on a beefy laptop, in the CI on Windows this takes ~15s)

nodejs-github-bot pushed a commit that referenced this issue Feb 22, 2024
It has failed 22 PRs from the latest 100 CI runs.

PR-URL: #51834
Refs: #51822
Refs: nodejs/reliability#790
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
@joyeecheung
Copy link
Member

#51836 shows that only the test case of poll() is flaking.

nodejs-github-bot pushed a commit that referenced this issue Feb 24, 2024
Move the child process code into a fixture and split the test
so that it can be run in parallel and it's easier to identify
where the failure is coming from. Also use the
spawnSyncAndExitWithoutError() utility so that the test shows
complete information on failure.

Instead of marking all the wasi tests as flaky, only mark the
wasi-poll one which is flaking in the CI now.

PR-URL: #51836
Refs: #51822
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
marco-ippolito pushed a commit that referenced this issue Feb 26, 2024
It has failed 22 PRs from the latest 100 CI runs.

PR-URL: #51834
Refs: #51822
Refs: nodejs/reliability#790
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
marco-ippolito pushed a commit that referenced this issue Feb 26, 2024
Move the child process code into a fixture and split the test
so that it can be run in parallel and it's easier to identify
where the failure is coming from. Also use the
spawnSyncAndExitWithoutError() utility so that the test shows
complete information on failure.

Instead of marking all the wasi tests as flaky, only mark the
wasi-poll one which is flaking in the CI now.

PR-URL: #51836
Refs: #51822
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
marco-ippolito pushed a commit that referenced this issue Feb 26, 2024
Move the child process code into a fixture and split the test
so that it can be run in parallel and it's easier to identify
where the failure is coming from. Also use the
spawnSyncAndExitWithoutError() utility so that the test shows
complete information on failure.

Instead of marking all the wasi tests as flaky, only mark the
wasi-poll one which is flaking in the CI now.

PR-URL: #51836
Refs: #51822
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
marco-ippolito pushed a commit that referenced this issue Feb 27, 2024
It has failed 22 PRs from the latest 100 CI runs.

PR-URL: #51834
Refs: #51822
Refs: nodejs/reliability#790
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
marco-ippolito pushed a commit that referenced this issue Feb 27, 2024
Move the child process code into a fixture and split the test
so that it can be run in parallel and it's easier to identify
where the failure is coming from. Also use the
spawnSyncAndExitWithoutError() utility so that the test shows
complete information on failure.

Instead of marking all the wasi tests as flaky, only mark the
wasi-poll one which is flaking in the CI now.

PR-URL: #51836
Refs: #51822
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
richardlau pushed a commit that referenced this issue Mar 5, 2024
It has failed 22 PRs from the latest 100 CI runs.

PR-URL: #51834
Refs: #51822
Refs: nodejs/reliability#790
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
richardlau pushed a commit that referenced this issue Mar 25, 2024
Move the child process code into a fixture and split the test
so that it can be run in parallel and it's easier to identify
where the failure is coming from. Also use the
spawnSyncAndExitWithoutError() utility so that the test shows
complete information on failure.

Instead of marking all the wasi tests as flaky, only mark the
wasi-poll one which is flaking in the CI now.

PR-URL: #51836
Refs: #51822
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
richardlau pushed a commit that referenced this issue Mar 25, 2024
Move the child process code into a fixture and split the test
so that it can be run in parallel and it's easier to identify
where the failure is coming from. Also use the
spawnSyncAndExitWithoutError() utility so that the test shows
complete information on failure.

Instead of marking all the wasi tests as flaky, only mark the
wasi-poll one which is flaking in the CI now.

PR-URL: #51836
Refs: #51822
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
rdw-msft pushed a commit to rdw-msft/node that referenced this issue Mar 26, 2024
It has failed 22 PRs from the latest 100 CI runs.

PR-URL: nodejs#51834
Refs: nodejs#51822
Refs: nodejs/reliability#790
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
rdw-msft pushed a commit to rdw-msft/node that referenced this issue Mar 26, 2024
Move the child process code into a fixture and split the test
so that it can be run in parallel and it's easier to identify
where the failure is coming from. Also use the
spawnSyncAndExitWithoutError() utility so that the test shows
complete information on failure.

Instead of marking all the wasi tests as flaky, only mark the
wasi-poll one which is flaking in the CI now.

PR-URL: nodejs#51836
Refs: nodejs#51822
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
nodejs-github-bot pushed a commit that referenced this issue Apr 17, 2024
PR-URL: #52538
Fixes: #51822
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Michael Dawson <midawson@redhat.com>
aduh95 pushed a commit that referenced this issue Apr 29, 2024
PR-URL: #52538
Fixes: #51822
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Michael Dawson <midawson@redhat.com>
marco-ippolito pushed a commit that referenced this issue May 2, 2024
PR-URL: #52538
Fixes: #51822
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Michael Dawson <midawson@redhat.com>
marco-ippolito pushed a commit that referenced this issue May 3, 2024
PR-URL: #52538
Fixes: #51822
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Michael Dawson <midawson@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
flaky-test Issues and PRs related to the tests with unstable failures on the CI. windows Issues and PRs related to the Windows platform.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants