Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: add automated CI testing with —no-intl node #3015

Merged
merged 7 commits into from
Mar 31, 2024

Conversation

mweberxyz
Copy link
Contributor

@mweberxyz mweberxyz commented Mar 29, 2024

This relates to...

Rationale

This PR extends the Node CI workflow to run undici tests against Node 20 and 21 built with --without-intl flag, which would have prevented the regression fixed by #2999 before the commit even merged to undici/main.

The full test suite currently failing against main, so problematic tests have been fixed where possible, otherwise excluded from running in the w/o intl build. If possible, these problematic tests/functionality should be fixed to run in w/o intl builds and the tests added to test:javascript:withoutintl (or even remove it entirely and both the w/o intl and regular builds can use test:javascript).

Note that the initial run, and occasional runs thereafter, of the workflow will take a long time (around 1 hour for the clean Node 21 build) -- but subsequent runs will be much faster due to ccache caching.

Changes

  • Workflows:
    • add test-without-intl job to workflows/nodejs.yml
  • package.json:
    • add test:javascript:withoutintl script to run tests known good with an w/o intl node build
    • add test:eventsource:nobuild, test:fetch:nobuild scripts to pull test running logic out of test:eventsource and test:fetch because undici is built prior to running w/o intl tests
    • add test:wpt:withoutintl script to run wpt tests except those problematic for w/o intl, namely:
      • test/wpt/start-websockets.mjs, and
      • test/wpt/start-FileAPI.mjs
  • various test tweaks to keep existing tests running without disabling for w/o intl

Features

N/A

Bug Fixes

N/A

Breaking Changes and Deprecations

N/A

Status

@codecov-commenter
Copy link

codecov-commenter commented Mar 29, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93.51%. Comparing base (9278de9) to head (bd4273e).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #3015   +/-   ##
=======================================
  Coverage   93.51%   93.51%           
=======================================
  Files          89       89           
  Lines       24231    24234    +3     
=======================================
+ Hits        22659    22662    +3     
  Misses       1572     1572           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mweberxyz
Copy link
Contributor Author

mweberxyz commented Mar 29, 2024

I started looking into the actual test failures, the problems might not be all that arduous.

First, there's a failing test which is simply a misaligned console.table assertion.

Then, some tests run npm run build:node which fails, but maybe that doesn't maybe doesn't need to run in this configuration because the package can be built with a regular node?

I've gotten down into the test:wpt which is failing on steps that are clearly unicode related:
CleanShot 2024-03-28 at 21 56 56@2x

Will update with more findings and if I get the tests to fully pass.

@KhafraDev
Copy link
Member

that failure will be ignored once #2728 lands cc @ronag @mcollina

@mweberxyz
Copy link
Contributor Author

mweberxyz commented Mar 29, 2024

With most recent commits, w/o intl build tests are green, PR description has been updated, and ready for review.

As a test, I reintroduced the commit which caused nodejs CI to fail its w/o intl build, and the check failed as expected -- see mweberxyz#6

test/node-test/debug.js Outdated Show resolved Hide resolved
mweberxyz and others added 2 commits March 29, 2024 09:51
Co-authored-by: Carlos Fuentes <me@metcoder.dev>
.bar is a valid, though unpopular, TLD. should be safer to use a domain in .invalid as specified in proposed RFC6761.
"test:cookies": "borp -p \"test/cookie/*.js\"",
"test:node-fetch": "borp -p \"test/node-fetch/**/*.js\"",
"test:eventsource": "npm run build:node && borp --expose-gc -p \"test/eventsource/*.js\"",
"test:fetch": "npm run build:node && borp --expose-gc -p \"test/fetch/*.js\" && borp -p \"test/webidl/*.js\" && borp -p \"test/busboy/*.js\"",
"test:eventsource": "npm run build:node && npm run test:eventsource:nobuild",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is building bad?

Copy link
Contributor Author

@mweberxyz mweberxyz Mar 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Build fails with icu-less node build:

CleanShot 2024-03-29 at 14 25 05@2x

But we shouldn't need to support building undici with icu-less node, only running it.

This is why the workflow installs regular node then builds undici prior to switching to the icu-less build for running tests.

I tried to minimize changes to package.json, so as to make people not need to think about keeping the test:javascript:withoutintl script up-to-date with test strategy changes.

@mweberxyz
Copy link
Contributor Author

One other omission I realize I haven't mentioned: jest doesn't work with icu-less node:

CleanShot 2024-03-29 at 14 55 14@2x

@mweberxyz
Copy link
Contributor Author

@fraxken Does anything in here conflict with your auditing re: #3012? I tried to be more mindful of security implications this time around (ie- I reviewed the code for this newly called action and made sure to pin it), but would appreciate an extra set of eyes.

@fraxken
Copy link
Member

fraxken commented Mar 30, 2024

@mweberxyz seems ok to me 😉

Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollina
Copy link
Member

Could this be landed?

@mcollina mcollina merged commit 74bd88e into nodejs:main Mar 31, 2024
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants