Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: use @web/test-runner instead of karma #1816

Merged
merged 18 commits into from
Mar 8, 2024

Conversation

SimenB
Copy link
Contributor

@SimenB SimenB commented Nov 20, 2023

Which problem is this PR solving?

  • Attempting to unblock feat: bring back browser extension #1152 - i.e. bringing back the browser extension
  • Note that this just affects @opentelemetry/instrumentation-document-load for now - I wanna get some feedback from the team that that this is something you want before spending more time on it 🙂

Short description of the changes

  • As mentioned in feat: bring back browser extension #1152 (comment), karma is deprecated, and they recommend using Jest or @web/test-runner as a replacement. Seeing as I guess these theses should run in the browser, I elected to go with @web/test-runner
  • Under the hood wtr already uses mocha, so that migration is free. However, we also need to replace webpack with rollup/esbuild - that requires some more changes. I was unable to get assert to work, so I replaced it with chai.
  • Coverage is collected, but I'm unsure if CI will correctly pick it up (EDIT: See later comments - the coverage is picked up, but something's weird with the merged report on codecov)

@SimenB SimenB requested a review from a team November 20, 2023 08:51
'http://localhost:9876/context.html'
assert.isOk(
(fetchSpan.attributes['http.url'] as string).startsWith(
'http://localhost:8000/?wtr-session-id='
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

URL is different, and the session ID unique per run. I think this is fine, but happy to change to whatever you want 🙂

Comment on lines +38 to +39
// @ts-expect-error: not an export, but we want the prebundled version
import chai from 'chai/chai.js';
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ideally we'd just get assert to work in the browser, but for some reason it gives an error

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We used the webpack node polyfills for assert (see

assert: true,
// The assert polyfill from github.com/browserify/commonjs-assert
// also requires the `global` polyfill.
global: true,
).

IMO not using node polyfills for tests that are only intended for the browser is a good idea - I'd be fine for this to switch to chai, would like to get an opinion on it by @martinkuba or @pkanal first though as they're the subject matter experts 🙂

Copy link

codecov bot commented Nov 20, 2023

Codecov Report

Merging #1816 (0b71795) into main (dfb2dff) will decrease coverage by 0.13%.
Report is 3 commits behind head on main.
The diff coverage is n/a.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1816      +/-   ##
==========================================
- Coverage   90.97%   90.85%   -0.13%     
==========================================
  Files         146      143       -3     
  Lines        7492     7368     -124     
  Branches     1502     1468      -34     
==========================================
- Hits         6816     6694     -122     
+ Misses        676      674       -2     

see 4 files with indirect coverage changes

@SimenB
Copy link
Contributor Author

SimenB commented Nov 20, 2023

Hmm, we need to upgrade the node version used to run the browser tests. Not sure how to best do that in a way that still works with the karma ones? or maybe karma works on latest release

@SimenB
Copy link
Contributor Author

SimenB commented Nov 20, 2023

OK, using Node 16 works with both old webpack and the WTR runner 👍

@SimenB
Copy link
Contributor Author

SimenB commented Nov 20, 2023

Hmm, unsure why code coverage was lost - the output file was located: https://github.com/open-telemetry/opentelemetry-js-contrib/actions/runs/6928406995/job/18844148357?pr=1816#step:8:79

Pushed an attempt to also produce json output - let's see 😀

@SimenB
Copy link
Contributor Author

SimenB commented Nov 20, 2023

Wait, is it correct? Codecov UI is confusing me 😅

image

But it is covered if I open coverage/lcov-report/index.html

image

No clue 😅

@pichlermarc
Copy link
Member

pichlermarc commented Nov 20, 2023

Wait, is it correct? Codecov UI is confusing me 😅

Looks like these are the indirect changes, as far as I understand (codecov is confusing me too 😅) that's changes in coverage to files that have themselves not changed in the PR.

This link should be the one to look at (when Unit Tests / browser-test (16) (pull_request) check is green, the latest reports are uploaded and it'll accurately reflect what's there) - if instrumentation.ts is in src/ and has some coverage reported for it then it's working. 🙂

That being said, the codecov upload in this repo is sometimes flaky (we (the repo maintainers) may have to add a token for it to work properly, though no idea why other runs work fine without it, EDIT: it may be related to this). In the current run it did not upload because of an error, but that error is not related to your PR. The file is actually here, https://github.com/open-telemetry/opentelemetry-js-contrib/actions/runs/6929736595/job/18848043256?pr=1816#step:8:69 so coverage seems to be working fine 🙂 - it's just codecov that's not uploading it.

@SimenB
Copy link
Contributor Author

SimenB commented Nov 20, 2023

At least running npm run test:browser && open coverage/lcov-report/index.html locally shows correct coverage information, so I think the test runner changes are correct. Getting codecov to agree shouldn't be that hard afterwards if it's some sort of permission/from fork weirdness that makes the codecov app report other coverage data.

@dyladan
Copy link
Member

dyladan commented Nov 20, 2023

Under the hood wtr already uses mocha, so that migration is free. However, we also need to replace webpack with rollup/esbuild - that requires some more changes. I was unable to get assert to work, so I replaced it with chai.

Removing webpack is something we've wanted to do anyway. In the core repo we've been working on updating tooling that was held back by the necessity to support old node versions which are being dropped soon.

Coverage is collected, but I'm unsure if CI will correctly pick it up (EDIT: See later comments - the coverage is picked up, but something's weird with the merged report on codecov)

Coverage has been in a weird state in this repo for a while. It is likely not your fault.

@SimenB
Copy link
Contributor Author

SimenB commented Nov 20, 2023

Cool! In that case, should I remove karma from the other packages as well so we don't have to run both systems?

This PR can also be merged on its own, and then I can follow up for the other packages. Whatever's most convenient to you 👍

Copy link
Member

@svrnm svrnm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will unblock the browser extension:-) , thank you @SimenB

@pichlermarc
Copy link
Member

Cool! In that case, should I remove karma from the other packages as well so we don't have to run both systems?

This PR can also be merged on its own, and then I can follow up for the other packages. Whatever's most convenient to you 👍

I think having follow ups to adapt the other instrumentations would be appreciated (once this PR has merged). We'd like to keep tooling in sync as much as possible. 🙂

@SimenB
Copy link
Contributor Author

SimenB commented Nov 21, 2023

Yeah, happy to do follow-ups - should be quick. I was more asking if you want one big bang in a single PR or split up.

@pichlermarc
Copy link
Member

pichlermarc commented Nov 21, 2023

Yeah, happy to do follow-ups - should be quick. I was more asking if you want one big bang in a single PR or split up.

Amazing, thx! I'd prefer having it split up :)

@SimenB
Copy link
Contributor Author

SimenB commented Nov 21, 2023

Sweet 👍 will do once this is merged

@pichlermarc
Copy link
Member

cc @martinkuba @pkanal (component-owners) - any objections? 🙂
If not then I'd merge this in. 🙂

Copy link
Member

@pichlermarc pichlermarc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Brought this up in the SIG Meeting this week and there were no objections to switching. 🙂

@pichlermarc
Copy link
Member

@SimenB would you mind giving that PR an update? 🙂 I had been planning to merge it in a while ago but due to conflicts merging was blocked.

@SimenB
Copy link
Contributor Author

SimenB commented Mar 8, 2024

image

hopefully CI is happy 🙂

@SimenB
Copy link
Contributor Author

SimenB commented Mar 8, 2024

looks like it 👍

@pichlermarc
Copy link
Member

awesome, thanks 🙂

@pichlermarc pichlermarc merged commit 19ebbaf into open-telemetry:main Mar 8, 2024
17 checks passed
@SimenB SimenB deleted the replace-karma branch March 8, 2024 14:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants