Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix v5 tests #2791

Merged
merged 5 commits into from
Jan 30, 2023
Merged

Fix v5 tests #2791

merged 5 commits into from
Jan 30, 2023

Conversation

krystofwoldrich
Copy link
Member

@krystofwoldrich krystofwoldrich commented Jan 27, 2023 •
edited
Loading

📢 Type of change

  • Bugfix
  • New feature
  • Enhancement
  • Refactoring

📜 Description

  • After merging the 4.14.0 changes which included the jest testing library bump. The jsdom test environment is missing.
  • RN Matrix E2E needed to fix iOS SDK version -> latest flag
  • iOS bundles are large (60 KiB more)

💡 Motivation and Context

Failing tests

💚 How did you test it?

📝 Checklist

  • I reviewed submitted code
  • I added tests to verify changes
  • No new PII added or SDK only sends newly added PII if sendDefaultPII is enabled
  • All tests passing
  • No breaking changes

🔮 Next steps

#skip-changelog

@krystofwoldrich
Fix missing jest jsdom
afc744f 
Touch events test needs update for the changed behavior
@krystofwoldrich
Fix missing react-native e2e tests build
c56a112
@krystofwoldrich krystofwoldrich changed the title Fix missing jest jsdom Fix v5 tests Jan 27, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Jan 27, 2023 •
edited
Loading

Android (legacy) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 363.33 ms 410.64 ms 47.31 ms
Size 17.73 MiB 20.03 MiB 2.29 MiB

Baseline results on branch: 5.0.0

Startup times

Revision Plain With Sentry Diff
b868b53+dirty 339.52 ms 369.84 ms 30.32 ms
d091239+dirty 425.61 ms 447.11 ms 21.50 ms

App size

Revision Plain With Sentry Diff
b868b53+dirty 17.73 MiB 19.95 MiB 2.22 MiB
d091239+dirty 17.73 MiB 19.95 MiB 2.22 MiB

@github-actions
Copy link
Contributor

github-actions bot commented Jan 27, 2023 •
edited
Loading

Android (new) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 331.04 ms 338.81 ms 7.77 ms
Size 7.15 MiB 8.07 MiB 948.94 KiB

Baseline results on branch: 5.0.0

Startup times

Revision Plain With Sentry Diff
b868b53+dirty 321.42 ms 342.74 ms 21.32 ms
d091239+dirty 340.16 ms 366.42 ms 26.26 ms

App size

Revision Plain With Sentry Diff
b868b53+dirty 7.15 MiB 8.00 MiB 874.27 KiB
d091239+dirty 7.15 MiB 8.00 MiB 874.38 KiB

@krystofwoldrich krystofwoldrich marked this pull request as ready for review January 27, 2023 19:16
@krystofwoldrich
Copy link
Member Author

This is ready for review.

@krystofwoldrich krystofwoldrich mentioned this pull request Jan 30, 2023
Merged
10 tasks
@github-actions
Copy link
Contributor

iOS (legacy) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 1234.08 ms 1235.48 ms 1.40 ms
Size 2.36 MiB 2.81 MiB 460.59 KiB

Baseline results on branch: 5.0.0

Startup times

Revision Plain With Sentry Diff
b868b53+dirty 1274.90 ms 1305.68 ms 30.78 ms
d091239+dirty 1251.49 ms 1276.75 ms 25.26 ms

App size

Revision Plain With Sentry Diff
b868b53+dirty 2.36 MiB 2.74 MiB 384.64 KiB
d091239+dirty 2.36 MiB 2.74 MiB 384.64 KiB

marandaneto
marandaneto reviewed Jan 30, 2023
View reviewed changes
test/perf/metrics-ios.yml
@@ -11,4 +11,4 @@ startupTimeTest:

binarySizeTest:
diffMin: 200 KiB
diffMax: 400 KiB
diffMax: 600 KiB
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason for the increase?

Copy link
Member Author

@krystofwoldrich krystofwoldrich Jan 30, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tracked it down to this JS SDK upgrade. It seems to make both android and iOS bundles larger, but iOS was closer to the edge before so it broke.

I'm comparing this as the last one where it succeeded #2695 (cd7054e) and this 30307a0 as first where it failed.

It's hard to notice because often the metrics fail because of the start times flakiness. Sometimes both start times and sizes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vaind would be fine for you? since you came up with the numbers

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Screenshot 2023-01-30 at 16 17 13

Here are the numbers before and after in one table for easier comparison.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@marandaneto @vaind Maybe we can solve the sizes in a new PR/Issue as the v4 with the JS SDK that likely caused this is out.

And this RP is blocking the new v5 alpha that I would like to push ASAP.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just going back to this I've missed that replay is part of browser now.

That clearly explains the difference.

https://github.com/getsentry/sentry-javascript/blob/d1934af091ce65ebd3ca7964d2f8e694fe2ef85a/CHANGELOG.md#L35

@krystofwoldrich krystofwoldrich merged commit 7f3249b into 5.0.0 Jan 30, 2023
@krystofwoldrich krystofwoldrich deleted the fix-missing-jest-jsdom branch January 30, 2023 18:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@marandaneto marandaneto marandaneto approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@krystofwoldrich @marandaneto