Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: rework flaky replay test #2543

Merged
merged 1 commit into from
Jan 2, 2025
Merged

ci: rework flaky replay test #2543

merged 1 commit into from
Jan 2, 2025

Conversation

vaind
Copy link
Collaborator

@vaind vaind commented Dec 30, 2024

📜 Description

💡 Motivation and Context

💚 How did you test it?

📝 Checklist

  • I reviewed submitted code
  • I added tests to verify changes
  • No new PII added or SDK only sends newly added PII if sendDefaultPii is enabled
  • I updated the docs if needed
  • All tests passing
  • No breaking changes

🔮 Next steps

Copy link

codecov bot commented Dec 30, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.99%. Comparing base (f735167) to head (eb870bb).
Report is 4 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2543      +/-   ##
==========================================
+ Coverage   87.15%   91.99%   +4.84%     
==========================================
  Files         266       89     -177     
  Lines        9432     3087    -6345     
==========================================
- Hits         8220     2840    -5380     
+ Misses       1212      247     -965     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@vaind vaind changed the title ci: rework flakyreplay test ci: rework flaky replay test Dec 30, 2024
Copy link
Contributor

Android Performance metrics 🚀

  Plain With Sentry Diff
Startup time 391.82 ms 416.14 ms 24.33 ms
Size 6.46 MiB 7.48 MiB 1.01 MiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
43abc4f 406.54 ms 475.53 ms 68.99 ms
559d28f 302.35 ms 339.53 ms 37.18 ms
7f75f32 347.36 ms 419.58 ms 72.22 ms
e0f6628 380.21 ms 450.39 ms 70.17 ms
d5696bf 424.06 ms 425.58 ms 1.52 ms
0764150 469.49 ms 541.86 ms 72.37 ms
be8cafe 405.40 ms 470.95 ms 65.56 ms
d089990 361.67 ms 442.50 ms 80.83 ms
2261c15 370.00 ms 455.88 ms 85.88 ms
eecbbca 324.37 ms 352.49 ms 28.12 ms

App size

Revision Plain With Sentry Diff
43abc4f 6.35 MiB 7.34 MiB 1007.72 KiB
559d28f 5.94 MiB 6.92 MiB 1001.70 KiB
7f75f32 6.26 MiB 7.20 MiB 959.18 KiB
e0f6628 6.33 MiB 7.29 MiB 987.23 KiB
d5696bf 6.49 MiB 7.55 MiB 1.06 MiB
0764150 6.46 MiB 7.48 MiB 1.01 MiB
be8cafe 6.34 MiB 7.29 MiB 969.12 KiB
d089990 6.34 MiB 7.28 MiB 967.79 KiB
2261c15 6.27 MiB 7.20 MiB 957.75 KiB
eecbbca 5.94 MiB 6.89 MiB 975.78 KiB

Previous results on branch: ci/flaky-replay-test

Startup times

Revision Plain With Sentry Diff
22db335 433.89 ms 529.94 ms 96.05 ms

App size

Revision Plain With Sentry Diff
22db335 6.46 MiB 7.48 MiB 1.01 MiB

@vaind vaind marked this pull request as ready for review December 31, 2024 10:17
Copy link
Contributor

iOS Performance metrics 🚀

  Plain With Sentry Diff
Startup time 1240.74 ms 1268.81 ms 28.07 ms
Size 8.42 MiB 9.86 MiB 1.44 MiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
4d75417 1240.57 ms 1261.76 ms 21.19 ms
24b6e60 1250.69 ms 1268.63 ms 17.94 ms
08a7b4f 1277.10 ms 1303.37 ms 26.27 ms
333903e 1251.15 ms 1270.21 ms 19.06 ms
bd37365 1236.22 ms 1243.27 ms 7.04 ms
6325c3b 1266.52 ms 1291.06 ms 24.54 ms
6a5a65d 1237.22 ms 1250.29 ms 13.07 ms
aed5947 1268.67 ms 1281.10 ms 12.43 ms
7954fb3 1247.20 ms 1272.15 ms 24.94 ms
ed2ae08 1222.10 ms 1226.57 ms 4.47 ms

App size

Revision Plain With Sentry Diff
4d75417 8.38 MiB 9.71 MiB 1.34 MiB
24b6e60 8.32 MiB 9.38 MiB 1.06 MiB
08a7b4f 8.16 MiB 9.16 MiB 1.01 MiB
333903e 8.10 MiB 9.16 MiB 1.06 MiB
bd37365 8.28 MiB 9.34 MiB 1.06 MiB
6325c3b 8.16 MiB 9.17 MiB 1.01 MiB
6a5a65d 8.34 MiB 9.65 MiB 1.31 MiB
aed5947 8.16 MiB 9.17 MiB 1.01 MiB
7954fb3 8.38 MiB 9.75 MiB 1.37 MiB
ed2ae08 8.28 MiB 9.34 MiB 1.06 MiB

Previous results on branch: ci/flaky-replay-test

Startup times

Revision Plain With Sentry Diff
22db335 1234.69 ms 1253.25 ms 18.56 ms

App size

Revision Plain With Sentry Diff
22db335 8.42 MiB 9.86 MiB 1.44 MiB

@vaind vaind merged commit 8de999e into main Jan 2, 2025
58 checks passed
@vaind vaind deleted the ci/flaky-replay-test branch January 2, 2025 12:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants