Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RUM-6583: Fix flaky test for RUM CPU vital #2129

Merged

Conversation

simaoseica-dd
Copy link
Contributor

@simaoseica-dd simaoseica-dd commented Dec 4, 2024

What and why?

The test testWhenCPUUnderHeavyLoadItMeasuresHigherCPUTicks() compares the CPU ticks under heavy and light processing and it started being flaky in GitLab. The CPU ticks measured should be greater on heavy processing.

This PR mitigates the flakiness of the test.

How?

  • To avoid false positives, the blocks of processing are run more than once. This way we can have an average of measures that dilute the false positives.
  • The duration of both tasks are now the same, to avoid comparisons of CPU ticks under different time intervals.
  • It uses CFAbsoluteTimeGetCurrent instead of Date to get the time since it is more suitable for performance measurements providing high-precision timing.

Review checklist

  • Feature or bugfix MUST have appropriate tests (unit, integration)
  • Make sure each commit and the PR mention the Issue number or JIRA reference
  • Add CHANGELOG entry for user facing changes
  • Add Objective-C interface for public APIs (see our guidelines [internal]) and run make api-surface)

@simaoseica-dd simaoseica-dd self-assigned this Dec 4, 2024
@simaoseica-dd simaoseica-dd requested review from a team as code owners December 4, 2024 11:28
Copy link
Member

@mariedm mariedm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice improvement! 🙌 Thank you for taking care of this flakiness. ❄

@simaoseica-dd simaoseica-dd force-pushed the simaoseica/RUM-6583/fix-flaky-test-for-rum-cpu-vital branch from 38fc91e to acdf435 Compare December 10, 2024 10:54
@simaoseica-dd simaoseica-dd force-pushed the simaoseica/RUM-6583/fix-flaky-test-for-rum-cpu-vital branch 2 times, most recently from 7940f33 to 6f5d819 Compare December 10, 2024 11:31
@datadog-datadog-prod-us1
Copy link

Datadog Report

Branch report: simaoseica/RUM-6583/fix-flaky-test-for-rum-cpu-vital
Commit report: 6f5d819
Test service: dd-sdk-ios

✅ 0 Failed, 3571 Passed, 0 Skipped, 2m 37.06s Total Time
🔻 Test Sessions change in coverage: 1 decreased (-0.02%), 7 increased, 6 no change

🔻 Code Coverage Decreases vs Default Branch (1)

  • test DatadogCrashReportingTests tvOS 26.66% (-0.02%) - Details

@simaoseica-dd simaoseica-dd merged commit 4f92b45 into develop Dec 10, 2024
16 checks passed
@maxep maxep mentioned this pull request Dec 11, 2024
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants