Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Re-enable CI Tests visibility #2108

Merged
merged 3 commits into from
Nov 14, 2024

Conversation

ncreated
Copy link
Member

What and why?

📦 In #2096, we temporarily disabled CI Test instrumentation due to sporadic test failures caused by the following error:

🔥 Failed to delete TestsDirectory: Error Domain=NSCocoaErrorDomain Code=512 ""com.datadoghq.ios-sdk-tests-434C4A49-7195-4957-AE16-9A943737BDBF" couldn't be removed."

I traced the root cause to the CI Tests SDK, which exceeded the open file descriptor limit when processing coverage files for the Test Impact Analysis feature. This led to failures with the {Error Domain=NSPOSIXErrorDomain Code=24 "Too many open files"} error.

My whole investigation is documented here: internal doc.

How?

Since the issue arises only when Test Impact Analysis is enabled, I disabled it for dd-sdk-ios in the CI Test Optimization Settings page. This stops the CI Tests SDK from processing coverage files, preventing the file descriptor limit from being exceeded.

A corresponding issue (SDTEST-1200) has been opened in CI Visibility for further investigation. Once resolved, we can re-enable TIA.

Additionally, this PR refines error messages related to temporary directory management, improving clarity by pointing more directly to the failing code.

Review checklist

  • Feature or bugfix MUST have appropriate tests (unit, integration)
  • Make sure each commit and the PR mention the Issue number or JIRA reference
  • Add CHANGELOG entry for user facing changes
  • Add Objective-C interface for public APIs (see our guidelines [internal]) and run make api-surface)

@ncreated ncreated self-assigned this Nov 14, 2024
@ncreated ncreated requested review from a team as code owners November 14, 2024 09:00
Copy link
Member

@maxep maxep left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch!!

@ncreated
Copy link
Member Author

/merge

@dd-devflow
Copy link

dd-devflow bot commented Nov 14, 2024

Devflow running: /merge

View all feedbacks in Devflow UI.


2024-11-14 14:26:25 UTC ℹ️ MergeQueue: waiting for PR to be ready

This merge request is not mergeable yet, because of pending checks/missing approvals. It will be added to the queue as soon as checks pass and/or get approvals.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.


2024-11-14 14:26:27 UTC ℹ️ MergeQueue: waiting for PR to be ready

This merge request is not mergeable yet, because of pending checks/missing approvals. It will be added to the queue as soon as checks pass and/or get approvals.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.


2024-11-14 15:09:03 UTC ℹ️ MergeQueue: pull request added to the queue

The median merge time in develop is 29m.

@datadog-datadog-prod-us1
Copy link

datadog-datadog-prod-us1 bot commented Nov 14, 2024

Datadog Report

Branch report: ncreated/chore/enable-ci-visibility
Commit report: 4c149aa
Test service: dd-sdk-ios

✅ 0 Failed, 3543 Passed, 0 Skipped, 2m 30.07s Total Time
❄️ 1 New Flaky
🔻 Test Sessions change in coverage: 4 decreased, 8 increased, 2 no change

New Flaky Tests (1)

  • testWhenEnabled_itSendsConfigurationTelemetry_withNewApi - SessionReplayTests - Last Failure

    Expand for error
     Assertion Failure at SessionReplayTests.swift:249: XCTUnwrap failed: expected non-nil value of type "ConfigurationTelemetry"
    

🔻 Code Coverage Decreases vs Default Branch (4)

  • test DatadogCrashReportingTests tvOS 26.67% (-0.11%) - Details
  • test DatadogCrashReportingTests iOS 26.66% (-0.09%) - Details
  • test DatadogRUMTests iOS 81.13% (-0.04%) - Details
  • test DatadogRUMTests tvOS 81.25% (-0.03%) - Details

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants