Skip to content

Commit 554482a

Browse files
Add Flaky Tests <> Bits Dev documentation (#32368)
* Add Flaky Tests <> Bits Dev documentation * Update content/en/tests/flaky_management/_index.md Co-authored-by: May Lee <may.lee@datadoghq.com> * add bits ai docs --------- Co-authored-by: May Lee <may.lee@datadoghq.com>
1 parent 369b09e commit 554482a

File tree

3 files changed

+25
-10
lines changed

3 files changed

+25
-10
lines changed

content/en/bits_ai/bits_ai_dev_agent.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -25,12 +25,12 @@ Bits AI Dev Agent is a generative AI coding assistant that uses observability da
2525

2626
Bits AI Dev Agent is available for the following Datadog products:
2727

28-
| Product | Availability | Issues Detected |
29-
|---------------------------|----------------------|--------------------------------------------------------------------|
30-
| [Error Tracking][1] | Preview | Crashes, panics, exceptions, unhandled errors |
31-
| [Code Security][2] | Preview | Security issues in first-party code and open source dependencies |
32-
| [Continuous Profiler][3] | Preview | Code-level performance issues |
33-
| [Test Optimization][7] | Preview | Flaky tests |
28+
| Product | Availability | Issues Detected |
29+
| ------------------------ | ------------ | ------------------------------------------------------------------- |
30+
| [Error Tracking][1] | Preview | Crashes, panics, exceptions, unhandled errors |
31+
| [Code Security][2] | Preview | Security issues in first-party code and open source dependencies |
32+
| [Continuous Profiler][3] | Preview | Code-level performance issues |
33+
| [Test Optimization][7] | Preview | [Flaky tests](/tests/flaky_management/#ai-powered-flaky-test-fixes) |
3434

3535
## Setup
3636

content/en/tests/flaky_management/_index.md

Lines changed: 19 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -87,10 +87,24 @@ When you fix a flaky test, Test Optimization's remediation flow can confirm the
8787
1. Copy the unique flaky test key that is displayed (for example, `DD_ABC123`).
8888
1. Include the test key in your Git commit title or message for the fix (for example, `git commit -m "DD_ABC123"`).
8989
1. When Datadog detects the test key in your commit, it automatically triggers the remediation flow for that test:
90-
- Retries any tests you're attempting to fix 20 times.
91-
- Runs tests even if they are marked as `Disabled`.
92-
- If all retries pass, updates the test's status to `Fixed`.
93-
- If any retry fails, keeps the test's current status (`Active`, `Quarantined`, or `Disabled`).
90+
- Retries any tests you're attempting to fix 20 times.
91+
- Runs tests even if they are marked as `Disabled`.
92+
- If all retries pass, updates the test's status to `Fixed`.
93+
- If any retry fails, keeps the test's current status (`Active`, `Quarantined`, or `Disabled`).
94+
95+
## AI-powered flaky test fixes
96+
97+
{{< callout url="http://datadoghq.com/product-preview/bits-ai-dev-agent" >}}
98+
Bits AI Dev Agent is in Preview. To sign up, click <strong>Request Access</strong> and complete the form.
99+
{{< /callout >}}
100+
101+
Bits AI Dev Agent can automatically diagnose and fix flaky tests that have been detected by Test Optimization. When a flaky test is identified, Bits AI analyzes the test failure patterns and generates production-ready fixes that can be submitted as GitHub pull requests.
102+
103+
{{< img src="tests/bits_ai_flaky_test_fixes.png" alt="Bits AI Dev Agent displaying a proposed fix for a flaky test" style="width:100%;" >}}
104+
105+
### Setup
106+
107+
To enable AI-powered flaky test fixes, enable Bits AI Dev Agent for Test Optimization by following the setup instructions in the [Bits AI Dev Agent documentation][16]. Bits AI Dev Agent automatically create fixes for flaky tests detected by Test Optimization.
94108

95109
## AI-powered flaky test categorization
96110

@@ -156,3 +170,4 @@ To use Flaky Test Management features, you must use Datadog's native instrumenta
156170
[11]: /tests/setup/ruby/
157171
[12]: /tests/setup/swift/
158172
[13]: https://app.datadoghq.com/ci/settings/test-optimization/flaky-test-management
173+
[16]: /bits_ai/bits_ai_dev_agent/
297 KB
Loading

0 commit comments

Comments
 (0)