Skip to content

Commit cad9c17

Browse files
Add Flaky Tests <> Bits Dev documentation
1 parent aa54b0a commit cad9c17

File tree

2 files changed

+19
-4
lines changed

2 files changed

+19
-4
lines changed

content/en/tests/flaky_management/_index.md

Lines changed: 19 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -86,10 +86,24 @@ When you fix a flaky test, Test Optimization's remediation flow can confirm the
8686
1. Copy the unique flaky test key that is displayed (for example, `DD_ABC123`).
8787
1. Include the test key in your Git commit title or message for the fix (for example, `git commit -m "DD_ABC123"`).
8888
1. When Datadog detects the test key in your commit, it automatically triggers the remediation flow for that test:
89-
- Retries any tests you're attempting to fix 20 times.
90-
- Runs tests even if they are marked as `Disabled`.
91-
- If all retries pass, updates the test's status to `Fixed`.
92-
- If any retry fails, keeps the test's current status (`Active`, `Quarantined`, or `Disabled`).
89+
- Retries any tests you're attempting to fix 20 times.
90+
- Runs tests even if they are marked as `Disabled`.
91+
- If all retries pass, updates the test's status to `Fixed`.
92+
- If any retry fails, keeps the test's current status (`Active`, `Quarantined`, or `Disabled`).
93+
94+
## AI-powered flaky test fixes
95+
96+
{{< callout url="http://datadoghq.com/product-preview/bits-ai-dev-agent" >}}
97+
Bits AI Dev Agent is in Preview. To sign up, click <strong>Request Access</strong> and complete the form.
98+
{{< /callout >}}
99+
100+
{{< img src="tests/bits_ai_flaky_test_fixes.png" alt="Bits AI Dev Agent displaying a proposed fix for a flaky test" style="width:100%;" >}}
101+
102+
Bits AI Dev Agent can automatically diagnose and fix flaky tests that have been detected by Test Optimization. When a flaky test is identified, Bits AI analyzes the test failure patterns and generates production-ready fixes that can be submitted as GitHub pull requests.
103+
104+
### Setup
105+
106+
To enable AI-powered flaky test fixes, enable Bits AI Dev Agent for Test Optimization by following the setup instructions in the [Bits AI Dev Agent documentation][16]. Bits AI Dev Agent will automatically create fixes for flaky tests detected by Test Optimization.
93107

94108
## AI-powered flaky test categorization
95109

@@ -155,3 +169,4 @@ To use Flaky Test Management features, you must use Datadog's native instrumenta
155169
[11]: /tests/setup/ruby/
156170
[12]: /tests/setup/swift/
157171
[13]: https://app.datadoghq.com/ci/settings/test-optimization/flaky-test-management
172+
[16]: /bits_ai/bits_ai_dev_agent/
297 KB
Loading

0 commit comments

Comments
 (0)