[test optimization] [SDTEST-1529] Add quarantined tests logic

[juan-fernandez]

What does this PR do?

• Read test_management configuration parameter
• If it's true, request quarantined tests
• Ignore failed tests that are also quarantined (*)

(*): this is test framework dependent: playwright and cypress actually skip the test altogether (I couldn't find a way to do it any other way). The other frameworks still run the test but prevent the test session from failing.

Motivation

Quarantine allows turning your pipelines green by removing the effect of flaky tests. Flaky tests can then be tackled without your pipelines being red.

Plugin Checklist

• Unit tests.

### Additional notes

There's a configuration change, so I need to update the system tests. Feel free to ignore system tests errors for the moment.

[github-actions]

Overall package size

Self size: 8.73 MB
Deduped: 94.93 MB
No deduping: 95.44 MB

Dependency sizes

| name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.4.0 | 29.44 MB | 29.44 MB | | @datadog/native-appsec | 8.4.0 | 19.25 MB | 19.26 MB | | @datadog/native-iast-taint-tracking | 3.3.0 | 13.77 MB | 13.78 MB | | @datadog/pprof | 5.5.1 | 9.79 MB | 10.17 MB | | protobufjs | 7.2.5 | 2.77 MB | 5.16 MB | | @datadog/native-iast-rewriter | 2.8.0 | 2.6 MB | 2.74 MB | | @opentelemetry/core | 1.14.0 | 872.87 kB | 1.47 MB | | @datadog/native-metrics | 3.1.0 | 1.06 MB | 1.46 MB | | @opentelemetry/api | 1.8.0 | 1.21 MB | 1.21 MB | | import-in-the-middle | 1.11.2 | 112.74 kB | 835.4 kB | | source-map | 0.7.4 | 226 kB | 226 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | lru-cache | 7.18.3 | 133.92 kB | 133.92 kB | | pprof-format | 2.1.0 | 111.69 kB | 111.69 kB | | @datadog/sketches-js | 2.1.0 | 109.9 kB | 109.9 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 5.3.2 | 53.63 kB | 53.63 kB | | shell-quote | 1.8.1 | 44.96 kB | 44.96 kB | | istanbul-lib-coverage | 3.2.0 | 29.34 kB | 29.34 kB | | rfdc | 1.3.1 | 25.21 kB | 25.21 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | dc-polyfill | 0.1.4 | 23.1 kB | 23.1 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | semifies | 1.0.0 | 15.84 kB | 15.84 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | ttl-set | 1.0.0 | 4.61 kB | 9.69 kB | | path-to-regexp | 0.1.12 | 6.6 kB | 6.6 kB | | koalas | 1.0.2 | 6.47 kB | 6.47 kB | | module-details-from-path | 1.0.3 | 4.47 kB | 4.47 kB |

_{🤖 This report was automatically generated by heaviest-objects-in-the-universe}

[codecov]

Codecov Report

Attention: Patch coverage is 10.00000% with 90 lines in your changes missing coverage. Please review.

Project coverage is 80.93%. Comparing base (8e8898d) to head (7661f31).
Report is 2 commits behind head on master.

Files with missing lines	Patch %	Lines
packages/datadog-instrumentations/src/cucumber.js	0.00%	24 Missing ⚠️
...ackages/datadog-instrumentations/src/mocha/main.js	0.00%	24 Missing ⚠️
...ibility/quarantined-tests/get-quarantined-tests.js	14.28%	18 Missing ⚠️
...ckages/datadog-instrumentations/src/mocha/utils.js	0.00%	12 Missing ⚠️
packages/datadog-plugin-cucumber/src/index.js	0.00%	4 Missing ⚠️
packages/datadog-plugin-mocha/src/index.js	0.00%	4 Missing ⚠️
.../ci-visibility/exporters/ci-visibility-exporter.js	20.00%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #5236      +/-   ##
==========================================
- Coverage   81.24%   80.93%   -0.32%     
==========================================
  Files         487      488       +1     
  Lines       21703    21800      +97     
==========================================
+ Hits        17633    17643      +10     
- Misses       4070     4157      +87     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[datadog-datadog-prod-us1]

Datadog Report

Branch report: juan-fernandez/add-quarantined-tests
Commit report: 99382f6
Test service: dd-trace-js-integration-tests

✅ 0 Failed, 669 Passed, 0 Skipped, 13m 51.73s Total Time

[pr-commenter]

Benchmarks

Benchmark execution time: 2025-02-13 17:13:58

Comparing candidate commit 7661f31 in PR branch juan-fernandez/add-quarantined-tests with baseline commit 8e8898d in branch master.

Found 0 performance improvements and 1 performance regressions! Performance is the same for 905 metrics, 27 unstable metrics.

scenario:plugin-graphql-with-async-hooks-18

• 🟥 max_rss_usage [+85.947MB; +89.597MB] or [+16.260%; +16.950%]