Manual test tracing API

[anmarchenko]

What does this PR do?
This PR refactors Datadog::CI internals and provides a public API for manual test instrumentation via Datadog::CI module.

How to use the new API:

# trace a test
Datadog::CI.trace_test(
  "my test",
  tags: {
    "test.framework" => "my framework",
    "test.suite" => "my suite"
  },
  service_name: "service",
  operation_name: "test.test"
) do |test|
   # test here
   
   if result == :pass
     test.passed!
   else
     test.failed!
   end
end

# trace any custom span inside the test
Datadog::CI.trace(
  "step",           # span type
  "my step",     # span name
  tags: {
      "test.framework" => "my framework",
  }
) do |span|
  # code here
end

There is also a convenience method CI.start_test that does not accept a block:

my_test = Datadog::CI.start_test(
  "my test",
  tags: {
    "test.framework" => "my framework",
    "test.suite" => "my suite"
  },
  service_name: "service",
  operation_name: "test.test"
)

# ....

my_test.finish

These methods return or yield new Datadog::CI::Span object that represents tracing span in CI context adding some domain-specific behaviour (like a possibility to set status with passed!, skipped!, and failed! methods)

Next steps (in subsequent PRs)

• create Datadog::CI::ContextProvider module to store active_test span
• expand manual API to support TestSession, TestModule, and TestSuite traces
• create domain objects for Test, TestSuite, etc (like we have for Span now)
• use the manual API to instrument RSpec, Minitest, Cucuber with test suite level visibility
• serialize and submit test suite level visibility spans to the backend

How to test the change?
No behaviour change, unit&integration tests make sure that it still works

[codecov-commenter]

Codecov Report

Merging #61 (c16bc0d) into main (9f6d671) will increase coverage by 0.01%.
Report is 2 commits behind head on main.
The diff coverage is 98.43%.

@@            Coverage Diff             @@
##             main      #61      +/-   ##
==========================================
+ Coverage   99.15%   99.17%   +0.01%     
==========================================
  Files         103      106       +3     
  Lines        3533     3616      +83     
  Branches      134      131       -3     
==========================================
+ Hits         3503     3586      +83     
  Misses         30       30              

Files	Coverage Δ
lib/datadog/ci.rb	100.00% <100.00%> (ø)
lib/datadog/ci/configuration/components.rb	100.00% <100.00%> (ø)
lib/datadog/ci/contrib/minitest/hooks.rb	96.77% <100.00%> (-0.20%)	⬇️
lib/datadog/ci/recorder.rb	100.00% <100.00%> (ø)
lib/datadog/ci/span.rb	100.00% <100.00%> (ø)
spec/datadog/ci/configuration/components_spec.rb	96.39% <100.00%> (+0.10%)	⬆️
spec/datadog/ci/recorder_spec.rb	100.00% <100.00%> (ø)
spec/datadog/ci/span_spec.rb	100.00% <100.00%> (ø)
spec/datadog/ci_spec.rb	100.00% <100.00%> (ø)
lib/datadog/ci/contrib/rspec/example.rb	96.55% <87.50%> (-0.12%)	⬇️
... and 1 more

📣 Codecov offers a browser extension for seamless coverage viewing on GitHub. Try it in Chrome or Firefox today!

[marcotc]

A bit of an unrelated comment, but this before_setup method cannot be nested with itself right? I ask because setting Thread.current[:_datadog_test_span] = test_span twice before calling the respective after_teardown in the correct order will cause issues.

[anmarchenko]

good point, I will check that - should not be the case, I have a bunch of minitest projects for integration testing, did not encounter lost test traces

[anmarchenko]

anyway this is something to replace with Datadog::CI::Context in the subsequent PR