Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CIVIS-2934] ITR test skipping implementation #160

Merged
merged 16 commits into from
Apr 22, 2024

Conversation

anmarchenko
Copy link
Member

@anmarchenko anmarchenko commented Apr 17, 2024

What does this PR do?

  • Adds actual test skipping by ITR for minitest, rspec, cucumber with unit tests
  • Adds test.parameters support to skippable tests logic
  • Updates datadog dependency to 2.0.0.beta2 and solves test failures related to it

How to test the change?
Tested on DD staging.

Results for Rubocop:
image

Results for sidekiq:
image

@anmarchenko anmarchenko changed the base branch from main to 1.0 April 17, 2024 09:16
@codecov-commenter
Copy link

codecov-commenter commented Apr 17, 2024

Codecov Report

Attention: Patch coverage is 99.39577% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 98.90%. Comparing base (4f59bc7) to head (8c4a167).

Files Patch % Lines
lib/datadog/ci/itr/runner.rb 92.30% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##              1.0     #160      +/-   ##
==========================================
- Coverage   99.05%   98.90%   -0.16%     
==========================================
  Files         195      196       +1     
  Lines        9032     9326     +294     
  Branches      390      404      +14     
==========================================
+ Hits         8947     9224     +277     
- Misses         85      102      +17     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@anmarchenko anmarchenko marked this pull request as ready for review April 22, 2024 09:41
@anmarchenko anmarchenko requested review from a team as code owners April 22, 2024 09:41
@anmarchenko anmarchenko merged commit 8183c3a into 1.0 Apr 22, 2024
23 checks passed
@anmarchenko anmarchenko deleted the anmarchenko/itr_skips_tests branch April 22, 2024 11:03
def execute(*args)
test_span = CI.active_test
if test_span&.skipped_by_itr?
@action.skip(*args)
Copy link
Member

@marcotc marcotc Apr 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm guessing Cucumber doesn't have a way to report the message CI::Ext::Test::ITR_TEST_SKIP_REASON, unlike Minitest and RSpec?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, at least not easily findable.

skip_this_scenario method does not accept arguments with skip reason for example

@anmarchenko anmarchenko added this to the 1.0 milestone Apr 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants