Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CIVIS-8552] ci-queue runner support for minitest #110

Merged
merged 5 commits into from
Jan 23, 2024

Conversation

anmarchenko
Copy link
Member

@anmarchenko anmarchenko commented Jan 19, 2024

What does this PR do?
Adds ci-queue support when using Minitest

Additional Notes
ci-queue is a test runner developed and open sourced by Shopify.
It runs Minitest and RSpec (deprecated) test suites in parallel using several workers and coordinating using redis queue.

ci-queue instrumentation for Minitest posed several challenges:

  • it turned out that datadog-ci-rb currently abuses Minitest plugin system by accessing Minitest.extensions directly which caused ci-queue to fail completely
  • even when using the correct way of working with minitest plugins by putting the plugin code in minitest/datadog_ci_plugin.rb file it still fails: ci-queue uses minitest-reporters gem that does not correctly work with minitest plugin system. Minitest-reporters gem simply replaces the framework reporters with its own reporters essentially discarding them. Author of minitest framework created an issue in minitest-reporters repository about this problem but it isn't resolved to this day
  • above issues were solved by directly instrumenting Minitest::CompositeReporter class that is not being overwritten by minitest-reporters (yet?)
  • ci-queue does not include Minitest::Parallel::Test in Minitest::Test so we missed test suites - solved by checking if Minitest::Queue is included

How to test the change?
Unit tests are provided

@codecov-commenter
Copy link

codecov-commenter commented Jan 22, 2024

Codecov Report

Attention: 11 lines in your changes are missing coverage. Please review.

Comparison is base (b8ce942) 99.05% compared to head (897abcc) 97.75%.

Files Patch % Lines
lib/datadog/ci/contrib/minitest/reporter.rb 33.33% 8 Missing ⚠️
lib/datadog/ci/contrib/minitest/helpers.rb 0.00% 2 Missing ⚠️
lib/datadog/ci/contrib/minitest/patcher.rb 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #110      +/-   ##
==========================================
- Coverage   99.05%   97.75%   -1.31%     
==========================================
  Files         147      144       -3     
  Lines        6276     5887     -389     
  Branches      288      288              
==========================================
- Hits         6217     5755     -462     
- Misses         59      132      +73     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@anmarchenko anmarchenko force-pushed the anmarchenko/minitest_instrumentation_ci_queue branch from 76a9b7e to decef1d Compare January 22, 2024 11:47
@anmarchenko anmarchenko marked this pull request as ready for review January 22, 2024 14:06
@anmarchenko anmarchenko requested review from a team as code owners January 22, 2024 14:06
@anmarchenko anmarchenko changed the title ci-queue runner support for minitest [CIVIS-8552] ci-queue runner support for minitest Jan 22, 2024
@anmarchenko anmarchenko merged commit c3ccfee into main Jan 23, 2024
26 checks passed
@anmarchenko anmarchenko deleted the anmarchenko/minitest_instrumentation_ci_queue branch January 23, 2024 08:27
@github-actions github-actions bot added this to the 0.7.0 milestone Jan 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants