Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement @console_rule for aggregate test runner/orchestrator #6006

Closed
kwlzn opened this issue Jun 22, 2018 · 1 comment
Closed

Implement @console_rule for aggregate test runner/orchestrator #6006

kwlzn opened this issue Jun 22, 2018 · 1 comment
Assignees

Comments

@kwlzn
Copy link
Member

kwlzn commented Jun 22, 2018

breakout task from the v2 FE HLD here


This issue covers adding a @console_rule (and possibly some support @rules) that implements basic rendering of test result values.

This should look like extracting the TestResult rendering logic from TestrunnerTaskMixin into a static function:

for partition in sorted(results):
rv = results[partition]
failed_targets = set(rv.failed_targets)
pre_execution_error = not failed_targets and not rv.success
for target in partition:
if pre_execution_error:
log = self.context.log.warn
result = 'NOT RUN'
elif target in failed_targets:
log = self.context.log.error
result = rv
else:
log = self.context.log.info
result = self.result_class.successful()
log('{0:80}.....{1:>10}'.format(target.address.reference(), str(result)))
msgs = [str(_rv) for _rv in results.values() if not _rv.success]
failed_targets = [target
for _rv in results.values() if not _rv.success
for target in _rv.failed_targets]
if len(failed_targets) > 0:
raise ErrorWhileTesting('\n'.join(msgs), failed_targets=failed_targets)
elif failure:
# A low-level test execution failure occurred before tests were run.
raise TaskError()

...and then consuming it from a new @console_rule (similar to fast_list) that will look roughly like:

@console_rule('test', [Select(Console), Select(HydratedTargets)])
def fast_test(console, hydrated_targets):
  test_results = yield [Get(TestResult, HydratedTarget, ht) for ht in hydrated_targets]

  ... # use extracted static function to render the `TestResult` instances.

So the meat of the change is 1) extracting the static function to use it from an @console_rule, 2) adding a coverage test for the @console_rule (described below).


A coverage test of these @rules will look like:

  1. extending TestBase
  2. registering the new rules using a def rules(cls): definition on your TestBase instance (similar to this one) in the python backend's register.py
  3. additionally registering some basic/dummy "TestResults" @rules in def rules that will provide the TestResult inputs that the @console_rule will need.
  4. calling
# Request the `test` goal for any dummy target (perhaps just a `target(..)` alias).
specs = Specs((SingleAddress('testprojects/my/dummy/project'),))
# Request execution of the synthetic `test` product using GoalProduct from src/python/pants/engine/rules.py
test_result, = self.scheduler.product_request(_GoalProduct.for_name('test'), specs)

...in the test, which should cause your registered @console_rule to execute*. The test should just confirm that the rule execution does not fail.

* NB: this will not actually allow for testing the rendered output of the @console_rule, but it will not be possible to do that until #6571 is completed.

@illicitonion
Copy link
Contributor

illicitonion commented Oct 19, 2018

WIP in #6646 - last commit needs to be reverted and replaced with a better approach, which should take at most an hour or so :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants