Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: (automatically) set backends through a fixture #276

Merged
merged 17 commits into from
Sep 20, 2018

Conversation

kratsg
Copy link
Contributor

@kratsg kratsg commented Sep 19, 2018

Description

This PR tries to make testing a lot easier for us moving forward as we start having complexity problems with testing combinations of backends+optimizers and so on and lots of duplicated code. In particular, the fixture is called backend so adding a backend parameter to any test function you have will automatically wrap that function in setting up the backend correctly, and breaking it down after the test runs. This can be useful with certain backends like tensorflow that need a new session or its internal graph cleared.

Additionally, some extra decorations (pytest.marks) are specified to make this even easier to fine-tune which backends run. Right now, we add 5: numpy, tensorflow, pytorch, mxnet, numpy_minuit. The last one is numpy with the minuit optimizer.

One can specify, for example, to only run numpy and numpy_minuit

@pytest.mark.only_numpy
@pytest.mark.only_numpy_minuit

or

@pytest.mark.skip_tensorflow
@pytest.mark.skip_pytorch
@pytest.mark.skip_mxnet

will be equivalent. The latter will not skip any new backends that get added after this PR, but the former will still only consider those two.

Similarly, if you expect a particular backend to fail for any reason, you can mark that specific backend as failing via

@pytest.mark.fail_tensorflow

for example.

Checklist Before Requesting Approver

  • Tests are passing
  • "WIP" removed from the title of the pull request

Sorry, something went wrong.

@kratsg kratsg added the feat/enhancement New feature or request label Sep 19, 2018
@kratsg
Copy link
Contributor Author

kratsg commented Sep 19, 2018

Tests are breaking currently, see scikit-hep/uproot3#130.

@kratsg
Copy link
Contributor Author

kratsg commented Sep 19, 2018

Fixed with scikit-hep/uproot3-methods#1.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.2%) to 97.345% when pulling f6a4228 on feature/backendFixtureTest into 47c032e on master.

1 similar comment
@coveralls
Copy link

Coverage Status

Coverage decreased (-0.2%) to 97.345% when pulling f6a4228 on feature/backendFixtureTest into 47c032e on master.

@coveralls
Copy link

coveralls commented Sep 19, 2018

Coverage Status

Coverage increased (+0.6%) to 98.15% when pulling 9efd25b on feature/backendFixtureTest into 47c032e on master.

Copy link
Member

@matthewfeickert matthewfeickert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kratsg kratsg merged commit 272f1a9 into master Sep 20, 2018
@kratsg kratsg deleted the feature/backendFixtureTest branch September 20, 2018 04:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feat/enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants