Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-enable the Integration Tests in xfail mode again #3682

Closed
wants to merge 10 commits into from
Closed

Re-enable the Integration Tests in xfail mode again #3682

wants to merge 10 commits into from

Conversation

cclauss
Copy link
Collaborator

@cclauss cclauss commented Aug 11, 2020

Does anyone have an instance of 'auth.yaml' on their local disk?

That file was not checked into git but would help use to reconstruct the configuration values.

On HOLD: Pending further progress on Python 3.
Closes #672 #2725 #3674 which got in a bad git state.

Instead of getting rid of our Integration Tests as discussed in #2725, let's re-enable them and set the ones that are not working in pytest.mark.xfail mode so that they can be fixed over time.

This approach allows us to keep the eight tests that already work and add new tests to cover the new Python 3 code.

This PR also adds a new Travis CI job to automatically run these user interface tests on all pull requests. The problem is that building the Docker images from scratch takes 8+ minutes on Travis CI so the total job takes 12 minutes! Perhaps this PR should be landed after #3615 so we can reuse that Docker Hub image.

Technical

Testing

The testing steps are described at
https://github.com/internetarchive/openlibrary/pull/3674/files#diff-6b1c0422e0532f9de8b00ee73de3d6acR21

Screenshot

Stakeholders

pushd tests/integration
pytest .

================================ test session starts ======================================
platform darwin -- Python 3.8.5, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /Users/cclauss/Python/itinerant_futurizer/openlibrary
collected 44 items

test_affiliate_links.py ..                                                                                                                                                                                                                                                        [  4%]
test_auth.py .xxxxxxxxxxxxxx.xxxxxxx.                                                                                                                                                                                                                                             [ 59%]
test_landing.py xxxxx                                                                                                                                                                                                                                                             [ 70%]
test_loans.py xxx                                                                                                                                                                                                                                                                 [ 77%]
test_microdata.py x.                                                                                                                                                                                                                                                              [ 81%]
test_search.py .xxx                                                                                                                                                                                                                                                               [ 90%]
test_sharing.py xx.x                                                                                                                                                                                                                                                              [100%]

=========================== 8 passed, 36 xfailed in 169.30s (0:02:49) ===========================

@cclauss cclauss added Python Theme: Testing Involves work related to testing infrastructure, development, and ops. [managed] labels Aug 11, 2020
@cclauss cclauss added this to the Active Sprint milestone Aug 11, 2020
@cclauss cclauss mentioned this pull request Aug 11, 2020
77 tasks
@cdrini cdrini self-assigned this Aug 11, 2020
@cclauss cclauss marked this pull request as draft August 17, 2020 10:40
@cclauss
Copy link
Collaborator Author

cclauss commented Aug 28, 2020

Update: docker pull openlibrary/olbase:latest allows us to cut execution time in half to just under 5 min 30 sec on Travis.

@cdrini cdrini removed this from the Active Sprint milestone Sep 8, 2020
@cdrini cdrini removed their assignment Dec 14, 2020
@cclauss cclauss closed this Jan 21, 2021
@cclauss cclauss deleted the re-enable-integration-tests branch January 21, 2021 15:27
@mekarpeles mekarpeles added the python Pull requests that update Python code label Sep 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
python Pull requests that update Python code Theme: Testing Involves work related to testing infrastructure, development, and ops. [managed]
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Integration tests fail looking for auth.yaml
3 participants