Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update Prf unit test #513

Merged
merged 23 commits into from
Apr 29, 2021
Merged

Update Prf unit test #513

merged 23 commits into from
Apr 29, 2021

Conversation

yuki617
Copy link
Member

@yuki617 yuki617 commented Apr 27, 2021

  • Add test for cross validation
  • Add scripts to generate run files while checking the score

yuki617 added 22 commits December 19, 2020 23:06
@yuki617
add trec-covid pre-built index to pyserini
634accf
@yuki617
fix a bug on total_terms,documents, and unique_terms number for trec-…
05f3722 
…covid pre-built index
@yuki617
add full-text and paragraph pre-built index for trec covid round 1-5
4647ba1
@yuki617
update the github from raw html landing to raw tarball
f064818
@yuki617
add test for trec_covid_r3 scripts
b370561
@yuki617
merge with origin
924fc79
@yuki617
fix bug on pre-built index and terdown function
fa4d129
@yuki617
fix typo on function name
6ea7db8
@yuki617
add script for downloading files to runs
e9828ee
@yuki617
allow the script to be called outside integration
7c85542
@yuki617
add trec covid ranker file to pyserini
1ae8c58
@yuki617
fix a typo on simplecheck
eb49795
@yuki617
update the evaluating qrel files
2de0942
@yuki617
fix a typo
bb3232e
@yuki617
Merge https://github.com/castorini/pyserini into prf
612ca09
@yuki617
add row1-3 unit tests for prf results table
a9f4346
@yuki617
allow user to change different python version
1d61a31
@yuki617
add unit tests for core17, robust04, robust05 for prf
c4a1fb4
@yuki617
add exception if the run file tar.gz is not downloaded properly
d4895b4
@yuki617
update python3 to python
077a21b
@yuki617
merge with master
8616ab1
@yuki617
add prf test for cross validation, run file generation and score check
ed30f2a
@yuki617 yuki617 requested a review from lintool April 27, 2021 14:25
@lintool lintool mentioned this pull request Apr 27, 2021
Merged
@lintool
Copy link
Member

lintool commented Apr 28, 2021

Ran the tests on my iMac Pro:

======================================================================
FAIL: test_cross_validation (integrations.test_simplesearcher_check_prf.TestSearchIntegration)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/jimmylin/workspace/pyserini/integrations/test_simplesearcher_check_prf.py", line 93, in test_cross_validation
    self.assertEqual(status, 0)
AssertionError: 256 != 0

----------------------------------------------------------------------
Ran 41 tests in 77365.158s

FAILED (failures=1)

That's ~21 hours for the tests...

@yuki617
Copy link
Member Author

yuki617 commented Apr 28, 2021

That's caused by the missing of mkdir script for folder core17. Let me update it and send the review request again

@lintool lintool merged commit 42db07b into castorini:master Apr 29, 2021
@lintool
Copy link
Member

lintool commented Apr 29, 2021

I've confirmed that the fix works!

@lintool lintool mentioned this pull request Apr 29, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lintool lintool lintool approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@yuki617 @lintool