Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added formal regression for MS MARCO passage task #692

Merged
merged 3 commits into from
Jun 8, 2019
Merged

Added formal regression for MS MARCO passage task #692

merged 3 commits into from
Jun 8, 2019

Conversation

lintool
Copy link
Member

@lintool lintool commented Jun 8, 2019

Ref #690

  • added MS MARCO passage regression into Anserini's framework
  • added tsv topic reader
  • "msmarco" -> "msmarco-passage" to make way for doc task
  • fixed bug with Java timing of MS MACRO retrieve class

@lintool lintool requested a review from rodrigonogueira4 June 8, 2019 16:19
@lintool lintool mentioned this pull request Jun 8, 2019
Closed
rodrigonogueira4
rodrigonogueira4 reviewed Jun 8, 2019
View reviewed changes
docs/experiments-msmarco-passage.md
First, we need to download and extract the MS MARCO dataset:

```
DATA_DIR=./msmarco_data
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Much better!

rodrigonogueira4
rodrigonogueira4 reviewed Jun 8, 2019
View reviewed changes
docs/regressions-msmarco-passage.md

MAP | BM25 (Default)| +RM3 | BM25 (Tuned)| +RM3 |
:---------------------------------------|-----------|-----------|-----------|-----------|
[MS MARCO Passage Ranking: Dev Queries](https://github.com/microsoft/MSMARCO-Passage-Ranking)| 0.1924 | 0.1661 | 0.1956 | 0.1766 |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing!

rodrigonogueira4
rodrigonogueira4 reviewed Jun 8, 2019
View reviewed changes
Copy link
Member

@rodrigonogueira4 rodrigonogueira4 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this!

docs/regressions-msmarco-passage.md Outdated Show resolved Hide resolved
@lintool lintool merged commit 781d9ed into master Jun 8, 2019
@lintool lintool deleted the msmarco-regression branch June 8, 2019 19:16
@lintool lintool mentioned this pull request Jun 9, 2019
Closed
@lintool lintool mentioned this pull request Jul 28, 2019
Closed
crystina-z pushed a commit to crystina-z/anserini that referenced this pull request Oct 28, 2022
@ArthurChen189
Fixed issue 689 (castorini#692)
c9f44b2 
fix curatedTREC dpr integration tests
Co-authored-by: Arthur Chen <haonan.chen@uwaterloo.ca>
crystina-z pushed a commit to crystina-z/anserini that referenced this pull request Oct 28, 2022
@lintool
DPR documentation fix (castorini#703)
5d3f459 
Following castorini#692 - which fixes castorini#689 - changes need to be propagated to the documentation also.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rodrigonogueira4 rodrigonogueira4 rodrigonogueira4 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@lintool @rodrigonogueira4