Anserini: #2122

[AileenLin]

No description provided.

[lintool]

Hi @AileenLin - if we implement the quantization and API changes we discussed on the Java end, we wouldn't need this on the Python end, right?

[lintool]

Hi @AileenLin - with the Anserini changes, something like this should work right?

$ python -m pyserini.search.lucene \
>   --threads 16 --batch-size 128 \
>   --index msmarco-v1-passage-unicoil \
>   --topics dl19-passage-unicoil \
>   --output run.msmarco-v1-passage.unicoil.dl19.txt \
>   --hits 1000 --impact --rocchio

I'm getting:

Initializing msmarco-v1-passage-unicoil...
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 192, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tuna1/scratch/jimmylin/pyserini/pyserini/search/lucene/__main__.py", line 217, in <module>
    searcher.set_rocchio()
  File "/tuna1/scratch/jimmylin/pyserini/pyserini/search/lucene/_impact_searcher.py", line 318, in set_rocchio
    elif self.prebuilt_index_name in ['msmarco-v1-passage', 'msmarco-v1-doc', 'msmarco-v1-doc-segmented']:
AttributeError: 'LuceneImpactSearcher' object has no attribute 'prebuilt_index_name'

Something I'm missing?

[lintool]

Results so far:

python -m pyserini.search.lucene \
  --threads 16 --batch-size 128 \
  --index msmarco-v1-passage-splade-pp-ed \
  --topics dl19-passage \
  --onnx-encoder SpladePlusPlusEnsembleDistil \
  --output runs/run.msmarco-v1-passage.splade-pp-ed-onnx.dl19.txt \
  --hits 1000 --impact

python -m pyserini.eval.trec_eval -c -l 2 -m map dl19-passage runs/run.msmarco-v1-passage.splade-pp-ed-onnx.dl19.txt
python -m pyserini.eval.trec_eval -c -m ndcg_cut.10 dl19-passage runs/run.msmarco-v1-passage.splade-pp-ed-onnx.dl19.txt
python -m pyserini.eval.trec_eval -c -l 2 -m recall.1000 dl19-passage runs/run.msmarco-v1-passage.splade-pp-ed-onnx.dl19.txt

map                   	all	0.5050
ndcg_cut_10           	all	0.7308
recall_1000           	all	0.8728

# Note this is different from
# SPLADE++ EnsembleDistil: query inference with ONNX	0.5054	0.7320	0.8724	
# but the same as here: https://github.com/castorini/anserini/blob/master/docs/regressions-dl19-passage-splade-pp-ed-onnx.md

python -m pyserini.search.lucene \
  --threads 16 --batch-size 128 \
  --index msmarco-v1-passage-splade-pp-ed-docvectors \
  --topics dl19-passage \
  --onnx-encoder SpladePlusPlusEnsembleDistil \
  --output runs/run.msmarco-v1-passage.splade-pp-ed-onnx.rocchio1.dl19.txt \
  --hits 1000 --impact --rocchio

python -m pyserini.eval.trec_eval -c -l 2 -m map dl19-passage runs/run.msmarco-v1-passage.splade-pp-ed-onnx.rocchio1.dl19.txt
python -m pyserini.eval.trec_eval -c -m ndcg_cut.10 dl19-passage runs/run.msmarco-v1-passage.splade-pp-ed-onnx.rocchio1.dl19.txt
python -m pyserini.eval.trec_eval -c -l 2 -m recall.1000 dl19-passage runs/run.msmarco-v1-passage.splade-pp-ed-onnx.rocchio1.dl19.txt

# matches https://github.com/castorini/anserini/blob/master/docs/regressions-dl19-passage-splade-pp-ed-onnx.md
map                   	all	0.5140
ndcg_cut_10           	all	0.7119
recall_1000           	all	0.8799

python -m pyserini.search.lucene \
  --threads 16 --batch-size 128 \
  --index msmarco-v1-passage-splade-pp-ed-text \
  --topics dl19-passage \
  --onnx-encoder SpladePlusPlusEnsembleDistil \
  --output runs/run.msmarco-v1-passage.splade-pp-ed-onnx.rocchio2.dl19.txt \
  --hits 1000 --impact --rocchio

python -m pyserini.eval.trec_eval -c -l 2 -m map dl19-passage runs/run.msmarco-v1-passage.splade-pp-ed-onnx.rocchio2.dl19.txt
python -m pyserini.eval.trec_eval -c -m ndcg_cut.10 dl19-passage runs/run.msmarco-v1-passage.splade-pp-ed-onnx.rocchio2.dl19.txt
python -m pyserini.eval.trec_eval -c -l 2 -m recall.1000 dl19-passage runs/run.msmarco-v1-passage.splade-pp-ed-onnx.rocchio2.dl19.txt

# matches https://github.com/castorini/anserini/blob/master/docs/regressions-dl19-passage-splade-pp-ed-onnx.md
map                   	all	0.5140
ndcg_cut_10           	all	0.7119
recall_1000           	all	0.8799