diff --git a/docs/regressions-clef06-fr.md b/docs/regressions-clef06-fr.md index 349cc7a0b1..36d6103da3 100644 --- a/docs/regressions-clef06-fr.md +++ b/docs/regressions-clef06-fr.md @@ -62,8 +62,8 @@ With the above commands, you should be able to reproduce the following results: | **MAP** | **BM25** | |:-------------------------------------------------------------------------------------------------------------|-----------| -| [CLEF 2006 (Monolingual French)](../src/main/resources/topics-and-qrels/topics.clef06fr.mono.fr.txt) | 0.3111 | +| [CLEF 2006 (Monolingual French)](../src/main/resources/topics-and-qrels/topics.clef06fr.mono.fr.txt) | 0.3115 | | **P20** | **BM25** | | [CLEF 2006 (Monolingual French)](../src/main/resources/topics-and-qrels/topics.clef06fr.mono.fr.txt) | 0.3184 | | **nDCG@20** | **BM25** | -| [CLEF 2006 (Monolingual French)](../src/main/resources/topics-and-qrels/topics.clef06fr.mono.fr.txt) | 0.4458 | +| [CLEF 2006 (Monolingual French)](../src/main/resources/topics-and-qrels/topics.clef06fr.mono.fr.txt) | 0.4457 | diff --git a/docs/regressions-hc4-neuclir22-ru.md b/docs/regressions-hc4-neuclir22-ru.md index 9f6919a395..462dde1955 100644 --- a/docs/regressions-hc4-neuclir22-ru.md +++ b/docs/regressions-hc4-neuclir22-ru.md @@ -156,21 +156,21 @@ With the above commands, you should be able to reproduce the following results: | **MAP** | **BM25 (default)**| **+RM3** | **+Rocchio**| |:-------------------------------------------------------------------------------------------------------------|-----------|-----------|-----------| -| [HC4 (Russian): test-topic title](https://github.com/hltcoe/HC4) | 0.0964 | 0.0811 | 0.1245 | -| [HC4 (Russian): test-topic description](https://github.com/hltcoe/HC4) | 0.0926 | 0.0605 | 0.1064 | -| [HC4 (Russian): test-topic description+title](https://github.com/hltcoe/HC4) | 0.1113 | 0.0771 | 0.1341 | +| [HC4 (Russian): test-topic title](https://github.com/hltcoe/HC4) | 0.1040 | 0.0841 | 0.1231 | +| [HC4 (Russian): test-topic description](https://github.com/hltcoe/HC4) | 0.0963 | 0.0640 | 0.0964 | +| [HC4 (Russian): test-topic description+title](https://github.com/hltcoe/HC4) | 0.1264 | 0.0825 | 0.1314 | | **nDCG@20** | **BM25 (default)**| **+RM3** | **+Rocchio**| -| [HC4 (Russian): test-topic title](https://github.com/hltcoe/HC4) | 0.1380 | 0.1257 | 0.1668 | -| [HC4 (Russian): test-topic description](https://github.com/hltcoe/HC4) | 0.1459 | 0.0963 | 0.1643 | -| [HC4 (Russian): test-topic description+title](https://github.com/hltcoe/HC4) | 0.1640 | 0.1318 | 0.1899 | +| [HC4 (Russian): test-topic title](https://github.com/hltcoe/HC4) | 0.1445 | 0.1283 | 0.1655 | +| [HC4 (Russian): test-topic description](https://github.com/hltcoe/HC4) | 0.1495 | 0.1037 | 0.1569 | +| [HC4 (Russian): test-topic description+title](https://github.com/hltcoe/HC4) | 0.1762 | 0.1411 | 0.1875 | | **J@20** | **BM25 (default)**| **+RM3** | **+Rocchio**| -| [HC4 (Russian): test-topic title](https://github.com/hltcoe/HC4) | 0.0860 | 0.0730 | 0.0940 | -| [HC4 (Russian): test-topic description](https://github.com/hltcoe/HC4) | 0.0790 | 0.0610 | 0.0890 | -| [HC4 (Russian): test-topic description+title](https://github.com/hltcoe/HC4) | 0.0900 | 0.0750 | 0.0980 | +| [HC4 (Russian): test-topic title](https://github.com/hltcoe/HC4) | 0.0860 | 0.0720 | 0.0930 | +| [HC4 (Russian): test-topic description](https://github.com/hltcoe/HC4) | 0.0790 | 0.0620 | 0.0890 | +| [HC4 (Russian): test-topic description+title](https://github.com/hltcoe/HC4) | 0.0900 | 0.0760 | 0.0980 | | **Recall@1000** | **BM25 (default)**| **+RM3** | **+Rocchio**| -| [HC4 (Russian): test-topic title](https://github.com/hltcoe/HC4) | 0.6319 | 0.6154 | 0.6887 | +| [HC4 (Russian): test-topic title](https://github.com/hltcoe/HC4) | 0.6319 | 0.6154 | 0.6982 | | [HC4 (Russian): test-topic description](https://github.com/hltcoe/HC4) | 0.6640 | 0.5408 | 0.6407 | -| [HC4 (Russian): test-topic description+title](https://github.com/hltcoe/HC4) | 0.6667 | 0.6221 | 0.6743 | +| [HC4 (Russian): test-topic description+title](https://github.com/hltcoe/HC4) | 0.6667 | 0.6254 | 0.6810 | ## Reproduction Log[*](reproducibility.md) diff --git a/docs/regressions-hc4-v1.0-ru.md b/docs/regressions-hc4-v1.0-ru.md index adf1b34fdb..a873453c5b 100644 --- a/docs/regressions-hc4-v1.0-ru.md +++ b/docs/regressions-hc4-v1.0-ru.md @@ -246,32 +246,32 @@ With the above commands, you should be able to reproduce the following results: | **MAP** | **BM25 (default)**| **+RM3** | **+Rocchio**| |:-------------------------------------------------------------------------------------------------------------|-----------|-----------|-----------| | [HC4 (Russian): dev-topic title](https://github.com/hltcoe/HC4) | 0.2937 | 0.2390 | 0.3995 | -| [HC4 (Russian): dev-topic description](https://github.com/hltcoe/HC4) | 0.2374 | 0.0844 | 0.2817 | -| [HC4 (Russian): dev-topic description+title](https://github.com/hltcoe/HC4) | 0.3209 | 0.2150 | 0.3565 | -| [HC4 (Russian): test-topic title](https://github.com/hltcoe/HC4) | 0.2186 | 0.2369 | 0.2592 | -| [HC4 (Russian): test-topic description](https://github.com/hltcoe/HC4) | 0.1880 | 0.1874 | 0.2252 | -| [HC4 (Russian): test-topic description+title](https://github.com/hltcoe/HC4) | 0.2267 | 0.2290 | 0.2703 | +| [HC4 (Russian): dev-topic description](https://github.com/hltcoe/HC4) | 0.2373 | 0.0844 | 0.2817 | +| [HC4 (Russian): dev-topic description+title](https://github.com/hltcoe/HC4) | 0.3186 | 0.2116 | 0.3564 | +| [HC4 (Russian): test-topic title](https://github.com/hltcoe/HC4) | 0.2186 | 0.2371 | 0.2641 | +| [HC4 (Russian): test-topic description](https://github.com/hltcoe/HC4) | 0.1883 | 0.1868 | 0.2250 | +| [HC4 (Russian): test-topic description+title](https://github.com/hltcoe/HC4) | 0.2265 | 0.2302 | 0.2732 | | **nDCG@20** | **BM25 (default)**| **+RM3** | **+Rocchio**| | [HC4 (Russian): dev-topic title](https://github.com/hltcoe/HC4) | 0.3942 | 0.3376 | 0.4719 | | [HC4 (Russian): dev-topic description](https://github.com/hltcoe/HC4) | 0.2580 | 0.1838 | 0.3168 | -| [HC4 (Russian): dev-topic description+title](https://github.com/hltcoe/HC4) | 0.3993 | 0.3412 | 0.4400 | -| [HC4 (Russian): test-topic title](https://github.com/hltcoe/HC4) | 0.2954 | 0.3200 | 0.3108 | -| [HC4 (Russian): test-topic description](https://github.com/hltcoe/HC4) | 0.2446 | 0.2402 | 0.2759 | -| [HC4 (Russian): test-topic description+title](https://github.com/hltcoe/HC4) | 0.2983 | 0.2955 | 0.3234 | +| [HC4 (Russian): dev-topic description+title](https://github.com/hltcoe/HC4) | 0.3972 | 0.3367 | 0.4400 | +| [HC4 (Russian): test-topic title](https://github.com/hltcoe/HC4) | 0.2944 | 0.3201 | 0.3163 | +| [HC4 (Russian): test-topic description](https://github.com/hltcoe/HC4) | 0.2456 | 0.2396 | 0.2767 | +| [HC4 (Russian): test-topic description+title](https://github.com/hltcoe/HC4) | 0.2989 | 0.2994 | 0.3265 | | **J@20** | **BM25 (default)**| **+RM3** | **+Rocchio**| | [HC4 (Russian): dev-topic title](https://github.com/hltcoe/HC4) | 0.4375 | 0.4500 | 0.5125 | | [HC4 (Russian): dev-topic description](https://github.com/hltcoe/HC4) | 0.5125 | 0.3625 | 0.5500 | | [HC4 (Russian): dev-topic description+title](https://github.com/hltcoe/HC4) | 0.5000 | 0.4625 | 0.5875 | -| [HC4 (Russian): test-topic title](https://github.com/hltcoe/HC4) | 0.3480 | 0.3620 | 0.3950 | -| [HC4 (Russian): test-topic description](https://github.com/hltcoe/HC4) | 0.3180 | 0.2960 | 0.3510 | -| [HC4 (Russian): test-topic description+title](https://github.com/hltcoe/HC4) | 0.3650 | 0.3520 | 0.3960 | +| [HC4 (Russian): test-topic title](https://github.com/hltcoe/HC4) | 0.3470 | 0.3620 | 0.3930 | +| [HC4 (Russian): test-topic description](https://github.com/hltcoe/HC4) | 0.3180 | 0.2970 | 0.3510 | +| [HC4 (Russian): test-topic description+title](https://github.com/hltcoe/HC4) | 0.3670 | 0.3520 | 0.3990 | | **Recall@1000** | **BM25 (default)**| **+RM3** | **+Rocchio**| | [HC4 (Russian): dev-topic title](https://github.com/hltcoe/HC4) | 0.8432 | 0.7598 | 0.8710 | | [HC4 (Russian): dev-topic description](https://github.com/hltcoe/HC4) | 0.5942 | 0.3886 | 0.6171 | | [HC4 (Russian): dev-topic description+title](https://github.com/hltcoe/HC4) | 0.7619 | 0.6428 | 0.7639 | -| [HC4 (Russian): test-topic title](https://github.com/hltcoe/HC4) | 0.7182 | 0.7223 | 0.7713 | -| [HC4 (Russian): test-topic description](https://github.com/hltcoe/HC4) | 0.7355 | 0.6475 | 0.7669 | -| [HC4 (Russian): test-topic description+title](https://github.com/hltcoe/HC4) | 0.7721 | 0.7273 | 0.8230 | +| [HC4 (Russian): test-topic title](https://github.com/hltcoe/HC4) | 0.7182 | 0.7223 | 0.7728 | +| [HC4 (Russian): test-topic description](https://github.com/hltcoe/HC4) | 0.7355 | 0.6480 | 0.7680 | +| [HC4 (Russian): test-topic description+title](https://github.com/hltcoe/HC4) | 0.7721 | 0.7273 | 0.8271 | ## Reproduction Log[*](reproducibility.md) diff --git a/docs/regressions-mrtydi-v1.1-fi.md b/docs/regressions-mrtydi-v1.1-fi.md index 3bedb53772..e14d16e557 100644 --- a/docs/regressions-mrtydi-v1.1-fi.md +++ b/docs/regressions-mrtydi-v1.1-fi.md @@ -68,9 +68,9 @@ With the above commands, you should be able to reproduce the following results: | **MRR@100** | **BM25** | |:-------------------------------------------------------------------------------------------------------------|-----------| | [Mr. TyDi (Finnish): train](https://github.com/castorini/mr.tydi) | 0.4101 | -| [Mr. TyDi (Finnish): dev](https://github.com/castorini/mr.tydi) | 0.4133 | +| [Mr. TyDi (Finnish): dev](https://github.com/castorini/mr.tydi) | 0.4136 | | [Mr. TyDi (Finnish): test](https://github.com/castorini/mr.tydi) | 0.2836 | | **R@100** | **BM25** | | [Mr. TyDi (Finnish): train](https://github.com/castorini/mr.tydi) | 0.8198 | | [Mr. TyDi (Finnish): dev](https://github.com/castorini/mr.tydi) | 0.8285 | -| [Mr. TyDi (Finnish): test](https://github.com/castorini/mr.tydi) | 0.7193 | +| [Mr. TyDi (Finnish): test](https://github.com/castorini/mr.tydi) | 0.7196 | diff --git a/docs/regressions-mrtydi-v1.1-ja.md b/docs/regressions-mrtydi-v1.1-ja.md index dd7b02514c..6223e48f5c 100644 --- a/docs/regressions-mrtydi-v1.1-ja.md +++ b/docs/regressions-mrtydi-v1.1-ja.md @@ -67,10 +67,10 @@ With the above commands, you should be able to reproduce the following results: | **MRR@100** | **BM25** | |:-------------------------------------------------------------------------------------------------------------|-----------| -| [Mr. TyDi (Japanese): train](https://github.com/castorini/mr.tydi) | 0.2236 | -| [Mr. TyDi (Japanese): dev](https://github.com/castorini/mr.tydi) | 0.2241 | -| [Mr. TyDi (Japanese): test](https://github.com/castorini/mr.tydi) | 0.2112 | +| [Mr. TyDi (Japanese): train](https://github.com/castorini/mr.tydi) | 0.2262 | +| [Mr. TyDi (Japanese): dev](https://github.com/castorini/mr.tydi) | 0.2250 | +| [Mr. TyDi (Japanese): test](https://github.com/castorini/mr.tydi) | 0.2125 | | **R@100** | **BM25** | -| [Mr. TyDi (Japanese): train](https://github.com/castorini/mr.tydi) | 0.7282 | -| [Mr. TyDi (Japanese): dev](https://github.com/castorini/mr.tydi) | 0.7274 | -| [Mr. TyDi (Japanese): test](https://github.com/castorini/mr.tydi) | 0.6451 | +| [Mr. TyDi (Japanese): train](https://github.com/castorini/mr.tydi) | 0.7290 | +| [Mr. TyDi (Japanese): dev](https://github.com/castorini/mr.tydi) | 0.7252 | +| [Mr. TyDi (Japanese): test](https://github.com/castorini/mr.tydi) | 0.6431 | diff --git a/docs/regressions-mrtydi-v1.1-ru.md b/docs/regressions-mrtydi-v1.1-ru.md index df8af8f3af..e7de862317 100644 --- a/docs/regressions-mrtydi-v1.1-ru.md +++ b/docs/regressions-mrtydi-v1.1-ru.md @@ -67,10 +67,10 @@ With the above commands, you should be able to reproduce the following results: | **MRR@100** | **BM25** | |:-------------------------------------------------------------------------------------------------------------|-----------| -| [Mr. TyDi (Russian): train](https://github.com/castorini/mr.tydi) | 0.2205 | -| [Mr. TyDi (Russian): dev](https://github.com/castorini/mr.tydi) | 0.2152 | -| [Mr. TyDi (Russian): test](https://github.com/castorini/mr.tydi) | 0.3129 | +| [Mr. TyDi (Russian): train](https://github.com/castorini/mr.tydi) | 0.2229 | +| [Mr. TyDi (Russian): dev](https://github.com/castorini/mr.tydi) | 0.2202 | +| [Mr. TyDi (Russian): test](https://github.com/castorini/mr.tydi) | 0.3163 | | **R@100** | **BM25** | -| [Mr. TyDi (Russian): train](https://github.com/castorini/mr.tydi) | 0.5706 | -| [Mr. TyDi (Russian): dev](https://github.com/castorini/mr.tydi) | 0.5673 | -| [Mr. TyDi (Russian): test](https://github.com/castorini/mr.tydi) | 0.6482 | +| [Mr. TyDi (Russian): train](https://github.com/castorini/mr.tydi) | 0.5779 | +| [Mr. TyDi (Russian): dev](https://github.com/castorini/mr.tydi) | 0.5760 | +| [Mr. TyDi (Russian): test](https://github.com/castorini/mr.tydi) | 0.6541 | diff --git a/docs/regressions-msmarco-doc.md b/docs/regressions-msmarco-doc.md index e901284e2c..b01548885e 100644 --- a/docs/regressions-msmarco-doc.md +++ b/docs/regressions-msmarco-doc.md @@ -277,9 +277,9 @@ With the above commands, you should be able to reproduce the following results: | **AP@1000** | **BM25 (default)**| **+RM3** | **+Rocchio**| **+Rocchio***| **+Ax** | **+PRF** | **BM25 (tuned)**| **+RM3** | **+Rocchio**| **+Rocchio***| **+Ax** | **+PRF** | **BM25 (tuned2)**| **+RM3** | **+Rocchio**| **+Rocchio***| **+Ax** | **+PRF** | |:-------------------------------------------------------------------------------------------------------------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------| -| [MS MARCO Doc: Dev](https://github.com/microsoft/MSMARCO-Document-Ranking) | 0.2305 | 0.1631 | 0.1632 | 0.1630 | 0.1146 | 0.1357 | 0.2784 | 0.2289 | 0.2280 | 0.2271 | 0.1888 | 0.1559 | 0.2774 | 0.2239 | 0.2248 | 0.2231 | 0.1886 | 0.1530 | +| [MS MARCO Doc: Dev](https://github.com/microsoft/MSMARCO-Document-Ranking) | 0.2305 | 0.1631 | 0.1632 | 0.1630 | 0.1146 | 0.1357 | 0.2784 | 0.2289 | 0.2280 | 0.2271 | 0.1888 | 0.1559 | 0.2773 | 0.2239 | 0.2248 | 0.2231 | 0.1886 | 0.1530 | | **RR@100** | **BM25 (default)**| **+RM3** | **+Rocchio**| **+Rocchio***| **+Ax** | **+PRF** | **BM25 (tuned)**| **+RM3** | **+Rocchio**| **+Rocchio***| **+Ax** | **+PRF** | **BM25 (tuned2)**| **+RM3** | **+Rocchio**| **+Rocchio***| **+Ax** | **+PRF** | -| [MS MARCO Doc: Dev](https://github.com/microsoft/MSMARCO-Document-Ranking) | 0.2299 | 0.1622 | 0.1624 | 0.1622 | 0.1135 | 0.1347 | 0.2778 | 0.2282 | 0.2274 | 0.2264 | 0.1880 | 0.1550 | 0.2768 | 0.2231 | 0.2242 | 0.2224 | 0.1878 | 0.1521 | +| [MS MARCO Doc: Dev](https://github.com/microsoft/MSMARCO-Document-Ranking) | 0.2299 | 0.1622 | 0.1624 | 0.1622 | 0.1135 | 0.1347 | 0.2778 | 0.2282 | 0.2274 | 0.2264 | 0.1880 | 0.1550 | 0.2767 | 0.2231 | 0.2242 | 0.2224 | 0.1877 | 0.1521 | | **R@100** | **BM25 (default)**| **+RM3** | **+Rocchio**| **+Rocchio***| **+Ax** | **+PRF** | **BM25 (tuned)**| **+RM3** | **+Rocchio**| **+Rocchio***| **+Ax** | **+PRF** | **BM25 (tuned2)**| **+RM3** | **+Rocchio**| **+Rocchio***| **+Ax** | **+PRF** | | [MS MARCO Doc: Dev](https://github.com/microsoft/MSMARCO-Document-Ranking) | 0.7281 | 0.6767 | 0.6763 | 0.6792 | 0.5754 | 0.6374 | 0.8069 | 0.7878 | 0.7901 | 0.7922 | 0.7560 | 0.6852 | 0.8070 | 0.7791 | 0.7878 | 0.7863 | 0.7526 | 0.6825 | | **R@1000** | **BM25 (default)**| **+RM3** | **+Rocchio**| **+Rocchio***| **+Ax** | **+PRF** | **BM25 (tuned)**| **+RM3** | **+Rocchio**| **+Rocchio***| **+Ax** | **+PRF** | **BM25 (tuned2)**| **+RM3** | **+Rocchio**| **+Rocchio***| **+Ax** | **+PRF** | diff --git a/src/main/resources/regression/beir-v1.0.0-bioasq-flat.yaml b/src/main/resources/regression/beir-v1.0.0-bioasq-flat.yaml index 319fd0491a..8ee3205395 100644 --- a/src/main/resources/regression/beir-v1.0.0-bioasq-flat.yaml +++ b/src/main/resources/regression/beir-v1.0.0-bioasq-flat.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 14914603 documents (non-empty): 14914602 - total terms: 2257541768 + total terms: 2257541758 metrics: - metric: nDCG@10 diff --git a/src/main/resources/regression/beir-v1.0.0-bioasq-multifield.yaml b/src/main/resources/regression/beir-v1.0.0-bioasq-multifield.yaml index 0a6f0233df..e96a3c1f2b 100644 --- a/src/main/resources/regression/beir-v1.0.0-bioasq-multifield.yaml +++ b/src/main/resources/regression/beir-v1.0.0-bioasq-multifield.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw -fields title index_stats: documents: 14914602 documents (non-empty): 14914585 - total terms: 2099554317 + total terms: 2099554307 metrics: - metric: nDCG@10 diff --git a/src/main/resources/regression/beir-v1.0.0-climate-fever-flat.yaml b/src/main/resources/regression/beir-v1.0.0-climate-fever-flat.yaml index 234ed8bb76..42d30d5a19 100644 --- a/src/main/resources/regression/beir-v1.0.0-climate-fever-flat.yaml +++ b/src/main/resources/regression/beir-v1.0.0-climate-fever-flat.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 5416593 documents (non-empty): 5416593 - total terms: 325185077 + total terms: 325185072 metrics: - metric: nDCG@10 diff --git a/src/main/resources/regression/beir-v1.0.0-climate-fever-multifield.yaml b/src/main/resources/regression/beir-v1.0.0-climate-fever-multifield.yaml index fba2b5e1f9..f6b2427994 100644 --- a/src/main/resources/regression/beir-v1.0.0-climate-fever-multifield.yaml +++ b/src/main/resources/regression/beir-v1.0.0-climate-fever-multifield.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw -fields title index_stats: documents: 5396163 documents (non-empty): 5396117 - total terms: 310661482 + total terms: 310661477 metrics: - metric: nDCG@10 diff --git a/src/main/resources/regression/beir-v1.0.0-cqadupstack-android-flat.yaml b/src/main/resources/regression/beir-v1.0.0-cqadupstack-android-flat.yaml index b4a86191eb..5d07831208 100644 --- a/src/main/resources/regression/beir-v1.0.0-cqadupstack-android-flat.yaml +++ b/src/main/resources/regression/beir-v1.0.0-cqadupstack-android-flat.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 22998 documents (non-empty): 22998 - total terms: 1760761 + total terms: 1760762 metrics: - metric: nDCG@10 diff --git a/src/main/resources/regression/beir-v1.0.0-cqadupstack-android-multifield.yaml b/src/main/resources/regression/beir-v1.0.0-cqadupstack-android-multifield.yaml index b74f66fa99..3456bf42c4 100644 --- a/src/main/resources/regression/beir-v1.0.0-cqadupstack-android-multifield.yaml +++ b/src/main/resources/regression/beir-v1.0.0-cqadupstack-android-multifield.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw -fields title index_stats: documents: 22998 documents (non-empty): 22998 - total terms: 1591284 + total terms: 1591285 metrics: - metric: nDCG@10 diff --git a/src/main/resources/regression/beir-v1.0.0-cqadupstack-tex-flat.yaml b/src/main/resources/regression/beir-v1.0.0-cqadupstack-tex-flat.yaml index faa0ef90e4..ff4f5995cb 100644 --- a/src/main/resources/regression/beir-v1.0.0-cqadupstack-tex-flat.yaml +++ b/src/main/resources/regression/beir-v1.0.0-cqadupstack-tex-flat.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 68184 documents (non-empty): 68184 - total terms: 9556422 + total terms: 9556423 metrics: - metric: nDCG@10 diff --git a/src/main/resources/regression/beir-v1.0.0-cqadupstack-tex-multifield.yaml b/src/main/resources/regression/beir-v1.0.0-cqadupstack-tex-multifield.yaml index c0234fe486..a74cecc282 100644 --- a/src/main/resources/regression/beir-v1.0.0-cqadupstack-tex-multifield.yaml +++ b/src/main/resources/regression/beir-v1.0.0-cqadupstack-tex-multifield.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw -fields title index_stats: documents: 68184 documents (non-empty): 68184 - total terms: 9155404 + total terms: 9155405 metrics: - metric: nDCG@10 diff --git a/src/main/resources/regression/beir-v1.0.0-dbpedia-entity-flat.yaml b/src/main/resources/regression/beir-v1.0.0-dbpedia-entity-flat.yaml index 8d7f6caa58..5f20408181 100644 --- a/src/main/resources/regression/beir-v1.0.0-dbpedia-entity-flat.yaml +++ b/src/main/resources/regression/beir-v1.0.0-dbpedia-entity-flat.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 4635922 documents (non-empty): 4635922 - total terms: 164794987 + total terms: 164794982 metrics: - metric: nDCG@10 diff --git a/src/main/resources/regression/beir-v1.0.0-dbpedia-entity-multifield.yaml b/src/main/resources/regression/beir-v1.0.0-dbpedia-entity-multifield.yaml index 2f13af5f61..2b46303f2b 100644 --- a/src/main/resources/regression/beir-v1.0.0-dbpedia-entity-multifield.yaml +++ b/src/main/resources/regression/beir-v1.0.0-dbpedia-entity-multifield.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw -fields title index_stats: documents: 4635922 documents (non-empty): 4635863 - total terms: 152205484 + total terms: 152205479 metrics: - metric: nDCG@10 diff --git a/src/main/resources/regression/beir-v1.0.0-fever-flat.yaml b/src/main/resources/regression/beir-v1.0.0-fever-flat.yaml index 025716aac9..a3f296793d 100644 --- a/src/main/resources/regression/beir-v1.0.0-fever-flat.yaml +++ b/src/main/resources/regression/beir-v1.0.0-fever-flat.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 5416568 documents (non-empty): 5416568 - total terms: 325179170 + total terms: 325179165 metrics: - metric: nDCG@10 diff --git a/src/main/resources/regression/beir-v1.0.0-fever-multifield.yaml b/src/main/resources/regression/beir-v1.0.0-fever-multifield.yaml index 9c691d3815..793ce0b958 100644 --- a/src/main/resources/regression/beir-v1.0.0-fever-multifield.yaml +++ b/src/main/resources/regression/beir-v1.0.0-fever-multifield.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw -fields title index_stats: documents: 5396138 documents (non-empty): 5396092 - total terms: 310655704 + total terms: 310655699 metrics: - metric: nDCG@10 diff --git a/src/main/resources/regression/beir-v1.0.0-hotpotqa-flat.yaml b/src/main/resources/regression/beir-v1.0.0-hotpotqa-flat.yaml index 692ce3cb22..ff1108bdb2 100644 --- a/src/main/resources/regression/beir-v1.0.0-hotpotqa-flat.yaml +++ b/src/main/resources/regression/beir-v1.0.0-hotpotqa-flat.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 5233329 documents (non-empty): 5233328 - total terms: 172477063 + total terms: 172477066 metrics: - metric: nDCG@10 diff --git a/src/main/resources/regression/beir-v1.0.0-hotpotqa-multifield.yaml b/src/main/resources/regression/beir-v1.0.0-hotpotqa-multifield.yaml index 194e554ca4..d3cd71b70b 100644 --- a/src/main/resources/regression/beir-v1.0.0-hotpotqa-multifield.yaml +++ b/src/main/resources/regression/beir-v1.0.0-hotpotqa-multifield.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw -fields title index_stats: documents: 5233235 documents (non-empty): 5233230 - total terms: 158180689 + total terms: 158180692 metrics: - metric: nDCG@10 diff --git a/src/main/resources/regression/beir-v1.0.0-nq-flat.yaml b/src/main/resources/regression/beir-v1.0.0-nq-flat.yaml index 48660425ac..3f33f3a691 100644 --- a/src/main/resources/regression/beir-v1.0.0-nq-flat.yaml +++ b/src/main/resources/regression/beir-v1.0.0-nq-flat.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 2681468 documents (non-empty): 2681468 - total terms: 151249287 + total terms: 151249294 metrics: - metric: nDCG@10 diff --git a/src/main/resources/regression/beir-v1.0.0-nq-multifield.yaml b/src/main/resources/regression/beir-v1.0.0-nq-multifield.yaml index bfea73938d..77b2723d3c 100644 --- a/src/main/resources/regression/beir-v1.0.0-nq-multifield.yaml +++ b/src/main/resources/regression/beir-v1.0.0-nq-multifield.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw -fields title index_stats: documents: 2680961 documents (non-empty): 2680763 - total terms: 144050884 + total terms: 144050891 metrics: - metric: nDCG@10 diff --git a/src/main/resources/regression/beir-v1.0.0-signal1m-flat.yaml b/src/main/resources/regression/beir-v1.0.0-signal1m-flat.yaml index 5b20c340e2..2a3a16f0e7 100644 --- a/src/main/resources/regression/beir-v1.0.0-signal1m-flat.yaml +++ b/src/main/resources/regression/beir-v1.0.0-signal1m-flat.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 2866315 documents (non-empty): 2866094 - total terms: 32240067 + total terms: 32240069 metrics: - metric: nDCG@10 diff --git a/src/main/resources/regression/beir-v1.0.0-signal1m-multifield.yaml b/src/main/resources/regression/beir-v1.0.0-signal1m-multifield.yaml index 1d4f0c04c3..958ec3623a 100644 --- a/src/main/resources/regression/beir-v1.0.0-signal1m-multifield.yaml +++ b/src/main/resources/regression/beir-v1.0.0-signal1m-multifield.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw -fields title index_stats: documents: 2866315 documents (non-empty): 2866094 - total terms: 32240067 + total terms: 32240069 metrics: - metric: nDCG@10 diff --git a/src/main/resources/regression/beir-v1.0.0-trec-covid-flat.yaml b/src/main/resources/regression/beir-v1.0.0-trec-covid-flat.yaml index 5df02bf85d..e25b390f07 100644 --- a/src/main/resources/regression/beir-v1.0.0-trec-covid-flat.yaml +++ b/src/main/resources/regression/beir-v1.0.0-trec-covid-flat.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 171331 documents (non-empty): 171331 - total terms: 20822810 + total terms: 20822821 metrics: - metric: nDCG@10 diff --git a/src/main/resources/regression/beir-v1.0.0-trec-covid-multifield.yaml b/src/main/resources/regression/beir-v1.0.0-trec-covid-multifield.yaml index 763b95b3ea..81032b6ccd 100644 --- a/src/main/resources/regression/beir-v1.0.0-trec-covid-multifield.yaml +++ b/src/main/resources/regression/beir-v1.0.0-trec-covid-multifield.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw -fields title index_stats: documents: 129192 documents (non-empty): 129184 - total terms: 19060111 + total terms: 19060122 metrics: - metric: nDCG@10 diff --git a/src/main/resources/regression/car17v1.5.yaml b/src/main/resources/regression/car17v1.5.yaml index 4ddbea3337..096e6be4f6 100644 --- a/src/main/resources/regression/car17v1.5.yaml +++ b/src/main/resources/regression/car17v1.5.yaml @@ -9,8 +9,8 @@ index_threads: 1 index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 29678360 - documents (non-empty): 29674425 - total terms: 1257909884 + documents (non-empty): 29674431 + total terms: 1257909856 metrics: - metric: MAP diff --git a/src/main/resources/regression/car17v2.0-doc2query.yaml b/src/main/resources/regression/car17v2.0-doc2query.yaml index 6a52c9c5d1..c2b7601d9e 100644 --- a/src/main/resources/regression/car17v2.0-doc2query.yaml +++ b/src/main/resources/regression/car17v2.0-doc2query.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 29794697 documents (non-empty): 29794694 - total terms: 2541082416 + total terms: 2541082328 metrics: - metric: MAP diff --git a/src/main/resources/regression/car17v2.0.yaml b/src/main/resources/regression/car17v2.0.yaml index 95ae7e54b9..27ae53306b 100644 --- a/src/main/resources/regression/car17v2.0.yaml +++ b/src/main/resources/regression/car17v2.0.yaml @@ -9,8 +9,8 @@ index_threads: 1 index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 29794689 - documents (non-empty): 29791059 - total terms: 1249754054 + documents (non-empty): 29791065 + total terms: 1249754017 metrics: - metric: MAP diff --git a/src/main/resources/regression/clef06-fr.yaml b/src/main/resources/regression/clef06-fr.yaml index 731aff95d5..9cbd12ecb7 100644 --- a/src/main/resources/regression/clef06-fr.yaml +++ b/src/main/resources/regression/clef06-fr.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw -language fr index_stats: documents: 171109 documents (non-empty): 171109 - total terms: 34352833 + total terms: 35303467 metrics: - metric: MAP @@ -50,8 +50,8 @@ models: params: -bm25 -language fr results: MAP: - - 0.3111 + - 0.3115 P20: - 0.3184 nDCG@20: - - 0.4458 + - 0.4457 diff --git a/src/main/resources/regression/cw09b.yaml b/src/main/resources/regression/cw09b.yaml index 55f84805c7..1127041741 100644 --- a/src/main/resources/regression/cw09b.yaml +++ b/src/main/resources/regression/cw09b.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 50220186 documents (non-empty): 50220156 - total terms: 31300822176 + total terms: 31300873283 metrics: - metric: MAP diff --git a/src/main/resources/regression/cw12.yaml b/src/main/resources/regression/cw12.yaml index 754f52d502..d8fe9c860f 100644 --- a/src/main/resources/regression/cw12.yaml +++ b/src/main/resources/regression/cw12.yaml @@ -9,8 +9,8 @@ index_threads: 44 index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 731645141 - documents (non-empty): 731542236 - total terms: 429234508918 + documents (non-empty): 731542315 + total terms: 429236660701 metrics: - metric: MAP diff --git a/src/main/resources/regression/cw12b13.yaml b/src/main/resources/regression/cw12b13.yaml index 284ae49312..0c52052d9f 100644 --- a/src/main/resources/regression/cw12b13.yaml +++ b/src/main/resources/regression/cw12b13.yaml @@ -9,8 +9,8 @@ index_threads: 44 index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 52244809 - documents (non-empty): 52237520 - total terms: 30660015721 + documents (non-empty): 52237521 + total terms: 30660169333 metrics: - metric: MAP diff --git a/src/main/resources/regression/dl19-doc-docTTTTTquery.yaml b/src/main/resources/regression/dl19-doc-docTTTTTquery.yaml index d9066e8931..c03b4214cb 100644 --- a/src/main/resources/regression/dl19-doc-docTTTTTquery.yaml +++ b/src/main/resources/regression/dl19-doc-docTTTTTquery.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 3213835 documents (non-empty): 3213835 - total terms: 3748333319 + total terms: 3748343494 metrics: - metric: AP@100 diff --git a/src/main/resources/regression/dl19-doc-segmented-docTTTTTquery.yaml b/src/main/resources/regression/dl19-doc-segmented-docTTTTTquery.yaml index 919277a503..e258060088 100644 --- a/src/main/resources/regression/dl19-doc-segmented-docTTTTTquery.yaml +++ b/src/main/resources/regression/dl19-doc-segmented-docTTTTTquery.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 20545677 documents (non-empty): 20545677 - total terms: 4206639543 + total terms: 4206646183 metrics: - metric: AP@100 diff --git a/src/main/resources/regression/dl19-doc-segmented.yaml b/src/main/resources/regression/dl19-doc-segmented.yaml index 2031df273b..50c208d331 100644 --- a/src/main/resources/regression/dl19-doc-segmented.yaml +++ b/src/main/resources/regression/dl19-doc-segmented.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 20545677 documents (non-empty): 20545677 - total terms: 3200515914 + total terms: 3200522554 metrics: - metric: AP@100 diff --git a/src/main/resources/regression/dl19-doc.yaml b/src/main/resources/regression/dl19-doc.yaml index 273ad94126..12402b0dbd 100644 --- a/src/main/resources/regression/dl19-doc.yaml +++ b/src/main/resources/regression/dl19-doc.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 3213835 documents (non-empty): 3213835 - total terms: 2742209690 + total terms: 2742219865 metrics: - metric: AP@100 diff --git a/src/main/resources/regression/dl20-doc-docTTTTTquery.yaml b/src/main/resources/regression/dl20-doc-docTTTTTquery.yaml index 554eb98f78..943a6b5732 100644 --- a/src/main/resources/regression/dl20-doc-docTTTTTquery.yaml +++ b/src/main/resources/regression/dl20-doc-docTTTTTquery.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 3213835 documents (non-empty): 3213835 - total terms: 3748333319 + total terms: 3748343494 metrics: - metric: AP@100 diff --git a/src/main/resources/regression/dl20-doc-segmented-docTTTTTquery.yaml b/src/main/resources/regression/dl20-doc-segmented-docTTTTTquery.yaml index 0c61d523c7..d538bf12a3 100644 --- a/src/main/resources/regression/dl20-doc-segmented-docTTTTTquery.yaml +++ b/src/main/resources/regression/dl20-doc-segmented-docTTTTTquery.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 20545677 documents (non-empty): 20545677 - total terms: 4206639543 + total terms: 4206646183 metrics: - metric: AP@100 diff --git a/src/main/resources/regression/dl20-doc-segmented.yaml b/src/main/resources/regression/dl20-doc-segmented.yaml index 7d17970c6f..3fd37d5fcd 100644 --- a/src/main/resources/regression/dl20-doc-segmented.yaml +++ b/src/main/resources/regression/dl20-doc-segmented.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 20545677 documents (non-empty): 20545677 - total terms: 3200515914 + total terms: 3200522554 metrics: - metric: AP@100 diff --git a/src/main/resources/regression/dl20-doc.yaml b/src/main/resources/regression/dl20-doc.yaml index 89641c9792..3c47217a0f 100644 --- a/src/main/resources/regression/dl20-doc.yaml +++ b/src/main/resources/regression/dl20-doc.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 3213835 documents (non-empty): 3213835 - total terms: 2742209690 + total terms: 2742219865 metrics: - metric: AP@100 diff --git a/src/main/resources/regression/dl21-doc-d2q-t5.yaml b/src/main/resources/regression/dl21-doc-d2q-t5.yaml index 6bb1bbeba5..3027f36655 100644 --- a/src/main/resources/regression/dl21-doc-d2q-t5.yaml +++ b/src/main/resources/regression/dl21-doc-d2q-t5.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 11959635 documents (non-empty): 11959635 - total terms: 19760777295 + total terms: 19760783236 metrics: - metric: MAP@100 diff --git a/src/main/resources/regression/dl21-doc-segmented-d2q-t5.yaml b/src/main/resources/regression/dl21-doc-segmented-d2q-t5.yaml index 372640716f..b5b55370d6 100644 --- a/src/main/resources/regression/dl21-doc-segmented-d2q-t5.yaml +++ b/src/main/resources/regression/dl21-doc-segmented-d2q-t5.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 124131414 documents (non-empty): 124131414 - total terms: 30376032067 + total terms: 30376034132 metrics: - metric: MAP@100 diff --git a/src/main/resources/regression/dl21-doc-segmented.yaml b/src/main/resources/regression/dl21-doc-segmented.yaml index d04d2434b0..62557d9752 100644 --- a/src/main/resources/regression/dl21-doc-segmented.yaml +++ b/src/main/resources/regression/dl21-doc-segmented.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 124131414 documents (non-empty): 124131414 - total terms: 24780915974 + total terms: 24780918039 metrics: - metric: MAP@100 diff --git a/src/main/resources/regression/dl21-doc.yaml b/src/main/resources/regression/dl21-doc.yaml index b67764a3d7..14861f6873 100644 --- a/src/main/resources/regression/dl21-doc.yaml +++ b/src/main/resources/regression/dl21-doc.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 11959635 documents (non-empty): 11959635 - total terms: 14165661202 + total terms: 14165667143 metrics: - metric: MAP@100 diff --git a/src/main/resources/regression/dl21-passage-augmented-d2q-t5.yaml b/src/main/resources/regression/dl21-passage-augmented-d2q-t5.yaml index 05d4301a3a..ae19e5826c 100644 --- a/src/main/resources/regression/dl21-passage-augmented-d2q-t5.yaml +++ b/src/main/resources/regression/dl21-passage-augmented-d2q-t5.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 138364198 documents (non-empty): 138364198 - total terms: 27561177420 + total terms: 27561177716 metrics: - metric: MAP@100 diff --git a/src/main/resources/regression/dl21-passage-augmented.yaml b/src/main/resources/regression/dl21-passage-augmented.yaml index 929a910f22..8a0963c871 100644 --- a/src/main/resources/regression/dl21-passage-augmented.yaml +++ b/src/main/resources/regression/dl21-passage-augmented.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 138364198 documents (non-empty): 138364198 - total terms: 15272964956 + total terms: 15272965252 metrics: - metric: MAP@100 diff --git a/src/main/resources/regression/dl21-passage-d2q-t5.yaml b/src/main/resources/regression/dl21-passage-d2q-t5.yaml index 2b2d27aff4..f83b0c07e9 100644 --- a/src/main/resources/regression/dl21-passage-d2q-t5.yaml +++ b/src/main/resources/regression/dl21-passage-d2q-t5.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 138364198 documents (non-empty): 138364198 - total terms: 16961479226 + total terms: 16961479264 metrics: - metric: MAP@100 diff --git a/src/main/resources/regression/dl21-passage.yaml b/src/main/resources/regression/dl21-passage.yaml index 0af33d8698..101df98f61 100644 --- a/src/main/resources/regression/dl21-passage.yaml +++ b/src/main/resources/regression/dl21-passage.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 138364198 documents (non-empty): 138364197 - total terms: 4673266762 + total terms: 4673266800 metrics: - metric: MAP@100 diff --git a/src/main/resources/regression/fever.yaml b/src/main/resources/regression/fever.yaml index 73e5d90b53..e2768af541 100644 --- a/src/main/resources/regression/fever.yaml +++ b/src/main/resources/regression/fever.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 5396106 documents (non-empty): 5396060 - total terms: 322660819 + total terms: 322660814 metrics: - metric: R@100 diff --git a/src/main/resources/regression/gov2.yaml b/src/main/resources/regression/gov2.yaml index 477865555b..784ebffcd2 100644 --- a/src/main/resources/regression/gov2.yaml +++ b/src/main/resources/regression/gov2.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 25170853 documents (non-empty): 25170665 - total terms: 17345663488 + total terms: 17345805954 metrics: - metric: MAP diff --git a/src/main/resources/regression/hc4-neuclir22-ru.yaml b/src/main/resources/regression/hc4-neuclir22-ru.yaml index 33301ecfd1..17d9731f22 100644 --- a/src/main/resources/regression/hc4-neuclir22-ru.yaml +++ b/src/main/resources/regression/hc4-neuclir22-ru.yaml @@ -64,13 +64,13 @@ models: params: -bm25 -language ru results: MAP: - - 0.0964 - - 0.0926 - - 0.1113 + - 0.1040 + - 0.0963 + - 0.1264 nDCG@20: - - 0.1380 - - 0.1459 - - 0.1640 + - 0.1445 + - 0.1495 + - 0.1762 J@20: - 0.0860 - 0.0790 @@ -84,38 +84,38 @@ models: params: -bm25 -rm3 -language ru results: MAP: - - 0.0811 - - 0.0605 - - 0.0771 + - 0.0841 + - 0.0640 + - 0.0825 nDCG@20: - - 0.1257 - - 0.0963 - - 0.1318 + - 0.1283 + - 0.1037 + - 0.1411 J@20: - - 0.0730 - - 0.0610 - - 0.0750 + - 0.0720 + - 0.0620 + - 0.0760 Recall@1000: - 0.6154 - 0.5408 - - 0.6221 + - 0.6254 - name: bm25-default+rocchio display: +Rocchio params: -bm25 -rocchio -language ru results: MAP: - - 0.1245 - - 0.1064 - - 0.1341 + - 0.1231 + - 0.0964 + - 0.1314 nDCG@20: - - 0.1668 - - 0.1643 - - 0.1899 + - 0.1655 + - 0.1569 + - 0.1875 J@20: - - 0.0940 + - 0.0930 - 0.0890 - 0.0980 Recall@1000: - - 0.6887 + - 0.6982 - 0.6407 - - 0.6743 + - 0.6810 diff --git a/src/main/resources/regression/hc4-v1.0-ru.yaml b/src/main/resources/regression/hc4-v1.0-ru.yaml index 8aefbabb1f..a4709802ae 100644 --- a/src/main/resources/regression/hc4-v1.0-ru.yaml +++ b/src/main/resources/regression/hc4-v1.0-ru.yaml @@ -78,25 +78,25 @@ models: results: MAP: - 0.2937 - - 0.2374 - - 0.3209 + - 0.2373 + - 0.3186 - 0.2186 - - 0.1880 - - 0.2267 + - 0.1883 + - 0.2265 nDCG@20: - 0.3942 - 0.2580 - - 0.3993 - - 0.2954 - - 0.2446 - - 0.2983 + - 0.3972 + - 0.2944 + - 0.2456 + - 0.2989 J@20: - 0.4375 - 0.5125 - 0.5000 - - 0.3480 + - 0.3470 - 0.3180 - - 0.3650 + - 0.3670 Recall@1000: - 0.8432 - 0.5942 @@ -111,30 +111,30 @@ models: MAP: - 0.2390 - 0.0844 - - 0.2150 - - 0.2369 - - 0.1874 - - 0.2290 + - 0.2116 + - 0.2371 + - 0.1868 + - 0.2302 nDCG@20: - 0.3376 - 0.1838 - - 0.3412 - - 0.3200 - - 0.2402 - - 0.2955 + - 0.3367 + - 0.3201 + - 0.2396 + - 0.2994 J@20: - 0.4500 - 0.3625 - 0.4625 - 0.3620 - - 0.2960 + - 0.2970 - 0.3520 Recall@1000: - 0.7598 - 0.3886 - 0.6428 - 0.7223 - - 0.6475 + - 0.6480 - 0.7273 - name: bm25-default+rocchio display: +Rocchio @@ -143,29 +143,29 @@ models: MAP: - 0.3995 - 0.2817 - - 0.3565 - - 0.2592 - - 0.2252 - - 0.2703 + - 0.3564 + - 0.2641 + - 0.2250 + - 0.2732 nDCG@20: - 0.4719 - 0.3168 - 0.4400 - - 0.3108 - - 0.2759 - - 0.3234 + - 0.3163 + - 0.2767 + - 0.3265 J@20: - 0.5125 - 0.5500 - 0.5875 - - 0.3950 + - 0.3930 - 0.3510 - - 0.3960 + - 0.3990 Recall@1000: - 0.8710 - 0.6171 - 0.7639 - - 0.7713 - - 0.7669 - - 0.8230 + - 0.7728 + - 0.7680 + - 0.8271 diff --git a/src/main/resources/regression/mrtydi-v1.1-ar.yaml b/src/main/resources/regression/mrtydi-v1.1-ar.yaml index 122a46af00..174aa6e352 100644 --- a/src/main/resources/regression/mrtydi-v1.1-ar.yaml +++ b/src/main/resources/regression/mrtydi-v1.1-ar.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw -language ar index_stats: documents: 2106586 documents (non-empty): 2106586 - total terms: 92529014 + total terms: 92529032 metrics: - metric: MRR@100 diff --git a/src/main/resources/regression/mrtydi-v1.1-bn.yaml b/src/main/resources/regression/mrtydi-v1.1-bn.yaml index b083bc5706..f846742278 100644 --- a/src/main/resources/regression/mrtydi-v1.1-bn.yaml +++ b/src/main/resources/regression/mrtydi-v1.1-bn.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw -language bn index_stats: documents: 304059 documents (non-empty): 304059 - total terms: 15236598 + total terms: 15236599 metrics: - metric: MRR@100 diff --git a/src/main/resources/regression/mrtydi-v1.1-en.yaml b/src/main/resources/regression/mrtydi-v1.1-en.yaml index 0c703b6c06..ddf4fc769a 100644 --- a/src/main/resources/regression/mrtydi-v1.1-en.yaml +++ b/src/main/resources/regression/mrtydi-v1.1-en.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw -language en index_stats: documents: 32907100 documents (non-empty): 32907100 - total terms: 1507060955 + total terms: 1507060932 metrics: - metric: MRR@100 diff --git a/src/main/resources/regression/mrtydi-v1.1-fi.yaml b/src/main/resources/regression/mrtydi-v1.1-fi.yaml index 73b850ef7b..f8b5656154 100644 --- a/src/main/resources/regression/mrtydi-v1.1-fi.yaml +++ b/src/main/resources/regression/mrtydi-v1.1-fi.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw -language fi index_stats: documents: 1908757 documents (non-empty): 1908757 - total terms: 69431615 + total terms: 69416543 metrics: - metric: MRR@100 @@ -52,9 +52,9 @@ models: results: MRR@100: - 0.4101 - - 0.4133 + - 0.4136 - 0.2836 R@100: - 0.8198 - 0.8285 - - 0.7193 + - 0.7196 diff --git a/src/main/resources/regression/mrtydi-v1.1-ja.yaml b/src/main/resources/regression/mrtydi-v1.1-ja.yaml index a63271c9a6..7f43ecfad2 100644 --- a/src/main/resources/regression/mrtydi-v1.1-ja.yaml +++ b/src/main/resources/regression/mrtydi-v1.1-ja.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw -language ja index_stats: documents: 7000027 documents (non-empty): 7000027 - total terms: 303640353 + total terms: 300761975 metrics: - metric: MRR@100 @@ -51,10 +51,10 @@ models: params: -bm25 -hits 100 -language ja results: MRR@100: - - 0.2236 - - 0.2241 - - 0.2112 + - 0.2262 + - 0.2250 + - 0.2125 R@100: - - 0.7282 - - 0.7274 - - 0.6451 + - 0.7290 + - 0.7252 + - 0.6431 diff --git a/src/main/resources/regression/mrtydi-v1.1-ko.yaml b/src/main/resources/regression/mrtydi-v1.1-ko.yaml index 8265665cbc..dbfffb89a5 100644 --- a/src/main/resources/regression/mrtydi-v1.1-ko.yaml +++ b/src/main/resources/regression/mrtydi-v1.1-ko.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw -language ko index_stats: documents: 1496126 documents (non-empty): 1496126 - total terms: 122217290 + total terms: 122217295 metrics: - metric: MRR@100 diff --git a/src/main/resources/regression/mrtydi-v1.1-ru.yaml b/src/main/resources/regression/mrtydi-v1.1-ru.yaml index 51b8c6fce1..a808138e13 100644 --- a/src/main/resources/regression/mrtydi-v1.1-ru.yaml +++ b/src/main/resources/regression/mrtydi-v1.1-ru.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw -language ru index_stats: documents: 9597504 documents (non-empty): 9597504 - total terms: 346329152 + total terms: 346329117 metrics: - metric: MRR@100 @@ -51,10 +51,10 @@ models: params: -bm25 -hits 100 -language ru results: MRR@100: - - 0.2205 - - 0.2152 - - 0.3129 + - 0.2229 + - 0.2202 + - 0.3163 R@100: - - 0.5706 - - 0.5673 - - 0.6482 + - 0.5779 + - 0.5760 + - 0.6541 diff --git a/src/main/resources/regression/msmarco-doc-docTTTTTquery.yaml b/src/main/resources/regression/msmarco-doc-docTTTTTquery.yaml index b66a21daa1..3557efa369 100644 --- a/src/main/resources/regression/msmarco-doc-docTTTTTquery.yaml +++ b/src/main/resources/regression/msmarco-doc-docTTTTTquery.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 3213835 documents (non-empty): 3213835 - total terms: 3748333319 + total terms: 3748343494 metrics: - metric: AP@1000 diff --git a/src/main/resources/regression/msmarco-doc-segmented-docTTTTTquery.yaml b/src/main/resources/regression/msmarco-doc-segmented-docTTTTTquery.yaml index 544f8e4bb3..ca8883eb72 100644 --- a/src/main/resources/regression/msmarco-doc-segmented-docTTTTTquery.yaml +++ b/src/main/resources/regression/msmarco-doc-segmented-docTTTTTquery.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 20545677 documents (non-empty): 20545677 - total terms: 4206639543 + total terms: 4206646183 metrics: - metric: AP@1000 diff --git a/src/main/resources/regression/msmarco-doc-segmented.yaml b/src/main/resources/regression/msmarco-doc-segmented.yaml index 98ce7c7bfc..bb09b36ec5 100644 --- a/src/main/resources/regression/msmarco-doc-segmented.yaml +++ b/src/main/resources/regression/msmarco-doc-segmented.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 20545677 documents (non-empty): 20545677 - total terms: 3200515914 + total terms: 3200522554 metrics: - metric: AP@1000 diff --git a/src/main/resources/regression/msmarco-doc.yaml b/src/main/resources/regression/msmarco-doc.yaml index 1c629e8c9c..6b9fe28277 100644 --- a/src/main/resources/regression/msmarco-doc.yaml +++ b/src/main/resources/regression/msmarco-doc.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 3213835 documents (non-empty): 3213835 - total terms: 2742209690 + total terms: 2742219865 metrics: - metric: AP@1000 @@ -201,9 +201,9 @@ models: params: -bm25 -bm25.k1 4.46 -bm25.b 0.82 results: AP@1000: - - 0.2774 + - 0.2773 RR@100: - - 0.2768 + - 0.2767 R@100: - 0.8070 R@1000: @@ -251,7 +251,7 @@ models: AP@1000: - 0.1886 RR@100: - - 0.1878 + - 0.1877 R@100: - 0.7526 R@1000: diff --git a/src/main/resources/regression/msmarco-v2-doc-d2q-t5.yaml b/src/main/resources/regression/msmarco-v2-doc-d2q-t5.yaml index 273ea83bc5..fcf73772ea 100644 --- a/src/main/resources/regression/msmarco-v2-doc-d2q-t5.yaml +++ b/src/main/resources/regression/msmarco-v2-doc-d2q-t5.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 11959635 documents (non-empty): 11959635 - total terms: 19760777295 + total terms: 19760783236 metrics: - metric: MAP@100 diff --git a/src/main/resources/regression/msmarco-v2-doc-segmented-d2q-t5.yaml b/src/main/resources/regression/msmarco-v2-doc-segmented-d2q-t5.yaml index f67f50a216..2c79d77d86 100644 --- a/src/main/resources/regression/msmarco-v2-doc-segmented-d2q-t5.yaml +++ b/src/main/resources/regression/msmarco-v2-doc-segmented-d2q-t5.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 124131414 documents (non-empty): 124131414 - total terms: 30376032067 + total terms: 30376034132 metrics: - metric: MAP@100 diff --git a/src/main/resources/regression/msmarco-v2-doc-segmented.yaml b/src/main/resources/regression/msmarco-v2-doc-segmented.yaml index 68f21ceabd..ff8fd0921a 100644 --- a/src/main/resources/regression/msmarco-v2-doc-segmented.yaml +++ b/src/main/resources/regression/msmarco-v2-doc-segmented.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 124131414 documents (non-empty): 124131414 - total terms: 24780915974 + total terms: 24780918039 metrics: - metric: MAP@100 diff --git a/src/main/resources/regression/msmarco-v2-doc.yaml b/src/main/resources/regression/msmarco-v2-doc.yaml index ea9672aff1..3c7e90312c 100644 --- a/src/main/resources/regression/msmarco-v2-doc.yaml +++ b/src/main/resources/regression/msmarco-v2-doc.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 11959635 documents (non-empty): 11959635 - total terms: 14165661202 + total terms: 14165667143 metrics: - metric: MAP@100 diff --git a/src/main/resources/regression/msmarco-v2-passage-augmented-d2q-t5.yaml b/src/main/resources/regression/msmarco-v2-passage-augmented-d2q-t5.yaml index 42c2160b81..ae4e106f3b 100644 --- a/src/main/resources/regression/msmarco-v2-passage-augmented-d2q-t5.yaml +++ b/src/main/resources/regression/msmarco-v2-passage-augmented-d2q-t5.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 138364198 documents (non-empty): 138364198 - total terms: 27561177420 + total terms: 27561177716 metrics: - metric: MAP@100 diff --git a/src/main/resources/regression/msmarco-v2-passage-augmented.yaml b/src/main/resources/regression/msmarco-v2-passage-augmented.yaml index 78fffb7bc5..708e8f4e0d 100644 --- a/src/main/resources/regression/msmarco-v2-passage-augmented.yaml +++ b/src/main/resources/regression/msmarco-v2-passage-augmented.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 138364198 documents (non-empty): 138364198 - total terms: 15272964956 + total terms: 15272965252 metrics: - metric: MAP@100 diff --git a/src/main/resources/regression/msmarco-v2-passage-d2q-t5.yaml b/src/main/resources/regression/msmarco-v2-passage-d2q-t5.yaml index 546ba3d88b..3fa7baffd8 100644 --- a/src/main/resources/regression/msmarco-v2-passage-d2q-t5.yaml +++ b/src/main/resources/regression/msmarco-v2-passage-d2q-t5.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 138364198 documents (non-empty): 138364198 - total terms: 16961479226 + total terms: 16961479264 metrics: - metric: MAP@100 diff --git a/src/main/resources/regression/msmarco-v2-passage.yaml b/src/main/resources/regression/msmarco-v2-passage.yaml index 13f395742b..ecba9ea147 100644 --- a/src/main/resources/regression/msmarco-v2-passage.yaml +++ b/src/main/resources/regression/msmarco-v2-passage.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 138364198 documents (non-empty): 138364197 - total terms: 4673266762 + total terms: 4673266800 metrics: - metric: MAP@100 diff --git a/src/main/resources/regression/wikipedia-dpr-100w-bm25.yaml b/src/main/resources/regression/wikipedia-dpr-100w-bm25.yaml index 99c56d370f..ace7220b35 100644 --- a/src/main/resources/regression/wikipedia-dpr-100w-bm25.yaml +++ b/src/main/resources/regression/wikipedia-dpr-100w-bm25.yaml @@ -10,7 +10,7 @@ index_options: -storeRaw index_stats: documents: 21015324 documents (non-empty): 21015324 - total terms: 1512973270 + total terms: 1512973244 conversions: - command: python -m pyserini.eval.convert_trec_run_to_dpr_retrieval_run diff --git a/src/main/resources/regression/wt10g.yaml b/src/main/resources/regression/wt10g.yaml index 4a9f01f5b7..7c8805b77b 100644 --- a/src/main/resources/regression/wt10g.yaml +++ b/src/main/resources/regression/wt10g.yaml @@ -10,7 +10,7 @@ index_options: -storePositions -storeDocvectors -storeRaw index_stats: documents: 1688390 documents (non-empty): 1688299 - total terms: 752785964 + total terms: 752795264 metrics: - metric: MAP