From 282c42dce6f07af66c1a33939ba7279794154165 Mon Sep 17 00:00:00 2001 From: Divyajyoti Panda <67417654+Divyajyoti02@users.noreply.github.com> Date: Sat, 23 Nov 2024 14:10:03 -0800 Subject: [PATCH 1/2] Update experiments-msmarco-passage.md There were issues in building the Anserini package on the Macbook Air M2 chip with 16 GB RAM and 256 GB memory. On building the package with maven (mvn clean package), the following error happens: [ERROR] Failures: [ERROR] ControllerTest.testSearch:42 expected:<10> but was:<0> The error went away when I passed the DskipTests flag. The results were then reproducible successfully. Also, it would be better for beginners if it is explained why an inverted index is necessary. --- docs/experiments-msmarco-passage.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/experiments-msmarco-passage.md b/docs/experiments-msmarco-passage.md index f68f3cb2b..1c4494241 100644 --- a/docs/experiments-msmarco-passage.md +++ b/docs/experiments-msmarco-passage.md @@ -535,3 +535,4 @@ The BM25 run with default parameters `k1=0.9`, `b=0.4` roughly corresponds to th + Results reproduced by [@a-y-m-a-n-c-h](https://github.com/a-y-m-a-n-c-h) on 2024-10-16 (commit [`0346842`](https://github.com/castorini/anserini/commit/03468423c820e1c0c38c9f48dc25d1f2f315831c)) + Results reproduced by [@Samantha-Zhan](https://github.com/Samantha-Zhan) on 2024-10-20 (commit [`daceb40`](https://github.com/castorini/anserini/commit/daceb4084c8e8103e3e86c81a8e0d597d409220e)) + Results reproduced by [@pxlin-09](https://github.com/pxlin-09) on 2024-10-26 (commit [`e2eb203`](https://github.com/castorini/anserini/commit/e2eb203b83dd643a356ee90f299c8877f6e108bd)) ++ Results reproduced by [@Divyajyoti02](https://github.com/Divyajyoti02) on 2024-11-23 (commit [`3bc1f8b`](https://github.com/castorini/anserini/commit/3bc1f8ba6c2eb4da6a1761938fc16631bcc3d6dd)) From d0fe0375b296bde6bca264dbd66114dac96dea56 Mon Sep 17 00:00:00 2001 From: Divyajyoti Panda <67417654+Divyajyoti02@users.noreply.github.com> Date: Sat, 23 Nov 2024 20:00:56 -0800 Subject: [PATCH 2/2] Update start-here.md Merging #2634 into this PR. Regarding the failed case, this happens while following step 2 in "Foundations of Retrieval" pathway ("BM25 Baselines for MS MARCO Passage Ranking in Anserini"). In the Indexing step, the tutorial asks us to build anserini using the "mvn clean package" command. Upon running the following error occurs. [ERROR] Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.054 s <<< FAILURE! - in io.anserini.server.ControllerTest [ERROR] io.anserini.server.ControllerTest.testSearch Time elapsed: 0.032 s <<< FAILURE! java.lang.AssertionError: expected:<10> but was:<0> at org.junit.Assert.fail(Assert.java:89) at org.junit.Assert.failNotEquals(Assert.java:835) at org.junit.Assert.assertEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:633) at io.anserini.server.ControllerTest.testSearch(ControllerTest.java:42) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) at java.base/java.lang.reflect.Method.invoke(Method.java:580) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:316) at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:240) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:214) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:155) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) I suspect this test case does not impact the task because if I skip this test case by running "mvn clean package -DskipTests" everything works fine, and the code is reproducible. Maven version: 3.9.9 Java version: 21.0.2 --- docs/start-here.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/start-here.md b/docs/start-here.md index 0e472ef42..01294bb72 100644 --- a/docs/start-here.md +++ b/docs/start-here.md @@ -420,3 +420,4 @@ If you think this guide can be improved in any way (e.g., you caught a typo or t + Results reproduced by [@a-y-m-a-n-c-h](https://github.com/a-y-m-a-n-c-h) on 2024-10-16 (commit [`0346842`](https://github.com/castorini/anserini/commit/03468423c820e1c0c38c9f48dc25d1f2f315831c)) + Results reproduced by [@Samantha-Zhan](https://github.com/Samantha-Zhan) on 2024-10-20 (commit [`daceb40`](https://github.com/castorini/anserini/commit/daceb4084c8e8103e3e86c81a8e0d597d409220e)) + Results reproduced by [@pxlin-09](https://github.com/pxlin-09) on 2024-10-26 (commit [`e2eb203`](https://github.com/castorini/anserini/commit/e2eb203b83dd643a356ee90f299c8877f6e108bd)) ++ Results reproduced by [@Divyajyoti02](https://github.com/Divyajyoti02) on 2024-11-22 (commit [`3bc1f8b`](https://github.com/castorini/anserini/commit/3bc1f8ba6c2eb4da6a1761938fc16631bcc3d6dd))