Retrieve and generation logic in the function level

[pppyb]

Hi there!

I would like to ask if the retrieve and generation logic for the function part is exactly the same as for the API and line parts(like we just need to add function benchmark?). Is the only difference in evaluation, where unit tests are used instead of EM and ES?

Thanks in advance!

[zfj1998]

Unit tests were employed to evaluate the correctness of the completed function body. Sorry we cannot share the source code as stated in the README (We utilize a private library to handle the execution and evaluation of the function-level completion. Due to the license issue, we cannot release the code. ) FYI, here is a repo claiming the reproduction of RepoEval's function-level dataset, https://github.com/code-rag-bench/code-rag-bench. Hope it helps with your research.

[pppyb]

Unit tests were employed to evaluate the correctness of the completed function body. Sorry we cannot share the source code as stated in the README (We utilize a private library to handle the execution and evaluation of the function-level completion. Due to the license issue, we cannot release the code. ) FYI, here is a repo claiming the reproduction of RepoEval's function-level dataset, https://github.com/code-rag-bench/code-rag-bench. Hope it helps with your research.

Thanks for your reply~ I still want to make sure that the code completion process for function level is the same as api and line level? cuz I didn't found any function level completion process like function benchmark in code.

Thanks again for your patient reply~