Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Duplicate bug in HumanEvalJava #2

Open
andre15silva opened this issue Nov 14, 2023 · 1 comment
Open

Duplicate bug in HumanEvalJava #2

andre15silva opened this issue Nov 14, 2023 · 1 comment

Comments

@andre15silva
Copy link

Hi,

I noticed that there is a duplicate bug in the humaneval_loc.txt file: CORRECT_BRACKETING

This might explain the discrepancy I was observing between the number of java files (163) and the number of bugs reported in that file and in your paper.

Would you confirm that this is indeed what is happening?

Thanks a lot :)

@jiang719
Copy link
Collaborator

Hi, this is because the original Python code generation benchmark, HumanEval dataset from OpenAI has this program duplicated, so we just keep it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants