Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

eval generation path issue #27

Open
sia-cerebras opened this issue Jan 16, 2024 · 0 comments
Open

eval generation path issue #27

sia-cerebras opened this issue Jan 16, 2024 · 0 comments

Comments

@sia-cerebras
Copy link

Hello, I'm trying to use your eval pipeline.
I ran ./inference_pipeline.sh -b pubmedqa -c gpt2 -s 0 -m 0 -out_dir out_dir
After it is done with generation I get the following error:

Stored pubmedqa generations to the following path: ../benchmarks/generations/pubmedqa-gpt2.jsonl
Traceback (most recent call last):
  File "evaluate.py", line 475, in <module>
    main(args)
  File "evaluate.py", line 390, in main
    data = load_jsonl(path)
  File "evaluate.py", line 39, in load_jsonl
    with open(filename, 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: '../benchmarks/generations/pubmedqa/pubmedqa-gpt2.jsonl'

It looks like the generations are saved in: meditron/benchmarks/generations/pubmedqa-gpt2.jsonl
But eval looks for them in: meditron/benchmarks/generations/pubmedqa/pubmedqa-gpt2.jsonl'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant