Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

How to validate VQA1 on CloudCV/EvalAI server? #5

Open
monajalal opened this issue Nov 7, 2018 · 1 comment
Open

How to validate VQA1 on CloudCV/EvalAI server? #5

monajalal opened this issue Nov 7, 2018 · 1 comment

Comments

@monajalal
Copy link

monajalal commented Nov 7, 2018 •
edited
Loading

We are not able to evaluate VQA1 on CloudCV/EvalAI server. They only have the evaluation code for VQA2.

I have attached the JSON file for VQA1 for you. Please let us know how you have reported evaluation numbers?
https://drive.google.com/drive/folders/1Moq6qOimMpyCmT499KQgwoONGHaXVTgK?usp=sharing

We evaluated it here:
http://evalai.cloudcv.org/

For VQA1:
screen shot 2018-11-07 at 3 41 42 pm

screen shot 2018-11-07 at 3 42 25 pm

screen shot 2018-11-07 at 3 42 45 pm

Doing the same procedure for VQA2 JSON file we get a ~90% accuracy.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@monajalal @mynameischaos