New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CLOSED] Abductive NLI (aNLI) #922

Closed

jeswan opened this issue Sep 17, 2020 · 1 comment

Contributor

jeswan commented Sep 17, 2020

Issue by zphang
Tuesday Oct 01, 2019 at 23:33 GMT
Originally opened as nyu-mll/jiant#922

See: https://leaderboard.allenai.org/anli/submissions/get-started

No test data is provided for the task: instead the expectation is that the model is uploaded for evaluation.

zphang included the following code: https://github.com/nyu-mll/jiant/pull/922/commits

Contributor Author

jeswan commented Sep 17, 2020

Comment by zphang
Monday Oct 07, 2019 at 07:43 GMT

bert-large-cased: 66.2 (dev)
roberta-large: 83.8 (dev)

Compared to paper:

bert-large (case unclear): 68.9 (test)
human: 91.4 (test)

Good to merge when tests pass.

jeswan closed this as completed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment