Natural Questions (MRQA), NewsQA, Quoref (nyu-mll#1207)

* race * black * race * quoref * newsqa * NewsQATask * newsqa fix * race * race data proc * race update * mrqa_nq * comma * answers * remove hi * remove gzip * remove race and ropes temporarily * docs * black * flake * flake * remove ropes * remove ropes * remove ropes * doc
leo-liuzy · Oct 22, 2020 · 79df467 · 79df467
1 parent fe75e87
commit 79df467
Show file tree

Hide file tree

Showing 13 changed files with 626 additions and 14 deletions.
diff --git a/examples/notebooks/simple_api_fine_tuning.ipynb b/examples/notebooks/simple_api_fine_tuning.ipynb
@@ -233,4 +233,4 @@
    "outputs": []
   }
  ]
-}
+}
diff --git a/guides/tasks/supported_tasks.md b/guides/tasks/supported_tasks.md
@@ -30,8 +30,11 @@
 | MNLI-mismatched | mnli_mismatched | ✅ | ✅ | mnli_mismatched | GLUE |
 | MultiRC | multirc | ✅ | ✅ | multirc | SuperGLUE |
 | MRPC | mrpc | ✅ | ✅ | mrpc | GLUE |
+| Natural Questions | mrqa_natural_questions | ✅ | ✅ | mrqa_natural_questions | [MRQA](https://mrqa.github.io/) version of task |
+| NewsQA | newsqa | ✅ | ✅ | newsqa |  |
 | QAMR | qamr | ✅ | ✅ | qamr |  |
 | QA-SRL | qasrl | ✅ | ✅ | qasrl |  |
+| Quoref | quoref | ✅ | ✅ | quoref |  |
 | EP-NER | ner | ✅ |  | ner | Edge-Probing |
 | PAWS-X | `pawsx_{lang}` | ✅ | ✅ | pawsx | XTREME, multi-lang |
 | WikiAnn | `panx_{lang}` | ✅ | ✅ | panx | XTREME, multi-lang |

diff --git a/jiant/scripts/download_data/constants.py b/jiant/scripts/download_data/constants.py
@@ -6,7 +6,14 @@
     "multirc": f"https://dl.fbaipublicfiles.com/glue/superglue/data/v2/MultiRC.zip",
     "record": f"https://dl.fbaipublicfiles.com/glue/superglue/data/v2/ReCoRD.zip",
 }
-OTHER_DOWNLOAD_TASKS = {"abductive_nli", "swag", "qamr", "qasrl"}
+OTHER_DOWNLOAD_TASKS = {
+    "abductive_nli",
+    "swag",
+    "qamr",
+    "qasrl",
+    "newsqa",
+    "mrqa_natural_questions",
+}
 DIRECT_DOWNLOAD_TASKS = set(
     list(SQUAD_TASKS) + list(DIRECT_SUPERGLUE_TASKS_TO_DATA_URLS) + list(OTHER_DOWNLOAD_TASKS)
 )