Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
ducdauge authored Nov 17, 2020
1 parent 8b950f7 commit c1d1be5
Showing 1 changed file with 11 additions and 8 deletions.
19 changes: 11 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,14 +50,17 @@ The "translate test" data obtained via Google Translate are available in the fol

## Leaderboard

If you want to see your scores reported, please make a pull request changing the table below.

| Model | Paper | et | ht | id | it | qu | sw | ta | th | tr | vi | zh |
|---|---|---|---|---|---|---|---|---|---|---|---| ---|
| Human | [Ponti et al. (2020)](https://ducdauge.github.io/files/xcopa.pdf) | 98.2 | 96.4 | 100.0 | 97.0 | 94.8 | 99.0 | 98.6 | 98.2 | 96.4 | 98.4 | 96.6 |
| RoBERTa Large (Translate test)| [Ponti et al. (2020)](https://ducdauge.github.io/files/xcopa.pdf) | 81.0 | 73.8 | 82.2 | 77.8 | - | 74.2 | 79.6 | 71.4 | 79.6 | 81.0 | 86.0 |
| XLM-R Large | [Ponti et al. (2020)](https://ducdauge.github.io/files/xcopa.pdf) | 71.4 | - | 79.8 | 72.6 | - | 59.2 | 73.0 | 72.8 | 74.4 | 73.8 | 78.6 |
| MAD-X Base | [Pfeiffer et al. (2020)](https://arxiv.org/pdf/2005.00052.pdf) | 61.3 |53.7 | 65.8 | 63.0 | 52.5 | 56.3 | 61.9 | 61.8 | 60.3 | 66.1 | 67.6 |
If you want to see your results reported, please:

1) submit them on the [XCOPA state-of-the-art page on Papers with Code](https://paperswithcode.com/sota/cross-lingual-transfer-on-xcopa);
2) make a pull request changing the table below.

| Model | Paper | *avg* | et | ht | id | it | qu | sw | ta | th | tr | vi | zh |
|---|---|---|---|---|---|---|---|---|---|---|---|---| ---|
| Human | [Ponti et al. (2020)](https://ducdauge.github.io/files/xcopa.pdf) | 97.60 | 98.2 | 96.4 | 100.0 | 97.0 | 94.8 | 99.0 | 98.6 | 98.2 | 96.4 | 98.4 | 96.6 |
| RoBERTa Large (Translate test)| [Ponti et al. (2020)](https://ducdauge.github.io/files/xcopa.pdf) | 76.05 | 81.0 | 73.8 | 82.2 | 77.8 | (50.0) | 74.2 | 79.6 | 71.4 | 79.6 | 81.0 | 86.0 |
| XLM-R Large | [Ponti et al. (2020)](https://ducdauge.github.io/files/xcopa.pdf) | 68.69 | 71.4 | (50.0) | 79.8 | 72.6 | (50.0) | 59.2 | 73.0 | 72.8 | 74.4 | 73.8 | 78.6 |
| MAD-X Base | [Pfeiffer et al. (2020)](https://arxiv.org/pdf/2005.00052.pdf) | 60.94 | 61.3 |53.7 | 65.8 | 63.0 | 52.5 | 56.3 | 61.9 | 61.8 | 60.3 | 66.1 | 67.6 |

The performance of other multilingual pre-trained encoders is shown in the figure.
![](baselines.png)
Expand Down

0 comments on commit c1d1be5

Please sign in to comment.