Unify the input order for text (NLG) metrics

[stancld]

🚀 Enhancement

Unify the input order for text (NLG) metrics

Motivation

We aim to unify text (NLG) metrics in a way that all of them will allow evaluating a hypothesis against multiple references. Nonetheless, there remains another distinction as some metrics presume preds, targets input order, while some expect targets, preds order. With respect to the whole TM ecosystem, we will go with preds, targets.

Pitch

Change input order for the following NLG metrics:

• BLEUScore - B.C (add a warning suggested here)
• CHRFScore - non B.C. (a new metrics in v0.7)
• SacreBLEUScore - B.C (add a warning suggested here)
• TER - non B.C. (a new metrics in v0.7)

Standardize naming convention to have: predictions and targets everywhere.

Alternatives

Keep it as it is.

@SkafteNicki @Borda @ashutoshml

[Borda]

I would try to make it for this 0.7

[SkafteNicki]

If this is as simple as changing the order in a couple of metrics, then lets fit it into 0.7

[stancld]

Yes, I think it's a simple change that affects only ROUGEScore and BERTScore:]

[ashutoshml]

@stancld Incase you aren't working on this, I can take it up.

[stancld]

@ashutoshml I've been finishing some other stuff, so it'd be great if you can have a look! O:]

[SkafteNicki]

@stancld is there a reason why we should standardize to
target, preds
instead of
preds, target
as the remaining codebase?

[stancld]

@SkafteNicki I considered that mainly w.r.t. text metrics. Should we rather convert all NLP metrics to preds, target instead? I guess it makes perfect sense. (Sorry for the confusion :| )

[stancld]

And actually I see that in the TM version 0.6.x there were only BLEUScore/SacreBLEUScore which had target, preds pattern @SkafteNicki ... In that case, it makes really perfect sense to have everything as target, preds.

[ashutoshml]

Also, naming standardization would help
targets or references?
preds or hypothesis?

I personally would prefer references and predictions.

[Borda]

I think that standardization would be needed, for example, if you have a collection and one of the metrics has its inputs in reverse order, could be bad...

[ashutoshml]

In case we decide on the order targets, preds, the current PR is ready for review.
If preds, targets, we might need to change the requirements/description of the issue, and change all files, other than the ones I have currently changed.

[stancld]

@SkafteNicki So I guess we will go for preds, targets order for all metrics? :]
cc: @ashutoshml

[ashutoshml]

Slightly confused.
What should be the final order then? targets, preds or preds, targets?

[stancld]

As also suggested in #687, we will go with preds, targets. I'll re-formulate the issue. Sorry for the confusion @ashutoshml :]

[ashutoshml]

@stancld Thanks for updating the issue. I'll start working on it this weekend.

[ashutoshml]

@stancld: Should I modify all the files in torchmetrics/text to have predictions and targets or only the ones you've mentioned in the description of this issue?

[mathemusician]

@ashutoshml @stancld Now I'm really confused. I thought it was preds and targets, not predictions and targets?

[ashutoshml]

@mathemusician Use predictions and targets as naming.
The earlier comments were primarily for the order of inputs by the metrics.