infer entailment label id on zero shot pipeline

[joeddav]

What does this PR do?

Adds an optional argument to the zero shot pipeline constructor to specify the label id of the NLI model that corresponds to "entailment", which it needs to calculate each candidate label's score. Most models in the hub use the last last label id, but some differ (e.g. the recent ynie/roberta-large-snli_mnli_fever_anli_R1_R2_R3-nli).

If the argument is not passed, the pipeline will attempt to look up the entailment dimension in the model config's id2label mapping. If the config does not specify the entailment dimension, the value will be set to -1, indicating the last dimension of the model output.

With this logic in place, the arg only needs to be passed when both (1) the model's entailment label id is not the last id and (2) when the model config's label2id doesn't specify the entailment id.

[julien-c]

Wouldn't it be better to:

• ask the model authors for the relevant models to update their config.json to include a id2label
• and/or to modify them automatically for them on the hub?

I feel this PR is adding a feature that works around an issue when we should be fixing the root issue (also fixes the displayed labels, other future features, etc.). Wdyt?

[joeddav]

@julien-c Well, to be clear this PR does did two things:

1. Switches from always using the last index to using the index determined by looking in the model config if present
2. Gives the user the option to manually override the index in case the information isn't present in the config

If I understand correctly, your issue is just with (2). I think it's a fair point. I don't think we'll be able to ensure that all NLI models have a clearly defined label mapping though. But instead of an override arg, I think it might be better to just add a warning if the entailment label ID can't be found in the config.

[julien-c]

@joeddav Ok yes 1/ is great.

I don't think we'll be able to ensure that all NLI models have a clearly defined label mapping though.

Why not?

[LysandreJik]

This is great! Do you mind running make fixup to patch the style/quality issues?

[joeddav]

@julien-c Just because there are almost 100 results for "NLI" on the model hub and I'd guess from a quick sampling that the majority don't have a label mapping defined. For each model we'd have to figure out which label is which, which would mean either getting the author to look it up and tell us or else running tests on the correct dataset to figure it ourselves.

Do you think it'd be worthwhile to warn the user when uploading or creating configs with generic/missing label mappings (or with any other important fields missing) going forward? Defining a label2id seems like a rather obscure property that I would assume is purely cosmetic if I were uploading a model, i.e. I wouldn't expect it to actually impact code behavior for someone using my model.