Tf longformer for sequence classification

[elk-cloner]

What does this PR do?

implement SequenceClassification, MultipleChoice and TokenClassification classes for TFLongformer.
Resolves #6401

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to the it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

[LysandreJik]

The PR looks good, but there are a few things to do before merging.

• You introduced a bunch of spelling errors in the variable names/documentation?
• You should run make style and make quality to fix the code quality test
• You should implement the tests related to the added classes
• You should add these classes to the TF auto models

[patrickvonplaten]

@elk-cloner - thanks a lot for taking a look into this!

Would be awesome to fix the TFLongformer related tests. There seem to be some obvious bug: UnboundLocalError: local variable 'input_ids' referenced before assignment .

I'll do a longer review once these tests are fixed :-) Lemme know if you need help at some point.

[elk-cloner]

@patrickvonplaten i have passed all the tests but got stuck in test_inputs_embeds when it's checking TFLongformerForMultipleChoice model, i debugged my code and found out that inputs_embeds shape is not same when TFLongformerEmbeddings get call from here(test_inputs_embeds) and here(TFLongformerForMultipleChoice), but don't know how to fix it, can you help me ?

[patrickvonplaten]

Hey @elk-cloner,

yeah this problem was not at all obvious! Thanks for letting me know :-) For Multiple Choice, we have to make sure that the position_ids stay 2-dimensional, which is only relevant for TFLongformer, but not for other TF models -> so we need this if fix here.

Feel free to ping me again, when you're ready with the PR or need help :-)

[elk-cloner]

@patrickvonplaten all tests have passed, can you take a look ?

[patrickvonplaten]

Hey @elk-cloner - the signature of the function calls should be done analogs to the one in other modeling_tf_....py files. Woud be great if you can fix that before we merge.

[patrickvonplaten]

Good to merge IMO!

[patrickvonplaten]

Checked the slow tests and everything passes. Great job @elk-cloner! Longformer is definitely not the easiest model

[patrickvonplaten]

Would be awesome if @LysandreJik and @sgugger can take a final look, then we're good to merge.

[sgugger]

Thanks for your contribution! This looks good to merge, just a little thing missing on the doc side.

[LysandreJik]

Very clean implementation. LGTM.