Improve LayoutLM

[NielsRogge]

What does this PR do?

• Improve documentation of LayoutLM, explaining how people can normalize bounding boxes before passing them to the model, add links to the various datasets on which the model achieves state-of-the-art results, add code examples in the documentation for the various models
• Add notebook to the list of community notebooks showcasing how to fine-tune LayoutLMForTokenClassification on the FUNSD dataset (on which the model achieves SOTA results)
• Add integration tests, which confirm that the model outputs the same tensors as the original implementation on the same input data
• Add LayoutLMForSequenceClassification, which makes it possible to fine-tune LayoutLM for document image classification tasks (such as the RVL-CLIP dataset), extra tests included.

Fixes the following issues:

• Differences between original implementation and HuggingFace implementation #9228
• Is the LayoutLM working now? #9097
• different embedding weights for base-uncased with different transformers versions #8866
• LayoutLM Token Classification not learning #8524

Who can review?

@LysandreJik, @patrickvonplaten, @sgugger

[patrickvonplaten]

atol=1e-3 would not pass here?

[patrickvonplaten]

Very nice PR! Thanks so much for taking care of this. The notebook looks great as well. Left a couple of comments. If possible it would be awesome if we could make the example a bit more concise (e.g. to just use tokenizer(...) instead of tokenize(...) and conevrt_tokens_to_ids(...).

[sgugger]

Thanks for all your work on this!

[LysandreJik]

LGTM, great job @NielsRogge! Thanks a lot for your contribution!

[NielsRogge]

Thanks for the reviews, I've addressed all comments. There are 2 things remaining:

• in the code examples, I use both tokenize() and convert_tokens_to_ids as the bounding boxes (which are at word-level) need to be converted to token-level. Is there a better solution?

words = ["Hello", "world"]
normalized_word_boxes = [637, 773, 693, 782], [698, 773, 733, 782]
tokens = []
token_boxes = []
for word, box in zip(words, normalized_word_boxes):
    word_tokens = tokenizer.tokenize(word)
    tokens.extend(word_tokens)
    token_boxes.extend([box] * len(word_tokens))

• according to @sgugger the input data on which the integration tests are ran are maybe too long, and black formatting causes them to be flattened vertically. Could you maybe fix this @LysandreJik?

[LysandreJik]

I pushed the reformat you asked for @NielsRogge, make sure to pull before doing any more changes!

[NielsRogge]

Ok thank you, so the only thing remaining is make the code examples more efficient? Is there a way to make the code block (see comment above) better?