Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Fix saving FlaubertTokenizer configs (#14991)
All specific tokenizer config properties must be passed to its base class (XLMTokenizer) in order to be saved. This was not the case for do_lowercase config. Thus it was not saved by save_pretrained() method and saving and reloading the tokenizer changed its behaviour. This commit fixes it.
- Loading branch information