[Tokenizer] Support fast tokenizer within AutoTokenizer import

[DrownFish19]

PR types

New features

PR changes

Others

Description

Support fast tokenizer import.

[paddle-bot]

Thanks for your contribution!

[codecov]

Codecov Report

Attention: Patch coverage is 60.00000% with 2 lines in your changes missing coverage. Please review.

Project coverage is 52.87%. Comparing base (d74df4d) to head (8fe34fa).
Report is 23 commits behind head on develop.

Files with missing lines	Patch %	Lines
paddlenlp/transformers/auto/tokenizer.py	60.00%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #9466      +/-   ##
===========================================
+ Coverage    52.76%   52.87%   +0.11%     
===========================================
  Files          679      679              
  Lines       108342   108372      +30     
===========================================
+ Hits         57162    57306     +144     
+ Misses       51180    51066     -114     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.