Multilingual OCR Development Plan

[D-DanielYang]

model name	description	model size	download	Update Date
ch	Chinese and English	3.71M	inference model / trained model	2020.9.22
ch_tra	chinese traditional	5.63M	inference model / trained model	2021.1.21
en	English	2.56M	inference model / trained model	2020.9.22
fr	French	2.65M	inference model / trained model	2021.9.22
ar	Arabic	2.53M	inference model / trained model	2021.1.21
es	Spanish	2.53M	inference model / trained model	2021.1.21
pt	Portuguese	2.63M	inference model / trained model	2021.1.21
ru	Russia	2.63M	inference model / trained model	2021.1.21
ge	german	2.65M	inference model / trained model	2020.9.22
kr	Korean	3.9M	inference model / trained model	2020.9.22
jp	Japanese	4.23M	inference model / trained model	2020.9.22
it	Italian	2.53M	inference model / trained model	2021.1.21
hi	Hindi	2.63M	inference model / trained model	2021.1.21
ug	Uyghur	2.63M	inference model / trained model	2021.1.21
fa	Persian	2.63M	inference model / trained model	2021.1.21
ur	Urdu	2.63M	inference model / trained model	2021.1.21
oc	Occitan	2.53M	inference model / trained model	2021.1.21
mr	Marathi	2.63M	inference model / trained model	2021.1.21
ne	Nepali	2.63M	inference model / trained model	2021.1.21
rs_cyrillic	Serbian(cyrillic)	2.63M	inference model / trained model	2021.1.21
rs_latin	Serbian(latin)	2.53M	inference model / trained model	2021.1.21
bg	Bulgarian	2.63M	inference model / trained model	2021.1.21
uk	Ukranian	2.63M	inference model / trained model	2021.1.21
be	Belarusian	2.63M	inference model / trained model	2021.1.21
te	Telugu	2.63M	inference model / trained model	2021.1.21
kn	Kannada	2.63M	inference model / trained model	2021.1.21
ta	Tamil	2.63M	inference model / trained model	2021.1.21
mg	Mongolian	--	Ongoing
bg	Bangla	--	Need dict and corpus
bm	Burmese	--	Need dict and corpus	call for contribution
ku_cent	kurdish central	--	PR8347	call for contribution
od	Odia	--	PR6348	call for contribution
th	thai	--	PR6719 issue chat	call for contribution
	More		TBC

Guideline for new language requests

If you want to request a new language support, a PR with 2 following files are needed：

1. In folder ppocr/utils/dict,
   it is necessary to submit the dict text to this path and name it with {language}_dict.txt that contains a list of all characters. Please see the format example from other files in that folder.

2. In folder ppocr/utils/corpus,
   it is necessary to submit the corpus to this path and name it with {language}_corpus.txt that contains a list of words in your language.
   Maybe, 50000 words per language is necessary at least.
   Of course, the more, the better.

3. call for contributions to add new language support for PaddleOCR.
   For anyone might be insterested in traing the new language model, Guidance to train the model is provided. We are calling contributions to add new language support for PaddleOCR.

If your language has unique elements, please tell me in advance within any way, such as useful links, wikipedia and so on.

[saheya]

Traditional Mongolian

[omar16100]

I would love to work on "Bangla"

[levanpon98]

I very happy if you do that with Vietnamese

[HusseinYoussef]

How about Arabic? That would be great.

[Hieung28]

I've find out that PADDLE OCR algorithm cannot recognize some special characters (such as comma, semicolon, or dot...) when the language is english. Is there any possible way that i can fix this problem

[GmGniap]

I would like to contribute to add the Burmese language. Is it only needed to submit two text files - dict & corpus? How further process do we need to provide?

[xeron56]

Adding "Bangla" will be grate for the people in south Asia

[giranntu]

Adding "Traditional Chinese (zh-TW)" would be great support.

[Ru-Van]

Do you have preTrained Russian recognition model?

[SasiAravind]

Hi adding " Tamil" language will be very grateful.

Tamil_dict.txt
Tamil_corpus.txt

Need more help plz refer this issue:
JaidedAI/EasyOCR#39

[fcakyon]

I can help with Turkish language.

[krzynio]

I can help with polish language.

[xmy0916]

@GmGniap Hello, Can you provide the corpus file of Burmese Language？

[xmy0916]

@shahidul56 Hello, Can you provide the corpus file of Bangla Languag？

[azmat21]

All models updated in 2021.1.21 cannot be downloaded with following Error：
{ code: "NoSuchKey", message: "The specified key does not exist.", requestId: "aa1bfeff-f572-40aa-8935-6129b1533ed1" }

[D-DanielYang]

All models updated in 2021.1.21 cannot be downloaded with following Error：
{ code: "NoSuchKey", message: "The specified key does not exist.", requestId: "aa1bfeff-f572-40aa-8935-6129b1533ed1" }

Sorry for the invalid links and all of them have been revised now, you can try again.

[redcinelli]

I very happy if you do that with Vietnamese

#1847, seems to be ongoing.

[xmy0916]

@redcinelli Thank you very much. The Vietnamese model is in training and will be available soon~

[fcakyon]

model name description model size download Update Date
ch Chinese and English 3.71M inference model / trained model 2020.9.22
cht chinese traditional 5.63M inference model / trained model 2021.1.21
en English 2.56M inference model / trained model 2020.9.22
fr French 2.65M inference model / trained model 2021.9.22
ar Arabic 2.53M inference model / trained model 2021.1.21
xi Spanish 2.53M inference model / trained model 2021.1.21
pu Portuguese 2.63M inference model / trained model 2021.1.21
ru Russia 2.63M inference model / trained model 2021.1.21
ge german 2.65M inference model / trained model 2020.9.22
kr Korean 3.9M inference model / trained model 2020.9.22
jp Japanese 4.23M inference model / trained model 2020.9.22
it Italian 2.53M inference model / trained model 2021.1.21
hi Hindi 2.63M inference model / trained model 2021.1.21
ug Uyghur 2.63M inference model / trained model 2021.1.21
fa Persian 2.63M inference model / trained model 2021.1.21
ur Urdu 2.63M inference model / trained model 2021.1.21
rs Serbian(latin) 2.53M inference model / trained model 2021.1.21
oc Occitan 2.53M inference model / trained model 2021.1.21
mr Marathi 2.63M inference model / trained model 2021.1.21
ne Nepali 2.63M inference model / trained model 2021.1.21
rsc Serbian(cyrillic) 2.63M inference model / trained model 2021.1.21
bg Bulgarian 2.63M inference model / trained model 2021.1.21
uk Ukranian 2.63M inference model / trained model 2021.1.21
be Belarusian 2.63M inference model / trained model 2021.1.21
te Telugu 2.63M inference model / trained model 2021.1.21
ka Kannada 2.63M inference model / trained model 2021.1.21
ta Tamil 2.63M inference model / trained model 2021.1.21
mg Mongolian -- Ongoing
bg Bangla -- Need dict and corpus
vi Vietnamese -- Need dict and corpus
bm Burmese -- Need dict and corpus
tk Turkish -- Need dict and corpus
po polish -- Need dict and corpus
More TBC

Guideline for new language requests

If you want to request a new language support, a PR with 2 following files are needed：

1. In folder [ppocr/utils/dict](./ppocr/utils/dict),
   it is necessary to submit the dict text to this path and name it with `{language}_dict.txt` that contains a list of all characters. Please see the format example from other files in that folder.

2. In folder [ppocr/utils/corpus](./ppocr/utils/corpus),
   it is necessary to submit the corpus to this path and name it with `{language}_corpus.txt` that contains a list of words in your language.
   Maybe, 50000 words per language is necessary at least.
   Of course, the more, the better.

If your language has unique elements, please tell me in advance within any way, such as useful links, wikipedia and so on.

@grasswolfs model name for Turkish should be "tr" instead of "tk", it is the widely used abbreviation for Turkish.

[fcakyon]

I have also opened a pr for Turkish dict and corpora: #1856

[tink2123]

Thanks @habout632 for adding Southeast Asian languages via #1896

[yumeliu]

Here is a dictionary for Greek.
el_dict.txt

[alenma04]

Hi , did we have a model to detect all English characters along with special characters like.,"()

[Jane-Ding]

hi, thank you for the great work! I just wonder whether you will add traditional Chinese to the general model? Right now, the general model can support Chinese(sim), English and numbers.

[AzizDZH]

Please add Tajik Language
tajik_corpus.txt
tajik_dict.txt

[ssavi-ict]

@fcakyon @D-DanielYang @xmy0916

I would like to contribute to Bangla Dictionary and Corpus. Can I do that?

Also, I have a few queries to ask -

1. Could not clearly understand this line - If your language has unique elements, please tell me in advance within any way, such as useful links, wikipedia and so on.
2. In the corpus of at least 50000 words, I am guessing all of them should be unique. Am I right?
3. Is there any particular category of corpus words? Like except stop words or something similar to that?
4. In ppocr/utils path I can not see any corpus directory.

Thanks in advance

[ariefwijaya]

Please add Indonesia (id) and English (en) together

[Truong-Thanh-Quang]

Do you have any plan for Vietnamese release?

[ursfan]

Is it sufficient to change the file german_dict.txt if one wants to detect Fraktur a historic german script instead of the current script form? The dictionary which was learnt for the German language should be the same? For tesseract there is one trained file for Fraktur to ocr scan historic documents.

[runachan19]

need indonesian language please

[zahamed]

Hi Dear plz add the bangla and english support. I have attach both the file for bangla
bangla_dict.txt

bangla_corpus.txt

[EdwardYGLi]

Hi team. Great work on Paddle, it's an amazing OCR engine! Can we please have Hebrew support in multilanguage models ?

Thanks !

[zahamed]

Dear Team, Tnx for your reply. I am from Bangladesh. I have already submitted both files like dict and corpus for bangla. I would appreciate if you could add bangla support. Thank you. Zahir

…

On Fri, Jul 28, 2023, 1:50 AM Edward Li ***@***.***> wrote: Hi team. Great work on Paddle, it's an amazing OCR engine! Can we please have *Hebrew* support in multilanguage models ? Thanks ! — Reply to this email directly, view it on GitHub <#1048 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD6CAOC6MTVJDVXY4W65TWDXSLBCHANCNFSM4TCPRJ6Q> . You are receiving this because you commented.Message ID: ***@***.***>

[shreyaahh]

Can you provide for any ancient scripts?

[hungtrieu07]

Truong

I'm trying with my private data, but the result very poor

[SUPERustam]

Sorry for my stupid question, I am novice at DL: What difference between Inference model and trained model?

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[taeefnajib]

I created a PR for Bangla

[juvebogdan]

Does this list contain the latest models? If i want to fine tune for example german model do i use this link from this page to download the pretrained model? If so what yml file should i use? How do i know what is the architecture of these models?

[AzizDZH]

Please add Tajik Language
tajik_corpus.txt
tajik_dict.txt