Releases: shibing624/pycorrector
Releases · shibing624/pycorrector
1.1.1
v1.1.1
- update get errors by difflib
- accelerate proper corrector with tire tree.
What's Changed
- feat:添加HillZhang1999的两个开源纠错模型 by @smartmark-pro in #527
New Contributors
- @smartmark-pro made their first contribution in #527
Full Changelog: 1.1.0...1.1.1
1.1.0
v1.1.0版本
- 新增了基于Qwen2.5的中文文本纠错模型,支持多字、少字、错字、词序、语法等错误纠正,发布了shibing624/chinese-text-correction-1.5b和shibing624/chinese-text-correction-7b模型,及其对应的LoRA模型。
![image](https://private-user-images.githubusercontent.com/10249622/376106640-13a16d18-631e-437e-b870-b680e0de9618.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkxODI4MzUsIm5iZiI6MTczOTE4MjUzNSwicGF0aCI6Ii8xMDI0OTYyMi8zNzYxMDY2NDAtMTNhMTZkMTgtNjMxZS00MzdlLWI4NzAtYjY4MGUwZGU5NjE4LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAyMTAlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMjEwVDEwMTUzNVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWUwZTdmOThjYjFjYTEwNmM0YmY3NjMyZTk1ZjlmYjY2NDM0MTlmNTczZWVmYWNlNGE2NmUwNzQ2NmJjZWE2ZGEmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.ro-4OBp4yWSQ5GigD5Mp6yZOsLwFog8rsHN61BKIZX8)
模型效果评估:
![eval](https://private-user-images.githubusercontent.com/10249622/376147225-a2e8419c-74e1-4749-9b67-7e8341df4fd1.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkxODI4MzUsIm5iZiI6MTczOTE4MjUzNSwicGF0aCI6Ii8xMDI0OTYyMi8zNzYxNDcyMjUtYTJlODQxOWMtNzRlMS00NzQ5LTliNjctN2U4MzQxZGY0ZmQxLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAyMTAlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMjEwVDEwMTUzNVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTVlMDVkYzc2ZjlmNTI4YWFhNTkzZjQ4NjY0NWFkY2UyZjAwMjU3OTFiOWEwNDZjMWNjZjkzZjQ5MTkxMjE2M2ImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.S2uNIvqncEtd-YU89k9wYrTWdm_w6RMa8OR85cvRgVY)
Full Changelog: 1.0.2...1.1.0
1.0.2
What's Changed
- Llm by @shibing624 in #431
- Llm by @shibing624 in #432
- 库依赖跟新和最低版本要求更新 by @Vela-zz in #444
- Fix issue #470: Improve how confusion words are located and/or replaced by @treya-lin in #471
New Contributors
- @Vela-zz made their first contribution in #444
- @treya-lin made their first contribution in #471
Full Changelog: 1.0.0...1.0.2
1.0.0
v1.0.0版本
- 新增了ChatGLM3/LLaMA2等GPT模型用于中文文本纠错,发布了基于ChatGLM3-6B的shibing624/chatglm3-6b-csc-chinese-lora拼写和语法纠错模型;
- 重写了DeepContext、ConvSeq2Seq、T5等模型的实现。
![image](https://private-user-images.githubusercontent.com/10249622/280926322-dbf10b29-bf72-4d29-8d1f-dfbc53ad4d28.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkxODI4MzYsIm5iZiI6MTczOTE4MjUzNiwicGF0aCI6Ii8xMDI0OTYyMi8yODA5MjYzMjItZGJmMTBiMjktYmY3Mi00ZDI5LThkMWYtZGZiYzUzYWQ0ZDI4LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAyMTAlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMjEwVDEwMTUzNlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWQ5NjliMGY1NTBjNDI5MTMxZGNkNjMxMDVmYWRjYjYyYjAwMjVmOWU2MTJlYzFlOTNkZWQ3NGZmOTRlOTUyNzAmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.AAY4xb9xEwcN1ZQjz53Avc84vmZnWt6d_QOqyvP7Nuk)
Full Changelog: 0.4.8...1.0.0
0.4.6
Full Changelog: 0.4.5...0.4.6
0.4.5
Full Changelog: 0.4.4...0.4.5
0.4.3
What's Changed
- Add ernie-csc model by @joey12300 in #252
- Update: the description for MacBERT. by @okcd00 in #259
- Solve the misalignment problem on UNK tokens (MacBERT). by @okcd00 in #261
New Contributors
- @joey12300 made their first contribution in #252
- @okcd00 made their first contribution in #259
Full Changelog: 0.4.1...0.4.3
0.3.1
0.3.1,__init__ delete torch import.
0.2.4
update demo.