Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(config): Make ONNX Runtime DirectML an optional configuration for better compatibility #178

Merged
merged 1 commit into from
May 15, 2024

Conversation

ccddos
Copy link
Contributor

@ccddos ccddos commented May 15, 2024

This commit introduces an optional configuration for using ONNX Runtime DirectML.

DirectML can significantly improve performance on newer devices, but it has stricter requirements and may not be suitable for all users.

By making it an optional configuration, we ensure better compatibility for users with older devices or who are running the CLI version, where the GPU's potential may not be fully utilized due to single inference models.

For more information on DirectML requirements, please refer to: https://github.com/microsoft/DirectML

根据讨论 #175

dml ep 在旧设备上表现不好,而且cli版本无法充分利用gpu 故切换至可选项

… better compatibility

This commit introduces an optional configuration for using ONNX Runtime DirectML.

DirectML can significantly improve performance on newer devices, but it has stricter requirements and may not be suitable for all users.

By making it an optional configuration, we ensure better compatibility for users with older devices or who are running the CLI version, where the GPU's potential may not be fully utilized due to single inference models.

For more information on DirectML requirements, please refer to: https://github.com/microsoft/DirectML
@ccddos
Copy link
Contributor Author

ccddos commented May 15, 2024

Usage example

cpu version

(p311) D:\work\RapidOCR\python>python -m rapidocr_onnxruntime.main -img testimg.png
<frozen runpy>:128: RuntimeWarning: 'rapidocr_onnxruntime.main' found in sys.modules after import of package 'rapidocr_onnxruntime', but prior to execution of 'rapidocr_onnxruntime.main'; this may result in unpredictable behaviour
[[[[25.0, 36.0], [251.0, 36.0], [251.0, 71.0], [25.0, 71.0]], 'Introduction', 0.9092294665483328], [[[86.0, 164.0], [776.0, 166.0], [775.0, 203.0], [85.0, 200.0]], 'OCR known to support rapid 
offline deployment.', 0.9705755609147092], [[[65.0, 222.0], [1593.0, 222.0], [1593.0, 258.0], [65.0, 258.0]], ' Supported Languages: The default is Chinese and English, other language recognition requires self-service', 0.9868534831243141], [[[87.0, 272.0], [633.0, 272.0], [633.0, 305.0], [87.0, 305.0]], 'conversion. For specific reference here.', 0.9651962751295509], [[[79.0, 327.0], [1665.0, 327.0], [1665.0, 360.0], [79.0, 360.0]], ' Cause: PaddleOCR is not well engineered, and to make it easier for people to do OCR inference on various ends,', 0.9737989907818181], [[[84.0, 372.0], [1445.0, 375.0], [1445.0, 411.0], [83.0, 408.0]], 'we converted the model in PaddleOCR to ONNX format and ported it to various platforms using', 0.985789726609769], [[[94.0, 426.0], [373.0, 425.0], [373.0, 456.0], [95.0, 457.0]], 'Python/C++/Java/C#', 0.9460111009447199], [[[70.0, 481.0], [1677.0, 482.0], [1677.0, 515.0], [70.0, 514.0]], ' Name Source: Light, fast, economical and smart. OCR technology based on deep learning technology focuses on', 0.9736786440971794], [[[87.0, 530.0], [1596.0, 530.0], [1596.0, 562.0], [87.0, 562.0]], 'artificial intelligence advantages and small models, with speed as the mission and effect as the leading role.', 0.9896172558939135], [[[65.0, 583.0], [186.0, 586.0], [185.0, 621.0], [64.0, 618.0]], 'Usage:', 0.854411244392395], [[[150.0, 735.0], [1027.0, 734.0], [1027.0, 771.0], [150.0, 772.0]], 'this repo is helpful to you, please click on a small star ★ Bah!', 0.9612960668710562]]

dml version

(p311) D:\work\RapidOCR\python>python -m rapidocr_onnxruntime.main --img testimg.png --rec_use_dml --det_use_dml --cls_use_dml
<frozen runpy>:128: RuntimeWarning: 'rapidocr_onnxruntime.main' found in sys.modules after import of package 'rapidocr_onnxruntime', but prior to execution of 'rapidocr_onnxruntime.main'; this may result in unpredictable behaviour
Windows 10 or above detected, try to use DirectML as primary provider
[[[[25.0, 36.0], [251.0, 36.0], [251.0, 71.0], [25.0, 71.0]], 'Introduction', 0.9092294665483328], [[[86.0, 164.0], [776.0, 166.0], [775.0, 203.0], [85.0, 200.0]], 'OCR known to support rapid 
offline deployment.', 0.9705755609147092], [[[65.0, 222.0], [1593.0, 222.0], [1593.0, 258.0], [65.0, 258.0]], ' Supported Languages: The default is Chinese and English, other language recognition requires self-service', 0.9868534831243141], [[[87.0, 272.0], [633.0, 272.0], [633.0, 305.0], [87.0, 305.0]], 'conversion. For specific reference here.', 0.9651962751295509], [[[79.0, 327.0], [1665.0, 327.0], [1665.0, 360.0], [79.0, 360.0]], ' Cause: PaddleOCR is not well engineered, and to make it easier for people to do OCR inference on various ends,', 0.9737989907818181], [[[84.0, 372.0], [1445.0, 375.0], [1445.0, 411.0], [83.0, 408.0]], 'we converted the model in PaddleOCR to ONNX format and ported it to various platforms using', 0.985789726609769], [[[94.0, 426.0], [373.0, 425.0], [373.0, 456.0], [95.0, 457.0]], 'Python/C++/Java/C#', 0.9460111009447199], [[[70.0, 481.0], [1677.0, 482.0], [1677.0, 515.0], [70.0, 514.0]], ' Name Source: Light, fast, economical and smart. OCR technology based on deep learning technology focuses on', 0.9736786440971794], [[[87.0, 530.0], [1596.0, 530.0], [1596.0, 562.0], [87.0, 562.0]], 'artificial intelligence advantages and small models, with speed as the mission and effect as the leading role.', 0.9896172558939135], [[[65.0, 583.0], [186.0, 586.0], [185.0, 621.0], [64.0, 618.0]], 'Usage:', 0.854411244392395], [[[150.0, 735.0], [1027.0, 734.0], [1027.0, 771.0], [150.0, 772.0]], 'this repo is helpful to you, please click on a small star ★ Bah!', 0.9612960668710562]]

testimg example

testimg

@SWHL
Copy link
Member

SWHL commented May 15, 2024

我看到你们的讨论了,作为一个可选项,在当前来看,的确是较为稳妥的做法。
我低估了在旧设备上的推理速度,想当然认为再慢,也就是onnxruntime本来的速度。看来,我错了。

@SWHL SWHL merged commit 8523a2e into RapidAI:main May 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants