Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Rust API to optimize COCOPageMapper performance #1120

Merged
merged 6 commits into from
Aug 10, 2023

Conversation

vinnamkim
Copy link
Contributor

@vinnamkim vinnamkim commented Aug 9, 2023

Summary

  • Ticket no. 117181
  • Migrate COCOPageMapper algorithm from Python to Rust implementation
Python impl. Rust impl.
stream coco-rust

This improvement shows that

  1. The time from starting the Python script to creating the COCO page map is decreased from ~200 secs (Python impl.) to ~10 secs (Rust impl.).
  2. Memory cost for constructing the COCO page map is decreased from ~350 Mb (Python impl.) to ~260 Mb (Rust impl.).

How to test

Existing COCO tests with stream=True cover these changes.

Checklist

  • I have added unit tests to cover my changes.​
  • I have added integration tests to cover my changes.​
  • I have added the description of my changes into CHANGELOG.​
  • I have updated the documentation accordingly

License

  • I submit my code changes under the same MIT License that covers the project.
    Feel free to contact the maintainers if that's a concern.
  • I have updated the license header for each file (see an example below).
# Copyright (C) 2023 Intel Corporation
#
# SPDX-License-Identifier: MIT

 - Migrate COCOPageMapper algorithm from Python to Rust implementation

Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>
@vinnamkim vinnamkim marked this pull request as ready for review August 9, 2023 03:49
@vinnamkim vinnamkim requested review from a team as code owners August 9, 2023 03:49
@vinnamkim vinnamkim requested review from bonhunko and removed request for a team August 9, 2023 03:49
Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>
@vinnamkim vinnamkim changed the title Add datumaro.rust_api Add Rust API to optimize COCOPageMapper Aug 9, 2023
Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>
@vinnamkim vinnamkim added this to the 1.4.0 milestone Aug 9, 2023
@vinnamkim vinnamkim added the FEATURE New feature & functionality label Aug 9, 2023
@vinnamkim vinnamkim changed the title Add Rust API to optimize COCOPageMapper Add Rust API to optimize COCOPageMapper performance Aug 9, 2023
Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>
Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>
@codecov-commenter
Copy link

codecov-commenter commented Aug 9, 2023

Codecov Report

Patch coverage: 85.18% and project coverage change: -0.04% ⚠️

Comparison is base (c84a13d) 80.00% compared to head (c95f9b8) 79.97%.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #1120      +/-   ##
===========================================
- Coverage    80.00%   79.97%   -0.04%     
===========================================
  Files          258      258              
  Lines        29329    29140     -189     
  Branches      5745     5709      -36     
===========================================
- Hits         23465    23304     -161     
+ Misses        4516     4506      -10     
+ Partials      1348     1330      -18     
Flag Coverage Δ
macos-11_Python-3.8 79.05% <85.18%> (-0.04%) ⬇️
ubuntu-20.04_Python-3.8 79.95% <85.18%> (-0.04%) ⬇️
windows-2019_Python-3.8 79.88% <85.18%> (-0.04%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
.../datumaro/plugins/data_formats/coco/page_mapper.py 82.35% <85.18%> (-2.41%) ⬇️

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

CHANGELOG.md Outdated Show resolved Hide resolved
Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>
Copy link
Contributor

@bonhunko bonhunko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@vinnamkim vinnamkim modified the milestones: 1.4.0, 1.5.0 Aug 10, 2023
Copy link
Contributor

@wonjuleee wonjuleee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vinnamkim vinnamkim merged commit 383dce7 into openvinotoolkit:develop Aug 10, 2023
4 checks passed
@vinnamkim vinnamkim deleted the feature/rust-api branch August 10, 2023 02:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
FEATURE New feature & functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants