Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize unit conversion by reusing cv_converters #420

Merged
merged 1 commit into from
Jul 13, 2022

Conversation

mattw-nws
Copy link
Contributor

@mattw-nws mattw-nws commented Jun 10, 2022

Callgrind analysis showed a >10% of runtime spent in get_converted_value largely because of ut_parse when creating unit representations. This was a known unoptimized point, but the impact was higher than expected. This PR optimizes the unit conversion pipeline.

Additions

  • Adds a static/global cache of converters that are reused instead of recreated. Basic thread safety is attempted for future-proofing.

Removals

Changes

Testing

  1. Automated tests are passing. No API surface changes.

Screenshots

Notes

Todos

Checklist

  • PR has an informative and human-readable title
  • Changes are limited to a single goal (no scope creep)
  • Code can be automatically merged (no conflicts)
  • Code follows project standards (link if applicable)
  • Passes all existing automated tests
  • Any change in functionality is tested
  • New functions are documented (with a description, list of inputs, and expected output)
  • Placeholder code is flagged / future todos are captured in comments
  • Project documentation has been updated (including the "Unreleased" section of the CHANGELOG)
  • Reviewers requested with the Reviewers tool ➡️

Testing checklist (automated report can be put here)

Target Environment support

  • Linux

const std::lock_guard<std::mutex> lock(converters_mutex);

std::string key = in_units + "|" + out_units; //Better solution? Good enough? Bother with nested maps?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

depends on what is "faster", string concat and then hash, or two hashes?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that, but I was actually thinking more about if | is somehow a valid character in a unit definition (probably not??) and/or if we could hit string encoding issues that this operator and std::string can't handle...no idea there.

@mattw-nws
Copy link
Contributor Author

image
⬇️
image

@mattw-nws
Copy link
Contributor Author

Hmm... noting that this will probably conflict with #405, which should probably be merged first.

@mattw-nws mattw-nws requested a review from donaldwj June 21, 2022 18:02
@mattw-nws mattw-nws marked this pull request as ready for review June 21, 2022 18:02
@mattw-nws
Copy link
Contributor Author

Rebased after #405 ... please reassess!

@mattw-nws mattw-nws requested a review from hellkite500 July 12, 2022 12:57
Copy link
Contributor

@donaldwj donaldwj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have not seen the use of closure used in creation of the mapped converters before, but it looks like it has test coverage.

@hellkite500 hellkite500 merged commit fa278a5 into NOAA-OWP:master Jul 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants