Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove in-place optimizations of patch and merge #1217

Merged
merged 4 commits into from
Sep 15, 2021
Merged

Conversation

mfelsche
Copy link
Member

@mfelsche mfelsche commented Sep 15, 2021

Pull request

Description

This PR removes the patch and merge in-place optimizations, as they can lead to memory unsafety.

a script like:

# initialize state
let state = match state of
  case null => {}
  default state
end;
let state = merge state of event end;
emit state

will happily mutate state and insert values from event, which will no longer be valid after the event is gone, and so allow a view into freed up or reallocated memory via string keys or values.
We try to get back to those optimizations in a safer way.

Related

Checklist

  • The RFC, if required, has been submitted and approved
  • Any user-facing impact of the changes is reflected in docs.tremor.rs
  • The code is tested
  • Use of unsafe code is reasoned about in a comment
  • Update CHANGELOG.md appropriately, recording any changes, bug fixes, or other observable changes in behavior
  • The performance impact of the change is measured (see below)

Performance

Matthias Wahl added 2 commits September 15, 2021 13:40
They are too dangerous and can lead to memory unsafety.

Also added proper state handling to script based tests.

Signed-off-by: Matthias Wahl <mwahl@wayfair.com>
So state is not reset to 'null' after every event but lasts the whole 'tremor run' session when running a tremor script.

Signed-off-by: Matthias Wahl <mwahl@wayfair.com>
Licenser
Licenser previously approved these changes Sep 15, 2021
Copy link
Member

@Licenser Licenser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch! 🚀

Signed-off-by: Matthias Wahl <mwahl@wayfair.com>
Licenser
Licenser previously approved these changes Sep 15, 2021
@coveralls
Copy link
Collaborator

coveralls commented Sep 15, 2021

Coverage Status

Coverage increased (+0.02%) to 85.244% when pulling 70f7e11 on remove_in_place into bffc2a5 on main.

Signed-off-by: Matthias Wahl <mwahl@wayfair.com>
@mfelsche mfelsche marked this pull request as ready for review September 15, 2021 13:48
Copy link
Member

@darach darach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@mfelsche mfelsche merged commit ec22294 into main Sep 15, 2021
@mfelsche mfelsche deleted the remove_in_place branch September 15, 2021 14:01
@mfelsche
Copy link
Member Author

mfelsche commented Sep 15, 2021

For performance we see a slight increase for patch and merge with the optimization removed, which is honestly surprising.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

transmute in _in_place functions
4 participants