Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ub left shifting negative signed values #1839

Closed
nmm0 opened this issue Jun 1, 2022 · 0 comments · Fixed by #1840
Closed

ub left shifting negative signed values #1839

nmm0 opened this issue Jun 1, 2022 · 0 comments · Fixed by #1840
Assignees

Comments

@nmm0
Copy link
Collaborator

nmm0 commented Jun 1, 2022

Describe the bug
Some bitfields trigger undefined behavior by left-shifting a negative value.

To Reproduce
Adding this assert to setFieldDynamic in bits_packer.impl.h will reliably fail in our tests:

  auto tmp = ((static_cast<BitField>(segment) << nbits) >> nbits);
  if ( std::is_signed< decltype(tmp) >::value && ( tmp < 0 ) )
    vtAbort(fmt::format(
      "hit a negative value of {}, segment: {}, nbits: {}", tmp,
      static_cast<BitField>(segment), nbits ) );

The next line is

field = field | (((static_cast<BitField>(segment) << nbits) >> nbits) << start);

So this issue being that the tmp value here is negative (BitField is signed in some cases) and being left shifted by start, which is undefined behavior.

Expected behavior
we need to make sure the shifted value is unsigned.

@nmm0 nmm0 added the type: bug label Jun 1, 2022
@nmm0 nmm0 self-assigned this Jun 1, 2022
@nmm0 nmm0 linked a pull request Jun 1, 2022 that will close this issue
nmm0 added a commit that referenced this issue Jun 1, 2022
…py instead of static_cast to ensure correct bit casts from signed to unsigned
nmm0 added a commit that referenced this issue Jun 2, 2022
… of the different bit values are conssistent
nmm0 added a commit that referenced this issue Jun 2, 2022
…py instead of static_cast to ensure correct bit casts from signed to unsigned
nmm0 added a commit that referenced this issue Jun 2, 2022
… of the different bit values are conssistent
@nmm0 nmm0 closed this as completed in #1840 Jun 6, 2022
nmm0 added a commit that referenced this issue Jun 6, 2022
…ative-signed-values

#1839: use unsigned types for left shifts in BitPacker
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant