Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid Unsafe.As in BitConverter #112616

Merged
merged 4 commits into from
Mar 7, 2025
Merged

Conversation

xtqqczze
Copy link
Contributor

@xtqqczze xtqqczze commented Feb 16, 2025

Doesn't depend upon implementation specific behaviour, i.e. that the array data is properly aligned.

@dotnet-policy-service dotnet-policy-service bot added the community-contribution Indicates that the PR has been added by a community member label Feb 16, 2025
Copy link
Contributor

Tagging subscribers to this area: @dotnet/area-system-numerics
See info in area-owners.md if you want to be subscribed.

@stephentoub
Copy link
Member

Why is this better?

@xtqqczze
Copy link
Contributor Author

We should really just use MemoryMarshal.Write, but the codegen is suboptimal, see sharplab

@xtqqczze
Copy link
Contributor Author

Why is this better?

It doesn't depend upon implementation specific behaviour, i.e. that the array data is properly aligned.

@hughbe
Copy link
Contributor

hughbe commented Feb 16, 2025

We should really just use MemoryMarshal.Write, but the codegen is suboptimal, see sharplab

Is this something that can be improved? Might be worth opening an issue :)

@stephentoub
Copy link
Member

Why is this better?

It doesn't depend upon implementation specific behaviour, i.e. that the array data is properly aligned.

In the future, please include your reasoning for the proposed changes in the PR description. Thanks.

@EgorBo
Copy link
Member

EgorBo commented Feb 16, 2025

We should really just use MemoryMarshal.Write, but the codegen is suboptimal, see sharplab

Looks like a simple addressing mode was not contained. It's just that this PR replaces one unsafe code with another unsafe (a tiny bit better, but still).
These APIs allocate, it is unlikely an extra lea makes perf difference here

@xtqqczze
Copy link
Contributor Author

We should really just use MemoryMarshal.Write, but the codegen is suboptimal, see sharplab

Looks like a simple addressing mode was not contained. It's just that this PR replaces one unsafe code with another unsafe (a tiny bit better, but still). These APIs allocate, it is unlikely an extra lea makes perf difference here

@jakobbotsch do we have an issue tracking this?

@jakobbotsch
Copy link
Member

@jakobbotsch do we have an issue tracking this?

I don't think we do. Feel free to open one.

@xtqqczze xtqqczze changed the title Avoid Unsafe.As in BitConvertor Avoid Unsafe.As in BitConverter Feb 19, 2025
@xtqqczze
Copy link
Contributor Author

@MihuBot

@EgorBo
Copy link
Member

EgorBo commented Feb 27, 2025

Thanks! The diffs look sane considering these APIs allocate (and tracked by #104538). I have a general question to @stephentoub @jkotas

So in this case MemoryMarshal.Write is perfectly safe, but if we mark it as "[CallersRequiresUnsafeContext]", it's likely going to require unsafe for it. We have BinaryPrimitives.* APIs which are 100% memory safe, the problem that they're Endianess agnostic:

BinaryPrimitives.WriteInt32LittleEndian
BinaryPrimitives.WriteInt32BigEndian

Do we want to add new APIs there which check the endianess inside?

+BinaryPrimitives.WriteInt32

So then for any perfectly safe Read/Write for primitives we can use BinaryPrimitives.

@stephentoub
Copy link
Member

if we mark it as "[CallersRequiresUnsafeContext]"

What's the rationale for why MemoryMarshal.Write(Span<T>, T) would be marked as unsafe? (This gets to our definition for what exactly we're covering with CallersRequiresUnsafeContext.)

@EgorBo
Copy link
Member

EgorBo commented Feb 27, 2025

What's the rationale for why MemoryMarshal.Write(Span, T) would be marked as unsafe?

I think mainly to protect from cases where structs with paddings/non-blittable fields (like bool) are used as T (probably can be protected by some theoretical where T : bitwise-writeable) resulting in storing garbage (or maybe even secrets?) into a byte stream. If whatever attribute we come up with can relax its behavior for primitives - that'd solve the issue as well

@stephentoub
Copy link
Member

What's the rationale for why MemoryMarshal.Write(Span, T) would be marked as unsafe?

I think mainly to protect from cases where structs with paddings/non-blittable fields (like bool) are used as T (probably can be protected by some theoretical where T : bitwise-writeable) resulting in storing garbage (or maybe even secrets?) into a byte stream. If whatever attribute we come up with can relax its behavior for primitives - that'd solve the issue as well

Before we start talking about introducing new APIs, we should agree on the definition.

If we decide that such functionality needs to be considered "unsafe", and if the relevant APIs are considered important enough, and if there isn't an existing counterpart, we could certainly introduce additional methods. But there are several ifs there. I'm also not sure how such a BinaryPrimitives.WriteInt32 would differ from the existing BitConverter.TryWriteBytes, other than the try-ness.

@EgorBo
Copy link
Member

EgorBo commented Feb 27, 2025

Good point!

I'm also not sure how such a BinaryPrimitives.WriteInt32 would differ from the existing BitConverter.TryWriteBytes, other than the try-ness.

TBH, it's a bit confusing we have these primitives spread in BitConverter, BinaryPrimitives, BitOperations, MemoryMarshal 🙂

@jkotas
Copy link
Member

jkotas commented Feb 27, 2025

resulting in storing garbage (or maybe even secrets?) into a byte stream.

If the bar for CallersRequiresUnsafeContext is memory safety and nothing else, I do not think this justification would meet the bar. This is not a memory safety violation. You can leak secrets into a byte stream in 100% safe code in many different ways.

BinaryPrimitives.WriteInt32 would differ from the existing BitConverter.TryWriteBytes

Can this change use TryWriteBytes and friends instead? It would make it more future proof.

@EgorBo
Copy link
Member

EgorBo commented Mar 1, 2025

Can this change use TryWriteBytes and friends instead? It would make it more future proof.

@xtqqczze could you please try that?

@EgorBo
Copy link
Member

EgorBo commented Mar 5, 2025

@MihuBot

@xtqqczze
Copy link
Contributor Author

xtqqczze commented Mar 5, 2025

@MihuBot

@xtqqczze
Copy link
Contributor Author

xtqqczze commented Mar 6, 2025

Can this change use TryWriteBytes and friends instead? It would make it more future proof.

@xtqqczze could you please try that?

Diffs are as expected with this approach

@EgorBo EgorBo merged commit 496824c into dotnet:main Mar 7, 2025
138 of 140 checks passed
@EgorBo
Copy link
Member

EgorBo commented Mar 7, 2025

@xtqqczze Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-System.Numerics community-contribution Indicates that the PR has been added by a community member reduce-unsafe
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants