Explain one-past-the-end pointer in std library

[xizheyin]

Fixes #138969

r? libs

[jieyouxu]

(Not a libs reviewer, just a drive-by remark)

[lolbinarycat]

I think this is the wrong place to add this remark.

I think the existing docs are correct, just confusingly worded, specifically this part on offset:

If the computed offset is non-zero, then self must be derived from a pointer to some allocated object, and the entire memory range between self and the result must be in bounds of that allocated object. In particular, this range must not “wrap around” the edge of the address space.

I believe that "the range between self and the result" is meant to be a half-open range, so all the bytes up to, but not including the result must be within the bounds of the allocated object.

[xizheyin]

The original documentation was correct, it just wasn't clear about this boundary case, so that caused the confusion. There seems to be ambiguity in what different people understand. The additions were just to add explanations to alleviate the confusion. : )

Do you have any good suggestions for modifications? @lolbinarycat

[lolbinarycat]

For one, I don't think this should be addressed under allocated object, since by my reading this has nothing to do with the definition of an allocated object. Additionally, calling a pointer valid has a very specific meaning, which has to do with if you can dereference it, so "valid, but you can't dereference it" doesn't really make sense.

I think something like this would be good (additions italicized):

If the computed offset is non-zero, then self must be [derived from][crate::ptr#provenance] a pointer to some [allocated object], and the entire memory range between self and the result (not including the result itself) must be in bounds of that allocated object. In particular, this range must not "wrap around" the edge of the address space.

We can then clarify the generality of the example later on:

This means implies, for instance, that offsetting to one past the end of an allocation is always safe, such as in vec.as_ptr().add(vec.capacity()) (for vec: Vec<T>) is always safe.

I'm not on t-libs, so ultimately I have no authority on this (and maybe we should wait to see what someone from t-libs thinks about this), but I did rewrite the "Pointer to reference conversion" section, so I have a bit of experience with pointer semantics.

[lolbinarycat]

I'm pretty sure t-libs is actually the wrong team for this.

@rust-lang/opsem

[RalfJung]

We are not using "one-past-the-end" terminology in Rust (I think it's terrible terminology).

Also, as far as I know all the existing docs already handle this properly.

[RalfJung]

I believe that "the range between self and the result" is meant to be a half-open range, so all the bytes up to, but not including the result must be within the bounds of the allocated object.

Correct. Happy for suggestions for how to clarify this, but I think calling this "past the end" is needlessly confusing. The pointer is not "past" the end, it is at the end.

For one, I don't think this should be addressed under allocated object,

👍 , the confusion is entirely around the offset docs, so only those docs should be change (if at all).

[lolbinarycat]

@xizheyin sounds like the only change that is needed is the "(not including the result itself)" line I mentioned earlier.

[xizheyin]

Make sense! Thank you for your help. I will revise this PR. @lolbinarycat @RalfJung

[RalfJung]

@xizheyin sounds like the only change that is needed is the "(not including the result itself)" line I mentioned earlier.

I don't think that's better than what we have now.

I'd rather state explicitly that when we talk about a "range" (as we already do), we mean a a..b range, i.e. left-inclusive right-exclusive -- as usual in Rust.

[lolbinarycat]

Yeah, that might make more sense, since the allocated object docs are also in terms of ranges.

[xizheyin]

I'd rather state explicitly that when we talk about a "range" (as we already do), we mean a a..b range, i.e. left-inclusive right-exclusive -- as usual in Rust.

Thank you. I've revised this submission to add a sentence of explanation. That doesn't seem to be causing any confusion. Does that make sense? @lolbinarycat @RalfJung

[RalfJung]

LGTM, thanks. :)

@bors r+ rollup

[bors]

📌 Commit 861f425 has been approved by RalfJung

It is now in the queue for this repository.

[xizheyin]

OH, I just squashed the commits, will it matter?

[RalfJung]

It's fine, this just kicked the PR out of the queue so I have to put it back in. :)

@bors r+ rollup

[bors]

📌 Commit 074edbd has been approved by RalfJung

It is now in the queue for this repository.

[xizheyin]

Thanks, I learned a lot. 😊