Support custom allocators in `Box` #77187

TimDiekmann · 2020-09-25T14:52:47Z

This pull request requires a crater run.

Prior work:

Currently blocked on:

~~Stability annotations on generic parameters (take 2.5) #77118~~
~~Implement clauses to allow multi-parameter fundamental types chalk#615 (Update to chalk 0.31 #77515)~~

TimDiekmann · 2020-09-25T15:09:24Z

The chalk error is legit in the sense that we haven't defined what fundamental means for a type with multiple type parameters. I guess we will ignore all but the first, but we're going to have to spend at least some time thinking about how to use this to break coherence :)

What is the state of this?

jackh726 · 2020-09-25T17:56:06Z

@nikomatsakis wrote in #71873 (comment)

The chalk error is legit in the sense that we haven't defined what fundamental means for a type with multiple type parameters. I guess we will ignore all but the first, but we're going to have to spend at least some time thinking about how to use this to break coherence :)

What is the state of this?

No change. But I'll be sure to bring this up in the wg-traits meeting on tuesday.

TimDiekmann · 2020-09-25T18:11:12Z

That would be great, thank you!

TimDiekmann · 2020-09-27T20:30:05Z

Rebased onto master to include #77118

library/alloc/src/boxed.rs

TimDiekmann · 2020-09-29T23:13:20Z

library/alloc/src/boxed.rs

    where
-        T: 'a, // Technically not needed, but kept to be explicit.
+        A: 'a,


I added a test, which ensures, that the allocator must not be moved while the leaked data is in use.

jackh726 · 2020-09-29T23:30:52Z

Just wanted to check in to say we did talk about this today at our @rust-lang/wg-traits meeting. We worked out a design that we're hoping to get into Chalk by next release on Saturday. Then it'll just be a Chalk upgrade and this should be unblocked from that.

TimDiekmann · 2020-09-29T23:40:40Z

That's awesome, thank you!

TimDiekmann · 2020-09-30T01:04:24Z

Rebased onto master to include #77289

bors · 2020-10-02T03:33:31Z

☔ The latest upstream changes (presumably #77436) made this pull request unmergeable. Please resolve the merge conflicts.

Note that reviewers usually do not review pull requests until merge conflicts are resolved! Once you resolve the conflicts, you should change the labels applied by bors to indicate that your PR is ready for review. Post this as a comment to change the labels:

@rustbot modify labels: +S-waiting-on-review -S-waiting-on-author

library/alloc/src/boxed.rs

bors · 2020-10-05T03:10:00Z

☔ The latest upstream changes (presumably #77466) made this pull request unmergeable. Please resolve the merge conflicts.

Note that reviewers usually do not review pull requests until merge conflicts are resolved! Once you resolve the conflicts, you should change the labels applied by bors to indicate that your PR is ready for review. Post this as a comment to change the labels:

@rustbot modify labels: +S-waiting-on-review -S-waiting-on-author

Remove `Box::leak_with_alloc` Add leak-test for box with allocator Rename `AllocErr` to `AllocError` in leak-test Add `Box::alloc` and adjust examples to use the new API

TimDiekmann · 2020-10-07T01:15:43Z

Rebased onto master to include #77515.

@rustbot modify labels: +S-waiting-on-review -S-blocked

rust-highfive · 2020-10-26T23:23:36Z

📣 Toolstate changed by #77187!

Tested on commit fd54259.
Direct link to PR: #77187

💔 miri on windows: test-pass → test-fail (cc @oli-obk @eddyb @RalfJung).
💔 miri on linux: test-pass → test-fail (cc @oli-obk @eddyb @RalfJung).

@oli-obk

Tested on commit rust-lang/rust@fd54259. Direct link to PR: <rust-lang/rust#77187> 💔 miri on windows: test-pass → test-fail (cc @oli-obk @eddyb @RalfJung). 💔 miri on linux: test-pass → test-fail (cc @oli-obk @eddyb @RalfJung).

RalfJung · 2020-10-27T09:53:40Z

library/alloc/src/boxed.rs

+
+        // The box is unitiliazed later when moving out the allocator. The pointer is stored
+        // beforehand.
+        let ptr = b.0;


I'm afraid this code broke Stacked Borrows. The code before was very careful to go through a mutable reference, as the comment explains. That comment was retained but the mutable reference is no longer used, so this code no longer works.

I'll submit a PR to fix this. In the future, when you find comments like this and are not sure what to do about them, feel free to ping me. :)

test Box::into_raw aliasing Directly test aliasing problems caused by `Box::into_raw` issues (like we have them again right now due to rust-lang/rust#77187, but the pinned rustc is older than that so this should still be able to land).

fix Box::into_unique rust-lang#77187 broke Stacked Borrows pointer tagging around `Box::into_unique` (this is caused by `Box` being a special case in the type system, which box-internal code needs to account for). This PR fixes that. r? `@Amanieu` Cc `@TimDiekmann` Fixes rust-lang#78419.

H2CO3 · 2020-11-28T18:08:48Z

I would also like to raise awareness about the issue of this breaking ABI here, more in my comment over at the related PR for Vec.

yoshuawuyts · 2021-01-15T13:15:07Z

Is there a write-up available on this design? I've been reading through the custom allocator PRs, and don't have a good sense why for example #58457 was abandoned in favor of the current design. The designs appear to differ, and I'd like to better understand why that choice was made.

edit: I guess the right place to ask would be in #32838 which tracks this work. I didn't see the issue linked from here, so I missed it. Apologies!

edit: The allocator WG appears to be tracking the methods introduced in this PR here: rust-lang/wg-allocators#7. This is the exact resource I was looking for.

BTreeMap: Support custom allocators (v1.5) Related: rust-lang/wg-allocators#7 https://github.com/TimDiekmann/alloc-wg Blocked on: ~~rust-lang#77187~~ ~~rust-lang#78459~~ ~~rust-lang#95036~~ previous: rust-lang#77438

RalfJung · 2024-08-31T09:20:39Z

FWIW this PR caused a lot of pain in the compiler, a long stream of ICEs, accidental UB -- altogether causing a lot of work for a lot of people. See #95453 for more details.

This should never have landed without a discussion with the compiler team on how to support Box with an extra field. Box is a language-primitive type, its definition cannot change without adjusting many parts of the compiler and even the language spec.

rust-highfive assigned Amanieu Sep 25, 2020

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Sep 25, 2020

TimDiekmann marked this pull request as ready for review September 27, 2020 15:50

TimDiekmann mentioned this pull request Sep 27, 2020

Unstable type parameters on stable types rust-lang/wg-allocators#2

Closed

TimDiekmann force-pushed the box-alloc branch from c0e060b to 39bf29f Compare September 27, 2020 20:28

nagisa reviewed Sep 28, 2020

View reviewed changes

library/alloc/src/boxed.rs Outdated Show resolved Hide resolved

nagisa reviewed Sep 28, 2020

View reviewed changes

library/alloc/src/boxed.rs Outdated Show resolved Hide resolved

TimDiekmann commented Sep 29, 2020

View reviewed changes

jackh726 mentioned this pull request Sep 29, 2020

Implement clauses to allow multi-parameter fundamental types rust-lang/chalk#615

Closed

TimDiekmann force-pushed the box-alloc branch from 44cfe3d to db359a0 Compare September 30, 2020 01:04

exrook mentioned this pull request Oct 2, 2020

BTreeMap: Support custom allocators #77438

Closed

TimDiekmann mentioned this pull request Oct 3, 2020

Tracking Issue for structs which needs an allocator rust-lang/wg-allocators#7

Open

15 tasks

exrook reviewed Oct 5, 2020

View reviewed changes

library/alloc/src/boxed.rs Outdated Show resolved Hide resolved

Support custom allocators in Box

f288cd2

Remove `Box::leak_with_alloc` Add leak-test for box with allocator Rename `AllocErr` to `AllocError` in leak-test Add `Box::alloc` and adjust examples to use the new API

TimDiekmann force-pushed the box-alloc branch from dacca08 to f288cd2 Compare October 7, 2020 01:15

rustbot removed the S-blocked Status: Marked as blocked ❌ on something else such as an RFC or other implementation work. label Oct 7, 2020

rust-highfive mentioned this pull request Oct 26, 2020

miri no longer builds after rust-lang/rust#77187 #78419

Closed

bors merged commit fd54259 into rust-lang:master Oct 26, 2020

rustbot added this to the 1.49.0 milestone Oct 26, 2020

TimDiekmann deleted the box-alloc branch October 26, 2020 23:23

This was referenced Oct 26, 2020

Associate an allocator to boxes #58457

Closed

Compat of adding parameters to stable types rust-lang/wg-allocators#1

Closed

This was referenced Oct 26, 2020

Rework diagnostics for wrong number of generic args (fixes #66228 and #71924) #77524

Merged

BTreeMap: replace Root with NodeRef<Owned, ...> #78104

Merged

RalfJung reviewed Oct 27, 2020

View reviewed changes

RalfJung mentioned this pull request Oct 27, 2020

test Box::into_raw aliasing rust-lang/miri#1602

Merged

RalfJung mentioned this pull request Oct 27, 2020

fix Box::into_unique #78446

Merged

exrook mentioned this pull request Oct 28, 2020

ICE when using Box<T, A> with non-zero sized A #78459

Closed

RalfJung mentioned this pull request Oct 28, 2020

BTree's insert_fit breaks pointer provenance rules #78477

Closed

ozkriff mentioned this pull request Nov 1, 2020

Ecosystem At Risk: Custom Allocators (and more?) in std rust-gamedev/wg#60

Closed

TimDiekmann mentioned this pull request Nov 15, 2020

Rename AllocRef to Allocator and alloc to allocate rust-lang/wg-allocators#76

Closed

KodrAus mentioned this pull request Nov 18, 2020

Add support for custom allocators in Vec #78461

Merged

exrook mentioned this pull request Jun 14, 2022

BTreeMap: Support custom allocators (v1.5) #98103

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support custom allocators in `Box` #77187

Support custom allocators in `Box` #77187

TimDiekmann commented Sep 25, 2020 •

edited

Loading

TimDiekmann commented Sep 25, 2020

jackh726 commented Sep 25, 2020

TimDiekmann commented Sep 25, 2020

TimDiekmann commented Sep 27, 2020

TimDiekmann Sep 29, 2020

jackh726 commented Sep 29, 2020

TimDiekmann commented Sep 29, 2020

TimDiekmann commented Sep 30, 2020

bors commented Oct 2, 2020

bors commented Oct 5, 2020

TimDiekmann commented Oct 7, 2020

rust-highfive commented Oct 26, 2020

RalfJung Oct 27, 2020

H2CO3 commented Nov 28, 2020

yoshuawuyts commented Jan 15, 2021 •

edited

Loading

RalfJung commented Aug 31, 2024

Support custom allocators in Box #77187

Support custom allocators in Box #77187

Conversation

TimDiekmann commented Sep 25, 2020 • edited Loading

Prior work:

TimDiekmann commented Sep 25, 2020

jackh726 commented Sep 25, 2020

TimDiekmann commented Sep 25, 2020

TimDiekmann commented Sep 27, 2020

TimDiekmann Sep 29, 2020

Choose a reason for hiding this comment

jackh726 commented Sep 29, 2020

TimDiekmann commented Sep 29, 2020

TimDiekmann commented Sep 30, 2020

bors commented Oct 2, 2020

bors commented Oct 5, 2020

TimDiekmann commented Oct 7, 2020

rust-highfive commented Oct 26, 2020

RalfJung Oct 27, 2020

Choose a reason for hiding this comment

H2CO3 commented Nov 28, 2020

yoshuawuyts commented Jan 15, 2021 • edited Loading

RalfJung commented Aug 31, 2024

Support custom allocators in `Box` #77187

Support custom allocators in `Box` #77187

TimDiekmann commented Sep 25, 2020 •

edited

Loading

yoshuawuyts commented Jan 15, 2021 •

edited

Loading