Access to traits' associated functions and constants from trait objects

[kotauskas]

This RFC proposes a relaxation of the trait object safety rules. More specifically, it specifies a minor syntax addition for accessing associated functions and constants from a trait object, as long as the size of the results is known at compile time (acheived by using Box<dyn Trait> instead of Self in associated functions and methods).

Rendered.

[kotauskas]

My bad. I applied the fixes, is there anything in the text that needs clarification or a correction right now?

[burdges]

I'll note that mem::size_of_val and mem::align_of_val already do exactly this: If given a trait object then they looks up the size and alignment in the vtable. I think they work fine even when the data portion of &self gets corrupted.

Also, if given [T] then mem::size_of_val computes the size using the fat pointer.

As an aside, You cannot make a trait object from a true DST in Rust, meaning no Box<dyn Hash>. https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=a5cc5857c2338bf6a3bba1fce2fb0df8 If we want that we'll need either

• some 24 byte wide fat dyn Trait, or else
• some dyn internal_size Trait that says the trait knows its own size.

[kotauskas]

I'll note that mem::size_of_val and mem::align_of_val already do exactly this: If given a trait object then they looks up the size and alignment in the vtable. I think they work fine even when the data portion of &self gets corrupted.

Also, if given [T] then mem::size_of_val computes the size using the fat pointer.

As an aside, You cannot make a trait object from a true DST in Rust, meaning no Box<dyn Hash>. https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=a5cc5857c2338bf6a3bba1fce2fb0df8 If we want that we'll need either

* some 24 byte wide `fat dyn Trait`, or else

* some `dyn internal_size Trait` that says the trait knows its own size.

@burdges Thanks for telling, I actually didn't know that trait objects store their size in a retrievable way. (You're talking about the "sized trait object" thing I wrote about in the future possibilities section, right?)

[kotauskas]

As an aside, you cannot make a trait object from a true DST in Rust, meaning no Box<dyn Hash> (https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=a5cc5857c2338bf6a3bba1fce2fb0df8). If we want that we'll need either:

• some 24 byte wide fat dyn Trait, or else
• some dyn internal_size Trait that says the trait knows its own size.

I think dyn Trait + ?Sized would be a good way of doing that.

[burdges]

Is trait Trait: ?Sized { } synonymous? Or should one always write dyn Trait + ?Sized?

We still choose between either

1. alter the vtable by replacing the size field with a size method, or else
2. expand the fat pointer with a size.

I suppose the fatter pointer works best for Box<dyn Trait+?Sized>.

[kotauskas]

Is trait Trait: ?Sized { } synonymous?

No. The Sized bound should be on the trait object rather than the trait because the property is specific only to trait objects, not the trait itself.

2. expand the fat pointer with a size

Feels like a bad idea, we don't want to change the layout of the fat pointer because that'd break code depending on makeshift TraitObject from #![feature(raw)] (trait object fat pointer introspection) in the stable channel (the TraitObject struct, copypasted into a crate). Even though such code might break anyway when multibound trait objects arrive, I think it's reasonable to put that off as long as possible, since I doubt that it'll get implemented any time soon. Furthermore, it's possible that multibound traits will receive special vtables which cover multiple traits behind one pointer. As long as we can avoid changing the trait object pointer ABI, we should.

alter the vtable by replacing the size field with a size method, or else

A good idea is to have two vtables per trait (Sized with a field and !Sized with a method). This way, the call to the method will be inlined in the case of Sized, which is the majority of cases.

[burdges]

An unsized type like [T] is literally unsized in that it does not know its own size, so if you cast a Box<[T]> or &[T] to a Box<dyn Borrow<[T]>> then you cannot store the size anywhere except inside the fat pointer, so the vtable pointer must enlarger fat pointer. Ain't so hard to add a UnsizedTraitObject type.

We've previous discussed types that manage their own size, but this requires another builtin trait like

unsafe trait SelfSized: !Sized {
    fn size(&self) -> usize;
}

I initially confused myself by poorly remembering those discussions, but doing self sized or truly unsized types requires far more work, and must avoid breaking size_of_val, etc.

---

Rust should add a SmallBox<T> type with the property that if size_of_val(&self) < 8 then self gets stored inside the pointer. In this, SmallBox<dyn Trait> currently works because the vtable knows size_of::<T>() and the fatter pointer design knows size_of_val(&self), but..

SmallBox<T> requires that size_of_val(&self) work without dereferencing the pointer since that usize contains arbitrary user data. If you cannot dereference the pointer then SmallBox<T> requires the negative reasoning T: !SelfSized.

Are small box optimizations so important? Yes:

SmallBox<dyn error::Error> would become an error type that (a) remains zero-cost when the underlying error type is smaller than 8 bytes, which covers most current error types, but (b) still supports expanding with arbitrarily complex error types, ala backtraces, etc.

We could even make SmallBox<T> work both with and without alloc, but without alloc SmallBox::new panics for values larger than 8 bytes. We could then make io::Error be SmallBox<dyn error::Error> so that io::* worked without std or even alloc.

We always have alloc available in practice, but actually supporting the niche cases without alloc creates significant overhead in Rust. We'd therefore save considerable work accross the ecosystem by simply making io::* available in this way.

[brain0]

What does this code do with your proposal?

trait Foo {
    const CONST: usize;
    
    fn assoc();
}

fn foo<T: Foo + ?Sized>() {
    let _ = T::CONST;
    T::assoc();
}

fn main() {
    foo::<dyn Foo>();
}

In Rust dyn Trait is a type that implements Trait. Your proposal does not describe how dyn Trait as a type would implement associated functions and constants.

[Centril]

@kotauskas

Feels like a bad idea, we don't want to change the layout of the fat pointer because that'd break code depending on makeshift TraitObject from #![feature(raw)] (trait object fat pointer introspection) in the stable channel (the TraitObject struct, copypasted into a crate). Even though such code might break anyway when multibound trait objects arrive, I think it's reasonable to put that off as long as possible, since I doubt that it'll get implemented any time soon.

Not that I have a particular reason to change the layout, but such code should never have been written as we have never given any guarantees of ABI stability for trait objects (or ABI guarantees in general unless explicitly stated). People who copy-pasted TraitObject should have known that it was unstable for a reason. People who intentionally attempt to circumvent non-guarantees and try to make the spec bend around them shouldn't get to complain about it later.

[comex]

It's a moot point. Adding an extra word to existing fat pointers is a non-starter for performance reasons. Adding a new kind of trait object type that would be fatter (dyn Trait + ?Sized) is a possibility, but that won't break compatibility, since no existing code is using such types.

[woshilapin]

Hi, as I happen to read and try to understand this RFC (but my Rust skills are clearly not enough 😄), I noticed a few typos. I hope it's not too early in the review process to tackle these kind of fix.

[kotauskas]

What does this code do with your proposal?

trait Foo {
    const CONST: usize;
    
    fn assoc();
}

fn foo<T: Foo + ?Sized>() {
    let _ = T::CONST;
    T::assoc();
}

fn main() {
    foo::<dyn Foo>();
}

In Rust dyn Trait is a type that implements Trait. Your proposal does not describe how dyn Trait as a type would implement associated functions and constants.

I added sections describing this. Thanks for pointing out!

[kotauskas]

Hi, as I happen to read and try to understand this RFC (but my Rust skills are clearly not enough 😄), I noticed a few typos. I hope it's not too early in the review process to tackle these kind of fix.

I think you'd have more luck understanding things one step at a time. There isn't anything that scarily complex in the RFC if you read it at a slow pace. Thanks for the corrections, by the way!

[burdges]

In Rust dyn Trait is a type that implements Trait. Your proposal does not describe how dyn Trait as a type would implement associated functions and constants.

We should probably stick to your actual RFC here @kotauskas so apologies for the derail, but..

If you've a &self method for a dyn Trait + ?Sized type then &self and &mut self must be fat pointers that contain the length, so vtables for dyn Trait + Sized and dyn Trait + !Sized require disjoint entry points for all methods. You might want one additional pointer for casts between the dyn Trait + Sized and dyn Trait + !Sized vtables.

There are no vtable changes for associated constants. Also SmallBox<dyn Trait> requires no vtable changes.

Initially I mistakenly imagined a !Sized vtable could provide some size method, but this fails: You cannot store the size anywhere except inside the fat pointer because current unsized types never contain their own size.

I think dyn Trait + ?Sized sounds fairly uninteresting right now because unsized types mostly provide an implementation detail for another type.

As hinted above, there was some limited past interest in "self sized" types with multiple unsized fields within the same allocation:

struct Foo { as: [A], bs: [B], }
struct Bar { 
    a_len: usize,
    b_len: usize,
    as: [A; dyn a_len], 
    bs: [B; dyn u_len],
}

An &Foo must be a fatter pointer with two lengths for 24 bytes, and makes &Foo as &dyn Trait+?Sized impossible even with our above discussions, while a &Bar needs only a simply pointer, but requires representing [T] types via these [T, dyn n]. You can implement Bar today with some unsafe code though, so not really high priority.

I think your RFC here sounds much more interesting than any of this unsized stuff actually. ;)

[PoignardAzur]

This RFC really doesn't seem bulletproof enough.

I think the author is seriously underestimating the number of edge cases and semantic questions that need to be addressed here, and is approaching the question of trait objects with a mindset of "Here's the feature I want and here is how we can implement it"; rather than methodically inspecting the solution space.

For instance, here is one edge case:

trait Foo {
    const CONST: usize;
}

fn foo<T: Foo + ?Sized>(t1: &T, t2: &T) -> usize {
    T::CONST
}


fn main() {
    let tobject1: Box<dyn Foo> = Box::from(...);
    let tobject2: Box<dyn Foo> = Box::from(...);

    foo(&tobject1, &tobject2);
}

Does this code compile? Does it fail? If it fails, what is the error message? If it compiles, which vtable does foo use?

(keeping in mind that, if dyn Foo is considered trait-safe, then it should be possible to pass to any function taking a T parameter; and calling foo should not fail to compile based on something that isn't in foo's type signature)

I'm not saying these questions have no possible answers.

I'm saying that this RFC should be backed up by a through analysis of Rust's template system; an analysis which as a matter of course would find and address edge cases like the one above.

As it is, I think the RFC is simply under-specified for how broad its effects are.

[burdges]

We cannot compile that code in current Rust:

• error[E0277]: the trait bound std::boxed::Box<dyn Foo>: Foo is not satisfied
• error[E0277]: the trait bound &dyn Foo: Foo is not satisfied

I've forgotten if previous discussions around that issue ever reached a conclusion of whether addressing E0277 is desirable.

[kotauskas]

I think the author is seriously underestimating the number of edge cases and semantic questions that need to be addressed here, and is approaching the question of trait objects with a mindset of "Here's the feature I want and here is how we can implement it"; rather than methodically inspecting the solution space.

As it seems now, I truly am. That's what the RFC process is, though: no one can realistically know every single edge case when writing an RFC. That being said, it's my first ever time writing an RFC and if it's needed, I'm okay with discussing every edge case to be found, for as long as needed. It's likely that the more people review the RFC, the less the chance of some edge-cases not being found.

For instance, here is one edge case:

trait Foo {
    const CONST: usize;
}

fn foo<T: Foo + ?Sized>(t1: &T, t2: &T) -> usize {
    T::CONST
}


fn main() {
    let tobject1: Box<dyn Foo> = Box::from(...);
    let tobject2: Box<dyn Foo> = Box::from(...);

    foo(&tobject1, &tobject2);
}

Does this code compile? Does it fail? If it fails, what is the error message? If it compiles, which vtable does foo use?

(keeping in mind that, if dyn Foo is considered trait-safe, then it should be possible to pass to any function taking a T parameter; and calling foo should not fail to compile based on something that isn't in foo's type signature)

I added a section about this, called "the vtable ambiguity rule". This should fully cover this edge case.

I'm not saying these questions have no possible answers.

I'm saying that this RFC should be backed up by a through analysis of Rust's template system; an analysis which as a matter of course would find and address edge cases like the one above.

As it is, I think the RFC is simply under-specified for how broad its effects are.

There's nothing bad in having an RFC in development phase for a long time. Doing this analysis collectively via the RFC process is much more realistic than simply relying on a single participant digging through the entire template system and finding all edge cases, something that can be done collectively in a much shorter period of time. Once again, that's why the RFC process exists.

[RustyYato]

The compiler may have assumed that the v-table has the same layout as a [usize; N] where N is the number of elements in the v-table (drop info, and functions).

[kotauskas]

I doubt it ever did. Something like this:

struct Vtable {
    size: usize,
    methods_and_associated_fns: [*const (), N],
    constants: (TypeOfConstant1, TypeOfConstant2, TypeOfConstantN)
}

would still work just fine for that purpose.

[RustyYato]

One way around this is through arbitrary self-types. Take a raw pointer as the receiver (although this isn't supported just yet, it could be). This way it is illegal to try and access the raw pointer (because it could be invalid), but even raw pointers have a valid vtable, so you can rely on that to dispatch on the required parts.

This way you must construct at least a raw pointer to access associated functions/constants.

This in combo with rust-lang/rust#64490 would allow you to ignore the borrow checker for these sorts of associated functions/constants.

[RustyYato]

You can always convert a trait into one that can be used as a trait object.

// lib a
trait Foo {
    fn foo() {}
    // ...
}

// lib b
trait DynFoo {
    fn dyn_foo(&self) {}
    // ...
}

impl<T: Foo + ?Sized> DynFoo for T {
    fn dyn_foo(&self) { T::foo() }
}

Then, whenever you need dynamic dispatch you use DynFoo, and this can be seemlessly integrated into an existing code base.

[kotauskas]

That workaround is also wrong from a semantic standpoint, since trait methods by design should use the fields of the trait object in some way, either directly or indirectly. Again, an unused &self just to combat language limitations introduces confusion.

As mentioned by the RFC, the "newtrait" solution is not less ugly. Why create a macro-worthy construct if you can implement the same thing idiomatically with a little bit of new syntax?

[burdges]

I should've guessed that, thank you! :) I'd think this RFC dies with the example in #2886 (comment) then.

If one really needs this, then they've many available tools, like Any::type_id, type erased traits, etc. At some point, someone could write helpers for making type erased traits.

[kotauskas]

I've removed support for using trait objects with associated members in compile-time generics. The intended use of such trait objects in these generics is wrapping them into non-generic newtypes which expose the trait object dynamic dispatch via their associated functions. This means that the RFC is now much less broad. Hopefully this prevents this sort of unsolvable issues.

[burdges]

I've two questions:

First, could we make associated types and constants object safe?

trait Foo {
    const CONST: usize where Self: Sized;
    type Target where Self: Sized;
}

Second, we cannot declare default implementations object safe, meaning this trait cannot be object safe:

trait Foo {
    fn assoc() { .. }
}

But could we perhaps declare object safe versions of non-object safe items, like

impl Foo for dyn Foo {
    const CONST: usize = ...;
    type Target = Box<dyn Foo>;
    fn assoc() -> Box<dyn Foo> { .. }
}

Yes, we could just mark this method where Self: Sized in the trait object and then declare an inherent method

impl dyn Foo {
    fn assoc() { .. }
}

except this cannot from a T: Trait+?Sized context.

I think one answer goes

impl<'a> Foo for &'a dyn Foo { .. }
impl<'a> Foo for &'a mut dyn Foo { .. } // delegate anything & to &
impl Foo for Box<dyn Foo { .. }> { .. } // delegate everything to & and &mut

I suspect impl Foo for dyn Foo { .. } might've some advantages though, not sure. If we could enumerate those advantages, then maybe such specialization of impl Foo for dyn Foo provides a sensible goal for after the main specialization work wraps up?

[PoignardAzur]

I've removed support for using trait objects with associated members in compile-time generics. The intended use of such trait objects in these generics is wrapping them into non-generic newtypes which expose the trait object dynamic dispatch via their associated functions. This means that the RFC is now much less broad. Hopefully this prevents this sort of unsolvable issues.

Not really.

trait Foo {
    fn get_value() -> usize;
}

fn foo<T: Foo + ?Sized>(t1: &T, t2: &T) -> usize {
    T::get_value();
}

fn main() {
    let tobject1: Box<dyn Foo> = Box::from(...);
    let tobject2: Box<dyn Foo> = Box::from(...);

    foo(&tobject1, &tobject2);
}

---

The fundamental problem at the heart of this RFC is the contradiction between the following rules:

• Given an object-safe trait MyTrait, a trait object dyn MyTrait can be passed to any function expecting T: MyTrait.
• Template functions taking MyTrait expect to be able to call MyTrait::static_function (and, in existing code, do so).
• Template functions that compile once must always compile. Template functions should never produce compile errors at monomorphisation.
• New rule: We want traits to be able to have static methods and still be object-safe.

These four rules, taken together, are fundamentally incompatible. This isn't a small edge case that can be ironed out, this is a fundamental contradiction. Any RFC to expand type-safe traits needs to address it.

---

My personal solution would be to have two types of template parameters: regular types and dyn? types. The syntax would look like:

trait FooBar {
    type AssociatedType;

    fn get_value(&self) -> AssociatedType;
    fn static_do_thing();
}

fn foo<T: Foo + ?dyn>(t1: &T, t2: &T) {
    let v1 = t1.get_value();   // ok
    let v2 = t2.get_value();   // ok

    T::static_get_value();     // error, T might be `dyn Foo`
    v1 = v2;                   // error, `t1::AssociatedType` and `t2::AssociatedType` might be different types
}

But that's a pretty huge RFC, and it would probably come with a lot of breaking changes that would require an edition switch.

[RustyYato]

My personal solution would be to have two types of template parameters: regular types and dyn? types. The syntax would look like:

You can already get that behaviour by adding a Self: Sized bound on static_do_thing.

[PoignardAzur]

Well, yes, but in my imaginary proposal you'd be able to do things you can't do right now (eg have associated types and call static methods in specific circumstances).

[burdges]

There are several more tangible goals here I think:

First, dyn Trait<const SIZE=32> does not currently work. It's merely a rustc bug I suppose, not a language design issue, but this sounds like the first task.

Second, trait aliases should similarly work, ala trait Bar = Foo<const SIZE=32>; Again, there is no new langauge design work here, but not sure if these first two need chalk.

Third, trait objects could exclude associated types and constants almost save exactly like they handle methods non object safe methods:

trait Foo {
    const CONST: usize where Self: Sized;
    type Target where Self: Sized;
}

We're closer to language design here, but this still sounds pretty uncontentious.

Fourth, an explicit impl Trait for dyn Trait { .. } could act like extension or specialization of the implicit impl provided by the compiler, like I explained above in #2886 (comment) so users can provide a similar type erased interface. I think this requires the most design work, but it's also the closest to this RFC in spirit.

[burdges]

I think another tangible goal might be &self = 0 methods:

impl MyType<..> {
    fn size_n_align(&self = 0) { (size_of_val(self), align_of_val(self)) }
}

Any &self = 0 methods is safe to call on a bare vtable pointer with an invalid/null data pointer. Associated constants act like &self = 0 methods. And &self=0 trait methods for associated types act like &self = 0 methods. You'd invoke them list v.SOME_CONST however because <dyn Trait>::SOME_CONST makes no sense.

There are already several such methods in std including: Any::type_id could easily become an associated constant. Interestingly, size_of_val and align_of_val act like &self = 0 methods, never deferencing the pointer itself, but only using the fat pointer and type for &[T], and use the vtable for dyn Trait.

In practice, rust would never use the syntax &self = 0, but instead some VTableOnlyPtr<&dyn Trait> type to which you cast real pointers.

[RustyYato]

@burdges witb arbitrary_self_types you can use raw pointers as recievers with much of the same semantics

[burdges]

True. We actually could provide *const dyn Foo with some mechanism for accessing the vtable for &dyn Foo::Target as a *const dyn Bar then, where

trait Foo {
    type Target: Bar where Self: Sized;
}

[kotauskas]

The latest commit reworks the concept of object safety entirely, patching the edge cases related to use of generics with trait objects. While this may bring increased complexity to the language, this may be the only way this RFC could ever work.

[nikomatsakis]

Hello! In today's "Backlog Bonanza" meeting, we discussed this RFC. Our consensus was that while we do feel the pain of having to add methods for access to constants and so forth from dyn types, we think that fixing this would be a non-trivial undertaking and that we don't really have the "time budget" to do that work properly right now. It's also true that the object safety rules have some known flaws (e.g., rust-lang/rust#57893) and that we are reluctant to make changes to them until we've gotten a better handle on their current state. In short, the time is not ripe to pursue this right now, though it may be something we would want to pursue later.

For those reasons, I am going to move to close. Thanks to the author for the suggestion.

@rfcbot fcp close

[rfcbot]

Team member @nikomatsakis has proposed to close this. The next step is review by the rest of the tagged team members:

• @cramertj
• @joshtriplett
• @nikomatsakis
• @pnkfelix
• @scottmcm
• @withoutboats

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[nikomatsakis]

@rfcbot fcp reviewed -- I checked the boxes for those folks who were in the backlog bonanza meeting.

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[sdroege]

GObject (the runtime type system behind GTK, GStreamer, etc.) also allows for such things, which shouldn't be much of a surprise as the vtable is manually managed there.

You can put fields into the vtable (-> associated constants) or function pointers without an instance parameter (-> associated functions). This is used in quite a few places in GStreamer in practice.

Vala, a C#-inspired language on top of GObject that compiles down to C handles these as with the class keyword AFAIU. Not to be confused with virtual, which requires an instance parameter, or static which can't be overridden/implemented by subclasses / interface implementations but exists only exactly once.

public class MyClass {
    public class int some_field;
    public class void some_function() { ... }
}

[rfcbot]

The final comment period, with a disposition to close, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

The RFC is now closed.