RFC: Partial Types (v2)

[VitWW]

This RFC proposes Partial Types as universal and type safe solution to "partial borrowing" like problems.
This RFC is a second try and it is based on Partial Types (v1) #3420
But I made it much nicer and easier to read.
I also add new abilities, and add a lot of simplifications in syntax.

This RFC makes a guarantee to borrow-checker that by & var.{x,y} or &mut var.{z,t} borrowing the whole variable and pretending to borrow just several fields is fully safe.

Rendered

struct Point {x: f64, y: f64, was_x: f64, was_y: f64}

let mut p1_full = Point {x: 1.0, y: 2.0, was_x: 4.0, was_y: 5.0};
    // p1_full : Point;

let p_just_x = Point {x: 1.0};
    // partial initializing
    // p_just_x : Point.%{x, %unfill};

let ref_p_just_x = & p_just_x;
    // partial referencing
    // ref_p_justx : & Point.%{x};

// late initialize unfilled fields
p_just_x.y let= 6.0;
     // p_just_x : Point.%{x, y, %unfill};
p_just_x.was_y let= 14.0;
     // p_just_x : Point.%{x, y, was_y, %unfill};
p_just_x.was_x let= 76.0;
     // p_just_x : Point.%full;
     // p_just_x : Point;

// partial parameters   
fn x_restore(&mut p1 : &mut Point.%{was_x, %any}, & p2 : & Point.%{x, %any}) {
    *p1.x = *p2.was_x;
}

// partial arguments
x_restore(&mut p1_full, & p1_full);

where

• %permit, %miss and %deny are field access;
• .%{..} is a detailed type access;
• %any is quasi-field filter

This is an alternative to Partial Types (v1) #3420, Partial borrowing issue#1215, View patterns internals#16879, Permissions #3380, Field projection #3318, Fields in Traits #1546, ImplFields issue#3269

[AaronKutch]

I just had an idea, I haven't read too much in this area so I don't know if something like this has already been thought of.

struct Vec<T> {
    len: usize,
    cap: usize,
    data: InternalDetails<T>,
}

impl Vec<T> {

    fn example0(param: &Self) {/* can use `param.len`, etc immutably */}
    // the above desugars into a future more finely grained thing that the compiler tracks
    fn example0(param: Self {&len, &cap, &data}) {
        /* Can use `param.len`, etc immutably. If explicitly typed out, we might
        grant the ability to use `len` as shorthand for `self.len` */
    }

    fn example1(param: &mut Self) {/* can use `param.len`, etc mutably */}
    // the above desugars into
    fn example1(param: Self {&mut len, &mut cap, &mut data}) {}

    // suppose we now selectively do
    fn example2(param: Self {&len, &mut data}) {
        // edit: important note: `Self {&len, &mut data}` is not a new partial type in the
        // sense of a new memory layout or something,
        // it is just a more specific borrow state of `Self`.
        /* can use `param.len` immutably, `param.data` mutably,
        and `param.cap` is never touched (otherwise a compiler error will result if it detects
        that `param.cap` is used or `param` is internally passed to a function that has
        desugared `Self {&/&mut cap, ..}`) */
    }

    // Note that for public documentation purposes, the signature of this
    // function will render as just `len(&self)` because the fields involved
    // here are private, but when a maintainer looks at the functions they
    // will see a promise they have made to the borrow checker that this
    // function only views the `len` field immutably. If they change the
    // function signature (in nontrivial ways besides renaming or
    // simple equivalences), then it may break situational usages of `Vec`.
    //
    // I think this is a universal implication, that any RFC of partial
    // borrowing will need to encode stuff in the function signatures,
    // otherwise previously trivial changes to internals may break some
    // usages if the compiler is inferring borrowability by itself
    // across function boundaries.
    pub fn len(self { &len }) -> usize {
        // can only use `len`
        len
    }

    // any size changing function is going to need the full `Self` regardless
    pub fn push(&mut self, element: T)

    // Here is where it gets interesting. `get_mut` only needs a copy of `len`
    // initially when checking bounds, `cap` is unused, and `data` is the
    // actual thing across which mutable lifetimes have to be handled.
    // the borrow on `len` gets dropped after the function call like normal,
    // but the `data` part retains its borrow for as long as 'a.
    pub fn get_mut(self { &len, &'a mut data }) -> &'a mut T {}
}

fn main() {

    let mut v: Vec<()> = ...;

    let element = v.get_mut(0);
    // the borrow checker now sees the borrow state of `v` as
    // "`Self { len, cap, &'lifetime_x mut data }`" with `len` and `cap` free to be used

    // the function signature `self { &len }` is compatible with the current
    // usage and state of `v`, so this just works at the same time that `element` is
    // mutably borrowed
    v.len();

    dbg!(element);

    // you can imagine a lot more complicated uses of this idea
}

[digama0]

There are a lot of previous RFCs and proposals regarding this feature, which indicates that the design question is quite hard. I think it is a necessary part of any new proposal for it to summarize how those previous proposals work, and how the new proposal fixes shortcomings in the previous proposals. The syntax in particular is very divergent from other rust syntax, and I think it might be helpful to see if it is possible to riff off of a previous proposal's syntax to get the same expressive power as this one in a more familiar packaging.

[AaronKutch]

@VitWW your current syntax with the % and especially %permit, %miss and %deny stuff looks quite foreign. What if you instead used a combination of my idea above and/or what issue #1215 does with its where clauses, and/or what View Patterns internals#16879 can do at the pattern level?

Looking your list of alternatives, Field Projection is its own orthogonal thing that may or may not have some intersection with what we want. Fields in Traits is another orthogonal thing. Permissions, View Types, and ImplFields don't look like the right direction to me.

I believe that what we are looking for is something like "borrowability augmented types" that are mainly applied at the function signature level. It allows giving borrow related information about internal borrowing flow needs to the compiler. After some more discussion here, I think you should close "Partial Types" because it is a misnomer, we don't want to bring memory layouts into the picture, just borrow checker assistance for parts of types. Also, please make it a pre-RFC, there are a lot of things to consider before making a serious RFC that has all bases covered.

[burdges]

I disagree that fields in traits is orthogonal. Any partial borrowing solution should work for types behind traits. We could typically make trait scheme work with inherent types, but not the reverse, so fields-in-traits superseded anything like this.

As previously discussed, we should've some far cleaner and more powerful solution in which

• "arithmetic" of relative lifetimes aka higher kinded lifetimes expresses the partial borrowing bounds, aka which methods, returns, closures, etc. coexist simultaneously, but
• rustc infers which fields actually belong under which higher kinded lifetimes.

We'll de facto loose lifetime elision anyway with any partial borrowing scheme. In particular, your %permit` etc merely change what goes into each lifetime, so their complexity necessitates complex explicit lifetime bounds too. We could always express the field assignment to relative lifetimes, but rustc could typically infer the field assignments for us.

impl T for MyT {
    type 'a<'self>: 'self = { field1, mut field2 };
}

[AaronKutch]

One more thing, augmentation is impl specific like how borrowing is instance specific and not a trait wide type thing, meaning that trait impls can have customized borrowability. In cases of calls on trait objects or generic parameters, the borrow checker simply applies wholesale borrowing like it normally does. We can make this dovetail nicely into specialization and preaugmented fields-in-traits.

[VitWW]

@AaronKutch , @digama0 Thanks a lot for ideas and suggestions!

Yes, % looks a bit alien, but it is changeable.

(1) First, in most cases we don't even need these constructions!

let p_just_x = Point {x: 1.0};
    // partial initializing

let ref_p_just_x = & p_just_x;
    // partial referencing

// partial arguments
x_restore(&mut p1_full, & p1_full);

let pxy = pfull.{x, y};

let rpwas = & pfull.{was_x,was_y};

let pfull2 = Point {..pxy, ..*rpwas};

(2) Second, for "extender", all simple combinations with assignment are already in use (except @=, #=)

p_just_x.y %%= 6.0;
p_just_x.y =% 6.0;

So, (%%=) or (=%) is a quite good choice.

(3) Third, since %permit is default access, we could easily remove it for nothing, like nothing is not-mut and not-ref

(4) Ok, we could change %deny and %miss to deny and miss. We could even get rid from %deny for omitted fields

---

UPD: (2) I've change (%%=) to (let=):

p_just_x.y let= 6.0;

[digama0]

I think you should try thinking about what is the absolute minimal amount of syntax required to express the semantics you want.

• The %%= operator appears to be completely unnecessary here. With an appropriate analysis of what is initialized, you should be able to do this with just =.
• The %permit qualifier is the default so you don't need it.
• The %deny qualifier is unnecessary if you just specify which fields are accessible. Specifying the fields that aren't accessible is a compatibility hazard in addition to being very verbose for single field getters.
• The difference between %miss and %deny is not clear to me. If you don't touch the field it shouldn't matter whether it is initialized or not.
• %ignore is also apparently a synonym for %miss and %deny, and %any is also explicitly a shorthand for this.

I'm not seeing a reason to have any qualifiers at all, except for the lifetime / mutability qualifiers suggested by @AaronKutch . Which would make this proposal quite similar to view types. Could you more directly address what you can do with these partial type qualifiers that are not supported by view types?

[VitWW]

@digama0 Thanks for advise.

1. I've already change access from "before type-name" to "after type-name".
   Instead of p : %full Point now is p : Point.%full, which is more Rust-friendly.
2. Maybe is needed to change %%= into let= which has more sense and is not confused with assignment =

p_just_x.y let= 6.0;

3. About %permit - agree, It is need for explanations only, just use <nothing>!
4. About %deny - almost agree, in most cases is need for explanations only, just use <omitted field>.
   Except binding deconstructions (but we alternatively force to use '_' ) . The only way it is useful is inside tuple initializing

let t1 = (%miss 2u32, %deny 6i64, 5i64);
//  same as (but simpler and nicer)
let t1 = (2u32, 6i64, 5i64).{2, %miss 0};

5. I see , more explanatory between %miss and %deny is needed. Ok, thank you, I'll add it

let p = Point{x : 1.0, y: 2.0};
let rp = & p.{x};
    // rp : & Point.%{x},   not  rp : & Point.%{x, %miss y}

rp.y let= &8.0;
// or
*rp.y let= 8.0;

By theory of types it is not forbidden, but in reality due Rust variable representations this "extender" rewrites p.y.
That's why we need to distinguish %deny from %miss.
So, all referenced variable has no %miss fields.
If we remove %miss access, type extension is no longer possible - (not E) alternative. Sure, (E) could become "future possibilities" in that case.

6. With %ignore - it is more complicated.
   6.1) First, it need for forward-compatibility (F)
   6.2) If we cut %ignore, we cannot allow flexible arguments (F):

fn upd_with_ignore (&mut p : &mut Point.%{x, %any}, newx : f64) {
   *p.x = newx;
}

fn upd_without (&mut p : &mut Point.%{x}, newx : f64) {
   *p.x = newx;
}

upd_with_ignore(&mut p2, 6.0); // Ok
//    same as
upd_with_ignore(&mut p2.%arg, 6.0); // Ok
upd_with_ignore(&mut p2.{x,y}, 6.0); // still Ok
upd_with_ignore(&mut p2.{x,y,z}, 6.0); // still Ok
upd_with_ignore(&mut p2.%max, 6.0); // still Ok
upd_with_ignore(&mut p2.%full, 6.0); // maybe Ok

upd_without(&mut p2, 6.0); // Ok
//    same as
upd_without(&mut p2.%arg, 6.0); // Ok
upd_without(&mut p2.{x,y}, 6.0); // error
upd_without(&mut p2.{x,y,z}, 6.0); // error
upd_without(&mut p2.%max, 6.0); // error
upd_without(&mut p2.%full, 6.0); // error

6.3) if we remain %ignore filter, but cut %ignore access we cannot return same type as function get (D):

    pub fn f_return_as_param(&self : &mut Self.%{t, %any}) -> &mut Self.%{t} {
        self.t = 0.0;
        &mut self.%max
    }

    pub fn f_return_as_arg(&self : &mut Self.%a@%{t, %any}) -> &mut Self.%a {
        self.t = 0.0;
        &mut self
        // same as
        &mut self.%exact
    }

Sure, if we cut %ignore, (D) and (F) could become "future possibilities" in that case.

[VitWW]

@AaronKutch I combined your Ideas and mine Partial Types idea and we get:
Partial Mutability #3428

pub fn mx_rstate(&mut p : &mix Point.%{mut x, state, %any}) { /* ... */ }

Now it looks quite Rusty.

[AaronKutch]

I'm thinking the root problem behind the ugliness of most of these attempts at partial things is that Rust currently binds the borrowability of structures too closely with the types of structures. I realize now that you are even trying to bind the state of initialization further into that mess. These things need to intersect with each other at some point, but could be more orthogonal in their intersection lines.
First of all, I would drop the attempts at partial initialization, because MaybeUninit handles the many technicalities of uninitialized data much better than any attempt could be at encoding it more directly in source code. I understand that you may be going for some higher level of uninitialization, but it would be better to reframe the problem as a Moveability/Borrowability/Visibility problem rather than an initialization problem.
What I mean about borrowability being bound too closely to types, is that &T and &mut T types both are pointers with no differences at the levels below borrow checking. The difference between them is how borrowck allows situational uses of them. If we had a kind of generic that only worked over the type of reference, we could start by deduplicating a lot of traits like Deref/DerefMut, and other Trait/TraitMut pairs (and if we wanted only one variation to be reacheable, we would just impl the trait over one specific reference type instead of using whatever the syntax for a generic reference would be).

This ability by itself wouldn't be that useful, what would be more useful is further recognizing that &kind Type {field_a, field_b} is really equivalent to having the power of (&kind field_a, &kind field_b) if we manually rewrote all usages of Type to tuples of its fields. In your signature above, there is a lot of redundancy in &mut p : &mix Point.%{mut x, state, %any}. Why should there be all of &mut applied to p, &mix applied to Point, and stuff applied within Point? It doesn't make any sense to me to have a borrowability applied to fields and higher level things at the same time (except note that when we write mut param: &mut ... in today's and future's rust, the first mut just turns off the in-function mutability guard that we can freely choose). Point { &mut x, &y}, Point { &x, &mut y }, Point { &mut x, &mut y }, and Point { &x, &y } are all the possible borrowabilities, and doing something like &kind Point { &x, &mut y } is either redundant or self contradictory.

[AaronKutch]

My ultimate idea is that we grant the same power as we could with rewriting everything into tuples, but without violating the privateness of fields and without multiplying the number of references in registers (Point { &x, &mut y} does not actually have two separate pointers to x and y, it just points to the outer struct and uses offsets like usual, only borrow checking compile time has some extra overhead).

One interesting consequence is that Struct { fields, &'a () } would have to act like Pin<Struct> for the lifetime of 'a.

[AaronKutch]

How about the most expanded form is &'a aug ident where a is the lifetime, aug is the borrow augmentation and ident is the type or binding that is being taken reference from. Then aug can be put in generic arguments like lifetimes are.

// edit: this is not some runtime dynamic thing, it gets monomorphized at compile time
fn general_ref<T, aug>(this: &aug Struct<T>) -> &aug T {
    &aug this.inner_t_field
}

If the fields of T are accessible by an implementor, they can use specialized aug that can deal with more complicated situations than wholesale & and &mut.
Call it like <... as Struct{&x, &mut y}>::general_ref(this), general_ref::<Point {&x, &mut y}>(this), or maybe general_ref(this {&x, &mut y}), I think there is enough room in parsing for this.
Are there any more useful augmentations besides unique exclusive references or immutable shared references?

[VitWW]

First of all, I would drop the attempts at partial initialization, because MaybeUninit handles the many technicalities of uninitialized data much better than any attempt could be at encoding it more directly in source code.

Partial initialization is just side effect of partial types. But yes, we could simplify a bit by getting rid of %miss access and (D)-part of proposal. And turn (D) into "future possibilities".

In your signature above, there is a lot of redundancy in &mut p : &mix Point.%{mut x, state, %any}. Why should there be all of &mut applied to p, &mix applied to Point, and stuff applied within Point?

This because of desugaring implicit rules

fn (&mut p : &mix Point.%{mut x, state, %any}) { /* ... */ }
// desugars into
fn (&mut.%lift p : &mut.%type Point.%{mut x, state, %ignore _}) { /* ... */ }

mut.%lift is "look how mutable at sharing part at type declaration"
and .mut.%type (and mix = mut.%type always) said that partial mutability is mixed with partial type.
And in this case in partiality of type we fix if field is mutable or not (otherwise)

How about the most expanded form is &'a aug ident where a is the lifetime, aug is the borrow augmentation and ident is the type or binding that is being taken reference from. Then aug can be put in generic arguments like lifetimes are.

fn general_ref<T, aug>(this: &aug Struct<T>) -> &aug T {
    &aug this.inner_t_field
}

Is is a way more abstract, then current Rust allows to write. And at first self-reflectiveness is needed for that.

[AaronKutch]

Is is a way more abstract, then current Rust allows to write. And at first self-reflectiveness is needed for that.

1. This is a generic that gets monomorphized at compile time, the compiler determines what aug is per call of general_ref. 2. Of course Rust doesn't allow currently writing this, both our changes require some deep compiler changes. 3. Why in the world do you think self-reflectiveness is needed, this isn't even something that would have entered my mind, all I am doing is a generalization of generics to use 3 kinds of things (lifetime, augmentation, type) instead of just 2, and we use much of the same inference machinery that is already present to infer the 3rd thing just like the first 2.

[VitWW]

3. Why in the world do you think self-reflectiveness is needed, this isn't even something that would have entered my mind, all I am doing is a generalization of generics to use 3 kinds of things (lifetime, augmentation, type) instead of just 2, and we use much of the same inference machinery that is already present to infer the 3rd thing just like the first 2.

Oops, sorry, I just thought about "any and every" Structs, but not specific ones.

In this case - yes, it is an alternative, but I'm afraid that Rust community don't want so drastic changes for just get rid of duplicate traits and implementations. ((

[VitWW]

I discuss more changes here: "[Pre? RFC] Partial types and partial mutability" internals.
Everyone is invited!
P.S. My proposal is focused on Partial Product Types(Structs, Tuples). But Partial types could also be applied to Sum Types (Enums)! (ok, it is a bit in a different way, but partial anyway)

enum MyEnum {
  A(u32),
  B { x: u32 },
}

fn print_A(a: MyEnum.%{A}) {
    println!("a is {}", a.0);
}

fn print_B(b: MyEnum.%{B}) {
    println!("b is {}", b.x);
}

fn print_no_pattern(e: MyEnum) {
  match e {
    MyEnum::A(_)  => print_A(e); // e.%{A}
    MyEnum::B(..) => print_B(e); // e.%{B}
  }
}

Main difference between partial Product types and partial Sum types, that Partial Sum types do require neither pretending borrowing, nor partial mutability fox maximum flexible partial Enums!

Based on alternative: Enum variant types #2593, Enum Variant Types iss_lng#122

[Aloso]

This RFC is a mess. It solves a real problem, but

• The proposed solution is more complicated than anything I could think of
• It tries to solve at least 5 unrelated problems at the same time
• It is poorly explained
• There's no proper motivation, and almost no discussion why the proposed features are needed.

I think this RFC would have a chance if 80% to 90% of the proposed features were removed and the syntax was simplified.

[Aloso]

Not clear why this distinction is needed — when a field is forbidden to use, it doesn't matter whether it is initialized or not.

[VitWW]

It is not necessary %deny and %miss fields-accesses are needed, maybe it is enough for variable be "fresh".

let p = Point{x : 1.0, y: 2.0};
let rp = & p.{x};
    // rp : & Point::{x};

*rp.y = 8.0;
// or
rp.y = &8.0;
// rp : & Point;

This is Ok from Type point of view, but due to internal Rust representation p.y in reality will be overwritten by this expression, so it must be forbidden.
Fresh initialized variables could be extended, but references, "de-reverences", moved partial variables cannot be extended.

[Aloso]

I understand, so this is to support initializing fields in a function, like this:

fn init(s: &MyStruct.{x, y, %miss}) -> &MyStruct {
    *s.x = 42;
    *s.y = 1337;
    s
}

A better name here would be uninit instead of miss. Also, I think it makes more sense to put it before the field:

s: &MyStruct.{uninit x, uninit y}

[Aloso]

I think the syntax should reflect this! Rust currently fails to compile when I forget to initialize a field, and opting out of that should be explicit. Maybe

let x = SR { val: 5i32, .. };

This syntax has been proposed for default-initializing fields, but I think leaving them uninitialized (requiring ..default() for initializing) would be ok if the compiler checks that all fields are initialized when they are used.

Also, please format code the way rustfmt would.

[VitWW]

Good point!
I think, default-initializing fields must be explicit (or similar):

let f = Foo { a: "Hello".to_owned(), .. };  // alternative syntax to make default use explicit

And difference of partly initialized is next:

let x = SR {val : 5i32 }; // complies Ok!
// x : SR::{val};

let x : SR = SR {val : 5i32 }; // complie ERROR!

let x  : SR::{val} = SR {val : 5i32 }; // complies Ok!

[Aloso]

I would much prefer having explicit syntax for partially initializing as well. I want SR { val: 42 } to return a compiler error when SR has more than 1 field.

[digama0]

I don't think this is a required feature. You can do without it by using let x: SR; x.val = 5i32, and you can make a macro wrapping that if you want a more struct-literal-based notation. So given that this RFC is bursting at the seams with new notation, I would recommend cutting it.

[VitWW]

Ok, let's cut!

[Aloso]

Why add two syntaxes that do the same thing? Are you trying to make this feature as difficult to learn as possible?

[VitWW]

Thank you, I'll cut this. First remains:

let pxy = pfull.{x, y};

[digama0]

Do we need this at all? Can't we just solve this with intra-procedural analysis, just as is done for lifetimes and partial borrowing today? You don't need to write let r = &'a x;, the compiler just figures out what 'a should be based on how it is used. Only types really require this annotation (like lifetimes but not partial borrowing today).

[VitWW]

@digama0 good question!
If we use such variables as argument to to the function or method, then yes, we can solve, which partiality is required.
But in rest of cases it is hard to say.
Hopefully, main specialization of partial types - is to be used as arguments!
So, if we add "implicit partiality" for arguments - we cover most useful cases

[digama0]

like I said, the compiler already does implicit partiality. The only issue that needs solving here is that there is no syntax for explicit partiality. The compiler can understand when you late initialize a field or borrow part of a struct - this is already expressible.

[VitWW]

@digama0 Do you suggest "use implicit rules for this if this do not use explicit rules" for partiality. Good point!

[VitWW]

@Aloso Thank you for feedback!
Wow! It was unexpected!
I'm trying to finish third version of this proposal, which is "almost" complete and much simpler:
Partial Types (v3) (it is not yet in "pull request")

[oli-obk]

The current lang team process is to pitch your idea in the lang team zulip and find a lang team member to shepherd this work. I recommend closing this RFC and first getting lang team buy in

[VitWW]

@oli-obk Ok, let's try your way!

💖 Thanks!

Thanks to @AaronKutch , @digama0 , @Aloso for for your reviews of the this proposal.