Defaultable Value Types

[RikkiGibson]

The following is an incomplete draft proposal. I have been thinking about this problem for a while now, but I haven't been able to come up with a proposal that addresses all the complications of the feature. The sections "Properties" and "Transitioning from previous C#" in particular point out some issues that need to be addressed before the proposal can go to LDM. Any feedback or suggestions you have would be appreciated.

Defaultable Value Types

Structs containing non-nullable reference types are a known hole in nullable analysis.

#nullable enable

var s = default(S);
M(s); // no warning, NRE at runtime

void M(S s)
{
    s.Field.ToString();
}

public struct S
{
    public object Field; // no warning
}

Previous discussion

• Championed: Non-defaultable value types #146
• Should non defaultable value types be per instance rather than per type? #3322
• Non-defaultable struct types and parameterless struct constructors (LDM notes)

Context

It's never been possible to truly encapsulate a struct's fields. Some behaviors affected by the set of fields in a struct include:

1. Definite assignment. A variable of a struct type becomes definitely assigned when all of its fields are definitely assigned.
2. Unsafe. The compiler digs through a struct's fields recursively to decide whether it's legal to take a pointer to it, for example.

If we follow the patterns set by these analyses, we could arrive at a solution for how to nullable analyze structs.

Defaultable annotation

When a struct type is used, the user indicates whether the struct is nullable-initialized using the "defaultable" annotation.

We need to choose a token to serve as this annotation. We can't use ? because that already means System.Nullable, and we don't think we can use ?? because it introduces too much parsing ambiguity with the ?? binary operator. Therefore, for this proposal we use the annotation ~.

If no annotation is used on the struct type, it is assumed to have all its non-nullable reference type fields initialized at all levels of nesting.

void M1(S s)
{
    s.Field.ToString(); // ok
}
M1(default); // warning

If a "defaultable" annotation is used, all non-nullable reference type fields in the struct might contain null.

void M2(S~ s)
{
    s.Field.ToString(); // warning
}
M2(default); // ok

If the flow analysis finds that all non-nullable reference type fields contain non-null values, we permit conversion from an "annotated" to "non-annotated" struct type. This is probably accomplished by tracking a flow state for the struct variable itself.

void M3(S s) { }

void M4(S~ s)
{
    if (s.Field != null)
    {
        M3(s); // ok
    }
}

~ will flow through type argument inference as expected.

T Identity<T>(T input) => input;

var s = Identity(default(S));
s.Field.ToString(); // warning

var s2 = Identity(new S { Field = "a" });
s2.Field.ToString(); // ok

~ is an all-or-nothing thing, so we won't be producing a distinct TypeWithState for every combination of field flow states for a struct, for example. This is because we would prefer to sidestep the possibility of indicating which individual fields on a member's inputs and outputs may be null or not--it invites some very complex type system problems.

public struct Pair
{
    public string Item1;
    public string Item2;
}

T Identity<T>(T input) => input;

var s = Identity(default(Pair));
s.Item1.ToString(); // warning

var s2 = Identity(new Pair { Item1 = "a" });
s2.Item1.ToString(); // warning

var s2 = Identity(new Pair { Item1 = "a", Item2 = "b" });
s2.Item1.ToString(); // ok

? on unconstrained generics will "turn into" ~ when the type parameter is substituted with a value type.

T? GetItemOrDefault<T>() => // ...

var s3 = GetItemOrDefault<S>();
s3.Field.ToString(); // warning

T GetItemOrFallback<T>(T? maybeItem, T fallback)
    => maybeItem ?? fallback;

var s3 = GetItemOrFallback(default, new S { Field = "a" });
s3.Field.ToString(); // ok

Flow analysis attributes

It will be necessary to have an AllowDefault parallel to AllowNull, and similarly with other common flow analysis attributes. Alternatively, we could adjust the meaning of existing attributes. For example, AllowNull could mean "allow default" when used with value types, it would be associated with the 'this' parameter when applied to methods, properties, etc.

readonly struct MyImmutableArray<T>
{
    private T[] _array;

    [AllowDefault, MaybeDefaultWhen(true), NotDefaultWhen(false)]
    public bool IsDefault => _array is null;

    // alternatively:
    [AllowNull, MaybeNullWhen(true), NotNullWhen(false)]
    public bool IsDefault => _array is null;
}

Opt-in

If a struct has no non-nullable reference type fields, then it is generally convertible from its "defaultable" to "non-defaultable" form. However, there are other structs out there which might benefit from an annotation which allows the struct to pretend that it does contain non-nullable reference type fields, and therefore can participate in nullable flow analysis.

In this case, the state of the struct wouldn't be determined by the state of its fields, but rather by annotations on members it is used with, etc.

[MaybeDefault] // some annotation that indicates that "this thing may be defaultable even if all its non-nullable reference fields are non-null".
public struct BlobHandle
{
    // ...
    [AllowDefault, MaybeDefaultWhen(true), NotDefaultWhen(false)]
    public bool IsNil => // ...

    public byte Read() => // ...
}

void M1(BlobHandle~ handle)
{
    if (!handle.IsNil)
    {
        handle.Read(); // ok
    }
}
M1(default); // ok

void M2(BlobHandle handle)
{
    handle.Read();
}
M2(default); // warning

operator default

There are multiple reasons we might want an operator default in the language.

public struct ImmutableArray<T>
{
    public static bool operator default(ImmutableArray<T> arr) => arr._array is null;
}

1. We can make it a "language-sanctioned" way of checking whether a struct is "valid" for usage according to the user's definition. Flow analysis would understand it without any additional annotation.
2. We can improve the behavior of scenarios like patterns on ImmutableArray. For example, there is some dissatisfaction with the fact that ImmutableArray<int> arr = default; arr is [1] throws an exception.
   • It's not quite as simple as just checking for the operator on the input type, because we would ideally like for this to also work when the pattern input is a type parameter and the value at runtime is a default ImmutableArray. It's not clear exactly how the codegen and runtime behavior would work in that scenario, but it's also not part of this proposal to solve it.

It might be reasonable to enforce that operator default only returns false in a code path where all non-nullable reference fields contain non-null values. Given this enforcement (via warning), we could assume the automatic notion of "non-default" is compatible the user-defined notion. However, if the user applies the "opt-in" attribute to the struct type, we might want to avoid any enforcement on the implementation, and try our best to just get out of the user's way.

Open question: should the user also be required to define operators ==/!= if they define operator default?

"defaultable" reference types?

This road leads to giving users the ability to define "null-like" values for non-value types. The Unity GameObject scenario is the biggest use case we are aware of currently.

This proposal does not include changing the analysis of reference types in this way, but it's good for us to have confidence that "defaultable value types" doesn't put us into a corner when it comes to reference types.

In a nutshell: It feels risky to introduce distinct "maybe-null" and "maybe-default" flow states for reference types. Rather, to solve the Unity scenario it might make more sense to introduce operator null for reference types, and allow the user to define how the language decides if something is null for the purposes of is null, ??, ?., etc. Effectively, operator default would be exclusive to value types and operator null would be exclusive to reference types.

Properties

OK, now for the hard part. Properties are supposed to be opaque, but they are known to frequently be simple encapsulations of fields. Definite assignment treats properties as truly opaque, and that provides an acceptable experience, particularly since assigning default/new() will always definitely assign all struct fields. However, it feels like a usable "defaultable struct types" feature will also need to be able to do the right thing in a scenario like the following:

public struct Person
{
    public string FirstName { get; set; }
    public string? LastName { get; set; }
}

void M(Person person) { }

M(default); // warning

var p1 = default(Person);
if (p1.FirstName is not null)
{
    M(p1); // ok
}

Ideally, we would like for it to also understand the equivalent manually implemented properties:

public struct Person
{
    private string _firstName;
    private string? _lastName;

    public string FirstName { get => _firstName; set => _firstName = value; }
    public string? LastName { get => _lastName; set => _lastName = value; }
}

The only practical way to do this is to use an annotation on the property. What would that look like? We probably wouldn't want something like the following, since it feels like the signature is oriented around the "uncommon" scenario rather than the "main" scenario, and it probably doesn't do the right thing in all scenarios:

    [NotNullIfNotNull(nameof(_firstName))]
    public string? FirstName { get => _firstName; set => _firstName = value; }

We would also want to appropriately handle the following additional scenarios (among others!):

The property throws if a non-nullable field is not initialized.

private SomeType _inner;
public string Item { get => _inner.Item; }

The property encapsulates a nested field, and a different property directly exposes the containing field.

private SomeType _inner;

public SomeType Inner { get => _inner; set => _inner = value; }
public string Item { get => _inner.Item; set => _inner.Item = value; }

The problem essentially is:

1. How do we account for the myriad set of behaviors properties can have on "non-nullable-initialized" or "partially initialized" structs?
2. How do we draw a line at a reasonable set of behaviors to support with annotations?
3. How do we degrade gracefully and allow people to still work with properties whose behaviors we've decided not to make "describable" using annotations?

Interaction with "required properties"

It's not completely clear how this feature interacts with the "required properties" feature. For example, if a struct contains required fields or properties, should it be allowed to simply take the default value of the struct and pass it around? Is a struct considered "initialized" once all its non-nullable fields and required members have been assigned? Is there additional expressiveness/enforcement needed to say "no, you need to run the constructor and assign the required properties to use this struct"?

Transitioning from previous C#

Another major problem with this proposal is that it's getting more difficult over time for us to allow new nullable diagnostics. We need to explore whether there is a straightforward way to make it so the new analysis level is "opt-in", and that people who update their SDK without updating to the latest language version won't be disrupted by a bunch of new warnings.

We may have to consider things like making flow analysis behavior vary depending on language version, or placing specific new diagnostics into a new warning wave.

[ufcpp]

Could the Opt-in annotation also work for enum?

[MaybeDefault]
enum E
{
    A = 1, B, C
}

class Program
{
    static void M1(E e) { }
    static void M2(E~ e) { }

    static void Main()
    {
        // warning
        M1(0);
        M1(default);

        // no warning
        M1(E.A);
        M2(0);
        M2(default);
    }
}

This example can be replaced by nullable E? and the difference is just one byte: Nullable<T>.HasValue. However, sometimes the one byte is critical to performance.

[RikkiGibson]

It might make sense to do this, but I'd like to know more about where this would be useful.

[ufcpp]

The only practical way to do this is to use an annotation on the property.

Is it possible make auto- and semi-auto-properties implicitly annotated? If possible, wouldn't significant percentage of defaultable analysis be solved?

[RikkiGibson]

I think it's possible, but it feels like there's risk in potentially attaching too many attributes implicitly.

Also, it's not yet clear to me what combination of default behaviors and special attributes would make it so we "just do the right thing" for auto-properties, while minimizing "surprises" for users in common manual property scenarios, and generally avoiding overcomplicating the compiler's analysis of initialization.

In particular, I am thinking of the "init list" which was proposed and rejected/deferred recently in the "required properties" feature. It feels like it would be best to avoid designing an equivalent feature/capability within this feature.

[RikkiGibson]

The nullability improvements working group discussed this feature yesterday.

[TahirAhmadov]

I totally missed it. It's unfortunate many more didn't voice their support for this feature. It looks like you put in a lot of effort, and this would be a great step towards ironing out the NRT feature. Furthermore, I think we should discuss what to do with arrays, too.