Add value types & value semantics for deep immutability (w/ compile-time enforcement)

[rchoi1712]

There are multiple issues that would benefit from having value semantics in Dart:

• Add data classes #314
• It is necessary to implement threads #333
• The final keyword is too long #136

Value semantics introduces deeply immutable objects whose transitive object graph consists only of objects which are themselves immutable, and which feature compiler-enforced read-only access to their underlying allocated memory. This allows value objects to be sent by reference (i.e., shared) over a SendPort instead of being copied, by providing a hard guarantee that the object cannot mutate in the normal course of events (barring situations where security is compromised).

Value objects come with a default amount of configurable support for comparison , hashing, and copy/modifies. Serialization and builder mechanics can also be provided, via a set of first-party and community-provided packages that leverage new language features to reduce or eliminate user-visible code-gen.

Immutable identifier bindings (a.k.a final variable assignments) are covered also, via variable declarations with the val keyword.

Note to the language team: Since the underlying user issues are well-understood, and the solution proposed here is low-risk (no breaking changes outside of a syntactic val for final swap), I decided to skip the 'request' phase and move ahead with a feature proposal. Hope that's ok!

Proposal

The proposal introduces use of the val keyword, and consists of three primary components - left- & right-val declarations, value classes & val objects, and the val() operator.

Left- & right-val variable declarations.

Left-val (or simply val) variable declarations are equivalent to current final declarations:

val x = ["hello", "world"];  // Immutable binding to a List of String's.
val int y;    // Late-assigned immutable binding.

x = ["good-night", "world"];  // Error: reassignment
x.add("!");  // OK

y = 1;
y++;  // Error: reassignment;

Right-val (or : val) declarations indicate the identifier is to be bound to a deeply-immutable object - a val object (more on that in the next section). Right-val declarations are by default left-val, unless otherwise specified, with the mut keyword.

List<int> x: val;   // Immutable binding to an immutable List of String's.
mut Map<String, String> y: val;   // Mutable binding to an immutable Map.

Put simply, a val declaration is one where the variable binding is immutable[1], whereas in : val declarations the target of the binding is either deeply immutable, or is itself a : val identifier.

value classes and val objects

A value class is a class declared with the value modifier, causing the compiler to enforce deep immutability at compile-time, and whose transitive object graph consists only of deeply immutable val objects. Fields of value classes are implicitly and invariably left- & right-val. Additionally, fields must be typed (dynamic is disallowed. EDIT: revisit) and initialized at time of construction (no late initialization). Instances of a value class are by definition val objects.

value classes can be extended, with its children required to also be value classes, and can have all class modifiers applied to them except interface. value classes can be declared with the mixin modifier, and mixed-in to other value classes.

TODO: Since value classes are immutable, consider generating default constructors, with positional arguments and named arguments that map to class fields.

value class SendMe {   
  SendMe(this.a, this.b, this.c);

  String a;
  int b;
  List<String> c;  // Right-val.  Needs to be bound to an object returned by the 'val' operator

  late int x; // Error: late declaration
}

A val object is either an instance of a value class, an immutable language primitive (like int, bool, or String), or an object returned by the val() operator. Only val objects are assignable to variables declared as right-val.

val() operator

Converts instances of sdk-defined collections to a deeply immutable object with the same type, via deep move semantics. The returned val object becomes the new owner of the underlying data wrapped by the collection (defined transitively), and the moved-from object is left in a destructible state, but no longer has access to the moved data - i.e., an empty state. This is similar to how move semantics work in C++ and Rust, with deep moves differing in that they are recursive in nature[2].

An example that constructs an instance of the SendMe class.

val obj = SendMe(
    "Hello, world!", 
    2023,
    val(["From:", "Paris"]),  // Returns an immutable List<String>
);

Example

Below is an example implementation of a sendMessage method that sends decoded JSON data from one isolate to another via a SendPort. The method takes two mutable maps - headers and content -, uses them to construct a Message object that is an instance of a value class, and sends it over the SendPort via memory sharing.

sendMessage(
    Map<String, String> headers, Map<String, JsonValue> content,
    SendPort port
) {
  val msg = Message(val(headers), val(content));   // Leaves 'headers' and 'content' in a moved-from state.
  port.send(msg);  // val objects can be shared instead of copied.
}

value class Message {
  Message(this.headers, this.data);
  
  Map<String, String> headers;  // Right-val
  Map<String, JsonValue> data;  // Right-val 
}

// Simplified JSON representation that only supports string and object values.
value class JsonValue {
  JsonMapEntry({this.stringValue, this.objValue});

  String? stringValue;
  Map<String, JsonValue>? objValue;
}

Further Considerations

• Corresponding changes to kernel spec.
• Non-Dart runtimes (i.e., JS, wasm).

EDITS:

8/22 - Fix wording around memory ownership and use transitive object graph to highlight the scope of deep immutability.
8/29 - Some more cleanup around wording and formatting.
9/6 - Remove section on value types, until it's re-written to include user-provided builders and support for serialization.
9/20 - Default constructors for value classes. Revisit dynamic fields in value classes. Update json model in example.

---

[1] A Dart variable can be thought of as an object with a char* field for its name, and a uint64 field containing the memory address of the assigned target. For a variable declared as val, this conceptual object can be thought of as being immutable.

[2] In deeply nested contexts, the depth of recursion need for a move is reduced by use of value classes, since they require right-val assignment for their collection fields via the val operator, which performs the move at the time of assignment.

[rchoi1712]

• Lightweight communication of typed data between long-lived isolates sdk#52577
• API to create deeply immutable typed data, from one or multiple chunks. sdk#50068
• Efficient communication of data between isolates #124

⬆️ Issues that this proposal would also resolve.

See also:

• Use isolates to download network image flutter/flutter#111927