Feature: Statically checked declaration-site variance #524
Comments
eernstg
added
the
feature
|
First draft of feature specification at PR dart-lang/sdk#557. |
|
The "exact" type already exists in Dart in certain places - the type of a literal or object creation expression may have an exact type, which is why For use-site invariance, we will have both As stated, the type arguments can only be used in type pattern expressions (type annotations), not class expressions or literals. The following should be allowed: List<exact String> foo(List<exact int> list) => (list..add(42)).map((x) => "$x").toList();It is meaningful and useful. What about: class C<T> {
List<exact T> mkList() => <T>[];
}Is this meaningful? Maybe. If you have a List<T> mkList() => <T>[];will never have a static return type of So, we need some substitution rules for type arguments to make this work. It also suggests that a lot of platform libraries should be changed to have That suggests a migration issue. If we change some platform libraries to return, say, If we need migration anyway, then we don't need to make the syntax backwards compatible, and we could do something else, like: class C<+T , -S, =U> { // out T, in S, inout U
List<+T> mkList() => <T>[]; // not exact T
List<U> mkList() => <U>[]; // exact U
}We do get the inherent complexity of a two-tier type system where it matters whether you write I expect For declarations site variance, we already have something similar for function types. For those, the variance is automatically computable from the co-/contra-variance of the occurrences of the type arguments. |
Right. The notion of an exact type is not specified, but it is used in some cases by our tools (even to raise an error for certain "impossible" casts). The notion of a type argument which is known exactly is a different concept (an object can have dynamic type
Both of those are instantiations of the generic type class A<X, Y> {}
class B<X> implements A<X, int> {}With that, We may or may not want to let
I'd prefer to say that a type accepts type arguments, each of which may have the modifier
class C<T> {
List<exactly T> mkList() => <T>[];
}That can be allowed, and would be safe, but the computation of the member access type must take into account that the statically known value of main() {
C<num> c = C<int>();
List<num> xs = c.mkList(); // Static type of `c.mkList()` is `List<num>`.
List<exactly num> ys = c.mkList(); // Error (downcast).
}Of course, with
I believe the most straightforward way to get this right is to consider the types of member accesses, with a case for each variance of the type parameters of the class. Type parameters with no variance modifier are the most complex ones, of course, because they are allowed to occur in any position in a member signature.
Indeed.
Right, but
I think that wouldn't ideally be such a common situation: The declaration site variance modifiers are required to match the usage (so if you have a type parameter The use of I would expect the reasonable trade-off to be somewhere between just using the legacy types as they are (with the current level of type safety, which has been used in practice for years without complete panic) and teasing out the very last drop of static type safety, by adding |
eernstg
changed the title
|
I more like the +T and -T syntax in Scala |
|
The current |
The primary beneficiary is the USER. The void addToList<T>(List<T> target, List<T> src) {
for(var x in src) {
target.add(x);
}
}
void test(List<num> nums) {
addToList(nums, [3, 4, 5]);
}
void main() {
List<double> doubles = [3.5];
test(doubles);
} |
Reading between the lines of this question, I think you are missing the primary effect of this feature, which is that it changes the subtyping relation between types. Describing |
I think tatumizer is suggesting |
Ah, sorry, I misunderstood your question. Here's the intution:
Does that help?
Neither. Classes are allowed to use type variables everywhere in a method, which corresponds most closely to
For completeness, here is the example rewritten to use an example of an invariant implementation of import 'dart:collection';
class Array<inout T> extends ListBase {
List<T> _store;
int get length => _store.length;
void set length(int i) => _store.length = i;
T operator[](int index) => _store[index];
void operator[]=(int index, T value) => _store[index] = value;
Array.fromList(List<T> this._store);
}
void addToList<T>(Array<T> target, Array<T> src) {
for(var x in src) {
target.add(x);
}
}
void test(Array<num> nums) {
Array<num> ints = Array.fromList([3, 4, 5]);
addToList(nums, ints);
}
void main() {
Array<double> doubles = Array.fromList([3.5]);
test(doubles);
} |
|
@tatumizer I think this is getting a bit far afield - variance is definitely a confusing subject, but I don't think we're going to fix that here. Is it fair to summarize your general take here as being that you find |
|
repeat the |
|
@leafpetersen I think the |
|
Any updates on the status of this feature for release? |
|
I don't think the team has any ETA, but the feature is behind an experimental flag ( It doesn't seems to be something the team is prioritizing, tho. |
I think the link should be #557 (dart-lang/language). Same for the other links in the issue description. |
Unfortunately the experiment is also unusable... The following basic snippet fails: // dummy class with contravariant type parameter
class Consumer<in T> {
void consume(T item) {}
}
final List<Consumer<String>> consumers = [];
Consumer<String> consumer = Consumer<Object>(); // OK, both static analysis and runtime
consumer.consume("hello"); // OK, both static analysis and runtime
print(consumer is Consumer<String>); // static analysis says this is always true, runtime prints false
consumers.add(consumer); // runtime error: "Consumer<Object> is not a Consumer<String>" (but it is) |
Consumer<String> consumer = Consumer<Object>(); // OK, both static analysis and runtimethis feels more like a bug than a new feature |
The bug is the current behaviour: // no variance annotation, defaults to covariant
class Consumer<T> {
void consume(T item) {}
}
Consumer<Object> consumer = Consumer<String>(); // OK, both static analysis and runtime
consumer.consume(123); // static analysis ok, fails at runtime: we passed an int to a method which expects a StringIt might feel strange because our intuition tells us that generics are covariant, and in some cases this is correct (if the type is an "output" type, as a rule of thumb). But some generics really should be contravariant (if the type is used for input only) or invariant (if the type is used for both input and output). For instance, a consumer that accepts strings is not a consumer of objects, because it doesn't accept objects that are not strings. Conversely, a consumer that accepts any kind of object is certainly a consumer of strings, because you can pass it a string and it will deal with it no problem. Unfortunately, Dart not only lacks variance annotation but defaults to covariant parameters, which leads to type unsoundess like the above example. A better default in terms of type safety would have been invariant. |
|
Why these have be keywords? Why not go with typescript way? It allows much more ways to expand. Like for example doing things like ReturnType. Maybe doing something like All objects except num and string. |
|
Late to the party, I must have overlooked these comments. Here we go: @plammens wrote:
True, the experiment is front-end only. The backends do not implement the subtyping relation for variance modifiers, which means that the backends will always assume that every class/mixin/enum type parameter is dynamically checked covariant (which is true as long as we do not have this feature). So the fact that this particular type parameter has the modifier
The static checking accepts the initialization of The situation at run time is now type correct, including the fact that a reference of type
The type test Finally, So, indeed, the experiment can only be used to try out the static analysis, you won't (yet) get the correct semantics at run time. However, if you're willing to use invariance rather than contravariance then you can emulate the variance modifier /*
Wanted:
class Consumer<inout T> {
void consume(T item) {}
}
*/
typedef Inv<X> = X Function(X);
typedef Consumer<T> = _Consumer<T, Inv<T>>;
class _Consumer<T, Invariance extends Inv<T>> {
void consume(T item) {}
}
void main() {
final List<Consumer<String>> consumers = [];
// Consumer<String> consumer = Consumer<Object>(); // Error, not a subtype.
Consumer<String> consumer = Consumer<String>(); // OK.
consumer.consume("hello"); // OK.
print(consumer is Consumer<String>); // Flagged as trivial. No harm, though.
consumers.add(consumer); // OK.
try {
consumers.add(Consumer<Object>() as dynamic); // Throws.
} catch (_) {
print('Did throw on `Consumer<Object>`.');
}
try {
consumers.add(Consumer<Never>() as dynamic); // Throws.
} catch (_) {
print('Did throw on `Consumer<Never>`.');
}
}The fact that the attempt to add a |
|
@plammens wrote:
True, invariance is the traditional approach, and it provides the guarantee that the run-time value of a type parameter is the same as the statically known value, which means that there will not be any failing run-time type checks. However, you probably can't find a recent language with subtyping whose generics do not include some kind of enhancement in this area. Plain invariant generics is awfully rigid. For example, Java introduced wildcarded types in Java 5 in 2004, and Scala had declaration-site variance at the same time. C# introduced Dart chose a different approach, based on the experiences with those explicitly specified variance related features in other languages. In particular, there was a lot of feedback in this area, claiming that variance related typing was too complex. So Dart adopted an approach which is used with arrays in several other languages: All type parameters are considered covariant, and every situation where the typing isn't guaranteed statically are checked at run time. (In Java and C# this is the ArrayStoreException/ArrayTypeMismatchException, and in Dart it's a I wasn't part of the Dart team at the time, and I was somewhat worried about this approach when I joined in 2014 because it isn't fully statically checked. However, I must admit that we have had amazingly few reports about difficulties created by those dynamic type checks. I've gathered 20 examples, not thousands (shown in the initial posting). Also, of course, soundness of the heap is strictly maintained because those run-time type checks are generated in every situation where the typing isn't guaranteed statically. In summary, it would indeed have been extremely easy to just specify that Dart subtyping with generics is invariant. However, dynamically checked covariance was chosen because it is simple, and it actually works almost all the time in practice (in particular, all scenarios that are purely read-only with respect to the given type parameter are safe). That said, I created this issue because I really want to give developers the option to specify a strict and fully statically checked approach to variance. This is especially relevant in the case where a functional-inspired style is used, and first class functions are used heavily. In any case, the way ahead is to get this feature fully implemented. If we get more votes for it, it gets more likely. |
This CL implements a new lint, `unsafe_variance`. This lint emits a warning whenever an instance member declaration has a signature where a type variable declared by the enclosing class/mixin/enum occurs in a non-covariant position in the return type (including the type of an instance variable). Issues: https://github.com/dart-lang/linter/issues/4111, with goals related to dart-lang/language#296 and dart-lang/language#524. Change-Id: I1352d71d61fece03a432ccf0d98825a69e3a457f Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/384700 Reviewed-by: Brian Wilkerson <brianwilkerson@google.com> Commit-Queue: Erik Ernst <eernst@google.com>
|
The experimental flag |
and others
This is the tracking issue for introducing a statically checked mechanism for declaration-site variance in Dart.
Background: The original request motivating this kind of feature is dart-lang/sdk#213; the initial proposal for declaration-site invariance is dart-lang/sdk#213. The initial proposal for the related feature known as use-site invariance is dart-lang/sdk#229, and the corresponding tracking issue is dart-lang/sdk#753.
Note that this issue does come up in practice. Here are a few examples gathered since April 2023:
not a subtype ofissue sdk#52826Object-inference induced type errors. #3156(int) => voidis not a subtype of type(dynamic) => voidsdk#53523The text below describes properties of this feature which are good candidates for being adopted. Many things can still change, and a full feature specification will be written and used to manage the discussions about the final design.
Variance in Dart Today
As of Dart 2.4 or earlier, every type variable declared for a generic class is considered covariant. The core meaning of this is that a parameterized type
C<T2>is a subtype ofC<T1>wheneverT2is a subtype ofT1. Other subtype rules can then be used to show subtype relationships likeList<int> <: Iterable<dynamic>andMap<String, String> <: Map<Object, Object> <: dynamic.This type rule is not sound; that is, in order to maintain heap soundness it is necessary to check certain types dynamically. This means that a program with no compile-time errors can fail with a type error at run time.
For instance, with the declaration
List<num> xsand some expressionewith static typenum, it is necessary to check during evaluation ofxs.add(e)that the value ofeactually has the type which is required byxs: It is possible that it is aList<int>or even aList<Never>, and it would then be a dynamic type error if the value ofeis adouble, even though the expression had no type errors at compile-time.Dynamically checked covariance enables many software designs that would be rejected by a traditional statically checked approach to variance (e.g., as in Java or C#). This allows developers to make a trade-off between more flexible types (e.g., a variable of type
List<num>is allowed to refer to aList<int>) in return for accepting the potential dynamic type errors (aList<int>will work safely under the typeList<num>in a lot of ways, just not all).We want to enable a statically checked typing discipline for variance as well (rejecting more programs, but providing a compile-time guarantee against the run-time type errors described above). This feature is concerned with the provision of support for that.
Declaration-site Variance
Declaration-site variance can be used to declare a strict and statically checked treatment of variance for each type variable of a generic class.
Syntactically, declaration-site variance consists in allowing each type parameter declaration of a generic class declaration to include one of the following modifiers:
out,in, orinout. We say that such a type parameter has explicit variance.The use of type parameters with explicit variance in the body of the enclosing class is restricted. It is a compile-time error for a type variable marked
outto occur in a non-covariant position in the signature of a member declaration; and for a type variable markedinto occur in a non-contravariant position. For example:Here are some core properties of declaration-site variance:
We obtain the following subtype relationships: Let
Cbe a generic class with one type parameterX. Assume thatSis a subtype ofT. IfXis markedoutthenC<S> <: C<T>; ifXis markedinthenC<T> <: C<S>. Note that there is no subtype relationship betweenC<S>andC<T>ifXis markedinout, unlessS == T.A type parameter with explicit variance can be used in the specification of a superinterface. For example:
Soundness is ensured via a number of rules like the following: It is a compile-time error if a type parameter
Xmarkedoutoccurs in a non-covariant position in an actual type argument for a superinterfaceDwhen the corresponding type parameter ofDis markedout; and ifXoccurs in a non-contravariant position in an actual type argument forDwhen the corresponding type parameter is markedin; and ifXoccurs at all in an actual type argument forDwhen the corresponding type parameter is markedinout.The interaction with dynamically checked covariant type parameters is similarly guarded: It is a compile-time error if a type parameter
Xmarkedoutoccurs in a non-covariant position in an actual type argument for a superinterfaceDwhen the corresponding type parameter ofDhas no explicit variance.In return for all these restrictions, we get static safety: For a class where all type parameters have explicit variance, every (non-dynamic) member access which is statically type correct is also dynamically safe (no type checks on parameter types etc. are needed at run time).
In general, declaration-site variance can be used for classes which are intended to be strictly type checked with respect to variance everywhere.
The text was updated successfully, but these errors were encountered: