diff --git a/working/augmentation-libraries/feature-specification.md b/working/augmentation-libraries/feature-specification.md index b1c376d64..aa14f815b 100644 --- a/working/augmentation-libraries/feature-specification.md +++ b/working/augmentation-libraries/feature-specification.md @@ -1,6 +1,6 @@ # Augmentations -Author: rnystrom@google.com, jakemac@google.com +Author: rnystrom@google.com, jakemac@google.com, lrn@google.com
Version: 1.21 (see [Changelog](#Changelog) at end) Augmentations allow spreading your implementation across multiple locations, @@ -64,259 +64,204 @@ part of macro expansion, it's still useful to have it be in a canonical well-specified form that users can understand. This proposal defines that format. The idea is that a Dart compiler executes -macros and then produces one or more library augmentation files that contain all -of the changes that the macros made to the library where they are applied. The -language then automatically merges those library augmentations into the -augmented library. +macros and then produces one or more new part files that contain all +of the changes that the macros made to the library where they are applied, as +new declarations to be added or augmentations that modify existing +declarations. The compiler then adds those part files to the existing libraries. -But library augmentations aren't *only* a serialization format for macros. They -are a first-class language feature that can be produced by non-macro code -generators or written by hand by users who simply want to break a giant library -or class into smaller files. +But improved part files and augmenting declarations are not *only* a +serialization format for macros. They are first-class language features that +can be produced by non-macro code generators or written by hand by users who +simply want to break a giant library or class into smaller files. -## Library Augmentations +## Part files with imports -A library augmentation is a separate file that *augments* an existing library, -referred to as the *augmented* library. A library augmentation is similar to -but more powerful than a part file, and it is not a library in its own right. +As part of the meta-programming and augmentation features, we expand the +capabilities of part files. See [“Parts with Imports”][parts_with_imports.md]. -* Like a library, it may contain all kinds of declarations—functions, - classes, variables, etc. +With that feature, a part file can now have its own `import` and `export` +directives, and further nested `part` files, with part files inheriting the +imports and prefixes of their parent (part or library) file. -* Like a library, it has its own import scope and may contain its own imports. +Augmentation declarations interact with part files mainly in restrictions on +where an augmenting declaration may occur relative to the declaration it +augments, as describe below. -* Like a part file, all of the top-level declarations it produces end up in - the top-level scope of the augmented library. +For this, we define the following relations on *declarations* based on the +relations between *files* of a library. -* Like a part file, it shares a private scope with the augmented library and - the two have mutual access to private declarations in the other file. +We say that a syntactic declaration *occurs in* a Dart file if the +declaration’s source code occurs in that Dart file. -Library augmentations may also contain declaration augmentations, which augment -existing declarations from the library. Some examples include: +We then say that a Dart file *contains* a declaration if the declaration occurs +in the file itself, or if any of the files included by the Dart file contains +the declaration. _That is, if the declaration occurs in a file in the sub-tree +of that Dart file._ -* Type augmentations, which can add new members to types, including adding new - values to enums, or even alter the type hierarchy by adding mixins, etc. - -* Function augmentations, which can replace the body of a function, or provide - a body if none was present. - -* Variable augmentations, which can wrap the initializer of a variable in the - augmented library, or provide an initializer if none was present. - -These can't be expressed today using only imports, exports, and part files. - -### Defining a library augmentation - -A library augmentation has almost the same syntax and semantics as a normal -Dart library. They are distinguished by a special `library` directive with -an `augment` modifier, like so: - -```dart -augment library 'main_library.dart'; -``` - -**TODO: Better syntax? How does this interact with [import short-hand -syntax][import]?** - -[import]: https://github.com/dart-lang/language/blob/master/working/0649%20-%20Import%20shorthand/proposal.md - -The URI points to the URI of the file which includes this library augmentation -via an `import augment ;`. - -After that, an augmentation may contain anything a regular Dart library can -contain: imports, exports, classes, functions, constants, etc. All -augmentations of an augmented library share the same top level declaration -scope. Declarations in any library augmentation or the augmented library are -visible to all of the others, including private ones. - -However, library augmentations do *not* share an import scope with the augmented -library or each other. The libraries one library augmentation imports are -visible only to that file. - -It is a compile-time error if: - -* A top-level declaration in a library augmentation has the same name as a - declaration in the augmented library or another of its library augmentations - (unless it is a declaration augmentation, described below). *This is the - same error conceptually as having a name collision in one file.* - -* An augmentation contains any `part` directives. - -* A library augmentation contains a normal `library` directive. They are not - self-contained libraries, only pieces of the augmented library. - -* An `import augment` directive has a `` that denotes an entity which - is not a library augmentation. *For example, it can not be a library.* - -* An `export` or `import` (not `import augment`) refers to an entity which - is not a library. *For example, it cannot be a library augmentation or - a part file.* - -* There is a cycle in the graph whose edges are the `import augment` - directives of an augmented library and of any library augmentations which - are directly or indirectly reachable from there via said edges. - -### Applying an augmentation - -A library applies an augmentation to itself using a new import directive with -the `augment` modifier, which looks like this: - -```dart -import augment 'some_augmentation.dart'; -``` - -This directive tells the compiler to read the given library augmentation and -merge its declarations into the augmented library. It is a compile-time error -if: - -* The URI referenced in an `import augment` directive is not a library - augmentation. +We then define a partial and a complete *ordering* of declarations of a library +as follows: -* The URI referenced in an `augment library` directive does not have an - `import augment` directive pointing back to this augmentation. +We define a partial ordering on syntactic declarations of a library, +*is above*, such that a syntactic declaration *A* is *above* a syntactic +declaration *B* if and only if: -* The same library augmentation is applied more than once. *In other words, - you can't have redundant `augment` directives that point to the same file.*. +* *A* and *B* occur in the same file, and the start of the *A* declaration is +syntactically before the start of the *B* declaration, in source order, or +* A file included by the file containing *A* contains *B*. -* The augmented library and its library augmentations do not all have the same - language version. There is only one library, and it should have a consistent - language version across its entire surface area. *A library augmentation - does not automatically inherit any language version from the augmented - library and may need an explicit language version comment of its own in order - to adhere to this requirement.* +We define a *total ordering relation* (transitive, anti-symmetric, irreflexive) +on declarations of a library, *is before* (and its reverse, *is after*) such +that for any two syntactic declarations *A*, and *B*: -Since the augmented library and its library augmentations both point to each -other, these rules imply that a given library augmentation can only be used to -augment a single library. +* If *A* and *B* occur in the same file, then: + * If the start of *A* is before the start of *B* in source order, + then *A* is before *B*. + * Otherwise *B* is before *A*. +* Otherwise *A* and *B* occur in different files. +* Let *F* be the least containing file for those two files. +* If *A* occurs in *F* then *A* is before *B*. +* If *B* occurs in *F* then *B* is before *A*. +* Otherwise *A* and *B* are contained in distinct included files of *F*. +* If the `part` directive including the file that contains *A* + is syntactically before the `part` directive including the file that + contains *B* in source order, then *A* is before *B*. +* Otherwise *B* is before *A*. -### Merge order +Then *B* *is after* *A* if and only if *A* *is before* *B*. -A library may apply multiple library augmentations to itself. Also, library -augmentations may themselves contain `import augment` directives. The entire -tree of library augmentations is recursively applied to the augmented library. -The merge order is defined as a depth-first pre-order traversal of the library -augmentations, in the source order of their `import augment` directives. +(Here the first five points can be summarized as “If *A* is above *B*, then *A* +is before *B*, and vice versa” and the remaining case covers when the two are +contained in sibling part directives, and at least one of those three cases +must occur.) -Within a single library augmentation, you may augment the same declaration -multiple times, whether it is a top level or nested declaration. The merge -order is defined as the source order of the declaration augmentations. +This order is total. It’s effectively ordering declarations as by a pre-order +depth-first traversal of the file-tree, visiting declarations of a file in +source order, and then recursing on `part`-directives in source order. -For example: +[parts_with_imports.md]: parts_with_imports.md "Parts with Imports Feature Specification" -``` -// main.dart -import augment 'a.dart'; -import augment 'c.dart'; +## Augmentation declarations -class C {} +Augmentation declarations are declarations marked with the new built-in +identifier `augment`, which makes the declaration augment an existing +declaration, which is itself a normal declaration with zero or more prior +augmentations applied. An augmentation declaration does not introduce a new +name into the surrounding scope, it attaches itself to the existing name. -void trace() { - print('main'); -} - -// a.dart -augment library 'main.dart'; - -import augment 'b.dart'; - -augment class C {} - -augment void trace() { - augmented(); - print('a'); -} - -// b.dart -augment library 'a.dart'; +Making `augment` a built-in identifier is language versioned, to make it +non-breaking for pre-feature code. -class D {} +Augmentation declarations include: -augment void trace() { - augmented(); - print('b'); -} - -// c.dart -augment library 'main.dart'; - -augment class D {} +* Type augmentations, which can add new members to types, including adding new + values to enums, or even alter the type hierarchy by adding mixin + applications to a class. -augment void trace() { - augmented(); - print('c'); -} +* Function augmentations, which can replace the body of a function, or provide + a body if none was present. -augment void trace() { - augmented(); - print('d'); -} -``` +* Variable augmentations, which can wrap the initializer of a variable in the + augmented library, or provide an initializer if none was present. -The merge order is `main.dart`, `a.dart`, `b.dart`, then `c.dart`. The -declarations in those library augmentations -—new declarations or augmentations— are processed in that order, -and source order within that. - -This order is user-visible in two ways: - -* A regular (i.e. non-augmenting) declaration must appear first before it can - be augmented. For example, `C` in `main.dart` is augmented by `C` in - `a.dart`. Likewise, `D` in `b.dart` is augmented by `D` in `c.dart`. Note - that the latter is allowed even though `b.dart` does not itself import - `c.dart`. - -* When the same declaration is augmented multiple times, merge order - determines the order that those wrappers are applied. When the `trace()` - function is called, it prints: - - ``` - main - a - b - c - d - ``` - -**TODO: Should it be a compile-time error if the augmented library and -library augmentation are in different packages?** - -## Augmenting declarations - -Unlike part files, which can only add entirely new declarations, a library -augmentation can also modify existing declarations in the augmented library. -This can mean adding new members to an existing type, or even modifying the code -of an existing declaration. There is a new built-in identifier, `augment`, which -is used to syntactically mark a declaration as an augmentation of an existing -one. The introduction of this new identifier will be language versioned in order -to make it non-breaking for old code. - -It is also allowed for a non-abstract class to have abstract members, if those -members are filled in by an augmentation. This is primarily useful for macros, -which may be used to provide a body for an abstract member. - -Often, an augmentation wants to also preserve and run the code of the original -declaration it augments (hence the name "augmentation"). It may want run before -the original code, after it, or both. To allow that, we allow a new expression -syntax inside the bodies of augmenting members. Inside a member marked -`augment`, the expression `augmented` can be used to refer to the original -function, getter, setter, or variable initializer. This is a contextual keyword +These operations cannot be expressed today using only imports, exports, or part +files. Any Dart file (library file or part file) can contain augmentation +declarations. + +An augmentation declaration can add new members to an existing type, or even +modify the code of an existing declaration. + +Because of augmentations, non-abstract class (or similar) declarations are now +allowed to contain abstract member declarations, as long as those +members are filled in by an augmentation declaration. _This is primarily useful +for macros, which may be used to provide a body for an abstract member._ + +An augmentation that replaces the body of a function, may also want to preserve +and run the code of the original declaration that it augments (hence the name +"augmentation"). It may want to run its own code before the original code, +after it, or both. To support that, we allow a new expression syntax inside the +“bodies” of augmenting declarations (function bodies, constructor bodies, and +variable initializers). Inside an expression of a member marked +`augment`, the identifier `augmented` can be used to refer to the original +function, getter, or setter body, or variable initializer. This is a contextual +keyword within `augment` members, and has no special meaning outside of that context. -See the next section for a full specification of what `augmented` actually -means, in the various contexts. - -The same declaration can be augmented multiple times by separate library -augmentations. When that happens, the merge order defined previously determines -which order the wrapping is applied. - -It is a compile-time error if: - -* An augmenting declaration has no corresponding original declaration to - apply to. - -* An augmenting declaration appears in a library before the library where the - original declaration occurs, according to merge order. *A library - augmentation can both declare a new declaration and augment it in the same - file.* +See the next section for a full specification of what `augmented` means, and +how it must be used, in the various contexts. + +The same declaration can be augmented multiple times by separate augmentation +declarations. When that happens, the *augmentation application order* defines +in which order the augmentations are applied, with later augmentations applying +to the result of applying all earlier augmentations to the original base +declaration. The augmentation application order is defined rather simply, +because of a further requirement on where augmentations of the same declaration +can occur relative to each other. + +It’s a **compile-time error** if a library contains an augmentation +declaration, but no non-augmentation declaration with the corresponding name in +the same scope. _(A mutable variable declaration counts as having both a getter +and a setter name.)_ + +For the following, we’ll say that one declaration of a library is *above* +another declaration of the same library if and only if: + +* The former declaration is in the same file as the latter declaration, and it + is textually earlier in the file (“above” in the source code as normally + presented), or +* The former declaration is in a file that is a direct or transitive parent + file of the file of the latter declaration (“above” in the file tree + hierarchy). + +We can similarly define *below* as the inverse of that relation. Both *before* +and *after* define *strict partial orders* on declarations in a library. + +It’s a **compile-time error** if a library contains an augmentation declaration +and a corresponding non-augmentation base declaration, and the base +declaration is not *above* the augmentation declaration. + +These requirements ensure that declarations that contribute to the same +effective declaration, one base declaration and zero or more augmentation +declarations, are *totally ordered* by the *above* relation, with the base +declaration at the top, and the declarations all being in files on a single +*path* down the file tree. + +The *augmentation application order* for a single base declaration’s (validly +positioned) augmentation declarations is then in *before* order: An augmentation +declaration is applied after any augmentation declarations that are *before* it, +and before augmentation declarations that it is before. + +This applies both to top-level declarations and to member declarations of, for +example, class declarations. + +#### Path requirement lint suggestion + +One issue with the augmentation application order is that it is not stable +under reordering of `part` directives. Sorting part directives can change the +order that augmentation applications in separate included sub-trees are applied +in. + +To help avoiding issues, we want to introduce a *lint* which warns if a library +is susceptible to part file reordering changing augmentation application order. +A possible name could be `augmentation_ordering`. + +It’s effect would be to **report a warning** *if* for any two (top-level) +augmenting declarations with name *n*, one is not *above* the other. + +The lint would only apply to user-written augmenting declarations, it should +not include macro generated augmentations. Those are placed where the macro +processor chooses to place them, usually after all other augmentations. + +If the lint is satisfied, then all augmenting declarations are ordered by the +*before* relation, which means that they no two can be in different sibling +parts of the same file, and therefore all the augmenting declarations occur +along a single path down the part-file tree. _That ensures that +*part file directive ordering* has no effect on augmentation application order._ + +The language specification doesn’t specify lints or warnings, so this lint +suggestion is not normative. We wish to have the lint, and preferably include +it in the “recommended” lint set, because it can help users avoid accidental +problems. We want it as a lint instead of a language restriction so that it +doesn’t interfere with macro-generated code, and so that users can `// ignore:` +it if they know what they’re doing. ### Augmented Expression @@ -401,50 +346,71 @@ augment class SomeClass { ``` This means that instead of creating a new declaration, the augmentation modifies -a corresponding declaration in the augmented library or one of its other -augmentations. +a corresponding declaration (above) in the library. A class, enum, extension type, or mixin augmentation may specify `extends`, -`implements`, `on`, and `with` clauses (when generally supported). The types -in these clauses are appended to the original declarations clauses of the same -kind, and if that clause did not exist previously then it is added with +`implements` and `with` clauses (when generally supported). The types +in these clauses are appended to the original declarations’ clauses of the same +kind, and if that clause did not exist previously, then it is added with the new types. All regular rules apply after this appending process, so you cannot have multiple `extends` on a class, or an `on` clause on an enum, etc. -Instance or static members defined in the body of the type, including enum -values, are added to the instance or static namespace of the corresponding type -in the augmented library. In other words, the augmentation can add new members -to an existing type. +Instance or static members defined in the body of the augmenting type, +including enum values, are added to the instance or static namespace of the +corresponding type in the augmented library. In other words, the augmentation +can add new members to an existing type. Instance and static members inside a type may themselves be augmentations. In -that case, they augment the corresponding members in the original type -declaration according to the rules in the following subsections. +that case, they augment the corresponding members in augmented type +declaration (a based declaration and zero or more augmentations that are all +above the current augmenting type declaration) according to the rules in the +following subsections. -It is a compile-time error if: +It is a **compile-time error** if: * The augmenting type and corresponding type are not the same kind: class, - mixin, enum, extension, or extension type. You can't augment a class with a + mixin, enum, extension, or extension type. You cannot augment a class with a mixin, etc. -* The augmenting type and corresponding type do not have all the same - modifiers (`abstract`, `base`, `final`, `interface`, `sealed`, and `mixin` - when it occurs immediately before `class`). +* The augmenting type and augmented type do not have all the same + modifiers: `abstract`, `base`, `final`, `interface`, `sealed` and `mixin` + for `class` declarations, and `base` for `mixin` declarations. *This is not a technical requirement, but it ensures that looking at either - declaration show the complete capabilities of the declaration. It also + declaration shows the complete capabilities of the declaration. It also deliberately prevents an augmentation from introducing a restriction that isn't visible to a reader of the main declaration.* +* The augmenting type declares an `extends` clause for a `class` declaration, + but one was already + present _(or the `class` was a `mixin class` declaration, which does not + allow `extends` clauses)_. We do not allow overwriting an existing + `extends`, but one can be filled in if none had been specified. + * The augmenting type declares an `extends` clause, but one was already present. We don't allow overwriting an existing `extends`, but one can be filled in if it wasn't present originally. -* The type parameters of the type augmentation do not match the original +* An augmenting extension declares an `on` clause. We don't allow replacing + this in for `extension` declarations, and the `on` clause is required on + the original declaration, so an `augment extension` cannot have any `on` + clause. We also do not allow adding further restrictions to a `mixin` + declaration, so no further types can be added to its `on` clause, if it + even has one. These restrictions could both be lifted later if we have a + compelling use case, as there is no fundamental reason it cannot be + allowed. It is a parse error today to have an `extension` declaration with + no `on` clause. + +* The type parameters of the augmenting type do not match the original type's type parameters. This means there must be the same number of type - parameters with the same bounds and names. + parameters with the exact same type parameter names (same identifiers) and + bounds (same *types*, even if they may not be written exactly the same in + case one of the declarations needs to refer to a type using an import + prefix). *Since repeating the type parameters is, by definition, redundant, this - doesn't accomplish anything semantically. But it ensures that anyone reading + restriction doesn't accomplish anything semantically. It ensures that + anyone reading the augmenting type can see the declarations of any type parameters that it uses in its body and avoids potential confusion with other top-level variables that might be in scope in the library augmentation.* @@ -452,7 +418,7 @@ It is a compile-time error if: ### Augmenting functions A top-level function, static method, instance method, or operator may be -augmented to wrap the original code in additional code: +augmented to replace or wrap the original body code in additional code: ```dart // Wrap the original function in profiling: @@ -464,32 +430,33 @@ augment int slowCalculation(int a, int b) { } ``` -The augmentation replaces the original function body with the augmenting code. -Inside the augmentation body, a special `augmented()` expression may be used to -execute the original function body. That expression takes an argument list -matching the original function's parameter list and returns the function's -return type. +The augmentation replaces the augmented function’s body with the augmenting +function’s body. +Inside the augmenting function’s body, a special `augmented(…)` expression may +be used to execute the original function body. That expression takes an +argument list matching the original function's parameter list and returns the +function's return type. The augmenting function does not have to pass the same arguments to -`augmented()` as were passed to it. It may call it once, more than once, or not -at all. +`augmented(…)` as were passed to it. It may invoke `augmented` once, more than +once, or not at all. It is a compile-time error if: -* The signature of the function augmentation does not exactly match the - original function. This means the return types must be the same; there must - be the same number of positional, optional, and named parameters; the types - of corresponding positional and optional parameters must be the same; the - names and types of named parameters must be the same; any type parameters - and bounds must be the same; and any `required` or `covariant` modifiers - must match. +* The function signature of the augmenting function does not exactly match the + function signature of the augmented function. This means the return types + must be the same type; there must be same number or required and optional + positional parameters, all with the same types, the sane number of named + parameters, each pairwise with the same name, same type and same `required` + and `covariant` modifiers, and any type parameters and their bounds must be + the same (like for type declarations). *Since repeating the signature is, by definition, redundant, this doesn't accomplish anything semantically. But it ensures that anyone reading the augmenting function can see the declarations of any parameters that it uses in its body.* -* The function augmentation specifies any default values. *Default values are +* The augmenting function specifies any default values. *Default values are defined solely by the original function.* * An augmenting declaration uses `augmented` when the original declaration has @@ -497,9 +464,6 @@ It is a compile-time error if: to have an implementation provided by another external source, and they will throw a runtime exception when called if not. -**TODO: Should we allow augmenting functions to add parameters? If so, how does -this interact with type checking calls to the function?** - ### Augmenting variables, getters, and setters While the language treats variables, getters, and setters as @@ -515,67 +479,81 @@ You can think of variable, getter, and setter declarations all as ways to define a higher-level "property" construct. A property has a name and a type. It may have one or more other capabilities: -* **A backing storage location.** You get this when you declare a variable. - This also enables an instance variable to be assigned in a constructor - initializer list. A variable may also have an **initializer** expression - that gets run either lazily for top-level variables and static fields or at - construction time when an instance is created. +* **A backing storage location.** You get this when you declare a + non-`external` variable. + Having a storage location enables (and often requires) having the variable + initialized by generative constructors. A variable may also have an + **initializer** expression that gets run either lazily for top-level and + static variables or at object construction/initialization time for instance + variables. -* **A getter function.** This function body is provided explicitly when you +* **A getter function.** This function’s body is provided explicitly when you declare a getter. A variable declaration provides an implicit getter body that returns the value in the backing storage location. (Late variables do some additional checking in that implicit body.) -* **A setter function.** A setter declaration provides this body explicitly. A +* **A setter function.** A setter declaration provides a body explicitly. A non-final variable declaration provides an implicit setter body that stores the given value in the storage location. (Again, late variables do some - additional checks.) + additional updates and/or checks.) Declarations may be marked `abstract` or `external` and, if so, those are -mapped over to the corresponding getter and setter functions. +mapped over to the corresponding getter and setter functions. An `abstract` +variable declaration is equivalent to an abstract getter declaration, and if +not `final`, also an abstract setter declaration. An `external` variable +similarly define an `external` getter and possibly an `external` setter, but +unlike abstract declarations, these are a valid implementations of the +signature. Augmentations on variables, getters, and setters works mostly at the level of these separate capabilities. For example, augmenting a variable with a getter -replaces the variable's implicit getter body with the augmenting getter's. +replaces the augmented variable's implicit getter body with the augmenting +getter's. More specifically: -* **Augmenting with a getter:** A getter in an augmentation can augment a - getter in the library or the implicit getter defined by a variable in the - library. Inside the augmenting body, an `augmented` expression invokes the - original getter. +* **Augmenting with a getter:** An augmenting getter can augment a getter + declaration, or the implicit getter of a variable declaration, with all + prior augmentations applied, by replacing the body of the augmented getter + with the body of the augmenting getter. Inside the augmenting getter’s + body, an `augmented` expression executes the augmented getter’s body. -* **Augmenting with a setter:** A setter in an augmentation can augment a - setter in the library or the implicit setter defined by a non-final variable - in the library. Inside the augmenting setter, an `augmented =` expression - invokes the original setter. +* **Augmenting with a setter:** An augmenting setter can augment a setter + declaration, or the implicit setter of a variable declaration, with all + prior augmentations applied, by replacing the augmented setter’s body with + the augmenting setter’s body. Inside the augmenting setter’s body, an + `augmented = ` assignment invokes the original setter with the + value of the expression. * **Augmenting a getter and/or setter with a variable:** This is a compile-time error in all cases. Augmenting an abstract or external variable with a variable is also a compile-time error, as those are actually just syntax sugar for getter/setter pairs and do not have an initializer that you - can augment. + can augment. *An augmenting variable replaces its augmented variable’s + initializer expression, and that can only be done on a declaration that can + have an initializer expression.* - We may decide in the future to allow augmenting abstract or external - getters, setters, or variables with variables, but for now you can instead - use the following workaround: + We may decide in the future to allow augmenting abstract getters, setters, + or variables with variables, but for now you can instead use the following + workaround: - Add a new field. - Augment the getter and/or setter to delegate to that field. - If a non-abstract, non-external variable is augmented by a getter or setter, - you **can** still augment the variable, as you are only augmenting the - initializer of the original variable. This is not considered to be - augmenting the augmenting getter or setter, since those are not actually - altered. + If a non-abstract, non-external variable is augmented by an augmenting + getter or setter, you **can** still augment the variable, as you are only + augmenting the initializer of the original variable. This is not considered + to be augmenting the augmenting getter or setter, since those are not + actually altered. The reason for this compile time error is that whether a member declaration - is a field versus a getter/setter is a visible property of the declaration: + is a field versus a getter/setter is a visible property of the declaration + inside the same class or even library: - - It determines whether the member can be initialized in a constructor - initializer list. - - It is also a visible distinction when introspecting on a program with the - analyzer, macros, or mirrors. + * It determines whether the member can be initialized in a constructor + initializer list. + * It is also a visible distinction when introspecting on a program with + the analyzer, macros, or mirrors. When a declaration is augmented, we don't want the augmentation to be able to change any of the known properties of the existing member being @@ -587,55 +565,76 @@ More specifically: * **Augmenting a variable with a variable:** Augmenting a variable with a variable only alters its initializer. External and abstract variables cannot - be augmented with variables, because they have no initializer to augment. + be augmented with variables, because they have no initializer expression to + augment. - Since the initializer is the only meaningful part of the augmenting - declaration, an initializer must be provided. This augmenting initializer - replaces the original initializer. The augmenting initializer may use an - `augmented` expression which executes the original initializer expression - when evaluated. + Since the initializer expression is the only meaningful part of the + augmenting declaration, an initializer expression must be provided. This + augmenting initializer replaces the original initializer. The augmenting + initializer may use an augmented` expression which executes the original + initializer expression when evaluated. The `late` property of a variable must always be consistent between the augmented variable and its augmenting variables. - If the variable declaration in the original library does not have a type - annotation, then the type is inferred only using the original library's - initializer. (If there is no initializer in the original library, then the - variable is inferred to have type `dynamic` like any non-augmented variable. - *This ensures that augmenting a variable doesn't change its type. This is - necessary to ensure that macros running after signatures are known can't - change the signature of a declaration.* - -It is a compile-time error if: - -* The original and augmenting declarations do not have the same type. - -* An augmenting declaration uses `augmented` when the original declaration - has no concrete implementation. Note that all external declarations are - assumed to have an implementation provided by another external source, - and they will throw a runtime exception when called if not. + If the original variable declaration does not have a type annotation, then + the variable's declared type is found using only that declaration, + without looking at any further augmenting declarations. + The type can either be inferred from an initializer expression of the + original variable declaration, be inherited from a superinterface for an + instance variable, or default to a type of `dynamic` if neither applies. + *This ensures that augmenting a variable doesn't change its type. That is + necessary to ensure that macros cannot change the signature of a + declaration, a signature which may have been depended on by other code, + or other macros.* -* An augmenting initializer uses `augmented` and the augmented declaration - is not an initializing variable declaration. +It is a **compile-time error** if: -* A final variable declaration is augmented with a setter declaration. - *Instead, the augmentation can declare a non-augmenting setter that - goes alongside the implicit getter defined by the final variable.* +* The original and augmenting declarations do not have the same declared + types (return type for getters, parameter type for setters, declared type + for variables). -* A non-final variable declaration is augmented with a final variable - declaration. *We don't want to leave the original setter declaration in - a weird state.* - -* A `late` variable declaration is augmented with a non-`late` variable - declaration. - -* A non-`late` variable declaration is augmented with a `late` variable - declaration. - -* A getter or setter declaration is augmented by a variable declaration. - -* An `abstract` or `external` variable declaration is augmented by a - variable declaration. +* An augmenting declaration uses `augmented` when the original declaration has + no concrete implementation. Note that all external declarations are assumed + to have an implementation provided by another external source, and + otherwise they will throw a runtime error when called. An `abstract` + variable introduces no implementation. + +* An augmenting variable’s initializer expression uses `augmented` and the + augmented variable is not a variable with an initializer. + +* A non-writable variable declaration is augmented with a setter. ( + Instead, the author can declare a *non-augmenting* setter that goes + alongside the implicit getter defined by the final variable.) + _Non-writable variable declarations are any that does not introduce a + setter, including non-`late` `final` variables, `late final` variables + with an initializer, and `const` variables._ + +* A non-final variable is augmented with a final variable. We don't want to + leave the original setter in a weird state. + * A final variable can be augmented with a non-`final` augmenting + variable, and that will not add any setter. An augmenting variable + declaration only affects the initializer expression, not setters. + +* A variable is augmented with another variable, and one is `late` and + the other is not. *(Augmentation cannot change `late`-ness, and since being + `late` does affect the initializer expression, the augmenting variable is + required to repeat the `late`.)* + +* A getter or setter base declaration is augmented by an augmenting variable. + +* An abstract or external variable base declarations is augmented by an + augmenting variable. + +* A late final variable with no initializer expression is augmented by an + augmenting variable with an initializer expression. + _A late final variable with no initializer has a setter, while one with an + initializer does not. An augmentation must not change whether there is a + setter._ + +* A `const` variable is augmented by an augmenting getter. **(TODO: Can a + const variable be augmented by another const variable, changing its value, + or is that too weird?)** ### Augmenting enum values @@ -650,8 +649,9 @@ the original enum value, or provide an argument list where none was present before. New enum values may also be defined in the augmentation, and they will be -appended to the original values in augmentation traversal order. Augmenting an -existing enum value never changes the order in which it appears in `values`. +appended to the current values of the declaration in augmentation application +order. Augmenting an existing enum value never changes the order in which it +appears in `values`. For example: @@ -693,7 +693,8 @@ Then `A.values` is `[A.first, A.second, A.third, A.fourth]`. It is a compile-time error if: -* An augmenting getter is defined for an enum value. +* An augmenting getter is defined for an enum value. _An enum value counts as + a constant variable._ ### Augmenting constructors @@ -704,31 +705,42 @@ constructor's initializers, but before any original super initializer or original redirecting initializer if there is one. In the augmenting constructor's body, an `augmented()` call invokes the -original constructor's body. +original constructor's body. The expression has type `void` and evaluates to +`null`. **(TODO: This is slightly under-specified. We can use the current +bindings of the parameters of the augmenting constructor as the initial binding +of parameter variables in the augmented body, or we can execute the body in the +current *scope*, using the same variables as the current body. The latter is +not what we do with functions elsewhere, and allows the `augmented` expression +to modify local variables, but the former introduces different variables than +the ones that existed when evaluating the initializer list. If the initializer +list captures variables in closures, that body may not work.)** It is a compile-time error if: -* The signature of the constructor augmentation does not match the original - constructor. This means the return types must be the same; there must be the - same number of positional, optional, and named parameters; the types of - corresponding positional and optional parameters must be the same; the names - and types of named parameters must be the same; any type parameters and - bounds must be the same; and any `required` or `covariant` modifiers must - match. Any initializing formals must be the same in both constructors. +* The function signature of the augmenting constructor does not match the + signature of the augmented constructor. This means that the parameters must + be the same (just as for augmenting functions, except here there is no + return type and no type parameters on the constructor itself). Any + initializing formals must be the same in both constructors. Any super + parameters must be the same in both constructors. **TODO: Is this the right way to handle initializing formals?** -* The constructor augmentation specifies any default values. *Default values - are defined solely by the original constructor.* +* The augmenting constructor parameters specify any default values. + *Default values are defined solely by the original constructor.* -* The original constructor is `const` and the augmenting constructor is not +* The original constructor is `const` and the augmenting constructor is not, or vice versa. -* The original constructor is a factory constructor and the augmenting - constructor has an initializer list. +* The original constructor is marked `factory` and the augmenting + constructor is not, or vice versa. -* The original constructor has a super initializer or redirecting initializer - and the augmenting constructor does too. +* The original constructor has a super initializer _(super constructor + invocation at the end of the initializer list)_ and the augmenting + constructor does too. _An augmentation can replace the implicit default + `super()` with a concrete super-invocation, but cannot replace a declared + super constructor._ **(TODO: Why not? We allow “replacing implementation”, + and this is *something* like that.)** **TODO: What about redirecting constructors?** @@ -755,8 +767,8 @@ https://github.com/dart-lang/language/blob/main/working/2364%20-%20primary%20con When augmenting an `external` member, it is assumed that a real implementation of that member has already been filled by some tool prior to any augmentations being applied. Thus, it is allowed to use `augmented` from augmenting members -on external declarations, but it may throw a `noSuchMethod` error at runtime if -no implementation was in fact provided. +on external declarations, but it may throw a `NoSuchMethodError` error at +runtime if no implementation was in fact provided. **NOTE**: Macros should _not_ be able to statically tell if an external body has been filled in by a compiler, because it could lead to a different result on @@ -767,40 +779,45 @@ whether there is an external implementation to call?** ### Metadata annotations and macro applications -An augmentation declaration may have metadata annotations or macro applications. -These are appended to the list of metadata annotations and macro applications on +An augmentation declaration may have metadata annotations, including macro +applications. These are appended to the list of metadata annotations on the original declaration. ## Scoping -Like part files, the augmented library and all of its library augmentations -share a single top-level scope where declarations are defined. They also share a -single private namespace. This means that private declarations in the augmented -library or an augmentation of it are visible to all augmentations. - -Unlike part files, a library augmentation has its own import scope surrounding -that shared top-level scope. Any libraries the augmentation imports are visible -only to that library augmentation. Likewise, libraries imported by the augmented -library are not implicitly imported by the library augmentation. - -Exports in a library augmentation are applied to the augmented library and -become exports from the augmented library's namespace. - -The static and instance member namespaces for an augmented type are shared -across the declaration of the type in the augmented library and all -augmentations of that type. Identifiers in the bodies of members (both implicit -ones and explicit uses like `this.` or `TypeName.`) are resolved against that -complete merged namespace. For example: +The static and instance member namespaces for a type or extension declaration, +augmenting or not, are lexical only. Only the declarations (augmenting or not) +declared inside the actual declaration are part of the lexical scope that +member declarations are resolved in. + +_This means that a static or instance member declared in the base declaration +of a class is not *lexically* in scope in an augmentation of that class, just +as an inherited instance member is not in the lexical scope of a class +declaration._ + +If a member declaration needs to reference a static or instance member declared +in another base or augmenting declaration of the same type, it can use `this. +name` for instance members an `TypeName.name` for static members to be +absolutely sure. Or it can rely on the default if `name` is not in the lexical +scope at all, in which case it’s interpreted as `this.name` if it occurs inside +a scope where a `this` is available. _This approach is always potentially +dangerous, since any third-party import adding a declaration with the same name +would break the code. In practice that’s almost never a problem, because +instance members and top-level declarations usually use different naming +strategies._ + +Example: ```dart // Main library "some_lib.dart": import 'other_lib.dart'; -import augment 'some_augment.dart'; +part 'some_augment.dart'; -const a = 1; +const b = 37; class C { + const int b = 42; bool isEven(int n) { if (n == 0) return true; return !_isOdd(n - 1); @@ -808,45 +825,50 @@ class C { } // Augmentation "some_augment.dart": -augment library 'some_lib.dart'; +part of 'some_lib.dart'; import 'also_lib.dart'; -const b = 2; - augment class C { - bool _isOdd(int n) => !isEven(n - 1); + bool _isOdd(int n) => !this.isEven(n - 1); + void printB() { print(b); } // Prints 37 } ``` -This code is fine. Code in C in the augmented library can refer to members added -in the augmentation like `_isOdd()`. Meanwhile, code in the augmentation can see -members like `isEven()` declared in the augmented library. +This code is fine. Code in `C.isEven` can refer to members added +in the augmentation like `_isOdd()` because there is no other `_isOdd` in +scope, and code in `C._isOdd` works too by explicitly using `this.isEvent` to +ensure it calls the correct method. You can visualize the namespace nesting sort of like this: ``` -some_lib.dart | some_augment.dart - | -.-----------------. | .-----------------. -| import scope: | | | import scope: | -| other_lib | | | also_lib | -'-----------------' | '-----------------' - ^ | ^ - | | | +some_lib.dart : + :` expression as - appropriate calls the augmented function body. - - **TODO: What is the syntax for calling a prefix operator's original - code?** - - 1. Else, the declaration is a variable: - - 1. Replace a matching variable, getter, and/or setter in the namespace - with the declaration. Inside the augmenting variable's initializer - expression, an `augmented` expression invokes the original variable - initializer. +The application of augmentation declarations to a base declaration produces +something that looks and behaves like a single declaration: It has a single +name, a single type or function signature, and it’s what all references +to the *name* refers to inside and outside of the library. + +Unlike before, that single *semantic declaration* now consists of multiple +*syntactic* declarations (one base declaration, the rest augmenting +declarations, with a given augmentation application order), and the properties +of the combined semantic declaration can be derived from the syntactic +declarations. + +We redefine a number of semantic functions to now work on a *stack* of +declarations (the declarations for a name in bottom to top order), so that +existing semantic definitions keep working. + +### Example: Class declarations + +#### Super-declarations + +The specification of class modifiers introduced a number of predicates on +*declarations*, to check whether the type hierarchy is well formed and the +class modifiers are as required, before the static semantics have even +introduced *types* yet. We modify those predicates to apply to a stack of +augmenting and base declarations as follows: + +* A a non-empty *stack* of syntactic class declarations, *C*, has a + declaration *D* as *declared super-class* if: + * *C* starts with an (augmenting or not) class declaration *C0* and either + * *C0* has an `extends` clause whose type clause denotes the + declaration *D*, or + * *C0* is an augmenting declaration, so *C* continues with a + non-empty *Crest*, and *Crest* has *D* as + declared super-class. +* A a non-empty *stack* of syntactic class declarations, *C*, has a + declaration *D* as *declared super-interface* if: + * *C* starts with an (augmenting or not) class declaration *C0* and either + * *C0* has an `implements` clause with an entry whose type clause + denotes the declaration *D*, or + * *C0* is an augmenting declaration, so *C* continues with a + non-empty *Crest*, and *Crest* has *D* as + declared super-interface. +* A a non-empty *stack* of syntactic class declarations, *C*, has a + declaration *D* as *declared super-mixin* if: + * *C* starts with an (augmenting or not) class declaration *C0* and either + * *C0* has a `with` clause with an entry whose type clause denotes + the declaration *D*, or + * *C0* is an augmenting declaration, so *C* continues with a + non-empty *Crest*, and *Crest* has *D* as + declared super-mixin. + +#### Members + +A class declaration stack, *C*, of a one non-augmenting and zero or more +augmenting class declarations, defines an *augmented interface* (member +signatures) and *augmented implementation* (instance members declarations) +based on the individual syntactic declarations. + +A non-empty class declaration stack, *C*, has the following set of instance +member declarations: + +* Let *Ctop* be the latest declaration of the stack, and + *Crest* the rest of the stack. +* If *Ctop* is a non-augmenting declaration, the declarations of + *C* is the set of syntactic instance member declarations of + *Ctop*. +* Otherwise let *P* be the set of member declarations of the non-empty stack + *Crest*. +* and the member declarations of *C* is the set *R* defined as containing + only the following elements: + * A singleton stack of each syntactic instance member declaration *M* of + *Ctop*, where *M* is a non-augmenting declaration. + * The elements *N* of *P* where *Ctop* does not contain an + augmenting instance member declaration with the same name _(mutable + variable declarations have both a setter and a getter name)_. + * The stacks of a declaration *M* on top of the stack *N*, where *N* is a + member of *P*, *M* is an augmenting instance member declaration of + *Ctop*, and *M* has the same name as *N*. + +And we can whether such an instance member declaration stack, *C*, *defines an +abstract method* as: + +* Let *Ctop* be the latest element of the stack and + *Crest* the rest of the stack. +* If *Ctop* is a non-variable declaration, and is not declared + `abstract`, the *C* doe +* If *Ctop* declares a function body, then *C* does not define an + abstract method. +* Otherwise *C* defines an abstract method if *Crest* defines an + abstract method. + +(This is just for methods, we will define it more generally for members, +including variable declarations.) + +### Example: Instance methods + +#### Properties + +Similarly we can define the properties of stacks of member declarations. + +For example, we define the *augmented parameter list* of a non-empty stack, +*C*, of augmentations on a base function declaration as: + +* Let *Ctop* be the latest element of the stack and + *Crest* the rest of the stack. +* If *Ctop* is not an augmenting declaration, its augmented + parameter list is its actual parameter list. _(And *Crest* is + known to be empty.)_ +* Otherwise *Ctop* is an augmenting declaration with a parameter + list which must have the same parameters (names, positions, optionality and + types) as its augmented declaration, except that it is not allowed to + declare default values for optional parameters. + * Let *P* be the augmented parameter list of *Crest*. + * The augmented parameter list of *Ctop* is then the parameter + list of *Ctop*, updated by adding to each optional parameter + the default value of the corresponding parameter in *P*, if any. + +_This will usually be exactly the parameter list of the original non-augmenting +declaration, but the ordering of named parameters may differ. This is mostly +intended as an example, in practice the augmented parameter list can just be +the parameter list of the original non-augmenting declaration, but it’s more +direct and clearly correct to use the actual parameter list of the declaration +when creating the parameter scope that its body will run in._ + +Similarly we define the _augmented function type_ of the declaration stack. +Because of the restrictions we place on augmentations, they will all have the +same function type as the original non-augmenting declaration, but again it’s +simpler to assign a function type to every declaration. + +#### Invocation + +When invoking an instance member on an object, the current specification looks +up the corresponding implementation on the class of the runtime-type of the +receiver, traversing super-classes, until it it finds a non-abstract +declaration or needs to search past `Object`. The specification then defines +how to invoke that method declaration, with suitable contexts and bindings. + +We still define the same thing, only the result of lookup is not a single +declaration, but a stack of augmenting declarations on top of a base +declaration, and while searching, we skip past *declaration stacks* that define +an abstract method. The resulting stack is the *member definition*, or +*semantic declaration*, which is derived from the syntactic declarations in the +source. + +Invoking a *stack*, *C*, of instance method declarations on a receiver object +*o* with an argument list *A* and type arguments *T*, is then defined as +follows: + +* Let *Ctop* be the latest declaration on the stack (the last + applied augmentation in augmentation application order), and + *C**rest* the rest of the stack. +* If *Ctop* has a function body *B* then: + * Bind actuals to formals (using the usual definition of that), binding + the argument list *A* and type arguments *T* to the *augmented + parameter list* of *C**top* and type parameters of + *Ctop*. This creates a runtime parameter scope which has the + runtime class scope as parent scope (the lexical scope of the class, + except that type parameters of the class are bound to the runtime type + arguments of those parameters for the instance *o*). + * Execute the body *B* in this parameter scope, with `this` bound to *o*. + * If *B* contains an expression of the form `augmented(args)` + (type arguments omitted if empty), then: + * The static type of `augmented` is the augmented function type of + *Crest*. The expression is type-inferred as a function + value invocation of a function with that static type. + * To evaluate the expression, evaluate `args` to an argument list + *A2*, invoke *Crest* with argument list *A2* and type + arguments that are the types of `TypeArgs`. The result of + `augmented(args)` is the same as the result of that + invocation (returned value or thrown error). + * _There would have been a compile-time error if there is no earlier + declaration with a body._ + * The result of invoking *C* is the returned or thrown result of + executing *B*. +* Otherwise, the result of the invocation of *C* is the result of invoke + *Crest* on *o* with argument list *A* and type arguments *T*. + * _This will eventually find a body to execute, otherwise *C* would have + defined an abstract method, and would not have been invoked to begin + with._ ## Documentation comments @@ -1069,48 +1166,59 @@ words, it is not the expectation that augmentations should duplicate the original documentation comments, but instead provide comments that are specific to the augmentation. -## Deprecating part files - -Part files have been [discouraged for many years][discourage]. They are still -fairly often used by code generators because it gives generated code access to -the main library's private namespace. However, it means that the generated part -file cannot have its own imports. - -Library augmentations can do everything part files can do but also support -their own imports and can modify members. With these, we can more strongly -recommend the few users using them migrate to library augmentations. In Dart -4.0, we can consider removing support for part files entirely, which would -simplify the language and our tools. - ## Changelog -## 1.21 +### 1.22 + +* Unify augmentation libraries and parts. + [Parts with imports specification][parts_with_imports.md] moved into + separate document, as a stand-alone feature that is not linked to + augmentations. +* Augmentation declarations can occur in any file, whether a library or part + file. Must occur ”below” the base declaration (later in same file or + sub-part) and “after” any prior applied augmentation that it modifies + (below, or in a later sub-part of a shared ancestor). +* Suggest a stronger ordering *lint*, where the augmentation must be “below” + the augmentation it is applied after. That imples that all declarations with + the same name are on the same path in the library file tree, so that + reordering `part` directives does not change augmentation application order. +* Change the lexical scope of augmenting class-like declarations to only + contain the member declarations that are syntactically inside the same + declaration, rather than collecting all member declarations from all + augmenting or non-augmenting declarations with the same name, and making + them all available in each declaration. +* Avoid defining a syntactic merging, since it requires very careful scope + management, which isn’t necessary if we can just extend properties that are + currently defined for single declarations to the combination of a + declaration plus zero or more augmentations. + +### 1.21 * Add a compile-time errors for wrong usages of `augmented`. -## 1.20 +### 1.20 * Change the `extensionDeclaration` grammar rule such that an augmenting extension declaration cannot have an `on` clause. Adjust other rules accordingly. -## 1.19 +### 1.19 * Change the phrase 'augmentation library' to 'library augmentation', to be consistent with the rename which was done in 1.15. -## 1.18 +### 1.18 * Add a grammar rule for `enumEntry`, thus allowing them to have the keyword `augment`. -## 1.17 +### 1.17 * Introduce compile-time errors about wrong structures in the graph of libraries and augmentation libraries formed by directives like `import` and `import augment` (#3646). -## 1.16 +### 1.16 * Update grammar rules and add support for augmented type declarations of all kinds (class, mixin, extension, extension type, enum, typedef). @@ -1119,31 +1227,31 @@ simplify the language and our tools. (which currently only exist for extension types) can be augmented like other constructors (#3177). -## 1.15 +### 1.15 -* Change `libary augment` to `augment library`. +* Change `library augment` to `augment library`. -## 1.14 +### 1.14 * Change `augment super` to `augmented`. -## 1.13 +### 1.13 * Clarify which clauses are (not) allowed in augmentations of certain declarations. * Allow adding an `extends` clause in augmentations. -## 1.12 +### 1.12 * Update the behavior for variable augmentations. -## 1.11 +### 1.11 * Alter and clarify the semantics around augmenting external declarations. -* Allow non-abstract classes to have implictly abstract members which are +* Allow non-abstract classes to have implicitly abstract members which are implemented in an augmentation. -## 1.10 +### 1.10 * Make `augment` a built-in identifier. @@ -1155,7 +1263,7 @@ simplify the language and our tools. ### 1.8 -* Specify that augmented libraries and thier augmentations must have the same +* Specify that augmented libraries and their augmentations must have the same language version. * Specifically call out that augmentations can add and augment enum values, diff --git a/working/augmentation-libraries/parts_with_imports.md b/working/augmentation-libraries/parts_with_imports.md new file mode 100644 index 000000000..804c55999 --- /dev/null +++ b/working/augmentation-libraries/parts_with_imports.md @@ -0,0 +1,631 @@ +# Part files with imports + +Authors: rnystrom@google.com, jakemac@google.com, lrn@google.com
+Version: 1.0 (See [Changelog](#Changelog) at end) + +This is a stand-along definition of _improved part files_, where the title of +this document/feature is highlighting only the most prominent part of the +feature. This document is extracted and distilled from the [Augmentations][] +specification. The original specification introduced special files for +declaring augmentations, and this document is the unification of those files +with the existing `part` files, generalizing library files, part files and +augmentation files into a consistent and (almost entirely) backwards compatible +extension of the existing part files. + +Because of that, the motivation and design is based on the needs of +meta-programming and augmentations. It’s defined as a stand-alone feature, but +design choices were made based on the augmentations and macro features, +combined with being backwards compatible. + +[Augmentations]: augmentations.md "Augmentations feature specification" + +## Motivation + +Dart libraries are the unit of code reuse. When an API is too large to fit into +a single file, you can usually split it into multiple libraries and then have +one main library export the others. That works well when the functionality in +each file is made of separate top-level declarations. + +There are two cases where that approach does not work optimally. + +If the separate classes are tightly coupled, and interact by accessing private +members of each other, they need to be in the same library. If the library file +becomes too big, the individual classes can be moved into separate *part +files*. Part files can be harder to work with than separate libraries because +they cannot declare their own imports, and all imports for the library must be +in the main library file. That makes it harder to manage and understand +imports, since the code that needs an import may not be in the same file as the +import itself. There have been requests to either loosen the “library privacy” +([#3125][]) or to allow better part files ([#519][]). This feature does not +loosen library privacy, but it improves part files to the point where it may +more tolerable to keep all the classes in the same library in some cases. + +Also, sometimes a single *class* declaration is too large to fit comfortably +in a file. Dart libraries and even part files are no help there. Because of +this, users have asked for something like partial classes in C# ([#252][] 71 👍, +[#678][] 18 👍). C# also supports splitting [the declaration and implementation +of methods into separate files][partial]. Splitting classes, or other +declarations, into separate parts is what the [Augmentations][] feature solves. +The improved part files gives augmentations, and specifically macro generated +augmentations, a structured and capable way to add new code, including new +imports and new exports, to a library. + +Finally, we take this opportunity to disallow the legacy +`part of library.name;` notation ([#2358][]). It won’t work some of the added +features, and the Dart language is moving away from giving libraries names. + +[#252]: https://github.com/dart-lang/language/issues/252 "Partial classes and methods" +[#678]: https://github.com/dart-lang/language/issues/678 "Partial classes" +[partial]: https://github.com/jaredpar/csharplang/blob/partial/proposals/extending-partial-methods.md +[#3125]: https://github.com/dart-lang/language/issues/3125 "Shared library privacy" +[#519]: https://github.com/dart-lang/language/issues/519 "Allow imports in part files" +[#2358]: https://github.com/dart-lang/language/issues/2358 "Disallow part of dotted.name" + +## Background + +In pre-feature code (Dart code before this feature is introduced), a library is +defined by one library file, and a number of part files referenced directly by +the library file using `part` directives like `part 'part_file_name.dart';`, +placed in the header section of the library file after `library`, `import` and +`export` declarations. Each part file must start with a `part of` directive +having one of the forms `part of 'library_file_name.dart';` or +`part of library.name;`, where the library name is the name declared by the +library file using a `library library.name;` directive. The part file +designating its containing library is intended to ensure that a part file can +only ever be part of one library file, which is essential for, for example, +having useful language support when editing the part file. + +The (URI) string version is the most useful for making analysis of a part file +possible and unique, because it uniquely defines the library by the URI that +the language itself uses to identify files and libraries. It’s technically +possible to have two separate libraries with the same declared library name, +which both include the same part file. The language has a restriction against +having two libraries with the same declared name in the same program *mainly* +to avoid this particular issue, but that still makes offline analysis of the +part file a problem. + +Pre-feature part files inherit the entire import scope from the library file. +Each declaration of the library file and each part file is included in the +library’s declaration scope. It’s viable to think of part files as being +textually included in the library file. There is even a rule against +declaring a `part` inclusion of the same file more than once, which matches +perfectly with that way of thinking. + +## Feature + +This feature allows a part file to have `import`, `export` and `part` directives +of its own, where `import` directives only affect the part file itself, and its +transitive part files. A library is defined by the source code of its library +file and *all* transitively included part files, which can be an arbitrarily +deep *tree*. A part file inherits the imported declarations and import prefixes +of all its transitive parent files (the library or part files that included it), +but can choose to ignore or shadow those using its own imports. + +The design goals and principles are: + +* *Backwards compatible*: If a part file has no `import`, `export` or `part` + directive, it works just like it always has. + * Because of that, it’s always safe to move one or more declarations into + a new part file. + * Similarly it’s always possible and safe to combine a part file with no + `import`s back into its parent file. + + _(Augmentations modify both of these properties slightly, because order of + declarations also matter.)_ + +* *Library member declarations are library-global*: All top-level declarations + in the library file and all transitive part files are equal, and are all in + scope in every file. They introduce declarations into the library’s + declaration scope, which is the most significant scope in all files of the + library. If there is any conflict with imported names, top-level + declarations win! + +* *The unit of ownership is the library*. It’s quite possible for one part + file to introduce a conflict with another part file. It always was, but + there are new ways too. If that happens, the library owner, who most likely + introduced the problem, is expected to fix it. There is no attempt to hide + name conflicts between declarations in separate tree-branches of the + library structure. + +* *Import inheritance is a only suggestion*: Aka. other files’ imports cannot + break your code (at least if you’re not depending on them). A part file is + never restricted by the imports it inherits from its parent file. It can + ignore and override all of them with imports of its own. That allows a + file, like a macro generated file, to import all its own dependencies and + be completely self-contained when it comes to imports. _It still needs to + fit into the library and not conflict with existing top-level names. That’s + why a macro should document any non-fresh names it introduces, so a library + using the macro can rename any declarations that would conflict._ + + * Because of that, it’s possible to convert an existing library into a + part file of another library. Since a library is self-contained and + imports all external names that it refers to, making it a part file will + not cause any conflict due to inherited imports. _(Obviously still need + to avoid conflicts with top-level declarations.)_ + * And similarly, if a part file *is* self-contained, it can be converted + into a separate library and imported back into the original library, or + it can be moved to another position in the part tree hierarchy. _(Again + augmentations introduce complications, which is why it’s usually a good + idea to keep all augmentations inside the same part sub-tree). + +### Grammar + +We extend the grammar of part files to allow `import`, `export` and `part` file +directives. We allow `part` files directives to use a configurable URI like the +other two. We restrict the `part of` directive to only allow the string version. + +```ebnf +-- Changed "" to "". + ::= `part' `;' + +-- Removed "" as option, retaining only "". + ::= `part' `of' `;' + +-- Added "* *" + ::= + * * ( + )* +``` + +The grammar change is small, mainly adding `import`, `export` and `part` +directives to part files. + +The change to `part of` directives to not allow a dotted name was made because +we want a part file of a part file to refer back to its parent part file, but a +dotted library name can only refer to a library. _That doesn’t mean that +part-of-dotted-name cannot be supported for part files that are part files of a +library file. It’s also that the Dart team wants to remove the feature, and has +been linting against its use for quite a while already. Dotted names in part-of +being partially incompatible with the new feature just means that now is a good +opportunity to get rid of them._ + +The change to a configurable URI for `part` files was made because it can ease +one of the shortcomings of using libraries for platform-dependent code: That +other libraries cannot provide implementations for private members, or code +that accesses private members, without duplicating the entire library. With +part files having their own imports, adding configurable URIs for `part` +directives gives a way to avoid that code duplication, possibly even more +conveniently if also using augmentations. + +The configurable URI for a `part` works just as for imports and exports, it +chooses the URI that the `part` directive refers to, and after that the +included file works just as any other part file. + +It’s a **compile-time error** if a Dart (parent) file with URI *P* has a `part` +directive with a URI *U*, and the source content for the URI *U* does not parse +as a ``, or if its leading ``'s `` string, +resolved as a URI reference against the URI *U*, does not denote the library of +*P*. _That is, if a Dart file has a part directive, its target must be a part +file whose “part of” directive points back to the first Dart file. Nothing new, +except that now the parent file may not be a library file.)_ + +### Resolution and scopes (part and import directives) + +A pre-feature library defines a *top-level scope* extending the import scope +(all declarations imported by non-prefixed import directives) with a +declaration scope containing all top-level declarations of the library file and +all part files, and all import prefixes declared by the library file. The +import prefixes are added to the same scope as library declarations, and there +is a name conflict if a top-level declaration has the same base name as an +import prefix. + +This feature splits the top-level declaration scope from the import prefix +scope to allow a part file to override the import prefix, but not the top-level +declaration. + +Each Dart file (library file or part file) defines a _combined import scope_ +which combines the combined import scope of its parent file with its own +imports and import prefixes. The combined import scope of a dart files is +defined as: + +* Let *C* be the combined import scope of the parent file, or an empty scope + if the current file is a library file. +* Let *I* be a scope containing all the imported declarations of all + non-prefixed `import` directives of the current file. The parent scope of + *I* is *C*. + * The import scope are computed the same way as for a pre-feature + library. The implicit import of `dart:core` only applies to the + library file. _As usual, it’s a compile-time error if any `import`‘s + target URI does not resolve to a valid Dart library file._ + * Let’s introduce *importsOf*(*S*), where *S* is a set of `import` + directives from a single Dart file, to refer to that computation, which + introduces a scope containing the declarations introduced by all the + `import` s (the declarations of the export scope of each imported + library, minus those hidden by a `show` or `hide` operator, combined + such that a name conflicts of different declarations is not an error, + but the name is marked as conflicted in the scope, and then referencing + it is an error.) +* Let *P* be a *prefix scope* containing all the import prefixes declared by + the current file. The parent scope of *P* is *I*. + * The *P* scope contains an entry for each name where the current file + has an `import` directive with that name as prefix, `as name`. (If an + import is `deferred`, it’s a compile-time error if more than one + `import` directive in the same file has that prefix name, as usual. + _It’s not an error if two import deferred prefixes have the same name + if they occur in different files, other file’s imports are only + suggestions._) + * The *P* scope binds each such name to a *prefix import scope*, + *P**name*, computed as *importsOf*(*S**name*) + where *S**name* is the set of import directives with that + prefix name. + * If an import is `deferred`, its *P**name* is a *deferred + scope* which has an extra `loadLibrary` member added, as usual, and the + import has an implicit `hide loadLibrary` modifier. + * If *P**name* is not `deferred`, and the parent scope in *C* + has a non-deferred prefix import scope with the same name, + *C**name*, then the parent scope of *P**name* is + *C**name*. _A part file can use the same prefix as a prefix + that it inherits, because inherited imports are only suggestions. If it + adds to that import scope, by importing into it, that can shadow + existing declarations, just like in the top-level declaration scope. A + deferred prefix import scope cannot be extended, and cannot extend + another prefix scope, deferred prefix scopes are always linked to a + single import directive._ + * _It’s possible to look further up in the import chain *C* for a prefix + scope to extend. Here it’s chosen that that importing parent file gets + to decide which names the part file has access to. If it wants to make + a transitive parent import prefix available, it should just not shadow + it._ + +That is: The combined import scope of a Dart file is a chain of the combined +import scopes of the file and its parent files, each step adding two scopes: +The (unnamed, top-level) import scope of the unprefixed imports and the prefix +scope with prefixed imports, each shadowing names further up in the chain. + +The *top-level scope* of a Dart file is a library *declaration scope* +containing every top-level library member declaration in every library or part +file of the library. The parent scope of the top-level scope of a Dart file is +the combined import scope of that Dart file. _Each Dart file has its own copy +of the library declaration scope, all containing the same declarations, because +the declaration scopes of different files have different parent scopes._ + +**It’s a compile-time error ** if any file declares an import prefix with the +same base name as a top-level declaration of the library. + +_We have split the prefixes out of the top-level scope, but we maintain that +they must not have the same names anyway. Any prefix that has the same name as +a top-level declaration of the library is impossible to reference, because the +library declaration scope always precedes the prefix scope in any scope chain +lookup. This does mean that adding a top-level declaration in one part file may +conflict with a prefix name in another part file in a completely different +branch of the library file tree. That is not a conflict with the “other file’s +imports cannot break your code” principle, rather the error is in the file +declaring the prefix. Other files’ top-level declarations can totally break +your code. Top-level declarations are global and the unit of ownership is the +library, so the library author should fix the conflict by renaming the prefix. +That such a name conflict is a compile-time error, makes it much easier to +detect if it happens._ + +### Export directives + +Any Dart file can contain an `export` directive. It makes no difference which +file an `export` is in, its declarations (filtered by any `hide` or `show` +modifiers) are added to the library’s single export scope, along with those of +any other `export`s in the library and the non-private declarations of the +library itself. Conflicts are handled as usual (as an error if it’s not the +*same* declaration). + +Allowing a part file to have its own export is mainly for consistency. +Most libraries will likely keep all `export` directives in the library file. + +## Terminology + +With libraries now being trees of files, not just a single level of parts, we +introduce terminology to concisely express relation of files. (Some of this has +already been defined above, but is also included here for completeness.) + +A *Dart file* is a file containing valid Dart source code, +and is identified by its URI. + +A Dart file is either a *library file* or a *part file*, +each having its own grammar. + +* _(With this feature those grammars differ only at the very top, + where a library file can have an optional script tag and an optional + `library` declaration, and a part file must start with a `part of` + declaration)_. + +We say that a Dart file *includes* a part file, or that the part file +_is included by_ a Dart file, if the Dart file has a `part` directive with a +URI denoting that part file. + +* _It’s a compile-time error if a Dart file has two `part` directives with + the same URI, so each included part file is included exactly once._ +* _It’s a compile-time error if a `part` directive denotes a file which is + not a part file._ + +The *parent file* of a part file is the file denoted by the URI of the +`part of` declaration of the part file. A library file has no parent file. + +* _It’s a compile-time error if a part file is included by any Dart file + other than the part file’s parent file._ +* The *includes* and *is the parent file of* properties are equivalent for + the files of a Dart program. A Dart file includes a part file if, and only + if, the Dart file is the parent file of the part file, otherwise there is a + compile-time error. (There are no restrictions on the parent file of a part + file which is not part of a library of a Dart program. Dart semantics is + only assigned to entire libraries and programs, not individual part files. + We’ll refer to the relation as both one file being included by another, and + the former file being the parent (file) of the latter file. + +Two or more part files are called *sibling part files* (or just +*sibling parts*) if if they are all included by the same (parent) file. + +A file is a *sub-part* of an *ancestor* Dart file, if the file is included by +the Dart file, or if the file is a *sub-part* of a file included by the +*ancestor* Dart file. + +* This “sub-part/ancestor” relation is the transitive closure of the + *included by* relation. We’ll refer to it by saying either that one Dart + file is an ancestor file of another part file, or that a part file is a + sub-part of another Dart file. +* _It’s a compile-time error if a part file is a sub-part of itself._ + That is, if the *includes* relation has a cycle. This is not a *necessary* + error from the language's perspective, since no library can contain such a + part file without introducing another error; at the first `part` directive + which includes a file from the cycle, the including file is not the parent + of that file. + The rule is included to as a help to tools that try to analyzer Dart code + starting at individual files, they can then assume that either a part file + has an ancestor which is a library file, or there is a compile-time error. + Or an infinite number of part files.) + +The *sub-tree* of a Dart file is the set of files containing the file itself +and all its sub-parts. The *root* of a sub-tree is the Dart file that all other +files in the tree are in the sub-parts of. + +We say that a Dart file *contains* another Dart file if the latter file is in +the sub-tree of the former (short for “the sub-tree set of the one file +contains the other file”). + +The *least containing sub-tree* or *least containing file* of a number of +Dart files from the same library, is the smallest sub-tree of the library +which contains all the files, or the root file of that sub-tree. +Here _least_ is by set inclusion, because any other sub-tree that contains the +two files also contains the entire smallest sub-tree. _A tree always has a +least containing sub-tree for any set of nodes._ + +* The least containing file of *two* distinct files is either one of those + two files, or the two files are contained in two *distinct* included part + files. The least containing file is the only file which contains *both* + files, and not in the *same* included file. +* Generally, the least containing file of any number of files +* is the *only* file which contains all the files, and which does not contain + them all in one sub-part. + _(If a file contains all the original files, then either they are in the + same included part file, and then that part file is a lesser containing + file, or not all are in the same included part file, so either in different + included parts or some in the file itself, and then no included part file + contains all the files, so there is no lesser containing file.)_ + +The *files of a library* is the entire sub-tree of the defining library file. +It contains the library file. +The sub-tree of a part file of a library contains no library file. + +In short: + +* A *parent* file *includes* a part file. Adding transitivity and reflexivity, + an *ancestor* file *contains* any *sub-part* file, and itself. +* A Dart file defines a *sub-tree* containing itself as *root* + and all the *sub-trees* of all the part files it *includes*. + As a tree, it trivially defines a partial ordering of files with a least + upper bound, which is the _least containing file_. + +## Language versioning and tooling + +Dart language versioning is an extra-linguistic feature which allows the SDK +tooling to compile programs containing libraries written for different versions +of the language. As such, the language semantics (usually) do not refer to +language versions at all. + +Similarly the language has no notion of a “package”, but tooling does consider +Dart files to belong to (at most) one package, and some of the files as +“having a `package:` URI”. This information is written into a metadata file, +`package_config.json` that is also used to resolve `package:` URIs, and to +assign default language versions to files that belong to a package. + +Because of that, the restrictions in this section are not *language* rules, +instead they are restrictions enforced by the *tooling* in order to allow +multi-language-version programs to be compiled. + +### Pre-feature code interaction and migration + +This feature is language versioned, so no existing code is affected at launch. + +The only non-backwards compatible change is to disallow `part of dotted.name;`. +That use has been discouraged by the +[`use_string_in_part_of_directives`][string_part_of_lint] lint, which was +introduced with Dart 2.19 in January 2023, and has been part of the official +“core” Dart lints since June 2023. The “core” lints are enforced more strongly +than the “recommended” lints, including counting against Pub score, so any +published code has had incentive to satisfy the lint. + +The lint has a quick-fix, so migration can be achieved by enabling the lint +(directly, or by including the “recommended” or “core” lint sets, which is +already itself recommended practice) and running `dart fix`, which will change +any `part of dotted.name;` to the future-safer `part of 'parent_file.dart';`. + +All in all, there is very little expected migration since all actively +developed code, which is expected to use and follow recommended or core lints, +will already be compatible. + +[string_part_of_lint]: https://dart.dev/tools/linter-rules/use_string_in_part_of_directives "use_string_in_part_of_directives lint" + +We will enforce a set of rules that weren’t as clearly defined before +(see next section), and therefore maybe not strictly enforced, so there is a +risk that some pathologically designed library may break one of those rules. +Other than that, a pre-feature library can be used as post-feature library as +long as it satisfies these very reasonable rules. + +The feature has no effect at the library boundary level, meaning the export +scope of a library, so pre-feature and post-feature libraries can safely +coexist. A library can start using the feature without any effect on client +libraries. There is no need to worry about migration order. + +### Explicit sanity rules + +The following rules are rules enforced by tooling, not the language, since they +rely on features that are not part of the language (files having a language +version, a language version marker, or belonging to a package). + +All pre-feature libraries should already be following these rules, which exist +mainly ensure that different files of a library will *always* have the same +language versions. _Some of these rules have not all been expressed explicitly +before, because they are considered blindingly obvious. We’re making them +explicit here, and will enforce the rules strictly for post-feature code, if we +didn’t already._ + +* It’s a **compile-time error** if two Dart files of a library do not have the + same language version._All Dart files in a library must have the same + language version._ Can be expressed locally as: + * It’s a compile-time error if the associated language version of a part + file is not the same as the language version of its parent file. + +* It’s a **compile-time error** if any file of a library has a + language-version override marker (a line like `// @dart=3.12` before any + Dart code), and any *other* file of the same library does not have a + language-version override marker. _While it’s still possible for that + library to currently have the same language version across all files, that + won’t stay true if the default language version for the package changes._ + Can be expressed locally as: + + * If a part file has a language version marker, then it’s a compile-time + error if its parent files does not have a language version marker. _The + version marker it has must be for the same version due to the previous + rule._ + + * If a part file has no language version marker, then it’s a compile-time + error if its parent file has a language version marker. + +* It’s a **compile-time error** if two Dart files of a library do not belong + to the same package. _Every file in a library must belong to the same + package to ensure that they always have the same default language version. + It’s also likely to break a lot of assumptions if they don’t._ Can be + expressed locally as: + + * It’s a compile-time error if a part file does not belong to the same + package as its parent file. + + The Dart SDK’s multi-language-version support, which based on files + belonging to packages, will not support libraries that are not entirely in a + single package. + +* We *may* want to also make it a **compile-time** error if two Dart files of + a library are not both inside the `lib/` directory or both outside of it. + _Having a parent file inside `lib/` with a part file outside will not + compile if the parent file is accessed using a `package:` URI. Having a + parent file outside of `lib/` with a part file inside works, but the part + file might as well be outside since the only way to use it is to go through + the parent file._ The only reason to maybe not enforce this rule would be a + file inside `lib/` that is *never* accessed using a`package:` URI, and which + depends on files outside of `lib/` for something. If some frameworks do + that, maybe a Flutter `main` file, then we should just keep giving warnings + about the pattern. The `lib/` directory should be self-contained because all + libraries in it can be accessed by other packages, and no other files can. + +### User guidance tooling + +The analyzer and analysis server needs to support and understand the new +feature. No new user-facing features are needed, but some error handling +and user guidance may be useful. The following are ideas for +hypothetical features that the tool may choose to add. + +##### Annotations applying to sub-tree + +Usually an annotation placed on the `library` declaration applies to the entire +library. There are no annotations that are defined as applying only to a single +file, or to be placed on `part of ` declarations. + +It might be useful to have some annotations that apply either to an entire +sub-tree or to a single file. The individual annotations should decide how they +can be applied. + +It may very well be that annotations affecting declarations (which is typically +what annotations on library declarations do) have no benefit from being limited +based on something as (so far) semantically arbitrary as source ordering. But on +the other hand, users may choose to order source depending on properties that +annotations apply to. _The analyzer may want to review annotations that apply to +a library for whether they can reasonably apply to any sub-tree of parts. For +example `@Deprecated(…)` could apply to every member in a sub-tree, allowing a +library to keep its deprecated API, and its necessary imports, separate from the +rest, so that it can all be removed as a single operation, and then marking all +that API as deprecated with one annotation._ + +##### An `// ignore` applying to a sub-tree + +The analyzer recognizes `// ignore: …` comments as applying to the same or next +line. For ignoring multiple warnings, there is a `// ignore_for_file: …` comment +which covers the entire file. There is no `ignore_for_library` that would apply +to the entire library, including parts. + +It can be considered whether to have an `// ignore_for_all_files: …` (or a +better name) which applies to an entire sub-tree, not just the current file, and +not the entire library. It would apply to the entire library if applied to the +library file. + +It may very well be better to *not* that, and have each sub-part write its own +`// ignore_for_file: ...`. That makes it very easy to see which ignores are in +effect for a file. + +##### Invalid part file structure correction + +When analyzing an incomplete or invalid Dart program, any and all of the +compile-time errors above may apply. + +It’s possible to have part files with parent-file cycles, part files with a +parent URI which doesn’t denote any existing file, or files with a `part` +directive with a URI that doesn’t denote any existing file. This isn’t *new* to +enhanced part files, other than the cycle where it used to immediately be an +error if the parent file wasn’t a library file. + +If a tool can see that one Dart file includes a part file, and the part file +has a non-existing file URI as its parent file, it could be a quick-fix to +update the URI in the part file’s `part of` directive to point to the file that +includes it. + +Similarly if a part file’s parent file doesn’t include the part file, then a +`part` directive can be added, or if the parent file has a `part ` directive +which doesn’t point to an existing file (and maybe only if the name is +*similar*), then that part directive can be updated to point to the part file. + +### Migration + +The only non-backwards compatible change is to disallow `part of dotted.name;`. +That use has been discouraged by the +[`use_string_in_part_of_directives`][string_part_of_lint] lint, which was +introduced with Dart 2.19 in January 2023, and has been part of the official +“core” Dart lints since June 2023. The “core” lints are enforced more strongly +than the “recommended” lints, including counting against Pub score, so any +published code has had incentive to satisfy the lint. + +The lint has a quick-fix, so migration can be achieved by enabling the lint +(directly, or by including the “recommended” or “core” lint sets, which is +already itself recommended practice) and running `dart fix`, which will change +any `part of dotted.name;` to the future-safer `part of 'parent_file.dart';`. + +All in all, there is very little expected migration since all actively +developed code, which is expected to use and follow recommended or core lints, +will already be compatible. + +[string_part_of_lint]: https://dart.dev/tools/linter-rules/use_string_in_part_of_directives "use_string_in_part_of_directives lint" + +## Changelog + +### 1.0 + +* Initial version. The corresponding version of [Augmentations], which refers + to part files with imports, is version 1.21. + +* Combines augmentation libraries, libraries and part files into just + libraries and part files, where the part files can have import, export and + further part directives. Those part directives can use configurable imports. +* Is backwards compatible with existing `part` files (other than disallowing + the long-discouraged `part of dotted.name;`). +* Unlike augmentation libraries, improved part files inherit the imports of + their parent file(s). A part file can still choose to ignore that and + import all its own dependencies directly. The feature ensures that + inherited imports cannot get in the way of a part file which wants to do. + +### Augmentations 1.20 + +Original specification which this feature was extracted from.