Will Dart 3 - static Metaprogramming make reflectable obsolete?

[dinbtechit]

Hi, I recently heard that in Dart-3 they are planning to introduce Static MetaProgramming wondering if that would make reflectable obsolete?

[Sten435]

Surprise...

They did not 🥲

[dinbtechit]

Interesting.... Curious what makes you say that? Lol 🤔

[Sten435]

Interesting.... Curious what makes you say that? Lol 🤔

Because... dart 3 is out and there is no static m prog yet. And macros are not fully released.

[eernstg]

Static metaprogramming is definitely happening, but there's a lot of work to do before it's mature enough to become a regular language feature. So right now we have just released, as an experimental feature, the ability to use a single macro. More to come.

Reflectable and static metaprogramming are somewhat orthogonal: Reflectable generates code that enables reflective actions at run time whereas static metaprogramming, as indicated by its very name, is completed before the program starts running. This means that there is no way static metaprogramming can emulate the dynamic semantics of reflectable, short of actually reimplementing the code generator of reflectable as a huge macro.

For example, reflectable allows us to do things like receiving a String from a network connection, treating that string as the name of a method, and then invoke that method on a given server object (properly vetting the whole thing for security/privacy/etc, obviously).

On the other hand, static metaprogramming will complete its work at compile time, which means that it will run at full speed at run time, we will never do things like comparing two strings in order to determine whether or not we've found the right method of a given object.

So it's a big difference that one is doing static metaprogramming and the other is doing dynamic metaprogramming (neither of which is "better", they are just good at different things, in addition to a lot of things that they can both do).

Another difference is that the static metaprogramming support in Dart will rely on the statically known scopes to enable operations. In other words, if you're working on a library L1 that imports another library L2 then a macro applied to a declaration in L1 will have access to information from L2 and information provided with the macro itself. This means that static metaprogramming can perform operations on source code entities which are valid for the given set of reachable declarations. In particular, the generated code would be equally correct for any program that contains L1.

In contrast, reflectable performs a closed-world analysis. This means that reflectable code generation needs to be performed with respect to a particular program, and we need to run the reflectable code generator afresh if we're considering a different program, even in the case where the two programs both contain many of the same libraries. In return for this need to run the analysis for a specific program, reflectable supports a number of global properties.

For instance, reflectable allows us to specify that "every subtype of a given type T is handled by this reflector". Static metaprogramming which is based on the available imports will never be able to do that, because we can (ignoring final classes and such) always add yet another subclass S to any given class hierarchy, and the static metaprogramming system wouldn't know anything about the existence of S.

I think they'll both continue to exist, and be useful. You would probably prefer to use static metaprogramming for the cases where that is sufficient, and you'd use reflectable in other situations where a global analysis is required, or in situations where some operations that we need to perform require some run-time information.

[gmpassos]

I like to think about "code compression": defining a basic piece of code or annotation (compressed code) that then automatically unfolds/expands into more detailed and complex code (uncompressed code).

Macros and reflection are two similar but distinct tools that can automate code generation or modify code behavior. They help avoid repetitive coding and enable the creation of frameworks that generalize many behaviors.

• Macros focus on code generation and implementation, allowing for compile-time code transformation.

• Reflection enables runtime code inspection, accessing fields, and calling functions but does not support runtime code generation.

[eernstg]

Makes sense!

There is one relevant distinction in the area of run-time reflection that we might keep in mind, namely 'introspection' vs. 'intercession'. The former is concerned with exploration of the given program, and the latter is concerned with modification of the program. Generation of new code is compatible with statically typed programs (e.g., a Dart program can create the source code of a new program and run it in an isolate, and it's just a run-time event that we can handle if compilation of the new program fails), but more general modification of the currently running program is probably not compatible with static typing (some research languages might have shown heroic efforts in that direction, but it's not going to be easy ;-).

However, I would certainly expect a reflection mechanism to be able to perform actions in the running program, in addition to the ability to investigate its properties (e.g., we should be able to invoke methods and such). This is where reflectable fits in: We can specify that the code generator should describe certain parts of the given program in terms of reflective support structures (maps from strings to methods of a given class, in some sense), and we can then use those structures (using the reflectable API that is based on mirrors) to perform operations like method invocations.

[sigurdm]

I wonder how much of reflectable that could be implemented as a macro? Are macros powerful enough to do the closed-world inspection of the program to create the support data structures for the reflectable runtime?

[eernstg]

If the macro is applied to the entry point (the library that exports the main function) of the program then it should be able to inspect and use everything in the program. I think that should work, but macro re-execution should probably not happen for every key press event. ;-)

[sigurdm]

but macro re-execution should probably not happen for every key press event

Why not? Too expensive?

Ideally the macro expansion should be able to only run for updated parts...

Maybe the dependencies are too hard to track here, because reflectable depends on everything...

[eernstg]

I'm just guessing, but '(potentially) depends on everything' sounds like a good reason. I do think macro execution may need to be under a little bit of user control (as in "took too long to run automatically, will now run on demand").

[sigurdm]

On the other hand - the output of the macro has no/little effect on the static types of anything - if the macro-system could realize this, it would not need to recompile during editing

[eernstg]

Good point! But it is probably not easy for the macro system to be aware of such properties.