Interactive Design Meeting Notes - 7/8/15

[kuhlenh]

Interactive Design Meeting 7/8/15

Scripting Dialect Review with C# LDM Committee

Design Team

The following individuals are members of the dream team:
Anthony D. Green
Chuck Stoner
Dustin Campbell
Kasey Uhlenhuth
Kevin Halverson
Kevin Pilch-Bisson
Lincoln Atkinson
Manish Jayaswal
Matt Warren
Tomas Matousek

Special Guests
Andrew Casey
Immo Landwerth
Jared Parsons
Mads Torgersen
Stephen Toub

---

AGENDA

The purpose of this meeting was to go over our C# scripting dialect design with the C# language design team. This would give us a chance to double-check our design as well as give them an opportunity to see what could potentially be incorporated into C# proper in the future.

Quotes of the Day

"No one likes hoops"..."Except basketball players"
"Scripting isn't about thinking!"

Basic Principles of Scripts

• We stitch together all top-level code and all code included via #load in a single script class at compile time.
• #r, #r!, and #load directives are only allowed at the start of the file. Other compiler directives (e.g., #pragma, #line, #if) are allowed anywhere.
• Scripts have three different types of variables:
  • static script variables (can't have an initializer with an await expression)
  • script variables, aka top-level variables available to entire script (can have await expressions in an initializer. Script will stop executing until task gets returned)
  • "real" locals (in a block)
• Design should follow the Copy/Paste Principle of Scripting (CPPoS): if you copy/paste from statement body or class declaration it should work the same way in a script--within reason. There are some cases that just don’t make sense, but they should be rare –- e.g., abstract.

Design Review

1. Top-level script methods vs local functions (should the feature make it into C# 7.0)
   There is a difference between top-level methods and local functions and scripts will support both. Top-level methods are not local functions. Scripting is supposed to be a lightweight, rapid coding experience. Script authors probably won't care if their top-level variables are technically fields.

   Confused? Read about local functions, the potential C# 7.0 feature, in the C# LDM Notes.

2. Async vs sync
   All top-level code is async in a script. As an optimization, the compiler can avoid emitting state machines when there are no await expressions used in the top-level code--but it has to preserve exception semantics. Is it a shame that there will no longer be a synchronous way of running scripts? Perhaps, but we think that having two entry-points for the sake of a simple scenario (e.g., the 1+1 script) isn't compelling enough. We planned on having the Interactive Window (aka REPL) be async, so having scripts be synchronous would violate the CPPoS.

3. Static top-level members. If allowed, await has to be disallowed in static field initializers.
   C# static fields and functions at the top-level are allowed. These will be processed when first loaded. await is allowed in script variable initializers but not in static script variable initializers. VB Static local variables are not allowed at top-level (cannot use static outside of a method). Static fields are the only way to compute something once and for all, no matter how many times the host re-evaluates the same script. Furthermore, one of the main complaints/scenarios with today's Immediate Window is that people want to copy/paste from APIs/libraries--meaning that without static top-level members developers would have to manually delete all statics from pasted code (i.e., a violation of the CPPoS).

   Await cannot be used in static script variable initializers but can be used in script variable initializers (e.g., var x = await Method()). Static readonly is allowed at the top-level.

4. Visibility of top-level methods, properties, types, script variables. Interaction with #load and #load into.
   The default should be that things in a script should be seen by the script that loads it. There should be a concept of script private (whether this means using braces or the private keyword). By default, top-level members and script variables declared in a script will be either public or internal (still discussing). Ultimately, we decided that the average script writer isn't going to be thinking about what variables he/she wants to expose to the public or not when initially writing a script and for the sake of rapid prototyping and experimentation scripts should be public/internal by default (if scripts were private by default aka file-private, script writers would have to remember to make things public that they want to use in other scripts).

   There should be some concept of "script private" so that a script writer may prevent another script from accessing variables/methods. Currently, you can just stick what you want to be private into braces but maybe we should have some sort of concept of "file private" to better represent this. We are leaning towards changing the meaning of the private keyword to mean "file private" for top-level script members.

   #load vs #load into
   Recall that #load essentially copy/pastes the content from the loaded script at the top of the new script. Recall that #load myscript.csx into MyVariable is a directive that would help prevent name-clashes between variables and methods in loaded scripts. The MyVariable variable is just a qualifier for all members of the loaded script. You can think of it as an aliasing mechanism of sorts (codegen that scrambles the names) that is used to avoid polluting the top-level space in a script (and make IntelliSense better). No new objects or instances are created with the qualifying MyVariable, and it is more like a using alias or a namespace (the path is the same).

[daveaglick]

If I'm reading this correctly, my understanding is that all scripts will be asynchronous going forward - is that correct? I understand why this is being considered, and it does make sense in the context of a REPL or other asynchronous script host.

However, there are probably lots of other hosts that wouldn't want to mess with asynchronous code. For example, the simple business rules engine that just needs to load and evaluate a script as configuration at startup. Or the old WinForms code that needs to add scripting for layout. The way that async/await tends to "bubble up" through your code for proper support makes it a pain to consume async methods from synchronous code. I'm also not sure that "a simple scenario (e.g., the 1+1 script)" isn't going to be just as, if not more important, than the more complex scenarios being envisioned. Personally, I've found lots of uses for simple "1+1" style scripts.

Granted, if you want to consume an asynchronous script from synchronous code you could always use a helper method or wrapper (like the AsyncHelper from ASP.NET 5), but that seems messy for something that is supposed to make working with dynamic code easier than firing up the full Roslyn compiler. I don't know what the answer is (or if there needs to be one) and I sympathize with not wanting to code up two parallel scripting environments, one for sync and one for async. What I do know is that this change in thinking from the current scripting engine is going to be mostly harmful for this specific developer.

[paulomorgado]

I'm just speculating that the script engine is asynchronous even if the script itself isn't.

And that also helps allowing asynchronous scripts.

Synchronous is just an illusion. You might as well embrace asynchronous.

[tmat]

Executing synchronously is gonna be simple:

int x = CSharpScript.EvalAsync<int>("x + y").Result;

[daveaglick]

Ah, great - so it'll just return a normal Task? Will there be an option to implicitly add ConfigureAwait(false) to the script runner so that there's less chance of deadlocking on the context?

[mattwar]

If you write

var result = Task.Run(() => CSharpScript.EvalAsync(script)).Result

Then the script should never need to specify ConfigureAwait(false) since its starting context is the thread pool.

note: but don't ever do this if you are already running out of the thread pool. How do you know? IDK.

[GSPP]

All top-level code is async in a script.

What does that mean? Does that have limit the kinds of code that one can write in a script?

Async does not seem particularly important for a script because there won't be many scripts running in any given CLR process. Potentially blocking threads is fine and has no performance impact.

A script that really wants to be async could return a Task. Then the host could say:

CSharpScript.Eval<Task>("new WebClient().DownloadTaskAsync(\"...\"));

And do whatever it likes with that task.

[paulomorgado]

An asynchronous script engine can handle synchronous scripts better than a synchronous engine can handle asynchronous scripts.

[GSPP]

@paulomorgado I don't know what you mean. I did not mean to suggest that the user experience must be synchronous or asynchronous. I did not intend to make any statement about how a REPL loop would work. If your comment was meant to answer my question I did not follow.

Script execution will always need to be initiated on some kind of background thread. Be it Task-based or not.

[MgSam]

Any chance of a CTP of the Interactive Window coming soon? We've been without C# Interactive since 2012. That's a long time to wait.

[kuhlenh]

@MgSam That's what we're hoping to do ASAP. The CTP would contain a subset of our 1.1 feature plans.