what world should Tasks run in?

[JeffBezanson]

Currently, we start Task execution in the newest world, like invokelatest. However, we might instead want them to capture the world they were created in. That would potentially enable us to infer the result of fetch(@spawn ...).

[tkf]

capture the world they were created in

Would it make task-local logger inferrable? cc @c42f

[c42f]

Oh really good question. I think the answer is no: even regardless of task spawning, dynamically scoped loggers currently can't be inferred at the use site. If we did want them to be fully inferred that's similar to passing them via contextual dispatch and has somewhat similar tradeoffs: extremely good runtime performance, but a lot more recompilation.

I think there's a middle ground between no specialization and full contextual dispatch... something like "taking dynamically scoped variables seriously": the error handling and logging cases have a similar structure where you may want to pass the context down a deep call stack without specializing the intermediate frames. But then specialize any leaf function calls which actually make use of the dynamic context. Anyway, this is quite different from what Jeff is proposing here.

Regarding the world age of Tasks, I'm definitely in favor of having it inherited from the parent Task. This seems like the correct dynamically scoped behavior that we use everywhere else for world age and is also very natural from a structured concurrency point of view.

[tkf]

However, we might instead want them to capture the world they were created in.

Another advantage with this approach is that this lets the chain/tree of calls started from Base.invoke_in_world to use a consistent world age even in child tasks.

Demo (from #35844 (comment)):

foo() = "First implementation of foo"
function bar()
    Base.invoke_in_world(_entry_point_world, foo)
end

afoo() = fetch(@async foo())
function abar()
    Base.invoke_in_world(_entry_point_world, afoo)
end

# Note `@async` creates a closure so the counter has to be fetched after that.
_entry_point_world = Base.get_world_counter()

@show bar()
@show abar()

println("Now, revise foo:")
foo() = "Second implementation of foo"

@show bar()
@show abar()

prints

bar() = "First implementation of foo"
abar() = "First implementation of foo"
Now, revise foo:
bar() = "First implementation of foo"
abar() = "Second implementation of foo"

Ideally, the last abar() should still return "First implementation of foo".

[StefanKarpinski]

This seems like a pretty compelling “minor change” ie technically breaking but probably undisruptive enough to be done in a minor release.

[tkf]

That would potentially enable us to infer the result of fetch(@spawn ...).

Here is my very hacky POC for inferable fetch(task): https://github.com/tkf/InferableTasks.jl/blob/master/src/InferableTasks.jl

Discussion: https://discourse.julialang.org/t/can-we-have-inferable-fetch-task/45541

[c42f]

How should we initialize the world age for a task? It's really easy to initialize from the parent ptls->world_age in jl_new_task, but this seems to have fairly loose semantics.

In particular, it would be insufficient to make fetch(some_task) inferrable, as some_task could be created in an unrelated context and passed into the current task.

Instead it seems we'd need a composite @fetch_spawn(...), which guarantees that the task which is fetch()ed from was created in the local context.

[jtrakk]

This kind of issue makes me wonder if world-age could be more like a partial order or lattice rather than an integer. The parent task defines f() = 0, then split and Task 1 defines f() = 1 while Task 2 defines f() = 2. Both children have higher age than the parent task but neither child is older than the other (they are twins).