Efficient communication of data between isolates

[leafpetersen]

A common pattern in flutter apps is to offload computationally intensive work (e.g. json decoding) to a separate isolate to avoid blocking the main UI thread. This breaks down when the isolate needs to return large amounts of data to the UI thread, since just the time to transfer the data back to the main isolate can block the main isolate for too long.

[leafpetersen]

cc @mraleph @jacob314

[jmesserly]

This would also be a huge help to the IDE too. Often they're written to take advantage of parallelism, so they can respond quickly to some requests (e.g. autocomplete) while taking more time to respond to others.

(C#'s IDE for example is designed to take advantage of .NET's Task Parallel Library)

[mraleph]

/cc @aam who is investigating transferrables.

[ramsestom]

I think that the dart-lang team should focus on providing a real threads support, with shared memory, for Dart rather than trying to improve the communication between Isolates which would just be a sticking-plaster solution (with many limitations like the immutability of shared objects between Isolates). Isolates should simply be used for isolated tasks, as their name suggests, requiring little or no data transfer between processes. For the rest Dart should offer the possibility to use real threads, like any modern programming language worthy of the name

[ping996]

I send message to the other isolate from the main isolate takes almost 8ms, it takes too long time of port communication.
2019-12-12 14:38:03.707 7699-7718/com.example.flutter_app I/flutter: >>>>sendMessage action: EXECUTE_JS 2019-12-12 14:38:03.717 7699-7730/com.example.flutter_app I/flutter: receive message from main.......

[modulovalue]

Is anybody working on this issue?

--

I've provided a rudimentary benchmark to measure skipped frames due to inefficient communication between isolates in dart-lang/sdk#40653

This is the result of a single run (json encoding and decoding):

13985982 bytes, please wait ~10 seconds.
 • Isolate Decode
   - Took          : 196
   - SkippedFrames : 152     (Frame 241 to 393)

 • Isolate Encode
   - Took          : 187
   - SkippedFrames : 6     (Frame 482 to 488)

 • Decode
   - Took          : 47
   - SkippedFrames : 47     (Frame 488 to 535)

 • Encode
   - Took          : 29
   - SkippedFrames : 30     (Frame 535 to 565)

This shows that it's impossible to have a smooth UI when a big json string has to be decoded and consumed entirely by the main isolate.

[mraleph]

We are actively working on implementing improvements to the concurrency story in the VM. The design space is vast, so we are focused mainly on implementing underlying infrastructure. One of the first approaches we are going to try, once the infrastructure is fully in place, is "send-and-exit" approach (tracked by dart-lang/sdk#37835).

[mkustermann]

The core of this issue seems to be

This breaks down when the isolate needs to return large amounts of data to the UI thread, since just the time to
transfer the data back to the main isolate can block the main isolate for too long.

This is now addressed with a newly exposed Isolate.exit() API (was recently added as part of dart-lang/sdk#47164) that can be used to hand ownership of large amounts of data to another isolate without making the receiver having a O(n) cost. There's a few follow-up items in dart-lang/sdk#37835 though the main functionality is implemented now.

Even when not using Isolate.exit() but rather send data using SendPort.send() one will transitively copy the mutable part of the message, though this happens on the sender side. So even in this case, receiving - in normal circumstances - is only a fixed O(1) cost.

What does remain is that isolates coordinate on garbage collections - so certain pauses due to GC are shared across isolates. Though even single-isolate scenarios pay this pause cost.

Closing this issue since most planned work has been implemented as part of dart-lang/sdk#36097

[MadOPcode]

Can't help but ask. What prevents from adding support for (almost) 0-cost transfer between isolates without copying of typed data and Lists, Maps, Sets that contain only primitive types and are not referenced by anything?
It could look something like this:

// Keyword "transient" prevents the object from being referenced
// from inside any other object or from being assigned to any variable
// and forces the object's type parameters for List, Map, etc to be primitives.
transient Uint8List? bytes = Uint8List(1 << 10);
sendPort.send(bytes);
// at this point bytes == null
bytes[0] = 1; // compile-time error

transient List<int>? listA = [0, 1, 2];
// for transient List, Map, etc. only primitive type parameters
// should be allowed as keys/values/elements
transient List<Object>? listB = []; // compile-time error
var listC = listA; // compile-time error
var map = { 'listA': listA }; // compile-time error

sendPort.send(listA);
// listA == null

void modifyList(transient List<int> list) {
  map['listA'] = listA; // compile-time error
  list.add(10); // ok
}

listA = [6, 7, 8, 9];
modifyList(listA); // see above
sendPort.send(listA);
listA.add(10); // compile-time error

[mkustermann]

What prevents from adding support for (almost) 0-cost transfer between isolates without copying of typed data and Lists, Maps, Sets that contain only primitive types and are not referenced by anything?

For 0-cost / sharing of objects, one has to choose between a) mutable b) immutable (and maybe c) transfer).

So far we always avoided the mutable option because it would allow one isolate to see side-effects (mutations of objects of any kinds) from other isolates. This is unexpected for the programmer and usually one needs to provide synchronization/locking primitives to allow a programmer to work safely in such an environment.

So we have chosen the immutable option for now. So transitively immutable objects (e.g. any constant, non-constant strings) are shared. For allowing other data structures (e.g. typed lists as you suggest) to be immutable one has to do more checking at runtime or have type-system guarantees (as you suggest with transient keyword).

The runtime option we have so far avoided due to cost in performance and code-size. The language option we have not explored yet: It is a long process to make language changes and one would want to make it generally applicable (ideally also for composites).