Support Dart_TypedDataAcquireData of Unmodifiable TypedData classes

[zanderso]

It is useful for embedders to be able to expose read-only memory regions to Dart code as external type data. This will let Flutter avoid expensive copies on the Dart thread of large amounts of image data, for example see here:

https://github.com/flutter/engine/blob/18fee597ab6ec957e6311ba2bc74d4732ad55bd7/lib/ui/painting/image_encoding.cc#L48

for Image.toByteData:

https://github.com/flutter/engine/blob/18fee597ab6ec957e6311ba2bc74d4732ad55bd7/lib/ui/painting.dart#L1610

(The caller of that API could then do a copy to a writable buffer if they need one.)

The best match for this in the Dart SDK appears to be the unmodifiable typed data view classes like UnmodifiableByteDataView. Unfortunately, when these buffers come back into the embedder for various reasons, a call to Dart_TypedDataAcquireData() will fail by returning an error because the Unmodifiable typed data classes don't pass the tests here:

sdk/runtime/vm/dart_api_impl.cc

Line 4065 in f66eb72

if (!IsExternalTypedDataClassId(class_id) &&

.

This issue is a feature request to integrate the Unmodifiable typed data classes more seamlessly with the VM so that they're a good match for the read-only buffers we can expose from Flutter as an optimization. If there's another approach that would solve the problem, then I'm interested in that, too.

/cc @a-siva @rmacnak-google @dnfield

[a-siva]

Since the embedder is creating External TypedData objects it should have direct access to the memory region, why does it need to call Dart_TypedDataAcquireData to access underlying buffer.

[rmacnak-google]

With the existing API, it is possible to extract the underlying buffer with Dart_GetField and pass that to Dart_TypedDataAcquireData. Though it does seem reasonable to expect Dart_TypedDataAcquireData to handle all the UnmodifiableX that are defined in dart:typed_data.

[zanderso]

Since the embedder is creating External TypedData objects it should have direct access to the memory region, why does it need to call Dart_TypedDataAcquireData to access underlying buffer.

The embedder can generate an external typed data object, and return it to Dart code, then that object can come back into the embedder through a Dart language API. A slightly contrived example would be the return from Image.toByteData() being passed back into Image.memory().

With the existing API, it is possible to extract the underlying buffer with Dart_GetField()

Yeah, it looks like a vm:entry-point annotation would be needed. I would also worry about the implications of reaching into the core libraries and grabbing a private field.

[rmacnak-google]

@mkustermann What do you think about giving the unmodifiable views fixed CIDs like the regular typed data views?

[mkustermann]

@mkustermann What do you think about giving the unmodifiable views fixed CIDs like the regular typed data views?

Right now I can discourage the usage of the unmodifiable views for the following reason:

The canonical representation of bytes in Dart code is Uint8List. All our Dart code uses Uint8List to access bytes (e.g. protobuf encoder/decoder, http parser, ...). We want to make reading / writing of bytes as fast as possible by hoisting bounds checks out of loops and directly accessing the bytes (i.e. no method calls).

Optimized access: Our AOT compiler does perform those optimizations - accessing Uint8List is around two machine instructions - as long as there are no 3rd party implementations of the Uint8List interface (**). We can do that due to our sound type system. (The JIT has dynamic type feedback and doesn't need to rely on this.)

3rd party implementations: The UnmodifiableUint8ListView class is right now tree-shakable. If Dart code does not use it we get the benefits mentioned above. If Dart code uses it, these optimizations are disabled in AOT. Call sites become polymorphic, any code working with bytes becomes significantly slower.

Short term option: We can turn those unmodifiable view classes to be VM-supported and use our unified typed data layout. That would allow us to have fast read access. BUT: The class would no longer be tree shakable and the Uint8List.[]= write access would always be polymorphic and suffer quite bad performance. To alleviate this bad performance, we could change our AOT optimization to specialize the code conditionally, i.e. if (ClassID.get(obj) is unmodifiable-view) { Call []= method; } else { store bytes directly }. Though it would still be slower and bigger.

Longer term option: I have talked to members of the language/library team about this over the last few months and suggested we could solve it via: a) Introduce a new ReadOnlyUint8List interface that does not implement the writable Uint8List interface. b) add VM-supported read-only class (which flutter could e.g. use) c) disallow 3rd party implementations (similar to int) d) migrate existing code to only . This would allows fast read access, but also fast write access). Though it might require re-writing a lot of library code to use ReadOnlyUint8List instead of Uint8List.

=> If we do the optimizations mentioned in the short term option above and take the performance hit for write-access we could go ahead. @sstrickl or @askeksa-google could take a look at this.

(**) Our internal implementations are _Uint8List, _Uint8ArrayView, _ExternalUint8Array. They have a unified layout that allows accessing bytes directly.

[dnfield]

We did just introduce an ImmutableBuffer class for dart:ui. Perhaps that would help here?

[zanderso]

Thanks for looking into this @mkustermann!

@dnfield Currently, the dart:ui ImmutableBuffer is neither readable nor writable ;-). To make it readable, a reference to the buffer would be cached in a field, and the type for that field that seems like the best fit would be an Unmodifiable typed data view. Alternately, if Unmodifiable typed data views had the right properties, we could investigate using them instead of introducing a new class for a somewhat similar purpose.

[dnfield]

It would probably help to understand the use cases we'd want to support with this.

If the goal is to avoid unnecessary copies, we'd want to know what we're eventually copying to and from - for example, if it's a file or a socket, users may not ever need to get access to the bytes directly in Dart, right?

If they need to do comparisons, we could probably have interfaces that boil down to doing a memcmp against a Uint8List after we acquire the typed data.

But if Dart could handle Unmodifiable more efficiently and avoid copies that would be a good thing, since you could then more easily compose Dart based APIs for this.

[jonahwilliams]

Another use case is in flutter/engine#21153: we currently have to copy the memory mapped bytes of all assets (potentially quite large!) in order to pass them back through the engine. If we could directly convert them to read only buffers, we'd be able to avoid tons of work

[mkustermann]

To speed up the discussion on this, I've written a proposal (see go/performant-typeddata-in-dart). If we follow this path we would ensure to have always fast access to bytes while safely allowing immutable bytes as well as sharing of such across isolates.