Skip to content

API to create deeply immutable typed data, from one or multiple chunks. #50068

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mkustermann opened this issue Sep 28, 2022 · 3 comments
Open
Labels
area-core-library SDK core library issues (core, async, ...); use area-vm or area-web for platform specific libraries.

Comments

@mkustermann
Copy link
Member

mkustermann commented Sep 28, 2022

The VM has support for deeply immutable typed data. Those can be shared by-pointer across isolates. Though right now one can only create them using an embedder API.

Using UnmodifiableUint8ListView(bytes) is not deeply immutable, because bytes may be mutable and someone may write to them.

So I propose to make an API that can make an UnmodifiableUint8ListView from any Uint8List and it may make a copy of it if the underlying bytes are mutable.

Since data is often available as chunks (e.g. loading from network results in List<Uint8List>), the API should support making UnmodifiableUint8ListView from such a set of chunks.

A possible API would be

class UnmodifiableUint8ListView {
    factory UnmodifiableUint8ListView.fromChunks(List<Uint8List> chunks);
}

/cc @lrhn opinions about the API?

@mkustermann mkustermann added the area-core-library SDK core library issues (core, async, ...); use area-vm or area-web for platform specific libraries. label Sep 28, 2022
@lrhn
Copy link
Member

lrhn commented Sep 28, 2022

This API sounds unrelated to being unmodifiable.

We should have a modifiable ByteBuffer and an unmodifiable ByteBuffer, and then (seemingly) use the same typed-data views on top.

Building a buffer from chunks would apply to both kinds of buffers.

So, we can have both Uint8List.fromChunks(Iterable<Uint8List> chunks) and Uint8List.unmodifiableFromChunks(Iterable<Uint8List> chunks)/UnmodifiableUint8ListView.fromChunks.

Or even ByteBuffer.fromChunks(Iterable<...something...> chunks).

(The UnmodifiableThingListView types are mostly bad design. They should just be UnmodifiableThingList.)

@mkustermann
Copy link
Member Author

Building a buffer from chunks would apply to both kinds of buffers.

So, we can have both Uint8List.fromChunks(Iterable<Uint8List> chunks) and
Uint8List.unmodifiableFromChunks(Iterable<Uint8List> chunks)/UnmodifiableUint8ListView.fromChunks.
Or even ByteBuffer.fromChunks(Iterable<...something...> chunks).

LGTM. @lrhn would this be something you'd like to do?

(Side note: Maybe fromList is better for consistency than fromChunks, since we have TransferableTypedData.fromList already?)

(The UnmodifiableThingListView types are mostly bad design. They should just be UnmodifiableThingList.)

Something for dart 3.0? (could make typedef now and migrate code)

@lrhn
Copy link
Member

lrhn commented Sep 30, 2022

LGTM. @lrhn would this be something you'd like to do?

Design the API, sure.

Can you elaborate a little more on how the native unmodifiable type data works?
Do you simply have an unmodifiable ByteBuffer which throws on modification attempts?

We can update the fromList to take a flag saying whether the result should be immutable or not:

external factory Uint8List.fromList(List<int> bytes, {bool unmodifiable = false});

The immutable list will have the same API and static type as the mutable list.

With that as a way to directly create an unmodifiable typed-data lis, I'd assume the .buffer of that would inherit the immutability, so immutableList.buffer.asUint16List() would give you an unmodifiable 16-bit list.
We can choose to only use UnmodifiableUint16ListView as views (what the unmodifiable ByteBuffer.asUint16ListView returns) so that it throws early on attempts to modify, or use the normal views and have the underlying buffer throw on modification attempts.

The second part, concatenating existing bytes, could be a separate constructor:

external factory Uint8List.fromLists(Iterable<List<int>> bytes, {bool unmodifiable = false});

Those two constructor changes should be the only needed API for this to work.
Then you can do everything you do today with the unmodiifable lists, like create views, as long as you don't try to change the contents.

We can consider adding a bool get isMutable to the lists, or to the buffer. It's not necessary, we don't do that for normal unmodifiable lists. (But it's technically possible because all typed-data types are sealed.)

WDYT?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-core-library SDK core library issues (core, async, ...); use area-vm or area-web for platform specific libraries.
Projects
None yet
Development

No branches or pull requests

2 participants