[vm/ffi] Array, Struct and Union, asTypedData

[dcharkes]

Our compounds are a view on a Pointer or TypedData.

These compounds can be created from a typed data:

• [vm/ffi] Struct and Union constructors fromTypedList #53418
• [vm/ffi] Array.fromTypedData constructors #54525

We should also enable getting a typed data pointing to the backing store, this will facilitate easier copying of large ranges of data.

API & implementation sketch:

• Extension method on subtypes of Struct and Union. Call site needs to be transformed to use the #sizeOf the subtype.
• Extension method on Array. Call site needs to be transformed to use the #sizeOf the type argument of Array.

The implementation needs to do branching on whether the backing store is pointer or a typed data. For pointers an external typed data is created. For typed datas a new view needs to be created if the offset != 0 (or length > sizeOf) after https://dart-review.googlesource.com/c/sdk/+/354226.

[FireSourcery]

This is great! Now we have c-style structs in dart, effectively named field views on TypeData/ByteBuffer.

Should retrieving the TypeData view also be supported? This would allow fluid casting between struct types. I suppose this can be done by the user implementation as well.

I do find the create method to be quite elegant. Although it might still of some consideration: individual constructors for alloc, and cast/view, which would be akin the TypedList unnamed constructor and TypedList .sublistView.

[FireSourcery]

If I may make a few more feature requests.

A fieldValueOrNull getter:

Since C does not restrict casting with regards to memory access safety, whereas dart must.

"Field declarations in a [Struct] subclass declaration are automatically given a setter and getter implementation which accesses the native struct's field in memory."

I don't except this to be too much on the code gen side, since Struct.create already checks the input buffer size. Having the OrNull method could relax the size requirement as well. Much like List elementAtOrNull()

A potential application:

@Packed(1)
final class VarReadRequest extends Struct implements Payload<VarReadRequestValues> {
  // Struct is useful for defining a region of memory, giving a name to each field.
  @Array(16)
  external Array<Uint16> ids;
 
  factory VarReadRequest.cast(TypedData typedData) => Struct.create<VarReadRequest>(typedData);

  static int get idCountMax => 16;

  @override
  PayloadMeta build(VarReadRequestValues args, MotPacket header) {
    if (args.length > idCountMax) throw ArgumentError('Max Ids: $idCountMax');
    var idSum = 0;
    for (final (index, id) in args.indexed) {
      ids[index] = id;
      idSum += id;
    } 
    return PayloadMeta(args.length * 2, (idSum, 0));
  }

  // Access may require a workaround, since the boundary must be the full extent.
  @override
  VarReadRequestValues parse(MotPacket header, void stateMeta) {
    // return Iterable.generate(header.payloadLength ~/ 2, (index) => ids[index]);
    return header.payloadAt<Uint16List>(0);
  }
}

// Another case where boundary checking must be done "manually"

  TypedField get startFieldPart;
  TypedField get idFieldPart;
  TypedField get lengthFieldPart;
  TypedField get checksumFieldPart;

  // Struct cannot cast less than full length
  int? get startFieldOrNull => startFieldPart.fieldValueOrNull(_byteData);
  int? get idFieldOrNull => idFieldPart.fieldValueOrNull(_byteData);
  int? get lengthFieldOrNull => lengthFieldPart.fieldValueOrNull(_byteData);
  int? get checksumFieldOrNull => checksumFieldPart.fieldValueOrNull(_byteData);

abstract mixin class TypedField<T extends NativeType> {
  const TypedField._();
  const factory TypedField(int offset) = TypedOffset<T>;

  int get offset;
  int get size => sizeOf<T>();
  int get end => offset + size;  
 
  // replaced by struct
  int fieldValue(ByteData byteData) => byteData.wordAt<T>(offset);
  void setFieldValue(ByteData byteData, int value) => byteData.setWordAt<T>(offset, value);
  // not yet replaceable. this class would be redundant otherwise, for the better
  int? fieldValueOrNull(ByteData byteData) => byteData.wordAtOrNull<T>(offset);
}

extension GenericWord on ByteData { 
  // throws range error
  int wordAt<R extends NativeType>(int byteOffset, [Endian endian = Endian.little]) {
    return switch (R) {
      const (Int8) => getInt8(byteOffset),
      const (Int16) => getInt16(byteOffset, endian),
      const (Int32) => getInt32(byteOffset, endian),
      const (Uint8) => getUint8(byteOffset),
      const (Uint16) => getUint16(byteOffset, endian),
      const (Uint32) => getUint32(byteOffset, endian),
      _ => throw UnimplementedError(),
    };
  }

  int? wordAtOrNull<R extends NativeType>(int byteOffset, [Endian endian = Endian.little]) {
    return (byteOffset + sizeOf<R>() <= lengthInBytes) ? wordAt<R>(byteOffset, endian) : null;
  }
}
```

[Atom735]

i used that code for it

  void loadingFromBytes(Uint8List bytes) {
    final allocated = malloc.allocate(sizeOf<TgaFileHeader>());
    final typedList = Pointer<Uint8>.fromAddress(allocated.address)
        .asTypedList(sizeOf<TgaFileHeader>());
    for (var i = 0; i < sizeOf<TgaFileHeader>(); i++) {
      typedList[i] = bytes[i];
    }
    header = Pointer<TgaFileHeader>.fromAddress(allocated.address).ref;
  }