How can I create a save view into external memory through TypedData?

[blaugold]

I have some externally allocated memory and would like to allow code to access it without having to know about the lifetime of the memory. I can get a Uint8List through Pointer.asTypedList but I have to manually manage the lifetime of the backing memory.

Is it safe to attach a native finalizer to the object returned from Pointer.asTypedList, which frees the external memory, through Dart_NewFinalizableHandle?
Also, what would be the implications for the Uint8List's ByteBuffer? I suspect it would not be safe to use it.

[devoncarew]

Perhaps @dcharkes would know this? Not sure if this relates to dart:ffi or not.

[dcharkes]

Is it safe to attach a native finalizer to the object returned from Pointer.asTypedList, which frees the external memory, through Dart_NewFinalizableHandle?

Yes, but you need to take care to hold on to the object which has the finalizer. Indeed, if you use the bytebuffer to make a new typed data and lose the old one, the finalizer could run.

Also, to ensure objects don't get collected prematurely you can add a Dart method to keeps it alive.

pragma('vm:never-inline')
void reachabilityFence(Object? object) {}

In Dart 2.17 we make using native finalizers much easier.

First, we make them available in Dart code, rather than using the Dart API.

sdk/sdk/lib/ffi/native_finalizer.dart

Lines 232 to 297 in 532c116

	/// A native finalizer which can be attached to Dart objects.
	///
	/// When [attach]ed to a Dart object, this finalizer's native callback is called
	/// after the Dart object is garbage collected or becomes inaccessible for other
	/// reasons.
	///
	/// Callbacks will happen as early as possible, when the object becomes
	/// inaccessible to the program, and may happen at any moment during execution
	/// of the program. At the latest, when an isolate group shuts down,
	/// this callback is guaranteed to be called for each object in that isolate
	/// group that the finalizer is still attached to.
	///
	/// Compared to the [Finalizer] from `dart:core`, which makes no promises to
	/// ever call an attached callback, this native finalizer promises that all
	/// attached finalizers are definitely called at least once before the program
	/// ends, and the callbacks are called as soon as possible after an object
	/// is recognized as inaccessible.
	abstract class NativeFinalizer {
	/// Creates a finalizer with the given finalization callback.
	///
	/// The [callback] must be a native function which can be executed outside of
	/// a Dart isolate. This means that passing an FFI trampoline (a function
	/// pointer obtained via [Pointer.fromFunction]) is not supported.
	///
	/// The [callback] might be invoked on an arbitrary thread and not necessary
	/// on the same thread that created [NativeFinalizer].
	// TODO(https://dartbug.com/47778): Implement isolate independent code and
	// update the above comment.
	external factory NativeFinalizer(Pointer<NativeFinalizerFunction> callback);
	/// Attaches this finalizer to [value].
	///
	/// When [value] is no longer accessible to the program,
	/// the finalizer will call its callback function with [token]
	/// as argument.
	///
	/// If a non-`null` [detach] value is provided, that object can be
	/// passed to [Finalizer.detach] to remove the attachment again.
	///
	/// The [value] and [detach] arguments do not count towards those
	/// objects being accessible to the program. Both must be objects supported
	/// as an [Expando] key. They may be the *same* object.
	///
	/// Multiple objects may be using the same finalization token,
	/// and the finalizer can be attached multiple times to the same object
	/// with different, or the same, finalization token.
	///
	/// The callback will be called exactly once per attachment, except for
	/// registrations which have been detached since they were attached.
	///
	/// The [externalSize] should represent the amount of native (non-Dart) memory
	/// owned by the given [value]. This information is used for garbage
	/// collection scheduling heuristics.
	void attach(Finalizable value, Pointer<Void> token,
	{Object? detach, int? externalSize});
	/// Detaches this finalizer from values attached with [detach].
	///
	/// If this finalizer was attached multiple times to the same object with
	/// different detachment keys, only those attachments which used [detach]
	/// are removed.
	///
	/// After detaching, an attachment won't cause any callbacks to happen if the
	/// object become inaccessible.
	void detach(Object detach);
	}

Note that this only works for void function(void*), including free but excluding WinHeapFree. So in Dart 2.17 this doesn't work for Windows yet. Supporting Windows is tracked in:

• Implement support for creating isolate independent native functions code in dart:ffi #47778

Secondly, we introduce Finalizable, which prevents the need manually writing reachabilityFence by automatically extending the lifetime of variables to the end of the syntactic scope.

sdk/sdk/lib/ffi/native_finalizer.dart

Lines 7 to 223 in 532c116

	/// Marker interface for objects which should not be finalized too soon.
	///
	/// Any local variable with a static type that _includes `Finalizable`_
	/// is guaranteed to be alive until execution exits the code block where
	/// the variable is in scope.
	///
	/// A type _includes `Finalizable`_ if either
	/// * the type is a non-`Never` subtype of `Finalizable`, or
	/// * the type is `T?` or `FutureOr<T>` where `T` includes `Finalizable`.
	///
	/// In other words, while an object is referenced by such a variable,
	/// it is guaranteed to *not* be considered unreachable,
	/// and the variable itself is considered alive for the entire duration
	/// of its scope, even after it is last referenced.
	///
	/// _Without this marker interface on the variable's type, a variable's
	/// value might be garbage collected before the surrounding scope has
	/// been completely executed, as long as the variable is definitely not
	/// referenced again. That can, in turn, trigger a `NativeFinalizer`
	/// to perform a callback. When the variable's type includes [Finalizable],
	/// The `NativeFinalizer` callback is prevented from running until
	/// the current code using that variable is complete._
	///
	/// For example, `finalizable` is kept alive during the execution of
	/// `someNativeCall`:
	///
	/// ```dart
	/// void myFunction() {
	/// final finalizable = MyFinalizable(Pointer.fromAddress(0));
	/// someNativeCall(finalizable.nativeResource);
	/// }
	///
	/// void someNativeCall(Pointer nativeResource) {
	/// // ..
	/// }
	///
	/// class MyFinalizable implements Finalizable {
	/// final Pointer nativeResource;
	///
	/// MyFinalizable(this.nativeResource);
	/// }
	/// ```
	///
	/// Methods on a class implementing `Finalizable` keep the `this` object alive
	/// for the duration of the method execution. _The `this` value is treated
	/// like a local variable._
	///
	/// For example, `this` is kept alive during the execution of `someNativeCall`
	/// in `myFunction`:
	///
	/// ```dart
	/// class MyFinalizable implements Finalizable {
	/// final Pointer nativeResource;
	///
	/// MyFinalizable(this.nativeResource);
	///
	/// void myFunction() {
	/// someNativeCall(nativeResource);
	/// }
	/// }
	///
	/// void someNativeCall(Pointer nativeResource) {
	/// // ..
	/// }
	/// ```
	///
	/// It is good practise to implement logic involving finalizables as methods
	/// on the class that implements [Finalizable].
	///
	/// If a closure is created inside the block scope declaring the variable, and
	/// that closure contains any reference to the variable, the variable stays
	/// alive as long as the closure object does, or as long as the body of such a
	/// closure is executing.
	///
	/// For example, `finalizable` is kept alive by the closure object and until the
	/// end of the closure body:
	///
	/// ```dart
	/// void doSomething() {
	/// final resourceAction = myFunction();
	/// resourceAction(); // `finalizable` is alive until this call returns.
	/// }
	///
	/// void Function() myFunction() {
	/// final finalizable = MyFinalizable(Pointer.fromAddress(0));
	/// return () {
	/// someNativeCall(finalizable.nativeResource);
	/// };
	/// }
	///
	/// void someNativeCall(Pointer nativeResource) {
	/// // ..
	/// }
	///
	/// class MyFinalizable implements Finalizable {
	/// final Pointer nativeResource;
	///
	/// MyFinalizable(this.nativeResource);
	/// }
	/// ```
	///
	/// Only captured variables are kept alive by closures, not all variables.
	///
	/// For example, `finalizable` is not kept alive by the returned closure object:
	///
	/// ```dart
	/// void Function() myFunction() {
	/// final finalizable = MyFinalizable(Pointer.fromAddress(0));
	/// final nativeResource = finalizable.nativeResource;
	/// return () {
	/// someNativeCall(nativeResource);
	/// };
	/// }
	///
	/// void someNativeCall(Pointer nativeResource) {
	/// // ..
	/// }
	///
	/// class MyFinalizable implements Finalizable {
	/// final Pointer nativeResource;
	///
	/// MyFinalizable(this.nativeResource);
	/// }
	/// ```
	///
	/// It's likely an error if a resource extracted from a finalizable object
	/// escapes the scope of the finalizable variable it's taken from.
	///
	/// The behavior of `Finalizable` variables applies to asynchronous
	/// functions too. Such variables are kept alive as long as any
	/// code may still execute inside the scope that declared the variable,
	/// or in a closure capturing the variable,
	/// even if there are asynchronous delays during that execution.
	///
	/// For example, `finalizable` is kept alive during the `await someAsyncCall()`:
	///
	/// ```dart
	/// Future<void> myFunction() async {
	/// final finalizable = MyFinalizable();
	/// await someAsyncCall();
	/// }
	///
	/// Future<void> someAsyncCall() async {
	/// // ..
	/// }
	///
	/// class MyFinalizable implements Finalizable {
	/// // ..
	/// }
	/// ```
	///
	/// Also in asynchronous code it's likely an error if a resource extracted from
	/// a finalizable object escapes the scope of the finalizable variable it's
	/// taken from. If you have to extract a resource from a `Finalizable`, you
	/// should ensure the scope in which Finalizable is defined outlives the
	/// resource by `await`ing any asynchronous code that uses the resource.
	///
	/// For example, `this` is kept alive until `resource` is not used anymore in
	/// `useAsync1`, but not in `useAsync2` and `useAsync3`:
	///
	/// ```dart
	/// class MyFinalizable {
	/// final Pointer<Int8> resource;
	///
	/// MyFinalizable(this.resource);
	///
	/// Future<int> useAsync1() async {
	/// return await useResource(resource);
	/// }
	///
	/// Future<int> useAsync2() async {
	/// return useResource(resource);
	/// }
	///
	/// Future<int> useAsync3() {
	/// return useResource(resource);
	/// }
	/// }
	///
	/// /// Does not use [resource] after the returned future completes.
	/// Future<int> useResource(Pointer<Int8> resource) async {
	/// return resource.value;
	/// }
	/// ```
	///
	/// _It is possible for an asynchronous function to *stall* at an
	/// `await`, such that the runtime system can see that there is no possible
	/// way for that `await` to complete. In that case, no code after the
	/// `await` will ever execute, including `finally` blocks, and the
	/// variable may be considered dead along with everything else._
	///
	/// If you're not going to keep a variable alive yourself, make sure to pass the
	/// finalizable object to other functions instead of just its resource.
	///
	/// For example, `finalizable` is not kept alive by `myFunction` after it has
	/// run to the end of its scope, while `someAsyncCall` could still continue
	/// execution. However, `finalizable` is kept alive by `someAsyncCall` itself:
	///
	/// ```dart
	/// void myFunction() {
	/// final finalizable = MyFinalizable();
	/// someAsyncCall(finalizable);
	/// }
	///
	/// Future<void> someAsyncCall(MyFinalizable finalizable) async {
	/// // ..
	/// }
	///
	/// class MyFinalizable implements Finalizable {
	/// // ..
	/// }
	/// ```
	// TODO(http://dartbug.com/44395): Add implicit await to Dart implementation.
	// This will fix `useAsync2` above.
	abstract class Finalizable {
	factory Finalizable._() => throw UnsupportedError("");
	}

And thirdly, we standardize on using a wrapper class wrapping the Pointer, TypedData or ByteBuffer (or file handle, etc.) and standardize on always using the native resource as methods of that class rather than ever leaking the resource form out of the class. An example can be found here:

sdk/tests/ffi/vmspecific_native_finalizer_test.dart

Lines 40 to 71 in 532c116

	class MyNativeResource implements Finalizable {
	final Pointer<Void> pointer;
	bool _closed = false;
	MyNativeResource._(this.pointer, {int? externalSize}) {
	print('pointer $pointer');
	freeFinalizer.attach(this, pointer,
	externalSize: externalSize, detach: this);
	}
	factory MyNativeResource() {
	const num = 1;
	const size = 16;
	final pointer = calloc(num, size);
	return MyNativeResource._(pointer, externalSize: size);
	}
	/// Eagerly stop using the native resource. Cancelling the finalizer.
	void close() {
	_closed = true;
	freeFinalizer.detach(this);
	free(pointer);
	}
	void useResource() {
	if (_closed) {
	throw UnsupportedError('The native resource has already been released');
	}
	print(pointer.address);
	}
	}

If you're using Dart 2.16 you can adopt the same patterns by wrapping your TypedData or Pointer in a wrapper class, attaching the finalizer with Dart_NewFinalizableHandle to the wrapper class, only using the typed data in the wrapper class, and adding a call to reachabilityFence(this) at the end of each method in the wrapper class.

It should be fairly easy to migrate this to the NativeFinalizer with Finalizable in Dart 2.17 (if you don't have to support Windows) or in Dart 2.18 then.

[blaugold]

Thanks for the detailed response!

In my use cases I have another API which takes a Uint8List as an argument and I cannot easily control for how long this list will be used. Currently, I copy the external data into a new Uint8List, but was hoping to be able to remove the extra allocation and copying.

As an aside, is there a difference between these two reachability fences?

@pragma('vm:never-inline')
void reachabilityFence(Object? object) {}

@pragma('vm:never-inline')
Object? reachabilityFence(Object? obj) {
  return obj;
}

I was using the first variant for the longest time but got crashes in AOT mode which looked like being caused by premature finalization of the native resource. I'm not seeing those crashes with the second variant.

[dcharkes]

I was using the first variant for the longest time but got crashes in AOT mode which looked like being caused by premature finalization of the native resource. I'm not seeing those crashes with the second variant.

That is very curious. I would have to investigate that.

In my use cases I have another API which takes a Uint8List as an argument and I cannot easily control for how long this list will be used. Currently, I copy the external data into a new Uint8List, but was hoping to be able to remove the extra allocation and copying.

Unfortunately that is currently not supported. I'll give it some thought. Maybe it would be possible to convert a Pointer into a TypedData that uses the VM's typed data finalizer instead of a manual Dart_NewFinalizableHandle or NativeFinalizer. In other words expose Dart_NewExternalTypedDataWithFinalizer as a method on Pointer.

[dcharkes]

Also, what would be the implications for the Uint8List's ByteBuffer? I suspect it would not be safe to use it.

I've dug some deeper on this: And it looks like it is actually safe to use the buffers.

The buffer wraps the typed data:

sdk/sdk/lib/_internal/vm/lib/typed_data_patch.dart

Line 2042 in 6faa5f3

_ByteBuffer get buffer => new _ByteBuffer(this);

And getting another TypedData creates a view around the same backing store:

sdk/sdk/lib/_internal/vm/lib/typed_data_patch.dart

Lines 1914 to 1920 in 6faa5f3

	Uint8List asUint8List([int offsetInBytes = 0, int? length]) {
	length ??=
	(this.lengthInBytes - offsetInBytes) ~/ Uint8List.bytesPerElement;
	_rangeCheck(
	this.lengthInBytes, offsetInBytes, length * Uint8List.bytesPerElement);
	return new _Uint8ArrayView._(this._data, offsetInBytes, length);
	}

The Dart_NewExternalTypedDataWithFinalizer simply allocates an external typed data (which is the same object for an asTypedList) and then attaches the finalizer:

sdk/runtime/vm/dart_api_impl.cc

Lines 3808 to 3813 in 6faa5f3

	result = ExternalTypedData::New(cid, reinterpret_cast<uint8_t*>(data), length,
	thread->heap()->SpaceForExternal(bytes));
	if (callback != nullptr) {
	AllocateFinalizableHandle(thread, result, peer, external_allocation_size,
	callback);
	}

So, doing asTypedList in Dart, and then Dart_NewFinalizableHandle in C/C++ or NativeFinalizer in Dart should be equal to using Dart_NewExternalTypedDataWithFinalizer in C/C++.

[blaugold]

Thanks for looking into this. That solves the issue for me.