renderFlutterWidget not working inside interactivity callback [BLOCKED BY dart:ui]

[naamapps]

Hey,
I'm trying to render a new image when the user interacts with the widget on iOS (android not tested), but I get an error.
Exception: Failed to render the widget: Exception: operation failed

Please advise,
Thanks

[ABausG]

Can you add a breakpoint to where the exception is thrown to determine what exactly might be going wrong?

[naamapps]

Hey @ABausG,
Thanks for the quick response.
I tried getting the stack trace but I can't because you used try-catch internally so it's swollen.
I think the error occurs because of the toImage function.
The callback is not running on the main thread, and I don't think flutter supports using ui functions on isolates.

See flutter/flutter#10647

[naamapps]

I wonder if there any solution at all in this case..
I wish we could use actual flutter widgets to render on the home screen widgets instead of images.

[ABausG]

Thanks for the investigation! According to flutter/flutter#10647 (comment) it might be solved if using flutter_isolate to spawn the isolate in the background callback.
Would you have time to try that out and open a PR if successful?
I agree having the ability to render Flutter natively would be ideal but given the performance restrictions the operating systems (rightfully) impose on the Homescreen Widgets I don't see it working any time soon

[naamapps]

I tried spawning a flutter isolate with this package, it still doesn't work, and throws the same error.
Maybe because I spawn the flutter isolate from an isolate?

[ctrl-aziz]

On Android it's run fine but on ios I have the same error
The error appeared when trying to convert repaintBoundary to image on

final ui.Image image =
          await repaintBoundary.toImage(pixelRatio: pixelRatio);

here is my flutter doctor

Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel stable, 3.13.7, on macOS 14.0 23A344 darwin-x64, locale en-TR)
[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
[✓] Xcode - develop for iOS and macOS (Xcode 15.0)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2022.3)
[✓] Connected device (3 available)
[✓] Network resources

• No issues found!

[naamapps]

@ctrl-aziz is right!
Only on iOS it's not working.

@ABausG Please is there anything we can do to fix this? this bug blocks me currently from updating my app to support interactive widgets in ios 17.

[ABausG]

Can you try if it maybe helps if you add the Override mentioned in
"interactive widgets > iOS > Point 5"

https://github.com/ABausG/home_widget/blob/dev/README.md

This might wake the app up a bit more such that dart:ui might work

[naamapps]

@ABausG I did that before opening the issue - no difference.
However I see a strange warning that might give us a clue on what's going on.
This only appears on iOS.

[VERBOSE-2:shell.cc(1004)] The 'home_widget/background' channel sent a message from native to Flutter on a non-platform thread. Platform channel messages must be sent on the platform thread. Failure to do so may result in data loss or crashes, and must be fixed in the plugin or application code creating that channel.
See https://docs.flutter.dev/platform-integration/platform-channels#channels-and-platform-threading for more information.

[naamapps]

Hi @ABausG,
I appreciate your work and the time you invest on maintaining the package.
Can we expect a fix for this issue soon?
Thanks.

[ABausG]

@naamapps I know of that Error Message. Unfortunately the fix for this that is behind the Link of registering the background task queue is not working as the header is not exposed in swift:
flutter/flutter#118832

Can you confirm that the function you define is being called however the image generation fails? Maybe @leighajarett has an idea how to work around this?

[naamapps]

@ABausG
The interactivity callback is definitely being executed. Only the image generation fails.
I hope this will be fixed soon 🤞

[mnefzger]

Hi @ABausG,
we also ran into the issue of not being able to render a flutter widget in the background callback.
By now, the issue you referenced (#118832) has been marked as fixed. Is there any chance that the home_widget package can be now updated to include a fix for this problem discussed here?
Thanks!

[dario-digregorio]

I have the same problem when renderFlutterWidget inside a background callback. I tried using flutter_isolates but I still get the error. I think the issue should be renamed since this has more to do with dart:ui in isolates then with interactivity callback. I will still try some things out and will update you here.

[ABausG]

So turns out the issue wasn't the one initially referenced but rather as @dario-digregorio mentioned the issue lays in dart:ui where you cannot capture the canvas while in the background (or in an isolate)

I think the appropriate ticket to track this on is this one flutter/flutter#10647