Sending large data between Dart and Rust

[laurenspriem]

I was wondering how well this package works with sending large amount of bytes between Dart and Rust.

To give more context, my use case would be to decode images in Flutter and then send the decoded bytes to Rust for more efficient image processing, and then sending the processed bytes back. I understand that it would be more efficient to just decode the image in Rust, but unfortunately I couldn't find a crate that supports all image formats I need (especially heic).

If anyone done something like this, sending large data back and forth, please let me know! I'm curious if it would work and how fast it would be. Or if anyone knows how to decode heic in Rust that information is also very welcome :)

[fzyzcjy]

Image processing is exactly why I created frb - I have some image processing in rust for my flutter app!

So, firstly, large amount of data from Rust to Dart is zero-copy, as long as it is Vec<u8> and you enable full_dep: true.

Or if anyone knows how to decode heic

If Rust cannot do it today, you can always use C/C++ for it. For example, a quick search gives https://github.com/strukturag/libheif.

Looks like Rust can decode heic: https://crates.io/crates/libheif-rs (which looks like a very thin wrapper around libheif C/C++ library - it is very common for Rust people to do this to enjoy the safety of Rust as well as the existing code in C/C++)

For other formats, there seems to be https://github.com/image-rs/image. I personally find https://github.com/twistedfall/opencv-rust also helpful, but if all you need is just decoding an image then it is an overkill.

As for speed, IIRC I do not see any issues. But I do not know how large your "large" is. So I would suggest you to make a simple benchmark and see whether it is fast enough for you. If you find it too slow, don't hesitate to report and we can discuss!

[laurenspriem]

Hey, back again!

I've just started actually working on it. So far it's going well, but I'm only about 30% there yet.

and you enable full_dep: true

What does this do exactly, and where do I set this?

[fzyzcjy]

So far it's going well, but I'm only about 30% there yet.

Feel free to ping me when there are questions!

What does this do exactly

tell frb that you are happy to install LLVM dependency on your host (indeed needed by https://pub.dev/packages/ffigen). then it can generate faster code utilizing that.

doc about how to install llvm dep: https://cjycode.com/flutter_rust_bridge/manual/ffigen-troubleshooting#llvm-needs-to-be-installed

and where do I set this?

set it in flutter_rust_bridge.yaml

[laurenspriem]

Thank you for the guidance! I have made the full_dep change.

So far I'm using it only to resize an image, for which I've written this rust function.

Now it's working correctly, but for some reason it's super slow. The above function takes around 5-6 seconds to complete when running my app in debug mode. I've added time checks, and it seems that there isn't one particular part of the function that's slow, rather the whole function is slow. Which leads me to believe I'm doing something wrong with the flutter rust bridge. Is there anything else I should do to make it actually performant, or can you point me towards some things to test out to find the issue?

Thanks again so much for the help!

[fzyzcjy]

Firstly, could you please try release mode (both release build Rust and build Dart)? IMHO debug mode is often super slow because it does a ton of sanity checks.

it seems that there isn't one particular part of the function that's slow, rather the whole function is slow

If it is part of the body of the rust function that is slow, then I guess it is not a bug of frb. Instead, maybe try the release build suggestion above, and if it is still slow, maybe try to see which exact operation is slow and raise an issue there. (e.g. suppose the "resize" crate is slow then we can ask there). Also feel free to discuss here as well.

[fzyzcjy]

Quickly glanced at the code and there are some places that may be speedup a little bit. For example, the "Return dst as a Vec" part may use Vec::with_capacity or even more advanced things that eliminates memory copy completely. But anyway, don't optimize prematurely - only if this part is too slow we need to worry about it.