refactor: dynamic dispatch async commands #13464

Legend-Master · 2025-05-18T16:20:04Z

Reference: #13382

From some testing, this could speed up the compile time by ~10s (80s -> 70s) on a full debug build on my machine with a minimal tauri app setup, and reduces the incremental compilation time from ~10s to ~4s (By adding and removing a new line inside an async command)

I'm not entirely sure the performance implications of doing dynamic dispatch for those commands though, maybe we could enable this for the debug build and still use the old static dispatch one for the release build? (did this for dev only right now, we can revisit this in the future)

Some cargo llvm-lines outputs

Before

  Lines                  Copies               Function name
  -----                  ------               -------------
  1231833                66222                (TOTAL)
    86143 (7.0%,  7.0%)   2497 (3.8%,  3.8%)  std::panic::catch_unwind
    42500 (3.5%, 10.4%)   2500 (3.8%,  7.5%)  std::panicking::try::do_catch
    34162 (2.8%, 13.2%)   2500 (3.8%, 11.3%)  std::panicking::try::do_call
    33280 (2.7%, 15.9%)    416 (0.6%, 11.9%)  tokio::runtime::task::core::Cell<T,S>::new
    33280 (2.7%, 18.6%)    416 (0.6%, 12.6%)  tokio::runtime::task::harness::poll_future
    29952 (2.4%, 21.1%)    832 (1.3%, 13.8%)  tokio::runtime::task::harness::Harness<T,S>::complete::{{closure}}
    24472 (2.0%, 23.0%)    648 (1.0%, 14.8%)  <alloc::boxed::Box<T,A> as core::ops::drop::Drop>::drop
    22464 (1.8%, 24.9%)    416 (0.6%, 15.4%)  tokio::runtime::task::harness::Harness<T,S>::poll_inner

After

  Lines                 Copies               Function name
  -----                 ------               -------------
  399528                17715                (TOTAL)
   19985 (5.0%,  5.0%)      2 (0.0%,  0.0%)  tauri_test::main::{{closure}}
   14321 (3.6%,  8.6%)    148 (0.8%,  0.8%)  std::sync::mpmc::context::Context::with::{{closure}}
   14268 (3.6%, 12.2%)     73 (0.4%,  1.3%)  tauri::window::plugin::init::{{closure}}::{{closure}}::{{closure}}
   10552 (2.6%, 14.8%)    203 (1.1%,  2.4%)  <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
    8829 (2.2%, 17.0%)    203 (1.1%,  3.6%)  futures_util::future::future::map::_::<impl futures_util::future::future::map::Map<Fut,F>>::project_replace
    5700 (1.4%, 18.4%)     21 (0.1%,  3.7%)  std::sync::mpmc::list::Channel<T>::start_recv
    5581 (1.4%, 19.8%)    203 (1.1%,  4.8%)  futures_util::future::future::map::_::<impl futures_util::future::future::map::Map<Fut,F>>::project
    5292 (1.3%, 21.2%)     63 (0.4%,  5.2%)  std::sync::mpmc::counter::Receiver<C>::release
    4597 (1.2%, 22.3%)     21 (0.1%,  5.3%)  std::sync::mpmc::list::Channel<T>::discard_all_messages
    4350 (1.1%, 23.4%)    108 (0.6%,  5.9%)  <alloc::boxed::Box<T,A> as core::ops::drop::Drop>::drop

Note: the zip file contains the cargo build --timings HTML output, and because GitHub doesn't seem to like uploading HTML files directly, I zipped them in

cargo-build-timings.zip

github-actions · 2025-05-18T16:20:33Z

Package Changes Through `9ff6c70`

There are 8 changes which include tauri-bundler with minor, tauri with minor, tauri-cli with minor, tauri-codegen with minor, tauri-utils with minor, @tauri-apps/api with minor, @tauri-apps/cli with minor, tauri-runtime-wry with patch

Planned Package Versions

The following package releases are the planned based on the context of changes in this pull request.

package	current	next
@tauri-apps/api	2.5.0	2.6.0
tauri-utils	2.4.0	2.5.0
tauri-bundler	2.4.0	2.5.0
tauri-runtime	2.6.0	2.6.1
tauri-runtime-wry	2.6.0	2.6.1
tauri-codegen	2.2.0	2.3.0
tauri-macros	2.2.0	2.2.1
tauri-plugin	2.2.0	2.2.1
tauri-build	2.2.0	2.2.1
tauri	2.5.1	2.6.0
@tauri-apps/cli	2.5.0	2.6.0
tauri-cli	2.5.0	2.6.0

Add another change file through the GitHub UI by following this link.

Read about change files or the docs at github.com/jbolda/covector

WSH032 · 2025-05-18T17:02:45Z

crates/tauri/src/ipc/mod.rs

-  pub fn respond_async_serialized<F>(self, task: F)
-  where
-    F: Future<Output = Result<InvokeResponseBody, InvokeError>> + Send + 'static,
-  {
+  pub fn respond_async_serialized(
+    self,
+    task: Pin<Box<dyn Future<Output = Result<InvokeResponseBody, InvokeError>> + Send + 'static>>,
+  ) {


Considering that respond_async_serialized is a pub API, isn't this a BREAKING CHANGE?

maybe we could enable this for the debug build and still use the old static dispatch one for the release build?

In my opinion, having different signatures in debug and release modes would be confusing.

Not sure if I am correctly: if we only pass in Box::pin(...) as Pin<Box<dyn ...>>, then only a single template instantiation of respond_async_serialized<Pin<Box<dyn ...>>> will occur, which is effectively the same as changing the signature to Pin<Box<dyn ...>>, right?

Considering that respond_async_serialized is a pub API, isn't this a BREAKING CHANGE?

I believe this part was marked as unstable?

In my opinion, having different signatures in debug and release modes would be confusing.

That's true though, I feel like the performance hit probably won't be a problem considering this is only a pretty small part of the call chain anyways

Not sure if I am correctly: if we only pass in Box::pin(...) type, then only a single template instantiation of respond_async_serialized will occur, which is effectively the same as changing the signature to Pin<Box<dyn ...>>, right?

I believe the box itself will create a new generic variation for every closure, so this probably won't work

Not entirely sure if this was marked as unstable or not, will double check that tomorrow, but since we manage it through macros, it should be easy to add a new function

it's probably meant to be unstable but we only communicated that on the "parent" Invoke struct.

honestly though? in my opinion marking anything pub in our main tauri crate as unstable only via a doc comment is bullshit. Ideally unstable APIs should not be public, or clearly unstable (feature flag, or in an unstable module path idk) but that's a discussion for later i guess.

Since we're working on v3 i think marking it as deprecated would be the more sensible choice here.

~~"working" as in praying it will somehow appear out of nowhere~~

I am already directly using Invoke downstream (in pytauri), so I always hope these APIs change as little as possible (v3) 😂

it's probably meant to be unstable but we only communicated that on the "parent" Invoke struct.

Yeah, I think so after checking

honestly though? in my opinion marking anything pub in our main tauri crate as unstable only via a doc comment is bullshit. Ideally unstable APIs should not be public, or clearly unstable (feature flag, or in an unstable module path idk) but that's a discussion for later i guess.

To be fair though, some of the things like generate_handler! and generate_context! and the related code are impossible to be stable and they still have to be public

Normally, users won't use them directly so it's fine to break, but it's probably a problem for the library authors (I pinned the tauri version in tauri-runtime-verso, and also we can see the impact on pytauri as well)

I am already directly using Invoke downstream (in pytauri), so I always hope these APIs change as little as possible (v3) 😂

I have tried to make a inner private function so the public API won't change from this, and it seems to only produce slightly more LLVM IR than doing it from the macro, pushed it

Thanks for the heads up @WSH032 about avoiding this breaking change!

And the question is now, do we want to use this for both the debug build and the release build, or do we only do this in debug build for now, since this is done with a private inner function, we should be able to change this without any breakages

I just tested this Box::pin(...) as Pin<Box<dyn ...>>, and from the assembly output it does seem that only one instantiation of respond_async_serialized<Pin<Box<dyn ...>>> occurs: https://godbolt.org/z/9sMMqdo1o

To be fair though, some things like generate_handler! and generate_context! and the related code are impossible to make stable, and they still have to be public

I think I'll use #[doc(hide)] to hide them.

And the question is now, do we want to use this for both the debug build and the release build, or do we only do this in debug build for now

I think everyone has their own preferences. Personally, I don't care about code bloat, so I would prefer using dyn in debug mode to speed up development, and using impl generics in release mode to optimize performance.

I just tested this Box::pin(...) as Pin<Box<dyn ...>>, and from the assembly output it does seem that only one instantiation of respond_async_serialized<Pin<Box<dyn ...>>> occurs: https://godbolt.org/z/9sMMqdo1o

Interesting, we could try that as well (I do want to avoid thing inside the macro in general though, so if the inner function method works, I might prefer that one)

I think I'll use #[doc(hide)] to hide them.

We can't hide the top level macros like generate_handler!, but we probably can hide the other ones, but to be fair, they still show up in IDEs so I'm not sure how effective would this be

I think everyone has their own preferences. Personally, I don't care about code bloat, so I would prefer using dyn in debug mode to speed up development, and using impl generics in release mode to optimize performance.

Good to know, I feel like this probably won't be a problem in terms of performance since the overhead of other things will heavily out weight the cost of this, but I'm fine with keeping it the same in release mode for now

To be fair though, some of the things like generate_handler! and generate_context! and the related code are impossible to be stable and they still have to be public

Well there are exceptions to everything, it just shouldn't be the standard.

…nc-commands

FabianLars

didn't test this yet. if you want me too i can do so by tomorrow. looks like a safe merge tho

Legend-Master · 2025-05-29T14:02:42Z

I'll merge this for now since I'm preparing a follow up PR, would be nice to have some comparison data on mac and linux though 🙃

FabianLars · 2025-05-29T14:54:47Z

I'll try to remember that lol

FabianLars · 2025-05-30T14:12:15Z

@Legend-Master My very professional testing on macos showed a decrease from 7secs to ~3.2-3.3secs so pretty similar to your result (40% vs 45%)

Legend-Master · 2025-05-30T14:32:48Z

Nice! Thanks for testing, incremental build is the most important factor to me, this should help a lot 😀

* Dynamic dispatch async commands * format * Preserve `'static` * Use a inner function instead * Only do it for dev for now * Add change file * Tag respond_async_serialized_dyn with debug

Dynamic dispatch async commands

07615f0

github-project-automation bot moved this to 📬Proposal in Roadmap May 18, 2025

github-project-automation bot added this to Roadmap May 18, 2025

Legend-Master added 2 commits May 19, 2025 00:22

format

e2f24f6

Preserve 'static

de15f05

WSH032 reviewed May 18, 2025

View reviewed changes

Legend-Master and others added 4 commits May 19, 2025 09:12

Use a inner function instead

5f817b9

Only do it for dev for now

0fb22eb

Add change file

d4fe467

Merge branch 'dev' into dynamic-dispatch-async-commands

549f319

Legend-Master marked this pull request as ready for review May 19, 2025 02:58

Legend-Master requested a review from a team as a code owner May 19, 2025 02:58

Legend-Master added 3 commits May 19, 2025 10:59

Tag respond_async_serialized_dyn with debug

5a64b09

Merge remote-tracking branch 'upstream/dev' into dynamic-dispatch-asy…

097c8d5

…nc-commands

Merge remote-tracking branch 'upstream/dev' into dynamic-dispatch-asy…

9ff6c70

…nc-commands

FabianLars approved these changes May 29, 2025

View reviewed changes

Legend-Master merged commit 6a39f49 into tauri-apps:dev May 29, 2025
19 of 20 checks passed

Legend-Master deleted the dynamic-dispatch-async-commands branch May 29, 2025 14:03

github-project-automation bot moved this from 📬Proposal to 🔎 In audit in Roadmap May 29, 2025

github-actions bot mentioned this pull request May 29, 2025

Apply Version Updates From Current Changes #13282

Merged

Uh oh!

refactor: dynamic dispatch async commands #13464

refactor: dynamic dispatch async commands #13464

Uh oh!

Conversation

Legend-Master commented May 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented May 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Package Changes Through 9ff6c70

Uh oh!

WSH032 May 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Legend-Master May 18, 2025

Choose a reason for hiding this comment

Uh oh!

Legend-Master May 18, 2025

Choose a reason for hiding this comment

Uh oh!

FabianLars May 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

WSH032 May 18, 2025

Choose a reason for hiding this comment

Uh oh!

Legend-Master May 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Legend-Master May 19, 2025

Choose a reason for hiding this comment

Uh oh!

WSH032 May 19, 2025

Choose a reason for hiding this comment

Uh oh!

Legend-Master May 19, 2025

Choose a reason for hiding this comment

Uh oh!

FabianLars May 19, 2025

Choose a reason for hiding this comment

Uh oh!

FabianLars left a comment

Choose a reason for hiding this comment

Uh oh!

Legend-Master commented May 29, 2025

Uh oh!

Uh oh!

FabianLars commented May 29, 2025

Uh oh!

FabianLars commented May 30, 2025

Uh oh!

Legend-Master commented May 30, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Legend-Master commented May 18, 2025 •

edited

Loading

github-actions bot commented May 18, 2025 •

edited

Loading

Package Changes Through `9ff6c70`

WSH032 May 18, 2025 •

edited

Loading

FabianLars May 18, 2025 •

edited

Loading

Legend-Master May 19, 2025 •

edited

Loading