Cargo does not install binaries with the profiles as defined in the installed package

[Robbepop]

Problem

Today I released a CLI tool for my Wasm interpreter: https://crates.io/crates/wasmi_cli

The entire tool depends heavily on the right Cargo profile to be used during compilation, namely:

[profile.release]
lto = "fat"
codegen-units = 1

Otherwise we see performance regressions of 50%-400%.

Cargo docs state that the --release profile is used by default for installs.
However, when installing the wasmi CLI tool via cargo install wasmi_cli and then benchmark it against a local build cargo run --release --package wasmi_cli -- etc.. I see very big differences in performance which leads me to the guess that Cargo does not apply the proper profiles to installed crates but uses the default configurations instead.

This bug is critical to the performance of the wasmi CLI application.

Steps

1. Install wasmi_cli via cargo install wasmi_cli
2. Clone the wasmi repository: git clone https://github.com/paritytech/wasmi.git
3. Set working directory to the wasmi repository: cd wasmi
4. Benchmark the installed wasmi CLI tool: time wasmi_cli ./crates/wasmi/benches/wat/fibonacci.wat fib_iterative 100000000
5. Benchmark the CLI from the repo: time cargo run --package wasmi_cli --profile bench ./crates/wasmi/benches/wat/fibonacci.wat fib_iterative 100000000

Observe that step 4. takes way more time than step 5.
Works with other examples as well.
For example using the recursive version of fibonacci: ./crates/wasmi/benches/wat/fibonacci.wat fib_recursive 40

Possible Solution(s)

Cargo should use the profiles as defined in the installed package and not the default settings.

Notes

How did I know that this is probably because Cargo uses the default release settings for installation? I simply compiled the repository wasmi_cli via the default release settings and then both performed equally bad.

Version

cargo 1.66.0 (d65d197ad 2022-11-15)

[weihanglo]

Just looked into Cargo.toml of wasmi_cli v0.23.0 from https://github.com/paritytech/wasmi/blob/v0.23.0/crates/cli/Cargo.toml, there is no [profile.release] being written. In step 5 you specified profile bench instead of release. It is likely that cargo install wasmi_cli just picks the default release profile, since no one overwrote the default. Maybe I miss any context?

[ehuss]

Thanks for the report! The issue is that workspace members do not pick up profiles from a workspace when published. This issue is tracked in #8264, so closing as a duplicate of that.

Unfortunately I can't offhand think of a good fix (other than duplicating the profile when publishing). It would definitely be something that would be good to fix.

[Robbepop]

Just looked into Cargo.toml of wasmi_cli v0.23.0 from https://github.com/paritytech/wasmi/blob/v0.23.0/crates/cli/Cargo.toml, there is no [profile.release] being written. In step 5 you specified profile bench instead of release. It is likely that cargo install wasmi_cli just picks the default release profile, since no one overwrote the default. Maybe I miss any context?

Thanks for the reply. Before publishing wasmi_cli I wrote the following into the root:

[profile.release]
lto = "fat"
codegen-units = 1

Also I tried installing wasmi_cli using --profile bench which is specified in the wasmi_cli repo just like the above profile.release but that changed nothing with the performance problem.

[Robbepop]

Thanks for the report! The issue is that workspace members do not pick up profiles from a workspace when published. This issue is tracked in #8264, so closing as a duplicate of that.

Unfortunately I can't offhand think of a good fix (other than duplicating the profile when publishing). It would definitely be something that would be good to fix.

Thanks a lot for the link to the other issue. I will try out that fix. Could be good enough as a temporary solution.

[Robbepop]

Just wanted to write here that the suggestion by @ehuss actually worked.
The fix for now is to manually copy over the [profile.release] settings from the workspace Cargo.toml to the CLI Cargo.toml before publishing a new version.
Now the CLI application performance is as expected. :)

[Gor-Madatyan]

@Robbepop ,

Just wanted to write here that the suggestion by @ehuss actually worked. The fix for now is to manually copy over the [profile.release] settings from the workspace Cargo.toml to the CLI Cargo.toml before publishing a new version. Now the CLI application performance is as expected. :)

---

I also checked it and tried to do these actions and publish the package, but I got

Warning:

warning: profiles for the non root package will be ignored, specify profiles at the workspace root:
package:   ROOT\crates\__CRATE_NAME__\Cargo.toml
workspace: ROOT\Cargo.toml

at the time of writing this message, simply copying profiles from root Cargo.toml into Cargo.toml of the package is not fixing the problem for me.

could you tell us a little more about how you solved it? The link that gave @ehuss (#8264) just said that in the next edition the behavior of the compiler may change

[Robbepop]

@MAKA-Rustcean I too get that warning when applying this fix and yet it works and properly applies the optimizations. I think the warning is unfortunate given the known bug. However, since this is a bug a warning is fair in order for people to rightfully get confused and eventually landing here and push awareness of this issue. :)

[Gor-Madatyan]

@MAKA-Rustcean I too get that warning when applying this fix and yet it works and properly applies the optimizations. I think the warning is unfortunate given the known bug. However, since this is a bug a warning is fair in order for people to rightfully get confused and eventually landing here and push awareness of this issue. :)

---

@Robbepop thank you for the really useful info!!!