Skip to content

Introduce function to create a cxxbridge target #244

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Nov 5, 2022

Conversation

trondhe
Copy link
Contributor

@trondhe trondhe commented Oct 26, 2022

Related issue: #241

Create a library target that will run cxxbridge on the given files on a crate that is assumed to contain a module with #[cxx::bridge].
Call with corrosion_cxxbridge(<target> CRATE <crate> PATH <tomlPath> FILES [<filePath>, ...])

  • <cxx_target> is the name of the created target that will host the cxx interface
  • CRATE <crate> is the name of the specific target generated by corrosion_import_crate
  • MANIFEST_PATH <path> is the relative path from where this function is called to Cargo.toml of <crate>
  • FILES [<filePath>, ...] is the relative paths to files with #[cxx::bridge]. These are relative to <tomlPath>/src/

given

- root
  - someCratePath
    - Cargo.toml
    - src
      - lib.rs
- CMakeLists.txt
- main.cpp

where the Cargo.toml has

[package]
name = "someCrate"
version = "0.1.0"
edition = "2021"

[lib]
crate-type = ["staticlib"]

[dependencies]
cxx = "1.0"

and the lib.rs has

#[cxx::bridge]
mod bridge {
    extern "Rust" {
        fn someRustFunction();
    }
}

fn someRustFunction() {
    println!("Hello world via cxxbridge!");
}

the import works by

cmake_minimum_required(VERSION 3.24)
project(example CXX)

find_package(corrosion REQUIRED)
corrosion_import_crate(MANIFEST_PATH someCratePath/Cargo.toml)
corrosion_add_cxxbridge(myCxxTarget CRATE someCrate MANIFEST_PATH someCratePath FILES lib.rs)

add_executable(example main.cpp)
target_link_libraries(example PRIVATE myCxxTarget)

then the main can look like

#include <myCxxTarget/lib.h>

int main() {
    someRustFunction();
    return 0;
}

Copy link
Collaborator

@jschwe jschwe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this PR!
Looks good, but I do have some comments.

In general I would like to first release v0.3 as a stable release before merging this change. That should be pretty soon though.

COMMAND
"${CMAKE_COMMAND}" -E make_directory ${cxx_generated_dir}
COMMAND
${CXXBRIDGE} ${CMAKE_CURRENT_LIST_DIR}/${crate_path}/${file} --header > ${cxx_generated_dir}/${cxx_header}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this even need to depend on cargo-build? The documentation does not say that the tool has to run before/after cargo build, so maybe it can be run independently?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that is true! It can be run whenever, and since it links in the imported rust target, it will regardless be built correctly, will fix!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update on this. If we depend on TARGET cargo-build_<importedtarget> then we will always rerun the cxxbridge generation if this target changes.
It might run a bit too optimistically, but again should always run when it needs to.
Im guessing this would be the preferable case?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think always rerunning is the only sensible thing for now. We can still optimize it later.
FYI, The cargo-build target is a custom command, so it is always out of date and rerun.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, yeah, Ill change it to the actual target, thanks!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't actually use that target since its an interface target. Depending on the exact rust source file instead.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, yeah, Ill change it to the actual target, thanks!

I think you may have misunderstood. I meant since the cargo-build_xxx target is a custom target and thus always out of date, the POST_BUILD step would also always be out of date. So the original approach was perfectly fine and the more conservative approach compared to what you changed to now.

Copy link
Contributor Author

@trondhe trondhe Oct 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup I definitely misunderstood, and agreed. I'll re-redo it and keep it as it originally was. It's the most safe case for now.

foreach(file ${_arg_FILES})
get_filename_component(filename ${file} NAME_WE)
set(cxx_header ${filename}.h)
set(cxx_source ${filename}.cpp)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For multiple files like the common mod.rs name but in different directories, we would generate just one source file, since we would overwrite it. I guess for a first version we could just leave it as is and document as a known limitation for now.
I'm not sure if we really want to preserve the directory structure though. At least the common src directory we would probably want to strip...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding the src, I thought it was helpful in the sense that they are all relative to each other. But src as far as I can see must always be included regardless, so maybe its not confusing to omit it.

Regarding if we have multiple levels of nesting, I could test out a change to see if I can match the folder structure as well!

Copy link
Collaborator

@jschwe jschwe Oct 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What we probably want to avoid is for C++ files having to include like this:

#include <Mytarget/src/myheader.h>
//                 ^^^ This would be strange

Maybe we could also change the directory structure slightly like this:

# Make a corrosion_generated directory in which also other artifacts, e.g. bindgen, cbindgen could be placed
set(corrosion_generated_dir ${CMAKE_CURRENT_BINARY_DIR}/corrosion_generated/${target})
# subdirectory for cxx artifacts
set(cxx_generated_dir ${corrosion_generated_dir }/cxx)
if(PREFIX_OPTION) # Would need a descriptive name for the option.
    target_include_directories(${target} PUBLIC ${corrosion_generated_dir}/)
else()
    target_include_directories(${target} PUBLIC ${cxx_generated_dir}/)
endif()

What do you think?

Copy link
Contributor Author

@trondhe trondhe Oct 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah the first thing there is definitely strange and is something that is already avoided with the current implementation.
It however places all files in the same directory as

#include "<cxxbridge_target_name>/<rust_source_filename_we>.h"

where

corrosion_cxxbridge(myCxxTarget CRATE someCrate PATH someCratePath FILES src/lib.rs src/foo/lib.rs)

will both add a file placed as myCxxTarget/lib.h, which is suboptimal.

Regarding the second part, I am a bit unsure. Have you some typo regarding which variable is used?
But yes, standardizing on a corrosion_generated folder with subtypes for different generations is definitely smart, will absolutely do!

Copy link
Contributor Author

@trondhe trondhe Oct 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to that it now follows the folder structure

corrosion_cxxbridge(myCxxTarget CRATE someCrate MANIFEST_PATH someCratePath FILES lib.rs foo/lib.rs)

will now result in

#include "myCxxTarget/lib.h"
#include "myCxxTarget/foo/lib.h"

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding the second part, I am a bit unsure. Have you some typo regarding which variable is used?

I edited my comment above. We could probably use CMake to strip the src/ prefix if present.

will now result in
#include "myCxxTarget/lib.h"
#include "myCxxTarget/foo/lib.h"

That sounds nice. I haven't had the time to look at that part again, I'll have another look on the weekend.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, but it is stripped. You will never have to write src anywhere with the current solution. So it should be as you wanted it to be, unless I misunderstood again 😊

@jschwe jschwe linked an issue Oct 27, 2022 that may be closed by this pull request
Copy link

@Be-ing Be-ing left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the cxxbridge CLI tool is a good idea and I don't think Corrosion should add anything for it. Instead, I think we should push cxx upstream to change cxx-build to write headers to a directory specified by an environment variable from the Cargo build script. This would be more robust and also much simpler for Corrosion to implement. dtolnay/cxx#462 (comment)

@jschwe
Copy link
Collaborator

jschwe commented Oct 27, 2022

@Be-ing I've read your linked comment, but I don't quite follow your concerns. What this PR proposes does not depend on the build.rs at all, and all generated files are known at configure time (because the user indirectly specifies which files are generated), so the problems you mentioned should not occur, right? Am I missing something?

@Be-ing
Copy link

Be-ing commented Oct 27, 2022

The problems are

  1. Distributing/acquiring a CLI tool separate from the Rust source code. Even in the best case, this is redundant build time.
  2. Version mismatches between the CLI tool and the Rust library using CXX

@jschwe
Copy link
Collaborator

jschwe commented Oct 27, 2022

Distributing/acquiring a CLI tool separate from the Rust source code. Even in the best case, this is redundant build time.

In the best case the correct version is already installed and in PATH and nothing has to be done, which is an improvement over the build.rs case as far as I can see. I will admit that if the tool is not installed having to do a Fetchcontentdeclare and using corrosion to build the tool is extra effort.

Version mismatches between the CLI tool and the Rust library using CXX

That can be easily checked I think. This PR does not add it, but we could check the version of CXX used in the Rust project via cargo metadata and compare that to the output of cxxbridge --version (I'm just assuming that the --version option exists here, but if it doesn't adding it should be not too much effort)

@Be-ing
Copy link

Be-ing commented Oct 27, 2022

In the best case the correct version is already installed and in PATH and nothing has to be done, which is an improvement over the build.rs case as far as I can see.

No, it isn't, because the cxx crates still have to be built for the Rust macro.

That can be easily checked

Even if those checks were implemented, it would still be bad for downstream users who would need to ensure these versions match. This problem is obviated by using cxx_build.

@trondhe
Copy link
Contributor Author

trondhe commented Oct 27, 2022

That seems like it can be easily modified at a later point in corrosion when cxx supports it. As far as I can see, it has to go through corrosion with a function of similar signature regardless.

@Be-ing
Copy link

Be-ing commented Oct 27, 2022

Please don't merge this and entrench this bad solution! I will implement a better solution today.

@Be-ing
Copy link

Be-ing commented Oct 27, 2022

Another downside to this for Corrosion is that this requires maintaining code specific to each code generator. With the environment variable approach, this would not be required if we can convince different code generators (cxx, cxx-qt, cbindgen, autocxx?, others?) to converge on using the same environment variable.

@trondhe
Copy link
Contributor Author

trondhe commented Oct 27, 2022

Please don't merge this and entrench this bad solution! I will implement a better solution today.

Haha, sure! Hope it will be good then 😉

@ogoffart
Copy link
Collaborator

As a datapoint, in Slint, we used to call cbindgen from cmake (via a xtask script). This caused a few problems like we had one more target to build, and also figuring out when to re-run the script, and also we needed to generate these to run the tests that are actually not using cmake.

But moving it into the build.rs fixed most problems.
We use this approach to generate the include file in a folder pointed at the SLINT_GENERATED_INCLUDE_DIR env variable set by cmake.
https://github.com/slint-ui/slint/blob/ea4ad7297bb0096f71d796befffd7e32a153f6b8/api/cpp/build.rs#L23

@jschwe
Copy link
Collaborator

jschwe commented Oct 27, 2022

Even if those checks were implemented, it would still be bad for downstream users who would need to ensure these versions match. This problem is obviated by using cxx_build.

The check would be a part of the corrosion_add_cxxbridge() function which would fail if the versions don't match.

Please don't merge this and entrench this bad solution! I will implement a better solution today.

There is plenty of time for discussion, no need to rush anything. I want to release v0.3 before merging this anyway, and there is still some testing effort needed before that happens - I'd estimate a week or more (If you have time, please do test the beta release).
Personally, I am fairly happy with what this PR proposes. It may not fit all use cases, but the cxx documentation does prominently mention cxxbridge as a tool for non cargo based setups. It also doesn't prevent us from supporting the build.rs approach too.

Another downside to this for Corrosion is that this requires maintaining code specific to each code generator. With the environment variable approach, this would not be required if we can convince different code generators (cxx, cxx-qt, cbindgen, autocxx?, others?) to converge on using the same environment variable.

The issue here is more that Corrosion would need to rely on the build-script being "Corrosion compatible" and doing what corrosion expects. If corrosion controls the code generator invocation directly, then Corrosion is in control.

I think ultimately we would want to support both ways. There are users which don't like build scripts and would prefer code generation to be visible in the CMake files and there will also be others which prefer the cargo style of using build scripts (and like you mentioned, there are valid reasons for preferring build scripts).

@trondhe
Copy link
Contributor Author

trondhe commented Oct 27, 2022

Aside from this discussion I want to mention one issue here.
Building multiple targets with this function and linking them together will likely always create multiple definitions errors to the cxx types. Someone reported it by my old hack solution here trondhe/rusty_cmake#6 (comment)

We must either create a pure cxx definition library, while the corrosion generated ones must exclude it.
Or cxx should supply it in a different fashion id recon.

@Be-ing
Copy link

Be-ing commented Oct 27, 2022

There is plenty of time for discussion, no need to rush anything. I want to release v0.3 before merging this anyway, and there is still some testing effort needed before that happens - I'd estimate a week or more (If you have time, please do test the beta release).

Sure! We're aiming to do a release of CXX-Qt imminently as well, so I'll give the Corrosion beta a try before we do.

@jschwe
Copy link
Collaborator

jschwe commented Oct 27, 2022

As a datapoint, in Slint, we used to call cbindgen from cmake (via a xtask script). This caused a few problems like we had one more target to build

I guess this depends on whether you need the build script anyway or you could get rid of it if you did it in CMake. In larger CMake project having the bindings generator as a CMake target should improve parallelism and build speed.
Corrosion calls cargo build once per target, so there is no parallelism when building build scripts.

and also figuring out when to re-run the script,

Is simply always rerunning not an option? I would have assumed that the binding generation is a rather small part of the overall build time.

and also we needed to generate these to run the tests that are actually not using cmake.

Yes, in such a case it defintely makes sense to put the generation in the build script.


Aside from this discussion I want to mention one issue here.
Building multiple targets with this function and linking them together will likely always create multiple definitions errors to the cxx types. Someone reported it by my old hack solution here trondhe/rusty_cmake#6 (comment)

We must either create a pure cxx definition library, while the corrosion generated ones must exclude it.

If this is possible that would be preferred.
Do the generated .cc files simply contain the definitions for the cxx builtin types or is additional source code generated depending on the rust code?


Sure! We're aiming to do a release of CXX-Qt imminently as well, so I'll give the Corrosion beta a try before we do.

Thanks!

@Be-ing
Copy link

Be-ing commented Oct 28, 2022

Upstream cxx PR: dtolnay/cxx#1120

@Be-ing
Copy link

Be-ing commented Oct 28, 2022

Alternative, simpler, and more robust solution for Corrosion: #247

@msvetkin
Copy link
Contributor

As a datapoint, in Slint, we used to call cbindgen from cmake (via a xtask script). This caused a few problems like we had one more target to build, and also figuring out when to re-run the script, and also we needed to generate these to run the tests that are actually not using cmake.

Could you share an example where it was hard to understand where to re-run the script?

In the proposed solution it will re-trigger cxxbridge-cli if an imported cargo target changes.
It could be additionally checked if cxxbridge-cli input files changes.

Alternative, simpler, and more robust solution for Corrosion: #247

The idea behind corrosion_cxxbridge is to allow integrate cxx into cmake project as a regular target and let cmake to take control.
I am sorry, but I am failing to see how your PR solves the described workflow.

@ogoffart
Copy link
Collaborator

Could you share an example where it was hard to understand where to re-run the script?

We need to make rules to rerun this script when the .rs files changes. And the build.rs has the infrastructure to do that, that was harder to describe in cmake.

@trondhe trondhe force-pushed the trondhe/add-cxxbridge branch 2 times, most recently from 779d4d8 to 007fc99 Compare October 28, 2022 14:07
@trondhe trondhe requested a review from jschwe October 28, 2022 14:08
@trondhe trondhe force-pushed the trondhe/add-cxxbridge branch 2 times, most recently from 006a89c to a75a761 Compare October 28, 2022 14:27
@Be-ing
Copy link

Be-ing commented Oct 28, 2022

I am sorry, but I am failing to see how your PR solves the described workflow.

#247 relies on cxx-build, not the cxxbridge CLI tool. I think the cxxbridge CLI tool should be deprecated.

@trondhe
Copy link
Contributor Author

trondhe commented Oct 28, 2022

@jschwe
I did the changes we discussed, hoping it should be better now!

I also added such that if you do

corrosion_cxxbridge(myCxxTarget CRATE someCrate MANIFEST_PATH someCratePath FILES lib.rs foo/mod.rs)

then for cpp it will be

#include <myCxxTarget/lib.h>
#include <myCxxTarget/foo/mod.h>

@trondhe trondhe force-pushed the trondhe/add-cxxbridge branch from c60db99 to 88aa8cc Compare October 29, 2022 07:40
@trondhe
Copy link
Contributor Author

trondhe commented Oct 29, 2022

@jschwe also, I did intentionally do nothing with how to fetch the cxxbridge executable. Any idea how we want to solve that for tests?

@Be-ing
Copy link

Be-ing commented Oct 29, 2022

I did intentionally do nothing with how to fetch the cxxbridge executable. Any idea how we want to solve that for tests?

That is precisely the problem with this approach. There's no good way to handle this. Every way would be error prone and easy to get the cxxbridge and cxx crate versions out of sync. And even if those versions do stay in sync, it's wasted build time because you have build cxx and its dependencies twice.

@dtolnay
Copy link

dtolnay commented Oct 29, 2022

To the extent that building dependencies twice is an issue (I don't know -- my expectation is that people would use Bazel rather than CMake if this kind of thing were a concern) it seems like something to fix totally independent of cxx. Today if someone uses corrosion_import_crate twice to import 2 different Rust crates, and those crates have some dependencies in common, would those common dependencies end up built twice or does corrosion make an effort to share a single CARGO_TARGET_DIR between the two cargo invocations? If they're built twice, that should be fixed. If they're not built twice, presumably neither would cxxbridge's dependencies be built twice.

As for versions being synced, synchronizing between a cxx and cxxbridge-cmd version doesn't seem different than synchronizing cxx with cxx-build. You have raised this in over and over in many comments on many threads but I have struggled to understand any of them. My best guess as to what you have in mind is that in the cxx + cxx-build case, you expect to leave synchronization of those up to the author of the CMake rules, whereas in the cxx + cxxbridge-cmd case your impression is that the build system or a distro package manager would somehow need to play a role in synchronization. That seems like an artificial and unwarranted distinction, but again this is just my guess as to what you mean, so apologies if you don't mean that. The author of the CMake rules can be the one responsible for synchronization either way; and they have exactly equal amount of synchronization to be responsible for either way. Optionally the build system can help them catch when a version is out of sync, but that's also the same either way.

@Be-ing
Copy link

Be-ing commented Oct 29, 2022

To the extent that building dependencies twice is an issue (I don't know -- my expectation is that people would use Bazel rather than CMake if this kind of thing were a concern) it seems like something to fix totally independent of cxx. Today if someone uses corrosion_import_crate twice to import 2 different Rust crates, and those crates have some dependencies in common, would those common dependencies end up built twice or does corrosion make an effort to share a single CARGO_TARGET_DIR between the two cargo invocations? If they're built twice, that should be fixed. If they're not built twice, presumably neither would cxxbridge's dependencies be built twice.

Corrosion does share build directories for crates in a workspace, but that's tangential to this discussion. When building a single crate, if the crate depends on both cxx and cxx-build, the build artifacts for their dependencies would be shared with or without a workspace.

As for versions being synced, synchronizing between a cxx and cxxbridge-cmd version doesn't seem different than synchronizing cxx with cxx-build.

This big difference is where the versions are maintained. In the simple case, they're right next to each other in Cargo.toml:

[dependencies]
cxx = "1.0.something"

[build-dependencies]
cxx-build = "1.0.something"

and if Cargo.lock is committed to the repository, they're also kept together there.

With cxxbridge, one version is in Cargo.toml, and the other is... somewhere in the CI script that has to be manually kept in sync with Cargo? And for local development...??

@dtolnay
Copy link

dtolnay commented Oct 29, 2022

Sharing a build directory is a matter of passing CARGO_TARGET_DIR in the environment or --target-dir on the command line, independent of workspaces. Nobody would be forced to use a workspace.

This big difference is where the versions are maintained. In the simple case, they're right next to each other in Cargo.toml, and if Cargo.lock is committed to the repository, they're also kept together there.

With cxxbridge, one version is in Cargo.toml, and the other is... somewhere in the CI script that has to be manually kept in sync with Cargo? And for local development...??

What's stopping you from doing exactly that for cxxbridge-cmd? You can put it as a dependency in Cargo.toml, it will appear in Cargo.lock, and you can update it using cargo update. That aspect should be no different between the two approaches.

@jschwe
Copy link
Collaborator

jschwe commented Oct 30, 2022

What's stopping you from doing exactly that for cxxbridge-cmd? You can put it as a dependency in Cargo.toml, it will appear in Cargo.lock, and you can update it using cargo update. That aspect should be no different between the two approaches.

As the maintainer of Corrosion - I agree with you and think this is a perfectly valid approach, and I don't see any reason why it should not work. We don't even have to cargo install, we could also fetch the repo, checkout the tag and build cxxbridge from source in the build tree (which I would prefer).

Today if someone uses corrosion_import_crate twice to import 2 different Rust crates, and those crates have some dependencies in common, would those common dependencies end up built twice or does corrosion make an effort to share a single CARGO_TARGET_DIR between the two cargo invocations? If they're built twice, that should be fixed. If they're not built twice, presumably neither would cxxbridge's dependencies be built twice.

The target directory depends on the current list directory of the CMakeLists.txt where corrosion_import_crate() is called. This is intentional to allow isolation in case different RUSTFLAGS are used, since those would invalidate all dependencies and always cause a rebuild.
Crates added from the same CMakeLists.txt share dependencies (and this can be a whole workspace).
Anyway, I don't see the rebuilding aspect as a huge issue. In CI you would presumably have the correct version installed, while locally you would only need to build cxxbridge the first time.

@jschwe also, I did intentionally do nothing with how to fetch the cxxbridge executable. Any idea how we want to solve that for tests?

For CI I would prefer to first restructure CI, so that the configure environment, configure cmake, build and test steps are reusable, but that is probably for another PR. For now I'd suggest commenting out the add_subdirectory() in tests, so that it is easy for me to test locally, but not tested in CI yet.

@Be-ing
Copy link

Be-ing commented Oct 30, 2022

could also fetch the repo, checkout the tag and build cxxbridge from source in the build tree (which I would prefer).

I would not prefer to compile Rust code during CMake's configure stage. That's one of the main reasons I like Corrosion: it nicely separates Cargo downloading dependencies and Cargo compiling into the CMake workflow.

In CI you would presumably have the correct version installed, while locally you would only need to build cxxbridge the first time.

... until you run cargo update

@jschwe
Copy link
Collaborator

jschwe commented Oct 30, 2022

I would not prefer to compile Rust code during CMake's configure stage. That's one of the main reasons I like Corrosion: it nicely separates Cargo downloading dependencies and Cargo compiling into the CMake workflow.

This would happen at build time. We already use this approach at work to run code generators before compiling, so I know it works.

In CI you would presumably have the correct version installed, while locally you would only need to build cxxbridge the first time.

... until you run cargo update

And then you would get an error which reminds you to update your Dockerfile.

@Be-ing
Copy link

Be-ing commented Nov 1, 2022

we could also fetch the repo, checkout the tag and build cxxbridge from source in the build tree (which I would prefer).

You'd need to redundantly download the cxx repository into the CMake build directory (redundant because Cargo has already done this, but doesn't put it in a path you could easily determine) and make the code generation target depend on the target for building cxxbridge. It'd be probably be a good idea to use cargo run to execute cxxbridge to avoid depending on the implementation details of how Cargo arranges its target directory and adding .exe to the command for Windows. There would need to be a CMake option to override this and explicitly provide the path to a cxxbridge executable, which would require checking the version of the executable. Without that override, this would be unpackagable.

This is basically reimplementing what Cargo can already do with build.rs (build and execute a Rust executable as part of the Rust build process), which is ridiculous to expect every build system to do and makes cxx users' and packagers' lives harder for no technical reason. CXX-Qt will not be supporting this overcomplicated and convoluted process, and even if we did, the code in this PR couldn't be reused for it because CXX-Qt generates more C++ files than CXX does so it'd need an incompatible CLI. So #247 is still relevant.

And after all this, cargo test might not link unless the user also uses cxx-build in build.rs, in which case they might run into one-definition-rule violations with the redundant compilation of the generated C++ by CMake. 🙃 Also, the crate that uses cxx wouldn't be usable within a Cargo-only build without a C++ build system without having a build.rs using cxx-build, so the crate author would have to choose one or the other, or guard everything in build.rs behind a feature flag.

@jschwe
Copy link
Collaborator

jschwe commented Nov 1, 2022

You'd need to redundantly download the cxx repository into the CMake build directory (redundant because Cargo has already done this, but doesn't put it in a path you could easily determine) and make the code generation target depend on the target for building cxxbridge.

Yes, this is somewhat redundant but imho also not a major issue.

It'd be probably be a good idea to use cargo run to execute cxxbridge to avoid depending on the implementation details of how Cargo arranges its target directory and adding .exe to the command for Windows.

Corrosion already depends on these implementation details and will copy the build artifacts from the target directory to the location CMake expects them to be. Using $<TARGET_FILE:target> already works. This is required already because a CMake build is not necessarily finished when the linker has linked the target. Perhaps the target is later packaged in some other form, or signed or otherwise post-processed.

CXX-Qt will not be supporting this overcomplicated and convoluted process, and even if we did, the code in this PR couldn't be reused for it because CXX-Qt generates more C++ files than CXX does so it'd need an incompatible CLI. So #247 is still relevant.

It's entirely possible that this solution is not fit for every usecase. Please kep the 247 discussion in 247 though.

And after all this, cargo test might not link

cargo test is already a problem in Corrosion and there are already usecases where cargo test does not work. In the future it is planned to support testing via ctest, but this is blocked on stabilization of a cargo flag which basically provides a cargo rustc like way to pass rustflags to cargo test.

unless the user also uses cxx-build in build.rs

This is not a Corrosion issue, because Corrosion does provide everything required to implement a build.rs solution.

@Be-ing
Copy link

Be-ing commented Nov 1, 2022

cargo test is already a problem in Corrosion and there are already usecases where cargo test does not work. In the future it is planned to support testing via ctest

That will be nice to have Cargo tests automatically integrated into ctest via Corrosion. Currently we're manually calling cargo test, so it will be nice to simplify that.

This is not a Corrosion issue

I didn't mean to imply that it was. I am just pointing out more downsides of this overcomplicated solution even if the work is put in to make it function for a narrow set of use cases.

Copy link
Collaborator

@jschwe jschwe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. I think the only thing missing now is the CI integration.
Could you add a CMake option in the tests directory to enable the cxxbridge test? It would be disabled by default, and I would then add one job to CI which also runs the cxxbridge test (in a follow-up PR).

@Be-ing
Copy link

Be-ing commented Nov 3, 2022

I don't think this should be merged without compiling cxxbridge as part of the build process. That's quite a lot more work, but this is awfully brittle without it.

@trondhe trondhe force-pushed the trondhe/add-cxxbridge branch from 88aa8cc to aeb64ec Compare November 3, 2022 21:05
trondhe and others added 2 commits November 3, 2022 22:06
Co-authored-by: Mikhail Svetkin <mikhail.svetkin@remarkable.no>
This is the more conservative solution.
Since  cargo-build_<target> is a custom command and is always out of date, this custom command
will then also be up to date.

If generation time becomes a problem it can be optimized later, and lean on
the safe side for it being up to date for now.

Co-authored-by: Mikhail Svetkin <mikhail.svetkin@remarkable.no>
@trondhe trondhe force-pushed the trondhe/add-cxxbridge branch from aeb64ec to 578e147 Compare November 3, 2022 21:06
@jschwe
Copy link
Collaborator

jschwe commented Nov 5, 2022

@trondhe Could you cherry-pick this commit: jschwe@62a687f to add a CI test. This also exposes a problem where the cpp file calls a fromLib function instead of print (probably due to some refactoring) causing the test to fail.

I don't think this should be merged without compiling cxxbridge as part of the build process. That's quite a lot more work, but this is awfully brittle without it.

It's not too much work and easier for me to add it after merging this PR to master.

@trondhe trondhe force-pushed the trondhe/add-cxxbridge branch from 578e147 to 07a33c1 Compare November 5, 2022 14:47
@jschwe jschwe merged commit 625fd43 into corrosion-rs:master Nov 5, 2022
@trondhe trondhe deleted the trondhe/add-cxxbridge branch November 5, 2022 17:18
@niyue
Copy link
Contributor

niyue commented Nov 6, 2022

@jschwe Thanks for the enhancement.

I have a C++ library that uses a rust library via cxx and corrosion previously.

I tried using the latest commit of corrosion (b921fd0) to use this enhancement to simplify the code, but ran into an issue:

CMake Error in CMakeLists.txt:
  Target "my-rust-lib-bridge" INTERFACE_INCLUDE_DIRECTORIES property contains
  path:

    "/Users/ss/projects/my-rust-lib/build-cmake-debug-osx/corrosion_generated/cxxbridge/my-rust-lib-bridge/include"

  which is prefixed in the build directory.


CMake Error in CMakeLists.txt:
  Target "my-rust-lib-bridge" INTERFACE_INCLUDE_DIRECTORIES property contains
  path:

    "/Users/ss/projects/my-rust-lib/build-cmake-debug-osx/corrosion_generated/cxxbridge/my-rust-lib-bridge/include"

  which is prefixed in the build directory.Target "my-rust-lib-bridge"
  INTERFACE_INCLUDE_DIRECTORIES property contains path:

    "/Users/ss/projects/my-rust-lib/build-cmake-debug-osx/corrosion_generated/cxxbridge/my-rust-lib-bridge/include"

  which is prefixed in the source directory.

This is probably because I exported my C++ library for external use.

I browsed the source code briefly, is it because

target_include_directories(${cxx_target} PUBLIC ${generated_dir}/include)

target_include_directories(${cxx_target} PUBLIC ${generated_dir}/include)

Should we change it to something like below?

target_include_directories(${cxx_target}
             PUBLIC
                 $<BUILD_INTERFACE:${generated_dir}/include>
                 $<INSTALL_INTERFACE:include>
             )

@niyue
Copy link
Contributor

niyue commented Nov 8, 2022

@trondhe @jschwe
I have a question about the usage of this feature. Suppose I have a C++ library my_lib that depends on the cxx_bridge_target, which uses this feature to generate. Currently, the generated headers are placed under ${CMAKE_CURRENT_BINARY_DIR}/corrosion_generated directory, and with some additional sub directory path like cxxbridge/${cxx_target}, which I have to read the corrosion code to figure out.

If my C++ library my_lib needs to be exported and installed for external consumption, I wonder how corrosion users are expected to setup the project for my_lib so that the generated cxx bridge related headers can be installed and consumed by my_lib users. Do you have any guidance on this? Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cxx integration
7 participants