Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Rust] Implement flatc build script wrapper #5252

Closed
wants to merge 1 commit into from

Conversation

rjsberry
Copy link

See also #5216.

This PR implements a wrapper around flatc using Rust code which is intended to be used in Rust build script files. Once merged it will be immediately useable:

[build-dependencies]
flatc = { git = "https://githib.com/google/flatbuffers.git", rev = "$SHA" }

@aardappel @rw Do you have any thoughts on distribution via crates.io?

For working with a specific commit of flatc, the above is sufficient and pushing to crates.io is not required. However with the current structure (crate as a subdirectory) it will not be possible to use the build script with revisions of flatc committed before the wrapper.

For crates.io distribution we could push on releases of FlatBuffers and tie the commit of this wrapper with the release version.

/// .compile();
/// }
/// ```
pub fn schema<P: AsRef<Path>>(&mut self, file: P) -> &mut Self {
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wasn't sure what to name this argument as (adding FBS files). I think it might be better of as distinct from --schema.

@aardappel aardappel requested a review from rw March 21, 2019 23:03
license = "Apache-2.0"
edition = "2018"
build = "build.rs"
description = "flatc Rust build script wrapper"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to make it clear that this new Cargo package is only part of the story -- how about making this description read: Companion package to flatbuffers, to generate code at compile time.?


//! # flatc
//!
//! A library for build scripts to compile FlatBuffer schemas.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again I think we need to point explicitly to the main flatbuffers package so that users don't get confused. Maybe a description like: Companion package to flatbuffers, to generate code at compile time.

fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(
f,
"An {} error occurred during FlatBuffer compilation: {}",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about we add a phrase here to say: "with crate-provided flatc"? Might save users a lot of time debugging flatc versions, especially if they have it installed in /usr/local/bin or something, as mac users do.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've extended the error reporting to include the path of the executable compiled by the crate and the commit SHA that it was built from.

@rw
Copy link
Collaborator

rw commented Mar 22, 2019

@rjsberry Also there is a CI failure related to how this PR is adding a Cargo Workspace to the project. WDYT about just keeping the crates totally separate?

https://travis-ci.org/google/flatbuffers/jobs/509127091#L1018

@rjsberry
Copy link
Author

WDYT about just keeping the crates totally separate?

This can be removed. I have always used a workspace when I have multiple crates in one repository but cargo can locate the crate under rust/flatc when specifying it as a dependency with a git URL.

@rjsberry
Copy link
Author

wrt. testing there is a small test in rust/flatc/tests/: do you want this moved to tests/ and run via RustTest.sh?

@rw
Copy link
Collaborator

rw commented Mar 22, 2019

@rjsberry

  1. That's a good suggestion, let's move the crate to rust/flatc.
  2. It would be great if it could be tested with RustTest.sh. I hope there aren't issues with testing on some platforms, though (due to the toolchain required to build this flatc crate.)


impl Build {
#[doc(hidden)]
pub fn output<P: AsRef<Path>>(&mut self, dir: P) -> &mut Self {
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not happy with having this as part of the public interface and simply hidden from the docs (as it would permit misuse of Builder), but we need to do this to test the crate from /tests/rust_usage_test.

@rjsberry
Copy link
Author

Is there a reason why the Travis pipeline is running with rustc 1.30.1? I'll roll the flatc crate back to the 2015 edition.

@rjsberry
Copy link
Author

@rw

It seems that the CMake components in before_scripts in CI are affecting the re-compilation of flatc in the build script for the crate as CMake artifacts exist in the root of the project.

How do you think we should resolve this? A couple of ideas come to mind:

  • Have before_scripts build in a subdirectory rather than the root (I'm against this as it would affect a lot of other components referencing executables at the project root).

  • Move Rust testing to the end of TestAll.sh and the AppVeyor scripts and clean the project root before we start.

@rw
Copy link
Collaborator

rw commented Mar 28, 2019

Not sure, yet. @aardappel any thoughts?

@aardappel
Copy link
Collaborator

I'm not an expert on the technologies involved, but would it make most sense to have the build for the crate be completely isolated in a subdir, such that it builds its own flatc if it needs to test it can do that?

Or, if we don't care to test that, can it reuse the existing flatc for the test?

@rjsberry
Copy link
Author

rjsberry commented Mar 29, 2019

We could use the existing flatc, but I think it's important we verify the CMake build done by the crate as part of CI.

Seems like there's also an open issue for arch handling of Visual Studio generators in the cmake crate which will affect the CI, too.

use std::{env, fs, path::Path};

#[test]
fn build_script_wrapper() {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rjsberry WDYT about comparing that the contents are the same as monster_test_generated.rs?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds reasonable. I've run a quick check locally and am seeing quite a few differences. When and how is monster_test_generated.rs generated?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rjsberry It's the result of generate_code.sh

@github-actions
Copy link

github-actions bot commented Apr 8, 2021

This pull request is stale because it has been open 6 months with no activity. Please comment or this will be closed in 14 days.

@github-actions github-actions bot added the stale label Apr 8, 2021
@github-actions github-actions bot closed this Apr 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants