-
Notifications
You must be signed in to change notification settings - Fork 839
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Move protoc generation to binary crate, unpin prost/tonic build (#3876) #3927
Conversation
path::Path, | ||
}; | ||
|
||
fn main() -> Result<(), Box<dyn std::error::Error>> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is just the code from build.rs, but with the conditionals on path existence removed (as we want it to error if it can't find the protobuf definitions)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The idea looks great -- thank you @tustvold
The subject of this PR says "move to binary crate" but the crate seems to be called "gen"
I left some comment / usability suggestions but otherwise 👍
arrow-flight/CONTRIBUTING.md
Outdated
./regen.sh | ||
``` | ||
|
||
The above script will run the Rust binary located in [gen](./gen) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it would be useful to put a note here on the context of why this doesn't use the standard protoc process (i.e. so that the code can be vendored and not automatically change if people have different versions of protoc)
It might be helpful to explicitly note that now any changes to protobuf files will require running these scripts manually
.github/workflows/arrow_flight.yml
Outdated
- name: Setup Rust toolchain | ||
uses: ./.github/actions/setup-builder | ||
- name: Run gen | ||
run: ./arrow-flight/regen.sh | ||
- name: Verify workspace clean |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- name: Verify workspace clean | |
- name: Verify workspace clean (if this fails, run ./arrow-flight/regen.sh and check in results) |
# specific language governing permissions and limitations | ||
# under the License. | ||
|
||
[package] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It might be worth pointing out here that this crate is not meant to be distributed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe add publish = false
here
@@ -64,13 +64,6 @@ tempfile = "3.3" | |||
tokio-stream = { version = "0.1", features = ["net"] } | |||
tower = "0.4.13" | |||
|
|||
[build-dependencies] | |||
# Pin specific version of the tonic-build dependencies to avoid auto-generated |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This change alone is 😍
FYI I've filled a corresponding ticket in DataFusion - apache/datafusion#5718 |
Which issue does this PR close?
Closes #3876
Rationale for this change
Inspired by @sunng87 's work in #3926
What changes are included in this PR?
Are there any user-facing changes?