-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Simplify and standardize (or replace) build.sh scripts #53
Comments
One input related to considering changes to The main benefit of "fitting in" is that when you arrive in a new-to-you project you end up feeling at home when things work the way they work in most other places. While this might only be a small comfort factor, when the opposite is true (standard way doesn't work) my thought process is "oh boy, this is going to take a while" - so the reason to enable "standard tooling" isn't because it is such a big win, more to avoid a big loss. Maybe I am getting too old and cynical :D But I do feel like anec-data-ly "custom build commands" tend to correlate with a less good developer experience. I often find "not excellent documentation", "exactly one way to do it and no other", "only works on one platform", "requires complex tools like bazel before you can do anything", "running the tests is not just TL;DR: Old man Tim yells at |
One of my main efforts over the years has been to try to get us closer to being able to use standard build mechanisms (run I know @rlratzel is one of the usual proponents of build.sh, so I'll let him chime in if he wants to respond to your other thoughts 😄 |
That sounds great! |
I'm in favor of these changes, especially if they make I suspect A bit more history: This might seem trivial but I like being able to switch to any RAPIDS repo and run One comment I'm confused about is:
|
Well to be fair this issue is only two weeks old, so I doubt it's responsible for triggering much discussion yet :)
I think this anecdote illustrates a few things. First, build.sh was absolutely valuable at some point for reducing the complexity of building when onboarding new developers. It collected common knowledge into a script so that not everyone needed to know the ins and outs of each build. Second, our builds were too specialized and complicated, and build.sh was used because fixing the underlying issues with our CMake or pip-based builds would have been much harder. That is no longer true, though; we ought to be able to create just as good of an experience with standard tools now as our build systems have matured. Third, we need to properly document our CMake options (and can improve that using presets, see #52). Also, we need to document our Python builds. I strongly prefer projects that use standard commands (autoconf/cmake/etc) and document their options to projects that use a custom script. That leads me to the notes below.
I'd argue for a broader point of view. I like being able to look at any piece of OSS and know that I can probably get started with a small, simple set of commands: |
Every RAPIDS project has a build.sh script at the root. It is a bash script that simplifies the process of building the various components of the project. It was designed with the idea of being run in an environment where all the dependencies are installed, but we want to build both the C++ and Python components with the Python packages using the built C++ library rather than prebuilt artifacts. Over time, the scope of this script has increased: it is invoked in all the conda builds, and it supports a number of additional local user conveniences on a per-repo basis such as building Python packages in editable mode, passing arbitrary extra CMake arguments through, building tests and/or benchmarks, etc. However, this script is probably not the best place for most of this logic. #52 should help address one of the most common use cases for build.sh: accumulating a set of CMake options into a common, easy-to-use grouping. Other cases may no longer be as important as they once were due to improvements in our Python build infrastructure since we now use scikit-build-core to effectively use CMake to coordinate the build of the compiled components of the Python package with the C++ package. Once #52 is complete we should do a thorough audit of build.sh scripts to see how far down we can simplify them. To the extent possible, we should find ways to make these scripts uniform across RAPIDS. If the scripts wind up being truly trivial, then we should consider if there are alternative ways to educate developers to obviate the need for these scripts.
The text was updated successfully, but these errors were encountered: