GH-37923: [R] Move macOS build system to nixlibs.R #37684
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
assignUser
force-pushed
the
r-macos-prebuild
branch
from
e0703ca to
52eecc2
Compare
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
assignUser
force-pushed
the
r-macos-prebuild
branch
from
aeeb70d to
25ae44e
Compare
|
@github-actions crossbow submit r-binary-packages |
|
Revision: 25ae44e Submitted crossbow builds: ursacomputing/crossbow @ actions-95e0198aa1
|
|
@github-actions crossbow submit r-binary-packages |
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
|
@github-actions crossbow submit r-binary-packages |
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
|
@github-actions crossbow submit r-binary-packages This should be it... test program compilation correctly detects openssl and chooses the right binary... 🤞 |
|
Revision: a30f3cd Submitted crossbow builds: ursacomputing/crossbow @ actions-d657866664
|
|
@github-actions crossbow submit r-binary-packages |
|
Revision: 4327943 Submitted crossbow builds: ursacomputing/crossbow @ actions-05b816ca9b
|
|
@github-actions crossbow submit r-binary-packages |
github-actions
bot
added
awaiting change review
|
@github-actions crossbow submit r-binary-packages |
|
Revision: 99cfe8a Submitted crossbow builds: ursacomputing/crossbow @ actions-a57a53186a
|
github-actions
bot
added
awaiting changes
paleolimbot
approved these changes
Oct 5, 2023
There was a problem hiding this comment.
It seems there's one CI failure on MacOS (which seems relevant given the MacOS-nature of this PR), but I'm not familiar with that set of tests to know if it is related.
I think there are probably some further changes that will be required; however, I do think it's important to merge this sooner than later so that there is time for the nightlies to run before the code freeze + release.
I'll defer to @kou to approve any changes to the cpp/ build system since I don't have expertise there!
github-actions
bot
added
awaiting merge
|
Thanks! As all checks pass now it seems that it was a flakey test |
github-actions
bot
added
awaiting merge
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
|
@github-actions crossbow submit r-binary-packages |
|
Revision: eca2134 Submitted crossbow builds: ursacomputing/crossbow @ actions-09ced6cc0b
|
|
After merging your PR, Conbench analyzed the 6 benchmarking runs that have been run so far on merge-commit dd8734d. There were 3 benchmark results indicating a performance regression:
The full Conbench report has more details. It also includes information about 3 possible false positives for unstable benchmarks that are known to sometimes produce them. |
JerAguilon
pushed a commit
to JerAguilon/arrow
that referenced
this pull request
Oct 23, 2023
This PR modifies the build system of the R package to no longer rely on auto/homebrew. Instead this PR adds the infrastructure and code paths to use the same type of pre-compiled libarrow binaries as we use for Linux. The main difference is the use of the binaries even on CRAN (as we previously also used binaries in form of brew bottles). The addition of the new artifacts to tasks.yml should ensure that they get uploaded to the nightly repo as well as to the artifactory during the release (@ kou please confirm). A summary of the changes in this PR: - update `r/configure` and `r/tools/nixlibs.R` to enable the source build on macOS and usage of precompiled binaries using the existing mechanism to test compile a program to detect the exisitng openssl version - added tests for the changes in nixlibs.R - update the binary allow-list - Add the build jobs for libarrow binaries for arm64 and x86_64 macos with openssl 1.1 and 3.0 to the `r-binary-packages` job - Use the binaries to build the nightly packages - bump snappy version to 1.1.10 (and patch it on 10.13) due to build issues with the current version. This also touches on a number of issues in regards to a sanitizer issue we have had for a long time: apache#32562 apache#31766 - Disable the centos binary test step: apache#37922 Follow up issues: - apache#37921 - apache#37941 - apache#37945 * Closes: apache#37923 Lead-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de> Co-authored-by: Jonathan Keane <jkeane@gmail.com> Co-authored-by: Sutou Kouhei <kou@cozmixng.org> Signed-off-by: Dewey Dunnington <dewey@voltrondata.com>
loicalleyne
pushed a commit
to loicalleyne/arrow
that referenced
this pull request
Nov 13, 2023
This PR modifies the build system of the R package to no longer rely on auto/homebrew. Instead this PR adds the infrastructure and code paths to use the same type of pre-compiled libarrow binaries as we use for Linux. The main difference is the use of the binaries even on CRAN (as we previously also used binaries in form of brew bottles). The addition of the new artifacts to tasks.yml should ensure that they get uploaded to the nightly repo as well as to the artifactory during the release (@ kou please confirm). A summary of the changes in this PR: - update `r/configure` and `r/tools/nixlibs.R` to enable the source build on macOS and usage of precompiled binaries using the existing mechanism to test compile a program to detect the exisitng openssl version - added tests for the changes in nixlibs.R - update the binary allow-list - Add the build jobs for libarrow binaries for arm64 and x86_64 macos with openssl 1.1 and 3.0 to the `r-binary-packages` job - Use the binaries to build the nightly packages - bump snappy version to 1.1.10 (and patch it on 10.13) due to build issues with the current version. This also touches on a number of issues in regards to a sanitizer issue we have had for a long time: apache#32562 apache#31766 - Disable the centos binary test step: apache#37922 Follow up issues: - apache#37921 - apache#37941 - apache#37945 * Closes: apache#37923 Lead-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de> Co-authored-by: Jonathan Keane <jkeane@gmail.com> Co-authored-by: Sutou Kouhei <kou@cozmixng.org> Signed-off-by: Dewey Dunnington <dewey@voltrondata.com>
dgreiss
pushed a commit
to dgreiss/arrow
that referenced
this pull request
Feb 19, 2024
This PR modifies the build system of the R package to no longer rely on auto/homebrew. Instead this PR adds the infrastructure and code paths to use the same type of pre-compiled libarrow binaries as we use for Linux. The main difference is the use of the binaries even on CRAN (as we previously also used binaries in form of brew bottles). The addition of the new artifacts to tasks.yml should ensure that they get uploaded to the nightly repo as well as to the artifactory during the release (@ kou please confirm). A summary of the changes in this PR: - update `r/configure` and `r/tools/nixlibs.R` to enable the source build on macOS and usage of precompiled binaries using the existing mechanism to test compile a program to detect the exisitng openssl version - added tests for the changes in nixlibs.R - update the binary allow-list - Add the build jobs for libarrow binaries for arm64 and x86_64 macos with openssl 1.1 and 3.0 to the `r-binary-packages` job - Use the binaries to build the nightly packages - bump snappy version to 1.1.10 (and patch it on 10.13) due to build issues with the current version. This also touches on a number of issues in regards to a sanitizer issue we have had for a long time: apache#32562 apache#31766 - Disable the centos binary test step: apache#37922 Follow up issues: - apache#37921 - apache#37941 - apache#37945 * Closes: apache#37923 Lead-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de> Co-authored-by: Jonathan Keane <jkeane@gmail.com> Co-authored-by: Sutou Kouhei <kou@cozmixng.org> Signed-off-by: Dewey Dunnington <dewey@voltrondata.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR modifies the build system of the R package to no longer rely on auto/homebrew. Instead this PR adds the infrastructure and code paths to use the same type of pre-compiled libarrow binaries as we use for Linux. The main difference is the use of the binaries even on CRAN (as we previously also used binaries in form of brew bottles).
The addition of the new artifacts to tasks.yml should ensure that they get uploaded to the nightly repo as well as to the artifactory during the release (@kou please confirm).
A summary of the changes in this PR:
r/configureandr/tools/nixlibs.Rto enable the source build on macOS and usage of precompiled binaries using the existing mechanism to test compile a program to detect the exisitng openssl versionr-binary-packagesjobFollow up issues: