Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[R] Move macOS build system to nixlibs.R #37923

Closed
assignUser opened this issue Sep 28, 2023 · 0 comments · Fixed by #37684
Closed

[R] Move macOS build system to nixlibs.R #37923

assignUser opened this issue Sep 28, 2023 · 0 comments · Fixed by #37684
Assignees
@assignUser
Labels
Component: R Priority: Blocker Marks a blocker for the release Type: enhancement
Milestone
14.0.0

Comments

@assignUser
Copy link
Member

Describe the enhancement requested

This will allow us to no longer rely on homebrew/autobrew to get the pre-compiled binaries for the R package build on macos. Instead we can utilize the same infrastructure used for the windows and linux binaries (artifactory/nightlies.a.o).

Component(s)

R
@github-actions github-actions bot mentioned this issue Sep 28, 2023
Merged
@assignUser assignUser added this to the 14.0.0 milestone Sep 28, 2023
@assignUser assignUser added the Priority: Blocker Marks a blocker for the release label Sep 28, 2023
This was referenced Sep 29, 2023
Closed
Closed
paleolimbot pushed a commit that referenced this issue Oct 5, 2023
@assignUser @jonkeane @kou
GH-37923: [R] Move macOS build system to nixlibs.R (#37684)
dd8734d 
This PR modifies the build system of the R package to no longer rely on auto/homebrew. Instead this PR adds the infrastructure and code paths to use the same type of  pre-compiled libarrow binaries as we use for Linux. The main difference is the use of the binaries even on CRAN (as we previously also used binaries in form of brew bottles).

The addition of the new artifacts to tasks.yml should ensure that they get uploaded to the nightly repo as well as to the artifactory during the release (@ kou please confirm). 

A summary of the changes in this PR:
- update `r/configure` and `r/tools/nixlibs.R` to enable the source build on macOS and usage of precompiled binaries using the existing mechanism to test compile a program to detect the exisitng openssl version
- added tests for the changes in nixlibs.R
- update the binary allow-list
- Add the build jobs for libarrow binaries for arm64 and x86_64 macos with openssl 1.1 and 3.0 to the `r-binary-packages` job
- Use the binaries to build the nightly packages  
- bump snappy version to 1.1.10 (and patch it on 10.13) due to build issues with the current version. This also touches on a number of issues in regards to a sanitizer issue we have had for a long time: #32562 #31766
- Disable the centos binary test step: #37922

Follow up issues:
- #37921
- #37941 
- #37945
* Closes: #37923

Lead-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Co-authored-by: Jonathan Keane <jkeane@gmail.com>
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Signed-off-by: Dewey Dunnington <dewey@voltrondata.com>
@assignUser assignUser mentioned this issue Oct 5, 2023
Closed
2 tasks
JerAguilon pushed a commit to JerAguilon/arrow that referenced this issue Oct 23, 2023
@assignUser @jonkeane @kou
apacheGH-37923: [R] Move macOS build system to nixlibs.R (apache#37684)
433aea0 
This PR modifies the build system of the R package to no longer rely on auto/homebrew. Instead this PR adds the infrastructure and code paths to use the same type of  pre-compiled libarrow binaries as we use for Linux. The main difference is the use of the binaries even on CRAN (as we previously also used binaries in form of brew bottles).

The addition of the new artifacts to tasks.yml should ensure that they get uploaded to the nightly repo as well as to the artifactory during the release (@ kou please confirm). 

A summary of the changes in this PR:
- update `r/configure` and `r/tools/nixlibs.R` to enable the source build on macOS and usage of precompiled binaries using the existing mechanism to test compile a program to detect the exisitng openssl version
- added tests for the changes in nixlibs.R
- update the binary allow-list
- Add the build jobs for libarrow binaries for arm64 and x86_64 macos with openssl 1.1 and 3.0 to the `r-binary-packages` job
- Use the binaries to build the nightly packages  
- bump snappy version to 1.1.10 (and patch it on 10.13) due to build issues with the current version. This also touches on a number of issues in regards to a sanitizer issue we have had for a long time: apache#32562 apache#31766
- Disable the centos binary test step: apache#37922

Follow up issues:
- apache#37921
- apache#37941 
- apache#37945
* Closes: apache#37923

Lead-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Co-authored-by: Jonathan Keane <jkeane@gmail.com>
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Signed-off-by: Dewey Dunnington <dewey@voltrondata.com>
loicalleyne pushed a commit to loicalleyne/arrow that referenced this issue Nov 13, 2023
@assignUser @jonkeane
@kou @loicalleyne
apacheGH-37923: [R] Move macOS build system to nixlibs.R (apache#37684)
4503fb0 
This PR modifies the build system of the R package to no longer rely on auto/homebrew. Instead this PR adds the infrastructure and code paths to use the same type of  pre-compiled libarrow binaries as we use for Linux. The main difference is the use of the binaries even on CRAN (as we previously also used binaries in form of brew bottles).

The addition of the new artifacts to tasks.yml should ensure that they get uploaded to the nightly repo as well as to the artifactory during the release (@ kou please confirm). 

A summary of the changes in this PR:
- update `r/configure` and `r/tools/nixlibs.R` to enable the source build on macOS and usage of precompiled binaries using the existing mechanism to test compile a program to detect the exisitng openssl version
- added tests for the changes in nixlibs.R
- update the binary allow-list
- Add the build jobs for libarrow binaries for arm64 and x86_64 macos with openssl 1.1 and 3.0 to the `r-binary-packages` job
- Use the binaries to build the nightly packages  
- bump snappy version to 1.1.10 (and patch it on 10.13) due to build issues with the current version. This also touches on a number of issues in regards to a sanitizer issue we have had for a long time: apache#32562 apache#31766
- Disable the centos binary test step: apache#37922

Follow up issues:
- apache#37921
- apache#37941 
- apache#37945
* Closes: apache#37923

Lead-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Co-authored-by: Jonathan Keane <jkeane@gmail.com>
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Signed-off-by: Dewey Dunnington <dewey@voltrondata.com>
dgreiss pushed a commit to dgreiss/arrow that referenced this issue Feb 19, 2024
@assignUser @jonkeane
@kou @dgreiss
apacheGH-37923: [R] Move macOS build system to nixlibs.R (apache#37684)
7125ca1 
This PR modifies the build system of the R package to no longer rely on auto/homebrew. Instead this PR adds the infrastructure and code paths to use the same type of  pre-compiled libarrow binaries as we use for Linux. The main difference is the use of the binaries even on CRAN (as we previously also used binaries in form of brew bottles).

The addition of the new artifacts to tasks.yml should ensure that they get uploaded to the nightly repo as well as to the artifactory during the release (@ kou please confirm). 

A summary of the changes in this PR:
- update `r/configure` and `r/tools/nixlibs.R` to enable the source build on macOS and usage of precompiled binaries using the existing mechanism to test compile a program to detect the exisitng openssl version
- added tests for the changes in nixlibs.R
- update the binary allow-list
- Add the build jobs for libarrow binaries for arm64 and x86_64 macos with openssl 1.1 and 3.0 to the `r-binary-packages` job
- Use the binaries to build the nightly packages  
- bump snappy version to 1.1.10 (and patch it on 10.13) due to build issues with the current version. This also touches on a number of issues in regards to a sanitizer issue we have had for a long time: apache#32562 apache#31766
- Disable the centos binary test step: apache#37922

Follow up issues:
- apache#37921
- apache#37941 
- apache#37945
* Closes: apache#37923

Lead-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Co-authored-by: Jonathan Keane <jkeane@gmail.com>
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Signed-off-by: Dewey Dunnington <dewey@voltrondata.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@assignUser assignUser

Labels
Component: R Priority: Blocker Marks a blocker for the release Type: enhancement
Projects
None yet
Milestone
14.0.0
Development

Successfully merging a pull request may close this issue.

GH-37923: [R] Move macOS build system to nixlibs.R assignUser/arrow
1 participant
@assignUser