Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-38079: [R] Allow override of version used to download libarrow static library #38080

Closed
wants to merge 2 commits into from

Conversation

paleolimbot
Copy link
Member

@paleolimbot paleolimbot commented Oct 6, 2023

Rationale for this change

Doing a bundled build of libarrow is time-consuming and makes casual contribution difficult. Downloading a recent nightly static library is sufficient in most cases; however, there is no route to override the version used to do this download.

What changes are included in this PR?

Adds an environment variable LIBARROW_BINARY_VERSION that can be used to override the VERSION for the purposes of downloading the binary.

Are these changes tested?

Not yet! Working on the best way to do this.

Are there any user-facing changes?

No (this is currently just a developer-facing tool).

@github-actions
Copy link

github-actions bot commented Oct 6, 2023

⚠️ GitHub issue #35049 has been automatically assigned in GitHub to PR creator.

@github-actions github-actions bot added Component: R awaiting committer review Awaiting committer review labels Oct 6, 2023
@paleolimbot paleolimbot changed the title GH-35049: [R] Allow override of version used to download libarrow static library GH-38079: [R] Allow override of version used to download libarrow static library Oct 6, 2023
@github-actions
Copy link

github-actions bot commented Oct 6, 2023

⚠️ GitHub issue #38079 has been automatically assigned in GitHub to PR creator.

@paleolimbot
Copy link
Member Author

Tested on MacOS 10.13 using

LIBARROW_BINARY_VERSION=13.0.0.100000258 LIBARROW_BINARY=true ARROW_R_DEV=true R CMD INSTALL .

Build log:

* installing to library ‘/Library/Frameworks/R.framework/Versions/4.2/Resources/library’
* installing *source* package ‘arrow’ ...
** using staged installation
*** Generating code with data-raw/codegen.R
*** > 542 functions decorated with [[arrow|acero|dataset|substrait|parquet|s3|gcs|json::export]]
*** > `src/arrowExports.cpp` not modified
*** > `R/arrowExports.R` not modified
*** Found libcurl and OpenSSL >= 3.0.0
trying URL 'https://nightlies.apache.org/arrow/r/libarrow/bin/darwin-x86_64-openssl-3.0/arrow-13.0.0.100000258.zip'
Content type 'application/zip' length 23952616 bytes (22.8 MB)
==================================================
downloaded 22.8 MB

*** Successfully retrieved C++ binaries (darwin-x86_64-openssl-3.0)
Warning message:
In dir.create(dst_dir, showWarnings = !quietly, recursive = TRUE) :
  'libarrow/arrow-13.0.0.9000' already exists
5
PKG_CFLAGS=-I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY
PKG_LIBS=-L/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/lib -larrow_acero -larrow_dataset -lparquet  -larrow -larrow_bundled_dependencies  -lcurl -lssl -lcrypto -lcurl -lssl -lcrypto
** libs
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c RTasks.cpp -o RTasks.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c altrep.cpp -o altrep.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c array.cpp -o array.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c array_to_vector.cpp -o array_to_vector.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c arraydata.cpp -o arraydata.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c arrowExports.cpp -o arrowExports.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c bridge.cpp -o bridge.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c buffer.cpp -o buffer.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c chunkedarray.cpp -o chunkedarray.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c compression.cpp -o compression.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c compute-exec.cpp -o compute-exec.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c compute.cpp -o compute.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c config.cpp -o config.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c csv.cpp -o csv.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c dataset.cpp -o dataset.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c datatype.cpp -o datatype.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c expression.cpp -o expression.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c extension-impl.cpp -o extension-impl.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c feather.cpp -o feather.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c field.cpp -o field.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c filesystem.cpp -o filesystem.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c io.cpp -o io.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c json.cpp -o json.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c memorypool.cpp -o memorypool.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c message.cpp -o message.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c parquet.cpp -o parquet.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c r_to_arrow.cpp -o r_to_arrow.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c recordbatch.cpp -o recordbatch.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c recordbatchreader.cpp -o recordbatchreader.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c recordbatchwriter.cpp -o recordbatchwriter.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c safe-call-into-r-impl.cpp -o safe-call-into-r-impl.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c scalar.cpp -o scalar.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c schema.cpp -o schema.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c symbols.cpp -o symbols.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c table.cpp -o table.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c threadpool.cpp -o threadpool.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include   -fPIC  -D_LIBCPP_DISABLE_AVAILABILITY -c type_infer.cpp -o type_infer.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o arrow.so RTasks.o altrep.o array.o array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o expression.o extension-impl.o feather.o field.o filesystem.o io.o json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o schema.o symbols.o table.o threadpool.o type_infer.o -L/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/lib -larrow_acero -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lcurl -lssl -lcrypto -lcurl -lssl -lcrypto -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
ld: warning: text-based stub file /System/Library/Frameworks//CoreFoundation.framework/CoreFoundation.tbd and library file /System/Library/Frameworks//CoreFoundation.framework/CoreFoundation are out of sync. Falling back to library file for linking.
installing to /Library/Frameworks/R.framework/Versions/4.2/Resources/library/00LOCK-r/00new/arrow/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (arrow)

...and we get the package built with S3:

> arrow::arrow_info()
Arrow package version: 13.0.0.9000

Capabilities:
               
acero      TRUE
dataset    TRUE
substrait FALSE
parquet    TRUE
json       TRUE
s3         TRUE
gcs        TRUE
utf8proc   TRUE
re2        TRUE
snappy     TRUE
gzip       TRUE
brotli     TRUE
zstd       TRUE
lz4        TRUE
lz4_frame  TRUE
lzo       FALSE
bz2        TRUE
jemalloc   TRUE
mimalloc   TRUE

Memory:
                  
Allocator mimalloc
Current    0 bytes
Max        0 bytes

Runtime:
                        
SIMD Level          avx2
Detected SIMD Level avx2

Build:
                                                             
C++ Library Version                           14.0.0-SNAPSHOT
C++ Compiler                                       AppleClang
C++ Compiler Version                          10.0.0.10001044
Git ID               e1d7459bce3f10c941cec1665b15d3388cbe5adb

@paleolimbot
Copy link
Member Author

Tested on Windows, too. The easiest way to get it there is to usethis::edit_r_environ() and add LIBARROW_BINARY_VERSION=13.0.0.100000258.

@paleolimbot paleolimbot marked this pull request as ready for review October 9, 2023 01:40
@paleolimbot
Copy link
Member Author

@assignUser is this fix worth merging? I do need this or something like it to debug the Windows issues we've been seeing. An alternative would be to go straight for LIBARROW_BINARY_URI (which could be file:// as well), which would also work for my personal degugging needs.

@assignUser
Copy link
Member

I like the idea but this will cause conflicts with #38115 but as that is done (imo) I will start on the 'get latest nightly binary'-feature now were I can integrate this.

For your local test we do have ARROW_DOWNLOADED_BINARIES and RWINLIB_LOCAL which can be used to build with a downloaded libarrow zip file, not a 1:1 match as you have to download manually once but pretty close. Would that work for you for now?

assignUser pushed a commit that referenced this pull request Oct 9, 2023
### Rationale for this change

We no longer use autobrew to install on MacOS, so we no longer need CI jobs or helper scripts that had previously been referenced from this install path.

### What changes are included in this PR?

This PR removes references to autobrew that are no longer needed. It also removes the R homebrew CI job, which has been failing for several months. We had supported homebrew because `brew install apache-arrow --HEAD` was one method of avoiding a full binary install for potential MacOS contributors; however, downloading nightly static libraries (e.g., #38080) is an even faster way to get a compatible libarrow built for the R package and works on MacOS, Windows, and Linux.

### Are these changes tested?

These changes remove tests and infrastructure that are no longer relevant.

### Are there any user-facing changes?

No.
* Closes: #38088

Authored-by: Dewey Dunnington <dewey@voltrondata.com>
Signed-off-by: Jacob Wujciak-Jens <jacob@wujciak.de>
@paleolimbot
Copy link
Member Author

Got it! I will rework this once there's a pre-calculatable URL for "the last one". I like URL better than a local file (because URL can be either) but the right way to do that would be to replace the existing "local file" option and unify it under "libarrow zip url".

@paleolimbot
Copy link
Member Author

Closing as #38236 is close to completion!

@assignUser assignUser closed this Oct 20, 2023
JerAguilon pushed a commit to JerAguilon/arrow that referenced this pull request Oct 23, 2023
…pache#38089)

### Rationale for this change

We no longer use autobrew to install on MacOS, so we no longer need CI jobs or helper scripts that had previously been referenced from this install path.

### What changes are included in this PR?

This PR removes references to autobrew that are no longer needed. It also removes the R homebrew CI job, which has been failing for several months. We had supported homebrew because `brew install apache-arrow --HEAD` was one method of avoiding a full binary install for potential MacOS contributors; however, downloading nightly static libraries (e.g., apache#38080) is an even faster way to get a compatible libarrow built for the R package and works on MacOS, Windows, and Linux.

### Are these changes tested?

These changes remove tests and infrastructure that are no longer relevant.

### Are there any user-facing changes?

No.
* Closes: apache#38088

Authored-by: Dewey Dunnington <dewey@voltrondata.com>
Signed-off-by: Jacob Wujciak-Jens <jacob@wujciak.de>
loicalleyne pushed a commit to loicalleyne/arrow that referenced this pull request Nov 13, 2023
…pache#38089)

### Rationale for this change

We no longer use autobrew to install on MacOS, so we no longer need CI jobs or helper scripts that had previously been referenced from this install path.

### What changes are included in this PR?

This PR removes references to autobrew that are no longer needed. It also removes the R homebrew CI job, which has been failing for several months. We had supported homebrew because `brew install apache-arrow --HEAD` was one method of avoiding a full binary install for potential MacOS contributors; however, downloading nightly static libraries (e.g., apache#38080) is an even faster way to get a compatible libarrow built for the R package and works on MacOS, Windows, and Linux.

### Are these changes tested?

These changes remove tests and infrastructure that are no longer relevant.

### Are there any user-facing changes?

No.
* Closes: apache#38088

Authored-by: Dewey Dunnington <dewey@voltrondata.com>
Signed-off-by: Jacob Wujciak-Jens <jacob@wujciak.de>
dgreiss pushed a commit to dgreiss/arrow that referenced this pull request Feb 19, 2024
…pache#38089)

### Rationale for this change

We no longer use autobrew to install on MacOS, so we no longer need CI jobs or helper scripts that had previously been referenced from this install path.

### What changes are included in this PR?

This PR removes references to autobrew that are no longer needed. It also removes the R homebrew CI job, which has been failing for several months. We had supported homebrew because `brew install apache-arrow --HEAD` was one method of avoiding a full binary install for potential MacOS contributors; however, downloading nightly static libraries (e.g., apache#38080) is an even faster way to get a compatible libarrow built for the R package and works on MacOS, Windows, and Linux.

### Are these changes tested?

These changes remove tests and infrastructure that are no longer relevant.

### Are there any user-facing changes?

No.
* Closes: apache#38088

Authored-by: Dewey Dunnington <dewey@voltrondata.com>
Signed-off-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[R] Allow overriding the libarrow static lib version
2 participants