Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a libgdal-arrow-parquet package #679

Merged
merged 24 commits into from
Dec 10, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
ce3276e
Add a libgdal-arrow-parquet package
rouault Nov 18, 2022
416dae1
update depends s/parquet-cpp/arrow-cpp
akrherz Dec 4, 2022
1bc4ddc
bump build number
akrherz Dec 4, 2022
3af0cd4
MNT: Re-rendered with conda-build 3.23.2, conda-smithy 3.22.0, and co…
Dec 4, 2022
78f886f
meta.yaml: add arrow-cpp >=10.0 constraint
rouault Dec 4, 2022
0e50716
MNT: Re-rendered with conda-build 3.23.2, conda-smithy 3.22.0, and co…
Dec 4, 2022
f008474
meta.yaml: swith to libarrow
rouault Dec 4, 2022
b03276a
MNT: Re-rendered with conda-build 3.23.2, conda-smithy 3.22.0, and co…
Dec 4, 2022
b12e3c5
recipe/meta.yaml: add @h-vetinari suggestions
rouault Dec 5, 2022
37e5878
MNT: Re-rendered with conda-build 3.23.2, conda-smithy 3.22.0, and co…
Dec 5, 2022
6833adc
manually add arrow_cpp10 migrator
h-vetinari Dec 7, 2022
0291c5f
remove arrow from global build deps; build new output for all arrow v…
h-vetinari Dec 8, 2022
f264909
MNT: Re-rendered with conda-build 3.23.3, conda-smithy 3.22.1, and co…
h-vetinari Dec 8, 2022
9c465fd
Merge pull request #1 from h-vetinari/arrow_parquet
rouault Dec 9, 2022
5a71951
Update recipe/meta.yaml
rouault Dec 9, 2022
9ab0faa
Update recipe/meta.yaml
rouault Dec 9, 2022
23e4694
Merge remote-tracking branch 'origin/main' into arrow_parquet
rouault Dec 9, 2022
32557d8
bump build number
rouault Dec 9, 2022
3a9796b
meta.yaml: arrow-cpp -> arrow_cpp
rouault Dec 9, 2022
65683e1
Update recipe/meta.yaml
rouault Dec 9, 2022
2458310
.ci_support: try to remove curl: 7
rouault Dec 9, 2022
0d56330
.ci_support: try to remove mentions to arrow-cpp in openssl3 files
rouault Dec 9, 2022
e2ec50c
Update recipe/meta.yaml
rouault Dec 9, 2022
cd13343
MNT: Re-rendered with conda-build 3.23.3, conda-smithy 3.22.1, and co…
Dec 9, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .azure-pipelines/azure-pipelines-win.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 7 additions & 0 deletions .ci_support/linux_64_openssl1.1.1.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
arrow_cpp:
- 10.0.1
- 7.0.1
- 8.0.1
- 9.0.0
c_compiler:
- gcc
c_compiler_version:
Expand All @@ -10,6 +15,8 @@ channel_sources:
- conda-forge
channel_targets:
- conda-forge main
curl:
- '7'
cxx_compiler:
- gxx
cxx_compiler_version:
Expand Down
7 changes: 7 additions & 0 deletions .ci_support/linux_64_openssl3.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
arrow_cpp:
- 10.0.1
- 7.0.1
- 8.0.1
- 9.0.0
c_compiler:
- gcc
c_compiler_version:
Expand All @@ -10,6 +15,8 @@ channel_sources:
- conda-forge
channel_targets:
- conda-forge main
curl:
- '7'
cxx_compiler:
- gxx
cxx_compiler_version:
Expand Down
7 changes: 7 additions & 0 deletions .ci_support/linux_aarch64_openssl1.1.1.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
BUILD:
- aarch64-conda_cos7-linux-gnu
arrow_cpp:
- 10.0.1
- 7.0.1
- 8.0.1
- 9.0.0
c_compiler:
- gcc
c_compiler_version:
Expand All @@ -14,6 +19,8 @@ channel_sources:
- conda-forge
channel_targets:
- conda-forge main
curl:
- '7'
cxx_compiler:
- gxx
cxx_compiler_version:
Expand Down
7 changes: 7 additions & 0 deletions .ci_support/linux_aarch64_openssl3.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
BUILD:
- aarch64-conda_cos7-linux-gnu
arrow_cpp:
- 10.0.1
- 7.0.1
- 8.0.1
- 9.0.0
c_compiler:
- gcc
c_compiler_version:
Expand All @@ -14,6 +19,8 @@ channel_sources:
- conda-forge
channel_targets:
- conda-forge main
curl:
- '7'
cxx_compiler:
- gxx
cxx_compiler_version:
Expand Down
7 changes: 7 additions & 0 deletions .ci_support/linux_ppc64le_openssl1.1.1.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
arrow_cpp:
- 10.0.1
- 7.0.1
- 8.0.1
- 9.0.0
c_compiler:
- gcc
c_compiler_version:
Expand All @@ -10,6 +15,8 @@ channel_sources:
- conda-forge
channel_targets:
- conda-forge main
curl:
- '7'
cxx_compiler:
- gxx
cxx_compiler_version:
Expand Down
7 changes: 7 additions & 0 deletions .ci_support/linux_ppc64le_openssl3.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
arrow_cpp:
- 10.0.1
- 7.0.1
- 8.0.1
- 9.0.0
c_compiler:
- gcc
c_compiler_version:
Expand All @@ -10,6 +15,8 @@ channel_sources:
- conda-forge
channel_targets:
- conda-forge main
curl:
- '7'
cxx_compiler:
- gxx
cxx_compiler_version:
Expand Down
17 changes: 17 additions & 0 deletions .ci_support/migrations/arrow_cpp10.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
__migrator:
build_number: 1
kind: version
migration_number: 1
arrow_cpp:
- 10.0.1
- 9.0.0
- 8.0.1
- 7.0.1
# make sure we have same length for zip, even
# though libarrow builds only exist since 10.x
libarrow:
- 10.0.1
- 9.0.0
- 8.0.1
- 7.0.1
migrator_ts: 1670146930.791868
7 changes: 0 additions & 7 deletions .ci_support/migrations/geos3111.yaml

This file was deleted.

7 changes: 7 additions & 0 deletions .ci_support/osx_64_openssl1.1.1.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
MACOSX_DEPLOYMENT_TARGET:
- '10.9'
arrow_cpp:
- 10.0.1
- 7.0.1
- 8.0.1
- 9.0.0
c_compiler:
- clang
c_compiler_version:
Expand All @@ -10,6 +15,8 @@ channel_sources:
- conda-forge
channel_targets:
- conda-forge main
curl:
- '7'
cxx_compiler:
- clangxx
cxx_compiler_version:
Expand Down
7 changes: 7 additions & 0 deletions .ci_support/osx_64_openssl3.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
MACOSX_DEPLOYMENT_TARGET:
- '10.9'
arrow_cpp:
- 10.0.1
- 7.0.1
- 8.0.1
- 9.0.0
c_compiler:
- clang
c_compiler_version:
Expand All @@ -10,6 +15,8 @@ channel_sources:
- conda-forge
channel_targets:
- conda-forge main
curl:
- '7'
cxx_compiler:
- clangxx
cxx_compiler_version:
Expand Down
7 changes: 7 additions & 0 deletions .ci_support/osx_arm64_openssl1.1.1.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
MACOSX_DEPLOYMENT_TARGET:
- '11.0'
arrow_cpp:
- 10.0.1
- 7.0.1
- 8.0.1
- 9.0.0
c_compiler:
- clang
c_compiler_version:
Expand All @@ -10,6 +15,8 @@ channel_sources:
- conda-forge
channel_targets:
- conda-forge main
curl:
- '7'
cxx_compiler:
- clangxx
cxx_compiler_version:
Expand Down
7 changes: 7 additions & 0 deletions .ci_support/osx_arm64_openssl3.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
MACOSX_DEPLOYMENT_TARGET:
- '11.0'
arrow_cpp:
- 10.0.1
- 7.0.1
- 8.0.1
- 9.0.0
c_compiler:
- clang
c_compiler_version:
Expand All @@ -10,6 +15,8 @@ channel_sources:
- conda-forge
channel_targets:
- conda-forge main
curl:
- '7'
cxx_compiler:
- clangxx
cxx_compiler_version:
Expand Down
7 changes: 7 additions & 0 deletions .ci_support/win_64_openssl1.1.1.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
arrow_cpp:
- 10.0.1
- 7.0.1
- 8.0.1
- 9.0.0
c_compiler:
- vs2019
cfitsio:
Expand All @@ -6,6 +11,8 @@ channel_sources:
- conda-forge
channel_targets:
- conda-forge main
curl:
- '7'
cxx_compiler:
- vs2019
expat:
Expand Down
7 changes: 7 additions & 0 deletions .ci_support/win_64_openssl3.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
arrow_cpp:
- 10.0.1
- 7.0.1
- 8.0.1
- 9.0.0
c_compiler:
- vs2019
cfitsio:
Expand All @@ -6,6 +11,8 @@ channel_sources:
- conda-forge
channel_targets:
- conda-forge main
curl:
- '7'
cxx_compiler:
- vs2019
expat:
Expand Down
7 changes: 4 additions & 3 deletions README.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
From eee1b439437788e9ecb65b600b8cd33d2012be32 Mon Sep 17 00:00:00 2001
From: Even Rouault <even.rouault@spatialys.com>
Date: Fri, 18 Nov 2022 02:09:18 +0100
Subject: [PATCH] Parquet: make sure that ArrowLayer destructor is available
(for plugin)

---
ogr/ogrsf_frmts/parquet/ogrparquetdriver.cpp | 1 +
1 file changed, 1 insertion(+)

diff --git a/ogr/ogrsf_frmts/parquet/ogrparquetdriver.cpp b/ogr/ogrsf_frmts/parquet/ogrparquetdriver.cpp
index ff112586ed..3303f999f4 100644
--- a/ogr/ogrsf_frmts/parquet/ogrparquetdriver.cpp
+++ b/ogr/ogrsf_frmts/parquet/ogrparquetdriver.cpp
@@ -35,6 +35,7 @@
#include "../arrow_common/ograrrowrandomaccessfile.h"
#include "../arrow_common/ograrrowwritablefile.h"
#include "../arrow_common/ograrrowdataset.hpp"
+#include "../arrow_common/ograrrowlayer.hpp" // for the destructor

/************************************************************************/
/* Identify() */
--
2.25.1

6 changes: 6 additions & 0 deletions recipe/bld.bat
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@ if %vc% GTR 9 set MSVC_VER=1900
if %vc% LEQ 9 set MSVC_TS_VER=90
if %vc% GTR 9 set MSVC_TS_VER=140

REM Make sure to disable Arrow/Parquet dependencies for now, so they are only
REM used in build_arrow_parquet

cmake -G "Ninja" ^
"%CMAKE_ARGS%" ^
-DMSVC_VERSION="%MSVC_VER%" ^
Expand All @@ -25,6 +28,9 @@ cmake -G "Ninja" ^
-DKEA_LIBRARY:PATH="%LIBRARY_LIB%\libkea.lib" ^
-DGDAL_USE_MYSQL:BOOL=OFF ^
-DGDAL_USE_MSSQL_ODBC:BOOL=OFF ^
-DGDAL_USE_PARQUET=OFF ^
-DGDAL_USE_ARROW=OFF ^
-DGDAL_USE_ARROWDATASET=OFF ^
"%SRC_DIR%"

if errorlevel 1 exit /b 1
Expand Down
5 changes: 5 additions & 0 deletions recipe/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,18 @@ export CXXFLAGS="${CXXFLAGS} -std=c++17 -D_LIBCPP_DISABLE_AVAILABILITY"
mkdir build
cd build

# Make sure to disable Arrow/Parquet dependencies for now, so they are only
# used in build_arrow_parquet
cmake -G "Unix Makefiles" \
${CMAKE_ARGS} \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_PREFIX_PATH=$PREFIX \
-DCMAKE_INSTALL_PREFIX=$PREFIX \
-DCMAKE_INSTALL_LIBDIR=lib \
-DBUILD_SHARED_LIBS=ON \
-DGDAL_USE_PARQUET=OFF \
-DGDAL_USE_ARROW=OFF \
-DGDAL_USE_ARROWDATASET=OFF \
-DBUILD_PYTHON_BINDINGS:BOOL=OFF \
-DBUILD_JAVA_BINDINGS:BOOL=OFF \
-DBUILD_CSHARP_BINDINGS:BOOL=OFF \
Expand Down
22 changes: 22 additions & 0 deletions recipe/build_arrow_parquet.bat
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
cd build
if errorlevel 1 exit 1

REM We reuse the same build directory as libgdal, so we just to have to
REM turn on the required dependency and drivers
cmake -DBUILD_PYTHON_BINDINGS:BOOL=OFF ^
-DGDAL_USE_PARQUET=ON ^
-DGDAL_USE_ARROW=ON ^
-DGDAL_USE_ARROWDATASET=ON ^
-DOGR_ENABLE_DRIVER_ARROW=ON ^
-DOGR_ENABLE_DRIVER_ARROW_PLUGIN=ON ^
-DOGR_ENABLE_DRIVER_PARQUET=ON ^
-DOGR_ENABLE_DRIVER_PARQUET_PLUGIN=ON ^
"%SRC_DIR%"

if errorlevel 1 exit /b 1

cmake --build . -j %CPU_COUNT% --verbose --config Release
if errorlevel 1 exit /b 1

cmake --build . --target install --config Release
if errorlevel 1 exit /b 1
Loading