Skip to content

Commit

Permalink
[GDAL] switch build to CMake (#5423)
Browse files Browse the repository at this point in the history 
* switch build over to CMake

The GCC version is increased to 8 since the Windows build hit an internal compiler error:

```
[20:26:11] In function ‘CPLErr GDALResampleChunk32R_ConvolutionT(double, double, double, double, const T*, int, const GByte*, int, int, int, int, int, int, int, int, GDALRasterBand*, void*, int, float, FilterFuncType, FilterFunc4ValuesType, int, bool, float) [with T = unsigned char]’:
[20:26:11] cc1plus: internal compiler error: in gimplify_modify_expr, at gimplify.c:5636
[20:26:11] [ 34%] Building CXX object ogr/ogrsf_frmts/mssqlspatial/CMakeFiles/ogr_MSSQLSpatial.dir/ogrmssqlgeometryparser.cpp.obj
[20:26:11] cd /workspace/srcdir/gdal-3.5.1/build/ogr/ogrsf_frmts/mssqlspatial && /opt/bin/x86_64-w64-mingw32-libgfortran4-cxx11/x86_64-w64-mingw32-g++ --sysroot=/opt/x86_64-w64-mingw32/x86_64-w64-mingw32/sys-root/ -DDEBUG -DDONT_DEPRECATE_SPRINTF -DGDAL_CMAKE_BUILD -DGDAL_COMPILATION -DHAVE_AVX_AT_COMPILE_TIME -DHAVE_SSE_AT_COMPILE_TIME -DHAVE_SSSE3_AT_COMPILE_TIME @CMakeFiles/ogr_MSSQLSpatial.dir/includes_CXX.rsp -fvisibility=hidden -g -Os -Wall -Wextra -Winit-self -Wunused-parameter -Wmissing-declarations -Wlogical-op -Wshadow -Wmissing-include-dirs -Wformat -Werror=format-security -Wno-format-nonliteral -Werror=vla -Wno-clobbered -Wdate-time -Wnull-dereference -Wfloat-conversion -Wnon-virtual-dtor -Woverloaded-virtual -Wsuggest-override -fno-operator-names -Wzero-as-null-pointer-constant -ftrapv -std=gnu++11 -MD -MT ogr/ogrsf_frmts/mssqlspatial/CMakeFiles/ogr_MSSQLSpatial.dir/ogrmssqlgeometryparser.cpp.obj -MF CMakeFiles/ogr_MSSQLSpatial.dir/ogrmssqlgeometryparser.cpp.obj.d -o CMakeFiles/ogr_MSSQLSpatial.dir/ogrmssqlgeometryparser.cpp.obj -c /workspace/srcdir/gdal-3.5.1/ogr/ogrsf_frmts/mssqlspatial/ogrmssqlgeometryparser.cpp
[20:26:12] 0x78493a gimplify_modify_expr
[20:26:12]      /workspace/srcdir/gcc-7.1.0/gcc/gimplify.c:5636
[20:26:12] 0x77c18e gimplify_expr(tree_node**, gimple**, gimple**, bool (*)(tree_node*), int)
[20:26:12]      /workspace/srcdir/gcc-7.1.0/gcc/gimplify.c:11237
[20:26:12] 0x77f148 gimplify_stmt(tree_node**, gimple**)
[20:26:12]      /workspace/srcdir/gcc-7.1.0/gcc/gimplify.c:6496
[20:26:12] 0x7823e3 gimplify_cond_expr
[20:26:12]      /workspace/srcdir/gcc-7.1.0/gcc/gimplify.c:3973
[20:26:12] 0x77c836 gimplify_expr(tree_node**, gimple**, gimple**, bool (*)(tree_node*), int)
[20:26:12]      /workspace/srcdir/gcc-7.1.0/gcc/gimplify.c:11194
[20:26:12] 0x77f148 gimplify_stmt(tree_node**, gimple**)
[20:26:12]      /workspace/srcdir/gcc-7.1.0/gcc/gimplify.c:6496
[20:26:12] 0x78203e gimplify_cond_expr
[20:26:12]      /workspace/srcdir/gcc-7.1.0/gcc/gimplify.c:3866
[20:26:12] 0x77c836 gimplify_expr(tree_node**, gimple**, gimple**, bool (*)(tree_node*), int)
[20:26:12]      /workspace/srcdir/gcc-7.1.0/gcc/gimplify.c:11194
[20:26:12] 0x77bbc1 gimplify_expr(tree_node**, gimple**, gimple**, bool (*)(tree_node*), int)
[20:26:12]      /workspace/srcdir/gcc-7.1.0/gcc/gimplify.c:11966
[20:26:12] 0x77c4ae gimplify_expr(tree_node**, gimple**, gimple**, bool (*)(tree_node*), int)
[20:26:12]      /workspace/srcdir/gcc-7.1.0/gcc/gimplify.c:11339
[20:26:12] 0x7879a9 force_gimple_operand_1(tree_node*, gimple**, bool (*)(tree_node*), tree_node*)
[20:26:12]      /workspace/srcdir/gcc-7.1.0/gcc/gimplify-me.c:78
[20:26:12] 0x787a7d force_gimple_operand_gsi_1(gimple_stmt_iterator*, tree_node*, bool (*)(tree_node*), tree_node*, bool, gsi_iterator_update)
[20:26:12]      /workspace/srcdir/gcc-7.1.0/gcc/gimplify-me.c:115
[20:26:12] 0xaa991d rewrite_use_nonlinear_expr
[20:26:12]      /workspace/srcdir/gcc-7.1.0/gcc/tree-ssa-loop-ivopts.c:7242
[20:26:12] 0xaa991d rewrite_groups
[20:26:12]      /workspace/srcdir/gcc-7.1.0/gcc/tree-ssa-loop-ivopts.c:7466
[20:26:12] 0xaa991d tree_ssa_iv_optimize_loop
[20:26:12]      /workspace/srcdir/gcc-7.1.0/gcc/tree-ssa-loop-ivopts.c:7827
[20:26:12] [ 34%] Building CXX object gcore/CMakeFiles/gcore.dir/gdalmultidim.cpp.obj
[20:26:12] 0xaa991d tree_ssa_iv_optimize()
[20:26:12]      /workspace/srcdir/gcc-7.1.0/gcc/tree-ssa-loop-ivopts.c:7859
[20:26:12] cd /workspace/srcdir/gdal-3.5.1/build/gcore && /opt/bin/x86_64-w64-mingw32-libgfortran4-cxx11/x86_64-w64-mingw32-g++ --sysroot=/opt/x86_64-w64-mingw32/x86_64-w64-mingw32/sys-root/ -DDEBUG -DDONT_DEPRECATE_SPRINTF -DGDAL_CMAKE_BUILD -DGDAL_COMPILATION -DHAVE_AVX_AT_COMPILE_TIME -DHAVE_GEOS -DHAVE_SSE_AT_COMPILE_TIME -DHAVE_SSSE3_AT_COMPILE_TIME -DPAM_ENABLED -DSQLITE_ENABLED @CMakeFiles/gcore.dir/includes_CXX.rsp -fvisibility=hidden -g -Os -Wall -Wextra -Winit-self -Wunused-parameter -Wmissing-declarations -Wlogical-op -Wshadow -Wmissing-include-dirs -Wformat -Werror=format-security -Wno-format-nonliteral -Werror=vla -Wno-clobbered -Wdate-time -Wnull-dereference -Wfloat-conversion -Wnon-virtual-dtor -Woverloaded-virtual -Wsuggest-override -fno-operator-names -Wzero-as-null-pointer-constant -ftrapv -Wold-style-cast -Weffc++ -std=gnu++11 -MD -MT gcore/CMakeFiles/gcore.dir/gdalmultidim.cpp.obj -MF CMakeFiles/gcore.dir/gdalmultidim.cpp.obj.d -o CMakeFiles/gcore.dir/gdalmultidim.cpp.obj -c /workspace/srcdir/gdal-3.5.1/gcore/gdalmultidim.cpp
[20:26:12] 0xac2900 execute
[20:26:12]      /workspace/srcdir/gcc-7.1.0/gcc/tree-ssa-loop.c:549
[20:26:12] Please submit a full bug report,
[20:26:12] with preprocessed source if appropriate.
[20:26:12] Please include the complete backtrace with any bug report.
[20:26:12] See <https://gcc.gnu.org/bugs/> for instructions.
[20:26:12] make[2]: *** [gcore/CMakeFiles/gcore.dir/build.make:617: gcore/CMakeFiles/gcore.dir/overview.cpp.obj] Error 1
[20:26:12] make[2]: *** Waiting for unfinished jobs....
```

* stop cmake from looking in the wrong place

without this it would pick up dependencies like

```
	-- Could NOT find ZSTD (missing: ZSTD_DIR)
	-- Found PkgConfig: /usr/bin/pkg-config (found version "1.8.0")
	-- Found ZSTD: /opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/local/lib/libzstd.so (found version "1.5.2")
```

* Add postgres support

Fixes #5349

* Revert "Add postgres support"

It seems to find the wrong postgres:

```
[07:34:13] In file included from /workspace/srcdir/gdal-3.5.1/ogr/ogrsf_frmts/pg/ogr_pg.h:35,
[07:34:13]                  from /workspace/srcdir/gdal-3.5.1/ogr/ogrsf_frmts/pg/ogrpgdatasource.cpp:31:
[07:34:13] /opt/aarch64-linux-gnu/aarch64-linux-gnu/sys-root/usr/local/include/libpq-fe.h:29:10: fatal error: postgres_ext.h: No such file or directory
[07:34:13]  #include "postgres_ext.h"
[07:34:13]           ^~~~~~~~~~~~~~~~
```

This reverts commit da9909d.

* allow undefined symbols on FreeBSD

This patch is based on a similar patch for the Glib build.

The error this fixes is:

```
[08:24:11] /opt/bin/x86_64-unknown-freebsd12.2-libgfortran5-cxx11/x86_64-unknown-freebsd12.2-clang++ --sysroot=/opt/x86_64-unknown-freebsd12.2/x86_64-unknown-freebsd12.2/sys-root/ -fPIC  -fno-finite-math-only -fvisibility=hidden -O3 -DNDEBUG  -Wl,--no-undefined -shared -Wl,-soname,libgdal.so.31 -o libgdal.so.31.0.1 @CMakeFiles/GDAL.dir/objects1.rsp  -Wl,-rpath,/workspace/destdir/lib: -lm /workspace/destdir/lib/libz.so /workspace/destdir/lib/libcurl.so /opt/x86_64-unknown-freebsd12.2/x86_64-unknown-freebsd12.2/sys-root/usr/lib/libcrypto.so /opt/x86_64-unknown-freebsd12.2/x86_64-unknown-freebsd12.2/sys-root/usr/lib/libssl.so /workspace/destdir/lib/libzstd.so /opt/x86_64-unknown-freebsd12.2/x86_64-unknown-freebsd12.2/sys-root/usr/lib/liblzma.so /workspace/destdir/lib/libz.so /workspace/destdir/lib/libexpat.so /workspace/destdir/lib/libjpeg.so /workspace/destdir/lib/libz.so /workspace/destdir/lib/libtiff.so /workspace/destdir/lib/libjpeg.so /workspace/destdir/lib/libtiff.so /workspace/destdir/lib/libjpeg.so /workspace/destdir/lib/libgeotiff.so.5.2.0 /workspace/destdir/lib/libtiff.so /workspace/destdir/lib/libjpeg.so /workspace/destdir/lib/libpng.so /workspace/destdir/lib/libz.so /workspace/destdir/lib/libLerc.so /workspace/destdir/lib/libzstd.so /workspace/destdir/lib/libpng.so /workspace/destdir/lib/libz.so /workspace/destdir/lib/libz.so /workspace/destdir/lib/libjpeg.so /workspace/destdir/lib/libz.so /workspace/destdir/lib/libpng.so /workspace/destdir/lib/libz.so /workspace/destdir/lib/libcurl.so /workspace/destdir/lib/libsqlite3.so /workspace/destdir/lib/libz.so /workspace/destdir/lib/libopenjp2.so /workspace/destdir/lib/libexpat.so /workspace/destdir/lib/libexpat.so /workspace/destdir/lib/libsqlite3.so /workspace/destdir/lib/libexpat.so /workspace/destdir/lib/libexpat.so /workspace/destdir/lib/libexpat.so /workspace/destdir/lib/libexpat.so /workspace/destdir/lib/libsqlite3.so /workspace/destdir/lib/libsqlite3.so /workspace/destdir/lib/libexpat.so /workspace/destdir/lib/libsqlite3.so /workspace/destdir/lib/libsqlite3.so /workspace/destdir/lib/libsqlite3.so /workspace/destdir/lib/libexpat.so /workspace/destdir/lib/libexpat.so /workspace/destdir/lib/libexpat.so /workspace/destdir/lib/libgeos_c.so.1.17.0 /workspace/destdir/lib/libproj.so.25.9.1.0 -lpthread /opt/x86_64-unknown-freebsd12.2/x86_64-unknown-freebsd12.2/sys-root/usr/lib/libcrypto.so -Wl,-rpath-link,/workspace/destdir/lib
[08:24:12] /opt/x86_64-unknown-freebsd12.2/bin/x86_64-unknown-freebsd12.2-ld: port/CMakeFiles/cpl.dir/cpl_spawn.cpp.o: in function `CPLSpawnAsync':
[08:24:12] cpl_spawn.cpp:(.text+0x822): undefined reference to `environ'
[08:24:12] clang-13: error: linker command failed with exit code 1 (use -v to see invocation)
```

* retry adding PostresQL, specify PostgreSQL_LIBRARY

* add PostgreSQL_INCLUDE_DIR

* leave out PostgreSQL for now

current error is:

```
[11:32:50] In file included from /workspace/srcdir/gdal-3.5.1/ogr/ogrsf_frmts/pg/ogr_pg.h:35,
[11:32:50]                  from /workspace/srcdir/gdal-3.5.1/ogr/ogrsf_frmts/pg/ogrpgtablelayer.cpp:31:
[11:32:50] /opt/aarch64-linux-gnu/aarch64-linux-gnu/sys-root/usr/local/include/libpq-fe.h:29:10: fatal error: postgres_ext.h: No such file or directory
[11:32:50]  #include "postgres_ext.h"
[11:32:50]           ^~~~~~~~~~~~~~~~
[11:32:50] compilation terminated.
```
  • Loading branch information
@visr
visr authored Sep 4, 2022
1 parent 728e7c9 commit 5df8f7f
Show file tree
Hide file tree
Showing 4 changed files with 29 additions and 170 deletions.
90 changes: 29 additions & 61 deletions G/GDAL/build_tarballs.jl
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ using BinaryBuilder, Pkg

name = "GDAL"
upstream_version = v"3.5.1"
version_offset = v"0.0.0"
version_offset = v"0.0.1"
version = VersionNumber(upstream_version.major * 100 + version_offset.major,
upstream_version.minor * 100 + version_offset.minor,
upstream_version.patch * 100 + version_offset.patch)
Expand All @@ -13,74 +13,42 @@ version = VersionNumber(upstream_version.major * 100 + version_offset.major,
sources = [
ArchiveSource("https://github.com/OSGeo/gdal/releases/download/v$upstream_version/gdal-$upstream_version.tar.gz",
"7c4406ca010dc8632703a0a326f39e9db25d9f1f6ebaaeca64a963e3fac123d1"),
DirectorySource("./bundled"),
]

# Bash recipe for building across all platforms
script = raw"""
cd $WORKSPACE/srcdir/gdal-*/
mkdir build
cd build
if [[ ${target} == *mingw* ]]; then
export LDFLAGS="-L${libdir}"
# Apply patch to customise PROJ library
atomic_patch -p1 "$WORKSPACE/srcdir/patches/configure_ac_proj_libs.patch"
autoreconf -vi
export PROJ_LIBS="proj_9_1"
elif [[ "${target}" == *-linux-* ]]; then
# Hint to find libstdc++, required to link against C++ libs when using C compiler
if [[ "${nbits}" == 32 ]]; then
export CFLAGS="-Wl,-rpath-link,/opt/${target}/${target}/lib"
else
export CFLAGS="-Wl,-rpath-link,/opt/${target}/${target}/lib64"
fi
# Use same flags also for GEOS
atomic_patch -p1 "$WORKSPACE/srcdir/patches/geos-m4-extra-cflags.patch"
export EXTRA_GEOS_CFLAGS="${CFLAGS}"
if [[ "${target}" == powerpc64le-* ]]; then
atomic_patch -p1 "$WORKSPACE/srcdir/patches/sqlite3-m4-extra-libs.patch"
export EXTRA_GEOS_LIBS="${EXTRA_GEOS_LIBS} -lm"
export EXTRA_SQLITE3_LIBS="-lm"
# libpthread and libldl are needed for libgdal, so let's always use them
export LDFLAGS="$LDFLAGS -lpthread -ldl"
fi
autoreconf -vi
if [[ "${target}" == *-freebsd* ]]; then
# Our FreeBSD libc has `environ` as undefined symbol, so the linker will
# complain if this symbol is used in the built library, even if this won't
# be a problem at runtime. This flag allows having undefined symbols.
export LDFLAGS="-undefined"
fi
# same fix as used for PROJ
if [[ "${target}" == x86_64-linux-musl* ]]; then
export LDFLAGS="$LDFLAGS -lcurl"
fi
# Clear out `.la` files since they're often wrong and screw us up
rm -f ${prefix}/lib/*.la
# Read the options in the log file
./configure --help
./configure --prefix=${prefix} --build=${MACHTYPE} --host=${target} \
--with-geos=${bindir}/geos-config \
--with-proj=$prefix \
--with-tiff=$prefix \
--with-geotiff=$prefix \
--with-libz=$prefix \
--with-expat=$prefix \
--with-zstd=$prefix \
--with-sqlite3=$prefix \
--with-curl=${bindir}/curl-config \
--with-openjpeg \
--with-python=no \
--enable-shared \
--disable-static
# Make sure that some important libraries are found
grep "HAVE_GEOS='yes'" config.log
grep "HAVE_SQLITE='yes'" config.log
grep "CURL_SETTING='yes'" config.log
grep "ZSTD_SETTING='yes'" config.log
grep "HAVE_EXPAT='yes'" config.log
cmake -DCMAKE_INSTALL_PREFIX=${prefix} \
-DCMAKE_TOOLCHAIN_FILE=${CMAKE_TARGET_TOOLCHAIN} \
-DCMAKE_PREFIX_PATH=${prefix} \
-DCMAKE_FIND_ROOT_PATH=${prefix} \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_PYTHON_BINDINGS=OFF \
-DBUILD_JAVA_BINDINGS=OFF \
-DBUILD_CSHARP_BINDINGS=OFF \
-DGDAL_USE_CURL=ON \
-DGDAL_USE_EXPAT=ON \
-DGDAL_USE_GEOTIFF=ON \
-DGDAL_USE_GEOS=ON \
-DGDAL_USE_OPENJPEG=ON \
-DGDAL_USE_SQLITE3=ON \
-DGDAL_USE_TIFF=ON \
-DGDAL_USE_ZLIB=ON \
-DGDAL_USE_ZSTD=ON \
..
make -j${nproc}
make install
cmake --build . -j${nproc}
cmake --build . -j${nproc} --target install
"""

# These are the platforms we will build for by default, unless further
Expand Down Expand Up @@ -127,4 +95,4 @@ dependencies = [

# Build the tarballs, and possibly a `build.jl` as well.
build_tarballs(ARGS, name, version, sources, script, platforms, products, dependencies;
julia_compat="1.6", preferred_gcc_version=v"7")
julia_compat="1.6", preferred_gcc_version=v"8")
69 changes: 0 additions & 69 deletions G/GDAL/bundled/patches/configure_ac_proj_libs.patch
View file

This file was deleted.

20 changes: 0 additions & 20 deletions G/GDAL/bundled/patches/geos-m4-extra-cflags.patch
View file

This file was deleted.

20 changes: 0 additions & 20 deletions G/GDAL/bundled/patches/sqlite3-m4-extra-libs.patch
View file

This file was deleted.

0 comments on commit 5df8f7f

Please sign in to comment.