-
Notifications
You must be signed in to change notification settings - Fork 224
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix issues for running on Windows #313
Closed
Changes from all commits
Commits
Show all changes
21 commits
Select commit
Hold shift + click to select a range
82e15db
Separate out the libgmt loading tests
leouieda fb04b6d
Try to find the Windows library
leouieda c93734b
Use .dll instead of .lib
leouieda aa70177
Set GMT_LIBRARY_PATH
leouieda b21f424
Update pygmt/clib/loading.py
leouieda dfd34e6
Remove the GMT_LIBRARY_PATH definition
leouieda efbd6ab
Temporarily disable clib info printing
leouieda 611e573
Update to RC2
leouieda 2f65ef9
Merge branch 'master' into windows
seisman ce7d12a
Update to GMT6.0.0rc3
seisman 0613fb2
Merge branch 'master' into windows
seisman 197ebb8
Re-enable Windows CI in current PR
seisman da2cfec
Merge branch 'master' into windows
seisman 5b2fb0d
Merge branch 'master' into windows
seisman 10f66c6
Merge branch 'master' into windows
weiji14 4d45c04
Merge branch 'master' into windows
weiji14 c0f9eef
Merge branch 'master' into windows
weiji14 22fd4b0
Try compiling GMT on Windows builds
leouieda 424e9b9
Merge branch 'master' into windows
leouieda 27afcde
Copy over Azure template from GMT
leouieda 0b7f674
Merge branch 'master' into windows
seisman File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,88 @@ | ||
# Template for Windows steps in Azure Pipelines | ||
|
||
steps: | ||
|
||
# Cache vcpkg installed libraries | ||
# Currently, caches are immutable and can't be updated. | ||
# To rebuild the cache, you have to change the date in the key list | ||
- task: CacheBeta@0 | ||
inputs: | ||
key: $(Agent.OS) | vcpkg | 20190802 | ||
path: $(VCPKG_INSTALLATION_ROOT)/installed/ | ||
cacheHitVar: CACHE_VCPKG_RESTORED | ||
displayName: Cache vcpkg libraries | ||
|
||
- bash: | | ||
set -x -e | ||
# By default, vcpkg builds both release and debug configurations. | ||
# Set VCPKG_BUILD_TYPE to build release only to save half time | ||
echo 'set (VCPKG_BUILD_TYPE release)' >> ${VCPKG_INSTALLATION_ROOT}/triplets/${WIN_PLATFORM}.cmake | ||
cat ${VCPKG_INSTALLATION_ROOT}/triplets/${WIN_PLATFORM}.cmake | ||
# install libraries | ||
#vcpkg install netcdf-c gdal pcre fftw3 clapack openblas --triplet ${WIN_PLATFORM} | ||
vcpkg install netcdf-c gdal pcre fftw3 --triplet ${WIN_PLATFORM} | ||
displayName: Install dependencies via vcpkg | ||
env: | ||
WIN_PLATFORM: "x64-windows" | ||
condition: ne(variables['CACHE_VCPKG_RESTORED'], true) | ||
|
||
- bash: | | ||
# hook up user-wide integration | ||
vcpkg integrate install | ||
# list installed libraries | ||
vcpkg list | ||
# Executable files search for DLL files in the directories listed in the PATH environment variable. | ||
echo "##vso[task.prependpath]${VCPKG_INSTALLATION_ROOT}/installed/${WIN_PLATFORM}/bin" | ||
# Tools like gdal_translate, ogr2ogr are located in tools/gdal | ||
echo "##vso[task.prependpath]${VCPKG_INSTALLATION_ROOT}/installed/${WIN_PLATFORM}/tools/gdal" | ||
displayName: Configure vcpkg | ||
env: | ||
WIN_PLATFORM: "x64-windows" | ||
|
||
- bash: | | ||
set -x -e | ||
choco install ninja | ||
choco install ghostscript --version 9.26 # gs 9.27 is buggy | ||
displayName: Install dependencies via chocolatey | ||
|
||
- bash: | | ||
echo "##vso[task.setvariable variable=INSTALLDIR]$BUILD_SOURCESDIRECTORY/gmt-install-dir" | ||
echo "##vso[task.setvariable variable=COASTLINEDIR]$BUILD_SOURCESDIRECTORY/coastline" | ||
displayName: Set install location and coastline location | ||
|
||
- bash: echo "##vso[task.prependpath]$INSTALLDIR/bin" | ||
displayName: Set PATH | ||
|
||
- task: CacheBeta@0 | ||
inputs: | ||
key: coastline | ||
path: $(COASTLINEDIR) | ||
cacheHitVar: CACHE_COASTLINE_RESTORED | ||
displayName: Cache GSHHG and DCW data | ||
|
||
- bash: ci/download-coastlines.sh | ||
displayName: Download coastlines | ||
condition: ne(variables['CACHE_COASTLINE_RESTORED'], true) | ||
|
||
- bash: ci/config-gmt-windows.sh | ||
displayName: Configure GMT | ||
|
||
# Azure Pipelines cannot find Visual Studio correctly if Ninja is used. | ||
# Here, we need | ||
# 1. call the vcvars64.bat script | ||
# 2. define CMAKE_C_COMPILER | ||
- script: | | ||
mkdir build | ||
cd build | ||
call "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Auxiliary\Build\vcvars64.bat" | ||
cmake .. -G "Ninja" -DCMAKE_TOOLCHAIN_FILE=%VCPKG_INSTALLATION_ROOT%\scripts\buildsystems\vcpkg.cmake -DCMAKE_C_COMPILER=cl.exe -DCMAKE_BUILD_TYPE=Release | ||
cmake --build . | ||
cmake --build . --target install | ||
displayName: Compile GMT | ||
|
||
- script: | | ||
gmt --version | ||
gmt defaults -Vd | ||
gmt pscoast -R0/10/0/10 -JM6i -Ba -Ggray -ENG+p1p,blue -P -Vd > test.ps | ||
gmt begin && gmt coast -R0/10/0/10 -JM6i -Ba -Ggray -ENG+p1p,blue -Vd && gmt end | ||
displayName: Check a few simple commands |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
#!/usr/bin/env bash | ||
# Build and install GMT | ||
|
||
# To return a failure if any commands inside fail | ||
set -e | ||
|
||
cat > cmake/ConfigUser.cmake << 'EOF' | ||
set (CMAKE_INSTALL_PREFIX "$ENV{INSTALLDIR}") | ||
set (GSHHG_ROOT "$ENV{COASTLINEDIR}/gshhg") | ||
set (DCW_ROOT "$ENV{COASTLINEDIR}/dcw") | ||
set (COPY_GSHHG TRUE) | ||
set (COPY_DCW TRUE) | ||
set (GMT_INSTALL_MODULE_LINKS FALSE) | ||
set (CMAKE_C_FLAGS "/D_CRT_SECURE_NO_WARNINGS /D_CRT_SECURE_NO_DEPRECATE ${CMAKE_C_FLAGS}") | ||
set (CMAKE_C_FLAGS "/D_CRT_NONSTDC_NO_DEPRECATE /D_SCL_SECURE_NO_DEPRECATE ${CMAKE_C_FLAGS}") | ||
EOF | ||
|
||
if [[ "$TEST" == "true" ]]; then | ||
cat >> cmake/ConfigUser.cmake << 'EOF' | ||
enable_testing() | ||
set (DO_EXAMPLES TRUE) | ||
set (DO_TESTS TRUE) | ||
set (DO_API_TESTS ON) | ||
set (SUPPORT_EXEC_IN_BINARY_DIR TRUE) | ||
EOF | ||
fi | ||
|
||
if [[ "$BUILD_DOCS" == "true" ]]; then | ||
cat >> cmake/ConfigUser.cmake << 'EOF' | ||
set (DO_ANIMATIONS TRUE) | ||
set (CMAKE_VERBOSE_MAKEFILE ON CACHE BOOL "ON") | ||
EOF | ||
fi | ||
|
||
echo "" | ||
echo "Using the following cmake configuration:" | ||
cat cmake/ConfigUser.cmake | ||
echo "" | ||
|
||
# Turn off exit on failure. | ||
set +e |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,55 @@ | ||
#!/usr/bin/env bash | ||
# Download and install the coastlines and boundaries datasets | ||
|
||
# General settings: | ||
EXT="tar.gz" | ||
GSHHG="gshhg-gmt-2.3.7" | ||
DCW="dcw-gmt-1.1.4" | ||
MD5_GSHHG=8ee2653f9daf84d49fefbf990bbfa1e7 | ||
MD5_DCW=4f30857a8b12af0f910222fceb591538 | ||
|
||
# Used for checking the downloaded files: | ||
check_md5 () | ||
{ | ||
md5_ref=$1 | ||
file=$2 | ||
|
||
test -f "$file" || return 1 | ||
md5=$(openssl dgst -md5 $file | cut -d ' ' -f 2) | ||
test "$md5" = "$md5_ref" | ||
} | ||
|
||
# To return a failure if any commands inside fail | ||
set -e | ||
|
||
# Create the directory for coastline data | ||
mkdir -p $COASTLINEDIR | ||
|
||
# GSHHG and DCW tarballs are cached here: | ||
test -d $HOME/cache-gshhg-dcw || mkdir $HOME/cache-gshhg-dcw | ||
cd $HOME/cache-gshhg-dcw | ||
|
||
# GSHHG (coastlines, rivers, and political boundaries): | ||
echo "" | ||
echo "Downloading and unpacking GSHHG" | ||
echo "================================================================================" | ||
# download when md5sums don't match: | ||
check_md5 $MD5_GSHHG $GSHHG.$EXT || curl -L -O --retry 10 "https://mirrors.ustc.edu.cn/gmt/$GSHHG.$EXT" | ||
check_md5 $MD5_GSHHG $GSHHG.$EXT | ||
tar xzf $GSHHG.$EXT | ||
mv $GSHHG $COASTLINEDIR/gshhg | ||
|
||
# DCW (country polygons): | ||
echo "" | ||
echo "Downloading and unpacking DCW" | ||
echo "================================================================================" | ||
# download when md5sums don't match: | ||
check_md5 $MD5_DCW $DCW.$EXT || curl -L -O --retry 10 "https://mirrors.ustc.edu.cn/gmt/$DCW.$EXT" | ||
check_md5 $MD5_DCW $DCW.$EXT | ||
tar xzf $DCW.$EXT | ||
mv $DCW $COASTLINEDIR/dcw | ||
|
||
ls $COASTLINEDIR/gshhg $COASTLINEDIR/dcw | ||
|
||
# Turn off exit on failure. | ||
set +e |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If both ctypes.CDLL and check_libgmt don't raise any errors, it means we find the library, then we should break out of the loop. Right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Of course! Thanks, Dongdong. I stopped coding yesterday because I couldn't even see this basic error anymore :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just realized that this change is part of what's been causing our Windows builds to fail... Since we have a list of dll files to check, if one of the dll fails, the variable
error
turnsTrue
and staysTrue
even if a later dll works. Our Linux and MacOS tests work because there's only one item in the list.