Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using Github CI to generate wheels and compiling to Windows too. #345

Merged
merged 2 commits into from
Apr 2, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
128 changes: 128 additions & 0 deletions .github/workflows/wheels.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
name: Build

on: [push, pull_request]

jobs:
build_wheels:
name: Build wheel for cp${{ matrix.python }}-${{ matrix.platform_id }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
include:
# Windows 64-bit
- os: windows-latest
python: 36
platform_id: win_amd64
- os: windows-latest
python: 37
platform_id: win_amd64
- os: windows-latest
python: 38
platform_id: win_amd64
- os: windows-latest
python: 39
platform_id: win_amd64
- os: windows-latest
python: 310
platform_id: win_amd64

# Linux 64-bit
- os: ubuntu-latest
python: 36
platform_id: manylinux_x86_64
- os: ubuntu-latest
python: 37
platform_id: manylinux_x86_64
- os: ubuntu-latest
python: 38
platform_id: manylinux_x86_64
- os: ubuntu-latest
python: 39
platform_id: manylinux_x86_64
- os: ubuntu-latest
python: 310
platform_id: manylinux_x86_64

# macOS on Intel 64-bit
- os: macos-latest
python: 36
arch: x86_64
platform_id: macosx_x86_64
- os: macos-latest
python: 37
arch: x86_64
platform_id: macosx_x86_64
- os: macos-latest
python: 38
arch: x86_64
platform_id: macosx_x86_64
- os: macos-latest
python: 39
arch: x86_64
platform_id: macosx_x86_64
- os: macos-latest
python: 310
arch: x86_64
platform_id: macosx_x86_64

# macOS on Apple M1 64-bit
# - os: macos-latest
# python: 38
# arch: arm64
# platform_id: macosx_arm64
# - os: macos-latest
# python: 39
# arch: arm64
# platform_id: macosx_arm64
# - os: macos-latest
# python: 310
# arch: arm64
# platform_id: macosx_arm64

steps:
- uses: actions/checkout@v2
with:
submodules: true

- uses: actions/setup-python@v2
name: Install Python host for cibuildwheel
with:
python-version: '3.9'

# Visual Studio
- name: Set up MSVC x64
if: matrix.platform_id == 'win_amd64'
uses: ilammy/msvc-dev-cmd@v1

- name: Install cibuildwheel
run: python -m pip install cibuildwheel==2.3.1

- name: Build wheels
env:
CIBW_MANYLINUX_X86_64_IMAGE: manylinux2014
CIBW_MANYLINUX_I686_IMAGE: manylinux2014
CIBW_BUILD: cp${{ matrix.python }}-${{ matrix.platform_id }}

# Include latest Python beta
CIBW_PRERELEASE_PYTHONS: True

CIBW_BEFORE_ALL_LINUX: |
yum install -y gcc-c++ libpng-devel libpng
pip install cmake ninja

CIBW_BEFORE_ALL_WINDOWS: |
pip install cmake ninja

CIBW_ARCHS_MACOS: ${{ matrix.arch }}
CIBW_BEFORE_ALL_MACOS: |
pip install cmake ninja

CIBW_ENVIRONMENT_MACOS: |
CMAKE_OSX_ARCHITECTURES=${{ matrix.arch }}

run: python -m cibuildwheel --output-dir wheelhouse

- uses: actions/upload-artifact@v2
with:
path: ./wheelhouse/*.whl
54 changes: 54 additions & 0 deletions scripts/configure_ANTsPy.bat
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
:: Converted with the help of https://daniel-sc.github.io/bash-shell-to-bat-converter/
:: @echo off

:: clone pybind11 into library directory
cd ants\lib
if not exist %USERPROFILE%\pybind11\ (
git clone https://github.com/stnava/pybind11.git
)
cd ..\..

echo %USERPROFILE%

:: create local ~/.antspy dir and move package data to it
if not exist %USERPROFILE%\.antspy\ (
mkdir %USERPROFILE%\.antspy
)
COPY data\* %USERPROFILE%\.antspy

:: clone ANTs and move all files into library directory
SET antsgit=https://github.com/ANTsX/ANTs.git
SET antstag=fc16efb8de42aa20955f694c31ea0af491e78d9e
echo "ANTS;%antstag%" REM UNKNOWN: {"type":"Redirect","op":{"text":">>","type":"dgreat"},"file":{"text":"./data/softwareVersions.csv","type":"Word"}}
cd ants\lib
echo "123"
:: if antscore doesnt exist, create it
if not exist "antscore\" (
echo "2"
git clone %antsgit% antsrepo
mkdir antscore
cd antsrepo

:: check out right branch
if exist ".git\" (
git checkout master
git pull
git checkout %antstag%
)
cd ..
COPY antsrepo\Examples\* antscore\
COPY antsrepo\Examples\include\* antscore
XCOPY antsrepo\Examples\include\ antscore\include\
COPY antsrepo\ImageRegistration\* antscore\
COPY antsrepo\ImageSegmentation\* antscore\
COPY antsrepo\Tensor\* antscore\
COPY antsrepo\Temporary\* antscore\
COPY antsrepo\Utilities\* antscore\
echo "HERE"

:: lil hack bc of stupid angled import bug in actual files
:: cp ReadWriteData.h antscore/ReadWriteData.h
:: lil hack bc ANTsVersionConfig.h is only created if you build ANTs...
COPY ANTsVersionConfig.h antscore\ANTsVersionConfig.h
)
cd ..\..
37 changes: 37 additions & 0 deletions scripts/configure_ITK.bat
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
:: Converted with the help of https://daniel-sc.github.io/bash-shell-to-bat-converter/
@echo off

SET CMAKE_BUILD_TYPE=Release

SET itkgit=https://github.com/InsightSoftwareConsortium/ITK.git
SET itktag=v5.3rc03

:: if there is a directory but no git, remove it
if exist itksource\ (
if exist itksource\.git (
DEL /S /Q itksource\
)
)
:: if no directory, clone ITK in `itksource` dir
if not exist itksource\ (
git clone %itkgit% itksource
)
cd itksource
if exist .git\ (
git checkout master
git checkout %itktag%
DEL /S /Q .git\
)

:: go back to main dir
cd ..\

echo Dependency;GitTag REM UNKNOWN: {"type":"Redirect","op":{"text":">","type":"great"},"file":{"text":"./data/softwareVersions.csv","type":"Word"}}
echo "ITK;%itktag%" REM UNKNOWN: {"type":"Redirect","op":{"text":">>","type":"dgreat"},"file":{"text":"./data/softwareVersions.csv","type":"Word"}}
mkdir itkbuild
cd itkbuild
SET compflags= -fPIC -O2
cmake -GNinja -DITK_USE_SYSTEM_PNG=OFF -DCMAKE_SH:BOOL=OFF -DCMAKE_BUILD_TYPE:STRING="%CMAKE_BUILD_TYPE%" -DITK_USE_GIT_PROTOCOL:BOOL=OFF -DBUILD_SHARED_LIBS:BOOL=OFF -DBUILD_TESTING:BOOL=OFF -DBUILD_EXAMPLES:BOOL=OFF -DCMAKE_INSTALL_PREFIX:PATH=%R_PACKAGE_DIR%\libs\ -DITK_LEGACY_REMOVE:BOOL=OFF -DITK_FUTURE_LEGACY_REMOVE:=BOOL=ON -DITK_BUILD_DEFAULT_MODULES:BOOL=OFF -DKWSYS_USE_MD5:BOOL=ON -DITK_WRAPPING:BOOL=OFF -DModule_MGHIO:BOOL=ON -DModule_ITKDeprecated:BOOL=OFF -DModule_ITKReview:BOOL=ON -DModule_ITKVtkGlue:BOOL=OFF -DModule_GenericLabelInterpolator:BOOL=ON -DITKGroup_Core=ON -DModule_ITKReview=ON -DITKGroup_Filtering=ON -DITKGroup_IO=ON -DITKGroup_Numerics=ON -DITKGroup_Registration=ON -DITKGroup_Segmentation=ON -DModule_AdaptiveDenoising:BOOL=ON -DModule_GenericLabelInterpolator:BOOL=ON -DCMAKE_C_VISIBILITY_PRESET:BOOL=hidden -DCMAKE_CXX_VISIBILITY_PRESET:BOOL=hidden -DCMAKE_VISIBILITY_INLINES_HIDDEN:BOOL=ON ..\itksource\
ninja

cd ..\
95 changes: 67 additions & 28 deletions setup.py
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
import os
import shutil
import re
import sys
import platform
import re
import shutil
import subprocess
import sys
import time
from distutils.version import LooseVersion
from functools import cmp_to_key

import setuptools
from setuptools import find_packages
from setuptools import Extension, setup
from setuptools.command.build_ext import build_ext
from distutils.version import LooseVersion
import setuptools.command.build_ext
import setuptools.command.install
import setuptools.command.develop
import setuptools.command.build_py
import setuptools.command.develop
import setuptools.command.install
from setuptools import Extension, find_packages, setup
from setuptools.command.build_ext import build_ext

setup_py_dir = os.path.dirname(os.path.realpath(__file__))
version = "0.3.2" # ANTsPy version
Expand Down Expand Up @@ -79,12 +79,22 @@ def run(self):
os.environ["ITK_DIR"] = os.path.join(setup_py_dir, "itkbuild")
else:
print("No local ITK installation found... Building ITK now...")
subprocess.check_call(["./scripts/configure_ITK.sh"], cwd=setup_py_dir)
if platform.system() == "Windows":
subprocess.check_call(
[".\\scripts\\configure_ITK.bat"], cwd=setup_py_dir
)
else:
subprocess.check_call(["./scripts/configure_ITK.sh"], cwd=setup_py_dir)
os.environ["ITK_DIR"] = os.path.join(setup_py_dir, "itkbuild")

## Find or Configure ANTs ##
print("Configuring ANTs core")
subprocess.check_call(["./scripts/configure_ANTsPy.sh"], cwd=setup_py_dir)
if platform.system() == "Windows":
subprocess.check_call(
[".\\scripts\\configure_ANTsPy.bat"], cwd=setup_py_dir
)
else:
subprocess.check_call(["./scripts/configure_ANTsPy.sh"], cwd=setup_py_dir)

## Configure ANTsPy library ##
print("Configuring ANTsPy library")
Expand All @@ -107,39 +117,55 @@ def run(self):

def build_extension(self, ext):
setup_py_dir = os.path.dirname(os.path.realpath(__file__))
extdir = os.path.join(setup_py_dir, "ants/lib/")
extdir = os.path.join(setup_py_dir, "ants", "lib")

cmake_args = [
"-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=" + extdir,
"-DPYTHON_EXECUTABLE=" + sys.executable,
"-DCMAKE_BUILD_TYPE=Release",
]

cfg = "Release"
build_args = ["--config", cfg]

if platform.system() == "Windows":
cmake_args += [
"-DCMAKE_LIBRARY_OUTPUT_DIRECTORY_{}={}".format(cfg.upper(), extdir)
"-DBUILD_SHARED_LIBS:BOOL=OFF",
"-DBUILD_ALL_ANTS_APPS:BOOL=OFF",
"-DCMAKE_LIBRARY_OUTPUT_DIRECTORY_{}={}".format(cfg.upper(), extdir),
]
if sys.maxsize > 2 ** 32:
cmake_args += ["-A", "x64"]
build_args += ["--", "/m"]
else:
cmake_args += ["-DCMAKE_BUILD_TYPE=" + cfg]
build_args += ["--", "-j2"]
# cmake_args += [
# "-DCMAKE_LIBRARY_OUTPUT_DIRECTORY_{}={}".format(cfg.upper(), extdir)
# ]
# if sys.maxsize > 2 ** 32:
# cmake_args += ["-A", "x64"]
# build_args += ["--", "/m"]
# else:
# cmake_args += ["-DCMAKE_BUILD_TYPE=" + cfg]
# build_args += ["--", "-j2"]

env = os.environ.copy()
env["CXXFLAGS"] = '{} {} -DVERSION_INFO=\\"{}\\"'.format(
"-Wno-inconsistent-missing-override",
env.get("CXXFLAGS", ""),
self.distribution.get_version(),
)
env["LINKFLAGS"] = "{}".format("-Wno-inconsistent-missing-override")

# if platform.system() != "Windows":
# env["CXXFLAGS"] = '{} {} -DVERSION_INFO=\\"{}\\"'.format(
# "-Wno-inconsistent-missing-override",
# env.get("CXXFLAGS", ""),
# self.distribution.get_version(),
# )
# env["LINKFLAGS"] = "{}".format("-Wno-inconsistent-missing-override")

print(cmake_args)
if not os.path.exists(self.build_temp):
os.makedirs(self.build_temp)
subprocess.check_call(
["cmake", ext.sourcedir] + cmake_args, cwd=self.build_temp, env=env
[
"cmake",
]
+ cmake_args
+ [
ext.sourcedir,
],
cwd=self.build_temp,
env=env,
)
subprocess.check_call(
["cmake", "--build", "."] + build_args, cwd=self.build_temp
Expand Down Expand Up @@ -176,7 +202,20 @@ def build_extension(self, ext):
cmdclass={"build_py": build_py, "build_ext": CMakeBuild, "install": BuildExtFirst},
zip_safe=False,
packages=find_packages(),
package_data={"ants": ["ants/lib/*.so*", "lib/*.so*", "ants/lib/*.so", "lib/*.so"]},
package_data={"ants": [
"ants/lib/*.so*",
"ants/lib/*.pyd",
"ants/lib/*.dll",
"lib/*.so*",
"lib/*.pyd",
"lib/*.dll",
"ants/lib/*.so",
"ants/lib/*.pyd",
"ants/lib/*.dll",
"lib/*.so"
"lib/*.pyd"
"lib/*.dll"
]},
url="https://github.com/ANTsX/ANTsPy",
classifiers=["Programming Language :: Python :: 3.6"],
)
19 changes: 19 additions & 0 deletions tests/run_tests.bat
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
SET PYCMD=python
pushd "tests"
echo "Running core tests"
%PYCMD% test_core_ants_image.py %@%
%PYCMD% test_core_ants_image_io.py %@%
%PYCMD% test_core_ants_transform.py %@%
%PYCMD% test_core_ants_transform_io.py %@%
%PYCMD% test_core_ants_metric.py %@%
echo "Running learn tests"
%PYCMD% test_learn.py %@%
echo "Running registation tests"
%PYCMD% test_registation.py %@%
echo "Running segmentation tests"
%PYCMD% test_segmentation.py %@%
echo "Running utils tests"
%PYCMD% test_utils.py %@%
echo "Running bug tests"
%PYCMD% test_bugs.py %@%
popd