Merge pull request #409 from rapidsai/branch-21.06

rapidsai · Jun 9, 2021 · b5e0810 · b5e0810
2 parents df1d93c + 4896437
commit b5e0810
Show file tree

Hide file tree

Showing 64 changed files with 3,624 additions and 652 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,3 +1,18 @@
+# cuSpatial 21.06.00 (9 Jun 2021)
+
+## 🚀 New Featues
+
+- `fom_geopandas` convets GeoPandas GeoSeies objects into cuspatial GeoAow fom. ([#300](https://github.com/rapidsai/cuspatial/pull/300)) [@thomcom](https://github.com/thomcom)
+
+## 🛠️ Impovements
+
+- Update envionment vaiable used to detemine `cuda_vesion` ([#407](https://github.com/rapidsai/cuspatial/pull/407)) [@ajschmidt8](https://github.com/ajschmidt8)
+- Update `CHANGELOG.md` links fo calve ([#404](https://github.com/rapidsai/cuspatial/pull/404)) [@ajschmidt8](https://github.com/ajschmidt8)
+- Move mm::device_buffe instead of copying ([#403](https://github.com/rapidsai/cuspatial/pull/403)) [@haism](https://github.com/haism)
+- Update docs build scipt ([#402](https://github.com/rapidsai/cuspatial/pull/402)) [@ajschmidt8](https://github.com/ajschmidt8)
+- Update cuspatial vesion fo calve, fix CMake FindPackage cudf ([#401](https://github.com/rapidsai/cuspatial/pull/401)) [@txcllnt](https://github.com/txcllnt)
+- Impove pefomance of quadtee point-to-polyline join ([#362](https://github.com/rapidsai/cuspatial/pull/362)) [@txcllnt](https://github.com/txcllnt)
+
 # cuSpatial 0.19.0 (21 Apr 2021)
 
 ## 🐛 Bug Fixes

diff --git a/README.md b/README.md
@@ -5,31 +5,38 @@
 **NOTE:** cuSpatial depends on [cuDF](https://github.com/rapidsai/cudf) and
 [RMM](https://github.com/rapidsai/rmm) from [RAPIDS](https://rapids.ai/).
 
-## Implemented operations:
+## Operations
+
 cuSpatial supports the following operations on spatial and trajectory data:
+
 1. Spatial window query
 2. Point-in-polygon test
 3. Haversine distance
 4. Hausdorff distance
 5. Deriving trajectories from point location data
 6. Computing distance/speed of trajectories
 7. Computing spatial bounding boxes of trajectories
+8. Quadtree-based indexing for large-scale point data
+9. Quadtree-based point-in-polygon spatial join
+10. Quadtree-based point-to-polyline nearest neighbor distance
+
+Future support is planned for the following operations:
 
-Future support is planned for the following operations.
 1. Temporal window query
 2. Temporal point query (year+month+day+hour+minute+second+millisecond)
-3. Point-to-polyline nearest neighbor distance
-4. Grid-based indexing for points and polygons
-5. Quadtree-based indexing for large-scale point data
-6. R-Tree-based indexing for Polygons/Polylines
+3. Grid-based indexing for points and polygons
+4. R-Tree-based indexing for Polygons/Polylines
 
 ## Install from Conda
+
 To install via conda:
-```
+
+```shell
 conda install -c conda-forge -c rapidsai-nightly cuspatial
 ```
 
 ## Install from Source
+
 To build and install cuSpatial from source:
 
 ### Install dependencies
@@ -46,28 +53,31 @@ environment created in step 3 is active.
 1. export `CUSPATIAL_HOME=$(pwd)/cuspatial`
 2. clone the cuSpatial repo
 
-```
-git clone --recurse-submodules https://github.com/rapidsai/cuspatial.git $CUSPATIAL_HOME
-```
+   ```shell
+   git clone --recurse-submodules https://github.com/rapidsai/cuspatial.git $CUSPATIAL_HOME
+   ```
 
-3. Compile and install 
-Similar to cuDF (version 0.11), simplely run 'build.sh' diectly under $CUSPATIAL_HOME<br>
-Note that a "build" dir is created automatically under $CUSPATIAL_HOME/cpp
+3. Compile and install
 
-4. Run C++/Python test code <br>
+   Similar to cuDF (version 0.20), simply run `build.sh` diectly under `$CUSPATIAL_HOME`.
 
-Some tests using inline data can be run directly, e.g.,
-```
-$CUSPATIAL_HOME/cpp/build/gtests/LEGACY_HAUSDORFF_TEST
-$CUSPATIAL_HOME/cpp/build/gtests/POINT_IN_POLYGON_TEST
-python python/cuspatial/cuspatial/tests/legacy/test_hausdorff_distance.py
-python python/cuspatial/cuspatial/tests/test_pip.py
-```
+   Note that a "build" dir is created automatically under `$CUSPATIAL_HOME/cpp`.
+
+4. Run C++/Python test code
+
+   Some tests using inline data can be run directly, e.g.:
+
+   ```shell
+   $CUSPATIAL_HOME/cpp/build/gtests/LEGACY_HAUSDORFF_TEST
+   $CUSPATIAL_HOME/cpp/build/gtests/POINT_IN_POLYGON_TEST
+   python python/cuspatial/cuspatial/tests/legacy/test_hausdorff_distance.py
+   python python/cuspatial/cuspatial/tests/test_pip.py
+   ```
 
-Some other tests involve I/O from data files under $CUSPATIAL_HOME/test_fixtures.
-For example, $CUSPATIAL_HOME/cpp/build/gtests/SHAPEFILE_READER_TEST requires three
-pre-generated polygon shapefiles that contain 0, 1 and 2 polygons, respectively. They are available at 
-$CUSPATIAL_HOME/test_fixtures/shapefiles <br>
+   Some other tests involve I/O from data files under $CUSPATIAL_HOME/test_fixtures.
+   For example, $CUSPATIAL_HOME/cpp/build/gtests/SHAPEFILE_READER_TEST requires three
+   pre-generated polygon shapefiles that contain 0, 1 and 2 polygons, respectively. They are available at 
+   $CUSPATIAL_HOME/test_fixtures/shapefiles <br>
 
 **NOTE:** Currently, cuSpatial supports reading point/polyine/polygon data using
 Structure of Array (SoA) format and a [shapefile reader](./cpp/src/io/shp)

diff --git a/ci/cpu/build.sh b/ci/cpu/build.sh
@@ -23,8 +23,9 @@ export CUDA_REL=${CUDA_VERSION%.*}
 export GPUCI_CONDA_RETRY_MAX=1
 export GPUCI_CONDA_RETRY_SLEEP=30
 
-# Use Ninja to build
+# Use Ninja to build, setup conda build dir
 export CMAKE_GENERATOR="Ninja"
+export CONDA_BLD_DIR="${WORKSPACE}/.conda-bld"
 
 # Switch to project root; also root of repo checkout
 cd $WORKSPACE
@@ -49,6 +50,11 @@ gpuci_logger "Activate conda env"
 . /opt/conda/etc/profile.d/conda.sh
 conda activate rapids
 
+# Remove rapidsai-nightly channel if we are building main branch
+if [ "$SOURCE_BRANCH" = "main" ]; then
+  conda config --system --remove channels rapidsai-nightly
+fi
+
 gpuci_logger "Check compiler versions"
 python --version
 $CC --version
@@ -69,19 +75,21 @@ conda config --set ssl_verify False
 if [ "$BUILD_LIBCUSPATIAL" == '1' ]; then
   gpuci_logger "Build conda pkg for libcuspatial"
   if [[ -z "$PROJECT_FLASH" || "$PROJECT_FLASH" == "0" ]]; then
-    gpuci_conda_retry build conda/recipes/libcuspatial
+    gpuci_conda_retry build --no-build-id --croot ${CONDA_BLD_DIR} conda/recipes/libcuspatial
   else
-    gpuci_conda_retry build --dirty --no-remove-work-dir conda/recipes/libcuspatial
+    gpuci_conda_retry build --no-build-id --croot ${CONDA_BLD_DIR} --dirty --no-remove-work-dir conda/recipes/libcuspatial
+    mkdir -p ${CONDA_BLD_DIR}/libcuspatial/work
+    cp -r ${CONDA_BLD_DIR}/work/* ${CONDA_BLD_DIR}/libcuspatial/work
   fi
 fi
 
 if [ "$BUILD_CUSPATIAL" == '1' ]; then
   gpuci_logger "Build conda pkg for cuspatial"
   if [[ -z "$PROJECT_FLASH" || "$PROJECT_FLASH" == "0" ]]; then
-    gpuci_conda_retry build conda/recipes/cuspatial
+    gpuci_conda_retry build --croot ${CONDA_BLD_DIR} conda/recipes/cuspatial
   else
-    gpuci_conda_retry build --dirty --no-remove-work-dir \
-        -c $WORKSPACE/ci/artifacts/cuspatial/cpu/conda-bld/ conda/recipes/cuspatial
+    gpuci_conda_retry build --croot ${CONDA_BLD_DIR} --dirty --no-remove-work-dir \
+        -c $WORKSPACE/ci/artifacts/cuspatial/cpu/.conda-bld/ conda/recipes/cuspatial
   fi
 fi
 

diff --git a/ci/cpu/prebuild.sh b/ci/cpu/prebuild.sh
@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 
 #Upload cuspatial once per PYTHON
-if [[ "$CUDA" == "10.1" ]]; then
+if [[ "$CUDA" == "11.0" ]]; then
     export UPLOAD_CUSPATIAL=1
 else
     export UPLOAD_CUSPATIAL=0

diff --git a/ci/cpu/upload.sh b/ci/cpu/upload.sh
@@ -26,8 +26,8 @@ fi
 ################################################################################
 
 gpuci_logger "Get conda file output locations"
-export LIBCUSPATIAL_FILE=`conda build conda/recipes/libcuspatial  --output`
-export CUSPATIAL_FILE=`conda build conda/recipes/cuspatial --python=$PYTHON --output`
+export LIBCUSPATIAL_FILE=`conda build --no-build-id --croot ${CONDA_BLD_DIR} conda/recipes/libcuspatial  --output`
+export CUSPATIAL_FILE=`conda build --croot ${CONDA_BLD_DIR} conda/recipes/cuspatial --python=$PYTHON --output`
 
 ################################################################################
 # UPLOAD - Conda packages
@@ -41,7 +41,7 @@ if [[ "$BUILD_LIBCUSPATIAL" == "1" && "$UPLOAD_LIBCUSPATIAL" == "1" ]]; then
   test -e ${LIBCUSPATIAL_FILE}
   echo "Upload libcuspatial"
   echo ${LIBCUSPATIAL_FILE}
-  gpuci_retry anaconda -t ${MY_UPLOAD_KEY} upload -u ${CONDA_USERNAME:-rapidsai} ${LABEL_OPTION} --skip-existing ${LIBCUSPATIAL_FILE}
+  gpuci_retry anaconda -t ${MY_UPLOAD_KEY} upload -u ${CONDA_USERNAME:-rapidsai} ${LABEL_OPTION} --skip-existing ${LIBCUSPATIAL_FILE} --no-progress
 fi
 
 if [[ "$BUILD_CUSPATIAL" == "1" && "$UPLOAD_CUSPATIAL" == "1" ]]; then
@@ -50,5 +50,5 @@ if [[ "$BUILD_CUSPATIAL" == "1" && "$UPLOAD_CUSPATIAL" == "1" ]]; then
   test -e ${CUSPATIAL_FILE}
   echo "Upload cuspatial"
   echo ${CUSPATIAL_FILE}
-  gpuci_retry anaconda -t ${MY_UPLOAD_KEY} upload -u ${CONDA_USERNAME:-rapidsai} ${LABEL_OPTION} --skip-existing ${CUSPATIAL_FILE}
+  gpuci_retry anaconda -t ${MY_UPLOAD_KEY} upload -u ${CONDA_USERNAME:-rapidsai} ${LABEL_OPTION} --skip-existing ${CUSPATIAL_FILE} --no-progress
 fi
diff --git a/ci/docs/build.sh b/ci/docs/build.sh
@@ -15,7 +15,6 @@ export PATH=/opt/conda/bin:/usr/local/cuda/bin:$PATH
 export HOME=$WORKSPACE
 export PROJECT_WORKSPACE=/rapids/cuspatial
 export LIBCUDF_KERNEL_CACHE_PATH="$HOME/.jitify-cache"
-export NIGHTLY_VERSION=$(echo $BRANCH_VERSION | awk -F. '{print $2}')
 export PROJECTS=(cuspatial)
 
 gpuci_logger "Check environment"

diff --git a/ci/gpu/build.sh b/ci/gpu/build.sh
@@ -18,6 +18,7 @@ export PARALLEL_LEVEL=${PARALLEL_LEVEL:-4}
 export CUDA_REL=${CUDA_VERSION%.*}
 export CUDF_HOME="${WORKSPACE}/cudf"
 export CUSPATIAL_HOME="${WORKSPACE}"
+export CONDA_ARTIFACT_PATH=${WORKSPACE}/ci/artifacts/cuspatial/cpu/.conda-bld/
 
 # Set home to the job's workspace
 export HOME=$WORKSPACE
@@ -130,11 +131,11 @@ else
 
     cd $WORKSPACE/python
 
-    CONDA_FILE=`find $WORKSPACE/ci/artifacts/cuspatial/cpu/conda-bld/ -name "libcuspatial*.tar.bz2"`
+    CONDA_FILE=`find ${CONDA_ARTIFACT_PATH} -name "libcuspatial*.tar.bz2"`
     CONDA_FILE=`basename "$CONDA_FILE" .tar.bz2` #get filename without extension
     CONDA_FILE=${CONDA_FILE//-/=} #convert to conda install
     gpuci_logger "Installing $CONDA_FILE"
-    conda install -c $WORKSPACE/ci/artifacts/cuspatial/cpu/conda-bld/ "$CONDA_FILE"
+    conda install -c ${CONDA_ARTIFACT_PATH} "$CONDA_FILE"
 
     export LIBCUGRAPH_BUILD_DIR="$WORKSPACE/ci/artifacts/cuspatial/cpu/conda_work/build"
 

diff --git a/ci/release/update-version.sh b/ci/release/update-version.sh
@@ -4,42 +4,25 @@
 #############################
 
 ## Usage
-# bash update-version.sh <type>
-#     where <type> is either `major`, `minor`, `patch`
-
-set -e
-
-# Grab argument for release type
-RELEASE_TYPE=$1
-
-# Get current version and calculate next versions
-CURRENT_TAG=`git tag | grep -xE 'v[0-9\.]+' | sort --version-sort | tail -n 1 | tr -d 'v'`
-CURRENT_MAJOR=`echo $CURRENT_TAG | awk '{split($0, a, "."); print a[1]}'`
-CURRENT_MINOR=`echo $CURRENT_TAG | awk '{split($0, a, "."); print a[2]}'`
-CURRENT_PATCH=`echo $CURRENT_TAG | awk '{split($0, a, "."); print a[3]}'`
-NEXT_MAJOR=$((CURRENT_MAJOR + 1))
-NEXT_MINOR=$((CURRENT_MINOR + 1))
-NEXT_PATCH=$((CURRENT_PATCH + 1))
+# bash update-version.sh <new_version>
+
+
+# Format is YY.MM.PP - no leading 'v' or trailing 'a'
+NEXT_FULL_TAG=$1
+
+# Get current version
+CURRENT_TAG=$(git tag --merged HEAD | grep -xE '^v.*' | sort --version-sort | tail -n 1 | tr -d 'v')
+CURRENT_MAJOR=$(echo $CURRENT_TAG | awk '{split($0, a, "."); print a[1]}')
+CURRENT_MINOR=$(echo $CURRENT_TAG | awk '{split($0, a, "."); print a[2]}')
+CURRENT_PATCH=$(echo $CURRENT_TAG | awk '{split($0, a, "."); print a[3]}')
 CURRENT_SHORT_TAG=${CURRENT_MAJOR}.${CURRENT_MINOR}
-NEXT_FULL_TAG=""
-NEXT_SHORT_TAG=""
-
-# Determine release type
-if [ "$RELEASE_TYPE" == "major" ]; then
-  NEXT_FULL_TAG="${NEXT_MAJOR}.0.0"
-  NEXT_SHORT_TAG="${NEXT_MAJOR}.0"
-elif [ "$RELEASE_TYPE" == "minor" ]; then
-  NEXT_FULL_TAG="${CURRENT_MAJOR}.${NEXT_MINOR}.0"
-  NEXT_SHORT_TAG="${CURRENT_MAJOR}.${NEXT_MINOR}"
-elif [ "$RELEASE_TYPE" == "patch" ]; then
-  NEXT_FULL_TAG="${CURRENT_MAJOR}.${CURRENT_MINOR}.${NEXT_PATCH}"
-  NEXT_SHORT_TAG="${CURRENT_MAJOR}.${CURRENT_MINOR}"
-else
-  echo "Incorrect release type; use 'major', 'minor', or 'patch' as an argument"
-  exit 1
-fi
-
-echo "Preparing '$RELEASE_TYPE' release [$CURRENT_TAG -> $NEXT_FULL_TAG]"
+
+#Get <major>.<minor> for next version
+NEXT_MAJOR=$(echo $NEXT_FULL_TAG | awk '{split($0, a, "."); print a[1]}')
+NEXT_MINOR=$(echo $NEXT_FULL_TAG | awk '{split($0, a, "."); print a[2]}')
+NEXT_SHORT_TAG=${NEXT_MAJOR}.${NEXT_MINOR}
+
+echo "Preparing release $CURRENT_TAG => $NEXT_FULL_TAG"
 
 # Inplace sed replace; workaround for Linux and Mac
 function sed_runner() {
@@ -56,4 +39,4 @@ sed_runner 's/release = .*/release = '"'${NEXT_FULL_TAG}'"'/g' docs/source/conf.
 # bump cudf
 for FILE in conda/environments/*.yml; do
   sed_runner "s/cudf=${CURRENT_SHORT_TAG}/cudf=${NEXT_SHORT_TAG}/g" ${FILE};
-done
+done
diff --git a/conda/environments/cuspatial_dev_cuda11.0.yml b/conda/environments/cuspatial_dev_cuda11.0.yml
@@ -4,15 +4,14 @@ channels:
   - nvidia
   - rapidsai-nightly
   - conda-forge
-  - defaults
 dependencies:
   - clang=8.0.1
   - clang-tools=8.0.1
-  - cudf=0.19.*
+  - cudf=21.06.*
   - cudatoolkit=11.0
   - gdal>=3.0.2
-  - geopandas=0.7.0
-  - cmake>=3.14
+  - geopandas>=0.9.0,<0.10.0a0
+  - cmake>=3.18
   - cython>=0.29,<0.30
   - gtest
   - gmock
diff --git a/...a/environments/cuspatial_dev_cuda10.1.yml → ...a/environments/cuspatial_dev_cuda11.1.yml b/...a/environments/cuspatial_dev_cuda10.1.yml → ...a/environments/cuspatial_dev_cuda11.1.yml
@@ -4,15 +4,14 @@ channels:
   - nvidia
   - rapidsai-nightly
   - conda-forge
-  - defaults
 dependencies:
   - clang=8.0.1
   - clang-tools=8.0.1
-  - cudf=0.19.*
-  - cudatoolkit=10.1
+  - cudf=21.06.*
+  - cudatoolkit=11.1
   - gdal>=3.0.2
-  - geopandas=0.7.0
-  - cmake>=3.14
+  - geopandas>=0.9.0,<0.10.0a0
+  - cmake>=3.18
   - cython>=0.29,<0.30
   - gtest
   - gmock
diff --git a/...a/environments/cuspatial_dev_cuda10.2.yml → ...a/environments/cuspatial_dev_cuda11.2.yml b/...a/environments/cuspatial_dev_cuda10.2.yml → ...a/environments/cuspatial_dev_cuda11.2.yml
@@ -4,15 +4,14 @@ channels:
   - nvidia
   - rapidsai-nightly
   - conda-forge
-  - defaults
 dependencies:
   - clang=8.0.1
   - clang-tools=8.0.1
-  - cudf=0.19.*
-  - cudatoolkit=10.2
+  - cudf=21.06.*
+  - cudatoolkit=11.2
   - gdal>=3.0.2
-  - geopandas=0.7.0
-  - cmake>=3.14
+  - geopandas>=0.9.0,<0.10.0a0
+  - cmake>=3.18
   - cython>=0.29,<0.30
   - gtest
   - gmock
diff --git a/conda/recipes/cuspatial/meta.yaml b/conda/recipes/cuspatial/meta.yaml
@@ -16,6 +16,9 @@ build:
   string: py{{ py_version }}_{{ GIT_DESCRIBE_HASH }}_{{ GIT_DESCRIBE_NUMBER }}
   script_env:
     - VERSION_SUFFIX
+    - CC
+    - CXX
+    - CUDAHOSTCXX
 
 requirements:
   build:
@@ -30,6 +33,7 @@ requirements:
     - cudf {{ minor_version }}.*
     - rmm {{ minor_version }}.*
     - gdal >=3.2.0,<3.3.0a0
+    - geopandas >=0.9.0,<0.10.0a0
 
 test:
   commands:

diff --git a/conda/recipes/libcuspatial/meta.yaml b/conda/recipes/libcuspatial/meta.yaml
@@ -2,14 +2,14 @@
 
 {% set version = environ.get('GIT_DESCRIBE_TAG', '0.0.0.dev').lstrip('v') + environ.get('VERSION_SUFFIX', '') %}
 {% set minor_version =  version.split('.')[0] + '.' + version.split('.')[1] %}
-{% set cuda_version='.'.join(environ.get('CUDA_VERSION', '9.2').split('.')[:2]) %}
+{% set cuda_version='.'.join(environ.get('CUDA', '9.2').split('.')[:2]) %}
 
 package:
   name: libcuspatial
   version: {{ version }}
 
 source:
-  path: ../../..
+  git_url: ../../..
 
 build:
   number: {{ GIT_DESCRIBE_NUMBER }}