Add initial Python unit/performance tests for TIFF loader module

.github/ISSUE_TEMPLATE/bug_report.md

@@ -1,6 +1,6 @@
 ---
 name: Bug report
-about: Create a bug report to help us improve ___PROJECT___
+about: Create a bug report to help us improve cuCIM
 title: "[BUG]"
 labels: "? - Needs Triage, bug"
 assignees: ''
@@ -18,7 +18,7 @@ A clear and concise description of what you expected to happen.
 
 **Environment details (please complete the following information):**
  - Environment location: [Bare-metal, Docker, Cloud(specify cloud provider)]
- - Method of ___PROJECT___ install: [conda, Docker, or from source]
+ - Method of cuCIM install: [conda, Docker, or from source]
    - If method of install is [Docker], provide `docker pull` & `docker run` commands used
 
 

.github/ISSUE_TEMPLATE/feature_request.md

@@ -1,14 +1,14 @@
 ---
 name: Feature request
-about: Suggest an idea for ___PROJECT___
+about: Suggest an idea for cuCIM
 title: "[FEA]"
 labels: "? - Needs Triage, feature request"
 assignees: ''
 
 ---
 
 **Is your feature request related to a problem? Please describe.**
-A clear and concise description of what the problem is. Ex. I wish I could use ___PROJECT___ to do [...]
+A clear and concise description of what the problem is. Ex. I wish I could use cuCIM to do [...]
 
 **Describe the solution you'd like**
 A clear and concise description of what you want to happen.

.github/ISSUE_TEMPLATE/submit-question.md

@@ -1,6 +1,6 @@
 ---
 name: Submit question
-about: Ask a general question about ___PROJECT___
+about: Ask a general question about cuCIM
 title: "[QST]"
 labels: "? - Needs Triage, question"
 assignees: ''

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,6 +1,6 @@
 <!--
 
-Thank you for contributing to ___PROJECT___ :)
+Thank you for contributing to cuCIM :)
 
 Here are some guidelines to help the review process go smoothly.
 

3rdparty/LICENSE.gputil

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2017 anderskm
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

3rdparty/LICENSE.psutil

@@ -0,0 +1,29 @@
+BSD 3-Clause License
+
+Copyright (c) 2009, Jay Loden, Dave Daeschler, Giampaolo Rodola'
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the following conditions are met:
+
+ * Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+ * Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+ * Neither the name of the psutil authors nor the names of its contributors
+   may be used to endorse or promote products derived from this software without
+   specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
+ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
+ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

3rdparty/LICENSE.pytest-lazy-fixture

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2016 Marsel Zaripov
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

LICENSE-3rdparty.md

@@ -237,3 +237,24 @@ libcuckoo
   - https://github.com/efficient/libcuckoo/blob/master/LICENSE
 - Copyright: Carnegie Mellon University and Intel Corporation
 - Usage: Using concurrent hash table implementation for cache mechanism.
+
+pytest-lazy-fixture
+- License: MIT License
+  - https://github.com/TvoroG/pytest-lazy-fixture/blob/master/LICENSE
+- Copyright: Marsel Zaripov
+- Usage: Using lazy fixture feature in PyTest.
+
+psutil
+- License: BSD-3-Clause License
+  - https://github.com/giampaolo/psutil/blob/master/LICENSE
+- Copyright:
+  - Jay Loden
+  - Dave Daeschler
+  - Giampaolo Rodola
+- Usage: Checking memory usage in Python applications.
+
+gputil
+- License: MIT License
+  - https://github.com/anderskm/gputil/blob/master/LICENSE.txt
+- Copyright: Anders Krogh Mortensen
+- Usage: Checking memory usage in Python applications.

README.md

@@ -16,15 +16,15 @@
 
 #### Conda (stable)
 
-> conda create -n cucim -c rapidsai -c conda-forge/label/cupy_rc -c conda-forge cucim cudatoolkit=`<CUDA version>`
+> conda create -n cucim -c rapidsai -c conda-forge cucim cudatoolkit=`<CUDA version>`
 
 `<CUDA version>` should be 11.0+ (e.g., `11.0`, `11.2`, etc.)
 
 **NOTE:** The first cuCIM conda package (v0.19.0) would be available on 4/19/2021.
 
 #### Conda (nightlies)
 
-> conda create -n cucim -c rapidsai-nightly -c conda-forge/label/cupy_rc -c conda-forge cucim cudatoolkit=`<CUDA version>`
+> conda create -n cucim -c rapidsai-nightly -c conda-forge cucim cudatoolkit=`<CUDA version>`
 
 `<CUDA version>` should be 11.0+ (e.g., `11.0`, `11.2`, etc)
 

ci/gpu/build.sh

@@ -109,6 +109,5 @@ else
     cat /etc/hosts
 
     gpuci_logger "Python py.test for cuCIM"
-    cd $WORKSPACE/python/cucim
-    py.test --cache-clear -v --ignore-glob . --rootdir=src
+    ./run test python all
 fi

cpp/src/cache/image_cache_manager.cpp

@@ -33,7 +33,7 @@ uint32_t preferred_memory_capacity(const std::vector<uint32_t>& image_size,
                                    const std::vector<uint32_t>& patch_size,
                                    uint32_t bytes_per_pixel)
 {
-    // https://godbolt.org/z/9884oh9sG for test
+    // https://godbolt.org/z/eMf946oE7 for test
 
     if (image_size.size() != 2 || tile_size.size() != 2 || patch_size.size() != 2)
     {
@@ -45,7 +45,8 @@ uint32_t preferred_memory_capacity(const std::vector<uint32_t>& image_size,
     uint32_t tile_accross_count = (image_size[0] + (tile_size[0] - 1)) / tile_size[0];
 
     // The maximal number of tiles (y-axis) overapped with the given patch
-    uint32_t patch_down_count = (patch_size[1] + (tile_size[1] - 1)) / tile_size[1] + 1;
+    uint32_t patch_down_count =
+        std::min(image_size[1] + (tile_size[1] - 1), patch_size[1] + (tile_size[1] - 1)) / tile_size[1] + 1;
 
     // (tile_accross_count) x (tile width) x (tile_height) x (patch_down_count) x (bytes per pixel)
     uint64_t bytes_needed =

docs/adr/README.md

@@ -1,8 +1,8 @@
 # Architecture Decision Record
 
-Documents in this folder followes the concept from [this blog - Why Write ADRs?](https://github.blog/2020-08-13-why-write-adrs/).
+Documents in this folder follow the concept from [this blog - Why Write ADRs?](https://github.blog/2020-08-13-why-write-adrs/).
 
-Please refer to to https://github.com/joelparkerhenderson/architecture_decision_record for examples.
+Please refer to https://github.com/joelparkerhenderson/architecture_decision_record for examples.
 
 
 ## ADR file name conventions

notebooks/Using_Cache.ipynb

@@ -174,7 +174,7 @@
     "\n",
     "cuCIM provide a utility method (`cucim.clara.cache.preferred_memory_capacity()`) to calculate a preferred cache memory size for the given image (image size and tile size) and the patch size.\n",
     "\n",
-    "Internal logic is available at <https://godbolt.org/z/Kj4e9rqd3>\n"
+    "Internal logic is available at <https://godbolt.org/z/jY7G84xzT>\n"
    ]
   },
   {
@@ -924,4 +924,4 @@
  },
  "nbformat": 4,
  "nbformat_minor": 5
-}
+}

python/cucim/.coveragerc

@@ -1,29 +1,29 @@
 # .coveragerc to control coverage.py
+# Refer to: https://coverage.readthedocs.io/en/latest/config.html#syntax
+
 [run]
+source =
+    cucim
 omit =
     # omit vendored files
     */_vendored/*
-    src/cucim/time.py
+    # somehow, above doesn't omit the following files so add below specifically.
+    */cucim/skimage/_vendored/*
     # omit versioneer file
-    src/cucim/_version.py
+    */cucim/_version.py
+    # omit non-relevant files
+    */cucim/__init__.py
+    */cucim/__main__.py
+    */cucim/time.py
+    */cucim/_misc.py
+    */cucim/clara/cli.py
 
+[paths]
+source =
+    src
+    */site-packages
 
-[html]
-directory = coverage_html_report
 
-# [paths]
-# source =
-#    src
-#    */site-packages
-#
-# [run]
-# branch = true
-# source =
-#     src/cucim
-#     tests
-# parallel = true
-#
-# [report]
-# show_missing = true
-# precision = 2
-# omit = *migrations*
+[report]
+show_missing = true
+precision = 2

python/cucim/requirements-test.txt

@@ -0,0 +1,8 @@
+GPUtil==1.4.0
+imagecodecs==2021.6.8
+openslide-python==1.1.2
+psutil==5.8.0
+pytest==6.2.4
+pytest-cov==2.12.1
+pytest-lazy-fixture==0.6.3
+tifffile==2021.7.2

python/cucim/setup.cfg

@@ -51,7 +51,9 @@ addopts =
     # --doctest-modules
     # --doctest-glob=\*.rst
     --tb=short
+    --ignore-glob build
 testpaths =
+    src
     tests
 
 [tool:isort]

python/cucim/tests/conftest.py

@@ -0,0 +1,17 @@
+#
+# Copyright (c) 2021, NVIDIA CORPORATION.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+# Fixtures
+from .fixtures.testimage import *  # noqa