Merge branch 'dev' into 279-test-coord-saving

.github/workflows/joss-draft-pdf.yml

@@ -0,0 +1,23 @@
+on: [push]
+
+jobs:
+  paper:
+    runs-on: ubuntu-latest
+    name: Paper Draft
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+      - name: Build draft PDF
+        uses: openjournals/openjournals-draft-action@master
+        with:
+          journal: joss
+          # This should be the path to the paper within your repo.
+          paper-path: paper/paper.md
+      - name: Upload
+        uses: actions/upload-artifact@v1
+        with:
+          name: paper
+          # This is the output path where Pandoc will write the compiled
+          # PDF. Note, this should be the same directory as the input
+          # paper.md
+          path: paper/paper.pdf

.github/workflows/mr_ci.yml

@@ -31,6 +31,7 @@ jobs:
       - name: Install dependencies
         run: |
           python -m pip install ".[dev]"
+          python -m pip install pytest-cov
 
       - name: Quality Assurance
         run: |
@@ -41,4 +42,15 @@ jobs:
 
       - name: Test with pytest
         run: |
-          python -m pytest ./tests
+          python -m pytest --cov=./ --cov-report=xml ./tests
+
+      - name: Upload coverage to Codecov
+        uses: codecov/codecov-action@v3
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          directory: ./coverage/reports/
+          env_vars: OS,PYTHON
+          fail_ci_if_error: true
+          files: ./coverage.xml,!./cache
+          flags: unittests
+          name: codecov-umbrella

README.md

@@ -18,6 +18,9 @@
     </a>
     <a href="https://zenodo.org/badge/latestdoi/430661738"><img src="https://zenodo.org/badge/430661738.svg" alt="DOI"></a>
     <br/>
+    <a href="https://codecov.io/github/Living-with-machines/MapReader" >
+    <img src="https://codecov.io/github/Living-with-machines/MapReader/graph/badge.svg?token=38GQ3O1GB5"/>
+    </a>
 </p>
 
 

docs/source/Project-cv.rst

@@ -1,4 +1,4 @@
-Project CV
+Project Curriculum Vitae
 ===========
 
 .. Notes to editors:

docs/source/User-guide/Classify/Classify.rst

@@ -20,13 +20,12 @@ This is all done within MapReader's ``ClassifierContainer()`` class, which is us
 - Predict labels of unannotated images (model inference).
 - Visualize datasets and predictions.
 
-If you already have a fine-tuned model, you can skip to the `Infer labels using a fine-tuned model <https://mapreader.readthedocs.io/en/latest/Classify/Infer.html>`_ page.
+If you already have a fine-tuned model, you can skip to the `Infer labels using a fine-tuned model <https://mapreader.readthedocs.io/en/latest/User-guide/Classify/Infer.html>`_ page.
 
-If not, you should proceed to the `Train/fine-tune a classifier <https://mapreader.readthedocs.io/en/latest/Classify/Train.html>`_ page.
+If not, you should proceed to the `Train/fine-tune a classifier <https://mapreader.readthedocs.io/en/latest/User-guide/Classify/Train.html>`_ page.
 
 .. toctree::
    :maxdepth: 1
 
    Train
    Infer
-

docs/source/User-guide/Classify/Train.rst

@@ -231,6 +231,8 @@ There are a number of options for the ``model`` argument:
         .. code-block:: python
 
             #EXAMPLE
+            from mapreader import ClassifierContainer
+
             my_classifier = ClassifierContainer("resnet18", annotated_images.labels_map, dataloaders)
 
         By default, this will load a pretrained form of the model and reshape the last layer to output the same number of nodes as labels in your dataset.
@@ -242,10 +244,13 @@ There are a number of options for the ``model`` argument:
 
         .. code-block:: python
 
+
             #EXAMPLE
             from torchvision import models
             from torch import nn
 
+            from mapreader import ClassifierContainer
+
             my_model = models.resnet18(pretrained=True)
 
             # reshape the final layer (FC layer) of the neural network to output the same number of nodes as label in your dataset
@@ -270,6 +275,8 @@ There are a number of options for the ``model`` argument:
             #EXAMPLE
             import torch
 
+            from mapreader import ClassifierContainer
+
             my_model = torch.load("./models/model_checkpoint_6.pkl")
 
             my_classifier = ClassifierContainer(my_model, annotated_images.labels_map, dataloaders)
@@ -285,6 +292,8 @@ There are a number of options for the ``model`` argument:
             .. code-block:: python
 
                 #EXAMPLE
+                from mapreader import ClassifierContainer
+
                 my_classifier = ClassifierContainer(None, None, None, load_path="./models/checkpoint_6.pkl")
 
             This will also load the corresponding model file (in this case "./models/model_checkpoint_6.pkl").
@@ -301,6 +310,8 @@ There are a number of options for the ``model`` argument:
             #EXAMPLE
             from transformers import AutoFeatureExtractor, AutoModelForImageClassification
 
+            from mapreader import ClassifierContainer
+
             extractor = AutoFeatureExtractor.from_pretrained("davanstrien/autotrain-mapreader-5000-40830105612")
             my_model = AutoModelForImageClassification.from_pretrained("davanstrien/autotrain-mapreader-5000-40830105612")
 
@@ -316,6 +327,8 @@ There are a number of options for the ``model`` argument:
             #EXAMPLE
             import timm
 
+            from mapreader import ClassifierContainer
+
             my_model = timm.create_model("hf_hub:timm/resnest101e.in1k", pretrained=True, num_classes=len(annotated_images.labels_map))
 
             my_classifier = ClassifierContainer(my_model, annotated_images.labels_map, dataloaders)

docs/source/User-guide/Load.rst

@@ -80,11 +80,13 @@ For example, if you have downloaded your maps using the default settings of our
     Other arguments you may want to specify when adding metadata to your images include:
 
     - ``index_col`` - By default, this is set to ``0`` so the first column of your csv/excel spreadsheet will be used as the index column when creating a pandas dataframe. If you would like to use a different column you can specify ``index_col``.
-    - ``columns`` - By default, the ``.add_metadata()`` method will add all the columns in your metadata to your ``MapImages`` object. If you would like to add only specific columns, you can pass a list of these as the ``columns``\s argument (e.g. ``columns=[`name`, `coordinates`, `region`]``) to add only these columns to your ``MapImages`` object.
+    - ``columns`` - By default, the ``add_metadata()`` method will add all the columns in your metadata to your ``MapImages`` object. If you would like to add only specific columns, you can pass a list of these as the ``columns``\s argument (e.g. ``columns=[`name`, `coordinates`, `region`]``) to add only these columns to your ``MapImages`` object.
     - ``ignore_mismatch``- By default, this is set to ``False`` so that an error is given if the images in your ``MapImages`` object are mismatched to your metadata. Setting ``ignore_mismatch`` to ``True`` (by specifying ``ignore_mismatch=True``) will allow you to bypass this error and add mismatched metadata. Only metadata corresponding to images in your ``MapImages`` object will be added.
     - ``delimiter`` - By default, this is set to ``|``. If your csv file is delimited using a different delimiter you should specify the delimiter argument.
 
 
+.. note:: In MapReader versions < 1.0.7, coordinates were miscalculated. To correct this, use the ``add_coords_from_grid_bb()`` method to calculate new, correct coordinates.
+
 Patchify
 ----------
 
@@ -184,7 +186,7 @@ As above, you can use the ``path_save`` argument to change where these patches a
     Other arguments you may want to specify when patchifying your images include:
 
     - ``square_cuts`` - By default, this is set to ``False``. Thus, if your ``patch_size`` is not a factor of your image size (e.g. if you are trying to slice a 100x100 pixel image into 8x8 pixel patches), you will end up with some rectangular patches at the edges of your image. If you set ``square_cuts=True``, then all your patches will be square, however there will be some overlap between edge patches. Using ``square_cuts=True`` is useful if you need square images for model training, and don't want to warp your rectangular images by resizing them at a later stage.
-    - ``add_to_parent`` - By default, this is set to ``True`` so that each time you run ``.patchify_all()`` your patches are added to your ``MapImages`` object. Setting it to ``False`` (by specifying ``add_to_parent=False``) will mean your patches are created, but not added to your ``MapImages`` object. This can be useful for testing out different patch sizes.
+    - ``add_to_parent`` - By default, this is set to ``True`` so that each time you run ``patchify_all()`` your patches are added to your ``MapImages`` object. Setting it to ``False`` (by specifying ``add_to_parent=False``) will mean your patches are created, but not added to your ``MapImages`` object. This can be useful for testing out different patch sizes.
     - ``rewrite`` - By default, this is set to ``False`` so that if your patches already exist they are not overwritten. Setting it to ``True`` (by specifying ``rewrite=True``) will mean already existing patches are recreated and overwritten.
 
 If you would like to save your patches as geo-referenced tiffs (i.e. geotiffs), use:
@@ -193,10 +195,12 @@ If you would like to save your patches as geo-referenced tiffs (i.e. geotiffs),
 
     my_files.save_patches_as_geotiffs()
 
-This will save each patch in your ``MapImages`` object as a ``.geotiff`` file in your patches directory.
+This will save each patch in your ``MapImages`` object as a georeferenced ``.tif`` file in your patches directory.
+
+.. note:: MapReader also has a ``save_parents_as_geotiff()`` method for saving parent images as geotiffs.
 
-After running the ``.patchify_all()`` method, you'll see that ``print(my_files)`` shows you have both 'parents' and 'patches'.
-To view an iterable list of these, you can use the ``.list_parents()`` and ``.list_patches()`` methods:
+After running the ``patchify_all()`` method, you'll see that ``print(my_files)`` shows you have both 'parents' and 'patches'.
+To view an iterable list of these, you can use the ``list_parents()`` and ``list_patches()`` methods:
 
 .. code-block:: python
 
@@ -229,7 +233,7 @@ or
 
 .. note:: These parent and patch dataframes **will not** automatically update so you will want to run this command again if you add new information into your ``MapImages`` object.
 
-At any point, you can also save these dataframes by passing the ``save`` argument to the ``.convert_images()`` method:
+At any point, you can also save these dataframes by passing the ``save`` argument to the ``convert_images()`` method:
 
 .. code-block:: python
 
@@ -280,7 +284,7 @@ If, however, you want to see a random sample of your patches use the ``tree_leve
 
 
 It can also be helpful to see your patches in the context of their parent image.
-To do this use the ``.show()`` method.
+To do this use the ``show()`` method.
 
 e.g. :
 
@@ -312,7 +316,7 @@ This will show you your chosen patches, by default highlighted with red borders,
 .. admonition:: Advanced usage
     :class: dropdown
 
-    Further usage of the ``.show()`` method is detailed in :ref:`Further_analysis`.
+    Further usage of the ``show()`` method is detailed in :ref:`Further_analysis`.
     Please head there for guidance on advanced usage.
 
 You may also want to see all the patches created from one of your parent images.
@@ -330,7 +334,7 @@ This can be done using:
 .. admonition:: Advanced usage
     :class: dropdown
 
-    Further usage of the ``.show_parent()`` method is detailed in :ref:`Further_analysis`.
+    Further usage of the ``show_parent()`` method is detailed in :ref:`Further_analysis`.
     Please head there for guidance on advanced usage.
 
 .. todo:: Move 'Further analysis/visualization' to a different page (e.g. as an appendix)
@@ -341,13 +345,13 @@ Further analysis/visualization (optional)
 -------------------------------------------
 
 If you have loaded geographic coordinates into your ``MapImages`` object, you may want to calculate the central coordinates of your patches.
-The ``.add_center_coord()`` method can used to do this:
+The ``add_center_coord()`` method can used to do this:
 
 .. code-block:: python
 
     my_files.add_center_coord()
 
-You can then rerun the ``.convert_images()`` method to see your results.
+You can then rerun the ``convert_images()`` method to see your results.
 
 i.e.:
 
@@ -358,15 +362,15 @@ i.e.:
 
 You will see that center coordinates of each patch have been added to your patch dataframe.
 
-The ``.calc_pixel_stats()`` method can be used to calculate means and standard deviations of pixel intensities of each of your patches:
+The ``calc_pixel_stats()`` method can be used to calculate means and standard deviations of pixel intensities of each of your patches:
 
 .. code-block:: python
 
     my_files.calc_pixel_stats()
 
-After rerunning the ``.convert_images()`` method (as above), you will see that mean and standard pixel intensities have been added to your patch dataframe.
+After rerunning the ``convert_images()`` method (as above), you will see that mean and standard pixel intensities have been added to your patch dataframe.
 
-The ``.show()`` and ``.show_parent()`` methods can be used to plot these values ontop of your patches.
+The ``show()`` and ``show_parent()`` methods can be used to plot these values ontop of your patches.
 This is done by specifying the ``column_to_plot`` argument.
 
 e.g. to view "mean_pixel_R" on your patches:
@@ -394,12 +398,12 @@ e.g. to view "mean_pixel_R" on your patches:
 .. image:: ../figures/show_par_RGB_0.5.png
     :width: 400px
 
-.. note:: The ``column_to_plot`` argument can also be used with the ``.show()`` method.
+.. note:: The ``column_to_plot`` argument can also be used with the ``show()`` method.
 
 .. admonition:: Advanced usage
     :class: dropdown
 
-    Other arguments you may want to specify when showing your images (for both the ``.show()`` and ``.show_parent()`` methods):
+    Other arguments you may want to specify when showing your images (for both the ``show()`` and ``show_parent()`` methods):
 
     - ``plot_parent`` - By default, this is set to ``True`` so that the parent image is shown. If you would like to remove the parent image, e.g. if you are plotting column values, you can set ``plot_parent=False``. This should speed up the code for plotting.
     - ``patch_border`` - By default, this is set to ``True`` so that borders are plotted around each patch. Setting ``patch_border`` to ``False`` (by specifying ``patch_border=False``) will stop patch borders being shown.