[REVIEW]: Semi-Automatic Classification Plugin: A Python tool for the download and processing of remote sensing images in QGIS

[whedon]

Submitting author: @semiautomaticgit (Luca Congedo)
Repository: https://github.com/semiautomaticgit/SemiAutomaticClassificationPlugin
Version: v7.9
Editor: @kthyng
Reviewer: @bstabler, @joferkington
Archive: 10.5281/zenodo.5259995

⚠️ JOSS reduced service mode ⚠️

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/d52b171e7799416ec7c95267698b1ae4"><img src="https://joss.theoj.org/papers/d52b171e7799416ec7c95267698b1ae4/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/d52b171e7799416ec7c95267698b1ae4/status.svg)](https://joss.theoj.org/papers/d52b171e7799416ec7c95267698b1ae4)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@bstabler & @joferkington, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @kthyng know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @bstabler

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@semiautomaticgit) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @joferkington

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@semiautomaticgit) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @bstabler, @joferkington it looks like you're currently assigned to review this paper 🎉.

⚠️ JOSS reduced service mode ⚠️

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

⭐ Important ⭐

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

2. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

PDF failed to compile for issue #3172 with the following error:

 Can't find any papers to compile :-(

[whedon]

Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=5.41 s (39.6 files/s, 32843.3 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          76           2050           7624          72310
SVG                            120            120            120          39983
Qt Linguist                      4              0              0          27642
Qt                               5              0              0          26955
XML                              5              0              0            503
Markdown                         2             71              0            117
make                             1             27             56             58
HTML                             1              0              0             13
-------------------------------------------------------------------------------
SUM:                           214           2268           7800         167581
-------------------------------------------------------------------------------


Statistical information for the repository 'decabcd1abe4ba2d7ebac2d6' was
gathered on 2021/04/15.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
Donovan Cameron                  1             3              0            0.00
Karthikeyan Singarav             1             8              9            0.00
Sayantan Majumdar                1             8              8            0.00
luca                            34         61551           7853            2.54
semiautomaticgit               145       1023829        1640416           97.46

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
Karthikeyan Singarav          7           87.5          4.2                0.00
luca                       6187           10.1         74.9               18.57
semiautomaticgit          75795            7.4         16.4                5.62

[whedon]

👋 @joferkington, please update us on how your review is going (this is an automated reminder).

[whedon]

👋 @bstabler, please update us on how your review is going (this is an automated reminder).

[kthyng]

Just a friendly reminder on this review, @bstabler and @joferkington

[kthyng]

Hi @bstabler and @joferkington! When will you be able to start your reviews?

[joferkington]

@kthyng - My apologies for the delay! I'm working on it now and hope to have it done this weekend.

[whedon]

I'm sorry human, I don't understand that. You can see what commands I support by typing:

@whedon commands

[whedon]

Here are some things you can ask me to do:

# List Whedon's capabilities
@whedon commands

# List of editor GitHub usernames
@whedon list editors

# List of reviewers together with programming language preferences and domain expertise
@whedon list reviewers

EDITORIAL TASKS

# Compile the paper
@whedon generate pdf

# Compile the paper from alternative branch
@whedon generate pdf from branch custom-branch-name

# Ask Whedon to check the references for missing DOIs
@whedon check references

# Ask Whedon to check repository statistics for the submitted software
@whedon check repository

[joferkington]

Apologies for the mixup on my part, @kthyng, but it seems I waited to long to accept the invitation link: https://github.com/openjournals/joss-reviews/invitations When I try I get an error with "Sorry, we couldn't find that repository invitation. It is possible that the invitation was revoked or that you are not logged into the invited account."

As a result, I can't check off any items in the review checklist.

Any chance I could get an updated invitation to review? My apologies again for the mixup!

[joferkington]

@whedon generate pdf

[whedon]

PDF failed to compile for issue #3172 with the following error:

 Can't find any papers to compile :-(

[joferkington]

@whedon generate pdf from branch paper

[whedon]

Attempting PDF compilation from custom branch paper. Reticulating splines etc...

[whedon]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[kthyng]

@whedon re-invite @joferkington as reviewer

[whedon]

OK, the reviewer has been re-invited.

@joferkington please accept the invite by clicking this link: https://github.com/openjournals/joss-reviews/invitations

[kthyng]

@joferkington give it a try now!

[joferkington]

@kthyng - That works. Thanks a ton!

[bstabler]

Hi @semiautomaticgit. Thank you for developing this powerful and extensive QGIS Semi-Automatic (Remote Sensing) Classification Plugin (SCP). I was able to install the plugin and work my through Tutorial 1 and Random Forest Classification. There is a lot of material here and so it takes a bit of time to get familiar with all the SCP plugin dialogs, settings, tools, etc. You can see above that I checked all boxes except for two:

• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?

According to your development guidelines, it appears that there is no automatic/organized/suggested testing framework? I think some kind of testing framework is needed for when people less familiar with the source code want to contribute and need a process for ensuring revisions not only work for their use case, but also work for other use cases (i.e. the revisions do not have unintended consequences). Essentially there needs to be adequate test coverage. I wonder if you can use the QGIS unittesting framework?

• A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?

I think the statement of need really starts at line 61. Before line 61, the section summarizes the need for remote sensing, which is good for the common reader, but I think it is not really needed (or could be condensed) since it is not the specific contribution of your work. Your work is a convenience, easy-to-use, open source toolkit for doing remote sensing in QGIS that makes use of third party algorithms.

And one small comment for the paper, the Python multiprocessing library uses subprocesses instead of threads so I suggest revising line 18.

Thanks!

[semiautomaticgit]

Hi @bstabler , thank you very much for reviewing my work!
I'm really glad that almost all the boxes were checked.
About your comments:

According to your development guidelines, it appears that there is no automatic/organized/suggested testing framework? I think some kind of testing framework is needed for when people less familiar with the source code want to contribute and need a process for ensuring revisions not only work for their use case, but also work for other use cases (i.e. the revisions do not have unintended consequences). Essentially there needs to be adequate test coverage. I wonder if you can use the QGIS unittesting framework?

Unfortunately I don't think I can use QGIS unit testing framework with my plugin.
At the moment there is a test unit for checking the correct installation of dependencies and basic functionalities (such as numpy, GDAL subprocess, internet connection), which can be run from the menu Settings > Debug.
Of course I'm planning to improve these test units also for other plugin functions.
Also, the steps for reporting a bug (activating the log file) are described in the documentation documentation .
Do you think it could be enough if I cite this in the paper?

I think the statement of need really starts at line 61. Before line 61, the section summarizes the need for remote sensing, which is good for the common reader, but I think it is not really needed (or could be condensed) since it is not the specific contribution of your work. Your work is a convenience, easy-to-use, open source toolkit for doing remote sensing in QGIS that makes use of third party algorithms.

I'm going to reduce the length of introduction about remote sensing, and highlight the characteristics of my work.

And one small comment for the paper, the Python multiprocessing library uses subprocesses instead of threads so I suggest revising line 18.

I'll correct this, and I'm going to submit a new version.
Thank you again for your time.

[kthyng]

@semiautomaticgit When you look at the paper generated just above, are the references corrected so that none of them have "http" in the doi? They all still have "http" when I look, but sometimes I keep looking at an old version and can't seem to find the newest version to look at.

[danielskatz]

@whedon check references from branch paper

[whedon]

Attempting to check references... from custom branch paper

[whedon]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1016/S0305-9006(03)00066-7 is OK
- 10.1002/9781119457107.ch4 is OK
- 10.3390/rs8040299 is OK
- 10.1016/j.rse.2019.111402 is OK
- 10.1016/j.autcon.2020.103477 is OK
- 10.1177/1940082918822411 is OK
- 10.1016/j.scitotenv.2018.08.430 is OK
- 10.3390/environments6030036 is OK
- 10.3390/ijgi8020056 is OK
- 10.3390/rs12193242 is OK
- 10.3390/w13040396 is OK
- 10.3390/rs11202337 is OK
- 10.3390/ijgi10020076 is OK
- 10.3390/su10051593 is OK
- 10.1016/0034-4257(93)90013-N is OK
- 10.1023/A:1010933404324  is OK
- 10.5281/zenodo.4066373 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[danielskatz]

@kthyng - The references seem fine to me

[kthyng]

@danielskatz Can you see them fixed in the pdf? I see that whedon isn't finding the dois to be invalid anymore but I think previously when it found 3 there were actually more that it wasn't finding for some reason. I wanted to be sure they were fixed by being able to see them.

[danielskatz]

I think JOSS always puts the DOIS as full URLs in the reference list. It wants then without the prefix in the bib file, and uses them in this way in the XML, but adds the prefix to make them clickable in the PDF

[kthyng]

ha! Oh I didn't realize that! Ok, thanks.

[kthyng]

@semiautomaticgit all good to go then!

[kthyng]

@whedon accept deposit=true

[whedon]

No archive DOI set. Exiting...

[kthyng]

@whedon set 10.5281/zenodo.5259995 as archive

[whedon]

OK. 10.5281/zenodo.5259995 is the archive.

[kthyng]

@whedon accept deposit=true

[whedon]

Doing it live! Attempting automated processing of paper acceptance...

[whedon]

PDF failed to compile for issue #3172 with the following error:

 Can't find any papers to compile :-(

[danielskatz]

try @whedon accept deposit=true from branch paper

[kthyng]

I'm having a hard time this morning huh.

[kthyng]

@whedon accept deposit=true from branch paper

[whedon]

Doing it live! Attempting automated processing of paper acceptance...

[whedon]

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

[whedon]

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

0. Check final PDF and Crossref metadata that was deposited 👉 Creating pull request for 10.21105.joss.03172 joss-papers#2542
1. Wait a couple of minutes, then verify that the paper DOI resolves https://doi.org/10.21105/joss.03172
2. If everything looks good, then close this review issue.
3. Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? Notify your editorial technical team...

[kthyng]

Congrats on your new publication @semiautomaticgit!! Thanks to reviewers @bstabler and @joferkington for your hard work, time, and expertise!

[whedon]

🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.03172/status.svg)](https://doi.org/10.21105/joss.03172)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.03172">
  <img src="https://joss.theoj.org/papers/10.21105/joss.03172/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.03172/status.svg
   :target: https://doi.org/10.21105/joss.03172

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://joss.theoj.org/reviewer-signup.html
• Making a small donation to support our running costs here: https://numfocus.org/donate-to-joss

[semiautomaticgit]

I am very happy of this publication in Journal of Open Source Software!

Thank you very much @bstabler and @joferkington for your valuable reviews!
Also, thanks to @danielskatz for your help!
Finally, I am very grateful to @kthyng for the hard work as editor!

Best wishes to all!