v2.1 (#170)

* Remove functionality to invite members to team (#167) * Create _copy_thread which just copies the threads, copy_thread now copies replies (#169) * Fix createchallenge bug (#171) * Make sure to not redefine 'challenge' * Dont pass object into rest post call * Fix tests * Add in sage name and year * Automate code coverage (#165) * Add in testing of synapse service module * Test code coverage * Fix * Add init * Update * Remove * Add inits * Add inits * Remove * Try uploading artifact * Add in flake8 * install all dependencies * Rename to build * Remove duplicated test func * Refactor mirror wiki (#143) * Update version * Refactor mirrowiki * Update mirrorwiki * Add dry run flag, fix docs * Remove comments * Increase update wiki pages and maps helper * Add test mirrorwiki * Make most functions convenience functions * Fix indent * add tests * Add test * Fix * Add integration test * Add integration tests * Remove synapseclient<2 support for challenge.py * Add -f * Add in stdout for wiki titles that exist in destionation but not entity * Update tests/test_mirrorwiki.py * Add to documentation (#175) * Add in development requirements * Add to contributing guide on how to develop * Update documentation * Update version and add to changelog * Update CONTRIBUTING.md * Add release procedure * Add in more contributing guide * Edit PR template * Update .github/PULL_REQUEST_TEMPLATE.md Co-authored-by: Verena Chung <9377970+vpchung@users.noreply.github.com> Co-authored-by: Verena Chung <9377970+vpchung@users.noreply.github.com> Co-authored-by: Verena Chung <9377970+vpchung@users.noreply.github.com> Co-authored-by: verena <vpchung@users.noreply.github.com>
Sage-Bionetworks · May 16, 2020 · 5a4722f · 5a4722f
1 parent d3fdaee
commit 5a4722f
Show file tree

Hide file tree

Showing 20 changed files with 702 additions and 204 deletions.
diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md
@@ -1,3 +1,3 @@
-- [ ] Did you reference the issue being addressed by number? (fixes #xxxx)
-- [ ] Did you run your tests locally (instructions [here](https://github.com/Sage-Bionetworks/challengeutils/blob/master/CONTRIBUTING.md)?
+- [ ] Fixes #xxxx (Link the github issue if applicable)
+- [ ] Did you run your tests locally (instructions [here](https://github.com/Sage-Bionetworks/challengeutils/blob/master/CONTRIBUTING.md))?
 - [ ] Did you add a good description about your changes or additions?
diff --git a/.github/workflows/pythonapp.yml b/.github/workflows/pythonapp.yml
@@ -1,7 +1,7 @@
 # This workflow will install Python dependencies, run tests and lint with a single version of Python
 # For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions
 
-name: challengeutils test
+name: build
 
 on: [pull_request]
 
@@ -21,15 +21,22 @@ jobs:
     - name: Install dependencies
       run: |
         python -m pip install --upgrade pip
+        pip install flake8 pytest pytest-cov
         pip install .
         if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
-   # - name: Lint with flake8
-   #   run: |
+    - name: Lint with flake8
+      run: |
         # stop the build if there are Python syntax errors or undefined names
-        # flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
+        flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
         # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
-        # flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
+        flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
     - name: Test with pytest
       run: |
-        pip install pytest
-        pytest
+        pytest tests/ --doctest-modules --cov=challengeutils --cov-report=html
+    - name: Upload pytest test results
+      uses: actions/upload-artifact@master
+      with:
+        name: pytest-results-${{ matrix.python-version }}
+        path: htmlcov
+      # Use always() to always run this step to publish test results when there are test failures
+      if: ${{ always() }}
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -1,6 +1,9 @@
 
 ## Contributing
 
+We welcome all contributions!  Please head to [issues](https://github.com/Sage-Bionetworks/challengeutils/issues) to either file any bugs/feature requests or find a task you want to assist with.  Make sure to assign yourself the task if you decide to work on it.
+
+
 ### Fork and clone this repository
 
 See the [Github docs](https://help.github.com/articles/fork-a-repo/) for how to make a copy (a fork) of a repository to your own Github account.
@@ -16,6 +19,13 @@ git checkout develop
 git pull upstream develop
 ```
 
+### Install development dependencies
+This will install all the dependencies of the package including the active branch of `challengeutils`.
+
+```
+pip install -r requirements-dev.txt
+```
+
 ### The development life cycle
 
 1. Pull the latest content from the `develop` branch of this central repository (not your fork).
@@ -40,3 +50,25 @@ pytest -vs tests/
 ```
 
 Tests are also run automatically by Travis on any pull request and are required to pass before merging.
+
+
+### Documentation
+
+`challengeutils` uses [Google style Python docstrings](https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html).  All functions you add must abide by these standards.  The Documentation of each python module can be found in `docs/python` folder and are auto generated by `sphinx`.  If you are adding a new module, you must add to a `.rst` file into this folder.  If you are adding or modifying a command line client function, you must add/modify the files in `docs/client`.
+
+To build documentation:
+
+```
+cd docs
+make html
+open _build/html/index.html
+```
+
+### Release Procedure (For Package Maintainers)
+
+* Always merge all new features into `develop` branch first (unless it is a minor patch into `master`)
+* update `__version__.py` to not have `-dev`
+* update `docs/about/changelog.md` with release information
+* Merge develop into master branch
+* Check documentation build for docs: https://sage-bionetworks.github.io/challengeutils/
+* Create release tag (`v...`) and include release notes.  Also include any known bugs for each release here.
diff --git a/LICENSE b/LICENSE
@@ -186,7 +186,7 @@
       same "printed page" as the copyright notice for easier
       identification within third-party archives.
 
-   Copyright [yyyy] [name of copyright owner]
+   Copyright 2017 Sage Bionetworks
 
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.

diff --git a/challengeutils/__main__.py b/challengeutils/__main__.py
@@ -21,17 +21,20 @@
 
 
 def command_mirrorwiki(syn, args):
-    """For all challenges, you should be editting the staging site and then
-    using the merge script to mirror staging to live site.  The script will
-    compare wiki titles between the staging and live site and update the live
-    site with respect to what has changed on the staging site.  Note, this is
-    different from copying the wikis. To copy the wikis, please look at
-    synapseutils.
+    """Mirrors (sync) wiki pages by using the wikipage titles between two
+    Synapse Entities.  This function only works if `entity` and `destination`
+    are the same type and both must have wiki pages.  Only wiki pages with the
+    same titles will be copied from `entity` to `destination` - if there is
+    a wiki page that you want to add, you will have to create a wiki page
+    first in the `destination` with the same name.
+
+    Note: this is different from copying a wiki - to copy a wiki, please look
+    at `synapseutils.copyWiki`.
 
     >>> challengeutils mirrorwiki syn12345 syn23456
     """
-    mirrorwiki.mirrorwiki(syn, args.entityid, args.destinationid,
-                          args.forceupdate)
+    mirrorwiki.mirror(syn, args.entityid, args.destinationid,
+                      force=args.force, dryrun=args.dryrun)
 
 
 def command_createchallenge(syn, args):
@@ -278,27 +281,39 @@ def build_parser():
               " there is already a live site"))
     parser_createChallenge.set_defaults(func=command_createchallenge)
 
-    parser_mirrorWiki = subparsers.add_parser(
+    parser_mirrorwiki = subparsers.add_parser(
         'mirrorwiki',
-        help=("This command mirrors wiki pages. It relies on the wiki titles "
-              "between two Synapse Projects to be the same and will merge the "
-              "updates from entity's wikis to destination's wikis. "
-              "Do not confuse this function with copy wiki."))
+        help="Mirrors (sync) wiki pages by using the wikipage titles between "
+             "two Synapse Entities. This function only works if `entity` and "
+             "`destination`are the same type and both must have wiki pages. "
+             "Only wiki pages with the same titles will be copied from "
+             "`entity` to `destination` - if there is a wiki page that you "
+             "want to add, you will have to create a wiki page first in the "
+             "`destination` with the same name."
+    )
 
-    parser_mirrorWiki.add_argument(
+    parser_mirrorwiki.add_argument(
         "entityid",
         type=str,
-        help="Synapse Id of the project's wiki changes you have staged")
-    parser_mirrorWiki.add_argument(
+        help="Synapse Id of the project's wiki changes you have staged"
+    )
+    parser_mirrorwiki.add_argument(
         "destinationid",
         type=str,
-        help=('Synapse Id of project whose wiki you want to update'
-              ' with the entityid'))
-    parser_mirrorWiki.add_argument(
-        "--forceupdate",
+        help="Synapse Id of project whose wiki you want to update "
+             "with the entityid"
+    )
+    parser_mirrorwiki.add_argument(
+        "-f", "--force",
+        action='store_true',
+        help='Update the wikipages even if they are the same'
+    )
+    parser_mirrorwiki.add_argument(
+        "--dryrun",
         action='store_true',
-        help='Update the wikipages even if they are the same')
-    parser_mirrorWiki.set_defaults(func=command_mirrorwiki)
+        help="Show the pages that have changed but don't update."
+    )
+    parser_mirrorwiki.set_defaults(func=command_mirrorwiki)
 
     parser_query = subparsers.add_parser(
         'query',

diff --git a/challengeutils/__version__.py b/challengeutils/__version__.py
@@ -1 +1 @@
-__version__ = "2.0.1"
+__version__ = "2.1.0"
diff --git a/challengeutils/challenge.py b/challengeutils/challenge.py
@@ -3,11 +3,7 @@
 from typing import Union, Iterator
 
 from synapseclient import Project, Synapse, Team
-try:
-    from synapseclient.core.utils import id_of
-except ModuleNotFoundError:
-    # For synapseclient < v2.0
-    from synapseclient.utils import id_of
+from synapseclient.core.utils import id_of
 
 from .synapseservices.challenge import Challenge
 
@@ -36,10 +32,10 @@ def create_challenge(self, teamid: str, projectid: str) -> Challenge:
             A synapseservices.Challenge
 
         """
-        new_challenge = Challenge(participantTeamId=teamid,
-                                  projectId=projectid)
+        challenge_object = {'participantTeamId': teamid,
+                            'projectId': projectid}
         challenge = self.syn.restPOST('/challenge',
-                                      str(new_challenge))
+                                      json.dumps(challenge_object))
         return Challenge(**challenge)
 
     def get_registered_challenges(self,
@@ -93,11 +89,11 @@ def update_challenge(self, challengeid: str, teamid: str = None,
             A synapseservices.Challenge
 
         """
-        new_challenge = Challenge(id=challengeid,
-                                  participantTeamId=teamid,
-                                  projectId=projectid)
+        challenge_object = {'id': challengeid,
+                            'participantTeamId': teamid,
+                            'projectId': projectid}
         challenge = self.syn.restPUT(f'/challenge/{challengeid}',
-                                     str(new_challenge))
+                                     json.dumps(challenge_object))
         return Challenge(**challenge)
 
     def delete_challenge(self, challengeid: str):

diff --git a/challengeutils/createchallenge.py b/challengeutils/createchallenge.py
@@ -1,20 +1,18 @@
-"""Creates challenge space in Synapse
-
-Input:  Challenge Project name
-Output: The skeleton for two challenges site with initial wiki, four teams
-        (admin, participants, organizers, and preregistrants), and
-        a challenge widget added on live site with a participant
-        team associated with it.
+"""The skeleton for two challenges site with initial wiki, four teams
+(admin, participants, organizers, and preregistrants), and
+a challenge widget added on live site with a participant
+team associated with it.
 
 For more information on challenges administration:
 https://docs.synapse.org/articles/challenge_administration.html
 
-Example (run on bash)
+Example::
 
->>> challengeutils createchallenge "Plouf Challenge"
+    import challengeutils
+    import synapseclient
+    syn = synapseclient.login()
+    challengeutils.createchallenge.main(syn, "Plouf Challenge")
 
-TODO Add participants
-TODO Add tests
 """
 import logging
 import sys
@@ -27,6 +25,7 @@
 
 logger = logging.getLogger(__name__)
 
+# TODO: Add participants
 # A pre-defined wiki project is used as initial template for challenge sites.
 # To copy over the template synapseutils.copyWiki() function is used with
 # template id as source and new challenge project entity synId as destination.
@@ -281,8 +280,8 @@ def main(syn, challenge_name, live_site=None):
     else:
         project_live = syn.get(live_site)
 
-    challenge = create_challenge_widget(syn, project_live,
-                                        teams['team_part_id'])
+    challenge_obj = create_challenge_widget(syn, project_live,
+                                            teams['team_part_id'])
     create_evaluation_queue(syn, '%s Project Submission' % challenge_name,
                             'Project Submission',
                             project_live.id)
@@ -304,7 +303,7 @@ def main(syn, challenge_name, live_site=None):
     for page in new_wikiids:
         wikipage = syn.getWiki(project_staging, page['id'])
         wikipage.markdown = _update_wikipage_string(wikipage.markdown,
-                                                    challenge.id,
+                                                    challenge_obj.id,
                                                     teams['team_part_id'],
                                                     challenge_name,
                                                     project_live.id)

diff --git a/challengeutils/discussion.py b/challengeutils/discussion.py
@@ -357,6 +357,23 @@ def create_thread_reply(syn, threadid, message):
 
 
 def copy_thread(syn, thread, project):
+    """Copies a discussion thread and its replies to a project
+
+    Args:
+        syn: synapse object
+        thread: Synapse Thread
+        project: Synapse Project or its id to copy thread to
+
+    Returns:
+        dict: Thread bundle
+    """
+    new_thread_obj = _copy_thread(syn, thread, project)
+    thread_replies = get_thread_replies(syn, thread['id'])
+    for reply in thread_replies:
+        copy_reply(syn, reply, new_thread_obj['id'])
+
+
+def _copy_thread(syn, thread, project):
     """Copies a discussion thread to a project
 
     Args:
@@ -374,8 +391,9 @@ def copy_thread(syn, thread, project):
     on_behalf_of = "On behalf of @{user}\n\n".format(user=username)
     text = get_thread_text(syn, thread['messageKey'])
     new_thread_text = on_behalf_of + text
+    new_thread_obj = create_thread(syn, projectid, title, new_thread_text)
 
-    return create_thread(syn, projectid, title, new_thread_text)
+    return new_thread_obj
 
 
 def copy_reply(syn, reply, thread):
@@ -408,7 +426,4 @@ def copy_forum(syn, project, new_project):
     """
     threads = get_forum_threads(syn, project)
     for thread in threads:
-        new_thread_obj = copy_thread(syn, thread, new_project)
-        thread_replies = get_thread_replies(syn, thread['id'])
-        for reply in thread_replies:
-            copy_reply(syn, reply, new_thread_obj['id'])
+        copy_thread(syn, thread, new_project)
diff --git a/challengeutils/evaluation_queue.py b/challengeutils/evaluation_queue.py
@@ -1,6 +1,8 @@
 """Evaluation Queue helper functions"""
 import time
 
+from synapseclient import Synapse
+
 
 def _convert_date_to_epoch(date_string):
     """Converts local time dates in YEAR-MM-DDTHH:MM:SS format
@@ -83,7 +85,7 @@ def _create_quota(round_start: str = None, round_end: str = None,
     return quota
 
 
-def set_evaluation_quota(syn: 'Synapse', evalid: int, **kwargs):
+def set_evaluation_quota(syn: Synapse, evalid: int, **kwargs):
     """Sets evaluation submission limit quota.  This WILL erase any old quota
     you had previously set. Note - round_start must be specified with either
     round_end or round_duration and number_of_rounds must be defined for the