Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Phenotips importer update and corresponding Migration Script #102

Merged
merged 22 commits into from
Jul 7, 2023

Conversation

fatimarabab
Copy link
Collaborator

@fatimarabab fatimarabab commented Jul 6, 2023

Checklist before requesting a review

  • I have performed a self-review of my code.
  • My code follows the style guidelines enforced by static analysis tools.
  • If it is a core feature, I have added thorough tests.
  • My changes generate no new warnings.
  • New and existing unit tests pass locally with my changes.

Pull Request Details

Wrike Tickets - Update the phenotips importer to create new sections that are needed from presentations as of Spring 2023
As a clinical researcher, I need a place to enter information regarding each gene's molecular mechanism data per gene in a case

Changes made:

  • Updated the sections in the "backend/src/core/phenotips_importer.py" to reflect the analysis section.
  • Crafted a migration script to migrate the old cases to the new structure (move data to different sections and create new sections)

To Review:

  • Static Analysis by Reviewer

  • The migration script migrates the production installation works as intended/rendered correctly.
    To check this run the following commands:

    1. Download the archived production database to your local machine. See @SeriousHorncat for the URL to download it from Box.

    2. Import the production database backup to Rosalution from the rosalution root directory

    # ./etc/database/restore-database.sh <container name> <absolute-file-path-to-backup-archive>
    ./etc/database/restore-database.sh rosalution-rosalution-db-1 rosalution-db-2023-07-06-1688674415.archive 
    1. Run the migration script
    docker exec -it rosalution-rosalution-db-1 mongosh /tmp/fixtures/migrations/migrate-new-sections-phenotips-importer.js
    1. Login to Rosalution as "developer" and view the cases below
    • CPAM0092
      image

    • CPAM0099
      image

    • CPAM0082
      image

  • Existing Fixtures are updated correctly to verify the new sections are added correctly

    • CPAM0084
      image

    • CPAM0002
      image

  • All Github Actions checks have passed.

@fatimarabab fatimarabab marked this pull request as ready for review July 7, 2023 14:38
@JmScherer
Copy link
Collaborator

JmScherer commented Jul 7, 2023

./etc/database/restore-database.sh rosalution-rosalution-db-1 /Users/jscherer/Downloads/rosalution-db-2023-07-06-1688674415.archive

Screenshot 2023-07-07 at 10 19 50 AM

docker exec -it rosalution-rosalution-db-1 mongosh /tmp/fixtures/migrations/migrate-new-sections-phenotips-importer.js

Screenshot 2023-07-07 at 10 18 51 AM

@JmScherer
Copy link
Collaborator

JmScherer commented Jul 7, 2023

There appears to be missing information for Clinical History the following Analyses:

Screenshot provided by the Pull Request:

Screenshot 2023-07-07 at 10 24 36 AM

Screenshot from my review:

Screenshot 2023-07-07 at 10 23 41 AM

This has been resolved:

Screenshot 2023-07-07 at 10 39 35 AM

@fatimarabab fatimarabab merged commit 103c171 into analysis-image-sections Jul 7, 2023
@fatimarabab fatimarabab deleted the phenotips-importer-update branch July 7, 2023 16:00
fatimarabab added a commit that referenced this pull request Jul 7, 2023
…mages Attachments (#103)

* wip to refactoring to supporting multiple sections in an analysis to add attachments

* wip for analysis backend integration with the adding/updating/removing

* debugging whats going on

* got it working?

* reduced the CSS footprint for the gene box component and the dom to be simpler

* wip for consolidating the css

* conslidated work that was here

* wip for refactoring the backend

* fixed frontend linting issues

* Refactoring to make the emits work, starting work on the updating section image in the backend

* Image edit update and removal working

* Replaced the component back into the sectionbox and am chaining emits from sectionimage to sectionbox to analysis view

* fixed removing an image, it now properly reacts to the image removed

* Updated the backend analysis collection attach image to use field instead of dataset to fix image attachment

* Updated the analysis image system tests, changed system-tests/e2e/attach_pedigree_image.cy.js to be attach_analysis_section_image.cy.js

* Knocked out some system tests, working on the others. I think there's a bug, need to check

* Working out some of the system tests,
they won't work until the collapsible and empty content boxes are resolved

* fixed collasping of the section boxes in annotationview and made preliminary changes to the system test for collapsing

* Work in progress to resolve CSS and remove the force render

* Fixed most of the linting for python

* Refactored to not require the force re-render for frontend images

* frontend css changes to make progress to editing  case"
git push

* System tests passing

* disabled duplicate code check

* formatted backend code with yapf

* Frontned  unit tests updated along with additional console logging on frontend removed for debugging

* Updated test_analysis_collection.py to reflect the new tests for the analysis section images. Also removed a few functions from analysis_collection.py pretaining to pedigree. Lastly, updated some text fixtures to more accurately reflect what the structure is

* Fixed up the integration tests for the analysis_router.py

* Linting

* Removed some comments

* Linting again

* removed special coloring to help resolve cleaning up and consolidating the CSS

* Phenotips importer update and corresponding Migration Script (#102)

* Added new sections to analysis

* Modified phenotips importer to show sections for multiple genes in analysis

* progress in migration script

* Fixed backend linting

* Fixed up phenotips_importer.py and added a test

* Added migration script to reconfigure analysis sections. Gene sections throwing error now

* Updated the CSS for the section text so that it wraps and doesn't go beyhond the bounds of its container.

* Updated gene sections to reconfigure where HPO terms are displayed

* Removed duplicate Clinical History/HPO terms loop

* added the pedigree image if it exists

* Removed print statements and included pedgree image migration

* added type to sections

* Added images-dataset type for fields

* script fix, verified can only run it once succesfully

* Updated the base fixtures to use the new analysis system

* Updated phenotips importer to have field names match header

* backend linting fixes

* Updated the system tests for the image attaching

* Fixed the formatting that came up with yapf

* Resovled the system tests

* Added some logging to the script to be informative of the output and fixed a few things found in the process

* Pushign up fix for the clinical history

---------

Co-authored-by: James Scherer <jscherer@uab.edu>
Co-authored-by: Angelina Elizabeth Uno-Antonison <ange.unoantonison@gmail.com>

* Updating system test to match the miggrated fixtures

---------

Co-authored-by: James Scherer <jscherer@uab.edu>
Co-authored-by: Rabab Fatima <rfatima@uab.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants