Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: New CLIs from lung project #261

Merged
merged 31 commits into from
Jan 13, 2022
Merged

feat: New CLIs from lung project #261

merged 31 commits into from
Jan 13, 2022

Conversation

aauker
Copy link
Contributor

@aauker aauker commented Dec 20, 2021

Summary of changes:

Library changes:

  • pyluna-common/luna/common/utils.py - Some whitespace issues, added a cli_runner method for CLI tools
  • pyluna-pathology/luna/pathology/common/utils.py - Bug fixes to rasterization (polygon -> bit mask code)
  • pyluna-radiology/luna/radiology/mirp/* -> Small changes to library

Deleted:

  • pyluna-pathology/luna/pathology/cli/extract_slide_texture_features.py - depreciated
  • pyluna-radiology/luna/radiology/cli/collect_csv_segment.py - depreciated
  • pyluna-radiology/luna/radiology/cli/extract_recist_radiomics.py - depreciated
  • pyluna-radiology/luna/radiology/cli/generate_scan.py - became dicom_to_itk
  • pyluna-radiology/luna/radiology/cli/load_dicom.py - unused

Changed CLI:

  • pyluna-pathology/luna/pathology/cli/infer_tile_labels.py

New Pathology CLIs:

  • pyluna-pathology/luna/pathology/cli/extract_stain_texture.py
  • pyluna-pathology/luna/pathology/cli/generate_mask.py

New (or newly updated) Radiology CLIs:

  • pyluna-radiology/luna/radiology/cli/coregister_volumes.py
  • pyluna-radiology/luna/radiology/cli/dicom_to_itk.py
  • pyluna-radiology/luna/radiology/cli/extract_radiomics.py
  • pyluna-radiology/luna/radiology/cli/match_metadata.py
  • pyluna-radiology/luna/radiology/cli/randomize_contours.py
  • pyluna-radiology/luna/radiology/cli/window_volume.py
  • pyluna-radiology/luna/radiology/cli/extract_voxels.py

Added new radiology CLI tests for:

  • pyluna-radiology/luna/radiology/cli/dicom_to_itk.py
  • pyluna-radiology/luna/radiology/cli/extract_radiomics.py
  • pyluna-radiology/luna/radiology/cli/window_volume.py
  • pyluna-radiology/luna/radiology/cli/extract_voxels.py

@codecov-commenter
Copy link

codecov-commenter commented Dec 20, 2021

Codecov Report

Merging #261 (0833af5) into dev (858c996) will increase coverage by 5.85%.
The diff coverage is 83.25%.

Impacted file tree graph

@@            Coverage Diff             @@
##              dev     #261      +/-   ##
==========================================
+ Coverage   53.57%   59.43%   +5.85%     
==========================================
  Files          84       80       -4     
  Lines        6087     4395    -1692     
==========================================
- Hits         3261     2612     -649     
+ Misses       2826     1783    -1043     
Impacted Files Coverage Δ
...tests/luna/pathology/cli/test_infer_tile_labels.py 0.00% <0.00%> (-100.00%) ⬇️
...ogy/tests/luna/pathology/common/test_preprocess.py 0.00% <0.00%> (-100.00%) ⬇️
...luna-radiology/luna/radiology/common/preprocess.py 42.56% <ø> (-18.41%) ⬇️
pyluna-pathology/luna/pathology/common/utils.py 61.98% <37.20%> (+1.21%) ⬆️
...-pathology/luna/pathology/cli/infer_tile_labels.py 67.92% <77.77%> (+2.77%) ⬆️
...-radiology/luna/radiology/cli/extract_radiomics.py 92.59% <93.87%> (ø)
...yluna-radiology/luna/radiology/cli/dicom_to_itk.py 94.23% <94.23%> (ø)
...luna-radiology/luna/radiology/cli/window_volume.py 96.66% <96.66%> (ø)
...una-radiology/luna/radiology/cli/extract_voxels.py 96.29% <100.00%> (ø)
...tests/luna/radiology/cli/test_extract_radiomics.py 100.00% <100.00%> (ø)
... and 28 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 858c996...0833af5. Read the comment docs.

@aauker aauker changed the title wip: Merge from Lung Analysis feat: Merge from Lung Analysis Jan 12, 2022
@aauker aauker changed the title feat: Merge from Lung Analysis feat: New CLIs from lung project Jan 12, 2022
@aauker aauker requested a review from doori January 13, 2022 19:49
Copy link
Contributor

@doori doori left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Massive effort and great improvement to the functions. Great job and thanks for adding the docstrings/unittests.

@aauker aauker merged commit d4df3fc into dev Jan 13, 2022
aauker added a commit that referenced this pull request Apr 14, 2022
* chore: merge updated release version to dev 0.1.1 (#264)

Automatically generated by python-semantic-release

Co-authored-by: Andy Aukerman <ataukerman@gmail.com>
Co-authored-by: github-actions <action@github.com>

* feat: New CLIs from lung project (#261)

All new CLI tools for radiology/pathology work from the lung project.

Co-authored-by: Aukerman <aukermaa@pllimsksparky3.mskcc.org>

* fix: Save size as list, not a tuple (safe for yaml)

Co-authored-by: Aukerman <aukermaa@pllimsksparky3.mskcc.org>

* feat: radiology - generate scan table cli (#265)

* 1074 final tutorial qc (#267)

chore: streamline docker-compose build and execution

* chore: 1074 final tutorial qc (#267)

chore: streamline docker-compose build and execution

* perf: Further refinements to pathology CLI calls (#269)

* feat: Stardist and Spatial Statistics (#270)

* feat: pathology - adding model trainer, samplers for tile classifiers (#268)

* 0.1.1

Automatically generated by python-semantic-release

* feat: pathology - adding model trainer, samplers for tile classifiers

* Update setup.cfg

sklearn dep

* fix: added ability to pass modules via yaml, resolved issue with parameterized types

* fix: removed custom StratifiedGroupKfold, small changes to get_group_stratified_sampler

* feat: adding cli for model training with ray

* fix: adding cli, dependencies for model training cli

* fix: adding updated dependencies to docker

* fix: fixing sklearn import

* fix: fixing mistake in Dataset

* Fix: changes to type system/param parsing for model training

Co-authored-by: Andy Aukerman <ataukerman@gmail.com>
Co-authored-by: github-actions <action@github.com>

* test: pathology - extract feature vector example (#273)

* docs: pathology - extract feature vector example

* update docstring

* perf: save tile data in h5 (#272)

* feat: New slide ETL (#271)

* fix: Imports for slide_etl (#274)

* fix: Imports for slide_etl

* Some fixes to generate tile to match conventions

* Fixing coordinates, updating test

* refactor

* row.tile_size

* schema check

* Updating tests

* -o

* ds test

* ds test

* new otsu tiles

* 10x

* Remove address assert

* Better warning with scale factor business

* spacing cleanup

* close hf file

* minor fix

* add subset test

* typo fix

Co-authored-by: Aukerman <aukermaa@pllimsksparky3.mskcc.org>
Co-authored-by: doori <doori.rose@gmail.com>

* fix: Inference Model (#275)

Refactoring and improvement of inference CLI

* feat: adding CLI for generating tile label masks  (#276)

* fix: removing explicit support for jsons, treat them like dict objects instead

* fix: fixing dict param in test cases

* fix: changing radiomic params from json to dict

* feat: adding CLI for generating tile mask from labels

* feat: Update tile schema, add save_tiles (#280)

* refactor: dsa visualization and upload tool (#281)

* docs: contribution guide (#278)

* Update Makefile

* feat: adding CLI for shape feature extraction  (#282)

* test: adding test data for mask generation

* feat: adding cli for hif extraction

* test: adding test for feature extractor

* doc: update docstrings

* doc: update docstring with more details

* fix: Updated DSA annotation (#279)

More complete ETL

* fix: Fixes on slide ingestion (#283)

* Fixes on slide ingestion

* Update adapters.py

Co-authored-by: Aukerman <aukermaa@pllimsksparky3.mskcc.org>

* feat: precommit w/ black and flake8 (#277)

* chore: adding pre-commit w. black/flake8

* sty: testing precommit

* chore: adding pre-commit to requirements

* docs: adding black badge

* fix: fixing flake8 config

* sty: remove comments

* fix: nifti scan orientation (#284)

* feat: pathology - dsa visualize bmps (#285)

* fix: Fixes from end-to-end tests (#286)

* Fixes from end-to-end tests + goodbye to preprocess.py

Co-authored-by: Aukerman <aukermaa@pllimsksparky3.mskcc.org>

* test: pathology - unit tests (#287)

* test: circleci config (#288)

* chore: reorganize and clean up pathology (#289)

* fix: run stardist cli with mind docker image (#290)

* Init post CLI tool

* feat: PET Pre-processing (SUVs) (#292)

* PET Processing

* Fix indent level

* Only propagate keys from non-aliases

* Reference origin

Co-authored-by: Aukerman <aukermaa@pllimsksparky3.mskcc.org>

* feat: Add dataset addition to eligible CLI tools  (#293)

* Make other tools dataset enabled

* add dataset post as part of CLI runner

* Add dataset ID options

* Revert uncomment

* Move to parquet

* More parquet migration

* Further migrate to parquet

* Parquet migration tests pass

* Forgot about run detection

* Combine ifs

* Better loggin

* Set language

* Better detect url scheme

* Docs post_to_dataset

* documentation

* Last few changes

* Fix tests

* PNG parquet fix

Co-authored-by: Aukerman <aukermaa@pllimsksparky3.mskcc.org>

* docs: inference,viz,dsa notebook updates (#294)

* docs: update inference tutorial

* docs: update dsa tools, add examples, remove configs

* docs: dsa annotation nb

Co-authored-by: armaan <kohli@cooper.edu>

Co-authored-by: doori <doori.rose@gmail.com>
Co-authored-by: github-actions <action@github.com>
Co-authored-by: Aukerman <aukermaa@pllimsksparky3.mskcc.org>
Co-authored-by: arfathpasha <arfathpasha@gmail.com>
Co-authored-by: armaan <kohli@cooper.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants