feat: Stardist and Spatial Statistics #270

aauker · 2022-01-25T19:16:52Z

New:

pyluna-common/.cli_template.py -> hopefully a nice template for writing new CLI tools
pyluna-pathology/luna/pathology/cli/extract_kfunction_statistics.py
pyluna-pathology/luna/pathology/cli/run_stardist_cell_detection.py

Changes:

pyluna-common/luna/common/utils.py -> add a code timer
pyluna-pathology/luna/pathology/cli/generate_tiles.py -> documentation fix
pyluna-pathology/luna/pathology/cli/visualize_tile_labels_png.py -> option to use um units instead of pixel units

Example notebook (tmp) showing stardist detection, to k-function, to aggregated statistics, and even UMAP!: cell-spatial-stat-examples.html

Update docs

codecov-commenter · 2022-01-26T15:02:11Z

Codecov Report

Merging #270 (fbee7ed) into dev (282b5bd) will increase coverage by 1.87%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##              dev     #270      +/-   ##
==========================================
+ Coverage   66.62%   68.50%   +1.87%     
==========================================
  Files          85       45      -40     
  Lines        4444     3077    -1367     
==========================================
- Hits         2961     2108     -853     
+ Misses       1483      969     -514

Impacted Files	Coverage Δ
...adiology/refined_table/annotation/test_generate.py
...ogy/tests/luna/radiology/common/test_preprocess.py
...ts/luna/arc_test_preprocess_feature_integration.py
pyluna-core/tests/luna/arc_test_dicom_to_graph.py
...ts/luna/arc_test_data_ingestion_template_schema.py
.../tests/luna/arc_test_dicom_to_delta_integration.py
...luna/api/radiologyPreprocessingLibrary/test_app.py
...logy/point_annotation/proxy_table/test_generate.py
...gy/point_annotation/refined_table/test_generate.py
...athology/tests/luna/pathology/common/test_utils.py
... and 30 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 282b5bd...fbee7ed. Read the comment docs.

Update docs

…o qupath-worker-cli-work

armaank

Adds stardist and spatial stats CLIs along with some small utility changes. Demo notebook looks great. Uses concurrent.futures instead of dask since these tasks are short lived.

* chore: merge updated release version to dev 0.1.1 (#264) Automatically generated by python-semantic-release Co-authored-by: Andy Aukerman <ataukerman@gmail.com> Co-authored-by: github-actions <action@github.com> * feat: New CLIs from lung project (#261) All new CLI tools for radiology/pathology work from the lung project. Co-authored-by: Aukerman <aukermaa@pllimsksparky3.mskcc.org> * fix: Save size as list, not a tuple (safe for yaml) Co-authored-by: Aukerman <aukermaa@pllimsksparky3.mskcc.org> * feat: radiology - generate scan table cli (#265) * 1074 final tutorial qc (#267) chore: streamline docker-compose build and execution * chore: 1074 final tutorial qc (#267) chore: streamline docker-compose build and execution * perf: Further refinements to pathology CLI calls (#269) * feat: Stardist and Spatial Statistics (#270) * feat: pathology - adding model trainer, samplers for tile classifiers (#268) * 0.1.1 Automatically generated by python-semantic-release * feat: pathology - adding model trainer, samplers for tile classifiers * Update setup.cfg sklearn dep * fix: added ability to pass modules via yaml, resolved issue with parameterized types * fix: removed custom StratifiedGroupKfold, small changes to get_group_stratified_sampler * feat: adding cli for model training with ray * fix: adding cli, dependencies for model training cli * fix: adding updated dependencies to docker * fix: fixing sklearn import * fix: fixing mistake in Dataset * Fix: changes to type system/param parsing for model training Co-authored-by: Andy Aukerman <ataukerman@gmail.com> Co-authored-by: github-actions <action@github.com> * test: pathology - extract feature vector example (#273) * docs: pathology - extract feature vector example * update docstring * perf: save tile data in h5 (#272) * feat: New slide ETL (#271) * fix: Imports for slide_etl (#274) * fix: Imports for slide_etl * Some fixes to generate tile to match conventions * Fixing coordinates, updating test * refactor * row.tile_size * schema check * Updating tests * -o * ds test * ds test * new otsu tiles * 10x * Remove address assert * Better warning with scale factor business * spacing cleanup * close hf file * minor fix * add subset test * typo fix Co-authored-by: Aukerman <aukermaa@pllimsksparky3.mskcc.org> Co-authored-by: doori <doori.rose@gmail.com> * fix: Inference Model (#275) Refactoring and improvement of inference CLI * feat: adding CLI for generating tile label masks (#276) * fix: removing explicit support for jsons, treat them like dict objects instead * fix: fixing dict param in test cases * fix: changing radiomic params from json to dict * feat: adding CLI for generating tile mask from labels * feat: Update tile schema, add save_tiles (#280) * refactor: dsa visualization and upload tool (#281) * docs: contribution guide (#278) * Update Makefile * feat: adding CLI for shape feature extraction (#282) * test: adding test data for mask generation * feat: adding cli for hif extraction * test: adding test for feature extractor * doc: update docstrings * doc: update docstring with more details * fix: Updated DSA annotation (#279) More complete ETL * fix: Fixes on slide ingestion (#283) * Fixes on slide ingestion * Update adapters.py Co-authored-by: Aukerman <aukermaa@pllimsksparky3.mskcc.org> * feat: precommit w/ black and flake8 (#277) * chore: adding pre-commit w. black/flake8 * sty: testing precommit * chore: adding pre-commit to requirements * docs: adding black badge * fix: fixing flake8 config * sty: remove comments * fix: nifti scan orientation (#284) * feat: pathology - dsa visualize bmps (#285) * fix: Fixes from end-to-end tests (#286) * Fixes from end-to-end tests + goodbye to preprocess.py Co-authored-by: Aukerman <aukermaa@pllimsksparky3.mskcc.org> * test: pathology - unit tests (#287) * test: circleci config (#288) * chore: reorganize and clean up pathology (#289) * fix: run stardist cli with mind docker image (#290) * Init post CLI tool * feat: PET Pre-processing (SUVs) (#292) * PET Processing * Fix indent level * Only propagate keys from non-aliases * Reference origin Co-authored-by: Aukerman <aukermaa@pllimsksparky3.mskcc.org> * feat: Add dataset addition to eligible CLI tools (#293) * Make other tools dataset enabled * add dataset post as part of CLI runner * Add dataset ID options * Revert uncomment * Move to parquet * More parquet migration * Further migrate to parquet * Parquet migration tests pass * Forgot about run detection * Combine ifs * Better loggin * Set language * Better detect url scheme * Docs post_to_dataset * documentation * Last few changes * Fix tests * PNG parquet fix Co-authored-by: Aukerman <aukermaa@pllimsksparky3.mskcc.org> * docs: inference,viz,dsa notebook updates (#294) * docs: update inference tutorial * docs: update dsa tools, add examples, remove configs * docs: dsa annotation nb Co-authored-by: armaan <kohli@cooper.edu> Co-authored-by: doori <doori.rose@gmail.com> Co-authored-by: github-actions <action@github.com> Co-authored-by: Aukerman <aukermaa@pllimsksparky3.mskcc.org> Co-authored-by: arfathpasha <arfathpasha@gmail.com> Co-authored-by: armaan <kohli@cooper.edu>

Aukerman added 7 commits January 21, 2022 18:22

New stardist thing

3c9db5b

Updates

c46355a

cli wrapper

17e2b8c

cell workthrough

6268c62

Timing

026ad32

Finalized

eaaa094

rever utils

63c7835

aauker requested a review from armaank January 26, 2022 14:54

Update run_stardist_cell_detection.py

b5ad69f

Update docs

aauker and others added 7 commits January 26, 2022 10:03

Update extract_kfunction_statistics.py

f793f82

Update docs

Delete Dockerfile

6a4d8ff

Delete stardist_simple.groovy

08e523a

update image

91735da

Merge branch 'qupath-worker-cli-work' of github.com:msk-mind/luna int…

1e1b7d2

…o qupath-worker-cli-work

Finalized

f88e444

prints

754e2d1

armaank approved these changes Jan 26, 2022

View reviewed changes

Nicer print transform::name

fbee7ed

aauker merged commit 8913e08 into dev Jan 26, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Stardist and Spatial Statistics #270

feat: Stardist and Spatial Statistics #270

aauker commented Jan 25, 2022 •

edited

Loading

codecov-commenter commented Jan 26, 2022 •

edited

Loading

armaank left a comment

feat: Stardist and Spatial Statistics #270

feat: Stardist and Spatial Statistics #270

Conversation

aauker commented Jan 25, 2022 • edited Loading

codecov-commenter commented Jan 26, 2022 • edited Loading

Codecov Report

armaank left a comment

Choose a reason for hiding this comment

aauker commented Jan 25, 2022 •

edited

Loading

codecov-commenter commented Jan 26, 2022 •

edited

Loading