Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Coverage tool #4

Merged
merged 33 commits into from
Aug 26, 2022
Merged

Coverage tool #4

merged 33 commits into from
Aug 26, 2022

Conversation

daisieh
Copy link
Member

@daisieh daisieh commented Aug 15, 2022

To test, use the data from https://github.com/CanDIG/mohccn-data. Check out this branch in the clinical_ETL_code submodule there, and run:

python validate_coverage.py --input ../Synthetic_Clinical_Data_2 --manifest ../mappings/synthetic_clinical/manifest.yml

The tool will:

  • validate the mapping with the katsu internal schema and report any mismatches.
  • quantify how much of the provided clinical data is covered by the mapping.
  • report which schema fields are not used by the mapping at all.

@daisieh daisieh requested review from kcranston and yavyx August 15, 2022 19:32
@yavyx
Copy link
Contributor

yavyx commented Aug 18, 2022

  • Coverage tool works as expected with synthetic data.
  • Add instructions to use coverage tool on README.
  • Consider using a DataFrame to store the mapping coverage table for pretty printing and easy cross-cohort comparisons/analyses.
  • Consider adding docstrings to functions to briefly describe their inputs and outputs.

BC POG data throws an error but appears to work properly anyways.

@kcranston
Copy link
Member

kcranston commented Aug 26, 2022

This is awesome! I agree with @yavyx that using a dataframe for a pretty printing option would be nice (since we already have a pandas requirement). That can be its own issue, though. Doesn't need to hold this up.

Yay! Data reports!

@kcranston kcranston merged commit 119c49a into main Aug 26, 2022
@kcranston kcranston deleted the daisieh/coverage branch August 26, 2022 00:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants