Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skip fields #67

Merged
merged 6 commits into from
May 23, 2023
Merged

Skip fields #67

merged 6 commits into from
May 23, 2023

Conversation

pipliggins
Copy link
Collaborator

This PR allows certain fields, or field patterns, to be skipped if not present in the data, to use when a single study has multiple datasets with a varying amount of data entry.

@pipliggins pipliggins requested a review from abhidg May 19, 2023 12:53
@pipliggins
Copy link
Collaborator Author

@abhidg - this works, but it's taking nearly double the time to run on larger datasets. Any ideas which bit might be slowing it down/ ways to get the speed back up?

@codecov-commenter
Copy link

codecov-commenter commented May 19, 2023

Codecov Report

Merging #67 (835c69d) into main (603fe6f) will decrease coverage by 1.48%.
The diff coverage is 70.90%.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

@@            Coverage Diff             @@
##             main      #67      +/-   ##
==========================================
- Coverage   89.10%   87.63%   -1.48%     
==========================================
  Files           4        4              
  Lines         716      760      +44     
==========================================
+ Hits          638      666      +28     
- Misses         78       94      +16     
Impacted Files Coverage Δ
adtl/__init__.py 82.31% <66.66%> (-2.16%) ⬇️
tests/test_parser.py 100.00% <100.00%> (ø)

adtl/__init__.py Outdated Show resolved Hide resolved
adtl/__init__.py Outdated Show resolved Hide resolved
Copy link
Contributor

@abhidg abhidg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good -- slowdown is probably due to more nested if blocks, Python is not that good with multiple conditionals in for-loops. Could you add some documentation examples? Thanks!

schemas/dev.schema.json Outdated Show resolved Hide resolved
@pipliggins pipliggins marked this pull request as ready for review May 22, 2023 11:57
@pipliggins pipliggins linked an issue May 22, 2023 that may be closed by this pull request
@abhidg
Copy link
Contributor

abhidg commented May 23, 2023

Thanks!

@abhidg abhidg merged commit c0309ee into main May 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow certain fields to be skipped if not present in data
3 participants