Add core__procedure table

[mikix]

Puts another brick in the wall of #139

Checklist

• Consider if documentation in docs/ needs to be updated
  • If you've changed the structure of a table, you may need to run generate-md
  • If you've added/removed core study fields that not in US Core, update our list of those in core-study-details.md
• Consider if tests should be added
• Update template repo if there are changes to study configuration in manifest.toml

[mikix]

So I think this deserves a tiny callout.

For context, the way completion works is that we ignore any encounters for which all the listed associated tables have not been loaded by the ETL yet.

So adding a new resource like this to the list of "required resources" means:

• Transition pain: if a site hasn't ETL'd procedures yet, the next time they rebuild core, all encounters will disappear.
• Ongoing pain: core requires more and more resources (like procedures) to be exported and ETL'd before it can function as expected

However... This pain is also... the point of completion tracking?

Would we want to draw a line between "resources we really care about" and "resources we kind of care about"? If we didn't add Procedure to the encounter-completion check, but your study used core__procedure, you would now be subject to the reasons we added the completion table - engineers ETL'ing data behind the scenes can cause inconsistent/incomplete results when querying.

This "ignore encounters" trick was so that studies didn't have to know about the whole completion table feature. They just would inner join on encounters at some point and incomplete data would be ignored. But another approach is maybe we have a list of 2nd-tier resources for which studies are expected to manually check the completion tables? (the logic isn't necessarily fun, but we could write some docs with examples)

Or we just accept the pain points listed above as we add resources.

[dogversioning]

I'm of two minds about this:

• Long term, I think this is ok/most correct.
• Short term, we are getting a lot of partial drips of data, and this makes it harder for folks to spin up/participate

I kinda want to have a discussion at the product level about what's best here and then circle back on the implementation side of things? We can take this as is for now and then backsolve later if we want.

[github-actions]

☂️ Python Coverage

current status: ✅

Overall Coverage

Lines	Covered	Coverage	Threshold	Status
2391	2386	100%	90%	🟢

New Files

File	Coverage	Status
cumulus_library/studies/core/builder_procedure.py	100%	🟢
TOTAL	100%	🟢

Modified Files

File	Coverage	Status
cumulus_library/builders/counts.py	100%	🟢
cumulus_library/builders/statistics_templates/counts_templates.py	100%	🟢
cumulus_library/studies/core/count_core.py	100%	🟢
TOTAL	100%	🟢

updated for commit: 542f40d by action🐍

[comorbidity]

Should include Procedure Category , which "should" be implemented at most sites because it is easy, BCH Cerner is not a good indicator for procedure coding because it was DIY home grown.

[comorbidity]

Should include Procedure Category , which "should" be implemented at most sites because it is easy, BCH Cerner is not a good indicator for procedure coding because it was DIY home grown.

STU4 https://hl7.org/fhir/us/core/STU4/StructureDefinition-us-core-procedure.html
https://hl7.org/fhir/R4/valueset-procedure-category.html

[mikix]

Should include Procedure Category , which "should" be implemented at most sites because it is easy, BCH Cerner is not a good indicator for procedure coding because it was DIY home grown.

Done!

[dogversioning]

I'm of two minds about this:

• Long term, I think this is ok/most correct.
• Short term, we are getting a lot of partial drips of data, and this makes it harder for folks to spin up/participate

I kinda want to have a discussion at the product level about what's best here and then circle back on the implementation side of things? We can take this as is for now and then backsolve later if we want.

[dogversioning]

you've got to be kidding me