Questionnaire Flattener

[AdritRao]

Questionnaire Flattener

♻️ Current situation & Problem

The issue is https://github.com/StanfordSpezi/SpeziDataPipelineTemplate/issues/11. The flattener for questionnaire responses should be able to use phoenix-generated surveys to convert the question and response IDs in Firebase to text.

⚙️ Release Notes

• Added the ability for a user to input Phoenix-generated JSON surveys to flatten_fhir_resources and for the question and answer mappings to be extracted. For the questions, I mapped the linkId to the text. For answers, because they have both a system ID and choice code, I combined the system id and code and used that as the mapping key. https://github.com/StanfordSpezi/SpeziDataPipelineTemplate/blob/cca4c5d2501a1b66234c240f0066aa39dfc0fc6b/data_flattening/fhir_resources_flattener.py#L508
• Extracts the question and answer IDs from the individual responses and then gets them from the survey mappings (if the answer is a valueCoding). If it is not a valueCoding, it will just use the integer or string value. https://github.com/StanfordSpezi/SpeziDataPipelineTemplate/blob/cca4c5d2501a1b66234c240f0066aa39dfc0fc6b/data_flattening/fhir_resources_flattener.py#L541
• The flattener then generates a data frame with the Date, UserID, SurveyID, QuestionID, QuestionText, and Answer. https://github.com/StanfordSpezi/SpeziDataPipelineTemplate/blob/cca4c5d2501a1b66234c240f0066aa39dfc0fc6b/data_flattening/fhir_resources_flattener.py#L573

📝 Code of Conduct & Contributing Guidelines

By submitting creating this pull request, you agree to follow our Code of Conduct and Contributing Guidelines:

• I agree to follow the Code of Conduct and Contributing Guidelines.

[Vicbi]

Thank you for your great work @AdritRao ! I went through the QuestionnaireResponseFlattener() and I've provided a couple of comments and suggestions. Let me know what you think and we can sync back! 🙌

[AdritRao]

Thank you for reviewing the PR @Vicbi! I have modified the code based on your suggestions in the recent commits. Please let me know if you have any other suggestions.

[Vicbi]

Hi @AdritRao,

Great job addressing the comments, and thanks for your efforts! 🚀

I've merged the main branch into survey_flattener and resolved the resulting conflicts. Feel free to review the updated branch. Also, it would be super helpful if you could address the lint issues reported in the GitHub Actions (e.g.,. adding documentation to the class/functions and performing some refactoring) before we proceed with the merge. Let me know if you need any clarifications!

[codecov]

Codecov Report

Attention: Patch coverage is 33.33333% with 54 lines in your changes are missing coverage. Please review.

Project coverage is 44.48%. Comparing base (c519aa9) to head (5b0cb7f).

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #12      +/-   ##
==========================================
- Coverage   46.05%   44.48%   -1.57%     
==========================================
  Files          10       10              
  Lines         721      796      +75     
==========================================
+ Hits          332      354      +22     
- Misses        389      442      +53     

Flag	Coverage Δ
unittests	44.48% <33.34%> (-1.57%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
data_visualization/data_visualizer.py	35.27% <ø> (ø)
data_flattening/fhir_resources_flattener.py	55.94% <33.34%> (-12.39%)	⬇️

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c519aa9...5b0cb7f. Read the comment docs.

[Vicbi]

Thank you @AdritRao for working on the pylint issues! 🚀 I've made some additional modifications and I am happy to merge the branch to main!