Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JP-3075: Updating dynamic_mask function to include inversion functionality #145

Closed

Conversation

kmacdonald-stsci
Copy link
Collaborator

…he 'dq_def' attribute to compress the DQ array.

Resolves JP-3075

This PR addresses round tripping issues for datamodels that use dq_def. The dq_def table details how DQ flags are compressed for saving and how to be uncompressed when opened. When opened, datamodels automatically call dynamic_mask to properly uncompress DQ arrays. However, the inverse was not true during save, which meant that if these datamodels got saved, the uncompressed DQ array got saved to disk, so that when opened again the decompression failed.

An inv parameter got added to the dynamic_mask function to compress uncompressed DQ arrays. This inversion is used for datamodels that use dq_def. In particular, in stdatamodels the inversion is used for ReferenceFileModel during save to ensure the uncompressed DQ array does not get saved, but is properly transformed using the dq_def table before saving.

Checklist

  • added entry in CHANGES.rst (either in Bug Fixes or Changes to API)
  • updated relevant tests
  • updated relevant documentation
  • updated relevant milestone(s)
  • added relevant label(s)

…he 'dq_def' attribute to compress the DQ array.
Copy link
Collaborator

@PaulHuwe PaulHuwe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This passes unit and regression tests for Roman.

@WilliamJamieson
Copy link
Collaborator

spacetelescope/stdatamodels#134 and #146 coordinate moving dynamicdq out of stcal and into stdatamodels. I think its best to just wait for these PRs to merge, and then you can add the dynamicdq changes in this PR to spacetelescope/stdatamodels#132.

@kmacdonald-stsci kmacdonald-stsci deleted the jp_3075 branch April 17, 2023 18:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants