-
Notifications
You must be signed in to change notification settings - Fork 16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Housekeeping data testing and validation for CoDICE l0 #176
Housekeeping data testing and validation for CoDICE l0 #176
Conversation
I added an initial pass at some unit tests (basically took what @tech3371 has for SWE). We will have to think about what to add to validate CoDICE housekeeping data specifically, but this adds in the basic testing structure for now. |
…ode review suggestions
…omparing it with the decommutated housekeeping packet
I had to change a few things that where outside the general change summary. I had to update the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice work here @GFMoraga!
0c6fcea
into
IMAP-Science-Operations-Center:dev
…Operations-Center#176) Validated housekeeping data with tests plus minor updates --------- Co-authored-by: Matthew Bourque <matthew.bourque@lasp.colorado.edu>
…Operations-Center#176) Validated housekeeping data with tests plus minor updates --------- Co-authored-by: Matthew Bourque <matthew.bourque@lasp.colorado.edu>
* update telemetry_generator to be more generic * Updates to telemetry generator * updating docs * updating to remove local-specific testing changes * first pass at glows xtce * update way of checking if description exists or not. update generator template. * updated generator to use correct IntegerParameterType * First pass at GLOWS decom * updating telemetry generator * Updating glows decom to properly process data into dictionary * Finished histogram decom * Updating decom to include direct events * Finalizing code, adding smaller test data file and combined XML, updating documentation and tests * Updating test file path * Fixing test runs * updating docs * Updating data classes to use attributes, and changing names * Removing keys from data class and getting attribute names from the packet * Move INT to SINT in telemetry_generator Co-authored-by: Tenzin Choedon <36522642+tech3371@users.noreply.github.com> * Addressing code review comments * Fixing test failure * Review updates * fixing typo * Updating IntEnum to Enum to fix automatic documentation * Move L0 data classes to a dataclass type with specific attributes * Initial GLOWS L1A data classes * Adding required attributes to L0 data class * Adding L1A histogram data class * fixing test failures * Finishing ccsds dataclass * Minor updates * Finishing GLOWS L1A histograms * Updating to include direct events data * Updating glows l1a * Adding useful GLOWS code from GLOWS team and Marek Strumik * Adding direct event information Co-authored-by: GLOWS team <maro@cbk.waw.pl> * Updates for direct event creation * Updates to L1A direct events * Updated release workflow to reflect removal of "main" branch and use of version branches * Reduced the filesize of git-workflow.png * Glows L0 changes to data class to accomodate L1A needs (#264) * Glows L0 changes to data class to accomodate L1A needs * DOC: Add intersphinx to documentation for external references * DOC: Add a clean option to the documentation Makefile build command * DOC: Update glows types in dataclasses * DOC: Change glows currentmodule references * DOC: Add ccsds header to glows documentation * DOC: Add inherited member description for glows This adds the inherited attributes from the dataclass directly into the subclasses. There may be a better way to do this, so perhaps some setting/configuration parameter needs to be updated? But this at least fixes the warnings for now. * FIX: Run black on glows test * PR updates --------- Co-authored-by: Greg Lucas <greg.m.lucas@gmail.com> * Housekeeping data testing and validation for CoDICE l0 (#176) Validated housekeeping data with tests plus minor updates --------- Co-authored-by: Matthew Bourque <matthew.bourque@lasp.colorado.edu> * L1 cdf creation (#179) * Extracting and adding new data to the IDEX L1 * Adding some of the values to the XML, rather than shifting bits in the code (work in progress, need to do a few more variables) * Finishing up having space packet parser decom * Getting rid of unecessary bit masksi8kt9gloh * making changes based on comments * Formatting the file after the upstream sync * Adding L1 CDF creation * Forgot to commit poetry changes * Making changes based on Greg's comments, as well as issues 1-8 from SPDF * Fixing import statements * Changing version into a string * Adding more descriptions, units, and labels in the attributes * Fixing the way strings are inserted into the CDF * Fixing issues with white spice in the text of attrs * Files are finally ISTP compliant! * Fixing a couple things for ISTP * Had the contents of the 32 bit things reversed * Fixing the packet definition for the CCSDS file * More fixes to ensure ISTP compliance * Updating the file name * touching finishes on the L1 CDF * Adding fixture for temp directory in tests * Fixing one of the ruff errors by adding a namedtuple * Adding new "write_cdf" function that writed cdfs based on attributes * Getting rid of a few more magic numbers * Updating the packet definition * initial hit l1a decom (#164) * initial hit l1a decom * using defaultdict, add docs * fixed hit l1a docstrings * fixed hit l1a docstring formatting * doc updates and minor fixes * minor docstring update * minor comment update * rebased with dev * fixed ruff issues * regenerated xtce * fixed unit test * github doc gen fix attempt * added class docstring attributes * fixed more doc issues * another doc fix attempt * doc fix attempt * doc fix attempt * changed doc python v to 3.10 * I-ALiRT: L0 parsing (#246) * adding ialirt * CDF documentation update (#273) * CDF documentation update * Adding the cdf_guide.rst to the toc * Formatting update! * Updating the docs for sections I missed * Formatting a little nicer * Adding links enabled in the docs * Fixing duplicate headers * Getting rid of duplicate headers * Fixing more documentation * Responding to a few code review comments * Breaking up the page into sub-pages * Adding a link to xarray_to_cdf page * Adding more links to different sections * fixing image link * Editting some of the wording now that we're on different pages * Typos and format fixed * Another format error * Create CDF class based system (#272) * Updating IDEX CDF attrs * remove comments, create constants class * Updating names and addressing PR comments * Updating docs * Adding tests * Updating types of L0 to properly read bytearrays * Updating tests * GLOWS direct events compressed and uncompressed * updating decom dataclasses * Updating GLOWS L0 * L1A tests * removing unneeded code * updating tests and fixing minor bug * fixing autodocs * addressing comments from code review * Updating constants to a frozen dataclass * Updating direct events packet reading to correctly read packet data, updating test data, adding tests * Updating docstrings --------- Co-authored-by: Tenzin Choedon <tenzin.choedon@lasp.colorado.edu> Co-authored-by: Tenzin Choedon <36522642+tech3371@users.noreply.github.com> Co-authored-by: GLOWS team <maro@cbk.waw.pl> Co-authored-by: Matthew Bourque <matthew.bourque@lasp.colorado.edu> Co-authored-by: Greg Lucas <greg.m.lucas@gmail.com> Co-authored-by: Gabriel M <104743000+GFMoraga@users.noreply.github.com> Co-authored-by: Bryan Harter <41062454+bryan-harter@users.noreply.github.com> Co-authored-by: Sean Hoyt <sean.hoyt@lasp.colorado.edu> Co-authored-by: Laura Sandoval <46567335+laspsandoval@users.noreply.github.com>
* update telemetry_generator to be more generic * Updates to telemetry generator * updating docs * updating to remove local-specific testing changes * first pass at glows xtce * update way of checking if description exists or not. update generator template. * updated generator to use correct IntegerParameterType * First pass at GLOWS decom * updating telemetry generator * Updating glows decom to properly process data into dictionary * Finished histogram decom * Updating decom to include direct events * Finalizing code, adding smaller test data file and combined XML, updating documentation and tests * Updating test file path * Fixing test runs * updating docs * Updating data classes to use attributes, and changing names * Removing keys from data class and getting attribute names from the packet * Move INT to SINT in telemetry_generator Co-authored-by: Tenzin Choedon <36522642+tech3371@users.noreply.github.com> * Addressing code review comments * Fixing test failure * Review updates * fixing typo * Updating IntEnum to Enum to fix automatic documentation * Move L0 data classes to a dataclass type with specific attributes * Initial GLOWS L1A data classes * Adding required attributes to L0 data class * Adding L1A histogram data class * fixing test failures * Finishing ccsds dataclass * Minor updates * Finishing GLOWS L1A histograms * Updating to include direct events data * Updating glows l1a * Adding useful GLOWS code from GLOWS team and Marek Strumik * Adding direct event information Co-authored-by: GLOWS team <maro@cbk.waw.pl> * Updates for direct event creation * Updates to L1A direct events * Updated release workflow to reflect removal of "main" branch and use of version branches * Reduced the filesize of git-workflow.png * Glows L0 changes to data class to accomodate L1A needs (IMAP-Science-Operations-Center#264) * Glows L0 changes to data class to accomodate L1A needs * DOC: Add intersphinx to documentation for external references * DOC: Add a clean option to the documentation Makefile build command * DOC: Update glows types in dataclasses * DOC: Change glows currentmodule references * DOC: Add ccsds header to glows documentation * DOC: Add inherited member description for glows This adds the inherited attributes from the dataclass directly into the subclasses. There may be a better way to do this, so perhaps some setting/configuration parameter needs to be updated? But this at least fixes the warnings for now. * FIX: Run black on glows test * PR updates --------- Co-authored-by: Greg Lucas <greg.m.lucas@gmail.com> * Housekeeping data testing and validation for CoDICE l0 (IMAP-Science-Operations-Center#176) Validated housekeeping data with tests plus minor updates --------- Co-authored-by: Matthew Bourque <matthew.bourque@lasp.colorado.edu> * L1 cdf creation (IMAP-Science-Operations-Center#179) * Extracting and adding new data to the IDEX L1 * Adding some of the values to the XML, rather than shifting bits in the code (work in progress, need to do a few more variables) * Finishing up having space packet parser decom * Getting rid of unecessary bit masksi8kt9gloh * making changes based on comments * Formatting the file after the upstream sync * Adding L1 CDF creation * Forgot to commit poetry changes * Making changes based on Greg's comments, as well as issues 1-8 from SPDF * Fixing import statements * Changing version into a string * Adding more descriptions, units, and labels in the attributes * Fixing the way strings are inserted into the CDF * Fixing issues with white spice in the text of attrs * Files are finally ISTP compliant! * Fixing a couple things for ISTP * Had the contents of the 32 bit things reversed * Fixing the packet definition for the CCSDS file * More fixes to ensure ISTP compliance * Updating the file name * touching finishes on the L1 CDF * Adding fixture for temp directory in tests * Fixing one of the ruff errors by adding a namedtuple * Adding new "write_cdf" function that writed cdfs based on attributes * Getting rid of a few more magic numbers * Updating the packet definition * initial hit l1a decom (IMAP-Science-Operations-Center#164) * initial hit l1a decom * using defaultdict, add docs * fixed hit l1a docstrings * fixed hit l1a docstring formatting * doc updates and minor fixes * minor docstring update * minor comment update * rebased with dev * fixed ruff issues * regenerated xtce * fixed unit test * github doc gen fix attempt * added class docstring attributes * fixed more doc issues * another doc fix attempt * doc fix attempt * doc fix attempt * changed doc python v to 3.10 * I-ALiRT: L0 parsing (IMAP-Science-Operations-Center#246) * adding ialirt * CDF documentation update (IMAP-Science-Operations-Center#273) * CDF documentation update * Adding the cdf_guide.rst to the toc * Formatting update! * Updating the docs for sections I missed * Formatting a little nicer * Adding links enabled in the docs * Fixing duplicate headers * Getting rid of duplicate headers * Fixing more documentation * Responding to a few code review comments * Breaking up the page into sub-pages * Adding a link to xarray_to_cdf page * Adding more links to different sections * fixing image link * Editting some of the wording now that we're on different pages * Typos and format fixed * Another format error * Create CDF class based system (IMAP-Science-Operations-Center#272) * Updating IDEX CDF attrs * remove comments, create constants class * Updating names and addressing PR comments * Updating docs * Adding tests * Updating types of L0 to properly read bytearrays * Updating tests * GLOWS direct events compressed and uncompressed * updating decom dataclasses * Updating GLOWS L0 * L1A tests * removing unneeded code * updating tests and fixing minor bug * fixing autodocs * addressing comments from code review * Updating constants to a frozen dataclass * Updating direct events packet reading to correctly read packet data, updating test data, adding tests * Updating docstrings --------- Co-authored-by: Tenzin Choedon <tenzin.choedon@lasp.colorado.edu> Co-authored-by: Tenzin Choedon <36522642+tech3371@users.noreply.github.com> Co-authored-by: GLOWS team <maro@cbk.waw.pl> Co-authored-by: Matthew Bourque <matthew.bourque@lasp.colorado.edu> Co-authored-by: Greg Lucas <greg.m.lucas@gmail.com> Co-authored-by: Gabriel M <104743000+GFMoraga@users.noreply.github.com> Co-authored-by: Bryan Harter <41062454+bryan-harter@users.noreply.github.com> Co-authored-by: Sean Hoyt <sean.hoyt@lasp.colorado.edu> Co-authored-by: Laura Sandoval <46567335+laspsandoval@users.noreply.github.com>
Change Summary
This is the first pass at l0 decom for codice- specifically NHK, which is the raw housekeeping data. This data has been validated. Testing is done in this PR and it validates against a csv we recieved from the instrument team at swRi.
Overview
Updated the decom code in
test_decom.py
Added testing code that validates SHCOARSE