initial hit l1a decom #164
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sdhoyt
requested review from
bourque,
greglucas,
tech3371,
bryan-harter,
laspsandoval,
GFMoraga and
maxinelasp
and removed request for
a team
sdhoyt
commented
Sep 21, 2023
greglucas
reviewed
Sep 22, 2023
|
Oh, one more thing, could you add this to the documentation as well by doing an autodoc similar to SWE under the instruments and adding it to the Instruments TOC tree. |
greglucas
approved these changes
Sep 22, 2023
| # if data for the APID exists, create the dataset | ||
| if data_dict != {}: | ||
| epoch = xr.DataArray( | ||
| name="Epoch", data=data_dict["SHCOARSE"], dims=("Epoch") |
There was a problem hiding this comment.
We should get @bryan-harter 's comment on how to name this.
I think this should be MET here since it is the mission elapsed time and then we won't confuse it with the reserved epoch name. Then later, I think we need to convert to an EPOCH which I believe needs to be all capitalized, but I also think it has a specific meaning and needs to be offset from a specific date for CDFs.
I don't know the details here though, so hopefully Bryan can give us some guidance.
There was a problem hiding this comment.
Merging with this comment unresolved. This will be fixed later by issue #269 .
This was
linked to
issues
Sep 22, 2023
tech3371
approved these changes
Sep 26, 2023
bourque
approved these changes
Sep 27, 2023
docs/source/reference/index.rst
Outdated
Comment on lines
18
to
19
| swe | ||
| hit |
There was a problem hiding this comment.
Real nit-picky here but maybe we should set the precedent here for keeping this list in alphabetical order?
There was a problem hiding this comment.
That's a good point. I'll update that
laspsandoval
pushed a commit
to laspsandoval/imap_processing
that referenced
this pull request
Nov 15, 2023
* initial hit l1a decom * using defaultdict, add docs * fixed hit l1a docstrings * fixed hit l1a docstring formatting * doc updates and minor fixes * minor docstring update * minor comment update * rebased with dev * fixed ruff issues * regenerated xtce * fixed unit test * github doc gen fix attempt * added class docstring attributes * fixed more doc issues * another doc fix attempt * doc fix attempt * doc fix attempt * changed doc python v to 3.10
maxinelasp
pushed a commit
to maxinelasp/imap_processing
that referenced
this pull request
Nov 16, 2023
* initial hit l1a decom * using defaultdict, add docs * fixed hit l1a docstrings * fixed hit l1a docstring formatting * doc updates and minor fixes * minor docstring update * minor comment update * rebased with dev * fixed ruff issues * regenerated xtce * fixed unit test * github doc gen fix attempt * added class docstring attributes * fixed more doc issues * another doc fix attempt * doc fix attempt * doc fix attempt * changed doc python v to 3.10
maxinelasp
added a commit
that referenced
this pull request
Jan 8, 2024
* update telemetry_generator to be more generic * Updates to telemetry generator * updating docs * updating to remove local-specific testing changes * first pass at glows xtce * update way of checking if description exists or not. update generator template. * updated generator to use correct IntegerParameterType * First pass at GLOWS decom * updating telemetry generator * Updating glows decom to properly process data into dictionary * Finished histogram decom * Updating decom to include direct events * Finalizing code, adding smaller test data file and combined XML, updating documentation and tests * Updating test file path * Fixing test runs * updating docs * Updating data classes to use attributes, and changing names * Removing keys from data class and getting attribute names from the packet * Move INT to SINT in telemetry_generator Co-authored-by: Tenzin Choedon <36522642+tech3371@users.noreply.github.com> * Addressing code review comments * Fixing test failure * Review updates * fixing typo * Updating IntEnum to Enum to fix automatic documentation * Move L0 data classes to a dataclass type with specific attributes * Initial GLOWS L1A data classes * Adding required attributes to L0 data class * Adding L1A histogram data class * fixing test failures * Finishing ccsds dataclass * Minor updates * Finishing GLOWS L1A histograms * Updating to include direct events data * Updating glows l1a * Adding useful GLOWS code from GLOWS team and Marek Strumik * Adding direct event information Co-authored-by: GLOWS team <maro@cbk.waw.pl> * Updates for direct event creation * Updates to L1A direct events * Updated release workflow to reflect removal of "main" branch and use of version branches * Reduced the filesize of git-workflow.png * Glows L0 changes to data class to accomodate L1A needs (#264) * Glows L0 changes to data class to accomodate L1A needs * DOC: Add intersphinx to documentation for external references * DOC: Add a clean option to the documentation Makefile build command * DOC: Update glows types in dataclasses * DOC: Change glows currentmodule references * DOC: Add ccsds header to glows documentation * DOC: Add inherited member description for glows This adds the inherited attributes from the dataclass directly into the subclasses. There may be a better way to do this, so perhaps some setting/configuration parameter needs to be updated? But this at least fixes the warnings for now. * FIX: Run black on glows test * PR updates --------- Co-authored-by: Greg Lucas <greg.m.lucas@gmail.com> * Housekeeping data testing and validation for CoDICE l0 (#176) Validated housekeeping data with tests plus minor updates --------- Co-authored-by: Matthew Bourque <matthew.bourque@lasp.colorado.edu> * L1 cdf creation (#179) * Extracting and adding new data to the IDEX L1 * Adding some of the values to the XML, rather than shifting bits in the code (work in progress, need to do a few more variables) * Finishing up having space packet parser decom * Getting rid of unecessary bit masksi8kt9gloh * making changes based on comments * Formatting the file after the upstream sync * Adding L1 CDF creation * Forgot to commit poetry changes * Making changes based on Greg's comments, as well as issues 1-8 from SPDF * Fixing import statements * Changing version into a string * Adding more descriptions, units, and labels in the attributes * Fixing the way strings are inserted into the CDF * Fixing issues with white spice in the text of attrs * Files are finally ISTP compliant! * Fixing a couple things for ISTP * Had the contents of the 32 bit things reversed * Fixing the packet definition for the CCSDS file * More fixes to ensure ISTP compliance * Updating the file name * touching finishes on the L1 CDF * Adding fixture for temp directory in tests * Fixing one of the ruff errors by adding a namedtuple * Adding new "write_cdf" function that writed cdfs based on attributes * Getting rid of a few more magic numbers * Updating the packet definition * initial hit l1a decom (#164) * initial hit l1a decom * using defaultdict, add docs * fixed hit l1a docstrings * fixed hit l1a docstring formatting * doc updates and minor fixes * minor docstring update * minor comment update * rebased with dev * fixed ruff issues * regenerated xtce * fixed unit test * github doc gen fix attempt * added class docstring attributes * fixed more doc issues * another doc fix attempt * doc fix attempt * doc fix attempt * changed doc python v to 3.10 * I-ALiRT: L0 parsing (#246) * adding ialirt * CDF documentation update (#273) * CDF documentation update * Adding the cdf_guide.rst to the toc * Formatting update! * Updating the docs for sections I missed * Formatting a little nicer * Adding links enabled in the docs * Fixing duplicate headers * Getting rid of duplicate headers * Fixing more documentation * Responding to a few code review comments * Breaking up the page into sub-pages * Adding a link to xarray_to_cdf page * Adding more links to different sections * fixing image link * Editting some of the wording now that we're on different pages * Typos and format fixed * Another format error * Create CDF class based system (#272) * Updating IDEX CDF attrs * remove comments, create constants class * Updating names and addressing PR comments * Updating docs * Adding tests * Updating types of L0 to properly read bytearrays * Updating tests * GLOWS direct events compressed and uncompressed * updating decom dataclasses * Updating GLOWS L0 * L1A tests * removing unneeded code * updating tests and fixing minor bug * fixing autodocs * addressing comments from code review * Updating constants to a frozen dataclass * Updating direct events packet reading to correctly read packet data, updating test data, adding tests * Updating docstrings --------- Co-authored-by: Tenzin Choedon <tenzin.choedon@lasp.colorado.edu> Co-authored-by: Tenzin Choedon <36522642+tech3371@users.noreply.github.com> Co-authored-by: GLOWS team <maro@cbk.waw.pl> Co-authored-by: Matthew Bourque <matthew.bourque@lasp.colorado.edu> Co-authored-by: Greg Lucas <greg.m.lucas@gmail.com> Co-authored-by: Gabriel M <104743000+GFMoraga@users.noreply.github.com> Co-authored-by: Bryan Harter <41062454+bryan-harter@users.noreply.github.com> Co-authored-by: Sean Hoyt <sean.hoyt@lasp.colorado.edu> Co-authored-by: Laura Sandoval <46567335+laspsandoval@users.noreply.github.com>
laspsandoval
added a commit
to laspsandoval/imap_processing
that referenced
this pull request
Apr 2, 2024
* update telemetry_generator to be more generic * Updates to telemetry generator * updating docs * updating to remove local-specific testing changes * first pass at glows xtce * update way of checking if description exists or not. update generator template. * updated generator to use correct IntegerParameterType * First pass at GLOWS decom * updating telemetry generator * Updating glows decom to properly process data into dictionary * Finished histogram decom * Updating decom to include direct events * Finalizing code, adding smaller test data file and combined XML, updating documentation and tests * Updating test file path * Fixing test runs * updating docs * Updating data classes to use attributes, and changing names * Removing keys from data class and getting attribute names from the packet * Move INT to SINT in telemetry_generator Co-authored-by: Tenzin Choedon <36522642+tech3371@users.noreply.github.com> * Addressing code review comments * Fixing test failure * Review updates * fixing typo * Updating IntEnum to Enum to fix automatic documentation * Move L0 data classes to a dataclass type with specific attributes * Initial GLOWS L1A data classes * Adding required attributes to L0 data class * Adding L1A histogram data class * fixing test failures * Finishing ccsds dataclass * Minor updates * Finishing GLOWS L1A histograms * Updating to include direct events data * Updating glows l1a * Adding useful GLOWS code from GLOWS team and Marek Strumik * Adding direct event information Co-authored-by: GLOWS team <maro@cbk.waw.pl> * Updates for direct event creation * Updates to L1A direct events * Updated release workflow to reflect removal of "main" branch and use of version branches * Reduced the filesize of git-workflow.png * Glows L0 changes to data class to accomodate L1A needs (IMAP-Science-Operations-Center#264) * Glows L0 changes to data class to accomodate L1A needs * DOC: Add intersphinx to documentation for external references * DOC: Add a clean option to the documentation Makefile build command * DOC: Update glows types in dataclasses * DOC: Change glows currentmodule references * DOC: Add ccsds header to glows documentation * DOC: Add inherited member description for glows This adds the inherited attributes from the dataclass directly into the subclasses. There may be a better way to do this, so perhaps some setting/configuration parameter needs to be updated? But this at least fixes the warnings for now. * FIX: Run black on glows test * PR updates --------- Co-authored-by: Greg Lucas <greg.m.lucas@gmail.com> * Housekeeping data testing and validation for CoDICE l0 (IMAP-Science-Operations-Center#176) Validated housekeeping data with tests plus minor updates --------- Co-authored-by: Matthew Bourque <matthew.bourque@lasp.colorado.edu> * L1 cdf creation (IMAP-Science-Operations-Center#179) * Extracting and adding new data to the IDEX L1 * Adding some of the values to the XML, rather than shifting bits in the code (work in progress, need to do a few more variables) * Finishing up having space packet parser decom * Getting rid of unecessary bit masksi8kt9gloh * making changes based on comments * Formatting the file after the upstream sync * Adding L1 CDF creation * Forgot to commit poetry changes * Making changes based on Greg's comments, as well as issues 1-8 from SPDF * Fixing import statements * Changing version into a string * Adding more descriptions, units, and labels in the attributes * Fixing the way strings are inserted into the CDF * Fixing issues with white spice in the text of attrs * Files are finally ISTP compliant! * Fixing a couple things for ISTP * Had the contents of the 32 bit things reversed * Fixing the packet definition for the CCSDS file * More fixes to ensure ISTP compliance * Updating the file name * touching finishes on the L1 CDF * Adding fixture for temp directory in tests * Fixing one of the ruff errors by adding a namedtuple * Adding new "write_cdf" function that writed cdfs based on attributes * Getting rid of a few more magic numbers * Updating the packet definition * initial hit l1a decom (IMAP-Science-Operations-Center#164) * initial hit l1a decom * using defaultdict, add docs * fixed hit l1a docstrings * fixed hit l1a docstring formatting * doc updates and minor fixes * minor docstring update * minor comment update * rebased with dev * fixed ruff issues * regenerated xtce * fixed unit test * github doc gen fix attempt * added class docstring attributes * fixed more doc issues * another doc fix attempt * doc fix attempt * doc fix attempt * changed doc python v to 3.10 * I-ALiRT: L0 parsing (IMAP-Science-Operations-Center#246) * adding ialirt * CDF documentation update (IMAP-Science-Operations-Center#273) * CDF documentation update * Adding the cdf_guide.rst to the toc * Formatting update! * Updating the docs for sections I missed * Formatting a little nicer * Adding links enabled in the docs * Fixing duplicate headers * Getting rid of duplicate headers * Fixing more documentation * Responding to a few code review comments * Breaking up the page into sub-pages * Adding a link to xarray_to_cdf page * Adding more links to different sections * fixing image link * Editting some of the wording now that we're on different pages * Typos and format fixed * Another format error * Create CDF class based system (IMAP-Science-Operations-Center#272) * Updating IDEX CDF attrs * remove comments, create constants class * Updating names and addressing PR comments * Updating docs * Adding tests * Updating types of L0 to properly read bytearrays * Updating tests * GLOWS direct events compressed and uncompressed * updating decom dataclasses * Updating GLOWS L0 * L1A tests * removing unneeded code * updating tests and fixing minor bug * fixing autodocs * addressing comments from code review * Updating constants to a frozen dataclass * Updating direct events packet reading to correctly read packet data, updating test data, adding tests * Updating docstrings --------- Co-authored-by: Tenzin Choedon <tenzin.choedon@lasp.colorado.edu> Co-authored-by: Tenzin Choedon <36522642+tech3371@users.noreply.github.com> Co-authored-by: GLOWS team <maro@cbk.waw.pl> Co-authored-by: Matthew Bourque <matthew.bourque@lasp.colorado.edu> Co-authored-by: Greg Lucas <greg.m.lucas@gmail.com> Co-authored-by: Gabriel M <104743000+GFMoraga@users.noreply.github.com> Co-authored-by: Bryan Harter <41062454+bryan-harter@users.noreply.github.com> Co-authored-by: Sean Hoyt <sean.hoyt@lasp.colorado.edu> Co-authored-by: Laura Sandoval <46567335+laspsandoval@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Change Summary
Overview
This is an initial pass at the HIT L1A decommutation. Not everything is here and testing was very limited because the data I have are packets from the HIT SIM that only contain SHCOARSE. This initial code does the following:
I'm still not sure the best way to organize all of this, but I wanted to get something out there for people to start giving feedback on. One thing I particularly struggled with is variable naming because we get a CCSDS packet, but that packet contains multiple APIDs that we also call packets(?). Please let me know if anyone has any better naming ideas.
This updates also adds all of the HIT XTCE files.
New Dependencies
None
New Files
HIT XTCE Files
hit_apid.py
hit_l1a_decom.py
Deleted Files
None
Updated Files
None
Testing