Test CCSDS header across all instrument packet definitions

[bourque]

Change Summary

Overview

This PR adds a new folder for storing tests that can be run on all instruments, and a new test that tests the existence and proper formatting of the CCSDS header in all packet definition XML files.

New Dependencies

None

New Files

• imap_processing/tests/test_decom.py
  • Tests for the decommutation process, applied to all instruments

Deleted Files

None

Updated Files

None

Testing

This test is currently failing on the idex_packet_defintiion.xml file (which currently doesn't contain a properly formatted header) and a handful of others (due to inconsistency with considering SHCOARSE as part of the header or not).

[bourque]

Some open questions for discussion:

1. Is SHCOARSE to be considered part of the header or not? Should we be consistent on how this is applied across all packet definition files?
2. Is this a sound method for testing the header (i.e. reading in the file and comparing strings)? Or should we try to actually decom each file and check if the header is in the results?
3. Are there any other generalized tests we can add to this file?

[greglucas]

I think this looks like a nice approach and is also a good way to make sure we are consistent as a team too!

Those are all really good questions that I don't know the answer to :)

[GFMoraga]

This is great! Need to change the typo on a broader scale but that can be done in another PR

[maxinelasp]

Looks good as a first pass! Definitely a few outstanding questions but I don't think any that would prevent a merge.

[bourque]

Ok, I believe this is good to go and the tests should pass once SWE, Ultra, and GLOWS update their XTCE files with the fixes implemented in #185

[maxinelasp]

Updated GLOWS XTCE to pass these tests in #244

[bourque]

pre-commit.ci autofix

[bourque]

I think we still need to update the packet definitions for SWE and Ultra with the changes from #185 to get these tests to pass.

@tech3371 @laspsandoval Could you update those files for SWE and Ultra (respectively)? Or if it is straightforward, I am happy to do it as part of this PR if you could share the Excel spreadsheet.

[greglucas]

Just an FYI, but you can always xfail Ultra for now and update it later the same way you're doing with idex I think? Or do you want to update idex as well and remove all xfails altogether? Just giving you an out to get your tests passing so you can merge this in and then let others update and remove xfails later so the onus isn't on you to do that work.

[bourque]

@laspsandoval Do you know what abstract="true" does here? I had to remove it in order to get these tests to pass, and the other instrument's packet definitions don't have it. I am thinking that it is not important?

[bourque]

I now see that removing this caused test_decom_apid_88*.py to fail, so I guess it is important afterall!

[bourque]

I ended up just making my header tests a bit more generalized to allow for this abstract parameter.

[bourque]

Just an FYI, but you can always xfail Ultra for now and update it later the same way you're doing with idex I think? Or do you want to update idex as well and remove all xfails altogether? Just giving you an out to get your tests passing so you can merge this in and then let others update and remove xfails later so the onus isn't on you to do that work.

It was an easy fix to get the Ultra packet definitions to pass, so I just did that. I think I will leave IDEX as xfail for now -- that one will require more work.