HIT L0 - data decompression #1169
Conversation
…all in parse_count_rates to decompress data. Consolidated two functions related to finding valid indices to reduce code tracing
vmartinez-cu
requested review from
sdhoyt,
a team,
laspsandoval and
maxinelasp
and removed request for
a team
vmartinez-cu
added
Ins: HIT
| power = packed >> MANTISSA_BITS | ||
|
|
||
| # Decompress the data depending on the value of the exponent | ||
| if power > 1: |
There was a problem hiding this comment.
Add comment here to explain what you are doing. You might consider breaking it down into different steps for clarity.
There was a problem hiding this comment.
I just took this code from the algorithm doc where it was written in C and translated it to python. I'm not super familiar with bit shifting, but it seems to be working as expected in my tests. I broke down this section into smaller components for readability. Let me know if this breakdown and the additional comments help.
|
|
||
| # Packed is the compressed integer | ||
| # Right bit shift to get the exponent | ||
| power = packed >> MANTISSA_BITS |
There was a problem hiding this comment.
Based on this, it looks power will only be greater than 1 if packed is >=8192. From the algorithm if the power is less than 1 then the decompressed number = the compressed number. So that makes sense with what you're seeing in the sample data, despite what the algorithm document seems to say
| @@ -177,6 +179,10 @@ def parse_count_rates(sci_dataset: xr.Dataset) -> None: | |||
| low_gain = data[1::2] # Items at odd indices 1, 3, 5, etc. | |||
| parsed_data[i] = [high_gain, low_gain] | |||
|
|
|||
| # Decompress data where needed | |||
| if all(x not in field for x in ["hdr", "spare", "pha"]): | |||
| parsed_data = np.vectorize(decompress_rates_16_to_32)(parsed_data) | |||
Add more comments that explain how the decompression algorithm works.
…or loop for the test values and expected values
…nteger. It was added to fix a mypy issue, but now the type is initialized earlier in the function
vmartinez-cu
merged commit c5bfa68
into
IMAP-Science-Operations-Center:dev
This PR adds a function to decom_hit.py to decompress rates data from 16-bit integers to 32-bit integers
Updated Files
-test_decom_hit.py
Closes issue #1055