Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

609 imap hi l1c pointing set cdf #627

Conversation

subagonsouth
Copy link
Contributor

Change Summary

Adds Hi l1c processing for pset data

Overview

Adds generation of empty l1c pset product to Hi processing.

New Dependencies

None

New Files

  • imap_processing/hi/l1c/init.py
    • added l1c directory
  • imap_processing/hi/l1c/hi_l1c.py
    • added processing for l1b_de -> l1c_pset
  • imap_processing/tests/hi/test_hi_l1c.py
    • added test coverage for all functions in imap_processing/hi/l1c/hi_l1c.py

Deleted Files

None

Updated Files

  • imap_processing/cdf/config/imap_hi_global_cdf_attrs.yaml
    • added global attributes for Hi l1c pset products
  • imap_processing/cdf/config/imap_hi_variable_attrs.yaml
    • Updated default fill values to match SPDF recommended values
    • Added default epoch definition for reuse
    • Added pset specific variable attribute definitions
  • imap_processing/cli.py
    • Added logic for Hi l1c processing
  • imap_processing/tests/test_cli.py
    • Parameterized tests for Hi l1a/l1b/l1c coverage into a single test
    • Added autospec=True to mock that mocks hi l1a/l1b/l1c function calls to ensure coverage of correct function signature in cli call

Testing

  • Full test coverage of new hi_l1c.py functions
  • Improved Hi tests in test_cli.py

parameterize hi cli tests so that all level1 is tested in a single function
add CdfAttributeManager to hi l1c processing
Update cli test for Hi to include coverage for mocked function signature
@subagonsouth subagonsouth requested review from a team, bourque, sdhoyt, greglucas, tech3371, maxinelasp, daralynnrhode and anamanica and removed request for a team June 14, 2024 17:00
@subagonsouth subagonsouth added Ins: Hi Related to the IMAP-Hi instrument CDF Related to CDF files labels Jun 14, 2024
Comment on lines +74 to +75
# SPDF says this should be the center of the time bin, but instrument
# teams may disagree.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For some reason I thought you could specify the left/center/right accumulation bin, but today I learned!

For ISTP the time value of a record refers to the center of the accumulation period for the record if the measurement is not an instantaneous one.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for grabbing that text. I should have put that into the TODO originally. I will add it in for clarity.

)

# Generate label variables
data_vars["esa_step_label"] = xr.DataArray(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, so these labels do work as data variables? We should make a ticket to update the other instruments later.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SKTEditor seemed fine with them being variables rather than coordinates. I wrote up ticket #628 to capture this.

return dataset


def full_dataarray(name, attrs, coords: dict, shape=None):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a really nice helper! For later: I think we should look at moving this out more generally and using it across instruments if possible.

@@ -107,56 +107,37 @@ def test_codice(mock_codice_l1a, mock_instrument_dependencies):
assert mocks["mock_upload"].call_count == 1


@mock.patch("imap_processing.cli.hi_l1a.hi_l1a")
def test_hi_l1a(mock_hi_l1a, mock_instrument_dependencies):
@pytest.mark.parametrize("data_level, n_prods", [("l1a", 2), ("l1b", 1), ("l1c", 1)])
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice use of parametrization here! I've been wondering if we could parameterize over some things in this file with all the duplication.

@subagonsouth subagonsouth merged commit 33a950a into IMAP-Science-Operations-Center:dev Jun 14, 2024
17 checks passed
@subagonsouth subagonsouth deleted the 609-imap-hi-l1c-pointing-set-cdf branch June 14, 2024 20:04
Copy link
Contributor

@tech3371 tech3371 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some late comments for future

DISPLAY_TYPE: no_plot
FORMAT: F5.2
LABLAXIS: Latitude
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Longitude here?

UNITS: deg
VALIDMIN: 0
VALIDMAX: 180
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we are starting at 0, would max be 179? I don't know for sure though.

UNITS: deg
VALIDMIN: 0
VALIDMAX: 360
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here if above needs update

LABL_PTR_1: esa_step_label
LABL_PTR_2: spin_bin_label
DISPLAY_TYPE: stack_plot
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool. Didn't know about this

Comment on lines +97 to +99
cdf_manager = CdfAttributeManager(imap_module_directory / "cdf" / "config")
cdf_manager.load_global_attributes("imap_hi_global_cdf_attrs.yaml")
cdf_manager.load_variable_attributes("imap_hi_variable_attrs.yaml")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is ok for now. After SIT-3, we will need to update this to this. I believe this does minor validation and sounds like we may expand this afterwards.

    # create the attribute manager for this data level
    attr_mgr = ImapCdfAttributes()
    attr_mgr.add_instrument_global_attrs(instrument="lo")
    attr_mgr.add_instrument_variable_attrs(instrument="lo", level="l1a")
    attr_mgr.add_global_attribute("Data_version", data_version)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CDF Related to CDF files Ins: Hi Related to the IMAP-Hi instrument
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

3 participants