1026 logical source parser

[subagonsouth]

Change Summary

Overview

• Added function to parse logical source into components
• Added function to parse sensor number from string

Updated Files

• imap_processing/hi/l1b/hi_l1b.py
  • Use new parser function
• imap_processing/hi/l1c/hi_l1c.py
  • Use new parser function
• imap_processing/hi/utils.py
  • Add utility function for parsing filename like strings such as logical source
  • Add function for parsing integer sensor number from string
• imap_processing/tests/hi/test_utils.py
  • Add test coverage for two new functions

Closes: #1026

[tech3371]

Do you need other fields or can you do something similar to this and then apply regex on descriptor? I am afraid of losing track of this in the future. I am wondering now if it's an feature we should implement in imap-data-access, especially update this function.

Suggested change

	regex_str = (
	r"^(?P<mission>imap)_" # Required mission
	r"(?P<instrument>[^_]+)_" # Required instrument
	r"(?P<data_level>[^_]+)_" # Required data level
	r"((?P<sensor>\d{2}sensor)?-)?" # Optional sensor number
	r"(?P<descriptor>[^_]+)" # Required descriptor
	r"(_(?P<start_date>\d{8}))?" # Optional start date
	r"(-repoint(?P<repointing>\d{5}))?" # Optional repointing field
	r"(?:_v(?P<version>\d{3}))?" # Optional version
	r"(?:\.(?P<extension>cdf|pkts))?$" # Optional extension
	)
	regex_str = (
	r"((?P<sensor>\d{2}sensor)?-)?" # Optional sensor number
	r"(?P<descriptor>[^_]+)" # Required descriptor
	)

[subagonsouth]

Yeah, perhaps it is overkill to parse all of the fields. I doubt I will need the mission or instrument field but I could see needing the start_date or repoint. I based this regex on the one in the ScienceFilePath class. I don't think that one should be modified because it needs to be very strict to have invalid file names fail where I don't want to be strict. Parsing using split() could be done but it is lots of hard coding indices and gets messy with some of the optional fields.

[tech3371]

If you like to keep it as it is, that's ok with me. I do want @laspsandoval and @sdhoyt to review these two function especially and see if they can benefit from it. If so, then we may want to move this in common utils function.

[sdhoyt]

I haven't come across a situation where I need to parse anything from the logical source. I agree with Tim though about the start_date and pointing. I could see parsing those two fields being useful for instruments that need to reprocess because there's data spread across CCSDS files for two pointings and they need to put some bits of data from Day 2 into the Day 1 data product. In that case I think there would be multiple L0 dependency files with different dates.

[subagonsouth]

@tech3371, I decided to move this function into the cdf/utils.py module.

[tech3371]

Tim, that's a good place to put it since it's related to CDF attrs. Cool. I will approve this PR then. Then if others find that function helpful, it's there to use.

[subagonsouth]

I should add a pointing number to this test case if the regex parsing stays.

[tech3371]

That would be helpful.

[tech3371]

Besides the previous comment about may be moving it to common utils, everything else looks good to me. I will let Sean and Laura approve this PR.

[tech3371]

That would be helpful.

[tech3371]

Can you add test with repointing number in the filename like you were thinking? Otherwise, it looks good to me.