Skip to content

Latest commit

 

History

History
554 lines (485 loc) · 13.9 KB

README.md

File metadata and controls

554 lines (485 loc) · 13.9 KB
Raw

PlanetaryPy

PlanetaryPy

Beta release: This will become (part of) the core package of the PlanetaryPy organisation.

Potential logo:image.png

Install

pip install planetarypy

This will pull in these other dependencies and their dependencies:

tomlkit pandas pvl numpy python-dateutil tqdm lxml yarl hirise-tools kalasiris

Suggested standard abbreviations:

  • Inside these docs the package will be called PLPY for brevity.
  • A standard Python import could be: plp or plpy
    • because the last p in plp can be pronounced out, we consider these equivalent for human conversation and pronounce these “plippy”.

General scope

First and foremost this package should provide support in working with planetary science data.

With working we mean:

  • locating
  • retrieving
  • reading
  • further processing

of data.

Locating

This library manages, via its PDS tools, multiple PDS3 index files per instrument that can be used for identifying data of interest. These index files are automatically downloaded and converted to the very performant (and cloud-ready) parquet file format. > Parquet is able to store advanced datatypes like nan-capable integer and full datetime objects, as opposed to HDF5.

Retrieving

The interface to getting data is via a path-retrieving function based on a PDS product-id. If that product-id is available locally, the path will be returned. If it is not, it will previously be downloaded, stored in a systematic fashion organized by mission and instrument, and then the local path will be returned.

Reading

For now, the library only returns the path to the object and the user needs to sort out the reading process. A recently funded NASA project Planetary Data Reader will be integrated here, so that basic reading into memory can be provided.

As such, we anticipate two classes of reading support: 1. basic reading into numpy and/or xarray 1. added reader functionality like basic plots and basic geospatial processing, as supported by interested parties

There will exist larger other packages that focus on working with a given instrument’s data, in which case that package could become an affiliated package with the planetarypy GitHub organization, if so desired.

Further processing

In the future, additional frequently used procedures will be added to this library, e.g. * frequently used GDAL/rasterio procedures * frequently used SPICE operations * like surface illumination on a given body

PDS tools

Look at the Apps docs to see what pds.apps exist for easily getting PDS indexes. The find_index app is specifically useful when you don’t know what index files exist.

So far, the following indexes are supported (but not necessarily all tested within PLPY):

  • Cassini
    • ISS (all)
    • UVIS (all)
  • MRO
    • CTX EDR
    • HiRISE
      • EDR, RDR, DTM
        • EDR index has a bug (as delivered by the team, reported), where I need to activate an existing fix for it.
  • LRO
    • Diviner (DLRE)
      • EDR, RDR
    • LOLA
      • EDR, RDR

More indexes

More indexes of other instruments can be easily added by following the existing structure of what has been copied into your config at ~/.planetarypy_config.toml.

Please consider submitting a pull request for adding further PDS index files into the config file at its source: https://github.com/michaelaye/nbplanetary/blob/master/planetarypy/data/planetarypy_config.toml

Utils

Find something in Utils for working with NASA timestamps and a well working URL download function url_retrieve, among other stuff.

Experiment/Instrument Specific

So far, planetarypy supports CTX EDR and HiRISE RGB.NOMAP data. Look at the CTX and HiRISE pages for descriptions of classes for working with these data.

Bug reports

Please submit bug reports at https://github.com/michaelaye/nbplanetary/issues

How to use

Indexes

See PDS apps for more details.

from planetarypy.pds.apps import get_index
ctrindex = get_index("mro.ctx", "edr", refresh=False)
ctrindex.sample(5, random_state=42)  # setting random_state to always get same files for docs
<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; } 
.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}
</style>
VOLUME_ID FILE_SPECIFICATION_NAME ORIGINAL_PRODUCT_ID PRODUCT_ID IMAGE_TIME INSTRUMENT_ID INSTRUMENT_MODE_ID LINE_SAMPLES LINES SPATIAL_SUMMING ... SUB_SOLAR_LATITUDE SUB_SPACECRAFT_LONGITUDE SUB_SPACECRAFT_LATITUDE SOLAR_DISTANCE SOLAR_LONGITUDE LOCAL_TIME IMAGE_SKEW_ANGLE RATIONALE_DESC DATA_QUALITY_DESC ORBIT_NUMBER
86101 MROX_2757 DATA/J07_047671_1256_XN_54S338W.IMG 4A_04_10F0036A00 J07_047671_1256_XN_54S338W 2016-09-27 00:24:54.682 CTX NIFL 2528 8192 1 ... -19.41 338.09 -54.43 207762612.4 230.58 15.48 90.1 Crater in Noachis Terra OK 47671
54420 MROX_1819 DATA/D10_031170_1808_XN_00N221W.IMG 4A_04_109C017D00 D10_031170_1808_XN_00N221W 2013-03-21 06:58:09.892 CTX NIFL 5056 7168 1 ... -24.39 221.52 0.88 209925149.5 286.15 14.49 90.1 Ride-along with HiRISE OK 31170
71053 MROX_2331 DATA/F10_039530_1470_XI_33S203W.IMG 4A_04_10C9000F00 F10_039530_1470_XI_33S203W 2015-01-01 16:20:01.840 CTX ITL 5056 52224 1 ... -25.3 203.96 -33.12 207070298.2 263.93 15.2 90.4 Terrain in Terra Cimmeria OK 39530
51385 MROX_1742 DATA/D04_028919_2026_XN_22N327W.IMG 4A_04_109002A800 D04_028919_2026_XN_22N327W 2012-09-26 21:17:00.483 CTX NIFL 5056 6144 1 ... 0.67 327.06 22.78 219814412.3 178.44 15.53 90.1 Ride-along with HiRISE OK 28919
51923 MROX_1757 DATA/D05_029283_1524_XN_27S180W.IMG 4A_04_1092020B00 D05_029283_1524_XN_27S180W 2012-10-25 05:44:21.786 CTX NIFL 5056 9216 1 ... -6.28 178.36 -27.56 214830233.7 194.72 15.57 90.1 Ride-along with HiRISE OK 29283

5 rows × 51 columns

hirise_rdr = get_index("mro.hirise", "rdr")
hirise_rdr.sample(5, random_state=42)
<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; } 
.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}
</style>
VOLUME_ID FILE_NAME_SPECIFICATION INSTRUMENT_HOST_ID INSTRUMENT_ID OBSERVATION_ID PRODUCT_ID PRODUCT_VERSION_ID TARGET_NAME ORBIT_NUMBER MISSION_PHASE_NAME ... LINE_PROJECTION_OFFSET SAMPLE_PROJECTION_OFFSET CORNER1_LATITUDE CORNER1_LONGITUDE CORNER2_LATITUDE CORNER2_LONGITUDE CORNER3_LATITUDE CORNER3_LONGITUDE CORNER4_LATITUDE CORNER4_LONGITUDE
144822 MROHR_0001 RDR/ESP/ORB_074100_074199/ESP_074107_1410/ESP_... MRO HIRISE ESP_074107_1410 ESP_074107_1410_COLOR 1 MARS 74107 Extended Science Phase ... -4568562.5 2009279.5 -38.7446 159.311 -38.7465 159.287 -38.614 159.269 -38.6121 159.293
22518 MROHR_0001 RDR/ESP/ORB_014000_014099/ESP_014080_1040/ESP_... MRO HIRISE ESP_014080_1040 ESP_014080_1040_RED 1 MARS 14080 Extended Science Phase ... -3193960.0 -845180.0 -76.0185 165.215 -76.0315 164.858 -75.6958 164.658 -75.6831 165.005
86634 MROHR_0001 RDR/ESP/ORB_046300_046399/ESP_046395_1730/ESP_... MRO HIRISE ESP_046395_1730 ESP_046395_1730_COLOR 1 MARS 46395 Extended Science Phase ... -1584330.0 -26573400.0 -6.9957 292.569 -6.9979 292.551 -6.6838 292.51 -6.6815 292.529
138163 MROHR_0001 RDR/ESP/ORB_069100_069199/ESP_069178_2345/ESP_... MRO HIRISE ESP_069178_2345 ESP_069178_2345_RED 1 MARS 69178 Extended Science Phase ... 12762196.0 15708938.0 53.8598 76.7591 53.8474 76.602 54.0013 76.5672 54.0137 76.7249
125490 MROHR_0001 RDR/ESP/ORB_063500_063599/ESP_063504_1995/ESP_... MRO HIRISE ESP_063504_1995 ESP_063504_1995_COLOR 1 MARS 63504 Extended Science Phase ... 4638178.5 12794202.0 19.4523 124.149 19.45 124.128 19.5686 124.113 19.5709 124.133

5 rows × 54 columns

Instrument tools

CTX

from planetarypy.ctx import CTX
pid = ctrindex.sample(random_state=42).squeeze().PRODUCT_ID
pid
'J07_047671_1256_XN_54S338W'

ctx = CTX(pid)
ctx
PRODUCT_ID: J07_047671_1256_XN_54S338W
URL: https://pds-imaging.jpl.nasa.gov/data/mro/mars_reconnaissance_orbiter/ctx/mrox_2757/data/J07_047671_1256_XN_54S338W.IMG
source_path: /remote/trove/geo/planet/Mars/CTX/pds/mrox_2757/J07_047671_1256_XN_54S338W.IMG
Shape: (8192, 2528)

ctx.source_path
Path('/remote/trove/geo/planet/Mars/CTX/pds/mrox_2757/J07_047671_1256_XN_54S338W.IMG')

ctx.download()
File exists. Use `overwrite=True` to download fresh.

Acknowledgements

The creation of this library was in part supported by:

  • NASA Cassini UVIS mission
  • NASA SSW grant NNX15AH36G
  • NASA PDART grant 80NSSC20K0875
  • NASA SSW grant 80NSSC20K0748
  • German Space Agency (DLR Bonn), grant 50 OO 2204, on behalf of the German Federal Ministry for Economic Affairs and Climate Action.
  • This research has made use of the USGS Integrated Software for Imagers and Spectrometers (ISIS)(Laura et al. 2022).

Laura, Jason, Alex Acosta, Travis Addair, Lauren Adoram-Kershner, James Alexander, Oleg Alexandrov, Stacey Alley, et al. 2022. Integrated Software for Imagers and Spectrometers (version 7.2.0_RC1). Zenodo. https://doi.org/10.5281/zenodo.7443567.