identification of coordinate systems #115
Comments
|
Jim Lewis mentioned this problem in his talk at IDHEA in 2020. |
|
SPEDAS uses a list of allowed coordinate system / frame names. |
|
possibly use some of NAIF conventions; also CDF has lots of examples; see also review paper on coordinate frames; |
|
Related to the semantics of identifying a variable as an actual spatial vector quantity. If you want to identify something as a vector in space, you need to provide it's representation (x,y,z OR r, theta, phi, etc) and also provide a coordiante from Also -- don't break existing servers that don't identify vectors as such |
|
The goal here is to describe what the data provider has indicted for coordinate frame names. SPASE uses a generally agreed upon list of coordinate names that many datasets use. Some frames, like GSM, are hard to re-create exactly numerically, so different implementations will not agree (beyond 5 places). HAPI is not really able to solve the ambiguities in the coordinate definitions. Documentation about coordinate frames is outside the scope of HAPI. The best approach for HAPI is to just provide a way to expose the info present in the data, and leave it to users to find out any details about precise definitions used. This is likely to work for a handful of well-known frames, and then there will be a lot of additional mission-specific frames that are dataset specific. SPASE has a CoordinateSystem object with a CoordinateRepresentation (constrained to be cartesian, cylindrical, sphereical), and then a CoordinateSystemName, also constrained to a lists of specific frame names. Two proposals are At top level: At the parameter level: |
|
Notes from telecon on 2021-08-02: Need to check with SPEDAS team (Eric and Jim) about their source of coordinate frame names. Jeremy to set up meeting. Possible sources of existing coordinate system definitions and names: FITS WCS: (Expert is Bill Thompson - could he give an overview? Bobby to ask for 20 minute session at future HAPI telecon.) Solar Orbiter has data in FITS and CDF, so they are following ISTP guidelines and FITS stanadards: Consistency with coordinate definition schemes within SpacePy, as well as SunPy and AstroPy (Note: SunPy and AstroPy do support FITS WCS) For Planetary - no definitive set of coordinate frame names - different for each mission and even for instruments within a mission. Still broad agreement that specifying a coordinate schema would be useful. Need to understand how complete existing systems are, and how they use names, add names, resolve conflicting views of frames, etc. Note: schema is for the full dataset. Other aspect of this is to be able to indicate which parameters are vectors, and then which frame they are in, as well as which coordinate representation (cartesian, cylindrical, spherical, other). SpacePy has CType (cartesian, cylindrical, etc) and CSys (frame name like GSM, GSE, etc) |
We get the coordinate systems from the CDF metadata - specifically, the 'COORDINATE_SYSTEM' variable attribute. These are set by the individual instrument teams (so sometimes it's simply 'GSE' or 'GSM', and sometimes it's mission/instrument specific, e.g., 'DBCS' for MMS/FPI's 'near-GSE' coordinate system). I think our transformation routines (at least the ones I'm familiar with) assume 3D cartesian coordinates (x, y, z). |
|
Bobby, Eric, Jon and I (Jeremy) met to talk about this issue. Eric is using CDF metadata in SPEDAS that identifies coordinate systems for CDF variables. The case we looked at was https://cdaweb.gsfc.nasa.gov/hapi/info?id=MMS1_FGM_SRVY_L2@1 or in Autoplot vap+cdaweb:ds=MMS1_FGM_SRVY_L2&id=mms1_fgm_r_gsm_srvy_l2 . For this one, the metadata contains COORDINATE_SYSTEM=GSM and REPRESENTATION_1=represent_vec_tot. This data is mms1_fgm_r_gsm_srvy_l2[n,4], with X,Y,Z, and magnitude of the 3-vector. We debated about whether this 4-component notation should be supported in HAPI, or if the server maintainer should be encouraged to split them into two separate parameters (r_vec and r_mag). Otherwise, the CDF has a model and we should probably just support this. So here we'd have something like: So here "CartesianPlusMagnitude" asserts that the data is [n,4] and the 4 channels are X, Y, Z, and magnitude. We also looked at ndcube. Jon captured the chat which had a bunch of interesting links, and will be posting it here. |
|
Here's the Zoom chat finally posted from Aug 5: These links and info were all from Bobby Candey: https://docs.sunpy.org/projects/ndcube/en/stable/ndcube.html#coordinate-transformations see page 53 of Solar Orbiter metadata: CDF metadata example: Two more examples: |
|
I'm going to create a very simple schema for identifying a coordinate system, and then we can use that for now, but with the idea that a larger effort should really expand on our toy model. It's not part of the HAPI spec, so I'll put something here: |
|
Based on discussion on 2022-02-21: This examples adds a The You can label a vector with a short-cut label if the elements follow a common convention. If none of the common conventions match, then you can specify what vector components you have, and in what order. Note: need to discuss “size” : [1] case – you could have this for a 1-D vector (i.e., x dimension only) These are only right-handed coordinate systems. component names: For angles:
Phi is the azimuthal angle:
For data with geographic coordinates, there might have only 2 elements (latitude and longitude). For ground magnetometer data, common vector component labels are: “coordinateSystemName” : “Legacy Ground Mag Frame” Sometimes, also the inclination angle I is specified. For spherical polar coordinates: Examples: |
|
Note that this approach only works if the vector quantities are present in the data as an array, with each element of the array being one of the vector components. There are plenty of datasets where this is not true, i.e., the vector quantities are split out as separate scalars. But aggregating these scalars into what would effectively be a new variable starts getting into a kind of semantic overlay on top of HAPI data. That is outside the scope of this kind of change. Jeremy and i have thought about how to convert what is present in HAPI (or other structured data, like CDF) to a more semantically uniform data structures that have science meaning. This kind of Science Data Interface would be a way of viewing the data as known semantic entities, and those entities would need to be assembled from whatever information and structures are present in the data. This will be entered in another ticket. |
|
For a list of coordinate frame names, the SPASE schema actually specifies a fairly lengthy list already, so we do not need to define our own schema! See this page for a list of frame names: The SPASE list has the eight coordinates that are in SSCWeb: And the three that are mentioned in COHOWeb: |
|
One other potential Also, there should probably be an |
|
To avoid the issue of vector and allow for generalization to higher dimensions, we could allow elements of labels to be objects: |
|
So would we be adding a new element "labels"? I like this idea. "comp" should be a list of somewhat reserved, so that we don't get "magnitude" and "mag". Also I think tag names should never be abbreviated, so: If a label contains a tag "coordinateSystem" then this overrides for just that component, allowing multiple coordinateSystems to be used. |
|
What I wrote was incomplete notes from my discussion with Jon. It gets around the problem of having stuff that is not a vector described under a vector key as in Jon's proposal. So I suggested something like This has two problems. (1) It does not indicate that "Magnitude" is the vector magnitude (was a used case we wanted to address) and (2) It is somewhat awkward to put this sort of information inside something called To address (1), we could add The reason for adding |
|
Why not "components" instead of "vectors" or "labels" ? All we need is something to more precisely describe what each component is, and it seems like with a set of allowed values (x,y,z,magnitude,theta,phi,rho,other) the client could easily figure out how to use the vector. We should keep this simple. The more complex it is, the more documentation we'll need, and the less likely it is that people will actually use it. |
|
Would that work for HDZ? |
|
We could qualify H, D, and Z, like allocate "magnetometer_H", "magnetometer_D", and "magnetomer_Z" |
|
Here's an example of where the components are not identified:
|
|
In this case (with the CDAWeb AC_H3_MFI data), it's pretty clear what these three components are. They are inside an array of length 3 that is representing magnetic field, so ideally there would be very little more HAPI metadata needed for a computer to recognize that this is a vector in a specific frame. To indicate that this is a Cartesian vector in a specific frame, just add this: The defaults representation is Cartesian, and the default component ordering is x,y,z. I still think having a separate (This kind of RTN frame is commonly used, and it is present int he SPASE list of standardized names. Note that SPASE is at 2.4.1 now: |
|
Yes, that would be sufficient to let me know what I can do with it. We still have the case of the n x 4 block, and it would be nice if you could specify the components explicitly (R,T,N,magnitude). |
|
Here is what we concluded at the end of yesterday’s telecon, which is basically to add an optional keywords. At the dataset level (this is consistent with earlier discussion): At the parameter level:
(The |
|
This was useful: https://en.wikipedia.org/wiki/Spherical_coordinate_system |
|
I talked with a math person about this, and he favored descriptive names, like latitude and colatitude. For cases where the branch cut is at a different place, he suggested we use these names: We could also use these: For angles starting at the minus Z axis: For west longitude: |
|
Looks good. I'd change |
|
Here is the collected set of what we've come up with so far. At the dataset level: At the parameter level:
Note that the separate These can also be used on scalar parameters too. Note: use unicode Pi and degrees |
|
For referencing the SPASE list of coordinate systems, we can for now just use the URL to the SPASE schema, since that has been stable as a way to reference the list of known systems there. https://spase-group.org/data/schema/spase-2.4.1.xsd FYI, here is Bobby's code for getting enumerated items out of SPASE (using jq to extract enumerated values For type = Enumeration) in case we wanted to extract is separately: For now, we won't extract - we will just reference the XSD and ask for a DOI (still). |
|
Bob and I reviewed the coordinate frame and vector component descriptions today, and we realized there is too much invented terminology involved in some of the named items for vector components. Plus we still don't cover all the potential weird cases, and it will be hard to expand our nomenclature consistently if we wanted to include them later. Standard vector components like x,y,z, r, rho, latitude, longitude, and colatitude are clear, and things like azimuth and azimuth0 are common enough that they can be justified (could be point of discussion), but all the "inverse" quantities (inverseLatitude, inverseAzimuth, inverseColatitude) take quite a bit of mental wrangling to wrap your head around, and these don't cover all the corner cases. The "inverse" terminology is something we invented, so this will be confusing and (more importantly) open to misinterpretation, especially when seen apart from the detailed descriptions. For example, we currently have We also have inverseColatitude, which is the angle away from the -Z axis, such that the -Z axis itself is at 0 degrees of inverseColatitude, and +Z is 180 degrees. There are multiple other combinatorics that you could envision that we don't have named items for. They are going to be rare, but some data really does come in weird coordinate frames. We think it's better to have just the simple components as named items, and then allow for a way to indicate angles that are relative to any plane or axis with any sense of direction (i.e., which way is positive for this angle). A plane is defined with two vectors, and the order is important because then the cross product tells you the direction of positive angles. So if you want to indicate what we now have as If you wanted Longitudinal type angles are clock angles around an axis, so you need to specify the rotation axis and the axis that defines the zero point (and also the angle range). Regular longitude would be this:
If you hand an angle that is West Longitude (starts at +X, but is positive towards -Y), you would have this: (Notice the -Z axis, so that the rotation goes the other way). This can then accommodate any possible angle from any of the Cartesian axes with any sense of direction. These I suggest that all ranges for these definition be given in degrees, regardless if the quantities themselves are in degrees or radians. I could be convinced, but radians means having users enter values for Pi, which is awkward. |
|
One other item worth adding: the ability to associate scalar parameters with each other so they could be linked together as a single vector quantity. If a dataset has |
|
Provide aq table to map our names to commonly used names in other fields (RA, DEC), and (latitude, longitude), etc. Include HDZ for ground magnetometers. Then have some full metadata examples using the vector components. |
|
Here's a way to express custom or non-standard angles. Since the custom angles may appear in more than one parameter, they can be defined less compactly outside the parameter list, and then when listing Here is draft text we could put in the spec:If the data you are presenting has an angle not represented in the table of standard There are three types of angles that can appear in data: It takes different criteria to define each of these types of angles: Note that the units of the angle are already specified within the units keyword for the parameter. You could also specify Allowed values in the "relevantAxes" are Examples follow for
Then to use these in a parameter definition, you just refer to the angle name: Description of HDZ from From the URL: Figure 1a shows a local Cartesian system in which the magnetic field vector, F, has components, H, D, and Z. H is defined as positive northwards, D as positive eastwards, and Z as positive downwards. Here "north" can mean one of two things: Either, "north" can mean geographic north. This is defined as the direction of the great circle path to the geographic north pole. Alternatively, "north" can mean geomagnetic north. This is generally not the same as geographic north for reasons which will be apparent later. Geomagnetic north is usually defined as the direction that the local field points in the horizontal plane, either on average or on a "quiet" day when there is little short time scale (< 1 day) variation. Figure 1b shows a local cylindrical polar coordinate system in which the magnetic field vector, F, has components, H, D, and Z. Here Z is the vertical component defined as positive downwards, as before, H is the component of the magnetic field in the horizontal plane, and D is the angle of the horizontal field component, H, from the eastward direction. D is referred to as the declination. (It is often to be seen on maps, indicating the angle between geographic and geomagnetic north.) Another angle that may be quoted is also shown in the figure. It is the inclination angle, I, between the vector, F, and the horizontal plane. We can see that (F, /2 -I , D) define a spherical polar coordinate system. |
|
This looks close. It captures the three types of angles:
When The only issue I see is that 1., defines a zero angle axis and no rotation direction (b/c not needed). Name is |
|
Yes, the last element for azimuthal angles is the zero angle (i.e., this: The types 1 and 2 above do tend to have fixed angle ranges, (0 to 180 for rel. to an axis) and then -90 to 90 for rel. to a plane. Can they ever be anything else? If so, we can drop that part. The angle in a plane can be either 0 to 360 or -180 to 180. I agree that Hmm.. If the defining axes are all different, then do we even need the |
|
Maybe have a keyword that is: and then those axes have different meanings for each type. We could say that axis2 must be null for |
|
I would require polar to always be 0-180 or 0 to pi. Otherwise, one can specify the same thing using an elevation angle. I'd also require elevation always to be -90 to 90 or -pi to pi. (I made a correction in my list for 2. I said "always 0-180".) Or, we could drop elevation and say the labeling starts at the axis. So latitude is polar = "z" with angles [90,-90] means label z=90, xy plane = 0, -z = 90 and colatitude is polar = "z" with angles [0, 180] so z=0, xy plane = 90, -z = 180. |
|
I agree with the angle range requirements. I favor keeping the Note that range designations are always in degrees, regardless of units of the data itself, since they are just specifying the intent of the vector component. This leads to a quick comment about 0 to 24 for MLT that Iused in an above example. This should not be there since it confuses units with angle range. Angle range should always be in degrees (regardless of the units of the actual data). This means that the angle range is only relevant for azimuthal items. And also, the angle range is really just a flag (is it 0 to 360, or -180 to 180). So rather than require a numeric range, we should just make that a flag. The flag indicates the location of the branch cut (i.e., where is the discontinuity: at 0 or at 180)? |
|
I think (and would like) the MLT to still be captured. If the units are "hours" and it is identified as an angle, then to get an angle in degrees, use a factor of 24. We do a similar thing with pi if the units are "radians". So we would allow angle units to be degrees, radians, and fractional hours. But then we'd have to answer the follow-up of how to handle the angle specified by integers "hours, minutes" and fractional seconds. |
|
MLT is still captured. You can represent a floating point MLT in hours without any need for a Example values would be: There is a confusing point here is that the way you specify the |
|
You could also define MLT to be a custom angle if you wanted to. It would really just become a synonym for longitude. |
|
Here then are the allowed
If the data you are presenting has an angle not represented in the table of standard There are three types of angles that can appear in data: It takes different criteria to define each of these types of angles: If your azimuthal angle has the discontinuity at zero (as in 0 to 360 or 0 to 2 PI), then use Note that the units of the angle are already specified within the units keyword for the parameter. Allowed values in the "relevantAxes" are Examples follow for
Then to use these in a parameter definition, you just refer to the angle name: |
|
At today's telecon, we decided to move forward with the standard list of vectorComponents, and delay inclusion of customAngles pending further external review. It should be easy to add that on later, and getting it wrong now means having to support non-optimal stuff for a potentially long time. |
|
Here are the parts that I took out of the draft spec, but want to save for next time. From the From the Description of If a vector component in the data is an angular quantity that is not in the Once you have defined a custom angle and given it a name, you can use that name as you There are three types of angles that can appear in data: It takes different criteria to define each of these types of angles: If your azimuthal angle has the discontinuity at zero (as in 0 to 360 or 0 to 2 PI), then use Note that the units of the angle are already specified within the units keyword for the parameter. All custome angle definitions are given in a list of objects with the Allowed values for axis1 and axis2 are: Examples follow for
Then to use these in a parameter definition, you just refer to the angle name: |
addresses issue #115 - coord frames
|
While installing astropy, I noticed a package named "asdf". The page https://docs.astropy.org/en/stable/io/asdf-schemas.html#coordinates-representation-1-0-0 has a description of representations of angles. The links to the schema are broken, but I did find this https://asdf-coordinates-schemas.readthedocs.io/en/latest/generated/schemas/representation-1.0.0.html, which does not provide definitions of terms. |
|
closed by #132 |
Like with the units, there is no standard way of identifying coordinate systems for vector quantities. For example,
vap+hapi:https://cdaweb.gsfc.nasa.gov/hapi?id=RBSP-A_MAGNETOMETER_4SEC-GEO_EMFISIS-L3¶meters=Mag&timerange=2019-10-14
is known by the human operator to be in GEO at RBSP-A, but there is no mechanism to let the computer know that.
So we propose that a coordinate system identifier be introduced, and schemas to provide a mechanism to interpret the identifier.
The text was updated successfully, but these errors were encountered: