Units & Prefixes

[MRichards99]

Description:
This is the least discussed part of the Search API and we've discussed using Pint for this. I have verified that Pint supports all units and prefixes defined by the documentation in the example implementation (May 2021) so it should be suitable for the task.

At this point in time, I'm not entirely sure how this will be implemented, but is an important part of the deliverable nonetheless.
Further thinking required.

Acceptance criteria:

• ADD CHECKLIST ONCE FURTHER THINKING HAS BEEN DONE

[VKTB]

Section Units and Conversions of the 3.1 API Definition (common search API) deliverable document contains more info about this.

[MRichards99]

Ultimately, we need the metadata to be cleaned up and made machine readable for Pint and other maths/unit libraries, but the overall workflow to deal with different units would look something like this:

• At startup (or daily cron job/equiv. for Python), get a distinct list of parameter types and store them
• Ingest them into Pint and put them into categories (e.g. temperature, weight)
• When a request comes in, search for the unit in the ingested list and get all other units of that type
• When constructing that part of the WHERE clause, have a condition for each of the units in the group (e.g. WHERE 100km or 62 miles) and maybe do something with SI units?
• Once the query has been executed and you have results back, iterate through the results and if a result contains units not requested by the user, convert them and change the unit name to what the user asked for (e.g. request asks for degC but result in K, convert result to degC and change unit name to degC)

[MRichards99]

As discussed with @louise-davies, this feature won't be used by ISIS currently, even if we implemented it.

ISIS don't use DatasetParameters, only InvestigationParameter is used and is also relevant for the search API. However, there are no investigations linked/have relationships with InvestigationParameters that also have meaningful unit names. If you send the following DataGateway API request to an instance pointing at ISIS dev ICAT, an empty list is returned (this is the same for any of the other unit names):

http://{{datagateway-api}}/investigations?include="parameters.type"&where={"parameters.type.units":{"ilike":"GPa"}}

This is because while there are meaningful units stored for InvestigationParameter, none of them are used/linked/related to any investigations stored in ISIS' ICAT. For context, it seems that ISIS use DatafileParameter to store units such as temperature and pressure; there is no equivalent entity in the PaNOSC data model.

As a result of these findings, I'm marking this issue with the 'wontfix' label. Currently, this is a metadata issue that's much bigger than the search API. Creating a meaningful test environment to implement this feature would be possible, but it would be time consuming and certainly not viable for the deadline of the deliverable.

We would welcome development help from other facilities who would make use of this feature.

[agbeltran]

thanks @MRichards99 and @louise-davies - I agree with the approach, and we can get back to implementing this feature once we look at the metadata issues. Indeed, other facilities might want to tackle this and contribute back to the code base and that would be great.