Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add per entry and per property meta data field. #463

Merged
merged 43 commits into from
Jun 16, 2023
Merged
Changes from 13 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
4170937
First draft metadata field.
JPBergsma Apr 6, 2023
92c4d92
updated example so the numbers make sense.
JPBergsma Apr 11, 2023
6fae33d
Added that metadata fields can have their own metadata.
JPBergsma Apr 11, 2023
d343a86
Add specifically that meta suffix must not be used other than for met…
JPBergsma Apr 18, 2023
54b632c
Moved database specific fields to a seperate section and renamed them…
JPBergsma Apr 18, 2023
5d186d9
Apply suggestions from code review
JPBergsma Apr 18, 2023
cb63bf2
Added 'metadata_for' field.
JPBergsma Apr 25, 2023
d0a4e65
Revert "Moved database specific fields to a seperate section and rena…
JPBergsma Apr 25, 2023
9e64a3a
Revert "Add specifically that meta suffix must not be used other than…
JPBergsma Apr 25, 2023
498f1f5
Merge branch 'JPBergsma_add_meta' of https://github.com/JPBergsma/OPT…
JPBergsma Apr 25, 2023
6cfe92f
Moved mention that suffix '_meta' should not be used for regular fiel…
JPBergsma Apr 25, 2023
1414c55
Add extra single quotes around the '_meta' suffix.
JPBergsma Apr 26, 2023
81f2f96
Added example of a property definition for a metadata field.
JPBergsma Apr 26, 2023
b69b0e8
Update optimade.rst
JPBergsma May 1, 2023
478a572
Clarifined nested meta data fields a bit.
JPBergsma May 1, 2023
d3b7050
Merge branch 'JPBergsma_add_meta' of https://github.com/JPBergsma/OPT…
JPBergsma May 1, 2023
8ed79b5
Placed meta_data_for field in the property definition.
JPBergsma May 1, 2023
f1e179c
Removed property definition metadata_for field as it is now part of t…
JPBergsma May 4, 2023
a60eb5c
Added comma to improve readability.
JPBergsma May 4, 2023
df40172
Added comma to improve readability.
JPBergsma May 4, 2023
d16e97c
Merge branch 'JPBergsma_add_meta' of https://github.com/JPBergsma/OPT…
JPBergsma May 4, 2023
1e81988
Postponed decision on whether or not nested metadata fields are allowed
JPBergsma May 4, 2023
6f56a17
Moved the metadata fields to the JSON API per entry meta field as sug…
JPBergsma Jun 1, 2023
97a0435
Removed x-optimade-metadata-for from example property declaration as …
JPBergsma Jun 1, 2023
00a7a08
Tried to make it clearer when a server SHOULD return the meta data.
JPBergsma Jun 1, 2023
7099a69
removed extra '.'
JPBergsma Jun 2, 2023
09fffea
Apply suggestions from review
rartino Jun 8, 2023
cceabb5
Fix where metadata properties appear based on discussion
rartino Jun 8, 2023
45b98b8
Remove the repetition of info about the data->meta field
rartino Jun 8, 2023
b95c465
Provide an example that is less objectable for metadata
rartino Jun 11, 2023
134a906
Fix examples as requested by workshop
rartino Jun 11, 2023
078f174
Apply suggestions from code review
JPBergsma Jun 12, 2023
e15aaff
Apply suggestions from review
rartino Jun 12, 2023
9c4417f
Fix JSON example spelling error
rartino Jun 12, 2023
b36c372
Update optimade.rst
JPBergsma Jun 14, 2023
cc42d8f
Apply suggestions from review
rartino Jun 14, 2023
52ad0c0
Remove global per-entry metadata properties
rartino Jun 14, 2023
62f987d
Remove ability for per-entry global metadata
rartino Jun 14, 2023
aaddc82
Apply suggestions from review
rartino Jun 15, 2023
656e1fb
Merged upstream develop
rartino Jun 16, 2023
4feb8e1
Update optimade.rst
merkys Jun 16, 2023
6992038
Cleanup trailing whitespace.
merkys Jun 16, 2023
6ffe550
Apply suggestions from review
rartino Jun 16, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
106 changes: 105 additions & 1 deletion optimade.rst
Original file line number Diff line number Diff line change
Expand Up @@ -381,7 +381,7 @@ API implementations SHOULD NOT make up and use new prefixes without first gettin

**Examples**:

- A database-provider-specific prefix: ``exmpl``. Used as a field name in a response: :field:`_exmpl_custom_field`.
- A database-provider-specific prefix: ``exmpl``. Used in a field name in a response: :field:`_exmpl_custom_field`.

The initial underscore indicates an identifier that is under a separate namespace under the ownership of that organization.
Identifiers prefixed with underscores will not be used for standardized names.
Expand Down Expand Up @@ -442,6 +442,109 @@ For example, the following query can be sent to API implementations `exmpl1` and

:filter:`filter=_exmpl1_band_gap<2.0 OR _exmpl2_band_gap<2.5`

Per property metadata fields
----------------------------

Implementations are allowed to specify an OPTIONAL field containing per entry metadata for a property.
The name of the metadata field consists of the name of the property for which it contains the metadata, suffixed with "_meta".
merkys marked this conversation as resolved.
Show resolved Hide resolved
For example, when the field is :property:`cartesian_site_positions` the metadata field would be :field:`cartesian_site_positions_meta`.

This metadata field consists of a dictionary which MAY contain database specific fields.
Fields in this metadata MAY also have a metadata field.
rartino marked this conversation as resolved.
Show resolved Hide resolved
If an implementation supports the metadata field, it SHOULD return the metadata field whenever the property to which the metadata field belongs is returned.

The metadata fields and their subfields should be described in the property definitions as described in the section `property definitions`_ just as regular fields.
If a subfield is present in multiple metadata fields these subfields should have a separate entry under each of these metadata fields.
The subfields SHOULD have the same value for the $id field if the $id field is present and the subfields are otherwise identical.
JPBergsma marked this conversation as resolved.
Show resolved Hide resolved

It SHOULD contain the field :

- **metadata_for**: which contains a string with the name of the property for which this meta data property contains the meta data,
rartino marked this conversation as resolved.
Show resolved Hide resolved

Example of a returned metadata field:

.. code:: jsonc

{
"element_ratios":[0.33336, 0.22229, 0.44425],
"element_ratios_meta": {
"metadata_for": "element_ratios",
"_exmpl_confidence_interval": [[0.33325,0.33347],[0.22190,0.22268],[0.44392,0.44458]],
"_exmpl_confidence_interval_meta": {
"_exmpl_confidence_level": 0.95,
"metadata_for": "_exmpl_confidence_interval",
}
rartino marked this conversation as resolved.
Show resolved Hide resolved
//...
}

Example of the property definition of a metadata field:

.. code:: jsonc
{
"element_ratios_meta": {
"$id": "https://properties.example.com/v1.2.0/element_ratios_meta",
"title": "Metadata for the element_ratios field",
"description": "This field contains the metadata for the element_ratios field that is specific to each individual entry.",
"x-optimade-property": {
"property-format": "1.2"
},
"x-optimade-type": "dictionary",
"x-optimade-unit": "inapplicable",
"type": ["object", "null"],
"properties" : {
"metadata_for": {
"$id": "https://properties.example.com/v1.2.0/element_ratios_meta/metadata_for",
"description" : "Defines the property for which this metadata field contains metadata.",
"x-optimade-type": "string",
"x-optimade-unit" : "inapplicable",
rartino marked this conversation as resolved.
Show resolved Hide resolved
"type": "string"
},
"_exmpl_confidence_interval": {
"$id": "https://properties.example.com/v1.2.0/element_ratios_meta/_exmpl_confidence_interval",
"description" : "This field consists of a list with a list that contains the lower and upper bounds of the confidence interval.",
"x-optimade-type": "list",
"x-optimade-unit" : "inapplicable",
"type": "array",
"items": {
"x-optimade-type": "list",
"description" : "The first value in this list is the lower bound for the confidence interval the second value is the upper bound.",
"x-optimade-unit" : "inapplicable",
"type": "array",
"items": {
"x-optimade-type": "float",
"x-optimade-unit" : "dimensionless",
"type": "number"
}
}
},
"_exmpl_confidence_interval_meta": {
"$id": "https://properties.example.com/v1.2.0/element_ratios_meta/_exmpl_confidence_interval_meta",
"title": "Metadata for the _exmpl_confidence_interval field",
"x-optimade-type": "dictionary",
"x-optimade-unit": "inapplicable",
"type": ["object", "null"],
"properties" : {
"metadata_for": {
"$id": "https://properties.example.com/v1.2.0/element_ratios_meta/metadata_for",
"description" : "Defines the property for which this metadata field contains metadata.",
"x-optimade-type": "string",
"x-optimade-unit" : "inapplicable",
"type": "string"
},
"_exmpl_confidence_level": {
"x-optimade-type": "float",
"description" : "The confidence level for this interval e.g. 0.95",
"maximum": 1.0,
"minimum": 0.0,
"x-optimade-unit" : "dimensionless",
"type": "number"
}
}
}
}
}
}

Responses
=========

Expand Down Expand Up @@ -2307,6 +2410,7 @@ database-provider-specific properties
Implementations are thus allowed to decide that some of these properties are part of what can be seen as the default value of :query-param:`response_fields` when that query parameter is omitted.
Implementations SHOULD NOT include database-provider-specific properties when the query parameter :query-param:`response_fields` is present but does not list them.
- These MUST be prefixed by a database-provider-specific prefix (see appendix `Database-Provider-Specific Namespace Prefixes`_).
- The suffix ``_meta`` is reserved for properties that contain metadata, as described in the section `Per property metadata fields`_ and MUST not be used for fields that do not follow the guidelines described there.
- Implementations MUST add the properties to the list of :property:`properties` under the respective entry listing :endpoint:`info` endpoint (see `Entry Listing Info Endpoints`_).

- **Examples**: A few examples of valid database-provided-specific property names follows:
Expand Down