Added a test to handle DIF10 records from NPI

[steingod]

NPI releases DIF10 records with an appearance like:
'''
<Metadata_Dates>
<Metadata_Creation>2023-12-04T17:11:12Z</Metadata_Creation>
<Metadata_Last_Revision>Not provided</Metadata_Last_Revision>
<Data_Creation>Not provided</Data_Creation>
<Data_Last_Revision>Not provided</Data_Last_Revision>
</Metadata_Dates>
'''
This cause issues on ingestion side for MMD records which have dates of type "Not provided". A hack to prevent such strings to pass through has been added. Full datetime validation would be easier with XSLT 2, but for now we are using XSLT 1.

[ferrighi]

I've tested the modifications, but this is not extracting the fields when it is a datetime element (the string is not of length 8). Since dif 10 supports a controlled vocabulary for Metadata_Last_Revision (and in principle also for other datetime fields) I have 2 suggestions:

• exclude the valid vocabulary entries:
  <xsl:if test="$datetimestr != 'Not provided' and $datetimestr !='unknown' and $datetimestr !='present' and $datetimestr !='unbounded' and $datetimestr !='future'">
• make sure there are numbers in the string:
  <xsl:if test="translate($datetimestr, '1234567890', '') != $datetimestr">

I've tested both and they seem to work. I can implement and push to this branch if you have any preference.

[steingod]

Sounds good, I am happy if you push and merge. Ø

…

On Wed, 2024-01-17 at 00:31 -0800, Lara Ferrighi wrote: I've tested the modifications, but this is not extracting the fields when it is a datetime element (the string is not of length 8). Since dif 10 supports a controlled vocabulary for Metadata_Last_Revision (and in principle also for other datetime fields) I have 2 suggestions:  * exclude the valid vocabulary entries:    <xsl:if test="$datetimestr != 'Not provided' and $datetimestr !='unknown' and $datetimestr !='present' and $datetimestr !='unbounded' and $datetimestr !='future'">  * make sure there are numbers in the string:    <xsl:if test="translate($datetimestr, '1234567890', '') != $datetimestr"> I've tested both and they seem to work. I can implement and push to this branch if you have any preference. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

-- Dr. Øystein Godøy Norwegian Meteorological Institute  P.O.BOX 43, Blindern, N-0313 OSLO, Norway Ph: (+47) 9802 4433

[ferrighi]

I've added the filtering of strings, and left the bit redundant check on the presence of number. It should create a valid mmd if datetime is provided.