"property is less than" tests are failing for valid data

[JohannaOtt]

Describe the bug
property Is Less Than_match All and property Is Less Than_match Any tests are failing with reason of failure: "java.lang.AssertionError: XPath expression cannot be evaluated: xs:dateTime(ns1:beginLifespanVersion) lt xs:dateTime('2020-08-15T00:00:00Z')Invalid dateTime value "" (too short). "
The element beginLifespanVersion is indeed empty for some objects in the WFS but as the attribute is nillable and a nilReason value is provided, that should be valid IMHO.

To Reproduce
Validate this WFS with the OGC Team Engine Validator.

Expected behavior
I would expect the handling of beginLifespanVersion to be valid.

Screenshots

[dstenger]

Thank you for reporting. I agree that this is most likely a shortcoming of the test suite.
We will do further investigation.

[bpross-52n]

@JohannaOtt The following request is sent to the service:

<wfs:GetFeature xmlns:wfs="http://www.opengis.net/wfs/2.0" count="10" service="WFS"
                version="2.0.0">
   <wfs:Query xmlns:ns75="http://inspire.ec.europa.eu/schemas/am/4.0"
              typeNames="ns75:ManagementRestrictionOrRegulationZone">
      <Filter xmlns="http://www.opengis.net/fes/2.0">
         <PropertyIsLessThan matchAction="All" matchCase="true">
            <Literal>2020-08-15T00:00:00Z</Literal>
            <ValueReference xmlns:tns="http://inspire.ec.europa.eu/schemas/am/4.0">tns:beginLifespanVersion</ValueReference>
         </PropertyIsLessThan>
      </Filter>
   </wfs:Query>
</wfs:GetFeature>

This returns only features, that have a nil value for the beginLifespanVersion property. IMHO this is not correct.
A query without the filter returns features, that have an earlier date than requested in the filter, as you can check with this request:

<wfs:GetFeature xmlns:wfs="http://www.opengis.net/wfs/2.0" count="10" service="WFS"
                version="2.0.0">
   <wfs:Query xmlns:ns75="http://inspire.ec.europa.eu/schemas/am/4.0"
              typeNames="ns75:ManagementRestrictionOrRegulationZone">
   </wfs:Query>
</wfs:GetFeature>

Could you please check this behavior?

[ghobona]

@pvretano @cportele Looking at the example requests above, can PropertyIsLessThan be applied to Date literal values?

The FilterEncoding spec describes in chapter 7.7 Comparison operators the different types of operators. PropertyIsLessThan is part of the binary comparisons. On the other hand, temporal operators are defined in chapter 7.9 Temporal operators.
As chapter 7.7 just contains mathematical comparisons, temporal comparisons should, in our opinion, not be supported here.

Thus, our conclusion is that PropertyIsLessThan should not work for temporal values.

[cportele]

@ghobona

I read section 7.7 differently. In general, the requirements are not as clear as they should be. But to me "mathematical comparison between two arguments" implies that the operators can be applied to all arguments for which there is a natural ordering of the values.

This is trivial for decimal numbers.

Implementations obviously have to support strings, too, since the matchCase attribute exists; this requires some additional assumptions in order to have an unambiguous ordering (in particular, the encoding that is used), but this is left to the implementations.

Since there is also a natural ordering of temporal instants of the same granularity, I do not see how to conclude that "PropertyIsLessThan should not work for temporal values". On the other hand, since the text does not explicitly require that the binary comparison operators also work for temporal values, it may also be difficult to argue that implementations must support them. However, I would say that either they reject them as a bad request or they have to properly apply the comparison operator.

This is independent from supporting additional temporal operators.

For comparison: In CQL2 we try to be clearer. The binary comparison operators have to support temporal values (but only if you compare a date against a date or a timestamp against a timestamp). In addition, there are the temporal operators (similar to FES, but we use OWL Time as the basis, not 19108) as a separate conformance class.

A different topic: I also notice from the requests that requests are sent as WFS 2.0.0 requests. WFS 2.0.0 and FES 2.0.0 have been deprecated 8 years ago and should not be used any longer, in particular in the official OGC CITE tests. Please update to WFS 2.0.2 and FES 2.0.2. These are also the only WFS 2.0 and FES 2.0 schema versions available in the OGC schema repository.

[dstenger]

@cportele Thank you for the answer.

Sorry, but we have an additional question as we do not find a clear answer in the spec.
As reported in this issue, a request using a "property is less than" filter with temporal values also returns features which have a nil value in the requested property (here beginLifespanVersion; also see #224 (comment)).
So, the question is how filters shall handle nil values. Shall features with nil values be returned when e.g. a "property is less than" filter is used on that property or not?

Thank you for pointing out the missing support of WFS 2.0.2 and FES 2.0.2. We created an issue for that and put it into our backlog: #248

[cportele]

@dstenger

Shall features with nil values be returned when e.g. a "property is less than" filter is used on that property or not?

No, they are not part of the result set (NULL < 2020-08-15T00:00:00Z evaluates to NULL).

You are correct, this is not explicit in FES, but basically implied by Filter being an XML encoding of a WHERE clause in an SQL SELECT statement. Again, we try to be more explicit in CQL2.

[dstenger]

@JohannaOtt
Thus, we conclude that the features with nil values shall not be returned by the service when a "property is less than" filter is used. Please also consider #224 (comment).

I propose to close this issue as the problem is not located in the test suite. Are you fine with that?

[JohannaOtt]

@dstenger As there is deegree/deegree3#1422 for the issue now, I am fine with closing this one.

[dstenger]

@JohannaOtt However, the problem with the nil value is not described in that issue (just that features with valid dates are not returned). If you are using deegree, can you please add that finding to the deegree issue?