Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Broken range queries on "date||epoch_millis" fields in 7.1 #42835

Closed
phil-bl opened this issue Jun 4, 2019 · 3 comments · Fixed by #43080
Closed

Broken range queries on "date||epoch_millis" fields in 7.1 #42835

phil-bl opened this issue Jun 4, 2019 · 3 comments · Fixed by #43080
Labels
>bug :Search/Search Search-related issues that do not fall into other categories

Comments

@phil-bl
Copy link

phil-bl commented Jun 4, 2019

After upgrading from 6.8 to 7.1.1 the following query results in error:

{
  "query": {
	  "bool": {
	    "filter": [
	      {
	        "range": {
	          "create_date": {
	            "to": 297276785531,
	            "include_upper": true
	          }
	        }
	      }
	    ]
	 }
  }
}

it used to work before the upgrade. The problem is in include_upper = true, it works if changed to false.

The error message is:

failed to parse date field [297276785531] with format [date||epoch_millis]: [Text '297276785531' could not be parsed, unparsed text found at index 0]

mapping for this field is:

      "create_date": {
        "type": "date",
        "format": "date||epoch_millis"
      }

another example of the same issue, but with include_lower:

{
  "query": {
	  "bool": {
	    "filter": [
	      {
	        "range": {
	          "create_date": {
	            "from": 297276785531,
	            "include_lower": false
	          }
	        }
	      }
	    ]
	 }
  }
}

Again, it works with "include_lower": true

This behavior doesn't affect "date_hour_minute_second_fraction||epoch_millis" fields, the issue seems to be related only to "date||epoch_millis" mapping format.

Elasticsearch version:
Version: 7.1.1, Build: oss/docker/7a013de/2019-05-23T14:04:00.380842Z, JVM: 12.0.1

JVM version:
OpenJDK 64 12.0.1

OS version
4.14.78-coreos

@not-napoleon not-napoleon added :Search Foundations/Mapping Index mappings, including merging and defining field types >bug labels Jun 4, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search

@not-napoleon not-napoleon added :Search/Search Search-related issues that do not fall into other categories and removed :Search Foundations/Mapping Index mappings, including merging and defining field types labels Jun 4, 2019
@not-napoleon
Copy link
Member

Possibly related to #41160 ?

@pgomulka
Copy link
Contributor

pgomulka commented Jun 5, 2019

It is related to #40100 where I changed the logic for parsing composite date formats (with ||) . I wrongly assumed that formats for round up can not have composite formats.

pgomulka added a commit to pgomulka/elasticsearch that referenced this issue Jun 13, 2019
roundUp parsers were losing the composite pattern information when new
JavaDateFormatter was created from methods withLocale or withZone.

The roundUp parser should be preserved when calling these methods. This is the same approach in withLocale/Zone methods as in https://github.com/elastic/elasticsearch/blob/daa2ec8a605d385a65b9ab3e89d016b3fd0dffe2/server/src/main/java/org/elasticsearch/common/time/JavaDateFormatter.java

closes elastic#42835
pgomulka added a commit to pgomulka/elasticsearch that referenced this issue Jun 13, 2019
roundUp parsers were losing the composite pattern information when new
JavaDateFormatter was created from methods withLocale or withZone.

The roundUp parser should be preserved when calling these methods. This is the same approach in withLocale/Zone methods as in https://github.com/elastic/elasticsearch/blob/daa2ec8a605d385a65b9ab3e89d016b3fd0dffe2/server/src/main/java/org/elasticsearch/common/time/JavaDateFormatter.java

closes elastic#42835
pgomulka added a commit that referenced this issue Jun 14, 2019
roundUp parsers were losing the composite pattern information when new
JavaDateFormatter was created from methods withLocale or withZone.

The roundUp parser should be preserved when calling these methods. This is the same approach in withLocale/Zone methods as in https://github.com/elastic/elasticsearch/blob/daa2ec8a605d385a65b9ab3e89d016b3fd0dffe2/server/src/main/java/org/elasticsearch/common/time/JavaDateFormatter.java

closes #42835
pgomulka added a commit that referenced this issue Jun 14, 2019
roundUp parsers were losing the composite pattern information when new
JavaDateFormatter was created from methods withLocale or withZone.

The roundUp parser should be preserved when calling these methods. This is the same approach in withLocale/Zone methods as in https://github.com/elastic/elasticsearch/blob/daa2ec8a605d385a65b9ab3e89d016b3fd0dffe2/server/src/main/java/org/elasticsearch/common/time/JavaDateFormatter.java

closes #42835
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug :Search/Search Search-related issues that do not fall into other categories
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants