Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DocValueFormat implementation for date range fields #47472

Merged
merged 5 commits into from
Oct 4, 2019
Merged

DocValueFormat implementation for date range fields #47472

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
61824ee
DocValueFormat implementation for date range fields
not-napoleon Oct 2, 2019
89ce21c
Fix documentation output
not-napoleon Oct 4, 2019
6e5e8f0
Quote json strings...
not-napoleon Oct 4, 2019
04b66fd
fix agg formatting
not-napoleon Oct 4, 2019
fab7821
Merge branch 'master' into bugfix/date-histo-range-format
not-napoleon Oct 4, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 11 additions & 2 deletions docs/reference/aggregations/bucket/range-field-note.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,8 @@ POST /range_index/_search?size=0
"november_data" : {
"date_histogram" : {
"field" : "time_frame",
"calendar_interval" : "day"
"calendar_interval" : "day",
"format": "yyyy-MM-dd"
}
}
}
Expand All @@ -135,35 +136,43 @@ calculated over the ranges of all matching documents.
"aggregations" : {
"november_data" : {
"buckets" : [
{
{
"key_as_string" : "2019-10-28",
"key" : 1572220800000,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the doc tests are failing because date_histo will show both the key and key_as_string. E.g.

                {
                    "key_as_string": "2015-01-01",
                    "key": 1420070400000,
                    "doc_count": 3
                },

"doc_count" : 1
},
{
"key_as_string" : "2019-10-29",
"key" : 1572307200000,
"doc_count" : 1
},
{
"key_as_string" : "2019-10-30",
"key" : 1572393600000,
"doc_count" : 1
},
{
"key_as_string" : "2019-10-31",
"key" : 1572480000000,
"doc_count" : 1
},
{
"key_as_string" : "2019-11-01",
"key" : 1572566400000,
"doc_count" : 1
},
{
"key_as_string" : "2019-11-02",
"key" : 1572652800000,
"doc_count" : 1
},
{
"key_as_string" : "2019-11-03",
"key" : 1572739200000,
"doc_count" : 1
},
{
"key_as_string" : "2019-11-04",
"key" : 1572825600000,
"doc_count" : 1
}
Expand Down
19 changes: 19 additions & 0 deletions server/src/main/java/org/elasticsearch/index/mapper/RangeFieldMapper.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,11 +45,13 @@
import org.elasticsearch.index.fielddata.IndexFieldData;
import org.elasticsearch.index.fielddata.plain.DocValuesIndexFieldData;
import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.search.DocValueFormat;

import java.io.IOException;
import java.net.InetAddress;
import java.net.UnknownHostException;
import java.time.ZoneId;
import java.time.ZoneOffset;
import java.util.HashSet;
import java.util.Iterator;
import java.util.List;
Expand Down Expand Up @@ -269,6 +271,23 @@ public Query existsQuery(QueryShardContext context) {
}
}

@Override
public DocValueFormat docValueFormat(String format, ZoneId timeZone) {
if (rangeType == RangeType.DATE) {
DateFormatter dateTimeFormatter = this.dateTimeFormatter;
if (format != null) {
dateTimeFormatter = DateFormatter.forPattern(format).withLocale(dateTimeFormatter.locale());
}
if (timeZone == null) {
timeZone = ZoneOffset.UTC;
}
// the resolution here is always set to milliseconds, as aggregations use this formatter mainly and those are always in
// milliseconds. The only special case here is docvalue fields, which are handled somewhere else
return new DocValueFormat.DateTime(dateTimeFormatter, timeZone, DateFieldMapper.Resolution.MILLISECONDS);
}
return super.docValueFormat(format, timeZone);
}

@Override
public Query termQuery(Object value, QueryShardContext context) {
Query query = rangeQuery(value, value, true, true, ShapeRelation.INTERSECTS, null, null, context);
Expand Down
16 changes: 16 additions & 0 deletions ...elasticsearch/search/aggregations/bucket/histogram/DateRangeHistogramAggregatorTests.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,22 @@ public void testBasics() throws Exception {
);
}

public void testFormat() throws Exception {
RangeFieldMapper.Range range = new RangeFieldMapper.Range(RangeType.DATE, asLong("2019-08-01T12:14:36"),
asLong("2019-08-01T15:07:22"), true, true);
testCase(
new MatchAllDocsQuery(),
builder -> builder.calendarInterval(DateHistogramInterval.DAY).format("yyyy-MM-dd"),
writer -> writer.addDocument(singleton(new BinaryDocValuesField(FIELD_NAME, RangeType.DATE.encodeRanges(singleton(range))))),
histo -> {
assertEquals(1, histo.getBuckets().size());
assertTrue(AggregationInspectionHelper.hasValue(histo));

assertEquals("2019-08-01", histo.getBuckets().get(0).getKeyAsString());
}
);
}

public void testUnsupportedRangeType() throws Exception {
RangeType rangeType = RangeType.LONG;
final String fieldName = "field";
Expand Down