Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Aggs: use time_zone in date_histogram to format bucket keys #9710

Closed
cbuescher opened this issue Feb 16, 2015 · 4 comments
Closed

Aggs: use time_zone in date_histogram to format bucket keys #9710

cbuescher opened this issue Feb 16, 2015 · 4 comments

Comments

@cbuescher
Copy link
Member

Follow up issue to the date_histogram clean up in #9062. Since we have only one time_zone option now, we could use this to print the keys of buckets in a date_histogram that uses other time zone than UTC to the specified time zone.

More details about how this could look like here: #9062 (comment)

@jpountz
Copy link
Contributor

jpountz commented Feb 18, 2015

+1

@cbuescher
Copy link
Member Author

I'm working on a solution with minimal code changes that would only require a setter for the time zone in the DateTime formatter and a small change in the DateHistogramParser. I ran some local tests, a query like

GET /dates/docs/_search?search_type=count 
{
  "aggs" : {
    "date_agg" : {
      "date_histogram": {
        "field": "date",
        "interval": "month",
        "time_zone": "Europe/Amsterdam"
      }
    }
  }
}

would now get results like:

"aggregations": {
      "date_agg": {
         "buckets": [
            {
               "key_as_string": "2014-03-01T00:00:00.000+01:00",
               "key": 1393628400000,
               "doc_count": 4
            },
            {
               "key_as_string": "2014-06-01T00:00:00.000+02:00",
               "key": 1401573600000,
               "doc_count": 2
            },
            [...]
         ]
      }
   }

The nice thing is that key_as_string is still a UTC date makes more sense in terms of the specified time zone because it seemingly starts at midnight but the zone is appended. This also works nicely with DST (as demostrated above).

Without specifying the time_zone the result would be something like:

"aggregations": {
      "date_agg": {
         "buckets": [
            {
               "key_as_string": "2014-03-01T00:00:00.000Z",
               "key": 1393632000000,
               "doc_count": 4
            },
            {
               "key_as_string": "2014-06-01T00:00:00.000Z",
               "key": 1401580800000,
               "doc_count": 2
            },
            [...]
         ]
      }
   }

Note that this doesn't affect custom formatting like:

"format" : "YY-MM-dd#HH-mm-ss#Z"  
-->
{
        "key_as_string": "14-03-01#00-00-00#+0100",
        "key": 1393628400000,
        "doc_count": 4
},

Also, the time zone can entirely be dropped with custom formatting so it looks like local time, but the key is still UTC:

"format" : "YY-MM-dd#HH:mm:ss"  
-->
{
        "key_as_string": "14-03-01#00:00:00",
        "key": 1393628400000,
        "doc_count": 4
},

@cbuescher
Copy link
Member Author

I wonder if this needs to be configurable because it is a nice default behaviour and one can always specify a format that drops the time zone using custom formatting. I will open a PR with my current changes for discussion.

@jpountz
Copy link
Contributor

jpountz commented Feb 18, 2015

@cbuescher The behaviour described above looks great to me. Also +1 to not introduce additional settings.

cbuescher pushed a commit to cbuescher/elasticsearch that referenced this issue Feb 18, 2015
…ng to `time_zone`

Change bucket key_as_string to reflect `time_zone` parameter. Currently `time_zone`
shifts bucket boundaries to other time zone, but keys are displayed in UTC, so e.g.
daily buckets in "+01:00" time zone have key_as_string like "2014-01-01T23:00:00Z". With this
change the default is to format this dates according to the local time zone, so the
above bucket key would be "2014-01-02T00:00:00+01:00".

Closes elastic#9710
@javanna javanna changed the title Aggs: use time_zone in date_hisogram to format bucket keys Aggs: use time_zone in date_histogram to format bucket keys Feb 25, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants