-
Notifications
You must be signed in to change notification settings - Fork 3.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support per-query timezone offsets #2074
Comments
Doing a simple +/- is not sufficient for full timezone support because of daylight savings (PST vs PDT depending on the time of year). For example, if you have a Right now, I am planning on implementing an index on top of influx for this purpose, but would love to not have to do that. |
IMO there should be both options. Query-wide parameter covers most of the cases but function that just returns offset based on supplied timezone (which would be necessary for per-query operator anyway) is much more convenient than doing that client-side |
Regarding "any other questions", can support be provided for falling edge v rising edge groupings? time_zone as a "number" should be a four number string too (some XX30 tz) |
Not sure if you're still debating format, but I'd recommend supporting the Olsen / tz database. See http://joda-time.sourceforge.net/timezones.html. It supports named timezones, as well as UTC offsets (which are confusingly backwards, ie. Etc/GMT+5 is actually GMT-5). This is the closest thing to a standard I've ever seen. |
+1 for Olsen tz ... with timezone('america/phoenix') |
why not write a timezone item in conf file? |
+1 |
1 similar comment
+1 |
another +1 for olsen db rather than literal offsets or any other named time zones though I should also point out that timezones aren't the only problem. if I do a group by time(30m), I might want my groups to be between 15 and 45, so that the "middle" of the grouping is on the hour and half hour, when I expect my peak load due to user traffic patterns. and that has nothing to do with timezone - it's more "offset the grouping by a certain amount" |
It may be the best |
I would use a configurable default timezone + query param to override it |
I eagerly awaited this feature. Make timezone can be configurable(in influxdb.conf) maybe is easier. |
+1 |
+1 for setting default timezone in config file. Currently, I'm using a quick workaround which consists in using a cron script or program to retrieve the average of the day (executed at 00:01 AM for example) of the desired series from InfluxDB, using the correct start and end timestamps in the WHERE clause of the query. After that, insert the data in a new series with the timestamp of the beginning of the day and you'll be able to retrieve the correct data in your queries. |
I really want to know which version will include this feature, is there a PR proposed already? |
There is no PR for this yet, and the work is currently unscheduled. |
+1 |
@hotsnow77 we will happily accept a PR for this feature, but I don't think you'll find it simple. It is definitely a highly desired addition but likely won't be tackled until clustering and other stability/availability concerns are considered mature. |
This is pointless. If the devs are busy with other features that they consider more important why bother complaining? Either get together some golang developers who can send a patch or let them work on their priorities without pointless complaints that will only slow them down because they are nice and they'll find the time to answer politely. We want this feature as bad as you so don't slow them down with useless comments. PS: this is an open source project, "without warranty of any kind" |
+1 for per-query timezone |
An offset of `time(1m, now())` will anchor the offset to the start time of the query. The default offset is `0s` which is the current default anyway. This fixes #2074 by making time zone offset support unnecessary. Time comparisons can use timezones inside of the time clause and the offset needed for non-hour timezone differences can be used as part of the offset argument.
An offset of `time(1m, now())` will anchor the offset to the start time of the query. The default offset is `0s` which is the current default anyway. This fixes #2074 by making time zone offset support unnecessary. Time comparisons can use timezones inside of the time clause and the offset needed for non-hour timezone differences can be used as part of the offset argument.
An offset of `time(1m, now())` will anchor the offset to the current time of the query. The default offset is `0s` which is the current default anyway. This fixes #2074 by making time zone offset support unnecessary. Time comparisons can use timezones inside of the time clause and the offset needed for non-hour timezone differences can be used as part of the offset argument.
This does seem to add a constant offset to the comparison so it does not handle summer/winter time changes in the dataset, right? |
Yeah, not sure you can close out a timezone requirement with a time offset solution. Sorry guys. Only partly resolved. |
@giko45 if you can explain a bit what you mean, we could add something more. What kind of output are you expecting? Times all get represented as UTC so the main concern is having the client interpreting the result. Even when you cross timezone boundaries due to going from something like PDT to PST, you can just do this:
These times will work properly and only select data within those time periods. It's just the output will always be in UTC. If you have an idea for something that can't be done with the currently existing tools, can you open an issue describing your use case? |
The offset bucket you added now allows me to account for the fact that I want my buckets at 06:00Z, but it does not account for the fact that in the specified interval for this query, one of the days has a bucket that is 23 hours, and that the UTC time of the boundaries between buckets will change from 07:00Z to 06:00Z on that day. This is not just a presentation problem, because the bucket size needs to dynamically adjust based on when the DST transition is. Also note that in the query as written right now, there's no way to know when the DST transition is. The offsets -0700 and -0600 don't tell you that - only the actual timezone ('America/Denver') has enough information to know when DST transitions are. Also note that while the offset solution is enough to help half-hour timezones (like you mentioned - India) get by (ignoring DST problems), if you provide a way for a timezone to be provided, you shouldn't also need to specify an offset, since you can infer the offsets from the time zone (i.e. current offset % 60 will tell you if it's a 30 minute, or possibly even 15/45, or :04 -- I'm pretty sure one of those zones exist). In fact, I would expect something like "SELECT MEAN(value) FROM series GROUP BY time(1h) AT TIME ZONE 'Asia/Kolkata'" to automatically infer an offset of 30m from UTC times for the grouping. |
A syntax like #6541 or @ccutrer However implementation requires |
+1 |
it would be nice if the time_zone option is available |
@mattbasta assuming 8h could be augmented with minute based offset, there's still the issue of DST which can't be dealt with like this :( |
@mattbasta all good and well to hold the opinion, however there are ample use cases and it's a fairly standard requirement for a database dealing with datetimes to be capable of dealing with the real world and its complexities. Imho this is a long outstanding issue that needs resolving by the Influx core team and I know I am not alone. Until that happens should we implement Influx to meet our tsdb needs we will be limited to aggregating at a half hour level in the database and then completing any aggregation in the application. This puts it at a disadvantage when comparing with other options. |
If one saves data in the database in UTC then daily aggregates can simply be done by converting to the right TZ before aggregration, this will handel DST too.
Normal select queries will return data in UTC (as the data is saved in UTC) but Clients can convert to local TZ (which is good practice anyway since clients might be in diffrent TZ) |
@jufemaiz Does any other tsdb handle that better ? Genuinely curious, haven't played with other ones much |
@mattbasta PST and PDT are not time zones. They are simply defined offsets (-8 and -7). Pacific Time is a time zone, and includes the information about when it changes offsets (DST), as well as nicknames for those offsets. And it's not hard for software to support it. There is a well maintained database of time zone information that software can use to know what the DST rules are for any given time zone in any given year (within practicality): https://en.wikipedia.org/wiki/Tz_database. Usually software will support the tzinfo data files, and rely on the operating system to keep them up to date. |
I deleted my comment. I don't want to be involved in this discussion. Feel free to argue all you want. |
Influx should store data with a UTC timestamp. Formatting timestamps should be a display detail. |
@greggwon you're correct except for the rather large use case of aggregation methods by calendar days (weeks, months, years etc), then InfluxDB needs to be able to manage timezone offsets. In this case it is not merely a display detail but a functional requirement :) |
+1 |
1 similar comment
+1 |
Aggregation becomes a "display detail", because you are now visualizing. The functional definition as shown in the sample query is not my point. My point is that there should be a common time reference that is automatically known for all data. Data should not float around in varied Timezones and then have to be coerced to another in random fashion. Instead, there should be a single base timezone reference and then "visualization" can specify what timezone is important. In the sample query, the 'time' comparison to "today()" also implies a visualization (extraction of the data for use) and so today() should also be offset by a timezone offset. today() needs the parameter, not a new function which will randomly interject problems to all other 'time' values in an expression. Each use of a time function, should by default use the server timezone offset but allow a parameter which specifies the timezone offset applicable. Yes, that becomes more places that the offset must be specified, but it allows the developer/application the explicit control needed to make sure that we don't suddenly have offset math strewn about the query with various +21600000, and -21600000 etc. |
I will be locking this issue. Litigating the result of a closed issue that was opened two years ago and resolved six months ago is unfair to the people subscribed to the issue and I think it is unfair to forcibly involve others in the discussion. With that said, I am happy to share the rationale for why the issue was resolved in the way it was resolved. But, the proper location for that question is on our community forums and not the issue tracker. @greggwon if you want to continue discussion, please create an account on that website and post a new topic. I will keep an eye out for the next day to see if there is any discussion about time zones there. If you have issues with the way it was resolved that pertain to a valid use case of time zone math, please open an issue. I do not want to fail to address valid use cases since I want the database to be as useful to as many people as possible. If there are problems with the existing implementation, I want to address them so that everyone benefits from a better database. One clarification that might be needed for most people (since I have seen this confusion before), the TZ parameter does not affect any time values in the actual query. It affects time boundaries. The time Time zones are a disabled feature by default. If you don't include the I hope that clarification helps and look forward to your continued engagement with the InfluxData community. |
Extending the discussion started in #2071 and re-starting the old discussion from #388, we should support time zones on a per-query basis.
A quick example might look like:
Or you could also do
time_zone(+8)
ortime_zone(-2)
.Any other suggestions?
The text was updated successfully, but these errors were encountered: