Timezone and unix (absolute?) conversion

[stebogit]

I've been reading throughout the docs and the issues in this repo to figure out how to use Temporal, but I haven't found an answer to this. My apologies in advance if I missed an obvious point.

The scenario I have in mind is I want to save the timestamp of an event in my database and store it as an unix (absolute?) timestamp so I can later display it in any timezone I prefer simply providing the expected timezone. However I couldn't find a straightforward way of doing that.

I tried:

const date = Temporal.DateTime.from('2020-01-03T19:41:00-08:00')
date.toAbsolute('UTC').toString(); // "2020-01-03T19:41:00Z"

but the returned string has not been converted to the right time. I'm not really clear on the meaning of "absolute", if we have to pass a timezone to the conversion... 🤔 Doesn't a Temporal.Absolute represent a UTC time?

Now, this seems to work in converting the string to the right UTC time:

const date = Temporal.Absolute.from('2020-01-03T19:41:00-08:00');
date.toString(); // "2020-01-04T03:41:00Z"

however I wonder how I would convert a Temporal.DateTime instance into a UTC value since

const date = Temporal.DateTime.from('2020-01-03T19:41:00-08:00')
const utc = Temporal.Absolute.from(date.toString());
OR
const utc = Temporal.Absolute.from(date);

they both error out with

// Uncaught RangeError: invalid ISO 8601 string: 2020-01-03T19:41:51
//    at Object.ParseISODateTime (ecmascript.mjs:98)
//    at Object.ParseTemporalAbsoluteString (ecmascript.mjs:142)
//    at Object.ParseTemporalAbsolute (ecmascript.mjs:243)
//    at Function.from (absolute.mjs:198)
//    at <anonymous>:1:19

From the docs toString should return an ISO string (i.e. one that includes date, time and timezone) and Absolut.from should accept any "thing" (which I assume include a DateTime); is that not correct?

---

As a side note, if I may suggest, I would expect in the Temporal API a straightforward way to (immutably) convert timezone of a Temporal.DateTime instance, something like

Temporal.DateTime.toTimezone(tz: Temporal.TimeZone)

[Ms2ger]

The thing you may be missing is that a DateTime has no concept of a time zone.

const date = Temporal.DateTime.from('2020-01-03T19:41:00-08:00')
// ^ you lose the time zone here; DateTime only stores the '2020-01-03T19:41:00' part.
date.toAbsolute('UTC').toString(); // "2020-01-03T19:41:00Z"
// ^ this interprets '2020-01-03T19:41:00' as being in UTC

As you noticed, Temporal.Absolute.from is how you convert a datetime+timezone string to Absolute. If you want the unix timestamp, you can use getEpochSeconds().

A string without a time zone such as '2020-01-03T19:41:51' does not define a single point in time, so Temporal.Absolute.from can't interpret it as one, which is why it throws.

[stebogit]

Thanks @Ms2ger for the insight; now that you mention this, it actually looks like none of the current Temporal classes actually represent an actual date-time instance aware of the timezone. Is that possible?

[justingrant]

it actually looks like none of the current Temporal classes actually represent an actual date-time instance aware of the timezone. Is that possible?

Hi @stebogit - It's coming soon. #700 specifies this new DateTime+offset+TimeZone type (called LocalDateTIme as a placeholder, although the name will change) that's being finalized now. The plan is for it to be implemented in the polyfill soon.

Could you clarify specifically the use case you're trying to support? Specifically, when you read the value back from your database, what type is that value? Is it an ISO 8601 string with offset, like 2020-01-03T19:41:00-08:00? An ISO string at UTC, e.g. 2020-01-04T03:41:00Z? A number of milliseconds since UNIX epoch? A number of seconds since UNIX epoch? Something else?

Regardless, here's how you'd get each of those formats into a Temporal.Absolute instance, and then how you'd get it into a Temporal.LocalDateTime instance once that type lands in the polyfill:

const abs = Temporal.Absolute.from(`2020-01-03T19:41:00-08:00`);
const abs = Temporal.Absolute.from(`2020-01-04T03:41:00Z`);
const abs = Temporal.Absolute.fromEpochSeconds(1599764621);
const abs = Temporal.Absolute.fromEpochMilliseconds(1599764621000);
// regardless of how you get it into Absolute, the conversion to LocalDateTime is the same
abs.toLocalDateTime('America/Los_Angeles', 'iso8601');  // first param is time zone, second is calendar

And here's how you'd get them out once LocalDateTime lands in the polyfill:

const ldt = Temporal.LocalDateTime.from(`2020-01-03T19:41:00-08:00[America/Los_Angeles]`);
ldt.toString(); // => "2020-01-03T19:41:00-08:00[America/Los_Angeles]"
ldt.toAbsolute().toString(); // => "2020-01-04T03:41Z"
ldt.toAbsolute().toString({ timeZone: 'America/Los_Angeles' }); // => "2020-01-03T19:41:00-08:00"
ldt.toAbsolute().getEpochSeconds(); // => 1578109260
ldt.toAbsolute().getEpochMilliseconds();  // => 1578109260000

Next question: how are you getting the time zone? Is it stored in a different column in the same table that also stores the from-epoch value that you're reading and writing? Or do you use the local timezone on the client? Or the server's system timezone? Or is the timezone stored in your database, just at a different granularity (e.g. per user) than the time data you're storing? Do you get it from some external system or API? Or some other way?

Regardless of where you get the timezone, what format is the timezone represented as? As an IANA name like America/Los_Angeles? As a localized name like "Pacific Daylight TIme"? As a pure offset like -08:00? As an OS-defined enumeration, like a Windows system timezone? Some other way of encoding the time zone?

[justingrant]

@stebogit - if you have a chance to reply to my questions in the previous comment, it'd be helpful for us as we figure out how best to address the concerns you raised. Thanks!

[stebogit]

@justingrant I sure will, tomorrow at latest. Sorry I planned to it today, but then carried away by #887 😅

[stebogit]

MySQL's DATETIME type does not account for timezone, therefore you can't store that information simply using the native field type. Even if there is a timezone function, still the stored value is always associated with the system timezone (it looks like though this has changed in v8 where the DATETIME does support the TZ offset).
Now you can store the timestamp in a VARCHAR field as ISO string including the timezone, or you store it as DATETIME (i.e. without timezone) or unix timestamp (i.e. an INT column) in one column and then you store the timezone in a separate column. Now in my understanding indexing an INT or a DATETIME is much more efficient than a string.
As far as I know the unix timestamp is usually expressed in seconds (see MySQL, PHP, GoLang) or milliseconds (see MongoDB/BSON, Moment.js ); I don't know of languages using nanoseconds.

Regarding the timezone names/definition, IANA names/abbreviations are a widely adopted standard. However in my specific case I have to specify the offset (-08:00) because the Pacific Standard Time (i.e. no DST) does not exist as string.
I have never seen anywhere the IANA name in the timestamp string (like 2020-01-03T19:41:00-08:00[America/Los_Angeles]) and I always used the ISO-8601 definition, which expects an offset time like +04:00, +0400, or +04. This seems to be the standard adopted in other languages.

For generating a local (user/system related) date/time object, usually both JS and PHP use the system timezone as default if none is passed to the constructor.

Thanks for the code you provided for the Temporal.LocalDateTime.
However I'd argue that the toString method, which is typically (in different languages) a "reserved" method to simply provide a string representation of the object, should not allow for side effects like change the timezone.
So I'd have instead:

const ldt = Temporal.LocalDateTime.from(`2020-01-03T19:41:00-08:00[America/Los_Angeles]`);
ldt.toString(); // => "2020-01-03T19:41:00-08:00[America/Los_Angeles]"
const abs = ldt.toAbsolute();
abs.toString(); // => "2020-01-04T03:41Z"
abs.toLocalDateTime('America/Los_Angeles').toString(); // => "2020-01-03T19:41:00-08:00"

As a final note, all the examples above starts with a timestamp to instantiate the Temporal object. In other contexts though you might get the instance as argument of a method or a parameter of an object and you might need to do stuff based on its timezone or pass that to another object or method. Is there a way?
I don't see something like getTimezone method in the Absolute or DateTime class, will LocalDateTime have one?

[justingrant]

I don't see something like getTimezone method in the Absolute or DateTime class, will LocalDateTime have one?

Yes. The most important thing that distinguishes LocalDateTime from other types is that it persists the time zone. LocalDateTime (or whatever its final name ends up being) will have a timeZone property that will return a Temporal.TimeZone instance. Furthermore, that time zone will be used to make all LocalDateTime operations "DST safe" which means that days won't automatically be assumed to be 24 hours long but instead the length of each day will be determined by the TZ database corresponding to the instance's IANA timezone. Some examples of how this works:

const midnight = Temporal.LocalDateTime.from('2020-03-08T00:00-08:00[America/Los_Angeles]');
midnight.hoursInDay;
// => 23

const noon = midnight.with({hour: 12});
noon.difference(midnight).toString()
// => "PT11H", because an hour is skipped by DST

midnight.plus({days: 1}).toString();
// => "2020-03-09T00:00-07:00[America/Los_Angeles]"
// adding days (or larger) units automatically adjust for DST, per RFC 5545

midnight.plus({hours: 24}).toString();
// => "2020-03-09T01:00-07:00[America/Los_Angeles]"
// adding hours (or smaller) units adds absolute time (ignoring DST) per RFC 5545

So I'd have instead:

const ldt = Temporal.LocalDateTime.from(`2020-01-03T19:41:00-08:00[America/Los_Angeles]`);
ldt.toString(); // => "2020-01-03T19:41:00-08:00[America/Los_Angeles]"
const abs = ldt.toAbsolute();
abs.toString(); // => "2020-01-04T03:41Z"
abs.toLocalDateTime('America/Los_Angeles').toString(); // => "2020-01-03T19:41:00-08:00"

Your code sample is correct except the last line. Because LocalDateTime persists its time zone, the time zone will be emitted by toString() to enable round-trip persistence.

abs.toLocalDateTime('America/Los_Angeles').toString(); // => "2020-01-03T19:41:00-08:00[America/Los_Angeles]"

For your use case where you can't persist the full string, we're considering an option for LocalDateTime.prototype.toString to hide the IANA timezone in its output. Syntax is still TBD, but you can look at #703 (comment) for some of the ideas we're considering.

You will also be able to emit the bracketless format directly from Absolute, as soon as the decisions from #741 (comment) get PR-ed. Example of how it will work:

> abs.toString('America/Los_Angeles'); // => "2020-01-03T19:41:00-08:00"

The reason this works is that Absolute has two valid string formats: one that's a UTC time ending in Z, and another that includes a numeric offset. The example above is how you'd emit the latter format. The time zone parameter is needed because Absolute doesn't persist the offset-- it only stores a nanoseconds-since-Epoch value.

However in my specific case I have to specify the offset (-08:00) because the Pacific Standard Time (i.e. no DST) does not exist as string.

Could you explain a little more about this use case?

Specifically when you say "Pacific Standard Time (i.e. no DST) does not exist as string.", do you mean that if you add 6 months to one of these values, you'd expect the result to still be in -08:00 even though six months later the Pacific Time Zone would have -07:00 offset thanks to DST?

Or do you simply mean that the back-end systems that you're working with don't have the ability to persist a value like 2020-01-03T19:41:00-08:00[America/Los_Angeles] so your only choice is to persist the IANA zone and the bracket-less ISO string separately?

Or do you mean something else? Please clarify. Thanks!

For generating a local (user/system related) date/time object, usually both JS and PHP use the system timezone as default if none is passed to the constructor.

This "implicit local time zone" behavior is specifically NOT how Temporal behaves, because this behavior in other platforms has been the source of countless bugs, especially in server apps where the local time zone often won't match users' time zones.

Instead, the only places where time zones are defaulted to the system time zone are the methods on Temporal.now, e.g. Temporal.now.localDateTime(). All other methods in Temporal that need a time zone (e.g. LocalDateTime.from, Absolute.prototype.toLocalDateTime, etc.) will require explicitly specifying the time zone.

[stebogit]

However in my specific case I have to specify the offset (-08:00) because the Pacific Standard Time (i.e. no DST) does not exist as string.

I merely mean that, for some odd reason, a Pacific Standard Time referenced IANA label does not exist. There is only America/Los_Angeles, which represents Pacific Daylight Time, i.e. subject to DST.
Anyway, we collect hourly weather records, which clearly ignore DST. So their timestamps are always in standard time and we should never have more or less than 24 records per day. This also means that all calculations of determining the timestamp, for collecting records in a date range for example, is done in standard time.

This "implicit local time zone" behavior is specifically NOT how Temporal behaves, because this behavior in other platforms has been the source of countless bugs, especially in server apps where the local time zone often won't match users' time zones.

The best way to align client and server(s) is to always use UTC timestamps everywhere (thus RFC3339). This will save you a lot of headaches and bugs. Only the client (as in "the user facing system") should apply the timezone offset, if necessary, as last step.
I partially agree that a local/system default timezone can be bug prone, if used improperly. However, now that I think about it, this means that if you want to "localize" a Temporal instance you need to know the system timezone. I'm not aware of a way to get the system timezone in JS, like a dedicated browser/node function returning that information, other than creating a Date instance, exactly because it defaults to the local timezone (I honestly don't know how it gets that value)

const sysOffset = -(new Date()).getTimezoneOffset() / 60;
// -7
const sysTz = Math.abs(sysOffset) > 9
    ? `${sysOffset}:00`
    : sysOffset > 0 ? `0${sysOffset}:00` : `-0${-sysOffset}:00`;
// -07:00

Does/will Temporal allow you to get it in a similar/equivalent way? I can't find a way in the current API, it seems everything assumes the timezone, when applicable, is part of the arguments passed to the constructor (even for a TimeZone).

[justingrant]

Does/will Temporal allow you to get it in a similar/equivalent way? I can't find a way in the current API, it seems everything assumes the timezone, when applicable, is part of the arguments passed to the constructor (even for a TimeZone).

Yes. Temporal.now is the API for fetching data from the local environment, including current time and time zone. Example:

Temporal.now.timeZone();
// => America/Los_Angeles
Temporal.now.absolute();
// => 2020-09-19T22:33:49.592762660Z
Temporal.now.localDateTime(); // when it lands in a few weeks
// => 2020-09-19T15:32:49.592762660-07:00[America/Los_Angeles]

I merely mean that, for some odd reason, a Pacific Standard Time referenced IANA label does not exist.

You can think of the IANA time zone database as a function that that accepts two parameters:

1. a time zone ID, which is usually a geographic key like America/Los_Angeles, but less commonly single-offset names like Etc/GMT+5 or UTC
2. a UTC time value.

This "function" returns a calendar date and local clock time for that key (geographic area or single offset) at that UTC time.

The reason why Pacific Standard Time isn't accepted is because the canonical name of the IANA time zone for that area is America/Los_Angeles.

But more fundamentally, Pacific Standard Time is an offset time zone corresponding to -08:00, just like Pacific Daylight Time is an offset time zone for -07:00 and UTC is an offset time zone for +00:00. The IANA time zone naming convention removes the ambiguity between Standard Time and Daylight time by keying off the largest city in a region.

[ptomato]

I'm assuming this is addressed by ZonedDateTime? Closing.