Remove time zone and calendar protocols and classes

[ptomato]

Conclusion in the champions meeting of 2024-04-18. We've been asked to reduce the surface area and complexity of the proposal. The callable hooks in the time zone and calendar protocols are the part that implementations have been most uncomfortable with. They also seem to be where we get the biggest reduction in complexity for the least loss of use cases. The TimeZone and Calendar classes themselves make less sense to keep if the protocols are gone. So our conclusion is to remove them.

Motivations for removing

• Requiring observable calls in a particular sequence makes it difficult for implementations to optimize, unless they maintain separate code paths for built-in vs. user-supplied calendars/time zones.
• Large increase in complexity of proposal for relatively little gain.
• This functionality made more sense back when you could monkeypatch Calendar.from() and TimeZone.from() to ‘register’ a custom ID. That ability was removed when the proposal went to stage 3 because the committee will not agree to global state.
• Without a calendar protocol, there doesn’t seem to be much point in having a Temporal.Calendar object. All the functionality is still easily available through PlainDate methods and properties, (except for Calendar.p.fields() and Calendar.p.mergeFields() which are weird methods that only exist to make it possible to build custom calendars with extra fields.)
• Without a time zone protocol, the usefulness of a Temporal.TimeZone object is heavily reduced, although it doesn’t become completely redundant like Temporal.Calendar.

Use cases that disappear, and their replacements

Work around incomplete/outdated support in CLDR calendars and TZDB

For example:

• Adjustment days in Hijri calendars
• https://github.com/khawarizmus/hijri-week-calendar
• Make sure program runs against known version of time zone database regardless of whether environment is ahead or behind

Replacement: Previously, you could implement a custom calendar or time zone by creating an object that implemented the protocol, and monkeypatching some of Temporal so that it would deserialize dates with your custom calendar/time zone's ID into a Temporal object with the correct protocol object. You would now have to monkeypatch more of Temporal. As part of moving this issue forward, we'll create a proof of concept for doing this, to make sure that it remains possible.

Converting directly between Instant and PlainDateTime

Replacement: Go through ZonedDateTime, instead of through TimeZone. e.g.

// Before
timeZone.getPlainDateTimeFor(instant)
timeZone.getInstantFor(plainDateTime, { disambiguation })

// After
instant.toZonedDateTimeISO(timeZoneID).toPlainDateTime()
plainDateTime.toZonedDateTime(timeZoneID, { disambiguation }).toInstant()

Determine if two time zone IDs are aliases

Replacement: Use ZonedDateTime.p.equals() with the same instant. (This is more inconvenient, but it's a pretty uncommon operation. If this really needs to be more ergonomic, a dedicated static method can be added in a follow up.)

// Before
Temporal.TimeZone.from('Asia/Kolkata').equals(Temporal.TimeZone.from('Asia/Calcutta'))

// After
zonedDateTime.withTimeZone('Asia/Kolkata').equals(zonedDateTime.withTimeZone('Asia/Calcutta'))

Look up previous/next UTC offset transition in a time zone

Replacement: We'll add two methods to ZonedDateTime that replace the two methods on TimeZone.

// Before
timeZone.getNextTransition(fromInstant)
timeZone.getPreviousTransition(fromInstant)

// After
const fromZonedDateTime = fromInstant.toZonedDateTimeISO(timeZoneID);
fromZonedDateTime.nextTransition()
fromZonedDateTime.previousTransition()

Custom disambiguation behaviour

A niche use case is implementing PlainDateTime-to-Instant conversions with disambiguation behaviours other than the built-in earlier, later, compatible modes. By request, we have a cookbook recipe for this.
Replacement: You can still do it with two conversions, for example:

// Before
arrayOfInstants = timeZone.getPossibleInstantsFor(plainDateTime)

// After
arrayOfInstants = [
  plainDateTime.toZonedDateTime(timeZoneID, { disambiguation: 'earlier' }),
  plainDateTime.toZonedDateTime(timeZoneID, { disambiguation: 'later' }),
]

Scope of issue

• Remove Temporal.Calendar
• Remove Temporal.TimeZone
• User-defined methods are not looked up nor called during calculations
• Calendar-taking APIs no longer accept objects, only string IDs, and ISO 8601 strings out of which a string ID is determined
• Time-zone-taking APIs no longer accept objects, only string IDs, and ISO 8601 strings out of which a string ID is determined
• [[Calendar]] internal slot of PlainDate, PlainDateTime, PlainYearMonth, PlainMonthDay, ZonedDateTime only stores string ID
• [[TimeZone]] internal slot of ZonedDateTime only stores string ID
• Remove getCalendar() methods from PlainDate, PlainDateTime, PlainYearMonth, PlainMonthDay, ZonedDateTime prototypes
• Remove ZonedDateTime.p.getTimeZone()
• calendar property of object returned by PlainDate, PlainDateTime, PlainYearMonth, PlainMonthDay, ZonedDateTime's getISOFields() methods can only be a string
• timeZone property of object returned by ZonedDateTime.p.getISOFields() can only be a string
• Add ZonedDateTime.p.nextTransition() and ZonedDateTime.p.previousTransition()
• Update TypeScript types to match the changes above.
• Create proof of concept for how you would polyfill a custom calendar or time zone going forward

[leftmostcat]

Datetimes with custom time zones are a pretty major part of working with iCalendar (RFC 5545; https://datatracker.ietf.org/doc/html/rfc5545), since the spec has no provision for reference to a common time zone database. It would be very helpful to get a picture of what this might look like with the revised scope of this proposal.

[khawarizmus]

This is a very heavy change considering that we are this far into the proposal.

Do i understand that there is a guarantee that use cases like this https://week-dates.netlify.app/calendars/iso-extended.html or this https://week-dates.netlify.app/calendars/hijri-week-calendars.html will still be possible with some additional implementation details?

Also for the proof of concept. I would be happy to collaborate and make it work in https://week-dates.netlify.app for both aforementioned use cases

[ptomato]

Datetimes with custom time zones are a pretty major part of working with iCalendar (RFC 5545; https://datatracker.ietf.org/doc/html/rfc5545), since the spec has no provision for reference to a common time zone database. It would be very helpful to get a picture of what this might look like with the revised scope of this proposal.

@leftmostcat Our resident RFC 5545 expert @justingrant may be able to give you a better answer about VTIMEZONEs. For TZID, It looks like it's recommended but not required to use identifiers from TZDB so my guess is that for most usages the builtin time zones would be sufficient; hopefully the usages where they weren't, would be covered by the proof of concept I intend to create.

This is a very heavy change considering that we are this far into the proposal.

Do i understand that there is a guarantee that use cases like this https://week-dates.netlify.app/calendars/iso-extended.html or this https://week-dates.netlify.app/calendars/hijri-week-calendars.html will still be possible with some additional implementation details?

Also for the proof of concept. I would be happy to collaborate and make it work in https://week-dates.netlify.app for both aforementioned use cases

@khawarizmus Yes, the intention is that you'll still be able to do things like this, but you'll have to either do it by not overriding built-in Temporal APIs, or monkeypatching more of Temporal if you want the custom calendar to appear built-in.

I hope to illustrate more with the proof of concept.

I understand it's a heavy change but I don't think we'll be able to move the proposal forward without it.

[justingrant]

Datetimes with custom time zones are a pretty major part of working with iCalendar (RFC 5545; https://datatracker.ietf.org/doc/html/rfc5545), since the spec has no provision for reference to a common time zone database. It would be very helpful to get a picture of what this might look like with the revised scope of this proposal.

@leftmostcat Our resident RFC 5545 expert @justingrant may be able to give you a better answer about VTIMEZONEs. For TZID, It looks like it's recommended but not required to use identifiers from TZDB so my guess is that for most usages the builtin time zones would be sufficient; hopefully the usages where they weren't, would be covered by the proof of concept I intend to create.

We understand that removing custom time zones from Temporal V1 will create challenges for any application that needs to take a datetime+VTIMEZONE from an iCalendar message and turn that into a Temporal.ZonedDateTime object, especially if the TZID isn't an IANA time zone name. (If it is an IANA zone, you can just use it... and you could validate that the built-in zone matches the VTIMEZONE data before doing that.)

But at this point it's not clear we have another option. We need to remove this functionality so that we can ship the rest of Temporal. Here's why:

• Without shrinking the size of this proposal, Google and Apple are reluctant to ship Temporal due to code size and overall complexity. (Especially how Temporal objects can call into user code for calendars and time zones)
• The current custom time zone protocol was defined when built-in time zones used the same protocol. But built-in time zones were changed to use strings, which made the current protocol inconsistent with built-in zones.
• Combining both the above concerns, the current consensus is that the best way to define a custom time zone is to use data, not code. There's lots of prior art to choose from for this, including iCalendar's VTIMEZONE or its JSCalendar equivalent, or the data format used in the IANA Time Zone Database itself. TC39 is very careful to avoid shipping the wrong API, even if it means a delay.
• Earlier changes in the Temporal proposal removed the ability to do string parsing of custom time zone IDs. This meant that support of custom time zones was partial and incomplete. If we do revisit custom time zones then we'd want to solve this too so that we could have round-trip ability including string parsing.

So to summarize, it seems very, very likely at this point that the current custom time zone implementation will be removed. But it's also likely that there will be lots of demand for a built-in custom timezone solution in a future proposal. And it seems likely that any future solution will use a data format (ideally one based on an existing standard, of which JSCalendar seems closest in spirit to ECMAScript's design) to define a time zone instead of custom userland code.

I know this isn't a great answer for folks who have been interested in making custom time zones, but it does seem like the only realistic path forward so we can get Temporal finally shipped into browsers.

[leftmostcat]

I understand the need for a change in the spec. I'm just hoping it's possible to find a way to provide for use cases where tzdb can't be the source of truth. As noted, it's possible to fall back to tzdb in a lot of cases, but there are common clients out there which don't use standard IDs (Microsoft Exchange, notably). I have trouble seeing a way to implement that with the current API minus custom time zones.

It would be a more invasive change to the proposal, but something like OffsetDateTime instead of ZonedDateTime could potentially offer the same functionality without much change in usability, while potentially also allowing custom time zone definitions.

OffsetDateTime would just store an offset instead of a reference to a time zone. A TimeZone would just need to implement a function mapping an Instant to an offset.

[ptomato]

@leftmostcat You can effectively have OffsetDateTime already, by using ZonedDateTime with a fixed UTC offset, which is also a valid time zone identifier. For example, Temporal.ZonedDateTime.from({ year, month, day, hour, ..., timeZone: '-04:30' }). Would that be sufficient for the use case you're proposing?

[leftmostcat]

@leftmostcat You can effectively have OffsetDateTime already, by using ZonedDateTime with a fixed UTC offset, which is also a valid time zone identifier. For example, Temporal.ZonedDateTime.from({ year, month, day, hour, ..., timeZone: '-04:30' }). Would that be sufficient for the use case you're proposing?

This solution feels problematic, as it means that common operations which are currently part of ZonedDateTime need to be reinvented for any consumer which can't use IANA time zones. For example, ZonedDateTime provides the instance method add() for getting a later ZonedDateTime in the same time zone, while accounting for offset changes, etc. With a pseudo-OffsetDateTime, it's now the consumer's responsibility to understand that the time zone on the ZonedDateTime doesn't represent a real time zone, but an offset, and that add() is inappropriate. It is also the consumer's responsibility to implement their own add() functionality which adds the duration and adjusts the offset. In the end, a "correct" implementation of custom time zones requires careful maintenance of private semantics/invariants, which is one of the big problems I encounter using Date.

Building the API around OffsetDateTime means that the semantics of time zones don't have to be abused to get a correct solution. The spec and naming can make it clear that adding a duration to OffsetDateTime will not account for time zone, and those semantics would be universal across implementations. zonedInstance.add(duration) becomes offsetInstance.add(duration).relativeTo(timeZone), where timeZone is anything with an offsetAt(instant) method. The Temporal global can provide something like Temporal.getIanaTimeZone() to get suitable implementations for standard time zones.

[leftmostcat]

In hindsight, since I don't understand the issues in optimization, I recognize that my posts may not be very helpful. Hopefully I can provide some insight into what an iCalendar implementation needs out of time zones:

As I mentioned, though IANA tzids are conventional, there are implementations producing non-IANA VTIMEZONE IDs which are in common use. Establishing equivalence to an IANA tzid is non-trivial given that most implementations provide a restricted definition of the time zone, and both time zone definitions and events may include recurrence rules which may recur indefinitely. As such, implementing iCalendar requires a mechanism outside of tzdb.

Using ZonedDateTime with added semantics around the "time zone" restricted to representing offset presents many of the same problems as the current Date implementation: the existing API is built around the semantics of a time zone, not an offset, and so an implementer needs to put significant effort into reshaping something against its intended purpose. The built-in APIs will produce incorrect results, so their functionality needs to be recreated. A lot of work also needs to go into making sure that the additional context that the ZonedDateTime isn't really a ZonedDateTime isn't lost. This is error-prone and a lot of wasted effort.

Data-driven custom time zones seem like a workable path forward, but will also require some care in specifying. iCalendar offers a lot of flexibility in specifying recurrence rules, and there isn't good data on how they are actually used in VTIMEZONE definitions. It's worth noting that the set of recurrence rules which can produce any occurrences (in our case, transitions from one offset to another) is a strict subset of the set of recurrence rules iCalendar can encode. For example, a recurrence rule can specify "the first Monday of the year, so long as it occurs between the 15th and the 21st of January". Detecting these is non-trivial and they could be a source of inconsistencies and bugs in implementations. (We had an infinite loop bug due to impossible recurrence rules at one point.)

icu4c's zoneinfo64 data uses a simpler model, with transition times stored as month, day, day of week, and time (with some complexity in how day and day-of-week interact), but due to the lack of data on how real-world implementations use RRULEs in iCalendar, it's hard to say if this is sufficient to encode what implementations produce or how difficult it is to convert from RRULE to zoneinfo64 rule.

[ljharb]

This is so large a change that if it’s going to be necessary, i think the proposal needs to drop to stage 2.7.

Separately, i don’t think requiring more monkeypatching is a good change - specifically, i think that “not requiring monkey patching” is enough of a use case to completely outweigh the concerns referenced in the OP.

[ptomato]

This is so large a change that if it’s going to be necessary, i think the proposal needs to drop to stage 2.7.

If that's what the plenary decides is appropriate, then that would be disappointing but so be it; we wouldn't have proposed this drastic of a change if it wasn't clear that implementation in engines just wasn't going to proceed without it.

That said, I'm not sure I see what purpose would be served by dropping the proposal to stage 2.7. Stage 3 means the proposal is recommended for implementation. IMO removing this obstacle for engines only strengthens that recommendation, not weakens it. If we make this change and then engines delay implementing it anyway because the proposal is back at stage 2.7, nobody wins.

Separately, i don’t think requiring more monkeypatching is a good change - specifically, i think that “not requiring monkey patching” is enough of a use case to completely outweigh the concerns referenced in the OP.

I'd like to address this later, maybe even in separate thread, after I dig up more of the discussion history around the use case of polyfilling the time zone database. But just to give a short answer for now, nothing requires any monkeypatching at all! However we do expect 'wanting to use my specific version of time zone info' to be a use case, although rare. Time zone info is inherently global data, so the only way to do this is either by polyfilling (i.e. monkeypatching) or by using user-supplied global data somehow. Way back during stage 2, we explored having a global registry for time zone IDs, but that was (probably rightly) shot down.

[ljharb]

The point of stage 2.7 is to mean that it's time to implement, but not to ship. Continuing normative changes mean it's not ready for stage 3.

In this case, are you saying that implementations are refusing to implement it without this change? The wording in the OP suggests merely that it's a request to explore reductions. Process-wise, which aspects of this feedback couldn't have been possible prior to implementation? The observability of the protocols was part of the reviewed stage 2 proposal.

[bakkot]

Process-wise, which aspects of this feedback couldn't have been possible prior to implementation?

I think it's simply impractical for an implementation to accurately assess how much code they're going to need to implement a proposal of this size without actually going and trying to implement it. That's why we have "implementation" as a separate stage prior to "landed in the spec", and why changes due to feedback from production-grade implementations are expected during stage 3.

[ljharb]

Which of the "motivation" bulletpoints in the OP require actual implementation work to determine? They all seem like purely design considerations (albeit with an implementor mindset applied).

[bakkot]

I can't speak for implementations, but speaking generally, the extent to which complexity and surface area in a proposed design actually translates to complexity and size in implementations can be hard to judge prior to implementing.

[anba]

For example it's difficult to determine how many different CalendarDateAdd functions are needed without having an actual implementation:

• The spec always creates a Temporal.Duration object for CalendarDateAdd.
• Temporal.Duration consist of ten slots which all hold a f64 floating point value.
• Creating a Temporal.Duration GC object is much more expensive when compared to using a plain C struct on the stack, which also has ten f64 floating point values, so implementation will like to avoid creating unnecessary GC objects.
• In most cases a more simple C struct with just four int64 values is actually only needed.
• The spec also always creates a Temporal.PlainDate result object, which is unnecessary when the object will directly consumed

The three possible cases for duration input:

• CalendarDateAdd is called with a Temporal.Duration GC object.
• CalendarDateAdd is called with an on-stack C/C++ struct with ten f64 floating point values.
• CalendarDateAdd is called with an on-stack C/C++ struct with four int64 values.

And two possible cases for the return value:

• Return a Temporal.PlainDate object which is later passed to user-code.
• Return a C struct PlainDate with three int32 fields.

It's difficult for me to identify which cases apply when just looking at the CalendarDateAdd callers in the spec text. Only during implementation I started to realise which GC objects are unobservable.

Without user-defined calendars the CalendarDateAdd implementation can likely be simplified to a single function which takes the C/C++ struct with four int64 values as the input and returns the C/C++ struct with three int32 fields.

This pattern repeats for many other abstract operations.

In addition to that, user-defined calendars and time zones can create invalid/unexpected results, which can easily lead to spec bugs: #2235, #2335, #2536, #2700, #2779, #2783, #2801, #2824, etc. This has led to clutter the spec with extra checks just to make sure no invariants are violated.

[FrankYFTang]

Here is my counterproposal "to reduce the surface area and complexity of the proposal" without removing any functionality of custom Calendar and TimeZone by removing 70 functions from Temporal spec (a 21.875% reduction in number of functions). How many functions this proposal can reduce comparing to my counter proposal?

[khawarizmus]

It would be fantastic if the team looks at ways to reduce the size without having to affect these features.

And i tend to agree with @ljharb on this:

Separately, i don’t think requiring more monkeypatching is a good change - specifically, i think that “not requiring monkey patching” is enough of a use case to completely outweigh the concerns referenced in the OP.

Also the question remains open

In this case, are you saying that implementations are refusing to implement it without this change?