Localized date-time formats

[dkhalanskyjb]

This is a discussion about the potential shape of the API dedicated to
locale-aware formats.

Problem statement

The use cases for date-time formatting are:

1. Machine-human communication: showing date-time information to the end user in the form they want. This typically involves locales.
2. Machine-machine communication: sharing information between programs in a way that a human operator can also understand from a glance.

A significant source of mistakes when using the formats are confusing between these two use cases:

• It is incorrect to show a user from Germany a string like May 29, 10:53 AM: the month name is wrong, and the time of day is not in the 24-hour format.
• When using RFC 2822 to communicate between computers, it's a mistake to make the format locale-aware: Fri, 21 Nov 1997 09:55:06 -0600 is a correct RFC 2822 string, but пт., 21 нояб. 1997 09:55:06 -0600 is not.

A good date-time formatting API should make it easy to write correct code for both of the use cases, which means avoiding the specified pitfalls.

Some prior art

C, Python, etc.: strftime

1. Has specific directives %x and %X for "preferred date format" and "preferred time format," respectively. No way to specify "I don't want to output seconds" or "output without the weekday."
2. There is no way to set the month names in stone: they will be taken from the current locale. This makes writing formats with preset month names error-prone.

Searching https://github.com/search?q=strftime+Tue+rfc+822+lang%3AC%2B%2B+&type=code, we can find several mistakes of the second kind and, occasionally, some explicit string manipulation to work around them:
https://github.com/killgcd/chromego/blob/1775962c565688a70cefc1a814a2654212e5942f/ChromeGo/XX-Net/code/default/python27/1.0/lib/rfc822.py#L971

Go

1. There is no localized formatting at all. A separate library, https://github.com/goodsign/monday/, handles this.
   Some predefined locale-aware formats exist but don't seem widely used: https://grep.app/search?q=ShortFormatsByLocale, which invites the issue of 18:55 vs. 06:55 PM for different locales.
2. There is no variance across locales, so all formats are suitable for machines unless someone confuses the standard Go date-time formatting with the one by the monday library.

JavaScript

1. There is localized formatting via the internationalization API: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Intl/DateTimeFormat/DateTimeFormat A good overview is given here: https://stackoverflow.com/questions/3552461/how-do-i-format-a-date-in-javascript
2. It looks like there's no built-in way to format/parse in custom machine-readable formats other than with libraries like Moment.js: https://stackoverflow.com/questions/28002261/parse-date-time-string-in-javascript-which-are-of-custom-format. However, this is not often needed because Date.parse typically supports both ISO 8601 and RFC 2822: https://stackoverflow.com/a/30421037/20886821. It looks like JS programmers just put their strings into Date.parse, and it usually does what they want it to. Its behavior other than parsing ISO 8601 is not guaranteed, though, so maybe this is a considerable footgun: see https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/parse#fall-back_to_implementation-specific_date_formats For formatting though, it looks like the internationalization API can get you pretty far: https://stackoverflow.com/a/63490548/20886821

Java

1. By default, the current system locale is used for formatting, so when run in Germany, the program will display the date with the correct field format.

   • However, the correct format is not always chosen. If the locale specifies that the dates should use a particular chronology, this will get silently ignored by default (from https://docs.oracle.com/en/java/javase/16/docs/api/java.base/java/time/format/DateTimeFormatter.html):

     "The chronology of the result is either the chronology that was parsed, or if no chronology was parsed, it is the chronology set on this class, or if that is null, it is IsoChronology."

     For example, in the locale ja_JP_JP_#u-ca-japanese, despite the calendar being explicitly set to the Japanese one, the chronology used is ISO. To correctly handle this case, one must call withChronology(Chronology.ofLocale(locale)) on the chosen date-time formatter. However, it is not clear whether this is important. There aren't many locales that include a chronology (despite it being possible in theory to mix and match locales and chronologies freely): https://play.kotlinlang.org/#eyJ2ZXJzaW9uIjoiMS43LjIxIiwicGxhdGZvcm0iOiJqYXZhIiwiYXJncyI6IiIsIm5vbmVNYXJrZXJzIjp0cnVlLCJ0aGVtZSI6ImlkZWEiLCJjb2RlIjoiaW1wb3J0IGphdmEudGltZS4qXG5pbXBvcnQgamF2YS50aW1lLmZvcm1hdC4qXG5pbXBvcnQgamF2YS51dGlsLipcblxuZnVuIG1haW4oKSB7XG4gICAgcHJpbnRsbihMb2NhbGUuZ2V0QXZhaWxhYmxlTG9jYWxlcygpLnRvTGlzdCgpLmZpbHRlciB7XG4gICAgICAgIGl0LnRvU3RyaW5nKCkuY29udGFpbnMoXCJjYS1cIilcbiAgICB9KVxufSJ9 Random sampling of Japanese websites didn't yield any of them using the Imperial calendar. It is possible that using the incorrect chronology is not an actual issue.

   • That the field format is correct does not mean that the format as a whole is. For example, if the format specifies the time with AM/PM markers, this does not get automatically corrected. To deal with this, Java provides DateTimePattern.ofLocalizedDateTime, DateTimePattern.ofLocalizedDate, and DateTimePattern.ofLocalizedDate, which return a particular magical formatter than automatically adapts the set and order of fields to the locale. Unfortunately, it's not very flexible, as there are only four variants of the output: "short", "medium", "long", and "full". The current date is formatted with them in the en_US locale as these strings, respectively: 2/28/23, Feb 28, 2023, February 28, 2023, and Tuesday, February 28, 2023. As you can see, there is no option like "I only need the month and the day" or "I need the short weekday name", which leads to people resorting to hardcoded non-localized formats.

2. On the other hand, since, by default, Java uses the current system locale, machine-readable formats need special care: one must explicitly fix the locale used for the format if they use some field names specific to that locale, like so: https://grep.app/search?q=MMM.%2Aocale&regexp=true.
   Examples of discussions of this:
   • https://stackoverflow.com/questions/4216745/java-string-to-date-conversion/4216767#4216767
   • https://stackoverflow.com/questions/44925840/java-time-format-datetimeparseexception-text-could-not-be-parsed-at-index-3/44928297#44928297

Swift, Objective-C

There are two APIs for date/time formatting.

https://developer.apple.com/documentation/foundation/dateformatter allows the usual format strings, but forces eone to set the locale before it can be used. This way, the programmer gets to choose whether they should use the POSIX locale (https://pubs.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap07.html#tag_07_03_05), or some specific locale, or the current system locale.

To deal with the issue of formats themselves being dependent on the locale, there's another API: https://developer.apple.com/documentation/foundation/date/formatstyle

It allows essentially listing the components to format in a builder-like manner. From the linked documentation:

let localeArray = ["en_US", "sv_SE", "en_GB", "th_TH", "fr_BE"]
for localeID in localeArray {
    print(meetingDate.formatted(.dateTime
             .day(.twoDigits)
             .month(.wide)
             .weekday(.short)
             .hour(.conversationalTwoDigits(amPM: .wide))
             .locale(Locale(identifier: localeID))))
}


// Th, November 12, 7 PM
// to 12 november 19
// Th 12 November, 19
// พฤ. 12 พฤศจิกายน 19
// je 12 novembre, 19 h

The order of the fields is not important: as we can see, it gets freely reordered to meet the needs of a particular locale.

Most likely, the API used internally to support this is https://unicode-org.github.io/icu/userguide/format_parse/datetime/#datetimepatterngenerator

These formats also allow parsing them "in a roundtrip manner".

Objective-C supports the same functionality, but with a slightly different API: https://developer.apple.com/documentation/foundation/nsdateformatter See setLocalizedDateFormatFromTemplate, which is essentially the same, but in a pseudo-format-string form.

Proposal

We can separate the APIs for localized and non-localized parsing and formatting.

Non-localized formatting API

The API for non-localized parsing and formatting includes simple format strings that do not allow specifying month names, weekday names, AM/PM markers, or any other locale-aware fields.

These fields are occasionally useful for machine-machine communication as well: for example, the RFC 2822 format uses English month and weekday names, and when parsing preset localized output (for example, logs), there's typically no choice. For cases when the established formats actually require localized strings, support functions like appendMonthName(names: List<String>) in the builder API.

This way, there is the option to deal with any strings life throws at you, but given that all the strings used for the format will be user-supplied, it will be completely clear that they are not localized.

Localized formatting API

We should provide and API like the one Swift and Objective-C have for specifying the set of fields to output. We should make the formats obtained in this manner impossible to use for parsing anything: for now, there are no clear use cases for this, as localized patterns can change with time as the CLDR (https://cldr.unicode.org/index, the source of knowledge typically used to select correct localized formats) gets updated.

One option is for the API to look like this:

LocalDateTimeFormat.fromFieldSetLocalized(
  DateField.FullMonthName(),
  DateField.Day(minDigits = 2),
  TimeField.Hour(),
  TimeField.Minute(),
)

Another way is to do something similar to what Swift does:

LocalDateTimeFormat.buildLocalized {
  addFullMonthName()
  addDay(minDigits = 2)
  addHour()
  addMinute()
}

The second option aligns more with the non-localized format builder API, leading to more uniformity. On the other hand, this uniformity can also be problematic: people may get confused about which builder they should copy-paste the code from the Internet into.

Also, the second option doesn't communicate clearly that the order of the fields will ultimately not matter, whereas the first option has "field set" in its name.

There is also the option to use format strings to define field sets, like Objective-C does, but this seems to be the worst of the two worlds: it's confusing which format strings can go where, and it's unclear that the field order does not matter.

Open questions

• How often do people need to define their own fixed, non-localized formats that also use some localized fields, like RFC-2822?
  • If it's rare and the set of such formats is reasonably bounded, we could fullfill the vast majority of needs by just providing those formats out of the box.
  • If it's often, what could be a nice footgun-free API that would help people keep their format builders reasonably bloat-free? For example, we could provide the set of weekday names [Mon, Tue, Wed, Thu, Fri, Sat, Sun] so that people don't have to redefine it on their own constantly, but just having it somewhere as a constant is completely non-discoverable. Maybe something like appendPosixShortWeekdayName() would be welcome.
• Does anyone want to parse something with a localized format they don't know in advance, like Java's DateTimeFormatter.ofLocalizedDate or Swift's Date.FormatStyle?
• There are various things that one can output in their formats: for example, a localized name of the timezone ("Central European Standard Time"), or characterization of the time of day ("in the evening"), or the era ("BC"). How often does one have to parse strings that contain something like this? Looks like the overwhelming majority of such strings are purely for the user to look at, and machines barely ever have to deal with this: has anyone seen "2023 Feb 28, 10:25 in the evening, AD" in their logs?
• (By @martinbonnin) Should locale be supplied at the moment of creating the format or at the point the format is used? In other words, should each format be specific to a single locale?
• Any other concerns?

[martinbonnin]

Many thanks for opening this 🙏 . As far as I'm concerned, I'm mostly interested in the machine-human communication and the formatting part. For machine-machine, I've relied on the existing Instant, LocalDateTime, etc.. and this served me very well so far.

About the API, I kind of like the Java DateTimeFormatter patterns just because I have a lot of brain muscle around it but maybe this is just resistance to change talking and what I really need is Switft-like APIs because it can handle order better (your option 2 above).

A few comments:

LocalDateTimeFormat.buildLocalized {

I'm not sure what LocalDateTimeFormat is? I would expect an extension on LocalDateTime instead?

// declaration
fun LocalDateTime.buildLocalized(block: (FormatContext) -> Unit)

// usage 
val displayed: String = localDateTime.buildLocalized { /* ... */ }

Maybe even make the locale mandatory?

fun LocalDateTime.buildLocalized(locale: Locale, block: (FormatContext) -> Unit)

the second option doesn't communicate clearly that the order of the fields will ultimately not matter, whereas the first option has "field set" in its name.

I'd be ok with removing the add prefix (that could mislead users to think it's a bit like adding to an ArrayList):

val displayed: String = localDateTime.buildLocalized(FR_BE) { 
   day()
   month()
   // ...
}

Also, swift has an "easy" API with just date and time, sounds like something that could be useful?

localDateTime.formatted(locale = EN_US, date = Complete, time = Omitted) // Sunday, January 17, 2021
localDateTime.formatted(locale = EN_US, date = Omitted, time = Complete) // 4:03:12 p.m. CST
localDateTime.formatted(locale = EN_US, date = Long, time = Shortened) // January 17, 2021, 4:03 PM
// etc...

In all cases, looking forward to this 😃 !

[dkhalanskyjb]

I'm not sure what LocalDateTimeFormat is?

A format that can be used to format LocalDateTime values, like

val myFormat = LocalDateTimeFormat.buildLocalized(locale) {
  // ...
}
localDateTime.format(myFormat)

This way, a format description could be reused instead of always being recreated. If you want most dates in your application to look the same way, it makes sense to define their format in one place.

The API you're suggesting is also interesting, Swift does something like that, but then it wouldn't be called buildLocalized, as build implies object creation. Maybe formatLocalized, or just an overload of format that accepts the locale and the builder.

Maybe even make the locale mandatory?

In the API you're suggesting, yes, the locale should be mandatory: otherwise, it's unclear how to format the localDateTime.

For the API in the document, this is an excellent question, I've added it to the list of questions. We can fix the locale either at the moment we define the format, or at the moment we use it.

• Pro for fixing at the moment of definition: this way, we can optimize the preset format more aggressively.
• Pro for fixing at the moment of use: if the used locale dynamically changes, we can immediately reflect that.

The question is, how often does the locale dynamically change? Are there existing code patterns people use to update the formats throughout the application when it happens?

I'd be ok with removing the add prefix

Interesting suggestion, thanks! The idea was that you add a field to the format, which is essentially a set of fields. So, it is a bit like adding to a Set.

Also, swift has an "easy" API with just date and time, sounds like something that could be useful?

Not sure. Is this useful? When experimenting with various formatting libraries, I personally found such options being more trouble than anything. What does the "full" format of a date include? How is "short" format of a date different from the "medium" one? This was virtually impossible to remember, I had to look up the documentation or re-check constantly. An explicit enumeration of fields is unambiguous and self-evident.

[martinbonnin]

Gotcha 👍 I didn't realize this was building a reusable LocalDateFormat. This makes a lot of sense. With this in mind, I think it makes sense to keep the locale out of it. It doesn't really belong to a "format".

val myFormat = LocalDateTimeFormat.buildLocalized(locale) {
  day()
  month()
}
localDateTime.format(locale, myFormat)

just an overload of format that accepts the locale and the builder

I think I'd use it to avoid breaking the typing Flow. Unless creating the format is costly and we want to encourage reusing them.

Also, swift has an "easy" API with just date and time, sounds like something that could be useful?
Not sure. Is this useful? When experimenting with various formatting libraries, I personally found such options being more trouble than anything.

Fair enough 👍

[hrach]

What we regularly use and is problematic (in our Android world) are custom date-time skeletons. The problematic part is that joda/java time depends on the platform support/implementation of Unicode.

We ended up copying over the patterns for specific skeletons to workaround old Android issues. (E.g. MMM skeleton is not interpreted as M pattern in cs on Android ~ 6+- or Huawei devices)

Also, the second option doesn't communicate clearly that the order of the fields will ultimately not matter, whereas the first option has "field set" in its name.

This is an important aspect of the API, until now, the majority of Java-world APIs (I'd say) prefer passing a pattern instead of a skeleton. Maybe using another prefix could help.

LocalDateTimeFormat.buildLocalized {
  withFullMonthName()
  withDay(minDigits = 2)
  withHour()
  withMinute()
}

Should locale be supplied at the moment of creating the format or at the point the format is used?

I'd ask a bit differently - is userland code expected to cache formatters on its own? Or is the skeleton->pattern resolution operation worth caching? But e.g. in Jetpack Compose world, both approaches can be easily done:

@Composable
fun LocalDate.formatLocalized(): String {
  val locale = LocalContext.current.getLocale()
  val formatter = remember(locale) { LocalDateFormat.buildLocalized(locale) { ... } }
  return format(formatter)
}

// vs

val formatter = LocalDateFormat.buildLocalized { ... } 

@Composable
fun LocalDate.formatLocalized(): String {
  return format(formatter, LocalContext.current.getLocale())
}

---

My questions:

• Is Kotlinx.Datetime going to bundle Unicode specs or rely on the "native" & "fragmented" implementation? (Also, iOS has some issues as well). We would appreciate the predictability here.
• Edit: Will be there a custom type for Locale? Value class for string? I.e. allocation-less conversion would be nice (alternatively it would affect when/where we wanna pass the locale)

[dkhalanskyjb]

What we regularly use and is problematic (in our Android world) are custom date-time skeletons.

Do you mean, the way Android's getBestDateTimePattern works?

But e.g. in Jetpack Compose world, both approaches can be easily done

Jetpack Compose is an important use case, but it's certainly not the only one. I'm sure many people will use the formatting capabilities in scenarios when it's much more tedious to cache values, we have to think about them, too.

Is Kotlinx.Datetime going to bundle Unicode specs or rely on the "native" & "fragmented" implementation?

By the Unicode specs, do you mean the CLDR? I don't think we even have an option but to bundle it, really. As far as I know, there's no way to access the CLDR on Java in general.

iOS has some issues as well

Are these issues also incompatibilities between versions similar to what you've shown?

[hrach]

Do you mean, the way Android's getBestDateTimePattern works?

Our basic use case was a localized date without a year. Not possible to obtain through standard joda/java time API. We tried getBestDateTimePattern() also others (DateTimePatternGenerator if I'm not mistaken) with the same buggy result on those buggy devices.

it's much more tedious to cache values, we have to think about them, too.

True, therefore I was asking if some caching will be on Kotlinx.Datetime side.

By the Unicode specs, do you mean the CLDR?

Yes

iOS has some issues as well

Are these issues also incompatibilities between versions similar to what you've shown?

By this, I meant that it renders some stuff differently compared to Android & CLDR spec. We didn't investigate why (outdated CLDR data or how it works inside).

[dkhalanskyjb]

the same buggy result on those buggy devices.

Why is this behavor buggy, though? The CLDR promises to yield the most natural-looking option for the reader, not one that preserves the fields from the skeleton at all costs. I look at the most recent CLDR data, and there, I also see MMM transforming into M or MM for various locales, including cs. The way I see it, for localized strings, how natural something looks and whether all the required information is included in some form are the only criteria. Did you have a use case that required exactly the month names?

From the https://unicode-org.github.io/icu/userguide/format_parse/datetime/#datetimepatterngenerator:

For example: [...] When using the Japanese calendar in the Japanese locale (locale “ja@calendar=japanese”), the skeleton "yMMMd" will produce the format pattern "Gy年M月d日" since the era must always be shown with the year in the Japanese calendar.

Seems completely intentional.

Do you say the modern Android do it differently, preserving the skeleton fields? If so, I wonder what the reasoning behind this is.

I was asking if some caching will be on Kotlinx.Datetime side.

We have yet to find out! In this discussion, we are asking everyone things like, "hey, is it okay if you have to cache things?" and are gathering answers like, "sure, I need to store the format in a constant anyway, so in a way, it is always cached" or "no way, my typical usage is to use one pattern a hundred times, then change it for another to be used in the same places, please don't make me cache it."

You can be sure though that, whatever the final decisions regarding this are, we'll reflect them in the documentation.

[hrach]

Why is this behavor buggy, though? The CLDR promises to yield the most natural-looking option for the reader, not one that preserves the fields from the skeleton at all costs. I look at the most recent CLDR data, and there, I also see MMM transforming into M or MM for various locales, including cs.

No, I'm wanted to say something different. The transformation is not consistent for the specific locale, moreover, it is not correct, in Czech we don't use Lis. as a shortcut for the month in dates. But older Androids produce that.

[brianguertin]

Is Kotlinx.Datetime going to bundle Unicode specs or rely on the "native" & "fragmented" implementation? (Also, iOS has some issues as well). We would appreciate the predictability here.

I've been doing localized date times in my Kotlin project across Android/iOS/Web, by deferring to the platform's skeleton APIs (or Intl.DateTimeFormat on web), and I found the implementations pretty reliable/consistent in our use case.

At the end of the day, I'm pretty sure Android, iOS, and web browsers are all backed by ICU libraries underneath, they just expose their own API. But I think the underlying ICU library actually brings consistent output across all of them.

tl;dr: I think kotlinx-datetime can defer to the platform instead of trying to ship a copy of ICU.

[OliverO2]

Not saying that they have to be in scope, but for completeness: Have relative formats ("in 20 minutes") been considered?
Cf. systemd.time - Time and date specifications.

For an API where the order of fields does not matter, I'd prefer something like this:

LocalDateTimeFormat.withLocalized {
    fullMonthName()
    day(minDigits = 2)
    hour()
    minute()
}

That means:

• No add prefix on components.
• Non-imperative function naming, replacing the build prefix, which suggests a side effect, with an with prefix, which suggests a returned value, and also suggests that its component are merely present, not necessarily ordered.

Does anyone want to parse something with a localized format they don't know in advance

"Smart" parsing is still a thing as part of the Robustness principle, or Postel's Law: "be conservative in what you send, be liberal in what you accept". I'm not arguing that sloppy machine-machine input should be accepted without warning, but in today's API world, having a lenient fallback for input data can be valuable.

[dkhalanskyjb]

Have relative formats ("in 20 minutes") been considered?

Eventually, yes, but for now, taking it one step at a time is probably better: by publishing first locale-invariant parsing and formatting, then locale-aware formatting, and only then the less often needed things, we will be able to find issues in the API before it becomes too large.

build prefix, which suggests a side effect

build refers to the builder pattern, which typically does not have side effects: https://en.wikipedia.org/wiki/Builder_pattern See https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.text/-string-builder/ for an example.

Other than that, an interesting idea, thanks!

"Smart" parsing is still a thing as part of the Robustness principle

This is unrelated to the exact question: does anyone want this? For example, did you happen to have such a use case? There are many guiding principles, but none apply in every circumstance. If, for the overwhelming majority of people, using such parsing facilities instead of providing a specific format would certainly be a mistake (see the fail fast principle https://en.wikipedia.org/wiki/Fail-fast), but no one would use this intentionally, we'd just be creating a severe pitfall for no reason. On the other hand, if there are such use cases, we could think of a more suitable API by knowing what exactly they are.

[OliverO2]

Have relative formats ("in 20 minutes") been considered?

Eventually, yes, but for now, taking it one step at a time is probably better

I absolutely agree.

build prefix, which suggests a side effect

build refers to the builder pattern, which typically does not have side effects

Yes, I'm aware, and I'm also aware that other builders exist with that naming. However, as also mentioned in the Coding conventions, in general using a verb communicates what a method does (producing a side effect). For methods returning a result, avoiding a verb better communicates intent. And in this case, the method could possibly avoid building something entirely, and instead return a previously cached format, given the fields specified.

"Smart" parsing is still a thing as part of the Robustness principle

This is unrelated to the exact question: does anyone want this?

Yes, actually I'd want something like this for an import API I'll be working on later this year. The application allows typed, user-defined fields, which are stored in a database. Field data can be imported by less technically inclined users via an API (or even an upload dialog). If a user chooses to import date fields scraped from web pages or from spreadsheets, such date values might come in various flavors.

However, that's JVM only and probably a bit too exotic, so I'd be absolutely OK with not having it as part of kotlinx-datetime. I could just use a special library or custom implementation to handle this case.

[dkhalanskyjb]

However, as also mentioned in the Coding conventions, in general using a verb communicates what a method does (producing a side effect).

I don't see any mentions of side effects, and if they were mentioned, I'd say the conventions are misleading: after all, we do have map, filter, fold, and such, which are all verbs, but typically considered to be side-effect-free.

Yes, actually I'd want something like this for an import API I'll be working on later this year

Would love to know the details.

If a user chooses to import date fields scraped from web pages or from spreadsheets, such date values might come in various flavors.

Do I understand correctly that the users push unstructured, non-typed data to you, and you must somehow make sense of it as much as you can and scrape things in various formats? Or do the users process the data and choose what is a date and what isn't? In any case, here's the huge question: if the user sends you the string "03/02/2023" and claims it's a date, is there a right answer as to which conventions to use: day/month/year or month/day/year? Both formats are used in some locales.

[OliverO2]

I don't see any mentions of side effects, and if they were mentioned, I'd say the conventions are misleading: after all, we do have map, filter, fold, and such, which are all verbs, but typically considered to be side-effect-free.

These just don't follow the coding conventions – see KT-43261.

• To quote Roman Elizarov: "I don't see any way out of map, filter, etc. Even though we now know that they would be more consistently named as mapped, filtered, etc. They are too widespread. These exceptions in the naming simply need to be mentioned in the coding style and learned."

If a user chooses to import date fields scraped from web pages or from spreadsheets, such date values might come in various flavors.

Do I understand correctly that the users push unstructured, non-typed data to you, and you must somehow make sense of it as much as you can and scrape things in various formats? Or do the users process the data and choose what is a date and what isn't? In any case, here's the huge question: if the user sends you the string "03/02/2023" and claims it's a date, is there a right answer as to which conventions to use: day/month/year or month/day/year? Both formats are used in some locales.

Users have to define data as a series of typed fields and supply them as fields (so not totally free-form). Ideally, the fields' contents would be well-defined and unambiguous, but in reality users may choose unreliable data sources they don't control. Then the user may decide: OK, even if the data is not 100% accurate, I'll take its lower quality anyway once I judge it to be "just goo enough" for me.

And yes, that means that the above can happen, that the above date strings are interpreted with a wrong order of day and month. I'm not in favor of "garbage in, garbage out", but if users decide otherwise (and may know better about the relevance of the above mistakes), who am I to force them?

And yes, it is a social experiment after all to which degree such an API will actually be used in this way. Hopefully it could help to produce happy users who'd prefer not to have an import of a gazillion items fail (or being incomlpete) because of a small number of data interpretation issues.

Does this sufficiently describe the use case?

[mahozad]

Wanted to repeat what I mentioned in this pull request.

Please consider supporting other calendar systems too (like Solar calendar).
Microsoft Windows operating system (and also Microsoft Office programs) support different calendars when using different locales.

JavaScript as well supports formatting dates based on the calendar of the given locale:

let formatted = new Date().toLocaleDateString('fa');

See https://stackoverflow.com/q/35570884/8583692

Also, please take into account that some locales use different symbols (different Unicode characters) to represent digits and numbers.

For example:

1. English (Latin script) uses Western Arabic digits : 0, 1, 2, 3, 4, 5, 6, 7, 8, 9.
2. ‭Arabic uses Eastern Arabic digits: ٠, ١, ٢, ٣, ٤, ٥, ٦, ٧, ٨, ٩.
3. ‭Persian (also called Farsi) uses another variant of Eastern Arabic digits: ۰, ۱, ۲, ۳, ۴, ۵, ۶, ۷, ۸, ۹.

And, for the 2nd and 3rd example, sometimes users may want to deviate from the standard (which is eastern) and have western digits or to mix and match them when formatting.

Java correctly formats numbers (for example when using NumberFormat.getInstance(locale).format(1234)) with
proper locale digits but there is no way to override the type of digits (number system) (as far as I know).

Relevant:

• https://youtrack.jetbrains.com/issue/KT-58122/StringtoDouble-does-not-parse-Eastern-Arabic-and-Persian-Farsi-valid-numbers
• A bug I filed for Java: https://bugs.java.com/bugdatabase/view_bug?bug_id=8243162
  which was resolved in Java 15: https://www.oracle.com/java/technologies/javase/15-relnote-issues.html#JDK-8244245

[mutarock]

If possible, I think if we can still support java DateTimeFormatter ofPattern with given time format might be better.
I believe some Android world still use joda time which is label as Deprecated and want to move to java time or Kotlin.

Also, we have a use case that locale and format pattern is from server/api calls not local pre-defined, so it is not able to set up formatter with pattern like

LocalDateTimeFormat.withLocalized {
    fullMonthName()
    day(minDigits = 2)
    hour()
    minute()
}

we must getting the formatter like

val localizedSimpleFormat = DateTimeFormatter.ofPattern(formatterStringResponseFromServer).withLocale(getLocale())

Reference:
https://www.joda.org/joda-time/apidocs/org/joda/time/format/DateTimeFormat.html#forPattern-java.lang.String-
https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatter.html#ofPattern-java.lang.String-java.util.Locale-

[mutarock]

Our use case is just showing the information to the user. Once the user access the existing content or if we want to promote any up coming content, we will show the release date to user and localized it. The offset string can be a past/future date based on the content is released or not.

And for client/app side, we show it based on the offset string and format we received from server response.
We will show different format + localized string based on user's status
example:
1 March 2017 vs 2017년 3월 1일

The comma for EN_format is intentional, but it is designed as this before I joined the team which I can not tell why they decide to do so.

[dkhalanskyjb]

Our use case is just showing the information to the user. Once the user access the existing content or if we want to promote any up coming content, we will show the release date to user and localized it. The offset string can be a past/future date based on the content is released or not.
And for client/app side, we show it based on the offset string and format we received from server response.
We will show different format + localized string based on user's status
example:
1 March 2017 vs 2017년 3월 1일

Yes, this is all clear. This is why localized date-time formats are needed. What is not clear is:

1. Imagine that kotlinx-datetime has with things like ES_ES_FORMAT = "d 'de' MMMM 'de' yyyy" built in for each locale. You'd write LocalDate.Format(Locale("es_es")) { year(); monthName(); dayOfMonth() }, and it would output 12 de marzo de 1994. All without any server. Would you still need to use a server then?
2. You mentioned parsing localized dates (converting a string to a date). When do you need to do that?

[mutarock]

1. magine that kotlinx-datetime has with things like ES_ES_FORMAT = "d 'de' MMMM 'de' yyyy" built in for each locale. You'd write LocalDate.Format(Locale("es_es")) { year(); monthName(); dayOfMonth() }, and it would output 12 de marzo de 1994. All without any server. Would you still need to use a server then?

I think If there are all necessary format built-in, we do not really need the server, only if there is any specific format/locale that we want to achieve but not exist in kotlinx-datetime. So I think keeping ofPattern in kotlinx-datetime can provide more flexibility?
Also, we have to make sure there does exist 1 to 1 mapping to all our current format to cover all usage we had now.

2. You mentioned parsing localized dates (converting a string to a date). When do you need to do that?

When user access existing/up coming video content.
Every content has its own release date, so before user plays it, we want to let user know when is the release date of the content on the UI.
Server will provide the release date by ISO offset string, and app/client uses locale/format to parse/show it

[dkhalanskyjb]

Server will provide the release date by ISO offset string, and app/client uses locale/format to parse/show it

So, you only need to parse ISO strings, but output them the result nicely to the user, right? You never need to convert things like 12 de marzo de 1994 back to LocalDate?

[mutarock]

Yes, We just need to parse ISO string and shows the nicely parsing result to the user.

we do not need to convert it back
thanks

[morki]

It seems wierd that we will have locale aware formatting of dates and times, when we have no locale concept in language or standard library or other kotlinx library. We even don't have locale aware number formatting, which for me is the first step.

Other thing is bundling CLDR data with datetime library seems wierd too. What about only including locales that the app is going to use?

From my point of view, we should create something like kotlinx-locale library which:

• will introduce concept of a locale
• will include CLDR data as separatly installable modules for every locale
• will support basic operations like formatting numbers
• will become dependency of this library extension module of this library

What do you think?

[dkhalanskyjb]

We even don't have locale aware number formatting, which for me is the first step.

For datetime formatting, number formatting only comes up when you need to output fractional seconds, so I don't understand why formatting numbers is a prerequisite.

Other thing is bundling CLDR data with datetime library seems wierd too. What about only including locales that the app is going to use?

That's a reasonable request, sure, we'll have to keep in mind the need to exclude unneeded locales (or delegate to the OS if the CLDR data is already available). I don't think we can reasonably provide literally 1000 packages for different locales or their variants, but at least we could provide, for example, instructions for Proguard/R8 or their equivalents.

From my point of view, we should create something like kotlinx-locale library which:

will introduce concept of a locale

Yes, kotlinx-datetime probably shouldn't provide kotlinx.datetime.Locale.

will include CLDR data

No, this doesn't seem feasible. If we look at https://github.com/unicode-org/cldr/blob/main/common/main/en.xml, we see an immense amount of data. If you just want to formats dates and times, you only need a tiny fraction of that. If you want to format the SI units, you need another tiny fraction of that. If you also need to format the region names, again, only a tiny fraction of that. It's hard to think of a program that would use a significant portion of all the data there is for a given locale, so it seems very excessive to say, "I need the CLDR data for en".

So, it makes more sense for domain-specific libraries (like kotlinx-datetime) to bundle only the locale data that they need to provide formatting functionality.

will support basic operations like formatting numbers

The variety of ways to format numbers encountered in date-time formats is just a tiny fraction of ways to format arbitrary numbers in a localized way (please look at https://github.com/unicode-org/cldr/blob/41709835279abbe797de897ec8fbfe86c1e5cb95/common/main/en.xml#L4936-L6640). Stating that supporting formatting of numbers is a prerequisite for this does derail us from the goal quite a bit.

will become dependency of this library extension module of this library

I don't understand the proposed scheme, but the more realistic option seems to be the following:

• Someone (in my view, the stdlib) provides a Locale class, just a wrapper around the locale name;
• The datetime library ships the tiny amount of data needed to format dates and times in different locales, supporting the exclusion of unneeded locales or delegating to the OS where possible.

[morki]

Thank you for your reply, I now understand better your point of view and agree with it.

The only clarification I want to state is the number formatting "prerequisite". I didn't mean it like prerequisite for formating dates, I mean it from application developer point of view as: There is little value in ability to format a localized date in Kotlin multiplatform, if I am unable to format a number (and use the same Locale representation consistently in whole application).

[westnordost]

It seems wierd that we will have locale aware formatting of dates and times, when we have no locale concept in language or standard library or other kotlinx library.

Well, I don't know, Java Locale is essentially just a container data class for what can be expressed by a IETF BCP 47 language tag or Unicode Locale Identifier which AFAIK are the same (e.g. de, en-US, sr-Latn).

So, the API might as well accept a language tag (a string) instead of some data class. That is how I solved the case of a missing Locale data class in the Kotlin standard library for a small library that I recently ported from Java to KMP (see e.g. https://github.com/search?q=repo%3Awestnordost/osmfeatures%20defaultLanguage&type=code)

[westnordost]

In my use case, I need also the localized symbols as-such, e.g. the weekday names, or the short weekday names etc. additionally to normal date formatting.

Additionally, I need them not always in the current system locale, but in a specified locale, e.g. get the weekday names in Polish.
For opening hours, I need to display the weekdays in the local language next to the weekdays in the users's language so that he knows which is which when comparing if the signed opening hours are still correct.

[Khazbs]

Hello! I think it's also important to support platform-specific formatting configurations and native system defaults. For example, Android has android.text.format.DateFormat with factory methods that provide instances of java.text.DateFormat tuned appropriately to the user's system formatting preferences, including not just locale, but also 12 hour vs. 24 hour clock. We need a way to honor these preferences!

[mtrakal]

@Khazbs you can use platform specific formatters. I did it as well and working pretty well.

dateTime.toJavaLocalDateTime() it convert to standard Java8 DateTime which you can format with standard platform formatters.

Simple example is here with my PR to change to platform formatters. It's KMP project, but the same is doable with Android only...

https://github.com/philipplackner/CMP-DateTime/pull/1/files

Description: https://github.com/mtrakal/CMP-DateTime/tree/feature/use-standard-formatters?tab=readme-ov-file#recommendation

Code which you are searching could be:

import kotlinx.datetime.*
import java.time.format.DateTimeFormatter
import java.time.format.FormatStyle

    fun formatLocalDateTime(dateTime: LocalDateTime): String {
        return dateTime.toJavaLocalDateTime().format(
            DateTimeFormatter.ofLocalizedDateTime(FormatStyle.MEDIUM, FormatStyle.MEDIUM)
        )
    }

    fun formatLocalTime(time: LocalTime): String = time.toJavaLocalTime().format(
        DateTimeFormatter.ofLocalizedTime(FormatStyle.MEDIUM)
    )

    fun formatLocalDate(date: LocalDate): String = date.toJavaLocalDate().format(
        DateTimeFormatter.ofLocalizedDate(FormatStyle.MEDIUM)
    )

In non-kmp porjet is better way use extensions for it I think.

[Khazbs]

Yeah, so far I thought about doing it by converting the date to something from Java. However, wouldn't LocalDateTime require Android API level 26+?

[dkhalanskyjb]

I think it's also important to support platform-specific formatting configurations and native system defaults.

This is a great point, one that we haven't considered before, thank you! Yes, we certainly should look into how users specify the preferred formats on various systems.

However, wouldn't LocalDateTime require Android API level 26+?

Not with library desugaring enabled, see https://github.com/Kotlin/kotlinx-datetime?tab=readme-ov-file#using-in-your-projects