Change default textual serialization of java.util.Date/Calendar to include colon in timezone offset

[cowtowncoder]

(for background, see #1624 and #1744)

So. While older JDK versions (up to JDK 7) always wrote timezone offset, if any, without colon, like:

+0000

ISO-8601 specification expects minute/hour part to be separated by a colon like

+00:00

While #1744 added an option to enable this behavior, it has not been default for Jackson.
Let's change it in 2.11.

A note on implementation: a new SerializationFeature setting was briefly considered, but wiring of such setting to be used proved difficult. Users can still use method in StdDateFormat

[michael-o]

That's not competely true. ISO 8601 provides two formats, basic and extended. So both outputs are valid as long as the rest of the timestamp adheres to the same format. Since Jackson uses extended format exclusively, the current behavior is wrong.

[cowtowncoder]

@michael-o Ok. I think this was mentioned in an earlier discussion, but was not sure how to describe it. So, two questions:

1. Do you agree with the change itself, going forward? (I assume yes, just want to make 100% certain)
2. Could you suggest an improved wording to use here (feel free to edit if you can; if not let me know)

Also related, for overall change plans wrt Date/Time type handling:

https://github.com/FasterXML/jackson-future-ideas/wiki/JSTEP-5

where it'd be good to first group distinct ideas of improvements needed, in detail.

[michael-o]

1. Yes, I do agree.
2. Do you refer to 4b51149? If yes, it definitvely needs improvement. Because Z has ben designed for RFC dates and not for ISO dates like telescoping X.

[cowtowncoder]

@michael-o Meant specifically wrt Javadoc for SerializationFeature.WRITE_DATES_AS_TIMESTAMPS primarily, and perhaps this issue title secondarily.

[michael-o]

New title: Change default textual serialization of java.util.Date/Calendar for the timezone offset to comply with ISO 8601 extended format.

I will review the Javadoc of SerializationFeature.WRITE_DATES_AS_TIMESTAMPS.

[michael-o]

See this part:

https://github.com/FasterXML/jackson-databind/blob/master/src/main/java/com/fasterxml/jackson/databind/util/StdDateFormat.java#L18-L22

This statement is problematic, even wrong. X cannot be used for RFC and ISO. The subtile difference is that is will always serialize Zulu with Z, never +0000 or 00:00.

I'd say that the Javadoc of WRITE_DATES_AS_TIMESTAMPS is acceptable. StdDateFormat needs imrovements for 3.0. It is a bit of a mess with respect to ISO 8601.

[cowtowncoder]

@michael-o I want to make sure I understand this part:

X cannot be used for RFC and ISO. The subtile difference is that is will always serialize Zulu with Z, never +0000 or 00:00.

As far as I can read JDK javadocs, X does produce Z for zero timezone offset; one that is legal (as well as alternate +00[:00]) for ISO-8601 as well. So I assume those are not problem parts.

So I assume you refer to the current (2.11) implementation of StdDateFormat, which does (for now) produce only numeric offset. This is easy to change, as comments already indicate (although claiming older specs being problematic, I'll change comment) that behavior was chosen for backwards compatibility.
Since changing that part (+00:00 -> Z) is yet another theoretically tiny, but in practice highly visible -- most likely causing tons of bogus test failures, but also breaking some amount of fragile code -- it seems best to only change colon inclusion for 2.11, and do +00:00 -> Z for 3.0 now. We can consider switch in 2.12 as well, depending on how 2.11 release goes.

Does above make sense?

[michael-o]

Yes, retain 00:00 for the 2.x line and move fully to XXX in 3.0. I wouldn't really touch that in 2.x as it cause too many confusion with broken stuff out there. Don't make the hodge-podge even worse for now.

[jocull]

This isn't clearly documented at all, but after digging through the source code I figured out what you need to do to go back to the old behavior:

final ObjectMapper mapper = JsonMapper.builder()
        .defaultDateFormat(new StdDateFormat().withColonInTimeZone(false)) // <<<<
        .build();

Hope that helps someone else. It's really difficult to get to these everywhere in your code (and shared libraries!) that it might exist. It also means that doing dependency alignment for things like security reasons (e.g. https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-28491 ) is extremely difficult.

[cowtowncoder]

Thank you for sharing this @jocull. Configurability of date/time handling is quite problematic, alas, both because of variety of differences across platforms and because Java has a few alternatives ("old" JDK, newer Java 8, Joda). And date/time handling is a huge swamp in and of itself, as a domain.

This is known as a problem domain so there have been plans:

https://github.com/FasterXML/jackson-future-ideas/wiki/JSTEP-5

to try to address settings, defaults, configurability. Currently no one has time to tackle this one, unfortunately.