[QUERY] Failed to de-serialize JSON on date 2020-02-18T10:14:43.06

[weidongxu-microsoft]

Query/Question

De-serialize JSON failed on incorrect date e.g. 2020-02-18T10:14:43.06. (lack Z at last char)

Error message

com.fasterxml.jackson.databind.exc.InvalidFormatException: Cannot deserialize value of type `java.time.OffsetDateTime` from String "2020-02-18T10:14:43.06": Failed to deserialize java.time.OffsetDateTime: (java.time.format.DateTimeParseException) Text '2020-02-18T10:14:43.06' could not be parsed at index 22

This probably is web service problem, but I don't think server will fix.

It happens for multiple mgmt services (include web, sql, etc.).

Let me know whether azure core would adapt for this.

---

Sample request

PUT https://management.azure.com/subscriptions/###/resourcegroups/javacsmrga8d720064/Microsoft.Web/sites/java-webapp-484058?api-version=2019-08-01

Sample response

{"id":"/subscriptions/ec0aa5f7-9e78-40c9-85cd-535c6305b380/resourceGroups/javacsmrga8d720064/providers/Microsoft.Web/sites/java-webapp-484058","name":"java-webapp-484058","type":"Microsoft.Web/sites","kind":"app","location":"westus","tags":{},"properties":{"name":"java-webapp-484058","state":"Running","hostNames":["java-webapp-484058.azurewebsites.net"],"webSpace":"javacsmrga8d720064-WestUSwebspace","selfLink":"https://waws-prod-bay-115.api.azurewebsites.windows.net:454/subscriptions/ec0aa5f7-9e78-40c9-85cd-535c6305b380/webspaces/javacsmrga8d720064-WestUSwebspace/sites/java-webapp-484058","repositorySiteName":"java-webapp-484058","owner":null,"usageState":"Normal","enabled":true,"adminEnabled":true,"enabledHostNames":["java-webapp-484058.azurewebsites.net","java-webapp-484058.scm.azurewebsites.net"],"siteProperties":{"metadata":null,"properties":[{"name":"LinuxFxVersion","value":""},{"name":"WindowsFxVersion","value":null}],"appSettings":null},"availabilityState":"Normal","sslCertificates":null,"csrs":[],"cers":null,"siteMode":null,"hostNameSslStates":[{"name":"java-webapp-484058.azurewebsites.net","sslState":"Disabled","ipBasedSslResult":null,"virtualIP":null,"thumbprint":null,"toUpdate":null,"toUpdateIpBasedSsl":null,"ipBasedSslState":"NotConfigured","hostType":"Standard"},{"name":"java-webapp-484058.scm.azurewebsites.net","sslState":"Disabled","ipBasedSslResult":null,"virtualIP":null,"thumbprint":null,"toUpdate":null,"toUpdateIpBasedSsl":null,"ipBasedSslState":"NotConfigured","hostType":"Repository"}],"computeMode":null,"serverFarm":null,"serverFarmId":"/subscriptions/ec0aa5f7-9e78-40c9-85cd-535c6305b380/resourceGroups/javacsmrga8d720064/providers/Microsoft.Web/serverfarms/java-asp-c9a952873","reserved":false,"isXenon":false,"hyperV":false,"lastModifiedTimeUtc":"2020-02-18T08:42:27.8166667","storageRecoveryDefaultState":"Running","contentAvailabilityState":"Normal","runtimeAvailabilityState":"Normal","siteConfig":null,"deploymentId":"java-webapp-484058","trafficManagerHostNames":null,"sku":"Basic","scmSiteAlsoStopped":false,"targetSwapSlot":null,"hostingEnvironment":null,"hostingEnvironmentProfile":null,"clientAffinityEnabled":true,"clientCertEnabled":false,"clientCertExclusionPaths":null,"hostNamesDisabled":false,"domainVerificationIdentifiers":null,"customDomainVerificationId":"2D9D2C35841FDE47EE3955E78F943C99EF6D4FB99D295897A5BC167F8837561D","kind":"app","inboundIpAddress":"40.112.192.69","possibleInboundIpAddresses":"40.112.192.69","outboundIpAddresses":"40.112.192.69,40.112.196.57,40.112.198.128,40.78.3.182,40.78.2.143","possibleOutboundIpAddresses":"40.112.192.69,40.112.196.57,40.112.198.128,40.78.3.182,40.78.2.143,40.112.193.168,40.78.1.240,40.78.1.3","containerSize":0,"dailyMemoryTimeQuota":0,"suspendedTill":null,"siteDisabledReason":0,"functionExecutionUnitsCache":null,"maxNumberOfWorkers":null,"homeStamp":"waws-prod-bay-115","cloningInfo":null,"hostingEnvironmentId":null,"tags":{},"resourceGroup":"javacsmrga8d720064","defaultHostName":"java-webapp-484058.azurewebsites.net","slotSwapStatus":null,"httpsOnly":false,"redundancyMode":"None","inProgressOperationId":null,"geoDistributions":null,"privateEndpointConnections":null,"buildVersion":null,"targetBuildVersion":null}}

Why is this not a Bug or a feature Request?
A clear explanation of why is this not a bug or a feature request?

More likely a service bug. I need to know whether core would adapt it.

Setup (please complete the following information if applicable):

• OS: Windows
• IDE : IntelliJ
• Version of the Library used: core 1.2.0

Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

• Query Added
• Setup information Added

[joshfree]

/cc @jianghaolu

[weidongxu-microsoft]

The difference with track1 clientruntime-for-java is that track1 uses JodaModule, while track2 uses JavaTimeModule (well, of course).

Code to reproduce:

        String time = "{\"lastModifiedTimeUtc\":\"2020-02-18T08:42:27.8166667\"}";
        SerializerAdapter serializerAdapter = new JacksonAdapter();
        Map<String, OffsetDateTime> timeObj = new HashMap<>();
        timeObj = serializerAdapter.deserialize(time, new TypeReference<Map<String, OffsetDateTime>>() {}.getType(), SerializerEncoding.JSON);

[weidongxu-microsoft]

LocalDateTime works for both, but I don't think service means local time.

[weidongxu-microsoft]

I kind of able to hack it like this. But this is hard to be done in user space to JacksonAdapter.

    static class OffsetDeserializer extends InstantDeserializer<OffsetDateTime> {

        public OffsetDeserializer() {
            super(OffsetDateTime.class, DateTimeFormatter.ISO_DATE_TIME, (temporal) -> {
                        if (temporal.query(TemporalQueries.offset()) == null) {
                            return LocalDateTime.from(temporal).atOffset(ZoneOffset.UTC);
                        } else {
                            return OffsetDateTime.from(temporal);
                        }
                    },
                    (a) -> OffsetDateTime.ofInstant(Instant.ofEpochMilli(a.value), a.zoneId),
                    (a) -> OffsetDateTime.ofInstant(Instant.ofEpochSecond(a.integer, (long)a.fraction), a.zoneId),
                    (d, z) -> d.withOffsetSameInstant(z.getRules().getOffset(d.toLocalDateTime())), true);
        }
    }

        ObjectMapper mapper = new ObjectMapper();
        JavaTimeModule module = new JavaTimeModule();
        module.addDeserializer(OffsetDateTime.class, new OffsetDeserializer());
        mapper.registerModule(module);

[weidongxu-microsoft]

I will try unblock myself by this

    static class DateTimeDeserializer extends JsonDeserializer<OffsetDateTime> {

        public static SimpleModule getModule() {
            SimpleModule module = new SimpleModule();
            module.addDeserializer(OffsetDateTime.class, new DateTimeDeserializer());
            return module;
        }

        @Override
        public OffsetDateTime deserialize(JsonParser jsonParser, DeserializationContext deserializationContext) throws IOException, JsonProcessingException {
            String string = jsonParser.getText();
            TemporalAccessor temporal = DateTimeFormatter.ISO_DATE_TIME.parseBest(string, OffsetDateTime::from, LocalDateTime::from);
            if (temporal.query(TemporalQueries.offset()) == null) {
                return LocalDateTime.from(temporal).atOffset(ZoneOffset.UTC);
            } else {
                return OffsetDateTime.from(temporal);
            }
        }
    }

        serializerAdapter.serializer().registerModule(DateTimeDeserializer.getModule());

[alzimmermsft]

Hi @weidongxu-microsoft, thanks for investigating into this, it looks like we could add a custom deserializer to handle ISO 8601 better (it appears we already have a custom serializer for it). There remains an open question on the proper way to handle an ISO 8601 date time without a time zone indicator, I feel that leaving it local may be the safest but it may be a service-by-service basis.

Any chance we could follow-up with the service owners to determine what the behavior should be in this scenario?

[weidongxu-microsoft]

@alzimmermsft @jianghaolu

Agree. Normally it should be LocalDateTime. On mgmt. the code is auto-generated, so for same swagger definition we get same code.
And they are same swagger definition... Though in live run, one gives time zone, the other does not.

storage account
https://github.com/Azure/azure-rest-api-specs/blob/master/specification/storage/resource-manager/Microsoft.Storage/stable/2019-06-01/storage.json#L2216

webapp site
https://github.com/Azure/azure-rest-api-specs/blob/master/specification/web/resource-manager/Microsoft.Web/stable/2019-08-01/CommonDefinitions.json#L1746

If data-plane has not encountered this behavior, I could put the customized de-serializer to core-management. That would leave a simpler core.

Then my question is that whether the above DateTimeDeserializer looks OK to you? If not, any advice?

For contacting service, if we encounter same issue on another service, I will contact them. Webapp service is not really keen to response to my query (I've got a much more critically query on authentication still pending).

Meantime you can close the issue. Just let me know your opinion about the above DateTimeDeserializer.

[weidongxu-microsoft]

Reopen it since @JonathanGiles requires us to remove serializer package in core-management. Hence it had to be mitigated in core.

Tasks for management team:
• Remove CloudException type from azure-core-management
• Move CloudError into .exception package
• Supply build instructions
• Move PollerFactory out of implementation package and into polling
• Remove serializer package