Set Desired Property with DeviceTwinDevice to null doesn't work

[MaximilianKarl]

Hello,

I got an issue when I try to delete a property from the DeviceTwin. When I set the value to null, then the value will not be deleted.

Setup:

Windows 8
Java Runtime: 1.8

 <dependency>
           <groupId>com.microsoft.azure.sdk.iot</groupId>
           <artifactId>iot-service-client</artifactId>
           <version>1.13.0</version>
       </dependency>
       <dependency>
           <groupId>com.google.code.gson</groupId>
           <artifactId>gson</artifactId>
           <version>2.8.2</version>
       </dependency>

Code:

        DeviceTwin twinClient = DeviceTwin.createFromConnectionString(connString);
        DeviceTwinDevice device = new DeviceTwinDevice(deviceId);

        twinClient.getTwin(device);
        System.out.println("Started with Device:");
        System.out.println(device);

        JsonObject mainObj = new JsonObject();
        mainObj.addProperty("test","set");
        mainObj.addProperty("test2","set2");
        System.out.println(mainObj);

        HashSet<Pair> tags = new HashSet<Pair>();
        tags.add(new Pair("eConfig", mainObj));
        device.setDesiredProperties(tags);
        twinClient.updateTwin(device);
        twinClient.getTwin(device);
        System.out.println(device);

        JsonObject mainObj2 = new JsonObject();
        mainObj2.addProperty("test","nope");
        mainObj2.add("test2",null);
        System.out.println(mainObj2);

        HashSet<Pair> tags2 = new HashSet<Pair>();
        tags2.add(new Pair("eConfig", mainObj2));
        device.setDesiredProperties(tags2);
        twinClient.updateTwin(device);
        twinClient.getTwin(device);
        System.out.println(device);

Console:

Started with Device:
Device ID: TestDevice
ETag: AAAAAAAAABE=
Tags:{}
Reported Properties{}
Desired Properties: {"eConfig":{"test2":"set2","test":"nope"},"$version":17,"$metadata":{"$lastUpdated":"2018-04-05T13:37:40.68Z","$lastUpdatedVersion":17,"eConfig":{"$lastUpdated":"2018-04-05T13:37:40.68Z","$lastUpdatedVersion":17}}}

{"test":"set","test2":"set2"}
Device ID: TestDevice
ETag: AAAAAAAAABI=
Tags:{}
Reported Properties{}
Desired Properties: {"eConfig":{"test2":"set2","test":"set"},"$version":18,"$metadata":{"$lastUpdated":"2018-04-05T13:38:15.177Z","$lastUpdatedVersion":18,"eConfig":{"$lastUpdated":"2018-04-05T13:38:15.177Z","$lastUpdatedVersion":18}}}

{"test":"nope","test2":null}
Device ID: TestDevice
ETag: AAAAAAAAABM=
Tags:{}
Reported Properties{}
Desired Properties: {"eConfig":{"test2":"set2","test":"nope"},"$version":19,"$metadata":{"$lastUpdated":"2018-04-05T13:38:15.240Z","$lastUpdatedVersion":19,"eConfig":{"$lastUpdated":"2018-04-05T13:38:15.240Z","$lastUpdatedVersion":19}}}

Process finished with exit code 0

At the last print there should be no test2 key and value anymore
Even :
tags.add(new Pair("eConfig", null));

doesn't do something

Greetings

[MaximilianKarl]

iot-service-client version 1.11.0 works everything well!

[timtay-microsoft]

I take it that this doesn't work in Service client 1.12.0 as well?

[MaximilianKarl]

Yes 1.12.0 doesn't work as well.

Debugging 1.13.0 :
I think the problem is in the DeviceTwin.class.

public synchronized void updateTwin(DeviceTwinDevice device) throws IotHubException, IOException {

Line 68 where the JsonElement should be serialized.
String twinJson = twinState.toJsonElement().toString();

the twinState in my Example is:
{"tags":{},"properties":{"desired":{"eConfig":{"test":"nope","test2":null}}}}

and converted to String twinJson:
{"tags":{},"properties":{"desired":{"eConfig":{"test":"nope"}}}}

The null value is missing. :/

The function toJsonElement() in

com.microsoft.azure.sdk.iot.deps.twin.TwinState;

    public JsonElement toJsonElement()
    {
        /* SRS_TWIN_STATE_21_002: [The toJsonElement shall return a JsonElement with the information in this class in a JSON format.] */
        /* SRS_TWIN_STATE_21_003: [If the tags is null, the toJsonElement shall not include the `tags` in the final JSON.] */
        /* SRS_TWIN_STATE_21_004: [If the property is null, the toJsonElement shall not include the `properties` in the final JSON.] */
        Gson gson = new GsonBuilder().disableHtmlEscaping().create();
        return gson.toJsonTree(this).getAsJsonObject();
    }

Null properties will be just ignored.

In Version 1.11.0 the final string contains the null property:
twinJson = "{"deviceId":"TestDevice","etag":"AAAAAAAAADo=","version":59,"tags":{},"properties":{"desired":{"eConfig":{"test":"nope","test2":null}},"reported":{}}}"
I hope this helps you!

[timtay-microsoft]

Looks like you are spot on. Thanks for the help! I'm working to get this committed into master soon. It will definitely make the next release we do. I'll update this thread when you can update to get the fix.

[timtay-microsoft]

This fix is scheduled to be released tomorrow. Please give it a try then and let me know if it works for you!

[MaximilianKarl]

Sure I will test it on Monday and give you a feedback.

[timtay-microsoft]

The fix for this issue has been released as of Device Client version 1.11.0. Feel free to re-open this issue if you still see this bug.

@karlchen7 thank you for your contribution to our open-sourced project! Please help us improve by filling out this 2-minute customer satisfaction survey.

[MaximilianKarl]

Thanks this is now working!