msrest 0.4.12 Serialization change

This page to describe all the new features related to serialization in msrest 0.4.12. This is already tested with Autorest / SDK / CLI and is fully backward compatible.

Improved automatic Model parsing

Given a Model like this one:

        class TestKeyTypeObj(Model):

            _validation = {}
            _attribute_map = {
                'attr_d': {'key':'properties.KeyD', 'type': 'int'},
            }

Assuming the create_or_update operation takes this object as parameter, all these syntaxes are equivalent:

client.operations.create_or_update({'attr_d': 42})
client.operations.create_or_update({'ATTR_D': 42})
client.operations.create_or_update({'keyd': 42})
client.operations.create_or_update({'KEYD': 42})
client.operations.create_or_update({'properties': {'keyd': 42}})
client.operations.create_or_update({'PROPERTIES': {'KEYD': 42}})

But since "explicit is better than implicit", the recommended way is now:

client.operations.create_or_update(TestKeyTypeObj.from_dict({'attr_d': 42}))

Note that the two implementations have been unified, so this is exactly the same parsing. There is no advantage to use from_dict or direct dict.

Validation

All models have now a validate method that returns a list with the validation that fails:

From this class:

        class TestObj(Model):

            _validation = {
                'name': {'min_length': 3},
            }                
            _attribute_map = {
                'name': {'key':'RestName', 'type':'str'},
            }
            
            def __init__(self, name):
                self.name = name

We can call validate directly:

In [5]: obj = TestObj("ab")

In [7]: obj.validate()
Out[7]: [msrest.exceptions.ValidationError("Parameter 'TestObj.name' must have length greater than 3.")]

This will recursively validate the entire model, and return the complete list.

Serialization

All model now have two new methods: serialize and as_dict:

In [11]: obj.serialize()
Out[11]: {'RestName': 'ab'}

In [12]: obj.as_dict()
Out[12]: {'name': 'ab'}

serialize will return the JSON for the Azure RestAPI. Which means this also trim readonly values (but there is a keep_readonly parameter). as_dict can be configured to:

• Change the key used using a callback that receive attribute name, attribute meta and value. A list can be used to imply hierarchy. Value can be changed as well to tweak serialization.
• Keep or not the read only values

Examples:

In [13]: obj.as_dict(key_transformer=lambda attr, attr_desc, value: ("prefix_"+attr, value))
Out[13]: {'prefix_name': 'ab'}

In [15]: obj.as_dict(key_transformer=lambda attr, attr_desc, value: (["prefix", attr], value))
Out[15]: {'prefix': {'name': 'ab'}}

Three callbacks are available by default:

• attribute_transformer : just use the attribute name
• full_restapi_key_transformer : use RestAPI complete syntax and hierarchy (like serialize)
• last_restapi_key_transformer : use RestAPI syntax, but not hierarchy (flatten object, but RestAPI case, close to CLI to_dict)

These transformers can be used to change on the fly the value if necessary:

        testobj = self.TestObj()
        testobj.attr_a = "myid"
        testobj.attr_b = 42
        testobj.attr_c = True
        testobj.attr_d = [1,2,3]
        testobj.attr_e = {"pi": 3.14}
        testobj.attr_f = timedelta(1)
        testobj.attr_g = "RecursiveObject"

        def value_override(attr, attr_desc, value):
            key, value = last_restapi_key_transformer(attr, attr_desc, value)
            if key == "AttrB":
                value += 1
            return key, value

        jsonable = json.dumps(testobj.as_dict(key_transformer=value_override))
        expected = {
            "id": "myid",
            "AttrB": 43,
            "Key_C": True,
            "AttrD": [1,2,3],
            "AttrE": {"pi": 3.14},
            "AttrF": "P1D",
            "AttrG": "RecursiveObject"
        }
        self.assertDictEqual(expected, json.loads(jsonable))

Deserialization

All model now have two new class methods: deserialize and from_dict:

In [19]: a = TestObj.deserialize({'RestName': 'ab'}) ; print(type(a), a)
<class '__main__.TestObj'> {'name': 'ab'}

In [20]: a = TestObj.from_dict({'name': 'ab'}) ; print(type(a), a)
<class '__main__.TestObj'> {'name': 'ab'}

from_dict takes a key extraction callback list. By default, this is the case insensitive RestAPI extractor, the case insensitive attribute extractor and the case insensitive last part of RestAPI key extractor.

This can be used to tweak the deserialisation process if necessary:

        # Scheduler returns duration as "00:00:10", which is not ISO8601 valid
        class TestDurationObj(Model):
            _attribute_map = {
                'attr_a': {'key':'attr_a', 'type':'duration'},
            }

        with self.assertRaises(DeserializationError):
            obj = TestDurationObj.from_dict({
                "attr_a": "00:00:10"
            })

        def duration_rest_key_extractor(attr, attr_desc, data):
            value = rest_key_extractor(attr, attr_desc, data)
            if attr == "attr_a":
                # Will return PT10S, which is valid ISO8601
                return "PT"+value[-2:]+"S"

        obj = TestDurationObj.from_dict(
            {"attr_a": "00:00:10"},
            key_extractors=[duration_rest_key_extractor]
        )
        self.assertEqual(timedelta(seconds=10), obj.attr_a)

Misc

Roundtrip

• serialize / deserialize : No roundtrip in most cases, since serialize removes the read-only attributes. But you can override it if necessary:

In [7]: a = TestObj.deserialize(TestObj('ab').serialize(keep_readonly=True)) ; print(type(a), a)
<class '__main__.TestObj'> {'name': 'ab'}

• from_dict / to_dict : Should support roundtrip, or it's a bug

In [6]: TestObj.from_dict({'name': 'ab'}).as_dict()
Out[6]: {'name': 'ab'}

In [7]: a = TestObj.from_dict(TestObj('ab').as_dict()) ; print(type(a), a)
<class '__main__.TestObj'> {'name': 'ab'}