[REQ][Python] ApiAttributeError when converting a json response that constains field not listed in the openapi schema

[ztd0ng]

Is your feature request related to a problem? Please describe.

During development, we noticed a problem related to the python generator.
It would raise "ApiAttributeError" when converting a json response that constains field not listed in the openapi schema the python package is derived from.

For example, consider the following schema:

{
   "openapi":"3.0.1",
   "info":{ },
   "servers":[ ],
   "paths":{
      "/api/Images/{id}/MetaData":{
         "get":{
            "tags":[
               "Images"
            ],
            "summary":"Looks for the whole slide image with given id and extracts corresponding meta data.",
            "operationId":"RetrieveMetaDataForId",
            "parameters":[
               {
                  "name":"id",
                  "in":"path",
                  "description":"The unique identifier of the whole slide image we want to extract metadata from.",
                  "required":true,
                  "schema":{
                     "type":"string",
                     "format":"uuid"
                  }
               }
            ],
            "responses":{
               "200":{
                  "description":"Success",
                  "content":{
                     "application/json":{
                        "schema":{
                           "$ref":"#/components/schemas/ImgMetaDataDtoApiResponse"
                        }
                     }
                  }
               }
            }
         }
      },
      "components":{
         "schemas":{
            "ImgMetaDataDto":{
               "type":"object",
               "properties":{
                  "name":{
                     "type":"string",
                     "description":"The filename of the whole slide image metadata file.",
                     "nullable":true
                  },
                  "id":{
                     "type":"string",
                     "description":"The unique id of the corresponding slide image could be used as correlation id.",
                     "format":"uuid"
                  }
               },
               "additionalProperties":false,
               "description":"Contains metadata that describes the outcome of one image."
            },
            "ImgMetaDataDtoApiResponse":{
               "type":"object",
               "properties":{
                  "apiMeta":{
                     "type":"array",
                     "items":{
                        "$ref":"#/components/schemas/IApiMeta"
                     },
                     "description":"List of items describing what happened on the backend side.",
                     "nullable":true
                  },
                  "data":{
                     "$ref":"#/components/schemas/ImgMetaDataDto"
                  }
               },
               "additionalProperties":false,
               "description":"Response contains a single object."
            },
            "IApiMeta":"..."
         },
         "securitySchemes":{ ... }
      },
      "security":[ ... ]
   }
}

And during development, another property "version" is added to "ImgMetaDataDto" so it becomes:

"ImgMetaDataDto":{
      "type":"object",
      "properties":{
         "version":{
            "type":"string",
            "description":"The version of the whole slide image metadata.",
            "nullable":true
         },
         "name":{
            "type":"string",
            "description":"The filename of the whole slide image metadata file.",
            "nullable":true
         },
         "id":{
            "type":"string",
            "description":"The unique id of the corresponding slide image could be used as correlation id.",
            "format":"uuid"
         }
      },
      "additionalProperties":false,
      "description":"Contains metadata that describes the outcome of one image."
   }

This addition is not meant to be a breaking change. But the python client generated on the original schema would fail on "ApiAttributeError" because the backend now has an additional field "version" in the response.

Describe the solution you'd like

The failure should not happen.
Regenerating python client for these property additions are unnecessary.
There is no way anyone can "predict" what new peoperty would be added ahead of time to utilize "additional_property".

Describe alternatives you've considered

Currently, we implement a little "hack" in the model_utils.py so the OpenApiModel set_attribute would look like this:

class OpenApiModel(object):
    """The base class for all OpenAPIModels"""

    def set_attribute(self, name, value):
        # this is only used to set properties on self
        ...
        if name in self.openapi_types:
            required_types_mixed = self.openapi_types[name]
        elif self.additional_properties_type is None:
            return
        elif self.additional_properties_type is not None:
            required_types_mixed = self.additional_properties_type
        ...

insted of:

class OpenApiModel(object):
    """The base class for all OpenAPIModels"""

    def set_attribute(self, name, value):
        # this is only used to set properties on self
        ...
        if name in self.openapi_types:
            required_types_mixed = self.openapi_types[name]
        elif self.additional_properties_type is None:
            raise ApiAttributeError(
                "{0} has no attribute '{1}'".format(
                    type(self).__name__, name),
                path_to_item
            )
        elif self.additional_properties_type is not None:
            required_types_mixed = self.additional_properties_type
        ...

Nevertheless, we recommand using a global config bool flag such as serializationSkipPropertyifNotInAttr, so that the generated client would not fail for not having exact types as described in the schema during serialization.

Additional context

[spacether]

If the additional properties are not defined in the source schema and your source schema defines additionalProperties as false, then that schema is saying any additional properties are not allowed which is why the exception happens.
This is a feature of validating the payloads to json schema, not a bug.

Your general generation options on fixing this are:

• regenerate your client with the new properties added to the schema
• regenerate your client with additionalProperties set to True or additionalProperties: false removed from that schema

[ztd0ng]

Thanks a lot for the comment and suggustion.

I noticed that other issues raised the problem of setting the additional property tags.
domaindrivendev/Swashbuckle.AspNetCore#1502

Probably it is agianst the design intention. But if there is the possibiity to ignore the additional properties on the generator side, that would be helpful.

[spacether]

it is agianst the design intention

Yes your suggestion is against the intentions of json schema.

[spacether]

Closing this issue because its root cause is the openapi spec. If different behavior is needed, one can update the json schema constraints that you use to allow in these unused additional properties.
If you want to keep only a subset of those properties, you could take the deserialized instance and its class, and only select the keys that exist in the properties of that class.

# example from the v6.2.0 python generator

inst = SomeClass(**kwargs)
desired_keys = set(SomeClass.MetaOapg.properties.__annotations__)
dict_with_only_desired_keys = {k: inst[k] for k in desired_keys}