Incorrect error raised when validation fails inside child object and unevaluatedProperties: false is applied to the parent object

[jsolbrig]

The Problem

If a schema sets unevaluatedProperties: false at the top level and contains an object, errors that occur while validating the properties of the inner object are incorrectly reported. If any error occurs within the object, that object is reported as "unevaluated". This obscures the real error.

Examples

Incorrect type results in unevaluated parent object

For example, given the following schema:

{
  "type": "object",
  "unevaluatedProperties": false,
  "properties": {
    "foo": {
      "type": "object",
      "properties": {
        "bar": {
          "type": "number"
        }
      }
    }
  }
}

The following will validate correctly:

{
  "foo": {
    "bar": 1
  }
}

However, the following will raise ValidationError: Unevaluated properties are not allowed ('foo' was unexpected) when ValidationError: 'string-should-be-number' is not of type 'number' would be expected.

{
  "foo": {
    "bar": "string-should-be-number"
  }
}

Missing required property results in unevaluated parent object

The same kind of problem occurs if a required property is missing. For example using this schema:

{
  "type": "object",
  "properties": {
    "foo": {
      "type": "object",
      "required": [
        "bar"
      ],
      "properties": {
        "bar": {
          "type": "number"
        }
      }
    }
  }
}

to validate this json:

{
  "foo": {}
}

should raise ValidationError: 'bar' is a required property but actually raises ValidationError: Unevaluated properties are not allowed ('foo' was unexpected).

Unevaluated property inside object results in unevaluated parent object

This is even true when the error is violation of unevaluatedProperties: false on the inner object. So, given:

{
  "type": "object",
  "unevaluatedProperties": false,
  "properties": {
    "foo": {
      "type": "object",
      "required": [
        "bar"
      ],
      "unevaluatedProperties": false,
      "properties": {
        "bar": {
          "type": "number"
        }
      }
    }
  }
}

to validate this:

{
  "foo": {
    "bar": 1,
    "meh": 2
  }
}

The expected error would indicate that meh is an unevaluated property. Instead, the error falls to indicating that foo is unevaluated.

Expected: ValidationError: Unevaluated properties are not allowed ('meh' was unexpected)

Actual: ValidationError: Unevaluated properties are not allowed ('foo' was unexpected)

What I've found

It appears that, when an error occurs inside of an object, that object does not get reported as having been evaluated when find_evaluated_property_keys_by_schema is called. The unevaluated properties error appears to take precedence over any other error that occurred inside the object.

[Julian]

I haven't really stared at your examples carefully, but this would seem already fixed in the prerelease, as both give what you listed as your expected result. Feel free to have a look?

[jsolbrig]

@Julian Yep, I guess I should have checked out the most recent prerelease before writing this issue. You're right that the prerelease fixes my problems. I'll close this issue. Thanks for the help!

Do you know when the next full release will be made?

[Julian]

There's at least one issue still blocking a beta release (and then a final one) -- that's #1061, so you could keep an eye on that one, I'm hoping to get it done in the next week but other things keep cropping up.

[jsolbrig]

Great, thanks @Julian. Also, thanks for the great package! I was able to pick it up and start using it very easily.

[Julian]

Very glad to hear it!