Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resolves improper anchoring of patternProperties #783

Merged
merged 2 commits into from
May 23, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
147 changes: 84 additions & 63 deletions doc/compatibility.md
Original file line number Diff line number Diff line change
@@ -1,76 +1,97 @@

### Legend

Symbol | Meaning |
:-----:|---------|
🟢 | Fully implemented
🟡 | Partially implemented
🔴 | Not implemented
🚫 | Not defined in Schema Version.
| Symbol | Meaning |
|:------:|:----------------------|
| 🟢 | Fully implemented |
| 🟡 | Partially implemented |
| 🔴 | Not implemented |
| 🚫 | Not defined |

### Compatibility with JSON Schema versions

Validation Keyword/Schema | Draft 4 | Draft 6 | Draft 7 | Draft 2019-09 |
---------------- |:--------------:|:-------: |:-------: |:-------------:|
$ref | 🟢 | 🟢 | 🟢 | 🟢
additionalProperties | 🟢 | 🟢 | 🟢 | 🟢
additionalItems | 🟢 | 🟢 | 🟢 | 🟢
allOf | 🟢 | 🟢 | 🟢 | 🟢
anyOf | 🟢 | 🟢 | 🟢 | 🟢
const | 🚫 | 🟢 | 🟢 | 🟢
contains | 🚫 | 🟢 | 🟢 | 🟢
contentEncoding | 🚫 | 🚫 | 🔴 | 🔴
contentMediaType | 🚫 | 🚫 | 🔴 | 🔴
dependencies | 🟢 | 🟢 |🟢 | 🟢
enum | 🟢 | 🟢 | 🟢 | 🟢
exclusiveMaximum (boolean) | 🟢 | 🚫 | 🚫 | 🚫
exclusiveMaximum (numeric) | 🚫 | 🟢 | 🟢 | 🟢
exclusiveMinimum (boolean) | 🟢 | 🚫 | 🚫 | 🚫
exclusiveMinimum (numeric) | 🚫 | 🟢 | 🟢 | 🟢
items | 🟢 | 🟢 | 🟢 | 🟢
maximum | 🟢 | 🟢 | 🟢 | 🟢
maxItems | 🟢 | 🟢 | 🟢 | 🟢
maxLength | 🟢 | 🟢 | 🟢 | 🟢
maxProperties | 🟢 | 🟢 | 🟢 | 🟢
minimum | 🟢 | 🟢 | 🟢 | 🟢
minItems | 🟢 | 🟢 | 🟢 | 🟢
minLength | 🟢 | 🟢 | 🟢 | 🟢
minProperties | 🟢 | 🟢 | 🟢 | 🟢
multipleOf | 🟢 | 🟢 | 🟢 | 🟢
not | 🟢 | 🟢 | 🟢 | 🟢
oneOf | 🟢 | 🟢 | 🟢 | 🟢
pattern | 🟢 | 🟢 | 🟢 | 🟢
patternProperties | 🟢 | 🟢 | 🟢 | 🟢
properties | 🟢 | 🟢 | 🟢 | 🟢
propertyNames | 🚫 | 🔴 | 🔴 | 🔴
required | 🟢 | 🟢 | 🟢 | 🟢
type | 🟢 | 🟢 | 🟢 | 🟢
uniqueItems | 🟢 | 🟢 | 🟢 | 🟢
| Keyword | Draft 4 | Draft 6 | Draft 7 | Draft 2019-09 | Draft 2020-12 |
|:---------------------------|:-------:|:-------:|:-------:|:-------------:|:-------------:|
| $anchor | 🚫 | 🚫 | 🚫 | 🔴 | 🔴 |
| $dynamicAnchor | 🚫 | 🚫 | 🚫 | 🚫 | 🔴 |
| $dynamicRef | 🚫 | 🚫 | 🚫 | 🚫 | 🔴 |
| $id | 🟡 | 🟡 | 🟡 | 🟡 | 🟡 |
| $recursiveAnchor | 🚫 | 🚫 | 🚫 | 🔴 | 🔴 |
| $recursiveRef | 🚫 | 🚫 | 🚫 | 🔴 | 🔴 |
| $ref | 🟡 | 🟡 | 🟡 | 🟡 | 🟡 |
| $vocabulary | 🚫 | 🚫 | 🚫 | 🔴 | 🔴 |
| additionalItems | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| additionalProperties | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| allOf | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| anyOf | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| const | 🚫 | 🟢 | 🟢 | 🟢 | 🟢 |
| contains | 🚫 | 🟢 | 🟢 | 🟢 | 🟢 |
| contentEncoding | 🚫 | 🚫 | 🔴 | 🔴 | 🔴 |
| contentMediaType | 🚫 | 🚫 | 🔴 | 🔴 | 🔴 |
| contentSchema | 🚫 | 🚫 | 🚫 | 🔴 | 🔴 |
| definitions | 🟢 | 🟢 | 🟢 | 🚫 | 🚫 |
| defs | 🚫 | 🚫 | 🚫 | 🟢 | 🟢 |
| dependencies | 🟢 | 🟢 | 🟢 | 🚫 | 🚫 |
| dependentRequired | 🚫 | 🚫 | 🚫 | 🟢 | 🟢 |
| dependentSchemas | 🚫 | 🚫 | 🚫 | 🟢 | 🟢 |
| enum | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| exclusiveMaximum (boolean) | 🟢 | 🚫 | 🚫 | 🚫 | 🚫 |
| exclusiveMaximum (numeric) | 🚫 | 🟢 | 🟢 | 🟢 | 🟢 |
| exclusiveMinimum (boolean) | 🟢 | 🚫 | 🚫 | 🚫 | 🚫 |
| exclusiveMinimum (numeric) | 🚫 | 🟢 | 🟢 | 🟢 | 🟢 |
| if-then-else | 🚫 | 🚫 | 🟢 | 🟢 | 🟢 |
| items | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| maxContains | 🚫 | 🚫 | 🚫 | 🟢 | 🟢 |
| minContains | 🚫 | 🚫 | 🚫 | 🟢 | 🟢 |
| maximum | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| maxItems | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| maxLength | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| maxProperties | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| minimum | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| minItems | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| minLength | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| minProperties | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| multipleOf | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| not | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| oneOf | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| pattern | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| patternProperties | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| prefixItems | 🚫 | 🚫 | 🚫 | 🚫 | 🟢 |
| properties | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| propertyNames | 🚫 | 🟢 | 🟢 | 🟢 | 🟢 |
| readOnly | 🚫 | 🚫 | 🔴 | 🔴 | 🔴 |
| required | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| type | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| unevaluatedItems | 🚫 | 🚫 | 🚫 | 🟢 | 🟢 |
| unevaluatedProperties | 🚫 | 🚫 | 🚫 | 🟢 | 🟢 |
| uniqueItems | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| writeOnly | 🚫 | 🚫 | 🔴 | 🔴 | 🔴 |

### Semantic Validation (Format)

Format | Draft 4 | Draft 6 | Draft 7 | Draft 2019-09 |
-------|---------|---------|---------|---------------|
date |🚫 | 🚫 | 🟢 | 🟢
date-time | 🟢 | 🟢 | 🟢 | 🟢
duration | 🚫 | 🚫 | 🔴 | 🔴
email | 🟢 | 🟢 | 🟢 | 🟢
hostname | 🟢 | 🟢 | 🟢 | 🟢
idn-email | 🚫 | 🚫 | 🔴 | 🔴
idn-hostname | 🚫 | 🚫 | 🔴 | 🔴
ipv4 | 🟢 | 🟢 | 🟢 | 🟢
ipv6 | 🟢 | 🟢 | 🟢 | 🟢
iri | 🚫 | 🚫 | 🔴 | 🔴
iri-reference | 🚫 | 🚫 | 🔴 | 🔴
json-pointer | 🚫 | 🔴 | 🔴 | 🔴
relative-json-pointer | 🚫 | 🔴 | 🔴 | 🔴
regex | 🚫 | 🚫 | 🔴 | 🔴
time | 🚫 | 🚫 | 🟢 | 🟢
uri | 🟢 | 🟢 | 🟢 | 🟢
uri-reference | 🚫 | 🔴 | 🔴 | 🔴
uri-template | 🚫 | 🔴 | 🔴 | 🔴
uuid | 🚫 | 🚫 | 🟢 | 🟢
| Format | Draft 4 | Draft 6 | Draft 7 | Draft 2019-09 | Draft 2020-12 |
|:----------------------|:-------:|:-------:|:-------:|:-------------:|:-------------:|
| date | 🚫 | 🚫 | 🟢 | 🟢 | 🟢 |
| date-time | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| duration | 🚫 | 🚫 | 🚫 | 🟢 | 🟢 |
| email | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| hostname | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| idn-email | 🚫 | 🚫 | 🟢 | 🟢 | 🟢 |
| idn-hostname | 🚫 | 🚫 | 🟢 | 🟢 | 🟢 |
| ipv4 | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| ipv6 | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| iri | 🚫 | 🚫 | 🟢 | 🟢 | 🟢 |
| iri-reference | 🚫 | 🚫 | 🟢 | 🟢 | 🟢 |
| json-pointer | 🚫 | 🟢 | 🟢 | 🟢 | 🟢 |
| relative-json-pointer | 🚫 | 🟢 | 🟢 | 🟢 | 🟢 |
| regex | 🚫 | 🚫 | 🟢 | 🟢 | 🟢 |
| time | 🚫 | 🚫 | 🟢 | 🟢 | 🟢 |
| uri | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| uri-reference | 🚫 | 🟢 | 🟢 | 🟢 | 🟢 |
| uri-template | 🚫 | 🟢 | 🟢 | 🟢 | 🟢 |
| uuid | 🚫 | 🚫 | 🟢 | 🟢 | 🟢 |

### Footnotes
1. Note that the validation are only optional for some of the keywords/formats.
2. Refer to the corresponding JSON schema for more information on whether the keyword/format is optional or not.

Original file line number Diff line number Diff line change
@@ -1,17 +1,23 @@
package com.networknt.schema.regex;

import java.util.regex.Matcher;
import java.util.regex.Pattern;

class JDKRegularExpression implements RegularExpression {
private final Pattern pattern;
private final boolean hasStartAnchor;
private final boolean hasEndAnchor;

JDKRegularExpression(String regex) {
this.pattern = Pattern.compile(regex);
this.hasStartAnchor = '^' == regex.charAt(0);
this.hasEndAnchor = '$' == regex.charAt(regex.length() - 1);
}

@Override
public boolean matches(String value) {
return this.pattern.matcher(value).matches();
Matcher matcher = this.pattern.matcher(value);
return matcher.find() && (!this.hasStartAnchor || 0 == matcher.start()) && (!this.hasEndAnchor || matcher.end() == value.length());
}

}
20 changes: 10 additions & 10 deletions src/test/java/com/networknt/schema/AbstractJsonSchemaTestSuite.java
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@
import static org.junit.jupiter.api.DynamicTest.dynamicTest;

public abstract class AbstractJsonSchemaTestSuite extends HTTPServiceSupport {
protected static final TypeReference<List<TestCase>> testCaseType = new TypeReference<List<TestCase>>() {};
protected static final TypeReference<List<TestCase>> testCaseType = new TypeReference<List<TestCase>>() { /* intentionally empty */};
protected static final Map<String, VersionFlag> supportedVersions = new HashMap<>();
static {
supportedVersions.put("draft2019-09", VersionFlag.V201909);
Expand Down Expand Up @@ -99,7 +99,7 @@ private DynamicNode buildContainer(VersionFlag defaultVersion, TestCase testCase
String msg = e.getMessage();
if (msg.endsWith("' is unrecognizable schema")) {
return dynamicContainer(testCase.getDisplayName(), unsupportedMetaSchema(testCase));
};
}
throw e;
}
}
Expand All @@ -109,7 +109,7 @@ private JsonSchemaFactory buildValidatorFactory(VersionFlag defaultVersion, Test
JsonSchemaFactory base = JsonSchemaFactory.getInstance(specVersion);
return JsonSchemaFactory
.builder(base)
.objectMapper(mapper)
.objectMapper(this.mapper)
.addUriTranslator(URITranslator.combine(
URITranslator.prefix("https://", "http://"),
URITranslator.prefix("http://json-schema.org", "resource:")
Expand All @@ -124,21 +124,21 @@ private DynamicNode buildTest(JsonSchemaFactory validatorFactory, TestSpec testS

SchemaValidatorsConfig config = new SchemaValidatorsConfig();
config.setTypeLoose(typeLoose);
config.setEcma262Validator(true);
config.setEcma262Validator(TestSpec.RegexKind.JDK != testSpec.getRegex());
testSpec.getStrictness().forEach(config::setStrict);
URI testCaseFileUri = URI.create("classpath:" + toForwardSlashPath(testSpec.getTestCase().getSpecification()));
JsonSchema schema = validatorFactory.getSchema(testCaseFileUri, testSpec.getTestCase().getSchema(), config);

return dynamicTest(testSpec.getDescription(), () -> executeAndReset(schema, testSpec));
}

private String toForwardSlashPath(Path file) {
private static String toForwardSlashPath(Path file) {
return file.toString().replace('\\', '/');
}

// For 2019-09 and later published drafts, implementations that are able to
// detect the draft of each schema via $schema SHOULD be configured to do so
private VersionFlag detectVersion(TestCase testCase, VersionFlag defaultVersion) {
private static VersionFlag detectVersion(TestCase testCase, VersionFlag defaultVersion) {
return Stream.of(
detectOptionalVersion(testCase.getSchema()),
detectVersionFromPath(testCase.getSpecification())
Expand All @@ -152,7 +152,7 @@ private VersionFlag detectVersion(TestCase testCase, VersionFlag defaultVersion)
// For draft-07 and earlier, draft-next, and implementations unable to
// detect via $schema, implementations MUST be configured to expect the
// draft matching the test directory name
private Optional<VersionFlag> detectVersionFromPath(Path path) {
private static Optional<VersionFlag> detectVersionFromPath(Path path) {
return StreamSupport.stream(path.spliterator(), false)
.map(Path::toString)
.map(supportedVersions::get)
Expand All @@ -168,7 +168,7 @@ private void executeAndReset(JsonSchema schema, TestSpec testSpec) {
}
}

private void executeTest(JsonSchema schema, TestSpec testSpec) {
private static void executeTest(JsonSchema schema, TestSpec testSpec) {
Set<ValidationMessage> errors = schema.validate(testSpec.getData());

if (testSpec.isValid()) {
Expand Down Expand Up @@ -246,7 +246,7 @@ private List<Path> findTestCases(String basePath) {

private Stream<TestCase> loadTestCases(Path testCaseFile) {
try (InputStream in = new FileInputStream(testCaseFile.toFile())) {
return mapper.readValue(in, testCaseType)
return this.mapper.readValue(in, testCaseType)
.stream()
.peek(testCase -> testCase.setSpecification(testCaseFile))
.filter(this::enabled);
Expand All @@ -258,7 +258,7 @@ private Stream<TestCase> loadTestCases(Path testCaseFile) {
}
}

private Iterable<? extends DynamicNode> unsupportedMetaSchema(TestCase testCase) {
private static Iterable<? extends DynamicNode> unsupportedMetaSchema(TestCase testCase) {
return Collections.singleton(
dynamicTest("Detected an unsupported schema", () -> {
String schema = testCase.getSchema().asText();
Expand Down
18 changes: 17 additions & 1 deletion src/test/java/com/networknt/schema/TestSpec.java
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,11 @@ public class TestSpec {
*/
private final boolean typeLoose;

/**
* Identifies the regular expression engine to use for this test-case.
*/
private final RegexKind regex;

/**
* The TestCase that contains this TestSpec.
*/
Expand All @@ -107,7 +112,8 @@ public TestSpec(
@JsonProperty("strictness") Map<String, Boolean> strictness,
@JsonProperty("validationMessages") Set<String> validationMessages,
@JsonProperty("isTypeLoose") Boolean isTypeLoose,
@JsonProperty("disabled") Boolean disabled
@JsonProperty("disabled") Boolean disabled,
@JsonProperty(value = "regex", defaultValue = "unspecified") RegexKind regex
) {
this.description = description;
this.comment = comment;
Expand All @@ -116,6 +122,7 @@ public TestSpec(
this.validationMessages = validationMessages;
this.disabled = Boolean.TRUE.equals(disabled);
this.typeLoose = Boolean.TRUE.equals(isTypeLoose);
this.regex = regex;
if (null != strictness) {
this.strictness.putAll(strictness);
}
Expand Down Expand Up @@ -211,4 +218,13 @@ public boolean isTypeLoose() {
return typeLoose;
}

public RegexKind getRegex() {
return this.regex;
}

public static enum RegexKind {
@JsonProperty("unspecified") UNSPECIFIED,
@JsonProperty("ecma-262") JONI,
@JsonProperty("jdk") JDK
}
}
64 changes: 64 additions & 0 deletions src/test/resources/draft2020-12/issue495.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
[
{
"description": "issue495 using ECMA-262",
"regex": "ecma-262",
"schema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"pattern": "^[a-z]{1,10}$",
"unevaluatedProperties": false
},
"tests": [
{
"description": "an expected property name",
"data": { "aaa": 3 },
"valid": true
},
{
"description": "trailing newline",
"data": { "aaa\n": 3 },
"valid": false
},
{
"description": "embedded newline",
"data": { "aaa\nbbb": 3 },
"valid": false
},
{
"description": "leading newline",
"data": { "\nbbb": 3 },
"valid": false
}
]
},
{
"description": "issue495 using Java Pattern",
"regex": "jdk",
"schema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"pattern": "^[a-z]{1,10}$",
"unevaluatedProperties": false
},
"tests": [
{
"description": "an expected property name",
"data": { "aaa": 3 },
"valid": true
},
{
"description": "trailing newline",
"data": { "aaa\n": 3 },
"valid": false
},
{
"description": "embedded newline",
"data": { "aaa\nbbb": 3 },
"valid": false
},
{
"description": "leading newline",
"data": { "\nbbb": 3 },
"valid": false
}
]
}
]
Loading