[vividus-plugin-json] Add step to validate JSON against schema #3597

IBrun · 2023-01-30T10:40:20Z

No description provided.

codecov · 2023-01-30T10:47:05Z

Codecov Report

Merging #3597 (11c72b6) into master (321c018) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##             master    #3597   +/-   ##
=========================================
  Coverage     96.98%   96.99%           
- Complexity     6068     6076    +8     
=========================================
  Files           855      856    +1     
  Lines         17350    17384   +34     
  Branches       1123     1127    +4     
=========================================
+ Hits          16827    16861   +34     
  Misses          417      417           
  Partials        106      106

Impacted Files	Coverage Δ
.../vividus/json/steps/JsonSchemaValidationSteps.java	`100.00% <100.00%> (ø)`

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

valfirst · 2023-01-31T09:22:17Z

docs/modules/plugins/pages/plugin-json.adoc

+Then JSON `$json` is valid against schema `$schema`
+----
+
+* `$json` - The JSON.


Suggested change

* `$json` - The JSON.

* `$json` - The JSON to validate.

valfirst · 2023-01-31T09:25:23Z

vividus-plugin-json/src/main/java/org/vividus/json/steps/JsonSteps.java

+ @Then("JSON `$json` is valid against schema `$schema`")
+ public void validateJsonAgainstSchema(String json, String schema)
+ {
+ JsonSchemaFactory factory = JsonSchemaFactory.getInstance(SpecVersion.VersionFlag.V202012);


I guess, it makes sense to use version of the schema specified in it, otherwise we should default to 2020-12

@valfirst could you please elaborate

@ikalinin1 the schema itself specfies its version "$schema": "https://json-schema.org/draft/2020-12/schema",

valfirst · 2023-01-31T09:25:53Z

vividus-plugin-json/src/main/java/org/vividus/json/steps/JsonSteps.java

+ if (!errors.isEmpty())
+ {
+ softAssert.recordFailedAssertion(
+ String.format("JSON is not valid against schema: %s", StringUtils.join(errors, ", ")));


how does error look like?

ikalinin1 · 2023-01-31T10:07:17Z

vividus-plugin-json/src/main/java/org/vividus/json/steps/JsonSteps.java

@@ -485,6 +490,25 @@ public void assertNumberOfJsonElements(String json, String jsonPath, ComparisonR
 assertJsonElementsNumber(jsonPath, actualNumber, comparisonRule, elementsNumber);
 }

+ /**
+ * Validates json against json schema


I believe javadoc should mention the version of the schema

ikalinin1 · 2023-01-31T10:51:01Z

docs/modules/plugins/pages/plugin-json.adoc

@@ -500,3 +502,59 @@ Then number of JSON elements from `
 ]
 ` by JSON path `$..accountId` is equal to 2
 ----
+
+=== Validate JSON document


Suggested change

=== Validate JSON document

=== Validate JSON against schema

ikalinin1 · 2023-01-31T11:06:10Z

vividus-plugin-json/src/main/java/org/vividus/json/steps/JsonSteps.java

+ if (!errors.isEmpty())
+ {
+ softAssert.recordFailedAssertion(
+ String.format("JSON is not valid against schema: %s", StringUtils.join(errors, ", ")));


java.lang.String#join(java.lang.CharSequence, java.lang.Iterable<? extends java.lang.CharSequence>) ?

uarlouski · 2023-01-31T11:24:22Z

vividus-plugin-json/build.gradle

@@ -13,6 +13,7 @@ dependencies {
 implementation(group: 'io.qameta.allure', name: 'allure-jsonunit', version: versions.allure) {
 exclude (group: 'io.qameta.allure')
 }
+ implementation(group: "com.networknt", name: "json-schema-validator", version: "1.0.76");


singlequotes as per all other cases where string interpolation doesn't happen

ikalinin1 · 2023-01-31T11:53:01Z

vividus-tests/src/main/resources/story/integration/JSON.story

+ "description": "Tags for the product",
+ "type": "array",
+ "items": {
+ "type": "string"


please update this example to validate that the array contains a specific list of items not any strigns

valfirst · 2023-01-31T15:42:59Z

docs/modules/plugins/pages/plugin-json.adoc

+:schema-specification: https://json-schema.org/specification-links.html[schema specification]
+:meta-schema-2020-12: https://json-schema.org/specification-links.html#2020-12[2020-12]


you can inline attributes used once

valfirst · 2023-01-31T15:43:26Z

docs/modules/plugins/pages/plugin-json.adoc

+----
+"$schema": "https://json-schema.org/draft/2020-12/schema"
+----
+If it is not present in the schema than JSON is validated using {meta-schema-2020-12} specification.


Suggested change

If it is not present in the schema than JSON is validated using {meta-schema-2020-12} specification.

If the version is not present in the schema then JSON is validated according to {meta-schema-2020-12} version.

valfirst · 2023-01-31T15:45:05Z

docs/modules/plugins/pages/plugin-json.adoc

+====
+:schema-specification: https://json-schema.org/specification-links.html[schema specification]
+:meta-schema-2020-12: https://json-schema.org/specification-links.html#2020-12[2020-12]
+Step validates JSON according {schema-specification} given in the schema. E.g.


Suggested change

Step validates JSON according {schema-specification} given in the schema. E.g.

The step validates JSON according to {schema-specification} version provided in the schema itself, e.g.:

valfirst · 2023-01-31T16:06:53Z

vividus-plugin-json/src/main/java/org/vividus/json/steps/JsonSteps.java

@@ -485,6 +493,45 @@ public void assertNumberOfJsonElements(String json, String jsonPath, ComparisonR
 assertJsonElementsNumber(jsonPath, actualNumber, comparisonRule, elementsNumber);
 }

+ /**
+ * Validates json against json schema


make javadoc as much possible close to the documentation

valfirst · 2023-01-31T16:07:55Z

vividus-plugin-json/src/main/java/org/vividus/json/steps/JsonSteps.java

+ {
+ JsonNode schemaNode = jsonUtils.readTree(schema);
+ SpecVersion.VersionFlag version;
+ if (schemaNode.has(SCHEMA_TAG))


having this complexity added it makes sense to move the step to a new class

valfirst · 2023-01-31T16:12:49Z

vividus-plugin-json/src/main/java/org/vividus/json/steps/JsonSteps.java

+ JsonSchema jsonSchema = factory.getSchema(schemaNode);
+ JsonNode jsonNode = jsonUtils.readTree(json);
+ Set<ValidationMessage> errors = jsonSchema.validate(jsonNode);
+ if (!errors.isEmpty())


we need to record passed assertion in case of success

valfirst · 2023-01-31T16:13:29Z

vividus-plugin-json/src/main/java/org/vividus/json/steps/JsonSteps.java

+ if (!errors.isEmpty())
+ {
+ softAssert.recordFailedAssertion("JSON is not valid against schema");
+ errors.forEach(error -> softAssert.recordFailedAssertion(error.toString()));


there should be a single assertion error with all schema violations listed

valfirst · 2023-02-01T10:08:00Z

vividus-plugin-json/src/main/java/org/vividus/json/steps/JsonValidationSteps.java

+import org.vividus.softassert.ISoftAssert;
+import org.vividus.util.json.JsonUtils;
+
+public class JsonValidationSteps


Suggested change

public class JsonValidationSteps

public class JsonSchemaValidationSteps

valfirst · 2023-02-01T10:09:38Z

vividus-plugin-json/src/main/java/org/vividus/json/steps/JsonValidationSteps.java

+ List<String> messages = new ArrayList<>();
+ int index = 1;
+ for (ValidationMessage validationMessage : validationMessages)
+ {
+ messages.add(String.format("%d) %s", index, validationMessage.toString()));
+ index++;
+ }
+ return String.join(String.format(",%n"), messages);


why not to use StringBuilder?

valfirst · 2023-02-01T10:10:56Z

vividus-plugin-json/src/main/resources/vividus-plugin/spring.xml

- <bean id="jsonSteps" class="org.vividus.json.steps.JsonSteps">
- <constructor-arg>
- <bean class="org.vividus.json.softassert.JsonSoftAssert" parent="softAssert" />
- </constructor-arg>
- </bean>


why is this change needed?

To move definition of JsonSoftAssert bean from definition of jsonSteps bean

valfirst · 2023-02-01T11:52:10Z

vividus-plugin-json/src/main/java/org/vividus/json/steps/JsonSchemaValidationSteps.java

+ {
+ if (index != 1)
+ {
+ messagesBuilder.append(",").append(System.lineSeparator());


Suggested change

messagesBuilder.append(",").append(System.lineSeparator());

messagesBuilder.append(',').append(System.lineSeparator());

valfirst · 2023-02-01T12:31:30Z

vividus-plugin-json/src/main/java/org/vividus/json/steps/JsonSchemaValidationSteps.java

+ }
+ else
+ {
+ softAssert.recordFailedAssertion(String.format("JSON is not valid against schema:%n%s",


here String.format is used, but below StringBuilder is used, it makes sense to use single StringBuilder to build full string

IBrun marked this pull request as ready for review January 30, 2023 11:03

IBrun requested a review from a team as a code owner January 30, 2023 11:03

IBrun requested review from valfirst, ikalinin1 and uarlouski January 30, 2023 11:03

IBrun changed the title ~~[vividus-plugin-json] Add step for validate JSON against schema~~ [vividus-plugin-json] Add step to validate JSON against schema Jan 30, 2023

IBrun force-pushed the json_validation branch from 7e44625 to 76ba0d0 Compare January 30, 2023 14:18

valfirst requested changes Jan 31, 2023

View reviewed changes

ikalinin1 reviewed Jan 31, 2023

View reviewed changes

uarlouski reviewed Jan 31, 2023

View reviewed changes

ikalinin1 reviewed Jan 31, 2023

View reviewed changes

IBrun marked this pull request as draft January 31, 2023 12:15

IBrun force-pushed the json_validation branch 2 times, most recently from 9f83bcb to 7cd5728 Compare January 31, 2023 12:31

IBrun marked this pull request as ready for review January 31, 2023 12:53

IBrun requested review from ikalinin1, uarlouski and valfirst and removed request for ikalinin1 and uarlouski January 31, 2023 12:53

IBrun force-pushed the json_validation branch 2 times, most recently from ff7bdf6 to d63f8ec Compare January 31, 2023 13:57

valfirst requested review from ikalinin1 and uarlouski January 31, 2023 15:41

valfirst requested changes Jan 31, 2023

View reviewed changes

IBrun force-pushed the json_validation branch from d63f8ec to 0e9466d Compare February 1, 2023 09:32

IBrun removed the request for review from ikalinin1 February 1, 2023 10:00

IBrun requested review from valfirst and removed request for uarlouski February 1, 2023 10:00

valfirst requested changes Feb 1, 2023

View reviewed changes

IBrun force-pushed the json_validation branch from 0e9466d to 1147ba4 Compare February 1, 2023 11:06

IBrun requested review from valfirst, uarlouski and ikalinin1 and removed request for valfirst, uarlouski and ikalinin1 February 1, 2023 11:07

valfirst requested changes Feb 1, 2023

View reviewed changes

[vividus-plugin-json] Add step to validate JSON against schema

11c72b6

IBrun force-pushed the json_validation branch from 1147ba4 to 11c72b6 Compare February 1, 2023 12:59

IBrun requested review from valfirst, uarlouski and ikalinin1 and removed request for valfirst and uarlouski February 1, 2023 13:09

valfirst approved these changes Feb 1, 2023

View reviewed changes

valfirst requested a review from uarlouski February 1, 2023 13:22

uarlouski approved these changes Feb 1, 2023

View reviewed changes

ikalinin1 approved these changes Feb 1, 2023

View reviewed changes

valfirst merged commit 42d4fcb into vividus-framework:master Feb 1, 2023

IBrun deleted the json_validation branch June 29, 2023 08:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[vividus-plugin-json] Add step to validate JSON against schema #3597

[vividus-plugin-json] Add step to validate JSON against schema #3597

IBrun commented Jan 30, 2023

codecov bot commented Jan 30, 2023 •

edited

Loading

valfirst Jan 31, 2023

valfirst Jan 31, 2023 •

edited

Loading

ikalinin1 Jan 31, 2023

uarlouski Jan 31, 2023

valfirst Jan 31, 2023

ikalinin1 Jan 31, 2023

ikalinin1 Jan 31, 2023

ikalinin1 Jan 31, 2023

uarlouski Jan 31, 2023

ikalinin1 Jan 31, 2023

valfirst Jan 31, 2023

valfirst Jan 31, 2023

valfirst Jan 31, 2023

valfirst Jan 31, 2023

valfirst Jan 31, 2023

valfirst Jan 31, 2023

valfirst Jan 31, 2023

valfirst Feb 1, 2023

valfirst Feb 1, 2023

valfirst Feb 1, 2023

IBrun Feb 1, 2023

valfirst Feb 1, 2023

valfirst Feb 1, 2023

		:schema-specification: https://json-schema.org/specification-links.html[schema specification]
		:meta-schema-2020-12: https://json-schema.org/specification-links.html#2020-12[2020-12]

	If it is not present in the schema than JSON is validated using {meta-schema-2020-12} specification.
	If the version is not present in the schema then JSON is validated according to {meta-schema-2020-12} version.

	Step validates JSON according {schema-specification} given in the schema. E.g.
	The step validates JSON according to {schema-specification} version provided in the schema itself, e.g.:

	public class JsonValidationSteps
	public class JsonSchemaValidationSteps

	messagesBuilder.append(",").append(System.lineSeparator());
	messagesBuilder.append(',').append(System.lineSeparator());

[vividus-plugin-json] Add step to validate JSON against schema #3597

[vividus-plugin-json] Add step to validate JSON against schema #3597

Conversation

IBrun commented Jan 30, 2023

codecov bot commented Jan 30, 2023 • edited Loading

Codecov Report

Choose a reason for hiding this comment

valfirst Jan 31, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Jan 30, 2023 •

edited

Loading

valfirst Jan 31, 2023 •

edited

Loading