tests(Java): run test against actual schema files

[ppkarwasz]

This change modifies the Java Unit tests to use the schemas in this repository to validate the examples, instead of those bundled in the cyclonedx-java-core artifact.

Closes #256

[jkowalleck]

this seams to hit not the entirety of #592 .

I had the tests run in an air-gapped offline environment, to see weather it actually uses the shipped schema files, and does not load some from the web.
I saw errors like

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.cyclonedx.schema.XmlCatalogVerificationTest
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.376 s -- in org.cyclonedx.schema.XmlCatalogVerificationTest
[INFO] Running org.cyclonedx.schema.XmlSchemaVerificationTest
[INFO] Tests run: 321, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.168 s -- in org.cyclonedx.schema.XmlSchemaVerificationTest
[INFO] Running org.cyclonedx.schema.JsonSchemaVerificationTest
[main] ERROR com.networknt.schema.JsonSchemaFactory - Failed to load json schema from http://cyclonedx.org/schema/spdx.schema.json
java.net.UnknownHostException: cyclonedx.org
 ...

[ppkarwasz]

@jkowalleck,

I had the tests run in an air-gapped offline environment, to see weather it actually uses the shipped schema files, and does not load some from the web. I saw errors like

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.cyclonedx.schema.XmlCatalogVerificationTest
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.376 s -- in org.cyclonedx.schema.XmlCatalogVerificationTest
[INFO] Running org.cyclonedx.schema.XmlSchemaVerificationTest
[INFO] Tests run: 321, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.168 s -- in org.cyclonedx.schema.XmlSchemaVerificationTest
[INFO] Running org.cyclonedx.schema.JsonSchemaVerificationTest
[main] ERROR com.networknt.schema.JsonSchemaFactory - Failed to load json schema from http://cyclonedx.org/schema/spdx.schema.json
java.net.UnknownHostException: cyclonedx.org
 ...

Nice catch! Fixed in fd405f4

[jkowalleck]

thank you so much for this, @ppkarwasz. 👍

[jkowalleck]

@Nicolas-Peiffer , could I ask you for a review?
maybe you have an idea how the XML tests could benefit from incorporating the new catalog from #479? :-)

[ppkarwasz]

maybe you have an idea how the XML tests could incorporate the new catalog from #479? :-)

The XML Catalog can replace specifying the location of spdx.xml by hand:

specification/tools/src/test/java/org/cyclonedx/schema/XmlSchemaVerificationTest.java

Lines 50 to 52 in dc4a2ab

	factory.setProperty(
	"http://apache.org/xml/properties/schema/external-schemaLocation",
	"http://cyclonedx.org/schema/spdx spdx.xsd");

However, I am not sure if we should do it, because it makes the Unit test less "unit".

[jkowalleck]

However, I am not sure if we should do it, because it makes the Unit test less "unit".

they are functional tests, not unit test. (they don't care why something is [in]valid, as long it is)
The catalog is just a lookup table. as long as we "ask" the code to validate based on a specific schema (which looks like a URL, but is actually just a name) we should be good, i think?

[ppkarwasz]

The catalog is just a lookup table. as long as we "ask" the code to validate based on a specific schema (which looks like a URL, but is actually just a name) we should be good, i think?

In that case I can update the tests to use the XML Catalog for lookups. Are there any problems if I bump the Java version of the module to Java 9? That version introduced out-of-the-box support for XML Catalogs.

[jkowalleck]

The catalog is just a lookup table. as long as we "ask" the code to validate based on a specific schema (which looks like a URL, but is actually just a name) we should be good, i think?

In that case I can update the tests to use the XML Catalog for lookups. Are there any problems if I bump the Java version of the module to Java 9? That version introduced out-of-the-box support for XML Catalogs.

not at all.

As far as I see it:
all we want here it so see whether all of our test data behave according to plan. :D Everything else is basically a bonus.
We use several different implementations of XML/JSON validation libraries for that, to cover as much as possible.