Skip to content

Conversation

ppkarwasz
Copy link
Contributor

This change modifies the Java Unit tests to use the schemas in this repository to validate the examples, instead of those bundled in the cyclonedx-java-core artifact.

Closes #256

@ppkarwasz ppkarwasz requested a review from a team as a code owner February 9, 2025 14:59
This change modifies the Java Unit tests to use the schemas in this repository to validate the examples, instead of those bundled in the `cyclonedx-java-core` artifact.

Closes CycloneDX#256

Signed-off-by: Piotr P. Karwasz <piotr.github@karwasz.org>
@ppkarwasz ppkarwasz force-pushed the fix/256_java_unit_tests branch from ee77f65 to 08e3851 Compare February 9, 2025 14:59
@jkowalleck
Copy link
Member

this seams to hit not the entirety of #592 .

I had the tests run in an air-gapped offline environment, to see weather it actually uses the shipped schema files, and does not load some from the web.
I saw errors like

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.cyclonedx.schema.XmlCatalogVerificationTest
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.376 s -- in org.cyclonedx.schema.XmlCatalogVerificationTest
[INFO] Running org.cyclonedx.schema.XmlSchemaVerificationTest
[INFO] Tests run: 321, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.168 s -- in org.cyclonedx.schema.XmlSchemaVerificationTest
[INFO] Running org.cyclonedx.schema.JsonSchemaVerificationTest
[main] ERROR com.networknt.schema.JsonSchemaFactory - Failed to load json schema from http://cyclonedx.org/schema/spdx.schema.json
java.net.UnknownHostException: cyclonedx.org
 ...

@jkowalleck jkowalleck added chore: QA A chore related to Quality Assurance test-data related to test-resources and -data labels Feb 9, 2025
@ppkarwasz
Copy link
Contributor Author

@jkowalleck,

I had the tests run in an air-gapped offline environment, to see weather it actually uses the shipped schema files, and does not load some from the web. I saw errors like

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.cyclonedx.schema.XmlCatalogVerificationTest
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.376 s -- in org.cyclonedx.schema.XmlCatalogVerificationTest
[INFO] Running org.cyclonedx.schema.XmlSchemaVerificationTest
[INFO] Tests run: 321, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.168 s -- in org.cyclonedx.schema.XmlSchemaVerificationTest
[INFO] Running org.cyclonedx.schema.JsonSchemaVerificationTest
[main] ERROR com.networknt.schema.JsonSchemaFactory - Failed to load json schema from http://cyclonedx.org/schema/spdx.schema.json
java.net.UnknownHostException: cyclonedx.org
 ...

Nice catch! Fixed in fd405f4

Signed-off-by: Piotr P. Karwasz <piotr.github@karwasz.org>
@ppkarwasz ppkarwasz force-pushed the fix/256_java_unit_tests branch from fd405f4 to c935467 Compare February 9, 2025 16:25
@jkowalleck
Copy link
Member

thank you so much for this, @ppkarwasz. 👍

@jkowalleck jkowalleck changed the title Run test against actual schema files (Java) tests: run test against actual schema files (Java) Feb 9, 2025
@jkowalleck jkowalleck changed the title tests: run test against actual schema files (Java) tests(Java): run test against actual schema files Feb 9, 2025
@jkowalleck jkowalleck self-requested a review February 9, 2025 16:36
The XML test did not access the SPDX schema from the network, but we just want to make sure that this will not happen in the future.

Signed-off-by: Piotr P. Karwasz <piotr.github@karwasz.org>
@jkowalleck
Copy link
Member

jkowalleck commented Feb 10, 2025

@Nicolas-Peiffer , could I ask you for a review?
maybe you have an idea how the XML tests could benefit from incorporating the new catalog from #479? :-)

@ppkarwasz
Copy link
Contributor Author

maybe you have an idea how the XML tests could incorporate the new catalog from #479? :-)

The XML Catalog can replace specifying the location of spdx.xml by hand:

factory.setProperty(
"http://apache.org/xml/properties/schema/external-schemaLocation",
"http://cyclonedx.org/schema/spdx spdx.xsd");

However, I am not sure if we should do it, because it makes the Unit test less "unit".

@jkowalleck
Copy link
Member

jkowalleck commented Feb 11, 2025

However, I am not sure if we should do it, because it makes the Unit test less "unit".

they are functional tests, not unit test. (they don't care why something is [in]valid, as long it is)
The catalog is just a lookup table. as long as we "ask" the code to validate based on a specific schema (which looks like a URL, but is actually just a name) we should be good, i think?

@ppkarwasz
Copy link
Contributor Author

The catalog is just a lookup table. as long as we "ask" the code to validate based on a specific schema (which looks like a URL, but is actually just a name) we should be good, i think?

In that case I can update the tests to use the XML Catalog for lookups. Are there any problems if I bump the Java version of the module to Java 9? That version introduced out-of-the-box support for XML Catalogs.

@jkowalleck
Copy link
Member

The catalog is just a lookup table. as long as we "ask" the code to validate based on a specific schema (which looks like a URL, but is actually just a name) we should be good, i think?

In that case I can update the tests to use the XML Catalog for lookups. Are there any problems if I bump the Java version of the module to Java 9? That version introduced out-of-the-box support for XML Catalogs.

not at all.

As far as I see it:
all we want here it so see whether all of our test data behave according to plan. :D Everything else is basically a bonus.
We use several different implementations of XML/JSON validation libraries for that, to cover as much as possible.

@jkowalleck jkowalleck merged commit d570ffb into CycloneDX:master Feb 17, 2025
9 checks passed
@jkowalleck
Copy link
Member

merged as is.

further improvements are welcome.
(e.g. using the xml catalog etc...)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
chore: QA A chore related to Quality Assurance test-data related to test-resources and -data
Projects
None yet
Development

Successfully merging this pull request may close these issues.

tests: run test against actual schema files, not snapshots
3 participants