Unable to parse text exception with in models with `group-as/@in-json="SINGLETON_OR_ARRAY"` behavior #221

aj-stein-gsa · 2024-10-28T21:48:01Z

Describe the bug

When using metaschema-java in the latest stable version to deserialize models with fields (not assemblies) with the group-as/@in-json="SINGLETON_OR_ARRAY", you will consistently receive a java.io.IOException: Unable to parse field value as text exception.

Who is the bug affecting

Developers of Metaschema processors operating on constraints and developers maintaining constraints

How do we replicate this issue

Download the reference model
Download the example document instance
Run the command below and confirm the error is reproducible.

% metaschema-cli --version
metaschema-java 1.2.0 built at 2024-10-08T20:00:42+0000 from branch 46df8d8fc25c5de1d7cb0485e534f31efe61b2b7 (46df8d8) at https://github.com/metaschema-framework/metaschema-java
metaschema  built at 2024-10-08T20:00:42+0000 from branch 7c03ce5844e46cf9d047193a37e44422ae6a7d61 (7c03ce5) at https://github.com/metaschema-framework/metaschema.git
% metaschema-cli validate-content -m=src/validations/constraints/unit-tests/unit_test_metaschema.xml src/validations/constraints/unit-tests/security-level-PASS.yaml --show-stack-trace
Loading 'file:/home/meunit_test_metaschema.xml'
Generating Java classes in: /var/folders/_q/y32s9yw56lb81l1sjgtp7k080000gn/T/validation-7255337077282254142
An uncaught runtime error occurred. Unable to generate class for definition 'test-case' in Module 'file:/home/meunit_test_metaschema.xml'
java.lang.IllegalStateException: Unable to generate class for definition 'test-case' in Module 'file:/home/meunit_test_metaschema.xml'
        at gov.nist.secauto.metaschema.databind.codegen.typeinfo.DefaultMetaschemaClassFactory.lambda$generateClass$1(DefaultMetaschemaClassFactory.java:156) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) ~[?:?]
        at java.base/java.util.stream.Streams$StreamBuilderImpl.forEachRemaining(Streams.java:411) ~[?:?]
        at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:762) ~[?:?]
        at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:276) ~[?:?]
        at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133) ~[?:?]
        at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1939) ~[?:?]
        at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:734) ~[?:?]
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[?:?]
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?]
        at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921) ~[?:?]
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
        at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682) ~[?:?]
        at gov.nist.secauto.metaschema.databind.codegen.typeinfo.DefaultMetaschemaClassFactory.generateClass(DefaultMetaschemaClassFactory.java:174) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.databind.codegen.ProductionImpl.addModule(ProductionImpl.java:43) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.databind.codegen.IProduction.of(IProduction.java:97) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.databind.codegen.JavaGenerator.generate(JavaGenerator.java:79) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.databind.codegen.JavaGenerator.generate(JavaGenerator.java:50) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.databind.codegen.ModuleCompilerHelper.compileModule(ModuleCompilerHelper.java:94) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.databind.codegen.ModuleCompilerHelper.compileMetaschema(ModuleCompilerHelper.java:70) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.databind.DefaultBindingContext.registerModule(DefaultBindingContext.java:251) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.cli.commands.ValidateContentUsingModuleCommand$OscalCommandExecutor.getBindingContext(ValidateContentUsingModuleCommand.java:129) ~[dev.metaschema.java.metaschema-cli-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.cli.commands.AbstractValidateContentCommand$AbstractValidationCommandExecutor.execute(AbstractValidateContentCommand.java:219) ~[dev.metaschema.java.metaschema-cli-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.cli.processor.CLIProcessor$CallingContext.invokeCommand(CLIProcessor.java:405) ~[dev.metaschema.java.cli-processor-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.cli.processor.CLIProcessor$CallingContext.processCommand(CLIProcessor.java:376) [dev.metaschema.java.cli-processor-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.cli.processor.CLIProcessor.parseCommand(CLIProcessor.java:175) [dev.metaschema.java.cli-processor-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.cli.processor.CLIProcessor.process(CLIProcessor.java:158) [dev.metaschema.java.cli-processor-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.cli.CLI.runCli(CLI.java:59) [dev.metaschema.java.metaschema-cli-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.cli.CLI.main(CLI.java:31) [dev.metaschema.java.metaschema-cli-1.2.0.jar:?]
Caused by: java.lang.NullPointerException
        at gov.nist.secauto.metaschema.core.util.ObjectUtils.requireNonNull(ObjectUtils.java:53) ~[dev.metaschema.java.metaschema-core-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.core.model.AbstractFieldInstance.getDefinition(AbstractFieldInstance.java:46) ~[dev.metaschema.java.metaschema-core-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.core.model.AbstractFieldInstance.getDefinition(AbstractFieldInstance.java:25) ~[dev.metaschema.java.metaschema-core-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.core.model.IFeatureDefinitionReferenceInstance.getEffectiveName(IFeatureDefinitionReferenceInstance.java:49) ~[dev.metaschema.java.metaschema-core-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.core.model.AbstractNamedInstance.lambda$new$0(AbstractNamedInstance.java:35) ~[dev.metaschema.java.metaschema-core-1.2.0.jar:?]
        at nl.talsmasoftware.lazy4j.Lazy.forceEagerEvaluation(Lazy.java:85) ~[nl.talsmasoftware.lazy4j-2.0.0.jar:?]
        at nl.talsmasoftware.lazy4j.Lazy.get(Lazy.java:101) ~[nl.talsmasoftware.lazy4j-2.0.0.jar:?]
        at gov.nist.secauto.metaschema.core.model.AbstractNamedInstance.getXmlQName(AbstractNamedInstance.java:42) ~[dev.metaschema.java.metaschema-core-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.core.model.xml.impl.XmlAssemblyModelContainer.append(XmlAssemblyModelContainer.java:186) ~[dev.metaschema.java.metaschema-core-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.core.model.xml.impl.XmlAssemblyModelContainer.handleField(XmlAssemblyModelContainer.java:119) ~[dev.metaschema.java.metaschema-core-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.core.model.xml.impl.XmlAssemblyModelContainer$1.lambda$identifyHandler$0(XmlAssemblyModelContainer.java:68) ~[dev.metaschema.java.metaschema-core-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.core.model.xml.impl.XmlObjectParser.parse(XmlObjectParser.java:192) ~[dev.metaschema.java.metaschema-core-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.core.model.xml.impl.XmlAssemblyModelContainer.of(XmlAssemblyModelContainer.java:109) ~[dev.metaschema.java.metaschema-core-1.2.0.jar:?]
alexanderjstein@FCOH2J-JH2LXQ2H fedramp-automation % metaschema-cli validate-content -m=src/validations/constraints/unit-tests/unit_test_metaschema.xml src/validations/constraints/unit-tests/security-level-PASS.yaml --show-stack-trace
Loading 'file:/home/meunit_test_metaschema.xml'
Generating Java classes in: /var/folders/_q/y32s9yw56lb81l1sjgtp7k080000gn/T/validation-2779578180586592859
Validating 'file:/home/mesecurity-level-PASS.yaml' as YAML.
java.io.IOException: Unable to parse field value as text
java.io.IOException: java.io.IOException: Unable to parse field value as text
        at gov.nist.secauto.metaschema.databind.io.AbstractDeserializer.deserializeToNodeItem(AbstractDeserializer.java:81) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.databind.io.IDeserializer.deserializeToNodeItem(IDeserializer.java:175) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.databind.io.DefaultBoundLoader.loadAsNodeItem(DefaultBoundLoader.java:239) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.databind.io.DefaultBoundLoader.loadAsNodeItem(DefaultBoundLoader.java:215) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.databind.io.DefaultBoundLoader.loadAsNodeItem(DefaultBoundLoader.java:205) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.databind.IBindingContext.validateWithConstraints(IBindingContext.java:414) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.cli.commands.AbstractValidateContentCommand$AbstractValidationCommandExecutor.execute(AbstractValidateContentCommand.java:289) ~[dev.metaschema.java.metaschema-cli-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.cli.processor.CLIProcessor$CallingContext.invokeCommand(CLIProcessor.java:405) ~[dev.metaschema.java.cli-processor-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.cli.processor.CLIProcessor$CallingContext.processCommand(CLIProcessor.java:376) [dev.metaschema.java.cli-processor-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.cli.processor.CLIProcessor.parseCommand(CLIProcessor.java:175) [dev.metaschema.java.cli-processor-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.cli.processor.CLIProcessor.process(CLIProcessor.java:158) [dev.metaschema.java.cli-processor-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.cli.CLI.runCli(CLI.java:59) [dev.metaschema.java.metaschema-cli-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.cli.CLI.main(CLI.java:31) [dev.metaschema.java.metaschema-cli-1.2.0.jar:?]
Caused by: java.io.IOException: Unable to parse field value as text
        at gov.nist.secauto.metaschema.core.datatype.AbstractDataTypeAdapter.parse(AbstractDataTypeAdapter.java:127) ~[dev.metaschema.java.metaschema-core-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.databind.io.json.MetaschemaJsonReader.readScalarItem(MetaschemaJsonReader.java:358) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.databind.io.json.MetaschemaJsonReader.readItemField(MetaschemaJsonReader.java:269) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.databind.model.IBoundInstanceModelFieldScalar.readItem(IBoundInstanceModelFieldScalar.java:74) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.databind.io.json.MetaschemaJsonReader$ModelInstanceReadHandler.readItem(MetaschemaJsonReader.java:763) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.databind.model.info.IModelInstanceReadHandler.readSingleton(IModelInstanceReadHandler.java:18) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.databind.model.info.SingletonCollectionInfo.readItems(SingletonCollectionInfo.java:67) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.databind.io.json.MetaschemaJsonReader.readModelInstance(MetaschemaJsonReader.java:238) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.databind.io.json.MetaschemaJsonReader.readObjectProperty(MetaschemaJsonReader.java:253) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.databind.io.json.MetaschemaJsonReader$PropertyBodyHandler.accept(MetaschemaJsonReader.java:548) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.databind.io.json.MetaschemaJsonReader.readComplexDefinitionObject(MetaschemaJsonReader.java:421) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.databind.io.json.MetaschemaJsonReader.readItemAssembly(MetaschemaJsonReader.java:346) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.databind.model.IBoundDefinitionModelAssembly.readItem(IBoundDefinitionModelAssembly.java:75) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.databind.model.IBoundDefinitionModelAssembly.readItem(IBoundDefinitionModelAssembly.java:29) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.databind.io.json.MetaschemaJsonReader.readObjectRoot(MetaschemaJsonReader.java:193) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.databind.io.json.DefaultJsonDeserializer.deserializeToNodeItemInternal(DefaultJsonDeserializer.java:99) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
        at gov.nist.secauto.metaschema.databind.io.AbstractDeserializer.deserializeToNodeItem(AbstractDeserializer.java:79) ~[dev.metaschema.java.metaschema-databind-1.2.0.jar:?]
                ... 12 more

Expected behavior (i.e. solution)

The model is properly deserialized and content validation is processed to complete on this example document instance as valid.

Other comments

No response

The text was updated successfully, but these errors were encountered:

aj-stein-gsa · 2024-10-28T22:36:20Z

This an invalid model because of the conflicting requirements between the parent max-occurs and the inner group-as. When max-occurs="1" is set outside such a group-as, the latter is ignored by the metaschema-java core parser.

Dave is going to write improved error messaging around this issue in the metaschema-core related to this poor incorrect model defintion.
AJ to write constraints in the Metaschema module module to give a hard error for group-as/@in-json as ARRAY or SINGLETON_OR_ARRAY where parent max-occurs MUST be greater than 1 exclusive (2 to unbounded).

This new constraint implements a check for the following issue comment. metaschema-framework/metaschema-java#221 (comment)

This addresses changes that should be handled as determined by upstream guidance from metaschema-framework maintainers. metaschema-framework/metaschema-java#221 metaschema-framework/metaschema#44

@wandmagic

* [skip ci] Metaschema module for unit test struct We are proposing changes to the structure of the JSON/YAML unit test file used for all constraints test suite management. This version documents the model of that YAML file before the proposed change in the comments of the #817 PR, linked below. #817 (comment) * [skip ci] Document unit test metaschema elements * [skip ci] Do not explicitly use default types * [skip ci] Add missing `@id` and `@level` to model * [skip ci] Add pipeline to pipeline, thx @wandmagic * [skip ci] Backport #219 docs into model remarks * [skip ci] Fix max-occurs re bad model cardinality This addresses changes that should be handled as determined by upstream guidance from metaschema-framework maintainers. metaschema-framework/metaschema-java#221 metaschema-framework/metaschema#44 * [skip ci] Is a pipeline with a ref even a pipeline at all? Thanks to @Rene2mt for PR feedback about the missing ref to the assembly definition of pipeline in the model. Co-authored-by: Rene Tshiteya <rene-claude.tshiteya@gsa.gov> --------- Co-authored-by: Rene Tshiteya <rene-claude.tshiteya@gsa.gov>

Changed expected behavior in unit test to provide null values in JSON and empty values in XML, which is consistent with current behavior. Improved error handling for data type conformance issues. Related to GSA/fedramp-automation#823.

@wandmagic

* [skip ci] Metaschema module for unit test struct We are proposing changes to the structure of the JSON/YAML unit test file used for all constraints test suite management. This version documents the model of that YAML file before the proposed change in the comments of the #817 PR, linked below. #817 (comment) * [skip ci] Document unit test metaschema elements * [skip ci] Do not explicitly use default types * [skip ci] Add missing `@id` and `@level` to model * [skip ci] Add pipeline to pipeline, thx @wandmagic * [skip ci] Backport #219 docs into model remarks * [skip ci] Fix max-occurs re bad model cardinality This addresses changes that should be handled as determined by upstream guidance from metaschema-framework maintainers. metaschema-framework/metaschema-java#221 metaschema-framework/metaschema#44 * [skip ci] Is a pipeline with a ref even a pipeline at all? Thanks to @Rene2mt for PR feedback about the missing ref to the assembly definition of pipeline in the model. Co-authored-by: Rene Tshiteya <rene-claude.tshiteya@gsa.gov> --------- Co-authored-by: Rene Tshiteya <rene-claude.tshiteya@gsa.gov>

@wandmagic

* [skip ci] Metaschema module for unit test struct We are proposing changes to the structure of the JSON/YAML unit test file used for all constraints test suite management. This version documents the model of that YAML file before the proposed change in the comments of the GSA#817 PR, linked below. GSA#817 (comment) * [skip ci] Document unit test metaschema elements * [skip ci] Do not explicitly use default types * [skip ci] Add missing `@id` and `@level` to model * [skip ci] Add pipeline to pipeline, thx @wandmagic * [skip ci] Backport GSA#219 docs into model remarks * [skip ci] Fix max-occurs re bad model cardinality This addresses changes that should be handled as determined by upstream guidance from metaschema-framework maintainers. metaschema-framework/metaschema-java#221 metaschema-framework/metaschema#44 * [skip ci] Is a pipeline with a ref even a pipeline at all? Thanks to @Rene2mt for PR feedback about the missing ref to the assembly definition of pipeline in the model. Co-authored-by: Rene Tshiteya <rene-claude.tshiteya@gsa.gov> --------- Co-authored-by: Rene Tshiteya <rene-claude.tshiteya@gsa.gov>

aj-stein-gsa added the bug Something isn't working label Oct 28, 2024

david-waltermire added a commit that referenced this issue Oct 28, 2024

Improved error handling to better report errors such as in #221.

9baaac0

aj-stein-gsa added a commit to aj-stein-gsa/metaschema that referenced this issue Oct 29, 2024

Add a constraint for #221 to error on bad group-as

9cb705d

This new constraint implements a check for the following issue comment. metaschema-framework/metaschema-java#221 (comment)

aj-stein-gsa mentioned this issue Oct 29, 2024

Constraint to error on check group-as/@in-json with parent @max-occurs metaschema-framework/metaschema#44

Open

8 tasks

aj-stein-gsa mentioned this issue Oct 29, 2024

Add unit test metaschema conformant with ADR#6 GSA/fedramp-automation#826

Merged

5 tasks

david-waltermire added this to Tooling Work Board Oct 29, 2024

github-project-automation bot moved this to To Triage in Tooling Work Board Oct 29, 2024

david-waltermire moved this from To Triage to Backlog in Tooling Work Board Oct 29, 2024

david-waltermire moved this from Backlog to In progress in Tooling Work Board Oct 29, 2024

david-waltermire linked a pull request Oct 29, 2024 that will close this issue

Improve NULL field value and data type error handling #227

Merged

8 tasks

david-waltermire moved this from In progress to In review in Tooling Work Board Oct 29, 2024

david-waltermire closed this as completed Oct 30, 2024

github-project-automation bot moved this from In review to Done in Tooling Work Board Oct 30, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unable to parse text exception with in models with `group-as/@in-json="SINGLETON_OR_ARRAY"` behavior #221

Unable to parse text exception with in models with `group-as/@in-json="SINGLETON_OR_ARRAY"` behavior #221

aj-stein-gsa commented Oct 28, 2024 •

edited

Loading

aj-stein-gsa commented Oct 28, 2024

Unable to parse text exception with in models with group-as/@in-json="SINGLETON_OR_ARRAY" behavior #221

Unable to parse text exception with in models with group-as/@in-json="SINGLETON_OR_ARRAY" behavior #221

Comments

aj-stein-gsa commented Oct 28, 2024 • edited Loading

Describe the bug

Who is the bug affecting

How do we replicate this issue

Expected behavior (i.e. solution)

Other comments

aj-stein-gsa commented Oct 28, 2024

Unable to parse text exception with in models with `group-as/@in-json="SINGLETON_OR_ARRAY"` behavior #221

Unable to parse text exception with in models with `group-as/@in-json="SINGLETON_OR_ARRAY"` behavior #221

aj-stein-gsa commented Oct 28, 2024 •

edited

Loading