Validate ontologies loaded from the triplestore

[benjamingeer]

Checks on classes

• Within the cardinalities of a class, there must be a link value property for each link property and vice versa.
• Each class must be a subclass of all the classes that are subject class constraints of the properties in its cardinalities.

If a class is defined in a project-specific ontology:

• It must be either a resource class or a standoff class, but not both (note that this forbids project-specific subclasses of knora-base:Value).
• All its cardinalities must be on properties that are defined in the triplestore.
• If it's a resource class, all its directly defined cardinalities must be on Knora resource properties (subproperties of knora-base:hasValue or knora-base:hasLinkTo), and all its base classes with Knora IRIs must also be resource classes. A cardinality on knora-base:resourceProperty or knora-base:hasValue is forbidden. It must also have an rdfs:label.
• If it's a standoff class, none of its cardinalities may be on Knora resource properties, and all its base classes with Knora IRIs must also be standoff classes.

Checks on properties

• The property's subject class constraint, if provided, must be a Knora class, owl:Class, or owl:Restriction, and must be a subclass of the subject class constraints of all its base properties.
• The property's object class constraint, if provided, must be a subclass of the object class constraints of all its base properties.
• If the property is a Knora resource property, it must have an object class constraint and an rdfs:label.

If a property is defined in a project-specific ontology:

• Its subject class constraint, if provided, must be a Knora resource or standoff class.

If a property is defined in a project-specific ontology and is a Knora resource property:

• It can't be a subproperty of both knora-base:hasValue and knora-base:hasLinkTo.
• It can't be a subproperty of knora-base:hasFileValue.
• Each of its base properties that has a Knora IRI must also be a Knora resource property.

Other checks

• The predicate salsah-gui:guiOrder is allowed in an owl:Restriction, not in a property definition.
• The predicates salsah-gui:guiElement and salsah-gui:guiAttribute are allowed in a property definition, not in an owl:Restriction.

Other tasks

• Get rid of the redundant predicate knora-api:projectIri, use knora-api:attachedToProject instead.
• Fix ontology errors found by these additional checks.
• Add tests that load invalid ontologies from the triplestore.
• Check that SALSAH 1 e2e tests pass.
• Update docs.
• Update release notes.

Resolves #778.

[benjamingeer]

@subotic Cool, that fixed it, thanks. :)

[subotic]

you're welcome :-)

[benjamingeer]

@loicjaouen @mrivoal The ontology verification introduced in this PR found some problems in drawings-gods_ontology.ttl:

• The class drawings-gods:PersonLinkedFile has a cardinality for drawings-gods:hasPersonLinkedFileComment, but that property had a knora-base:subjectClassConstraint of drawings-gods:Person. I've changed its subject class constraint to drawings-gods:PersonLinkedFile.
• drawings-gods:QuestionnaireIndividual had cardinalities for two nonexistent properties: drawings-gods:ifIToldYouGodCouldYouDrawDoNotKnow and drawings-gods:otherLanguages3. I've removed both cardinalities.

Please check whether these fixes are correct.

Also, in this PR I've upgraded the RDF4J Turtle library to 2.3.0, so the output of TransformOntology (which moves salsah-gui:guiOrder to the right place) now contains anonymous blank nodes for cardinalities. I've used it to reformat drawings-gods_ontology.ttl.

[benjamingeer]

@tobiasschweizer @SepidehAlassi The ontology verification introduced in this PR found a typo in biblio-onto.ttl:

https://github.com/dhlab-basel/Knora/pull/806/files#diff-d4765708f69ccf775ec8c8803fbc1a24

[tobiasschweizer]

@tobiasschweizer @SepidehAlassi The ontology verification introduced in this PR found a typo in biblio-onto.ttl:

Great job! Thanks for fixing this!

[janCstoffregen]

Dear dhlab,
I'm not sure how many necessary changes this will cause in our ontology. Are the error messages of those checks written in a way that we will understand what is not working in our ontology when the time comes?

As you know, we have quite a bit of ontologies already here: https://github.com/nie-ine/Ontologies/tree/master/nie-ontologies
Those ontologies work with an older version of Knora, we can import them locally in GraphDB and display them in Salsah. Is there a smart way to perform the necessary changes, maybe in a certain order? Any advice would be much appreciated since this looks like it will be a time intensive procedure for us.

Also, we wonder which goals the restrictions have, are the changes written to improve the performance or for another reason?

Thank you so much for your answer.
Best regards,
Jan

[benjamingeer]

Are the error messages of those checks written in a way that we will understand what is not working in our ontology when the time comes?

Yes, that's the intention.

we wonder which goals the restrictions have

The goal is to prevent errors. None of these restrictions are new. I think all of them are already described in the knora-base ontology documentation:

http://www.knora.org/documentation/manual/rst/02-knora-ontologies/knora-base.html

Please read that document if you haven't already.

If a project-specific ontology doesn't conform to the requirements of knora-base, the Knora API server won't be able to use it correctly.

Currently it's possible to violate some of these restrictions without getting an error message, but you will get unexpected results. The purpose of this PR is to have all such violations cause error messages on startup.

[janCstoffregen]

ok, thank you for this clarification, I did not realise that there are no new restrictions in this pr.

[benjamingeer]

I did not realise that there are no new restrictions in this pr.

I’ll double-check the docs and document any restrictions that weren’t documented before.

[mrivoal]

@benjamingeer, yes, your changes to drawings-gods_ontology.ttl are correct, thanks!
Glad to see that such mistakes can now be pointed out!

[SepidehAlassi]

@benjamingeer wow, I am impressed!

[benjamingeer]

@subotic I'll be on holiday next week, so there's no need to hurry to review this. :)

[subotic]

'A description of an institution'

[benjamingeer]

Fixed.

[subotic]

you are fast :-)

[subotic]

As a tip, you could write

expectMsgPF(timeout) {
                 case msg: akka.actor.Status.Failure => msg.cause.isInstanceOf[InconsistentTriplestoreDataException] should ===(true)
             }

as

expectMsgType[akka.actor.Status.Failure](timeout).cause should be (true)

expectMsgPF is useful when you expect more than one message type.

[benjamingeer]

Thanks, I'll do that next time. :)

[subotic]

Is there a test, that loads an ontology with a missing knora-base:attachedToProject?

[benjamingeer]

Good idea, I'll add one now.

[subotic]

Thanks :-)

[benjamingeer]

Thanks for the review! It's a relief to finally see this done.

[subotic]

Thanks for doing it :-) It's a huge thing.