Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create V2 for Search #478

Merged
merged 185 commits into from
Jul 25, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
185 commits
Select commit Hold shift + click to select a range
28af692
refactor (move Responder): move Responder up the directory tree to be…
Mar 27, 2017
847973d
feature (add support for V2): create responder manager for V2
Mar 27, 2017
8dc2ef2
Merge branch 'develop' into wip/design-V2-search
tobiasschweizer Mar 27, 2017
7e211b5
refactor (optimize imports): remove unnecessary import statement
Mar 27, 2017
1cc6813
feature (search v2): use SPARQL construct query to perform a fulltext…
Mar 28, 2017
8ff99c2
feature (search V2): return results
Mar 28, 2017
ba3e110
feature (generate JSON-LD from case class): ongoing ...
Mar 28, 2017
e4871e9
feature (generate JSON-LD from case class) ongoing ...
Mar 28, 2017
d70a4c5
feature (search results as JSON-LD): return a list of values for each…
Mar 28, 2017
cfa2e74
feature (JSON-LD): add more members to search response
Mar 28, 2017
482da87
feature (search results as JSON-LD): show information about value obj…
Mar 29, 2017
943b0ae
refactor (fulltext search): rename variables in SPARQL query
Mar 29, 2017
a2a30f9
feature (add resources route V2): ongoing ...
Mar 29, 2017
491874f
feature (v2 resource query): add SPARQL for v2 resource query
Mar 29, 2017
e88b98f
refactor (add util for construct query results): try to process searc…
Mar 29, 2017
a01d41d
Merge branch 'develop' into wip/design-V2-search
tobiasschweizer Mar 29, 2017
d6071ec
refactor (util for construct query v2): add utility function to get a…
Mar 29, 2017
ec6c051
feature (resource request v2): try to reuse methods needed for search
Mar 29, 2017
bdab263
refactor (unify response messages for v2): make a generic resource re…
Mar 29, 2017
7f5b7ff
feature (typed literals): represent value literals in a type specific…
Mar 30, 2017
c587705
feature (value literal types): support more literal value types
Mar 30, 2017
f8f3859
refactor (rename v2 response messages components)
Mar 30, 2017
1a480b3
refactor (V2 case class): try to make case classes response format in…
Apr 3, 2017
25cb869
refactor (v2 case classes): convert Sparql results to generic case cl…
Apr 3, 2017
b859870
refactor (v2 case classes): try to reuse the same case class componen…
Apr 3, 2017
6055405
refactor (v2 response format): convert value types to JSON
Apr 3, 2017
d63393d
refactor (restructure v2 case class): reuse v2 case classes in differ…
Apr 3, 2017
87cc6d6
refactor (v2 resource response): include standoff in search results
Apr 4, 2017
92917a2
refactor (v2 sparql results): separate resources from standoff nodes
Apr 4, 2017
9ae5f03
refactor (v2 resource query): handle value props and linking props se…
Apr 4, 2017
cd2ddb4
refactor (v2): split results (ongoing ...)
Apr 4, 2017
d0481b7
refactor (v2 response): create case classes from CONSTRUCT query resp…
Apr 5, 2017
247728b
refactor (v2 results): unify the creation of responses to the highest…
Apr 5, 2017
1e24134
refactor (v2 response): include information about a referred resource…
Apr 5, 2017
e81a5c1
refactor (v2 Sparql results): handle Sparql Construct results in a ge…
Apr 6, 2017
c59094a
refactor (construct response for v2): add more documentation and make…
Apr 6, 2017
01e254b
Merge branch 'develop' into wip/design-V2-search
tobiasschweizer Apr 6, 2017
e1a0f08
feature (v2 date): represent a JDN in an understandable string repres…
Apr 6, 2017
30008ed
refactor (v2 fulltext search): return the values of a value object
Apr 10, 2017
61bdccc
refactor (v2 sparql queries): use UNION instead of OPTIONAL
Apr 10, 2017
2f3db02
refactor (unfify standoff handling): put the creation of standoff nod…
Apr 10, 2017
355411b
refactor (fix some minor inconsistencies)
Apr 11, 2017
922f1b4
refactor (reapply some changes from another branch): improve handling…
Apr 11, 2017
dfbb746
feature (default XSL transformation): add an optional default XSL tra…
Apr 11, 2017
53b126c
refactor (v2 handling of mapping): represent mapping and default XSL …
Apr 11, 2017
f0a9e65
feature (apply XSL transformation to XML): convert XML before returni…
Apr 12, 2017
ec363ec
feature (v2 permission check): check for permissions on resources and…
Apr 12, 2017
e7d871f
fix (set cookie for all paths): set path to root to make cookie valid…
Apr 12, 2017
098fcec
feature (limit query results in fulltext search): use a SELECT subque…
Apr 18, 2017
1ba1508
refactor (fix typo)
Apr 19, 2017
0e8b4e1
fix (authentication test): fix test (cookie path)
Apr 19, 2017
1684f1a
refactor (webapi): Rename some V2 classes for clarity.
Apr 19, 2017
fe81c7f
refactor (v2 fulltext search): do not query standoff nodes
Apr 20, 2017
355b949
fix (bloody authentication test): set absolute cookie path
Apr 20, 2017
4209c31
Merge branch 'develop' into wip/design-V2-search
Apr 20, 2017
41aaf38
refactor (webapi): Rename ValueV2 -> IOValueV2 for clarity.
Apr 20, 2017
a73cae9
feature (knora api ontology): add more properties for value types (on…
Apr 20, 2017
15233a7
Merge branch 'develop' into wip/design-V2-search
Apr 20, 2017
9c43f1e
refactor (api ontology): define knora api entities (ongoing)
Apr 20, 2017
77207c7
refactor (v2 knora api ontology): ongoing
Apr 20, 2017
7a9fcca
feature (webapi): Add JSON-LD library.
Apr 20, 2017
758c2f9
feature (knora api date handling) add utility methods (ongoing)
Apr 20, 2017
1189e42
feature (knora api values): convert a knora-base date to a knora-api …
Apr 21, 2017
7bb4439
feature (knora api values): convert a knora-base text value to a knor…
Apr 21, 2017
5d0d37a
feature (knora api values): integer and decimal value types
Apr 21, 2017
b420cda
refactor (knora api ontology): link value
Apr 21, 2017
c735b4c
refactor (knora-api values): handle link values (recursion not suppor…
Apr 21, 2017
a03c95b
Merge branch 'develop' into wip/design-V2-search
tobiasschweizer Apr 21, 2017
0838d48
feature (v2: flag resources): indicate if a resource is the main reso…
Apr 21, 2017
965a266
Merge branch 'develop' into wip/design-V2-search
tobiasschweizer Apr 21, 2017
f38d3e8
Merge branch 'develop' into wip/design-V2-search
tobiasschweizer Apr 24, 2017
eb1182e
refactor (v2 api response): check for isMainResource
Apr 24, 2017
fe0343f
fix (value content v2 case classes): fix wrong constant
Apr 24, 2017
48e034b
feature (v2 knora api): support still image file values
Apr 24, 2017
9aa8ad5
Merge branch 'develop' into wip/design-V2-search
tobiasschweizer Apr 24, 2017
d0e4529
refactor (webapi): Refactor code for processing resource with values …
Apr 24, 2017
2f59301
feature (values v2): add missing value types
Apr 24, 2017
cdef17a
feature (v2 knora api): add more value types to ontology
Apr 25, 2017
0ef3927
feature (knora api v2): construct missing ValueContentV2 case classes…
Apr 25, 2017
82fabf0
feature (handling of referred resources): follow links recursively
Apr 25, 2017
700fbad
feature (referred resources): represent referred resources as nested …
Apr 25, 2017
a566d16
refactor (referred resources): follow references to resources when tu…
Apr 25, 2017
0e68841
refactor (resources route): accept a list of resource Iris
Apr 26, 2017
99ef1b5
feature (ontologies responder v2): prepare the ontologies responder v2
Apr 26, 2017
a17477b
fix (webapi): Don’t follow link cycles in an infinite loop.
Apr 26, 2017
85e8ea8
refactor (ontology responder v2): make ontology responder v1 call ont…
Apr 26, 2017
c09964b
refactor (ontology responder v2): move subclass checks to V2 responder
Apr 27, 2017
59ccd16
feature (api v2): provide information for IIIF image API client
Apr 28, 2017
784fa45
refactor (unify responder manager): unify v1 and v2 responder manager
May 2, 2017
f969b19
refactor (consistency of ontology responder v2): retun v2 messages
May 2, 2017
967676c
refactor (ontology information v2 JSON-LD): return an object for reso…
May 5, 2017
48841f4
Merge branch 'develop' into wip/design-V2-search
tobiasschweizer May 5, 2017
a54d16d
feature (ontology responder v2): get property definitions v2
May 5, 2017
3ace169
refactor (named graph v2): get entity definitions for named graphs
May 5, 2017
e2a2281
Merge branch 'develop' into wip/design-V2-search
tobiasschweizer May 7, 2017
3916feb
feature (ontologies v2): return Iris of existing named graphs
May 8, 2017
52f3d51
feature (JSONLD lib): use JSONLD library to construct response
May 9, 2017
45bd313
feature (fallback language): use a preferred and a fallback language …
May 9, 2017
5212f74
Merge branch 'develop' into wip/design-V2-search
May 9, 2017
923fb04
refactor (knora-api ontology): use consistent naming schema for knora…
May 10, 2017
8a0fdbb
Merge branch 'develop' into wip/design-V2-search
tobiasschweizer May 12, 2017
925d0c7
fix (typo): remove superfluous #
May 22, 2017
7abedb3
ben (apply changes from bulk import branch): apply refactoring of Ont…
May 22, 2017
b467215
refactor (knora api constants): stick to new refactoring scheme
May 22, 2017
8c6e8a9
refactor (expose only knora-api Iris to the outside): translate inter…
May 23, 2017
a2379d3
feature (external to internal Iris): convert external ontology and en…
May 24, 2017
43a58a5
feature (provide default XSL transformation): define a default XSL tr…
May 29, 2017
4750a42
refactor (increase timeout): increase timeout when connecting to the …
Jun 1, 2017
130e365
Merge branch 'develop' into wip/design-V2-search
tobiasschweizer Jun 1, 2017
c9a2d75
Merge branch 'develop' into wip/design-V2-search
tobiasschweizer Jun 1, 2017
a9b6780
Merge branch 'develop' into wip/design-V2-search
Jun 1, 2017
15dbfdb
feature (search resources by label): search for resources by their label
Jun 5, 2017
57d0879
feature (webapi): Start proof-of-concept of #530.
Jun 6, 2017
7f584d0
feature (resource preview V2): get the preview of a resource
Jun 7, 2017
f42fb5c
feature (webapi): Start making a data structure to represent a simple…
Jun 7, 2017
d4f384a
feature (webapi): Parse simple SPARQL CONSTRUCT queries for extended …
Jun 7, 2017
7515ae5
refactor (webapi): Remove unused ANTLR grammar and library.
Jun 9, 2017
eb5111c
test: Add more tests for SearchParserV2.
Jun 9, 2017
f772212
test: Add tests for rejecting nested UNIONs in a SPARQL search query.
Jun 9, 2017
f3b4f1e
test: Add a test for rejecting an OPTIONAL in a search query.
Jun 9, 2017
d989bdf
test (add test case for query parser): CONSTRUCT query is not handled…
Jun 14, 2017
5d8ecc3
feature (webapi): Fix parsing bug in SearchParserV2.
Jun 14, 2017
78dccc2
tests (add Sparql parser test): add test for Sparql parser
Jun 16, 2017
5c10d4f
Merge branch 'develop' into wip/design-V2-search
tobiasschweizer Jun 16, 2017
6845fb9
refactor (remove docs for inexistent parameter)
Jun 16, 2017
693d49a
feature (extended search): prepare extended search
Jun 19, 2017
6c6b151
Merge branch 'develop' into wip/design-V2-search
tobiasschweizer Jun 19, 2017
852a32b
refactor (simplified sparql): allow for Boolean operators in FILTER
Jun 19, 2017
b23a59e
Merge branch 'develop' into wip/design-V2-search
tobiasschweizer Jun 20, 2017
8d11387
refactor (rename regex)
Jun 20, 2017
4ba4834
refactor (SearchParserSpec): adapt test for Boolean operator in test
Jun 20, 2017
bd66f50
feature (webapi): Support BIND in a CONSTRUCT query for extended search.
Jun 20, 2017
c8bfb79
feature (convert external Iris to internal Iris): handle external Iri…
Jun 20, 2017
c1db077
feature (webapi): Add ontology entities and SPARQL for key-value stor…
Jun 21, 2017
df0bbda
feature (webapi): Start implementing persistent maps.
Jun 21, 2017
1ce0aea
refactor (webapi): Keep the messages.store package, just not in v1.
Jun 21, 2017
3a97921
refactor (webapi): Rename MapMessagesV2.scala -> PersistentMapMessage…
Jun 21, 2017
ccdd0bb
feature (extended search): prepare extended search with user provided…
Jun 22, 2017
393ff19
refactor (settings): pass settings obj in the route
Jun 22, 2017
61f8390
feature (sparql processing for extended search) ongoing fight ...
Jun 22, 2017
445a136
feature (extended search): provide more sample sparql queries
Jun 23, 2017
35ad691
feature (extended search): automatically create additional Sparql sta…
Jun 26, 2017
c886fd0
feature (webapi): Implement PersistentMapResponderV2 (not tested yet).
Jun 26, 2017
97e258f
feature (webapi): Inspect search SPARQL to extract type information.
Jun 27, 2017
2e7f5ff
feature (webapi): Remove the type annotations from a search WHERE cla…
Jun 27, 2017
e45faf5
feature (extended search): crazy stuff at the end of a long workday
Jun 27, 2017
f0a4399
feature (webapi): fix type inspector Scaladoc.
Jun 27, 2017
0ee22b2
feature (extended search): process statements and add more statements…
Jun 28, 2017
7f9ee5c
refactor (extended search): disable inference in the Twirl template
Jun 28, 2017
bc2e66e
refactor (extended search): create additional statements from type in…
Jun 28, 2017
c313045
refactor (extended search): handle property and non property types in…
Jun 29, 2017
6e5a369
feature (extended search): handle value props (string)
Jun 29, 2017
dfd4dd9
Merge branch 'develop' into wip/design-V2-search
Jun 30, 2017
c7b103b
Merge branch 'develop' into wip/design-V2-search
tobiasschweizer Jun 30, 2017
5b7848e
feature (extended search): keep original where patterns and additiona…
Jul 3, 2017
9c401bd
refactor (extended search): adapt sample query
Jul 3, 2017
c85a4e1
feature (extended search): transform FILTER patterns
Jul 3, 2017
8d1c846
refactor (extended search): accept url encoded string as Sparql query
Jul 4, 2017
5f8536a
refactor (extended search): make some clarifications
Jul 4, 2017
f6af2ca
refactor (extended search): improve separation of statements provided…
Jul 5, 2017
efd37f0
feature (extended search): check for isDeleted flag
Jul 5, 2017
7b70c97
feature (extended search): support file and geom values
Jul 6, 2017
be6f2b5
feature (extended search): find out if predicate variables represent …
Jul 6, 2017
cdcb301
feature (extended search): handle color values, start Date handling
Jul 6, 2017
cd8d85e
refactor (provide assertions and add comments)
Jul 7, 2017
123c2a6
Merge branch 'develop' into wip/design-V2-search
Jul 18, 2017
c678efb
Merge branch 'develop' into wip/design-V2-search
tobiasschweizer Jul 18, 2017
a58ac81
refactor (webapi): Remove unused method.
Jul 18, 2017
e4d6a1c
feature (webapi): Add missing case in receive method.
Jul 18, 2017
0fe4246
refactor (webapi): Use a consistent naming convention for Akka router…
Jul 18, 2017
57f7b03
refactor (webapi): Use java.time.Instant for timestamps instead of ja…
Jul 19, 2017
0cab86c
feature (extended search V2): add more comparison operators to search…
Jul 19, 2017
d66377c
Merge branch 'develop' into wip/design-V2-search
tobiasschweizer Jul 20, 2017
788f120
Merge branch 'develop' into wip/design-V2-search
tobiasschweizer Jul 20, 2017
59fbdea
tests (add tests for extended search v2): tests for user provided Sp…
Jul 21, 2017
c436965
tests (extended search v2): add tests for date values
Jul 21, 2017
aeb27ff
refactor (remove redundant tests): remove redundant tests
Jul 21, 2017
fc59aed
feature (extended search v2): support Filter expressions with a logic…
Jul 24, 2017
fdd8bac
refactor (extended search v2): add assertions
Jul 24, 2017
2ca7aa1
Merge branch 'develop' into wip/design-V2-search
tobiasschweizer Jul 24, 2017
a348c34
test: Add tests for PersistentMapResponderV2.
Jul 24, 2017
f86f4e3
test: Add copyright.
Jul 24, 2017
79e251d
fix (webapi): Fix SPARQL bugs in persistent map code.
Jul 24, 2017
6331c9b
feature (extended search v2): support decimal and Boolean values
Jul 25, 2017
8804a4d
test: Disable v2 search tests for now, because they don't yet work wi…
Jul 25, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/rst/knora-api-server/api_v1/create-a-mapping.rst
Original file line number Diff line number Diff line change
Expand Up @@ -59,13 +59,16 @@ Basically, a mapping expresses the relations between XML elements and attributes
The relations expressed in a mapping are one-to-one relations, so the XML can be recreated from the data in RDF. However, since HTML offers a very limited set of elements, Knora mappings support the combination of element names
and classes. In this way, the same element can be used several times in combination with another classname (please note that ``<a>`` without a class is a mere hyperlink whereas ``<a class="salsah-link">`` is an internal link/standoff link).

With a mapping, a default XSL transformation may be provided to transform the XML to HTML before sending it back to the client. This is useful when the client is a web-browser expecting HTML (instead of XML).

----------------------------
Basic Structure of a Mapping
----------------------------

The mapping is written in XML itself (for a formal description, see ``webapi/src/resources/mappingXMLToStandoff.xsd``). It has the following structure (the indentation corresponds to the nesting in XML):

- ``<mapping>``: the root element
- ``<defaultXSLTransformation> (optional)``: the Iri of the default XSL transformation to be applied to the XML when reading it back from Knora. The XSL transformation is expected to produce HTML. If given, the Iri has to refer to a resource of type ``knora-base:XSLTransformation``.
- ``<mappingElement>``: an element of the mapping (at least one)
- ``<tag>``: information about the XML element that is mapped to a standoff class
- ``<name>``: name of the XML element
Expand All @@ -86,6 +89,7 @@ The mapping is written in XML itself (for a formal description, see ``webapi/src
XML structure of a mapping::

<mapping>
<defaultXSLTransformation>Iri of a knora-base:XSLTransformation</defaultXSLTransformation>
<mappingElement>
<tag>
<name>XML element name</name>
Expand Down
2 changes: 1 addition & 1 deletion docs/rst/knora-ontologies/knora-base.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1216,7 +1216,7 @@ has permission to see the source and target resources.
Internal Links in a TextValue
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Internal links in a ``TextValue`` can be using the data type standoff class ``StandoffInternalReferenceTag`` or a subclass of it.
Internal links in a ``TextValue`` can be represented using the data type standoff class ``StandoffInternalReferenceTag`` or a subclass of it.
It has the following property:

``standoffTagHasInternalReference`` (1)
Expand Down
72 changes: 72 additions & 0 deletions knora-ontologies/knora-admin.ttl
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,43 @@



:isInMap rdf:type owl:ObjectProperty ;

rdfs:subPropertyOf :objectCannotBeMarkedAsDeleted ;

:subjectClassConstraint :MapEntry;

:objectClassConstraint :Map ;

rdfs:comment "Associates a MapEntry with a Map"@en .



:mapEntryKey rdf:type owl:DatatypeProperty ;

rdfs:subPropertyOf :objectCannotBeMarkedAsDeleted ;

:subjectClassConstraint :MapEntry;

:objectDatatypeConstraint xsd:string ;

rdfs:comment "Represents the key of a MapEntry"@en .



:mapEntryValue rdf:type owl:DatatypeProperty ;

rdfs:subPropertyOf :objectCannotBeMarkedAsDeleted ;

:subjectClassConstraint :MapEntry;

:objectDatatypeConstraint xsd:string ;

rdfs:comment "Represents the value of a MapEntry"@en .




#################################################################
#
# Datatype properties
Expand Down Expand Up @@ -597,6 +634,41 @@



:Map rdf:type owl:Class ;

rdfs:subClassOf [ rdf:type owl:Restriction ;
owl:onProperty :lastModificationDate ;
owl:maxCardinality "1"^^xsd:nonNegativeInteger
] ;

rdfs:comment "Represents a simple map of key-value pairs"@en .



:MapEntry rdf:type owl:Class ;

rdfs:subClassOf [ rdf:type owl:Restriction ;
owl:onProperty :isInMap ;
owl:cardinality "1"^^xsd:nonNegativeInteger
],
[ rdf:type owl:Restriction ;
owl:onProperty :mapEntryKey ;
owl:cardinality "1"^^xsd:nonNegativeInteger
],
[ rdf:type owl:Restriction ;
owl:onProperty :mapEntryValue ;
owl:cardinality "1"^^xsd:nonNegativeInteger
],
[ rdf:type owl:Restriction ;
owl:onProperty :lastModificationDate ;
owl:cardinality "1"^^xsd:nonNegativeInteger
];

rdfs:comment "Represents a key-value pair in a Map"@en .




#################################################################
#
# Individuals
Expand Down
Loading