feature (webapi): make project shortcode required #824

subotic · 2018-04-12T16:58:04Z

Summary

make project shortcode required
add shortcode to projects that did not have one up until now (incunabula, biblio and anything)
changed dokubib shortcode to one from the production range as these projects are going to be added in production
had to revert some changes in 7687651 and 90a68aa to make it work again. These changes are backed by two new tests. Maybe you could take a look to see if I understood it correctly and that the tests are correct.
change all data.knora.org to rdfh.ch

Issues

resolves Making Project-Shortcode required #787
resolves Convert existing ontology names to NCNames #667

…ject shortcodes.

…tter. # Conflicts: # webapi/src/main/scala/org/knora/webapi/responders/v2/OntologyResponderV2.scala

subotic · 2018-04-13T15:02:50Z

@benjamingeer so this is more work then I anticipated.

I'm not sure if I have broken something or if simply the expected result needs to be changed.

Test output of JSONLDHandlingV2R2RSpec.scala:

Map("rdf" -> "http://www.w3.org/1999/02/22-rdf-syntax-ns#", "knora-api" -> "http://api.knora.org/ontology/knora-api/v2#", "schema" -> "http://schema.org/", "rdfs" -> "http://www.w3.org/2000/01/rdf-schema#", "p0803-incunabula" -> "http://0.0.0.0:3333/ontology/0803/incunabula/v2#") did not equal Map("rdf" -> "http://www.w3.org/1999/02/22-rdf-syntax-ns#", "knora-api" -> "http://api.knora.org/ontology/knora-api/v2#", "schema" -> "http://schema.org/", "rdfs" -> "http://www.w3.org/2000/01/rdf-schema#", "incunabula" -> "http://0.0.0.0:3333/ontology/0803/incunabula/v2#") @context incorrect (JSONLDHandlingV2R2RSpec.scala:113)

In particular, "p0803-incunabula" -> "http://0.0.0.0:3333/ontology/0803/incunabula/v2#" vs "incunabula" -> "http://0.0.0.0:3333/ontology/0803/incunabula/v2#". Is p0803-incunabula the correct result?

benjamingeer · 2018-04-13T15:04:58Z

p0803-incunabula is correct. StringFormatter makes prefix labels like this because there might be two different ontologies called incunabula in different projects.

benjamingeer · 2018-04-13T15:08:08Z

Does this branch include the commits I did to require project shortcodes in #788?

bb930c4

3698588

benjamingeer · 2018-04-13T15:08:33Z

Never mind, I just saw that it does include them. :)

subotic · 2018-04-13T15:17:19Z

thanks :-)

# Conflicts: # webapi/src/test/scala/org/knora/webapi/responders/v2/OntologyResponderV2Spec.scala

subotic · 2018-04-24T08:34:06Z

@benjamingeer whenever you have some spare time :-)

benjamingeer · 2018-04-24T08:44:13Z

webapi/src/main/scala/org/knora/webapi/util/KnoraIdUtil.scala

-        } else {
-            s"http://$IriDomain/${projectInfo.shortname}/$knoraResourceID"
-        }
+        s"http://$IriDomain/${projectInfo.shortcode}/${projectInfo.shortname}/$knoraResourceID"


Why include both the project shortcode and the project shortname?

Good catch. Only the shortcode would be more than enough.

benjamingeer · 2018-04-24T08:44:59Z

webapi/_test_data/all_data/anything-data.ttl


-<http://data.knora.org/0C-0L1kORryKzJAJxxRyRQ> a anything:Thing ;
+<http://rdfh.ch/0C-0L1kORryKzJAJxxRyRQ> a anything:Thing ;


If you really want to make this standard, it would have to include the project shortcode as in KnoraIdUtil.makeRandomResourceIri.

yes, you are right.

if we have:

lists: `http://rdfh.ch/lists/0001/xxxx

users: `http://rdfh.ch/users/xxxxxxx

projects: `http://rdfh.ch/projects/0001

shouldn't we have for resources: http://rdfh.ch/resources/0001/xxx`?

but this is another discussion. I will do it as described here

That is how we formatted our data too, so, so far, our prod data is compliant to the doc.

But our (Lausanne) next upgrade will be a big one, so I would be in favour to push as much changes as we want to push now and have smoother releases in the future.

Did you want to do this for incunabula-data.ttl, too?

Oh, please no. This will take too long. The anything data is small, while incunabula is huge. I need to finish some things for Lukas before my vacation next week and after I'm back, prepare the production server, so I won't be working on Knora until June.

No problem, I was just wondering how masochistic you were feeling. :)

benjamingeer · 2018-04-24T08:52:39Z

webapi/_test_data/all_data/admin-data.ttl

@@ -413,7 +416,7 @@ Die Internetpublikation macht das digitalisierte Korpus dieser Frühdrucke  durc
 # BEOL                            #
 ###################################

-<http://rdfh.ch/projects/0801> a knora-base:knoraProject ;
+<http://rdfh.ch/projects/yTerZGyxjZVqFMNNKXCDPF> a knora-base:knoraProject ;


Why does the BEOL project have a UUID in its IRI, while all the other projects have shortcodes in their IRIs?

Because then I would also need to change the data on the production server, which I'm reluctant to do at the moment.

OK, no problem.

benjamingeer · 2018-04-24T10:41:04Z

webapi/src/main/scala/org/knora/webapi/util/StringFormatter.scala

@@ -1779,7 +1781,7 @@ class StringFormatter private(val knoraApiHostAndPort: Option[String]) {
      */
    def xmlImportNamespaceToInternalOntologyIriV1(namespace: String, errorFun: => Nothing): SmartIri = {
        namespace match {
-            case ProjectSpecificXmlImportNamespaceRegex(_, Optional(projectCode), ontologyName) if !isBuiltInOntologyName(ontologyName) =>
+            case ProjectSpecificXmlImportNamespaceRegex(projectCode, ontologyName) if !isBuiltInOntologyName(ontologyName) =>


Sorry, this was my bug here, fixed now.

benjamingeer · 2018-04-24T10:49:36Z

had to revert some changes in 7687651 and 90a68aa to make it work again

These were both my mistakes, sorry:

The code that uses the regex that matches project-specific API v1 XML import namespaces was incorrect (it expected the wrong number of capturing groups). I've fixed that, and made the project shortcode required in the regex again, and it works now.

The regex that matches API v2 ontology URL paths was incorrect because it can't require the shortcode (which I mistakenly made it do), because it handles built-in ontologies as well as project-specific ones, and built-in ontologies don't have shortcodes.

Thanks for the tests. :)

subotic · 2018-04-24T14:45:14Z

Thanks for adding the fix :-)

benjamingeer · 2018-04-26T13:24:09Z

This was really a lot of work, thanks for being patient enough to do it.

subotic · 2018-04-26T14:23:48Z

thanks :-)

* develop: feature (webapi): make project shortcode required (#824) doc (webapi): jekyll / paradox based documentation website (#751) Validate ontologies loaded from the triplestore (#806) fix (webapi): add searchbox gui element to hasOtherSomething property (#828) feature (webapi): allow loading data into graphdb on windows (internal PR) (#814) build (webapi): add missing library (#822) feature (webapi): allow changing of user’s password by system admin (#815) feature (webapi): OpenAPI / Swagger API documentation generation (#812)

subotic and others added 8 commits April 11, 2018 16:14

feature (webapi): add required project shortcode

1985dd5

feature (webapi): Reject project-specific definition IRIs without pro…

f1a7f32

…ject shortcodes.

fix (webapi): Finish making project shortcode required in StringForma…

cae64b5

…tter. # Conflicts: # webapi/src/main/scala/org/knora/webapi/responders/v2/OntologyResponderV2.scala

feature (webapi): add required project shortcode

5a304b8

feature (webapi): add required project shortcode

541b0bf

feature (webapi): add required project shortcode

12f5a0b

feature (webapi): add required project shortcode

a67f327

feature (webapi): add required project shortcode

013c098

subotic added the enhancement improve existing code or new feature label Apr 12, 2018

subotic added this to the v1.4.0 milestone Apr 12, 2018

subotic self-assigned this Apr 12, 2018

subotic added 3 commits April 12, 2018 19:00

Merge branch 'develop' into wip/required-project-shortcode

49e14de

feature (webapi): add required project shortcode

0e556cf

feature (webapi): add required project shortcode

bd38582

subotic added 8 commits April 13, 2018 17:42

feature (webapi): add required project shortcode

6f90774

feature (webapi): add required project shortcode

613e9eb

fix (webapi): xml import namespace

7687651

test (webapi): fix test

3528d01

fix (webapi): parsing of internal ontology path

90a68aa

feature (webapi): add required project shortcode

da0a6bc

Merge branch 'develop' into wip/required-project-shortcode

196fda7

feature (webapi): add required project shortcode

72a7b65

subotic requested a review from benjamingeer April 20, 2018 11:18

subotic added the ready-for-review label Apr 20, 2018

Merge branch 'develop' into wip/required-project-shortcode

22c1af1

# Conflicts: # webapi/src/test/scala/org/knora/webapi/responders/v2/OntologyResponderV2Spec.scala

subotic added breaking any breaking change breaking (existing data) and removed breaking any breaking change labels Apr 23, 2018

subotic added 5 commits April 23, 2018 15:43

refactor (webapi): change IRI to rdfh.ch

e6d2e3b

refactor (webapi): change IRI to rdfh.ch

39f36ab

refactor (webapi): change IRI to rdfh.ch

54de937

refactor (webapi): change IRI to rdfh.ch

5b58ea9

refactor (webapi): change IRI to rdfh.ch

7d89217

Benjamin Geer added 2 commits April 24, 2018 11:11

test (webapi): Re-enable ignored tests.

480420d

fix (webapi): Fix regex matching for API v1 XML import namespace.

698cecb

benjamingeer suggested changes Apr 24, 2018

View reviewed changes

subotic added 5 commits April 25, 2018 09:15

Merge branch 'develop' into wip/required-project-shortcode

c62435d

test (webapi): standardize IRIs in test data

0ccc01a

test (webapi): standardize IRIs in test data

11cdbf4

test (webapi): standardize IRIs in test data

ce84669

test (webapi): standardize IRIs in test data

e9fc6b8

benjamingeer approved these changes Apr 26, 2018

View reviewed changes

subotic merged commit 66e03b3 into develop Apr 26, 2018

subotic deleted the wip/required-project-shortcode branch April 26, 2018 14:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feature (webapi): make project shortcode required #824

feature (webapi): make project shortcode required #824

subotic commented Apr 12, 2018 •

edited by benjamingeer

Loading

subotic commented Apr 13, 2018

benjamingeer commented Apr 13, 2018

benjamingeer commented Apr 13, 2018

benjamingeer commented Apr 13, 2018

subotic commented Apr 13, 2018

subotic commented Apr 24, 2018

benjamingeer Apr 24, 2018

subotic Apr 24, 2018

subotic Apr 26, 2018

benjamingeer Apr 24, 2018

subotic Apr 24, 2018

subotic Apr 25, 2018

subotic Apr 25, 2018

loicjaouen Apr 25, 2018

subotic Apr 26, 2018

benjamingeer Apr 26, 2018

subotic Apr 26, 2018

benjamingeer Apr 26, 2018

benjamingeer Apr 24, 2018

subotic Apr 24, 2018

benjamingeer Apr 24, 2018

benjamingeer Apr 24, 2018

subotic Apr 24, 2018

benjamingeer commented Apr 24, 2018 •

edited

Loading

subotic commented Apr 24, 2018

benjamingeer commented Apr 26, 2018

subotic commented Apr 26, 2018


		<http://data.knora.org/0C-0L1kORryKzJAJxxRyRQ> a anything:Thing ;
		<http://rdfh.ch/0C-0L1kORryKzJAJxxRyRQ> a anything:Thing ;

feature (webapi): make project shortcode required #824

feature (webapi): make project shortcode required #824

Conversation

subotic commented Apr 12, 2018 • edited by benjamingeer Loading

Summary

Issues

subotic commented Apr 13, 2018

benjamingeer commented Apr 13, 2018

benjamingeer commented Apr 13, 2018

benjamingeer commented Apr 13, 2018

subotic commented Apr 13, 2018

subotic commented Apr 24, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

benjamingeer commented Apr 24, 2018 • edited Loading

subotic commented Apr 24, 2018

benjamingeer commented Apr 26, 2018

subotic commented Apr 26, 2018

subotic commented Apr 12, 2018 •

edited by benjamingeer

Loading

benjamingeer commented Apr 24, 2018 •

edited

Loading