Simplify and specify REST API format (JSKOS)

[nichtich]

The current REST API data format is JSON-LD without further specification:

The data consists of a single JSON object which includes JSON-LD context information (in the @context field) and one or more fields which contain the actual data.

The point of JSON-LD, however, is to provide a JSON format that can be used both, as plain JSON and as RDF. To make use of plain JSON, the format has to be specified in more detail, because the same RDF graph can be encoded in JSON-LD in many different ways. I already invested some effort to specify a JSON encoding of SKOS, called JSKOS, see http://gbv.github.io/jskos/jskos.html

To give one example, instead of

{
      "inScheme": { "uri": "http://www.yso.fi/onto/yso/" },
      "prefLabel": [ { 
          "lang": "en", "value": "adrenaline"
        }, {
          "lang": "fi", "value": "adrenaliini"
        }, {
          "lang": "sv", "value": "adrenalin"
        } ]
}

Skosmos should better return JSKOS with the approriate JSON-LD context document like this:

{
      "inScheme": [ "http://www.yso.fi/onto/yso/" ],
      "prefLabel": { 
          "en": "adrenaline",
          "fi": "adrenaliini",
          "sv": "adrenalin"
        }
}

Some features, such as concept collections and mappings may need to be discussed. Comments and questions are welcome!

[osma]

Thanks for your input @nichtich ! I agree with the general principle, but JSKOS didn't exist at the time the API was started and we had to invent our own conventions, balancing between the needs of web developers who are aware of only JSON, and making everything parsable as JSON-LD.

What we have done is that each method returns a result that is "as simple as possible" when interpreted as plain JSON, and also contains a JSON-LD context appropriate for that response. We have also tried to verify that when this JSON-LD response is interpreted as RDF triples, the result is a subset of the triples in the original dataset (i.e. round-tripping should work). So we don't have one big JSON-LD context like JSKOS, but separate contexts for each method and these are returned inline, not via a link.

Your example seems to be from the data method, which is a bit of an exception, since in that method we return all triples relevant to the requested resource and we don't really know in advance what kind of triples might be in the original data. So what we do is to serialize all the triples with EasyRdf (which uses the JsonLD library by Markus Lanthaler under the hood) using a generic JSON-LD context. In this case we have a bit less control over how the result looks. We just provide the context and the JsonLD library does its best to follow that.

What we could perhaps do is to use the JSKOS context for the data method instead and see if it affects the result. I'm just worried about breaking API compatibility - there might be API users out there that rely on the current JSON serialization (e.g. we use expanded term definition to express language tags, instead of language maps as you suggest). So what to do? At some point publish a REST API v2 that uses JSKOS instead of the current (varying) contexts?

[nichtich]

I'm just worried about breaking API compatibility - there might be API users out there that rely on the current JSON serialization

That's exactly why a specified JSON-LD profile is necessary. The current output format is not documented, so nobody can rely on the current JSON serialization. Creating a REST API v2 in parallel to the existing v1 may be easier to implement and better for experimenting - for instance we could pass every request to both API versions, expand both JSON-LD to RDF triples and test that both graphs are identical. As far as I understand, the data method might need special treatment but maybe the JSKOS context document already helps.

[osma]

The current output format is not documented, so nobody can rely on the current JSON serialization.

Only for the data method is the current output format (as JSON) not documented. For all other methods, there is a fair bit of documentation, mainly in the form of tables listing the returned fields: https://github.com/NatLibFi/Skosmos/wiki/REST-API
(I see now that there are some broken links between methods, which has happened during migration of the wiki from Google Code to GitHub, these should be fixed of course)

As mentioned above the data method is an exception since it has to be able to handle any triples, whereas the other methods only provide specific information known in advance.

I think JSKOS is definitely a good starting point for REST API v2 if we start implementing one.

[nichtich]

thanks, I see. The current JSON format differs from JSKOS at some points (of course) - where is the best place to compare both and to raise questions? For instance I don't understand why topConcepts has a different response format than lookup and search and some fields like localname are not clear to me. By the way JSKOS uses a different approach to express missing values (not hasChildren).

[osma]

The API has grown organically over time and the design of many methods have been at least somewhat affected by the requirements of the Skosmos web interface, which makes use of its own API for many AJAX-y things. What each method returns is based on some idea of a use case for it (sometimes the Skosmos UI, sometimes an external API client).

The current JSON format differs from JSKOS at some points (of course) - where is the best place to
compare both and to raise questions?

I don't have any suggestions - this issue tracker is probably not the best place though :) Maybe a wiki somewhere?

I think it would help if there existed also an API spec with methods/operations that can be performed on a SKOS dataset. So one level up from JSKOS. Any plans for that?

For instance I don't understand why topConcepts has a different response format than lookup and
search

I see two main differences:

1. For topConcepts, the uri (subject of all triples) is the concept scheme, as we are basically just generating triples like "x skos:hasTopConcept y". For search and lookup, the uri/subject is set to "" which stands for the original query, because each query is different.
2. search and lookup return more data fields per concept than topConcepts. I agree this is not consistent, we could easily let the topConcepts method return more information about concepts. Do you think that would be good?

and some fields like localname are not clear to me.

Agreed this is not very elegant. This is basically for the Skosmos web UI, which differentiates between concept URIs in the concept scheme's main namespace (for which shorter URLs can be used) and other URIs. In case of a URI that can be shortened, the localname field is used to return the last part.

By the way JSKOS uses a different approach to express missing values (not hasChildren).

Yes, I can see that you're using boolean values for the same properties, e.g. narrower. Seems clever. But doesn't this cause problems for a naïve (non-JSKOS-aware) but JSON-LD aware client that will then generate triples like "x skos:narrower true"? I see the spec says "The following JSON-LD context document can be used to map JSKOS to map JSKOS without closed world statements to RDF triples." but if the JSON-LD context is referenced in a JSON file with closed world statements, how can you forbid using that context?

[osma]

Closing and archiving issue due to lack of activity. Please reopen if there are concrete reasons for reconsidering.