Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Branch for jim #82

Merged
merged 7 commits into from
Nov 10, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
.idea/
3 changes: 0 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,10 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## 2022-09

### Changed
- fixed missing mandatory elements in multipart message
- Fixed wrong references to IDS-G-pre and RAM in CONTRIBUTING.md


## [Q2/2022]

### Added
Expand Down
38 changes: 20 additions & 18 deletions Communication/protocols/multipart/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ Content-Type: application/json
Content-Length: 409

{
"@context" : "https://w3id.org/idsa/contexts/context.jsonld",
"@type" : "ids:DescriptionResponseMessage",
"issued" : {
"@value" : "2019-12-21T13:11:14.596+02:00",
Expand Down Expand Up @@ -264,24 +265,25 @@ Content-Type: application/json; charset=utf-8
Content-Disposition: form-data; name="header"

{
"@type" : "ids:ConnectorUnavailableMessage",
"@id" : "http://industrialdataspace.org/connectorUnavailableMessage/1a421b8c-3407-44a8-aeb9-253f145c869a",
"issued" : {
"@value" : "2019-10-25T15:35:34.589+02:00",
"@type" : "http://www.w3.org/2001/XMLSchema#dateTimeStamp"
},
"modelVersion" : "4.1.0",
"ids:issuerConnector" : {
"@id" : "https://test.connector.de/"
},
"securityToken" : {
"@type" : "ids:DynamicAttributeToken",
"@id" : "http://industrialdataspace.org/token/e43c08e1-157b-4207-94a8-754e53f48839",
"ids:tokenFormat" : {
"@id" : "idsc:JWT"
},
"tokenValue" : "eyJhbGciOiJSUzI1NiIsInR5cCI..."
}
"@context" : "https://w3id.org/idsa/contexts/context.jsonld",
"@type" : "ids:ConnectorUnavailableMessage",
"@id" : "http://industrialdataspace.org/connectorUnavailableMessage/1a421b8c-3407-44a8-aeb9-253f145c869a",
"issued" : {
"@value" : "2019-10-25T15:35:34.589+02:00",
"@type" : "http://www.w3.org/2001/XMLSchema#dateTimeStamp"
},
"modelVersion" : "4.1.0",
"ids:issuerConnector" : {
"@id" : "https://test.connector.de/"
},
"securityToken" : {
"@type" : "ids:DynamicAttributeToken",
"@id" : "http://industrialdataspace.org/token/e43c08e1-157b-4207-94a8-754e53f48839",
"ids:tokenFormat" : {
"@id" : "idsc:JWT"
},
"tokenValue" : "eyJhbGciOiJSUzI1NiIsInR5cCI..."
}
}
--msgpart--
```
Expand Down
13 changes: 13 additions & 0 deletions Specifications/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Getting Started

The [Information Model document](./model/information.model.md) defines the core concepts, entities, and relationships that underpin a `Dataspace`.

The [Catalog Protocol document](./catalog/catalog.protocol.md) defines a how a `Catalog` is requested from a catalog service by a consumer using an abstract message exchange format.

The [Catalog Binding document](./catalog/catalog.binding.https.md) defines a RESTful API over HTTPS for the `Catalog Protocol`.

The [Bibliography](./notes/bibliography.md) contains links to relevant standards referenced by the above documents.

Note that PlantUML diagrams will be replaced by Draw-IO alternatives.


134 changes: 134 additions & 0 deletions Specifications/catalog/catalog.binding.https.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
# Catalog HTTPS Binding

## 1 Introduction

This specification defines a RESTful API over HTTPS for the [Catalog Protocol].

The OpenAPI definitions for this specification can be accessed [here](TBD).

## 2 Path Bindings

### 2.1 Prerequisites

1. The `<base>` notation indicates the base URL for a catalog service endpoint. For example, if the base catalog URL is `api.example.com`, the URL `https://<base>/catalog/request`
will map to `https//api.example.com/catalog/request`.

2. All request and response messages must use the `application/json` media type.

### 2.2 CatalogErrorMessage

In the event of a request error, the catalog service must return an appropriate HTTP code and a [CatalogErrorMessage](./catalog.protocol.md#) in the response body.

| Field | Type | Description |
|---------|---------------|-------------------------------------------------------------|
| code | string | An optional implementation-specific error code. |
| reasons | Array[object] | An optional array of implementation-specific error objects. |

### 2.3 The `catalog/request` endpoint

#### 2.3.1 POST

The [CatalogRequestMessage](catalog.protocol.md#1-catalogrequestmessage) corresponds to `POST https://<base>/catalog/request`:

```
POST https://provider.com/catalog/request

Authorization: ...

{
"@context": {
"ids": "https://idsa.org/"
},
"@type": "ids:CatalogRequest"
"ids:filter": {}
}
```

The `Authorization` header is optional if the catalog service does not require authorization. If present, the contents of the `Authorization` header are detailed in the
[Authorization section](#authorization).

The `filter` property is optional. If present, the `filter` property can contain an implementation-specific filter expression or query to be executed as part of the catalog
request.

#### 2.3.2 OK (200) Response

If the request is successful, the catalog service must return a response body containing a [CatalogMessage](./message/catalog.message.json) which is a profiled DCAT Catalog type
described by the [Catalog Protocol Specification](catalog.protocol.md).

## 3 Technical Considerations

### 3.1 Authorization

A catalog service may require authorization. If the catalog service requires authorization, requests must include an HTTP `Authorization` header with a token. The contents of
the token are undefined by may be an OAUTH2, Web DID, or other access token type.

### 3.2 Versioning

- Versioning will be done via URLs. TBD.

### 3.3 Pagination

A catalog service may paginate the results of a `CatalogRequestMessage`. Pagination data is specified using [Web Linking](https://datatracker.ietf.org/doc/html/rfc5988)
and the HTTP `Link` header. The `Link` header will contain URLs for navigating to previous and subsequent results. The following request sequence demonstrates pagination:

```
Link: <https://provider.com/catalog?page=2&per_page=100>; rel="next"
{
"@context": {
"dcat": "http://www.w3.org/ns/dcat/"
},
"@type": "dcat:Catalog"
...
}

```

Second page response:

```
Link: <https://provider.com/catalog?page=1&per_page=100>; rel="previous"
Link: <https://provider.com/catalog?page=3&per_page=100>; rel="next"

{
"@type": "dcat:Catalog"
...
}
```

Last page response:

```
Link: <https://provider.com/catalog?page=2&per_page=100>; rel="previous"

{
"@type": "dcat:Catalog"
...
}
```

### 3.4 Compression

Catalog services MAY compress responses to a catalog request by setting the `Content-Encoding` header to `gzip` as described in
the [HTTP 1.1 Specification](https://www.rfc-editor.org/rfc/rfc9110.html#name-gzip-coding).

## 4 Notes

### 4.1 Asynchronous Interactions

We may want to specify optional support for asynchronous callbacks for the catalog response. This would require addling a `callbackAddress` property and an `@id` to the request:

```
POST https://provider.com/catalog/request

Authorization: ...

{
"@context:{},
"@type": "ids:CatalogRequest"
"@id: "..."
"ids:callbackAddress": "https://example.com/endpoint"
}
```

The `CatalogResponseMessage` would be POSTed back to the endpoint. the response message could be posted mutiple times for paginated results and would need to include the
original `@id` value as a `correlationId` and a property indicating if the contents are complete (or additional responses will be sent).
131 changes: 131 additions & 0 deletions Specifications/catalog/catalog.protocol.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
# Catalog Protocol

## 1 Introduction: Terms

This document outlines the catalog protocol. The following terms are used:

- A _**message type**_ defines the structure of a _message_.
- A _**message**_ is an instantiation of a _message type_.
- A _**catalog**_ is a [DCAT catalog](https://www.w3.org/TR/vocab-dcat-3/) offered by a _provider_
- a _**catalog service**_ is a provider participant agent that advertises offered assets
- A _**consumer**_ is a participant agent that requests access to an offered asset.

The catalog protocol defines a how a `Catalog` is requested from a catalog service by a consumer using an abstract message exchange format. The concrete message exchange wire
format is defined in binding specifications.

## 2 Message Types

All messages must be serialized in JSON-LD compact form as specified in the [JSON-LD 1.1 Processing Algorithms and API](https://www.w3.org/TR/json-ld11-api/#compaction-algorithms).
Future IDS specifications may define additional optional serialization formats.

### 2.1 CatalogRequestMessage

**Sent by**: Consumer

**Example**:

```
{
"@context: {
"ids": "https://idsa.org/"
},
"@type": "ids:CatalogRequest"
"ids:filter": {}
}
```

**Response**: [CatalogMessage](./message/catalog.message.json) containing the [DCAT catalog](https://www.w3.org/TR/vocab-dcat-3/#Class:Catalog).

**Schema**: (xx)[]

#### Description

The `CatalogRequestMessage` is message sent by a consumer to a catalog service. The catalog service must respond with a `CatalogMessage,` which is a
valid [DCAT Catalog](https://www.w3.org/TR/vocab-dcat-3/#Class:Catalog).

The `CatalogRequestMessage` may have a `filter` property which contains an implementation-specific query or filter expression type supported by the catalog service.

The catalog service may require an authorization token. Details for including that token can be found in the relevant catalog binding specification. Similarly, pagination may
be defined in the relevant catalog binding specification.

## 3 DCAT Vocabulary Mapping

This section describes how the IDS Information Model maps to DCAT resources.

### 3.1 Asset Entry

An `Asset Entry` is a [DCAT Dataset](https://www.w3.org/TR/vocab-dcat-3/#Class:Dataset) with the following attributes:

#### 3.1.1 dcat:haPolicy

An asset entry Dataset may have 0..N `hasPolicy` attributes that contain an ODRL `Offer` defining the usage control policy associated with the asset. Offers must NOT contain any
target attributes. The target of an offer is the asset associated with the containing asset entry. If an entry is has no associated policies (hasPolicy does not contain any offer
entries), the asset is does not have any usage control policy.

### 3.2 Distributions

An asset may contain 0..N [DCAT Distributions](https://www.w3.org/TR/vocab-dcat-3/#Class:Distribution). Each distribution must have at least one `DataService` which specifies where
the distribution is obtained. Specifically, a `DataService` specifies the endpoint for initiating a `ContractNegotiation` and `AssetTransfer`.

### 3.3 DataService

A DataService may specify an IDS service endpoint such as a `Connector`.

#### 3.3.1 ids:dataServiceType

If the DataService refers to an IDS service endpoint, it must include the property `ids:dataServiceType`:

| Category | Description |
|------------|----------------------------------------------------------------------------|
| Definition | Specifies the IDS service type |
| Domain | [dcat:DataService](https://www.w3.org/TR/vocab-dcat-2/#Class:Data_Service) |
| Type | xsd:string |
| Note | The value of this field is left intentionally open for future extension. |

The following table lists well-know IDS endpoint types:

| Value | Description |
|---------------|----------------------|
| ids:connector | A Connector endpoint |
| | |


#### 3.3.2 dcat:servesDataset

Note that the property `dcat:servesDataset` should be omitted from the `DataService` since `DataSets` are included as top-level entries. Clients are not required to process the
contents of `dcat:servesDataset`.

## 4 Technical Considerations

### 4.1 Queries and Filter Expressions

A _**catalog service**_ may support catalog queries or filter expressions as an implementation-specific feature. However, it is expected that query capabilities will be implemented
by the consumer against the results of a `CatalogRequestMessage,` as the latter is an RDF vocabulary. Client-side querying can be scaled by periodically crawling provider catalog
services, caching the results, and executing queries against the locally-stored catalogs.

### 4.2 Replication Protocol

The catalog protocol is designed to be used by federated services without the need for a replication protocol. Each consumer is responsible for issuing requests
to 1..N catalog services, and managing the results. It follows that a specific replication protocol is not needed, or more precisely, each consumer replicates data from catalog
services by issuing `CatalogRequestMessages`.

The discovery protocol adopted by a particular dataspace defines how a consumer discovers catalog services.

### 4.3 Security

It is expected (although not required) that catalog services implement access control. A catalog as well as individual catalog _datasets_ may be restricted to trusted parties.
The catalog service may require consumers to include a security token along with a `CatalogRequestMessage.` The specifics of how this is done can be found in the relevant
catalog binding specification. In addition, this specification does not define the contents of the security token. It is expected different security mechanisms may be used such
as [OAuth2 2.1](https://datatracker.ietf.org/doc/html/draft-ietf-oauth-v2-1-06) or [did:web](https://w3c-ccg.github.io/did-method-web/).

### 4.4 Catalog Brokers

A dataspace may include _**catalog brokers**_. A catalog broker is a consumer that has trusted access to 1..N upstream catalog services and advertises their respective catalogs as
a single catalog service. The broker is expected to honor upstream access control requirements.

## 5 DCAT and ODRL Profiles

The catalog is a DCAT catalog with the following restrictions:

1. Each ODRL `Offer` must be unique to a dataset since the target of the offer is derived from its enclosing context.
2. Each ODRL `Offer` must NOT include an explicit `target` attribute.
11 changes: 11 additions & 0 deletions Specifications/catalog/message/catalog.error.message.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"@context": {
"ids": "https://idsa.org/"
},
"@type": "ids:CatalogErrorMessage",
"code": "123:A",
"reasons": [
{},
{}
]
}
Loading