This system component is a RDF-Connect-based pipeline that reads data events from Linked Data Event Stream (LDES) and writes them into a target RDF graph store.
In this particular case, we define a pipeline using a CA processor that encapsulates this LDES client implementation, and writes all extracted entity updates into a Virtuoso RDF graph store (see the rdf-graph-store folder). Next, a schematic view of this pipeline is shown followed by a description of each processor.
This diagram corresponds to the pipeline description found at rdfc-pipeline.ttl.
This pipeline use 2 CA processors:
-
js:LdesClient: This processor encapsulates this TypeScript implementation of an LDES client, which allows to replicate and synchronize with any LDES over the Web. In this case, we use it to replicate and track the changes published by the RINF LDES, published by the generation pipeline. One instance of this processor is defined in the consumption pipeline.
-
js:SPARQLIngest: This processor takes a stream of entities/members described by the SDS metadata, and generates SPARQL Update queries that are executed in a pre-configured target RDF graph store. Additional configuration can be given to specify the semantics (property and values) of specific changes (i.e., creations, updates and deletions). Furthermore, a set of SHACL shapes can be provided to the processor to define properly SPARQL DELETE queries of entities that don't carry all their properties. One instance of this processor is defined in the consumption pipeline.
First build a container from the Dockerfile present in this repository:
docker build -t consumption-pipeline .Then execute the container as follows:
docker run --volume={STATE_PATH}:/opt/consumption-pipeline/state consumption-pipelineReplace {STATE_PATH} by the local path on disk, where the state of the LDES client will be persisted.