-
Notifications
You must be signed in to change notification settings - Fork 52
QLever support for SPARQL 1.1 Update
QLever now has basic support for SPARQL 1.1 Update, as of https://github.com/ad-freiburg/qlever/pull/1608, which was merged into the QLever master on 15.11.2024. In particular, QLever now supports the following four core SPARQL 1.1 Update operations:
INSERT DATA { ... }
DELETE DATA { ... }
INSERT { ... } WHERE { ... }
DELETE { ... } WHERE { ... }
This is now being thoroughly evaluated and expanded. A first performance evaluation can be found on https://github.com/ad-freiburg/qlever/wiki/First-tests-with-SPARQL-1.1-Update . The current update speed is 0.5 million triples / second, which is already sufficient for Wikidata (10 - 30 million page edits per month). The speed will very likely improve in future versions.
Two important features that are currently still missing, but will be added soon: (1) only a single update per request is allowed, not multiple updates that are treated atomically (that is, they either all succeed or all fail); (2) updates are not yet persistent and need to be remembered outside of QLever in case of a server crash and restart.
Most of the other missing features are syntactic sugar in the sense that they can be simulated by the four core operations above. But they will of course be added, too.
A first proof of concept for SPARQL 1.1 Update was implemented by March 2023, see https://github.com/ad-freiburg/qlever/pull/916. It supported updates via a special syntax. The goal of that first version was to evaluate the efficiency of various design choices. In the following months, many preparatory changes deep down in QLever's index data structures and index processing routines were made.
A first functional version for SPARQL 1.1 Update (with standard-conform syntax) has been available since May 2024, see https://github.com/ad-freiburg/qlever/pull/1351. It supports the four principle ways to add or delete triples: INSERT DATA, DELETE DATA, INSERT ... WHERE, and DELETE ... WHERE.
What we are currently working on (as of 06.09.2024):
-
So far, the implementation of SPARQL 1.1 Update is still in a separate branch and under review (see above). We are in the process of integrating this code into QLever's main (master) branch. [DONE 15.11.2024]
-
So far, SPARQL 1.1 Update works for all triples in the default graph. We are working on extending this to arbitrary named graphs. Note that for Wikidata, all triples are in the default graph. [DONE 15.11.2024]
-
The SPARQL 1.1 Update operations are already reasonably efficient, but not yet as efficient as they could be. Our goal is to be able to support thousands (possibly millions) of updates per second, with a negligible effect on query processing times. For example, Wikidata receives around 20 million edits per month. [DONE 15.11.2024]
-
We would love to test our current code on Wikidata and provide a SPARQL endpoint that is in sync with https://query.wikidata.org . However, Wikidata currently provides only weekly dumps of their data and no publicly available update streams. We are currently working on obtaining the updates indirectly via other means. [ONGOING, see https://phabricator.wikimedia.org/T294133]