Bind prefixes choices #1686

nicholascar · 2022-01-23T23:57:29Z

Proposed Changes

a bunch of reST documentation updates
updates to inline docc for term.py classes (RDF 1.1, not 1.0 explanations)
new param for Graph(): bind_namespaces
- with options for none, core prefixes (default), all RDFLib Namespaces or use of prefix.cc

Since this PR is really needed - see the comments - I'm pushing it up to review before the prefix.cc option is implemented.

docs/index.rst

docs/rdf_terms.rst

aucampia · 2022-01-24T00:15:08Z

rdflib/namespace/__init__.py

+        else:  # bind_namespaces is None
+            pass  # bind nothing


May be a bit better to rather distinguish between explicit None and invalid/unknown string so that sending the wrong string results in a failure, will make a PR against your branch tomorrow to show you what I mean.

I think I get it. I haven't finished all the implementation here of course, so was going to add checks for None and other strings too but just haven't got to it so far.

Feel free to put in a PR or commits to the branch!

PR here: #1693

Co-authored-by: Iwan Aucamp <aucampia@gmail.com>

wmelder · 2022-02-25T15:09:32Z

Noticed this dependency in the PR: This is in DRAFT until I implement the prexif.cc lookup option
For me, that lookup option is a nice to have. It would already be an improvement if the options: none, core prefixes (default), all RDFLib Namespaces .. can be chosen. Using 'none' I can add all namespaces I need and no other defaults would bloat my RDF.

Co-authored-by: Iwan Aucamp <aucampia@gmail.com>

cmungall · 2022-03-27T00:52:54Z

I agree with @wmelder's comments - I don't think having prefix.cc lookup should be core functionality. It would be really awesome if we could get this PR merged as currently these additional prefixes are causing a lot of problems for us.

nicholascar · 2022-03-28T14:26:22Z

I'm held up in completing this PR for, due to my testing, I've think I'm seeing that bind with replace=True doesn't actually work! This test fails:

from rdflib import Graph
from rdflib.namespace import OWL

def test_rebinding():
    g = Graph()  # 'core' bind_namespaces (default)
    print()
    # 'owl' should be bound
    assert "owl" in [x for x, y in list(g.namespaces())]

    # replace 'owl' with 'sowa'
    # 'sowa' should be bound
    # 'owl' should not be bound
    g.bind("sowa", OWL, replace=True)

    assert "sowa" in [x for x, y in list(g.namespaces())]
    assert "owl" not in [x for x, y in list(g.namespaces())]

ghost · 2022-03-28T17:39:00Z

I'm held up in completing this PR for, due to my testing, I've think I'm seeing that bind with replace=True doesn't actually work! This test fails:
    assert "owl" not in [x for x, y in list(g.namespaces())]

I believe that's a correctly-failing test according to the implementation, if not according to the API spec. It rather looks like replace should be read as "replace the namespace bound to this prefix" rather than "replace the prefix bound to this namespace".

In namespace/__init.py__::NamespaceManager.bind(), LoC#566 results in bound_namespace=None and so the if bound_namespace and bound_namespace != namespace: of LoC#573 fails and the execution skips to the else at Loc#598, where it skips to the else on Loc#674 - resulting in the new binding of sowa to the OWL namespace, that gets added to the trie and returned.

~~I'll hazard a guess that for replace to work as originally intended, the old binding needs to be deleted from the trie.~~

Um, wrong again. Turns out that the persistence-backed stores handle this (sort of) gracefully and automatically perform a replace: BerkeleyDB has:

    def bind(self, prefix, namespace):
        prefix = prefix.encode("utf-8")
        namespace = namespace.encode("utf-8")
        bound_prefix = self.__prefix.get(namespace)
        if bound_prefix:
            self.__namespace.delete(bound_prefix)
        self.__prefix[namespace] = prefix
        self.__namespace[prefix] = namespace

So, for persistence stores using that architecture, the test passes but the SimpleMemory and Memory implementation ignores any already-bound prefix and so the test fails:

    def bind(self, prefix, namespace):
        self.__prefix[namespace] = prefix
        self.__namespace[prefix] = namespace

Making the following (fairly straightforward) change to memory.py allows the test to succeed (with one small change)

diff --git a/rdflib/plugins/stores/memory.py b/rdflib/plugins/stores/memory.py
index 5a137b5a..135f2a22 100644
--- a/rdflib/plugins/stores/memory.py
+++ b/rdflib/plugins/stores/memory.py
@@ -149,6 +149,9 @@ class SimpleMemory(Store):
         return i
 
     def bind(self, prefix, namespace):
+        bound_prefix = self.__prefix.get(namespace)
+        if bound_prefix:
+            self.__namespace.delete(bound_prefix)
         self.__prefix[namespace] = prefix
         self.__namespace[prefix] = namespace
 
@@ -400,6 +403,9 @@ class Memory(Store):
                             yield triple, self.__contexts(triple)
 
     def bind(self, prefix, namespace):
+        bound_prefix = self.__prefix.get(namespace)
+        if bound_prefix:
+            del self.__namespace[bound_prefix]
         self.__prefix[namespace] = prefix
         self.__namespace[prefix] = namespace

def test_rebinding():
    g = Graph()  # 'core' bind_namespaces (default)

    # 'owl' should be bound
    assert "owl" in [x for x, y in list(g.namespaces())]

    NOWL = g.store.namespace('owl')  # see ¹

    # replace 'owl' with 'sowa'
    # 'sowa' should be bound
    # 'owl' should not be bound

    g.bind("sowa", NOWL)

    assert "sowa" in [x for x, y in list(g.namespaces())]
    assert "owl" not in [x for x, y in list(g.namespaces())]

¹ NOWL is necessary because OWL is "<class 'rdflib.namespace.DefinedNamespaceMeta'>" and so the Store's test of self.__prefix.get(namespace) fails, whereas NOWL evals to the actually-bound URIRef and succeeds.

nicholascar · 2022-03-28T23:15:45Z

I'll hazard a guess that for replace to work as originally intended, the old binding needs to be deleted from the trie.
the SimpleMemory and Memory implementation ignores any already-bound prefix and so the test fails

That's what I though I was seeing, that it was Store-specific and that the Memory stores were failing! I was hoping to be able to restrict my PR changes to the namespace module but clearly it's not possible to sort this our without also touching the Memory stores.

OK, I'll make the updates as outlined above later today.

aucampia · 2022-03-29T20:09:03Z

That's what I though I was seeing, that it was Store-specific and that the Memory stores were failing! I was hoping to be able to restrict my PR changes to the namespace module but clearly it's not possible to sort this our without also touching the Memory stores.

OK, I'll make the updates as outlined above later today.

Happy to help here if you don't find time, can try make a separate PR to fix it tomorrow if you want, just ping me.

nicholascar · 2022-03-29T23:22:11Z

Thanks for the offer @aucampia. I think I'd like to give this a go initially. The fix for Memory stores is easy, even though I was trying to avoid having to fiddle with the Stores! - but I'm keen to test out other stores too, especially the SPARQL stores. So I'll give it a shot shortly and reach out to you if I need a hand.

nicholascar · 2022-03-30T13:56:59Z

@aucampia any ideas as to why everything's failing? It looks like a system fault with errors such as ERROR: py38: commands failed rather than application code errors.

Running locally, everything passes so I think I'm done with the fixes for Stores.

ghost · 2022-03-31T10:16:23Z

Just FTR, there's a relevant extant issue from a while ago - #543 (which contains some informative observations by gromgull of the intended replace and override functionality) and an unfinished open PR #1094, both of which can be closed when this PR is merged.

aucampia · 2022-03-31T12:20:35Z

@aucampia any ideas as to why everything's failing? It looks like a system fault with errors such as ERROR: py38: commands failed rather than application code errors.

Running locally, everything passes so I think I'm done with the fixes for Stores.

mypy is failing, will make a fix to your branch in a moment.

aucampia · 2022-04-12T17:10:26Z

@nicholascar let me know if you want me to split things off from this PR, can easily use rebase to keep your commits the same just put them in a seperate branch with you as author, I think a lot of these doc changes are very helpful and very unobjectionable. Up to you though.

aucampia · 2022-04-15T09:31:37Z

Will look at tying up all loose ends here next on 2022-04-18 if there is no further feedback.

ghost

Well in hand now.

aucampia · 2022-04-15T13:52:44Z

Will rebase and fix the docs warning.

nicholascar · 2022-04-15T14:21:25Z

pre-commit.ci autofix

for more information, see https://pre-commit.ci

nicholascar · 2022-04-15T14:38:28Z

This is only failing the documentation building check, but that is being addressed elsewhere, so I'm going to merge this in now.

aucampia · 2022-04-15T14:39:44Z

This is only failing the documentation building check, but that is being addressed elsewhere, so I'm going to merge this in now.

Happy with that, I figured out what the problem is and will make a PR to fix on master shortly.

cmungall · 2022-04-25T19:52:45Z

it would be awesome to have a new release with this PR in - I'm currently blocked on upgrading rdflib in a few projects as the namespace pollution is causing issues

nicholascar · 2022-04-26T06:12:58Z

@cmungall we're going to get a release out ASAP. Yes, I'm in the same boat as you!

cmungall · 2022-06-06T21:11:20Z

Just a friendly ping to see if there was a timeline for a new release with the namespace pollution removed?

… back the `wgs` binding <RDFLib#1686> inadvertently changed prefix for `https://schema.org/` from `schema` to `sdo` and removed the `wgs` prefix. This PR changes the `https://schema.org/` binding back to `schema` and adds the `wgs` binding back. - Closes <RDFLib#2196>.

… the `wgs` binding back <RDFLib#1686> inadvertently changed the prefix for `https://schema.org/` from `schema` to `sdo` and removed the `wgs` prefix. This PR changes the `https://schema.org/` binding back to `schema` and adds the `wgs` binding back. - Closes <RDFLib#2196>.

<RDFLib#1686> inadvertently removed the `wgs` prefix. This change adds it back. - Closes <RDFLib#2196>.

<#1686> inadvertently removed the `wgs` prefix. This change adds it back. - Closes <#2196>.

nicholascar added 2 commits January 23, 2022 20:38

improved Terms documentation in reST and py files

fed6bf5

bind NSes according to options; tests

c6e2fd7

nicholascar requested review from ashleysommer, white-gecko and aucampia January 23, 2022 23:57

nicholascar changed the title ~~Default prefixes~~ Bind prefixes choices Jan 23, 2022

nicholascar mentioned this pull request Jan 23, 2022

Replace long list of namespaces with list of prefixes used when using serialize. #1679

Closed

aucampia reviewed Jan 24, 2022

View reviewed changes

docs/index.rst Outdated Show resolved Hide resolved

aucampia reviewed Jan 24, 2022

View reviewed changes

docs/rdf_terms.rst Outdated Show resolved Hide resolved

aucampia reviewed Jan 24, 2022

View reviewed changes

Update docs/rdf_terms.rst

4164e63

Co-authored-by: Iwan Aucamp <aucampia@gmail.com>

nicholascar and others added 2 commits February 26, 2022 02:27

Merge branch 'master' into default_prefixes

4a55c26

Update docs/index.rst

f9a2617

Co-authored-by: Iwan Aucamp <aucampia@gmail.com>

nicholascar added 3 commits March 27, 2022 17:33

Merge branch 'master' into default_prefixes

56bee18

docco improvements

6a6f632

'none' option

c24f444

nicholascar added 2 commits March 30, 2022 15:58

ensure all core Stores contain prefix replace option

3652855

remove prints

2cbea1b

ghost mentioned this pull request Mar 30, 2022

Isvaliduri optimization #1779

Merged

3 tasks

Fix type error

de3c61b

This was referenced Apr 14, 2022

Black tests #1798

Merged

docs: fix sphinx warnings #1818

Merged

ghost approved these changes Apr 15, 2022

View reviewed changes

Merge branch 'master' into default_prefixes

2f9cccf

Merge remote-tracking branch 'upstream/master' into default_prefixes

11f2357

[pre-commit.ci] auto fixes from pre-commit.com hooks

808aa09

for more information, see https://pre-commit.ci

nicholascar merged commit b5ec096 into master Apr 15, 2022

nicholascar deleted the default_prefixes branch April 15, 2022 14:39

aucampia mentioned this pull request Apr 17, 2022

fix: namespace rebinding in Memory, SimpleMemory and BerkelyDB stores. #1843

Merged

4 tasks

cmungall mentioned this pull request May 2, 2022

Allow earlier versions of rdflib RDFLib/sparqlwrapper#219

Open

aucampia mentioned this pull request Mar 19, 2023

fix: add the wgs namespace binding back #2294

Merged

4 tasks

aucampia mentioned this pull request Mar 20, 2023

dcterms no longer included among default namespaces in 6.2.0 #2103

Closed

aucampia added a commit to aucampia/rdflib that referenced this pull request Mar 20, 2023

fix: add the wgs namespace binding back

6c1d9e8

<RDFLib#1686> inadvertently removed the `wgs` prefix. This change adds it back. - Closes <RDFLib#2196>.

aucampia added a commit that referenced this pull request Mar 21, 2023

fix: add the wgs namespace binding back (#2294)

adf8eb2

<#1686> inadvertently removed the `wgs` prefix. This change adds it back. - Closes <#2196>.

aucampia added the concept: namespace label Jun 13, 2023

aucampia mentioned this pull request Jun 13, 2023

rdflib 6.2.0 SPARQL parsing regression: multiple prefixes for same IRI not supported? #2077

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bind prefixes choices #1686

Bind prefixes choices #1686

nicholascar commented Jan 23, 2022 •

edited

Loading

aucampia Jan 24, 2022 •

edited

Loading

nicholascar Jan 24, 2022

aucampia Jan 24, 2022

wmelder commented Feb 25, 2022

cmungall commented Mar 27, 2022

nicholascar commented Mar 28, 2022 •

edited

Loading

ghost commented Mar 28, 2022 •

edited by ghost

Loading

nicholascar commented Mar 28, 2022

aucampia commented Mar 29, 2022

nicholascar commented Mar 29, 2022

nicholascar commented Mar 30, 2022

ghost commented Mar 31, 2022

aucampia commented Mar 31, 2022

aucampia commented Apr 12, 2022

aucampia commented Apr 15, 2022

ghost left a comment

aucampia commented Apr 15, 2022

nicholascar commented Apr 15, 2022

nicholascar commented Apr 15, 2022

aucampia commented Apr 15, 2022

cmungall commented Apr 25, 2022

nicholascar commented Apr 26, 2022

cmungall commented Jun 6, 2022

Bind prefixes choices #1686

Bind prefixes choices #1686

Conversation

nicholascar commented Jan 23, 2022 • edited Loading

Proposed Changes

aucampia Jan 24, 2022 • edited Loading

Choose a reason for hiding this comment

nicholascar Jan 24, 2022

Choose a reason for hiding this comment

aucampia Jan 24, 2022

Choose a reason for hiding this comment

wmelder commented Feb 25, 2022

cmungall commented Mar 27, 2022

nicholascar commented Mar 28, 2022 • edited Loading

ghost commented Mar 28, 2022 • edited by ghost Loading

nicholascar commented Mar 28, 2022

aucampia commented Mar 29, 2022

nicholascar commented Mar 29, 2022

nicholascar commented Mar 30, 2022

ghost commented Mar 31, 2022

aucampia commented Mar 31, 2022

aucampia commented Apr 12, 2022

aucampia commented Apr 15, 2022

ghost left a comment

Choose a reason for hiding this comment

aucampia commented Apr 15, 2022

nicholascar commented Apr 15, 2022

nicholascar commented Apr 15, 2022

aucampia commented Apr 15, 2022

cmungall commented Apr 25, 2022

nicholascar commented Apr 26, 2022

cmungall commented Jun 6, 2022

nicholascar commented Jan 23, 2022 •

edited

Loading

aucampia Jan 24, 2022 •

edited

Loading

nicholascar commented Mar 28, 2022 •

edited

Loading

ghost commented Mar 28, 2022 •

edited by ghost

Loading