Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid that tripper.Namespace crashes if the cache directory cannot be accessed #235

Merged
merged 2 commits into from
Aug 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ repos:
rev: 24.8.0
hooks:
- id: black
line-length: 79

- repo: https://github.com/PyCQA/bandit
rev: 1.7.9
Expand Down
19 changes: 11 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,13 @@ Tripper
[![DOI](https://zenodo.org/badge/547162834.svg)](https://zenodo.org/badge/latestdoi/547162834)


Getting started
---------------
* [Tutorial]
* [Discovery of custom backends]
* [Reference manual]
* [Known issues]


Basic concepts
--------------
Expand Down Expand Up @@ -39,13 +46,6 @@ The submodules `mappings` and `convert` provide additional functionality beyond
- **tripper.convert**: convert between RDF and other data representations.


Documentation
-------------
* Getting started: See the [tutorial](docs/tutorial.md)
* [Discovery of custom backends](docs/backend_discovery.md)
* [Reference manual]


Available backends
------------------
The following backends are currently available, either in Tripper or other packages.
Expand Down Expand Up @@ -104,11 +104,14 @@ We gratefully acknowledge the following projects for supporting the development



[Tutorial]: docs/tutorial.md
[Discovery of custom backends]: docs/backend_discovery.md
[Reference manual]: https://emmc-asbl.github.io/tripper/latest/api_reference/triplestore/
[Known issues]: https://emmc-asbl.github.io/tripper/latest/known-issues.md
[tripper]: https://emmc-asbl.github.io/tripper
[rdflib]: https://rdflib.readthedocs.io/en/stable/
[PyPI]: https://pypi.org/project/tripper
[PyBackTrip]: https://github.com/EMMC-ASBL/PyBackTrip/
[Reference manual]: https://emmc-asbl.github.io/tripper/latest/api_reference/triplestore/
[Literal]: https://emmc-asbl.github.io/tripper/latest/api_reference/triplestore/#tripper.triplestore.Literal
[Namespace]: https://emmc-asbl.github.io/tripper/latest/api_reference/triplestore/#tripper.triplestore.Namespace
[Triplestore]: https://emmc-asbl.github.io/tripper/latest/api_reference/triplestore/#tripper.triplestore.Triplestore
Expand Down
19 changes: 11 additions & 8 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,13 @@ Tripper
[![DOI](https://zenodo.org/badge/547162834.svg)](https://zenodo.org/badge/latestdoi/547162834)


Getting started
---------------
* [Tutorial]
* [Discovery of custom backends]
* [Reference manual]
* [Known issues]


Basic concepts
--------------
Expand Down Expand Up @@ -39,13 +46,6 @@ The submodules `mappings` and `convert` provide additional functionality beyond
- **tripper.convert**: convert between RDF and other data representations.


Documentation
-------------
* Getting started: See the [tutorial](tutorial.md)
* [Discovery of custom backends](backend_discovery.md)
* [Reference manual]


Available backends
------------------
The following backends are currently available, either in Tripper or other packages.
Expand Down Expand Up @@ -104,11 +104,14 @@ We gratefully acknowledge the following projects for supporting the development



[Tutorial]: tutorial.md
[Discovery of custom backends]: backend_discovery.md
[Reference manual]: https://emmc-asbl.github.io/tripper/latest/api_reference/triplestore/
[Known issues]: https://emmc-asbl.github.io/tripper/latest/known-issues.md
[tripper]: https://emmc-asbl.github.io/tripper
[rdflib]: https://rdflib.readthedocs.io/en/stable/
[PyPI]: https://pypi.org/project/tripper
[PyBackTrip]: https://github.com/EMMC-ASBL/PyBackTrip/
[Reference manual]: https://emmc-asbl.github.io/tripper/latest/api_reference/triplestore/
[Literal]: https://emmc-asbl.github.io/tripper/latest/api_reference/triplestore/#tripper.triplestore.Literal
[Namespace]: https://emmc-asbl.github.io/tripper/latest/api_reference/triplestore/#tripper.triplestore.Namespace
[Triplestore]: https://emmc-asbl.github.io/tripper/latest/api_reference/triplestore/#tripper.triplestore.Triplestore
Expand Down
10 changes: 10 additions & 0 deletions docs/known-issues.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
Known issues
============

* If you use the rdflib backend and don't have write permissions to
the cache directory (which e.g. can happen if you run Python in
docker as a non-root user), you may get a `urllib.error.HTTPError`
error when accessing an online rdf resource.

Setting the environment variable `XDG_CACHE_HOME` to a directory
that you have write access to will solve the problem.
6 changes: 6 additions & 0 deletions tripper/backends/rdflib.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,12 @@

For developers: The usage of `s`, `p`, and `o` represent the different parts of
an RDF Triple: subject, predicate, and object.

There is a issue with rdflib raising an `urllib.error.HTTPError`
exception if you don't have write permissions to the cache directory.
See [Known issues](https://emmc-asbl.github.io/tripper/latest/known-issues.md)
for more details.

"""

# pylint: disable=line-too-long
Expand Down
57 changes: 41 additions & 16 deletions tripper/namespace.py
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@
if s.startswith(iri)
)

def _get_cachefile(self) -> "Path":
def _get_cachefile(self) -> Path:
"""Return path to cache file for this namespace."""
# pylint: disable=too-many-function-args
name = self._iri.rstrip("#/").rsplit("/", 1)[-1]
Expand All @@ -162,34 +162,59 @@
def _save_cache(self):
"""Save current cache."""
# pylint: disable=invalid-name
cachefile = self._get_cachefile()
if self._iris and not sys.is_finalizing():
with open(cachefile, "wb") as f:
pickle.dump(self._iris, f)
try:
cachefile = self._get_cachefile()
if self._iris and not sys.is_finalizing():
with open(cachefile, "wb") as f:
pickle.dump(self._iris, f)
except OSError as exc:
warnings.warn(

Check warning on line 171 in tripper/namespace.py

View check run for this annotation

Codecov / codecov/patch

tripper/namespace.py#L170-L171

Added lines #L170 - L171 were not covered by tests
f"Cannot access cache file: {exc}\n\n"
"You can select cache directory with the XDG_CACHE_HOME "
"environment variable."
)

def _load_cache(self) -> bool:
"""Update cache with cache file.

Returns true if there exists a cache file to load from.
"""
# pylint: disable=invalid-name
cachefile = self._get_cachefile()
if self._iris is None:
self._iris = {}
if cachefile.exists():
with open(cachefile, "rb") as f:
self._iris.update(pickle.load(f)) # nosec
return True
return False
try:
cachefile = self._get_cachefile()
if self._iris is None:
self._iris = {}

Check warning on line 186 in tripper/namespace.py

View check run for this annotation

Codecov / codecov/patch

tripper/namespace.py#L186

Added line #L186 was not covered by tests
if cachefile.exists():
with open(cachefile, "rb") as f:
self._iris.update(pickle.load(f)) # nosec
return True
return False
except OSError as exc:
warnings.warn(

Check warning on line 193 in tripper/namespace.py

View check run for this annotation

Codecov / codecov/patch

tripper/namespace.py#L192-L193

Added lines #L192 - L193 were not covered by tests
f"Cannot create cache directory: {exc}\n\n"
"You can select cache directory with the XDG_CACHE_HOME "
"environment variable."
)
return False

Check warning on line 198 in tripper/namespace.py

View check run for this annotation

Codecov / codecov/patch

tripper/namespace.py#L198

Added line #L198 was not covered by tests

def __getattr__(self, name):
if self._iris and name in self._iris:
return self._iris[name]
if self._check:
msg = ""
cachefile = self._get_cachefile()
if cachefile.exists():
msg = f"\nMaybe you have to remove the cache file: {cachefile}"
try:
cachefile = self._get_cachefile()
if cachefile.exists():
msg = (
"\nMaybe you have to remove the cache file: "
f"{cachefile}"
)
except OSError as exc:
warnings.warn(

Check warning on line 213 in tripper/namespace.py

View check run for this annotation

Codecov / codecov/patch

tripper/namespace.py#L212-L213

Added lines #L212 - L213 were not covered by tests
f"Cannot access cache file: {exc}\n\n"
"You can select cache directory with the XDG_CACHE_HOME "
"environment variable."
)
raise NoSuchIRIError(self._iri + name + msg)
return self._iri + name

Expand Down