How to use local Unix paths with Pathy

[b2m]

Context

spaCy is describing on it's website that it can store project data either locally or remotely using Pathy.

Example Paths from the spaCy's documentation:

• s3://my-spacy-bucket
• /mnt/scratch/cache
• ssh://myserver.example.com/whatever

But using local paths like /mnt/scratch/cache does not work.

Problem

Using a local Unix path currently seems not to be supported by Pathy.
I modified the example code from the project's README;

from pathy import Pathy
# Create a bucket
Pathy("/tmp/test").mkdir(exist_ok=True)
# An excellent blob
greeting = Pathy("/tmp/test/greeting.txt")
# But it doesn't exist yet
assert not greeting.exists()
# Create it by writing some text
greeting.write_text("Hello World!")
# Now it exists
assert greeting.exists()
# Delete it
greeting.unlink()
# Now it doesn't
assert not greeting.exists()

This will result in a PermissionError: [Errno 13] Permission denied: '/test'.
Note the missing /tmp part in the path!

  Traceback (most recent call last):
  File "test_pathy.py", line 9, in <module>
    greeting.write_text("Hello World!")
  File "/usr/lib/python3.8/pathlib.py", line 1255, in write_text
    with self.open(mode='w', encoding=encoding, errors=errors) as f:
  File "/pathy_test/venv/lib/python3.8/site-packages/pathy/__init__.py", line 672, in open
    return self._accessor.open(
  File "/pathy_test/venv/lib/python3.8/site-packages/pathy/__init__.py", line 412, in open
    return self.client(path).open(
  File "/pathy_test/venv/lib/python3.8/site-packages/pathy/__init__.py", line 1001, in open
    full_path.mkdir(parents=True, exist_ok=True)
  File "/usr/lib/python3.8/pathlib.py", line 1288, in mkdir
    self._accessor.mkdir(self, mode)
PermissionError: [Errno 13] Permission denied: '/test'

So I am wondering, whether using Pathy('/some/unix/path') should be discouraged in favor of Pathy.fluid('/some/unix/path') or whether this is a bug in Pathy.

Possibly related issues:

• Support forward slash separated paths on Windows #69

[adrianeboyd]

This is a bug in spacy. I ran into the same issue when trying to use local remote storage for testing, see explosion/spaCy#11762.

[justindujardin]

Sorry for the late reply, @b2m, you should use Pathy.fluid when you are working with a path of an unknown type. You could alternatively use the pathlib.Path class if you want to explicitly represent a unix path.

[justindujardin]

I've thought about this some more, and I think there should be an error if you initialize Pathy with Unix paths instead of using Pathy.fluid. This should resolve your problem and keep people from running into the same problem in the future.

Changes coming in #95

[github-actions]

🎉 This issue has been resolved in version 0.9.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀