UPath created with specific filesystem does not use that filesystem

[alex-at-reliant]

I am setting up

fs = fsspec.filesystem(
  "filecache",
  target_protocol="gs", 
  cache_storage="/tmp/some/cache/folder/", 
)

to do

path = UPath("gs://bucket/some/path/to/some/file.json", fs=fs)

Then path.read_text() returns the right file content, but no file is cached in /tmp/some/cache/folder/. Moreover, path.fs returns the vanilla gcsfs.core.GCSFileSystem, not the expected WholeFileCacheFileSystem.

Am I using the API wrong? How does one force a filesystem to be used with upath?

[ap--]

Hi Alex,

there are two issues at hand here.

1. UPath.__init__ does not support a fs keyword argument, which means you can't provide an instantiated filesystem to UPath.

2. chained filesystems as you're trying to use with the filecache are currently not supported. Once wip: start implementing chain functionality #346 is completed and merged this functionality will be available and supported.

That being said, you could try the following right now, though it is untested and might not work:

# if this works I would not recommend to using it.
# better wait for the next release when chained urlpath support is available
pth = UPath(
  "bucket/some/path/to/some/file.json"
  protocol="filecache",
  target_protocol="gs", 
  cache_storage="/tmp/some/cache/folder/", 
)

Also, if you don't mind me asking: Was there some text in the documentation, that made you assume you can pass fs=filesystem_instance to UPath as a keyword argument?

[ap--]

Partial duplicate of #207

[alex-at-reliant]

Thanks a lot for the quick feedback. I don't remember how I came about this reference in the flurry of searches I did to see how I could cache stuff with fsspec and upath, unfortunately. And I remember not being able to confirm if filesystem or fs is a viable kwarg for upath.UPath by just looking at the code, due to the indirection around how kwargs are blindly passed around.