Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add metadata methods #118

Merged
merged 5 commits into from
Nov 12, 2018
Merged

Add metadata methods #118

merged 5 commits into from
Nov 12, 2018

Conversation

martindurant
Copy link
Member

Fixes #38

@martindurant
Copy link
Member Author

(tests not created, recordings not updated)

@martindurant
Copy link
Member Author

@yiga2 , a quick implementation that seems to work

Copy link
Contributor

@yan-hic yan-hic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Forgot you had the getsetters cooked up too. Nice !

Copy link
Contributor

@yan-hic yan-hic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@martindurant , here my quick-n-dirty test script - which passed fine:

import gcsfs

fs = gcsfs.GCSFileSystem()
bucket='gcsfs-metadata-test'

def upload(name, block_size=fs.default_block_size, metadata=None, patch=False):
    name = '{}/{}'.format(bucket, name)

    with fs.open(name, 'wb', block_size=block_size, metadata=None if patch else metadata) as f:
        f.write(b'0' * (f.blocksize - 100))

    if patch and metadata is not None:
        fs.setxattrs(name, myname='test')

    if metadata is not None:
       attr = next(iter(metadata))
       return fs.getxattr(name, attr)


if not fs.exists(bucket):
    fs.mkdir(bucket)

# small (< block), no meta, no setter
assert upload('small_wo_meta') is None

# small (< block), meta, no setter
assert upload('small_meta_creation', metadata={'myname': 'test'}) == 'test'

# small (< block), meta, setter
assert upload('small_meta_patch', metadata={'myname': 'test'}, patch=True) == 'test'

# large (> block), no meta, no setter
assert upload('large_wo_meta', block_size=2 * fs.default_block_size) is None

# large (> block), meta, no setter
assert upload('large_meta_creation', block_size=2 * fs.default_block_size, metadata={'myname': 'test'}) == 'test'

# large (> block), meta, setter
assert upload('large_meta_patch', block_size=2 * fs.default_block_size, metadata={'myname': 'test'}, patch=True) == 'test'

for f in fs.glob(bucket + '/*'):
    fs.rm(f)
fs.rmdir(bucket)

@martindurant
Copy link
Member Author

@yiga2 , you are welcome to play with this before I merge

@yan-hic
Copy link
Contributor

yan-hic commented Nov 12, 2018

@martindurant I did - works fine ! Tested the put too.

Unless no need to hold off, may want to bundle with another (unrelated) change - if applicable - as per my comment here

@martindurant martindurant merged commit d805f1a into fsspec:master Nov 12, 2018
@martindurant martindurant deleted the metadata branch November 12, 2018 19:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants