-
Notifications
You must be signed in to change notification settings - Fork 2.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[DataLake][SetExpiry]Set Expiry of DataLake File #11629
Conversation
* [Blob][Swagger]Regenerate Swagger Code * fix container test failure caused by list_containers include type change
* initial avro parser * try fixing test... * falling in love with python compatibility... * make linter happy. * raise StopIteration when there is no more bytes instead of tracking file length. * async avro parser * fix syntax for Python 3.5 * get rid of 'readers_schema' as we only honor schema that has been written to file ('writer_schema'). * pr feedback * trim unused code. * pr feedback. * simplify skip sync in next. * move avro tests from _shared.
* wip * initial test coverage. * wip. * wip * single upload. * add async tests. * disable 50k block tests. * datalake append. * async datalake * disable tests that send large payload over network. * pr feedback.
cpk_info=cpk_info, | ||
**kwargs) | ||
except StorageErrorException as error: | ||
process_storage_error(error) | ||
blob_props.name = self.blob_name | ||
blob_props.container = self.container_name | ||
if isinstance(blob_props, BlobProperties): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please use
try:
##
except AttributeError:
###
If mypy is complaining, use cast()
cpk_info=cpk_info, | ||
**kwargs) | ||
except StorageErrorException as error: | ||
process_storage_error(error) | ||
blob_props.name = self.blob_name | ||
blob_props.container = self.container_name | ||
if isinstance(blob_props, BlobProperties): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same here
https://docs.python.org/2/glossary.html#term-eafp
interesting read about why - Easier to ask for forgiveness than permission
return self._get_path_properties(cls=FileProperties._deserialize_file_properties, **kwargs) # pylint: disable=protected-access | ||
|
||
def set_file_expiry(self, expiry_options, expires_on=None, **kwargs): | ||
# type: (**Any) -> None |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
wrong type hint
self.lease = None | ||
self.last_modified = None | ||
self.creation_time = None | ||
self.metadata = kwargs.get('metadata') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should these all be kwargs.get('metadata', None)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's not needed - get
already returns None by default.
self.state = None | ||
self.duration = None | ||
def __init__(self, **kwargs): | ||
super(LeaseProperties, self).__init__( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i dont think this init is needed to just call the super
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
agree!
return await self._get_path_properties(cls=FileProperties._deserialize_file_properties, **kwargs) # pylint: disable=protected-access | ||
|
||
async def set_file_expiry(self, expiry_options, expires_on=None, **kwargs): | ||
# type: (**Any) -> None |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
wrong type hint
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, just a few minor comments :)
a4f697b
to
60ada0b
Compare
No description provided.