-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add: do not verify hardlink if file is empty #3428
Conversation
Codecov Report
@@ Coverage Diff @@
## master #3428 +/- ##
=======================================
Coverage 93.08% 93.08%
=======================================
Files 140 140
Lines 8515 8515
=======================================
Hits 7926 7926
Misses 589 589 Continue to review full report at Codecov.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What do you think to fix the issue another way:
- in
_verify_link
check the case when the file is not a link and has size0 bytes
- if it's True - just don't raise an Exception
- put the comment with the issue and short description about the case
It allows to fix it in one place without passing inconsistent return value
UPD
I see @pared propsed the same in the comment above
We (@efiop and I) discussed this on 1o1. A better way to fix this would be to try to create a temporary file to check if the System supports |
if link_type == "hardlink" and self.getsize(path_info) == 0: | ||
return | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not great that it will be checking the size for each link it creates, might get expensive. Though, it does that in hardlink
anyway...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We did that anyway on the hardlink()
though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@skshetry Maybe there is some nicer way to do this? Like making hardlink
(and other link class methods) verify themselves? This is an honest question, I don't know myself either π
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@skshetry Could keep it as is and create a ticket for it to reconsider later. Just asking.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like this suggestion #3428 (comment) more. Running verification on hardlink
itself for only once (by setting self.cache_type_confirmed
) is also hackish.
I'd say, we revisit this on next sprint and fix it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@skshetry Ok, please create an issue and add it to the next sprint.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could just check for cache_type_verified
at the beggining again. I know that is duplication, but don't see any obvious workaround.
@skshetry Check your tests, travis failed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So, the conclusion for this issue is that we will crete issue to fix the way we handle confirmation of cache type?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One additional comment in discussion with @efiop, not anything major.
@skshetry Please check the tests. |
Fixes #3390
β Have you followed the guidelines in the Contributing to DVC list?
π Check this box if this PR does not require documentation updates, or if it does and you have created a separate PR in dvc.org with such updates (or at least opened an issue about it in that repo). Please link below to your PR (or issue) in the dvc.org repo.
β Have you checked DeepSource, CodeClimate, and other sanity checks below? We consider their findings recommendatory and don't expect everything to be addressed. Please review them carefully and fix those that actually improve code or fix bugs.
Thank you for the contribution - we'll try to review it as soon as possible. π