[MRESOLVER-331] Make DefaultTrackingFileManager write directly to tracking files #259
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
asfgit
force-pushed
the
direct-write
branch
2 times, most recently
from
6d4ab15 to
4694f15
Compare
…cking files This closes #259
michael-o
changed the title
cstamas
approved these changes
Feb 26, 2023
cstamas
added a commit
that referenced
this pull request
Nov 17, 2023
Fixes: * move() call should NOT perform the move, as writer stream to tmp file may still be open * move the file move logic to close, make it happen only when closing collocated temp file * perform fsync before atomic move to ensure there is no OS dirty buffers related to newly written file * on windows go with old code that for some reason works (avoid NIO2) * on non-Win OS fsync the parent directory as well. So, to recap: * original issue is not related to locking at all, is about change done in 1.9 where file processor was altered to use nio2 atomic move (to prevent possibility of partial download being read by other process and prevent incomplete files in case of crash). Maven LRM uses layout, so randomized temp files does not matter (if they remain after crash), as they would merely just clutter the local repository but not corrupt it. * nuking local repo is part of healthy hygiene anyway * seems windows (or java on windows) have issues with atomic fs ops * resolver have no deps on p-u or any commons, api and util modules are zero dependency jars and should remain as such * one thing bugs me still: why does the old code work? Or in reverse, the new why does not work? As opposed to "high frequency writes", this is not that case (fixed by @olamy ... Um, sorry @michael-o ) To explain this last bullet: there was https://issues.apache.org/jira/browse/MRESOLVER-325 with solution #259 that "seems similar", as originally this code was used by both, and in that case "high frequency atomic moves" was applicable. But this now happens seemingly on "install" and "cache" (download, place it in local repo) that has way less frequency that tracking file write.... --- https://issues.apache.org/jira/browse/MRESOLVER-372
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.