-
Notifications
You must be signed in to change notification settings - Fork 63
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Prefetch all files on clone #56
Prefetch all files on clone #56
Conversation
Will disable later when doing sparse clone Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
/azp run microsoft.scalar |
Azure Pipelines successfully started running 1 pipeline(s). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice speedup!
...adding a comment made the Mac functional tests sit forever. Ordering the tests that were not reporting logs fixed the problem... I don't even know anymore. I'll wait until morning for this one. |
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
@@ -180,8 +180,10 @@ private void ExpandOneTempPack(bool copyPackBackToPackDirectory) | |||
string[] packFiles = Directory.GetFiles(this.TempPackRoot, "pack-*.pack"); | |||
Assert.Greater(packFiles.Length, 0); | |||
|
|||
// Pick the first one found | |||
string packFile = packFiles[0]; | |||
List<FileInfo> fileInfos = packFiles.Select(file => new FileInfo(file)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was able to repro the functional test hang, and it is related to unpacking the prefetch pack. To fix the problem long-term, we will want to make these tests use the sparse mode, which avoids the prefetch. Temporarily, we can select the smallest pack to turn into loose objects.
Long term, we will not have a mount, so this feature will eventually be meaningless. In the meantime, it is only causing possible problems with hydrating files. Resolves #26 permanently. In a normal clone, we already resolved it in #56 by prefetching all files on clone. However, we still dynamically prefetch on a sparse clone (#54).
Will disable later when doing sparse clone. This speeds up our functional tests (and any clone we do for local testing).
With the current count of functional tests, they run in 3-4 minutes instead of 5-7 minutes. As that list grows, this will be even more important.