This repository was archived by the owner on Jan 23, 2023. It is now read-only.
[release/3.1] Single file: Guard against partial cleanup of extracted files #9013
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
swaroop-sridhar
force-pushed
the
restore
branch
from
8a30c53 to
f602a7d
Compare
… files ** Issue dotnet/runtime#3778 ** Customer Scenario Single-file apps rely on extractiong their contents to disk. The extaction is performed to a temp-directory by default, which may be cleaned up by the OS. The app can re-extract itself if the entire extraction is cleaned up -- but not partial deletions. Customers have noticed that for some apps, the extracted files were removed, but the extraction directory itself wasn't. This causes the app to fail to start. ** Problem When executing single-file apps, if a pre-existing extraction exists, it is assumed to be valid. ** Solution When executing single-file apps, if a pre-existing extraction exists, the contents of the extraction directory are now verified. If files are missing, they are recovered. **Extraction Algorithm** `ExtractionDir` = `$DOTNET_BUNDLE_EXTRACT_BASE_DIR/<app>/<bundle-id>` `WorkingDir` = `$DOTNET_BUNDLE_EXTRACT_BASE_DIR/<app>/<process-id>` If `ExtractionDir` does not exist, then Create `WorkingDir`, and extract bundle contents within it Attempt to rename `WorkingDir` as `ExtractionDir` If the rename succeeds, continue execution of the app If not, another process completed extraction; Verify and reuse this extraction (as in step 2). If `ExtractionDir` exists, then verify that it contains all files listed in the manifest. If all contents are available, Remove `WorkingDir` Continue execution of the app, reusing the contents. If certain files are missing within `ExtractionDir`, then For each missing file, do the following individually Extract the files within `WorkingDir` Create sub-paths within `ExtractionDir` if necessary Rename the file from `WorkingDir/path/<file>` to `ExtractionDir/path/<file>` unless `ExtractionDir/path/<file>` exists (extracted there by another process in the meantime) Remove `WorkingDir` Continue execution of the app. All of the renames above are done with appropriate retries to circumvent interference from anti-virus apps. ** Startup time impact * Console HelloWorld execution time: * Framework dependent app: Windows/Linux No measurable difference * Self-contained app: Windows: ~10ms additional * Self-contained app: Linux: No measurable difference * Greenshot Startup: * Self-contained Windows: No noticiable/measurable difference * NugetPackageExplorer Startup: * Self-contained Windows: No noticiable/measurable difference ** Risk Medium. The change is well scoped, but isn't trivial. It affects all single-file apps. ** Master Branch dotnet/runtime#32649
swaroop-sridhar
force-pushed
the
restore
branch
from
f602a7d to
cce3b15
Compare
swaroop-sridhar
added
area-Host
Servicing-consider
lpereira
approved these changes
Feb 29, 2020
jeffschwMSFT
approved these changes
Mar 3, 2020
leecow
added
Servicing-approved
|
@swaroop-sridhar can you please resolve the merge conflict here ? |
|
OK @Anipik I'll do it now |
swaroop-sridhar
force-pushed
the
restore
branch
from
1c87572 to
cce3b15
Compare
|
@Anipik the conflict is now resolved. Thanks. |
|
Really glad to see a fix for this. Has a date been set for the 3.1.4 release? |
|
@Anipik do you have the schedule for the May release? Thanks. |
|
The current date for release is Mid-May but it might change |
This was referenced May 6, 2020
Viir
added a commit
to Viir/bots
that referenced
this pull request
May 14, 2020
Let's see if the improvements coming with dotnet core SDK 3.1.202 are sufficient to fix the problem that caused us to avoid the single-file bundling. Last year, several customers were blocked by issues specific to the single-file distribution, causing us to avoid it until now. The error reports showed several people got the "The application to execute does not exist" error on application startup: + 6a41758 + #3 (comment) + dotnet/core#3830 (comment) + https://forum.botengine.org/t/how-to-automate-mining-asteroids-in-eve-online/628/94?u=viir For the recent improvement in .net core, see (also): + dotnet/core-setup#9013 + https://github.com/dotnet/core/blob/0e083d8449ab3c11c1dd7a769e21d12eaceb782e/release-notes/3.1/3.1.4/3.1.4.md + https://devblogs.microsoft.com/dotnet/net-core-may-2020/
Viir
added a commit
to Viir/bots
that referenced
this pull request
May 14, 2020
Let's see if the improvements coming with dotnet core SDK 3.1.202 are sufficient to fix the problem that caused us to avoid the single-file bundling. Last year, several users were blocked by issues specific to the single-file distribution, causing us to avoid it until now. The error reports showed several people got the "The application to execute does not exist" error on application startup: + 6a41758 + #3 (comment) + dotnet/core#3830 (comment) + https://forum.botengine.org/t/how-to-automate-mining-asteroids-in-eve-online/628/94?u=viir For the recent improvement in .net core, see (also): + dotnet/core-setup#9013 + https://github.com/dotnet/core/blob/0e083d8449ab3c11c1dd7a769e21d12eaceb782e/release-notes/3.1/3.1.4/3.1.4.md + https://devblogs.microsoft.com/dotnet/net-core-may-2020/
Viir
added a commit
to pine-vm/pine
that referenced
this pull request
May 15, 2020
Let's see if the single-file bundling works with dotnet core SDK 3.1.202. For the recent improvement in .net core, see (also): + dotnet/core-setup#9013 + https://github.com/dotnet/core/blob/0e083d8449ab3c11c1dd7a769e21d12eaceb782e/release-notes/3.1/3.1.4/3.1.4.md + https://devblogs.microsoft.com/dotnet/net-core-may-2020/ Last year, several users were blocked by issues specific to the single-file distribution, causing us to avoid it until now. The error reports showed several people got the "The application to execute does not exist" error on application startup: + Viir/bots@6a41758 + Viir/bots#3 (comment) + dotnet/core#3830 (comment) + https://forum.botengine.org/t/how-to-automate-mining-asteroids-in-eve-online/628/94?u=viir
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue
dotnet/runtime#3778
Customer Scenario
Single-file apps rely on extractiong their contents to disk.
The extaction is performed to a temp-directory by default, which may be cleaned up by the OS.
The app can re-extract itself if the entire extraction is cleaned up -- but not partial deletions.
Customers have noticed that for some apps, the extracted files were removed, but the extraction directory itself wasn't.
This causes the app to fail to start.
Problem
When executing single-file apps, if a pre-existing extraction exists, it is assumed to be valid.
Solution
When executing single-file apps, if a pre-existing extraction exists, the contents of the extraction directory are now verified. If files are missing, they are recovered.
Extraction Algorithm
ExtractionDir=$DOTNET_BUNDLE_EXTRACT_BASE_DIR/<app>/<bundle-id>WorkingDir=$DOTNET_BUNDLE_EXTRACT_BASE_DIR/<app>/<process-id>If
ExtractionDirdoes not exist, thenCreate
WorkingDir, and extract bundle contents within itAttempt to rename
WorkingDirasExtractionDirIf the rename succeeds, continue execution of the app
If not, another process completed extraction; Verify and reuse this extraction (as in step 2).
If
ExtractionDirexists, then verify that it contains all files listed in the manifest.If all contents are available,
Remove
WorkingDirContinue execution of the app, reusing the contents.
If certain files are missing within
ExtractionDir, thenFor each missing file, do the following individually
Extract the files within
WorkingDirCreate sub-paths within
ExtractionDirif necessaryRename the file from
WorkingDir/path/<file>toExtractionDir/path/<file>unlessExtractionDir/path/<file>exists (extracted there by another process in the meantime)Remove
WorkingDirContinue execution of the app.
All of the renames above are done with appropriate retries to circumvent interference from anti-virus apps.
Startup time impact
Risk
Medium.
The change is well scoped, but isn't trivial.
It affects all single-file apps.
Master Branch
dotnet/runtime#32649