Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data loss possibility calling 'CopyAsync' or 'CopyDirectoryAsync' with CopyMethod.ServiceSideSyncCopy (To fix in 2.0.1) #244

Closed
vinjiang opened this issue Nov 20, 2020 · 0 comments
Assignees
Labels

Comments

@vinjiang
Copy link
Member

vinjiang commented Nov 20, 2020

We just identified a code defect that may result unexpected data on destination under the following conditions:

  • Invoke DMLib with TransferManager.CopyAsync(source, dest, CopyMethod.ServiceSideSyncCopy) and TransferManager.CopyDirectoryAsync(sourceDir, destDir, CopyMethod.ServiceSideSyncCopy)
  • source and dest are both Page Blob or File
  • Blob/File in transfer has size smaller than 148MB won't hit this issue. The larger size file will have higher possibility on hitting this issue
  • DMLib version is 1.3 or 2.0

This regression was introduced by this commit and the fix is in this PR. We are going to release a hotfix once the release validation is done. Before the hotfix is ready, we strongly recommend to stop using the specific version(s) of DMLib under the above situation; alternatively, you may use SyncCopy or ServicesideAsyncCopy to workaround this issue. For more details about the difference among the copy methods, please refer to CopyMethod.

To identify potential Page Blob objects impacted by this issue, you may check the logging (Blob only) on the copy destination account with the following criteria:

  • The Operation can be: PutPageFromUrl(For page blob)
  • The Usage Agent can contain "DataMovement/2.0.0.0", "DataMovement/2.0.0.3", "DataMovement/1.3.0.0"

Here is a sample:
1.0;2020-11-19T10:21:22.0000345Z;PutPageFromUrl;Success;201;562;562;authenticated;emmauswest;emmauswest;blob;"https://emmauswest.blob.core.windows.net/vhddest/VHD8?comp=page&update&timeout=300";"/emmauswest/vhddest/VHD8";80cb6796-c01e-006f-615d-be8bbf000000;0;167.220.255.40;2019-07-07;834;0;318;0;0;;"5e6eLz/Oe1ax1CnJQAlNGg==";;Thursday, 19-Nov-20 10:21:22 GMT;;"DataMovement/2.0.0.0 Azure-Storage/11.2.2 (.NET CLR 4.0.30319.42000; Win32NT 6.2.9200.0)";;"c82268e2-9a01-4484-901f-d7d9fd796f5d"

We sincerely apologize for the impact this issue is causing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants