Fix azcopy on architectures that are strict about alignment (ARM) #1567
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The current version of azcopy dies on ARM systems with a SIGBUS and the following stacktrace:
INFO: Scanning...
INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support
unexpected fault address 0xffff72640a5f
fatal error: fault
[signal SIGBUS: bus error code=0x1 addr=0xffff72640a5f pc=0x733d8c]
goroutine 25 [running]:
runtime.throw(0x9de596, 0x5)
/usr/lib/go-1.16/src/runtime/panic.go:1117 +0x54 fp=0x4000d982f0 sp=0x4000d982c0 pc=0x43914
runtime.sigpanic()
/usr/lib/go-1.16/src/runtime/signal_unix.go:731 +0x284 fp=0x4000d98330 sp=0x4000d982f0 pc=0x5b214
github.com/Azure/azure-storage-azcopy/v10/common.(*TransferStatus).AtomicLoad(...)
/home/jeremi/github/azure-storage-azcopy-2/common/fe-ste-models.go:690
github.com/Azure/azure-storage-azcopy/v10/ste.(*JobPartPlanTransfer).TransferStatus(...)
/home/jeremi/github/azure-storage-azcopy-2/ste/JobPartPlan.go:377
github.com/Azure/azure-storage-azcopy/v10/ste.(*jobPartMgr).ScheduleTransfers(0x4000adc000, 0xb54930, 0x4000ad2000)
/home/jeremi/github/azure-storage-azcopy-2/ste/mgr-JobPartMgr.go:396 +0x42c fp=0x4000d99f20 sp=0x4000d98340 pc=0x733d8c
github.com/Azure/azure-storage-azcopy/v10/ste.(*jobsAdmin).scheduleJobParts(0x4000496000)
/home/jeremi/github/azure-storage-azcopy-2/ste/JobsAdmin.go:287 +0x44 fp=0x4000d99fd0 sp=0x4000d99f20 pc=0x724ee4
runtime.goexit()
/usr/lib/go-1.16/src/runtime/asm_arm64.s:1130 +0x4 fp=0x4000d99fd0 sp=0x4000d99fd0 pc=0x78a44
created by github.com/Azure/azure-storage-azcopy/v10/ste.initJobsAdmin
/home/jeremi/github/azure-storage-azcopy-2/ste/JobsAdmin.go:212 +0x5ec
What happens is that a JobPartPlanTransfer struct is created inside a memory
mapped 'plan' file and there are several members of that struct that are
accessed atomically. This requires that those members have the natural
alignment for their type, which is guaranteed at the struct level, but the
struct itself is not properly aligned inside the plan file. JobPartPlanTransfer
is located after the fixed size JobPartPlanHeader struct and a variable size
'CommandString'. It is that variable sized string that breaks the whole
alignment (most of the time). x86 doesn't care but ARM does not like it
one bit and SIGBUS.
To remedy this, ensure that the start of the first JobPartPlanTransfer is 8
byte aligned both when writing the file and when accessing it. This makes
azcopy work on ARM64 systems that I have tested.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
|
Running tests to check for regression. |
|
Hey @jepio , Thanks for your contribution. If you wished to compile AzCopy for Linux with 64-bit ARM, consider adding azure-storage-azcopy/azure-pipelines.yml Line 30 in 0a14fbd and here - azure-storage-azcopy/azure-pipelines.yml Line 87 in 0a14fbd |
|
Hi @mohsha-msft, |
|
I can add the lines on your behalf. 👍 |
|
Not causing any regressions. Need to check why E2E tests are failing on ARM. Branch with changes in |
|
I don't think your CI has arm64 nodes, it might be able to build the executables but not test them. For legacy arm I have another PR that I plan to submit, but it requires this jiacfan/keyctl#4 PR in a dependency first. |
mohsha-msft
added
the
pr-on-hold
mohsha-msft
added
testing-required
and removed
pr-on-hold
|
Thanks, this fixes the issue for me #882 |
|
Replaced by #1577 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR fixes azcopy on ARM architectures (tested on ARM64) by ensuring that JobPartPlanTransfer is properly 8 byte aligned inside the plan file. JobPartPlanTransfer is located after a variable size string and so its alignment was dependent on what was being copied.
Tested using go 1.16.2 and
GOARCH=arm64 go build.Fixes #972
Fixes #882
Fixes #427