Build libsodium.dll with VC++ for win-arm64
#1431
Conversation
- update version check to use **greater-than-or-equal**, i.e. to include VS 2019 **and** 2022 (or later versions) - select the `ARM64` environment (`x86_arm64` is not valid)
Build `win-arm64` native library with VC++ on Windows; may help with #1430
|
@jedisct1 @mlugg I may need your help with the GitHub CI failure in https://github.com/jedisct1/libsodium/actions/runs/12160019675
Is there anything I can do in my fork to get CI back on its feet and able to run on this PR? Thank you in advance! |
|
Could we fix what's wrong with the files created by Zig instead? |
|
I do not know what's wrong with them, and I'm afraid I don't know how to debug Zig or native code. Happy to take a deeper look if someone can advise what exactly I should look at; I had hoped that by providing repro steps we'd have a better chance of figuring out what's happening. In any case, given that in the past several other people mentioned |
|
I understand that this hack may fix your problem on a specific architecture, but fixing the root cause would be better and would address a much broader problem possibly affecting other applications. If there's a bug in Zig that builds libraries that .NET cannot parse, it's a good opportunity to understand why. As I don't use Windows and don't know anything about .NET, testing the Visual Studio builds is very painful. Cross-compiling everything with the same compiler makes things a little bit easier. |
|
I understand your point of view, and I admire Zig as a language and a toolset that can cross-compile for many platforms. For instance, we built Here is a little bit of information that I could gather from Comparing Dependents: NuGet package (Zig)Dependents: locally-built (VC++)A couple of differences that could (perhaps?) be relevant:
One step further, the list of symbols imported from each dependent DLLs, from Imports: NuGet package (Zig)Imports: locally-built (VC++)This shows that the Zig-built DLL imports a few functions from
While the VC++-built DLL imports two of them ( I do not know if this can be related to the fact that the DLL doesn't load as expected. Perhaps there is a C runtime API, specific to ARM64, that VCRUNTIME abstracts away from apps? I read that Zig tries to minimize platform dependencies as much as possible, and on Windows strives to only use kernel-exposed APIs, and perhaps the DLL load failure might be related? |
|
The MS docs at https://learn.microsoft.com/en-us/cpp/windows/universal-crt-deployment?view=msvc-170#local-deployment mention this:
Comparing the That is, > cd "C:\Program Files (x86)\Windows Kits\10\Redist\10.0.26100.0\ucrt\DLLs"
> dir ARM64\api-ms-win-crt-private-l1-1-0.dll
File Not Found
> dir x64\api-ms-win-crt-private-l1-1-0.dll
2024-09-05 14:56 75,328 api-ms-win-crt-private-l1-1-0.dll
1 File(s) 75,328 bytes
> dir x86\api-ms-win-crt-private-l1-1-0.dll
2024-09-05 14:56 67,624 api-ms-win-crt-private-l1-1-0.dll
1 File(s) 67,624 bytesI hope this helps, and am very much open to suggestion to anything else I could look at. |
|
@jedisct1 I tried removing .NET from the equation, and instead wrote a simple C program to load the arm64 #include <stdio.h>
#include <windows.h>
int main()
{
HMODULE module = LoadLibraryW(L"C:\\Users\\nil4\\.nuget\\packages\\libsodium\\1.0.20\\runtimes\\win-arm64\\native\\libsodium.dll");
if (module == NULL) {
printf("LoadLibrary failed: %08x\n", GetLastError());
return 1;
}
typedef int(__cdecl *sodium_init_p)(void);
sodium_init_p sodium_init = (sodium_init_p)GetProcAddress(module, "sodium_init");
if (sodium_init == NULL) {
printf("GetProcAddress: %08x\n", GetLastError());
return 1;
}
int result = sodium_init();
printf("sodium_init() returned %d", result);
FreeLibrary(module);
return 0;
}This failed as well, showing a Windows error message box:
|
|
I also tried loading > zig env
{
[...]
"version": "0.13.0",
"target": "aarch64-windows.win10_fe...win10_fe-gnu",
}
> mkdir reprozig && cd reprozig
> zig init
> zig fetch --save=zigwin32 https://github.com/marlersoft/zigwin32/archive/main.tar.gzReplace the entire const builtin = @import("builtin");
const std = @import("std");
pub fn build(b: *std.Build) void {
const target = b.standardTargetOptions(.{ .default_target = .{ .os_tag = .windows, .cpu_arch = .aarch64 } });
const optimize = b.standardOptimizeOption(.{});
const exe = b.addExecutable(.{
.name = "repro-zig",
.root_source_file = b.path("src/repro.zig"),
.target = target,
.optimize = optimize,
.link_libc = false,
});
const zigwin32 = b.dependency("zigwin32", .{});
exe.root_module.addImport("zigwin32", zigwin32.module("zigwin32"));
b.installArtifact(exe);
}Delete const std = @import("std");
const win32 = struct {
usingnamespace @import("zigwin32").zig;
usingnamespace @import("zigwin32").foundation;
usingnamespace @import("zigwin32").system.library_loader;
};
const L = win32.L;
pub fn main() void {
const libsodium = L("C:\\Users\\nil4\\.nuget\\packages\\libsodium\\1.0.20\\runtimes\\win-arm64\\native\\libsodium.dll");
const module: win32.HINSTANCE = win32.LoadLibraryW(libsodium) orelse {
std.debug.panic("LoadLibrary failed: {}", .{win32.GetLastError()});
};
const ptr = win32.GetProcAddress(module, "sodium_init") orelse {
std.debug.panic("GetProcAddress failed: {}", .{win32.GetLastError()});
};
const sodium_init_p = *const fn () callconv(.C) c_int;
const sodium_init: sodium_init_p = @ptrCast(ptr);
const result = sodium_init();
std.debug.print("sodium_init() returned {}", .{result});
_ = win32.FreeLibrary(module);
}Build and run the program to confirm the issue: > zig build && zig-out\bin\repro-zig.exe
thread 13708 panic: LoadLibrary failed: win32.foundation.WIN32_ERROR.ERROR_BAD_EXE_FORMAT
C:\dev\reprozig\src\repro.zig:13:24: 0x7ff78fa610b3 in main (repro-zig.exe.obj)
std.debug.panic("LoadLibrary failed: {}", .{win32.GetLastError()});
^
C:\dev\zig\lib\std\start.zig:363:53: 0x7ff78fa6100f in WinStartup (repro-zig.exe.obj)
std.os.windows.ntdll.RtlExitUserProcess(callMain());
^
???:?:?: 0x7ff88195873f in ??? (KERNEL32.DLL)
???:?:?: 0x7ff884cc0083 in ??? (ntdll.dll)It looks like neither .NET, C or Zig are able to load the current @jedisct1 if there is anything else I can do to help inform a decision about this PR, please let me know -- I would appreciate feedback. |
|
I spent a couple hours fighting with UTM to run Windows. And... the const std = @import("std");
const windows = std.os.windows;
const W = std.unicode.utf8ToUtf16LeStringLiteral;
pub fn main() !void {
const lib = try windows.LoadLibraryW(W("libsodium.dll"));
const addr = windows.kernel32.GetProcAddress(lib, "sodium_version_string").?;
const sodium_version_string: *fn () [*:0]u8 = @ptrCast(@alignCast(addr));
std.debug.print("libsodium version: {s}\n", .{sodium_version_string()});
}PS Z:\> .\dll-load
libsodium version: 1.0.20So I'm really confused. |
|
Ahah! The DLL loads fine if I build it locally, but it fails with the |
|
Alright, was about to reply that now I'm stumped and starting to question my reality! Glad to hear that the difference is in fact there! 😌 I really appreciate you going the extra mile to set up UTM and ARM64 Windows -- good that it wasn't in vain after all! If there's anything I can do to help untangle more mysteries, I'm happy to chime in if I can. |
|
Could it be that the build host being ARM or non-ARM makes a difference to Zig? (Sounds like both of our local build hosts are ARM64 vs. GitHub CI's x64) Or maybe the Zig version used in CI has an issue when cross-compiling to another Windows architecture? In any case, something's off with the CI binary -- good that we have it confirmed now! |
|
Updating Zig in the CI to Zig 0.14 fixes the issue! |
Update
buildbase.batforwin-arm64:ARM64environment (x86_arm64is not valid)Update
dotnet-core.ymlforwin-arm64:libsodium.dllwith VC++ on Windows.libsodiumNuGet package unusable onwin-arm64#1430