Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to zig version 0.14.0-dev.2802+257054a14 #173

Conversation

rohlem
Copy link
Collaborator

@rohlem rohlem commented Jan 10, 2025

Continuation of #119 .
Some dependencies also aren't upstream-updated yet, so I created updated forks and linked to them in build.zig.zon for now.

With this, on Windows 11 I was able to build and run all examples (sdl-(standalone|ontop), dx11-(standalone|ontop), raylib-(standalone|ontop), web-test).
I also tried to get the -Dsdl3 flag working, and got it from compile errors to "unable to find artifact 'sdl3'".
EDIT: However, web-test currently requires a workaround that breaks for non-Debug optimization modes. (See below.)

Most of the changes should be straightforward, the main motivators being ziglang/zig#21225 and ziglang/zig#16865 / ziglang/zig#21817 .

The only unfortunate change/workaround is in build.zig:
ziglang/zig#22240 removed libc support for the target wasm32-freestanding-musl.
This means for web-test, when compiling module dvui_mod_web for this target, libc headers aren't found:

...\dvui\src/stb/stb_truetype.h:441:13: error: 'math.h' file not found
   #include <math.h>
            ^~~~~~~~~
...\dvui\src/stb/stb_truetype_impl.c:8:10: note: in file included from ...\dvui\src/stb/stb_truetype_impl.c:8:
#include "stb_truetype.h"
         ^
...\dvui\src/stb/stb_image.h:385:10: error: 'stdlib.h' file not found
#include <stdlib.h>
         ^~~~~~~~~~~
...\dvui\src/stb/stb_image_impl.c:10:10: note: in file included from ...\dvui\src/stb/stb_image_impl.c:10:
#include "stb_image.h"
         ^
src\dvui.zig:63:15: error: C import failed
pub const c = @cImport({
              ^~~~~~~~
...\dvui\src\stb/stb_image.h:370:10: error: 'stdio.h' file not found
#include <stdio.h>
         ^

Fwict the primary motivation for this was that freestanding means having no system interface, while musl as an ABI implies a particular system interface.
However, the headers here are primarily math helper functions. Previously zig would provide these for this target, now it doesn't anymore.

The fix should be to instead specify a libc-available target, such as wasm32-wasi-musl, however linking two objects for this target together currently results in a duplicate symbol error:

error: wasm-ld: duplicate symbol: __stack_chk_guard
    note: defined in ...\zig\o\19a6733837159280ecaa4b92d137e967\libc.a(>..\zig\o\f831e7109012a241701a8177894b04e1\__stack_chk_fail.o)
    note: defined in ...\dvui\.zig-cache\o\c632e504dcd72bcac6a51b59beb9385f\web-test.wasm.o
error: wasm-ld: duplicate symbol: __stack_chk_fail
    note: defined in ...\zig\o\19a6733837159280ecaa4b92d137e967\libc.a(...\zig\o\f831e7109012a241701a8177894b04e1\__stack_chk_fail.o)
    note: defined in ...\dvui\.zig-cache\o\c632e504dcd72bcac6a51b59beb9385f\web-test.wasm.o

I think this might well be an upstream zig issue (those two symbols should be weak or something).
I haven't found mention of this on the GitHub issue tracker (though there's similar issue ziglang/zig#22096 ),
however someone reported it in a #zig-help discord thread: https://discord.com/channels/605571803288698900/1302304150792568862

The workaround for now seems to be to target wasi for only one object, and freestanding for all others.
An upstream report would probably be good, however I don't know much about wasm, so if someone else were able to reduce it to a minimal example, that would be great.

Also note that the target workaround actually only seems to work for Debug optimization mode.
With any other optimization mode, zig refuses to link because wasm-ld warns wasm-ld: warning: Linking two modules of different target triples: '...\dvui\.zig-cache\o\6222e68294243b2dfd92cc297ecb1962\web-test.wasm.o' is 'wasm32-unknown-unknown-musl' whereas 'ld-temp.o' is 'wasm32-unknown-wasi0.1.0-musl'.
So this really is just a brittle, temporary workaround.

@rohlem rohlem changed the title Update zig to 0.14.0 dev.2628+5b5c60f43 Update to zig version 0.14.0-dev.2628+5b5c60f43 Jan 10, 2025
@david-vanderson
Copy link
Owner

This is awesome, I really appreciate it!

@rohlem rohlem force-pushed the update-zig-to-0.14.0-dev.2628+5b5c60f43 branch from 92d3993 to 71e4c81 Compare January 11, 2025 15:53
@MichaelBelousov
Copy link
Collaborator

MichaelBelousov commented Jan 17, 2025

I ran into this earlier (the #zig-help thread and one of the gh issues is actually mine).
I plan on taking a look at the web workaround this week... hopefully if it's a compiler bug we can get it on the radar for the 0.14.0 release

@MichaelBelousov
Copy link
Collaborator

@rohlem I can't seem to build your branch on your fork:

mike@mikeframework ~/projects/dvui (update-zig-to-0.14.0-dev.2628+5b5c60f43) 11:09:05.874
$ zig build
/home/mike/.cache/zig/p/12207ab67f1a9f7994f5ffcba0b8294360d95ab583026934e750848179fc48dab735/build.zig:101:43: error: expected 1 argument(s), found 2
        const win_sdk = std.zig.WindowsSdk.find(b.allocator, target.result.cpu.arch) catch std.zig.WindowsSdk{ .windows10sdk = null, .windows81sdk = null, .msvc_lib_dir = null }; //try

do you know why off the top of your head? Looks potentially related to forks you made of the dependencies.

@rohlem
Copy link
Collaborator Author

rohlem commented Jan 18, 2025

@MichaelBelousov that function signature was changed from 1 to 2 arguments in ziglang/zig@ba37a43 , so the easiest explanation would be if you're using a Zig version before that.
I just tried cleaning all my caches and rebuilding, and the web-test step failed with "file not found" this time (still on version 0.14.0-dev.2628+5b5c60f43 though).
I'll try updating to current downloadable master, getting it to work again, and reporting the steps to get there if simple zig build doesn't cut it.

@MichaelBelousov
Copy link
Collaborator

that makes perfect sense, thank you for pointing that out, I did in fact just try using a an older 0.14.0 I had lying around and hoped it would work. Sorry I should have tried upgrading earlier. I'll take a look later again

@rohlem
Copy link
Collaborator Author

rohlem commented Jan 18, 2025

Okay, when initially building step web-test with zig version 0.14.0-dev.2628+5b5c60f43, the build for me (on Windows 11) fails with what looks like a linker / file management error internal to zig (maybe LLVM/lld?):

error: warning(compilation): failed to delete '...\dvui\.zig-cache\tmp\5a56515740f53e9a-stb_truetype_impl.o.d': FileNotFound
warning(compilation): failed to delete '...\dvui\.zig-cache\tmp\a8a96324d3ea619b-stb_image_impl.o.d': FileNotFound

To make these sub-compilations succeed, webtarget_exe.os_tag has to be .wasi instead of .freestanding.
This will fail the combined compilation, but the sub-compilation succeeds, after which combined compilation (with webtarget_exe.os_tag = .freestanding) works again.
(I'm adding a commit with that an inline comment in build.zig.)
It's possible that the new wasm-linker work in ziglang/zig#22220 could fix / interact with this, no idea.
(Or maybe in this configuration the build process still uses lld and that is unrelated, idk.)

On the front of updating to current ziglang.org-downloadable master (0.14.0-dev.2802+257054a14), that's a bit more messy.
Some translate-c output of an SDL header now generates a type error on bit-or | between c_int and c_uint.
I didn't see any commits referencing translate-c, so maybe this is a side-effect of a change in zig's own type analysis?
I'll try reducing it into an upstream issue. (EDIT: opened ziglang/zig#22532)
Since a workaround would probably be too involved I'll keep that version (with broken SDL integration) on a separate branch for now.

@david-vanderson
Copy link
Owner

david-vanderson commented Jan 18, 2025

Some translate-c output of an SDL header now generates a type error on bit-or | between c_int and c_uint.

ziglang/zig#21220 and I saw somebody on discord work around it by doing an explicit @as(ENUM_TYPE, ENUM_VALUE) but unsure if that helps here?

@rohlem
Copy link
Collaborator Author

rohlem commented Jan 18, 2025

Some translate-c output of an SDL header now generates a type error on bit-or | between c_int and c_uint.

ziglang/zig#21220 and I saw somebody on discord work around it by doing an explicit @as(ENUM_TYPE, ENUM_VALUE) but unsure if that helps here?

Eh, that's also related to types, but a bit of a different scenario.
I know how I would fix the code (either wrap X in @as(c_uint, @bitCast(X)) in Zig, or explicitly cast the argument in C),
but if the C code is valid (and I believe it is), it's a translate-c issue that should be fixed upstream.

The explicit cast in C would probably be the better workaround in the meantime.
I didn't really want to invest more time, but I guess I'll try it.

@david-vanderson
Copy link
Owner

The explicit cast in C would probably be the better workaround in the meantime.
I didn't really want to invest more time, but I guess I'll try it.

Ah I see - yes that makes sense. I don't meant to pressure you into sinking more time into it. I very much appreciate the effort you've already put in!

@rohlem rohlem force-pushed the update-zig-to-0.14.0-dev.2628+5b5c60f43 branch from 3f570a1 to af56131 Compare January 18, 2025 16:41
@rohlem rohlem changed the title Update to zig version 0.14.0-dev.2628+5b5c60f43 Update to zig version 0.14.0-dev.2802+257054a14 Jan 18, 2025
@rohlem
Copy link
Collaborator Author

rohlem commented Jan 18, 2025

I don't meant to pressure you into sinking more time into it. I very much appreciate the effort you've already put in!

No worries! I was just wary of maybe a dozen more issues creeping up, but that constant was the only place that triggered a compile error.

I rebased onto current main branch and added a commit to update to 0.14.0-dev.2802+257054a14.
(But kept my branch name because iirc renaming can cause issues like the PR closing.)

zig build of all steps now works for me, except that web-test requires the webtarget_exe workaround in build.zig for the first build - see comment here and earlier issue comment.
Since target .wasi succeeds with the sub-compilation, compiling both modules to .wasi would be the ideal solution (and is probably intended),
the only blocker being the duplicate symbol error on __stack_chk_guard and __stack_chk_fail. (still unreduced, not yet reported upstream)

(EDIT: new version 0.14.0-dev.2837+f38d7a92c shows the same behavior.)

@david-vanderson
Copy link
Owner

Thank you!!

@david-vanderson david-vanderson merged commit e50cfbf into david-vanderson:main Jan 21, 2025
@david-vanderson
Copy link
Owner

@rohlem I'm getting this when compiling against sdl2:

/home/dvanderson/.cache/zig/p/12200aa79b05aaeefff144b9e376371e2e7ddc982b9207d146163baf56361331a834/src/stdlib/SDL_string.c:373:12: error: call to undeclared function 'wcslcpy'; ISO C99 and later do not support implicit function declarations
return wcslcpy(dst, src, maxlen);

I'm going to work around it for now, but does that ring any bells for you?

@rohlem
Copy link
Collaborator Author

rohlem commented Jan 21, 2025

Nope, haven't run into that. It reads like a missing #include in SDL's source?
Apparently it's a POSIX function; #include <wchar.h> should be the correct include for it, which looks like it's included in SDL_string.c via SDL_stdinc.h.
Are you compiling with system libc or zig provided? gnu.org says it's "missing on many systems".

@david-vanderson
Copy link
Owner

Good question, I'm not sure. We are setting .link_libc = true in build.zig and I'm getting the error from:

zig build-lib <many SDL files> -ODebug -I /home/dvanderson/.cache/zig/p/12200aa79b05aaeefff144b9e376371e2e7ddc982b9207d146163baf56361331a834/include -I /home/dvanderson/code/dvui/.zig-cache/o/8baaa61a3c1a2bdde19252fa228d348c/ -I /home/dvanderson/code/dvui/.zig-cache/o/76093cf1769599eadcd71b2e28c303f0/ -DSDL_USE_BUILTIN_OPENGL_DEFINITIONS=1 -DHAVE_GCC_ATOMICS=1 -DHAVE_GCC_SYNC_LOCK_TEST_AND_SET=1 -DUSING_GENERATED_CONFIG_H= -DSDL_VIDEO_RENDER_SW=1 -Mroot -lc --cache-dir /home/dvanderson/code/dvui/.zig-cache --global-cache-dir /home/dvanderson/.cache/zig --name SDL2 -static --zig-lib-dir /home/dvanderson/apps/zig-linux-x86_64-0.14.0-dev.2851+b074fb7dd/lib/ --listen=-

I always assumed it was the system libc when I'm not cross compiling. This is on Linux Mint 21.3.

@rohlem
Copy link
Collaborator Author

rohlem commented Jan 21, 2025

I always assumed it was the system libc when I'm not cross compiling. This is on Linux Mint 21.3.

I believe that's true. It might change as soon as you specify a -Dtarget parameter because that then counts as cross-compilation (which also doesn't optimize to your specific cpu model) iirc?
What should definitely work is specifying a particular version of glibc to make Zig provide it. If I'm googling correctly these functions were introduced in glibc 2.38,
so adding "-Dtarget=x86_64-linux-gnu.2.38" / "-Dtarget=native-native-gnu.2.38" as argument to zig build should do the trick.
(It might still make sense to disable them in the SDL header by default though I guess.)

@david-vanderson
Copy link
Owner

Thanks very much for the info - you are totally right. Looks like my system is still on glibc 2.35. I agree - we'll keep them disabled for now. Thanks!

@MichaelBelousov
Copy link
Collaborator

MichaelBelousov commented Jan 22, 2025

am I still building this wrong @rohlem?

$ cd dvui
$ git checkout e50cfbf7d3cba03552cd85a9b368a0940b1f9d9b # the merge commit from this PR
$ ~/opensource/zig-linux-x86_64-0.14.0-dev.2802+257054a14/zig
0.14.0-dev.2802+257054a14
$ ~/opensource/zig-linux-x86_64-0.14.0-dev.2802+257054a14/zig build web-test

My version looks correct according to the PR title, I see it changed once in the convo.

I get:

/home/mike/projects/dvui/src/stb/stb_truetype.h:441:13: error: 'math.h' file not found
   #include <math.h>
            ^~~~~~~~~
/home/mike/projects/dvui/src/stb/stb_truetype_impl.c:8:10: note: in file included from /home/mike/projects/dvui/src/stb/stb_truetype_impl.c:8:
#include "stb_truetype.h"
         ^
/home/mike/projects/dvui/src/stb/stb_image.h:385:10: error: 'stdlib.h' file not found
#include <stdlib.h>
         ^~~~~~~~~~~
/home/mike/projects/dvui/src/stb/stb_image_impl.c:10:10: note: in file included from /home/mike/projects/dvui/src/stb/stb_image_impl.c:10:
#include "stb_image.h"

...

@MichaelBelousov
Copy link
Collaborator

MichaelBelousov commented Jan 22, 2025

nevermind, I now see the comment referenced in the conversation about having to build .wasi first

@MichaelBelousov
Copy link
Collaborator

non-debug mode builds for me with the following patch which I could turn into a PR,
but there is a runtime error which I would guess is a zig compiler bug, and I will try to make an MVR and submit a bug report:

diff --git a/build.zig b/build.zig
index 2559c8b..21ab254 100644
--- a/build.zig
+++ b/build.zig
@@ -51,7 +51,7 @@ pub fn build(b: *std.Build) !void {
         };
         const webtarget_exe = std.Target.Query{
             .cpu_arch = .wasm32,
-            .os_tag = .freestanding, //the initial build of this fails for me. => change to .wasi once, build fails, change back to .freestanding, build succeeds
+            .os_tag = .wasi, //the initial build of this fails for me. => change to .wasi once, build fails, change back to .freestanding, build succeeds
             .abi = .musl,
         };

diff --git a/examples/web-test.zig b/examples/web-test.zig
index cc3f67a..0b2ba9e 100644
--- a/examples/web-test.zig
+++ b/examples/web-test.zig
@@ -219,3 +219,7 @@ fn dvui_frame() !void {
         win.content_scale = ns;
     }
 }
+
+export fn main(_: u32, _: u32) u32 {
+    return 0;
+}
diff --git a/src/backends/WebBackend.zig b/src/backends/WebBackend.zig
index 832f780..511c02d 100644
--- a/src/backends/WebBackend.zig
+++ b/src/backends/WebBackend.zig
@@ -57,8 +57,8 @@ pub const wasm = struct {
     pub extern fn wasm_add_noto_font() void;
 };

-export const __stack_chk_guard: c_ulong = 0xBAAAAAAD;
-export fn __stack_chk_fail() void {}
+//export const __stack_chk_guard: c_ulong = 0xBAAAAAAD;
+//export fn __stack_chk_fail() void {}

 export fn dvui_c_alloc(size: usize) ?*anyopaque {
     //std.log.debug("dvui_c_alloc {d}", .{size});

Basically, in my zig-help thread I encountered a similar issue and noticed that dvui exports its own __stack_chk_guard for some reason, which I assume was due to using a freestanding and not a wasi target. Now that we need to ship wasi, I assume this isn't necessary, but I have not git blame'd to find out why this was there in the first case.

Also note that I had to add a fake main function for the wasm linker, I assume this zig version's compiler or build system's handling of exe.entry = .disabled is broken.

So it builds now but (regardless of optimization setting) at runtime the webassembly fails to import with:

Uncaught (in promise) TypeError: WebAssembly.instantiate(): Import #0 "wasi_snapshot_preview1": module is not an object or function

I will look into that.

@MichaelBelousov
Copy link
Collaborator

I have an MVR, I will talk to zig folks, it doesn't like like wasm is in a good state on master

@MichaelBelousov
Copy link
Collaborator

MichaelBelousov commented Jan 22, 2025

After some testing, thinking and asking zig-help, it looks like the way we should do this is
by linking a non-standard and ideally partial libc to satisfy the C dependencies like stb_image.h, and then keep building freestanding. WASI is not really a good option for the browser target.

An alternative is to use something like jco to polyfill the wasi env when loading the web wasm. I am afraid this may significantly bloat dvui with stuff like file system APIs that are very much unused in most applications.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants