Zig - Sample build #1815

kassane · 2023-04-02T20:46:05Z

The sample riscv ( random choose) was chosen for reference and was used as basis for developing the wrapper/binding.
In addition to a basic build-system that could trigger cmake and its necessary commands to build.

How to use

Need libunicorn:

$> zig build cmake

When building, it will create the build folder containing the required library.

build sample_riscv.zig

$> zig build run -Doptimize=ReleaseSafe

Note: It is recommended that you try to link the libunicorn-shared or build libunicorn with the same compiler (zig cc).
Unfortunately there are conflicts with gcc/mingw.

Ref.: #1809

kassane · 2023-04-02T23:18:34Z

Hi @lupyuen,

I remember you made a pinephone guide using unicorn-rs.
Do you have any suggestions for suitable zig support based on your experiment?

lupyuen · 2023-04-03T00:01:22Z

@kassane Sorry I haven't thought about how we might support Zig in Unicorn. (I used the standard Rust Bindings for Unicorn)

But I think Zig support would be really useful.

.github/workflows/build-uc2.yml

kassane · 2023-04-03T13:05:47Z

@wtdcode
Test outupt (Zig CI test): https://github.com/kassane/unicorn/actions/runs/4596902849/jobs/8118942852

--edit

https://github.com/kassane/unicorn/actions/runs/4640759805

wtdcode · 2023-04-08T16:51:41Z

Sorry for late because I was trying to use zig to try your samples and stuck on some dumb questions. I will review it asap.

kassane · 2023-04-08T17:53:44Z

Sorry for late because I was trying to use zig to try your samples and stuck on some dumb questions. I will review it asap.

No worries! If you need help, I'm available.

kassane · 2023-04-11T15:51:22Z

Post commit 765c84d was retested on windows mingw and no segfault occurred.
Currently zig toolchain/mingw doesn't provide winpthreads support. However, in this build the libunicorn already includes winpthread.

Before

Msys2 terminal

Matheus@DESKTOP-BUHRSIO CLANG64 ~/unicorn
$ PATH=$PATH:$PWD/build zig build run
info(unicorn): Emulate RISCV code: recover_from_illegal
info(unicorn): >>> Allocating block at 0x4096 (0x4096), block size = 0x2 (0x4096)
info(unicorn): >>> Tracing basic block at 0x4096, block size = 0x0
error(unicorn): Expected Illegal Instruction error, got: error.EXCEPTION
info(unicorn): >>> Tracing basic block at 0x65536, block size = 0x8
info(unicorn): >>> Tracing instruction at 0x65536, instruction size = 0x4
info(unicorn): >>> Emulation done. Below is the CPU context
info(unicorn): >>> A0 = 0x1
info(unicorn): >>> A1 = 0x30864
info(unicorn): ------------------
info(unicorn): Emulate RISCV code: split emulation
info(unicorn): >>> Tracing basic block at 0x65536, block size = 0x4
info(unicorn): >>> Tracing basic block at 0x65536, block size = 0x4
info(unicorn): >>> A0 = 0x1
info(unicorn): >>> A1 = 0x30864
info(unicorn): >>> Tracing basic block at 0x65540, block size = 0x4
info(unicorn): >>> Tracing basic block at 0x65540, block size = 0x4
info(unicorn): >>> Emulation done. Below is the CPU context
info(unicorn): >>> A0 = 0x1
info(unicorn): >>> A1 = 0x30864
Segmentation fault at address 0x18d4e268270
thread 11496 panic: start index 18181400 is larger than end index 17866752
C:\Users\Matheus\Downloads\zig\master\files\lib\std\coff.zig:1213:54: 0x7ff73974ef9b in getStrtab (sample_riscv.exe.obj)
        const size = mem.readIntLittle(u32, self.data[offset..][0..4]);
                                                     ^
C:\Users\Matheus\Downloads\zig\master\files\lib\std\coff.zig:1235:42: 0x7ff73973c3c5 in getSectionName (sample_riscv.exe.obj)
            const strtab = self.getStrtab().?;
                                         ^
C:\Users\Matheus\Downloads\zig\master\files\lib\std\coff.zig:1244:48: 0x7ff739706837 in getSectionByName__anon_10896 (sample_riscv.exe.obj)
            if (mem.eql(u8, self.getSectionName(sect), name)) {
                                               ^
C:\Users\Matheus\Downloads\zig\master\files\lib\std\debug.zig:884:38: 0x7ff73972448d in readCoffDebugInfo (sample_riscv.exe.obj)
        if (coff_obj.getSectionByName(".debug_info")) |sec| {
                                     ^
C:\Users\Matheus\Downloads\zig\master\files\lib\std\debug.zig:1414:54: 0x7ff73970578e in lookupModuleWin32 (sample_riscv.exe.obj)
                obj_di.* = try readCoffDebugInfo(self.allocator, mapped_module);
                                                     ^
C:\Users\Matheus\Downloads\zig\master\files\lib\std\debug.zig:1338:42: 0x7ff73970542f in getModuleForAddress (sample_riscv.exe.obj)
            return self.lookupModuleWin32(address);
                                         ^
C:\Users\Matheus\Downloads\zig\master\files\lib\std\debug.zig:767:50: 0x7ff739735c36 in printSourceAtAddress__anon_10126 (sample_riscv.exe.obj)
    const module = debug_info.getModuleForAddress(address) catch |err| switch (err) {
                                                 ^
C:\Users\Matheus\Downloads\zig\master\files\lib\std\debug.zig:186:29: 0x7ff73975c3b4 in dumpStackTraceFromBase (sample_riscv.exe.obj)
        printSourceAtAddress(debug_info, stderr, ip, tty_config) catch return;
                            ^
C:\Users\Matheus\Downloads\zig\master\files\lib\std\debug.zig:2069:45: 0x7ff73974b262 in handleSegfaultWindowsExtra__anon_14009 (sample_riscv.exe.obj)  
        dumpStackTraceFromBase(regs.bp, regs.ip);
                                            ^
C:\Users\Matheus\Downloads\zig\master\files\lib\std\debug.zig:2046:73: 0x7ff739738840 in handleSegfaultWindows (sample_riscv.exe.obj)
        windows.EXCEPTION_ACCESS_VIOLATION => handleSegfaultWindowsExtra(info, 1, null),
                                                                        ^
???:?:?: 0x7ffb9aac7b09 in ??? (???)
???:?:?: 0x7ffb9aa6e8c1 in ??? (???)
???:?:?: 0x7ffb9aaf2e9d in ??? (???)
Panicked during a panic. Aborting.
run sample_riscv: error: the following command exited with error code 3:
C:\msys64\home\Matheus\unicorn\zig-out\bin\sample_riscv.exe

After

Msys2 terminal

Matheus@DESKTOP-BUHRSIO CLANG64 ~/unicorn
$ PATH=$PATH:$PWD/build zig build run 
info(unicorn): Emulate RISCV code: recover_from_illegal
info(unicorn): >>> Allocating block at 0x4096 (0x4096), block size = 0x2 (0x4096)
info(unicorn): >>> Tracing basic block at 0x4096, block size = 0x0
error(unicorn): Expected Illegal Instruction error, got: error.EXCEPTION
info(unicorn): >>> Tracing basic block at 0x65536, block size = 0x8
info(unicorn): >>> Tracing instruction at 0x65536, instruction size = 0x4
info(unicorn): >>> Emulation done. Below is the CPU context
info(unicorn): >>> A0 = 0x1
info(unicorn): >>> A1 = 0x30864
info(unicorn): ------------------
info(unicorn): Emulate RISCV code: split emulation
info(unicorn): >>> Tracing basic block at 0x65536, block size = 0x4
info(unicorn): >>> Tracing basic block at 0x65536, block size = 0x4
info(unicorn): >>> A0 = 0x1
info(unicorn): >>> A1 = 0x30864
info(unicorn): >>> Tracing basic block at 0x65540, block size = 0x4
info(unicorn): >>> Tracing basic block at 0x65540, block size = 0x4
info(unicorn): >>> Emulation done. Below is the CPU context
info(unicorn): >>> A0 = 0x1
info(unicorn): >>> A1 = 0x30864
info(unicorn): ------------------
info(unicorn): Emulate RISCV code: return from func
info(unicorn): ========
info(unicorn): >>> Tracing basic block at 0x65540, block size = 0x2
info(unicorn): >>> Tracing instruction at 0x65540, instruction size = 0x2
info(unicorn): >>> Tracing basic block at 0x65542, block size = 0x4
info(unicorn): Good, PC == RA
info(unicorn): >>> Emulation done

sample_build.tar.gz

Edit
Sadly still haven't finalized the readaptation from build.zig ( like standalone zig) to QEMU, because it needs to have the configs.h (host and target) predefined already in some build steps.
The winpthread issue would probably get fixed with that: https://github.com/kassane/winpthreads-zigbuild

kassane · 2023-04-11T17:46:13Z

New output: (Build all samples)

Help

$> zig build -h

Usage: /home/kassane/zig/0.11.0-dev.2545+311d50f9d/files/zig build [steps] [options]

Steps:
  install (default)            Copy build artifacts to prefix path
  uninstall                    Remove build artifacts from prefix path
  sample_riscv_zig             Run the sample_riscv_zig sample
  sample_arm                   Run the sample_arm sample
  sample_arm64                 Run the sample_arm64 sample
  sample_ctl                   Run the sample_ctl sample
  sample_batch_reg             Run the sample_batch_reg sample
  sample_m68k                  Run the sample_m68k sample
  sample_riscv                 Run the sample_riscv sample
  sample_sparc                 Run the sample_sparc sample
  sample_s390x                 Run the sample_s390x sample
  shellcode                    Run the shellcode sample
  sample_tricore               Run the sample_tricore sample
  sample_x86                   Run the sample_x86 sample
  sample_x86_32_gdt_and_seg_regs Run the sample_x86_32_gdt_and_seg_regs sample
  cmake                        Run cmake build

Build

Build Summary: 29/29 steps succeeded
sample_x86_32_gdt_and_seg_regs success
└─ run sample_x86_32_gdt_and_seg_regs success 1ms MaxRSS:4M
   ├─ zig build-exe sample_x86_32_gdt_and_seg_regs Debug native success 355ms MaxRSS:353M
   └─ install success
      ├─ install sample_riscv_zig success
      │  └─ zig build-exe sample_riscv_zig Debug native success 2s MaxRSS:356M
      ├─ install sample_arm success
      │  └─ zig build-exe sample_arm Debug native success 344ms MaxRSS:352M
      ├─ install sample_arm64 success
      │  └─ zig build-exe sample_arm64 Debug native success 345ms MaxRSS:352M
      ├─ install sample_ctl success
      │  └─ zig build-exe sample_ctl Debug native success 368ms MaxRSS:352M
      ├─ install sample_batch_reg success
      │  └─ zig build-exe sample_batch_reg Debug native success 386ms MaxRSS:352M
      ├─ install sample_m68k success
      │  └─ zig build-exe sample_m68k Debug native success 415ms MaxRSS:353M
      ├─ install sample_riscv success
      │  └─ zig build-exe sample_riscv Debug native success 343ms MaxRSS:352M
      ├─ install sample_sparc success
      │  └─ zig build-exe sample_sparc Debug native success 527ms MaxRSS:353M
      ├─ install sample_s390x success
      │  └─ zig build-exe sample_s390x Debug native success 381ms MaxRSS:352M
      ├─ install shellcode success
      │  └─ zig build-exe shellcode Debug native success 1s MaxRSS:352M
      ├─ install sample_tricore success
      │  └─ zig build-exe sample_tricore Debug native success 241ms MaxRSS:351M
      ├─ install sample_x86 success
      │  └─ zig build-exe sample_x86 Debug native success 386ms MaxRSS:352M
      └─ install sample_x86_32_gdt_and_seg_regs success
         └─ zig build-exe sample_x86_32_gdt_and_seg_regs Debug native (reused)

kassane · 2023-04-12T00:19:04Z

cc: @wtdcode @aquynh

Following the theme proposed in the title of this contribution is enough.
It only remains to improve the necessary later after the review.

The next step (another PR) will be to make zig a fully standalone build alternative (change build.zig).

Why? Would CMake not solve it?

zig is not well tolerated by cmake or any trivial build system because there is 'space' in the zig cc/zig c++ command causing certain conflicts.
By using only the zig toolchain, it would allow, in theory, to unify the configuration between the library and the built examples (cross compilation).

wtdcode · 2023-05-09T13:15:36Z

cc: @wtdcode @aquynh

Following the theme proposed in the title of this contribution is enough. It only remains to improve the necessary later after the review.

The next step (another PR) will be to make zig a fully standalone build alternative (change build.zig).

Why? Would CMake not solve it?

zig is not well tolerated by cmake or any trivial build system because there is 'space' in the zig cc/zig c++ command causing certain conflicts. By using only the zig toolchain, it would allow, in theory, to unify the configuration between the library and the built examples (cross compilation).

Just curious about whether is there any other workaround for this? One straightforward workaround I think is something like ln -s "zig cc" "zigc".

kassane · 2023-05-09T14:54:43Z

Just curious about whether is there any other workaround for this? One straightforward workaround I think is something like ln -s "zig cc" "zigc".

Usually it has been tried to create a false clang (zigcc) in this way below:

Unix-like

#!/usr/bin/env bash
`which zig` cc $@

Windows - cmd

@echo off
zig cc %*

@mrexodia - zig-cmake
https://github.com/mrexodia/zig-cross

mrexodia · 2023-05-09T15:54:08Z

The real issue with cmake is that tools like ranlib and friends cannot have arguments in the compiler path. As you can see in zig-cross this is worked around by pointing cmake to a script file that wraps it as mentioned above.

kassane · 2023-05-19T17:49:01Z

Latest change - CI: https://github.com/kassane/unicorn/actions/runs/5026896166

zigcc.cmd

wtdcode · 2023-05-19T20:47:54Z

The only change I propose is to move the shell scripts to bindings/zig. Other looks fine to me. Sorry for the delay because I was sick for a few weeks.

kassane · 2023-05-20T13:29:27Z

zig-mingw build CI was disabled.
It found a compiler failure (on zig cc w/ cmake), described here:

x86_64: https://github.com/kassane/unicorn/actions/runs/5032387218/jobs/9025992784#step:6:21

ERROR: Your compiler does not support the __thread specifier for 
       Thread-Local Storage (TLS). Please upgrade to a version that does.

aarch64-windows: https://github.com/kassane/unicorn/actions/runs/5032387218/jobs/9025992877#step:6:13

run cmake: error: unable to spawn cmake: InvalidExe   # x86_64 executable?

kassane · 2023-08-04T13:16:50Z

Zig 0.11.0 released: https://ziglang.org/download/0.11.0/release-notes.html

* zig binding - sample added * zig build CI * split mingw (shared/static) CI/CD * unicorn log added * build C/C++ samples

wtdcode · 2023-08-06T14:10:56Z

@kassane Is it the final version? I'm going to merge it if it's done.

kassane · 2023-08-06T14:14:04Z

@kassane Is it the final version? I'm going to merge it if it's done.

Yes! Merge - works in 0.11.0 stable

wtdcode · 2023-08-06T14:15:05Z

@kassane Is it the final version? I'm going to merge it if it's done.

Yes! Merge - works in 0.11.0 stable

Here you go. Thanks!

kassane marked this pull request as draft April 2, 2023 23:02

kassane marked this pull request as ready for review April 2, 2023 23:07

wtdcode reviewed Apr 3, 2023

View reviewed changes

.github/workflows/build-uc2.yml Show resolved Hide resolved

kassane marked this pull request as draft April 3, 2023 12:21

kassane requested a review from wtdcode April 3, 2023 12:59

kassane marked this pull request as ready for review April 3, 2023 13:06

wtdcode reviewed May 19, 2023

View reviewed changes

zigcc.cmd Outdated Show resolved Hide resolved

kassane requested a review from wtdcode May 20, 2023 13:22

kassane mentioned this pull request May 20, 2023

zig cc on MinGW does not support the __thread specifier ziglang/zig#15793

Open

Zig binding

4fb4b3e

* zig binding - sample added * zig build CI * split mingw (shared/static) CI/CD * unicorn log added * build C/C++ samples

wtdcode merged commit 6ae0c97 into unicorn-engine:dev Aug 6, 2023

kassane deleted the zig-build branch August 6, 2023 14:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Zig - Sample build #1815

Zig - Sample build #1815

kassane commented Apr 2, 2023 •

edited

Loading

kassane commented Apr 2, 2023

lupyuen commented Apr 3, 2023 •

edited

Loading

kassane commented Apr 3, 2023 •

edited

Loading

wtdcode commented Apr 8, 2023

kassane commented Apr 8, 2023

kassane commented Apr 11, 2023 •

edited

Loading

kassane commented Apr 11, 2023 •

edited

Loading

kassane commented Apr 12, 2023

wtdcode commented May 9, 2023

Why? Would CMake not solve it?

kassane commented May 9, 2023 •

edited

Loading

mrexodia commented May 9, 2023

kassane commented May 19, 2023

wtdcode commented May 19, 2023

kassane commented May 20, 2023

kassane commented Aug 4, 2023

wtdcode commented Aug 6, 2023

kassane commented Aug 6, 2023 •

edited

Loading

wtdcode commented Aug 6, 2023

Zig - Sample build #1815

Zig - Sample build #1815

Conversation

kassane commented Apr 2, 2023 • edited Loading

How to use

kassane commented Apr 2, 2023

lupyuen commented Apr 3, 2023 • edited Loading

kassane commented Apr 3, 2023 • edited Loading

wtdcode commented Apr 8, 2023

kassane commented Apr 8, 2023

kassane commented Apr 11, 2023 • edited Loading

kassane commented Apr 11, 2023 • edited Loading

kassane commented Apr 12, 2023

Why? Would CMake not solve it?

wtdcode commented May 9, 2023

Why? Would CMake not solve it?

kassane commented May 9, 2023 • edited Loading

mrexodia commented May 9, 2023

kassane commented May 19, 2023

wtdcode commented May 19, 2023

kassane commented May 20, 2023

kassane commented Aug 4, 2023

wtdcode commented Aug 6, 2023

kassane commented Aug 6, 2023 • edited Loading

wtdcode commented Aug 6, 2023

kassane commented Apr 2, 2023 •

edited

Loading

lupyuen commented Apr 3, 2023 •

edited

Loading

kassane commented Apr 3, 2023 •

edited

Loading

kassane commented Apr 11, 2023 •

edited

Loading

kassane commented Apr 11, 2023 •

edited

Loading

kassane commented May 9, 2023 •

edited

Loading

kassane commented Aug 6, 2023 •

edited

Loading