Support Cosmo target #666

ivmai · 2024-10-09T17:08:36Z

Cosmo (cosmopolitan) is a Toolchain (and Libc) for producing universal fat binaries running on any supported platform.

Some porting efforts are needed to support Cosmo in libgc, at least partially. And initial porting has already been done (for v8.2.4):

First, the patch is to be rebased to master and polished.

Links:

https://github.com/jart/cosmopolitan/releases/download/3.9.3/cosmocc-3.9.3.zip - latest Toolchain release (binary distributive)
https://github.com/ahgamut/superconfigure - a collection of programs ported to run as fat binaries using Cosmopolitan
https://ahgamut.github.io/2022/10/23/debugging-c-with-cosmo/ - How to debug Cosmo apps

Issue #666 (bdwgc).

Issue #666 (bdwgc). * Makefile.direct (clean): Remove also *.com.dbg and *.elf files.

Issue #666 (bdwgc). By using "?=" operator we only set the AR variable in case it does not already have a value. * Makefile.direct (AR): Assign value using operator "?=" instead of "=" one.

…irect Issue #666 (bdwgc). * Makefile.direct (NODIST_OBJS): Move addition of atomic_ops_sysdeps.o item to a separate line (to be easily commented out by client if needed); add comment.

Issue #666 (bdwgc). * include/private/gcconfig.h [__COSMOPOLITAN__] (COSMO): New macro. * include/private/gcconfig.h [__aarch64__ && !ANY_BSD && !DARWIN && !LINUX && !KOS && !QNX] (NOSYS): Do not define if COSMO. * include/private/gcconfig.h [__aarch64__ && COSMO] (AARCH64, mach_type_known): Define macro. * include/private/gcconfig.h [(__x86_64 || __x86_64__ || __amd64__ || __X86_64__) && COSMO] (X86_64, mach_type_known): Likewise. * include/private/gcconfig.h [COSMO] (OS_TYPE, LINUX_STACKBOTTOM, DATASTART, DATAEND, USE_MMAP_ANON): Likewise. * include/private/gcconfig.h [COSMO] (UNIX_LIKE): Likewise. * include/private/gcconfig.h [COSMO] (__data_start, _end): Declare global variable. * tests/gctest.c [!BIG && COSMO && THREADS] (BIG): Define to 500. Co-authored-by: Ivan Maidanski <ivmai@mail.ru>

Issue #666 (bdwgc).

ivmai · 2024-10-12T09:05:54Z

How to build & test with cosmocc (works at least for Linux/x86_64):

Using configure:
CC=cosmocc CXX=cosmoc++ AR=cosmoar RANLIB='cosmoar s' ./configure --disable-shared --enable-threads=posix && make check (w/o -j, --enable-cplusplus does not work on MacOS)
Using cmake (does not work on MacOS):
mkdir out && cd out && CC=cosmocc CXX=cosmoc++ cmake -DCMAKE_AR=`type -p cosmoar` -DBUILD_SHARED_LIBS=OFF -Denable_cplusplus=ON -Dbuild_tests=ON .. && cmake --build . && ctest`
Using Makefile.direct:
CC=cosmocc CXX=cosmoc++ AR=cosmoar RANLIB='cosmoar s' make -f Makefile.direct check (this requires commenting out NODIST_OBJS+= atomic_ops_sysdeps.o in Makefile.direct)
Directly using cosmocc:
../cosmocc/bin/cosmocc -I include -DALL_INTERIOR_POINTERS -DNO_EXECUTE_PERMISSION -D GC_THREADS -D GC_BUILTIN_ATOMIC -o gctest tests/gctest.c extra/gc.c

How to clear output files generated by cosmocc:
find . -name .aarch64 -o -name '*.aarch64.elf' -o -name '*.com.dbg' | xargs rm -r

Tested with: cosmocc-3.9.3

with -fexceptions support, all the tests appear to pass. sometimes the gctest check takes a bit on aarch64 on QEMU.

ivmai · 2024-10-15T09:46:56Z

Note on configure: we have to pass --enable-threads=posix option to configure because mosmocc does not support -v option (as of cosmocc 14.1.0).
libgc configure uses -v compiler option to figure out the default threads API by grepping "Thread model:" line of the compiler output. E.g. on Linux gcc -v produces such a line: Thread model: posix

Issue #666 (bdwgc). These symbols are intercepted now, to match that on Linux. * include/gc/gc_config_macros.h [!GC_HAVE_PTHREAD_EXIT]: Check HOST_ANDROID and __ANDROID__ macros only if GC_LINUX_THREADS. * include/gc/gc_config_macros.h [!GC_HAVE_PTHREAD_EXIT && __COSMOPOLITAN__] (GC_HAVE_PTHREAD_EXIT): Define.

Issue #666 (bdwgc). This symbol is intercepted now, to match that on Linux. * include/gc/gc_pthread_redirects.h [!GC_PTHREAD_REDIRECTS_ONLY && !GC_NO_PTHREAD_SIGMASK && __COSMOPOLITAN__] (GC_pthread_sigmask): Declare. * include/gc/gc_pthread_redirects.h [!GC_PTHREAD_REDIRECTS_ONLY && !GC_NO_PTHREAD_SIGMASK && !(GC_PTHREAD_SIGMASK_NEEDED || GC_HAVE_PTHREAD_SIGMASK || _BSD_SOURCE || _GNU_SOURCE || _NETBSD_SOURCE || _POSIX_C_SOURCE>=199506L || _XOPEN_SOURCE>=500)] (GC_NO_PTHREAD_SIGMASK): Do not define if __COSMOPOLITAN__.

Issue #666 (bdwgc). * include/private/gcconfig.h [COSMO] (LINUX_STACKBOTTOM): Do not define. * include/private/gcconfig.h [COSMO] (HEURISTIC1, USE_GET_STACKBASE_FOR_MAIN): Define macro. * include/private/gcconfig.h [COSMO && NO_PTHREAD_GETATTR_NP] (HAVE_PTHREAD_GETATTR_NP): Likewise. * include/private/gcconfig.h [COSMO] (STACK_GRAN): Define macro (with some fake value).

Issue #666 (bdwgc). * include/private/gcconfig.h [COSMO && !HAVE_CLOCK_GETTIME] (HAVE_CLOCK_GETTIME): Define (to 1). * include/private/gcconfig.h [COSMO && !HAVE_PTHREAD_SETNAME_NP_WITH_TID] (HAVE_PTHREAD_SETNAME_NP_WITH_TID): Likewise.

Issue #666 (bdwgc). In thread-local allocation implementation, use "__thread" keyword to declare GC_thread_key and directly access the key instead of use of pthread_set/getspecific. * include/private/thread_local_alloc.h [!USE_PTHREAD_SPECIFIC && !USE_WIN32_SPECIFIC && !USE_WIN32_COMPILER_TLS && !USE_COMPILER_TLS && !USE_CUSTOM_SPECIFIC && !GC_WIN32_THREADS && !HOST_ANDROID && !(LINUX && GC_GNUC_PREREQ(3,3)) && COSMO && (GC_GNUC_PREREQ(4,4) || GC_CLANG_PREREQ(3,9))] (USE_COMPILER_TLS): Define.

Issue #666 (bdwgc). Optionally, if -D FORCE_MPROTECT_BEFORE_MADVISE is passed to CFLAGS_EXTRA, mprotect(PROT_NONE) is called before madvise(MADV_DONTNEED). * os_dep.c [USE_MUNMAP && !USE_WINALLOC && !SN_TARGET_PS3 && COSMO] (block_unmap_inner): Call madvise(MADV_DONTNEED) instead of mmap(PROT_NONE). * os_dep.c [USE_MUNMAP && !USE_WINALLOC && !SN_TARGET_PS3 && COSMO && FORCE_MPROTECT_BEFORE_MADVISE] (block_unmap_inner): Call mprotect(PROT_NONE). * os_dep.c [USE_MUNMAP && !USE_WINALLOC && !SN_TARGET_PS3 && !FORCE_MPROTECT_BEFORE_MADVISE&& COSMO] (GC_remap): Do nothing instead of mprotect(PROT_READ|PROT_WRITE). * os_dep.c [USE_MUNMAP && !USE_WINALLOC && COSMO && FORCE_MPROTECT_BEFORE_MADVISE] (GC_remap): Call mmap(PROT_READ|PROT_WRITE).

ivmai · 2024-10-17T05:40:15Z

Unclear if memory unmapping really works on Darwin. How to check it?

(fix of commit a72ac8b) Issue #666 (bdwgc). Note: the error was observed when compiling on Darwin only. * include/private/gc_priv.h [!__linux__ && __COSMOPOLITAN__ && USE_MUNMAP && !_GNU_SOURCE] (_GNU_SOURCE): Define.

Issue #666 (bdwgc). This currently works for single-threaded builds or when pthread_atfork is not used. * include/private/gcconfig.h [COSMO && (!GC_THREADS || NO_HANDLE_FORK || GC_NO_CAN_CALL_ATFORK)] (MPROTECT_VDB): Define; add FIXME item. * include/private/gcconfig.h [(HAVE_SYS_TYPES_H || !(__CC_ARM || OS2 || MSWINCE || SN_TARGET_ORBIS || SN_TARGET_PSP2)) && COSMO && MPROTECT_VDB && !_GNU_SOURCE] (_GNU_SOURCE): Define (before include sys/types.h). * include/private/gcconfig.h [COSMO] (USE_SEGV_SIGACT, USE_BUS_SIGACT): Define. * os_dep.c [MPROTECT_VDB && !DARWIN && !MSWIN32 && !MSWINCE && COSMO] (CODE_OK): Define (to TRUE).

Issue #666 (bdwgc). Prevously, GC_get_nprocs() always reported 1 core. * include/private/gcconfig.h [COSMO] (PARALLEL_MARK): Undefine; add FIXME item. * pthread_support.c [!HPUX && COSMO] (GC_get_nprocs): Use sysconf(_SC_NPROCESSORS_ONLN).

Issue #666 (bdwgc). Now, all cord*.o and gctest.o files are put to base folder (instead of "cord" and "tests" folders, respectively). * Makefile.direct (CORD_OBJS): Move (and update) comment from cordbscs.o target. * Makefile.direct (CORD_OBJS): Remove "cord/" prefix for cord*.o files. * Makefile.direct (cord/cordbscs.o, cord/cordxtra.o, cord/cordprnt.o): Likewise. * Makefile.direct (tests/gctest.o): Remove "tests/" prefix for gctest.o file. * Makefile.direct (gctest, gctest_dyn_link, gctest_irix_dyn_link): Likewise. * Makefile.direct (tests/gctest.o): Do not run "mv" command. * Makefile.direct (cord/cordbscs.o, cord/cordxtra.o, cord/cordprnt.o): Do not run "mkdir" and "mv" commands. * Makefile.direct (clean): Remove tests/*.o and $(CORD_OBJS) items.

Issue #666 (bdwgc). By using "?=" operator we only set the RANLIB variable in case it does not already have a value. * Makefile.direct (RANLIB): Assign value using operator "?=" instead of "=" one; place comment before the definition; mention Cosmo in comment.

Issue #666 (bdwgc).

ivmai · 2024-10-18T21:49:54Z

Remaining issues:

gctest sometimes hang (or got "Signals delivery fail") on Linux if multi-threaded build
gctest got "Signals delivery fail" on MacOS if multi-threaded build
deadlock in parallel marker (at least on Linux) - W/A applied (undefine PARALLE_MARK)
cmake-based build fails on MacOS
child process fail in gctest if MPROTECT_VDB and multi-threaded build on Linux (unless pthread_atfork is not used) - W/A applied (do not define MPROTECT_VDB if multi-threaded build)
add testing on MacOS and Windows (using Github actions)

ivmai added the feature-request label Oct 9, 2024

ivmai added a commit that referenced this issue Oct 12, 2024

.gitignore: Ignore *.dbg and *.elf files produced by Cosmo toolchain

0c86f66

Issue #666 (bdwgc).

ivmai added a commit that referenced this issue Oct 12, 2024

Add removal of Cosmo-specific output files to Makefile.direct clean

6ec9f46

Issue #666 (bdwgc). * Makefile.direct (clean): Remove also *.com.dbg and *.elf files.

ivmai added a commit that referenced this issue Oct 12, 2024

Update AUTHORS file (add Gautham V)

c06c709

Issue #666 (bdwgc).

ivmai referenced this issue in ahgamut/superconfigure Oct 12, 2024

libgc make check test passes

eb5fd41

with -fexceptions support, all the tests appear to pass. sometimes the gctest check takes a bit on aarch64 on QEMU.

ivmai referenced this issue in ahgamut/superconfigure Oct 12, 2024

libgc diff

39905b0

ivmai mentioned this issue Oct 15, 2024

Bug: unsupported -v option by cosmocc jart/cosmopolitan#1316

Open

ivmai added a commit that referenced this issue Oct 18, 2024

GH Actions: Add workflow for cmake-based build/test using Cosmo

837c2aa

Issue #666 (bdwgc).

ivmai added a commit that referenced this issue Oct 18, 2024

Add cmake-cosmo status badge to README.md

01d10a2

Issue #666 (bdwgc).

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support Cosmo target #666

Support Cosmo target #666

ivmai commented Oct 9, 2024 •

edited

Loading

ivmai commented Oct 12, 2024 •

edited

Loading

ivmai commented Oct 15, 2024 •

edited

Loading

ivmai commented Oct 17, 2024

ivmai commented Oct 18, 2024

Support Cosmo target #666

Support Cosmo target #666

Comments

ivmai commented Oct 9, 2024 • edited Loading

ivmai commented Oct 12, 2024 • edited Loading

ivmai commented Oct 15, 2024 • edited Loading

ivmai commented Oct 17, 2024

ivmai commented Oct 18, 2024

ivmai commented Oct 9, 2024 •

edited

Loading

ivmai commented Oct 12, 2024 •

edited

Loading

ivmai commented Oct 15, 2024 •

edited

Loading