Haiku: Add Haiku support for CoreCLR/PAL #93907

trungnt2910 · 2023-10-24T04:40:55Z

This contains the code required to build this repo's native components and get paltests to pass on Haiku.

Most implementation details related to this commit are documented here.

Part of #55803.

src/native/minipal/errno.h

janvorli · 2023-10-24T17:21:07Z

src/coreclr/minipal/Unix/doublemapping.cpp

-    sprintf(name, "/shm-dotnet-%d", getpid());
-    name[sizeof(name) - 1] = '\0';
-    shm_unlink(name);
-    int fd = shm_open(name, O_RDWR | O_CREAT | O_EXCL | O_NOFOLLOW, 0600);
 #else // TARGET_FREEBSD


I would prefer the #elif defined(TARGET_LINUX) here and final #else with the posix fallback instead of hierarchical ifdef.

Done. The final #else assigns the fd variable to -1 instead of doing the POSIX fallback itself, in case any of the platform-specific functions above return an error.

This should solve #72192 from the .NET 7 era.

src/coreclr/pal/src/include/pal/palinternal.h

src/native/corehost/hostmisc/pal.unix.cpp

src/coreclr/pal/inc/rt/safecrt.h

src/coreclr/utilcode/sstring.cpp

janvorli · 2023-10-24T22:38:19Z

src/native/libs/System.Native/pal_networkchange.c

-    // unreachable
-    abort();
-    return Error_SUCCESS;
+    return Error_EINVAL;


Should we keep the old aborting code as a fallback and add separate #if for HAIKU instead? I am not familiar enough with the contract of this function though.

I've added back the abort() logic, as an assertion that the function is unreachable.

This is true because on unsupported platforms, the higher-level C# libraries should already throw while trying to construct a network events socket.

Returning Error_SUCCESS does not make much sense though.

trungnt2910 · 2023-10-25T07:34:18Z

Strange, I'm not modifying any FreeBSD-related codepaths, what's wrong with the FreeBSD build?

jkotas · 2023-10-25T13:08:55Z

FreeBSD build error is:

/__w/1/s/src/coreclr/pal/src/misc/sysinfo.cpp:215:57: error: use of undeclared identifier 'NUPML4E'
    lpSystemInfo->lpMaximumApplicationAddress = (PVOID) VM_MAXUSER_ADDRESS;
                                                        ^
/crossrootfs/x64/usr/include/machine/vmparam.h:186:35: note: expanded from macro 'VM_MAXUSER_ADDRESS'
#define VM_MAXUSER_ADDRESS      UVADDR(NUPML4E, 0, 0, 0)
                                       ^

I guess you have introduced it by changing conditions around some of the includes.

trungnt2910 · 2023-10-25T13:41:22Z

This is suspicious:

-- Looking for include file sys/user.h
-- Looking for include file sys/user.h - not found

I wonder how I could get more detailed CMake logs for the CI runs.

trungnt2910 · 2023-10-25T13:55:53Z

When looking at other files, it seems like sys/user.h is only used on FreeBSD. Somehow sysinfo.cpp includes this header for all platforms.

This time, instead of checking for this header during configuration as in #86391, a FreeBSD #ifdef guard is added instead.

huoyaoyuan · 2023-10-30T06:13:58Z

src/coreclr/minipal/Unix/doublemapping.cpp

+    // POSIX fallback
+    if (fd == -1)
+    {
+        char name[24];
+        sprintf(name, "/shm-dotnet-%d", getpid());
+        name[sizeof(name) - 1] = '\0';
+        shm_unlink(name);
+        fd = shm_open(name, O_RDWR | O_CREAT | O_EXCL | O_NOFOLLOW, 0600);
+        shm_unlink(name);
+    }


Shouldn't this block be in #else?

No, it should not, as stated in #93907 (comment).

Even for OSes with platform-specific ways to get a special memory file open, should their method fail, we can still always use the POSIX fallback.

huoyaoyuan · 2023-10-30T06:15:16Z

src/coreclr/gc/unix/numasupport.cpp

+#ifdef TARGET_LINUX
+#include <sys/syscall.h>
+#endif


Is this used by other Unix systems?

No, it is only used on Linux:

runtime/src/coreclr/gc/unix/numasupport.cpp

Lines 52 to 63 in e5c631d

#ifdef TARGET_LINUX

if (syscall(__NR_get_mempolicy, NULL, NULL, 0, 0, 0) < 0 && errno == ENOSYS)

return;

int highestNumaNode = GetNodeNum("/sys/devices/system/node", false);

// we only use this implementation when there are two or more NUMA nodes available

if (highestNumaNode < 1)

return;

g_numaAvailable = true;

g_highestNumaNode = highestNumaNode;

#endif

and

runtime/src/coreclr/gc/unix/numasupport.cpp

Lines 81 to 83 in e5c631d

#ifdef TARGET_LINUX

return syscall(__NR_mbind, (long)start, len, 1, (long)nodemask, maxnode, 0);

#else

huoyaoyuan · 2023-10-30T06:17:55Z

src/coreclr/pal/inc/pal_mstypes.h

 #define _SIZE_T_DEFINED
+#endif // !PAL_STDCPP_COMPAT


What's the impact range of this?

What do you mean by "impact range"?

The line should affect all platforms. However, I believe that this is how things should be, since with PAL_STDCPP_COMPAT the header should not define things that potentially mess up system headers.

Also, CoreCLR builds are working, so it should not break the definition on any other platforms.

Some more comments about this change here: #55803 (comment)

trungnt2910 · 2024-04-30T14:27:50Z

I have edited this pull request to include only files in src/coreclr/pal.
Other parts will come in other PRs.
Please let me know if this needs to be split further.

trungnt2910 · 2024-04-30T15:57:47Z

Latest Haiku builds are failing with this error:

  [  7%] Building CXX object inc/CMakeFiles/corguids.dir/__/pal/prebuilt/idl/cordebug_i.cpp.o
  In file included from /home/runner/work/dotnet-builds/dotnet-builds/runtime/src/coreclr/pal/inc/rt/palrt.h:630,
                   from /home/runner/work/dotnet-builds/dotnet-builds/runtime/src/coreclr/pal/inc/rt/rpc.h:15,
                   from /home/runner/work/dotnet-builds/dotnet-builds/runtime/src/coreclr/pal/prebuilt/idl/cordebug_i.cpp:29:
  /home/runner/work/dotnet-builds/dotnet-builds/runtime/src/coreclr/pal/inc/rt/safecrt.h:1093:16: error: conflicting declaration of C function 'size_t wcsnlen(const WCHAR*, size_t)'
   1093 | size_t __cdecl wcsnlen(const WCHAR *inString, size_t inMaxSize);
        |                ^~~~~~~
  In file included from /home/runner/work/dotnet-builds/dotnet-builds/dotnet-rootfs/boot/system/develop/headers/posix/wctype.h:10,
                   from /home/runner/work/dotnet-builds/dotnet-builds/runtime/src/coreclr/pal/inc/pal.h:51,
                   from /home/runner/work/dotnet-builds/dotnet-builds/runtime/src/coreclr/pal/inc/rt/palrt.h:136:
  /home/runner/work/dotnet-builds/dotnet-builds/dotnet-rootfs/boot/system/develop/headers/posix/wchar.h:122:17: note: previous declaration 'size_t wcsnlen(const wchar_t*, size_t)'
    122 | extern size_t   wcsnlen(const wchar_t *wcs, size_t maxLength);
        |                 ^~~~~~~
  /home/runner/work/dotnet-builds/dotnet-builds/runtime/src/coreclr/pal/inc/rt/safecrt.h:1098:16: error: conflicting declaration of C function 'size_t wcsnlen(const WCHAR*, size_t)'
   1098 | size_t __cdecl wcsnlen(const WCHAR *inString, size_t inMaxSize)
        |                ^~~~~~~
  /home/runner/work/dotnet-builds/dotnet-builds/dotnet-rootfs/boot/system/develop/headers/posix/wchar.h:122:17: note: previous declaration 'size_t wcsnlen(const wchar_t*, size_t)'
    122 | extern size_t   wcsnlen(const wchar_t *wcs, size_t maxLength);
        |                 ^~~~~~~
  make[2]: *** [inc/CMakeFiles/corguids.dir/build.make:76: inc/CMakeFiles/corguids.dir/__/pal/prebuilt/idl/cordebug_i.cpp.o] Error 1
  make[1]: *** [CMakeFiles/Makefile2:3250: inc/CMakeFiles/corguids.dir/all] Error 2
  make[1]: *** Waiting for unfinished jobs....

Seems like whoever included wctype.h did not expect wchar.h to be included as well.

According to the POSIX Standard, however:

[CX] [Option Start] Inclusion of the <wctype.h> header may make visible all symbols from the headers <ctype.h>, <stdarg.h>, <stddef.h>, <stdio.h>, <stdlib.h>, <string.h>, <time.h>, and <wchar.h>. [Option End]

What should I do in this case? Removing <wchar.h> from Haiku's system include just breaks everything. And according to POSIX Haiku isn't doing anything wrong in the first place...

jkotas · 2024-04-30T16:18:35Z

Does anything break if our local wcsnlen implementation is deleted?

trungnt2910 · 2024-04-30T17:02:37Z

Hasn't broken my local Linux build and moved my local Haiku build to a different error. Let the CI inform the rest of the answer.

trungnt2910 · 2024-04-30T17:24:46Z

The next Haiku error is:

  asmparse.y:176: error: "_IMPORT" redefined [-Werror]
  In file included from /home/trung/dotnet-rootfs/boot/system/develop/headers/posix/sys/types.h:11,
                   from /home/trung/dotnet-rootfs/boot/system/develop/headers/posix/stdio.h:9,
                   from /home/trung/dotnet-rootfs/boot/system/develop/headers/bsd/stdio.h:9,
                   from /home/trung/dotnet-runtime/src/coreclr/pal/inc/pal.h:39,
                   from /home/trung/dotnet-runtime/src/coreclr/pal/inc/rt/palrt.h:136,
                   from /home/trung/dotnet-runtime/src/coreclr/pal/inc/rt/rpc.h:15,
                   from /home/trung/dotnet-runtime/src/coreclr/pal/inc/rt/objidl.h:12,
                   from /home/trung/dotnet-runtime/src/coreclr/pal/inc/rt/ole2.h:8,
                   from /home/trung/dotnet-runtime/src/coreclr/inc/cor.h:16,
                   from /home/trung/dotnet-runtime/src/coreclr/ilasm/./ilasmpch.h:8,
                   from asmparse.y:9:
  /home/trung/dotnet-rootfs/boot/system/develop/headers/os/BeBuild.h:95: note: this is the location of the previous definition
     95 | #define _IMPORT
        |
  asmparse.y:262: error: "_EXPORT" redefined [-Werror]
  /home/trung/dotnet-rootfs/boot/system/develop/headers/os/BeBuild.h:90: note: this is the location of the previous definition
     90 | # define _EXPORT __attribute__((visibility("default")))
        |

Currently I am solving this by adding these lines to ilasmpch.h:

#include "mdfileformat.h"
#include "stgpooli.h"

+#ifdef _EXPORT
+#undef _EXPORT
+#endif

+#ifdef _IMPORT
+#undef _IMPORT
+#endif

#endif

I wonder whether these lines should be put directly to pal.h instead.

mangod9 · 2024-07-29T16:19:05Z

Hey @trungnt2910, what is the status of this PR. Are you continuing to work on it?

trungnt2910 · 2024-07-29T20:50:31Z

There's still an unanswered question from me above.

When we have a solution for those macros, I can proceed to resolving the conflicts and the PR is ready for further review.

mangod9 · 2024-07-30T19:59:05Z

ok, so the question is about whether to add the undefs in pal.h? We are also triaging PRs for .NET 9, assume this can wait a few weeks ?

trungnt2910 · 2024-07-30T23:07:25Z

Yes, the question is related to where I should add the #undefs, or otherwise how I should fix the redefined error.
I believe that one of the uses of pal.h is to isolate parts of CoreCLR code from system headers, so I initially thought putting those macros there would work.

assume this can wait a few weeks ?

Yeah. This has been waiting for quite a few months now, waiting a few more weeks wouldn't hurt.

mangod9 · 2024-09-16T14:44:19Z

adding @janvorli on advise on #undefs in pal.h.

janvorli · 2024-11-05T18:08:20Z

I wonder whether these lines should be put directly to pal.h instead.

This is strictly ilasm specific, so let's keep it where you have put it.

janvorli

Besides the two comments, it all looks good.

src/coreclr/pal/src/include/pal/thread.hpp

src/coreclr/pal/src/map/map.cpp

src/native/minipal/debugger.c

This contains a part of the code required to build CoreCLR and get `paltests` to pass on Haiku. This commit covers `src/coreclr/pal/**`. Co-authored-by: Jessica Hamilton <jessica.l.hamilton@gmail.com>

Some OSes like Haiku define the `_EXPORT` and `_IMPORT` marcos. Symbols beginning with an underscore followed immediately by an uppercase letter are reserved anyway. Add the guards to get them out of the way.

This flag is a compatibility flag and not required on most OSes. Some, including Haiku, does not even define it.

janvorli

LGTM, thank you!

janvorli · 2024-11-05T22:46:38Z

/azp run runtime-coreclr outerloop

azure-pipelines · 2024-11-05T22:46:58Z

Azure Pipelines successfully started running 1 pipeline(s).

janvorli · 2024-11-06T10:48:18Z

The outerloop failure is a known crossgen2 issue #104340

trungnt2910 · 2024-11-06T12:44:40Z

Thanks for the review!

Following up with #109580, hope you could have a look at that.

trungnt2910 requested review from vargaz, lambdageek and MichalStrehovsky as code owners October 24, 2023 04:40

ghost added the community-contribution Indicates that the PR has been added by a community member label Oct 24, 2023

dotnet-issue-labeler bot added the area-PAL-coreclr label Oct 24, 2023

jkotas reviewed Oct 24, 2023

View reviewed changes

src/native/minipal/errno.h Outdated Show resolved Hide resolved

jkotas requested a review from janvorli October 24, 2023 04:55

This was referenced Oct 24, 2023

Tracking issue for CI build timeouts #76454

Closed

Test failure: Wasm.Build.Tests.NativeBuildTests.MonoAOTCross_WorksWithNoTrimming #93522

Closed

trungnt2910 force-pushed the dev/trungnt2910/haiku-pal branch from f98a038 to 3ff7e01 Compare October 24, 2023 21:58

janvorli reviewed Oct 24, 2023

View reviewed changes

trungnt2910 force-pushed the dev/trungnt2910/haiku-pal branch 2 times, most recently from 790029d to 2131127 Compare October 25, 2023 04:03

This was referenced Oct 25, 2023

Test_EventSource_EtwManifestGeneration* tests failing in CI #48798

Closed

[wasm][debugger] Debugger tests getting randomly aborted on CI #89409

Open

The active test run was aborted. Reason: Test host process crashed dotnet/dnceng#451

Open

trungnt2910 force-pushed the dev/trungnt2910/haiku-pal branch from ed1af42 to 4a8cc01 Compare October 25, 2023 13:53

trungnt2910 force-pushed the dev/trungnt2910/haiku-pal branch 4 times, most recently from 0a97fb8 to f1f4369 Compare October 28, 2023 20:58

This was referenced Oct 28, 2023

Timeout in System.Net.Quic.Functional.Tests #86019

Closed

CI error: System.Net.Quic.QuicException: The connection timed out from inactivity #91757

Closed

System.Security.Cryptography.Tests timing out #93840

Closed

huoyaoyuan reviewed Oct 30, 2023

View reviewed changes

trungnt2910 force-pushed the dev/trungnt2910/haiku-pal branch from f1f4369 to 72e7cc3 Compare November 5, 2023 00:40

trungnt2910 force-pushed the dev/trungnt2910/haiku-pal branch from a7afb88 to c11ed1a Compare April 30, 2024 14:46

trungnt2910 force-pushed the dev/trungnt2910/haiku-pal branch from ed24de7 to 56ff51f Compare April 30, 2024 17:02

build-analysis bot mentioned this pull request Apr 30, 2024

System.Numerics.Tensors.Tests.SingleGenericTensorPrimitives.SpanScalarDestination_SpecialValues fails #101721

Closed

janvorli reviewed Nov 5, 2024

View reviewed changes

src/coreclr/pal/src/include/pal/thread.hpp Outdated Show resolved Hide resolved

src/coreclr/pal/src/map/map.cpp Outdated Show resolved Hide resolved

trungnt2910 force-pushed the dev/trungnt2910/haiku-pal branch from 56ff51f to 7e63089 Compare November 5, 2024 22:34

janvorli reviewed Nov 5, 2024

View reviewed changes

src/native/minipal/debugger.c Outdated Show resolved Hide resolved

trungnt2910 and others added 3 commits November 6, 2024 09:39

Haiku: Add Haiku support for CoreCLR/PAL

850ab1a

This contains a part of the code required to build CoreCLR and get `paltests` to pass on Haiku. This commit covers `src/coreclr/pal/**`. Co-authored-by: Jessica Hamilton <jessica.l.hamilton@gmail.com>

ilasm: Haiku build fixes

84a1502

Some OSes like Haiku define the `_EXPORT` and `_IMPORT` marcos. Symbols beginning with an underscore followed immediately by an uppercase letter are reserved anyway. Add the guards to get them out of the way.

pal: Remove usage of MAP_FILE

40b806a

This flag is a compatibility flag and not required on most OSes. Some, including Haiku, does not even define it.

trungnt2910 force-pushed the dev/trungnt2910/haiku-pal branch from 7e63089 to 40b806a Compare November 5, 2024 22:40

janvorli approved these changes Nov 5, 2024

View reviewed changes

This was referenced Nov 6, 2024

chrome-DebuggerTests timing out #108078

Open

[debugger] Inspector exceptions in debugger tests #109025

Open

janvorli merged commit ffcef3d into dotnet:main Nov 6, 2024
208 of 213 checks passed

am11 mentioned this pull request Nov 6, 2024

Haiku: add platform support #55803

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Haiku: Add Haiku support for CoreCLR/PAL #93907

Haiku: Add Haiku support for CoreCLR/PAL #93907

trungnt2910 commented Oct 24, 2023

janvorli Oct 24, 2023

trungnt2910 Oct 25, 2023

janvorli Oct 24, 2023

trungnt2910 Oct 25, 2023

trungnt2910 commented Oct 25, 2023

jkotas commented Oct 25, 2023 •

edited

Loading

trungnt2910 commented Oct 25, 2023

trungnt2910 commented Oct 25, 2023

huoyaoyuan Oct 30, 2023

trungnt2910 Oct 30, 2023

huoyaoyuan Oct 30, 2023

trungnt2910 Oct 30, 2023

huoyaoyuan Oct 30, 2023

trungnt2910 Oct 30, 2023 •

edited

Loading

trungnt2910 commented Apr 30, 2024

trungnt2910 commented Apr 30, 2024 •

edited

Loading

jkotas commented Apr 30, 2024

trungnt2910 commented Apr 30, 2024

trungnt2910 commented Apr 30, 2024

mangod9 commented Jul 29, 2024

trungnt2910 commented Jul 29, 2024 •

edited

Loading

mangod9 commented Jul 30, 2024

trungnt2910 commented Jul 30, 2024

mangod9 commented Sep 16, 2024

janvorli commented Nov 5, 2024

janvorli left a comment

janvorli left a comment

janvorli commented Nov 5, 2024

azure-pipelines bot commented Nov 5, 2024

janvorli commented Nov 6, 2024

trungnt2910 commented Nov 6, 2024

	#ifdef TARGET_LINUX
	if (syscall(__NR_get_mempolicy, NULL, NULL, 0, 0, 0) < 0 && errno == ENOSYS)
	return;

	int highestNumaNode = GetNodeNum("/sys/devices/system/node", false);
	// we only use this implementation when there are two or more NUMA nodes available
	if (highestNumaNode < 1)
	return;

	g_numaAvailable = true;
	g_highestNumaNode = highestNumaNode;
	#endif

	#ifdef TARGET_LINUX
	return syscall(__NR_mbind, (long)start, len, 1, (long)nodemask, maxnode, 0);
	#else

Haiku: Add Haiku support for CoreCLR/PAL #93907

Haiku: Add Haiku support for CoreCLR/PAL #93907

Conversation

trungnt2910 commented Oct 24, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

trungnt2910 commented Oct 25, 2023

jkotas commented Oct 25, 2023 • edited Loading

trungnt2910 commented Oct 25, 2023

trungnt2910 commented Oct 25, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

trungnt2910 Oct 30, 2023 • edited Loading

Choose a reason for hiding this comment

trungnt2910 commented Apr 30, 2024

trungnt2910 commented Apr 30, 2024 • edited Loading

jkotas commented Apr 30, 2024

trungnt2910 commented Apr 30, 2024

trungnt2910 commented Apr 30, 2024

mangod9 commented Jul 29, 2024

trungnt2910 commented Jul 29, 2024 • edited Loading

mangod9 commented Jul 30, 2024

trungnt2910 commented Jul 30, 2024

mangod9 commented Sep 16, 2024

janvorli commented Nov 5, 2024

janvorli left a comment

Choose a reason for hiding this comment

janvorli left a comment

Choose a reason for hiding this comment

janvorli commented Nov 5, 2024

azure-pipelines bot commented Nov 5, 2024

janvorli commented Nov 6, 2024

trungnt2910 commented Nov 6, 2024

jkotas commented Oct 25, 2023 •

edited

Loading

trungnt2910 Oct 30, 2023 •

edited

Loading

trungnt2910 commented Apr 30, 2024 •

edited

Loading

trungnt2910 commented Jul 29, 2024 •

edited

Loading