[MUSL] [chromium-based]: Build support for Musl #376

Kawanaao · 2024-09-24T07:17:12Z

Improvements for working with dragons #375

Added upstream patch for perfetto due to include linux and musl conflict, disabled shim allocator as it causes multiple errors without mallinfo implementation, ref_ptr_support requiring it and added mallinfo stub for particularly tricky code

I test my changes without unnecessary ifs, just in case

Kawanaao · 2024-09-24T10:57:42Z

Whoosh! Almost done!! :)

[48492/49360], compilation ongoing

Kawanaao · 2024-09-24T13:40:13Z

Whoosh! Almost done!! :)

[48492/49360], compilation ongoing

Error at the linking stage with symbols, solvable, but I think it is necessary to completely replace all static config.h, defining the behavior of musl equivalent to glibc

Kawanaao · 2024-09-24T16:52:55Z

Great! Electron built successfully

PF4Public · 2024-09-24T18:35:21Z

Are you sure those two patches do not negatively affect other platforms?

You don't have to put a single patch into a separate folder, while holding other patches in files, it just doesn't make sense.

Kawanaao · 2024-09-25T00:19:55Z

Are you sure those two patches do not negatively affect other platforms?

You don't have to put a single patch into a separate folder, while holding other patches in files, it just doesn't make sense.

config.h changes only if musl is used
__RES versioning method for resolv, it is defined as a standard and is present in musl, glibc, uClibc, bionic
cdefs is more likely a bug and a hard link to private glibc header
These patches should not affect the build on other architectures

I have never worked with ebuild, I thought to mark the header files somehow, will move it to files, thx!

Some declarations remove implementations instead of making the implementation empty, which causes errors at link stage when these symbols should have been there, but they were not included in the final build due to incorrect declarations.

Some processes launched by electron as part of the tests are too lightweight and can lead to a race between the parent process and the child process

CLONE_SETTLS is not used correctly within Chromium, this behavior has been deprecated in musl and now returns EINVAL

Kawanaao · 2024-09-25T10:54:03Z

Electron successfully assembled and works! :)
Next I think we need to select minimal versions for patches and apply them to other chromium-based packages?

Within the chromium sandbox, some function definitions could use syscalls that are forbidden for glibc, such as: shed_setschedulerer, mremap, membarrier, set_tid_address

This file is only required for the Kate editor

…blic-gentoo-overlay into musl-chromium-fix

PF4Public · 2024-09-25T19:51:02Z

minimal versions for patches

Are the files in this PR not minimal? What do you mean by "minimal"?

apply them to other chromium-based packages

I won't be able to verify whether they work, if you can ensure it works on musl (and doesn't break other platforms) then why not.

Kawanaao · 2024-09-25T21:49:16Z

Are the files in this PR not minimal? What do you mean by "minimal"?

I mean, these patches can be ported to earlier ebuilds for electron, but now all patches are applied exclusively to the latest version, so is there any point in doing this?

I won't be able to verify whether they work, if you can ensure it works on musl (and doesn't break other platforms) then why not.

I'll try to provide full functionality while I check if it behaves correctly in general at the system call level compared to glibc from flatpaks, but even with memory reallocation, alas, not without problems, huh

PF4Public · 2024-09-26T18:07:19Z

I mean, these patches can be ported to earlier ebuilds for electron

If you're not using older versions of electron, why bother adapting patches? :D

not without problems, huh

Is it capable of building vscode, does it work?

Kawanaao · 2024-09-27T15:58:53Z

If you're not using older versions of electron, why bother adapting patches? :D

hah touche ;)

Is it capable of building vscode, does it work?

electron is fully functional, even managed to get a normal browsing experience, except perhaps debugging, for now I need to figure out if it's because of shim or a memory leak in pthread, but vscode itself can't compile at the stage of installing packages, SQLite at least complains about something

Kawanaao · 2024-09-27T16:06:38Z

make: Entering directory '/var/tmp/portage/app-editors/vscode-1.93.1/work/vscode-1.93.1/node_modules/@vscode/sqlite3/build'
  CC(target) Release/obj.target/nothing/../../node-addon-api/nothing.o
error: invalid argument '-std=c++20' not allowed with 'C'
make: *** [../../node-addon-api/nothing.target.mk:112: Release/obj.target/nothing/../../node-addon-api/nothing.o] Error 1
make: Leaving directory '/var/tmp/portage/app-editors/vscode-1.93.1/work/vscode-1.93.1/node_modules/@vscode/sqlite3/build'

much ado about nothing :\

PF4Public · 2024-09-27T16:20:00Z

much ado about nothing :\

unless you were paying more attention to warnings:

gentoo-overlay/app-editors/vscode/vscode-1.93.1.ebuild

Lines 2241 to 2245 in 5565721

    
           if use electron-32; then 
        
           	CPPFLAGS="${CPPFLAGS} -std=c++20"; 
        
           	use build-online || eerror "build-online should be enabled for node-addon-api substitution to work" || die; 
        
           	sed -i 's$"resolutions": {$"resolutions": {"node-addon-api": "^7.1.0",$' package.json || die; 
        
           fi

;)

Although I must admit, I haven't tried building 1.93 myself jet :-/

but vscode itself can't compile at the stage of installing packages,

I'm not sure what fails on your system, but vscode needs at least 12G of ram to build and even then it might fail with OOM.

PS: That condition is indeed badly formed and perhaps does not function as intended :D

Kawanaao · 2024-09-27T17:03:09Z

unless you were paying more attention to warnings:

don't blame me, blame portage! ;)

 * build-online should be enabled for node-addon-api substitution to work
 * Installing node_modules ...

just lost sight of it, and portage didn't stop working on eerror.. pkill -SIGSEGV emerge

I'm not sure what fails on your system, but vscode needs at least 12G of ram to build and even then it might fail with OOM.

PS: That condition is indeed badly formed and perhaps does not function as intended :D

can only tell how important rest is :0
i simply didn't enable the build-online flag, resulting in a build error for @vscode/sqlite but.. enabled it and got the same error again huh

PF4Public · 2024-09-27T18:32:11Z

Give 1.92 a try:

PF4Public · 2024-09-27T19:05:38Z

Update: -std=c++20 seems to be no longer needed, I've updated 1.93 ebuild.

In the Chromium and Node.js sandbox, using pthread_getattr_np becomes impossible, as the virtual environment deceives libc, forcing it to cross protection boundaries at best. At worst, it will first allocate ~1.2TB of virtual memory 12 times in a loop using mremap

Kawanaao · 2024-09-28T08:12:23Z

Update: -std=c++20 seems to be no longer needed, I've updated 1.93 ebuild.

my first VSCode on musl after so long... 🥹

didn't have to apply patches, works great, of course, still need to check how it works with extensions, and there's a SIGSEGV when clicking the File button, but it's already awesome that it runs at all ;)

Kawanaao · 2024-09-29T21:51:29Z

huh there is a problem

gtk4 seems not to be applicable to chromium yet, due to changes in API, some functions were marked as deprecated
libcxx gives errors inside
system-libvpx seems to be broken too, gives errors
system-av1 is also broken

and the necessary debug flag is broken, it asks for

assert(!(is_debug && is_official_build), "Can't do official debug builds")

fixed via

sed -i '/is_official_build/d' build/args/release.gn || die
myconf_gn+=" is_official_build=false"

now the build error looks like this

>>> Source configured.
>>> Compiling source in /var/tmp/portage/dev-util/electron-32.1.2/work/chromium-128.0.6613.137 ...
 * Checking whether python3_13 is suitable ...
 *   dev-lang/python:3.13[xml(+)] ...                                                                                                                                                                       [ ok ]
 *   python_check_deps ...
 *     dev-python/setuptools[python_targets_python3_13(-)] ...                                                                                                                                              [ ok ]
 * Using python3.13 to build (via EPYTHON)
ninja -j24 -l24 -C out/Release electron:node_headers
ninja: Entering directory `out/Release'
[131/139] ACTION //electron:generate_config_gypi(//build/toolchain/linux/unbundle:default)
/var/tmp/portage/dev-util/electron-32.1.2/work/chromium-128.0.6613.137/out/Release/../../electron/script/generate-config-gypi.py:43: SyntaxWarning: invalid escape sequence '\w'
  m = re.match('(\w+) = (.+)', line)
Package http_parser was not found in the pkg-config search path.
Perhaps you should add the directory containing `http_parser.pc'
to the PKG_CONFIG_PATH environment variable
Package 'http_parser' not found
INFO: configure completed successfully
[139/139] STAMP obj/electron/node_headers.stamp
ninja -j24 -l24 -C out/Release electron
ninja: Entering directory `out/Release'
[16/48004] ACTION //third_party/devtools-frontend/src/front_end/core/platform:platform(//build/toolchain/linux/unbundle:default)
FAILED: gen/third_party/devtools-frontend/src/front_end/core/platform/platform-tsconfig.json gen/third_party/devtools-frontend/src/front_end/core/platform/ArrayUtilities.js gen/third_party/devtools-frontend/src/front_end/core/platform/ArrayUtilities.js.map gen/third_party/devtools-frontend/src/front_end/core/platform/Brand.js gen/third_party/devtools-frontend/src/front_end/core/platform/Brand.js.map gen/third_party/devtools-frontend/src/front_end/core/platform/DOMUtilities.js gen/third_party/devtools-frontend/src/front_end/core/platform/DOMUtilities.js.map gen/third_party/devtools-frontend/src/front_end/core/platform/DateUtilities.js gen/third_party/devtools-frontend/src/front_end/core/platform/DateUtilities.js.map gen/third_party/devtools-frontend/src/front_end/core/platform/DevToolsPath.js gen/third_party/devtools-frontend/src/front_end/core/platform/DevToolsPath.js.map gen/third_party/devtools-frontend/src/front_end/core/platform/KeyboardUtilities.js gen/third_party/devtools-frontend/src/front_end/core/platform/KeyboardUtilities.js.map gen/third_party/devtools-frontend/src/front_end/core/platform/MapUtilities.js gen/third_party/devtools-frontend/src/front_end/core/platform/MapUtilities.js.map gen/third_party/devtools-frontend/src/front_end/core/platform/MimeType.js gen/third_party/devtools-frontend/src/front_end/core/platform/MimeType.js.map gen/third_party/devtools-frontend/src/front_end/core/platform/NumberUtilities.js gen/third_party/devtools-frontend/src/front_end/core/platform/NumberUtilities.js.map gen/third_party/devtools-frontend/src/front_end/core/platform/PromiseUtilities.js gen/third_party/devtools-frontend/src/front_end/core/platform/PromiseUtilities.js.map gen/third_party/devtools-frontend/src/front_end/core/platform/StringUtilities.js gen/third_party/devtools-frontend/src/front_end/core/platform/StringUtilities.js.map gen/third_party/devtools-frontend/src/front_end/core/platform/Timing.js gen/third_party/devtools-frontend/src/front_end/core/platform/Timing.js.map gen/third_party/devtools-frontend/src/front_end/core/platform/TypedArrayUtilities.js gen/third_party/devtools-frontend/src/front_end/core/platform/TypedArrayUtilities.js.map gen/third_party/devtools-frontend/src/front_end/core/platform/TypescriptUtilities.js gen/third_party/devtools-frontend/src/front_end/core/platform/TypescriptUtilities.js.map gen/third_party/devtools-frontend/src/front_end/core/platform/UIString.js gen/third_party/devtools-frontend/src/front_end/core/platform/UIString.js.map gen/third_party/devtools-frontend/src/front_end/core/platform/UserVisibleError.js gen/third_party/devtools-frontend/src/front_end/core/platform/UserVisibleError.js.map 
python3.13 ../../third_party/devtools-frontend/src/third_party/typescript/ts_library.py --tsconfig_output_location gen/third_party/devtools-frontend/src/front_end/core/platform/platform-tsconfig.json --front_end_directory ../../third_party/devtools-frontend/src/front_end/core/platform --reset_timestamps --use-esbuild --sources ../../third_party/devtools-frontend/src/front_end/core/platform/ArrayUtilities.ts ../../third_party/devtools-frontend/src/front_end/core/platform/Brand.ts ../../third_party/devtools-frontend/src/front_end/core/platform/DOMUtilities.ts ../../third_party/devtools-frontend/src/front_end/core/platform/DateUtilities.ts ../../third_party/devtools-frontend/src/front_end/core/platform/DevToolsPath.ts ../../third_party/devtools-frontend/src/front_end/core/platform/KeyboardUtilities.ts ../../third_party/devtools-frontend/src/front_end/core/platform/MapUtilities.ts ../../third_party/devtools-frontend/src/front_end/core/platform/MimeType.ts ../../third_party/devtools-frontend/src/front_end/core/platform/NumberUtilities.ts ../../third_party/devtools-frontend/src/front_end/core/platform/PromiseUtilities.ts ../../third_party/devtools-frontend/src/front_end/core/platform/StringUtilities.ts ../../third_party/devtools-frontend/src/front_end/core/platform/Timing.ts ../../third_party/devtools-frontend/src/front_end/core/platform/TypedArrayUtilities.ts ../../third_party/devtools-frontend/src/front_end/core/platform/TypescriptUtilities.ts ../../third_party/devtools-frontend/src/front_end/core/platform/UIString.ts ../../third_party/devtools-frontend/src/front_end/core/platform/UserVisibleError.ts
Traceback (most recent call last):
  File "/var/tmp/portage/dev-util/electron-32.1.2/work/chromium-128.0.6613.137/out/Release/../../third_party/devtools-frontend/src/third_party/typescript/ts_library.py", line 370, in <module>
    sys.exit(main())
             ~~~~^^
  File "/var/tmp/portage/dev-util/electron-32.1.2/work/chromium-128.0.6613.137/out/Release/../../third_party/devtools-frontend/src/third_party/typescript/ts_library.py", line 331, in main
    return runEsbuild(opts)
  File "/var/tmp/portage/dev-util/electron-32.1.2/work/chromium-128.0.6613.137/out/Release/../../third_party/devtools-frontend/src/third_party/typescript/ts_library.py", line 217, in runEsbuild
    p = subprocess.run(cmd)
  File "/usr/lib/python3.13/subprocess.py", line 554, in run
    with Popen(*popenargs, **kwargs) as process:
         ~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/subprocess.py", line 1036, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                        pass_fds, cwd, env,
                        ^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
                        gid, gids, uid, umask,
                        ^^^^^^^^^^^^^^^^^^^^^^
                        start_new_session, process_group)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/subprocess.py", line 1966, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/var/tmp/portage/dev-util/electron-32.1.2/work/chromium-128.0.6613.137/third_party/devtools-frontend/src/third_party/esbuild/esbuild'
[39/48004] CXX obj/third_party/protobuf/protobuf_lite/wire_format_lite.o
ninja: build stopped: subcommand failed.

but at the same time

ls -la /var/tmp/portage/dev-util/electron-32.1.2/work/chromium-128.0.6613.137/third_party/devtools-frontend/src/third_party/esbuild/esbuild
-rwxr-xr-x 1 portage portage 11019212 Sep 10 21:51 /var/tmp/portage/dev-util/electron-32.1.2/work/chromium-128.0.6613.137/third_party/devtools-frontend/src/third_party/esbuild/esbuild

ldd /var/tmp/portage/dev-util/electron-32.1.2/work/chromium-128.0.6613.137/third_party/devtools-frontend/src/third_party/esbuild/esbuild
        /lib64/ld-linux-x86-64.so.2 (0x75b496467000)
        libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x75b496467000)
        libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x75b496467000)

and it looks like this error is related to the fact that esbuild is linked to glibc, and it needs to be built manually under musl?

PF4Public · 2024-10-01T16:14:18Z

ungoogled-chromium has official USE flag for that, I didn't bother implementing it in electron. Merely disabling official is not enough, there is a bug upstream (which you've just rediscovered), which I fixed in ungoogled this way:

gentoo-overlay/www-client/ungoogled-chromium/ungoogled-chromium-129.0.6668.70_p1.ebuild

Line 1620 in 479732e

myconf_gn+=" devtools_skip_typecheck=false"

Kawanaao added 2 commits September 24, 2024 10:02

First steps of fixes [4381/49360]

3476720

Added upstream patch for perfetto due to include linux and musl conflict, disabled shim allocator as it causes multiple errors without mallinfo implementation, ref_ptr_support requiring it and added mallinfo stub for particularly tricky code

Moving from Dev to Prod environment

c9d3c2a

I test my changes without unnecessary ifs, just in case

Kawanaao marked this pull request as draft September 24, 2024 07:20

Kawanaao added 4 commits September 24, 2024 12:02

Versioning of resolver based on __RES [7344/49360]

daf37dd

+ Resolver patch

0fbc6fe

[ugly] llvm-subzero config patch [5657/49360]

1bb6b51

removed cdefs from sync.h and ptrace.h, no-macro-rdf

e9c11b1

Kawanaao added 3 commits September 25, 2024 03:34

headers to files, fix decls

f2136ed

Some declarations remove implementations instead of making the implementation empty, which causes errors at link stage when these symbols should have been there, but they were not included in the final build due to incorrect declarations.

add resheduling for way too fast child processes

4a6931d

Some processes launched by electron as part of the tests are too lightweight and can lead to a race between the parent process and the child process

disabled CLONE_SETTLS and dependent optimizations

7c4151e

CLONE_SETTLS is not used correctly within Chromium, this behavior has been deprecated in musl and now returns EINVAL

Kawanaao and others added 4 commits September 25, 2024 17:48

support for isolating necessary musl syscalls

fd6e4fe

Within the chromium sandbox, some function definitions could use syscalls that are forbidden for glibc, such as: shed_setschedulerer, mremap, membarrier, set_tid_address

Delete dev-util/electron/.electron-32.1.2.ebuild.kate-swp

6fedb6e

This file is only required for the Kate editor

only the bare minimum syscals should be published

b1d53f0

Merge branch 'musl-chromium-fix' of https://github.com/Kawanaao/pf4pu…

d414fb1

…blic-gentoo-overlay into musl-chromium-fix

Fixed debug use flag, added project for nodejs utilities

c3408a2

libcxx conflicts with system-re2

28faff4

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[MUSL] [chromium-based]: Build support for Musl #376

[MUSL] [chromium-based]: Build support for Musl #376

Kawanaao commented Sep 24, 2024

Kawanaao commented Sep 24, 2024 •

edited

Loading

Kawanaao commented Sep 24, 2024

Kawanaao commented Sep 24, 2024 •

edited

Loading

PF4Public commented Sep 24, 2024

Kawanaao commented Sep 25, 2024 •

edited

Loading

Kawanaao commented Sep 25, 2024

PF4Public commented Sep 25, 2024 •

edited

Loading

Kawanaao commented Sep 25, 2024

PF4Public commented Sep 26, 2024 •

edited

Loading

Kawanaao commented Sep 27, 2024 •

edited

Loading

Kawanaao commented Sep 27, 2024

PF4Public commented Sep 27, 2024 •

edited

Loading

Kawanaao commented Sep 27, 2024

PF4Public commented Sep 27, 2024

PF4Public commented Sep 27, 2024

Kawanaao commented Sep 28, 2024

Kawanaao commented Sep 29, 2024

PF4Public commented Oct 1, 2024

[MUSL] [chromium-based]: Build support for Musl #376

Are you sure you want to change the base?

[MUSL] [chromium-based]: Build support for Musl #376

Conversation

Kawanaao commented Sep 24, 2024

Kawanaao commented Sep 24, 2024 • edited Loading

Kawanaao commented Sep 24, 2024

Kawanaao commented Sep 24, 2024 • edited Loading

PF4Public commented Sep 24, 2024

Kawanaao commented Sep 25, 2024 • edited Loading

Kawanaao commented Sep 25, 2024

PF4Public commented Sep 25, 2024 • edited Loading

Kawanaao commented Sep 25, 2024

PF4Public commented Sep 26, 2024 • edited Loading

Kawanaao commented Sep 27, 2024 • edited Loading

Kawanaao commented Sep 27, 2024

PF4Public commented Sep 27, 2024 • edited Loading

Kawanaao commented Sep 27, 2024

PF4Public commented Sep 27, 2024

PF4Public commented Sep 27, 2024

Kawanaao commented Sep 28, 2024

Kawanaao commented Sep 29, 2024

PF4Public commented Oct 1, 2024

Kawanaao commented Sep 24, 2024 •

edited

Loading

Kawanaao commented Sep 24, 2024 •

edited

Loading

Kawanaao commented Sep 25, 2024 •

edited

Loading

PF4Public commented Sep 25, 2024 •

edited

Loading

PF4Public commented Sep 26, 2024 •

edited

Loading

Kawanaao commented Sep 27, 2024 •

edited

Loading

PF4Public commented Sep 27, 2024 •

edited

Loading