jar() rule fails with "Argument list too long" error

[madushan1000]

Describe the bug
Where there are a lot of java files in a jar rule meson fails with something like

[0/8] Compiling Java sources for libcore/core.jarninja: fatal: posix_spawn: Argument list too long

To Reproduce

I'm trying to compile libcore from aosp, but this should work with any sufficiently large java library(1500+ java files), so to reproduce

git clone https://android.googlesource.com/platform/libcore -b android15-release --depth=1

put this meson.build file in the root directory. https://gist.github.com/madushan1000/e164c18c8f290e1064fa2f1959086e65#file-meson-build

meson setup builddir && meson compile -C builddir

Expected behavior

meson should not error with "fatal: posix_spawn: Argument list too long"
javac has a mechanism to read the source file list from another file. you can put all the relative file paths in sources.list and then javac @sources.list consider using this method for the jar rule. setting ulimit -s unlimited doesn't seem to help.

system parameters

• Is this a cross build or just a plain native build ? This is a native build
• what operating system : ubuntu 24.04
• what Python version are you using : 3.12.3
• what meson --version: 1.6.0
• what ninja --version if it's a Ninja build: 1.11.1
• jdk version: 17.0.13

[madushan1000]

I tried this patch, it seems to work but I'm not sure if follows meson best practices.

diff --git a/mesonbuild/backend/ninjabackend.py b/mesonbuild/backend/ninjabackend.py
index 5716ea29e..52c7008db 100644
--- a/mesonbuild/backend/ninjabackend.py
+++ b/mesonbuild/backend/ninjabackend.py
@@ -1570,8 +1570,14 @@ class NinjaBackend(backends.Backend):
             plain_class_paths.append(plain_class_path)
             rel_obj = os.path.join(self.get_target_private_dir(target), plain_class_path)
             rel_objs.append(rel_obj)
-        element = NinjaBuildElement(self.all_outputs, rel_objs, self.compiler_to_rule_name(compiler), rel_srcs)
-        element.add_dep(deps)
+
+        source_list_file_path = os.path.join(self.environment.get_build_dir(), self.get_target_private_dir(target), "sources.list")
+        with open(source_list_file_path, 'w', encoding='utf-8') as sourcelist:
+            sourcelist.writelines([os.path.join(self.environment.get_source_dir(),f.relative_name()) + '\n' for f in srcs])
+
+        element = NinjaBuildElement(self.all_outputs, rel_objs, self.compiler_to_rule_name(compiler), [])
+        #element.add_dep(deps)
+        args.append('@'+source_list_file_path)
         element.add_item('ARGS', args)
         element.add_item('FOR_JAR', self.get_target_filename(target))
         self.add_build(element)

[eli-schwartz]

Maybe what this means is we should be plugging the javac compiler handling up to rspfile support? See how we do the similar thing for the GCC handler.

[madushan1000]

It wasn't very difficult to get rspfile working. had to change $out.rsp to something static because meson tries to include all class file names in $out and throws a file name too long error. But meson compiles all the java files on every run. I'm not sure why.

diff --git a/mesonbuild/backend/ninjabackend.py b/mesonbuild/backend/ninjabackend.py
index 5716ea29e..b70cf1518 100644
--- a/mesonbuild/backend/ninjabackend.py
+++ b/mesonbuild/backend/ninjabackend.py
@@ -260,8 +260,8 @@ class NinjaRule:
         for rsp in rule_iter():
             outfile.write(f'rule {self.name}{rsp}\n')
             if rsp == '_RSP':
-                outfile.write(' command = {} @$out.rsp\n'.format(' '.join([self._quoter(x) for x in self.command])))
-                outfile.write(' rspfile = $out.rsp\n')
+                outfile.write(' command = {} @sources.rsp\n'.format(' '.join([self._quoter(x) for x in self.command])))
+                outfile.write(' rspfile = sources.rsp\n')
                 outfile.write(' rspfile_content = {}\n'.format(' '.join([self._quoter(x, rspfile_quote_func) for x in rspfile_args])))
             else:
                 outfile.write(' command = {}\n'.format(' '.join([self._quoter(x) for x in self.command + self.args])))
@@ -2393,9 +2393,10 @@ class NinjaBackend(backends.Backend):
 
     def generate_java_compile_rule(self, compiler) -> None:
         rule = self.compiler_to_rule_name(compiler)
-        command = compiler.get_exelist() + ['$ARGS', '$in']
+        command = compiler.get_exelist() + ['$ARGS']
+        args = ['$in']
         description = 'Compiling Java sources for $FOR_JAR'
-        self.add_rule(NinjaRule(rule, command, [], description))
+        self.add_rule(NinjaRule(rule, command, args, description, rspable=True))
 
     def generate_cs_compile_rule(self, compiler: 'CsCompiler') -> None:
         rule = self.compiler_to_rule_name(compiler)