Should support filename longer than 260 characters

[6ziv]

Well, I can see that it is a limitation from Microsoft Windows.
But MSDN suggests that this limitation may be removed.
Enable Long Paths in Windows 10, Version 1607, and Later
And there are cases we have to use long paths (when handling some really complicated 3rd-party projects).
Maybe ninja could only raise a warning, but do not stop the build, like cmake does?

[jhasse]

Duplicate of #1514.

[jhasse]

#1514 was removed by GitHub because the user who created it was banned or something. No idea why they won't simply remove his posts only ...

Anyway, I will reopen this one then. #1514 can still be accessed via the GitHub API: https://api.github.com/repos/ninja-build/ninja/issues/1514/comments

[jhasse]

https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation

[LDong-Arm]

https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation

Hi @jhasse,

From the page you linked, the user needs to edit the registry to enable long path support, and also the application needs to declare it in its own manifest (pasted from that page):

<application xmlns="urn:schemas-microsoft-com:asm.v3">
    <windowsSettings xmlns:ws2="http://schemas.microsoft.com/SMI/2016/WindowsSettings">
        <ws2:longPathAware>true</ws2:longPathAware>
    </windowsSettings>
</application>

Presumably adding this to ninja.manifest should work? Or how would this impact compatibility with old Windows?

Thanks.

[LDong-Arm]

Oh #1939 already does that actually.

[emmenlau]

So, what are the next steps for this issue?

[parbo]

RealDiskInterFace uses posix file functions like fopen, and also the *A versions of win32 calls, while the long paths page says only *W versions are affected. Another option might be to prepend \\?\, but this only works for absolute paths.

[parbo]

I made a PR: #2289

[vbaderks]

Hi @parbo

FYI: It seems that the Windows Universal C CRT only calls CreateFileW. It performs internally a codepage\utf-8 conversion to a wchar_t before calling CreateFileW:
C:\Program Files (x86)\Windows Kits\10\Source\10.0.22621.0\ucrt\lowio\open.cpp (method _sopen_nolock at line 833 for example)

[parbo]

Hmm, why didn't it work with fopen then? Maybe the longPathsAware manifest doesn't extend through to linked libraries/dll:s?

…

On Fri, Apr 21, 2023, 10:26 Victor Derks ***@***.***> wrote: Hi @parbo <https://github.com/parbo> FYI: It seems that the Windows Universal C CRT only calls CreateFileW. It performs internally a codepage\utf-8 conversion to a wchar_t before calling CreateFileW: C:\Program Files (x86)\Windows Kits\10\Source\10.0.22621.0\ucrt\lowio\open.cpp (method _sopen_nolock at line 833 for example) — Reply to this email directly, view it on GitHub <#1900 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA52CQ7SL6436E4ZQDQI67LXCJACXANCNFSM4VDY46VA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[vbaderks]

@parbo. I am assuming that the longPathsAware manifest option applies to the .exe and is enabled process wide at startup or not. I didn't do a debug session in Ninja, just a debug-trace of a local app that uses fopen on Windows 11.

[parbo]

Maybe only some of the `*A` functions needed replacing, and not the `fopen` et al.

…

On Fri, Apr 21, 2023, 16:16 Victor Derks ***@***.***> wrote: @parbo <https://github.com/parbo>. I am assuming that the longPathsAware manifest option applies to the .exe and is enabled process wide at startup or not. I didn't do a debug session in Ninja, just a debug-trace of a local app that uses fopen on Windows 11. — Reply to this email directly, view it on GitHub <#1900 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA52CQ36XK7XM3Q4M5GONNDXCKJE7ANCNFSM4VDY46VA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[jhasse]

@XantreGodlike Because the last release of ninja was in 2022.

[XantreDev]

Do new releases be planned?

[smalik007]

With #2321 and other PRs now merged, it seems to be fixed! We're just missing an official release, unfortunately. Here's the workaround I used to compile a RN New Architecture app on Windows:

• Enable Long Paths first! Instructions to do that here.
• Download the latest build artifact of Ninja (I used this one) and put it in C:\ninja\ninja.exe.
• Go to android/app/build.gradle inside of your RN project and add this bit of code inside of android.defaultConfig:

externalNativeBuild {
    cmake {
        arguments "-DCMAKE_MAKE_PROGRAM=C:\\ninja\\ninja.exe", "-DCMAKE_OBJECT_PATH_MAX=1024"
    }
}

To explain why this is needed: Even if you've added C:\ninja to your path, Gradle still uses ninja from the built-in CMake from the Android SDK. Of course, it doesn't strictly need to be in C:\ninja, but chances are that most people have it there already. With just the first argument, it should probably compile already. However, CMake is still warning us about potentially broken builds because the path is too long, so we set it to 1024 to fix that with the second argument.

Lastly, if you want to make this more "contributor-friendly", you can run something like where ninja in your build.gradle, instead of having a hardcoded path like this. Just make sure to check the version beforehand :)

I tried with extra cmake flags but the problem still persist for me on latest master. The registry flag is enabled for long path , I tried to add some debugging prints to see what's happening and found the flag is not getting set, the ntdll.dll library call for registry flag of long path returns false. Also there are some build warnings about incompatible casting of function types, if that could be any of concern?

[jhasse]

You're using MinGW, @Sculas was referring to the artifact from GitHub Actions which builds with MSVC. Can you try that instead? Maybe it makes a difference because of the UCRT.

[smalik007]

The above built artifact link has expired. But I tried the latest built from the master (ninja-build-artifact), I no longer see the ninja error due long path however my build still fails, Now this could be the MinGW that might be causing it to fail now.
Thanks for the quick response :)

[Rc85]

With #2321 and other PRs now merged, it seems to be fixed! We're just missing an official release, unfortunately. Here's the workaround I used to compile a RN New Architecture app on Windows:

• Enable Long Paths first! Instructions to do that here.
• Download the latest build artifact of Ninja here and put it in C:\ninja\ninja.exe.
• Go to android/app/build.gradle inside of your RN project and add this bit of code inside of android.defaultConfig:

externalNativeBuild {
    cmake {
        arguments "-DCMAKE_MAKE_PROGRAM=C:\\ninja\\ninja.exe", "-DCMAKE_OBJECT_PATH_MAX=1024"
    }
}

To explain why this is needed: Even if you've added C:\ninja to your path, Gradle still uses ninja from the built-in CMake from the Android SDK. Of course, it doesn't strictly need to be in C:\ninja, but chances are that most people have it there already. With just the first argument, it should probably compile already. However, CMake is still warning us about potentially broken builds because the path is too long, so we set it to 1024 to fix that with the second argument.

Lastly, if you want to make this more "contributor-friendly", you can run something like where ninja in your build.gradle, instead of having a hardcoded path like this. Just make sure to check the version beforehand :)

edit 21/03/2024: replaced expired artifact link with a link to the latest workflow runs

The artifact link doesn't link to anywhere to download anything, so I assume you mean the latest version 1.11.1?

I have an RN project and a library (react-native-vision-camera) is giving me this error. I enabled long paths and tried your method by adding the code to my build.gradle and also to the camera's build.gradle, none worked. So maybe it's the ninja.exe that I was using?

The only thing that worked was rename my project path from F:/Projects/<project-name>/<dir1>/<dir2>/<dir3> to F:/p/r/m/a/c, which is ridiculous. It's so sad that in 2024, we still can't deal with long paths.

[jhasse]

There seem to be some issues left (only MinGW build? Without setting the registry value it should still not work?), but some are fixed. I will close this PR, please create new issues with specific failures that you see with 1.12.0 and in which circumstances they occur.

The artifact link doesn't link to anywhere to download anything, so I assume you mean the latest version 1.11.1?

No, it was a master build with some changes not in 1.11.1. Please try with 1.12.0 again.

[carrot93]

ninjia 1.12.1 still has not been solved "Filename longer than 260 characters Error" on windows( even Enablelongpath). So a temporary way is to shorten the file name(directory name). For python -m pip install, Temporarily "uuid.uuid4().hex[:8]" at "pip/_internal/req/req_install.py", and after install package, recovery to default.
May be risky, temporary way, wait for ninja to update to new version.

[SidneyLann]

ninjia 1.10.2 still has not been solved "Filename longer than 260 characters Error" on windows( even Enablelongpath).

[digit-google]

Can you provide exact reproduction steps so we can verify that ourselves (and verify the Ninja version being used too)?

[SidneyLann]

1. Clone https://github.com/SidneyLann/rn75.git to a long path dir
2. cd to android
3. yarn
4. yarn android
5. should have error: Filename longer than 260 characters Error

[Sculas]

ninjia 1.10.2 still has not been solved "Filename longer than 260 characters Error" on windows( even Enablelongpath).

You're running Ninja 1.10.2, correct? This issue has been fixed in v1.12.0 (get the latest version here).
Make sure you actually have long paths enabled in Windows as well, as this obviously doesn't work without that.

@SidneyLann Next time though, please read the issue comments before you post your own comment saying a version from 4 years ago doesn't work. Numerous fixes have been posted here already, and it was even closed with a "fixed in 1.12.0" comment.

Lastly, it seems your reproducer project is missing the React Native workaround specified here: #1900 (comment)
The Android SDK includes an old version of Ninja, which it uses by default unless you use that workaround (see comment for info).
(@carrot93 make sure you've done this as well if your project is also React Native!)

[sylque]

The provided workaround (#1900 (comment)) doesn't work for me. I get the error even with the Windows fix, the new ninja.exe installed in c:\ninja, and the modified build.gradle:

C:\xxxxxxxxxx\Tech\dev\xxxxxxxxxxxxxx\dev\test01\my-app2>npx expo run:android
› Building app...
Configuration on demand is an incubating feature.

> Configure project :expo

Using expo modules
  - expo-asset (11.0.1)
  - expo-blur (14.0.1)
  - expo-constants (17.0.3)
  - expo-file-system (18.0.6)
  - expo-font (13.0.2)
  - expo-haptics (14.0.0)
  - expo-keep-awake (14.0.1)
  - expo-linking (7.0.3)
  - expo-modules-core (2.1.2)
  - expo-splash-screen (0.29.18)
  - expo-system-ui (4.0.6)
  - expo-web-browser (14.0.1)


> Configure project :react-native-reanimated
Android gradle plugin: 8.6.0
Gradle: 8.10.2

> Task :react-native-reanimated:buildCMakeDebug[arm64-v8a] FAILED
C/C++: ninja: error: mkdir(src/main/cpp/reanimated/CMakeFiles/reanimated.dir/C_/xxxxxxxxxx/Tech/dev/xxxxxxxxxxxxxx/dev/test01/my-app2/node_modules/react-native-reanimated): No such file or directory

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':react-native-reanimated:buildCMakeDebug[arm64-v8a]'.
> com.android.ide.common.process.ProcessException: ninja: Entering directory `C:\xxxxxxxxxx\Tech\dev\xxxxxxxxxxxxxx\dev\test01\my-app2\node_modules\react-native-reanimated\android\.cxx\Debug\5v1g473x\arm64-v8a'
  [0/2] Re-checking globbed directories...
  ninja: build stopped: .

  C++ build system [build] failed while executing:
      @echo off
      "C:\\Users\\Sylvain\\AppData\\Local\\Android\\Sdk\\cmake\\3.22.1\\bin\\ninja.exe" ^
        -C ^
        "C:\\xxxxxxxxxx\\Tech\\dev\\xxxxxxxxxxxxxx\\dev\\test01\\my-app2\\node_modules\\react-native-reanimated\\android\\.cxx\\Debug\\5v1g473x\\arm64-v8a" ^
        reanimated ^
        worklets
    from C:\xxxxxxxxxx\Tech\dev\xxxxxxxxxxxxxx\dev\test01\my-app2\node_modules\react-native-reanimated\android
  ninja: error: mkdir(src/main/cpp/reanimated/CMakeFiles/reanimated.dir/C_/xxxxxxxxxx/Tech/dev/xxxxxxxxxxxxxx/dev/test01/my-app2/node_modules/react-native-reanimated): No such file or directory

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org.

BUILD FAILED in 18s
421 actionable tasks: 25 executed, 1 from cache, 395 up-to-date
Error: C:\xxxxxxxxxx\Tech\dev\xxxxxxxxxxxxxx\dev\test01\my-app2\android\gradlew.bat app:assembleDebug -x lint -x test --configure-on-demand --build-cache -PreactNativeDevServerPort=8081 -PreactNativeArchitectures=arm64-v8a,armeabi-v7a exited with non-zero code: 1
Error: C:\xxxxxxxxxx\Tech\dev\xxxxxxxxxxxxxx\dev\test01\my-app2\android\gradlew.bat app:assembleDebug -x lint -x test --configure-on-demand --build-cache -PreactNativeDevServerPort=8081 -PreactNativeArchitectures=arm64-v8a,armeabi-v7a exited with non-zero code: 1
    at ChildProcess.completionListener (C:\xxxxxxxxxx\Tech\dev\xxxxxxxxxxxxxx\dev\test01\my-app2\node_modules\@expo\spawn-async\src\spawnAsync.ts:67:13)
    at Object.onceWrapper (node:events:639:26)
    at ChildProcess.emit (node:events:524:28)
    at ChildProcess.cp.emit (C:\xxxxxxxxxx\Tech\dev\xxxxxxxxxxxxxx\dev\test01\my-app2\node_modules\cross-spawn\lib\enoent.js:34:29)
    at maybeClose (node:internal/child_process:1101:16)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:304:5)
    ...
    at spawnAsync (C:\xxxxxxxxxx\Tech\dev\xxxxxxxxxxxxxx\dev\test01\my-app2\node_modules\@expo\spawn-async\src\spawnAsync.ts:28:21)
    at spawnGradleAsync (C:\xxxxxxxxxx\Tech\dev\xxxxxxxxxxxxxx\dev\test01\my-app2\node_modules\@expo\cli\src\start\platforms\android\gradle.ts:134:28)
    at assembleAsync (C:\xxxxxxxxxx\Tech\dev\xxxxxxxxxxxxxx\dev\test01\my-app2\node_modules\@expo\cli\src\start\platforms\android\gradle.ts:83:16)
    at runAndroidAsync (C:\xxxxxxxxxx\Tech\dev\xxxxxxxxxxxxxx\dev\test01\my-app2\node_modules\@expo\cli\src\run\android\runAndroidAsync.ts:48:24)

The new ninja .exe is installed and used: if I rename c:\ninja to c:\ninja2, the build fails, complaining it cannot found ninja.exe. But, as you can see above, it seems the old (default) ninja.exe is still called at some point.

Moving my project closer to the root of my drive solves the issue.

EDIT: I ran "npx expo prebuild --clean" and "gradlew clean" but it didn't solve the issue.

EDIT2: replacing the old ninja.exe by the new one in its own folder solves the issue.