Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[clang][lex] Fix non-portability diagnostics with absolute path #74782

Merged
merged 6 commits into from
Dec 18, 2023

Conversation

jansvoboda11
Copy link
Contributor

The existing code incorrectly assumes that Path can be empty. It can't, it always contains at least < or ". On Unix, this patch fixes an incorrect diagnostics that instead of "/Users/blah" suggested "Userss/blah". In assert builds, this would outright crash.

rdar://91172342

@llvmbot llvmbot added clang Clang issues not falling into any other category clang:frontend Language frontend issues, e.g. anything involving "Sema" labels Dec 7, 2023
The existing code incorrectly assumes that `Path` can be empty. It can't, it always contains at least `<` or `"`. On Unix, this patch fixes an incorrect diagnostics that instead of `"/Users/blah"` suggested `"Userss/blah"`. In assert builds, this would outright crash.

rdar://91172342
@llvmbot
Copy link
Member

llvmbot commented Dec 7, 2023

@llvm/pr-subscribers-clang

Author: Jan Svoboda (jansvoboda11)

Changes

The existing code incorrectly assumes that Path can be empty. It can't, it always contains at least &lt; or ". On Unix, this patch fixes an incorrect diagnostics that instead of "/Users/blah" suggested "Userss/blah". In assert builds, this would outright crash.

rdar://91172342


Full diff: https://github.com/llvm/llvm-project/pull/74782.diff

2 Files Affected:

  • (modified) clang/lib/Lex/PPDirectives.cpp (+14-8)
  • (added) clang/test/Lexer/case-insensitive-include-absolute.c (+13)
diff --git a/clang/lib/Lex/PPDirectives.cpp b/clang/lib/Lex/PPDirectives.cpp
index 956e2276f25b7..576b3c5925353 100644
--- a/clang/lib/Lex/PPDirectives.cpp
+++ b/clang/lib/Lex/PPDirectives.cpp
@@ -2466,15 +2466,21 @@ Preprocessor::ImportAction Preprocessor::HandleHeaderIncludeOrImport(
         // The drive letter is optional for absolute paths on Windows, but
         // clang currently cannot process absolute paths in #include lines that
         // don't have a drive.
-        // If the first entry in Components is a directory separator,
-        // then the code at the bottom of this loop that keeps the original
-        // directory separator style copies it. If the second entry is
-        // a directory separator (the C:\ case), then that separator already
-        // got copied when the C: was processed and we want to skip that entry.
-        if (!(Component.size() == 1 && IsSep(Component[0])))
+        if (Component.size() == 1 && IsSep(Component[0])) {
+          // Note: Path always contains at least '<' or '"'.
+          if (Path.size() == 1) {
+            // If the first entry in Components is a directory separator,
+            // then the code at the bottom of this loop that keeps the original
+            // directory separator style copies it.
+          } else {
+            // If the second entry is a directory separator (the C:\ case),
+            // then that separator already got copied when the C: was processed
+            // and we want to skip that entry.
+            continue;
+          }
+        } else {
           Path.append(Component);
-        else if (!Path.empty())
-          continue;
+        }
 
         // Append the separator(s) the user used, or the close quote
         if (Path.size() > NameWithoriginalSlashes.size()) {
diff --git a/clang/test/Lexer/case-insensitive-include-absolute.c b/clang/test/Lexer/case-insensitive-include-absolute.c
new file mode 100644
index 0000000000000..48f3d59421bd2
--- /dev/null
+++ b/clang/test/Lexer/case-insensitive-include-absolute.c
@@ -0,0 +1,13 @@
+// REQUIRES: case-insensitive-filesystem
+
+// RUN: rm -rf %t && split-file %s %t
+// RUN: sed "s|DIR|%/t|g" %t/tu.c.in > %t/tu.c
+// RUN: %clang_cc1 -fsyntax-only %t/tu.c 2>&1 | FileCheck %s --DPREFIX=%t
+
+//--- header.h
+//--- tu.c.in
+#import "DIR/Header.h"
+// CHECK:      tu.c:1:9: warning: non-portable path to file '"[[PREFIX]]/header.h"'; specified path differs in case from file name on disk [-Wnonportable-include-path]
+// CHECK-NEXT:    1 | #import "[[PREFIX]]/Header.h"
+// CHECK-NEXT:      |         ^~~~~~~~~~~~~~~~~~~~~
+// CHECK-NEXT:      |         "[[PREFIX]]/header.h"

@jansvoboda11 jansvoboda11 force-pushed the users/jansvoboda11/absolute-case-mismatch branch from 6dc0ca6 to 6ab18ed Compare December 7, 2023 23:27
Copy link
Collaborator

@cachemeifyoucan cachemeifyoucan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with small comments.

clang/lib/Lex/PPDirectives.cpp Outdated Show resolved Hide resolved
clang/lib/Lex/PPDirectives.cpp Outdated Show resolved Hide resolved
@jansvoboda11
Copy link
Contributor Author

jansvoboda11 commented Dec 8, 2023

My first version of the test put backslashes into the source file (i.e. #include "C:\foo\bar") which doesn't really work, because Clang treats those as string escape sequences. Instead of trying to replace \ with \\ in the test, I chose to use forward slashes, which should be fine.

Interestingly, that prevented Clang to diagnose the case mismatch. That's because in trySimplifyPath(), any path component that differs from the "real path" component in anything other than case prevents the diagnostic. Importantly, this also applies to mismatching separators (e.g. 'C:/' in source, 'C:\' in real path). I don't think that's intended or reasonable, so I pushed a fix that allows the diagnostic to be emitted regardless of separator differences.

Note that the diagnostic won't suggest fixing up the separators themselves, since those get inherited from the in-source spelling, not from the real path.

@jansvoboda11
Copy link
Contributor Author

Ping.

@jansvoboda11 jansvoboda11 merged commit f0691bc into main Dec 18, 2023
4 checks passed
@jansvoboda11 jansvoboda11 deleted the users/jansvoboda11/absolute-case-mismatch branch December 18, 2023 20:11
jansvoboda11 added a commit to swiftlang/llvm-project that referenced this pull request Dec 18, 2023
…#74782)

The existing code incorrectly assumes that `Path` can be empty. It
can't, it always contains at least `<` or `"`. On Unix, this patch fixes
an incorrect diagnostics that instead of `"/Users/blah"` suggested
`"Userss/blah"`. In assert builds, this would outright crash.

This patch also fixes a bug on Windows that would prevent the diagnostic
being triggered due to separator mismatch.

rdar://91172342
(cherry picked from commit f0691bc)
jansvoboda11 added a commit to swiftlang/llvm-project that referenced this pull request Dec 19, 2023
…mismatch-fix

[clang][lex] Fix non-portability diagnostics with absolute path (llvm#74782)
@jrmwng
Copy link

jrmwng commented Jan 3, 2024

492>Failed Tests (1):
492> Clang :: Lexer/case-insensitive-include-absolute.c

I built with "clang", "X86" and "AArch64" under Windows 11 and VS2022 community edition

492>------ Build started: Project: check-all, Configuration: Release x64 ------
492>Running all regression tests
492>llvm-lit.py: C:\GitHub\llvm-project\llvm\utils\lit\lit\llvm\config.py:46: note: using lit tools: C:\Program Files\Git\usr\bin
492>llvm-lit.py: C:\GitHub\llvm-project\llvm\utils\lit\lit\llvm\config.py:488: note: using clang: c:\github\llvm-project\build\release\bin\clang.exe
492>-- Testing: 70944 tests, 32 workers --
492>FAIL: Clang :: Lexer/case-insensitive-include-absolute.c (1 of 70944)
492>******************** TEST 'Clang :: Lexer/case-insensitive-include-absolute.c' FAILED ********************
492>Exit Code: 2
492>
492>Command Output (stdout):
492>--
492># RUN: at line 3
492>rm -rf C:\GitHub\llvm-project\build\tools\clang\test\Lexer\Output\case-insensitive-include-absolute.c.tmp && split-file C:\GitHub\llvm-project\clang\test\Lexer\case-insensitive-include-absolute.c C:\GitHub\llvm-project\build\tools\clang\test\Lexer\Output\case-insensitive-include-absolute.c.tmp
492># executed command: rm -rf 'C:\GitHub\llvm-project\build\tools\clang\test\Lexer\Output\case-insensitive-include-absolute.c.tmp'
492># executed command: split-file 'C:\GitHub\llvm-project\clang\test\Lexer\case-insensitive-include-absolute.c' 'C:\GitHub\llvm-project\build\tools\clang\test\Lexer\Output\case-insensitive-include-absolute.c.tmp'
492># RUN: at line 4
492>sed "s|DIR|C:/GitHub/llvm-project/build/tools/clang/test/Lexer/Output/case-insensitive-include-absolute.c.tmp|g" C:\GitHub\llvm-project\build\tools\clang\test\Lexer\Output\case-insensitive-include-absolute.c.tmp/tu.c.in > C:\GitHub\llvm-project\build\tools\clang\test\Lexer\Output\case-insensitive-include-absolute.c.tmp/tu.c
492># executed command: sed 's|DIR|C:/GitHub/llvm-project/build/tools/clang/test/Lexer/Output/case-insensitive-include-absolute.c.tmp|g' 'C:\GitHub\llvm-project\build\tools\clang\test\Lexer\Output\case-insensitive-include-absolute.c.tmp/tu.c.in'
492># RUN: at line 5
492>c:\github\llvm-project\build\release\bin\clang.exe -cc1 -internal-isystem F:\GitHub\llvm-project\build\Release\lib\clang\18\include -nostdsysteminc -fsyntax-only C:\GitHub\llvm-project\build\tools\clang\test\Lexer\Output\case-insensitive-include-absolute.c.tmp/tu.c 2>&1 | c:\github\llvm-project\build\release\bin\filecheck.exe C:\GitHub\llvm-project\clang\test\Lexer\case-insensitive-include-absolute.c --DDIR=C:/GitHub/llvm-project/build/tools/clang/test/Lexer/Output/case-insensitive-include-absolute.c.tmp
492># executed command: 'c:\github\llvm-project\build\release\bin\clang.exe' -cc1 -internal-isystem 'F:\GitHub\llvm-project\build\Release\lib\clang\18\include' -nostdsysteminc -fsyntax-only 'C:\GitHub\llvm-project\build\tools\clang\test\Lexer\Output\case-insensitive-include-absolute.c.tmp/tu.c'
492># executed command: 'c:\github\llvm-project\build\release\bin\filecheck.exe' 'C:\GitHub\llvm-project\clang\test\Lexer\case-insensitive-include-absolute.c' --DDIR=C:/GitHub/llvm-project/build/tools/clang/test/Lexer/Output/case-insensitive-include-absolute.c.tmp
492># .---command stderr------------
492>CUSTOMBUILD : # | FileCheck error : '<stdin>' is empty.
492># | FileCheck command line:  c:\github\llvm-project\build\release\bin\filecheck.exe C:\GitHub\llvm-project\clang\test\Lexer\case-insensitive-include-absolute.c --DDIR=C:/GitHub/llvm-project/build/tools/clang/test/Lexer/Output/case-insensitive-include-absolute.c.tmp
492># `-----------------------------
492>CUSTOMBUILD : # error : command failed with exit status: 2
492>
492>--
492>
492>********************
492>********************
492>Failed Tests (1):
492>  Clang :: Lexer/case-insensitive-include-absolute.c
492>
492>
492>Testing Time: 434.26s
492>
492>Total Discovered Tests: 92696
492>  Skipped          :    90 (0.10%)
492>  Unsupported      : 24219 (26.13%)
492>  Passed           : 68283 (73.66%)
492>  Expectedly Failed:   103 (0.11%)
492>  Failed           :     1 (0.00%)
492>C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(254,5): error MSB8066: Custom build for 'C:\GitHub\llvm-project\build\CMakeFiles\5344a242bf0d5667ee3e0d3564612537\check-all.rule;C:\GitHub\llvm-project\llvm\CMakeLists.txt' exited with code 1.
492>Done building project "check-all.vcxproj" -- FAILED.
========== Build: 491 succeeded, 1 failed, 55 up-to-date, 0 skipped ==========

@jansvoboda11
Copy link
Contributor Author

Thanks for reporting @jrmwng, I'll look into it.

@jansvoboda11
Copy link
Contributor Author

Thanks for reporting @jrmwng, I'll look into it.

I wasn't able to reproduce locally on Windows 10. Can you please share detailed reproduction steps and any other config that may be relevant? It seems your realpath to the LLVM repo is "C:\GitHub\llvm-project" and the check-all target is built using "c:\github\llvm-project". This works on my machine, so something else must be at play.

@jrmwng
Copy link

jrmwng commented Jan 4, 2024

@jansvoboda11 One thing is special.

The path C:\GitHub is a symbolic link to another harddisk.

<SYMLINKD>     GitHub [F:\GitHub]

@jrmwng
Copy link

jrmwng commented Jan 4, 2024

@jansvoboda11 it should be the following steps. (I try to repeat it now)

  1. CMake (cmake-gui)
    • Where is the source code: C:/GitHub/llvm-project/
    • Where to build the binaries: C:/GitHub/llvm-project/build
    • Click the "Configure" button
      • Create Directory --> "Yes"
      • Specify the generator for this project: Visual Studio 17 2022
      • Optional platform for generator: x64
      • Optional toolset to use: ClangCL
      • Use default native compilers
      • Click the "Finish" button
    • Modify the following entries
      • LLVM_ENABLE_PROJECTS: clang
      • LLVM_TARGETS_TO_BUILD: X86;AArch64
    • Click the "Configure" button again
    • Click the "Generate" button
    • Click the "Open Project" button
  2. Visual Studio is opened
    • Change configuration from "Debug" to "Release"
    • Build the "check-all" project

@jrmwng
Copy link

jrmwng commented Jan 4, 2024

@jansvoboda11 the same error using the above steps

547>------ Build started: Project: check-all, Configuration: Release x64 ------
547>Running all regression tests
547>llvm-lit.py: C:\GitHub\llvm-project\llvm\utils\lit\lit\llvm\config.py:46: note: using lit tools: C:\Program Files\Git\usr\bin
547>llvm-lit.py: C:\GitHub\llvm-project\llvm\utils\lit\lit\llvm\config.py:488: note: using clang: c:\github\llvm-project\build\release\bin\clang.exe
547>-- Testing: 70944 tests, 32 workers --
547>FAIL: Clang :: Lexer/case-insensitive-include-absolute.c (11482 of 70944)
547>******************** TEST 'Clang :: Lexer/case-insensitive-include-absolute.c' FAILED ********************
547>Exit Code: 2
547>
547>Command Output (stdout):
547>--
547># RUN: at line 3
547>rm -rf C:\GitHub\llvm-project\build\tools\clang\test\Lexer\Output\case-insensitive-include-absolute.c.tmp && split-file C:\GitHub\llvm-project\clang\test\Lexer\case-insensitive-include-absolute.c C:\GitHub\llvm-project\build\tools\clang\test\Lexer\Output\case-insensitive-include-absolute.c.tmp
547># executed command: rm -rf 'C:\GitHub\llvm-project\build\tools\clang\test\Lexer\Output\case-insensitive-include-absolute.c.tmp'
547># executed command: split-file 'C:\GitHub\llvm-project\clang\test\Lexer\case-insensitive-include-absolute.c' 'C:\GitHub\llvm-project\build\tools\clang\test\Lexer\Output\case-insensitive-include-absolute.c.tmp'
547># RUN: at line 4
547>sed "s|DIR|C:/GitHub/llvm-project/build/tools/clang/test/Lexer/Output/case-insensitive-include-absolute.c.tmp|g" C:\GitHub\llvm-project\build\tools\clang\test\Lexer\Output\case-insensitive-include-absolute.c.tmp/tu.c.in > C:\GitHub\llvm-project\build\tools\clang\test\Lexer\Output\case-insensitive-include-absolute.c.tmp/tu.c
547># executed command: sed 's|DIR|C:/GitHub/llvm-project/build/tools/clang/test/Lexer/Output/case-insensitive-include-absolute.c.tmp|g' 'C:\GitHub\llvm-project\build\tools\clang\test\Lexer\Output\case-insensitive-include-absolute.c.tmp/tu.c.in'
547># RUN: at line 5
547>c:\github\llvm-project\build\release\bin\clang.exe -cc1 -internal-isystem F:\GitHub\llvm-project\build\Release\lib\clang\18\include -nostdsysteminc -fsyntax-only C:\GitHub\llvm-project\build\tools\clang\test\Lexer\Output\case-insensitive-include-absolute.c.tmp/tu.c 2>&1 | c:\github\llvm-project\build\release\bin\filecheck.exe C:\GitHub\llvm-project\clang\test\Lexer\case-insensitive-include-absolute.c --DDIR=C:/GitHub/llvm-project/build/tools/clang/test/Lexer/Output/case-insensitive-include-absolute.c.tmp
547># executed command: 'c:\github\llvm-project\build\release\bin\clang.exe' -cc1 -internal-isystem 'F:\GitHub\llvm-project\build\Release\lib\clang\18\include' -nostdsysteminc -fsyntax-only 'C:\GitHub\llvm-project\build\tools\clang\test\Lexer\Output\case-insensitive-include-absolute.c.tmp/tu.c'
547># executed command: 'c:\github\llvm-project\build\release\bin\filecheck.exe' 'C:\GitHub\llvm-project\clang\test\Lexer\case-insensitive-include-absolute.c' --DDIR=C:/GitHub/llvm-project/build/tools/clang/test/Lexer/Output/case-insensitive-include-absolute.c.tmp
547># .---command stderr------------
547>CUSTOMBUILD : # | FileCheck error : '<stdin>' is empty.
547># | FileCheck command line:  c:\github\llvm-project\build\release\bin\filecheck.exe C:\GitHub\llvm-project\clang\test\Lexer\case-insensitive-include-absolute.c --DDIR=C:/GitHub/llvm-project/build/tools/clang/test/Lexer/Output/case-insensitive-include-absolute.c.tmp
547># `-----------------------------
547>CUSTOMBUILD : # error : command failed with exit status: 2
547>
547>--
547>
547>********************
547>********************
547>Failed Tests (1):
547>  Clang :: Lexer/case-insensitive-include-absolute.c
547>
547>
547>Testing Time: 433.52s
547>
547>Total Discovered Tests: 92696
547>  Skipped          :    90 (0.10%)
547>  Unsupported      : 24219 (26.13%)
547>  Passed           : 68283 (73.66%)
547>  Expectedly Failed:   103 (0.11%)
547>  Failed           :     1 (0.00%)
547>C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(254,5): error MSB8066: Custom build for 'C:\GitHub\llvm-project\build\CMakeFiles\5344a242bf0d5667ee3e0d3564612537\check-all.rule;C:\GitHub\llvm-project\llvm\CMakeLists.txt' exited with code 1.
547>Done building project "check-all.vcxproj" -- FAILED.

@jansvoboda11
Copy link
Contributor Author

Thank you. I think the symlink is most likely the culprit. I'll be able to confirm on my Windows machine tomorrow. For now, I have a speculative fix: #76985

@jansvoboda11
Copy link
Contributor Author

@jrmwng PR #76985 (landed in 853b133) seems to have resolved this issue for me. Can you confirm that on your end?

@jrmwng
Copy link

jrmwng commented Jan 6, 2024

@jrmwng PR #76985 (landed in 853b133) seems to have resolved this issue for me. Can you confirm that on your end?

Yes, your fix is effective 💯

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
clang:frontend Language frontend issues, e.g. anything involving "Sema" clang Clang issues not falling into any other category
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants