Crash with invalid preamble and concepts

[zyn0217]

(I think I should file the report to llvm repository, but I couldn't reproduce the crash directly with clang since it would stop building the AST once the including header is invalid/missing.)

The problematic code was observed from gnu.gold when I was randomly looking for something without a configured compile_commands.json.

// crash.cpp
// Crashes with clangd -check=crash.cpp. Note I'm using clangd with gcc-11's libstdc++ and c++20 enabled.

#include "gold.h" // Some header the clang couldn't find without a compile command. 

#include <vector>

class Object;

// Class Cref_inputs.  This is used to hold the list of input files
// for cross referencing.
class Cref_inputs {
private:
  // A list of input objects.
  typedef std::vector<Object *> Objects;

  // Report symbol counts for a list of Objects.
  void print_objects_symbol_counts(const Objects *) const;

  // List of input objects.
  Objects objects_;
};

// Report symbol counts for a list of inputs.

void Cref_inputs::print_objects_symbol_counts(const Objects *objects) const {
  objects->begin() != objects->end();
}

Relevant output:

I[14:39:39.888] argv[0]: ../llvm-project-Build/BuildDebug/bin/clangd
I[14:39:39.888] argv[1]: --check=/clangd-test/crash.cc
I[14:39:39.888] Entering check mode (no LSP server)
I[14:39:39.888] Testing on source file /clangd-test/crash.cc
I[14:39:39.889] Loading compilation database...
I[14:39:39.889] Failed to find compilation database for /clangd-test/crash.cc
I[14:39:39.890] Generic fallback command is: [/clangd-test] /usr/local/bin/clang -std=c++20 -resource-dir=/repo/llvm-project-Build/BuildDebug/lib/clang/18 -- /clangd-test/crash.cc

And the stacktrace:

clangd: /repo/llvm-project/llvm/include/llvm/ADT/PointerUnion.h:156: T llvm::PointerUnion<clang::Expr *, clang::concepts::Requirement::SubstitutionDiagnostic *>::get() const [PTs = <clang::Expr *, clang::concepts::Requirement::SubstitutionDiagnostic *>, T = clang::concepts::Requirement::SubstitutionDiagnostic *]: Assertion `isa<T>(*this) && "Invalid accessor called"' failed.
 #0 0x00007fc5259f5ced llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) /repo/llvm-project/llvm/lib/Support/Unix/Signals.inc:602:11
 #1 0x00007fc5259f618b PrintStackTraceSignalHandler(void*) /repo/llvm-project/llvm/lib/Support/Unix/Signals.inc:675:1
 #2 0x00007fc5259f4446 llvm::sys::RunSignalHandlers() /repo/llvm-project/llvm/lib/Support/Signals.cpp:104:5
 #3 0x00007fc5259f6945 SignalHandler(int) /repo/llvm-project/llvm/lib/Support/Unix/Signals.inc:413:1
 #4 0x00007fc524e86f60 __restore_rt sigaction.c:0:0
 #5 0x00007fc52433747a raise /root/glibc-2.33/signal/../sysdeps/unix/sysv/linux/raise.c:50:0
 #6 0x00007fc52433845e abort /root/glibc-2.33/stdlib/abort.c:81:0
 #7 0x00007fc52433011a __assert_fail_base /root/glibc-2.33/assert/assert.c:89:0
 #8 0x00007fc524330192 (/lib64/libc.so.6+0x33192)
 #9 0x00007fc52700f5bc clang::concepts::Requirement::SubstitutionDiagnostic* llvm::PointerUnion<clang::Expr*, clang::concepts::Requirement::SubstitutionDiagnostic*>::get<clang::concepts::Requirement::SubstitutionDiagnostic*>() const /repo/llvm-project/llvm/include/llvm/ADT/PointerUnion.h:0:5
#10 0x00007fc52700f489 clang::concepts::ExprRequirement::getExprSubstitutionDiagnostic() const /repo/llvm-project/clang/include/clang/AST/ExprConcepts.h:389:5
#11 0x00007fc526fa39ad diagnoseUnsatisfiedRequirement(clang::Sema&, clang::concepts::ExprRequirement*, bool) /repo/llvm-project/clang/lib/Sema/SemaConcept.cpp:925:13
#12 0x00007fc526fa379b diagnoseWellFormedUnsatisfiedConstraintExpr(clang::Sema&, clang::Expr*, bool) /repo/llvm-project/clang/lib/Sema/SemaConcept.cpp:1111:11
#13 0x00007fc526f9c800 void diagnoseUnsatisfiedConstraintExpr<std::pair<clang::SourceLocation, llvm::StringRef>>(clang::Sema&, clang::Expr const*, llvm::PointerUnion<clang::Expr*, std::pair<clang::SourceLocation, llvm::StringRef>*> const&, bool) /repo/llvm-project/clang/lib/Sema/SemaConcept.cpp:1140:1
#14 0x00007fc526f9c3a1 clang::Sema::DiagnoseUnsatisfiedConstraint(clang::ConstraintSatisfaction const&, bool) /repo/llvm-project/clang/lib/Sema/SemaConcept.cpp:1149:11
#15 0x00007fc527b674d2 DiagnoseBadDeduction(clang::Sema&, clang::NamedDecl*, clang::Decl*, clang::DeductionFailureInfo&, unsigned int, bool) /repo/llvm-project/clang/lib/Sema/SemaOverload.cpp:11240:3
#16 0x00007fc527b83974 DiagnoseBadDeduction(clang::Sema&, clang::OverloadCandidate*, unsigned int, bool) /repo/llvm-project/clang/lib/Sema/SemaOverload.cpp:11386:1
#17 0x00007fc527b65bf4 NoteFunctionCandidate(clang::Sema&, clang::OverloadCandidate*, unsigned int, bool, clang::LangAS) /repo/llvm-project/clang/lib/Sema/SemaOverload.cpp:11545:5
#18 0x00007fc527b4f0e0 clang::OverloadCandidateSet::NoteCandidates(clang::Sema&, llvm::ArrayRef<clang::Expr*>, llvm::ArrayRef<clang::OverloadCandidate*>, llvm::StringRef, clang::SourceLocation) /repo/llvm-project/clang/lib/Sema/SemaOverload.cpp:12100:7
#19 0x00007fc527b710cf clang::Sema::CreateOverloadedBinOp(clang::SourceLocation, clang::BinaryOperatorKind, clang::UnresolvedSetImpl const&, clang::Expr*, clang::Expr*, bool, bool, clang::FunctionDecl*) /repo/llvm-project/clang/lib/Sema/SemaOverload.cpp:14283:14
#20 0x00007fc52744b281 BuildOverloadedBinOp(clang::Sema&, clang::Scope*, clang::SourceLocation, clang::BinaryOperatorKind, clang::Expr*, clang::Expr*) /repo/llvm-project/clang/lib/Sema/SemaExpr.cpp:15995:12
#21 0x00007fc52744ac48 clang::Sema::BuildBinOp(clang::Scope*, clang::SourceLocation, clang::BinaryOperatorKind, clang::Expr*, clang::Expr*) /repo/llvm-project/clang/lib/Sema/SemaExpr.cpp:16097:14
#22 0x00007fc527407120 clang::Sema::ActOnBinOp(clang::Scope*, clang::SourceLocation, clang::tok::TokenKind, clang::Expr*, clang::Expr*) /repo/llvm-project/clang/lib/Sema/SemaExpr.cpp:15945:10
#23 0x00007fc52ade08ef clang::Parser::ParseRHSOfBinaryExpression(clang::ActionResult<clang::Expr*, true>, clang::prec::Level) /repo/llvm-project/clang/lib/Parse/ParseExpr.cpp:631:21
#24 0x00007fc52addf4b6 clang::Parser::ParseAssignmentExpression(clang::Parser::TypeCastState) /repo/llvm-project/clang/lib/Parse/ParseExpr.cpp:178:10
#25 0x00007fc52addf34f clang::Parser::ParseExpression(clang::Parser::TypeCastState) /repo/llvm-project/clang/lib/Parse/ParseExpr.cpp:126:18
#26 0x00007fc52ae5fef8 clang::Parser::ParseExprStatement(clang::Parser::ParsedStmtContext) /repo/llvm-project/clang/lib/Parse/ParseStmt.cpp:530:19
#27 0x00007fc52ae5e3ce clang::Parser::ParseStatementOrDeclarationAfterAttributes(llvm::SmallVector<clang::Stmt*, 32u>&, clang::Parser::ParsedStmtContext, clang::SourceLocation*, clang::ParsedAttributes&, clang::ParsedAttributes&) /repo/llvm-project/clang/lib/Parse/ParseStmt.cpp:279:14
#28 0x00007fc52ae5da4b clang::Parser::ParseStatementOrDeclaration(llvm::SmallVector<clang::Stmt*, 32u>&, clang::Parser::ParsedStmtContext, clang::SourceLocation*) /repo/llvm-project/clang/lib/Parse/ParseStmt.cpp:117:20
#29 0x00007fc52ae667d2 clang::Parser::ParseCompoundStatementBody(bool) /repo/llvm-project/clang/lib/Parse/ParseStmt.cpp:1205:11
#30 0x00007fc52ae67eb1 clang::Parser::ParseFunctionStatementBody(clang::Decl*, clang::Parser::ParseScope&) /repo/llvm-project/clang/lib/Parse/ParseStmt.cpp:2467:21
#31 0x00007fc52ae8986f clang::Parser::ParseFunctionDefinition(clang::ParsingDeclarator&, clang::Parser::ParsedTemplateInfo const&, clang::Parser::LateParsedAttrList*) /repo/llvm-project/clang/lib/Parse/Parser.cpp:1471:3
#32 0x00007fc52ad9267b clang::Parser::ParseDeclGroup(clang::ParsingDeclSpec&, clang::DeclaratorContext, clang::ParsedAttributes&, clang::SourceLocation*, clang::Parser::ForRangeInit*) /repo/llvm-project/clang/lib/Parse/ParseDecl.cpp:2195:27
#33 0x00007fc52ae886e5 clang::Parser::ParseDeclOrFunctionDefInternal(clang::ParsedAttributes&, clang::ParsedAttributes&, clang::ParsingDeclSpec&, clang::AccessSpecifier) /repo/llvm-project/clang/lib/Parse/Parser.cpp:1210:10
#34 0x00007fc52ae87c3f clang::Parser::ParseDeclarationOrFunctionDefinition(clang::ParsedAttributes&, clang::ParsedAttributes&, clang::ParsingDeclSpec*, clang::AccessSpecifier) /repo/llvm-project/clang/lib/Parse/Parser.cpp:1225:12
#35 0x00007fc52ae87513 clang::Parser::ParseExternalDeclaration(clang::ParsedAttributes&, clang::ParsedAttributes&, clang::ParsingDeclSpec*) /repo/llvm-project/clang/lib/Parse/Parser.cpp:1040:14
#36 0x00007fc52ae853ec clang::Parser::ParseTopLevelDecl(clang::OpaquePtr<clang::DeclGroupRef>&, clang::Sema::ModuleImportState&) /repo/llvm-project/clang/lib/Parse/Parser.cpp:742:12
#37 0x00007fc52ad7e2e7 clang::ParseAST(clang::Sema&, bool, bool) /repo/llvm-project/clang/lib/Parse/ParseAST.cpp:163:16
#38 0x00007fc52692c1e6 clang::ASTFrontendAction::ExecuteAction() /repo/llvm-project/clang/lib/Frontend/FrontendAction.cpp:1175:1
#39 0x00007fc52692bc0c clang::FrontendAction::Execute() /repo/llvm-project/clang/lib/Frontend/FrontendAction.cpp:1067:7
#40 0x00007fc5287761cd clang::clangd::ParsedAST::build(llvm::StringRef, clang::clangd::ParseInputs const&, std::unique_ptr<clang::CompilerInvocation, std::default_delete<clang::CompilerInvocation>>, llvm::ArrayRef<clang::clangd::Diag>, std::shared_ptr<clang::clangd::PreambleData const>) /repo/llvm-project/clang-tools-extra/clangd/ParsedAST.cpp:669:19
#41 0x00007fc52824981a clang::clangd::(anonymous namespace)::Checker::buildAST() /repo/llvm-project/clang-tools-extra/clangd/tool/Check.cpp:248:11
#42 0x00007fc528248a30 clang::clangd::check(llvm::StringRef, clang::clangd::ThreadsafeFS const&, clang::clangd::ClangdLSPServer::Options const&) /repo/llvm-project/clang-tools-extra/clangd/tool/Check.cpp:487:7
#43 0x00007fc528216bc6 clang::clangd::clangdMain(int, char**) /repo/llvm-project/clang-tools-extra/clangd/tool/ClangdMain.cpp:971:12
#44 0x00007fc52586ab02 main /repo/llvm-project/clang-tools-extra/clangd/tool/ClangdToolMain.cpp:12:3
#45 0x00007fc5243235e0 __libc_start_main /root/glibc-2.33/csu/../csu/libc-start.c:366:0
#46 0x00007fc52550624a _start /root/glibc-2.33/csu/../sysdeps/x86_64/start.S:122:0
[1]    4562 abort      ../llvm-project-Build/BuildDebug/bin/clangd --check=/clangd-test/crash.cc

The stacktrace shows that we're extracting SubstitutionDiagnostic from an ExprRequirement, while the stored pointer in ExprRequirement actually points to an Expr, not a SubstitutionDiagnostic. After some digging, I can conclude that this is caused by D129499, in which the ExprRequirement is constructed with SS_ExprSubstitutionFailure but without specifying any SubstitutionDiagnostic which is expected in Sema if the status is set to SS_ExprSubstitutionFailure.

Then I patched the code with a SubstitutionDiagnostic, and the crash was indeed solved. Now I can see the real diagnostic indicating that,

1. In template: no member named 'value' in 'std::is_convertible<bool, bool>' [Ln 27, Col 20]
2. Invalid operands to binary expression ('const_iterator' (aka '__normal_iterator<int, std::vector<Object *, std::allocator<Object *>>>') and 'const_iterator') [Ln 27, Col 20]

I've tracked down the SubstExpr and found that the substitution failure happened in Sema::BuildDeclarationNameExpr, where a RecoveryExpr might be built due to the invalidity of VD, and then we'll exercise the isSFINAEContext code path, resulting in an empty invalid expression instead of a RecoveryExpr, while it would be returned from calculateConstraintSatisfaction without storing diagnostic information in Satisfaction parameter. (I suppose this could somehow relate to the deferred concept instantiation strategy, but I'm not sure.)

Regardless, before sending out the patch, I think it'd be better to get rid of headers from the code and reproduce this crash in clang. Any kind help/comment would be appreciated.