Infinite recursion while detecting SFINAE

[Nerixyz]

I was trying to compile MrDocs and ran into a stack-overflow while building the documentation for MrDocs itself (which is build by default).

The error occurs when trying to detect whether fmt::formatter is such a template. One of its specializations is in mrdocs/Dom/String.hpp. In this specialization, the fmt::formatter extends its own template. When checking for SFINAE, the base template is checked as well, leading to infinite recursion.

The following change "fixes" the issue, but I don't know if it's correct to just "ignore" that base class.

diff --git a/src/lib/AST/ASTVisitor.cpp b/src/lib/AST/ASTVisitor.cpp
index 1d48999..893997e 100644
--- a/src/lib/AST/ASTVisitor.cpp
+++ b/src/lib/AST/ASTVisitor.cpp
@@ -1723,6 +1723,8 @@ public:
                             return true;
                         continue;
                     }
+                    if (sfinae_info->Template == TD)
+                        continue;
                     auto sfinae_result = isSFINAETemplate(
                         sfinae_info->Template, Member);
                     if(! sfinae_result)

Minimal example:

template <typename T> struct Foo {};
template<> struct Foo<char> : public Foo<int> {};

This isn't caught in CI, as the docs generation is disabled there.

---

Aside:

• The documentation generation from CMake doesn't work right now, because it tries to find a local-antora-playbook.yml - this should probably be antora-playbook.yml.
• MSVC fails to compile lib/AST/ASTVisitor.cpp, because CompleteExternalDeclaration is overridden with a DeclaratorDecl*, but declared with a VarDecl* in the version referenced on https://mrdocs.com/docs/mrdocs/install.html (7a28a5b3fee6c78ad59af79a3d03c00db153c49f; I suppose that version is outdated)

[alandefreitas]

I just reproduced the problem and can confirm it. For reference, I built the MrDocs documentation with:

mrdocs \
	--config="path\to\mrdocs\docs\mrdocs.yml" \
	--output="path\to\mrdocs\build\reference" \
	--addons=path/to/mrdocs/share/mrdocs/addons \
	--stdlib-includes=path/to/libraries/llvm/include \
	--concurrency 1 \
	--compilation-database=path\to\mrdocs\build\release-msvc\compile_commands.json

(omitting the path/to/mrdocs for brevity and privacy)

The stack trace is something like:

• mrdocs/src/lib/Lib/CorpusImpl.cpp

  Line 198 in 8537d3d

  processFile(path);

• mrdocs/src/lib/Lib/CorpusImpl.cpp

  Line 155 in 8537d3d

  if (Tool.run(action.get()))

• clang::tooling::ClangTool::run
• ...
• clang::FrontEndAction::Execute

• mrdocs/src/lib/AST/ASTVisitor.cpp

  Line 4756 in 8537d3d

  ParseAST(

• clang::ParseAST

• mrdocs/src/lib/AST/ASTVisitor.cpp

  Line 4623 in 8537d3d

  visitor.build();

• mrdocs/src/lib/AST/ASTVisitor.cpp

  Line 426 in 8537d3d

  traverseDecl(context_.getTranslationUnitDecl());

• mrdocs/src/lib/AST/ASTVisitor.cpp

  Line 3366 in 8537d3d

  visit(D, [&]<typename DeclTy>(DeclTy* DD)

• include/clang/AST/DeclNodes.inc: TRANSLATIONUNIT(TranslationUnit, Decl)

• mrdocs/src/lib/AST/ASTVisitor.cpp

  Line 3395 in 8537d3d

  traverse(DD, std::forward<Args>(args)...);

• mrdocs/src/lib/AST/ASTVisitor.cpp

  Line 3343 in 8537d3d

  traverseContext(DC);

• mrdocs/src/lib/AST/ASTVisitor.cpp

  Line 3405 in 8537d3d

  traverseDecl(D);

• mrdocs/src/lib/AST/ASTVisitor.cpp

  Line 3366 in 8537d3d

  visit(D, [&]<typename DeclTy>(DeclTy* DD)

• include/clang/AST/DeclNodes.inc: NAMESPACE(Namespace, NamedDecl)

• mrdocs/src/lib/AST/ASTVisitor.cpp

  Line 3395 in 8537d3d

  traverse(DD, std::forward<Args>(args)...);

• mrdocs/src/lib/AST/ASTVisitor.cpp

  Line 3041 in 8537d3d

  traverseContext(D);

• mrdocs/src/lib/AST/ASTVisitor.cpp

  Line 3405 in 8537d3d

  traverseDecl(D);

• mrdocs/src/lib/AST/ASTVisitor.cpp

  Line 3366 in 8537d3d

  visit(D, [&]<typename DeclTy>(DeclTy* DD)

• include/clang/AST/DeclNodes.inc: NAMESPACE(Namespace, NamedDecl)

• mrdocs/src/lib/AST/ASTVisitor.cpp

  Line 3395 in 8537d3d

  traverse(DD, std::forward<Args>(args)...);

• mrdocs/src/lib/AST/ASTVisitor.cpp

  Line 3041 in 8537d3d

  traverseContext(D);

• mrdocs/src/lib/AST/ASTVisitor.cpp

  Line 3405 in 8537d3d

  traverseDecl(D);

• mrdocs/src/lib/AST/ASTVisitor.cpp

  Line 3366 in 8537d3d

  visit(D, [&]<typename DeclTy>(DeclTy* DD)

• include/clang/AST/DeclNodes.inc: CLASSTEMPLATE(ClassTemplate, RedeclarableTemplateDecl)

• mrdocs/src/lib/AST/ASTVisitor.cpp

  Line 3377 in 8537d3d

  traverseDecl(DD->getTemplatedDecl(), DD);

• mrdocs/src/lib/AST/ASTVisitor.cpp

  Line 3366 in 8537d3d

  visit(D, [&]<typename DeclTy>(DeclTy* DD)

• include/clang/AST/DeclNodes.inc: CXXRECORD(CXXRecord, RecordDecl)

• mrdocs/src/lib/AST/ASTVisitor.cpp

  Line 3395 in 8537d3d

  traverse(DD, std::forward<Args>(args)...);

• mrdocs/src/lib/AST/ASTVisitor.cpp

  Line 3201 in 8537d3d

  traverseContext(D);

• mrdocs/src/lib/AST/ASTVisitor.cpp

  Line 3405 in 8537d3d

  traverseDecl(D);

• mrdocs/src/lib/AST/ASTVisitor.cpp

  Line 3366 in 8537d3d

  visit(D, [&]<typename DeclTy>(DeclTy* DD)

• include/clang/AST/DeclNodes.inc: TYPEALIASTEMPLATE(TypeAliasTemplate, RedeclarableTemplateDecl)

• mrdocs/src/lib/AST/ASTVisitor.cpp

  Line 3377 in 8537d3d

  traverseDecl(DD->getTemplatedDecl(), DD);

• mrdocs/src/lib/AST/ASTVisitor.cpp

  Line 3366 in 8537d3d

  visit(D, [&]<typename DeclTy>(DeclTy* DD)

• include/clang/AST/DeclNodes.inc: TYPEALIAS(TypeAlias, TypedefNameDecl)

• mrdocs/src/lib/AST/ASTVisitor.cpp

  Line 3395 in 8537d3d

  traverse(DD, std::forward<Args>(args)...);

• mrdocs/src/lib/AST/ASTVisitor.cpp

  Line 3333 in 8537d3d

  buildTypedef(I, created, D);

• mrdocs/src/lib/AST/ASTVisitor.cpp

  Line 2295 in 8537d3d

  I.Type = buildTypeInfo(

• mrdocs/src/lib/AST/ASTVisitor.cpp

  Line 4268 in 8537d3d

  if(auto SFINAE = isSFINAEType(qt); SFINAE.has_value())

• mrdocs/src/lib/AST/ASTVisitor.cpp

  Line 1870 in 8537d3d

  auto sfinae_result = isSFINAETemplate(

• Then the infinite loop...

  • mrdocs/src/lib/AST/ASTVisitor.cpp

    Line 1792 in 8537d3d

    if(CTSD->isExplicitSpecialization() &&

  • mrdocs/src/lib/AST/ASTVisitor.cpp

    Line 1726 in 8537d3d

    auto sfinae_result = isSFINAETemplate(

  • ...

@sdkrystian Please have a look at this logic and the proposed solution:

+                    if (sfinae_info->Template == TD)
+                        continue;

My intuition is the solution is going to be more complex than that.

The documentation generation from CMake doesn't work right now, because it tries to find a local-antora-playbook.yml - this should probably be antora-playbook.yml.

I opened another issue for this one: #680

MSVC fails to compile lib/AST/ASTVisitor.cpp, because CompleteExternalDeclaration is overridden with a DeclaratorDecl*, but declared with a VarDecl* in the version referenced on https://mrdocs.com/docs/mrdocs/install.html (7a28a5b3fee6c78ad59af79a3d03c00db153c49f; I suppose that version is outdated)

Interesting. But yes, this should be done in develop because we have no problems in CI.