Skip to content

Conversation

@ilovepi
Copy link
Contributor

@ilovepi ilovepi commented Mar 21, 2025

The last version of this patch had memory leaks due to using the
BumpPtrAllocator for data types that required destructors to run to
release heap memory (e.g. via std::vector and std::string). This version
avoids that by using smart pointers, and dropping support for
BumpPtrAllocator.

We should refactor this code to use the BumpPtrAllocator again, but that
can be addressed in future patches, since those are more invasive
changes that need to refactor many of the core data types to avoid
owning allocations.

Adds Support for the Mustache Templating Language. See specs here:
https://mustache.github.io/mustache.5.html This patch implements
support+tests for majority of the features of the language including:

- Variables
- Comments
- Lambdas
- Sections

This meant as a library to support places where we have to generate
HTML, such as in clang-doc.

Co-authored-by: Peter Chou peter.chou@mail.utoronto.ca

Copy link
Contributor Author

ilovepi commented Mar 21, 2025

This stack of pull requests is managed by Graphite. Learn more about stacking.

@github-actions
Copy link

github-actions bot commented Mar 21, 2025

✅ With the latest revision this PR passed the C/C++ code formatter.

@ilovepi ilovepi marked this pull request as ready for review March 21, 2025 20:26
@llvmbot
Copy link
Member

llvmbot commented Mar 21, 2025

@llvm/pr-subscribers-llvm-support

Author: Paul Kirth (ilovepi)

Changes

The last version of this patch had memory leaks due to using the
BumpPtrAllocator for data types that required destructors to run to
release heap memory (e.g. via std::vector and std::string). This version
avoids that by using smart pointers, and dropping support for
BumpPtrAllocator.

We should refactor this code to use the BumpPtrAllocator again, but that
can be addressed in future patches, since those are more invasive
changes that need to refactor many of the core data types to avoid
owning allocations.

Adds Support for the Mustache Templating Language. See specs here:
https://mustache.github.io/mustache.5.html This patch implements
support+tests for majority of the features of the language including:

- Variables
- Comments
- Lambdas
- Sections

This meant as a library to support places where we have to generate
HTML, such as in clang-doc.

Co-authored-by: Peter Chou <peter.chou@mail.utoronto.ca>


Patch is 68.00 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/132467.diff

5 Files Affected:

  • (added) llvm/include/llvm/Support/Mustache.h (+127)
  • (modified) llvm/lib/Support/CMakeLists.txt (+1)
  • (added) llvm/lib/Support/Mustache.cpp (+763)
  • (modified) llvm/unittests/Support/CMakeLists.txt (+1)
  • (added) llvm/unittests/Support/MustacheTest.cpp (+1226)
diff --git a/llvm/include/llvm/Support/Mustache.h b/llvm/include/llvm/Support/Mustache.h
new file mode 100644
index 0000000000000..0ee9a344c0d68
--- /dev/null
+++ b/llvm/include/llvm/Support/Mustache.h
@@ -0,0 +1,127 @@
+//===--- Mustache.h ---------------------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// Implementation of the Mustache templating language supports version 1.4.2
+// currently relies on llvm::json::Value for data input.
+// See the Mustache spec for more information
+// (https://mustache.github.io/mustache.5.html).
+//
+// Current Features Supported:
+// - Variables
+// - Sections
+// - Inverted Sections
+// - Partials
+// - Comments
+// - Lambdas
+// - Unescaped Variables
+//
+// Features Not Supported:
+// - Set Delimiter
+// - Blocks
+// - Parents
+// - Dynamic Names
+//
+// The Template class is a container class that outputs the Mustache template
+// string and is the main class for users. It stores all the lambdas and the
+// ASTNode Tree. When the Template is instantiated it tokenizes the Template
+// String and creates a vector of Tokens. Then it calls a basic recursive
+// descent parser to construct the ASTNode Tree. The ASTNodes are all stored
+// in an arena allocator which is freed once the template class goes out of
+// scope.
+//
+// Usage:
+// \code
+//   // Creating a simple template and rendering it
+//   auto Template = Template("Hello, {{name}}!");
+//   Value Data = {{"name", "World"}};
+//   std::string Out;
+//   raw_string_ostream OS(Out);
+//   T.render(Data, OS);
+//   // Out == "Hello, World!"
+//
+//   // Creating a template with a partial and rendering it
+//   auto Template = Template("{{>partial}}");
+//   Template.registerPartial("partial", "Hello, {{name}}!");
+//   Value Data = {{"name", "World"}};
+//   std::string Out;
+//   raw_string_ostream OS(Out);
+//   T.render(Data, OS);
+//   // Out == "Hello, World!"
+//
+//   // Creating a template with a lambda and rendering it
+//   Value D = Object{};
+//   auto T = Template("Hello, {{lambda}}!");
+//   Lambda L = []() -> llvm::json::Value { return "World"; };
+//   T.registerLambda("lambda", L);
+//   std::string Out;
+//   raw_string_ostream OS(Out);
+//   T.render(D, OS);
+//   // Out == "Hello, World!"
+// \endcode
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_SUPPORT_MUSTACHE
+#define LLVM_SUPPORT_MUSTACHE
+
+#include "Error.h"
+#include "llvm/ADT/StringMap.h"
+#include "llvm/Support/Allocator.h"
+#include "llvm/Support/JSON.h"
+#include "llvm/Support/StringSaver.h"
+#include <functional>
+#include <vector>
+
+namespace llvm::mustache {
+
+using Lambda = std::function<llvm::json::Value()>;
+using SectionLambda = std::function<llvm::json::Value(std::string)>;
+
+class ASTNode;
+using AstPtr = std::unique_ptr<ASTNode>;
+
+// A Template represents the container for the AST and the partials
+// and Lambdas that are registered with it.
+class Template {
+public:
+  Template(StringRef TemplateStr);
+
+  Template(const Template &) = delete;
+
+  Template &operator=(const Template &) = delete;
+
+  Template(Template &&Other) noexcept;
+
+   // Define this in the cpp file to  work around ASTNode being an incomplete type.
+  ~Template();
+
+  Template &operator=(Template &&Other) noexcept;
+
+  void render(const llvm::json::Value &Data, llvm::raw_ostream &OS);
+
+  void registerPartial(std::string Name, std::string Partial);
+
+  void registerLambda(std::string Name, Lambda Lambda);
+
+  void registerLambda(std::string Name, SectionLambda Lambda);
+
+  // By default the Mustache Spec Specifies that HTML special characters
+  // should be escaped. This function allows the user to specify which
+  // characters should be escaped.
+  void overrideEscapeCharacters(DenseMap<char, std::string> Escapes);
+
+private:
+  StringMap<AstPtr> Partials;
+  StringMap<Lambda> Lambdas;
+  StringMap<SectionLambda> SectionLambdas;
+  DenseMap<char, std::string> Escapes;
+  AstPtr Tree;
+};
+} // namespace llvm::mustache
+
+#endif // LLVM_SUPPORT_MUSTACHE
diff --git a/llvm/lib/Support/CMakeLists.txt b/llvm/lib/Support/CMakeLists.txt
index 49a26a618de83..2754c97fce6c1 100644
--- a/llvm/lib/Support/CMakeLists.txt
+++ b/llvm/lib/Support/CMakeLists.txt
@@ -220,6 +220,7 @@ add_llvm_component_library(LLVMSupport
   MD5.cpp
   MSP430Attributes.cpp
   MSP430AttributeParser.cpp
+  Mustache.cpp      
   NativeFormatting.cpp
   OptimizedStructLayout.cpp
   Optional.cpp
diff --git a/llvm/lib/Support/Mustache.cpp b/llvm/lib/Support/Mustache.cpp
new file mode 100644
index 0000000000000..1ef4f71cd7a12
--- /dev/null
+++ b/llvm/lib/Support/Mustache.cpp
@@ -0,0 +1,763 @@
+//===-- Mustache.cpp ------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+#include "llvm/Support/Mustache.h"
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/Support/Error.h"
+#include "llvm/Support/raw_ostream.h"
+#include <sstream>
+
+using namespace llvm;
+using namespace llvm::mustache;
+
+namespace {
+
+using Accessor = SmallVector<std::string>;
+
+static bool isFalsey(const json::Value &V) {
+  return V.getAsNull() || (V.getAsBoolean() && !V.getAsBoolean().value()) ||
+         (V.getAsArray() && V.getAsArray()->empty());
+}
+
+static Accessor splitMustacheString(StringRef Str) {
+  // We split the mustache string into an accessor.
+  // For example:
+  //    "a.b.c" would be split into {"a", "b", "c"}
+  // We make an exception for a single dot which
+  // refers to the current context.
+  Accessor Tokens;
+  if (Str == ".") {
+    Tokens.emplace_back(Str);
+    return Tokens;
+  }
+  while (!Str.empty()) {
+    StringRef Part;
+    std::tie(Part, Str) = Str.split(".");
+    Tokens.emplace_back(Part.trim());
+  }
+  return Tokens;
+}
+} // namespace
+
+namespace llvm::mustache {
+
+class Token {
+public:
+  enum class Type {
+    Text,
+    Variable,
+    Partial,
+    SectionOpen,
+    SectionClose,
+    InvertSectionOpen,
+    UnescapeVariable,
+    Comment,
+  };
+
+  Token(std::string Str)
+      : TokenType(Type::Text), RawBody(std::move(Str)), TokenBody(RawBody),
+        AccessorValue({}), Indentation(0){};
+
+  Token(std::string RawBody, std::string TokenBody, char Identifier)
+      : RawBody(std::move(RawBody)), TokenBody(std::move(TokenBody)),
+        Indentation(0) {
+    TokenType = getTokenType(Identifier);
+    if (TokenType == Type::Comment)
+      return;
+    StringRef AccessorStr(this->TokenBody);
+    if (TokenType != Type::Variable)
+      AccessorStr = AccessorStr.substr(1);
+    AccessorValue = splitMustacheString(StringRef(AccessorStr).trim());
+  }
+
+  Accessor getAccessor() const { return AccessorValue; }
+
+  Type getType() const { return TokenType; }
+
+  void setIndentation(size_t NewIndentation) { Indentation = NewIndentation; }
+
+  size_t getIndentation() const { return Indentation; }
+
+  static Type getTokenType(char Identifier) {
+    switch (Identifier) {
+    case '#':
+      return Type::SectionOpen;
+    case '/':
+      return Type::SectionClose;
+    case '^':
+      return Type::InvertSectionOpen;
+    case '!':
+      return Type::Comment;
+    case '>':
+      return Type::Partial;
+    case '&':
+      return Type::UnescapeVariable;
+    default:
+      return Type::Variable;
+    }
+  }
+
+  Type TokenType;
+  // RawBody is the original string that was tokenized.
+  std::string RawBody;
+  // TokenBody is the original string with the identifier removed.
+  std::string TokenBody;
+  Accessor AccessorValue;
+  size_t Indentation;
+};
+
+class ASTNode {
+public:
+  enum Type {
+    Root,
+    Text,
+    Partial,
+    Variable,
+    UnescapeVariable,
+    Section,
+    InvertSection,
+  };
+
+  ASTNode(llvm::StringMap<AstPtr> &Partials, llvm::StringMap<Lambda> &Lambdas,
+          llvm::StringMap<SectionLambda> &SectionLambdas,
+          llvm::DenseMap<char, std::string> &Escapes)
+      : Partials(Partials), Lambdas(Lambdas), SectionLambdas(SectionLambdas),
+        Escapes(Escapes), Ty(Type::Root), Parent(nullptr),
+        ParentContext(nullptr) {}
+
+  ASTNode(std::string Body, ASTNode *Parent, llvm::StringMap<AstPtr> &Partials,
+          llvm::StringMap<Lambda> &Lambdas,
+          llvm::StringMap<SectionLambda> &SectionLambdas,
+          llvm::DenseMap<char, std::string> &Escapes)
+      : Partials(Partials), Lambdas(Lambdas), SectionLambdas(SectionLambdas),
+        Escapes(Escapes), Ty(Type::Text), Body(std::move(Body)), Parent(Parent),
+        ParentContext(nullptr) {}
+
+  // Constructor for Section/InvertSection/Variable/UnescapeVariable Nodes
+  ASTNode(Type Ty, Accessor Accessor, ASTNode *Parent,
+          llvm::StringMap<AstPtr> &Partials, llvm::StringMap<Lambda> &Lambdas,
+          llvm::StringMap<SectionLambda> &SectionLambdas,
+          llvm::DenseMap<char, std::string> &Escapes)
+      : Partials(Partials), Lambdas(Lambdas), SectionLambdas(SectionLambdas),
+        Escapes(Escapes), Ty(Ty), Parent(Parent),
+        AccessorValue(std::move(Accessor)), ParentContext(nullptr) {}
+
+  void addChild(AstPtr Child) { Children.emplace_back(std::move(Child)); };
+
+  void setRawBody(std::string NewBody) { RawBody = std::move(NewBody); };
+
+  void setIndentation(size_t NewIndentation) { Indentation = NewIndentation; };
+
+  void render(const llvm::json::Value &Data, llvm::raw_ostream &OS);
+
+private:
+  void renderLambdas(const llvm::json::Value &Contexts, llvm::raw_ostream &OS,
+                     Lambda &L);
+
+  void renderSectionLambdas(const llvm::json::Value &Contexts,
+                            llvm::raw_ostream &OS, SectionLambda &L);
+
+  void renderPartial(const llvm::json::Value &Contexts, llvm::raw_ostream &OS,
+                     ASTNode *Partial);
+
+  void renderChild(const llvm::json::Value &Context, llvm::raw_ostream &OS);
+
+  const llvm::json::Value *findContext();
+
+  StringMap<AstPtr> &Partials;
+  StringMap<Lambda> &Lambdas;
+  StringMap<SectionLambda> &SectionLambdas;
+  DenseMap<char, std::string> &Escapes;
+  Type Ty;
+  size_t Indentation = 0;
+  std::string RawBody;
+  std::string Body;
+  ASTNode *Parent;
+  // TODO: switch implementation to SmallVector<T>
+  std::vector<AstPtr> Children;
+  const Accessor AccessorValue;
+  const llvm::json::Value *ParentContext;
+};
+
+// A wrapper for arena allocator for ASTNodes
+AstPtr createRootNode(llvm::StringMap<AstPtr> &Partials,
+                      llvm::StringMap<Lambda> &Lambdas,
+                      llvm::StringMap<SectionLambda> &SectionLambdas,
+                      llvm::DenseMap<char, std::string> &Escapes) {
+  return std::make_unique<ASTNode>(Partials, Lambdas, SectionLambdas, Escapes);
+}
+
+AstPtr createNode(ASTNode::Type T, Accessor A, ASTNode *Parent,
+                  llvm::StringMap<AstPtr> &Partials,
+                  llvm::StringMap<Lambda> &Lambdas,
+                  llvm::StringMap<SectionLambda> &SectionLambdas,
+                  llvm::DenseMap<char, std::string> &Escapes) {
+  return std::make_unique<ASTNode>(T, std::move(A), Parent, Partials, Lambdas,
+                                   SectionLambdas, Escapes);
+}
+
+AstPtr createTextNode(std::string Body, ASTNode *Parent,
+                      llvm::StringMap<AstPtr> &Partials,
+                      llvm::StringMap<Lambda> &Lambdas,
+                      llvm::StringMap<SectionLambda> &SectionLambdas,
+                      llvm::DenseMap<char, std::string> &Escapes) {
+  return std::make_unique<ASTNode>(std::move(Body), Parent, Partials, Lambdas,
+                                   SectionLambdas, Escapes);
+}
+
+// Function to check if there is meaningful text behind.
+// We determine if a token has meaningful text behind
+// if the right of previous token contains anything that is
+// not a newline.
+// For example:
+//  "Stuff {{#Section}}" (returns true)
+//   vs
+//  "{{#Section}} \n" (returns false)
+// We make an exception for when previous token is empty
+// and the current token is the second token.
+// For example:
+//  "{{#Section}}"
+bool hasTextBehind(size_t Idx, const ArrayRef<Token> &Tokens) {
+  if (Idx == 0)
+    return true;
+
+  size_t PrevIdx = Idx - 1;
+  if (Tokens[PrevIdx].getType() != Token::Type::Text)
+    return true;
+
+  const Token &PrevToken = Tokens[PrevIdx];
+  StringRef TokenBody = StringRef(PrevToken.RawBody).rtrim(" \r\t\v");
+  return !TokenBody.ends_with("\n") && !(TokenBody.empty() && Idx == 1);
+}
+
+// Function to check if there's no meaningful text ahead.
+// We determine if a token has text ahead if the left of previous
+// token does not start with a newline.
+bool hasTextAhead(size_t Idx, const ArrayRef<Token> &Tokens) {
+  if (Idx >= Tokens.size() - 1)
+    return true;
+
+  size_t NextIdx = Idx + 1;
+  if (Tokens[NextIdx].getType() != Token::Type::Text)
+    return true;
+
+  const Token &NextToken = Tokens[NextIdx];
+  StringRef TokenBody = StringRef(NextToken.RawBody).ltrim(" ");
+  return !TokenBody.starts_with("\r\n") && !TokenBody.starts_with("\n");
+}
+
+bool requiresCleanUp(Token::Type T) {
+  // We must clean up all the tokens that could contain child nodes.
+  return T == Token::Type::SectionOpen || T == Token::Type::InvertSectionOpen ||
+         T == Token::Type::SectionClose || T == Token::Type::Comment ||
+         T == Token::Type::Partial;
+}
+
+// Adjust next token body if there is no text ahead.
+// For example:
+// The template string
+//  "{{! Comment }} \nLine 2"
+// would be considered as no text ahead and should be rendered as
+//  " Line 2"
+void stripTokenAhead(SmallVectorImpl<Token> &Tokens, size_t Idx) {
+  Token &NextToken = Tokens[Idx + 1];
+  StringRef NextTokenBody = NextToken.TokenBody;
+  // Cut off the leading newline which could be \n or \r\n.
+  if (NextTokenBody.starts_with("\r\n"))
+    NextToken.TokenBody = NextTokenBody.substr(2).str();
+  else if (NextTokenBody.starts_with("\n"))
+    NextToken.TokenBody = NextTokenBody.substr(1).str();
+}
+
+// Adjust previous token body if there no text behind.
+// For example:
+//  The template string
+//  " \t{{#section}}A{{/section}}"
+// would be considered as having no text ahead and would be render as
+//  "A"
+// The exception for this is partial tag which requires us to
+// keep track of the indentation once it's rendered.
+void stripTokenBefore(SmallVectorImpl<Token> &Tokens, size_t Idx,
+                      Token &CurrentToken, Token::Type CurrentType) {
+  Token &PrevToken = Tokens[Idx - 1];
+  StringRef PrevTokenBody = PrevToken.TokenBody;
+  StringRef Unindented = PrevTokenBody.rtrim(" \r\t\v");
+  size_t Indentation = PrevTokenBody.size() - Unindented.size();
+  if (CurrentType != Token::Type::Partial)
+    PrevToken.TokenBody = Unindented.str();
+  CurrentToken.setIndentation(Indentation);
+}
+
+// Simple tokenizer that splits the template into tokens.
+// The mustache spec allows {{{ }}} to unescape variables,
+// but we don't support that here. An unescape variable
+// is represented only by {{& variable}}.
+SmallVector<Token> tokenize(StringRef Template) {
+  SmallVector<Token> Tokens;
+  StringLiteral Open("{{");
+  StringLiteral Close("}}");
+  size_t Start = 0;
+  size_t DelimiterStart = Template.find(Open);
+  if (DelimiterStart == StringRef::npos) {
+    Tokens.emplace_back(Template.str());
+    return Tokens;
+  }
+  while (DelimiterStart != StringRef::npos) {
+    if (DelimiterStart != Start)
+      Tokens.emplace_back(Template.substr(Start, DelimiterStart - Start).str());
+    size_t DelimiterEnd = Template.find(Close, DelimiterStart);
+    if (DelimiterEnd == StringRef::npos)
+      break;
+
+    // Extract the Interpolated variable without delimiters.
+    size_t InterpolatedStart = DelimiterStart + Open.size();
+    size_t InterpolatedEnd = DelimiterEnd - DelimiterStart - Close.size();
+    std::string Interpolated =
+        Template.substr(InterpolatedStart, InterpolatedEnd).str();
+    std::string RawBody = Open.str() + Interpolated + Close.str();
+    Tokens.emplace_back(RawBody, Interpolated, Interpolated[0]);
+    Start = DelimiterEnd + Close.size();
+    DelimiterStart = Template.find(Open, Start);
+  }
+
+  if (Start < Template.size())
+    Tokens.emplace_back(Template.substr(Start).str());
+
+  // Fix up white spaces for:
+  //   - open sections
+  //   - inverted sections
+  //   - close sections
+  //   - comments
+  //
+  // This loop attempts to find standalone tokens and tries to trim out
+  // the surrounding whitespace.
+  // For example:
+  // if you have the template string
+  //  {{#section}} \n Example \n{{/section}}
+  // The output should would be
+  // For example:
+  //  \n Example \n
+  size_t LastIdx = Tokens.size() - 1;
+  for (size_t Idx = 0, End = Tokens.size(); Idx < End; ++Idx) {
+    Token &CurrentToken = Tokens[Idx];
+    Token::Type CurrentType = CurrentToken.getType();
+    // Check if token type requires cleanup.
+    bool RequiresCleanUp = requiresCleanUp(CurrentType);
+
+    if (!RequiresCleanUp)
+      continue;
+
+    // We adjust the token body if there's no text behind or ahead.
+    // A token is considered to have no text ahead if the right of the previous
+    // token is a newline followed by spaces.
+    // A token is considered to have no text behind if the left of the next
+    // token is spaces followed by a newline.
+    // eg.
+    //  "Line 1\n {{#section}} \n Line 2 \n {{/section}} \n Line 3"
+    bool HasTextBehind = hasTextBehind(Idx, Tokens);
+    bool HasTextAhead = hasTextAhead(Idx, Tokens);
+
+    if ((!HasTextAhead && !HasTextBehind) || (!HasTextAhead && Idx == 0))
+      stripTokenAhead(Tokens, Idx);
+
+    if ((!HasTextBehind && !HasTextAhead) || (!HasTextBehind && Idx == LastIdx))
+      stripTokenBefore(Tokens, Idx, CurrentToken, CurrentType);
+  }
+  return Tokens;
+}
+
+// Custom stream to escape strings.
+class EscapeStringStream : public raw_ostream {
+public:
+  explicit EscapeStringStream(llvm::raw_ostream &WrappedStream,
+                              DenseMap<char, std::string> &Escape)
+      : Escape(Escape), WrappedStream(WrappedStream) {
+    SetUnbuffered();
+  }
+
+protected:
+  void write_impl(const char *Ptr, size_t Size) override {
+    llvm::StringRef Data(Ptr, Size);
+    for (char C : Data) {
+      auto It = Escape.find(C);
+      if (It != Escape.end())
+        WrappedStream << It->getSecond();
+      else
+        WrappedStream << C;
+    }
+  }
+
+  uint64_t current_pos() const override { return WrappedStream.tell(); }
+
+private:
+  DenseMap<char, std::string> &Escape;
+  llvm::raw_ostream &WrappedStream;
+};
+
+// Custom stream to add indentation used to for rendering partials.
+class AddIndentationStringStream : public raw_ostream {
+public:
+  explicit AddIndentationStringStream(llvm::raw_ostream &WrappedStream,
+                                      size_t Indentation)
+      : Indentation(Indentation), WrappedStream(WrappedStream) {
+    SetUnbuffered();
+  }
+
+protected:
+  void write_impl(const char *Ptr, size_t Size) override {
+    llvm::StringRef Data(Ptr, Size);
+    SmallString<0> Indent;
+    Indent.resize(Indentation, ' ');
+    for (char C : Data) {
+      WrappedStream << C;
+      if (C == '\n')
+        WrappedStream << Indent;
+    }
+  }
+
+  uint64_t current_pos() const override { return WrappedStream.tell(); }
+
+private:
+  size_t Indentation;
+  llvm::raw_ostream &WrappedStream;
+};
+
+class Parser {
+public:
+  Parser(StringRef TemplateStr) : TemplateStr(TemplateStr) {}
+
+  AstPtr parse(llvm::StringMap<AstPtr> &Partials,
+               llvm::StringMap<Lambda> &Lambdas,
+               llvm::StringMap<SectionLambda> &SectionLambdas,
+               llvm::DenseMap<char, std::string> &Escapes);
+
+private:
+  void parseMustache(ASTNode *Parent, llvm::StringMap<AstPtr> &Partials,
+                     llvm::StringMap<Lambda> &Lambdas,
+                     llvm::StringMap<SectionLambda> &SectionLambdas,
+                 ...
[truncated]

The last version of this patch had memory leaks due to using the
BumpPtrAllocator for data types that required destructors to run to
release heap memory (e.g. via std::vector and std::string). This version
avoids that by using smart pointers, and dropping support for
BumpPtrAllocator.

We should refactor this code to use the BumpPtrAllocator again, but that
can be addressed in future patches, since those are more invasive
changes that need to refactor many of the core data types to avoid
owning allocations.

Adds Support for the Mustache Templating Language. See specs here:
https://mustache.github.io/mustache.5.html This patch implements
support+tests for majority of the features of the language including:

    - Variables
    - Comments
    - Lambdas
    - Sections

This meant as a library to support places where we have to generate
HTML, such as in clang-doc.

Co-authored-by: Peter Chou <peter.chou@mail.utoronto.ca>
@ilovepi ilovepi force-pushed the users/ilovepi/reland-mustache branch from f721cf3 to 29e9450 Compare March 24, 2025 17:26
@ilovepi
Copy link
Contributor Author

ilovepi commented Mar 24, 2025

@PeterChou1 I'd like to land this soon. Can you take a look at the reland? All the ASan failures should be addressed. Also, once this lands I'd like to land the clang-doc support soon after, so I'd appreciate it if we could get those ready for a speedy review.

Copy link
Contributor Author

ilovepi commented Mar 25, 2025

Merge activity

  • Mar 24, 8:22 PM EDT: A user started a stack merge that includes this pull request via Graphite.
  • Mar 24, 8:23 PM EDT: A user merged this pull request with Graphite.

@ilovepi ilovepi merged commit ece59a8 into main Mar 25, 2025
11 checks passed
@ilovepi ilovepi deleted the users/ilovepi/reland-mustache branch March 25, 2025 00:23
@llvm-ci
Copy link
Collaborator

llvm-ci commented Mar 25, 2025

LLVM Buildbot has detected a new failure on builder lldb-x86_64-debian running on lldb-x86_64-debian while building llvm at step 6 "test".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/162/builds/18805

Here is the relevant piece of the build log for the reference
Step 6 (test) failure: build (failure)
...
XFAIL: lldb-api :: functionalities/thread/break_after_join/TestBreakAfterJoin.py (601 of 2801)
PASS: lldb-api :: functionalities/thread/concurrent_events/TestConcurrentDelayWatchBreak.py (602 of 2801)
PASS: lldb-api :: functionalities/thread/concurrent_events/TestConcurrentDelaySignalBreak.py (603 of 2801)
PASS: lldb-api :: functionalities/thread/concurrent_events/TestConcurrentDelayedCrashWithBreakpointSignal.py (604 of 2801)
PASS: lldb-api :: functionalities/thread/concurrent_events/TestConcurrentDelaySignalWatch.py (605 of 2801)
PASS: lldb-api :: functionalities/inferior-assert/TestInferiorAssert.py (606 of 2801)
PASS: lldb-api :: functionalities/step-avoids-no-debug/TestStepNoDebug.py (607 of 2801)
PASS: lldb-api :: functionalities/thread/concurrent_events/TestConcurrentSignalDelayBreak.py (608 of 2801)
PASS: lldb-api :: functionalities/thread/concurrent_events/TestConcurrentSignalDelayWatch.py (609 of 2801)
PASS: lldb-api :: functionalities/thread/concurrent_events/TestConcurrentSignalWatchBreak.py (610 of 2801)
FAIL: lldb-api :: functionalities/thread/concurrent_events/TestConcurrentDelayedCrashWithBreakpointWatchpoint.py (611 of 2801)
******************** TEST 'lldb-api :: functionalities/thread/concurrent_events/TestConcurrentDelayedCrashWithBreakpointWatchpoint.py' FAILED ********************
Script:
--
/usr/bin/python3 /home/worker/2.0.1/lldb-x86_64-debian/llvm-project/lldb/test/API/dotest.py -u CXXFLAGS -u CFLAGS --env LLVM_LIBS_DIR=/home/worker/2.0.1/lldb-x86_64-debian/build/./lib --env LLVM_INCLUDE_DIR=/home/worker/2.0.1/lldb-x86_64-debian/build/include --env LLVM_TOOLS_DIR=/home/worker/2.0.1/lldb-x86_64-debian/build/./bin --arch x86_64 --build-dir /home/worker/2.0.1/lldb-x86_64-debian/build/lldb-test-build.noindex --lldb-module-cache-dir /home/worker/2.0.1/lldb-x86_64-debian/build/lldb-test-build.noindex/module-cache-lldb/lldb-api --clang-module-cache-dir /home/worker/2.0.1/lldb-x86_64-debian/build/lldb-test-build.noindex/module-cache-clang/lldb-api --executable /home/worker/2.0.1/lldb-x86_64-debian/build/./bin/lldb --compiler /home/worker/2.0.1/lldb-x86_64-debian/build/./bin/clang --dsymutil /home/worker/2.0.1/lldb-x86_64-debian/build/./bin/dsymutil --make /usr/bin/gmake --llvm-tools-dir /home/worker/2.0.1/lldb-x86_64-debian/build/./bin --lldb-obj-root /home/worker/2.0.1/lldb-x86_64-debian/build/tools/lldb --lldb-libs-dir /home/worker/2.0.1/lldb-x86_64-debian/build/./lib -t /home/worker/2.0.1/lldb-x86_64-debian/llvm-project/lldb/test/API/functionalities/thread/concurrent_events -p TestConcurrentDelayedCrashWithBreakpointWatchpoint.py
--
Exit Code: -11

Command Output (stdout):
--
lldb version 21.0.0git (https://github.com/llvm/llvm-project.git revision ece59a8cb9c82227ccd304b1cd26a2c216ddb13e)
  clang revision ece59a8cb9c82227ccd304b1cd26a2c216ddb13e
  llvm revision ece59a8cb9c82227ccd304b1cd26a2c216ddb13e

Watchpoint 1 hit:
old value: 0
new value: 1
Skipping the following test categories: ['libc++', 'dsym', 'gmodules', 'debugserver', 'objc']

--
Command Output (stderr):
--
Change dir to: /home/worker/2.0.1/lldb-x86_64-debian/llvm-project/lldb/test/API/functionalities/thread/concurrent_events
runCmd: settings clear -all

output: 

runCmd: settings set symbols.enable-external-lookup false

output: 

runCmd: settings set target.inherit-tcc true

output: 

runCmd: settings set target.disable-aslr false

output: 


@llvm-ci
Copy link
Collaborator

llvm-ci commented Mar 25, 2025

LLVM Buildbot has detected a new failure on builder lld-x86_64-win running on as-worker-93 while building llvm at step 7 "test-build-unified-tree-check-all".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/146/builds/2572

Here is the relevant piece of the build log for the reference
Step 7 (test-build-unified-tree-check-all) failure: test (failure)
******************** TEST 'LLVM-Unit :: Support/./SupportTests.exe/82/95' FAILED ********************
Script(shard):
--
GTEST_OUTPUT=json:C:\a\lld-x86_64-win\build\unittests\Support\.\SupportTests.exe-LLVM-Unit-2528-82-95.json GTEST_SHUFFLE=0 GTEST_TOTAL_SHARDS=95 GTEST_SHARD_INDEX=82 C:\a\lld-x86_64-win\build\unittests\Support\.\SupportTests.exe
--

Script:
--
C:\a\lld-x86_64-win\build\unittests\Support\.\SupportTests.exe --gtest_filter=ProgramEnvTest.CreateProcessLongPath
--
C:\a\lld-x86_64-win\llvm-project\llvm\unittests\Support\ProgramTest.cpp(160): error: Expected equality of these values:
  0
  RC
    Which is: -2

C:\a\lld-x86_64-win\llvm-project\llvm\unittests\Support\ProgramTest.cpp(163): error: fs::remove(Twine(LongPath)): did not return errc::success.
error number: 13
error message: permission denied



C:\a\lld-x86_64-win\llvm-project\llvm\unittests\Support\ProgramTest.cpp:160
Expected equality of these values:
  0
  RC
    Which is: -2

C:\a\lld-x86_64-win\llvm-project\llvm\unittests\Support\ProgramTest.cpp:163
fs::remove(Twine(LongPath)): did not return errc::success.
error number: 13
error message: permission denied




********************


Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants