Skip to content

[HLSL] Constant buffers codegen #124886

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 20 commits into from
Feb 20, 2025
Merged

[HLSL] Constant buffers codegen #124886

merged 20 commits into from
Feb 20, 2025

Conversation

hekota
Copy link
Member

@hekota hekota commented Jan 29, 2025

Translates cbuffer declaration blocks to target("dx.CBuffer") type. Creates global variables in hlsl_constant address space for all cbuffer constant and adds metadata describing which global constant belongs to which constant buffer. Also adds metadata describing the layout of the constant buffer and embedded structs.

These globals are temporary and will be removed in upcoming pass HLSLConstantAccess that will translate load instructions in the hlsl_constant address space to constant buffer load intrinsics calls off a CBV handle (#124630, #112992).

See Constant buffer design doc for more details.

Fixes #113514, #106596

- create structs with public fields instead of classes with private fields
- add Packed attribute to prevent struct padding
- use __cblayout_ prefix in name
- filter out arrays of resources (bug fix)
- don't create implicit initializer for constant buffer decls
- update tests
@hekota hekota changed the base branch from main to users/hekota/pr-124840 January 29, 2025 06:19
@hekota hekota marked this pull request as ready for review January 29, 2025 07:10
@llvmbot llvmbot added clang Clang issues not falling into any other category clang:frontend Language frontend issues, e.g. anything involving "Sema" clang:codegen IR generation bugs: mangling, exceptions, etc. backend:DirectX HLSL HLSL Language Support labels Jan 29, 2025
@llvmbot
Copy link
Member

llvmbot commented Jan 29, 2025

@llvm/pr-subscribers-clang

@llvm/pr-subscribers-clang-codegen

Author: Helena Kotas (hekota)

Changes

Translates cbuffer declaration blocks to target("dx.CBuffer") type. Creates global variables in hlsl_constant address space for all cbuffer constant and adds metadata describing which global constant belongs to which constant buffer. Also adds metadata describing the layout of the constant buffer and embedded structs.

These globals are temporary and will be removed in upcoming pass HLSLConstantAccess that will translate load instructions in the hlsl_constant address space to constant buffer load intrinsics calls off a CBV handle (#124630, #112992).

See Constant buffer design doc for more details.

Fixes #113514


Patch is 48.54 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/124886.diff

15 Files Affected:

  • (modified) clang/include/clang/AST/Decl.h (+6)
  • (modified) clang/include/clang/AST/Type.h (+2-2)
  • (modified) clang/lib/AST/Decl.cpp (+16-1)
  • (modified) clang/lib/CodeGen/CGHLSLRuntime.cpp (+297-124)
  • (modified) clang/lib/CodeGen/CGHLSLRuntime.h (+19-13)
  • (modified) clang/lib/CodeGen/Targets/DirectX.cpp (+7-3)
  • (modified) clang/lib/Sema/SemaHLSL.cpp (+3)
  • (removed) clang/test/CodeGenHLSL/cbuf.hlsl (-33)
  • (removed) clang/test/CodeGenHLSL/cbuf_in_namespace.hlsl (-29)
  • (added) clang/test/CodeGenHLSL/cbuffer.hlsl (+199)
  • (added) clang/test/CodeGenHLSL/cbuffer_and_namespaces.hlsl (+63)
  • (added) clang/test/CodeGenHLSL/cbuffer_with_packoffset.hlsl (+40)
  • (added) clang/test/CodeGenHLSL/cbuffer_with_static_global_and_function.hlsl (+32)
  • (modified) clang/test/CodeGenHLSL/resource-bindings.hlsl (+4)
  • (removed) clang/test/CodeGenHLSL/static_global_and_function_in_cb.hlsl (-22)
diff --git a/clang/include/clang/AST/Decl.h b/clang/include/clang/AST/Decl.h
index 16403774e72b31..e1c7e3817699ce 100644
--- a/clang/include/clang/AST/Decl.h
+++ b/clang/include/clang/AST/Decl.h
@@ -5032,6 +5032,9 @@ class HLSLBufferDecl final : public NamedDecl, public DeclContext {
   SourceLocation KwLoc;
   /// IsCBuffer - Whether the buffer is a cbuffer (and not a tbuffer).
   bool IsCBuffer;
+  /// HasValidPackoffset - Whether the buffer has valid packoffset annotations
+  //                       on all declarations
+  bool HasPackoffset;
 
   HLSLBufferDecl(DeclContext *DC, bool CBuffer, SourceLocation KwLoc,
                  IdentifierInfo *ID, SourceLocation IDLoc,
@@ -5052,6 +5055,9 @@ class HLSLBufferDecl final : public NamedDecl, public DeclContext {
   SourceLocation getRBraceLoc() const { return RBraceLoc; }
   void setRBraceLoc(SourceLocation L) { RBraceLoc = L; }
   bool isCBuffer() const { return IsCBuffer; }
+  void setHasPackoffset(bool PO) { HasPackoffset = PO; }
+  bool hasPackoffset() const { return HasPackoffset; }
+  const CXXRecordDecl *getLayoutStruct() const;
 
   // Implement isa/cast/dyncast/etc.
   static bool classof(const Decl *D) { return classofKind(D->getKind()); }
diff --git a/clang/include/clang/AST/Type.h b/clang/include/clang/AST/Type.h
index 1d9743520654eb..c3ff7ebd88516c 100644
--- a/clang/include/clang/AST/Type.h
+++ b/clang/include/clang/AST/Type.h
@@ -6266,8 +6266,8 @@ class HLSLAttributedResourceType : public Type, public llvm::FoldingSetNode {
     LLVM_PREFERRED_TYPE(bool)
     uint8_t RawBuffer : 1;
 
-    Attributes(llvm::dxil::ResourceClass ResourceClass, bool IsROV,
-               bool RawBuffer)
+    Attributes(llvm::dxil::ResourceClass ResourceClass, bool IsROV = false,
+               bool RawBuffer = false)
         : ResourceClass(ResourceClass), IsROV(IsROV), RawBuffer(RawBuffer) {}
 
     Attributes() : Attributes(llvm::dxil::ResourceClass::UAV, false, false) {}
diff --git a/clang/lib/AST/Decl.cpp b/clang/lib/AST/Decl.cpp
index beb5fcaefac535..fa7d03354a9937 100644
--- a/clang/lib/AST/Decl.cpp
+++ b/clang/lib/AST/Decl.cpp
@@ -1747,6 +1747,10 @@ void NamedDecl::printNestedNameSpecifier(raw_ostream &OS,
       }
     }
 
+    // Suppress transparent contexts like export or HLSLBufferDecl context
+    if (Ctx->isTransparentContext())
+      continue;
+
     // Skip non-named contexts such as linkage specifications and ExportDecls.
     const NamedDecl *ND = dyn_cast<NamedDecl>(Ctx);
     if (!ND)
@@ -5713,7 +5717,7 @@ HLSLBufferDecl::HLSLBufferDecl(DeclContext *DC, bool CBuffer,
                                SourceLocation IDLoc, SourceLocation LBrace)
     : NamedDecl(Decl::Kind::HLSLBuffer, DC, IDLoc, DeclarationName(ID)),
       DeclContext(Decl::Kind::HLSLBuffer), LBraceLoc(LBrace), KwLoc(KwLoc),
-      IsCBuffer(CBuffer) {}
+      IsCBuffer(CBuffer), HasPackoffset(false) {}
 
 HLSLBufferDecl *HLSLBufferDecl::Create(ASTContext &C,
                                        DeclContext *LexicalParent, bool CBuffer,
@@ -5743,6 +5747,17 @@ HLSLBufferDecl *HLSLBufferDecl::CreateDeserialized(ASTContext &C,
                                     SourceLocation(), SourceLocation());
 }
 
+const CXXRecordDecl *HLSLBufferDecl::getLayoutStruct() const {
+  // Layout struct is the last decl in the HLSLBufferDecl.
+  if (CXXRecordDecl *RD = llvm::dyn_cast_or_null<CXXRecordDecl>(LastDecl)) {
+    assert(RD->getName().starts_with(
+               ("__cblayout_" + getIdentifier()->getName()).str()) &&
+           "expected buffer layout struct");
+    return RD;
+  }
+  llvm_unreachable("HLSL buffer is missing a layout struct");
+}
+
 //===----------------------------------------------------------------------===//
 // ImportDecl Implementation
 //===----------------------------------------------------------------------===//
diff --git a/clang/lib/CodeGen/CGHLSLRuntime.cpp b/clang/lib/CodeGen/CGHLSLRuntime.cpp
index 2ce54cc3c52efa..52247173b69901 100644
--- a/clang/lib/CodeGen/CGHLSLRuntime.cpp
+++ b/clang/lib/CodeGen/CGHLSLRuntime.cpp
@@ -16,15 +16,20 @@
 #include "CGDebugInfo.h"
 #include "CodeGenModule.h"
 #include "TargetInfo.h"
+#include "clang/AST/ASTContext.h"
 #include "clang/AST/Decl.h"
+#include "clang/AST/Type.h"
 #include "clang/Basic/TargetOptions.h"
+#include "llvm/IR/DerivedTypes.h"
 #include "llvm/IR/GlobalVariable.h"
 #include "llvm/IR/LLVMContext.h"
 #include "llvm/IR/Metadata.h"
 #include "llvm/IR/Module.h"
+#include "llvm/IR/Type.h"
 #include "llvm/IR/Value.h"
 #include "llvm/Support/Alignment.h"
 
+#include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/FormatVariadic.h"
 
 using namespace clang;
@@ -32,6 +37,9 @@ using namespace CodeGen;
 using namespace clang::hlsl;
 using namespace llvm;
 
+static void createResourceInitFn(CodeGenModule &CGM, llvm::GlobalVariable *GV,
+                                 unsigned Slot, unsigned Space);
+
 namespace {
 
 void addDxilValVersion(StringRef ValVersionStr, llvm::Module &M) {
@@ -54,54 +62,11 @@ void addDxilValVersion(StringRef ValVersionStr, llvm::Module &M) {
   auto *DXILValMD = M.getOrInsertNamedMetadata(DXILValKey);
   DXILValMD->addOperand(Val);
 }
+
 void addDisableOptimizations(llvm::Module &M) {
   StringRef Key = "dx.disable_optimizations";
   M.addModuleFlag(llvm::Module::ModFlagBehavior::Override, Key, 1);
 }
-// cbuffer will be translated into global variable in special address space.
-// If translate into C,
-// cbuffer A {
-//   float a;
-//   float b;
-// }
-// float foo() { return a + b; }
-//
-// will be translated into
-//
-// struct A {
-//   float a;
-//   float b;
-// } cbuffer_A __attribute__((address_space(4)));
-// float foo() { return cbuffer_A.a + cbuffer_A.b; }
-//
-// layoutBuffer will create the struct A type.
-// replaceBuffer will replace use of global variable a and b with cbuffer_A.a
-// and cbuffer_A.b.
-//
-void layoutBuffer(CGHLSLRuntime::Buffer &Buf, const DataLayout &DL) {
-  if (Buf.Constants.empty())
-    return;
-
-  std::vector<llvm::Type *> EltTys;
-  for (auto &Const : Buf.Constants) {
-    GlobalVariable *GV = Const.first;
-    Const.second = EltTys.size();
-    llvm::Type *Ty = GV->getValueType();
-    EltTys.emplace_back(Ty);
-  }
-  Buf.LayoutStruct = llvm::StructType::get(EltTys[0]->getContext(), EltTys);
-}
-
-GlobalVariable *replaceBuffer(CGHLSLRuntime::Buffer &Buf) {
-  // Create global variable for CB.
-  GlobalVariable *CBGV = new GlobalVariable(
-      Buf.LayoutStruct, /*isConstant*/ true,
-      GlobalValue::LinkageTypes::ExternalLinkage, nullptr,
-      llvm::formatv("{0}{1}", Buf.Name, Buf.IsCBuffer ? ".cb." : ".tb."),
-      GlobalValue::NotThreadLocal);
-
-  return CBGV;
-}
 
 } // namespace
 
@@ -119,48 +84,288 @@ llvm::Triple::ArchType CGHLSLRuntime::getArch() {
   return CGM.getTarget().getTriple().getArch();
 }
 
-void CGHLSLRuntime::addConstant(VarDecl *D, Buffer &CB) {
-  if (D->getStorageClass() == SC_Static) {
-    // For static inside cbuffer, take as global static.
-    // Don't add to cbuffer.
-    CGM.EmitGlobal(D);
-    return;
+// Returns true if the type is an HLSL resource class
+static bool isResourceRecordType(const clang::Type *Ty) {
+  return HLSLAttributedResourceType::findHandleTypeOnResource(Ty) != nullptr;
+}
+
+// Returns true if the type is an HLSL resource class or an array of them
+static bool isResourceRecordTypeOrArrayOf(const clang::Type *Ty) {
+  while (const ConstantArrayType *CAT = dyn_cast<ConstantArrayType>(Ty))
+    Ty = CAT->getArrayElementTypeNoTypeQual();
+  return isResourceRecordType(Ty);
+}
+
+static ConstantAsMetadata *getConstIntMetadata(LLVMContext &Ctx, uint32_t value,
+                                               bool isSigned = false) {
+  return ConstantAsMetadata::get(
+      ConstantInt::get(Ctx, llvm::APInt(32, value, isSigned)));
+}
+
+static unsigned getScalarOrVectorSize(llvm::Type *Ty) {
+  assert(Ty->isVectorTy() || Ty->isIntegerTy() || Ty->isFloatingPointTy());
+  if (Ty->isVectorTy()) {
+    llvm::FixedVectorType *FVT = cast<llvm::FixedVectorType>(Ty);
+    return FVT->getNumElements() *
+           (FVT->getElementType()->getScalarSizeInBits() / 8);
+  }
+  return Ty->getScalarSizeInBits() / 8;
+}
+
+// Returns size of a struct in constant buffer layout. The sizes are cached
+// in StructSizesForBuffer map. The map is also an indicator if a layout
+// metadata for this struct has been added to the module.
+// If the struct type is not in the map, this method will calculate the struct
+// layout, add a metadata node describing it to the module, and add the struct
+// size to the map.
+size_t
+CGHLSLRuntime::getOrCalculateStructSizeForBuffer(llvm::StructType *StructTy) {
+  // check if we already have a side for this struct
+  auto SizeIt = StructSizesForBuffer.find(StructTy);
+  if (SizeIt != StructSizesForBuffer.end())
+    return SizeIt->getSecond();
+
+  // if not, calculate the struct layout and create a metadata node
+  LLVMContext &Ctx = CGM.getLLVMContext();
+  SmallVector<llvm::Metadata *> LayoutItems;
+
+  // start metadata list with a struct name and reserve one slot for its size
+  LayoutItems.push_back(MDString::get(Ctx, StructTy->getName()));
+  LayoutItems.push_back(nullptr);
+
+  // add element offsets
+  size_t StructSize = 0;
+  for (llvm::Type *ElTy : StructTy->elements()) {
+    size_t Offset = calculateBufferElementOffset(ElTy, &StructSize);
+    LayoutItems.push_back(getConstIntMetadata(CGM.getLLVMContext(), Offset));
+  }
+  // set the size of the buffer to the reserved slot
+  LayoutItems[1] = getConstIntMetadata(Ctx, StructSize);
+
+  // add the struct layout to metadata
+  CGM.getModule()
+      .getOrInsertNamedMetadata("hlsl.cblayouts")
+      ->addOperand(MDNode::get(CGM.getLLVMContext(), LayoutItems));
+
+  // add struct size to list and return it
+  StructSizesForBuffer[StructTy] = StructSize;
+  return StructSize;
+}
+
+// Calculates offset of a single element in constant buffer layout.
+// The provided LayoutEndOffset marks the end of the layout so far (end offset
+// of the buffer or struct). After the element offset calculations are done it
+// will be updated the new end of layout value.
+// If the PackoffsetAttrs is not nullptr the offset will be based on the
+// packoffset annotation.
+size_t CGHLSLRuntime::calculateBufferElementOffset(
+    llvm::Type *LayoutTy, size_t *LayoutEndOffset,
+    HLSLPackOffsetAttr *PackoffsetAttr) {
+
+  size_t ElemOffset = 0;
+  size_t ElemSize = 0;
+  size_t ArrayCount = 1;
+  size_t ArrayStride = 0;
+  size_t EndOffset = *LayoutEndOffset;
+  size_t NextRowOffset = llvm::alignTo(EndOffset, 16U);
+
+  if (LayoutTy->isArrayTy()) {
+    llvm::Type *Ty = LayoutTy;
+    while (Ty->isArrayTy()) {
+      ArrayCount *= Ty->getArrayNumElements();
+      Ty = Ty->getArrayElementType();
+    }
+    ElemSize =
+        Ty->isStructTy()
+            ? getOrCalculateStructSizeForBuffer(cast<llvm::StructType>(Ty))
+            : getScalarOrVectorSize(Ty);
+    ArrayStride = llvm::alignTo(ElemSize, 16U);
+    ElemOffset =
+        PackoffsetAttr ? PackoffsetAttr->getOffsetInBytes() : NextRowOffset;
+
+  } else if (LayoutTy->isStructTy()) {
+    ElemOffset =
+        PackoffsetAttr ? PackoffsetAttr->getOffsetInBytes() : NextRowOffset;
+    ElemSize =
+        getOrCalculateStructSizeForBuffer(cast<llvm::StructType>(LayoutTy));
+
+  } else {
+    size_t Align = 0;
+    if (LayoutTy->isVectorTy()) {
+      llvm::FixedVectorType *FVT = cast<llvm::FixedVectorType>(LayoutTy);
+      size_t SubElemSize = FVT->getElementType()->getScalarSizeInBits() / 8;
+      ElemSize = FVT->getNumElements() * SubElemSize;
+      Align = SubElemSize;
+    } else {
+      assert(LayoutTy->isIntegerTy() || LayoutTy->isFloatingPointTy());
+      ElemSize = LayoutTy->getScalarSizeInBits() / 8;
+      Align = ElemSize;
+    }
+    if (PackoffsetAttr) {
+      ElemOffset = PackoffsetAttr->getOffsetInBytes();
+    } else {
+      ElemOffset = llvm::alignTo(EndOffset, Align);
+      // if the element does not fit, move it to the next row
+      if (ElemOffset + ElemSize > NextRowOffset)
+        ElemOffset = NextRowOffset;
+    }
   }
 
-  auto *GV = cast<GlobalVariable>(CGM.GetAddrOfGlobalVar(D));
-  GV->setExternallyInitialized(true);
-  // Add debug info for constVal.
-  if (CGDebugInfo *DI = CGM.getModuleDebugInfo())
-    if (CGM.getCodeGenOpts().getDebugInfo() >=
-        codegenoptions::DebugInfoKind::LimitedDebugInfo)
-      DI->EmitGlobalVariable(cast<GlobalVariable>(GV), D);
-
-  // FIXME: support packoffset.
-  // See https://github.com/llvm/llvm-project/issues/57914.
-  uint32_t Offset = 0;
-  bool HasUserOffset = false;
-
-  unsigned LowerBound = HasUserOffset ? Offset : UINT_MAX;
-  CB.Constants.emplace_back(std::make_pair(GV, LowerBound));
+  // Update end offset of the buffer/struct layout; do not update it if
+  // the provided EndOffset is already bigger than the new one value
+  // (which may happen with packoffset annotations)
+  unsigned NewEndOffset =
+      ElemOffset + (ArrayCount - 1) * ArrayStride + ElemSize;
+  *LayoutEndOffset = std::max<size_t>(EndOffset, NewEndOffset);
+
+  return ElemOffset;
 }
 
-void CGHLSLRuntime::addBufferDecls(const DeclContext *DC, Buffer &CB) {
-  for (Decl *it : DC->decls()) {
-    if (auto *ConstDecl = dyn_cast<VarDecl>(it)) {
-      addConstant(ConstDecl, CB);
-    } else if (isa<CXXRecordDecl, EmptyDecl>(it)) {
+// Emits constant global variables for buffer declarations, creates metadata
+// linking the constant globals with the buffer. Also calculates the buffer
+// layout and creates metadata node describing it.
+void CGHLSLRuntime::emitBufferGlobalsAndMetadata(const HLSLBufferDecl *BufDecl,
+                                                 llvm::GlobalVariable *BufGV) {
+  LLVMContext &Ctx = CGM.getLLVMContext();
+  llvm::StructType *LayoutStruct = cast<llvm::StructType>(
+      cast<llvm::TargetExtType>(BufGV->getValueType())->getTypeParameter(0));
+
+  // Start metadata list associating the buffer global variable with its
+  // constatns
+  SmallVector<llvm::Metadata *> BufGlobals;
+  BufGlobals.push_back(ValueAsMetadata::get(BufGV));
+
+  // Start layout metadata list with a struct name and reserve one slot for
+  // the buffer size
+  SmallVector<llvm::Metadata *> LayoutItems;
+  LayoutItems.push_back(MDString::get(Ctx, LayoutStruct->getName()));
+  LayoutItems.push_back(nullptr);
+
+  size_t BufferSize = 0;
+  bool UsePackoffset = BufDecl->hasPackoffset();
+  const auto *ElemIt = LayoutStruct->element_begin();
+  for (Decl *D : BufDecl->decls()) {
+    if (isa<CXXRecordDecl, EmptyDecl>(D))
       // Nothing to do for this declaration.
-    } else if (isa<FunctionDecl>(it)) {
-      // A function within an cbuffer is effectively a top-level function,
-      // as it only refers to globally scoped declarations.
-      CGM.EmitTopLevelDecl(it);
+      continue;
+    if (isa<FunctionDecl>(D)) {
+      // A function within an cbuffer is effectively a top-level function.
+      CGM.EmitTopLevelDecl(D);
+      continue;
+    }
+    VarDecl *VD = dyn_cast<VarDecl>(D);
+    if (!VD)
+      continue;
+
+    QualType VDTy = VD->getType();
+    if (VDTy.getAddressSpace() != LangAS::hlsl_constant) {
+      if (VD->getStorageClass() == SC_Static ||
+          VDTy.getAddressSpace() == LangAS::hlsl_groupshared ||
+          isResourceRecordTypeOrArrayOf(VDTy.getTypePtr())) {
+        // Emit static and groupshared variables and resource classes inside
+        // cbuffer as regular globals
+        CGM.EmitGlobal(VD);
+      }
+      // Anything else that is not in the hlsl_constant address space must be
+      // an empty struct or a zero-sized array and can be ignored
+      continue;
     }
+
+    assert(ElemIt != LayoutStruct->element_end() &&
+           "number of elements in layout struct does not match");
+    llvm::Type *LayoutType = *ElemIt++;
+
+    // Make sure the type of the VarDecl type matches the type of the layout
+    // struct element, or that it is a layout struct with the same name
+    assert((CGM.getTypes().ConvertTypeForMem(VDTy) == LayoutType ||
+            (LayoutType->isStructTy() &&
+             cast<llvm::StructType>(LayoutType)
+                 ->getName()
+                 .starts_with(("struct.__cblayout_" +
+                               VDTy->getAsCXXRecordDecl()->getName())
+                                  .str()))) &&
+           "layout type does not match the converted element type");
+
+    // there might be resources inside the used defined structs
+    if (VDTy->isStructureType() && VDTy->isHLSLIntangibleType())
+      // FIXME: handle resources in cbuffer structs
+      llvm_unreachable("resources in cbuffer are not supported yet");
+
+    // create global variable for the constant and to metadata list
+    GlobalVariable *ElemGV =
+        cast<GlobalVariable>(CGM.GetAddrOfGlobalVar(VD, LayoutType));
+    BufGlobals.push_back(ValueAsMetadata::get(ElemGV));
+
+    // get offset of the global and and to metadata list
+    assert(((UsePackoffset && VD->hasAttr<HLSLPackOffsetAttr>()) ||
+            !UsePackoffset) &&
+           "expected packoffset attribute on every declaration");
+    size_t Offset = calculateBufferElementOffset(
+        LayoutType, &BufferSize,
+        UsePackoffset ? VD->getAttr<HLSLPackOffsetAttr>() : nullptr);
+    LayoutItems.push_back(getConstIntMetadata(Ctx, Offset));
   }
+  assert(ElemIt == LayoutStruct->element_end() &&
+         "number of elements in layout struct does not match");
+  // set the size of the buffer
+  LayoutItems[1] = getConstIntMetadata(Ctx, BufferSize);
+
+  // add buffer metadata to the module
+  CGM.getModule()
+      .getOrInsertNamedMetadata("hlsl.cbs")
+      ->addOperand(MDNode::get(Ctx, BufGlobals));
+
+  CGM.getModule()
+      .getOrInsertNamedMetadata("hlsl.cblayouts")
+      ->addOperand(MDNode::get(Ctx, LayoutItems));
 }
 
-void CGHLSLRuntime::addBuffer(const HLSLBufferDecl *D) {
-  Buffers.emplace_back(Buffer(D));
-  addBufferDecls(D, Buffers.back());
+// Creates resource handle type for the HLSL buffer declaration
+static const clang::HLSLAttributedResourceType *
+createBufferHandleType(const HLSLBufferDecl *BufDecl) {
+  ASTContext &AST = BufDecl->getASTContext();
+  QualType QT = AST.getHLSLAttributedResourceType(
+      AST.HLSLResourceTy,
+      QualType(BufDecl->getLayoutStruct()->getTypeForDecl(), 0),
+      HLSLAttributedResourceType::Attributes(ResourceClass::CBuffer));
+  return cast<HLSLAttributedResourceType>(QT.getTypePtr());
+}
+
+// Codegen for HLSLBufferDecl
+void CGHLSLRuntime::addBuffer(const HLSLBufferDecl *BufDecl) {
+
+  assert(BufDecl->isCBuffer() && "tbuffer codegen is not supported yet");
+
+  // create resource handle type for the buffer
+  const clang::HLSLAttributedResourceType *ResHandleTy =
+      createBufferHandleType(BufDecl);
+
+  // empty constant buffer is ignored
+  if (ResHandleTy->getContainedType()->getAsCXXRecordDecl()->isEmpty())
+    return;
+
+  // create global variable for the constant buffer
+  llvm::Module &M = CGM.getModule();
+  llvm::TargetExtType *TargetTy =
+      cast<llvm::TargetExtType>(convertHLSLSpecificType(ResHandleTy));
+  llvm::GlobalVariable *BufGV =
+      new GlobalVariable(TargetTy, /*isConstant*/ true,
+                         GlobalValue::LinkageTypes::ExternalLinkage, nullptr,
+                         llvm::formatv("{0}{1}", BufDecl->getName(),
+                                       BufDecl->isCBuffer() ? ".cb" : ".tb"),
+                         GlobalValue::NotThreadLocal);
+  M.insertGlobalVariable(BufGV);
+
+  // Add globals for constant buffer elements and create metadata nodes
+  emitBufferGlobalsAndMetadata(BufDecl, BufGV);
+
+  // Resource initialization
+  const HLSLResourceBindingAttr *RBA =
+      BufDecl->getAttr<HLSLResourceBindingAttr>();
+  // FIXME: handle implicit binding if no binding attribute is found
+  if (RBA)
+    createResourceInitFn(CGM, BufGV, RBA->getSlotNumber(),
+                         RBA->getSpaceNumber());
 }
 
 void CGHLSLRuntime::finishCodeGen() {
@@ -173,28 +378,8 @@ void CGHLSLRuntime::finishCodeGen() {
   generateGlobalCtorDtorCalls();
   if (CGM.getCodeGenOpts().OptimizationLevel == 0)
     addDisableOptimizations(M);
-
-  const DataLayout &DL = M.getDataLayout();
-
-  for (auto &Buf : Buffers) {
-    layoutBuffer(Buf, DL);
-    GlobalVariable *GV = replaceBuffer(Buf);
-    M.insertGlobalVariable(GV);
-    llvm::hlsl::ResourceClass RC = Buf.IsCBuffer
-     ...
[truncated]

@llvmbot
Copy link
Member

llvmbot commented Jan 29, 2025

@llvm/pr-subscribers-hlsl

Author: Helena Kotas (hekota)

Changes

Translates cbuffer declaration blocks to target("dx.CBuffer") type. Creates global variables in hlsl_constant address space for all cbuffer constant and adds metadata describing which global constant belongs to which constant buffer. Also adds metadata describing the layout of the constant buffer and embedded structs.

These globals are temporary and will be removed in upcoming pass HLSLConstantAccess that will translate load instructions in the hlsl_constant address space to constant buffer load intrinsics calls off a CBV handle (#124630, #112992).

See Constant buffer design doc for more details.

Fixes #113514


Patch is 48.57 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/124886.diff

15 Files Affected:

  • (modified) clang/include/clang/AST/Decl.h (+6)
  • (modified) clang/include/clang/AST/Type.h (+2-2)
  • (modified) clang/lib/AST/Decl.cpp (+16-1)
  • (modified) clang/lib/CodeGen/CGHLSLRuntime.cpp (+297-124)
  • (modified) clang/lib/CodeGen/CGHLSLRuntime.h (+19-13)
  • (modified) clang/lib/CodeGen/Targets/DirectX.cpp (+7-3)
  • (modified) clang/lib/Sema/SemaHLSL.cpp (+3)
  • (removed) clang/test/CodeGenHLSL/cbuf.hlsl (-33)
  • (removed) clang/test/CodeGenHLSL/cbuf_in_namespace.hlsl (-29)
  • (added) clang/test/CodeGenHLSL/cbuffer.hlsl (+199)
  • (added) clang/test/CodeGenHLSL/cbuffer_and_namespaces.hlsl (+63)
  • (added) clang/test/CodeGenHLSL/cbuffer_with_packoffset.hlsl (+40)
  • (added) clang/test/CodeGenHLSL/cbuffer_with_static_global_and_function.hlsl (+32)
  • (modified) clang/test/CodeGenHLSL/resource-bindings.hlsl (+4)
  • (removed) clang/test/CodeGenHLSL/static_global_and_function_in_cb.hlsl (-22)
diff --git a/clang/include/clang/AST/Decl.h b/clang/include/clang/AST/Decl.h
index 16403774e72b31c..e1c7e3817699ce0 100644
--- a/clang/include/clang/AST/Decl.h
+++ b/clang/include/clang/AST/Decl.h
@@ -5032,6 +5032,9 @@ class HLSLBufferDecl final : public NamedDecl, public DeclContext {
   SourceLocation KwLoc;
   /// IsCBuffer - Whether the buffer is a cbuffer (and not a tbuffer).
   bool IsCBuffer;
+  /// HasValidPackoffset - Whether the buffer has valid packoffset annotations
+  //                       on all declarations
+  bool HasPackoffset;
 
   HLSLBufferDecl(DeclContext *DC, bool CBuffer, SourceLocation KwLoc,
                  IdentifierInfo *ID, SourceLocation IDLoc,
@@ -5052,6 +5055,9 @@ class HLSLBufferDecl final : public NamedDecl, public DeclContext {
   SourceLocation getRBraceLoc() const { return RBraceLoc; }
   void setRBraceLoc(SourceLocation L) { RBraceLoc = L; }
   bool isCBuffer() const { return IsCBuffer; }
+  void setHasPackoffset(bool PO) { HasPackoffset = PO; }
+  bool hasPackoffset() const { return HasPackoffset; }
+  const CXXRecordDecl *getLayoutStruct() const;
 
   // Implement isa/cast/dyncast/etc.
   static bool classof(const Decl *D) { return classofKind(D->getKind()); }
diff --git a/clang/include/clang/AST/Type.h b/clang/include/clang/AST/Type.h
index 1d9743520654ebd..c3ff7ebd88516c7 100644
--- a/clang/include/clang/AST/Type.h
+++ b/clang/include/clang/AST/Type.h
@@ -6266,8 +6266,8 @@ class HLSLAttributedResourceType : public Type, public llvm::FoldingSetNode {
     LLVM_PREFERRED_TYPE(bool)
     uint8_t RawBuffer : 1;
 
-    Attributes(llvm::dxil::ResourceClass ResourceClass, bool IsROV,
-               bool RawBuffer)
+    Attributes(llvm::dxil::ResourceClass ResourceClass, bool IsROV = false,
+               bool RawBuffer = false)
         : ResourceClass(ResourceClass), IsROV(IsROV), RawBuffer(RawBuffer) {}
 
     Attributes() : Attributes(llvm::dxil::ResourceClass::UAV, false, false) {}
diff --git a/clang/lib/AST/Decl.cpp b/clang/lib/AST/Decl.cpp
index beb5fcaefac5353..fa7d03354a99379 100644
--- a/clang/lib/AST/Decl.cpp
+++ b/clang/lib/AST/Decl.cpp
@@ -1747,6 +1747,10 @@ void NamedDecl::printNestedNameSpecifier(raw_ostream &OS,
       }
     }
 
+    // Suppress transparent contexts like export or HLSLBufferDecl context
+    if (Ctx->isTransparentContext())
+      continue;
+
     // Skip non-named contexts such as linkage specifications and ExportDecls.
     const NamedDecl *ND = dyn_cast<NamedDecl>(Ctx);
     if (!ND)
@@ -5713,7 +5717,7 @@ HLSLBufferDecl::HLSLBufferDecl(DeclContext *DC, bool CBuffer,
                                SourceLocation IDLoc, SourceLocation LBrace)
     : NamedDecl(Decl::Kind::HLSLBuffer, DC, IDLoc, DeclarationName(ID)),
       DeclContext(Decl::Kind::HLSLBuffer), LBraceLoc(LBrace), KwLoc(KwLoc),
-      IsCBuffer(CBuffer) {}
+      IsCBuffer(CBuffer), HasPackoffset(false) {}
 
 HLSLBufferDecl *HLSLBufferDecl::Create(ASTContext &C,
                                        DeclContext *LexicalParent, bool CBuffer,
@@ -5743,6 +5747,17 @@ HLSLBufferDecl *HLSLBufferDecl::CreateDeserialized(ASTContext &C,
                                     SourceLocation(), SourceLocation());
 }
 
+const CXXRecordDecl *HLSLBufferDecl::getLayoutStruct() const {
+  // Layout struct is the last decl in the HLSLBufferDecl.
+  if (CXXRecordDecl *RD = llvm::dyn_cast_or_null<CXXRecordDecl>(LastDecl)) {
+    assert(RD->getName().starts_with(
+               ("__cblayout_" + getIdentifier()->getName()).str()) &&
+           "expected buffer layout struct");
+    return RD;
+  }
+  llvm_unreachable("HLSL buffer is missing a layout struct");
+}
+
 //===----------------------------------------------------------------------===//
 // ImportDecl Implementation
 //===----------------------------------------------------------------------===//
diff --git a/clang/lib/CodeGen/CGHLSLRuntime.cpp b/clang/lib/CodeGen/CGHLSLRuntime.cpp
index 2ce54cc3c52efa0..52247173b69901b 100644
--- a/clang/lib/CodeGen/CGHLSLRuntime.cpp
+++ b/clang/lib/CodeGen/CGHLSLRuntime.cpp
@@ -16,15 +16,20 @@
 #include "CGDebugInfo.h"
 #include "CodeGenModule.h"
 #include "TargetInfo.h"
+#include "clang/AST/ASTContext.h"
 #include "clang/AST/Decl.h"
+#include "clang/AST/Type.h"
 #include "clang/Basic/TargetOptions.h"
+#include "llvm/IR/DerivedTypes.h"
 #include "llvm/IR/GlobalVariable.h"
 #include "llvm/IR/LLVMContext.h"
 #include "llvm/IR/Metadata.h"
 #include "llvm/IR/Module.h"
+#include "llvm/IR/Type.h"
 #include "llvm/IR/Value.h"
 #include "llvm/Support/Alignment.h"
 
+#include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/FormatVariadic.h"
 
 using namespace clang;
@@ -32,6 +37,9 @@ using namespace CodeGen;
 using namespace clang::hlsl;
 using namespace llvm;
 
+static void createResourceInitFn(CodeGenModule &CGM, llvm::GlobalVariable *GV,
+                                 unsigned Slot, unsigned Space);
+
 namespace {
 
 void addDxilValVersion(StringRef ValVersionStr, llvm::Module &M) {
@@ -54,54 +62,11 @@ void addDxilValVersion(StringRef ValVersionStr, llvm::Module &M) {
   auto *DXILValMD = M.getOrInsertNamedMetadata(DXILValKey);
   DXILValMD->addOperand(Val);
 }
+
 void addDisableOptimizations(llvm::Module &M) {
   StringRef Key = "dx.disable_optimizations";
   M.addModuleFlag(llvm::Module::ModFlagBehavior::Override, Key, 1);
 }
-// cbuffer will be translated into global variable in special address space.
-// If translate into C,
-// cbuffer A {
-//   float a;
-//   float b;
-// }
-// float foo() { return a + b; }
-//
-// will be translated into
-//
-// struct A {
-//   float a;
-//   float b;
-// } cbuffer_A __attribute__((address_space(4)));
-// float foo() { return cbuffer_A.a + cbuffer_A.b; }
-//
-// layoutBuffer will create the struct A type.
-// replaceBuffer will replace use of global variable a and b with cbuffer_A.a
-// and cbuffer_A.b.
-//
-void layoutBuffer(CGHLSLRuntime::Buffer &Buf, const DataLayout &DL) {
-  if (Buf.Constants.empty())
-    return;
-
-  std::vector<llvm::Type *> EltTys;
-  for (auto &Const : Buf.Constants) {
-    GlobalVariable *GV = Const.first;
-    Const.second = EltTys.size();
-    llvm::Type *Ty = GV->getValueType();
-    EltTys.emplace_back(Ty);
-  }
-  Buf.LayoutStruct = llvm::StructType::get(EltTys[0]->getContext(), EltTys);
-}
-
-GlobalVariable *replaceBuffer(CGHLSLRuntime::Buffer &Buf) {
-  // Create global variable for CB.
-  GlobalVariable *CBGV = new GlobalVariable(
-      Buf.LayoutStruct, /*isConstant*/ true,
-      GlobalValue::LinkageTypes::ExternalLinkage, nullptr,
-      llvm::formatv("{0}{1}", Buf.Name, Buf.IsCBuffer ? ".cb." : ".tb."),
-      GlobalValue::NotThreadLocal);
-
-  return CBGV;
-}
 
 } // namespace
 
@@ -119,48 +84,288 @@ llvm::Triple::ArchType CGHLSLRuntime::getArch() {
   return CGM.getTarget().getTriple().getArch();
 }
 
-void CGHLSLRuntime::addConstant(VarDecl *D, Buffer &CB) {
-  if (D->getStorageClass() == SC_Static) {
-    // For static inside cbuffer, take as global static.
-    // Don't add to cbuffer.
-    CGM.EmitGlobal(D);
-    return;
+// Returns true if the type is an HLSL resource class
+static bool isResourceRecordType(const clang::Type *Ty) {
+  return HLSLAttributedResourceType::findHandleTypeOnResource(Ty) != nullptr;
+}
+
+// Returns true if the type is an HLSL resource class or an array of them
+static bool isResourceRecordTypeOrArrayOf(const clang::Type *Ty) {
+  while (const ConstantArrayType *CAT = dyn_cast<ConstantArrayType>(Ty))
+    Ty = CAT->getArrayElementTypeNoTypeQual();
+  return isResourceRecordType(Ty);
+}
+
+static ConstantAsMetadata *getConstIntMetadata(LLVMContext &Ctx, uint32_t value,
+                                               bool isSigned = false) {
+  return ConstantAsMetadata::get(
+      ConstantInt::get(Ctx, llvm::APInt(32, value, isSigned)));
+}
+
+static unsigned getScalarOrVectorSize(llvm::Type *Ty) {
+  assert(Ty->isVectorTy() || Ty->isIntegerTy() || Ty->isFloatingPointTy());
+  if (Ty->isVectorTy()) {
+    llvm::FixedVectorType *FVT = cast<llvm::FixedVectorType>(Ty);
+    return FVT->getNumElements() *
+           (FVT->getElementType()->getScalarSizeInBits() / 8);
+  }
+  return Ty->getScalarSizeInBits() / 8;
+}
+
+// Returns size of a struct in constant buffer layout. The sizes are cached
+// in StructSizesForBuffer map. The map is also an indicator if a layout
+// metadata for this struct has been added to the module.
+// If the struct type is not in the map, this method will calculate the struct
+// layout, add a metadata node describing it to the module, and add the struct
+// size to the map.
+size_t
+CGHLSLRuntime::getOrCalculateStructSizeForBuffer(llvm::StructType *StructTy) {
+  // check if we already have a side for this struct
+  auto SizeIt = StructSizesForBuffer.find(StructTy);
+  if (SizeIt != StructSizesForBuffer.end())
+    return SizeIt->getSecond();
+
+  // if not, calculate the struct layout and create a metadata node
+  LLVMContext &Ctx = CGM.getLLVMContext();
+  SmallVector<llvm::Metadata *> LayoutItems;
+
+  // start metadata list with a struct name and reserve one slot for its size
+  LayoutItems.push_back(MDString::get(Ctx, StructTy->getName()));
+  LayoutItems.push_back(nullptr);
+
+  // add element offsets
+  size_t StructSize = 0;
+  for (llvm::Type *ElTy : StructTy->elements()) {
+    size_t Offset = calculateBufferElementOffset(ElTy, &StructSize);
+    LayoutItems.push_back(getConstIntMetadata(CGM.getLLVMContext(), Offset));
+  }
+  // set the size of the buffer to the reserved slot
+  LayoutItems[1] = getConstIntMetadata(Ctx, StructSize);
+
+  // add the struct layout to metadata
+  CGM.getModule()
+      .getOrInsertNamedMetadata("hlsl.cblayouts")
+      ->addOperand(MDNode::get(CGM.getLLVMContext(), LayoutItems));
+
+  // add struct size to list and return it
+  StructSizesForBuffer[StructTy] = StructSize;
+  return StructSize;
+}
+
+// Calculates offset of a single element in constant buffer layout.
+// The provided LayoutEndOffset marks the end of the layout so far (end offset
+// of the buffer or struct). After the element offset calculations are done it
+// will be updated the new end of layout value.
+// If the PackoffsetAttrs is not nullptr the offset will be based on the
+// packoffset annotation.
+size_t CGHLSLRuntime::calculateBufferElementOffset(
+    llvm::Type *LayoutTy, size_t *LayoutEndOffset,
+    HLSLPackOffsetAttr *PackoffsetAttr) {
+
+  size_t ElemOffset = 0;
+  size_t ElemSize = 0;
+  size_t ArrayCount = 1;
+  size_t ArrayStride = 0;
+  size_t EndOffset = *LayoutEndOffset;
+  size_t NextRowOffset = llvm::alignTo(EndOffset, 16U);
+
+  if (LayoutTy->isArrayTy()) {
+    llvm::Type *Ty = LayoutTy;
+    while (Ty->isArrayTy()) {
+      ArrayCount *= Ty->getArrayNumElements();
+      Ty = Ty->getArrayElementType();
+    }
+    ElemSize =
+        Ty->isStructTy()
+            ? getOrCalculateStructSizeForBuffer(cast<llvm::StructType>(Ty))
+            : getScalarOrVectorSize(Ty);
+    ArrayStride = llvm::alignTo(ElemSize, 16U);
+    ElemOffset =
+        PackoffsetAttr ? PackoffsetAttr->getOffsetInBytes() : NextRowOffset;
+
+  } else if (LayoutTy->isStructTy()) {
+    ElemOffset =
+        PackoffsetAttr ? PackoffsetAttr->getOffsetInBytes() : NextRowOffset;
+    ElemSize =
+        getOrCalculateStructSizeForBuffer(cast<llvm::StructType>(LayoutTy));
+
+  } else {
+    size_t Align = 0;
+    if (LayoutTy->isVectorTy()) {
+      llvm::FixedVectorType *FVT = cast<llvm::FixedVectorType>(LayoutTy);
+      size_t SubElemSize = FVT->getElementType()->getScalarSizeInBits() / 8;
+      ElemSize = FVT->getNumElements() * SubElemSize;
+      Align = SubElemSize;
+    } else {
+      assert(LayoutTy->isIntegerTy() || LayoutTy->isFloatingPointTy());
+      ElemSize = LayoutTy->getScalarSizeInBits() / 8;
+      Align = ElemSize;
+    }
+    if (PackoffsetAttr) {
+      ElemOffset = PackoffsetAttr->getOffsetInBytes();
+    } else {
+      ElemOffset = llvm::alignTo(EndOffset, Align);
+      // if the element does not fit, move it to the next row
+      if (ElemOffset + ElemSize > NextRowOffset)
+        ElemOffset = NextRowOffset;
+    }
   }
 
-  auto *GV = cast<GlobalVariable>(CGM.GetAddrOfGlobalVar(D));
-  GV->setExternallyInitialized(true);
-  // Add debug info for constVal.
-  if (CGDebugInfo *DI = CGM.getModuleDebugInfo())
-    if (CGM.getCodeGenOpts().getDebugInfo() >=
-        codegenoptions::DebugInfoKind::LimitedDebugInfo)
-      DI->EmitGlobalVariable(cast<GlobalVariable>(GV), D);
-
-  // FIXME: support packoffset.
-  // See https://github.com/llvm/llvm-project/issues/57914.
-  uint32_t Offset = 0;
-  bool HasUserOffset = false;
-
-  unsigned LowerBound = HasUserOffset ? Offset : UINT_MAX;
-  CB.Constants.emplace_back(std::make_pair(GV, LowerBound));
+  // Update end offset of the buffer/struct layout; do not update it if
+  // the provided EndOffset is already bigger than the new one value
+  // (which may happen with packoffset annotations)
+  unsigned NewEndOffset =
+      ElemOffset + (ArrayCount - 1) * ArrayStride + ElemSize;
+  *LayoutEndOffset = std::max<size_t>(EndOffset, NewEndOffset);
+
+  return ElemOffset;
 }
 
-void CGHLSLRuntime::addBufferDecls(const DeclContext *DC, Buffer &CB) {
-  for (Decl *it : DC->decls()) {
-    if (auto *ConstDecl = dyn_cast<VarDecl>(it)) {
-      addConstant(ConstDecl, CB);
-    } else if (isa<CXXRecordDecl, EmptyDecl>(it)) {
+// Emits constant global variables for buffer declarations, creates metadata
+// linking the constant globals with the buffer. Also calculates the buffer
+// layout and creates metadata node describing it.
+void CGHLSLRuntime::emitBufferGlobalsAndMetadata(const HLSLBufferDecl *BufDecl,
+                                                 llvm::GlobalVariable *BufGV) {
+  LLVMContext &Ctx = CGM.getLLVMContext();
+  llvm::StructType *LayoutStruct = cast<llvm::StructType>(
+      cast<llvm::TargetExtType>(BufGV->getValueType())->getTypeParameter(0));
+
+  // Start metadata list associating the buffer global variable with its
+  // constatns
+  SmallVector<llvm::Metadata *> BufGlobals;
+  BufGlobals.push_back(ValueAsMetadata::get(BufGV));
+
+  // Start layout metadata list with a struct name and reserve one slot for
+  // the buffer size
+  SmallVector<llvm::Metadata *> LayoutItems;
+  LayoutItems.push_back(MDString::get(Ctx, LayoutStruct->getName()));
+  LayoutItems.push_back(nullptr);
+
+  size_t BufferSize = 0;
+  bool UsePackoffset = BufDecl->hasPackoffset();
+  const auto *ElemIt = LayoutStruct->element_begin();
+  for (Decl *D : BufDecl->decls()) {
+    if (isa<CXXRecordDecl, EmptyDecl>(D))
       // Nothing to do for this declaration.
-    } else if (isa<FunctionDecl>(it)) {
-      // A function within an cbuffer is effectively a top-level function,
-      // as it only refers to globally scoped declarations.
-      CGM.EmitTopLevelDecl(it);
+      continue;
+    if (isa<FunctionDecl>(D)) {
+      // A function within an cbuffer is effectively a top-level function.
+      CGM.EmitTopLevelDecl(D);
+      continue;
+    }
+    VarDecl *VD = dyn_cast<VarDecl>(D);
+    if (!VD)
+      continue;
+
+    QualType VDTy = VD->getType();
+    if (VDTy.getAddressSpace() != LangAS::hlsl_constant) {
+      if (VD->getStorageClass() == SC_Static ||
+          VDTy.getAddressSpace() == LangAS::hlsl_groupshared ||
+          isResourceRecordTypeOrArrayOf(VDTy.getTypePtr())) {
+        // Emit static and groupshared variables and resource classes inside
+        // cbuffer as regular globals
+        CGM.EmitGlobal(VD);
+      }
+      // Anything else that is not in the hlsl_constant address space must be
+      // an empty struct or a zero-sized array and can be ignored
+      continue;
     }
+
+    assert(ElemIt != LayoutStruct->element_end() &&
+           "number of elements in layout struct does not match");
+    llvm::Type *LayoutType = *ElemIt++;
+
+    // Make sure the type of the VarDecl type matches the type of the layout
+    // struct element, or that it is a layout struct with the same name
+    assert((CGM.getTypes().ConvertTypeForMem(VDTy) == LayoutType ||
+            (LayoutType->isStructTy() &&
+             cast<llvm::StructType>(LayoutType)
+                 ->getName()
+                 .starts_with(("struct.__cblayout_" +
+                               VDTy->getAsCXXRecordDecl()->getName())
+                                  .str()))) &&
+           "layout type does not match the converted element type");
+
+    // there might be resources inside the used defined structs
+    if (VDTy->isStructureType() && VDTy->isHLSLIntangibleType())
+      // FIXME: handle resources in cbuffer structs
+      llvm_unreachable("resources in cbuffer are not supported yet");
+
+    // create global variable for the constant and to metadata list
+    GlobalVariable *ElemGV =
+        cast<GlobalVariable>(CGM.GetAddrOfGlobalVar(VD, LayoutType));
+    BufGlobals.push_back(ValueAsMetadata::get(ElemGV));
+
+    // get offset of the global and and to metadata list
+    assert(((UsePackoffset && VD->hasAttr<HLSLPackOffsetAttr>()) ||
+            !UsePackoffset) &&
+           "expected packoffset attribute on every declaration");
+    size_t Offset = calculateBufferElementOffset(
+        LayoutType, &BufferSize,
+        UsePackoffset ? VD->getAttr<HLSLPackOffsetAttr>() : nullptr);
+    LayoutItems.push_back(getConstIntMetadata(Ctx, Offset));
   }
+  assert(ElemIt == LayoutStruct->element_end() &&
+         "number of elements in layout struct does not match");
+  // set the size of the buffer
+  LayoutItems[1] = getConstIntMetadata(Ctx, BufferSize);
+
+  // add buffer metadata to the module
+  CGM.getModule()
+      .getOrInsertNamedMetadata("hlsl.cbs")
+      ->addOperand(MDNode::get(Ctx, BufGlobals));
+
+  CGM.getModule()
+      .getOrInsertNamedMetadata("hlsl.cblayouts")
+      ->addOperand(MDNode::get(Ctx, LayoutItems));
 }
 
-void CGHLSLRuntime::addBuffer(const HLSLBufferDecl *D) {
-  Buffers.emplace_back(Buffer(D));
-  addBufferDecls(D, Buffers.back());
+// Creates resource handle type for the HLSL buffer declaration
+static const clang::HLSLAttributedResourceType *
+createBufferHandleType(const HLSLBufferDecl *BufDecl) {
+  ASTContext &AST = BufDecl->getASTContext();
+  QualType QT = AST.getHLSLAttributedResourceType(
+      AST.HLSLResourceTy,
+      QualType(BufDecl->getLayoutStruct()->getTypeForDecl(), 0),
+      HLSLAttributedResourceType::Attributes(ResourceClass::CBuffer));
+  return cast<HLSLAttributedResourceType>(QT.getTypePtr());
+}
+
+// Codegen for HLSLBufferDecl
+void CGHLSLRuntime::addBuffer(const HLSLBufferDecl *BufDecl) {
+
+  assert(BufDecl->isCBuffer() && "tbuffer codegen is not supported yet");
+
+  // create resource handle type for the buffer
+  const clang::HLSLAttributedResourceType *ResHandleTy =
+      createBufferHandleType(BufDecl);
+
+  // empty constant buffer is ignored
+  if (ResHandleTy->getContainedType()->getAsCXXRecordDecl()->isEmpty())
+    return;
+
+  // create global variable for the constant buffer
+  llvm::Module &M = CGM.getModule();
+  llvm::TargetExtType *TargetTy =
+      cast<llvm::TargetExtType>(convertHLSLSpecificType(ResHandleTy));
+  llvm::GlobalVariable *BufGV =
+      new GlobalVariable(TargetTy, /*isConstant*/ true,
+                         GlobalValue::LinkageTypes::ExternalLinkage, nullptr,
+                         llvm::formatv("{0}{1}", BufDecl->getName(),
+                                       BufDecl->isCBuffer() ? ".cb" : ".tb"),
+                         GlobalValue::NotThreadLocal);
+  M.insertGlobalVariable(BufGV);
+
+  // Add globals for constant buffer elements and create metadata nodes
+  emitBufferGlobalsAndMetadata(BufDecl, BufGV);
+
+  // Resource initialization
+  const HLSLResourceBindingAttr *RBA =
+      BufDecl->getAttr<HLSLResourceBindingAttr>();
+  // FIXME: handle implicit binding if no binding attribute is found
+  if (RBA)
+    createResourceInitFn(CGM, BufGV, RBA->getSlotNumber(),
+                         RBA->getSpaceNumber());
 }
 
 void CGHLSLRuntime::finishCodeGen() {
@@ -173,28 +378,8 @@ void CGHLSLRuntime::finishCodeGen() {
   generateGlobalCtorDtorCalls();
   if (CGM.getCodeGenOpts().OptimizationLevel == 0)
     addDisableOptimizations(M);
-
-  const DataLayout &DL = M.getDataLayout();
-
-  for (auto &Buf : Buffers) {
-    layoutBuffer(Buf, DL);
-    GlobalVariable *GV = replaceBuffer(Buf);
-    M.insertGlobalVariable(GV);
-    llvm::hlsl::ResourceClass RC = Buf.IsCBuffe...
[truncated]

@llvmbot
Copy link
Member

llvmbot commented Jan 29, 2025

@llvm/pr-subscribers-backend-directx

Author: Helena Kotas (hekota)

Changes

Translates cbuffer declaration blocks to target("dx.CBuffer") type. Creates global variables in hlsl_constant address space for all cbuffer constant and adds metadata describing which global constant belongs to which constant buffer. Also adds metadata describing the layout of the constant buffer and embedded structs.

These globals are temporary and will be removed in upcoming pass HLSLConstantAccess that will translate load instructions in the hlsl_constant address space to constant buffer load intrinsics calls off a CBV handle (#124630, #112992).

See Constant buffer design doc for more details.

Fixes #113514


Patch is 48.57 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/124886.diff

15 Files Affected:

  • (modified) clang/include/clang/AST/Decl.h (+6)
  • (modified) clang/include/clang/AST/Type.h (+2-2)
  • (modified) clang/lib/AST/Decl.cpp (+16-1)
  • (modified) clang/lib/CodeGen/CGHLSLRuntime.cpp (+297-124)
  • (modified) clang/lib/CodeGen/CGHLSLRuntime.h (+19-13)
  • (modified) clang/lib/CodeGen/Targets/DirectX.cpp (+7-3)
  • (modified) clang/lib/Sema/SemaHLSL.cpp (+3)
  • (removed) clang/test/CodeGenHLSL/cbuf.hlsl (-33)
  • (removed) clang/test/CodeGenHLSL/cbuf_in_namespace.hlsl (-29)
  • (added) clang/test/CodeGenHLSL/cbuffer.hlsl (+199)
  • (added) clang/test/CodeGenHLSL/cbuffer_and_namespaces.hlsl (+63)
  • (added) clang/test/CodeGenHLSL/cbuffer_with_packoffset.hlsl (+40)
  • (added) clang/test/CodeGenHLSL/cbuffer_with_static_global_and_function.hlsl (+32)
  • (modified) clang/test/CodeGenHLSL/resource-bindings.hlsl (+4)
  • (removed) clang/test/CodeGenHLSL/static_global_and_function_in_cb.hlsl (-22)
diff --git a/clang/include/clang/AST/Decl.h b/clang/include/clang/AST/Decl.h
index 16403774e72b31c..e1c7e3817699ce0 100644
--- a/clang/include/clang/AST/Decl.h
+++ b/clang/include/clang/AST/Decl.h
@@ -5032,6 +5032,9 @@ class HLSLBufferDecl final : public NamedDecl, public DeclContext {
   SourceLocation KwLoc;
   /// IsCBuffer - Whether the buffer is a cbuffer (and not a tbuffer).
   bool IsCBuffer;
+  /// HasValidPackoffset - Whether the buffer has valid packoffset annotations
+  //                       on all declarations
+  bool HasPackoffset;
 
   HLSLBufferDecl(DeclContext *DC, bool CBuffer, SourceLocation KwLoc,
                  IdentifierInfo *ID, SourceLocation IDLoc,
@@ -5052,6 +5055,9 @@ class HLSLBufferDecl final : public NamedDecl, public DeclContext {
   SourceLocation getRBraceLoc() const { return RBraceLoc; }
   void setRBraceLoc(SourceLocation L) { RBraceLoc = L; }
   bool isCBuffer() const { return IsCBuffer; }
+  void setHasPackoffset(bool PO) { HasPackoffset = PO; }
+  bool hasPackoffset() const { return HasPackoffset; }
+  const CXXRecordDecl *getLayoutStruct() const;
 
   // Implement isa/cast/dyncast/etc.
   static bool classof(const Decl *D) { return classofKind(D->getKind()); }
diff --git a/clang/include/clang/AST/Type.h b/clang/include/clang/AST/Type.h
index 1d9743520654ebd..c3ff7ebd88516c7 100644
--- a/clang/include/clang/AST/Type.h
+++ b/clang/include/clang/AST/Type.h
@@ -6266,8 +6266,8 @@ class HLSLAttributedResourceType : public Type, public llvm::FoldingSetNode {
     LLVM_PREFERRED_TYPE(bool)
     uint8_t RawBuffer : 1;
 
-    Attributes(llvm::dxil::ResourceClass ResourceClass, bool IsROV,
-               bool RawBuffer)
+    Attributes(llvm::dxil::ResourceClass ResourceClass, bool IsROV = false,
+               bool RawBuffer = false)
         : ResourceClass(ResourceClass), IsROV(IsROV), RawBuffer(RawBuffer) {}
 
     Attributes() : Attributes(llvm::dxil::ResourceClass::UAV, false, false) {}
diff --git a/clang/lib/AST/Decl.cpp b/clang/lib/AST/Decl.cpp
index beb5fcaefac5353..fa7d03354a99379 100644
--- a/clang/lib/AST/Decl.cpp
+++ b/clang/lib/AST/Decl.cpp
@@ -1747,6 +1747,10 @@ void NamedDecl::printNestedNameSpecifier(raw_ostream &OS,
       }
     }
 
+    // Suppress transparent contexts like export or HLSLBufferDecl context
+    if (Ctx->isTransparentContext())
+      continue;
+
     // Skip non-named contexts such as linkage specifications and ExportDecls.
     const NamedDecl *ND = dyn_cast<NamedDecl>(Ctx);
     if (!ND)
@@ -5713,7 +5717,7 @@ HLSLBufferDecl::HLSLBufferDecl(DeclContext *DC, bool CBuffer,
                                SourceLocation IDLoc, SourceLocation LBrace)
     : NamedDecl(Decl::Kind::HLSLBuffer, DC, IDLoc, DeclarationName(ID)),
       DeclContext(Decl::Kind::HLSLBuffer), LBraceLoc(LBrace), KwLoc(KwLoc),
-      IsCBuffer(CBuffer) {}
+      IsCBuffer(CBuffer), HasPackoffset(false) {}
 
 HLSLBufferDecl *HLSLBufferDecl::Create(ASTContext &C,
                                        DeclContext *LexicalParent, bool CBuffer,
@@ -5743,6 +5747,17 @@ HLSLBufferDecl *HLSLBufferDecl::CreateDeserialized(ASTContext &C,
                                     SourceLocation(), SourceLocation());
 }
 
+const CXXRecordDecl *HLSLBufferDecl::getLayoutStruct() const {
+  // Layout struct is the last decl in the HLSLBufferDecl.
+  if (CXXRecordDecl *RD = llvm::dyn_cast_or_null<CXXRecordDecl>(LastDecl)) {
+    assert(RD->getName().starts_with(
+               ("__cblayout_" + getIdentifier()->getName()).str()) &&
+           "expected buffer layout struct");
+    return RD;
+  }
+  llvm_unreachable("HLSL buffer is missing a layout struct");
+}
+
 //===----------------------------------------------------------------------===//
 // ImportDecl Implementation
 //===----------------------------------------------------------------------===//
diff --git a/clang/lib/CodeGen/CGHLSLRuntime.cpp b/clang/lib/CodeGen/CGHLSLRuntime.cpp
index 2ce54cc3c52efa0..52247173b69901b 100644
--- a/clang/lib/CodeGen/CGHLSLRuntime.cpp
+++ b/clang/lib/CodeGen/CGHLSLRuntime.cpp
@@ -16,15 +16,20 @@
 #include "CGDebugInfo.h"
 #include "CodeGenModule.h"
 #include "TargetInfo.h"
+#include "clang/AST/ASTContext.h"
 #include "clang/AST/Decl.h"
+#include "clang/AST/Type.h"
 #include "clang/Basic/TargetOptions.h"
+#include "llvm/IR/DerivedTypes.h"
 #include "llvm/IR/GlobalVariable.h"
 #include "llvm/IR/LLVMContext.h"
 #include "llvm/IR/Metadata.h"
 #include "llvm/IR/Module.h"
+#include "llvm/IR/Type.h"
 #include "llvm/IR/Value.h"
 #include "llvm/Support/Alignment.h"
 
+#include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/FormatVariadic.h"
 
 using namespace clang;
@@ -32,6 +37,9 @@ using namespace CodeGen;
 using namespace clang::hlsl;
 using namespace llvm;
 
+static void createResourceInitFn(CodeGenModule &CGM, llvm::GlobalVariable *GV,
+                                 unsigned Slot, unsigned Space);
+
 namespace {
 
 void addDxilValVersion(StringRef ValVersionStr, llvm::Module &M) {
@@ -54,54 +62,11 @@ void addDxilValVersion(StringRef ValVersionStr, llvm::Module &M) {
   auto *DXILValMD = M.getOrInsertNamedMetadata(DXILValKey);
   DXILValMD->addOperand(Val);
 }
+
 void addDisableOptimizations(llvm::Module &M) {
   StringRef Key = "dx.disable_optimizations";
   M.addModuleFlag(llvm::Module::ModFlagBehavior::Override, Key, 1);
 }
-// cbuffer will be translated into global variable in special address space.
-// If translate into C,
-// cbuffer A {
-//   float a;
-//   float b;
-// }
-// float foo() { return a + b; }
-//
-// will be translated into
-//
-// struct A {
-//   float a;
-//   float b;
-// } cbuffer_A __attribute__((address_space(4)));
-// float foo() { return cbuffer_A.a + cbuffer_A.b; }
-//
-// layoutBuffer will create the struct A type.
-// replaceBuffer will replace use of global variable a and b with cbuffer_A.a
-// and cbuffer_A.b.
-//
-void layoutBuffer(CGHLSLRuntime::Buffer &Buf, const DataLayout &DL) {
-  if (Buf.Constants.empty())
-    return;
-
-  std::vector<llvm::Type *> EltTys;
-  for (auto &Const : Buf.Constants) {
-    GlobalVariable *GV = Const.first;
-    Const.second = EltTys.size();
-    llvm::Type *Ty = GV->getValueType();
-    EltTys.emplace_back(Ty);
-  }
-  Buf.LayoutStruct = llvm::StructType::get(EltTys[0]->getContext(), EltTys);
-}
-
-GlobalVariable *replaceBuffer(CGHLSLRuntime::Buffer &Buf) {
-  // Create global variable for CB.
-  GlobalVariable *CBGV = new GlobalVariable(
-      Buf.LayoutStruct, /*isConstant*/ true,
-      GlobalValue::LinkageTypes::ExternalLinkage, nullptr,
-      llvm::formatv("{0}{1}", Buf.Name, Buf.IsCBuffer ? ".cb." : ".tb."),
-      GlobalValue::NotThreadLocal);
-
-  return CBGV;
-}
 
 } // namespace
 
@@ -119,48 +84,288 @@ llvm::Triple::ArchType CGHLSLRuntime::getArch() {
   return CGM.getTarget().getTriple().getArch();
 }
 
-void CGHLSLRuntime::addConstant(VarDecl *D, Buffer &CB) {
-  if (D->getStorageClass() == SC_Static) {
-    // For static inside cbuffer, take as global static.
-    // Don't add to cbuffer.
-    CGM.EmitGlobal(D);
-    return;
+// Returns true if the type is an HLSL resource class
+static bool isResourceRecordType(const clang::Type *Ty) {
+  return HLSLAttributedResourceType::findHandleTypeOnResource(Ty) != nullptr;
+}
+
+// Returns true if the type is an HLSL resource class or an array of them
+static bool isResourceRecordTypeOrArrayOf(const clang::Type *Ty) {
+  while (const ConstantArrayType *CAT = dyn_cast<ConstantArrayType>(Ty))
+    Ty = CAT->getArrayElementTypeNoTypeQual();
+  return isResourceRecordType(Ty);
+}
+
+static ConstantAsMetadata *getConstIntMetadata(LLVMContext &Ctx, uint32_t value,
+                                               bool isSigned = false) {
+  return ConstantAsMetadata::get(
+      ConstantInt::get(Ctx, llvm::APInt(32, value, isSigned)));
+}
+
+static unsigned getScalarOrVectorSize(llvm::Type *Ty) {
+  assert(Ty->isVectorTy() || Ty->isIntegerTy() || Ty->isFloatingPointTy());
+  if (Ty->isVectorTy()) {
+    llvm::FixedVectorType *FVT = cast<llvm::FixedVectorType>(Ty);
+    return FVT->getNumElements() *
+           (FVT->getElementType()->getScalarSizeInBits() / 8);
+  }
+  return Ty->getScalarSizeInBits() / 8;
+}
+
+// Returns size of a struct in constant buffer layout. The sizes are cached
+// in StructSizesForBuffer map. The map is also an indicator if a layout
+// metadata for this struct has been added to the module.
+// If the struct type is not in the map, this method will calculate the struct
+// layout, add a metadata node describing it to the module, and add the struct
+// size to the map.
+size_t
+CGHLSLRuntime::getOrCalculateStructSizeForBuffer(llvm::StructType *StructTy) {
+  // check if we already have a side for this struct
+  auto SizeIt = StructSizesForBuffer.find(StructTy);
+  if (SizeIt != StructSizesForBuffer.end())
+    return SizeIt->getSecond();
+
+  // if not, calculate the struct layout and create a metadata node
+  LLVMContext &Ctx = CGM.getLLVMContext();
+  SmallVector<llvm::Metadata *> LayoutItems;
+
+  // start metadata list with a struct name and reserve one slot for its size
+  LayoutItems.push_back(MDString::get(Ctx, StructTy->getName()));
+  LayoutItems.push_back(nullptr);
+
+  // add element offsets
+  size_t StructSize = 0;
+  for (llvm::Type *ElTy : StructTy->elements()) {
+    size_t Offset = calculateBufferElementOffset(ElTy, &StructSize);
+    LayoutItems.push_back(getConstIntMetadata(CGM.getLLVMContext(), Offset));
+  }
+  // set the size of the buffer to the reserved slot
+  LayoutItems[1] = getConstIntMetadata(Ctx, StructSize);
+
+  // add the struct layout to metadata
+  CGM.getModule()
+      .getOrInsertNamedMetadata("hlsl.cblayouts")
+      ->addOperand(MDNode::get(CGM.getLLVMContext(), LayoutItems));
+
+  // add struct size to list and return it
+  StructSizesForBuffer[StructTy] = StructSize;
+  return StructSize;
+}
+
+// Calculates offset of a single element in constant buffer layout.
+// The provided LayoutEndOffset marks the end of the layout so far (end offset
+// of the buffer or struct). After the element offset calculations are done it
+// will be updated the new end of layout value.
+// If the PackoffsetAttrs is not nullptr the offset will be based on the
+// packoffset annotation.
+size_t CGHLSLRuntime::calculateBufferElementOffset(
+    llvm::Type *LayoutTy, size_t *LayoutEndOffset,
+    HLSLPackOffsetAttr *PackoffsetAttr) {
+
+  size_t ElemOffset = 0;
+  size_t ElemSize = 0;
+  size_t ArrayCount = 1;
+  size_t ArrayStride = 0;
+  size_t EndOffset = *LayoutEndOffset;
+  size_t NextRowOffset = llvm::alignTo(EndOffset, 16U);
+
+  if (LayoutTy->isArrayTy()) {
+    llvm::Type *Ty = LayoutTy;
+    while (Ty->isArrayTy()) {
+      ArrayCount *= Ty->getArrayNumElements();
+      Ty = Ty->getArrayElementType();
+    }
+    ElemSize =
+        Ty->isStructTy()
+            ? getOrCalculateStructSizeForBuffer(cast<llvm::StructType>(Ty))
+            : getScalarOrVectorSize(Ty);
+    ArrayStride = llvm::alignTo(ElemSize, 16U);
+    ElemOffset =
+        PackoffsetAttr ? PackoffsetAttr->getOffsetInBytes() : NextRowOffset;
+
+  } else if (LayoutTy->isStructTy()) {
+    ElemOffset =
+        PackoffsetAttr ? PackoffsetAttr->getOffsetInBytes() : NextRowOffset;
+    ElemSize =
+        getOrCalculateStructSizeForBuffer(cast<llvm::StructType>(LayoutTy));
+
+  } else {
+    size_t Align = 0;
+    if (LayoutTy->isVectorTy()) {
+      llvm::FixedVectorType *FVT = cast<llvm::FixedVectorType>(LayoutTy);
+      size_t SubElemSize = FVT->getElementType()->getScalarSizeInBits() / 8;
+      ElemSize = FVT->getNumElements() * SubElemSize;
+      Align = SubElemSize;
+    } else {
+      assert(LayoutTy->isIntegerTy() || LayoutTy->isFloatingPointTy());
+      ElemSize = LayoutTy->getScalarSizeInBits() / 8;
+      Align = ElemSize;
+    }
+    if (PackoffsetAttr) {
+      ElemOffset = PackoffsetAttr->getOffsetInBytes();
+    } else {
+      ElemOffset = llvm::alignTo(EndOffset, Align);
+      // if the element does not fit, move it to the next row
+      if (ElemOffset + ElemSize > NextRowOffset)
+        ElemOffset = NextRowOffset;
+    }
   }
 
-  auto *GV = cast<GlobalVariable>(CGM.GetAddrOfGlobalVar(D));
-  GV->setExternallyInitialized(true);
-  // Add debug info for constVal.
-  if (CGDebugInfo *DI = CGM.getModuleDebugInfo())
-    if (CGM.getCodeGenOpts().getDebugInfo() >=
-        codegenoptions::DebugInfoKind::LimitedDebugInfo)
-      DI->EmitGlobalVariable(cast<GlobalVariable>(GV), D);
-
-  // FIXME: support packoffset.
-  // See https://github.com/llvm/llvm-project/issues/57914.
-  uint32_t Offset = 0;
-  bool HasUserOffset = false;
-
-  unsigned LowerBound = HasUserOffset ? Offset : UINT_MAX;
-  CB.Constants.emplace_back(std::make_pair(GV, LowerBound));
+  // Update end offset of the buffer/struct layout; do not update it if
+  // the provided EndOffset is already bigger than the new one value
+  // (which may happen with packoffset annotations)
+  unsigned NewEndOffset =
+      ElemOffset + (ArrayCount - 1) * ArrayStride + ElemSize;
+  *LayoutEndOffset = std::max<size_t>(EndOffset, NewEndOffset);
+
+  return ElemOffset;
 }
 
-void CGHLSLRuntime::addBufferDecls(const DeclContext *DC, Buffer &CB) {
-  for (Decl *it : DC->decls()) {
-    if (auto *ConstDecl = dyn_cast<VarDecl>(it)) {
-      addConstant(ConstDecl, CB);
-    } else if (isa<CXXRecordDecl, EmptyDecl>(it)) {
+// Emits constant global variables for buffer declarations, creates metadata
+// linking the constant globals with the buffer. Also calculates the buffer
+// layout and creates metadata node describing it.
+void CGHLSLRuntime::emitBufferGlobalsAndMetadata(const HLSLBufferDecl *BufDecl,
+                                                 llvm::GlobalVariable *BufGV) {
+  LLVMContext &Ctx = CGM.getLLVMContext();
+  llvm::StructType *LayoutStruct = cast<llvm::StructType>(
+      cast<llvm::TargetExtType>(BufGV->getValueType())->getTypeParameter(0));
+
+  // Start metadata list associating the buffer global variable with its
+  // constatns
+  SmallVector<llvm::Metadata *> BufGlobals;
+  BufGlobals.push_back(ValueAsMetadata::get(BufGV));
+
+  // Start layout metadata list with a struct name and reserve one slot for
+  // the buffer size
+  SmallVector<llvm::Metadata *> LayoutItems;
+  LayoutItems.push_back(MDString::get(Ctx, LayoutStruct->getName()));
+  LayoutItems.push_back(nullptr);
+
+  size_t BufferSize = 0;
+  bool UsePackoffset = BufDecl->hasPackoffset();
+  const auto *ElemIt = LayoutStruct->element_begin();
+  for (Decl *D : BufDecl->decls()) {
+    if (isa<CXXRecordDecl, EmptyDecl>(D))
       // Nothing to do for this declaration.
-    } else if (isa<FunctionDecl>(it)) {
-      // A function within an cbuffer is effectively a top-level function,
-      // as it only refers to globally scoped declarations.
-      CGM.EmitTopLevelDecl(it);
+      continue;
+    if (isa<FunctionDecl>(D)) {
+      // A function within an cbuffer is effectively a top-level function.
+      CGM.EmitTopLevelDecl(D);
+      continue;
+    }
+    VarDecl *VD = dyn_cast<VarDecl>(D);
+    if (!VD)
+      continue;
+
+    QualType VDTy = VD->getType();
+    if (VDTy.getAddressSpace() != LangAS::hlsl_constant) {
+      if (VD->getStorageClass() == SC_Static ||
+          VDTy.getAddressSpace() == LangAS::hlsl_groupshared ||
+          isResourceRecordTypeOrArrayOf(VDTy.getTypePtr())) {
+        // Emit static and groupshared variables and resource classes inside
+        // cbuffer as regular globals
+        CGM.EmitGlobal(VD);
+      }
+      // Anything else that is not in the hlsl_constant address space must be
+      // an empty struct or a zero-sized array and can be ignored
+      continue;
     }
+
+    assert(ElemIt != LayoutStruct->element_end() &&
+           "number of elements in layout struct does not match");
+    llvm::Type *LayoutType = *ElemIt++;
+
+    // Make sure the type of the VarDecl type matches the type of the layout
+    // struct element, or that it is a layout struct with the same name
+    assert((CGM.getTypes().ConvertTypeForMem(VDTy) == LayoutType ||
+            (LayoutType->isStructTy() &&
+             cast<llvm::StructType>(LayoutType)
+                 ->getName()
+                 .starts_with(("struct.__cblayout_" +
+                               VDTy->getAsCXXRecordDecl()->getName())
+                                  .str()))) &&
+           "layout type does not match the converted element type");
+
+    // there might be resources inside the used defined structs
+    if (VDTy->isStructureType() && VDTy->isHLSLIntangibleType())
+      // FIXME: handle resources in cbuffer structs
+      llvm_unreachable("resources in cbuffer are not supported yet");
+
+    // create global variable for the constant and to metadata list
+    GlobalVariable *ElemGV =
+        cast<GlobalVariable>(CGM.GetAddrOfGlobalVar(VD, LayoutType));
+    BufGlobals.push_back(ValueAsMetadata::get(ElemGV));
+
+    // get offset of the global and and to metadata list
+    assert(((UsePackoffset && VD->hasAttr<HLSLPackOffsetAttr>()) ||
+            !UsePackoffset) &&
+           "expected packoffset attribute on every declaration");
+    size_t Offset = calculateBufferElementOffset(
+        LayoutType, &BufferSize,
+        UsePackoffset ? VD->getAttr<HLSLPackOffsetAttr>() : nullptr);
+    LayoutItems.push_back(getConstIntMetadata(Ctx, Offset));
   }
+  assert(ElemIt == LayoutStruct->element_end() &&
+         "number of elements in layout struct does not match");
+  // set the size of the buffer
+  LayoutItems[1] = getConstIntMetadata(Ctx, BufferSize);
+
+  // add buffer metadata to the module
+  CGM.getModule()
+      .getOrInsertNamedMetadata("hlsl.cbs")
+      ->addOperand(MDNode::get(Ctx, BufGlobals));
+
+  CGM.getModule()
+      .getOrInsertNamedMetadata("hlsl.cblayouts")
+      ->addOperand(MDNode::get(Ctx, LayoutItems));
 }
 
-void CGHLSLRuntime::addBuffer(const HLSLBufferDecl *D) {
-  Buffers.emplace_back(Buffer(D));
-  addBufferDecls(D, Buffers.back());
+// Creates resource handle type for the HLSL buffer declaration
+static const clang::HLSLAttributedResourceType *
+createBufferHandleType(const HLSLBufferDecl *BufDecl) {
+  ASTContext &AST = BufDecl->getASTContext();
+  QualType QT = AST.getHLSLAttributedResourceType(
+      AST.HLSLResourceTy,
+      QualType(BufDecl->getLayoutStruct()->getTypeForDecl(), 0),
+      HLSLAttributedResourceType::Attributes(ResourceClass::CBuffer));
+  return cast<HLSLAttributedResourceType>(QT.getTypePtr());
+}
+
+// Codegen for HLSLBufferDecl
+void CGHLSLRuntime::addBuffer(const HLSLBufferDecl *BufDecl) {
+
+  assert(BufDecl->isCBuffer() && "tbuffer codegen is not supported yet");
+
+  // create resource handle type for the buffer
+  const clang::HLSLAttributedResourceType *ResHandleTy =
+      createBufferHandleType(BufDecl);
+
+  // empty constant buffer is ignored
+  if (ResHandleTy->getContainedType()->getAsCXXRecordDecl()->isEmpty())
+    return;
+
+  // create global variable for the constant buffer
+  llvm::Module &M = CGM.getModule();
+  llvm::TargetExtType *TargetTy =
+      cast<llvm::TargetExtType>(convertHLSLSpecificType(ResHandleTy));
+  llvm::GlobalVariable *BufGV =
+      new GlobalVariable(TargetTy, /*isConstant*/ true,
+                         GlobalValue::LinkageTypes::ExternalLinkage, nullptr,
+                         llvm::formatv("{0}{1}", BufDecl->getName(),
+                                       BufDecl->isCBuffer() ? ".cb" : ".tb"),
+                         GlobalValue::NotThreadLocal);
+  M.insertGlobalVariable(BufGV);
+
+  // Add globals for constant buffer elements and create metadata nodes
+  emitBufferGlobalsAndMetadata(BufDecl, BufGV);
+
+  // Resource initialization
+  const HLSLResourceBindingAttr *RBA =
+      BufDecl->getAttr<HLSLResourceBindingAttr>();
+  // FIXME: handle implicit binding if no binding attribute is found
+  if (RBA)
+    createResourceInitFn(CGM, BufGV, RBA->getSlotNumber(),
+                         RBA->getSpaceNumber());
 }
 
 void CGHLSLRuntime::finishCodeGen() {
@@ -173,28 +378,8 @@ void CGHLSLRuntime::finishCodeGen() {
   generateGlobalCtorDtorCalls();
   if (CGM.getCodeGenOpts().OptimizationLevel == 0)
     addDisableOptimizations(M);
-
-  const DataLayout &DL = M.getDataLayout();
-
-  for (auto &Buf : Buffers) {
-    layoutBuffer(Buf, DL);
-    GlobalVariable *GV = replaceBuffer(Buf);
-    M.insertGlobalVariable(GV);
-    llvm::hlsl::ResourceClass RC = Buf.IsCBuffe...
[truncated]

@hekota hekota requested a review from bogner February 3, 2025 18:56
@hekota hekota marked this pull request as draft February 10, 2025 22:02
@hekota hekota marked this pull request as ready for review February 12, 2025 04:39
Copy link
Contributor

@bogner bogner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking pretty good, though I have some misgivings about CommonHLSLTargetCodeGenInfo and think it might be better to introduce a HLSLLayoutBuilder that can be used by targets as necessary instead. If you take that suggestion a couple of my comments become moot.

Copy link

github-actions bot commented Feb 19, 2025

✅ With the latest revision this PR passed the C/C++ code formatter.

@hekota hekota merged commit 19af858 into llvm:main Feb 20, 2025
9 checks passed
llvm-beanz added a commit to llvm-beanz/llvm-project that referenced this pull request Feb 21, 2025
We do handle EmptyDecls in codegen already as of llvm#124886, but we were
blocking them in Sema. EmptyDecls tend to be caused by extra semicolons
which are not illegal.

Fixes llvm#128238
llvm-beanz added a commit that referenced this pull request Feb 26, 2025
We do handle EmptyDecls in codegen already as of #124886, but we were
blocking them in Sema. EmptyDecls tend to be caused by extra semicolons
which are not illegal.

Fixes #128238
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend:DirectX clang:codegen IR generation bugs: mangling, exceptions, etc. clang:frontend Language frontend issues, e.g. anything involving "Sema" clang Clang issues not falling into any other category HLSL HLSL Language Support
Projects
Status: Closed
Development

Successfully merging this pull request may close these issues.

[HLSL] Translate cbuffer {...} declarations to target type dx.CBuffer
5 participants