Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DXIL: Use correct type ID when writing ValueAsMetadata. #94337

Merged
merged 1 commit into from
Sep 11, 2024

Conversation

maleadt
Copy link
Contributor

@maleadt maleadt commented Jun 4, 2024

Small drive-by PR (I'm not working on DXIL, but am working on a similar IR downgrader based on the opaque pointer handling developed for DXIL): It looks like the ValueAsMetadata handling currently is broken when writing references to functions. For example:

target datalayout = "e-m:e-p:32:32-i1:32-i8:8-i16:16-i32:32-i64:64-f16:16-f32:32-f64:64-n8:16:32:64"
target triple = "dxil-unknown-shadermodel6.7-library"

define void @kernel(ptr addrspace(1)) {
    ret void
}

!kernel = !{!0}
!0 = !{ptr @kernel}

Assembling this with llc and disassembling the contained IR with llvm-dis from LLVM 12 (again, I'm not working on DXIL so I don't have dxil-dis handy) breaks:

Assertion failed: (V && "Unexpected null Value"), function get, file Metadata.cpp, line 350.
PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace.
Stack dump:
0.	Program arguments: /Users/tim/Julia/src/llvm-12/build/bin/llvm-dis reduced.dxil -o -
Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
0  llvm-dis                 0x0000000102f421bc llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 80
1  llvm-dis                 0x0000000102f42720 PrintStackTraceSignalHandler(void*) + 28
2  llvm-dis                 0x0000000102f40890 llvm::sys::RunSignalHandlers() + 140
3  llvm-dis                 0x0000000102f43c14 SignalHandler(int) + 276
4  libsystem_platform.dylib 0x000000018611f584 _sigtramp + 56
5  libsystem_pthread.dylib  0x00000001860eec20 pthread_kill + 288
6  libsystem_c.dylib        0x0000000185ffba30 abort + 180
7  libsystem_c.dylib        0x0000000185ffad20 err + 0
8  llvm-dis                 0x0000000102ce8db4 llvm::ValueAsMetadata::get(llvm::Value*) + 100
9  llvm-dis                 0x00000001029d672c llvm::MetadataLoader::MetadataLoaderImpl::parseOneMetadata(llvm::SmallVectorImpl<unsigned long long>&, unsigned int, (anonymous namespace)::(anonymous namespace)::PlaceholderQueue&, llvm::StringRef, unsigned int&) + 2184
10 llvm-dis                 0x00000001029d5a78 llvm::MetadataLoader::MetadataLoaderImpl::parseMetadata(bool) + 1448
11 llvm-dis                 0x00000001029df7fc llvm::MetadataLoader::parseMetadata(bool) + 60
12 llvm-dis                 0x000000010298634c llvm::MetadataLoader::parseModuleMetadata() + 40
13 llvm-dis                 0x0000000102982f2c (anonymous namespace)::BitcodeReader::parseModule(unsigned long long, bool, llvm::function_ref<llvm::Optional<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > (llvm::StringRef)>) + 2272

IIUC, this is because the value emitted here is of type typedptr(void (ptr addrspace(1)), 0), while the type is simply void (i8*)* because it's not looked up in the PointerAnalysisMap. Passing V along fixes this.

cc @bogner @llvm-beanz

@llvmbot
Copy link
Member

llvmbot commented Jun 4, 2024

@llvm/pr-subscribers-backend-directx

Author: Tim Besard (maleadt)

Changes

Small drive-by PR (I'm not working on DXIL, but am working on a similar IR downgrader based on the opaque pointer handling developed for DXIL): It looks like the ValueAsMetadata handling currently is broken when writing references to functions. For example:

target datalayout = "e-m:e-p:32:32-i1:32-i8:8-i16:16-i32:32-i64:64-f16:16-f32:32-f64:64-n8:16:32:64"
target triple = "dxil-unknown-shadermodel6.7-library"

define void @<!-- -->kernel(ptr addrspace(1)) {
    ret void
}

!kernel = !{!0}
!0 = !{ptr @<!-- -->kernel}

Assembling this with llc and disassembling the contained IR with llvm-dis from LLVM 12 (again, I'm not working on DXIL so I don't have dxil-dis handy) breaks:

Assertion failed: (V &amp;&amp; "Unexpected null Value"), function get, file Metadata.cpp, line 350.
PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace.
Stack dump:
0.	Program arguments: /Users/tim/Julia/src/llvm-12/build/bin/llvm-dis reduced.dxil -o -
Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
0  llvm-dis                 0x0000000102f421bc llvm::sys::PrintStackTrace(llvm::raw_ostream&amp;, int) + 80
1  llvm-dis                 0x0000000102f42720 PrintStackTraceSignalHandler(void*) + 28
2  llvm-dis                 0x0000000102f40890 llvm::sys::RunSignalHandlers() + 140
3  llvm-dis                 0x0000000102f43c14 SignalHandler(int) + 276
4  libsystem_platform.dylib 0x000000018611f584 _sigtramp + 56
5  libsystem_pthread.dylib  0x00000001860eec20 pthread_kill + 288
6  libsystem_c.dylib        0x0000000185ffba30 abort + 180
7  libsystem_c.dylib        0x0000000185ffad20 err + 0
8  llvm-dis                 0x0000000102ce8db4 llvm::ValueAsMetadata::get(llvm::Value*) + 100
9  llvm-dis                 0x00000001029d672c llvm::MetadataLoader::MetadataLoaderImpl::parseOneMetadata(llvm::SmallVectorImpl&lt;unsigned long long&gt;&amp;, unsigned int, (anonymous namespace)::(anonymous namespace)::PlaceholderQueue&amp;, llvm::StringRef, unsigned int&amp;) + 2184
10 llvm-dis                 0x00000001029d5a78 llvm::MetadataLoader::MetadataLoaderImpl::parseMetadata(bool) + 1448
11 llvm-dis                 0x00000001029df7fc llvm::MetadataLoader::parseMetadata(bool) + 60
12 llvm-dis                 0x000000010298634c llvm::MetadataLoader::parseModuleMetadata() + 40
13 llvm-dis                 0x0000000102982f2c (anonymous namespace)::BitcodeReader::parseModule(unsigned long long, bool, llvm::function_ref&lt;llvm::Optional&lt;std::__1::basic_string&lt;char, std::__1::char_traits&lt;char&gt;, std::__1::allocator&lt;char&gt; &gt; &gt; (llvm::StringRef)&gt;) + 2272

IIUC, this is because the value emitted here is of type typedptr(void (ptr addrspace(1)), 0), while the type is simply void (i8*)* because it's not looked up in the PointerAnalysisMap. Passing V along fixes this.

cc @bogner @llvm-beanz


Full diff: https://github.com/llvm/llvm-project/pull/94337.diff

1 Files Affected:

  • (modified) llvm/lib/Target/DirectX/DXILWriter/DXILBitcodeWriter.cpp (+1-1)
diff --git a/llvm/lib/Target/DirectX/DXILWriter/DXILBitcodeWriter.cpp b/llvm/lib/Target/DirectX/DXILWriter/DXILBitcodeWriter.cpp
index bc7637a3e3def..054bcd5eaf42e 100644
--- a/llvm/lib/Target/DirectX/DXILWriter/DXILBitcodeWriter.cpp
+++ b/llvm/lib/Target/DirectX/DXILWriter/DXILBitcodeWriter.cpp
@@ -1348,7 +1348,7 @@ void DXILBitcodeWriter::writeValueAsMetadata(
     Ty = TypedPointerType::get(F->getFunctionType(), F->getAddressSpace());
   else if (GlobalVariable *GV = dyn_cast<GlobalVariable>(V))
     Ty = TypedPointerType::get(GV->getValueType(), GV->getAddressSpace());
-  Record.push_back(getTypeID(Ty));
+  Record.push_back(getTypeID(Ty, V));
   Record.push_back(VE.getValueID(V));
   Stream.EmitRecord(bitc::METADATA_VALUE, Record, 0);
   Record.clear();

@maleadt
Copy link
Contributor Author

maleadt commented Jun 4, 2024

A similar case for loading pointer values:

define void @kernel(ptr %0) {
  %box = alloca ptr addrspace(1)
  %x = load ptr addrspace(1), ptr %box, align 8
  ret void
}

This currently throws a Mismatch in load/store type: expected i8* but found i8 addrspace(1)* when reading the bitcode because the load type is set to i8*, while i8 addrspace(1)* is expected. Passing the instruction along similarly to the change proposed here makes this work:

diff --git a/llvm/lib/Bitcode/Writer50/BitcodeWriter50.cpp b/llvm/lib/Bitcode/Writer50/BitcodeWriter50.cpp
index c5c52932512f..20b58f7d1846 100644
--- a/llvm/lib/Bitcode/Writer50/BitcodeWriter50.cpp
+++ b/llvm/lib/Bitcode/Writer50/BitcodeWriter50.cpp
@@ -2845,7 +2845,7 @@ void ModuleBitcodeWriter50::writeInstruction(const Instruction &I,
       if (!pushValueAndType(I.getOperand(0), InstID, Vals)) // ptr
         AbbrevToUse = FUNCTION_INST_LOAD_ABBREV;
     }
-    Vals.push_back(getTypeID(I.getType()));
+    Vals.push_back(getTypeID(I.getType(), &I));
     Vals.push_back(getEncodedAlign(cast<LoadInst>(I).getAlign()));
     Vals.push_back(cast<LoadInst>(I).isVolatile());
     if (cast<LoadInst>(I).isAtomic()) {

And another for phis:

define i64 @kernel2(ptr %0, ptr %1, i1 %cond) {
entry:
  br i1 %cond, label %trueBlock, label %falseBlock

trueBlock:
  br label %select

falseBlock:
  br label %select

select:
  %ifelse = phi ptr [ %0, %trueBlock ], [ %1, %falseBlock ]
  %value = load i64, ptr %ifelse
  ret i64 %value
}

Resulting in an Invalid record during phi deserialization; doing Vals64.push_back(getTypeID(PN.getType(), &PN)) instead correctly preserves that type too.


I guess the DXIL serializer isn't really made for this use case, presumably because it doesn't support pointer values (as I understand from a comment in the code)?

Copy link
Contributor

@bogner bogner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pointer values are a bit interesting because DXIL (and its usual source language, HLSL) don't generally expose pointers, so these code paths are probably not really reachable without fiddling with the IR we input to the DXIL backend. That said, I think it's reasonable to take the obvious fixes either way.

It would be better if this came with a test. Can you come up with something sensible or do you need someone (like me) with access to dxil-dis to take a stab at that?

@maleadt
Copy link
Contributor Author

maleadt commented Jun 7, 2024

Pointer values are a bit interesting because DXIL (and its usual source language, HLSL) don't generally expose pointers, so these code paths are probably not really reachable without fiddling with the IR we input to the DXIL backend.

Yeah, I noticed, and there turned out to be several more issues than the ones mentioned here. I ultimately took another approach, instead of inferring pointer types based on some later use (which can lead to conflicts that fall back to i8*, and can cause issues with instructions like select and phi that assume identically-typed inputs, JuliaLLVM#7), only cast to and use typed pointers when an instruction requires it and cast back to opaque pointers right after (see JuliaLLVM#5 for the implementation). That results in a lot of bitcasts, but it's assumed that the back-end could easily optimize those away.

I'll see about adding a test for the changes in here.

@llvm-beanz
Copy link
Collaborator

Unfortunately the always casting to and from the typed pointer doesn't work for DXIL because DXIL disallows bitcasts for a lot of cases, and that is enforced by the DXIL validator which operates on typed pointer IR.

@maleadt
Copy link
Contributor Author

maleadt commented Aug 19, 2024

Sorry for the delay. Added a test (verifying that the test crashes without this change), and also changed the dxil-dis CMakeLists.txt to checkout the main branch from https://github.com/microsoft/DirectXShaderCompiler instead of the master branch my CMake defaults to.

@maleadt
Copy link
Contributor Author

maleadt commented Sep 5, 2024

@bogner Anything else you want me to add here?

Copy link
Contributor

@bogner bogner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks!

@maleadt
Copy link
Contributor Author

maleadt commented Sep 7, 2024

Feel free to merge, I don't have commit bit.

@vchuravy vchuravy merged commit 49b57df into llvm:main Sep 11, 2024
8 checks passed
@llvm-ci
Copy link
Collaborator

llvm-ci commented Sep 11, 2024

LLVM Buildbot has detected a new failure on builder lld-x86_64-win running on as-worker-93 while building llvm at step 7 "test-build-unified-tree-check-all".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/146/builds/1127

Here is the relevant piece of the build log for the reference
Step 7 (test-build-unified-tree-check-all) failure: test (failure)
******************** TEST 'LLVM-Unit :: Support/./SupportTests.exe/43/86' FAILED ********************
Script(shard):
--
GTEST_OUTPUT=json:C:\a\lld-x86_64-win\build\unittests\Support\.\SupportTests.exe-LLVM-Unit-24436-43-86.json GTEST_SHUFFLE=0 GTEST_TOTAL_SHARDS=86 GTEST_SHARD_INDEX=43 C:\a\lld-x86_64-win\build\unittests\Support\.\SupportTests.exe
--

Script:
--
C:\a\lld-x86_64-win\build\unittests\Support\.\SupportTests.exe --gtest_filter=ProgramEnvTest.CreateProcessLongPath
--
C:\a\lld-x86_64-win\llvm-project\llvm\unittests\Support\ProgramTest.cpp(160): error: Expected equality of these values:
  0
  RC
    Which is: -2

C:\a\lld-x86_64-win\llvm-project\llvm\unittests\Support\ProgramTest.cpp(163): error: fs::remove(Twine(LongPath)): did not return errc::success.
error number: 13
error message: permission denied



C:\a\lld-x86_64-win\llvm-project\llvm\unittests\Support\ProgramTest.cpp:160
Expected equality of these values:
  0
  RC
    Which is: -2

C:\a\lld-x86_64-win\llvm-project\llvm\unittests\Support\ProgramTest.cpp:163
fs::remove(Twine(LongPath)): did not return errc::success.
error number: 13
error message: permission denied




********************


Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

6 participants