Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate location assignments #3308

Merged
merged 10 commits into from
Jun 25, 2020

Conversation

alan-baker
Copy link
Contributor

Validate that location, component and index decorations do not cause an overlap in any interface.

Checks:

  • location can only be applied to variables or structure members
  • variable that require a location have one
  • index can only be on fragment outputs
  • no conflicting location, component, index assignments
    • utility functions calculate the consumed locations and components

Tests include common corners such as a scalar being packed after 3-component 64-bit vector.

Tested against CTS.

@alan-baker alan-baker requested a review from dneto0 April 17, 2020 19:44
@alan-baker alan-baker self-assigned this Apr 17, 2020
@alan-baker
Copy link
Contributor Author

The glslang tests appear to because the common test runner in glslang doesn't perform the IO mapping to actually assign auto-mapped locations.

Copy link
Collaborator

@dneto0 dneto0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks really good.
I think the main issue is stripping away the arrayness needs to be conditional on stage and some other decorations.

// This function assumes a base location has been determined already. As such
// any further location decorations are invalid.
// TODO: if this code turns out to be slow, there is an opportunity to cache
// the for a given type id.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cache the ... result?

std::tie(is_int, is_const, value) =
_.EvalInt32IfConst(type->GetOperandAs<uint32_t>(2));
if (is_int && is_const) *num_locations *= value;
break;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should return an error if is not a constant?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That should be validated elsewhere. Also, the validator only really evaluates 32-bit integer constants so is_int would be false for other integer sizes.

break;
}
default:
break;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should return an error here? E.g. pointer type.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That validation belongs on the variable, but it might currently be missing.

@@ -114,6 +419,16 @@ spv_result_t ValidateInterfaces(ValidationState_t& _) {
}
}

if (spvIsVulkanEnv(_.context()->target_env)) {
for (auto& inst : _.ordered_instructions()) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Could exit as soon as you see OpTypeVoid. That way we don't scan the entire module.

has_index = true;
index = dec.params()[0];
} else if (dec.dec_type() == SpvDecorationBuiltIn) {
// Don't check built-ins.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or should the validator error out if builtins have any kind of location annotation?

Vulkan 14.1.1: "Built-ins must not have any Location or Component decorations."

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about that, but I'm not sure this is the best place to do it. Ideally the validator would check for all levels of variable/struct and I didn't want to do all that searching here.

auto ptr_type = _.FindDef(ptr_type_id);
auto type_id = ptr_type->GetOperandAs<uint32_t>(2);
auto type = _.FindDef(type_id);
// Unpack optional arrayness.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This only applies to the ones that are required to be arrays:
tessellation control and mesh per-vertex shader outputs
tessellation evaluation, tessellation control, geometry shader per-vertex inputs.

See https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/vkspec.html#interfaces-iointerfaces-matching

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For "mesh per-vertex output", that means on Task shader the variables with PerTaskNV decoration.
http://htmlpreview.github.io/?https://github.com/KhronosGroup/SPIRV-Registry/blob/master/extensions/NV/SPV_NV_mesh_shader.html

For the inputs cases noted above, see GLSL 4.5:

Tessellation control, evaluation, and geometry shader input variables get the per-vertex values written out by output variables of the same names in the previous active shader stage. For these inputs, centroid and interpolation qualifiers are allowed, but have no effect. Since tessellation control, tessellation evaluation, and geometry shaders operate on a set of vertices, each input variable (or input block, see interface blocks below) needs to be declared as an array.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For tessellation control outputs, outputs are partitioned into per-vertex and per-patch. Per-patch variables are decorated with Patch. Per-vertex variables are always arrayed, and an invocation can only write to its own lane, i.e. with index gl_InvocationID.

Example:

#version 460

layout(vertices=5) out;

layout(location=0) out uint arr[];
layout(location=1) patch out uint count;

void main() {
  const uint c = arr.length();
  count = c;
  arr[gl_InvocationID] = c;
}

In the SPIR-V. arr is an array and c is not:

               OpCapability Tessellation
          %1 = OpExtInstImport "GLSL.std.450"
               OpMemoryModel Logical GLSL450
               OpEntryPoint TessellationControl %main "main" %count %arr %gl_InvocationID
               OpExecutionMode %main OutputVertices 5
               OpSource GLSL 460
               OpName %main "main"
               OpName %count "count"
               OpName %arr "arr"
               OpName %gl_InvocationID "gl_InvocationID"
               OpDecorate %count Patch
               OpDecorate %count Location 1
               OpDecorate %arr Location 0
               OpDecorate %gl_InvocationID BuiltIn InvocationId
       %void = OpTypeVoid
          %3 = OpTypeFunction %void
       %uint = OpTypeInt 32 0
%_ptr_Output_uint = OpTypePointer Output %uint
      %count = OpVariable %_ptr_Output_uint Output
     %uint_5 = OpConstant %uint 5
%_arr_uint_uint_5 = OpTypeArray %uint %uint_5
%_ptr_Output__arr_uint_uint_5 = OpTypePointer Output %_arr_uint_uint_5
        %arr = OpVariable %_ptr_Output__arr_uint_uint_5 Output
        %int = OpTypeInt 32 1
%_ptr_Input_int = OpTypePointer Input %int
%gl_InvocationID = OpVariable %_ptr_Input_int Input
       %main = OpFunction %void None %3
          %5 = OpLabel
               OpStore %count %uint_5
         %16 = OpLoad %int %gl_InvocationID
         %17 = OpAccessChain %_ptr_Output_uint %arr %16
               OpStore %17 %uint_5
               OpReturn
               OpFunctionEnd

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, what I implemented is slightly more permissive (since variables will consume fewer locations).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've made a bunch of changes related to this. The validator now accounts for these implicitly arrayed types and also accounts for component decorations getting used on arrayed types to interleave variables in the location component space. @dneto0 PTAL.

Copy link
Collaborator

@dneto0 dneto0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your patience.
LGTM

@alan-baker
Copy link
Contributor Author

@dneto0 this latest patch fixes up some arrayed interfaces checks after digging into the remaining glslang failures.

@alan-baker alan-baker requested a review from dneto0 May 21, 2020 17:45
* Add validation that input/output locations are assigned correctly
* Many tests
* Account for 4 components per location and track the combined
coordinate
* Track index1 for outputs separately
* More tests
* More comments
* Formatting
* handle specifically arrayed variables (WIP)
  * update a test
* fix comment
* early exit
* Arrayed variables that specify a component get locations determined
per index of the array for the sub type
  * Added tests that check locations and components can be assigned
  to interleave an array's locations and components
* Fix up which interfaces are allowed to be arrayed for various shader
stages based on glslang
* rebase on master
@alan-baker alan-baker merged commit a1fb255 into KhronosGroup:master Jun 25, 2020
@alan-baker alan-baker deleted the validate-location branch June 25, 2020 17:22
dneto0 pushed a commit to dneto0/SPIRV-Tools that referenced this pull request Sep 14, 2024
Roll third_party/glslang/ 8397044..f5ed7a6 (30 commits)

KhronosGroup/glslang@8397044...f5ed7a6

$ git log 8397044..f5ed7a6 --date=short --no-merges --format='%ad %ae %s'
2020-07-03 marcin.slusarz Add --quiet option.
2020-07-05 ShabbyX gn: Fix dawn tests in Chromium
2020-07-05 ShabbyX gn: Fix `gn gen --check` by adding missing dependency
2020-07-03 bclayton Add GLSLANG_BUILD_PIC CMake flag
2020-07-03 ShabbyX gn: Optionally disable optimizations and HLSL
2020-07-03 bclayton Don't use add_link_options() on old CMake versions
2020-07-03 bclayton License headers: s/Google/The Khronos Group
2020-07-03 bclayton Kokoro: Correct the `build_file' path to build.sh
2020-07-02 bclayton Add config for license-checker and Kokoro scripts.
2020-07-02 bclayton Fix GLSLANG_IS_SHARED_LIBRARY define
2020-07-01 bclayton Add missing copyright headers
2020-07-02 cepheus Bump revision.
2020-07-01 cepheus SPIRV-Tools and tests: Update to location-validation in SPIRV-Tools.
2020-07-01 cepheus Tests: More broadly use automapping binding/location.
2020-07-01 bclayton Add additional licenses in use to LICENSE.txt
2020-07-01 cepheus HLSL: Catch error cases earlier, preventing a later assert.
2020-06-29 bclayton glslang: Only export public interface for SOs
2020-06-29 bclayton CMake: break up glslang into smaller static libs
2020-06-30 cepheus SPV: RelaxedPrecision: use the result precision for texture sampling.
2020-06-30 cepheus SPV: RelaxedPrecision: Generalize fix KhronosGroup#2293 to cover more operations.
2020-06-24 e.proydakov Fixed GCC -Wunused-parameter in hlslParseables.cpp.
2020-06-29 bclayton CMake: Compile with -fPIC when building SOs
2020-06-29 bclayton CMake: Error on unresolved symbols
2020-06-29 bclayton Remove root kokoro/linux-*-cmake configs
2020-06-26 cepheus SPV: Fix KhronosGroup#2293: keep relaxed precision on arg passed to relaxed param
2020-06-26 cepheus SPV: Partially address KhronosGroup#2293: correct "const in" precision matching.
2020-06-25 lriki.net Add pack_matrix test
2020-06-12 lriki.net HLSL: Fix #pragma pack_matrix(row_major) not work on global uniforms
2020-06-24 bclayton Kokoro: Split linux cmake cfgs into static/shared
2020-06-23 e.proydakov Fixed msvc 2019 nmake compiler warnings with RTTI. By default cmake generates cxx_flags with `/GR` parameter. I updated CMAKE_CXX_FLAGS string and replaced `/GR` -> `/GR-`

Created with:
  roll-dep third_party/glslang

Roll third_party/googletest/ c6e309b26..356f2d264 (10 commits)

google/googletest@c6e309b...356f2d2

$ git log c6e309b26..356f2d264 --date=short --no-merges --format='%ad %ae %s'
2020-07-01 absl-team Googletest export
2020-06-26 absl-team Googletest export
2020-06-25 absl-team Googletest export
2020-06-24 absl-team Googletest export
2020-06-24 absl-team Googletest export
2020-06-19 mayur.shingote Updated googletest issue tracker url.
2020-06-10 rharrison Fix build issue for MinGW
2020-02-21 nini16041988-gitbucket Add missing call for gtest_list_output_unittest_ unitTest. Add unitTest for fixed TEST_P line number. Use CodeLocation TestInfo struct.
2020-02-18 nini16041988-gitbucket Fix: shadow member
2020-02-18 nini16041988-gitbucket Add correct line number to TEST_P test cases for gtest_output.

Created with:
  roll-dep third_party/googletest

Roll third_party/re2/ 14d319322..fe8a81adc (3 commits)

google/re2@14d3193...fe8a81a

$ git log 14d319322..fe8a81adc --date=short --no-merges --format='%ad %ae %s'
2020-07-05 junyer Bump SONAME, which missing ')' versus unexpected ')' needed.
2020-07-03 courbet Make the compiler inline the hot RE2::DFA loop.
2020-06-25 Shikugawa change bazel cpu symbol from wasm to wasm32

Created with:
  roll-dep third_party/re2

Roll third_party/spirv-cross/ f9ae06512..559b21c6c (14 commits)

KhronosGroup/SPIRV-Cross@f9ae065...559b21c

$ git log f9ae06512..559b21c6c --date=short --no-merges --format='%ad %ae %s'
2020-07-06 dsinclair Roll deps.
2020-07-01 post MSL: Do not emit swizzled writes in packing fixups.
2020-07-01 post MSL: Workaround broken vector -> scalar access chain in MSL.
2020-07-06 post MSL: Use input attachment index directly for resource index fallback.
2020-07-06 post GLSL: Support I/O flattening with arrays as final type.
2020-07-03 post GLSL: Support multi-level struct flattening for I/O.
2020-07-01 post Run format_all.sh.
2020-07-01 post test: Use --hlsl-dx9-compatible when attempting to compile SM 3.0 shaders.
2020-06-30 post GLSL: Fix nested legacy switch workarounds.
2020-06-29 post GLSL: Implement switch on ESSL 1.0.
2020-06-29 post GLSL: Use for-loop fallback instead of do/while for legacy ESSL.
2020-06-29 post Implement context-sensitive expression read tracking.
2020-06-29 post Fix bug with control dependent expression tracking.
2020-06-23 post HLSL: Workaround FXC bugs with degenerate switch blocks.

Created with:
  roll-dep third_party/spirv-cross

Roll third_party/spirv-headers/ 11d7637..308bd07 (1 commit)

KhronosGroup/SPIRV-Headers@11d7637...308bd07

$ git log 11d7637..308bd07 --date=short --no-merges --format='%ad %ae %s'
2020-06-26 dj2 Register the Tint compiler

Created with:
  roll-dep third_party/spirv-headers

Roll third_party/spirv-tools/ d4b9f57..6a4da9d (16 commits)

KhronosGroup/SPIRV-Tools@d4b9f57...6a4da9d

$ git log d4b9f57..6a4da9d --date=short --no-merges --format='%ad %ae %s'
2020-07-06 jaebaek Debug info preservation in copy-prop-array pass (KhronosGroup#3444)
2020-07-03 vasniktel spirv-fuzz: TransformationInvertComparisonOperator (KhronosGroup#3475)
2020-07-02 vasniktel Fix regression (KhronosGroup#3481)
2020-07-02 vasniktel spirv-fuzz: Add fuzzerutil::FindOrCreate* (KhronosGroup#3479)
2020-06-30 vasniktel spirv-fuzz: Add FuzzerPassAddCopyMemoryInstructions (KhronosGroup#3391)
2020-06-30 vasniktel spirv-fuzz: Add one parameter at a time (KhronosGroup#3469)
2020-06-29 jaebaek Fix ADCE pass bug for mulitple entries (KhronosGroup#3470)
2020-06-26 ehsannas Add gl_BaseInstance to the name mapper. (KhronosGroup#3462)
2020-06-26 andreperezmaselco.developer Implement the OpMatrixTimesScalar linear algebra case (KhronosGroup#3450)
2020-06-25 jaebaek Clear debug information for kill and replacement (KhronosGroup#3459)
2020-06-25 alanbaker Validate location assignments (KhronosGroup#3308)
2020-06-23 ehsannas Support OpCompositeExtract pattern in desc_sroa (KhronosGroup#3456)
2020-06-23 vasniktel spirv-fuzz: Implement FuzzerPassAddParameters (KhronosGroup#3399)
2020-06-23 vasniktel spirv-fuzz: Add GetParameters (KhronosGroup#3454)
2020-06-23 vasniktel spirv-fuzz: Permute OpPhi instruction operands (KhronosGroup#3421)
2020-06-22 rharrison Add support for different default/trunks in roll-deps (KhronosGroup#3442)

Created with:
  roll-dep third_party/spirv-tools
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants