Allow the user to control the MaxVectorTBitWidth

[tannergooding]

This will also allow Vector<T> to eventually be larger than 256-bits via explicit opt-in and for the user to explicitly opt-in to a smaller size, if desired, without requiring ISA disablement

To achieve this we've done three primary things:

1. We now track as part of CG2/NAOT compilation whether a type is Vector<T> or transitively has a Vector<T> field. Such structs are marked as requiring a type layout check on load
2. We now explicitly track the Vector<T> size as an instruction set flag. Only one Vector<T> size is allowed to be specified at a time and we have a set of asserts validating that is the case
3. The user can specify the maximum desired size for Vector<T> via an environment variable (JIT) or command line switch (CG2/NAOT). This plays into the selected Vector<T> size

[ghost]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This resolves #85543 and allows a bit more fine grained control without requiring an ISA to be disabled.

This will also allow Vector<T> to eventually be larger than 256-bits via explicit opt-in.

Author:	tannergooding
Assignees:	tannergooding
Labels:	area-CodeGen-coreclr
Milestone:	-

[tannergooding]

This doesn't yet cover a corresponding switch for NAOT/R2R.

[tannergooding]

LLVM/GCC actually extend this prefer-vector-width=256 behavior all the way up to the latest generation.

However, the general throttling issue has been fixed since Ice Lake and there are only a few instructions with "false dependencies" that can still cause some slowdown if used incorrectly. Since we don't have a general purpose auto-vectorizer and really just use this to control simple memory operations and comparisons, we should be fine limiting it to just the below.

[anthonycanino]

Thanks for this Tanner.

[BruceForstall]

@dotnet/jit-contrib

[tannergooding]

Resolved merge conflicts

[sebastienros]

/benchmark fortunes_ef aspnet-citrine-win runtime

[pr-benchmarks]

Benchmark started for fortunes_ef on aspnet-citrine-win with runtime. Logs: link

[tannergooding]

Ping @dotnet/jit-contrib, @jkotas for review/feedback

[BruceForstall]

This generally looks good to me. I had a couple suggestions on namings that (unfortunately) would be somewhat pervasive. You can decide whether to take them or not.

Presumably, where before using arm64 altjit on x64 we would set DOTNET_SIMD16ByteOnly=1, now we would set DOTNET_MaxVectorTBitWidth=128?

[jkotas]

Have you given any thought to how this should work with AOT compilers?

• It would be nice to be able to pass the Vector<T> size to the AOT compiler
• We need check to validate that the Vector<T> size chosen at AOT time matches the size chosen at runtime

[tannergooding]

Have you given any thought to how this should work with AOT compilers?

• It would be nice to be able to pass the Vector size to the AOT compiler
• We need check to validate that the Vector size chosen at AOT time matches the size chosen at runtime

Right. My thought was that much as you can pass --instruction-set avx,avx2,bmi1,bmi2; you should also be able to pass --preferred-vector-bitwidth 256 and --max-vector-t-bitwidth 128 (or similar names).

This would function much as the environment variables do and just help instruct how codegen should occur. It would choose the minimum of the value passed in by the user and the largest actually supported given the target ISAs. So, if you said --preferred-vector-bitwidth 256 but didn't also specify --instruction-set avx2, then you would still end up with 128-bit codegen.

It then functions essentially identically to how the JIT functions in this PR, just in R2R/NAOT instead. What is a bit "unclear" is how exactly Vector<T> would work if the user opted for 128-bit in R2R and t hen ended up with 256-bit at runtime. This is something we could be doing already, but we don't as the handling to throw away just the functions which mismatch doesn't exist today. That may mean we need to differentiate between "user specified" and "system default" as we don't want the entire R2R image being thrown away by default just because R2R targets an SSE2 baseline.

[jkotas]

What is a bit "unclear" is how exactly Vector would work if the user opted for 128-bit in R2R and then ended up with 256-bit at runtime.

It should work same way as other instruction sets specified at AOT time. The system can treat Vector128/256/512 as a "virtual" vector instruction set. It is defined like that in https://github.com/dotnet/runtime/blob/main/src/coreclr/tools/Common/JitInterface/ThunkGenerator/InstructionSetDesc.txt#L92-L94.

his is something we could be doing already, but we don't as the handling to throw away just the functions which mismatch doesn't exist today.

We do have infrastructure to throw away just functions which mismatch (look for needPerMethodInstructionSetFixup). It probably needs work to hook it up to the Vector128/256/512 size configuration.

[tannergooding]

It should work same way as other instruction sets specified at AOT time.

So you're indicating rather than DOTNET_MaxVectorTBitWidth we should instead provide InstructionSet_VectorT128, InstructionSet_VectorT256, and InstructionSet_VectorT512, with the last one being supported but off by default and requiring explicit opt-in, such as via DOTNET_VectorT512=1?

We do have infrastructure to throw away just functions which mismatch (look for needPerMethodInstructionSetFixup). It probably needs work to hook it up to the Vector128/256/512 size configuration.

My understanding is that this doesn't quite work as expected and has a number of larger work items pending. One example the general issue that if you pre-compile for --instruction-set avx2 and you run on hardware without AVX2, then the entire R2R image is thrown away, not just the methods that actually used the VEX encoding.

I expect that trying to plug the vector size information into this same thing may likewise be problematic today.

From #61471 (comment):

The current expectation is that the --instruction-set argument to crossgen2 should make an image where all of the compiled code will be dropped if the application is run on a machine which does not support the specified instruction set. The end result will be significantly degraded startup time on machines without the specific instruction sets. Due to engineering concerns in the current implementation of the JIT/crossgen2 compiler, we're not currently able to enable AVX (or SSE4.2) support selectively on a subset of methods compiled into the application.

[tannergooding]

I've converted this to a draft while I work through the last of the issues. I believe I've nearly got everything handled now.

[tannergooding]

This should be ready for review again. It's been trimmed down to just the Vector<T> sizing support and we now have (from other recent PRs) relevant tests validating baseline vs avx vs avx2 vs avx512f for JIT, CG2, and NAOT.

[davidwrighton]

Thank you for the updated checkin comment, and for splitting up this work into the set of PRs you made. It now looks good.