TensorPrimitives.IndexOfMax produces incorrect results with vectorized paths

[vcsjones]

For TensorPrimitives.IndexOfMax<T> where the number of elements is larger that the largest value T can represent and can be vectorized, the incorrect index of can be calculated.

Consider the following example

using System;
using System.Numerics.Tensors;

byte[] data = new byte[(int)byte.MaxValue + 3];
data[(int)byte.MaxValue + 2] = 255;

int result = TensorPrimitives.IndexOfMax(data);
Console.WriteLine(result);

This will print "1". ((255 + 2) % 256).

Another example, using a signed short

using System;
using System.Numerics.Tensors;

short[] data = new short[(int)short.MaxValue + 3];
data[(int)short.MaxValue + 2] = 255;

int result = TensorPrimitives.IndexOfMax(data);
Console.WriteLine(result);

This will print -32767.

This likely affects IndexOfMin, IndexOfMaxMagnitude, and IndexOfMinMagnitude since they share IndexOfMinMaxCore.

This also only impacts vectorized code paths. When .NET is forced to using scalar code paths using DOTNET_EnableHWIntrinsic=0, then the outputs are 257 and 32769, respectively.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-numerics-tensors
See info in area-owners.md if you want to be subscribed.

[pine919]

@vcsjones I am interested in this issue. can I pick this up?