Test discovery may fail and does not discover all tests

[Galad]

The test discovery fails every times in one of our project with the error message
Operation failed. The index entry of length 903 bytes for the index 'Idx_Stale_StateValue_DisplayName' exceeds the maximum length of 900 bytes.

When it fails the discovery is stopped and a lot of tests are not being discovered.

It seems that the test adapter may generate too long names or ids.
NUnit had the same issue, if found some bugs about it
nunit/nunit-vs-adapter#68

I have not been able to find the test triggering the error but I eventually update the issue if I do.

[akrock]

I can second that one. It fails for me as well:

Operation failed. The index entry of length 903 bytes for the index 'Idx_Stale_StateValue_DisplayName' exceeds the maximum length of 900 bytes.

When this occurs and the test explorer updates it will show a very limited number of my tests. Closing and re-opening visual studio will show the expected number of tests after a rebuild.

[tomap]

Hi,
It seems it occurs when the test name is longer that 900 char
They seem to have had the same issue on NUnit
nunit/nunit-vs-adapter#68
nunit/nunit-vs-adapter#3
https://bugs.launchpad.net/nunit-vs-adapter/+bug/1065462
https://bugs.launchpad.net/nunit-vs-adapter/+bug/1221780

[Galad]

Hello,
I had some time investigate it.
It seems that one test display name has indeed a size in byte longer than what Visual Studio expected.
The size of this test is 1200 bytes (which means 600 characters) and is over the 900 bytes limits. I only one test in this case.
The test case come from an InlineData:

[InlineData("String, Dictionary<Number+, Array<Number>>", new[] { typeof(string), typeof(EbDictionary<double, INabsicArray<double?>>) })]
public void TypeUtils_ParseNabsicTypes(string types, Type[] expectedTypes)
{
}

And the test display name generated by xunit is:

Enablon.Server.Nabsic.Tests.Tools.TypeUtilsFixture.TypeUtils_ParseNabsicTypes(types: "String, Dictionary<Number+, Array<Number>>", expectedTypes: [typeof(System.String), typeof(Enablon.Server.Wizz.CalcObjects.EbDictionary`2[[System.Double, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[Enablon.Server.Nabsic.Runtime.Objects.INabsicArray`1[[System.Nullable`1[[System.Double, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], Enablon.Server.Nabsic, Version=7.8.0.0, Culture=neutral, PublicKeyToken=bce3f989f4a9ff0b]])])`

I was not able to understand why this name is so long.

You can reproduce the issue with this snippet:
https://gist.github.com/Galad/5262df941c4b0a42cea3
The expected number of tests is 901, but the actual number around 480 (it is not the same everytime)

Basically you need to things to have the issue:

• A test with a long display name
• Many tests in the solution

@bradwilson
I can submit a fix, but I'd like some guidance on where and how I should handle this.
Do you think that the name size limitation should be handle in xunit.visualstudio, or can it be a defect in xunit.core ?
What would be the behavior when truncating the long names ?

[bradwilson]

The limitation is a Visual Studio limitation, so the truncation should happen there (in particular, when creating the Visual Studio test case object). No need to punish everybody else. :) I think basically saying anything > 450 characters would just to a .Substring(0, 447) with a ... tacked on the end.

[bradwilson]

Fixed by #70