ConvertToGeneratedDllImportFixerTests.PreferredAttributeOrder failed in CI

[eerhardt]

https://dev.azure.com/dnceng/public/_build/results?buildId=1442108&view=logs&jobId=4724cd84-0a26-56bc-9b1d-ca1983f340b0

Test DllImportGenerator.UnitTests.ConvertToGeneratedDllImportFixerTests.PreferredAttributeOrder has failed. Check the Test tab or this console log: https://helix.dot.net/api/2019-06-17/jobs/3e2d5fb0-c474-4dff-8b8d-61c7e8241ca3/workitems/DllImportGenerator.Unit.Tests/console

DllImportGenerator.UnitTests.ConvertToGeneratedDllImportFixerTests.RemoveableNamedArguments(usePreprocessorDefines: False) [SKIP]
      Skipping test due to issue with test-packages (The package is missing the required nuspec file. Path: /var/folders/3z/h5nxtjjj6lg28plvh_twzfs80000gq/T/test-packages/Microsoft.NETCore.App.Ref.6.0.0-preview.6.21352.12). See https://github.com/dotnet/roslyn-sdk/issues/590.
    DllImportGenerator.UnitTests.ConvertToGeneratedDllImportFixerTests.PreferredAttributeOrder(usePreprocessorDefines: True) [FAIL]
      Microsoft.DotNet.XUnitExtensions.SkipTestException : Skipping test due to issue with test-packages (The package is missing the required nuspec file. Path: /var/folders/3z/h5nxtjjj6lg28plvh_twzfs80000gq/T/test-packages/Microsoft.NETCore.App.Ref.6.0.0-preview.6.21352.12). See https://github.com/dotnet/roslyn-sdk/issues/590.
      Stack Trace:
        /_/src/libraries/System.Runtime.InteropServices/tests/DllImportGenerator.UnitTests/TestUtils.cs(170,0): at DllImportGenerator.UnitTests.TestUtils.ThrowSkipExceptionIfPackagingException(Exception e)
        /_/src/libraries/System.Runtime.InteropServices/tests/DllImportGenerator.UnitTests/Verifiers/CSharpCodeFixVerifier.cs(128,0): at DllImportGenerator.UnitTests.Verifiers.CSharpCodeFixVerifier`2.Test.RunImplAsync(CancellationToken cancellationToken)
        /_/src/Microsoft.CodeAnalysis.Testing/Microsoft.CodeAnalysis.Analyzer.Testing/AnalyzerTest`1.cs(180,0): at Microsoft.CodeAnalysis.Testing.AnalyzerTest`1.RunAsync(CancellationToken cancellationToken)
        /_/src/libraries/System.Runtime.InteropServices/tests/DllImportGenerator.UnitTests/Verifiers/CSharpCodeFixVerifier.cs(67,0): at DllImportGenerator.UnitTests.Verifiers.CSharpCodeFixVerifier`2.VerifyCodeFixAsync(String source, DiagnosticResult[] expected, String fixedSource, String codeActionEquivalenceKey)
        /_/src/libraries/System.Runtime.InteropServices/tests/DllImportGenerator.UnitTests/Verifiers/CSharpCodeFixVerifier.cs(49,0): at DllImportGenerator.UnitTests.Verifiers.CSharpCodeFixVerifier`2.VerifyCodeFixAsync(String source, String fixedSource, String codeActionEquivalenceKey)
        /_/src/libraries/System.Runtime.InteropServices/tests/DllImportGenerator.UnitTests/ConvertToGeneratedDllImportFixerTests.cs(403,0): at DllImportGenerator.UnitTests.ConvertToGeneratedDllImportFixerTests.PreferredAttributeOrder(Boolean usePreprocessorDefines)
        --- End of stack trace from previous location ---
    DllImportGenerator.UnitTests.ConvertToGeneratedDllImportFixerTests.PreferredAttributeOrder(usePreprocessorDefines: False) [FAIL]
      Microsoft.DotNet.XUnitExtensions.SkipTestException : Skipping test due to issue with test-packages (The package is missing the required nuspec file. Path: /var/folders/3z/h5nxtjjj6lg28plvh_twzfs80000gq/T/test-packages/Microsoft.NETCore.App.Ref.6.0.0-preview.6.21352.12). See https://github.com/dotnet/roslyn-sdk/issues/590.
      Stack Trace:
        /_/src/libraries/System.Runtime.InteropServices/tests/DllImportGenerator.UnitTests/TestUtils.cs(170,0): at DllImportGenerator.UnitTests.TestUtils.ThrowSkipExceptionIfPackagingException(Exception e)
        /_/src/libraries/System.Runtime.InteropServices/tests/DllImportGenerator.UnitTests/Verifiers/CSharpCodeFixVerifier.cs(128,0): at DllImportGenerator.UnitTests.Verifiers.CSharpCodeFixVerifier`2.Test.RunImplAsync(CancellationToken cancellationToken)
        /_/src/Microsoft.CodeAnalysis.Testing/Microsoft.CodeAnalysis.Analyzer.Testing/AnalyzerTest`1.cs(180,0): at Microsoft.CodeAnalysis.Testing.AnalyzerTest`1.RunAsync(CancellationToken cancellationToken)
        /_/src/libraries/System.Runtime.InteropServices/tests/DllImportGenerator.UnitTests/Verifiers/CSharpCodeFixVerifier.cs(67,0): at DllImportGenerator.UnitTests.Verifiers.CSharpCodeFixVerifier`2.VerifyCodeFixAsync(String source, DiagnosticResult[] expected, String fixedSource, String codeActionEquivalenceKey)
        /_/src/libraries/System.Runtime.InteropServices/tests/DllImportGenerator.UnitTests/Verifiers/CSharpCodeFixVerifier.cs(49,0): at DllImportGenerator.UnitTests.Verifiers.CSharpCodeFixVerifier`2.VerifyCodeFixAsync(String source, String fixedSource, String codeActionEquivalenceKey)
        /_/src/libraries/System.Runtime.InteropServices/tests/DllImportGenerator.UnitTests/ConvertToGeneratedDllImportFixerTests.cs(403,0): at DllImportGenerator.UnitTests.ConvertToGeneratedDllImportFixerTests.PreferredAttributeOrder(Boolean usePreprocessorDefines)
        --- End of stack trace from previous location ---

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[jkoritzinsky]

cc @elinor-fung looks like the idea to skip tests with the exception isn’t working quite right.

@safern whats the best way to conditionally skip a test in an imperative fashion (ie determining during a test to skip)

[safern]

You make it a conditional fact or conditional theory without any condition and then at runtime you throw a SkipTestException.

I.e:

runtime/src/libraries/System.Linq.Expressions/tests/Dynamic/ExpandoObjectProxyTests.cs

Line 91 in 57bfe47

throw new SkipTestException($"Didn't find DebuggerTypeProxyAttribute on {eo}.");

[elinor-fung]

Ergh... This was just me failing at resolving merge conflicts, such that that one test ended Theory instead of ConditionalTheory.