Stryker failure : System.InvalidOperationException: Sequence contains no elements

[andrey-noskov]

Describe the bug
I don't have a clear enough description, but in Azure DevOps Stryker constantly fails on the "analyzing project" step and exits with code 1. May be I am missing something obvious.

Logs
The tool is executed in Azure DevOps, so the log is copied from the step logs.
stryker-log.log

Expected behavior
Analyze the project and run mutation tests.

Desktop (please complete the following information):

• Windows nodes in Azure DevOps
• framework
• 4.6
• 0.13.0 (beta)

Additional context
Add any other context about the problem here.

[richardwerkman]

Thanks for submitting this issue! Does this behaviour only occur on Azure Devops? It looks like a dependency of ours (buildalyzer) cannot open your project file CoreTests\CoreTests.csproj.

I'm curious if this happens too if you run stryker locally.

[rouke-broersma]

It looks like we are failing to read your test project file for some reason. Does this only happen on azure devops? Could you try it locally and use -f to produce the logfile?

Could you also perhaps share the project file with us?

[andrey-noskov]

yes, it fails in Azure Devops only.
it works fine on my machine (the log is attached), though we seem to have a flaky test
log-20190911.txt

[andrey-noskov]

and here is the project file with tests
CoreTests.csproj.txt

[richardwerkman]

One difference I can think of is the msbuild version. There might be a version installed on DevOps that is not compatible with buildalyzer.

Unfortunately we don't log the msbuild version before stryker crashes... So I can't compare the versions.

[andrey-noskov]

is there any script or a command I can execute locally and on the agent to collect additional details?

[rouke-broersma]

If you add -f to the stryker run in your pipeline you could use publish pipeline artifact to publish the stryker logs to your build pipeline. You can then download them and send them to us.

Other than that I'm not sure yet. We are failing here:

stryker-net/src/Stryker.Core/Stryker.Core/Initialisation/ProjectFileReader.cs

Line 58 in ccebdee

var analyzerResult = manager.GetProject(projectFilePath).Build().First();

Which to me indicates that for some reason we are unable to build your targetframework(s) using buildalyzer.

I'll try to repro using your csproj. Could you perhaps share the solutionfile and project under test projectfile as well? You can remove any sensitive information and I don't think I need any source files.

Also, which vm images are you using in your hosted pipelines? Just windows-latest or some other version? And are they the normal versions everyone else uses or are they some microsoft internal agents?

[andrey-noskov]

Here is the log from the agent:
log-20190913.txt

the solution file:
AsyncMediaShared.sln.txt

the project I want to test:
Core.csproj.txt

The agents are internal, most likely with the latest Windows and pre-installed VS2017 and Azure SDK

[rouke-broersma]

I'm not seeing anything at all suspicious about those project files, SDK style and using package reference is pretty much ideal for buildalyzer and stryker (short of being dotnet core). Have you tried it with any other projects and/or agents?

[andrey-noskov]

yeah, I did. it fails with the same exception:

[07:15:03 ERR] An error occurred during the mutation test run
System.InvalidOperationException: Sequence contains no elements
at System.Linq.Enumerable.First[TSource](IEnumerable`1 source)

[rouke-broersma]

Would it be possible to get access to the build agent or perhaps a list of all software and versions so I can try to reproduce? Software versions for windows, visual studio, msbuild, nuget and dotnet (core) installed on the system as a minimum to try to rebuild the agent on a vm.

[rouke-broersma]

Closing until repro is available

[samvandenbossche]

Is there any progress on this issue? I'm having the exact same behaviour but on my local machine

[richardwerkman]

@samvandenbossche Do you have a reproduction project that we can use to analyse the problem? With the current info we cannot research what went wrong.

[samvandenbossche]

I'm afraid not because it's against employer guidelines.
it's a mulpti-project solution ( also multiple test projects )
All projects have target framework 4.7.2 and are using the new Dependency format.
MsBuild version 4.8.3752.0

[rouke-broersma]

Could you perhaps create a reproduction using your current project as a template? Just remove anything that's not necessary to produce the failure, anything proprietary. I know that might be a lot of work but if you can't help us create a reproduction there's unfortunately nothing we can do to investigate.

[samvandenbossche]

I'll give it a try

[eNeRGy164]

If the builds were running under Linux, this issue might have fixed it: phmonte/Buildalyzer#142
This is part of Buildalyzer 3.0.1

[rouke-broersma]

Thanks for the ping @eNeRGy164.

I don't think they were running on linux as they were all dotnet framework projects as far as I can tell but I'm gonna go and close this issue anyway since there hasn't been any activity for a while.