No CodeCoverage with Tools from Microsoft.TestPlatform / CodeCoverage only in Tempfolder #1963

CSmith1111 · 2019-03-19T15:28:43Z

Description

A buildagent on a Windows 10 Ent. with Visual Studio 2017 Ent. 15.9.9 is able to test and cover a build. The same build over the same TFS (2018 Update 3.2) with a buildagent on the buildserver (Windows Server 2019 Standard) without Visual Studio is not able to cover a build.

Steps to reproduce

Create 2 agents and 1 build with

Build SLN
Install VsTest-Tools (16.0.1)
Run VsTest

Run on local pc with VS installed (chrisLokalerPC)
Run on buildserver without VS installed (Default)

Expected behavior

If there is a .coverage-File on the "Win10 PC"/"chrisLokalerPC"-Agent in the "./X/s/TestResults/[guid]"-Folder there should be also a .coverage-File on the "Windows Server 2019"/"Default"-Agent.

Actual behavior

There is an empty TestResults-Folder on the server (Default) and a non empty folder on the local pc (chrisLokalerPC).
If i look in the file "
C:\Users\TFSbuild\AppData\Local\Temp\CodeCoverage.exe.log" there is
[Verbose]: 3.19.2019 9:14:15:871 Code coverage log file size in bytes : 33836641 [Verbose]: 3.19.2019 15:28:14:811 Code coverage log file size in bytes : 11688544 [Verbose]: 3.19.2019 16:1:49:293 Code coverage log file size in bytes : 11688836
There are .coverage-Files in the tempfolder like "C:\Users\TFSbuild\AppData\Local\Temp\MTM_e3914e24-7b63-4890-8fa5-ab59698da2cf\f1544061-855c-44b2-8d90-ac5b52f84dc0\TFSbuild_BUILDSRV2019 2019-03-18 09_20_52.coverage".

Diagnostic logs

6_Installer für die VsTest-Plattform.txt
8_VsTest - testAssemblies.txt

Environment

"chrisLokalerPC": Windows 10 Ent. with Visual Studio 2017 Ent. 15.9.9
"Default": Windows Server 2019 Standard with Visual Studio Build Tools 2017 15.9.9

The text was updated successfully, but these errors were encountered:

ShreyasRmsft · 2019-03-20T09:43:34Z

Add the node
<UseVerifiableInstrumentation>False</UseVerifiableInstrumentation>

to your runsettings file. It's position in the runsetttings is as shown below.

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
  <!-- Configurations for data collectors -->
  <DataCollectionRunSettings>
    <DataCollectors>
      <DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0" assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
        <Configuration>
          <CodeCoverage>
            <ModulePaths>
              <Exclude>
                <ModulePath>.*CPPUnitTestFramework.*</ModulePath>
              </Exclude>
            </ModulePaths>


            <!-- Add the below node with value False to your runsettings: -->
            <UseVerifiableInstrumentation>False</UseVerifiableInstrumentation>


            <AllowLowIntegrityProcesses>True</AllowLowIntegrityProcesses>
            <CollectFromChildProcesses>True</CollectFromChildProcesses>
            <CollectAspDotNet>False</CollectAspDotNet>

          </CodeCoverage>
        </Configuration>
      </DataCollector>

      <DataCollector uri="datacollector://microsoft/VideoRecorder/1.0" assemblyQualifiedName="Microsoft.VisualStudio.TestTools.DataCollection.VideoRecorder.VideoRecorderDataCollector, Microsoft.VisualStudio.TestTools.DataCollection.VideoRecorder, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" friendlyName="Screen and Voice Recorder">
        <!--Video data collector was introduced in Visual Studio 2017 version 15.5 -->
      </DataCollector>

    </DataCollectors>
  </DataCollectionRunSettings>
</RunSettings>

CSmith1111 · 2019-03-20T10:57:39Z

This is my runsettings file, no change on the buildserver agent

<RunSettings>
  <DataCollectionRunSettings>
    <DataCollectors>
      <DataCollector friendlyName="Code Coverage" 
                     uri="datacollector://Microsoft/CodeCoverage/2.0"
                     assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Culture=neutral">
        <Configuration>
          <CodeCoverage>
            <ModulePaths>
              <Include>
              </Include>
              <Exclude>
                <ModulePath>.*AutoMapper.*</ModulePath>
                <ModulePath>.*BouncyCastle.*</ModulePath>
                <ModulePath>.*Caliburn.*</ModulePath>
                <ModulePath>.*DevExpress.*</ModulePath>
                <ModulePath>.*grace.*</ModulePath>
                <ModulePath>.*MsgReader.*</ModulePath>
                <ModulePath>.*Newtonsoft.*</ModulePath>
                <ModulePath>.*NLog.*</ModulePath>
                <ModulePath>.*Antlr3.*</ModulePath>
                <ModulePath>.*AutoMapper.*</ModulePath>
                <ModulePath>.*EnvDTE*.*</ModulePath>
                <ModulePath>.*fluentassertions.*</ModulePath>
                <ModulePath>.*fluentmigrator.*</ModulePath>
                <ModulePath>.*HibernatingRhinos.*</ModulePath>
                <ModulePath>.*Iesi.*</ModulePath>
                <ModulePath>.*Irony.*</ModulePath>
                <ModulePath>.*moq.*</ModulePath>
                <ModulePath>.*MailKit.*</ModulePath>
                <ModulePath>.*Microsoft.*</ModulePath>
                <ModulePath>.*MimeKit.*</ModulePath>
                <ModulePath>.*MsgReader.*</ModulePath>
                <ModulePath>.*netstandard.*</ModulePath>
                <ModulePath>.*NHibernate.*</ModulePath>
                <ModulePath>.*Npgsql.*</ModulePath>
                <ModulePath>.*Owin.*</ModulePath>
                <ModulePath>.*Remotion.*</ModulePath>
                <ModulePath>.*stdole.*</ModulePath>
                <ModulePath>.*System.*</ModulePath>
                <ModulePath>.*UnitsNet.*</ModulePath>
                <ModulePath>.*VSLangProj.*</ModulePath>
                <ModulePath>.*WebActivatorEx.*</ModulePath>
                <ModulePath>.*\.tests\.dll</ModulePath>
              </Exclude>
            </ModulePaths>
            <UseVerifiableInstrumentation>False</UseVerifiableInstrumentation>
            <AllowLowIntegrityProcesses>True</AllowLowIntegrityProcesses>
            <CollectFromChildProcesses>True</CollectFromChildProcesses>
            <CollectAspDotNet>False</CollectAspDotNet>
          </CodeCoverage>
        </Configuration>
      </DataCollector>
    </DataCollectors>
  </DataCollectionRunSettings>
</RunSettings>`

ShreyasRmsft · 2019-03-20T11:09:27Z

Can you set system.debug = true and share the full logs zip file?

CSmith1111 · 2019-03-20T11:59:41Z

i posted the installer and vstest logs already, do you need new logs? the problem is still the same

ShreyasRmsft · 2019-03-20T12:05:11Z

I need the entire zip please. We have additional diagnostic logs we publish there.

for instance this: /diag:"C:\Users\TFSbuild\AppData\Local\Temp\9e528890-4a57-11e9-8c2a-fbe322e0eac3.txt"

there are logs that come form the code coverage data collector also. Those would help with the investigation here. Thanks.

ShreyasRmsft · 2019-03-20T12:06:14Z

And also can you provide the entire set of logs for both the successful and the run where code coverage was not found. So that I can check for differences if any.

CSmith1111 · 2019-03-20T12:26:09Z

is there a private way to send you the logs?

ShreyasRmsft · 2019-03-21T05:27:50Z

you can send it to devops_tools@microsoft.com

CSmith1111 · 2019-03-21T10:09:34Z

i send you a email with a wetransfer.com download, your mailserver told me this when i try to send the logs directly to you

BatchSize Runsettings #550 5.3.4 ROUTING.SizeLimit; message size exceeds fixed maximum size for route
Enable Trx logger in TPv2 #5.7.133 smtp;550 5.7.133 RESOLVER.RST.SenderNotAuthenticatedForGroup; authentication required; Delivery restriction check failed because the sender was not authenticated when sending to this group

ShreyasRmsft · 2019-03-21T10:40:05Z

got it

ShreyasRmsft · 2019-03-21T10:41:16Z

It's failing because of this. You can find this in the 8_43b1e510-4b11-11e9-b119-03b0490c3407.datacollector.19-03-2.txt log file.

TpTrace Error: 0 : 6228, 1, 2019/03/20, 14:10:53.823, 18274459502187, datacollector.exe, DataCollectionEvents.SessionEnd: Exception occurred while calling handler of type Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector for SessionEndEventArgs: System.Reflection.TargetInvocationException: Ein Aufrufziel hat einen Ausnahmefehler verursacht. ---> System.IO.FileNotFoundException: Die Datei oder Assembly "Microsoft.VisualStudio.Coverage.Interop.dll" oder eine Abhängigkeit davon wurde nicht gefunden. Das angegebene Modul wurde nicht gefunden.
   bei Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollectorImpl.StopVanguard(DataCollectionContext context)
   bei Microsoft.VisualStudio.Coverage.UnitTestDataCollector.SessionEnd(Object sender, SessionEndEventArgs e)
   bei Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector.SessionEnd(Object sender, SessionEndEventArgs e)
   --- Ende der internen Ausnahmestapelüberwachung ---
   bei System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
   bei System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
   bei System.Delegate.DynamicInvokeImpl(Object[] args)
   bei Microsoft.VisualStudio.TestPlatform.Utilities.MulticastDelegateUtilities.SafeInvoke(Delegate delegates, Object sender, EventArgs args, String traceDisplayName)
TpTrace Error: 0 : 6228, 1, 2019/03/20, 14:10:54.549, 18274466774998, datacollector.exe, DataCollectionAttachmentManager.GetAttachments: Fail to get attachments: System.Collections.Generic.KeyNotFoundException: Der angegebene Schlüssel war nicht im Wörterbuch angegeben.
   bei System.ThrowHelper.ThrowKeyNotFoundException()
   bei System.Collections.Generic.Dictionary`2.get_Item(TKey key)
   bei Microsoft.VisualStudio.TestPlatform.Common.DataCollector.DataCollectionAttachmentManager.GetAttachments(DataCollectionContext dataCollectionContext)

CSmith1111 · 2019-03-21T10:43:14Z

so its a missing dependency. shouldn't that be installed with the installation script? if not, where can i find the setup?

ShreyasRmsft · 2019-03-21T11:03:20Z

The dll is present actually. The logs in 8_VsTest - testAssemblies.txt indicate that.

My suspicion is that on Windows 10 Ent. with Visual Studio 2017 Ent. it is loading this dll from the GAC and here it is unable to load this.

I think I know the reason.

https://github.com/Microsoft/azure-pipelines-tasks/blob/b22f64b916457afe3d8ca426dcd4c71ede651a9d/Tasks/VsTestV2/vstest.ts#L272

If you look at this code here, we add 2 more env vars which code coverage needs when running on a machine without VS.

To complicate things there is a known bug in TFS 2018 wherein if you tick the code coverage box (which enables these variables to get set) the default code coverage settings are used and your exclude filters etc. will not work. If you did not have custom coverage settings then ticking the enable code coverage checkbox should get you the coverage on a machine without VS installed. You can try this to validate my theory.

This is fixed in TFS2019 and above.

As a workaround I think you will manually need to set these environment variables.

If you go through the source code of the task they point to the location of the code coverage profiler dlls that were fetched in the vstestplatform tool installer task

Or the simpler option is to upgrade to TFS2019 where we had found and fixed the issue

CSmith1111 · 2019-03-21T13:13:43Z

Thx for your help, with you info its working now by installing the static code analyse tools over the build tools.
we don't want to switch to tfs2019 right now and setting the environment variables manually is not our preferred way to deal with this.

CSmith1111 closed this as completed Mar 21, 2019

singhsarab added the question label Apr 8, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

No CodeCoverage with Tools from Microsoft.TestPlatform / CodeCoverage only in Tempfolder #1963

No CodeCoverage with Tools from Microsoft.TestPlatform / CodeCoverage only in Tempfolder #1963

CSmith1111 commented Mar 19, 2019

ShreyasRmsft commented Mar 20, 2019 •

edited

Loading

CSmith1111 commented Mar 20, 2019

ShreyasRmsft commented Mar 20, 2019

CSmith1111 commented Mar 20, 2019

ShreyasRmsft commented Mar 20, 2019

ShreyasRmsft commented Mar 20, 2019

CSmith1111 commented Mar 20, 2019

ShreyasRmsft commented Mar 21, 2019

CSmith1111 commented Mar 21, 2019 •

edited

Loading

ShreyasRmsft commented Mar 21, 2019

ShreyasRmsft commented Mar 21, 2019 •

edited

Loading

CSmith1111 commented Mar 21, 2019

ShreyasRmsft commented Mar 21, 2019 •

edited

Loading

CSmith1111 commented Mar 21, 2019

No CodeCoverage with Tools from Microsoft.TestPlatform / CodeCoverage only in Tempfolder #1963

No CodeCoverage with Tools from Microsoft.TestPlatform / CodeCoverage only in Tempfolder #1963

Comments

CSmith1111 commented Mar 19, 2019

Description

Steps to reproduce

Expected behavior

Actual behavior

Diagnostic logs

Environment

ShreyasRmsft commented Mar 20, 2019 • edited Loading

CSmith1111 commented Mar 20, 2019

ShreyasRmsft commented Mar 20, 2019

CSmith1111 commented Mar 20, 2019

ShreyasRmsft commented Mar 20, 2019

ShreyasRmsft commented Mar 20, 2019

CSmith1111 commented Mar 20, 2019

ShreyasRmsft commented Mar 21, 2019

CSmith1111 commented Mar 21, 2019 • edited Loading

ShreyasRmsft commented Mar 21, 2019

ShreyasRmsft commented Mar 21, 2019 • edited Loading

CSmith1111 commented Mar 21, 2019

ShreyasRmsft commented Mar 21, 2019 • edited Loading

CSmith1111 commented Mar 21, 2019

ShreyasRmsft commented Mar 20, 2019 •

edited

Loading

CSmith1111 commented Mar 21, 2019 •

edited

Loading

ShreyasRmsft commented Mar 21, 2019 •

edited

Loading

ShreyasRmsft commented Mar 21, 2019 •

edited

Loading