Run tests fail intermittently with a disconnected from server exception.

[AbhitejJohn]

Description

Running a large set of tests sometimes throws the following exception from the Test Platform:
"Error: An exception occurred while invoking executor 'executor://mstestadapter/v2': Object '/ddbf5695_3650_4819_8ba2_77ea8202b458/ec2ln5fvtuhgu_udubuqgkpz_27.rem' has been disconnected or does not exist at the server. "

Steps to reproduce:

This is observed when running a large set of tests on a machine with lower config.

Expected behavior

The tests should all run.

Actual behavior

They intermittently fail mid-way.

[AbhitejJohn]

This is most likely tied to us not setting a life time service for the objects we marshal across the app domain.

[smalgin]

Hate to break the news, but I just got this intermittent failure happen on VSTS.

Error: An exception occurred while invoking executor 'executor://mstestadapter/v2': Object '/782fd4c0_59e2_45df_bcfc_589cfdf6b164/vuahegdjhrpj2hrwzqbadpw+_27.rem' has been disconnected or does not exist at the server.

Let me know if I can help with any extra information.

[jayaranigarg]

@smalgin : Which version of adapter and framework are you using?

[smalgin]

@jayaranigarg here's relevant VSTS log

017-12-27T18:06:44.5162987Z ==============================================================================
2017-12-27T18:06:44.5163174Z Task : Visual Studio Test
2017-12-27T18:06:44.5163331Z Description : Run tests with Visual Studio test runner
2017-12-27T18:06:44.5163498Z Version : 2.3.4
2017-12-27T18:06:44.5163642Z Author : Microsoft Corporation
2017-12-27T18:06:44.5163823Z Help : More Information
2017-12-27T18:06:44.5164023Z ==============================================================================
2017-12-27T18:06:45.2982011Z Run the tests locally using vstest.console.exe
2017-12-27T18:06:45.2982220Z ========================================================
2017-12-27T18:06:45.2982535Z Test selector : Test assemblies
2017-12-27T18:06:45.2983009Z Test assemblies : **\AGOL.UnitTests.dll
2017-12-27T18:06:45.2983205Z Test filter criteria : null
2017-12-27T18:06:45.2983426Z Search folder : d:\a\1\s\Main\BuildOutput\UnitTestBinaries
2017-12-27T18:06:45.2983691Z Run settings file : d:\a\1\s\Main\AGOL.UnitTests\AGOL.UnitTests\test.runsettings
2017-12-27T18:06:45.2983975Z Run in parallel : false
2017-12-27T18:06:45.2984159Z Run in isolation : false
2017-12-27T18:06:45.2985482Z Path to custom adapters : null
2017-12-27T18:06:45.2985704Z Other console options : null
2017-12-27T18:06:45.2985918Z Code coverage enabled : false
2017-12-27T18:06:45.2986340Z Rerun failed tests: false
2017-12-27T18:06:45.2986653Z VisualStudio version selected for test execution : latest
2017-12-27T18:06:46.6282007Z ========================================================
2017-12-27T18:06:46.7029111Z [command]"C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" @d:\a_temp\b3cf8a11-eb30-11e7-a930-b5ba0dfefc2d.txt
2017-12-27T18:06:53.7341332Z Microsoft (R) Test Execution Command Line Tool Version 15.0.26929.2
2017-12-27T18:06:53.7341634Z Copyright (c) Microsoft Corporation. All rights reserved.
2017-12-27T18:06:53.7341732Z
2017-12-27T18:06:53.7341867Z vstest.console.exe
2017-12-27T18:06:53.7342020Z "d:\a\1\s\Main\BuildOutput\UnitTestBinaries\AGOL.UnitTests.dll"
2017-12-27T18:06:53.7342193Z /Settings:"d:\a\1\s\Main\AGOL.UnitTests\AGOL.UnitTests\test.runsettings"
2017-12-27T18:06:53.7342499Z /logger:"trx"
2017-12-27T18:06:53.7342662Z /TestAdapterPath:"d:\a\1\s\Main\BuildOutput\UnitTestBinaries"
2017-12-27T18:06:56.7324052Z Starting test execution, please wait...
2017-12-27T18:06:57.4887193Z Warning: Using Isolation mode to run tests as required by effective Platform:X64 and .Net Framework:Framework45 settings for test run. Use the /inIsolation parameter to suppress this warning.
2017-12-27T18:07:19.3184750Z Passed aaa
. . . skipped some passed tests
2017-12-27T18:12:29.5842147Z Passed bbb
2017-12-27T18:12:29.6321068Z Error: An exception occurred while invoking executor 'executor://mstestadapter/v2': Object '/782fd4c0_59e2_45df_bcfc_589cfdf6b164/vuahegdjhrpj2hrwzqbadpw+_27.rem' has been disconnected or does not exist at the server.
2017-12-27T18:12:29.6321631Z
2017-12-27T18:12:30.2256934Z Results File: d:\a\1\s\Main\BuildOutput\UnitTestBinaries\TestResults\VssAdministrator_factoryvm-az176 2017-12-27 18_07_19.trx
2017-12-27T18:12:30.2860869Z
2017-12-27T18:12:30.2861657Z Total tests: 26. Passed: 26. Failed: 0. Skipped: 0.
2017-12-27T18:12:30.2861903Z Test Run Failed.
2017-12-27T18:12:30.2862438Z Test execution time: 5.5201 Minutes
2017-12-27T18:12:35.0874412Z ##[warning]Vstest failed with error. Check logs for failures. There might be failed tests.
2017-12-27T18:12:35.2569953Z ##[error]Error: C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe failed with return code: 1
2017-12-27T18:12:37.3890686Z ##[error]VsTest task failed.
2017-12-27T18:12:37.4339480Z ##[section]Async Command Start: Publish test results
2017-12-27T18:12:37.4791011Z Publishing test results to test run '57'
2017-12-27T18:12:37.4791283Z Test results remaining: 26. Test run id: 57
2017-12-27T18:12:38.1175647Z Published Test Run : ccc
2017-12-27T18:12:38.1176111Z ##[section]Async Command End: Publish test results
2017-12-27T18:12:38.1177433Z ##[section]Finishing: Run AGOL.UnitTests

[jayaranigarg]

@smalgin: Thank you for sharing the logs.
In your UnitTestProject i.e (\UnitTestBinaries\AGOL.UnitTests.dll) can you upgrade the the nuget packages MSTest.TestAdapter and MSTest.TestFramework to point to latest version 1.2.0 ans see if you still face this issue?
If possible can you share a repro project with us? What are the machine configurations your tests are running on in VSTS?

[smalgin]

@jayaranigarg I upgraded my nuget packages to 1.2.0. I will let you know if we encounter this issue again. Unfortunately, we are going off VSTS for that specific project, so I will be most likely not able to test this.

[TsuyoshiUshio]

Hi @jayaranigarg I encounter the same issue with Azure DevOps. According to your advice I switch from the 1.1.11 to 1.2.0. It works. :) Thanks.

[ykelanemer]

Hi,
I got the same error:
Error] An exception occurred while invoking executor 'executor://mstestadapter/v2': Object '/xxxxxxxxxxxxxxxxxxxxxxxx/fzkml+xnaloancmasqkrp2mv_9.rem' has been disconnected or does not exist at the server.
I upgraded the MSTest.TestAdapter and MSTest.TestFramework to point to version 1.2.0 and the issue was fixed.
Thanks a lot!