Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dotnet test should accept an argument to overwrite testresults folder #299

Closed
sbaid opened this issue Dec 22, 2016 · 3 comments
Closed

dotnet test should accept an argument to overwrite testresults folder #299

sbaid opened this issue Dec 22, 2016 · 3 comments
Assignees
Labels

Comments

@sbaid
Copy link
Contributor

sbaid commented Dec 22, 2016

Description

vstest.console creates TestResults folder in the working directory to place test results. This folder is used by logger extension like logger:trx to create log file under it. This test results folder is overrideable using runsettings.

    <?xml version="1.0" encoding="utf-8"?>
    <RunSettings>
      <!-- Configurations that affect the Test Framework -->
      <RunConfiguration>
        <ResultsDirectory>c:\temp\TestResults</ResultsDirectory>
      </RunConfiguration>
    </RunSettings>

It need to be exposed as a first class argument --ResultsDirectory|/ResultsDirectory

dotnet test need to also accept such argument -rd | --resultsDirectory
but it will build a folder structure below it to support multi TFM. These nested folders will likely be passed by dotnet test to vstest using runsettings or --ResultsDirectory

   <PropertyGroup>
      <TargetFrameworks>netstandard1.4;net46</TargetFrameworks>
    </PropertyGroup>

In such cases - test results for multiple test runs one for each TFM needs to be placed under separate folders like build.

    C:\temp\b\bin\Debug\net46
    C:\temp\b\bin\Debug\netstandard1.4
    C:\temp\b\bin\Debug\net46\b.dll
    C:\temp\b\bin\Debug\net46\b.pdb
    C:\temp\b\bin\Debug\net46\Microsoft.Win32.Primitives.dll
    C:\temp\b\bin\Debug\net46\System.AppContext.dll
    C:\temp\b\bin\Debug\net46\System.Console.dll
    C:\temp\b\bin\Debug\net46\System.Diagnostics.DiagnosticSource.dll
    C:\temp\b\bin\Debug\net46\System.Globalization.Calendars.dll
    C:\temp\b\bin\Debug\net46\System.IO.Compression.dll
    C:\temp\b\bin\Debug\net46\System.IO.Compression.ZipFile.dll
    C:\temp\b\bin\Debug\net46\System.IO.FileSystem.dll
    C:\temp\b\bin\Debug\net46\System.IO.FileSystem.Primitives.dll
    C:\temp\b\bin\Debug\net46\System.Net.Http.dll
    C:\temp\b\bin\Debug\net46\System.Net.Sockets.dll
    C:\temp\b\bin\Debug\net46\System.Runtime.InteropServices.RuntimeInformation.dll
    C:\temp\b\bin\Debug\net46\System.Security.Cryptography.Algorithms.dll
    C:\temp\b\bin\Debug\net46\System.Security.Cryptography.Encoding.dll
    C:\temp\b\bin\Debug\net46\System.Security.Cryptography.Primitives.dll
    C:\temp\b\bin\Debug\net46\System.Security.Cryptography.X509Certificates.dll
    C:\temp\b\bin\Debug\netstandard1.4\b.deps.json
    C:\temp\b\bin\Debug\netstandard1.4\b.dll
    C:\temp\b\bin\Debug\netstandard1.4\b.pdb

Suggest folder structure like this.

    C:\temp\b\TestResults\net46
    C:\temp\b\TestResults\netstandard1.4
    C:\temp\b\TestResults\net46\sbaid_SBAID-LAPTOP1 2016-12-22 15_30_37.trx
    C:\temp\b\TestResults\netstandard1.4\sbaid_SBAID-LAPTOP1 2016-12-22 15_30_37.trx

Steps to reproduce

What steps can reproduce the defect?
Please share the setup, commandline for vstest.console, sample project, target
framework etc.

Expected behavior

Share the expected output

Actual behavior

What is the behavior observed?

Environment

Please share additional details about the test environment.
Operating system, Build version of vstest.console

@sbaid
Copy link
Contributor Author

sbaid commented Dec 22, 2016

@codito @smadala @pvlakshm @piotrpMSFT to read & comment

@codito
Copy link
Contributor

codito commented Dec 23, 2016

+1 on the idea. Tagging @blackdwarf for review as well.

@blackdwarf
Copy link

Looks good to me. For dotnet test, the switch should be properly titled --results-directory.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants