Explicit output format always outputs JSON but only on one machine

[CalvinWilkinson]

Issue Description

I am currently going open source with one of my projects and using/learning GitHub actions for the first time.
I am coming from Azure DevOps with a lot of experience with pipelines as well. During this process, I am at the point of generating code coverage reports that can be consumed by either Coveralls or CodeCov.

Once I got it all set up and building using my own hosted agents on my build server, I noticed that the report being generated is in the JSON format even though I explicitly used the dotnet test command to use the lcov format. I tried other formats as well but no matter what format I use, it generates it in JSON format.

The next thing I tried is I manually tried to run the dotnet test command on the project in the build agent itself on my build server to see if I could get it work properly and of course it did not work, because I did not have the .NET SDK 5.0.x installed. So I manually installed the SDK so I could do some troubleshooting and attempted the command again. This time the command of course ran successfully, but I still had the same behavior as the agent itself and the coverage report was still in JSON format even though my command was explicitly using the lcov format.

At this point I tried running the dotnet test command on my local dev machine and low and behold, it works just fine with no issue.

I also tried this using a GitHub Hosted server instead of my server, and it still does not work.

Everything points to some kind of difference from the machines and that I am doing everything correctly.
Any guidance or help on this issue would be greatly appreciated. 🙂

Commands Used/Attempted:

dotnet test MyProject.csproj /p:CollectCoverage=true /p:CoverletOutput=TestResults/ /p:CoverletOuputFormat=lcov
and
dotnet test MyProject.csproj /p:CollectCoverage=true /p:CoverletOutput=TestResults/ /p:CoverletOuputFormat=cobertura

---

Workflow YAML Being Used:

name: .NET

on:
  workflow_dispatch:
    inputs:
      name:
        description: "Manual Run"
        required: true
        
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  Build_Project:
    runs-on: windows-latest
    steps:
    - uses: actions/checkout@v2
    - name: Setup .NET 5.0.x
      uses: actions/setup-dotnet@v1
      with:
        dotnet-version: 5.0.x
    - name: Restore Dependencies
      run: dotnet restore
    - name: Build
      run: dotnet build --no-restore
  
  Run_Unit_Tests:
    runs-on: windows-latest
    steps:
      - uses: actions/checkout@v2
      - name: Setup .NET 5.0.x
        uses: actions/setup-dotnet@v1
        with:
          dotnet-version: 5.0.x
      - name: Restore Dependencies
        run: dotnet restore
      - name: Execute Unit Tests
        run: dotnet test "${{ github.workspace }}\\Testing\\CASLTests\\CASLTests.csproj" /p:CollectCoverage=true /p:CoverletOutput=TestResults/ /p:CoverletOuputFormat=lcov
      - name: Publish Code Coverage Results
        uses: coverallsapp/github-action@v1.1.2
        with:
          github-token: ${{ secrets.COVERALLS_TOKEN }}
          path-to-lcov: "${{ github.workspace }}\\Testing\\CASLTests\\TestResults"

---

Local Machine Environment

• Windows 10 20H2 (OS Build 19042.985)
• .NET SDK 5.0.300 (x64) from Visual Studio

---

Personal Build Server Environment

• Windows 10 20H2 (OS Build 19042.985)
• Do not have Visual Studio or any **.NET SDK's ** installed
  • Not required. I use the setup-dotnet@v1 GitHub action

---

GitHub Hosted Environment

• I use the latest windows

---

Steps to reproduce

I suppose you could setup an agent on your local dev machine, setup one using the GitHub hosted server using windows-latest and then run the dotnet test command on the dev machine pointed to the repo of the project.

Expected Behavior

When running the dotnet test command with the proper code coverage parameters, the expected output format will be produced when it is a GitHub hosted server, a personal machine with a setup agent, or simply from the project on a local dev machine.

Actual Behavior

1. Generated output when running dotnet test command for project on local dev machine works just fine
2. Generated output when running dotnet test command for project on personal build server with agent only generates JSON format
3. Generated output when running dotnet test command using GitHub Hosted server only generates JSON format

[artoxian]

Did you get any resolution to this?

I'm seeing the same behaviour on my PC. I can get a "coverage.json" file to be created by using "/p:CollectCoverage=true", but all other coverlet related parameters don't seem to be respected:

dotnet test /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput=coverage.cobertura.xml

In the above example, neither the output foramt is changed nor the output name.

[jimdeane]

I've been messing with this for hours too. I get exactly the same as artoxian. set up a tiny project from a turorial and get coverage.json but never anythging else.

[petli]

@artoxian @jimdeane Are you using a .NET 7 SDK? If so, you might be affected by the problem in dotnet test that's being tracked in #1391 (which links to the vstest issue that includes some workarounds).

[daveMueller]

@artoxian @jimdeane, the issue was fixed by the microsoft guys with sdk 7.0.101 that was released last week microsoft/vstest#4014. I just checked it with a repro from another related issue and it works fine. Please give it a try again.

[daveMueller]

I'm closing this as the external issue was fixed and already released. Feel free to reopen.