Coveralls + Scala doesn't seem to work with GitHub Actions

[gzoller]

I tried using the Coveralls Github Action and received an error that it couldn't find the lcov.info file.

I've been using Coveralls with scoverage (via sbt plugin) for quite a while with Travis builds and wanted to see how Github Actions would work. Generating my coverage report doesn't appear to create a lcov.info file.

Is there a known pattern for getting code coverage for Scala into Coveralls via an Action? If not, this is a gap and issue.

[ruippeixotog]

Unfortunately, I tried asking both in coverallsapp/github-action#30 and in scoverage/sbt-coveralls#126 and I got no official answer. Seems like Coveralls is rather limited when it comes to Scala, despite announcing support for it in their webpage 🙁

[carueda]

oh, just hitting this issue in a project I'm "migrating" from travis to actions.. 🙁

[afinetooth]

@gzoller @ruippeixotog @carueda The task of integrating a scala project with coveralls can almost certainly be accomplished, but it looks like the community-created integration we're aware of, sbt-coverage with sbt-coveralls, does currently have some limitations; apparently with parallel builds and using service_job_id, which is the preferable method for identifying jobs through some CIs.

As it pertains to using the Coveralls Github Action, the limitation to plugging and playing that may just be that sbt-coverage doesn't generate coverage reports in LCOV format, which, as you've discovered @gzoller, is required by the Coveralls Github Action.

The next step, if you want to continue using the Coveralls Github Action, would be to find a way to convert the sbt-coverage report into LCOV, which you might be able to find off-the-shelf. You'll probably want to start back at scoverage and see if anyone's created any converters. Or perhaps use sbt-jacoco instead, which has some instructions on integrating with Coveralls.

If converting to LCOV proves too big a barrier, that won't prevent you from using Github Actions for CI without using the Coveralls Github Action. If you attempt that, reach out to support@coveralls.io with the issues you encounter and we'll try to help you.

Some guidelines to be aware of that may help as you explore different options:

• The Coveralls API Reference is your guide to understanding what the Coveralls API expects as inputs, in the form of JSON parameters, to process an incoming job as valid.
• Using verbose mode (debug mode) is your key to understanding what your integration is trying to send to the Coveralls API in the form of that JSON. As long as your integration has some debug flag or mode, you should enable it and be able to see debug info in your CI build logs. In some cases, the debug mode of an underlying integration or library may be available to add the debug output to your log if your current integration doesn't have a debug mode. If you're having trouble finding out how to enable verbose mode, email us.

And some details about using Github Actions vs Github Actions with the Coveralls Github Action:

• The Coveralls Github Action is one of the only coveralls integrations that requires the repo's GITHUB_TOKEN to identify it with the Coveralls API instead of the repo's coveralls repo token (sometimes referernced with the env var, COVERALLS_REPO_TOKEN).
• The Coveralls Github Action also co-opts the value github for the service_name JSON param (see Coveralls API Reference), such that when the Coveralls API sees this value for service_name on an incoming job, it assumes the job is coming via the Coveralls Github Action and tries to match it to an existing repo using secrets.GITHUB_TOKEN vs COVERALLS_REPO_TOKEN.
• If you're using Github Actions for CI without the Coveralls Github Action, you can continue using the COVERALLS_REPO_TOKEN as your identifier, you'll just need to make sure you're not passing github for service_name. In all likelyhood, you can override this by using the following env var in your GHA config yaml with a value like: COVERALLS_SERVICE_NAME=github-actions.
• Alternately, if you want to use the Coveralls Github Action, or you want to use Github Actions for CI without the Coveralls Github Action but you know you'll be passing service_name as github, you'll just want to make sure you're using {{ secrets.GITHUB_TOKEN }} as your identifier instead of the coveralls repo token, which you can set like this in your GHA config yaml: COVERALLS_REPO_TOKEN: ${{ secrets.GITHUB_TOKEN }}

[benthecarman]

Any update on getting parallel builds for sbt-coveralls? This issue has been open for over 3 years scoverage/sbt-coveralls#125