Feature Request: Post results as a comment in the PR

[lexicalunit]

I tried for a couple hours to get https://github.com/marketplace/actions/coveralls-github-action together with https://pypi.org/project/coveralls/ working correctly and it just never did.

But your action worked perfectly my first attempt! Thank you! 🎉

One thing that would be nice that I think coveralls-github-action does is that it posts a detailed comment to the PR with results linking to the coveralls report as well. Without this it's not obvious where to find the report, or that it even exists, unless you go looking for it on coveralls.io yourself.

[AndreMiras]

Thanks @lexicalunit,
Coud you share an example of how it should look like if you ever saw it working somewhere?

[lexicalunit]

Sadly I don't know of any examples. I'm just going off the documentation which says that it does make a comment with such details.

[AndreMiras]

Then it might be what we already support as we see in this screenshot:

Basically if the coverage is the same or increases it won't complain. But it still "comments" with a "Details" link.

[lexicalunit]

So those links work just fine, but those are only active when the checks are running or if one of the checks has failed. If the checks ran and ran without errors, then all those checks get rolled up and aren't immediately visible anymore. Also sometimes if you aren't using parallel tests you may have your coveralls step not broken out like that, so you'd only see one single check, and you'd have no indication that the test included any sort of coverage tests.

That's why most tools, including https://github.com/marketplace/actions/coveralls-github-action according to its own documentation, and indeed also codecov do things like you see in this PR: theastropath/turbot#68

Where the final step will be to create and/or update a comment in the PR (and only if the build is based on a PR, it wouldn't do it for a push event) with details about the run.

All I know is that https://github.com/marketplace/actions/coveralls-github-action claims to support this feature... I'm sure if it does there must be some project out there somewhere with an example of what their comment might look like.

[lexicalunit]

Ah, I found an example in the wild of a comment like this: repetere/jsonm#3 (comment)

[AndreMiras]

Very nice, I thought only codecov was capable of it. Have an example really helps. Well I'll look into it then

[ben-alkov]

With this config, I get the standard GH "CI box" (here, heading "Some checks were not successful"). The "Details" link from "coveralls_finish (pull_request)" just points to the Actions tab for the check, where there is no link to Coveralls.io, nor anything else useful, really (I verified that the results were actually posted correctly to Coveralls, although the "Commit Message" there (on Coveralls.io) is a garbled mess of HTML... 🤔).

Am I missing some necessary magic to get any link to Coveralls.io?

EIT: Self answer. It really does require coverage run --source=foo_src -m pytest tests/ instead of e.g. pytest --cov=foo_src tests/