coveralls crashes when there is GITHUB_{ACTIONS,TOKEN} but no GITHUB_REF in the environment (e.g. with tox)

[hartwork]

This is…

• with GITHUB_ACTIONS=true and GITHUB_TOKEN set but
• without variable GITHUB_REF:

# coveralls
Submitting coverage to coveralls.io...
Traceback (most recent call last):
  File "[..]/.tox/py38-coverage/bin/coveralls", line 8, in <module>
    sys.exit(main())
  File "[..]/.tox/py38-coverage/lib/python3.8/site-packages/coveralls/cli.py", line 95, in main
    result = coverallz.wear()
  File "[..]/.tox/py38-coverage/lib/python3.8/site-packages/coveralls/api.py", line 254, in wear
    json_string = self.create_report()
  File "[..]/.tox/py38-coverage/lib/python3.8/site-packages/coveralls/api.py", line 332, in create_report
    data = self.create_data()
  File "[..]/.tox/py38-coverage/lib/python3.8/site-packages/coveralls/api.py", line 387, in create_data
    self._data.update(git_info())
  File "[..]/.tox/py38-coverage/lib/python3.8/site-packages/coveralls/git.py", line 80, in git_info
    branch = git_branch()
  File "[..]/.tox/py38-coverage/lib/python3.8/site-packages/coveralls/git.py", line 36, in git_branch
    github_ref.startswith('refs/heads/')
AttributeError: 'NoneType' object has no attribute 'startswith'

Seen live today at https://github.com/asyncee/django-easy-select2/runs/4619646186?check_suite_focus=true#step:5:254 .

For a workaround in context of tox inside GitHub Actions:

1. Add passenv to the related environment in tox.ini:

   passenv = GITHUB_*
   

2. Invoke with --service=github to workaround issue Coveralls fails to upload with HTTP 422 "Unprocessable Entity" on retried builds #1435

[afinetooth]

@hartwork thank you. Can I ask for a clarification?:

Are you using one of the community-created python integrations for coveralls, or are you creating your own integration to be used via Github Actions? I'm not seeing any references in your source code, so I'm curious how to best contextualize this. TIA.

[hartwork]

Hi @afinetooth , happy to clarify. The integration is hand-made and part of a pull request — https://github.com/asyncee/django-easy-select2/pull/96/files — that has not been merged to master, yet. That's why you didn't see it. While building that manually from the ground up, I hit the issue above.

[afinetooth]

Ok, @hartwork, got it. Thanks.

One thing might be worth mentioning if you want to continue building upon your integration:

All integrations except one use the Coveralls repo token (env var: COVERALLS_REPO_TOKEN) to identify a repo to the Coveralls API (in POSTs to the /jobs endpoint). That one integration is the Coveralls Github Action, which uses the Github secret token (${{ secrets.GITHUB_TOKEN }}) to identify the repo.

Given that you are creating this integration to use Github Actions, but may want to extend it to other CI services, it might be worth changing the value you're sending for service_name. The reason is that the value service_name: github is "reserved" by the Coveralls Github Action, insofar as, by passing this value to the Coveralls API, it's letting the API know the POST is coming via the Coveralls Github Action, and the API executes some logic specific to what is expected from that integration.

The first such example being that, when the API receives service_name: github it looks for the GITHUB_TOKEN rather than the Coveralls repo token, to identify your repo.

A more generic implementation that would play with all CI services would be to change the service_name value to anything besides github, such as github-actions, and then, for the repo_token parameter, provide the value for the Coveralls repo token (found on your Coveralls repo page), and/or pass it in the env var COVERALLS_REPO_TOKEN.

Let me know if I can clarify that further or if you have any further questions.

[hartwork]

Hi @afinetooth,

thanks for your explanation. I'm only starting to understand why you suggest avoiding --service=github. (I have only settled on --service=github because it made the HTTP status 422 error go way, as you know from #1435). I would try trading --service=github by specifying COVERALLS_REPO_TOKEN instead but I don't have access to Coveralls project https://coveralls.io/github/asyncee/django-easy-select2 myself, and someone with permissions (CC @asyncee) would need to add that value as a named GitHub secret to the GitHub repository, it seems.

I looked at migrating to mentioned GitHub Action coverallsapp/github-action now, but learned that it only supports LCOV files (which is C land) while I have a .coverage file (which is Python land and SQLite technically). So moving to that GitHub Action doesn't seem to be an option.

That additional logic for magic service name github that you mention: is it in or against our favor? Does it break anything for https://github.com/asyncee/django-easy-select2 that I am not aware of? I am already wondering how to get those comment posts to pull requests back that we seem to have lost somehow, if possible. Is that related? Any ideas?

Best, Sebastian

[asyncee]

@hartwork i can give your access to coveralls of you need to tune django-easy-select2 configuration.

[hartwork]

@asyncee it will allow me to experiment with the above. Thank you!

[asyncee]

@hartwork unfortunately i found no way to give you access to coveralls via it's UI. So I added project's token COVERALLS_REPO_TOKEN to repository secrets.

[hartwork]

@asyncee thanks! Looks like the transition worked, see asyncee/django-easy-select2#97 .

[afinetooth]

@hartwork (Cc: @asyncee), it looks like you've got this worked out, but let me answer some open questions for future reference:

@asyncee in the future, you can add a github user as a collaborator to your project, or to its org so it has access, and, at that point, the user will have access to the repo at Coveralls.io and be able to see the repo token. If you, additionally, give that user Admin access to the repo, they will have the ability to modify the repo's settings at coveralls.io, including changing its repo token.

@hartwork to answer your questions / comments:

[...] I don't have access to Coveralls project https://coveralls.io/github/asyncee/django-easy-select2 myself, and someone with permissions (CC @asyncee) would need to add that value as a named GitHub secret to the GitHub repository, it seems.

I understand, and given that constraint, you could have implemented your solution with --service_name=github (or service=github if that's how you're having your users declare it), it's use would just be limited to Github Actions for CI. On the other hand, you could get around that by also implementing a version with COVERALLS_REPO_TOKEN as the identifier, for use with other CI systems; you'd just have the extra coding and the extra effort of providing usage instructions for the two different contexts in your README.

If @asyncee had not supplied the repo token, those would have been your options.

I looked at migrating to mentioned GitHub Action coverallsapp/github-action now, but learned that it only supports LCOV files (which is C land) while I have a .coverage file (which is Python land and SQLite technically). So moving to that GitHub Action doesn't seem to be an option.

That is true, the Coveralls Github Action currently requires coverage reports in LCOV format.

FWIW, the user from this comment reports success using coveragepy-lcov with the Coveralls Github Action.

That additional logic for magic service name github that you mention: is it in or against our favor? Does it break anything for https://github.com/asyncee/django-easy-select2 that I am not aware of? I am already wondering how to get those comment posts to pull requests back that we seem to have lost somehow, if possible. Is that related? Any ideas?

It's in your favor / doesn't break anything, if you're only using Github Actions for CI.

The rest of the "magic" is small and mostly limited to the API being able to render a correct link back to your Github Actions workflow next to the job on your build page.

Let me check out the status of your pull request comments to see if there's an issue I can help resolve. BRB here.

Update:
I'm seeing that PR Comments (with valid IDs) were created for your last three (2) pull_request builds:

Order: DATE DESC:

1. Build 1628080224: https://coveralls.io/builds/45251883 (PR: 98 - Comment ID: 1001714095)
2. Build 1624475090: https://coveralls.io/builds/45238604 (PR: 97 - Comment ID: 1001207817)

So it looks like PR Comments are working.

There was no comment sent for Build 1616664368 (PR 96):
https://coveralls.io/builds/45218685

But that's because we can't find a base build for that pull_request build. Which is why the top of your coverage table reads: FIRST BUILD ON RESTORE-COVERALLS AT 100.0%.

That just means we did not receive a coverage report for the commit on the branch that PR was meant to be merged into (likely master).

But again, it looks like PR Comments are working, the issue is just limited to that build. If I can help you fix that, lmk.

[hartwork]

@afinetooth coveragepy-lcov is an interesting pointer. Thanks for elaborating 🙏

[afinetooth]

@hartwork you bet. Just replied re: PR Comments above. :)

[hartwork]

@afinetooth we have moved away from --service=github (and GITHUB_TOKEN) to COVERALLS_REPO_TOKEN as suggested in the mean time and now today we're getting error coveralls.exception.CoverallsException: Running on Github Actions but GITHUB_TOKEN is not set. Add "env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}" to your step config. That's not cool 😃 Why?! 😃

CC @asyncee

[afinetooth]

Hi @hartwork.

From the PR you shared, I think the issue might be that you're not declaring any value for --service and something along the way is choosing github as the value, such that github is acting as the default.

That would explain why the integration complains, "Running on Github Actions but GITHUB_TOKEN is not set."

I'm hoping you can just pass a different explicit value for --service and it will work.

In other words, change this line:

bash -c 'set -x; [[ $\{GITHUB_ACTIONS:-false\} = false ]] || coveralls --service=github'

To:

bash -c 'set -x; [[ $\{GITHUB_ACTIONS:-false\} = false ]] || coveralls --service=github-actions'

Let me know if that doesn't fix it.

[hartwork]

Hi @afinetooth thanks for your reply! I had a look at the causing code in coveralls-python now. Because it's detection centers around variable GITHUB_ACTIONS, I have decided to take that variable away from it now, and that seems to work well. As a result, it should send "coveralls-python" for a service name now, and it did stop complaining. So the symptom is fixed.

What I don't get is how that code that was introduced in this coveralls-pyrthon commit for release 3.0.0 in 2021-01 did not break PR asyncee/django-easy-select2#97 back in 2021-12 but broke PRs yesterday with supposedly the exact same code. Weird 🤔 Any ideas?

[afinetooth]

Hi, @hartwork. I would have to see the PR associated with this build log, but I suspect the difference relates to the order of configuration precedence the author changed in that commit.

That is to say that, declaring the service in an env var in your GHA config yaml is not the same as declaring it as a flag in your command line command. I don't know exactly what the order of precedence is for that integration, but I suspect that declaring it in your command line is the last word and that, if you don't declare it there, some other value---an env var you're passing, or a default in the code---is getting used instead.

[hartwork]

@afinetooth I think you're arguing for --service over variables. Let me think about that.

[afinetooth]

@hartwork I'm actually agnostic. I think you could also use an environment variable (suggested variable names: COVERALLS_SERVICE_NAME or CI_NAME per the conventions set in the node-coveralls integration), or a .coveralls.yml file in the the root of your project with the service_name key, and either one would effectively override defaults in the code.

But --service is a good option if it simplifies your usage instructions (especially if your users don't need to mess with any other env vars or a new config file for any other reasons).

[stale]

This issue has been automatically marked for closure because it has not had recent activity. It will be closed if no further activity occurs. If your issue is still active please add a comment and we’ll review as soon as we can. Thank you for your contributions.