[BUG] Uploading test results from GitHub Actions CI/CD fails (in some repos)

[webknjaz]

I don't actually know if it's because the repo is private, that's just a wild guess.
Here's the log:

Run codecov/test-results-action@v1
  with:
    files: .tox/.tmp/.test-results/pytest-3.12/test.xml
  env:
    COLOR: yes
    FORCE_COLOR: 1
    MYPY_FORCE_COLOR: 1
    PIP_DISABLE_PIP_VERSION_CHECK: 1
    PIP_NO_PYTHON_VERSION_WARNING: 1
    PIP_NO_WARN_SCRIPT_LOCATION: 1
    PRE_COMMIT_COLOR: always
    PY_COLORS: 1
    PYTHONIOENCODING: utf-8
    PYTHONUTF8: 1
    TOX_PARALLEL_NO_SPINNER: 1
    TOX_TESTENV_PASSENV: COLOR FORCE_COLOR MYPY_FORCE_COLOR NO_COLOR PIP_DISABLE_PIP_VERSION_CHECK PIP_NO_PYTHON_VERSION_WARNING PIP_NO_WARN_SCRIPT_LOCATION PRE_COMMIT_COLOR PY_COLORS PYTEST_THEME PYTEST_THEME_MODE PYTHONIOENCODING PYTHONLEGACYWINDOWSSTDIO PYTHONUTF8
    TOXENV: py
    pythonLocation: /Users/runner/hostedtoolcache/Python/3.12.4/x64
    PKG_CONFIG_PATH: /Users/runner/hostedtoolcache/Python/3.12.4/x64/lib/pkgconfig
    Python_ROOT_DIR: /Users/runner/hostedtoolcache/Python/3.12.4/x64
    Python2_ROOT_DIR: /Users/runner/hostedtoolcache/Python/3.12.4/x64
    Python3_ROOT_DIR: /Users/runner/hostedtoolcache/Python/3.12.4/x64
==> macos OS detected
https://cli.codecov.io/latest/macos/codecov.SHA256SUM
==> Running version latest
gpg: key 806BB28AED779869: "Codecov Uploader (Codecov Uploader Verification Key) <security@codecov.io>" not changed
gpg: Total number processed: 1
gpg:              unchanged: 1

gpg: Signature made Fri Aug 16 23:23:25 2024 UTC
gpg:                using RSA key 27034E7FDB850E0BBC2C62FF806BB28AED779869
gpg: Good signature from "Codecov Uploader (Codecov Uploader Verification Key) <security@codecov.io>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 2703 4E7F DB85 0E0B BC2C  62FF 806B B28A ED77 9869

==> Running version v0.7.4
==> Running command '/Users/runner/work/_actions/codecov/test-results-action/v1/dist/codecov do-upload'
/Users/runner/work/_actions/codecov/test-results-action/v1/dist/codecov do-upload -f .tox/.tmp/.test-results/pytest-3.12/test.xml --report-type test_results
==> Uploader SHASUM verified (081d43f3fcd8176dc97acf6c77aac989a74383b9f0c1e7bdc56498a920720255  codecov)
info - 2024-08-20 20:25:50,370 -- ci service found: github-actions
info - 2024-08-20 20:25:50,435 -- Found 1 test_results files to report
info - 2024-08-20 20:25:50,435 -- > /Users/runner/work/<repo-name>/<repo-name>/.tox/.tmp/.test-results/pytest-3.12/test.xml
info - 2024-08-20 20:25:50,534 -- Process Upload complete
error - 2024-08-20 20:25:50,534 -- Upload failed: {"service":["This field may not be null."]}

I'm not reporting the bug against the action repo because I see in the log that it fails within codecov-cli. The junit file is produced by pytest and is found correctly. However, uploading fails with a This field may not be null. which suspiciously reminds me of errors Django would generate when working with the DB.
So my assumption is that the error is coming from the backend, which isn't something I'd be able to debug myself.

Hopefully, this gives you a good starting point. Let me know if you want some additional repro details!

[webknjaz]

The repo in question is now public, and the uploads are still failing the same way: https://github.com/ansible/awx-plugins/actions/runs/10581340011/job/29318439675#step:23:47.

@thomasrockhu-codecov any chance you could take a look? The token is set in the config file rather than a secret. The coverage upload succeeds, the tests upload fails.

[webknjaz]

UPD: I tried it out locally with an editable installation of this repo. I added an entry point to https://github.com/codecov/codecov-cli/blob/ae3fc7e/codecov_cli/helpers/request.py#L160 and it succeeded!

$ some-venv-py313/bin/codecovcli do-upload -f .tox/.tmp/.test-results/pytest-3.13/test.xml -F local-debugging --report-type test_results       
info - 2024-08-27 18:24:31,788 -- ci service found: local
info - 2024-08-27 18:24:32,481 -- Found 1 test_results files to report
info - 2024-08-27 18:24:32,481 -- > ~/src/github/ansible/awx-plugins/.tox/.tmp/.test-results/pytest-3.13/test.xml
info - 2024-08-27 18:24:33,794 -- Process Upload complete

The obvious difference in the log is

- info - 2024-08-20 20:25:50,370 -- ci service found: github-actions
+ info - 2024-08-27 18:24:31,788 -- ci service found: local

which leaves me puzzled... What could influence the process @ GHA env that's not present locally?

[webknjaz]

Extra observation: the job is in a reusable workflow and does not have a Git checkout (the source is coming from an sdist file).
Doing mv -v .git{,.bak} locally, makes it break differently. And --commit-sha reveals another traceback.

[joseph-sentry]

hi @webknjaz, i'm looking into this, I think you're right that this issue is due to something going wrong on the backend. I think I have a fix for it here: codecov/codecov-api#779

explanation: I was using the "git_service" arg of the upload instead of the "ci_service" param, ci_service is usually automatically set, i think git_service depends on you uploading from a git repo.

This should at least fix the current This field may not be null error

[webknjaz]

Got it, thanks!