Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bug] (v4) Unable to upload to same artifact name from multiple jobs #478

Closed
DanTup opened this issue Dec 18, 2023 · 49 comments
Closed

[bug] (v4) Unable to upload to same artifact name from multiple jobs #478

DanTup opened this issue Dec 18, 2023 · 49 comments
Labels
bug Something isn't working

Comments

@DanTup
Copy link

DanTup commented Dec 18, 2023

(Edit: There are solutions posted in this thread such as at #478 (comment), #478 (comment), #478 (comment))


What happened?

The PR from dependabot to upgrade to v4 is failing on my project with this error:

Error: Failed to CreateArtifact: Received non-retryable error: Failed request: (409) Conflict: an artifact with this name already exists on the workflow run

It seems like this is a breaking change that wasn't mentioned in the changelog and I'm not sure if it was deliberate.

There's some discussion about this behaviour in #279 and it suggests that it was fine to do this and there wouldn't be issues as long as the filenames within the artifact are unique. This was convenient to bundle the logs from several shards together into a single artifact rather than having lots of individual zip files to download.

What did you expect to happen?

I expected everything to work the same as in v3 unless it was noted as a deliberate breaking change.

How can we reproduce it?

Create multiple jobs that upload artifacts with the same name (but the files from each job are uniquely named).

Anything else we need to know?

No response

What version of the action are you using?

v4.0.0

What are your runner environments?

linux, window, macos

Are you on GitHub Enterprise Server? If so, what version?

No response

@DanTup DanTup added the bug Something isn't working label Dec 18, 2023
@DanTup
Copy link
Author

DanTup commented Dec 18, 2023

Actually, it seems this is called out here:

https://github.com/actions/upload-artifact?tab=readme-ov-file#v4---whats-new:~:text=The%20contents%20of%20an%20Artifact%20are%20uploaded%20together%20into%20an%20immutable%20archive.%20They%20cannot%20be%20altered%20by%20subsequent%20jobs.%20Both%20of%20these%20factors%20help%20reduce%20the%20possibility%20of%20accidentally%20corrupting%20Artifact%20files.

The contents of an Artifact are uploaded together into an immutable archive. They cannot be altered by subsequent jobs. Both of these factors help reduce the possibility of accidentally corrupting Artifact files.

It just wasn't included in the "What's changed" section of the Dependabot release notes because it just has a summary saying "Lots has changed". I should've followed the link through.

Seems like this is certainly intended though.

@RobertoPrevato
Copy link

RobertoPrevato commented Dec 31, 2023

Well, this is a bad news for me. I find convenient to use the upload-artifact to write different files to the same folder in a build matrix. For example to compile custom C extensions for several combinations of Python versions and operating systems, and publishing to a single folder.

like in https://github.com/Neoteroi/BlackSheep/actions/runs/7370452109/job/20056867940

Now if I want to upgrade my workflow, I need to publish to different folders and download artifacts from multiple sources - making the workflow look like a mess compared to how clean it used to look like. For now I stay with v3 and I hope this will be reconsidered in a future version of these actions.

@DanTup
Copy link
Author

DanTup commented Dec 31, 2023

Yeah, I rolled back to v3 too. Until I'm forced to upgrade, the old way is much more convenient for me.

Kontinuation referenced this issue in apache/sedona Jan 2, 2024
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 3 to 4.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](actions/upload-artifact@v3...v4)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
syoyo added a commit to lighttransport/jagger-python that referenced this issue Jan 2, 2024
semenko added a commit to mcwdsi/bam2tensor that referenced this issue Jan 3, 2024
Rixxan added a commit to EDCD/EDMarketConnector that referenced this issue Jan 4, 2024
github-merge-queue bot pushed a commit to onnx/onnx that referenced this issue Jan 5, 2024
@seanvaleo
Copy link

I had to roll back also

@ptr727
Copy link

ptr727 commented Jan 6, 2024

Same here, I also use a matrix to build multi platform releases in the same directory, and then zipping them all together, rolling back to v3 :(

@DanTup
Copy link
Author

DanTup commented Jan 6, 2024

Seems like a lot of people have been bitten by this, so although it appears to have been deliberate I'm re-opening for better visibility to see if the authors want to chime in (of course, it's very possible it may just be closed as WAI).

@DanTup DanTup reopened this Jan 6, 2024
flozz added a commit to wanadev/yoga that referenced this issue Jan 6, 2024
dvarrazzo added a commit to psycopg/psycopg that referenced this issue Jan 7, 2024
eymar added a commit to JetBrains/compose-multiplatform that referenced this issue Sep 11, 2024
According to
https://github.blog/changelog/2024-02-13-deprecation-notice-v1-and-v2-of-the-artifact-actions/
v2 is deprecated. Besides that the workflow doesn't work with v2
anoymore -
https://github.com/JetBrains/compose-multiplatform/actions/runs/10794252433/job/29978381803?pr=5134
:

> Error: This request has been automatically failed because it uses a
deprecated version of `actions/upload-artifact: v2`. Learn more:
https://github.blog/changelog/2024-02-13-deprecation-notice-v1-and-v2-of-the-artifact-actions/


The suggested version is v4. But it requires more changes to the
workflow configuration (see
actions/upload-artifact#478). Let's use v3 for
now.
benjamin-awd pushed a commit to benjamin-awd/streamlit that referenced this issue Sep 29, 2024
## Describe your changes

Update our GitHub actions versions to fix the deprecation warning.

With `actions/upload-artifact@v4`, multiple jobs cannot upload to the
same folder anymore. Hence, I have updated the flows according to [this
migration
guide](https://github.com/actions/upload-artifact/blob/main/docs/MIGRATION.md#multiple-uploads-to-the-same-named-artifact).
See also this discussion:
actions/upload-artifact#478

The workflows for cypress are kept on `v3` according to [this
comment](streamlit#8689 (review))
to keep the usage of it simple until we retire it soonish.

## GitHub Issue Link (if applicable)

## Testing Plan

- if the CI workflows run through, we should be all good

---

**Contribution License Agreement**

By submitting this pull request you agree that all contributions to this
project are made under the Apache 2.0 license.
@tvorogme
Copy link

tvorogme commented Oct 1, 2024

@DanTup I feel this migration so ugly.

Why it "just work" before, and now you need to do this strange shitty coding -_-

@DanTup
Copy link
Author

DanTup commented Oct 1, 2024

@tvorogme I don't disagree, but this ship has sailed and I don't have any ability to improve it (I'm not a contributor here, I'm just a user that was affected by this change).

linshokaku pushed a commit to linshokaku/onnx that referenced this issue Oct 2, 2024
Reverts onnx#5830. It is breaking the release pipeline.

actions/upload-artifact#478
Signed-off-by: Linsho Kaku <linsho@preferred.jp>
myhro added a commit to myhro/staticd that referenced this issue Oct 8, 2024
It was failing with:

Error: This request has been automatically failed because it uses a
deprecated version of `actions/upload-artifact: v2`. Learn more:
https://github.blog/changelog/2024-02-13-deprecation-notice-v1-and-v2-of-the-artifact-actions/

Upgrading directly to 'v4' isn't straightforward, given it includes
breaking changes[1]. Instead of uploading the whole folder, files are
uploaded individually and then downloaded together.

[1]: actions/upload-artifact#478
TenSeventy7 added a commit to FreshROMs/android_kernel_samsung_exynos9610_mint that referenced this issue Oct 16, 2024
GitHub did something stupid on v4 that we'd need to overhaul the workflow in the near future.

actions/upload-artifact#478

Roll back to v3 for now.
TenSeventy7 added a commit to FreshROMs/android_kernel_samsung_exynos9610_mint that referenced this issue Oct 16, 2024
GitHub did something stupid on v4 that we'd need to overhaul the workflow in the near future.

actions/upload-artifact#478

Roll back to v3 for now.
juergenhoetzel added a commit to juergenhoetzel/flowercare_exporter that referenced this issue Oct 24, 2024
king-phyte added a commit to king-phyte/uuid-utils that referenced this issue Oct 25, 2024
upload-artifact v4 is broken for systems using matrix of versions. See actions/upload-artifact#478
brianegge added a commit to brianegge/hobbes that referenced this issue Oct 25, 2024
clrpackages pushed a commit to clearlinux-pkgs/pypi-geventhttpclient that referenced this issue Nov 5, 2024
…12 to version 2.3.1

Lars Holmberg (8):
      update versioning comments
      Bump version to 2.1.1
      publish workflow: publish source package only once the binary builds have succeeded (so we're more confident that it will work and we wont publish source and no binaries)
      Bump version to 2.2.0
      Downgrade artifact actions due to actions/upload-artifact#478
      GH actions: Use more specific job names
      GH actions: use kebab-case (snake_case is deprecated and gives a warning)
      Bump version to 2.3.1

Michael (4):
      Drop support for outdated python and gevent versions (#192)
      Further spring cleaning (#195)
      Fix publishing build issue (#201)
      Fixes http.client patching (#200)

Michael Löffler (39):
      Fix SSL deprecation warnings
      Add .venv config to gitignore
      Testcases now use only one localhost:54323 for tests and not a whole zoo anymore
      Renamed pytest marker online to network as suggested in #189
      Fixes #142
      Redo code duplication to avoid import issues from other tests
      Upgrade llhttp v6.0.11
      Removed restkit benchmark as it seems discontinued
      Updated github actions for publish.yml
      Removed outdated test version forks
      Use pytest tmp_path for handling temporary files
      Fix cookilib compat testcase
      Parametrized some more tests; removed __main__ sections
      Updated benchmarks and results
      Removed restkit compatible useragent as of restkit deprecation
      Added useragent download testcase; improved download examples
      Rerun benchmarks without monkeypatching, which seems to slow down the other http clients
      Make httpbin host configurable; improve client tests; regroup network dependent tests; fix flaky httpbin test
      Switch to httpbingo.org as httpbin endpoint for better test reliability
      Fix link to benchmarks readme
      Make headers preserve the case even for headers with duplicate field names
      URL refactoring: left all the parsing to the trustworthy hands of the std lib
      Improve SSL tests
      Move common test mockup to conftest.py
      Move ClientPool back to httplib2
      Make max_retries and max_redirects not count the initial attempt
      Fixes #202 using test.pypi.org before publishing
      Add MIT license classifier
      Move license note about previous http parser to README.md
      Bump version to 2.2.1
      Clean up readme and example code
      Fixes #179
      Add a thin requests compatibility layer
      Fix some warnings on parser build
      Restructure folders, so that tests don't need to be included in the package and still can import from each other
      Move tox config into pyproject.toml
      Run codespell
      Add requests API compatibility in README
      Fix locust backwards compatibility
@georgik98
Copy link

So, what is the actual fix ? We must update to @v4 until 30 Nov.
Screenshot 2024-11-18 at 17 54 12
Is this going to fix the issue ? And for download @v4.1.7 ?

@air3ijai
Copy link

@georgik98, that works. My post and mentioned comment.

We've switched to the v4 for download/upload actions and slightly adjusted workflow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests