Large Actions artifacts (> ~8MB) will fail checksum when merging their chunks

[lunny]

ref: https://forum.gitea.com/t/large-actions-artifacts-8mb-will-fail-checksum-when-merging-their-chunks/9291

I'm using a self-hosted Gitea 1.22.0, with the patched version of upload-artifact@v4 (https://github.com/ChristopherHX/gitea-upload-artifact@v4), and am trying to upload a ~1.8GB artifact.

From the Actions step:

Uploaded bytes 1851809172
Finished uploading artifact content to blob storage!
SHA256 hash of uploaded artifact zip is 5445b7ac4b993cbc01f349b0ec90b8ebceee564cff62ce9a529fdb9ce1326886
Finalizing artifact upload
::error::Failed to FinalizeArtifact: Unexpected token 'E', "Error merge chunks%0A" is not valid JSON

When looking at the Gitea server logs, this matches up to this error message:

...artifactsv4.go:363:finalizeArtifact() [E] Error merge chunks: update artifact error checksum is invalid

I did some tests, and it looks like any artifacts that are bigger than roughly 8MB will fail to be merged when finalizing the upload.

[ChristopherHX]

Investigating... Interesting my original .net based implementation has no problem with 807MB Zip files for me (that doesn't do checksum checks like I did for Gitea, but the linux shasum tool confirms a correct upload)

Now I need to test Gitea Actions itself and bisect the problem

Related my nektos/act port seem to be affected as well (8 MB mentioned as well...), this could be indeed a porting problem from .net to go

• upload-artifact@v4 uploads corrupted archive nektos/act#2351
• upload-artifacts/download-artifacts@v4 not working with large files nektos/act#2371

[ChristopherHX]

Problem found for act:

Multiple chunks are uploaded with type "block" without using "appendBlock", my .net Implementation didn't have different handling for these upload type and this solves the issue for act itself

8MB seems to be the chunk size

[VAllens]

I also encountered the same problem.
Has there been any progress?