Research about actions/upload-artifact@v4 and actions/download-artifact@v4 #2135

ChristopherHX · 2023-12-19T20:48:50Z

Decompiled proto for artifacts from toolkit only typescript sources provided by GitHub.

syntax = "proto3";

import "google/protobuf/timestamp.proto";
import "google/protobuf/wrappers.proto";

package github.actions.results.api.v1;

message CreateArtifactRequest {
    string workflow_run_backend_id = 1;
    string workflow_job_run_backend_id = 2;
    string name = 3;
    google.protobuf.Timestamp expires_at = 4;
    int32 version = 5;
}

message CreateArtifactResponse {
    bool ok = 1;
    string signed_upload_url = 2;
}

message FinalizeArtifactRequest {
    string workflow_run_backend_id = 1;
    string workflow_job_run_backend_id = 2;
    string name = 3;
    int64 size = 4;
    google.protobuf.StringValue hash = 5;
}

message FinalizeArtifactResponse {
  bool ok = 1;
  int64 artifact_id = 2;
}

message ListArtifactsRequest {
    string workflow_run_backend_id = 1;
    string workflow_job_run_backend_id = 2;
    google.protobuf.StringValue name_filter = 3;
    google.protobuf.Int64Value id_filter = 4;
}

message ListArtifactsResponse {
    repeated ListArtifactsResponse_MonolithArtifact artifacts = 1;
}

message ListArtifactsResponse_MonolithArtifact {
    string workflow_run_backend_id = 1;
    string workflow_job_run_backend_id = 2;
    int64 database_id = 3;
    string name = 4;
    int64 size = 5;
    google.protobuf.Timestamp created_at = 6;
}

message GetSignedArtifactURLRequest {
    string workflow_run_backend_id = 1;
    string workflow_job_run_backend_id = 2;
    string name = 3;
}

message GetSignedArtifactURLResponse {
    string signed_url = 1;
}

message DeleteArtifactRequest {
    string workflow_run_backend_id = 1;
    string workflow_job_run_backend_id = 2;
    string name = 3;
}

message DeleteArtifactResponse {
    bool ok = 1;
    int64 artifact_id = 2;
}

Notice, this was decompiled by bing chatbot and only required include fixes
You can do it yourself using that prompt and place classes as long they don't hit the content length. I have converted multiple times to get the final list of all Messages.

Convert the following to the protobuf proto format
```

```

Update 2024-02-17: Delete Artifact Endpoint proto

The text was updated successfully, but these errors were encountered:

ChristopherHX · 2023-12-19T22:20:04Z

Hmm they are not using protobuf in the toolkit yet, but the code exists

They use the json representation of protobuf and not the binary protocol.

ChristopherHX · 2023-12-20T21:21:17Z

Only the origin of ACTIONS_RESULTS_URL is used so if it contains a baseurl path that is ignored.

Base url is always /twirp/github.actions.results.api.v1.ArtifactService of a user specified origin. Both http/https and port of the origin are preserved.

ACTIONS_RUNTIME_TOKEN must be a jwt token with claim scp containing a Actions.Results:{runid}:{jobId} value (replace {runid} and {jobId} with an id or guid, that is passed in each api request

GHES and GITEA are blocked by the current toolkit code and won't work as of now.

Gitea would have to switch to jwt tokens if they would want to support artifacts v4 from GitHub as soon it is allowed in GHES

TWiStErRob · 2024-01-26T12:36:45Z

@ChristopherHX what's the current state of ...-artifact@v4 support in act? and what are the steps required to get there?

When using new version, I'm getting

::error::Invalid token specified: Cannot read properties of undefined (reading 'replace')

which is consistent with your findings about jwt tokens above.

ChristopherHX · 2024-01-26T15:30:48Z

what's the current state of ...-artifact@v4 support in act

0%

in gitea 80%, it's also in go so not that much to fear.

We could port go-gitea/gitea@main...ChristopherHX:gitea:gitea-artifacts-v4 to act.

My time is currently limited due to exam period.

ACTIONS_RUNTIME_TOKEN has never been set by act itself...

- Issue: nektos/act#329 - Discussion: nektos/act#2135

zdenardi · 2024-02-05T22:54:22Z

I'm still getting
::error::Invalid token specified: Cannot read properties of undefined (reading 'replace')
Any update on this?

JoshMcCullough · 2024-04-10T16:25:56Z

Also seeing this with upload-artifact@v4 -- any fix?

ChristopherHX · 2024-04-10T16:49:33Z

The fix is #2224, but it doesn't meet the 50% (has 40%) coverage requirement of this project. Due to unused generated code of google protobuf.

go-gitea/gitea requirements are not that strict and it has been merged months ago

JoshMcCullough · 2024-04-10T16:53:48Z

@ChristopherHX thanks for your work on this, and for the update.

rorycl · 2024-05-01T14:30:46Z

I think I've hit the same issue on act version 0.2.62:

...
[artifacts/artifacts-component/artifacts-component]   🐳  docker cp src=/home/rory/.cache/act/actions-upload-artifact@v4/ dst=/var/run/act/actions/actions-upload-artifact@v4/
[artifacts/artifacts-component/artifacts-component]   🐳  docker exec cmd=[node /var/run/act/actions/actions-upload-artifact@v4/dist/upload/index.js] user= workdir=
...
| With the provided path, there will be 12 files uploaded 
[artifacts/artifacts-component/artifacts-component]   💬  ::debug::Root artifact directory is /home/<user/src/cexfind/artifacts
| Artifact name is valid! 
| Root directory input is valid!
[artifacts/artifacts-component/artifacts-component]   ❗  ::error::Invalid token specified: Cannot read properties of undefined (reading 'replace')
[artifacts/artifacts-component/artifacts-component]   ❌  Failure - Main archive
[artifacts/artifacts-component/artifacts-component] exitcode '1': failure
[artifacts/artifacts-component/artifacts-component] 🏁  Job failed
Error: Job 'artifacts-component' failed

This workflow works fine on github.

rorycl · 2024-05-20T22:02:38Z

Hi @ChristopherHX. Thanks very much for the note about the JWT token to fix the problem with artifacts-component. That seems to work fine.

However, running on e1e5671 I'm getting a related error: Unable to get the ACTIONS_RESULTS_URL env variable.

I'm invoking act as follows:

~/go/bin/act -v \
   -artifact-server-path=/tmp/artifacts \
   --env ACTIONS_RUNTIME_TOKEN=eyJhbGciOiJIUzI1NiJ9.eyJzY3AiOiJBY3Rpb25zLlJlc3VsdHM6e3J1bmlkfTp7am9iSWR9In0.uQOTIBLfmTt_NHUzaNcZHk8RcjWGwHxrfmon2-hUzPo

(That jwt has the contents you suggested, see jwt.io).

Apologies if I've missed something here. Let me know if I should file a new issue or specify an ACTIONS_RESULTS_URL in the environment.

Extract from output:

[arti...artifacts-component]   ✅  Success - Main build   
[arti...artifacts-component] ⭐ Run Main Copy Files 
[arti...artifacts-component]   🐳  docker exec cmd=[bash --noprofile --norc -e -o pipefail /var/run/act/workflow/3] user= workdir=
[arti...artifacts-component]   ✅  Success - Main Copy Files
[arti...artifacts-component] ⭐ Run Main archive
[arti...artifacts-component]   🐳  docker cp src=/home/user/.cache/act/actions-upload-artifact@v4/ dst=/var/run/act/actions/actions-upload-artifact@v4/
[arti...artifacts-component]   🐳  docker exec cmd=[node /var/run/act/actions/actions-upload-artifact@v4/dist/upload/index.js] user= workdir=
[arti...artifacts-component]   💬  ::debug::followSymbolicLinks 'true'
[arti...artifacts-component]   💬  ::debug::implicitDescendants 'true'
[arti...artifacts-component]   💬  ::debug::omitBrokenSymbolicLinks 'true'
[arti...artifacts-component]   💬  ::debug::followSymbolicLinks 'true'
[arti...artifacts-component]   💬  ::debug::implicitDescendants 'true'
[arti...artifacts-component]   💬  ::debug::matchDirectories 'true'
[arti...artifacts-component]   💬  ::debug::omitBrokenSymbolicLinks 'true'
[arti...artifacts-component]   💬  ::debug::Search path '/home/user/src/cexfind/artifacts'
[arti...artifacts-component]   💬  ::debug::Removing /home/user/src/cexfind/artifacts from rawSearchResults because it is a directory
[arti...artifacts-component]   💬  ::debug::File:/home/user/src/cexfind/artifacts/cli-darwin-amd64 was found using the provided searchPath
[arti...artifacts-component]   💬  ::debug::File:/home/user/src/cexfind/artifacts/cli-darwin-arm64 was found using the provided searchPath
[arti...artifacts-component]   💬  ::debug::File:/home/user/src/cexfind/artifacts/cli-linux-amd64 was found using the provided searchPath
[arti...artifacts-component]   💬  ::debug::File:/home/user/src/cexfind/artifacts/cli-win-amd64.exe was found using the provided searchPath
[arti...artifacts-component]   💬  ::debug::File:/home/user/src/cexfind/artifacts/console-darwin-amd64 was found using the provided searchPath
[arti...artifacts-component]   💬  ::debug::File:/home/user/src/cexfind/artifacts/console-darwin-arm64 was found using the provided searchPath
[arti...artifacts-component]   💬  ::debug::File:/home/user/src/cexfind/artifacts/console-linux-amd64 was found using the provided searchPath
[arti...artifacts-component]   💬  ::debug::File:/home/user/src/cexfind/artifacts/console-win-amd64.exe was found using the provided searchPath
[arti...artifacts-component]   💬  ::debug::File:/home/user/src/cexfind/artifacts/webserver-darwin-amd64 was found using the provided searchPath
[arti...artifacts-component]   💬  ::debug::File:/home/user/src/cexfind/artifacts/webserver-darwin-arm64 was found using the provided searchPath
[arti...artifacts-component]   💬  ::debug::File:/home/user/src/cexfind/artifacts/webserver-linux-amd64 was found using the provided searchPath
[arti...artifacts-component]   💬  ::debug::File:/home/user/src/cexfind/artifacts/webserver-win-amd64.exe was found using the provided searchPath
| With the provided path, there will be 12 files uploaded
[arti...artifacts-component]   💬  ::debug::Root artifact directory is /home/user/src/cexfind/artifacts
| Artifact name is valid!
| Root directory input is valid!
[arti...artifacts-component]   💬  ::debug::Workflow Run Backend ID: {runid}
[arti...artifacts-component]   💬  ::debug::Workflow Job Run Backend ID: {jobId}
[arti...artifacts-component]   ❗  ::error::Unable to get the ACTIONS_RESULTS_URL env variable
[arti...artifacts-component]   ❌  Failure - Main archive 
[arti...artifacts-component] exitcode '1': failure
[arti...artifacts-component] 🏁  Job failed

ChristopherHX · 2024-05-20T22:51:41Z

However, running on e1e5671 I'm getting a related error: Unable to get the ACTIONS_RESULTS_URL env variable.

I don't know what you are doing..... this is missing a dash (-)

-artifact-server-path=/tmp/artifacts

And doing --env ACTIONS_RUNTIME_TOKEN is a very bad idea

For me it works like that

../act -W test.yml -P ubuntu-latest=-self-hosted --env GITHUB_REPOSITORY=me/me --artifact-server-path $PWD

test.yml is from my comment in the PR.
--env GITHUB_REPOSITORY=me/me is only there because I didn't have any valid repo at my pwd

ChristopherHX · 2024-05-20T22:54:18Z

Also if ACTIONS_RUNTIME_TOKEN is not a JWT, you might be using a different version of act than you claim to be using

rorycl · 2024-05-21T08:57:41Z

Thanks very much for your response, @ChristopherHX . Apologies, the missing hyphen was the problem.

I can confirm that

~/go/bin/act \
   --artifact-server-path=/tmp/artifacts \
   --env ACTIONS_RUNTIME_TOKEN=$(<jwt)

runs successfully from a fresh compilation from e1e5671 and from the latest artifact run on linux amd64.

For anyone needing a jwt, please see the comment concerning making jwt with the contents

{"scp": "Actions.Results:{runid}:{jobId}"}

for example this one.

TWiStErRob · 2024-05-21T12:03:30Z

With the PR merged, do we still need to manually mint a JWT? Shouldn't act be doing that internally?

rorycl · 2024-05-21T12:07:53Z

You are right, @TWiStErRob

~/go/bin/act --artifact-server-path=/tmp/artifacts without a jwt worked for me on a local test run, with act on e1e5671.

Apologies for the noise.

tokidoki11 · 2024-05-22T08:43:02Z

sorry is there any release ETA?

ChristopherHX · 2024-05-22T13:03:16Z

I didn't even create this as a feature request, this issue contains much technical details for implementing the feature from scratch. Therefore are references to the jwt format or the google protobuf file part of this issue, all of them are mostly irrelevant for users of the act executable.

sorry is there any release ETA?

June 2024, cron job will create a release as long it doesn't fail to do so

Nobody prevents you from downloading the executable associated with all Pullrequests that are merged from the GitHub Artifact List or just clone or doing a go build.

ChristopherHX added the kind/discussion This is purely for discussing things label Dec 19, 2023

ChristopherHX mentioned this issue Dec 19, 2023

Add @actions/artifact v2.0.0+ and upload-artifact@v4+ support ChristopherHX/runner.server#205

Closed

This was referenced Jan 19, 2024

ACTIONS_RUNTIME_URL, ACTIONS_RUNTIME_TOKEN and ACTIONS_CACHE_URL environment variables are missing. #329

Closed

Support actions/upload-artifact@v4, actions/download-artifact@v4 go-gitea/gitea#28853

Closed

kholisrag added a commit to kholisrag/me.kholisrag.com that referenced this issue Feb 3, 2024

fix: add workaround for local artifact issue

0898c4f

- Issue: nektos/act#329 - Discussion: nektos/act#2135

ChristopherHX mentioned this issue Feb 19, 2024

Actions Artifacts v4 backend #2224

Merged

mergify bot closed this as completed in #2224 May 20, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Research about actions/upload-artifact@v4 and actions/download-artifact@v4 #2135

Research about actions/upload-artifact@v4 and actions/download-artifact@v4 #2135

ChristopherHX commented Dec 19, 2023 •

edited

Loading

ChristopherHX commented Dec 19, 2023 •

edited

Loading

ChristopherHX commented Dec 20, 2023 •

edited

Loading

TWiStErRob commented Jan 26, 2024 •

edited

Loading

ChristopherHX commented Jan 26, 2024

zdenardi commented Feb 5, 2024

JoshMcCullough commented Apr 10, 2024

ChristopherHX commented Apr 10, 2024

JoshMcCullough commented Apr 10, 2024

rorycl commented May 1, 2024

rorycl commented May 20, 2024 •

edited

Loading

ChristopherHX commented May 20, 2024 •

edited

Loading

ChristopherHX commented May 20, 2024

rorycl commented May 21, 2024

TWiStErRob commented May 21, 2024

rorycl commented May 21, 2024

tokidoki11 commented May 22, 2024

ChristopherHX commented May 22, 2024

Research about actions/upload-artifact@v4 and actions/download-artifact@v4 #2135

Research about actions/upload-artifact@v4 and actions/download-artifact@v4 #2135

Comments

ChristopherHX commented Dec 19, 2023 • edited Loading

ChristopherHX commented Dec 19, 2023 • edited Loading

ChristopherHX commented Dec 20, 2023 • edited Loading

TWiStErRob commented Jan 26, 2024 • edited Loading

ChristopherHX commented Jan 26, 2024

zdenardi commented Feb 5, 2024

JoshMcCullough commented Apr 10, 2024

ChristopherHX commented Apr 10, 2024

JoshMcCullough commented Apr 10, 2024

rorycl commented May 1, 2024

rorycl commented May 20, 2024 • edited Loading

ChristopherHX commented May 20, 2024 • edited Loading

ChristopherHX commented May 20, 2024

rorycl commented May 21, 2024

TWiStErRob commented May 21, 2024

rorycl commented May 21, 2024

tokidoki11 commented May 22, 2024

ChristopherHX commented May 22, 2024

ChristopherHX commented Dec 19, 2023 •

edited

Loading

ChristopherHX commented Dec 19, 2023 •

edited

Loading

ChristopherHX commented Dec 20, 2023 •

edited

Loading

TWiStErRob commented Jan 26, 2024 •

edited

Loading

rorycl commented May 20, 2024 •

edited

Loading

ChristopherHX commented May 20, 2024 •

edited

Loading