Add job_id into github.context

[alvarezfr]

Hi,

Right now, it seems that there is no way to identify the job that is being run from inside of the GitHub workflow run.

The github context only has the element job, with matches with the string id of the job.

But when you try to get the job from the GitHub API, there is no way to match against it. If there is no name for this job, you could match them but it is not safe.

I think you could add to the github or job context the id of the job run to be able to query the GitHub API about the current job.

I found these community entries regarding this request, but I couldn't find a ticket or discussion regarding this issue:

• https://github.saobby.my.eu.orgmunity/t/job-id-is-string-in-github-job-but-integer-in-actions-api/139060
• https://github.saobby.my.eu.orgmunity/t/get-action-job-id/17365/4

Thanks!

[ethomson]

Thanks for the feedback. We should add this, indeed. I'll talk to the team about this in the new year.

[nhooyr]

What I did to find the job run id with the workflow run id based on @ianhundere's solution:

GITHUB_JOB_ID="$(curl -fsSL -H "Authorization: token $GITHUB_TOKEN" "https://api.github.com/repos/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID/jobs?per_page=100" | \
  jq ".jobs[] | select( .name == \"$GITHUB_JOB\") | .id")"
if [ -z "$GITHUB_JOB_ID" ]; then
  printf "failed to query GITHUB_JOB_ID" >&2
  exit 1
fi

[ianhundere]

i don't think i updated this, but this is the final thing i came to:

jobs_url="https://api.github.com/repos/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID/jobs"
job_id=$(curl -s -H "Authorization: token ${{ env.GITHUB_TOKEN }}" $jobs_url | jq -r '.jobs[] | select(.name == "${{ github.job }}" and .status == "in_progress") | .id')
drift_url="$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/runs/$job_id?#step:3:${{ env.STEP }}"
repo_slug=$(echo $GITHUB_REPOSITORY | awk -F/ '{ print $NF }')

[sblask]

I am using something similar, but the problem is that github.job does not always contain the complete job name. Not sure it works with matrix alone, but it definitely does not work with re-usable workflows (workflow_call/uses) - it only contains the job name of the called workflow, but not the one of the calling one. So for example only callee where the jobs endpoint returns complete job names like caller (matrix_element) / callee.

[sblask]

That solution is also assuming that you don't set a name on the job that is different from the "job ID" aka the key in the yaml file.

[casper-jensen-maersk]

2022 is almost over. Do you happen to have any news on this? Running actions inside a matrix renders the context totally useless without the jobid

[casper-jensen-maersk]

Do you happen to have any news on this topic?
Not having the job id makes it impossible to use the jobs list for anything. Especially if you are running inside a matrix or someone has named their job differently.

[lucasmellos]

Having the same problem

[nomantis]

This would be a very good to have feature. When there is an existing action that uses the GitHub API, you know there is a need for it. Having it in the context, and especially as an environment variable (so that automation can use it too, without having to pass it as a jinja template variable), would be a godsend! 🙏🏻

[tvandinther]

Chiming in from 2023, we really need this. The job I am trying to get the ID for is a matrix job with a stringified JSON object as its value so the name is truncated making the workaround practically useless.

[davidboweninrupt]

I'm embarrassed to share this, but I know people are struggling.

Here is how I'm getting this to work in every scenario:

1. Add an environment variable to the step where you want to use the job name. Use the same logic as for the actual job name (yes, that's fragile).
2. Consume the environment variable as provided_job_name in the code that wants to correlate the job name in the API
3. Have a list of possible_job_names which might match. The provided_job_name from the environment variable and one that does this:

if len(provided_job_name) > 100:
    possible_job_names.append( provided_job_name[:97] + "..." )

4. Iterate over all the jobs in the API and compare each one to the possible_job_names. I always get a match this way. Raise an error to point the user to step 1 if there is no match.
5. Profit

I know this is not pretty but it has been very effective. I really hope Github don't see this as a workaround which delays implementing the fix. If I see that happening I may have to delete this post.

I should also add the customary reminder that you shouldn't risk your career or your user's happiness based on a snippet of code you saw someone post on the Internet.

[tlhunter]

Our use case for requiring a job Id is as follows:

We have software that runs against dozens of different npm packages. Each package run is done in a separate job. If this software fails then we send a Slack message to the team to look at the incompatible npm package. Ideally we would be able to provide a URL in this slack message so that a developer can click it for more information:

https://github.com/${{ github.repository_owner }}/${{ github.repository }}/actions/runs/${{ github.run_id }}/jobs/<😥>

[hermanbanken]

Is github.job the fix for this?

jobs.<job_id>
Use jobs.<job_id> to give your job a unique identifier. The key job_id is a string and its value is a map of the job's configuration data. You must replace <job_id> with a string that is unique to the jobs object. The <job_id> must start with a letter or _ and contain only alphanumeric characters, -, or _.

No it is not: this is the string key of the job, as in my_first_job and my_second_job.

jobs:
  my_first_job:
    name: My first job
  my_second_job:
    name: My second job

[bansalsulabh]

job-name is also not present in the github context.

[jsoref]

job-name is also begged for here: https://github.com/orgs/community/discussions/16614

[jkwinder]

We could really use the jobid in the github context so on a matrix job so we can provide a html_url to a workflow status dashboard...

[Felixoid]

So, it's one of the plenty cases, where GH actions aren't fixing their fundamental issues.

Just an example of how we solve it in python https://github.com/ClickHouse/clickhouse/blob/4b7aa30/tests/ci/env_helper.py#L49-L95

The code tries to address either the feature requested in this discussion or in the issue actions/runner#2577

[Felixoid]

We are experimenting with reusable workflows, and it's just the last border before the deepest inconsistency hell. The logs are
logs_33.zip, the run is https://github.com/Felixoid/actions-experiments/actions/runs/6774723377, the there's the tree

So, the job has a name parameter. And it's overwritable. Then the name is included in the jobs API. See call_test / CallTestname-one and another_call / CallTestname-two. But there's no ... any way to get this name from the job itself. Both jobs have a simple GITHUB_JOB=CallTest.

In Felixoid/actions-experiments@8eaa77a I tried to override the parameter - no luck.

update: in tests + concurrency_call (logs_39.zip) I tried to get this variable from at least any place. No luck, neither of the contexts github, runner, env, nor job contains this value.

So, the outcome: without kicking around with strange self-invented variables, GH actions not only prevent you from finding an easy way to get the job_id, but any tool you have out of the box will ultimately give a broken solution. Because who needs to know the full Job name, right?

[jsoref]

Yup.
I think I collect everything eventually. I think most of it is near here:
https://github.com/check-spelling/check-spelling/blob/15a82f1a4276abf802d30e8af2a2961b1786d909/unknown-words.sh#L2279-L2312

[philn-delphia]

It's almost 2024 and still no movement on this?
We require the job name in order to check if the current job run has changed from a failure to a success. Unfortunately, the only available environment variable (and context variable) is the yaml key for the job, but this causes issues if the job has been given a name. In that case all the API calls use the name, which isn't available anywhere else!

It's seems to be impossible to tie a current run to a previous run unless either the name of the job becomes available in environment or context, or the slug (it's not an id, it's not a name, it's got no nomenclature I can find) needs to be available in the API responses.

[Felixoid]

I've been having a long conversation with GH support, and it feels like I almost convinced them that it's an actual issue

[CYWong-Nick]

It's now 2024 and GitHub is still adding job ID in context. We need a way to distinguish one job from another, especially when using matrix. This should not be a difficult thing either, the job ID is already in the URL.

[enginoid]

The impact of this is ultimately an inability to link to logs from a test reporting system. This is critical for our evals setup where debuggable events within the job need to link back to the full job for logs, so I think this could mean we have to switch away from GitHub Actions.

[austinpray-mixpanel]

I said to hell with it and just started using random IDs as suffixes to get around duplicate artifact names. I would have just liked to use the job id for the workflow_call so I can at least trace an artifact back to the job that generated it.

Greatly simplified, but this guy runs into trouble when trying to upload artifacts from workflow_call invocations where it's called multiple times in a matrix (or just used multiple times at all)

name: Storybook
on:
  workflow_dispatch: 
  workflow_call:
  push:
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      # ////////////////////////////////////////////////////////////////////////
      # Setup tasks
      # ////////////////////////////////////////////////////////////////////////
      # ////////////////////////////////////////////////////////////////////////
      # Run tests
      # ////////////////////////////////////////////////////////////////////////
      # ////////////////////////////////////////////////////////////////////////
      # Upload test artifacts
      # ////////////////////////////////////////////////////////////////////////
      - name: Generate a unique id
        id: gen-id
        if: always()
        run: |
          echo "rand=$(openssl rand -hex 3)" >> "$GITHUB_OUTPUT"
      - name: archive storybook junit
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: storybook-junit-${{ github.run_id }}-${{ steps.gen-id.outputs.rand }}
          path: junit.xml
          retention-days: 7
      - name: archive storybook json
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: storybook-json-${{ github.run_id }}-${{ steps.gen-id.outputs.rand }}
          path: storybook-run.json
          retention-days: 7

[AFMiziara]

2 years and a half and this is still a HUGE gap on GitHub API. It makes a lot of possible integrations and useful actions very hard to be implemented.

[Luc45]

Job IDs are necessary when running in a matrix to accurately link specific tasks. Additionally, the recently-released version 4 of actions/upload-artifact requires unique artifact names. Without job IDs, using github.run_id could result in duplicate names for jobs within the same matrix, leading to conflicts.

[gregberge]

I also need it in Argos to support granular retries. In case of matrix builds (Playwright sharding for example) I have no way to know which shard is linked to which job. So I have no way to actually know in Argos how many shards are awaited for this retry attempt.

The lack of unique identifier of a job across retry attempts (except the name not exposed), is a big problem actually.

[tt-rkim]

Just adding our use case and to the chorus of voices here.

We are trying to upload some artifacts from a matrix of jobs just like another person explained above. However, we also want to know which specific job each artifact came from.

Without the job ID available to the runner at artifact generation + upload time within the job, I'm not sure how we can not only have non-conflicting artifact names, but also know which jobs generated them.

The current solution I've been thinking about was to use the idea from above with some UUID generation and pray there's no collision for artifact names. But that still doesn't solve the problem of knowing which job made them. For that, I was thinking of printing out a parsable string for the UUID I mentioned within the logs of the job and then searching for it by download the logs for each job for a given workflow run.

That solution is really cumbersome and still prone to error. This would all go so much smoother for us if we had the job ID available instead in a context.

[taylortn]

Your scenario is indeed a common challenge when working with GitHub Actions, especially when dealing with artifacts and needing to trace them back to their originating jobs. Here are a few strategies to consider that might help streamline this process:

1. Environment Variables: Utilize GitHub's environment variables to include the job ID or other job-specific identifiers in your artifact names. GitHub Actions provide a set of default environment variables that can be quite useful. You can access the job name using GITHUB_JOB, the workflow name using GITHUB_WORKFLOW, and other useful identifiers.

   name: CI
   
   on: [push]
   
   jobs:
     build:
       runs-on: ubuntu-latest
   
       steps:
       - name: Checkout code
         uses: actions/checkout@v2
   
       - name: Build
         run: |
           echo "Building..."
           # Your build commands here
   
       - name: Upload artifact
         uses: actions/upload-artifact@v2
         with:
           name: artifact-${{ github.run_id }}-${{ github.job }}
           path: path/to/artifact

   In this example, the artifact name includes the github.run_id and github.job, which will help ensure unique and traceable names.

2. GitHub Contexts and Expressions: You can leverage GitHub contexts and expressions to dynamically generate artifact names that include unique identifiers.

   - name: Upload artifact
     uses: actions/upload-artifact@v2
     with:
       name: artifact-${{ github.run_number }}-${{ github.job }}
       path: path/to/artifact

3. Custom Scripts: If the built-in environment variables do not meet your needs, consider using a custom script to generate a unique identifier and save it to a file that can be included with your artifact. You can also log this identifier for future reference.

   jobs:
     build:
       runs-on: ubuntu-latest
   
       steps:
       - name: Checkout code
         uses: actions/checkout@v2
   
       - name: Generate UUID
         run: |
           UUID=$(uuidgen)
           echo "UUID=$UUID" >> $GITHUB_ENV
           echo "Generated UUID: $UUID"
   
       - name: Build
         run: |
           echo "Building with UUID: $UUID"
           # Your build commands here
   
       - name: Upload artifact
         uses: actions/upload-artifact@v2
         with:
           name: artifact-${{ env.UUID }}
           path: path/to/artifact

4. Workflow Level Context: If you need to pass information between jobs within the same workflow, you can use the outputs feature in GitHub Actions. One job can set an output, which can then be used by another job.

   jobs:
     generate-id:
       runs-on: ubuntu-latest
       outputs:
         job_id: ${{ steps.set-output-job-id.outputs.job_id }}
   
       steps:
       - name: Generate UUID
         id: set-output-job-id
         run: |
           UUID=$(uuidgen)
           echo "::set-output name=job_id::$UUID"
   
     build:
       runs-on: ubuntu-latest
       needs: generate-id
   
       steps:
       - name: Checkout code
         uses: actions/checkout@v2
   
       - name: Build
         run: |
           echo "Building with Job ID: ${{ needs.generate-id.outputs.job_id }}"
           # Your build commands here
   
       - name: Upload artifact
         uses: actions/upload-artifact@v2
         with:
           name: artifact-${{ needs.generate-id.outputs.job_id }}
           path: path/to/artifact

By using these strategies, you should be able to create unique and traceable artifact names, reducing the risk of name collisions and improving your ability to track which job generated which artifact.

[taylortn]

name: Example Workflow

on: [push]

jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v2

  - name: Set job_id
    id: set_job_id
    run: echo "job_id=${{ github.job }}" >> $GITHUB_ENV

  - name: Display job_id
    run: echo "The job_id is ${{ env.job_id }}"

second_job:
needs: build
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v2

  - name: Retrieve job_id from previous job
    run: echo "The job_id from the build job is ${{ needs.build.outputs.job_id }}"

[tt-rkim]

You can only identify jobs uniquely within a workflow run by name in cases where you:

• don't use a matrix
• don't override the default name

[adamsir-et]

You can only identify jobs uniquely within a workflow run by name in cases where you:

• don't use a matrix
• don't override the default name

What are DRY principles .. 🤦

[qoomon]

Feel free to use my action to access current job_id (among other context information)

Example Workflow

jobs:
  example:
    runs-on: ubuntu-latest
    environment: playground
    permissions:
      actions:     read  # required for qoomon/actions--context action
      deployments: read  # required for qoomon/actions--context action
      contents: read
    steps:
      - uses: qoomon/actions--context@v1
        id: context

      - run: |
          echo "Current Environment: ${{ steps.context.outputs.environment }}"
          echo "Current Environment: ${{ steps.context.outputs.job_id }}"
          echo "Job Logs: ${{ steps.context.outputs.job_log_url }}"

[qoomon]

Oh you are right I have not tested the combination of of reusable workflow with matrix. I'll do gimme a moment.

[qoomon]

@Felixoid I guess I found an acceptable solution for reusable workflows.

Main Workflow Example

on:
  push:
    branches: ['main']

permissions:
  actions: read  # required for qoomon/actions--context action
  deployments: read  # required for qoomon/actions--context action
  contents: read

jobs:
 
  build-with-reusable-workflow-with-matrix:
    strategy:
      matrix:
        node-version: [ 22.x, 20.x ]
        chrome-version: [ latest, stable ]
    uses: ./.github/workflows/reusable-example.yml
    with:
      # IMPORTANT job value needs to be set to the job name of the reusable workflow
      # If the surrounding workflow is a reusable workflow itself,
      #  add ${{ inputs.workflow-context }} as first line of 'workflow-context' value
      workflow-context: |
        { "job": "build-with-reusable-workflow-with-matrix", "matrix": ${{ toJSON(matrix) }} }

Reusable Workflow Example (./.github/workflows/reusable-example.yml)

on:
  workflow_call:
    inputs:
      workflow-context:
        type: string

permissions:
  actions: read  # required for qoomon/actions--context action
  deployments: read  # required for qoomon/actions--context action
  contents: read

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: qoomon/actions--context@v1
         id: context
         with:
           workflow-context: ${{ inputs.workflow-context }}

      - run: |
          echo "Current Environment: ${{ steps.context.outputs.environment }}"
          echo "Job Logs: ${{ steps.context.outputs.job_log_url }}"

This test workflow run show a viarity of use case situations https://github.com/qoomon/actions--context/actions/runs/10235291031

[Felixoid]

Don't get me wrong, but it's so ugly workaround. I see it as a worse problem to maintain, not as a solution.

And this is exactly what is discussed in the topic.

Without embedded job_id/job_name we'll get similar Frankenstein's monsters over and over again.

[qoomon]

@Felixoid I agree completely, however this is the only workaround that is possible currently and the most beautiful 😁 I'm aware of. So if someone really need this context informations right now, this is at least a solution.

[Felixoid]

Yes, it's hard-coding the name on the top level and pushing it down the road.

The same is done in our project.

And it looks like nobody notice in the GH/m$, that this is an issue.

[nebuk89]

Hey, sorry for being quiet here. Let me find out how hard adding this and the job_name would be. In the mean time I know that one of our solution engineers made an Action that gets the ID back at least
https://github.com/austenstone/job-id

Hold tight and let me see what this woudl take <3

[qoomon]

The implementation does not work with matrix jobs nor with jobs in reusable workflows.

[Felixoid]

The details of why it's useless are in https://github.com/orgs/community/discussions/8945#discussioncomment-7497043

[nebuk89]

Thank you for sharing and pointing these my way! Both are good examples of why this remains hard. And both actually are great examples of why I am so keen that we start to burn down the paper cuts. There are lots of cool features in Actions (such as reusable workflows!) where other bits of Actions 'just don't quite play nice'. This is what we really want to get to <3

Thank you for taking the time to reply to me and help me understand more. 🫶

[jenseng]

While poking around at the actions/runner repo I found this closed pull request which would partially implement it. It's not entirely clear why it was approved but then closed without merging. Though it seems like there's probably a missing piece to implement on the server side, since AgentJobRequestMessage's variables don't currently include system.github.jobId (if you check a worker's debug logs, you can see all the other variables like system.github.job)

[qoomon]

Wow, I did not expect such a simple implementation for an issue open since 3 years. @nebuk89 maybe you could address this somewhere :-)

[nebuk89]

Without reviewing the PR I would guess you are right @jenseng , quite often I have found that a bunch of the simpler things that I hope can 'just' be done in the runner also require at least a small server side change from us :)

[qoomon]

@nebuk89 I've created a new updated PR actions/runner#3489 to implement the changes from actions/runner#1950

[seanlafferty-ibm]

Even having this "full" job name is a big improvement. I use pre/post job hooks on the runner to collect telemetry- but its useless without the full job name because I cannot distinguish any of the matrix jobs (of which, I have a ton) https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/running-scripts-before-or-after-a-job

[github-actions[bot]]

💬 Your Product Feedback Has Been Submitted 🎉

Thank you for taking the time to share your insights with us! Your feedback is invaluable as we build a better GitHub experience for all our users.

Here's what you can expect moving forward ⏩

• Your input will be carefully reviewed and cataloged by members of our product teams. 
  • Due to the high volume of submissions, we may not always be able to provide individual responses.
  • Rest assured, your feedback will help chart our course for product improvements.
• Other users may engage with your post, sharing their own perspectives or experiences. 
• GitHub staff may reach out for further clarification or insight. 
  • We may 'Answer' your discussion if there is a current solution, workaround, or roadmap/changelog post related to the feedback.

Where to look to see what's shipping 👀

• Read the Changelog for real-time updates on the latest GitHub features, enhancements, and calls for feedback.
• Explore our Product Roadmap, which details upcoming major releases and initiatives.

What you can do in the meantime 💻

• Upvote and comment on other user feedback Discussions that resonate with you.
• Add more information at any point! Useful details include: use cases, relevant labels, desired outcomes, and any accompanying screenshots.

As a member of the GitHub community, your participation is essential. While we can't promise that every suggestion will be implemented, we want to emphasize that your feedback is instrumental in guiding our decisions and priorities.

Thank you once again for your contribution to making GitHub even better! We're grateful for your ongoing support and collaboration in shaping the future of our platform. ⭐

[StevenBoutcher]

[jsoref]

Fwiw, you should do:

env:
  IS_SARIF: ${{ matrix.browser == 'safari' && 1 || 0 }}
  JOB_INDEX: ${{ matrix.shardIndex - (matrix.browser != 'safari' && 1 || 0) }}

You shouldn't use ${{...}} inside run: statements, it's a bad/dangerous practice.

[StevenBoutcher]

Appreciate the pro tip 🫡 If you can't tell, I don't work primarily in CI/CD or know tons about security lol fwiw I followed your advice and it's not only more secure now but looks much cleaner 🥳