No way to configure CODEQL_THREADS with an environment variable

[austinpray-mixpanel]

The source says this action respects both CODEQL_RAM and CODEQL_THREADS

https://github.com/github/codeql-action/blob/5eb3ed6614230b1931d5c08df9e096e4ba524f21/lib/init-action.js#L315C12-L319

this is true for CODEQL_RAM

codeql-action/lib/init-action.js

Line 320 in 5eb3ed6

core.exportVariable("CODEQL_RAM", process.env["CODEQL_RAM"] ||

but not true for CODEQL_THREADS

codeql-action/lib/init-action.js

Line 322 in 5eb3ed6

core.exportVariable("CODEQL_THREADS", (0, util_1.getThreadsFlagValue)((0, actions_util_1.getOptionalInput)("threads"), logger).toString());

Is this an oversight or is there a good reason for this?

My use case is I'm running this on a big 48 core kubernetes but the codeql runner pod is only allowed to use 16 cores. The autodetection is not factoring in the pod limits, it's looking at the node's available resources. I want to hint to codeql that it only has 16 threads available via the CODEQL_THREADS env var.

[austinpray-mixpanel]

Spiked up a PR for this #2891

[mbg]

The source says this action respects both CODEQL_RAM and CODEQL_THREADS

I think there's a key misunderstanding here: the comment in

codeql-action/src/init-action.ts

Lines 538 to 542 in 5eb3ed6

	// Limit RAM and threads for extractors. When running extractors, the CodeQL CLI obeys the
	// CODEQL_RAM and CODEQL_THREADS environment variables to decide how much RAM and how many
	// threads it would ask extractors to use. See help text for the "--ram" and "--threads"
	// options at https://codeql.github.com/docs/codeql-cli/manual/database-trace-command/
	// for details.

talks about the CodeQL CLI, not the CodeQL Action. In other words, it is the CLI that respects these environment variables and they are set by the Action based on the corresponding Actions inputs (if set).

That said, I am unsure why we respect the existing value of CODEQL_RAM (if set) over the explicit input to the Action and not CODEQL_THREADS (if set).

[mbg]

For your use case, if you are able to, ensure that you set the CODEQL_THREADS environment variable for the autobuild and analyze steps in your workflow.

[austinpray-mixpanel]

Thank you for the quick response!

More concrete details:

name: "GHAS JS CodeQL"

on:
  push:
    branches: [ "master" ]
    paths:
      - '**.js'
      - '**.jsx'
      - '**.ts'
      - '**.tsx'
      - '**.html'
      - '.github/workflows/ghas-js-codeql.yaml'
  pull_request:
    branches: [ "master" ]
    paths:
      - '**.js'
      - '**.jsx'
      - '**.ts'
      - '**.tsx'
      - '**.html'
      - '.github/workflows/ghas-js-codeql.yaml'
  workflow_dispatch: {}

jobs:
  analyze:
    name: Analyze
    runs-on: 'mxpnl-arc-32' # needs upsized runner or will OOM
    container:
      image: '<an ubuntu 24.04 base image>'
    concurrency:
      group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.head_ref || github.sha }}
      cancel-in-progress: ${{ github.event_name == 'pull_request' }}
    timeout-minutes: 30
    permissions:
      security-events: write
      packages: read
      actions: read
      contents: read

    steps:
    - name: Checkout repository
      uses: actions/checkout@v4
    - name: Install Node
      uses: actions/setup-node@v4.3.0
      with:
        node-version-file: .nvmrc

    # Initializes the CodeQL tools for scanning.
    - name: print env vars
      run: |
        echo "CODEQL_THREADS=$CODEQL_THREADS"
        echo "CODEQL_RAM=$CODEQL_RAM"

    - name: Initialize CodeQL
      uses: github/codeql-action/init@v3.28.14
      with:
        languages: javascript-typescript
        build-mode: none
        config-file: ./.github/codeql-config.yaml

    - name: Perform CodeQL Analysis
      uses: github/codeql-action/analyze@v3.28.14
      with:
        category: "/language:javascript-typescript"

mxpnl-arc-32 is a actions-runner-controller autoscaling runner set in kubernetes mode where the workflow pods have these env vars set

nets me

So it's correctly picking up the ram variable but ignoring the threads env var.

The CLI is not respecting the global env var because it is being overridden by the action

[mbg]

Like I said, you would need to set the environment variable(s) for the analyze step. They get overriden by the init step as you say, but you can set them yourself for the analyze step with e.g.:

    - name: Perform CodeQL Analysis
      uses: github/codeql-action/analyze@v3.28.14
      with:
        category: "/language:javascript-typescript"
      env:
        CODEQL_THREADS: # your value or expression here

[austinpray-mixpanel]

I'm still a little confused why we can't unify the behavior of the CODEQL_THREAD env var with the CODEQL_RAM env var. Like what's the downside to making them behave in the same way?

I'm rolling with updating each of our GHAS workflows with a step to pass the environment vars to the init step explicitly

name: "GHAS JS CodeQL"
# ...
jobs:
  analyze:
    # ...
    steps:
    # ...
    # 👉 added this step to pass the env vars to the init action
    - name: Set limit vars
      run: |
        echo "CODEQL_THREADS=$CODEQL_THREADS" >> "$GITHUB_ENV"
        echo "CODEQL_RAM=$CODEQL_RAM" >> "$GITHUB_ENV"
    - name: Initialize CodeQL
      uses: github/codeql-action/init@v3.28.14
      with:
        languages: javascript-typescript
        build-mode: none
        config-file: ./.github/codeql-config.yaml
        threads: ${{ env.CODEQL_THREADS }}
        ram: ${{ env.CODEQL_RAM }}
    - name: Perform CodeQL Analysis
      uses: github/codeql-action/analyze@v3.28.14
      with:
        category: "/language:javascript-typescript"

[mbg]

I'm still a little confused why we can't unify the behavior of the CODEQL_THREAD env var with the CODEQL_RAM env var. Like what's the downside to making them behave in the same way?

My previous replies were mainly about finding you a workaround for the immediate issue, while we need to review whether changing that behaviour would have any unintended side-effects. Your PR is appreciated and we will review that separately!

[austinpray-mixpanel]

My previous replies were mainly about finding you a workaround for the immediate issue, while we need to review whether changing that behaviour would have any unintended side-effects. Your PR is appreciated and we will review that separately!

Ohhh okay I misunderstood that as "this is how it's expected to work"

Yep I'm unblocked right now 👍. Thank you!

[austinpray-mixpanel]

Thank you @aeisenberg @mbg and friends 👍