fix: allow slash in variable key when retrieving variable value with task-sdk

[vanducng]

Problem

Variables with slashes in their keys (e.g., test/with_slash) fail when accessed through the Task SDK's Variable.get() method, even though they exist in the database and work fine in the Airflow UI.

Error encountered:

airflow.sdk.exceptions.AirflowRuntimeError: VARIABLE_NOT_FOUND: {'key': 'test/with_slash'}

Root Cause

The issue occurs because the execution API routes in /airflow-core/src/airflow/api_fastapi/execution_api/routes/variables.py use /{variable_key} instead of /{variable_key:path}. Without the :path converter, FastAPI cannot match URLs containing slashes, causing 404 errors when the Task SDK tries to retrieve variables with slashes in their keys.

Flow of the issue:

1. DAG contains: Variable.get("test/with_slash")
2. Task SDK calls execution API: GET /variables/test/with_slash
3. FastAPI route /{variable_key} doesn't match paths with slashes
4. Returns 404 → converted to VARIABLE_NOT_FOUND error

Example Reproduction

DAG code that fails:

from airflow.sdk import Variable

# This works fine
var_no_slash = Variable.get("test_without_slash")  

# This fails with VARIABLE_NOT_FOUND
var_with_slash = Variable.get("test/with_slash")  # ❌ Fails even if variable exists

Error in logs:

HTTP Request: GET http://in-process.invalid./variables/test/with_slash "HTTP/1.1 404 Not Found"
Variable not found [airflow.sdk.api.client] detail={'detail': {'reason': 'not_found', 'message': "Variable with key 'test/with_slash' not found"}} key=test/with_slash

Solution

Updated all execution API variable route patterns to use the :path converter, matching the public API pattern:

Before:

@router.get("/{variable_key}")
@router.put("/{variable_key}")
@router.delete("/{variable_key}")

After:

@router.get("/{variable_key:path}")
@router.put("/{variable_key:path}")
@router.delete("/{variable_key:path}")

Changes Made

1. Fixed route patterns in /airflow-core/src/airflow/api_fastapi/execution_api/routes/variables.py

   • Updated GET, PUT, DELETE routes to use /{variable_key:path}
   • Updated Path parameter with proper description

2. Added comprehensive tests in /airflow-core/tests/unit/api_fastapi/execution_api/versions/head/test_variables.py

   • Tests for variables with single slash (test/with_slash)
   • Tests for variables with multiple slashes (path/to/nested/variable)
   • Tests for all CRUD operations (GET, PUT, DELETE)
   • Tests for proper 404 handling with slash-containing keys

Testing

The fix has been tested with:

• ✅ Variables with single slash: test/with_slash
• ✅ Variables with multiple slashes: path/to/nested/variable
• ✅ All CRUD operations work correctly
• ✅ Backward compatibility maintained for existing variables without slashes
• ✅ Error handling works correctly for non-existent variables with slashes

Verification

After this fix, the following now works correctly:

# Previously failed, now works
Variable.set("test/with_slash", "hello with slash")
value = Variable.get("test/with_slash")  # ✅ Returns "hello with slash"

This change aligns the execution API with the public API design and ensures consistent behavior across all Airflow variable operations.

@ykultgen @dungtrantt

---

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

[vanducng]

Hi @ashb @kaxil, could you please help review?

The CI failures don’t seem related to the changes. It looks like there’s an issue with the Google provider changes in #55321

[vanducng]

follow up on this @ashb @kaxil @amoghrajesh

[ashb]

I'm honestly surprised this worked at all before. I wonder if that was intentional or just an oversight

[vanducng]

@ashb, could you please review the updated tests again?

[vanducng]

hi @ashb, can you help to review when you get a chance?