Skip to content

Conversation

@amoghrajesh
Copy link
Contributor

The deprecated import path 'airflow.sdk.execution_time.secrets_masker' was redirecting mask_secret() to a single-process implementation that only masked secrets in the task subprocess. This caused secrets to not be masked in logs because the supervisor process (which writes logs) did not receive masking instructions through this message: https://github.com/apache/airflow/blob/main/task-sdk/src/airflow/sdk/log.py#L244-L250

Users importing mask_secret from the old path would see their secrets appear unmasked in task logs despite calling mask_secret() successfully.

The compatibility shim now redirects mask_secret to the supervisor aware implementation in airflow.sdk.log that properly masks secrets in both the task and supervisor processes.

Testing

DAG used:

import sys
from time import sleep

from airflow.sdk import DAG, Variable
from airflow.providers.standard.operators.python import PythonOperator
from airflow.exceptions import AirflowTaskTimeout

from airflow.sdk.execution_time.secrets_masker import mask_secret

mask_secret("username")


def py_callable():
    print("Hello World!")
    print("This is an username: username")
    print(f"This is in dict form: {{'key': 'username'}}")
    print(f"This is in list form: {[1, 2, 3, 'username']}")


with DAG(
    dag_id="testing_secrets_masker",
    schedule=None,
    catchup=False,
    tags=["hello", "world"],
) as dag:
    my_task = PythonOperator(
        task_id="say_hello",
        python_callable=py_callable,
    )

Before the change:
image

After the change:

image

I also fixed the deprecation warning which was not being emitted.


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@amoghrajesh amoghrajesh requested a review from potiuk November 25, 2025 06:45
@amoghrajesh amoghrajesh self-assigned this Nov 25, 2025
@amoghrajesh amoghrajesh added this to the Airflow 3.1.4 milestone Nov 25, 2025
@amoghrajesh amoghrajesh added the backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch label Nov 25, 2025
Copy link
Collaborator

@sunank200 sunank200 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall good change

@amoghrajesh
Copy link
Contributor Author

I will try the fix for automatic backport on this PR: #58705

@amoghrajesh amoghrajesh merged commit 0afe6d3 into apache:main Nov 26, 2025
85 checks passed
github-actions bot pushed a commit that referenced this pull request Nov 26, 2025
…58662)

(cherry picked from commit 0afe6d3)

Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
@github-actions
Copy link

Backport successfully created: v3-1-test

Status Branch Result
v3-1-test PR Link

github-actions bot pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Nov 26, 2025
…pache#58662)

(cherry picked from commit 0afe6d3)

Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
potiuk pushed a commit that referenced this pull request Nov 28, 2025
…58662) (#58726)

(cherry picked from commit 0afe6d3)

Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
ephraimbuddy pushed a commit that referenced this pull request Dec 3, 2025
…58662) (#58726)

(cherry picked from commit 0afe6d3)

Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
Copilot AI pushed a commit to jason810496/airflow that referenced this pull request Dec 5, 2025
itayweb pushed a commit to itayweb/airflow that referenced this pull request Dec 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:task-sdk backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants