Skip to content

Conversation

@amoghrajesh
Copy link
Contributor

Unusual, but when Connection.get(None) is called, the stacktrace looks like:

[2025-09-30 16:58:30] WARNING - Skipping masking for a secret as it's too short (<5 chars) source=airflow._shared.secrets_masker.secrets_masker loc=secrets_masker.py:546
[2025-09-30 16:58:30] ERROR - Unable to retrieve connection from secrets backend (EnvironmentVariablesBackend). Checking subsequent secrets backend. source=task loc=context.py:162
AttributeError: 'NoneType' object has no attribute 'upper'
File "/opt/airflow/task-sdk/src/airflow/sdk/execution_time/context.py", line 156 in _get_connection

File "/opt/airflow/airflow-core/src/airflow/secrets/base_secrets.py", line 76 in get_connection

File "/opt/airflow/airflow-core/src/airflow/secrets/environment_variables.py", line 34 in get_conn_value

[2025-09-30 16:58:30] ERROR - Task failed with exception source=task loc=task_runner.py:993
ValidationError: 1 validation error for GetConnection
conn_id
  Input should be a valid string [type=string_type, input_value=None, input_type=NoneType]
    For further information visit https://errors.pydantic.dev/2.11/v/string_type
File "/opt/airflow/task-sdk/src/airflow/sdk/execution_time/task_runner.py", line 919 in run

File "/opt/airflow/task-sdk/src/airflow/sdk/execution_time/task_runner.py", line 1306 in _execute_task

File "/opt/airflow/task-sdk/src/airflow/sdk/bases/operator.py", line 416 in wrapper

File "/opt/airflow/providers/standard/src/airflow/providers/standard/operators/python.py", line 216 in execute

File "/opt/airflow/providers/standard/src/airflow/providers/standard/operators/python.py", line 239 in execute_callable

File "/opt/airflow/task-sdk/src/airflow/sdk/execution_time/callback_runner.py", line 82 in run

File "/files/dags/print_sensitive_data.py", line 13 in print_sensitive

File "/opt/airflow/task-sdk/src/airflow/sdk/definitions/connection.py", line 226 in get

File "/opt/airflow/task-sdk/src/airflow/sdk/execution_time/context.py", line 183 in _get_connection

File "/usr/python/lib/python3.10/site-packages/pydantic/main.py", line 253 in __init__

This is because when a connection is not found, the secrets backend failures log verbosely (log.exception) and continues to API call, allowing cryptic errors to bubble up. Changed that to log.debug, so that the task SDK now raises clean AirflowNotFoundException instead of cryptic AttributeError, matching airflow-core's error handling.

Example after:

from airflow.sdk import Connection as SDKConnection
SDKConnection.get(None)
Traceback (most recent call last):
  File "/Users/amoghdesai/Documents/OSS/repos/airflow/.venv/lib/python3.12/site-packages/IPython/core/interactiveshell.py", line 3699, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-9-21c62a62b135>", line 1, in <module>
    SDKConnection.get(None)
  File "/Users/amoghdesai/Documents/OSS/repos/airflow/task-sdk/src/airflow/sdk/definitions/connection.py", line 226, in get
    return _get_connection(conn_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/amoghdesai/Documents/OSS/repos/airflow/task-sdk/src/airflow/sdk/execution_time/context.py", line 172, in _get_connection
    raise AirflowNotFoundException(f"The conn_id `{conn_id}` isn't defined")
airflow.exceptions.AirflowNotFoundException: The conn_id `None` isn't defined

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@amoghrajesh amoghrajesh self-assigned this Oct 15, 2025
@amoghrajesh amoghrajesh added this to the Airflow 3.1.1 milestone Oct 15, 2025
@amoghrajesh amoghrajesh requested a review from feluelle October 15, 2025 07:52
@amoghrajesh amoghrajesh added the backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch label Oct 15, 2025
@amoghrajesh amoghrajesh merged commit 7a834d6 into apache:main Oct 15, 2025
152 of 153 checks passed
@amoghrajesh amoghrajesh deleted the log-better-for-connection branch October 15, 2025 09:41
github-actions bot pushed a commit that referenced this pull request Oct 15, 2025
…ore behavior (#56650)

(cherry picked from commit 7a834d6)

Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
@github-actions
Copy link

Backport successfully created: v3-1-test

Status Branch Result
v3-1-test PR Link

github-actions bot pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Oct 15, 2025
…ore behavior (apache#56650)

(cherry picked from commit 7a834d6)

Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
potiuk pushed a commit that referenced this pull request Oct 15, 2025
…ore behavior (#56650) (#56653)

(cherry picked from commit 7a834d6)

Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 17, 2025
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:task-sdk backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants