Fix/RE:Invalid uri created when extras contains non string elements #59339

Pyasma · 2025-12-11T20:29:25Z

RE:This PR fixes how Connection.get_uri() handles extras with non-string values (floats, booleans, nested dicts).

Extras are now stringified before encoding to produce valid, SQLAlchemy-compatible URIs.

Added a unit test test_get_uri_with_non_string_extras to verify the change.

All pre-commit checks passed.

Original PR is closed -> #57969

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

Pyasma · 2025-12-12T10:18:35Z

CodeQL workflow is awaiting maintainer approval (standard for PRs from forks). All unit tests pass locally.
@jason810496
@Lee-W

Pyasma · 2025-12-12T10:19:01Z

CodeQL workflow is awaiting maintainer approval (standard for PRs from forks). All unit tests pass locally. @jason810496 @Lee-W

Can you review this please

Pyasma · 2025-12-14T06:11:52Z

Hey @potiuk, thanks for re-running the tests. I’m a bit confused by these errors—do you have any advice on where I should start looking or how to approach fixing them?

potiuk · 2025-12-14T07:48:05Z

Hey @potiuk, thanks for re-running the tests. I’m a bit confused by these errors—do you have any advice on where I should start looking or how to approach fixing them?

There was a flaky test in main - so it failed often in other PRs as well. This is now hopefully fixed in #59394 so you should not worry about it.

henry3260 · 2025-12-14T09:29:57Z

Same 🫠

Pyasma · 2025-12-14T09:46:40Z

test still failed though 🥲
I will look at it later any advice for this one ?

potiuk · 2025-12-14T09:51:29Z

Yep. the hopefully part was unfortunatelly well placed.

henry3260 · 2025-12-14T09:51:59Z

test still failed though 🥲

I will look at it later any advice for this one ?

Don't worry. It's not you fault

potiuk · 2025-12-14T10:30:56Z

Yeah. Not your fault for sure. I have a likely fix for the main issue: #59406

Pyasma · 2025-12-15T03:13:06Z

Nice all test passed thanks @potiuk @henry3260

The issue is solved now?

henry3260 · 2025-12-15T04:00:32Z

Nice all test passed thanks @potiuk @henry3260

The issue is solved now?

Yes 😀

Pyasma · 2025-12-16T14:55:58Z

@jason810496 Hey jason can you please review the PR

Pyasma · 2025-12-16T14:56:26Z

hey @Lee-W Can you please review the PR

jason810496

Thanks for the update!

airflow-core/tests/unit/models/test_connection.py

airflow-core/src/airflow/models/connection.py

Lee-W

one nit, overall looks good

Pyasma · 2025-12-22T15:14:06Z

sorry about this push i am gonna push one more time i forgot to do prek test

…rialization and deserialization for various data types.

Pyasma · 2025-12-27T04:42:12Z

@jason810496 @Lee-W Thanks for approving the PR can you merge this to main?

jason810496 · 2025-12-27T12:26:05Z

Sorry that I forgot to add the backport label. @Pyasma, would you mind backporting this PR to the v3-1-test branch? Thanks!

We can backport the patch using the following command and resolve any conflicts if needed:

cherry_picker a377033 v3-1-test

…pache#59339) Co-authored-by: Pyasma <pranyasharms55555@gmail.com>

…g elements (apache#59339) (cherry picked from commit a377033) Co-authored-by: Piyush Mudgal <94688939+Pyasma@users.noreply.github.com> Co-authored-by: Pyasma <pranyasharms55555@gmail.com>

potiuk · 2026-01-07T14:46:48Z

I cherry-picked it here #60219 @jason810496

Pyasma · 2026-01-07T18:10:08Z

hey @jason810496 Sorry i missed the message and thanks @potiuk

…g elements (#59339) (#60219) (cherry picked from commit a377033) Co-authored-by: Piyush Mudgal <94688939+Pyasma@users.noreply.github.com> Co-authored-by: Pyasma <pranyasharms55555@gmail.com>

…pache#59339) Co-authored-by: Pyasma <pranyasharms55555@gmail.com>

frodo2000 · 2026-01-13T14:52:24Z

After migration to 3.1.6 still having error:

`[2026-01-13 15:46:29] ERROR - Task failed with exception source=task loc=task_runner.py:1056
AirflowException: ('Entra Id processing error:', TypeError("connect() got an unexpected keyword argument 'extra'"))
File "/home/airflow/airflow_venv/lib/python3.12/site-packages/airflow/sdk/execution_time/task_runner.py", line 1004 in run

File "/home/airflow/airflow_venv/lib/python3.12/site-packages/airflow/sdk/execution_time/task_runner.py", line 1405 in _execute_task

File "/home/airflow/airflow_venv/lib/python3.12/site-packages/airflow/sdk/bases/operator.py", line 417 in wrapper

File "/opt/airflow/plugins/com/softwaremind/transfers/entraid.py", line 148 in execute

TypeError: connect() got an unexpected keyword argument 'extra'
File "/opt/airflow/plugins/com/softwaremind/transfers/entraid.py", line 133 in execute

File "/home/airflow/airflow_venv/lib/python3.12/site-packages/airflow/providers/microsoft/mssql/hooks/mssql.py", line 97 in get_sqlalchemy_connection

File "/home/airflow/airflow_venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 3325 in connect

File "/home/airflow/airflow_venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 96 in init

File "/home/airflow/airflow_venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 3404 in raw_connection

File "/home/airflow/airflow_venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 3371 in _wrap_pool_connect

File "/home/airflow/airflow_venv/lib/python3.12/site-packages/sqlalchemy/pool/base.py", line 327 in connect

File "/home/airflow/airflow_venv/lib/python3.12/site-packages/sqlalchemy/pool/base.py", line 894 in _checkout

File "/home/airflow/airflow_venv/lib/python3.12/site-packages/sqlalchemy/pool/base.py", line 493 in checkout

File "/home/airflow/airflow_venv/lib/python3.12/site-packages/sqlalchemy/pool/impl.py", line 145 in _do_get

File "/home/airflow/airflow_venv/lib/python3.12/site-packages/sqlalchemy/util/langhelpers.py", line 70 in exit

File "/home/airflow/airflow_venv/lib/python3.12/site-packages/sqlalchemy/util/compat.py", line 211 in raise_

File "/home/airflow/airflow_venv/lib/python3.12/site-packages/sqlalchemy/pool/impl.py", line 143 in _do_get

File "/home/airflow/airflow_venv/lib/python3.12/site-packages/sqlalchemy/pool/base.py", line 273 in _create_connection

File "/home/airflow/airflow_venv/lib/python3.12/site-packages/sqlalchemy/pool/base.py", line 388 in init

File "/home/airflow/airflow_venv/lib/python3.12/site-packages/sqlalchemy/pool/base.py", line 690 in __connect

File "/home/airflow/airflow_venv/lib/python3.12/site-packages/sqlalchemy/util/langhelpers.py", line 70 in exit

File "/home/airflow/airflow_venv/lib/python3.12/site-packages/sqlalchemy/util/compat.py", line 211 in raise_

File "/home/airflow/airflow_venv/lib/python3.12/site-packages/sqlalchemy/pool/base.py", line 686 in __connect

File "/home/airflow/airflow_venv/lib/python3.12/site-packages/sqlalchemy/engine/create.py", line 574 in connect

File "/home/airflow/airflow_venv/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 598 in connect`

When extra fields contains as_dict attribute:

{ "tds_version": "7.3", "as_dict": true }

henry3260 · 2026-01-13T18:18:46Z

@frodo2000 Can you open an issue for that?
Thanks!

Pyasma requested review from XD-DENG and ashb as code owners December 11, 2025 20:29

Pyasma changed the title ~~Fix/connection uri~~ Fix/RE:Invalid uri created when extras contains non string elements Dec 11, 2025

potiuk force-pushed the Fix/ConnectionURI branch from 3431d69 to 41e7df8 Compare December 11, 2025 22:47

potiuk force-pushed the Fix/ConnectionURI branch from 29f80b2 to 355bf68 Compare December 13, 2025 16:03

potiuk force-pushed the Fix/ConnectionURI branch 2 times, most recently from 96581bd to 6e6b804 Compare December 14, 2025 07:47

potiuk force-pushed the Fix/ConnectionURI branch from 6e6b804 to 08db084 Compare December 14, 2025 23:50

jason810496 reviewed Dec 17, 2025

View reviewed changes

Lee-W reviewed Dec 17, 2025

View reviewed changes

airflow-core/src/airflow/models/connection.py Outdated Show resolved Hide resolved

airflow-core/src/airflow/models/connection.py Outdated Show resolved Hide resolved

Lee-W reviewed Dec 22, 2025

View reviewed changes

airflow-core/src/airflow/models/connection.py Outdated Show resolved Hide resolved

Lee-W approved these changes Dec 22, 2025

View reviewed changes

Pyasma added 4 commits December 27, 2025 00:12

feat: Enhance Connection extra parameter handling by improving URI se…

6cf45be

…rialization and deserialization for various data types.

PrekChanges

2879d7e

new changes

befff29

Testing Changes

b96a442

Pyasma and others added 8 commits December 27, 2025 00:12

finished

071bd6e

Solving the linting error

ac5375a

Synchronize TI and TIS attributes

324dc7d

Testing

e7d96bc

Retesting

d17f37f

Added the Type str removed by mistake

e54f9b2

Finished Prek

a4e8268

Reverted the get_extra_dejson

2dead81

Pyasma force-pushed the Fix/ConnectionURI branch from a934aed to 2dead81 Compare December 26, 2025 18:42

jason810496 approved these changes Dec 27, 2025

View reviewed changes

Lee-W merged commit a377033 into apache:main Dec 27, 2025
69 checks passed

amoghrajesh pushed a commit to astronomer/airflow that referenced this pull request Dec 29, 2025

Fix/RE:Invalid uri created when extras contains non string elements (a…

c3c1107

…pache#59339) Co-authored-by: Pyasma <pranyasharms55555@gmail.com>

Subham-KRLX pushed a commit to Subham-KRLX/airflow that referenced this pull request Jan 2, 2026

Fix/RE:Invalid uri created when extras contains non string elements (a…

e12170b

…pache#59339) Co-authored-by: Pyasma <pranyasharms55555@gmail.com>

potiuk added this to the Airflow 3.1.6 milestone Jan 7, 2026

potiuk mentioned this pull request Jan 7, 2026

[v3-1-test] Fix/RE:Invalid uri created when extras contains non strin… #60219

Merged

stegololz pushed a commit to stegololz/airflow that referenced this pull request Jan 9, 2026

Fix/RE:Invalid uri created when extras contains non string elements (a…

17f899f

…pache#59339) Co-authored-by: Pyasma <pranyasharms55555@gmail.com>

Fix/RE:Invalid uri created when extras contains non string elements #59339

Fix/RE:Invalid uri created when extras contains non string elements #59339

Conversation

Pyasma commented Dec 11, 2025

Uh oh!

Pyasma commented Dec 12, 2025

Uh oh!

Pyasma commented Dec 12, 2025

Uh oh!

Pyasma commented Dec 14, 2025

Uh oh!

potiuk commented Dec 14, 2025

Uh oh!

henry3260 commented Dec 14, 2025

Uh oh!

Pyasma commented Dec 14, 2025

Uh oh!

potiuk commented Dec 14, 2025

Uh oh!

henry3260 commented Dec 14, 2025

Uh oh!

potiuk commented Dec 14, 2025

Uh oh!

Pyasma commented Dec 15, 2025

Uh oh!

henry3260 commented Dec 15, 2025

Uh oh!

Pyasma commented Dec 16, 2025

Uh oh!

Pyasma commented Dec 16, 2025

Uh oh!

jason810496 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Lee-W left a comment

Choose a reason for hiding this comment

Uh oh!

Pyasma commented Dec 22, 2025

Uh oh!

Pyasma commented Dec 27, 2025

Uh oh!

Uh oh!

jason810496 commented Dec 27, 2025

Uh oh!

potiuk commented Jan 7, 2026

Uh oh!

Pyasma commented Jan 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

frodo2000 commented Jan 13, 2026

Uh oh!

henry3260 commented Jan 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Pyasma commented Jan 7, 2026 •

edited

Loading

henry3260 commented Jan 13, 2026 •

edited

Loading