Breeze is not picking up the connections defined in environment file

[sc250072]

Apache Airflow version

main (development)

If "Other Airflow 2 version" selected, which one?

No response

What happened?

I have configured AIRFLOW_CONN_TERADATA_DEFAULT='teradata://login:password@host/schema' in files/airflow-breeze-config/environment_variables.env but when running system test using breeze testing system-tests providers/teradata/tests/system/teradata/example_teradata.py, breeze considering default connection values (defined as localhost as host in source code) and not considering connection details defined in environment file. But when I start airflow using breeze start-airflow, example_teradata.py dag is working successfully.
I tried to export AIRFLOW_CONN_TERADATA_DEFAULT='teradata://login:password@host/schema' at bash level and also tried at bash shell but both are not working.

I also tried to change connection name from teradata_default to teradata_normal at teradata/tests/system/teradata/example_teradata.py and defined AIRFLOW_CONN_TERADATA_NORMAL='teradata://login:password@host/schema' in files/airflow-breeze-config/environment_variables.env and breeze testing system-tests failing with teradata_normal connection is not defined.

What you think should happen instead?

• breeze testing system-tests should consider connections defined in environment variables and system tests should get success with provided connection details.

How to reproduce

• git clone https://github.com/apache/airflow.git
• cd airflow
• uv tool install -e ./dev/breeze
• create directory under files/airflow-breeze-config and create environment_variables.env
• Define connection to Teradata database as AIRFLOW_CONN_TERADATA_DEFAULT='teradata://login:password@host/schema'
• Run ./scripts/ci/testing/run_system_tests.sh providers/teradata/tests/system/teradata/example_teradata.py

New Teradata database instance can be set using following steps.

• Create a new ClearScape Analytics™ Experience account.

If you don't already have one, sign up at:

Teradata ClearScape Analytics™ Experience

• Login

Sign in with your new account at:

ClearScape Analytics™ Experience Login

• Create new Teradata instance

use option Create new environment to create new Teradata instance

login: demo_user
schema: demo_user

Operating System

Ubuntu 22.04.5 LTS

Versions of Apache Airflow Providers

apache-airflow-providers-teradata - 3.2.1

Deployment

Docker-Compose

Deployment details

docker Version 27.5.1
Docker Compose version v2.32.4
Python 3.10.12

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[potiuk]

This is expected. Pytest common plugin we use, clears env variables when pytest is run (to make sure tests are reproducible - regardless of the env variables set accidentally in the environment). You can disable it by passing --keep-env-variables as exta argument:

[sc250072]

@potiuk I already tried this option but still the same issue. Please find the screenshots of this configuration.

Error log

[potiuk]

I think you should debig it @sc250072 . Just the usual way you debug things

1. print the variable after it is set in the init
2. check if environment variable is actually set where you want to read it in the Dag (simply printing it should work).
3. check if it is set before the "pytest" command (look and run_tests.py - there is a place where pytest command is run
4. Finally you can use --verbose to see the exact command that is executed by breeze (it's a copy-pesteable docker-compose command that is executed) and manually run the exact command that breeze runs

Maybe there are other conftetst.py rules that are removing the variable somewhere on your path - but only by this debugging you can find where it is likely - it's not a rocket science. It's just a docker-compose command that gets executed, so debugging it should be easy once you know --verbose and can follow it step-by-step.

[potiuk]

That will be way more efficient rather than me trying to guess and starting another set of hypotheses