remove: set_transport_variable import

[kgw7401]

Hi i am making some function on dataproc and i found unnessary import in there.
I don't know why this import line added but whenever i run the system code, import error is always raised due to that line.
So i suggest removing that line for local testing.

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

[potiuk]

Interesting @mobuchowski @kacpermuda @VladaZakharova -> do you know anything about it ?

[kacpermuda]

This may need to be updated after switch to new provider layout, what we should import is this auto-use fixture. Many google operators use OpenLineage so we're also testing OL events in google system tests, without this fixture and setting OL transport, the system tests where we check OL would fail.

[kyungjunleeme]

Intresting issue.

I have a problem in breeze development environment

I think that we shoud trace history!.

It can be very helpful to understand system-test

#22311 New design system

#45681

from providers.tests.system.openlineage.conftest import set_transport_variable  # noqa: F401

#46608

from system.openlineage.conftest import set_transport_variable  # noqa: F401

[kgw7401]

@kacpermuda then what about commenting that import line until provider layout update?

[kacpermuda]

Commenting out will not help, as we're also importing from system.openlineage within google system tests e.g. from system.openlineage.operator import OpenLineageTestOperator. Some tests may pass, but some will still fail. One option would be to copy needed modules from OL system tests to Google system tests, but maybe there is en easier way.

@potiuk Is here a way to have a cross-provider pytest fixture or some cross-provider code used for system tests ?

[potiuk]

There:

from tests_common.test_utils.system_tests import get_test_run 

[potiuk]

Each system test already uses it - you can add shared code there:

devel-common/src/tests_common/test_utils

[kacpermuda]

@kgw7401 @VladaZakharova I think that would the way to go, copying OL variable transport and operator to common test utils and then importing from there in google system tests. I can look into it maybe next week, so if you want to do it faster, I'll be happy to review the PR. Not sure how import from openlineage.X will behave in those common utils, so definitely something to check.

Also @mobuchowski, do we want to move the OL VariableTransport and check operator into common utils entirely or also keep a copy in OL provider? Not sure it makes sense to keep a copy, but then any changes to it are nicely labeled with provider:openlineage and it helps me keep track of any changes made by other contributors, maybe there is a workaround for this 😄

[kyungjunleeme]

I'm learning a lot through this process — thank you.

However, one thing I’m still a bit concerned about is the fact that we’re getting import errors in local development environments.

I think it would be a good idea to also review and ensure that these new from system... imports work reliably in local setups as well, not just in CI or Breeze.

It feels important to ensure a smooth experience for all contributors who are developing and testing locally.

like that.

Should we also check how this affects other providers as well?
It might be worth verifying whether the new import structure works consistently across different providers, especially in local development environments.

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.