Skip to content

Conversation

@kyungjunleeme
Copy link
Contributor

@kyungjunleeme kyungjunleeme commented Jul 1, 2025

Follow-up of #52292. Part of #52378


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@kyungjunleeme kyungjunleeme force-pushed the airflow3-compat/azure-provider-data-factory-operator-updates branch from 60c467b to 6a46418 Compare July 1, 2025 06:51
@kaxil
Copy link
Member

kaxil commented Jul 1, 2025

Test failures:

https://github.com/apache/airflow/actions/runs/15992030386/job/45107520283?pr=52618#step:6:1117

FAILED providers/microsoft/azure/tests/unit/microsoft/azure/operators/test_msgraph.py::TestMSGraphAsyncOperator::test_execute_with_old_result_processor_signature - KeyError: 'task_instance'
FAILED providers/microsoft/azure/tests/unit/microsoft/azure/operators/test_msgraph.py::TestMSGraphAsyncOperator::test_execute_with_new_result_processor_signature - KeyError: 'task_instance'
FAILED providers/microsoft/azure/tests/unit/microsoft/azure/operators/test_msgraph.py::TestMSGraphAsyncOperator::test_execute_with_old_paginate_function_signature - KeyError: 'task_instance'
FAILED providers/microsoft/azure/tests/unit/microsoft/azure/operators/test_powerbi.py::TestPowerBIDatasetRefreshOperator::test_powerbi_operator_async_get_refresh_status_success - KeyError: 'task_instance'
FAILED providers/microsoft/azure/tests/unit/microsoft/azure/operators/test_powerbi.py::TestPowerBIDatasetRefreshOperator::test_powerbi_operator_async_execute_complete_success - KeyError: 'task_instance'
FAILED providers/microsoft/azure/tests/unit/microsoft/azure/operators/test_powerbi.py::TestPowerBIDatasetRefreshOperator::test_powerbi_operator_async_execute_complete_fail - KeyError: 'task_instance'
FAILED providers/microsoft/azure/tests/unit/microsoft/azure/operators/test_powerbi.py::TestPowerBIDatasetRefreshOperator::test_powerbi_operator_refresh_fail - KeyError: 'task_instance'
FAILED providers/microsoft/azure/tests/unit/microsoft/azure/operators/test_powerbi_list.py::TestPowerBIDatasetListOperator::test_powerbi_operator_async_execute_complete_success - KeyError: 'task_instance'
FAILED providers/microsoft/azure/tests/unit/microsoft/azure/operators/test_powerbi_list.py::TestPowerBIDatasetListOperator::test_powerbi_operator_async_execute_complete_fail - KeyError: 'task_instance'
FAILED providers/microsoft/azure/tests/unit/microsoft/azure/operators/test_powerbi_list.py::TestPowerBIDatasetListOperator::test_powerbi_operator_dataset_list_fail - KeyError: 'task_instance'
FAILED providers/microsoft/azure/tests/unit/microsoft/azure/operators/test_powerbi_list.py::TestPowerBIWorkspaceListOperator::test_powerbi_operator_async_execute_complete_success - KeyError: 'task_instance'
FAILED providers/microsoft/azure/tests/unit/microsoft/azure/operators/test_powerbi_list.py::TestPowerBIWorkspaceListOperator::test_powerbi_operator_async_execute_complete_fail - KeyError: 'task_instance'
FAILED providers/microsoft/azure/tests/unit/microsoft/azure/operators/test_powerbi_list.py::TestPowerBIWorkspaceListOperator::test_powerbi_operator_workspace_list_fail - KeyError: 'task_instance'

@kyungjunleeme
Copy link
Contributor Author

kyungjunleeme commented Jul 1, 2025

@kaxil yeah, okay. Thank you. I will add testcode to pr.


I have a question about task.
In this file
providers/microsoft/azure/src/airflow/providers/microsoft/azure/transfers/sftp_to_wasb.py

I can not change from airflow.models import BaseOperator to from airflow.providers.microsoft.azure.version_compat import BaseOperator

In this case, I have to change test code?

class SFTPToWasbOperator(BaseOperator):
    ......

    def dry_run(self) -> None:
        super().dry_run()
        sftp_files: list[SftpFile] = self.get_sftp_files_map()
        for file in sftp_files:
            self.log.info(
                "Process will upload file from (SFTP) %s to wasb://%s as %s",
                file.sftp_file_path,
                self.container_name,
                file.blob_name,
            )
            if self.move_object:
                self.log.info("Executing delete of %s", file)

image
I have a problem with this file.

I was trying to fix the test code.

@kaxil
Copy link
Member

kaxil commented Jul 1, 2025

@kaxil yeah, okay. Thank you. I will add testcode to pr.

I have a question about task. In this file providers/microsoft/azure/src/airflow/providers/microsoft/azure/transfers/sftp_to_wasb.py

I can not change from airflow.models import BaseOperator to from airflow.providers.microsoft.azure.version_compat import BaseOperator

In this case, I have to change test code?

image I have a problem with this file.

imo, we can remove the dry_run method in providers/microsoft/azure/src/airflow/providers/microsoft/azure/transfers/sftp_to_wasb.py.

Or just add something like:

    def dry_run(self) -> None:
        if not AIRFLOW_V_3_0_PLUS:
            raise NotImplementedError("Not implemented for Airflow 3.")
        super().dry_run()
        sftp_files: list[SftpFile] = self.get_sftp_files_map()
        for file in sftp_files:
            self.log.info(
                "Process will upload file from (SFTP) %s to wasb://%s as %s",
                file.sftp_file_path,
                self.container_name,
                file.blob_name,
            )
            if self.move_object:
                self.log.info("Executing delete of %s", file)

fixup! fixup! ADD: type: ignore[no-redef]
@kaxil kaxil marked this pull request as ready for review July 1, 2025 20:19
Copy link
Member

@kaxil kaxil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pushed a fix for the tests: 790bbcb

@kaxil kaxil merged commit 7adb432 into apache:main Jul 1, 2025
102 of 110 checks passed
@kyungjunleeme
Copy link
Contributor Author

Thank you,
I’m currently getting used to contributing to open source projects.
Next time, I’ll try to be more thorough before opening a PR, keeping in mind that the first reviewer is always myself.
Thank you for your help!

@potiuk
Copy link
Member

potiuk commented Jul 2, 2025

Thank you, I’m currently getting used to contributing to open source projects. Next time, I’ll try to be more thorough before opening a PR, keeping in mind that the first reviewer is always myself. Thank you for your help!

Good job! no worries - we all started one day !

@kaxil
Copy link
Member

kaxil commented Jul 2, 2025

Thank you, I’m currently getting used to contributing to open source projects. Next time, I’ll try to be more thorough before opening a PR, keeping in mind that the first reviewer is always myself. Thank you for your help!

Yeah, we all still learn new things everyday. Keep contributing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants