Fixes 14528: skip statuses for tasks that are no longer present in Airflow DAG

[mgorsk1]

Describe your changes:

Fixes #14528

I worked on filtering out task instances which are related to tasks no longer present in serialized dag object because fetching statuses for such tasks results in errors during ingestion.

Type of change:

• Bug fix
• Improvement
• New feature
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation

Checklist:

• I have read the CONTRIBUTING document.
• My PR title is Fixes <issue-number>: <short explanation>
• I have commented on my code, particularly in hard-to-understand areas.
• [] For JSON Schema changes: I updated the migration scripts or explained why it is not needed.

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[harshach]

@mgorsk1 thanks for the contribution.

[github-actions]

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

[harshach]

@mgorsk1 can you please fix style check failures, looks like test case is also failing because of the change

[github-actions]

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

[github-actions]

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

[mgorsk1]

Hey @harshach I've addressed whatever was brought up by CI, should be 👌 now

[sonarqubecloud]

Quality Gate passed for 'open-metadata-ingestion'

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
25.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[harshach]

Thanks for your contributions @mgorsk1

[mgorsk1]

sure, always a pleasure 🤝🏼