Informatica provider

[cetingokhan]

Add Informatica Provider for Apache Airflow

This PR introduces a new provider for Informatica EDC, which includes:

• Hooks for connecting to Informatica EDC.
• Operator and listener for lineage tracking.
• Extractors for ingesting lineage information.
• Unit tests and system test stubs.
• Documentation updates under providers/informatica/docs.

Closes: N/A
Related: N/A

All new functionality is covered by unit tests and follows Airflow coding standards.
No new third-party dependencies are introduced.

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
• Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: dev@airflow.apache.org
  Slack: https://s.apache.org/airflow-slack

[potiuk]

We are currently on pause with adding new providers.

And in any case In order to add a new provider, one has to start discussion on devlist. We have a discussion opened on whether we want to add new providers and what expectations would be for those who submit the providers

https://lists.apache.org/thread/qrv0j4dxp2yg09gds40vh49dhkbrj5q9

If you would like to advance the discussion @cetingokhan you should comment the proposal we have there.

It explains things that you (the person who submits the provider) will have to commit to (maintenance) and what conditions will have to be met in order to move from default "incubation" state to graduate to "official".

Please add your comments to the discussion if you would like to advance the discussion - no providers will be accepted until the discussion concludes I guess.

[kaxil]

Hey @cetingokhan,

Following up on Jarek's earlier comment -- the AIP-95 Provider Governance vote has concluded and passed (result). This establishes a new process for adding providers to Airflow.

Governance requirements:

Before we can consider any new provider, contributors need to:

1. Start a discussion thread on dev@airflow.apache.org (you can check archive here ) proposing the provider and explaining the use case
2. Identify 2 stewards who will commit to long-term maintenance (each steward needs sponsorship from an existing Committer)
3. Accept that the provider starts in Incubation stage with specific graduation criteria

This applies to all new provider PRs -- we won't be merging any until the proposal goes through the mailing list.

Also, this PR also has unrelated changes that need to be removed:

• Deleted symlinks in airflow-core/src/airflow/_shared/ (logging, secrets_masker, timezones, example_dags/standard)

These look like local checkout issues that got committed accidentally.

Once you've gone through the governance process on the mailing list, you can rebase and clean up this PR.

Let us know if you have any questions or need clarifications.

[cetingokhan]

Hello @kaxil,

Thank you for your guidance and for highlighting the new management requirements.

I have reviewed the AIP-95 proposal. As you mentioned, I need a Committer sponsor to start the discussion on the email list. Are you open to sponsoring this provider, or can you suggest someone who might be interested? I can assure you I will handle the heavy lifting related to maintaining the provider.

Regarding the irrelevant changes (deleted symbolic links): As you guessed, this was an unintended side effect of running on a Windows machine where Git mishandles symbolic links. I will definitely clean these up and re-base the PR as I progress through the management process.

I look forward to your recommendations regarding sponsorship.

Best regards, Gökhan

[potiuk]

You do not need to have "sponsor" before you start discussion. You will need to get someone(s) committed to overlook your maintenance - but it's a bit your job to get someone to volunteer, and likely the discussion you start should be convincing, and ideally someone who signs up should be somehow knowledgeable about it - at least somewhat.

The discussion on devlist is the best way to attract someone - there is no single person that you can delegate finding who that sponsor might be to - it's really the first task of someone to commit to maintain a provider to convince a volunteer to sign up for it.

And a good way to convince someone is to "do stuff". For example continue rebasing and fixing issues with your PR and making it green - this is one of the signals that you are serious about further maintenance.

[jscheffl]

Note that build most probably fail on broken main which is fixed in parallel in #61651

[jscheffl]

Cassandra diver problems leading to failures in CI seem to be unrelated.
Fixed via #61685

[cetingokhan]

Hi @jscheffl,

The recent errors don't seem to be related to Informatica. Is it mandatory for all tests to pass for merge?

[jscheffl]

The recent errors don't seem to be related to Informatica. Is it mandatory for all tests to pass for merge?

No not needed but would be better for confidence. Last run just had a DB timeout... but last week there were still syntax errors.

@potiuk You still have "request changes", once turning green, devlist discussionw as positive, can you re-review from your current response. In my view LGTM.

[potiuk]

I rerun the failed check to be sure

[cetingokhan]

We finally saw the green :), thank you everyone for your support from the beginning, it was a wonderful experience for me too.

[jscheffl]

Thanks for all the efforts and being patient! Let's get is rolling!

[boring-cyborg]

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

[github-actions]

Backport failed to create: v3-1-test. View the failure log Run details

Status	Branch	Result
❌	v3-1-test

You can attempt to backport this manually by running:

cherry_picker 23e980d v3-1-test

This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

[jscheffl]

Besides the optional extra for Airflow core, there is actually no real core dependency. So I'd rather not back-port, which would just miss the convenience of being able to install apache-airflow[informatica]==3.1.8 - if somebody think this should be made, let me know then I can backport. Otherwise will be possible from 3.2.0 onwards.