When an Ingestion job has failures and it calls the Airflow callback send_failed_status_callback errors

[IceS2]

Affected module
Ingestion Framework

Describe the bug
When an ingestion job has failures, the Airflow callback errors.

Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line 1702, in _run_finished_callback
    callback(context)
TypeError: send_failed_status_callback() takes 2 positional arguments but 3 were given

To Reproduce

We can reproduce the error by:

1. Creating a new MySQL Connector pointing to the OpenMetadata database (using the docker-compose from development)
2. Triggering an Ingestion Job
3. Killing it fast as soon as it starts running

Then it takes a while to fetch the logs through the UI but you should see the error described.

Expected behavior
The Airflow callback does not fail.

Version:

• OS: macOS
• Python version: 3.9
• OpenMetadata version: main
• OpenMetadata Ingestion package version: 1.3.0

Additional context