Add compatibility note for Listeners (apache#39544)

While running tests for older versions of Airflow, it turned out that
forward-compatibility is maintained for implemented interfaces but if
someone implements listener using the latest version of listener,
they will not be backwards-compatible with older versions of Airflow.

This PR adds a note about it to Airflow documentation.

docs/apache-airflow/administration-and-deployment/listeners.rst

@@ -110,3 +110,67 @@ Airflow defines the specification as `hookspec <https://github.com/apache/airflo
 To include the listener in your Airflow installation, include it as a part of an :doc:`Airflow Plugin </authoring-and-scheduling/plugins>`.
 
 Listener API is meant to be called across all DAGs and all operators. You can't listen to events generated by specific DAGs. For that behavior, try methods like ``on_success_callback`` and ``pre_execute``. These provide callbacks for particular DAG authors or operator creators. The logs and ``print()`` calls will be handled as part of the listeners.
+
+
+Compatibility note
+------------------
+
+The listeners interface might change over time. We are using ``pluggy`` specifications which
+means that implementation of the listeners written for older versions of the interface should be
+forward-compatible with future versions of Airflow.
+
+However, the opposite is not guaranteed, so if your listener is implemented against a newer version of the
+interface, it might not work with older versions of Airflow. It is not a problem if you target single version
+of Airflow, because you can adjust your implementation to the version of Airflow you use, but it is important
+if you are writing plugins or extensions that could be used with different versions of Airflow.
+
+For example if a new field is added to the interface (like the ``error`` field in the
+``on_task_instance_failed`` method in 2.10.0), the listener implementation will not handle the case when
+the field is not present in the event object and such listeners will only work for Airflow 2.10.0 and later.
+
+In order to implement a listener that is compatible with multiple versions of Airflow including using features
+and fields added in newer versions of Airflow, you should check version of Airflow used and use newer version
+of the interface implementation, but for older versions of Airflow you should use older version of the
+interface.
+
+For example if you want to implement a listener that uses the ``error`` field in the
+``on_task_instance_failed``, you should use code like this:
+
+.. code-block:: python
+
+ from importlib.metadata import version
+ from packaging.version import Version
+ from airflow.listeners import hookimpl
+
+ airflow_version = Version(version("apache-airflow"))
+ if airflow_version >= Version("2.10.0"):
+
+ class ClassBasedListener:
+ ...
+
+ @hookimpl
+ def on_task_instance_failed(
+ self, previous_state, task_instance, error: None | str | BaseException, session
+ ):
+ # Handle error case here
+ pass
+
+ else:
+
+ class ClassBasedListener: # type: ignore[no-redef]
+ ...
+
+ @hookimpl
+ def on_task_instance_failed(self, previous_state, task_instance, session):
+ # Handle no error case here
+ pass
+
+
+List of changes in the listener interfaces since 2.8.0 when they were introduced:
+
+
++-----------------+-----------------------------+---------------------------------------+
+| Airflow Version | Affected method | Change |
++=================+=============================+=======================================+
+| 2.10.0 | ``on_task_instance_failed`` | An error field added to the interface |
++-----------------+-----------------------------+---------------------------------------+