Log serving fails for Celery with InvalidKeyError

[ianbuss]

Apache Airflow version

3.0.0

If "Other Airflow 2 version" selected, which one?

No response

What happened?

When running a task and viewing the live logs in the UI, the following is seen:

Could not read served logs: Expecting a EllipticCurvePrivateKey/EllipticCurvePublicKey. Wrong key provided for EdDSA algorithms

The apiserver logs the following stack trace:

[2025-04-03T17:31:27.300+0000] {file_task_handler.py:590} ERROR - Could not read served logs
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/airflow/utils/log/file_task_handler.py", line 567, in _read_from_logs_server
    response = _fetch_logs_from_service(url, rel_path)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/airflow/utils/log/file_task_handler.py", line 112, in _fetch_logs_from_service
    headers={"Authorization": generator.generate({"filename": log_relative_path})},
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/airflow/api_fastapi/auth/tokens.py", line 458, in generate
    return jwt.encode(claims, self.signing_arg, algorithm=self.algorithm, headers=headers)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/jwt/api_jwt.py", line 78, in encode
    return api_jws.encode(
           ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/jwt/api_jws.py", line 170, in encode
    key = alg_obj.prepare_key(key)
          ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/jwt/algorithms.py", line 738, in prepare_key
    raise InvalidKeyError(
jwt.exceptions.InvalidKeyError: Expecting a EllipticCurvePrivateKey/EllipticCurvePublicKey. Wrong key provided for EdDSA algorithms

What you think should happen instead?

The apiserver should be able to read logs from the worker using the pre-shared secret in webserver.secret_key

How to reproduce

Create an Airflow 3 Celery instance, run a task that runs for a while, and view the live logs in the UI.

Operating System

Linux

Versions of Apache Airflow Providers

No response

Deployment

Astronomer

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct