Skip to content

Comments

Set the JWTGenerator private_key field to None for log retrieval#54442

Merged
ashb merged 1 commit intoapache:mainfrom
ianbuss:do-not-load-pk-file-task-handler
Aug 13, 2025
Merged

Set the JWTGenerator private_key field to None for log retrieval#54442
ashb merged 1 commit intoapache:mainfrom
ianbuss:do-not-load-pk-file-task-handler

Conversation

@ianbuss
Copy link
Contributor

@ianbuss ianbuss commented Aug 13, 2025

In a recent change, the private_key parameter is left empty when instantiating the JWTGenerator used to authenticate log serving requests for in-progress tasks. Such requests use the shared secret_key instead. However, leaving private_key unset means that it is automatically populated through the attrs field factory and this leads to a failure in the post init which asserts mutual exclusivity of secret_key and private_key. The result is stack traces in the API Server logs like the following when retrieving "live" logs:

[2025-08-12T19:33:59.403+0000] {file_task_handler.py:907} ERROR - Could not read served logs
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/airflow/utils/log/file_task_handler.py", line 878, in _read_from_logs_server
    response = _fetch_logs_from_service(url, rel_path)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/airflow/utils/log/file_task_handler.py", line 173, in _fetch_logs_from_service
    generator = JWTGenerator(
                ^^^^^^^^^^^^^
  File "<attrs generated methods airflow.api_fastapi.auth.tokens.JWTGenerator>", line 34, in __init__
    self.__attrs_post_init__()
  File "/usr/local/lib/python3.12/site-packages/airflow/api_fastapi/auth/tokens.py", line 417, in __attrs_post_init__
    raise ValueError("Exactly one of private_key and secret_key must be specified")
ValueError: Exactly one of private_key and secret_key must be specified

The fix is to add back the explicit private_key=None, for which we have to add a mypy type ignore comment as it does not seem to understand that None is allowed for AllowedPrivateKeys | None.


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@rawwar rawwar requested a review from ashb August 13, 2025 08:45
@kaxil kaxil added the backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch label Aug 13, 2025
@ashb ashb added this to the Airflow 3.0.5 milestone Aug 13, 2025
@ashb ashb merged commit 4fa0d01 into apache:main Aug 13, 2025
59 checks passed
github-actions bot pushed a commit that referenced this pull request Aug 13, 2025
… for service logs retrieval (#54442)

In a recent change, the private_key parameter is left empty when instantiating the JWTGenerator used to authenticate log serving requests for in-progress tasks. Such requests use the shared secret_key instead. However, leaving private_key unset means that it is automatically populated through the attrs field factory and this leads to a failure in the post init which asserts mutual exclusivity of secret_key and private_key. The result is stack traces in the API Server logs like the following when retrieving "live" logs:

```
[2025-08-12T19:33:59.403+0000] {file_task_handler.py:907} ERROR - Could not read served logs
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/airflow/utils/log/file_task_handler.py", line 878, in _read_from_logs_server
    response = _fetch_logs_from_service(url, rel_path)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/airflow/utils/log/file_task_handler.py", line 173, in _fetch_logs_from_service
    generator = JWTGenerator(
                ^^^^^^^^^^^^^
  File "<attrs generated methods airflow.api_fastapi.auth.tokens.JWTGenerator>", line 34, in __init__
    self.__attrs_post_init__()
  File "/usr/local/lib/python3.12/site-packages/airflow/api_fastapi/auth/tokens.py", line 417, in __attrs_post_init__
    raise ValueError("Exactly one of private_key and secret_key must be specified")
ValueError: Exactly one of private_key and secret_key must be specified
```

The fix is to add back the explicit `private_key=None`, for which we have to add a mypy type ignore comment as it does not seem to understand that `None` is allowed for `AllowedPrivateKeys | None`.
(cherry picked from commit 4fa0d01)

Co-authored-by: Ian Buss <ianbuss@users.noreply.github.com>
@github-actions
Copy link

Backport successfully created: v3-0-test

Status Branch Result
v3-0-test PR Link

github-actions bot pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Aug 13, 2025
… for service logs retrieval (apache#54442)

In a recent change, the private_key parameter is left empty when instantiating the JWTGenerator used to authenticate log serving requests for in-progress tasks. Such requests use the shared secret_key instead. However, leaving private_key unset means that it is automatically populated through the attrs field factory and this leads to a failure in the post init which asserts mutual exclusivity of secret_key and private_key. The result is stack traces in the API Server logs like the following when retrieving "live" logs:

```
[2025-08-12T19:33:59.403+0000] {file_task_handler.py:907} ERROR - Could not read served logs
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/airflow/utils/log/file_task_handler.py", line 878, in _read_from_logs_server
    response = _fetch_logs_from_service(url, rel_path)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/airflow/utils/log/file_task_handler.py", line 173, in _fetch_logs_from_service
    generator = JWTGenerator(
                ^^^^^^^^^^^^^
  File "<attrs generated methods airflow.api_fastapi.auth.tokens.JWTGenerator>", line 34, in __init__
    self.__attrs_post_init__()
  File "/usr/local/lib/python3.12/site-packages/airflow/api_fastapi/auth/tokens.py", line 417, in __attrs_post_init__
    raise ValueError("Exactly one of private_key and secret_key must be specified")
ValueError: Exactly one of private_key and secret_key must be specified
```

The fix is to add back the explicit `private_key=None`, for which we have to add a mypy type ignore comment as it does not seem to understand that `None` is allowed for `AllowedPrivateKeys | None`.
(cherry picked from commit 4fa0d01)

Co-authored-by: Ian Buss <ianbuss@users.noreply.github.com>
ashb pushed a commit that referenced this pull request Aug 13, 2025
… for service logs retrieval (#54442) (#54444)

In a recent change, the private_key parameter is left empty when instantiating the JWTGenerator used to authenticate log serving requests for in-progress tasks. Such requests use the shared secret_key instead. However, leaving private_key unset means that it is automatically populated through the attrs field factory and this leads to a failure in the post init which asserts mutual exclusivity of secret_key and private_key. The result is stack traces in the API Server logs like the following when retrieving "live" logs:

```
[2025-08-12T19:33:59.403+0000] {file_task_handler.py:907} ERROR - Could not read served logs
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/airflow/utils/log/file_task_handler.py", line 878, in _read_from_logs_server
    response = _fetch_logs_from_service(url, rel_path)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/airflow/utils/log/file_task_handler.py", line 173, in _fetch_logs_from_service
    generator = JWTGenerator(
                ^^^^^^^^^^^^^
  File "<attrs generated methods airflow.api_fastapi.auth.tokens.JWTGenerator>", line 34, in __init__
    self.__attrs_post_init__()
  File "/usr/local/lib/python3.12/site-packages/airflow/api_fastapi/auth/tokens.py", line 417, in __attrs_post_init__
    raise ValueError("Exactly one of private_key and secret_key must be specified")
ValueError: Exactly one of private_key and secret_key must be specified
```

The fix is to add back the explicit `private_key=None`, for which we have to add a mypy type ignore comment as it does not seem to understand that `None` is allowed for `AllowedPrivateKeys | None`.
(cherry picked from commit 4fa0d01)

Co-authored-by: Ian Buss <ianbuss@users.noreply.github.com>
meldiner pushed a commit to faros-ai/airflow that referenced this pull request Feb 6, 2026
… for service logs retrieval (apache#54442) (apache#54444)

In a recent change, the private_key parameter is left empty when instantiating the JWTGenerator used to authenticate log serving requests for in-progress tasks. Such requests use the shared secret_key instead. However, leaving private_key unset means that it is automatically populated through the attrs field factory and this leads to a failure in the post init which asserts mutual exclusivity of secret_key and private_key. The result is stack traces in the API Server logs like the following when retrieving "live" logs:

```
[2025-08-12T19:33:59.403+0000] {file_task_handler.py:907} ERROR - Could not read served logs
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/airflow/utils/log/file_task_handler.py", line 878, in _read_from_logs_server
    response = _fetch_logs_from_service(url, rel_path)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/airflow/utils/log/file_task_handler.py", line 173, in _fetch_logs_from_service
    generator = JWTGenerator(
                ^^^^^^^^^^^^^
  File "<attrs generated methods airflow.api_fastapi.auth.tokens.JWTGenerator>", line 34, in __init__
    self.__attrs_post_init__()
  File "/usr/local/lib/python3.12/site-packages/airflow/api_fastapi/auth/tokens.py", line 417, in __attrs_post_init__
    raise ValueError("Exactly one of private_key and secret_key must be specified")
ValueError: Exactly one of private_key and secret_key must be specified
```

The fix is to add back the explicit `private_key=None`, for which we have to add a mypy type ignore comment as it does not seem to understand that `None` is allowed for `AllowedPrivateKeys | None`.
(cherry picked from commit 4fa0d01)

Co-authored-by: Ian Buss <ianbuss@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:logging backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants