Skip to content

Commit

Permalink
Make arguments 'offset' and 'length' not required (#28234)
Browse files Browse the repository at this point in the history 
* Make arguments 'offset' and 'length' not required

* Disable implicit optional for azure-storage (mypy)
  • Loading branch information
@malthe
malthe authored Dec 9, 2022
1 parent e247336 commit 7e776db
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 2 deletions.
2 changes: 1 addition & 1 deletion airflow/providers/microsoft/azure/hooks/wasb.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -385,7 +385,7 @@ def upload(
return blob_client.upload_blob(data, blob_type, length=length, **kwargs)

def download(
self, container_name, blob_name, offset: int, length: int, **kwargs
self, container_name, blob_name, offset: int | None = None, length: int | None = None, **kwargs
) -> StorageStreamDownloader:
"""
Downloads a blob to the StorageStreamDownloader
Expand Down
2 changes: 1 addition & 1 deletion setup.cfg
View file
Original file line number Diff line number Diff line change
Expand Up @@ -191,5 +191,5 @@ no_implicit_optional = False
#Let's assume all azure packages have no implicit optional
[mypy-azure.batch.*]
no_implicit_optional = False
[mypy-azure.batch.models.*]
[mypy-azure.storage.*]
no_implicit_optional = False

2 comments on commit 7e776db

@aresabalo
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This commit is causing failure retrieving logs from Azure Blog Storage when upgrade to airflow 2.5.0 from 2.4.3.

[2022-12-14 21:18:55,708] {wasb_task_handler.py:162} ERROR - Could not read logs from wasb-airflow-logs/dag_id=airflow-log-cleanup/run_id=scheduled__2022-12-13T00:00:00+00:00/task_id=log_cleanup_worker_num_1_dir_0/attempt=1.log
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.9/site-packages/airflow/providers/microsoft/azure/log/wasb_task_handler.py", line 159, in wasb_read
return self.hook.read_file(self.wasb_container, remote_log_location)
File "/home/airflow/.local/lib/python3.9/site-packages/airflow/providers/microsoft/azure/hooks/wasb.py", line 356, in read_file
return self.download(container_name, blob_name, **kwargs).content_as_text()
TypeError: download() missing 2 required positional arguments: 'offset' and 'length'

@potiuk
Copy link
Member

@potiuk potiuk commented on 7e776db Dec 30, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really. This commit actually fixed the problem introduced in #25426 . We released ad-hoc provider version 5.0.1 specifically to address this feature https://airflow.apache.org/docs/apache-airflow-providers-microsoft-azure/stable/index.html#id1 - please upgrede the provider to 5.0.1 to fix the problem.

Please sign in to comment.