Fix WASB remote logging base path handling (#58946)

[nailo2c]

Closes: #58946

Why

Following this doc causes Airflow to write logs under an unexpected <no name> folder in Azure Blob Storage.

How

wasb is used as a flag to select the Azure Blob remote logging implementation, but the current code path does not normalize the wasb:// scheme.

airflow/airflow-core/src/airflow/config_templates/airflow_local_settings.py

Lines 215 to 216 in 89f109b

	elif remote_base_log_folder.startswith("wasb"):
	from airflow.providers.microsoft.azure.log.wasb_task_handler import WasbRemoteLogIO

In the documentation prior to version 12.4.1, users were instructed to use a wasb- prefix (e.g. wasb-logs). However, PR #51988 notes that this could lead to ResourceNotFoundError and authentication failures.

As a result, the documentation was updated to the current guidance (using remote_base_log_folder = wasb://...):

• Current docs: https://airflow.apache.org/docs/apache-airflow-providers-microsoft-azure/stable/logging/index.html
• Older docs (12.4.1): https://airflow.apache.org/docs/apache-airflow-providers-microsoft-azure/12.4.1/logging/index.html

But the current implementation treats wasb:// as part of the blob name when constructing the remote log path:

airflow/providers/microsoft/azure/src/airflow/providers/microsoft/azure/log/wasb_task_handler.py

Line 60 in 89f109b

remote_loc = os.path.join(self.remote_base, path)

1. wasb-logs (works)

self.remote_base = "wasb-logs"
os.path.join("wasb-logs", "dag_id=xxx/task_id=yyy/attempt=1.log")                                                                       

# Result: "wasb-logs/dag_id=xxx/task_id=yyy/attempt=1.log"                                                                                
# This is a valid Azure Blob name

2. wasb://logs (problematic)

self.remote_base = "wasb://logs"
os.path.join("wasb://logs", "dag_id=xxx/task_id=yyy/attempt=1.log")

# Result: "wasb://logs/dag_id=xxx/task_id=yyy/attempt=1.log"
# The `wasb://` scheme is being treated as part of the blob name.
# In Azure Portal, this ends up looking like a weird virtual folder structure,
# e.g. `wasb:` / `{No Name}` / `logs` / ..

What

Normalize remote_base_log_folder by stripping the wasb:// prefix before constructing the blob key:

if remote_base_log_folder.startswith("wasb://"):
    wasb_remote_base = remote_base_log_folder.removeprefix("wasb://")
else:
    wasb_remote_base = remote_base_log_folder

I intentionally did not change the selection logic to elif remote_base_log_folder.startswith("wasb://"): because I do not want to impact users who currently configure wasb-... in their settings.

airflow/airflow-core/src/airflow/config_templates/airflow_local_settings.py

Line 215 in 89f109b

elif remote_base_log_folder.startswith("wasb"):

Result:

---

Was generative AI tooling used to co-author this PR?

• No (please specify the tool below)

[SameerMesiah97]

Looks good to me. Just one tiny nit.

Overall, this is a very safe change that fixes a legitimate issue.

[kalluripradeep]

Nice fix! This addresses a real issue with the current documentation guidance.

The solution is clean and backward-compatible - stripping the wasb:// prefix while still supporting the legacy wasb- format is the right approach.

I agree with @SameerMesiah97 suggestion to add a comment explaining the normalization. Something like:

if remote_base_log_folder.startswith("wasb://"):
    # Strip wasb:// scheme to get clean blob path
    # Docs recommend wasb://container format but os.path.join treats scheme as part of path
    wasb_remote_base = remote_base_log_folder.removeprefix("wasb://")
else:
    # Support legacy wasb- prefix format for backward compatibility  
    wasb_remote_base = remote_base_log_folder

This would help future maintainers understand why this normalization exists.

Otherwise ! ✅

[nailo2c]

Thanks for the suggestion! I'm hesitant to make that change right now as I'm not sure if it fits the project norms. I'd prefer to wait and see what a committer thinks first.

[nailo2c]

Applied all suggestions, please let me know if anything else needs adjustment. Thanks :)

[alessandro-peyrachia-rnc]

Thanks for the fix!
would it be a good idea to update docs to report this behaviour affecting versions from 12.4.1 up to 3.1.7?

[nailo2c]

Thanks for the suggestion! The bug fix will be recorded in the CHANGELOG, so users should be able to track affected versions from there. No need to worry :)

[jason810496]

Thanks for the fix!

[github-actions]

Backport successfully created: v3-1-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-1-test