Skip to content

Flattening the requirements input for python venv operators #50521

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
eladkal merged 2 commits into from
May 13, 2025
Merged

Flattening the requirements input for python venv operators #50521

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2f52f69
Flattening the requirements for python virtual env operator
amoghrajesh May 13, 2025
02c7861
Refine BigQuery test with field-wise comparison of access_entries
amoghrajesh May 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion providers/google/tests/unit/google/cloud/hooks/test_bigquery.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -323,7 +323,12 @@ def test_run_grant_dataset_view_access_granting(self, mock_update, mock_get):
)

mock_get.assert_called_once_with(project_id=PROJECT_ID, dataset_id=DATASET_ID)
assert view_access in dataset.access_entries
assert any(
entry.role == view_access.role
and entry.entity_type == view_access.entity_type
and entry.entity_id == view_access.entity_id
for entry in dataset.access_entries
), f"View access entry not found in {dataset.access_entries}"
mock_update.assert_called_once_with(
fields=["access"],
dataset_resource=dataset.to_api_repr(),
Expand Down
9 changes: 8 additions & 1 deletion providers/standard/src/airflow/providers/standard/operators/python.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -859,7 +859,14 @@ def _iter_serializable_context_keys(self):
# If we're using system packages, assume both are present
found_airflow = found_pendulum = True
else:
for raw_str in self.requirements:
requirements_iterable = []
if isinstance(self.requirements, str):
Copy link
Contributor

@yannlambret yannlambret May 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I missed the fact that requirements could be other thing than a list. But based on the constructor code, it looks impossible to get a string here:

        elif isinstance(requirements, str):
            self.requirements = [requirements]

Copy link
Contributor Author

@amoghrajesh amoghrajesh May 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call.

requirements_iterable = self.requirements.splitlines()
else:
for item in self.requirements:
requirements_iterable.extend(item.splitlines())

for raw_str in requirements_iterable:
line = raw_str.strip()
# Skip blank lines and full‐line comments
if not line or line.startswith("#"):
Expand Down
Loading