Skip to content

[Data] Decrease scale of map_groups release test #58711

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
bveeramani merged 1 commit into from
Nov 17, 2025
Merged

[Data] Decrease scale of map_groups release test #58711

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion release/release_data_tests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -193,7 +193,7 @@
run:
timeout: 3600
script: >
python groupby_benchmark.py --sf 100 --map-groups --group-by {{columns}}
python groupby_benchmark.py --sf 10 --map-groups --group-by {{columns}}
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Nov 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This change is a good temporary fix to unblock the release tests. To improve long-term maintainability, it would be beneficial to add a TODO comment in the code explaining why the scale factor was reduced and referencing the tracking issue (#58312).

Since this script block uses the folded block scalar (>), adding an inline comment will comment out the rest of the script. You could change the block scalar to literal style (|) on line 195 to add the comment on a separate line.

For example:

  run:
    timeout: 3600
    script: |
      # TODO(#58312): Revert to --sf 100 once the scalability issue is fixed.
      python groupby_benchmark.py --sf 10 --map-groups --group-by {{columns}}
      --shuffle-strategy {{shuffle_strategy}}

This will ensure the reason for this temporary change is not lost over time.

--shuffle-strategy {{shuffle_strategy}}
###############
Expand Down