Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatically enable /improve --extended mode for large PRs #564

Merged
merged 2 commits into from
Jan 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/IMPROVE.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,10 @@ Under the section 'pr_code_suggestions', the [configuration file](./../pr_agent/
- `include_improved_code`: if set to true, the tool will include an improved code implementation in the suggestion. Default is true.

#### params for '/improve --extended' mode
- `auto_extended_mode`: enable extended mode automatically for large PRs. Default is true.
- `auto_extended_mode_min_files`: minimum number of files in the PR to automatically enable extended mode. Default is 1.
- `auto_extended_mode_min_additions`: minimum number of line additions in the PR to automatically enable extended mode. Default is 500.
- `auto_extended_mode_min_deletions`: minimum number of line deletions in the PR to automatically enable extended mode. Default is 0.
- `num_code_suggestions_per_chunk`: number of code suggestions provided by the 'improve' tool, per chunk. Default is 8.
- `rank_extended_suggestions`: if set to true, the tool will rank the suggestions, based on importance. Default is true.
- `max_number_of_calls`: maximum number of chunks. Default is 5.
Expand Down
4 changes: 4 additions & 0 deletions pr_agent/settings/configuration.toml
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,10 @@ include_improved_code = true
extra_instructions = ""
rank_suggestions = false
# params for '/improve --extended' mode
auto_extended_mode=true
auto_extended_mode_min_files=1
auto_extended_mode_min_additions=500
auto_extended_mode_min_deletions=0
num_code_suggestions_per_chunk=8
rank_extended_suggestions = true
max_number_of_calls = 5
Expand Down
27 changes: 24 additions & 3 deletions pr_agent/tools/pr_code_suggestions.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ def __init__(self, pr_url: str, cli_mode=False, args: list = None,

# extended mode
try:
self.is_extended = any(["extended" in arg for arg in args])
self.is_extended = self._get_is_extended(args or [])
except:
self.is_extended = False
if self.is_extended:
Expand Down Expand Up @@ -206,6 +206,21 @@ def dedent_code(self, relevant_file, relevant_lines_start, new_code_snippet):

return new_code_snippet

def _get_is_extended(self, args: list[str]) -> bool:
"""Check if extended mode should be enabled by the `--extended` flag or automatically according to the PR"""
if any(["extended" in arg for arg in args]):
get_logger().info("Extended mode is enabled by the `--extended` flag")
return True
if (
get_settings().pr_code_suggestions.auto_extended_mode
and self.git_provider.pr.changed_files >= get_settings().pr_code_suggestions.auto_extended_mode_min_files
and self.git_provider.pr.additions >= get_settings().pr_code_suggestions.auto_extended_mode_min_additions
and self.git_provider.pr.deletions >= get_settings().pr_code_suggestions.auto_extended_mode_min_deletions
):
get_logger().info("Extended mode is enabled automatically based on the PR size")
return True
return False

async def _prepare_prediction_extended(self, model: str) -> dict:
get_logger().info('Getting PR diff...')
patches_diff_list = get_pr_multi_diffs(self.git_provider, self.token_handler, model,
Expand Down Expand Up @@ -271,8 +286,14 @@ async def rank_suggestions(self, data: List) -> List:
data_sorted[importance_order - 1] = suggestion_list[suggestion_number - 1]

if get_settings().pr_code_suggestions.final_clip_factor != 1:
new_len = int(0.5 + len(data_sorted) * get_settings().pr_code_suggestions.final_clip_factor)
data_sorted = data_sorted[:new_len]
max_len = max(
len(data_sorted),
get_settings().pr_code_suggestions.num_code_suggestions,
get_settings().pr_code_suggestions.num_code_suggestions_per_chunk,
)
new_len = int(0.5 + max_len * get_settings().pr_code_suggestions.final_clip_factor)
if new_len < len(data_sorted):
data_sorted = data_sorted[:new_len]
Comment on lines +289 to +296
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is intended to handle an edge-case where the total size of data_sorted before clipping is already small enough in comparison to num_code_suggestions or num_code_suggestions_per_chunk, and so clipping it will result it potentially much fewer suggestions.

For example:
Assume num_code_suggestions_per_chunk=10, final_clip_factor=0.5 and len(data_sorted)==2 before clipping. Based on these configurations, it would be reasonable to expect that the PR-Agent will give us all of the (2) suggestions it generated, but in reality it would have applied the final_clip_factor and cut the data_sorted list in half, giving us only the first suggestion.
With this "fix" the final_clip_factor won't reduce the size of the data_sorted list if it's already small enough relative to the max number of suggestions we set in the configuration.

except Exception as e:
if get_settings().config.verbosity_level >= 1:
get_logger().info(f"Could not sort suggestions, error: {e}")
Expand Down