Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix broken output format(yaml) prompt #1048

Merged
merged 1 commit into from
Jul 21, 2024

Conversation

woung717
Copy link
Contributor

@woung717 woung717 commented Jul 19, 2024

User description

Since the code_feedback field is a property of the YAML output, a colon (:) should follow it. Some LLM models might omit the colon, causing a parsing error.

  • Another suggestion
    I observed sometimes that text output in relevant_line property has broken indentation. I think adding multiple line example with indentation could better indent output.
  relevant_line: |
    xxx
    xxx # added

PR Type

bug fix, enhancement


Description

  • Fixed missing colon after code_feedback field to correct YAML format in pr_agent/settings/pr_reviewer_prompts.toml.
  • Improved indentation for relevant_line property to ensure proper formatting.

Changes walkthrough 📝

Relevant files
Bug fix
pr_reviewer_prompts.toml
Fix YAML output format and improve indentation                     

pr_agent/settings/pr_reviewer_prompts.toml

  • Fixed missing colon after code_feedback field to correct YAML format.
  • Improved indentation for relevant_line property to ensure proper
    formatting.
  • +1/-1     

    💡 PR-Agent usage:
    Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

    Copy link
    Contributor

    PR Reviewer Guide 🔍

    ⏱️ Estimated effort to review: 1 🔵⚪⚪⚪⚪
    🏅 Score: 95
    🧪 No relevant tests
    🔒 No security concerns identified
    🔀 No multiple PR themes
    ⚡ No key issues to review

    Copy link
    Contributor

    PR Code Suggestions ✨

    CategorySuggestion                                                                                                                                    Score
    Possible issue
    Correct the formatting of the 'code_feedback' block to ensure proper parsing

    Ensure that the 'code_feedback' block is properly formatted as a list in the TOML
    configuration. The current format might cause parsing issues. Use double quotes for
    string values and ensure proper indentation.

    pr_agent/settings/pr_reviewer_prompts.toml [168-171]

     code_feedback:
    -- relevant_file: |
    -    directory/xxx.py
    -  language: |
    +  - relevant_file: "directory/xxx.py"
    +    language: "toml"
     
    • Apply this suggestion
    Suggestion importance[1-10]: 8

    Why: The suggestion correctly identifies a potential formatting issue in the TOML configuration and provides a clear improvement to ensure proper parsing. Proper indentation and quoting string values are important for maintaining the integrity of the configuration file.

    8

    @KennyDizi
    Copy link
    Contributor

    I think it would be great if we moved the content to the right side a bit, like this
    CleanShot 2024-07-19 at 17 46 25

    @mrT23
    Copy link
    Collaborator

    mrT23 commented Jul 21, 2024

    thanks @woung717
    This code is not on by default, and is not a major feature. improve should be used instead. So I don't see a big reason to further optimize this prompt. improve generates a valid YAML in a stable and consistent way with GPT4 and Claude

    @KennyDizi output (and input) should be short and concise as possible, if it contains the same data. This saves time and money (fewer tokens).
    Adding extra space is against this principle

    @mrT23 mrT23 merged commit 6b9de6b into Codium-ai:main Jul 21, 2024
    @KennyDizi
    Copy link
    Contributor

    thanks @woung717 This code is not on by default, and is not a major feature. improve should be used instead. So I don't see a big reason to further optimize this prompt. improve generates a valid YAML in a stable and consistent way with GPT4 and Claude

    @KennyDizi output (and input) should be short and concise as possible, if it contains the same data. This saves time and money (fewer tokens). Adding extra space is against this principle

    That's good to know. If that's so, with the same approach, should we remove the space in the other similar places? @mrT23

    @woung717
    Copy link
    Contributor Author

    woung717 commented Jul 23, 2024

    I think the short and concise prompt principle is good. But modifying the prompt requires a fair amount of testing to make sure it keeps valid format, especially since this project parses structured text (YAML) directly from the LLM response. Maybe it is good to see if it is done correctly, but I think there is not much benefit for the effort. This is just my 2 cent.

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    3 participants