Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: work on programmatic interface, self-reviewing agent #199

Merged
merged 7 commits into from
Oct 15, 2024

Conversation

ErikBjare
Copy link
Owner

@ErikBjare ErikBjare commented Oct 15, 2024

  • refactored LogManager into mutable manager and immutable Log dataclass
  • added wip treeofthought script

TODO

  • Manually test confirmation refactor

Important

Refactor logging system with new Log dataclass and add tree-branching conversation script.

  • Refactor LogManager:
    • Introduced Log dataclass in logmanager.py for immutable message handling.
    • Updated LogManager to use Log for managing conversation logs.
    • Removed prepare_messages from LogManager, added as standalone function.
  • Scripts:
    • Added treeofthoughts.py for tree-branching conversation evaluation.
  • Imports and Usage:
    • Updated imports and usage of LogManager and Log in chat.py, cli.py, commands.py, server/api.py, and tools/chats.py.
    • Replaced Conversation with ConversationMeta in cli.py and list_user_messages.py.

This description was created by Ellipsis for eff6ab1. It will automatically update as commits are pushed.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ Changes requested. Reviewed everything up to 36d1b7f in 1 minute and 29 seconds

More details
  • Looked at 857 lines of code in 9 files
  • Skipped 0 files when reviewing.
  • Skipped posting 3 drafted comments based on config settings.
1. gptme/logmanager.py:6
  • Draft comment:
    Ensure that the replace method is imported from dataclasses to avoid potential NameError issues.
from dataclasses import dataclass, field, replace
  • Reason this comment was not posted:
    Comment did not seem useful.
2. gptme/server/api.py:113
  • Draft comment:
    Ensure manager.log is not empty before accessing manager.log.messages to prevent potential AttributeError.
if manager.log:
    msgs = prepare_messages(manager.log.messages)
else:
    msgs = []
  • Reason this comment was not posted:
    Comment did not seem useful.
3. scripts/treeofthoughts.py:23
  • Draft comment:
    Ensure that _step always yields valid Message objects to prevent runtime errors when appending to log.
  • Reason this comment was not posted:
    Comment did not seem useful.

Workflow ID: wflow_La8vX5FjzlZhjaLR


Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

gptme/logmanager.py Outdated Show resolved Hide resolved
@codecov-commenter
Copy link

codecov-commenter commented Oct 15, 2024

Codecov Report

Attention: Patch coverage is 77.07317% with 47 lines in your changes missing coverage. Please review.

Project coverage is 79.88%. Comparing base (798b754) to head (eff6ab1).
Report is 8 commits behind head on master.

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
gptme/commands.py 72.97% 10 Missing ⚠️
gptme/logmanager.py 87.50% 9 Missing ⚠️
gptme/tools/shell.py 65.21% 8 Missing ⚠️
gptme/tools/save.py 63.63% 4 Missing ⚠️
gptme/chat.py 84.21% 3 Missing ⚠️
gptme/message.py 40.00% 3 Missing ⚠️
gptme/server/api.py 72.72% 3 Missing ⚠️
gptme/tools/patch.py 60.00% 2 Missing ⚠️
gptme/tools/python.py 71.42% 2 Missing ⚠️
gptme/tools/tmux.py 66.66% 2 Missing ⚠️
... and 1 more
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #199      +/-   ##
==========================================
- Coverage   80.34%   79.88%   -0.47%     
==========================================
  Files          56       56              
  Lines        3327     3365      +38     
==========================================
+ Hits         2673     2688      +15     
- Misses        654      677      +23     
Flag Coverage Δ
anthropic/claude-3-haiku-20240307 78.72% <77.07%> (-0.54%) ⬇️
openai/gpt-4o-mini 78.63% <77.07%> (-0.24%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

scripts/treeofthoughts.py Outdated Show resolved Hide resolved
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Incremental review on e63d525 in 33 seconds

More details
  • Looked at 295 lines of code in 2 files
  • Skipped 0 files when reviewing.
  • Skipped posting 7 drafted comments based on config settings.
1. scripts/treeofthoughts.py:25
  • Draft comment:
    Consider handling exceptions like subprocess.CalledProcessError to ensure robustness when running subprocess commands.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The use of subprocess.run without handling potential exceptions can lead to unhandled errors if the command fails. It's a good practice to handle exceptions like subprocess.CalledProcessError to ensure robustness.
2. scripts/treeofthoughts.py:31
  • Draft comment:
    Consider handling exceptions like subprocess.CalledProcessError to ensure robustness when running subprocess commands.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The use of subprocess.run without handling potential exceptions can lead to unhandled errors if the command fails. It's a good practice to handle exceptions like subprocess.CalledProcessError to ensure robustness.
3. scripts/treeofthoughts.py:39
  • Draft comment:
    Consider handling exceptions like subprocess.CalledProcessError to ensure robustness when running subprocess commands.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The use of subprocess.run without handling potential exceptions can lead to unhandled errors if the command fails. It's a good practice to handle exceptions like subprocess.CalledProcessError to ensure robustness.
4. scripts/treeofthoughts.py:97
  • Draft comment:
    Consider handling exceptions like subprocess.CalledProcessError to ensure robustness when running subprocess commands.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The use of subprocess.run without handling potential exceptions can lead to unhandled errors if the command fails. It's a good practice to handle exceptions like subprocess.CalledProcessError to ensure robustness.
5. scripts/treeofthoughts.py:126
  • Draft comment:
    Consider handling exceptions like subprocess.CalledProcessError to ensure robustness when running subprocess commands.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The use of subprocess.run without handling potential exceptions can lead to unhandled errors if the command fails. It's a good practice to handle exceptions like subprocess.CalledProcessError to ensure robustness.
6. scripts/treeofthoughts.py:187
  • Draft comment:
    Consider handling exceptions like subprocess.CalledProcessError to ensure robustness when running subprocess commands.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The use of subprocess.run without handling potential exceptions can lead to unhandled errors if the command fails. It's a good practice to handle exceptions like subprocess.CalledProcessError to ensure robustness.
7. gptme/chat.py:164
  • Draft comment:
    Use Union[Log, List[Message]] for type hinting instead of Log | list[Message] for better clarity and compatibility.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The current implementation of the step function in gptme/chat.py uses Log | list[Message] as a type hint for the log parameter. This can be improved for clarity and type safety by using Union[Log, List[Message]].

Workflow ID: wflow_PxtchXuYJyA1Sxt8


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

@ErikBjare ErikBjare mentioned this pull request Oct 15, 2024
@ErikBjare ErikBjare force-pushed the dev/programmatic-api-and-treeofthoughts branch from e63d525 to 76b9abf Compare October 15, 2024 12:52
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Incremental review on 76b9abf in 48 seconds

More details
  • Looked at 295 lines of code in 2 files
  • Skipped 0 files when reviewing.
  • Skipped posting 5 drafted comments based on config settings.
1. scripts/treeofthoughts.py:47
  • Draft comment:
    Use triple backticks for code blocks in Python strings.
        context += f"

{f}\n"

- **Reason this comment was not posted:** 
Comment looked like it was already resolved.

</details>

<details>
<summary>2. <code>scripts/treeofthoughts.py:25</code></summary>

- **Draft comment:** 
Consider adding exception handling for subprocess.run to handle potential errors from the git command.
- **Reason this comment was not posted:** 
Decided after close inspection that this draft comment was likely wrong and/or not actionable:
The comment is relevant because subprocess.run is used in the new functions, and it can raise exceptions. Adding exception handling could improve the robustness of the code. However, the current usage does not use check=True, so the main concern would be handling FileNotFoundError. The comment is actionable and suggests a clear improvement to the code.
The comment does not specify which exceptions to handle or how to handle them, which might make it less actionable. Additionally, if the subprocess.run calls are expected to always succeed in the given environment, exception handling might be unnecessary.
Even if the environment is controlled, handling potential exceptions can prevent unexpected crashes and improve code robustness. The comment is a general suggestion that can be refined by the developer.
Keep the comment as it suggests a valid improvement to handle potential errors from subprocess.run, which is used in the new code.

</details>

<details>
<summary>3. <code>scripts/treeofthoughts.py:97</code></summary>

- **Draft comment:** 
Consider adding exception handling for subprocess.run to handle potential errors from the make command.
- **Reason this comment was not posted:** 
Marked as duplicate.

</details>

<details>
<summary>4. <code>scripts/treeofthoughts.py:187</code></summary>

- **Draft comment:** 
Consider adding exception handling for subprocess.run to handle potential errors from the git command.
- **Reason this comment was not posted:** 
Marked as duplicate.

</details>

<details>
<summary>5. <code>scripts/treeofthoughts.py:204</code></summary>

- **Draft comment:** 
Ensure that the Log class supports the pop method or use an appropriate method to remove the last message.
- **Reason this comment was not posted:** 
Comment did not seem useful.

</details>


Workflow ID: <workflowid>`wflow_jBkuZtgeBxNvQ5Nc`</workflowid>

</details>


----
You can customize Ellipsis with :+1: / :-1: [feedback](https://docs.ellipsis.dev/review), review rules, user-specific overrides, `quiet` mode, and [more](https://docs.ellipsis.dev/config).

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ Changes requested. Incremental review on 11df31e in 38 seconds

More details
  • Looked at 862 lines of code in 13 files
  • Skipped 0 files when reviewing.
  • Skipped posting 2 drafted comments based on config settings.
1. gptme/tools/save.py:9
  • Draft comment:
    The ask_execute function is no longer used and should be removed to clean up the code.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The ask_execute function is no longer used in gptme/tools/save.py and should be removed to clean up the code.
2. gptme/tools/save.py:80
  • Draft comment:
    The ask_execute function is no longer used and should be removed to clean up the code.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The ask_execute function is no longer used in gptme/tools/save.py and should be removed to clean up the code.

Workflow ID: wflow_MWodEYP7BDsjI8Z0


Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

gptme/tools/python.py Outdated Show resolved Hide resolved
@ErikBjare ErikBjare force-pushed the dev/programmatic-api-and-treeofthoughts branch from 1de2a02 to 8281fdc Compare October 15, 2024 14:25
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Incremental review on 8281fdc in 19 seconds

More details
  • Looked at 32 lines of code in 2 files
  • Skipped 0 files when reviewing.
  • Skipped posting 1 drafted comments based on config settings.
1. gptme/tools/base.py:23
  • Draft comment:
    The ask_confirm function was removed but not replaced. If this function was used elsewhere, it might cause issues. Consider re-adding it or ensuring all references are updated.
  • Reason this comment was not posted:
    Comment did not seem useful.

Workflow ID: wflow_AJLn0lwNXLQphfZ1


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

@ErikBjare ErikBjare force-pushed the dev/programmatic-api-and-treeofthoughts branch from ec9da6a to 36dfa95 Compare October 15, 2024 14:26
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Incremental review on 36dfa95 in 18 seconds

More details
  • Looked at 32 lines of code in 2 files
  • Skipped 0 files when reviewing.
  • Skipped posting 1 drafted comments based on config settings.
1. gptme/tools/base.py:23
  • Draft comment:
    The refactoring of ConfirmFunc from a Protocol to a TypeAlias is a good simplification. However, ensure that the removal of ask_confirm does not affect other parts of the codebase that might rely on it.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The refactoring of ConfirmFunc from a Protocol to a TypeAlias is a good change for simplicity, but the removal of the ask_confirm function might affect other parts of the codebase that rely on it.

Workflow ID: wflow_2v0sbAjmPJeNoMu7


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Incremental review on 9850780 in 25 seconds

More details
  • Looked at 13 lines of code in 1 files
  • Skipped 0 files when reviewing.
  • Skipped posting 1 drafted comments based on config settings.
1. gptme/commands.py:127
  • Draft comment:
    The lambda function lambda _: True is redundant since it always returns True. Consider removing it or replacing it with a direct True value if possible.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The lambda function in the 'impersonate' case is redundant since it always returns True.

Workflow ID: wflow_whPC4ptbtuzpWjHf


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Incremental review on 696f706 in 28 seconds

More details
  • Looked at 35 lines of code in 2 files
  • Skipped 0 files when reviewing.
  • Skipped posting 1 drafted comments based on config settings.
1. gptme/tools/save.py:9
  • Draft comment:
    ask_execute is removed from util.py but still used here. This will cause a runtime error. Consider replacing it with confirm or another appropriate function.
  • Reason this comment was not posted:
    Decided after close inspection that this draft comment was likely wrong and/or not actionable:
    The comment is accurate as it identifies a potential runtime error due to the removal of 'ask_execute' from the imports. The code change in the diff shows that 'ask_execute' was replaced with 'confirm', which resolves the issue. However, the comment is not needed because the change has already been made in the diff.
    I might be missing the fact that the comment is pointing out a potential issue if 'ask_execute' was used elsewhere in the file, but the diff shows that it was replaced with 'confirm', resolving the issue.
    The comment is unnecessary because the issue it points out has already been resolved in the diff by replacing 'ask_execute' with 'confirm'.
    Delete the comment because the issue it points out has already been resolved in the diff.

Workflow ID: wflow_evGXI1Q1pvdt2X6Q


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Incremental review on eff6ab1 in 27 seconds

More details
  • Looked at 55 lines of code in 1 files
  • Skipped 0 files when reviewing.
  • Skipped posting 1 drafted comments based on config settings.
1. gptme/chat.py:115
  • Draft comment:
    set_interruptible() is called multiple times unnecessarily. It should be called once before the try block to avoid redundancy. This applies to lines 115, 195, and 224.
  • Reason this comment was not posted:
    Decided after close inspection that this draft comment was likely wrong and/or not actionable:
    The comment is addressing a change made in the diff, where set_interruptible() was moved inside the try block. The suggestion to call it once before the try block seems to be a valid code quality improvement, as it reduces redundancy. The comment is actionable and clear, suggesting a specific refactor to improve code quality.
    The comment assumes that calling set_interruptible() multiple times is unnecessary without considering if there might be a reason for its placement inside the try block. There might be a specific reason for setting it at those points, such as ensuring it is set right before certain operations.
    The comment is focused on reducing redundancy, which is generally a good practice unless there is a specific reason for the redundancy. The suggestion is clear and actionable, and if there is no specific reason for the redundancy, it should be addressed.
    The comment is valid as it addresses a change made in the diff and suggests a clear and actionable improvement to reduce redundancy in the code. It should be kept.

Workflow ID: wflow_7ySeT37n9LFPNodm


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

@ErikBjare ErikBjare merged commit 440aedb into master Oct 15, 2024
7 checks passed
@ErikBjare ErikBjare deleted the dev/programmatic-api-and-treeofthoughts branch October 15, 2024 14:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants