Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add anthropic computer use tools #765

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
Open

Conversation

kaavee315
Copy link
Contributor

@kaavee315 kaavee315 commented Oct 24, 2024

Important

Add tools for executing bash commands, computer interactions, and text file editing with Docker setup for the environment.

  • Tools:
    • Adds BashCommand in bash.py for executing bash commands with session management.
    • Adds Computer in computer.py for screen, keyboard, and mouse interactions, supporting macOS and Linux.
    • Adds TextEditor in text_editor.py for file operations like view, create, replace, insert, and undo.
  • File Management:
    • Updates File class in file.py to include undo functionality and history tracking.
  • Enums:
    • Adds ANTHROPIC to App in _app.py.
    • Adds ANTHROPIC_BASH_COMMAND, ANTHROPIC_COMPUTER, ANTHROPIC_TEXT_EDITOR to Action in _action.py.
    • Adds ANTHROPIC related tags to Tag in _tag.py.
  • Docker:
    • Adds Dockerfile.computer and Dockerfile.computer.dev for setting up the environment.
    • Updates Makefile to include targets for building these Docker images.
  • Misc:
    • Adds WorkspaceTemplate.AnthropicComputer in factory.py for Docker workspace configuration.
    • Minor changes in host/shell.py and host/workspace.py for environment setup.

This description was created by Ellipsis for f107409. It will automatically update as commits are pushed.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ Changes requested. Reviewed everything up to a0740ac in 34 seconds

More details
  • Looked at 735 lines of code in 5 files
  • Skipped 0 files when reviewing.
  • Skipped posting 3 drafted comments based on config settings.
1. python/composio/tools/local/anthropic_computer_use/actions/bash.py:17
  • Draft comment:
    Consider providing a public method to safely access the _process attribute, or make it a public attribute if it needs to be accessed outside the class.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The use of private attributes in the BashSession class is inconsistent. The _process attribute is private, but the class does not provide a way to access it safely. This could lead to issues if the attribute needs to be accessed or modified outside the class.
2. python/composio/tools/local/anthropic_computer_use/actions/computer.py:190
  • Draft comment:
    Avoid using shell=True in subprocess calls unless necessary. Consider using a list of arguments instead for better security and reliability.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The method get_screen_size uses subprocess with shell=True, which can be a security risk if the command string is constructed from external input. Although in this case, the command is static, it's a good practice to avoid shell=True when not necessary.
3. python/composio/tools/local/anthropic_computer_use/actions/computer.py:90
  • Draft comment:
    Consider handling NotImplementedError where _get_mouse_tool and _get_screenshot_tool are called to provide a more user-friendly error message.
  • Reason this comment was not posted:
    Confidence changes required: 40%
    The method _get_mouse_tool and _get_screenshot_tool raise NotImplementedError for unsupported OS. This is a good practice, but it would be better to handle this exception where these methods are called to provide a more user-friendly error message.

Workflow ID: wflow_7IyB3rmSmZ96Wakh


Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

async with asyncio.timeout(self._timeout):
while True:
await asyncio.sleep(self._output_delay)
output = self._process.stdout._buffer.decode()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using _buffer to access the output of the process is not recommended as it is an internal attribute. Consider using await self._process.stdout.read() to read the output instead.

Suggested change
output = self._process.stdout._buffer.decode()
output = await self._process.stdout.read()

return TextEditorResponse(output="", error=str(e))

def _execute(self, request: TextEditorRequest, metadata: t.Dict) -> TextEditorResponse:
file_manager = self.filemanagers.get(request.file_manager_id)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a check to ensure file_manager is not None before proceeding with file operations to avoid potential AttributeError.

Copy link

github-actions bot commented Oct 24, 2024

This comment was generated by github-actions[bot]!

JS SDK Coverage Report

📊 Coverage report for JS SDK can be found at the following URL:
https://pub-92e668239ab84bfd80ee07d61e9d2f40.r2.dev/coverage-11589457190/coverage/index.html

📁 Test report folder can be found at the following URL:
https://pub-92e668239ab84bfd80ee07d61e9d2f40.r2.dev/html-report-11589457190/html-report/report.html

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Incremental review on e0e1968 in 54 seconds

More details
  • Looked at 1981 lines of code in 8 files
  • Skipped 0 files when reviewing.
  • Skipped posting 3 drafted comments based on config settings.
1. python/composio/tools/local/anthropic_computer_use/actions/bash.py:75
  • Draft comment:
    self.shells is not defined in this class or imported, which will cause an AttributeError. Ensure self.shells is properly defined or imported.
  • Reason this comment was not posted:
    Comment did not seem useful.
2. python/composio/tools/local/anthropic_computer_use/actions/computer.py:321
  • Draft comment:
    The shell method does not handle exceptions that might occur during subprocess execution. Consider adding exception handling to manage potential errors during command execution.
  • Reason this comment was not posted:
    Decided after close inspection that this draft comment was likely wrong and/or not actionable:
    The comment is relevant because the shell method does not currently handle exceptions that could arise from subprocess execution. This could lead to unhandled exceptions if a command fails, which is a valid concern. Adding exception handling would improve the robustness of the code.
    The comment does not specify what kind of exceptions might occur or how they should be handled, which could make it less actionable. However, the general suggestion to add exception handling is still valid.
    While the comment lacks specifics, the suggestion to add exception handling is a common best practice when dealing with subprocesses, as it can prevent the program from crashing unexpectedly.
    The comment is valid because it points out a potential issue with unhandled exceptions in the shell method. It should be kept as it suggests a clear improvement to the code's robustness.
3. python/composio/tools/local/anthropic_computer_use/actions/text_editor.py:65
  • Draft comment:
    self.filemanagers is not defined in this class or imported, which will cause an AttributeError. Ensure self.filemanagers is properly defined or imported.
  • Reason this comment was not posted:
    Marked as duplicate.

Workflow ID: wflow_u0dYq8DgdIarO6vo


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ Changes requested. Incremental review on 4f7bf8d in 38 seconds

More details
  • Looked at 25 lines of code in 1 files
  • Skipped 0 files when reviewing.
  • Skipped posting 0 drafted comments based on config settings.

Workflow ID: wflow_9V1ZQj24ojvzZ1dq


Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

@@ -10,6 +10,7 @@
from anthropic.types.tool_use_block import ToolUseBlock
from anthropic.types.tool_param import ToolParam
from anthropic.types.message import Message as ToolsBetaMessage
from anthropic.types.beta.beta_tool_use_block import BetaToolUseBlock
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The import for BetaToolUseBlock should be inside the try-except block to handle potential ModuleNotFoundError.

Suggested change
from anthropic.types.beta.beta_tool_use_block import BetaToolUseBlock
from anthropic.types.beta.beta_tool_use_block import BetaToolUseBlock

Copy link

vercel bot commented Oct 30, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
composio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Oct 30, 2024 8:28am

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Incremental review on 7d5aa09 in 30 seconds

More details
  • Looked at 336 lines of code in 5 files
  • Skipped 0 files when reviewing.
  • Skipped posting 4 drafted comments based on config settings.
1. python/composio/tools/local/anthropic_computer_use/actions/computer.py:128
  • Draft comment:
    Unnecessary use of elif after a return or raise statement. Use if instead for clarity. This pattern is repeated in several places in this file.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The code in computer.py has multiple instances where the elif statement is used unnecessarily after a return or raise statement. This can be simplified by using if instead of elif. This pattern is repeated in several places.
2. python/composio/tools/local/anthropic_computer_use/actions/text_editor.py:66
  • Draft comment:
    Consider logging the exception e for better debugging, in addition to returning it in the response.
  • Reason this comment was not posted:
    Confidence changes required: 33%
    In text_editor.py, the execute method catches all exceptions and returns them as part of the response. This is a good practice for robustness, but it would be better to log these exceptions for debugging purposes.
3. python/composio/tools/env/host/shell.py:166
  • Draft comment:
    Avoid using # type: ignore unless absolutely necessary. Consider addressing the root cause of the type issue.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    In shell.py, the exec method in HostShell and SSHShell classes have a # type: ignore comment. This might be suppressing important type checking errors. It's better to address the root cause of the type issue.
4. python/composio/tools/env/base.py:310
  • Draft comment:
    Remove commented-out breakpoint(). Debugging code should not be present in production code.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    In base.py, the check_for_missing_dependencies method in RemoteWorkspace class has a commented-out breakpoint(). This should be removed as it is not suitable for production code.

Workflow ID: wflow_CANBTH4tBLSLBM0t


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Incremental review on f107409 in 28 seconds

More details
  • Looked at 25 lines of code in 1 files
  • Skipped 0 files when reviewing.
  • Skipped posting 1 drafted comments based on config settings.
1. python/composio/utils/shared.py:225
  • Draft comment:
    Consider using prop_info.get("type") instead of prop_info["type"] to avoid potential KeyErrors if 'type' is missing or None.
  • Reason this comment was not posted:
    Comment was on unchanged code.

Workflow ID: wflow_BinBmKVNOJ5T6xqy


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants