Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add POST route for testing tool execution via tool_id #2139

Merged
merged 7 commits into from
Dec 3, 2024

Conversation

cpacker
Copy link
Collaborator

@cpacker cpacker commented Dec 2, 2024

@router.post("/run", response_model=FunctionReturn, operation_id="run_tool_from_source")
def run_tool_from_source(
    server: SyncServer = Depends(get_letta_server),
    request: ToolRunFromSource = Body(...),
    user_id: Optional[str] = Header(None, alias="user_id"),  # Extract user_id from header, default to None if not present
):
    """
    Attempt to build a tool from source, then run it on the provided arguments
    """
    actor = server.get_user_or_default(user_id=user_id)

    return server.run_tool_from_source(
        tool_source=request.source_code,
        tool_source_type=request.source_type,
        tool_args=request.args,
        tool_name=request.name,
        user_id=actor.id,
    )

Example tests:

EXAMPLE_TOOL_SOURCE_WITH_DISTRACTOR = '''
def util_do_nothing():
    """
    A util function that does nothing.

    Returns:
        str: Dummy output.
    """
    print("I'm a distractor")

def ingest(message: str):
    """
    Ingest a message into the system.

    Args:
        message (str): The message to ingest into the system.

    Returns:
        str: The result of ingesting the message.
    """
    util_do_nothing()
    return f"Ingested message {message}"

'''


def test_tool_run(server, user_id, agent_id):
    """Test that the server can run tools"""

    result = server.run_tool_from_source(
        user_id=user_id,
        tool_source=EXAMPLE_TOOL_SOURCE,
        tool_source_type="python",
        tool_args=json.dumps({"message": "Hello, world!"}),
        # tool_name="ingest",
    )
    print(result)
    assert result.status == "success"
    assert result.function_return == "Ingested message Hello, world!", result.function_return

    result = server.run_tool_from_source(
        user_id=user_id,
        tool_source=EXAMPLE_TOOL_SOURCE,
        tool_source_type="python",
        tool_args=json.dumps({"message": "Well well well"}),
        # tool_name="ingest",
    )
    print(result)
    assert result.status == "success"
    assert result.function_return == "Ingested message Well well well", result.function_return

    result = server.run_tool_from_source(
        user_id=user_id,
        tool_source=EXAMPLE_TOOL_SOURCE,
        tool_source_type="python",
        tool_args=json.dumps({"bad_arg": "oh no"}),
        # tool_name="ingest",
    )
    print(result)
    assert result.status == "error"
    assert "Error" in result.function_return, result.function_return
    assert "missing 1 required positional argument" in result.function_return, result.function_return

...

letta/server/server.py Outdated Show resolved Hide resolved
agent_state = None

try:
sandbox_run_result = ToolExecutionSandbox(tool.name, tool_args_dict, user_id).run(agent_state=agent_state)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a bit of a bit, but can't this be the same code as run_tool_from_source once the Tool object is ready?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh yeah this code I kind of stopped editing in the PR since I wasn't sure if we wanted to delete it?

e.g. if we're not going to use run_tool and only use run_tool_from_source, shouldn't we just add the later and kill the former (and add it if we need it later?)

@sarahwooders sarahwooders self-requested a review December 3, 2024 01:18
@cpacker cpacker merged commit 9417b11 into main Dec 3, 2024
31 of 32 checks passed
@cpacker cpacker deleted the allow-tool-exec-via-api branch December 3, 2024 02:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants