-
Notifications
You must be signed in to change notification settings - Fork 1.2k
add playwright arguments to agent #1066
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
🦋 Changeset detectedLatest commit: 8473aa4 The changes in this PR will be included in the next version bump. Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Greptile Summary
This PR restores the missing playwrightArguments field to the agent execute response, addressing issue #1060. The implementation adds observe calls to capture Playwright action details (selector, method, arguments, description) before executing actions, then passes these details through the tool result chain to be included in the final AgentAction.
Key changes:
- Modified
act.tsto callobserve()beforeact()to capture Playwright arguments - Updated
stagehandAgentHandler.tsto extract and includeplaywrightArgumentsfrom tool results - Added new TypeScript interfaces for
PlaywrightArgumentsandActToolResult - Handles both regular and iframe action scenarios
- Added proper changeset documentation
Issues identified:
- Inconsistent parameter usage between execution model paths in act tool
- Iframe observe options don't use
buildActObservePrompt()for consistency
Confidence score: 3/5
- This PR addresses a valid regression but has some logical inconsistencies that should be resolved
- The implementation successfully restores the missing playwrightArguments functionality, but there are two logical issues: inconsistent parameter usage between execution paths and inconsistent observe prompt building in iframe handling. These issues could cause runtime problems or unexpected behavior.
- Pay close attention to lib/agent/tools/act.ts for the parameter consistency issues
3 files reviewed, 2 comments
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Greptile Summary
This PR restores the playwrightArguments field to the AgentResult that was previously removed, addressing issue #1060. The changes implement a new approach that uses observe() first to get the playwright arguments, then executes the action with act().
Key changes:
- Modified
act.tstool to callobserve()beforeact()to capture playwright arguments - Updated
stagehandAgentHandler.tsto extract and include playwright arguments in the agent response - Added proper TypeScript types for
PlaywrightArgumentsandActToolResult - Handles both regular and iframe contexts appropriately
Confidence score: 4/5
- This PR is safe to merge with minor style improvements recommended
- Score reflects well-structured implementation that addresses the regression properly with comprehensive type safety and error handling, though there's a minor consistency issue with iframe prompt building
- No files require special attention, though the iframe observe logic could be made more consistent
1 file reviewed, 1 comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Greptile Summary
This PR restores the missing playwrightArguments field in agent execution responses, addressing a regression where playwright execution details were no longer being returned to users. The fix refactors the act tool to use the observe/act pattern consistently, extracting playwright action details from observe results and including them in all response paths (normal execution, iframe handling, and error cases).
Confidence score: 4/5
- This PR is safe to merge with low risk
- Score reflects a well-structured fix that restores expected functionality. The code properly handles all execution paths including iframe scenarios and error cases. The refactor to consistently use observe/act pattern improves code organization.
- No files require special attention
1 file reviewed, no comments
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## @browserbasehq/stagehand@2.5.1 ### Patch Changes - [#1082](#1082) [`8c0fd01`](8c0fd01) Thanks [@tkattkat](https://github.com/tkattkat)! - Pass stagehand object to agent instead of stagehand page - [#1104](#1104) [`a1ad06c`](a1ad06c) Thanks [@miguelg719](https://github.com/miguelg719)! - Fix logging for stagehand agent - [#1066](#1066) [`9daa584`](9daa584) Thanks [@tkattkat](https://github.com/tkattkat)! - Add playwright arguments to agent execute response - [#1077](#1077) [`7f38b3a`](7f38b3a) Thanks [@tkattkat](https://github.com/tkattkat)! - adds support for stagehand agent in the api - [#1032](#1032) [`bf2d0e7`](bf2d0e7) Thanks [@miguelg719](https://github.com/miguelg719)! - Fix for zod peer dependency support - [#1014](#1014) [`6966201`](6966201) Thanks [@tkattkat](https://github.com/tkattkat)! - Replace operator handler with base of new agent - [#1089](#1089) [`536f366`](536f366) Thanks [@miguelg719](https://github.com/miguelg719)! - Fixed info logs on api session create - [#1103](#1103) [`889cb6c`](889cb6c) Thanks [@tkattkat](https://github.com/tkattkat)! - patch custom tool support in anthropic cua client - [#1056](#1056) [`6a002b2`](6a002b2) Thanks [@chrisreadsf](https://github.com/chrisreadsf)! - remove need for duplicate project id if already passed to Stagehand - [#1090](#1090) [`8ff5c5a`](8ff5c5a) Thanks [@miguelg719](https://github.com/miguelg719)! - Improve failed act error logs - [#1014](#1014) [`6966201`](6966201) Thanks [@tkattkat](https://github.com/tkattkat)! - replace operator agent with scaffold for new stagehand agent - [#1107](#1107) [`3ccf335`](3ccf335) Thanks [@seanmcguire12](https://github.com/seanmcguire12)! - fix: url extraction not working inside an array - [#1102](#1102) [`a99aa48`](a99aa48) Thanks [@miguelg719](https://github.com/miguelg719)! - Add current page and date context to agent - [#1110](#1110) [`dda52f1`](dda52f1) Thanks [@miguelg719](https://github.com/miguelg719)! - Add support for new Gemini Computer Use models ## @browserbasehq/stagehand-evals@1.1.0 ### Minor Changes - [#1057](#1057) [`b7be89e`](b7be89e) Thanks [@filip-michalsky](https://github.com/filip-michalsky)! - added web voyager ground truth (optional), added web bench, and subset of OSWorld evals which run on a browser ### Patch Changes - [#1072](#1072) [`dc2d420`](dc2d420) Thanks [@filip-michalsky](https://github.com/filip-michalsky)! - improve evals screenshot service - add img hashing diff to add screenshots and change to screenshot intercepts from the agent - Updated dependencies \[[`8c0fd01`](8c0fd01), [`a1ad06c`](a1ad06c), [`9daa584`](9daa584), [`7f38b3a`](7f38b3a), [`bf2d0e7`](bf2d0e7), [`6966201`](6966201), [`536f366`](536f366), [`889cb6c`](889cb6c), [`6a002b2`](6a002b2), [`8ff5c5a`](8ff5c5a), [`6966201`](6966201), [`3ccf335`](3ccf335), [`a99aa48`](a99aa48), [`dda52f1`](dda52f1)]: - @browserbasehq/stagehand@2.5.1 ## @browserbasehq/stagehand-examples@1.0.10 ### Patch Changes - Updated dependencies \[[`8c0fd01`](8c0fd01), [`a1ad06c`](a1ad06c), [`9daa584`](9daa584), [`7f38b3a`](7f38b3a), [`bf2d0e7`](bf2d0e7), [`6966201`](6966201), [`536f366`](536f366), [`889cb6c`](889cb6c), [`6a002b2`](6a002b2), [`8ff5c5a`](8ff5c5a), [`6966201`](6966201), [`3ccf335`](3ccf335), [`a99aa48`](a99aa48), [`dda52f1`](dda52f1)]: - @browserbasehq/stagehand@2.5.1 Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
why
solves #1060
patch regression of playwright arguments being removed from agent execute response
what changed
agent.execute now returns playwright arguments in its response
test plan
tested locally