Add headless browser script #85
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
frankfeng98
requested changes
Oct 29, 2024
|
|
||
| (async () => { | ||
| // Set the AgentQL API key via the `configure` method. | ||
| configure({ apiKey: process.env.AGENTQL_API_KEY }); |
There was a problem hiding this comment.
We have decided to remove this line from example. It will be included in the installation guide.
|
|
||
| let response; | ||
| // Use queryElements() method to locate the search box from the page. | ||
| response = await page.queryElements(SEARCH_QUERY); |
There was a problem hiding this comment.
We could just use const response here instead of declaring it on separate line.
|
|
||
| // Use Playwright's API to fill the search box and press Enter. | ||
| await response.search_products_box.type('Charmander'); | ||
| page.keyboard.press('Enter'); |
There was a problem hiding this comment.
I think press is an async function that requires await. Could you double check?
…t-in-headless-browsers
frankfeng98
requested changes
Oct 30, 2024
| await normalPage.goto(URL); | ||
|
|
||
| // Wrap a new page instance to use with AgentQL API | ||
| const wrappedPage = wrap(normalPage); |
There was a problem hiding this comment.
Another nit: could we inline page creation? And only call page.goto() after wrap()
There was a problem hiding this comment.
Hey @frankfeng98, when trying to make this a one liner, I get the following error:
% node --env-file=.env test.js
node:internal/process/promises:394
triggerUncaughtException(err, true /* fromPromise */);
^
page.evaluate: Execution context was destroyed, most likely because of a navigation
at PageActivityMonitor.<anonymous> (/Users/zach/CS467/node_modules/agentql/dist/ext/playwright/page-monitor.js:42:29)
at fulfilled (/Users/zach/CS467/node_modules/agentql/dist/ext/playwright/page-monitor.js:5:58) {
name: 'Error'
}
Node.js v23.1.0Code snippet:
// Launch a headless browser using Playwright.
const browser = await chromium.launch({ headless: false });
// Create a new page in the browser and wrap it to get access to the AgentQL's querying API
const page = wrap(await browser.newPage());
await page.goto(URL);This is why I extracted it to two lines. If this issue only pertains to my node setup then I can change the code back into a one liner but wasn't sure if this affected anyone else. Could you maybe try running this code?
There was a problem hiding this comment.
Hi Zach! Thank you for flagging this. It help me find a bug in JS SDK. I just updated and published a new version of JS SDK. If you run npm install, it should contains the fix.
Note that in the latest version, wrap becomes async, so you need to await it.
There was a problem hiding this comment.
Thank you for the prompt fix!
…//github.com/tinyfish-io/agentql into zach/tf-3909-run-script-in-headless-browsers
frankfeng98
approved these changes
Oct 31, 2024
frankfeng98
added a commit
that referenced
this pull request
Nov 6, 2024
* Add js sdk folder to fish-tank * Update formatting * Add placeholder files * Update naming of the file * Add one more example * kate/tf 3913 application collect ecommerce pricing data (#81) * collect pricing data example * update lint * Update javascript-sdk/examples/collect-pricing-data/README.md Co-authored-by: R L Nabors <rachelnabors@users.noreply.github.com> --------- Co-authored-by: R L Nabors <rachelnabors@users.noreply.github.com> * Add js example for list query usage (#83) * Add js example for list query usage * lint * Address security concerns * lint * Add review comments * Lint * Lint * add save / load authenticated session js sdk example. ran script and saw successful reload (#84) * Add close-popup files (#76) * Add close-popup files * Format * Update example * format * Update javascript-sdk/examples/close-popup/README.md * Update --------- Co-authored-by: R L Nabors <rachelnabors@users.noreply.github.com> * Close cookie popup js sdk (#77) * js example cookie * readme * link * . * agentql env key * Update javascript-sdk/examples/close-cookie-dialog/README.md * remove api key setup --------- Co-authored-by: R L Nabors <rachelnabors@users.noreply.github.com> * Add log-into-sites examples (#79) * Add examples * Format * Update javascript-sdk/examples/log-into-sites/README.md * Update --------- Co-authored-by: R L Nabors <rachelnabors@users.noreply.github.com> * Compare Product Prices Example (#78) * Update * Update * Update * Update * Update * Update * Update * Update --------- Co-authored-by: R L Nabors <rachelnabors@users.noreply.github.com> * Add headless browser script (#85) * Add js headless browser script * Add Getting Started YT script (#90) * Add wait for entire page to load example script * Js stealth mode example (#80) * Add example script * Add readme * Fix await * Fix format * Fix lint error * Fix nits * Add xpath application example (#92) * Add xpath application example * Add extra comment * linting * `python` -> `Javascript` * Added javascript sentiment analysis script (#94) * Add wait for entire page to load example script (#95) * Add wait for entire page to load example script * Add `await` * `.py` -> `.js` * submit form js example (#89) * submit form js example * changes * ran lint * changes * Add get_by_prompt example * Fix formatting * add interact with existing browser example * Update interact with existing browser example * address comments & fix examples --------- Co-authored-by: Kate Zhang <54541538+KateZhang98@users.noreply.github.com> Co-authored-by: R L Nabors <rachelnabors@users.noreply.github.com> Co-authored-by: Urvish Thakker <141964033+thakkerurvish@users.noreply.github.com> Co-authored-by: Mark Peng <mark@tinyfish.io> Co-authored-by: Suveen Ellawela <suveen@tinyfish.io> Co-authored-by: Aaron Tan <157438789+aarontantf@users.noreply.github.com> Co-authored-by: Jin Yang <157438770+jinyangTF@users.noreply.github.com> Co-authored-by: Zachary Lyon <zachary@lyonfamily.org> Co-authored-by: Sergey Ryabov <944361+colriot@users.noreply.github.com> Co-authored-by: Julius Gambe <gambe@tinyfish.io> Co-authored-by: Josh Kung <josh@tinyfish.io>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.