Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Getting Started YT script #90

Merged
merged 1 commit into from
Nov 1, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Example script: interacting with YouTube website and extracting video information using AgentQL

This is an example of interacting with YouTube website and extracting video information using AgentQL.

## Run the script

- [Install AgentQL SDK](https://docs.agentql.com/javascript-sdk/installation)
- Save this JavaScript file locally as **collect_youtube_comments.js**
- Run the following command from the project's folder:

```bash
node collect_youtube_comments.js
```

## Play with the query

Install the [AgentQL Debugger Chrome extension](https://docs.agentql.com/installation/chrome-extension-installation) to play with the AgentQL query. [Learn more about the AgentQL query language](https://docs.agentql.com/agentql-query/query-intro)
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
const { chromium } = require('playwright');
const { configure, wrap } = require('agentql');

const URL = 'https://www.youtube.com/';

async function main() {
// Configure the AgentQL API key
configure({
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rachelnabors and I have decided to remove them from the example. We will include how to set up API key in installation guide.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be no so good, because some people don't read the installation guide (except the part with npm install), and go straight to the code examples to play with.
What kind of content do you plan in installation guide? To set OS global env var or smth else?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The installation guide currently contains guide on set session env var via export, but this is a good point. We should instruct users to set OS global env var

apiKey: process.env.AGENTQL_API_KEY, // This is the default and can be omitted.
});

const browser = await chromium.launch({ headless: false });
const page = wrap(await browser.newPage());
await page.goto(URL);

const SEARCH_QUERY = `
{
search_input
search_btn
}
`;

const VIDEO_QUERY = `
{
videos[] {
video_link
video_title
channel_name
}
}
`;

const VIDEO_CONTROL_QUERY = `
{
play_or_pause_btn
expand_description_btn
}
`;

const DESCRIPTION_QUERY = `
{
description_text
}
`;

const COMMENT_QUERY = `
{
comments[] {
channel_name
comment_text
}
}
`;

try {
// search query
const searchResponse = await page.queryElements(SEARCH_QUERY);
await searchResponse.search_input.type('machine learning', { delay: 75 });
await searchResponse.search_btn.click();

// video query
const videoResponse = await page.queryElements(VIDEO_QUERY);
console.log(
`Clicking Youtube Video: ${await videoResponse.videos[0].video_title.textContent()}`,
);
await videoResponse.videos[0].video_link.click(); // click the first youtube video

// video control query
const controlResponse = await page.queryElements(VIDEO_CONTROL_QUERY);
await controlResponse.expand_description_btn.click();

// description query
const descriptionData = await page.queryData(DESCRIPTION_QUERY);
console.log(`Captured the following description:\n${descriptionData.description_text}`);

// Scroll down the page to load more comments
for (let i = 0; i < 3; i++) {
await page.keyboard.press('PageDown');
await page.waitForLoadState();
}

// comment query
const commentResponse = await page.queryData(COMMENT_QUERY);
console.log(`Captured ${commentResponse.comments?.length || 0} comments!`);
} catch (error) {
console.error(`Found Error: ${error}`);
throw error;
}

// Used only for demo purposes. It allows you to see the effect of the script.
await page.waitForTimeout(10000);
}

main();
2 changes: 1 addition & 1 deletion javascript-sdk/examples/first-steps/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ This is an example of collecting pricing data from e-commerce website using Agen
## Run the script

- [Install AgentQL SDK](https://docs.agentql.com/javascript-sdk/installation)
- Save this python file locally as **first_steps.js**
- Save this JavaScript file locally as **first_steps.js**
- Run the following command from the project's folder:

```bash
Expand Down
1 change: 1 addition & 0 deletions javascript-sdk/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading