Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(tools): additional tools in SQL toolkit #212

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

tonxxd
Copy link

@tonxxd tonxxd commented Dec 3, 2024

Which issue(s) does this pull-request address?

#182

Description

This feature proposal introduces a toolkit for SQL and data analysis, by enhancing current SQL tool and introducing few other concepts

It adds excel and csv support for SQLTool in order to perform advanced querying on these type of files.
It also adds the ability to provide few shot examples for the connect db, useful for bigger complex dbs.

  • TODO:
  • tests
  • improve search values action
  • add embedding based schema linker
  • correction

Checklist

  • I have read the contributor guide
  • Linting passes: yarn lint or yarn lint:fix
  • Formatting is applied: yarn format or yarn format:fix
  • Unit tests pass: yarn test:unit
  • E2E tests pass: yarn test:e2e
  • Tests are included
  • Documentation is changed or added
  • Commit messages and PR title follow conventional commits

Signed-off-by: Enrico Toniato <2827496+tonxxd@users.noreply.github.com>
@tonxxd tonxxd requested a review from a team as a code owner December 3, 2024 07:34
Copy link
Contributor

@Tomas2D Tomas2D left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding the file support (public providers) that internally gets converted to SQL adds extra complexity (initialize method, mutation properties, checks) unrelated to the core SQL functionality.

I would add a static (async) factory method to convert and initiate the SQLTool with the SQLite connection. Something like this.

static async fromFile(provider: PublicProvider, ...) {
   // conversion
   return new SQLTool(...)
}

Comment on lines +26 to +50
// const sqlTool = new SQLTool({
// provider: "sqlite",
// examples: [
// {
// question: "Get wild albums",
// query: "SELECT * FROM Album where Title = 'Restless and Wild' LIMIT 1",
// },
// ],
// connection: {
// dialect: "sqlite",
// logging: false,
// storage: await fetch(
// "https://github.com/lerocha/chinook-database/releases/download/v1.4.5/chinook_sqlite.sqlite",
// ).then(async (response) => {
// if (!response.ok) {
// throw new Error("Failed to download Chinook database!");
// }

// const dbPath = path.join(os.tmpdir(), "bee_chinook.sqlite");
// const data = Buffer.from(await response.arrayBuffer());
// await fs.promises.writeFile(dbPath, data);
// return dbPath;
// }),
// },
// });
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either remove or create another example.

...old,
defaults: {
instructions:
"You are the Bee 🐝 Data Agent! If the user asks about data questions, use the FlowPilot tool, passing the user natural language question as 'question' to the tool. Do not render ascii tables, the user will already see it externally.",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would rather not use emojis.

];
}

function deduplicateAndSortResults(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider leveraging functionality from remeda library.

import { Sequelize } from "sequelize";
import { beforeEach, expect, vi } from "vitest";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this line can be removed

} as const;

export class SQLTool extends Tool<JSONToolOutput<any>, ToolOptions, ToolRunOptions> {
public options: ToolOptions;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can be removed as it is defined in the parent class.

});
}

public constructor(options: ToolOptions) {
super(options);
if (!options.connection.dialect) {
this.options = options;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can be removed as it is defined in the parent class.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants