Budi 8677 ai column type v1

[shogunpurple]

Description

This PR contains the new budibase AI column - a new column type that allows you to specify a function to be carried out by an LLM, pass in some of your row data, and have the LLM write the prompt into the Budibase DB. The AI functions Budibase provides are as follows:

• Summarise text
• Clean data
• Translate
• Categorise Text
• Sentiment Analysis
• Free Prompt
• Search Web

AI columns are generated when you save a new row, edit it, or use the right click context menu to generate the columns for a previously saved row.

Pro PR: https://github.com/Budibase/budibase-pro/pull/322

Addresses

• https://linear.app/budibase/issue/BUDI-8677/ai-column-type-v1

Screenshots

Launchcontrol

The budibase AI column allows you to perform LLM operations over your data - summarise text, run translations, perform sentiment analysis and much more simply by creating an AI column and saving/editing your data. You can also generate data from rows saved previously by right clicking and using the "Generate AI Columns" functionality on your row.

[linear]

BUDI-8677 AI column type V1

[mike12345567]

Looks good - would it be possible to get test cases for search.spec.ts to make sure the column is searchable, as well as a test case to row.spec.ts which makes sure that the column can be added/updated on a row as expected? We may have to mock the LLM in these cases, but it would be good to confirm this flows correctly in these test cases.

We have test cases in these which use something like isSqs && describe(.... to make sure the test case only runs for internal DB.

[aptkingston]

Doing a thorough review now, but the first snag I hit after generating a license to test this is that attempting to save a row (in a random table with no AI columns) threw an error The OPENAI_API_KEY environment variable is missing or empty; either provide it, or instantiate the OpenAI client with an apiKey option, like new OpenAI({ apiKey: 'My API Key' }). So I assume other self hosters on non-free licenses would also hit this. We should probably fail gracefully if you aren't using any AI features and so don't have any AI configs set up. Also possible I just did something wrong.

Edit: what's the correct way to get AI up and running? The default Budibase AI config is present and enabled but can't be configured, and I've tried adding the OPENAI_API_KEY env var but that didn't work either. Would just like to test the feature properly 👌

[shogunpurple]

Doing a thorough review now, but the first snag I hit after generating a license to test this is that attempting to save a row (in a random table with no AI columns) threw an error The OPENAI_API_KEY environment variable is missing or empty; either provide it, or instantiate the OpenAI client with an apiKey option, like new OpenAI({ apiKey: 'My API Key' }). So I assume other self hosters on non-free licenses would also hit this. We should probably fail gracefully if you aren't using any AI features and so don't have any AI configs set up. Also possible I just did something wrong.

Edit: what's the correct way to get AI up and running? The default Budibase AI config is present and enabled but can't be configured, and I've tried adding the OPENAI_API_KEY env var but that didn't work either. Would just like to test the feature properly 👌

@aptkingston This happens because the license generation on self host/dev env gives all the features - including Budibase AI (the cloud config) and custom configs. So it assumes you have the cloud config switched on, and tries to go to OpenAI. If you add OPENAI_API_KEY to your .env in server and restart, you should be able to use the AI features as normal.

[aptkingston]

Awesome work @shogunpurple 👌 great to see our first proper AI features.

One small improvement I think we should make is highlighting that AI cells are readonly in grids. We have a function in stores/columns.js called isReadonly that we use today to make things like formula cells readonly. So I think we should add the AI type in there too, and then it'll reflect its readonly nature properly by having a grey border.

When filtering on AI column in the design section it seems to bug out a little (the value field is disabled for some reason):

Maybe it's just not being considered as a searchable type properly? Haven't dug into the code to see why though.

The "generate AI columns" functionality is missing in client apps (maybe by design?) since the licensing value isn't being passed in. Not sure if we want to expose that or not though.

[shogunpurple]

@aptkingston thanks for the review - they have all been addressed, some nice catches there 🙏

[aptkingston]

LGTM!