Skip to content

Commit

Permalink
feat: Action redesign: Updating the config for Appsmith AI plugin to …
Browse files Browse the repository at this point in the history 
…use sections and zones format (#36089)

## Description

Action redesign: Updating the config for Appsmith AI plugin to use
sections and zones format

Fixes [#35486](#35486)

## Automation

/ok-to-test tags="@tag.All"

### 🔍 Cypress test results
<!-- This is an auto-generated comment: Cypress test results  -->
> [!TIP]
> 🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉
> Workflow run:
<https://github.com/appsmithorg/appsmith/actions/runs/10718755525>
> Commit: f53f783
> <a
href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=10718755525&attempt=1"
target="_blank">Cypress dashboard</a>.
> Tags: `@tag.All`
> Spec:
> <hr>Thu, 05 Sep 2024 11:41:02 UTC
<!-- end of auto-generated comment: Cypress test results  -->


## Communication
Should the DevRel and Marketing teams inform users about this change?
- [ ] Yes
- [ ] No


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit


- **New Features**
- Enhanced styling capabilities for the input text component with a new
specific class.
- Improved layout organization for image captioning, classification, and
entity extraction features through updated control structures.
- Modular design for text generation and summarization features,
enhancing user experience and clarity.

- **Bug Fixes**
- Resolved layout issues by restructuring child elements into zones,
improving responsiveness and usability.

- **Chores**
- Updated JSON configurations to reflect new control types and
structures for better maintainability.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
  • Loading branch information
@ankitakinger
ankitakinger authored Sep 5, 2024
1 parent 1c14d69 commit a683747
Show file tree
Hide file tree
Showing 9 changed files with 455 additions and 392 deletions.
78 changes: 42 additions & 36 deletions app/server/appsmith-plugins/appsmithAiPlugin/src/main/resources/editor/imageCaptioning.json
View file
Original file line number Diff line number Diff line change
@@ -1,51 +1,57 @@
{
"controlType": "SECTION_V2",
"identifier": "IMAGE_CAPTION",
"controlType": "SECTION",
"conditionals": {
"show": "{{actionConfiguration.formData.usecase.data === 'IMAGE_CAPTION'}}"
},
"children": [
{
"label": "Try out these examples",
"Description": "Try out these examples",
"subtitle": "",
"configProperty": "",
"controlType": "FORM_TEMPLATE",
"isRequired": false,
"options": [
"controlType": "DOUBLE_COLUMN_ZONE",
"identifier": "IMAGE-CAPTION-Z1",
"children": [
{
"label": "Product description",
"value": {
"actionConfiguration.formData.imageCaption.input.data": "https://i.imgur.com/Eiq5s0F.png",
"actionConfiguration.formData.imageCaption.instructions.data": "Write a 200 word product description"
}
"label": "Try out these examples",
"Description": "Try out these examples",
"subtitle": "",
"configProperty": "",
"controlType": "FORM_TEMPLATE",
"isRequired": false,
"options": [
{
"label": "Product description",
"value": {
"actionConfiguration.formData.imageCaption.input.data": "https://i.imgur.com/Eiq5s0F.png",
"actionConfiguration.formData.imageCaption.instructions.data": "Write a 200 word product description"
}
}
]
}
]
},
{
"label": "Input",
"Description": "Provide an image URL or the base64 encoded image",
"subtitle": "Provide an image URL or the base64 encoded image",
"configProperty": "actionConfiguration.formData.imageCaption.input.data",
"controlType": "QUERY_DYNAMIC_INPUT_TEXT",
"placeholderText": "Write some text or use {{ }} to reference a dynamic text value",
"isRequired": true,
"customStyles": {
"width": "700px"
}
},
{
"label": "Additional Instructions",
"Description": "Provide additional instructions to tweak the caption",
"subtitle": "Provide additional instructions to tweak the caption",
"configProperty": "actionConfiguration.formData.imageCaption.instructions.data",
"controlType": "QUERY_DYNAMIC_TEXT",
"placeholderText": "Write some text or use {{ }} to reference a dynamic text value",
"isRequired": false,
"initialValue": "",
"customStyles": {
"width": "700px"
}
"controlType": "SINGLE_COLUMN_ZONE",
"identifier": "IMAGE-CAPTION-Z2",
"children": [
{
"label": "Input",
"Description": "Provide an image URL or the base64 encoded image",
"subtitle": "Provide an image URL or the base64 encoded image",
"configProperty": "actionConfiguration.formData.imageCaption.input.data",
"controlType": "QUERY_DYNAMIC_INPUT_TEXT",
"placeholderText": "Write some text or use {{ }} to reference a dynamic text value",
"isRequired": true
},
{
"label": "Additional Instructions",
"Description": "Provide additional instructions to tweak the caption",
"subtitle": "Provide additional instructions to tweak the caption",
"configProperty": "actionConfiguration.formData.imageCaption.instructions.data",
"controlType": "QUERY_DYNAMIC_TEXT",
"placeholderText": "Write some text or use {{ }} to reference a dynamic text value",
"isRequired": false,
"initialValue": ""
}
]
}
]
}
117 changes: 60 additions & 57 deletions ...rver/appsmith-plugins/appsmithAiPlugin/src/main/resources/editor/imageClassification.json
View file
Original file line number Diff line number Diff line change
@@ -1,72 +1,75 @@
{
"controlType": "SECTION_V2",
"identifier": "IMAGE_CLASSIFY",
"controlType": "SECTION",
"conditionals": {
"show": "{{actionConfiguration.formData.usecase.data === 'IMAGE_CLASSIFY'}}"
},
"children": [
{
"label": "Try out these examples",
"Description": "Try out these examples",
"subtitle": "",
"configProperty": "",
"controlType": "FORM_TEMPLATE",
"isRequired": false,
"options": [
"controlType": "DOUBLE_COLUMN_ZONE",
"identifier": "IMAGE-CLASSIFY-Z1",
"children": [
{
"label": "Product category",
"value": {
"actionConfiguration.formData.imageClassify.input.data": "https://i.imgur.com/Eiq5s0F.png",
"actionConfiguration.formData.imageClassify.instructions.data": "Identify the category of clothing. Apply only one category.",
"actionConfiguration.formData.imageClassify.labels.data": "Jacket, Shirt, Pant, T-Shirt, Shorts, Dress, Skirt"
}
},
{
"label": "IT Asset Tagging",
"value": {
"actionConfiguration.formData.imageClassify.input.data": "https://i.imgur.com/EqfqRQY.png",
"actionConfiguration.formData.imageClassify.instructions.data": "Tag the IT hardware.",
"actionConfiguration.formData.imageClassify.labels.data": "Laptop, Phone, Headphones, Mouse, Keyboard, Monitor"
}
"label": "Try out these examples",
"Description": "Try out these examples",
"subtitle": "",
"configProperty": "",
"controlType": "FORM_TEMPLATE",
"isRequired": false,
"options": [
{
"label": "Product category",
"value": {
"actionConfiguration.formData.imageClassify.input.data": "https://i.imgur.com/Eiq5s0F.png",
"actionConfiguration.formData.imageClassify.instructions.data": "Identify the category of clothing. Apply only one category.",
"actionConfiguration.formData.imageClassify.labels.data": "Jacket, Shirt, Pant, T-Shirt, Shorts, Dress, Skirt"
}
},
{
"label": "IT Asset Tagging",
"value": {
"actionConfiguration.formData.imageClassify.input.data": "https://i.imgur.com/EqfqRQY.png",
"actionConfiguration.formData.imageClassify.instructions.data": "Tag the IT hardware.",
"actionConfiguration.formData.imageClassify.labels.data": "Laptop, Phone, Headphones, Mouse, Keyboard, Monitor"
}
}
]
}
]
},
{
"label": "Input",
"Description": "Provide an image URL or the base64 encoded image for AI to extract entities from",
"subtitle": "Provide an image URL or the base64 encoded image for AI to extract entities from",
"configProperty": "actionConfiguration.formData.imageClassify.input.data",
"placeholderText": "Write some text or use {{ }} to reference a dynamic text value",
"controlType": "QUERY_DYNAMIC_INPUT_TEXT",
"isRequired": true,
"customStyles": {
"width": "700px"
}
},
{
"label": "Labels",
"Description": "Provide labels as comma-separated string input for classification",
"subtitle": "Provide a comma separated list of labels to classify the Input on",
"configProperty": "actionConfiguration.formData.imageClassify.labels.data",
"controlType": "QUERY_DYNAMIC_INPUT_TEXT",
"placeholderText": "Write a list of comma separated text values or use {{ }} to reference a dynamic value",
"isRequired": true,
"customStyles": {
"width": "700px"
}
},
{
"label": "Additional Instructions",
"Description": "Provide additional instructions to tweak the classification",
"subtitle": "Provide additional instructions to tweak the classification",
"configProperty": "actionConfiguration.formData.imageClassify.instructions.data",
"controlType": "QUERY_DYNAMIC_TEXT",
"placeholderText": "Write some text or use {{ }} to reference a dynamic text value",
"isRequired": false,
"initialValue": "",
"customStyles": {
"width": "700px"
}
"controlType": "SINGLE_COLUMN_ZONE",
"identifier": "IMAGE-CLASSIFY-Z2",
"children": [
{
"label": "Input",
"Description": "Provide an image URL or the base64 encoded image for AI to extract entities from",
"subtitle": "Provide an image URL or the base64 encoded image for AI to extract entities from",
"configProperty": "actionConfiguration.formData.imageClassify.input.data",
"placeholderText": "Write some text or use {{ }} to reference a dynamic text value",
"controlType": "QUERY_DYNAMIC_INPUT_TEXT",
"isRequired": true
},
{
"label": "Labels",
"Description": "Provide labels as comma-separated string input for classification",
"subtitle": "Provide a comma separated list of labels to classify the Input on",
"configProperty": "actionConfiguration.formData.imageClassify.labels.data",
"controlType": "QUERY_DYNAMIC_INPUT_TEXT",
"placeholderText": "Write a list of comma separated text values or use {{ }} to reference a dynamic value",
"isRequired": true
},
{
"label": "Additional Instructions",
"Description": "Provide additional instructions to tweak the classification",
"subtitle": "Provide additional instructions to tweak the classification",
"configProperty": "actionConfiguration.formData.imageClassify.instructions.data",
"controlType": "QUERY_DYNAMIC_TEXT",
"placeholderText": "Write some text or use {{ }} to reference a dynamic text value",
"isRequired": false,
"initialValue": ""
}
]
}
]
}
114 changes: 60 additions & 54 deletions ...er/appsmith-plugins/appsmithAiPlugin/src/main/resources/editor/imageEntityExtraction.json
View file
Original file line number Diff line number Diff line change
@@ -1,69 +1,75 @@
{
"controlType": "SECTION_V2",
"identifier": "IMAGE_ENTITY_EXTRACT",
"controlType": "SECTION",
"conditionals": {
"show": "{{actionConfiguration.formData.usecase.data === 'IMAGE_ENTITY_EXTRACT'}}"
},
"children": [
{
"label": "Try out these examples",
"Description": "Try out these examples",
"subtitle": "",
"configProperty": "",
"controlType": "FORM_TEMPLATE",
"isRequired": false,
"options": [
"controlType": "DOUBLE_COLUMN_ZONE",
"identifier": "IMAGE-ENTITY-EXTRACT-Z1",
"children": [
{
"label": "KYC document",
"value": {
"actionConfiguration.formData.imageEntity.input.data": "https://i.imgur.com/5h9SfGf.jpgg",
"actionConfiguration.formData.imageEntity.labels.data": "name, date of birth, gender, licence number, height",
"actionConfiguration.formData.imageEntity.instructions.data": ""
}
},
{
"label": "Expense report",
"value": {
"actionConfiguration.formData.imageEntity.input.data": "https://i.imgur.com/z2PlaKB.jpg",
"actionConfiguration.formData.imageEntity.labels.data": "food items, tax, total cost, date of purchase",
"actionConfiguration.formData.imageEntity.instructions.data": "The date should be in dd/mm/yyyy format"
}
"label": "Try out these examples",
"Description": "Try out these examples",
"subtitle": "",
"configProperty": "",
"controlType": "FORM_TEMPLATE",
"isRequired": false,
"options": [
{
"label": "KYC document",
"value": {
"actionConfiguration.formData.imageEntity.input.data": "https://i.imgur.com/5h9SfGf.jpgg",
"actionConfiguration.formData.imageEntity.labels.data": "name, date of birth, gender, licence number, height",
"actionConfiguration.formData.imageEntity.instructions.data": ""
}
},
{
"label": "Expense report",
"value": {
"actionConfiguration.formData.imageEntity.input.data": "https://i.imgur.com/z2PlaKB.jpg",
"actionConfiguration.formData.imageEntity.labels.data": "food items, tax, total cost, date of purchase",
"actionConfiguration.formData.imageEntity.instructions.data": "The date should be in dd/mm/yyyy format"
}
}
]
}
]
},
{
"label": "Input",
"Description": "Provide an image URL or the base64 encoded image",
"subtitle": "Provide an image URL or the base64 encoded image",
"configProperty": "actionConfiguration.formData.imageEntity.input.data",
"placeholderText": "Write some text or use {{ }} to reference a dynamic text value",
"controlType": "QUERY_DYNAMIC_INPUT_TEXT",
"isRequired": true,
"customStyles": {
"width": "700px"
}
},
{
"label": "Entities",
"Description": "Provide a comma separated list of entities to extract from the Input",
"subtitle": "Provide a comma separated list of entities to extract from the Input",
"configProperty": "actionConfiguration.formData.imageEntity.labels.data",
"controlType": "QUERY_DYNAMIC_INPUT_TEXT",
"placeholderText": "Write a list of comma separated text values or use {{ }} to reference a dynamic value",
"isRequired": true,
"customStyles": {
"width": "700px"
}
},
{
"label": "Additional Instructions",
"Description": "Provide additional instructions for the AI to tweak the entity extraction",
"subtitle": "Provide additional instructions for the AI to tweak the entity extraction",
"configProperty": "actionConfiguration.formData.imageEntity.instructions.data",
"controlType": "QUERY_DYNAMIC_TEXT",
"placeholderText": "Write some text or use {{ }} to reference a dynamic text value",
"initialValue": "",
"isRequired": false
"controlType": "SINGLE_COLUMN_ZONE",
"identifier": "IMAGE-ENTITY-EXTRACT-Z2",
"children": [
{
"label": "Input",
"Description": "Provide an image URL or the base64 encoded image",
"subtitle": "Provide an image URL or the base64 encoded image",
"configProperty": "actionConfiguration.formData.imageEntity.input.data",
"placeholderText": "Write some text or use {{ }} to reference a dynamic text value",
"controlType": "QUERY_DYNAMIC_INPUT_TEXT",
"isRequired": true
},
{
"label": "Entities",
"Description": "Provide a comma separated list of entities to extract from the Input",
"subtitle": "Provide a comma separated list of entities to extract from the Input",
"configProperty": "actionConfiguration.formData.imageEntity.labels.data",
"controlType": "QUERY_DYNAMIC_INPUT_TEXT",
"placeholderText": "Write a list of comma separated text values or use {{ }} to reference a dynamic value",
"isRequired": true
},
{
"label": "Additional Instructions",
"Description": "Provide additional instructions for the AI to tweak the entity extraction",
"subtitle": "Provide additional instructions for the AI to tweak the entity extraction",
"configProperty": "actionConfiguration.formData.imageEntity.instructions.data",
"controlType": "QUERY_DYNAMIC_TEXT",
"placeholderText": "Write some text or use {{ }} to reference a dynamic text value",
"initialValue": "",
"isRequired": false
}
]
}
]
}
Loading

0 comments on commit a683747

Please sign in to comment.