Skip to content

Commit

Permalink
[custom tool] Provide guidance for configuring tool icon (#616)
Browse files Browse the repository at this point in the history
# Description

Provide guidance for configuring tool icon.

# All Promptflow Contribution checklist:
- [X] **The pull request does not introduce [breaking changes].**
- [ ] **CHANGELOG is updated for new features, bug fixes or other
significant changes.**
- [X] **I have read the [contribution guidelines](../CONTRIBUTING.md).**
- [ ] **Create an issue and link to the pull request to get dedicated
review from promptflow team. Learn more: [suggested
workflow](../CONTRIBUTING.md#suggested-workflow).**

## General Guidelines and Best Practices
- [X] Title of the pull request is clear and informative.
- [X] There are a small number of commits, each of which have an
informative message. This means that previously merged commits do not
appear in the history of the PR. For more information on cleaning up the
commits in your PR, [see this
page](https://github.com/Azure/azure-powershell/blob/master/documentation/development-docs/cleaning-up-commits.md).

### Testing Guidelines
- [ ] Pull request includes test coverage for the included changes.
  • Loading branch information
jiazengcindy authored Sep 28, 2023
1 parent 65f8537 commit 037f627
Show file tree
Hide file tree
Showing 11 changed files with 141 additions and 8 deletions.
3 changes: 2 additions & 1 deletion .cspell.json
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,8 @@
"tcsetattr",
"pysqlite",
"AADSTS700082",
"levelno"
"levelno",
"LANCZOS"
],
"allowCompoundWords": true
}
2 changes: 1 addition & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Below is a table of important doc pages.
|----------------|----------------|
|Quick start|[Get started with prompt flow](./how-to-guides/quick-start.md)|
|Concepts|[Flows](./concepts/concept-flows.md)<br> [Tools](./concepts/concept-tools.md)<br> [Connections](./concepts/concept-connections.md)<br> [Variants](./concepts/concept-variants.md)<br> |
|How-to guides|[How to initialize and test a flow](./how-to-guides/init-and-test-a-flow.md) <br>[How to run and evaluate a flow](./how-to-guides/run-and-evaluate-a-flow.md)<br> [How to tune prompts using variants](./how-to-guides/tune-prompts-with-variants.md)<br>[How to deploy a flow](./how-to-guides/deploy-a-flow/index.md)<br>[How to create and use your own tool package](./how-to-guides/how-to-create-and-use-your-own-tool-package.md)|
|How-to guides|[How to initialize and test a flow](./how-to-guides/init-and-test-a-flow.md) <br>[How to run and evaluate a flow](./how-to-guides/run-and-evaluate-a-flow.md)<br> [How to tune prompts using variants](./how-to-guides/tune-prompts-with-variants.md)<br>[How to deploy a flow](./how-to-guides/deploy-a-flow/index.md)<br>[How to create and use your own tool package](./how-to-guides/develop-a-tool/create-and-use-tool-package.md)|
|Tools reference|[LLM tool](./tools-reference/llm-tool.md)<br> [Prompt tool](./tools-reference/prompt-tool.md)<br> [Python tool](./tools-reference/python-tool.md)<br> [SERP API tool](./tools-reference/serp-api-tool.md)<br> [Embedding tool](./tools-reference/embedding_tool.md)||


Expand Down
2 changes: 1 addition & 1 deletion docs/concepts/concept-tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Our partners also contributes other useful tools for advanced scenarios, here ar
## Custom tools

You can create your own tools that can be shared with your team or anyone in the world.
Learn more on [Custom tool package creation and usage](../how-to-guides/how-to-create-and-use-your-own-tool-package.md)
Learn more on [Create and Use Tool Package](../how-to-guides/develop-a-tool/create-and-use-tool-package.md)

## Next steps

Expand Down
58 changes: 58 additions & 0 deletions docs/how-to-guides/develop-a-tool/add-a-tool-icon.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Add a Tool Icon
A tool icon serves as a graphical representation of your tool in the user interface (UI). Follow this guidance to add a custom tool icon when developing your own tool package.

Adding a custom tool icon is optional. If you do not provide one, the system uses a default icon.

## Prerequisites

- Create a tool package as described in [Create and Use Tool Package](create-and-use-tool-package.md).
- Prepare a custom tool icon image. The image should meet following requirements:
- The image should be in `PNG` or `JPG` format.
- 16x16 pixel image to avoid distortion from resizing.
- Avoid complex images with much detail or contrast as they may not resize well.
You could see for example.
- Install dependencies of generating icon data URI:

```
pip install pillow
```
## Generate a tool icon data URI
Run below command under the root folder to generate a data URI for your custom tool icon. Make sure the output file has an `.html` extension, this makes it easier to check the image's data URI:
```
python <path-to-scripts>\tool\convert_image_to_data_url.py --image-path <image_input_path> -o <html_output_path>
```
For example:
```
python D:\proj\github\promptflow\scripts\tool\convert_image_to_data_url.py --image-path D:\proj\github\promptflow\examples\tools\tool-package-quickstart\my_tool_package\icons\custom-tool-icon.png -o output.html
```
The content of `output.html` looks like the following, and you can open it in a web browser to see the result.
```html
<html>
<body>
<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAACq0lEQVR4nKWTPWxTVxiGn3N/bF/fGyd2YxsQ5LdSmoqGgKJMEQNZYGBFkVhY2JGYmFG3ioWZpWukqqpaqaVKq6pFIAECAkSxMA6RHZP4Jnbs+O/6xvdjcCLKT6ee7ZwjvfrOc55XiYjwP5bx740IZDf3+PVZicdrVeK2yfzJJBem02iaQv1XQCCCCNz+Lce91R1mRvtYd5uEDIfl9SqpWIjZLxOIgPooRQtE0JQiU6xx91mJfNkjZoX47vIkM2Nx6l6Xmz9kWHywgVIQBB++WImI1Nv7fP/XOqah0fKFdH8YQwXcf1Vh6WWZTrfLaDLK4rVZrJD+wSQGwJrbQtc0rs4PAXDr5xy/PHW5NJsmGQuhNI0/XrisFmtMjwxwOLVCYXTaTdq+kHagXq0iAo4phE2dn564XD8/zLlTRwn8gK1dQaHQtfcgDDMcwQo1Wc43mEp16YpibdsjEKHdEX5/8YZEpIhjCckBi9a+ibfvETEsIobdY1Bp+Pz4cAvP67C522IsbeN1A0zd5r77LWF7hebe1xxJrzKRmON56W/OHDnHwskbaCIQt03OTsbJljxeuz4rxSYXp2JcmYszedQhrNscj/ehJIKuQpiaBegHEFVPoOHBKOPpKMXdNtlSmzt/bpC0XTb9LxgcmGDq2CT5mpC0hhiO1UhGe8ANBYgCQ1dcnR9iJGnxT6ZMrtLmbV1H78/QrD0nagQ82ljCP+HzqLBEsB8wP7bQ+8ZDpoauuHA6xfnpFA3Px4mY3M2cJbeTZjTxFQYm44lv0MRkPDH1aRcOtdaUwon0rgrbBdbd10S1AXJbWRxzkLXNLDEz1VP54wDtQLHuQUl36xUKpTzl6jYFN89OdYeCm6eyV3mv8mdKxuFxueHS8PawTJuW3yAacmh26jiRfhL2IO8AhSUo7nmFnjUAAAAASUVORK5CYII=" alt="My Image">
</body>
</html>
```

## Use the tool icon data URI in the tool YAML
In the auto-generated tool YAML file, customize the tool icon by adding the data URI:
```yaml
hello_world.tools.hello_world_tool.get_greeting_message
function: get_greeting_message
inputs:
connection:
type:
- CustomConnection
input_text:
type:
- string
module: hello_world.tools.hello_world_tool
name: Hello World Tool
description: This is hello world tool
type: python
icon: data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAACq0lEQVR4nKWTPWxTVxiGn3N/bF/fGyd2YxsQ5LdSmoqGgKJMEQNZYGBFkVhY2JGYmFG3ioWZpWukqqpaqaVKq6pFIAECAkSxMA6RHZP4Jnbs+O/6xvdjcCLKT6ee7ZwjvfrOc55XiYjwP5bx740IZDf3+PVZicdrVeK2yfzJJBem02iaQv1XQCCCCNz+Lce91R1mRvtYd5uEDIfl9SqpWIjZLxOIgPooRQtE0JQiU6xx91mJfNkjZoX47vIkM2Nx6l6Xmz9kWHywgVIQBB++WImI1Nv7fP/XOqah0fKFdH8YQwXcf1Vh6WWZTrfLaDLK4rVZrJD+wSQGwJrbQtc0rs4PAXDr5xy/PHW5NJsmGQuhNI0/XrisFmtMjwxwOLVCYXTaTdq+kHagXq0iAo4phE2dn564XD8/zLlTRwn8gK1dQaHQtfcgDDMcwQo1Wc43mEp16YpibdsjEKHdEX5/8YZEpIhjCckBi9a+ibfvETEsIobdY1Bp+Pz4cAvP67C522IsbeN1A0zd5r77LWF7hebe1xxJrzKRmON56W/OHDnHwskbaCIQt03OTsbJljxeuz4rxSYXp2JcmYszedQhrNscj/ehJIKuQpiaBegHEFVPoOHBKOPpKMXdNtlSmzt/bpC0XTb9LxgcmGDq2CT5mpC0hhiO1UhGe8ANBYgCQ1dcnR9iJGnxT6ZMrtLmbV1H78/QrD0nagQ82ljCP+HzqLBEsB8wP7bQ+8ZDpoauuHA6xfnpFA3Px4mY3M2cJbeTZjTxFQYm44lv0MRkPDH1aRcOtdaUwon0rgrbBdbd10S1AXJbWRxzkLXNLDEz1VP54wDtQLHuQUl36xUKpTzl6jYFN89OdYeCm6eyV3mv8mdKxuFxueHS8PawTJuW3yAacmh26jiRfhL2IO8AhSUo7nmFnjUAAAAASUVORK5CYII=
```
## Verify the tool icon in VS Code extension
Follow [steps](create-and-use-tool-package.md#use-your-tool-from-vscode-extension) to use your tool from VS Code extension. Your tool displays with the custom icon:
![custom-tool-with-icon-in-extension](../../media/contributing/custom-tool-with-icon-in-extension.png)
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Custom tool package creation and usage
# Create and Use Tool Package
In this document, we will guide you through the process of developing your own tool package, offering detailed steps and advice on how to utilize your creation.

The custom tool is the prompt flow tool developed by yourself. If you find it useful, you can follow this guidance to make it a tool package. This will enable you to conveniently reuse it, share it with your team, or distribute it to anyone in the world.

After successful installation of the package, your custom "tool" will show up in VSCode extension as below:
![custom-tool-list](../media/contributing/custom-tool-list-in-extension.png)
![custom-tool-list](../../media/contributing/custom-tool-list-in-extension.png)

## Create your own tool package
Your tool package should be a python package. To try it quickly, just use [my-tools-package 0.0.1](https://pypi.org/project/my-tools-package/) and skip this section.
Expand Down Expand Up @@ -115,7 +115,7 @@ hello-world-proj/
```

* Step3: Go to the extension and open one flow folder. Click 'flow.dag.yaml' and preview the flow. Next, click `+` button and you will see your tools. You may need to reload the windows to clean previous cache if you don't see your tool in the list.
![auto-list-tool-in-extension](../media/contributing/auto-list-tool-in-extension.png)
![auto-list-tool-in-extension](../../media/contributing/auto-list-tool-in-extension.png)


## FAQ
Expand Down
10 changes: 10 additions & 0 deletions docs/how-to-guides/develop-a-tool/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Develop a tool
We provide guides on how to develop a tool and use it.

```{toctree}
:maxdepth: 1
:hidden:
create-and-use-tool-package
add-a-tool-icon
```
2 changes: 1 addition & 1 deletion docs/how-to-guides/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ enable-streaming-mode
manage-connections
manage-runs
column-mapping
how-to-create-and-use-your-own-tool-package
develop-a-tool/index
faq
```
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
64 changes: 64 additions & 0 deletions scripts/tool/convert_image_to_data_url.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
import argparse
import base64
import os
import io
from PIL import Image


def get_image_size(image_path):
with Image.open(image_path) as img:
width, height = img.size
return width, height


def get_image_storage_size(image_path):
file_size_bytes = os.path.getsize(image_path)
file_size_mb = file_size_bytes / (1024 * 1024)
return file_size_mb


def image_to_data_url(image_path):
with open(image_path, "rb") as image_file:
# Create a BytesIO object from the image file
image_data = io.BytesIO(image_file.read())

# Open the image and resize it
img = Image.open(image_data)
if img.size != (16, 16):
img = img.resize((16, 16), Image.Resampling.LANCZOS)

# Save the resized image to a data URL
buffered = io.BytesIO()
img.save(buffered, format="PNG")
img_str = base64.b64encode(buffered.getvalue())
data_url = 'data:image/png;base64,' + img_str.decode('utf-8')

return data_url


def create_html_file(data_uri, output_path):
html_content = '<html>\n<body>\n<img src="{}" alt="My Image">\n</body>\n</html>'.format(data_uri)

with open(output_path, 'w') as file:
file.write(html_content)


if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument(
"--image-path",
type=str,
required=True,
help="Your image input path",
)
parser.add_argument(
"--output",
"-o",
type=str,
required=True,
help="Your image output path",
)
args = parser.parse_args()
data_url = image_to_data_url(args.image_path)
print("Your image data uri: \n{}".format(data_url))
create_html_file(data_url, args.output)
2 changes: 1 addition & 1 deletion src/promptflow-tools/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,4 @@

Tools are the fundamental building blocks of a flow in Azure Machine Learning prompt flow. Each tool is a simple, executable unit with a specific function, allowing users to perform various tasks. By combining different tools, users can create a flow that accomplishes a wide range of goals. One of the key benefit of prompt flow tools is their seamless integration with third-party APIs and python open source packages. This not only improves the functionality of large language models but also makes the development process more efficient.

In this package, we provide a set of builtin tools of prompt flow, which are the most commonly used tools in the development of AI applications. We also provide a flexible way for users to create their own tools and share them with others. See [Custom tool package creation and usage](https://github.com/microsoft/promptflow/blob/main/docs/how-to-guides/how-to-create-and-use-your-own-tool-package.md) for more details.
In this package, we provide a set of builtin tools of prompt flow, which are the most commonly used tools in the development of AI applications. We also provide a flexible way for users to create their own tools and share them with others. See [Create and Use Tool Package](https://github.com/microsoft/promptflow/blob/main/docs/how-to-guides/develop-a-tool/create-and-use-tool-package.md) for more details.

1 comment on commit 037f627

@saskyagiyan
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ajarin saya dong

Please sign in to comment.