- Signup for an Azure Subscription
- Download VS Code
- Install the Azure CLI and run
az login
to get token credentials set. - Create an AI Studio Resource
- Create an AI Studio Project
To setup the development environment you can leverage codespaces, a local environment that you configure with Anaconda or venv, or a vs code docker container environment that leverages the Devcontainer.
git clone https://github.com/azure/contoso-chat
cd contoso-chat
code .
3. Install the Prompt Flow Extension
- Open the VS Code Extensions tab
- Search for "Prompt Flow"
- Install the extension
conda create -n contoso-chat python=3.9
conda activate contoso-chat
pip install -r requirements.txt
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
For codespaces click on the green code
button on the repository and select the codespaces
tab. Click create codespace...
to open the project in a Codespace container. This will automatically install all the dependencies and setup the environment. Proceed with "Create the prompt flow runtime in AI Studio".
If you're using Visual Studio Code and Devcontainer, clone the project, open it with code .
or as folder. VS Code will detect the devcontainer configuration and ask you to reopen the project in a container. Alternatively you will need to run this step manually. See the Remote Container Extension for more information. Proceed with "Create the prompt flow runtime in AI Studio".
First use command az login
to sign into the Azure Command Line SDK
Open config.json
file and add subscriptionid
, resourcegroup
, workspacename
for your Azure AI Studio instance
Follow the instructions and steps in the notebook create_compute_runtime.ipynb
under the runtime
folder.
- Follow these instructions to create an Azure Open AI resource
- Populate the
local.env
file with the endpoint and key from the Azure Open AI resource created in the previous step. - Now that the service is created use Azure AI Studio to deploy the following models to be used in the prompt flow:
GPT-4
,GPT-3.5 Turbo
, and the embedding modeltext-embedding-ada-002
. Follow these instructions to deploy the models.
- Follow these instructions to create an Azure AI Search service
- Populate the
local.env
file with the endpoint and key from the Azure AI Search service created in the previous step. - Now that the resource is created in Azure, use the notebook code and instructions
create-azure-search.ipynb
under thedata\product_info
folder to create the index and populate with the sample data
- Follow these instructions to create the resource: Create an Azure Cosmos DB
- Populate the
local.env
file with the endpoint and key from the Azure Cosmos DB resource created in the previous step. - Now that the resource is created in Azure, use the notebook code and instructions
create-cosmos-db.ipynb
under thedata\customer_info
folder to create the database, container and populate with the sample data.
To run the prompt flow, the connections need to be set up both locally and in the Azure AI Studio. When setting up the connections in the Azure AI Studio, make sure to use the same names as the local connections. Follow the instructions below to setup the connections.
-
Create the Azure AI Search connection named
contoso-search
. Follow the instructions here to setup the Azure AI Search connection -
Create Cosmos DB Custom connection named
contoso-cosmos
. Follow the instructions here to create the Custom connection to Cosmos DB. NOTE: Be sure to add all the key value pairs needed in this connection:endpoint
,key
,databaseId
,containerId
. -
Create Azure Open AI connection named
aoai-connection
. Follow the instructions here to setup the Azure Open AI connection
To simplify the local connection creation use the notebook create-connections.ipynb
under the connections
folder. This notebook will create the local connections with the naming above. Be sure to update the endpoints and keys in the notebook to create the connections to the resources created in Azure. If you prefer to create the connection mannually, follow the instructions here.
Now that the environment, resources and connections have been configured we can open up the prompt flow and take a look at how it works.
The prompt flow is a DAG (directed acyclic graph) that is made up of nodes that are connected together to form a flow. Each node in the flow is a python function tool that can be edited and customized to fit your needs.
Click on the flow.dag.yaml
file in the explorer. If everything was installed and the python environment was activated you should see the following and select visual editor
to view the prompt flow:
This will open up the prompt flow in the visual editor.
The prompt flow is made up of the following nodes:
-
inputs - This node is used to start the flow and is the entry point for the flow. It has the input parameters
customer_id
andquestion
, andchat_history
. Thecustomer_id
is used to lookup the customer information in the Cosmos DB. Thequestion
is the question the customer is asking. Thechat_history
is the chat history of the conversation with the customer. -
question_embedding - This node is used to embed the question text using the
text-embedding-ada-002
model. The embedding is used to find the most relevant documents from the AI Search index. -
retrieve_documents - This node is used to retrieve the most relevant documents from the AI Search index with the question vector.
-
customer_lookup - This node is used to get the customer information from the Cosmos DB.
-
customer_prompt - This node is used to generate the prompt with the information retrieved and added to the
customer_prompt.jinja2
template. -
llm_response - This node is used to generate the response to the customer using the
GPT-35-Turbo
model. -
outputs - This node is used to end the flow and return the response to the customer.
Now that we have the prompt flow open in the visual editor, we can run the flow and see the results. To run the flow, click on the Run
play button at the top. For more details on running the prompt flow, follow the instructions here.
Once the prompt flow is setup and working, its time to test it and evaluate the results. To do this we have included some evaluation prompt flows in this project that will use GPT-4 to test the prompt flow.
Follow the instructions and steps in the notebook evaluate-chat-prompt-flow.ipynb
under the eval
folder.
Now that you have validated and corrected any issues with the prompt flow performance. Its time to push the solution to the cloud and deploy.
Follow the instructions and steps in the notebook push_and_deploy_pf.ipynb
under the deployment
folder.
-
Login to Azure Shell
-
Follow the instructions to create a service principal here
-
Follow the instructions in steps 1 - 8 here to add create and add the user-assigned managed identity to the subscription and workspace.
-
Assign
Data Science Role
and theAzure Machine Learning Workspace Connection Secrets Reader
to the service principal. Complete this step in the portal under the IAM. -
Setup authentication with Github here
{
"clientId": <GUID>,
"clientSecret": <GUID>,
"subscriptionId": <GUID>,
"tenantId": <GUID>
}
- Add
SUBSCRIPTION
(this is the subscription) ,GROUP
(this is the resource group name),WORKSPACE
(this is the project name), andKEY_VAULT_NAME
to GitHub.
-
Follow the instructions to create a custom env with the packages needed here
- Select the
upload existing docker
option - Upload from the folder
runtime\docker
- Select the
-
Update the deployment.yml image to the newly created environemnt. You can find the name under
Azure container registry
in the environemnt details page.
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.
When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.
This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.