Skip to content

Commit

Permalink
Fix Typos and Add Header Links to RAG Demo (#9)
Browse files Browse the repository at this point in the history 
Fix typos, some grammatical changes and add linkable headers to [full
RAG Demo ](https://docs.tecton.ai/docs/tutorials/full_rag)
  • Loading branch information
@kodyfisher
kodyfisher authored Oct 24, 2024
1 parent 354e702 commit 88f6e46
Showing 1 changed file with 45 additions and 46 deletions.
91 changes: 45 additions & 46 deletions restaurant-recommender/full-rag-demo.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,12 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Using Tecton's GenAI AgentService to create a better Chatbot Experience\n",
"## Using Tecton's GenAI AgentService to create a better Chatbot Experience\n",
"\n",
"By combining the Tecton GenAI functions, you can provide much better context in for your LLM applications.\n",
"Tecton GenAI offers several functions, which you can use to provide better context in to your LLM applications.\n",
"- The *prompt* provides instructions for the LLM and is enriched with context from feature pipelines or directly from the real-time context.\n",
"- *Features as tools* give the LLM the ability to retrieve additional data only as needed to respond to the user question.\n",
"- *Knowledge* function provides RAG functionality. It gives the LLM access to domain specific information that it can select from to provide responses that are relevant to the business problem that you aim to solve with generative AI. By taking unstructured text and calculating embedding vectors for them and storing them in a vector database that provides similarity search capabilities to better respond to user queries. \n",
"- The *Knowledge* function provides RAG functionality. It gives the LLM access to domain specific information that it can select from to provide responses that are relevant to the business problem you aim to solve with generative AI. By taking unstructured text and calculating embedding vectors for them and storing them in a vector database that provides similarity search capabilities to better respond to user queries. \n",
"\n",
"\n",
"If you want to run it yourself, click on the Colab button below to open the notebook in an interactive environment.\n",
Expand All @@ -27,7 +27,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Install Packages"
"### Install Packages"
]
},
{
Expand All @@ -43,7 +43,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Setup"
"### Setup"
]
},
{
Expand Down Expand Up @@ -82,21 +82,22 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Example Support Chatbot\n",
"## Example Support Chatbot\n",
"\n",
"The following cell creates the `support_chatbot` function which you will use to test drive Tecton's GenAI functionality.\n",
"The following cell creates the `support_chatbot` function which you will use to test Tecton's GenAI functionality. This function includes several parameters:\n",
"\n",
"- The first parameter `service` is a Tecton AgentService that serves context as prompts, features as tools and knowledge.\n",
"- The first parameter, `service`, is a Tecton AgentService that serves context as prompts, features as tools and knowledge.\n",
"- The second parameter is the `user_query`, a string with any arbitrary question for the LLM.\n",
"- The third parameter provides the `context` as a dictionary of session parameters provided by a mobile app.\n",
"- It creates a LangChain agent from the `service` integrating Tecton's context into the LLM interactions.\n",
"\n",
"It creates a LangChain agent from the `service` integrating Tecton's context into the LLM interactions.\n",
"\n",
"The `service` definition shapes the personalization capabilities of the LLM-driven chatbot by providing:\n",
"- feature enriched *prompts* with specific instructions on how the LLM should respond. \n",
"- *features as tools* that the LLM can use to retrieve data it needs to respond to the user.\n",
"- *knowledge* it has access to when responding to the user.\n",
"\n",
"The `context` provides the personalization values needed by the `service` to provide data services to the LLM. These are any entity key columns needed to access the features views for enriching prompts and for features as tools. If request sources are used to provide real-time values these are also listed in the context, like location or any fields that are used for real-time transformations. "
"The `context` provides the personalization values needed by the `service` to provide data services to the LLM. These are any entity key columns needed to access the features views for enriching prompts and for features as tools. If request sources are used to provide real-time values they are also listed in the context, like location or any fields that are used for real-time transformations. "
]
},
{
Expand Down Expand Up @@ -131,7 +132,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Simple Prompt\n",
"## Simple Prompt \n",
"\n",
"This first example creates a simple *prompt* without any enrichment. \n",
"\n",
Expand Down Expand Up @@ -171,7 +172,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Chatbot with a Simple Prompt\n",
"#### Chatbot with a Simple Prompt \n",
"\n",
"Try running the following cell multiple times to see how the LLM output changes."
]
Expand Down Expand Up @@ -203,22 +204,22 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"At the time of writing this notebook, multiple runs of the cell above had varied responses.\n",
"It responded with hallucinations by making up restaurants and addresses, or it provided famous restaurants in New York, which are not close to me (the user). In some cases it provided slightly better outputs where the LLM asks for more information like this one:\n",
"At the time of writing, multiple runs of the cell above had varied responses.\n",
"It responded with hallucinations by making up restaurants and addresses, or it provided famous restaurants in New York, which are not close to me (the user). In some cases the LLM provided slightly better outputs where it asks for more information like this one:\n",
"```\n",
"Sure! Can you please provide me with your location or the type of cuisine you're interested in? \n",
"```\n",
"\n",
"The results are quite underwhelming but they can provide tips on what is missing like location and cuisine. It is clear that in order to create a personalized experience for the user, better context is needed."
"The results are quite underwhelming, but they can provide tips on what is missing like location and cuisine. It is clear that in order to create a personalized experience for the user, better context is needed."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# A Feature Enriched Prompt \n",
"## A Feature Enriched Prompt \n",
"\n",
"The following example incorporates user specific data into the *prompt*. In order to provide better personalization, the next example uses:\n",
"The following example incorporates user-specific data into the *prompt*. In order to provide better personalization, the next example uses:\n",
"\n",
"- a sample dataset with basic user information like name, age, and food_preference\n",
"- a *request source* to capture the user's current location from the mobile app\n",
Expand All @@ -234,7 +235,7 @@
"from tecton_gen_ai.testing import make_local_batch_feature_view\n",
"from tecton_gen_ai.utils.tecton_utils import make_request_source\n",
"\n",
"# what if we had more up to date information about the user, like their name, age and cuisines preference\n",
"# what if we had more up to date information about the user, like their name, age and cuisine preference\n",
"user_info = [\n",
" {\"user_id\": \"user1\", \"name\": \"Jim\", \"age\": 30, \"food_preference\": \"American\"},\n",
" {\"user_id\": \"user2\", \"name\": \"John\", \"age\": 40, \"food_preference\": \"Italian\"},\n",
Expand All @@ -254,7 +255,7 @@
"def sys_prompt(location_request, user_info_fv):\n",
" return f\"\"\"\n",
" Address the user by their name. Their name is {user_info_fv['name']}.\n",
" You are a consierge service that recommends restaurants.\n",
" You are a concierge service that recommends restaurants.\n",
" Only suggest restaurants that are in or near {location_request['location']}. \n",
" Always provide an address.\n",
" Always suggested menu items. \n",
Expand All @@ -272,7 +273,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Chatbot with some user context\n",
"#### Chatbot with Some User Context \n",
"\n",
"Try running the following cell a few times by changing the user between \"user1\",\"user2\",\"user3\" and by changing the location."
]
Expand Down Expand Up @@ -325,7 +326,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"The results have gotten better, the LLM is referring to the user by name and it provides some decent restaurant options that are actually in or near the location that the mobile app provides in the `mobile_app_context` variable. \n",
"The results have gotten better! The LLM is referring to the user by name and it provides some decent restaurant options that are actually in or near the location that the mobile app provides in the `mobile_app_context` variable. \n",
"\n",
"But what happens when trying a more personalized question? The next cell asks the chatbot take into account the user's preference."
]
Expand Down Expand Up @@ -370,22 +371,22 @@
"Sure, Jim! Could you please share your preferences, such as the type of cuisine you enjoy or any specific dietary \n",
"restrictions? That way, I can recommend a restaurant that you'll love.\n",
"```\n",
"This makes a lot of sense because it the user's food preference is not part of the *prompt*.\n",
"This makes a lot of sense because the user's food preference is not part of the *prompt*.\n",
"\n",
"Adding food preference to the prompt would be a good option and you can test that out on your own. But there is an alternative that we explore in the next section, Features as Tools."
"Adding food preference to the prompt would be a good option which you can test out on your own. But there is an alternative that we explore in the next section, Features as Tools."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Features as Tools\n",
"## Features as Tools \n",
"\n",
"With Features as Tools, the LLM has the option of querying features on-demand.\n",
"The `user_info_fv` used in the previous example contains the `food_preference` field but it isn't available to the LLM.\n",
"When adding the feature view `user_info_fv` in the list of *tools* in the *AgentService*, the LLM can query all the features in the feature view and make use of that information to drive a better response.\n",
"\n",
"Try it out running multiple times with different users and locations:"
"Try running multiple times with different users and locations:"
]
},
{
Expand Down Expand Up @@ -466,8 +467,8 @@
"The results get a lot better, user1 gets American food recommendations, user2 gets Italian and user3 gets Chinese.\n",
"The DEBUG message output shows that the LLM retrieved data from `user_info_fv` before answering the question.\n",
"\n",
"### More features to answer more complex questions\n",
"In the following example we add yet another feature as tools component th the mix.\n",
"#### More Features to Answer More Complex Questions \n",
"In the following example we add yet another feature as tools component to the mix.\n",
"\n",
"By adding a list of the restaurants that the user has been to recently, the LLM can now add variety to its recommendations."
]
Expand Down Expand Up @@ -512,7 +513,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Chatbot with more tools \n",
"#### Chatbot with More Tools \n",
"Again, try running the following cell multiple times while varying the user id."
]
},
Expand Down Expand Up @@ -582,23 +583,22 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"The results are getting even better, the user is not only getting recommendations for places that fit their cuisine preference, but also places that they have not been to before.\n",
"The results are getting even better! The user is not only getting recommendations for places that fit their cuisine preference, but also places that they have not been to before.\n",
"\n",
"### But why these restaurants and not others\n",
"Savvy readers may have noticed at this point that the recommendations come from the general training of the LLM. \n",
"But why these restaurants and not others? You may have noticed at this point that the recommendations come from the general training of the LLM. \n",
"\n",
"If you are the developer of this app, you probably want to promote the restaurants that have signed up for your service and not those that haven't. This is where more prompt instructions and *knowledge* comes in which are explored in the next section... "
"If you are the developer of this app, you probably want to promote the restaurants that have signed up for your service and not those that haven't. This is where more prompt instructions and *knowledge* comes in. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Knowledge\n",
"## Knowledge \n",
"\n",
"Adding knowledge to an LLM provides domain specific information that extends beyond its base training. With it, the LLM is able to select relevant text that helps respond to user queries and avoid hallucination. \n",
"Adding knowledge to an LLM provides domain-specific information that extends beyond its base training. With it, the LLM is able to select relevant text that helps respond to user queries and avoid hallucination. \n",
"\n",
"When creating a set of *knowledge* in Tecton, the platform takes care of calculating embedding vectors and storing the data in a vector search database which the LLM workflow agent can then use to do semantic search. \n",
"When creating a set of *knowledge* in Tecton, the platform takes care of calculating embedding vectors and storing the data in a vector search database, which the LLM workflow agent can then use to do semantic search. \n",
"\n",
"With Tecton knowledge, other fields can be specified as filters for the dataset narrowing the results prior to semantic search and therefore providing information that is even more relevant given the user's question and their current context.\n",
"\n",
Expand Down Expand Up @@ -751,13 +751,13 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Transform and load the knowledge\n",
"#### Transform and Load the Knowledge \n",
"\n",
"The following cell creates a sample *knowledge* base for the LLM using `make_local_source`, notice that the description `\"Restaurants that signed up for the service\"` provides the LLM the information it needs to follow the new instruction in the prompt created above. \n",
"The following cell creates a sample *knowledge* base for the LLM using `make_local_source`. Notice that the description `\"Restaurants that signed up for the service\"` provides the LLM the information it needs to follow the new instruction in the prompt created above. \n",
"\n",
"The `source_as_knowledge` function that creates `restaurant_knowledge` calculates embedding vectors and loads the data into a vector search database.\n",
"\n",
"The complete *AgentService* will now contains the prompt, two features as tools and the newly created `restaurant_knowledge` with the intent that it only recommend restaurants that have signed up for the service."
"The complete *AgentService* will now contains the prompt, two features as tools and the newly created `restaurant_knowledge` with the intent to only recommend restaurants that have signed up for the service."
]
},
{
Expand Down Expand Up @@ -801,9 +801,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Chatbot with prompt, feature tools & knowledge\n",
"#### Chatbot with Prompt, Feature as Tools & Knowledge \n",
"\n",
"The knowledge `restaurant_knowledge` created above specifies a zipcode based filter, so now we use zip codes for location to provide this additional filtering capability for knowledge.\n",
"The knowledge `restaurant_knowledge` created above specifies a zipcode based filter. Now, we can use zip codes for location to provide this additional filtering capability for knowledge.\n",
"\n",
"Run the next cell multiple times changing the user."
]
Expand Down Expand Up @@ -858,7 +858,6 @@
}
],
"source": [
"# test the app\n",
"mobile_app_context = {\"user_id\": \"user3\", \"location\": \"28277\"}\n",
"support_chatbot(service, \"recommend a restaurant with dry-aged steaks\", mobile_app_context)"
]
Expand All @@ -868,9 +867,9 @@
"metadata": {},
"source": [
"It only recommends restaurants in the database and since we are asking for specific items, their food preference is not considered.\n",
"Adding context in prompts, tools and knowledge can really control the behavior of the LLM and avoid hallucinations.\n",
"Adding context in prompts, tools and knowledge can control the behavior of the LLM and avoid hallucinations.\n",
"\n",
"Try experimenting a bit more, below is another type of question where the user wants to visit one of their recent visits but would also like dry-aged steak. Change the user between runs and see how the response adapts to the personalization:"
"Try experimenting a bit more. Below is another type of question where the user wants to visit one of their recent visits but would also like dry-aged steak. Change the user between runs and see how the response adapts to the personalization:"
]
},
{
Expand Down Expand Up @@ -1027,12 +1026,12 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Conclusions\n",
"## Conclusions \n",
"\n",
"Adding context to LLM based applications is a powerful technique to avoid hallucinations and produce highly relevant responses.\n",
"Adding context to LLM-based applications is a powerful technique to avoid hallucinations and produce highly relevant responses.\n",
"*Prompt* engineering with the addition of up-to-date features that describe and make use of the user's specific situation enhances personalization. *Features as tools* give the LLM access to additional data as needed, making it more responsive to the user's individual situation and immediate requests. *Knowledge* provides domain relevant informat\n",
"\n",
"Tecton helps deliver a new level of generative AI capabilities while providing the robust production-grade data pipelines and serving infrastructure that AI driven applications need to achieve in order to serve broad customer facing functionality."
"Tecton helps deliver a new level of generative AI capabilities while providing the robust production-grade data pipelines and serving infrastructure that AI driven applications need to achieve in order to serve broad customer-facing functionality."
]
}
],
Expand Down

0 comments on commit 88f6e46

Please sign in to comment.