How to make the new variables input available via query method? #394
Replies: 6 comments 12 replies
-
|
HI @xiechengmude, This is mostly an LCEL issue. To debug break things into smaller chunks and inspect the input and output schemas of your chains. Try to follow the examples here and modify them: Check how |
Beta Was this translation helpful? Give feedback.
-
|
Here's a self contained example: import re
from pathlib import Path
from typing import Callable, Union
import uvicorn
from fastapi import FastAPI, HTTPException
from langchain.chat_models import ChatOpenAI
from langchain.memory import FileChatMessageHistory
from langchain.prompts import PromptTemplate
from langchain_core.chat_history import BaseChatMessageHistory
from langchain_core.runnables.history import RunnableWithMessageHistory
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langserve import add_routes
def _is_valid_identifier(value: str) -> bool:
"""Check if the session ID is in a valid format."""
# Use a regular expression to match the allowed characters
valid_characters = re.compile(r"^[a-zA-Z0-9-_]+$")
return bool(valid_characters.match(value))
def create_session_factory(
base_dir: Union[str, Path],
) -> Callable[[str], BaseChatMessageHistory]:
"""Create a session ID factory that creates session IDs from a base dir.
Args:
base_dir: Base directory to use for storing the chat histories.
Returns:
A session ID factory that creates session IDs from a base path.
"""
base_dir_ = Path(base_dir) if isinstance(base_dir, str) else base_dir
if not base_dir_.exists():
base_dir_.mkdir(parents=True)
def get_chat_history(session_id: str) -> FileChatMessageHistory:
"""Get a chat history from a session ID."""
if not _is_valid_identifier(session_id):
raise HTTPException(
status_code=400,
detail=f"Session ID `{session_id}` is not in a valid format. "
"Session ID must only contain alphanumeric characters, "
"hyphens, and underscores.",
)
file_path = base_dir_ / f"{session_id}.json"
return FileChatMessageHistory(str(file_path))
return get_chat_history
llm = ChatOpenAI()
prompt = ChatPromptTemplate.from_messages([
("system", "You're an assistant. Answer the question to the best of your ability "
"based on the conversation, the lesson {lesson} and the affection number {affection}."),
MessagesPlaceholder(variable_name="history"),
("human", "{question}"),
])
app = FastAPI()
chain = prompt | llm
chain_with_history = RunnableWithMessageHistory(
chain,
create_session_factory("./chats"),
input_messages_key="question",
history_messages_key="history",
verbose=True,
)
chain.invoke(
{
"lesson": "Math Lesson",
"affection": 66,
"question": "what's the important to learn math well?",
'history': []
},
)
chain_with_history.invoke(
{
"lesson": "Math Lesson",
"affection": 66,
"question": "what's the important to learn math well?",
},
{
'configurable': {
'session_id': 'some_random_id3',
}
}
) |
Beta Was this translation helpful? Give feedback.
-
|
hanks for your detailed reply.
|
Beta Was this translation helpful? Give feedback.
-
|
The variables are not defined in Separate it out like this: llm = ChatOpenAI()
prompt = ChatPromptTemplate.from_messages([
("system", "You're an assistant. Answer the question to the best of your ability "
"based on the conversation, the lesson {lesson} and the affection number {affection}."),
MessagesPlaceholder(variable_name="history"),
("human", "{question}"),
])
chain = prompt | llm
chain_with_history = RunnableWithMessageHistory(
chain,
create_session_factory("./chats"),
input_messages_key="question",
history_messages_key="history",
verbose=True,
)All the input variables are defined in the prompt itself. That's why this code: chain.invoke(
{
"lesson": "Math Lesson",
"affection": 66,
"question": "what's the important to learn math well?",
'history': []
},
)works! RunnableWithMessageHistory only does the job of filling in the missing |
Beta Was this translation helpful? Give feedback.
-
|
so how to make the right way to define in the RunnableWithMessageHistory? |
Beta Was this translation helpful? Give feedback.
-
|
Could you show me the right way to query ? |
Beta Was this translation helpful? Give feedback.
-
Heres the full server code:#!/usr/bin/env python
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
import uvicorn
from operator import itemgetter
from langchain.prompts import PromptTemplate,ChatPromptTemplate, MessagesPlaceholder
from langchain.chat_models import ChatOpenAI
from langserve import add_routes
from langchain.schema.output_parser import StrOutputParser
from langchain.schema.runnable import RunnablePassthrough
from langchain.schema.chat_history import BaseChatMessageHistory
from langchain.schema.runnable.history import RunnableWithMessageHistory
from langchain import globals
from langchain.memory.chat_message_histories import RedisChatMessageHistory
from src.MyRedisChatMessageHistory import MyRedisChatMessageHistory
globals.set_debug(True)
OPENAI_API_KEY="sk-xxxxxx"
import os
os.environ['OPENAI_API_KEY'] = OPENAI_API_KEY
REDIS_URL = "redis://localhost:6379/0"
llm = ChatOpenAI()
# _DEFAULT_TEMPLATE = """You are excellent assistant.\n
# Based on the pieces of previous conversation：{history}\n
# The talking lession: {lession}\n
# affection number: {affection}\n
# Human: {question}\n
# \n
# Answer:
# """
BASIC_PROMPT = ChatPromptTemplate.from_messages([
("system", "You're an assistant. Answer the question to the best of your ability "
"based on the conversation, the lesson {lesson} and the affection number {affection}."),
MessagesPlaceholder(variable_name="history"),
("human", "{question}"),
])
# BASIC_PROMPT = PromptTemplate(
# input_variables=["history", "question","lession", "affection"],
# template=_DEFAULT_TEMPLATE
# )
app = FastAPI(
title="A simple Server",
version="0.1",
description="A Simple Api Server Using Runnable Interfaces",
)
chain_with_history = RunnableWithMessageHistory(
BASIC_PROMPT | llm ,
lambda session_id: RedisChatMessageHistory(session_id, url=REDIS_URL),
input_messages_key="question",
history_messages_key="history",
verbose=True,
max_message_history=5,
)
add_routes(
app,
chain_with_history,
path="/query-test",
)
if __name__ == "__main__":
uvicorn.run("test:app", host="0.0.0.0", port=7005, reload=True) |
Beta Was this translation helpful? Give feedback.
-
|
The curl that is using a variable called "input", but the template is using a variable called: "question". Before trying to issue a curl request, invoke it the chain itself without fast api layer in the middle. It'll help identify bugs in the code. Once a regular .invoke() works, translate that to a corresponding curl request. Or use /redoc/ or /docs to help formulate the request. Also please format your code in a readable way (I already edited your question to reformat the code). |
Beta Was this translation helpful? Give feedback.
-
|
Thank you ! Then I got the error codde: 422 So whats the correct JSON body here in query way?(I could invoke the message in chain.invoke method.) |
Beta Was this translation helpful? Give feedback.
-
|
I could run the invoke code in python. theres it : |
Beta Was this translation helpful? Give feedback.
-
|
the same code after deployed. error response like that:
|
Beta Was this translation helpful? Give feedback.
-
|
Invoke code work, but the curl type is not. why? |
Beta Was this translation helpful? Give feedback.
-
|
Alright, thats the whole thing I need.
|
Beta Was this translation helpful? Give feedback.
-
|
I'll take a look at the type inference to figure out if we can improve that. I assume everything works now? Apologies that it took so long to find the answer, i'll try try to figure out what can be done to improve things so it's easier to at least debug |
Beta Was this translation helpful? Give feedback.
-
|
how did you solve that i didn't understand. I have the same need. Trying to get use some variables with the api on langserve but gives me errors like these |
Beta Was this translation helpful? Give feedback.
-
|
@xiechengmude if could you share your solution i 'd be grateful. |
Beta Was this translation helpful? Give feedback.
-
Question:
If I create the new varialbels: input_variables=["history", "input","lession", "affection"],
and setting like the below code.
I cant make the right query via the swagger docs from langserve. Whats the problem here?
Thank you !
Beta Was this translation helpful? Give feedback.
All reactions