Added NeMo Guardrails Chat component #3331
Conversation
|
awesome! can we do |
|
Yes! I did it and now has just 3 warnings |
| _generate(messages: BaseMessage[], options: this['ParsedCallOptions'], runManager?: CallbackManagerForLLMRun): Promise<ChatResult> { | ||
| const generate = async (messages: BaseMessage[], client: NemoClient): Promise<ChatResult> => { | ||
| const chatMessages = await client.chat(messages) | ||
| const generations = chatMessages.map((message) => { |
There was a problem hiding this comment.
can we add await runManager?.handleLLMNewToken(token ?? '') so that we can trace it for analytic as well
There was a problem hiding this comment.
Hi @HenryHengZJ
I perceived this await runManager... is used when request is a streming. In this case, my request is direct. Do you think use this command is really necessary?
There was a problem hiding this comment.
this is used for both, you can do this:
await _runManager?.handleLLMNewToken(generationResult.generations?.length ? generationResult.generations[0].text : '')
There was a problem hiding this comment.
I got it. I added this line of code (with some adjusments) and it is working fine. Thanks for this!
| const chatMessages = await client.chat(messages) | ||
| const generations = chatMessages.map((message) => { | ||
| return { | ||
| text: '', |
There was a problem hiding this comment.
should the text be empty?
There was a problem hiding this comment.
Nops, I just fixed it. :)
| ] | ||
| } | ||
|
|
||
| async init(nodeData: INodeData, _: string, options: ICommonObject): Promise<any> { |
There was a problem hiding this comment.
we can just have async init(nodeData: INodeData)
There was a problem hiding this comment.
i got it. Done!
fix: generation text has been updated with content string
|
thank you @patrickreinan! For the next iteration, we can add support for streaming as well |
|
You are welcome. Yes, let do it! |
NeMo Guardrails Chat Component
This is a new component which allow connect in LLM models through NVidia Nemo Guardrails. It is a important component for application that need decouple conversational policies from its chatflows.