VertexAI with Nemoguardrails

[fedewj]

I am trying to integrate nemoguardrails in my RAG(built using Langchain Framework). This RAG can use both GPT and VertexAI models(gemini). With GPT all works perfectly, no error. However, when I try to use gemini I get this error:

**Error in ask**: LLM Call Exception: Task <Task pending name='Task-2953' coro=<BaseChatModel._agenerate_with_cache() running at /app/.venv/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py:715> cb=[gather.<locals>._done_callback() at /usr/local/lib/python3.10/asyncio/tasks.py:720]> got Future <Task pending name='Task-2954' coro=<UnaryUnaryCall._invoke() running at /app/.venv/lib/python3.10/site-packages/grpc/aio/_call.py:577>> attached to a different loop

These are my config:

1. config.yml:

models:
 - type: main
   engine: vertexai
   model: gemini-1.0-pro

rails:
  input:
    flows:
      - self check input

nemoguardrails=0.9.1.1
google-cloud-aiplatform=1.73.0
langchain = 0.1.13 
langchain-community= 0.0.38 
langchain-core = 0.1.53
langchain-google-vertexai = 0.1.2 

2. RAG:
   I've followed this documentation (Add Guardrails to a Chain): https://docs.nvidia.com/nemo/guardrails/user_guides/langchain/langchain-integration.html
   .....
   self.guardrails_config = RailsConfig.from_path("./data/configs_guardrails")
   .....
   return RunnableRails(self.guardrails_config, llm, input_key="question",
   output_key="answer", runnable=chain)
   .....

[Pouyanpi]

Hi @jingwangfe can you try out the most recent version (0.11.0) and see if you can reproduce this issue, and try to upgrade your dependencies.

[fedewj]

I have upgraded to the most recent version of nemoguardrails, but it did not resolve my problem.

However, I’ve implemented a workaround by modifying the invoke method of the RunnableRails class as follows (even though I received this warning : LangChainDeprecationWarning: Importing ZapierNLAWrapper from langchain.utilities is deprecated. Please replace deprecated imports:)

`    async def invoke(
        self,
        input: Input,
        config: Optional[RunnableConfig] = None,
        **kwargs: Optional[Any],
    ) -> Output:
        """Invoke this runnable synchronously."""
        input_messages = self._transform_input_to_rails_format(input)
        res = await self.rails.generate_async(
            messages=input_messages, options=GenerationOptions(output_vars=True)
        )
        context = res.output_data
        result = res.response`

Do you think there could be any issues with this modification? Do you think it's possible to solve this by upgrading all the libraries, rather than just upgrading nemoguardrails? Thank you in advance @Pouyanpi