Concurrent queries

[simondpalmer]

Hi Rohan,

I have you code up and running fine. However, I want to be able to do multiple queries concurrently. I guess multiple Threads using the top_agent. For instance

def query_llama_index(query, agent):
        response = agent.stream_chat(query)

queries = ["riddle me this", riddle me that"]
for query in queries:
  thread = Thread(target=query_llama_index, args=(query, agent))
  thread.start()
  thread.join()

This runs however I receive no output from the StreamingResponse.

Can OpenAIAgent's be used in multiple threads? If so, how do I 'queue' all these callbacks in the one queue and receive them? Or would I have to create a queue for each query?
I am kind of stuck and any help would be really appreciated!