Question-Answering Feature in BlenderBot 2.0

[atharvjairath]

Hi, I am a student and I believe that BlenderBot 2.0 is one of the best Chatbots out there. But I like the question-answering feature in GPT-3(Link). I would like to know is there a way to achieve the same while having all the goodies of Blenderbot 2.0.

Here are some of the solutions I have thought of -

1. A way, for storing the document (content from which questions will be asked) in the long-term memory?
2. In Pragmatic memory of FiD? ( Although if this approach is possible it will require training the model).

Describe alternatives you've considered
Since I have the GPT-3 API, I could direct any content bases question to GPT-3 but then I would need to develop a switch that decides which query should go to which model. (Any suggestions in this approach will be appreciated)

Therefore, it would be amazing if you can guide me on how to achieve the above solutions.
My main goal is to create an end-product which have the question-answering ability from a prompt/document as well as all the new advantages of Blenderbot 2.0

[klshuster]

If you know the documents you want to retrieve in advance, you can create a dense index for FiD, and retrieve from there (i.e., rather than retrieving from the internet). This would not require re-training the model, as you could use a trained DPR-style model for retrieving from the index - the model is trained to incorporate knowledge from "documents", regardless of where those documents come from (internet, wiki index, memory etc.)

As for Q/A ability specifically, note that BB2 was fine-tuned exclusively on dialogue datasets, so in a pure q/a setting it may not yield responses like what you'd expect from a QA-trained model

[atharvjairath]

@klshuster Thank you for answering. I get your point and it makes sense,

Here are some of the doubts/questions I have :

1. How can I create a dense index for FiD from my document and feed it to the model?. (Will this require to sacrifice replacing internet search or memory search)
2. I also understand that BB2 was fine-tuned exclusively of dialogue datasets, so it may not result in the best/accurate response. So, therefore, would it be better if I can redirect the question to GPT-3, I reckon that can be done in the query generator? If yes can you suggest me a way in that? That would be great.
3. Can I also directly write in the long-term memory? If yes then please let me know how!
4. I was also confused about the concatenation of data, correct me if I am wrong, After the query is generated we call on both Internet search and long-term memory. What happens if this is the first instance of chat or if there is no similarity between the query and summary, do the model skips contacting the data to model or passes an empty token or vector?. Also is there any importance or weights to any of the two methods (Internet and long-term memory) which decides to go with either of the response?.

Apologies in advance for asking quite a few questions and suggestions. It would be amazing if you could help me out in understanding these. Thank you!

[klshuster]

1. For generating your own index, see instructions here. Indeed in the current setup this would require sacrificing internet search (you would specify --rag-retriever-type dpr and the relevant index path args)
2. Not really sure I understand this - you're asking if there is a way to redirect the question to GPT3 instead of BB2? How does the query generator come into play here?
3. Yes, we actually write into it with the memory_decoder; you can take a look at this function to see how we read from and write to the memory. The memory module may also be useful to look at
4. If nothing is deemed relevant to provide to the model (the query generator does not generate a search query and the memory module does not have any memories), we pass "empty" documents to the model; note that it still encodes and attends in the FiD-style, but there are no real documents

[atharvjairath]

@klshuster Thank you for replying and answering all my queries.
2. With this I meant, calling gpt3 from the query generator, I wanted to know how does the query generator functions? does it use the same Retrieve and Refine with External Knowledge method?.

I was also wanted to know is there any complete paper on BB2 (except the individual components one), which can explain the full process from a chat query to a response?

Again thank you for taking the time and answering my queries. :)

[atharvjairath]

@klshuster
Also, About the approach where we create a Dense index(FAISS) for FiD, Is it still needed if we have just one document? (simple passage).
Is there any simple approach just to feed one document directly to the FiD and to the model to create an appropriate response?

[klshuster]

Is there any simple approach just to feed one document directly to the FiD and to the model to create an appropriate response?

See discussion in this issue for how to handle that scenario.

I wanted to know how does the query generator functions?

The query generator was multi-task fine-tuned on the search query generation task from the Wizard of the Internet dataset, and the persona generation task from the multi-session chat dataset. See the project pages for details (wizint, msc)