Proper way of preloading persona and conversation history for chat service implementation.

[renzs6]

Hi @klshuster, Most of the answered questions here for loading persona and conversation history for blender is for the interactive/cli mode, for the chat service implementations, like the websocket service, what is the correct process for:

• Setting up personas. what are the ways for chat service to use something like "your persona: ["you like football", "you are an engineer"], etc.
• Saving conversation history per various connections in websocket, and loading it again to continue conversation/maintain context.

I have tried the following method in loading up persona/conversation history/memory but it seems like it gets confused on who's memory it should be given to.

I am adding the following inside the parley function.
self.model.observe({"text": "your persona: I am from New York City", "episode_done": False}) self.model.observe({"text": "partner's persona: I am from London", "episode_done": False})
conversation log:

{"text": "begin"}
{"text": "Welcome to Blender chat chatbot demo. Please type "begin" to start, or "exit" to exit", "quick_replies": ["begin", "exit"]}
{"text": "begin"}
{"text": "Welcome to the Blender Chatbot demo. You are now paired with a bot - feel free to send a message.Type [DONE] to finish the chat, or [RESET] to reset the dialogue history.", "quick_replies": null}
{"text": "where are you from?"}
{"text": "I'm from the UK, but I've lived in New York my whole life. What about you?", "quick_replies": null}

It also seems like it doesn't write anything to the memories since it's empty even though i "preload" memories.

debug mode log:
Sending new message: {'id': 'BlenderBot2Fid', 'episode_done': False, 'text': "I'm from the UK, but I've lived in New Yo rk my whole life. What about you?", 'beam_texts': [("I'm from the UK, but I've lived in New York my whole life. What about you?", -7.090072154998779) , ("I'm originally from the UK, but I've lived in New York my whole life. What about you?", -7.531071186065674), ("I'm from the UK, but I've lived in New York my whole life. What about you?", -7.733624458312988), ("I'm from the UK, but I've lived in New York my whole life. What about yourself?", -8.317614555358887), ("I'm from the UK, but I've lived in New York my whole life. What about you? ", -8.448592185974121), ("I'm from the UK, but I've lived in New York for a few years now. ", -8.555205345153809), ("I'm originally from the UK, but I've lived in New York my whole life. ", -8.607127 18963623), ("I'm from the UK, but I've lived in New York my whole life. What about you? ", -8.62945556640625), ("I'm originally from the UK, but I'v e lived in New York my whole life. What about you? ", -8.845654487609863), ("I'm from the UK, but I've lived in New York my whole life. I love the ci ty.", -8.919466972351074), ("I'm from the UK, but I've lived in New York my whole life. I love it there.", -8.956177711486816)], 'memories': [], 'met rics': {'clen': AverageMetric(25), 'ctrunc': AverageMetric(0), 'ctrunclen': AverageMetric(0)}}

[volkerha]

Also interested in this.

I am also wondering if observe actually writes to memory since there would be a model encoding step required somewhere. If not, how could we initialize the memory from external (persistent) conversation data?

[klshuster]

Writing personas to memory will require the model to know where to look for those personas. In your current implementation you're passing the information through text, which means you'd want to set --memory-key full_text so that the model knows to look there to write to memory.

If you want to provide a consistent memory store for each turn in the conversation, you can compile the memories in a list (e.g., ["your persona: I am from New York", "partner's persona: I am from London"]) and pass those in through the personas key in the agent observe call:

personas = ["your persona: I am from New York", "partner's persona: I am from London"]
self.model.observe({"text": "", "personas": personas "episode_done": False})

This will have the model encode the memories appropriately, as long as you have --memory-key personas

[renzs6]

Hi @klshuster it doesn't seem to be working on my end:

worlds.py

lines = ['your persona: I am from Tokyo', "partner's persona: I am from the London"]
print("===act====")
print(a)
print("~~~~~~~~~~~")
self.model.observe({"text": a["text"], "personas": lines, "episode_done": False})
response = self.model.act()
print("===response====")
print(response)
print("~~~~~~~~~~~")
self.agent.observe(response)

config file

tasks:
        default:
                onboard_world: MessengerBotChatOnboardWorld
                task_world: MessengerBotChatTaskWorld
                timeout: 1800
                agents_required: 1
task_name: chatbot
world_module: parlai.chat_service.tasks.chatbot.worlds
overworld: MessengerOverworld
max_workers: 3000
opt:
        debug: True
        models:
                blenderbot2_3B:
                        model: projects.blenderbot2.agents.blenderbot2:BlenderBot2FidAgent
                        model_file: zoo:blenderbot2/blenderbot2_3B/model
                        interactive_mode: True
                        no_cuda: False
                        memory_key: personas
                        override:
                                search_server: 0.0.0.0:10002
                                model_parallel: true
                                memory_key: personas

debug mode logs

===act====
{'episode_done': False, 'text': 'Where are you from?', 'payload': None}
~~~~~~~~~~~
parlai/core/torch_generator_agent.py:1610: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  hyp_ids = best_idxs // voc_size
===response====
{'id': 'BlenderBot2Fid',
 'episode_done': False, 
'text': "I'm from the US. _POTENTIALLY_UNSAFE__", 
'beam_texts': [("I'm from the US. _POTENTIALLY_UNSAFE__", -3.7928786277770996), ("I'm from the United States. _POTENTIALLY_UNSAFE__", -3.980670928955078), ('I am from the United States. _POTENTIALLY_UNSAFE__', -4.051711082458496), ("I'm from New Zealand. _POTENTIALLY_UNSAFE__", -4.200631141662598), ("I'm from New Zealand. _POTENTIALLY_UNSAFE__", -7.555082321166992), ("I'm from the US. _POTENTIALLY_UNSAFE____", -7.88314962387085), ("I'm from the US. _POTENTIALLY_UNSAfE__", -8.07750415802002), ("I'm from New Zealand. _POTENTIALLY_UNSAFE__ US _Potentially_Unsubstantiated__U_U", -10.786835670471191), ("I'm from New Zealand. _POTENTIALLY_UNSAFE__ US _Potentially_Unsubstantiated___U__U", -11.387863159179688), ("I'm from New Zealand. _POTENTIALLY_UNSAFE__ US _Potentially_Unsubstantiated__POTentially__Uncertain__U_U", -12.659741401672363)], 

'memories': [], 
'metrics': {'clen': AverageMetric(6), 'ctrunc': AverageMetric(0), 'ctrunclen': AverageMetric(0)}}

Is there any way to visualize all the lines/memories written to LTM or is that the memories: [] field already and my assumption that preloading the personas are not being written is correct?

Also, do you have a suggestion for the number of personas to add? should your persona and partner's persona always be equal? are there minimum suggested lines? etc.

Using full_text works, so I'm not sure what's wrong with the personas approach. I could also verify that using full_text pre-populated the memories: [] field with personas.

persona_str = "Where are you from?\nI am from London\nDid you grow up there?\nyes where are you from?\nI am from Tokyo\nI haven't been there but I heard it's a great place\nYes and the food is delicious\nWhat's your favorite food to eat there?\nI love takoyaki\nThat sounds delicious"

self.model.observe({"text": persona_str, "episode_done": False})

{"text": "where are you from?"}
{"text": "I'm from the uk. Do you like to travel? _POTENTIALLY_UNSAFE__", "quick_replies": null}

[klshuster]

If full_text is working, you can feel free to continue using that.

Also, do you have a suggestion for the number of personas to add? should your persona and partner's persona always be equal? are there minimum suggested lines? etc

This is arbitrary, feel free to experiment and see what works (i.e., there are no set guidelines)

Is there any way to visualize all the lines/memories written to LTM or is that the memories: [] field already and my assumption that preloading the personas are not being written is correct?

memories is actually slightly different; these are the memories that are generated from the conversation with the memory decoder. If you run with --loglevel debug and BB2 reads from memory it will display what memories it is reading. You can also access them directly via self.model.model.long_term_memory.memory_dict (and you can decode them with self.model.model.long_term_memory._tokenizer.decode(memory_vec)

[github-actions]

This issue has not had activity in 30 days. Please feel free to reopen if you have more issues. You may apply the "never-stale" tag to prevent this from happening.