After summarization and reload previous messages are missing, resume crashes

[DOsinga]

Describe the bug

My conversation ran out of context and did autosummarization. it did also say that I ran out of context and had to start over, but it did do the auto summarization.

after that it showed me my previous messages and said, anything above this line is summarized, you can still the messages but they don't impact it directly. it also had the actual summarization message.

when I reloaded, I only saw the summarization message and not the previous messages. hitting resume made goose go back to the home screen instead of resuming

Expected behavior

I would expect to be able to see my previous conversation (and not the summarization message). and I would expect to be able to resume the conversation (possibly a different bug)

[iamrajiv]

hi @DOsinga i figured out whats happening here the summarization feature splits messages into two groups the summary which gets saved and the original conversation which doesnt can you try to fix this if that ok could assign me this?

[DOsinga]

yeah, if you want to have a look, that would be great - @katzdave is generally the person responsible for this feature. keep us posted!

[katzdave]

@DOsinga are you proposing we keep session messages around forever? Because the summaries replace those in the json file, so the previous messages aren't actually persisted.

If you close goose and re-open that session do you only want your messages + some line about auto-summarization (without actually seeing the summary, maybe you have to click)?

I think in that case we need to persist all the messages to the JSON file. Maybe we keep messages up to the last summary around (rather than everything)?

[DOsinga]

as discussed, we should get the conversation pr in and then tag messages in the conversation as summarized and then have a prepare_for_llm method that takes a conversation and returns a conversation with those messages filtered out. that way the client should still be able to display them, but the LLMs never see them

[DOsinga]

is this fixed on main @katzdave ?

[DOsinga]

1.11 should have this fixed