compact causes llm to forget convo

[tylerlaprade]

Claude seemingly forgot the message he just sent about remaining work after "finishing" (giving up halfway through) his task. This felt like he was starting a fresh session, even though I was referencing what he just said.

https://opencode.ai/s/t3wDHDQg

[rekram1-node]

ah yeah this is a due to compact

[rekram1-node]

we should fix

[OpeOginni]

Hey @rekram1-node I see you already have this issue assigned, but is it still okay if I take a look, I find it interesting.

[rekram1-node]

@OpeOginni u are welcome to look at it but I think compact needs some rework and that may require discussion w/ team so just a forewarning

[tylerlaprade]

Oh really? I didn't see a compact around that time.

[rekram1-node]

@tylerlaprade the second to last ai message is the llm auto compacting the conversation by the looks of it

[tylerlaprade]

Ah, interesting, that actually explains quite a lot. I often see similar messages when it seems to prematurely declare victory and give a summary of its accomplishments. I guess those were actually compacts.

[OpeOginni]

@rekram1-node from the code it seems like auto compacting is implemented as it should (checks if model limit is reached and then runs a summarize), if you could please explain what rewrite you think needed?

Only thing I can really see is the summarize prompt is a bit little, and maybe for this exact case priority could be made to the last message made by the assistant in the summary.

[rekram1-node]

imo I think compact shouldn't literally replace entire convo, last n messages should stay as is and the compact should apply to the messages before it to help prevent this "memory loss"

Compact prompt could probably also use work

but these are kinda core things

Ultimately the decision isn't up to me, I am a maintainer but I don't make authoritative decisions on this project, that is the up to the team

[OpeOginni]

Makes sense, I'll leave this one for now then, but will keep an eye out 👀

[robertmclaws]

Just had this one happen to me with Grok Code Free.

[rekram1-node]

This should be fixed on latest! Dax just did some compact updates