Model Feedback

[LoggeL]

Hi 👋
First I got to say that this is a really cool project. It's easy to install and to get started with.

I've played a bit around with it and had the problem that it would constantly fail the LLMSummaryCheck (using llama3:8b). I thought about giving the generator some feedback on why it failed last time. I did this by adding a "Reason" parameter to the json and adding that to the prompt which writes the scene. It doesn't work 100% but I it finds faster to a valid chapter now.
Would be interested to know if you have a better approach to this.

[datacrystals]

Thanks for the interest!

Hmm, would you mind sharing your arguments to write.py? I've noticed that it'll tend to fail with the same models, but actually a reason is pretty clever - If you're willing, I'd be interested in merging that in.

[LoggeL]

I just pushed my current implementation here:
LoggeL@95c986e

It currently sometimes results in the model mentioning that the chapter is revisioned.
E.g.: Here is the rewritten Chapter 1: The Desperation with added dialogue:
Still trying to fix that.

[datacrystals]

Ah yes - that's behavior that I've already seen - for now I wouldn't worry about it. The scrub step is supposed to fix that, where it just goes through the generated text and strips out that extra stuff. I've not tested that recently though - so it might have some issues.