Handling custom presets

[netrunnereve]

I'm retrying Kobold (normally I'm an Ooba user) and while I'm still digging through the codebase it looks like we can't create custom sampler and instruct presets without directly modifying klite.embd. Instead we're meant to create our configs directly in the UI and then save them on disk as a json session as mentioned in #127.

I'm currently saving blank sessions with only the system prompt in the W info box, but the issue is that there really isn't much granularity when restoring the settings as it basically treats General Settings as one section. Also if I say come up with some good sampler options in the middle of a conversation I'll have to save it along with the other options and the full chat history.

For the users with multiple models and custom templates that go beyond the presets, how do you do it? I guess I'm spoiled by how Ooba has configurable automatic model detection that allows me to automatically load a custom preset in Instruct mode.

[LostRuins]

Quite a good point.

For myself, I am doing exactly as you say: I have a few "Templated" save files that are blank, but configured to the settings that I need. When I wish to start a new story with a specific setup, I can load the template that suits it best. If I have an ongoing story already, I can load that story, then overwrite the settings with the ones I desire, while keeping chat history intact.

You say that the "General Settings" is not granular enough - may I ask what sub-sections would you need?

[netrunnereve]

I think as a first step it would be good to seperate out the sampler settings like temperature and top-k so those can be loaded without affecting say the context length options.

In the future though I think easily modifyable custom presets for both templates and sampler options are necessary so that people can keep up with all those new models that are coming out nowadays. Maybe klite.embd can source an optional file containing overrides in the same format? Also the GGUF format can store chat templates and it might be worth having an option to automatically get the template from the model itself.