Temperature application order non standard?

[electronjoe]

I was reading a really interesting piece on Reddit regarding samplers, and a particularly interesting exchange came up which appears to have highlighted a discrepancy between the order llama.cpp applies temperature (to probabilities) while research literature / other implementations apply temperature earlier in the chain (to logits).

I thought it would be unfortunate for this discussion to die without visibility and discussion, so I've tossed up a GH issue.

I would normally look for historical / closed issues that are related, but I'm on my phone and that's rather complex.

The interesting Reddit discussion:

https://www.reddit.com/r/LocalLLaMA/s/WonSDiMCoD

[MaggotHATE]

There was a discussion of this topic in min_p PR, but after it was already merged. I remember playing with temp being first after that, but a customizable samplers order seems like a better idea anyway.

[electronjoe]

There was a discussion of this topic in min_p PR, but after it was already merged. I remember playing with temp being first after that, but a customizable samplers order seems like a better idea anyway.

Thanks for the pointer! Closing this issue as it's on the radar of those that matter:)