Implementation of logit threshold sampler and confidence breaker

[anchortense]

Edit 23/10: Simplified the implementation of logit threshold sampler, removing the previous logit_min_threshold parameter and replacing it with a new temperature parameter applied only to logits above logit_temp_threshold. This should make the sampler easier to tune across different models, with conservative settings (logit_high_temp=3.0; logit_temp_threshold=16.0) improving text diversity while maintaining coherence across a number of tested models (Gemma 9b, Llama 3.1 8B, Llama 3.2 3B, Mistral Nemo 12B). The below explanations and example outputs have been updated to reflect these changes.

Implementation of two new samplers enabling coherent diverse text generation.

• The logit threshold temperature Enables the application of much higher temperatures to stronger candidates based on a logit value threshold, producing varied outputs without losing coherence.
• The confidence breaker sampler addresses repetitive text sequences by dynamically detecting them on the logit level, allowing the model to generate more diverse responses. CB builds upon the existing implementation of banned strings in exllamav2.

The key innovation is to use absolute logit values instead of softmax probabilities to retain the model’s raw confidence in each token. By using the raw logits to make decisions, we preserve the model's absolute confidence in each token, allowing for more nuanced sampling based on how confident the model is about each token.

Explanation: Logit threshold sampler

In language model text generation, adjusting the temperature parameter affects the randomness of the output:

• Low temperature produces highly probable and repetitive token selections, leading to coherent but less diverse and potentially uninteresting text.
• High temperature increases randomness by making less probable tokens more likely, enhancing diversity but often at the expense of coherence, introducing errors or nonsensical content.

Leading approaches to addressing these issues, such as min-p, rely upon the manipulation of token probabilities after softmax, however this can only go so far, as the softmax operation discards absolute logit values, so that we can no longer distinguish between model uncertainty in cases where probabilities are flat due to many good choices vs no good choices. This despite the fact that ideally we want to broaden our options in the first case and narrow them in the second.

On the other hand, Logit threshold temperature allows us to avoid the diversity-coherence trade-off to a far greater extent through selectively applying high temperature only to high-confidence tokens. This enhances creativity among the tokens the model is most confident about, without introducing incoherence. Those remaining tokens below this threshold are not eliminated, but neither do they increase their existing probabilities.

Primary Applications

• Coherent creativity in outputs
  • Finely tune the level of creativity in generated text. By applying higher temperatures selectively, the sampler allows for diverse word choices among tokens the model deems highly coherent.
• Enhancing agent diversity
  • In systems where multiple agents use the same language model, this sampler introduces variability in their responses. Each agent can generate unique yet coherent outputs, enabling more sophisticated interactions between differing perspectives.

Parameters

• logit_high_temp - The temperature applied to logits above logit_temp_threshold
  • As you raise this temperature, the threshold will also need to be raised to maintain coherence.
  • Tune this up or down to increase/decrease the diversity of outputs above the threshold.
• logit_temp_threshold - The threshold above which logits will be subjected to temperature scaling, based on the assumption that they are highly coherent choices.
  • Tune this up or down to increase/decrease the coherence floor

Suggested initial conservative starting values: logit_high_temp=3.0; logit_temp_threshold=16.0 - if this is a little too wild then bump up the threshold in increments of 1-2 and observe outputs for coherence.

As with all samplers, you may need to adjust these settings based on intended use case and model.

Explanation: Confidence breaker sampler

Current generation language models are well known for producing certain cliched phrases, which would not necessarily be problematic in a single instance, but are known to be produced repeatedly in response to varied prompts. This is the so-called ai-slop problem. In 'deterministic' use cases this is usually not an issue, as we are simply looking for the one correct answer. In scenarios where engaging, diverse language choices are valued, ai-slop represents a significant limitation.

Current approaches to resolving this issue involve user defined lists of banned strings. This can be reasonably effective, however fail to address the deeper issue, which is the tendency of language models to funnel their responses into unintentionally learned 'tram-track' token sequences, where token choices are strongly conditioned by their immediate predecessors in a manner which is not logically or grammatically implied by the prompt or by those preceding choices.

This is a far deeper and more pervasive problem than the well known handful of phrases commonly associated with the idea of ai-slop. These tram-tracks exist within trained models because the tokens within them are good predictors for text completion. Nevertheless a user passing either the same or similar prompts repeatedly, looking for diverse outputs, will quickly observe that model responses which seemed initially impressive are in fact tram-track patterns, and the apparent diversity of outputs is an illusion.

Using the confidence breaker to jump tracks

The confidence breaker sampler addresses the issue by looking for logit patterns that signal we have entered a tram-track, and extending exllamav2's banned string functionality to roll-back to the token directly before we entered the tram-tracks. The tram-tracked tokens are then discarded and replaced by a novel generation, which will take us down a different, less traveled path.

Based on empirical observation of logits and the conditions for the appearance of tram-tracks, the pattern that the confidence breaker looks for to identify these tram-tracks is a sequence of mid-high valued logits, logits which have been nudged higher than a good score by over-training, but which are not yet so guaranteed as logical or grammatical necessity.

Empirical observation also validates the decision to return back and alter the token before the tram-track, rather than the first tram-track. This is a token which had a reasonable range of viable alternatives, but once the model settled on the decision it made, the tram-track became nearly inevitable. So, this is the error we have to correct.

Parameters

• confidence_breaker - An integer setting the length of a sequence match. If there are confidence_breaker flagged tokens in a row, then rollback is triggered.
  • For Gemma 2 9B, a good starting point is confidence_breaker=8
  • Tune this up to lengthen the required number of tokens before a confidence breaker is triggered. Tune it down to lower them. You can drop this down to 3 or even to 2 at your own risk for detection of very short sequences, or extend it out to larger numbers like 20 or 30.
• cb_mid_threshold - If a logit is encountered above this threshold and below the high_threshold, it is counted as a flagged token.
  • If this value is <= 1, it is treated as a probability value, if it is > 1 it is treated as a logit threshold
  • For Gemma 2 9B, a good starting point is cb_mid_threshold=18.0 - it is recommended to use this as a logit threshold, as these reflect absolute token assessments
  • Tune this up to tighten the conditions for detecting tram-tracks, tune it down to loosen them
• cb_high_threshold - If a logit is encountered above this threshold, it is assigned a None flag, and neither adds to the sequence nor resets it. This way the model is permitted to provide a prediction that it is highly confident if it is the only viable option. For example if it follows logically or grammatically from previously generated tokens.
  • If this value is <= 1, it is treated as a probability value, if it is > 1 it is treated as a logit threshold
  • For Gemma 2 9B, a good starting point is cb_high_threshold=0.999 - it is recommended to use this as a probability value, as this reflects a relative confidence with respect to other options
  • Tune this down if the model starts escaping tram-tracks by resorting to incoherence, tune it up if you get too many tram-tracks slipping through
• confidence_breaker_debug - This is a boolean True/False (defaulting to False), which prints out a message containing the discarded text whenever a rollback is initiated.

Note on use

1. Confidence breaker is a workaround for model overfitting. Its use may be detrimental to a model that has no overfitting (however all major models do, for now).
2. Use the high_threshold parameter wisely, this is how you ensure that the model can still maintain coherence if following a tram-track is the only way to do so.
3. It is entirely possible to dial the settings for this sampler too high, resulting in the model never producing an accepted output, or only after a very large number of rejected solutions. Loosen the settings a little if you find that generation is taking longer than you can bear.
4. Sometimes discarded text may be shorter than the confidence breaker setting, where there has been a rollback and then the newly generated tokens combine with still-prior generated tokens to form a new tram-track pattern. There is no recursive rollback in this case to the beginning of that new tram-track, but rather we return directly to the previously triggered rollback point and try another token from there. In this way we avoid escaping from one tram-track by jumping into another.

Output comparison

Prompt: "Write at least 500 words, beginning with and in the same style as the following:\nOnce upon a time,"

Model: gemma-2-9b-it-exl2-6bpw

Note that the trials below are cut-off after the first paragraph block.

Baseline output for comparison

settings.temperature = 1.0

Trial 1.

Once upon a time, nestled amongst rolling hills and whispering willows, lay the village of Oakhaven. It was a place where life unfolded at a gentle pace, where the scent of freshly baked bread mingled with the earthy aroma of tilled soil, and where laughter echoed through cobblestone streets as children chased fireflies at dusk. But beneath the idyllic veneer, a shadow lurked. For generations, Oakhaven had been plagued by a mysterious enchantment, a creeping melancholy that settled over its inhabitants like a shroud.

Trial 2.

Once upon a time, nestled amongst rolling hills and whispering willows, lay the village of Willow Creek. It was a place where laughter echoed through cobblestone streets, where the warmth of freshly baked bread filled every home, and where the nights were alive with the twinkling tales spun by elder firekeepers. But beneath this idyllic surface, a shadow lurked – a whisper of forgotten magic, a secret yearning for something more.

Trial 3.

Once upon a time, nestled amongst rolling hills draped in emerald moss and kissed by the sun's golden rays, lay the village of Whispering Willows. It was a place where time seemed to flow a little slower, where the gentle murmur of the river Willowbrook was the village's constant lullaby, and where the villagers lived simple, contented lives.

Trial 4.

Once upon a time, nestled amidst rolling emerald hills and whispering willow trees, lay the quaint village of Sunhaven. Here, life flowed at a gentle pace, punctuated by the clang of the blacksmith's hammer, the melodic bleating of sheep, and the warm laughter of children chasing fireflies in the twilight. But beneath this idyllic surface, a secret slumbered, a tale whispered only in hushed tones around crackling hearths. A secret that spoke of an ancient prophecy, an echo of a forgotten magic, and a destiny

Trial 5.

Once upon a time, nestled amidst rolling emerald hills and whispering willow trees, lay the village of Whisperbrook. It was a place where life moved at the gentle pace of a babbling brook, where laughter echoed through cobblestone streets, and where the scent of freshly baked bread mingled with the sweet perfume of honeysuckle.

# With min_p + temperature last
settings.temperature = 1.5
settings.min_p = 0.025
settings.temperature_last = True

Trial 1.

Once upon a time, nestled amongst rolling hills that wore crowns of emerald pines, lived a young woman named Elara. Unlike other girls her age, who dreamt of grand balls and handsome princes, Elara yearned for adventures beyond the cobblestone streets of her village. Her heart thrummed with a wild, untamed rhythm, longing for the whispered secrets of ancient forests and the call of mountains shrouded in mist.

Trial 2.

Once upon a time, nestled amongst the emerald foothills of the Whispering Mountains, lay the kingdom of Atheria. Its people, known for their vibrant artistry and unyielding optimism, lived in harmony with nature, their lives interwoven with the rhythm of the seasons. Atheria was ruled by a benevolent queen, Elara, whose wisdom flowed like the silver river that snaked through the heart of the kingdom. But even in this realm of perpetual sunshine and song, a shadow was brewing.

Trial 3.

Once upon a time, nestled between towering redwoods and whispering ferns, lived a young girl named Elara. Her hair was the color of fallen leaves, her eyes like moss-covered stones, and her laughter rang like wind chimes in the forest breeze. Elara knew every inch of the ancient woods, every secret path, every whispering brook. She spoke the language of birds, understood the sighs of the trees, and could track a deer through the densest undergrowth.

Trial 4.

Once upon a time, nestled between towering emerald hills and a sapphire sea kissed by gentle waves, lay the quaint village of Sunhaven. Its cobblestone streets, worn smooth by generations of footsteps, wound around houses painted in hues reminiscent of ripened fruit and blushing sunsets. Laughter often spilled from open windows, mingling with the scent of freshly baked bread and the melodies of birdsong. Life in Sunhaven flowed at a leisurely pace, governed by the rhythm of the sun and the whispers of the wind.

Trial 5.

Once upon a time, nestled amongst the silver-leafed birch trees and whispering willows by the banks of the River Whisper, lived a girl named Elara. Her hair was the color of spun moonlight, her eyes like pools reflecting the twilight sky, and her spirit as untamed as the winds that rustled through the ancient trees. Elara wasn't like the other village girls, content with their embroidery hoops and quiet afternoons. She craved adventure, her heart yearning for the mysteries that lay beyond the familiar boundaries of

# With XTC
settings.temperature = 1.0
settings.xtc_threshold = 0.1
settings.xtc_probability = 0.5

Trial 1.

Once upon a time, nestled amongst the emerald foothills of a slumbering volcano, lay the village of Asteria. A place where sunlight dripped like honey through the leaves of ancient willow trees, and laughter echoed from cobblestone streets lined with quaint flower stalls and bakeries overflowing with the scent of warm bread. Life in Asteria moved at the leisurely pace of a hummingbird's wings, each day a gentle brushstroke on the canvas of existence.

Trial 2.

Once upon a time, nestled amidst rolling emerald hills and whispering willow trees, lived a village known as Sunhaven. Its name was well-earned, for the sun seemed to linger longer in that place, bathing the cobblestone streets and thatched rooftops in a glorious golden light. Life there flowed at a gentle pace, dictated by the rhythms of the seasons and the cheerful clanging of the blacksmith's hammer. The villagers were known for their kindness, their hearty laughter echoing through the market square, and their unwavering belief in the magic that whispered within the heart of their world.

Trial 3.

Once upon a time, nestled between the whispering pines and the silver ribbon of Willow Creek, sat the village of Whisperwind. It wasn't much to look at, cobbled streets winding through rows of gingerbread cottages painted in pastel hues, smoke curling lazily from chimneys topped with crooked teacup weathervanes. But within its humble walls simmered a secret, a magic hummed beneath the everyday chatter and laughter, as palpable as the scent of honeysuckle that drifted on the evening breeze.

Trial 4.

Once upon a time, nestled amidst emerald hills and whispering willows, lay the quaint village of Sunshadow. A ribbon of silver, the Sunstream River, snaked through its heart, reflecting the dancing sunlight and lending an air of perpetual serenity to the village. Life there unfolded at a measured pace, governed by the rising and setting of the sun, the blossoming and fading of flowers, and the gentle rhythm of the river’s flow.

Trial 5.

Once upon a time, nestled amidst emerald hills and whispering willows, lay the village of Willow Creek. Its inhabitants, hardy folk with hearts as warm as their hearth fires, lived simple lives, their days a tapestry woven from the rhythms of nature.

Logit threshold sampler

settings.temperature = 1.0 
settings.logit_high_temp = 3.0
settings.logit_temp_threshold = 14.0

Trial 1.

Once upon a time, the city of Lumiëres thrummed with an ethereal glow. Nestled amongst sun-kissed valleys and towering sapphire peaks, its houses shimmered like opals under the caress of perpetually setting suns. The streets, paved with obsidian that absorbed the light from ten painted heavens, glittered with every passing footstep. Here, magic pulsed through every vein of existence, not as spells whispered or incantations cast, but as an inherent, living tapestry that breathed life into every stone and bloom.

Trial 2.

Once upon a time, in a kingdom nestled between a shimmering desert and a mountain range cloaked in emerald mist, lived a girl named Elowen. She wasn't your average princess, adorned with tiaras and longing for ball gowns. Elowen’s crown was a tangle of windswept chestnut hair, and her ball dresses were the canvas of her sky-blue gown, splashed with paints she collected from the humming birds who favoured her flower garden.

Trial 3.

Once upon a time, in a sleepy village nestled amongst emerald hills and silver streams, lived a baker named Elara whose pies were sung about by birds and devoured by all who smelled the sweet warmth of her oven. Elara, with her sunshine smile and sprinkle-covered fingertips, was loved as much for her gentle spirit as for her delicious creations. But within her humble bakery, a shadow lurked. A fear that chilled Elara to her very core - the fear of losing her recipe, the one passed down for generations, whispered to her on a stormy night when wind rattled the shutters.

Trial 4.

Once upon a time, nestled amongst emerald hills and kissed by the golden touch of perpetual sunrises, lay the Kingdom of Lumios. It was a place woven from spun moonlight and vibrant flower meadows, where rivers flowed with the laughter of children and trees spoke wisdom only those with listening hearts could understand. The Queen, her crown adorned with stars she'd gleaned from fallen meteor showers, ruled with kindness and grace, while the King, a mountain forged into being by love and laughter, defended his people with unwavering courage.

Trial 5.

Once upon a time, in a kingdom perched precariously on the back of a giant turtle named Thales, lived Elara, a weaver whose hands could stitch dreams into tapestry. Unlike her fellow weavers who crafted floral and celestial scenes, Elara wove nightmares – creatures made of tangled threads and moonlight whispers, emotions woven into shivering silences and fleeting shadows. They hung in her workshop, grotesque and mesmerizing, each one pulsating with the echo of someone's forgotten fear.

# LTS, pushing the temperature even further up
settings.temperature = 1.5
settings.logit_high_temp = 10.0
settings.logit_temp_threshold = 14.0

Trial 1.

Once upon a time, nestled beneath the crimson cloak of the dying summer sun, sat a tiny teahouse called the Crimson Blossom. Perched on the crest of Willow Creek Hill, it commanded a panoramic view – an unending vista of meadows dusted with fire-toned lilies, orchards brimming with gold apples, and fields so wide, they stretched beyond the reach of eyesight, lost somewhere amongst emerald forests that teemed with twilight murmurs. For those who climbed the meandering pathway to the hill, the Crimson Blossom was a welcome haven.

Trial 2.

Once upon a time, in a hidden village carved into the side of a sapphire blue mountain, lived a weaver whose touch coaxed whispers of stories into every strand of shimmering silk. They say his hands, more attuned to the universe's hidden currents than to ordinary needles, were once the talons of a wise eagle who bargained for their artistry. How else, could he spin such mesmerizing tales into thread and fleshing them out, warp by warp, with colour that sang of moonlit forests and sun-kissed valleys.

Trial 3.

Once upon a time, there was a little silver dragon who could do almost anything but make wishes become true. Her scales sparkled like freshly fallen snow, her horns curled upwards towards the amethyst sunset, and her fire, when breathed, tasted of lemon tarts and dreams forgotten. All around her in the hidden valley where she lived, trees blossomed with jade flowers and sapphire ferns tickled the wind with their whispering laughter. Still, amidst this enchantment, little Flicker longed for the most impossible thing - she yearned to understand her brother's shimmering silver tears that sometimes stained his sapphire throat feathers. His were not tears of sadness or distress, for his sapphire eyes, usually twinkling with mischievous laughter, would shine bright even while his feathers trembled with silent sorrows. Flicker simply did not know why he cried.

Trial 4.

Once upon a time, in a realm veiled with shimmering moonlight and where forests whispered ancient secrets, dwelled Princess Elara. Unlike princesses renowned for their dainty fingers and delicate stitching, Elara bore callouses from braiding sturdy ropes and eyes that gleamed with a longing for the boundless freedom of the sky.

Trial 5.

Once upon a time, nestled between snow-capped mountains kissed by lavender clouds and sapphire glaciers that winked in the sun, lay the whimsical kingdom of Lumiér. Inhabited by mischievous Pixi flitting between vibrant fungi gardens, and benevolent Nymph whose song rippled through waterfalls and danced within crystal streams, Lumiér hummed with the enchantment of magic untamed. At its heart, draped like moonlight on spun glass, sat the magnificent Luminary – an ancient palace built not of stonel but starlight captured by benevolent star sprites and etched into towering gossamer pillars. Here, lived the celestial Sovereign, Lady Elara, known throughout the lands as the Heartkeeper, her luminous heart mirroring the glow that pulsed from her dwelling; the heartbeat of the entire kingdom.

Confidence breaker

# CB with moderate settings
settings.temperature = 1.0
settings.confidence_breaker = 3
settings.cb_mid_threshold = 18.0
settings.cb_high_threshold = 0.999

Trial 1.

Once upon a time, tucked between emerald-hued rice fields that swayed in rhythmic patterns to a whispered breeze, nestled a village unlike any other. Here the villagers, known for their radiant smiles and laughter as infectious as a summer sunshower, possessed a unique gift – each held a piece within the ancient, shimmering tapestry of dreams. They spun stories from starlight, wove emotions into moonlight, and painted landscapes with their whispered lullaby melodies, all captured within their individual threads of luminous fabric. These dream-tapestries, vibrant and ever-evolving, hung in each home like shimmering galaxies, a constant testament to the villagers' boundless imagination and shared history. The tapestry of Elder Esha's home, however, was strangely subdued tonight. It usually blazed with the colors and energy of a thousand suns, reflecting her spirit, as bright and boundless as a cerulean dawn sky, but tonight it hung dim and muted, a canvas of hushed twilight hues mirroring the sorrow that etched lines on her usually cheerful face

Trial 2.

Once upon a time, in a valley cloaked by emerald forests and kissed by silver moonlight streams, there existed a village named Evergleam, known for its harmonious melodies and vibrant laughter, where the air itself shimmered with a joyful aura. Its inhabitants, the Lunare, were a people of exceptional kindness, their skin as luminous as polished amber under twilight’s soft glow. Their laughter echoed through the ancient oak trees, and the sound of their weaving looms, crafting tapestries woven from moonlight and dreams, lulled the village into tranquil sleep. They lived in perfect rhythm, guided by whispers of the wind and the gentle pulse of nature.

Trial 3.

Once upon a time, there sat beneath a gnarled oak, as ancient as its surrounding forest, a young witch by the name Elora. Her cauldron, normally brimming over with fantastical concoctions, lay cold, abandoned beside a stack of unlit herbs and dried blossoms. The air around her throbbed with an unsettling stillness, a silence thick enough to choke on. Elora wasn't grieving, not precisely. It wasn't sorrow that weighed upon her heart, a heavy stone nestled deep within its hollow. It was… discontentment, an itch she could never fully satisfy. Years spent mastering the art of brewing potions, weaving charms and whispering spells for the good of the nearby villages had grown tedious, predictable.

Trial 4.

Once upon a time, in a valley cradled amidst the ancient mountains that touched a perpetually stormy sky, there existed an inn called The Waking Hourglass. Built into a gnarled and knobbly oak that had witnessed centuries pass, its walls thrummed and echoed the whispering stories carried by the wind.

Trial 5.

Once upon a time, in a valley nestled between shimmering waterfalls, there existed an ancient oak, its branches sprawling wide like a wise grandmother welcoming weary souls beneath its leafy shade. This oak was known throughout the valley and beyond as The Heartkeeper.

# CB with a longer breaker
settings.temperature = 1.0
settings.confidence_breaker = 8
settings.cb_mid_threshold = 17.0
settings.cb_high_threshold = 0.999

Trial 1.

Once upon a time, deep beneath a silver-threaded sky, nestled amongst rolling hills of lavender and sapphire grass, resided a town named Whisperwind. Its houses stood crafted from luminous moonstone, their spires reaching upwards like fingers grasping for starry blessings, their windows glowing with a thousand soft fires that banished even the deepest shadows. The townsfolk themselves were a peculiar breed – gentle whispers of wind woven from star-dust, their laughter a tinkling of distant chimes, their sorrows sighs echoing through rustling reeds.

Trial 2.

Once upon a time, in a forgotten pocket dimension veiled from our mundane reality, lived a peculiar race known as the Luminaries. Their bodies weren’t of flesh and bone, but intricate constellations of shimmering starlight, each twinkle a spark of cosmic energy. They existed on a plane where thoughts became constellations themselves, and emotions painted galaxies. Harmony was their nature, for their very essence was the cosmic ballet, a celestial orchestra where every being played a vital role in maintaining balance and wonder.

Trial 3.

Once upon a time, nestled amongst rolling hills of shimmering sunflowers, sat Oakhollow, a village seemingly stitched directly into a dreamscape. Its houses weren't merely built, they were crafted; thatched roofs curved gracefully into fantastical shapes, chimneys puffing out smoke swirls resembling ethereal dragons. Stone walls held the warmth of hearth fire within, and windows shone brightly, casting dancing patterns onto cobblestone streets. Laughter echoed from every doorway, mingling with the sweet music carried on the gentle breeze from hidden harps and wind chimes.

Trial 4.

Once upon a time, nestled between rolling green meadows where wild strawberries ripened, and whispering woods thick with ancient trees and mossy paths, there lived a peculiar creature. Not a beast or fairy tale troll or mischievous sprite, but something altogether different – an alchemist, who called himself Thistleweed. Now, most alchemists you might imagine were grand lords in their imposing towers, filled to bursting with brass retorts, glass bubbles, and the pungent aroma of bubbling elixirs. But Thistleweed was a simpler fellow.

Trial 5.

Once upon a time, nestled amidst rolling emerald hills and whispering willow streams lay Willow Creek, a village untouched by grand conquests or sweeping revolutions. Here life flowed at the measured rhythm of the seasons, guided by the warmth of sunrises and the silvered tapestry woven across moonbeams. Each resident bore the mark of their heritage, woven into the very fabric of their being – from the stonecutter with hands roughened by granite to the cobbler whose nimble fingers sang stories into every leather stitch.

Logit threshold sampler + Confidence breaker

# LTS + CB, conservative settings
settings.temperature = 1.0
settings.logit_high_temp = 3.0
settings.logit_temp_threshold = 14.0
settings.confidence_breaker = 8
settings.cb_mid_threshold = 17.0
settings.cb_high_threshold = 0.999

Trial 1.

Once upon a time, nestled amongst rolling hills of whispering barley and shimmering blackberry bushes, stood a village unlike any other. Named Whispering Willows after the long, mournful trees that lined its boundaries, the village was known for its inhabitants, all imbued with an extraordinary ability: they spoke with animals. From the squeaky chipmunks gathering nuts in the dappled sunlight to the watchful owls hooting secrets in the crisp twilight, the whispers of the wild carried easily on the gentle breeze and into the ears of every Villager.

Trial 2.

Once upon a time, nestled amongst rolling emerald hills kissed by a silver moon, resided a quaint village called Briarbrook. Cobblestone streets, whispering secrets carried by the wind, snaked past houses adorned with fragrant jasmine and rosebushes that bloomed in defiance of the seasons. It was a haven, a place where life unfolded like a perfectly spun tapestry woven from warmth, laughter, and the rhythmic thrum of community. At the heart of this idyll lived Anya. She wasn't royalty, nor possessed any special gift. Anya was simply the baker's daughter, her days spent swirling yeast, kneading dough, and dreaming of adventures beyond the frosted windowpanes that looked out onto the peaceful village square.

Trial 3.

Once upon a time, in a world woven from starfire dust and dreams, lived a tiny creature named Flicker. Now Flicker wasn't like the other denizens of Lumistella - winged nymphs with eyes that shimmered like sunrise, wise old trees that gossiped in rustling leaves, or even the glinting, chittering fireflies that danced through the dusk. No, Flicker was different. Flicker was a Wisp.

Trial 4.

Once upon a time, nestled between a whispering bamboo forest and a mountain capped with snow that glistened like fallen diamonds, lay the hidden village of Yumi. This wasn't a village known on earthly maps; it resided within the folds of the Hidden Kingdom, a realm accessible only to those whose hearts hummed with genuine compassion and whose souls held an unyielding desire for knowledge. Here, time flowed differently, etched not in seconds and minutes, but in the seasons and the lunar cycles.

Trial 5.

Once upon a time, the whispering wind sang tales of forgotten magic in the crumbling stonework of Elderon Keep. It hummed through the shattered windows, a lone mournful dirge in the echoing chambers, a song that carried secrets on its dusty breath. Centuries of slumber clung to the Keep, each inch coated in a velvet layer of moss and lichen, each battlements softened by time itself. Once, this place had thrummed with vibrant life, the clang of armor mingling with the booming voice of its once-commanding King, a figure whispered to have held dominion over more land than one could fathom. Now, the king was dust, his reign swallowed by the creeping hands of time, and Elderon lay as a tomb, guarding mysteries buried deep within its weathered walls.

# Pushing the temperature up, with a short breaker
settings.temperature = 1.5
settings.logit_high_temp = 10.0
settings.logit_temp_threshold = 14.0
settings.confidence_breaker = 3
settings.cb_mid_threshold = 17.0
settings.cb_high_threshold = 0.999

Trial 1.

Once upon a time, the sky hummed with lavender fireflies. Every night, these gentle beings swished between crimson maple blossoms that painted the countryside like watercolor splashes. Their bioluminescence bathed in soft lilac glow the winding roads that snaked through rolling meadows filled with fragrant thyme. These roads were patrolled not by soldiers but by Willowwisps, spectral entities made of smoke and stars.

Trial 2.

Once upon a time, in a world veiled by the fragrant mists of endless pineforests, lived an imp named Thistleknock, smaller than your thumb and pricklier than the thorns adorning ancient wisterias. While other impish sprites relished pranks that flitted harmlessly between mortals - loosening latches, tickling sleeping cats, rustling leaves to sound like warnings in the dark – Thistleknock had dreams grander than such frippery.

Trial 3.

Once upon a time, there existed, amidst a tangle of rolling emerald hills and bubbling emerald waterfalls, a bustling forest filled with a curious assortment of inhabitants. From gossipy sparrows with nests made under flamboyant wisteria to badger burrows with stories whispering deeper than tree roots, everyone carried the warmth of kinship woven into their beings, every rustling leaf and whispering breeze a secret language among them. They shared a collective belief in an unseen, ethereal source called "Whisperwind", an entity believed to nurture and weave their fate. Whisperwind, they said, carried the hopes and sorrows of all, dancing between sunbeams and stars, weaving stories into the tapestries of the woods forever changing with each turning moon.

Trial 4.

Once upon a time, there was a hidden meadow deep within Whisperwoods. Shimmering wildflowers stretched out as if spun by an artist's whimsical heart, each one a canvas painted with iridescent hues. There, lived Ellia Moonstone, a nymph who spun starlight into threads of gossamer, weaving tapestries more intricate than even a sunset in its splendor. But despite the magical wonders she crafted and her whimsical surroundings, Ellia yearned for a melody more complex than the soft rustlings of leaves carried atop a whispering wind, for her world seemed trapped in a tranquil stillness that had long turned bittersweet.

Trial 5.

Once upon a time, tucked between fields of sapphire blueberries and emerald fir forests, sat a gingerbread cottage that drippled frosting from its windows and whispered sugarplum dreams with every exhale from its frosted smoke stack. Not your typical Grimm Fairy Tale residence, this charming shack boasted bright-fringed, cotton candy gumdrop gabled roofs that beckoned visitors to clamber upon them and tickle their tips. Inside thrived a witch named Flitzy who, contrary to her fiery, fuchsia hued locks, harboured the most compassionate heart this side of Everlasting Gummie Stream, and whose potions did wondrously peculiar things: they wouldn't give you youth everlasting like most witches’brews, but would instead sprout butterflies from your socks on moonlit nights, or make you laugh so hard, tears tasted like champagne bubbles – experiences that were equally, if differently, wondrously unique!