Fix mixture endpoint

[dbobrenko]

Changes

• Fix mixture endpoint to take into account full completions.

Example from live test

Query: Say 10 random numbers between 1 and 100 with mixture=True:

It seems like all the models have generated the same list of 10 random numbers between 1 and 100. Here is the synthesized response:

Here are 10 random numbers between 1 and 100:

1. 43
2. 91
3. 14
4. 67
5. 85
6. 28
7. 52
8. 19
9. 73
10. 41

[Hollyqui]

Have you tested whether the response is still instance. Use a query like e.g.

Write a short story about an elephant shrew with streaming=True. We would expect that streaming only starts after ~10+ seconds seeing that the miners need to generate the initial stories. Is that confirmed to be the case?

[dbobrenko]

Have you tested whether the response is still instance. Use a query like e.g.

Write a short story about an elephant shrew with streaming=True. We would expect that streaming only starts after ~10+ seconds seeing that the miners need to generate the initial stories. Is that confirmed to be the case?

Mixture is 2x slower. In the PR description response it's obvious it uses the context of previous responses.

[dbobrenko]

Write a short story about an elephant shrew with streaming=True. We would expect that streaming only starts after ~10+ seconds seeing that the miners need to generate the initial stories. Is that confirmed to be the case?

mixture 46s

It appears that the provided responses are identical, and they all describe the same story about a tiny creature named Zuzu, an elephant shrew, who lives in the savannah and helps a group of young gazelles trapped in a thorny thicket.

To synthesize this information into a single, high-quality, and concise response, I will provide a rewritten version of the story, focusing on the essential elements and avoiding repetition:

In the scorching savannah, a tiny creature named Zuzu, an elephant shrew, lived a life full of adventure. With her trunk-like snout and agile body, she scavenged for insects, seeds, and fruits, using her keen senses to evade predators. Despite her small size, Zuzu was a force to be reckoned with, possessing a quick wit, fearlessness, and a mischievous glint in her eye.

One sunny afternoon, while out foraging, Zuzu stumbled upon a group of young gazelles trapped in a thorny thicket. The thorns were too dense for the gazelles to escape, and they bleated pitifully, their large eyes filled with worry. Without hesitation, Zuzu sprang into action, using her sharp claws and agile body to scurry up the th

23s

**The Tiny Titan of the Savannah**

In the scorching savannah, where acacia trees stretched towards the sky and grasses swayed in the breeze, a tiny creature scurried about, often overlooked by the larger inhabitants of the land. Her name was Zuzu, and she was an elephant shrew, a mouse-sized marvel with a trunk-like snout and a penchant for adventure.

Zuzu lived in a cozy burrow beneath the roots of a ancient baobab tree, surrounded by a network of tunnels and hiding spots. She spent her days scavenging for insects, seeds, and fruits, using her keen senses and agility to evade predators. Despite her small size, Zuzu was a force to be reckoned with. She was quick-witted, fearless, and possessed a mischievous glint in her eye.

One sunny afternoon, as Zuzu was out foraging, she stumbled upon a group of young gazelles, trapped in a thorny thicket. The thorns were too dense for the gazelles to escape, and they bleated pitifully, their large eyes filled with worry. Without hesitation, Zuzu sprang into action. She scurried up the thorny stems, using her sharp claws and agile body