Skip to content

Releases: argilla-io/distilabel

1.4.1

16 Oct 07:30
@gabrielmbmb gabrielmbmb
1.4.1
844165f
This commit was signed with the committer’s verified signature.
gabrielmbmb Gabriel Martín Blázquez
GPG key ID: F13D58A2031AB1CA
Learn about vigilant mode.
Compare
Choose a tag to compare
1.4.1 Latest
Latest

What's Changed

  • Fix not handling list of all primitive types in SignatureMixin by @gabrielmbmb in #1037

Full Changelog: 1.4.0...1.4.1

Contributors

gabrielmbmb
Assets 2
Loading

1.4.0

08 Oct 14:53
@gabrielmbmb gabrielmbmb
1.4.0
c0d798a
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
1.4.0

✨ Release highlights

Offline Batch Generation and OpenAI Batch API

We’ve updated the LLM interface so now LLMs using an external platform that offers a batch service can be integrated in distilabel. In addition, OpenAILLM has been updated so it can use the OpenAI Batch API to get 50% cost reductions.

distilabel-offline-batch-generation.mp4

Improved cache for maximum outputs reusability

We all know that running LLM is costly and most of the times we want to reuse as much as we can the outputs generated with them. Before this release, distilabel cache mechanism enabled to recover a pipeline execution that was stopped before finishing and to re-create the Distiset generated by one that finished its execution and was re-executed.

In this release, we've greatly improved the cache so the outputs of all the Steps are cached and therefore can be reused in other pipelines executions even if the pipeline has changed:

image

In addition, we've added a use_cache attribute in the Steps that allows toggling the use of the cache at step level.

Steps can generated artifacts

In some cases, Step produces some additional artifacts that are used to generate its outputs. These artifacts can take some time to be generated and they could be reused in the future. That’s why we’ve added a new method called Step.save_artifact that can be called within the step to store artifacts generated by it. The artifacts generated by the Step will also get uploaded to the Hugging Face Hub.

from typing import List, TYPE_CHECKING
from distilabel.steps import GlobalStep, StepInput, StepOutput
import matplotlib.pyplot as plt

if TYPE_CHECKING:
    from distilabel.steps import StepOutput


class CountTextCharacters(GlobalStep):
    @property
    def inputs(self) -> List[str]:
        return ["text"]

    @property
    def outputs(self) -> List[str]:
        return ["text_character_count"]

    def process(self, inputs: StepInput) -> "StepOutput":  # type: ignore
        character_counts = []

        for input in inputs:
            text_character_count = len(input["text"])
            input["text_character_count"] = text_character_count
            character_counts.append(text_character_count)

        # Generate plot with the distribution of text character counts
        plt.figure(figsize=(10, 6))
        plt.hist(character_counts, bins=30, edgecolor="black")
        plt.title("Distribution of Text Character Counts")
        plt.xlabel("Character Count")
        plt.ylabel("Frequency")

        # Save the plot as an artifact of the step
        self.save_artifact(
            name="text_character_count_distribution",
            write_function=lambda path: plt.savefig(path / "figure.png"),
            metadata={"type": "image", "library": "matplotlib"},
        )

        plt.close()

        yield inputs

New Tasks: CLAIR, APIGEN and many more!

  • New CLAIR task: CLAIR uses an AI system to minimally revise a solution A→A´ such that the resulting preference A preferred A’ is much more contrastive and precise.
  • New tasks to replicate APIGen framework: APIGenGenerator, APIGenSemanticChecker, APIGenExecutionChecker. These tasks allow generating datasets like the one presented in the paper: APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets
  • New URIAL task that allows using non-instruct models to generate a response for an instruction.
  • New TextClassification task to make zero-shot text classification based on a predefined but highly customizable prompt.
  • TextClustering, to generate clusters from text and group your generations, discovering labels from your data. Comes with 2 steps to run UMAP and DBSCAN algorithms.
  • Updated TextGeneration to simplify customization of tasks that don’t require further post-processing.

New Steps to sample data in your pipelines and remove duplicates

  • New DataSampler step to sample data from other datasets, which can be useful to inject different examples for few-shot examples in your prompts.
  • New EmbeddingDedup step to remove duplicates based on embeddings and a distance metric.
  • New MinHashDedup step to remove near duplicates from the text based on MinHash and MinHashLSH algorithm.
  • New TruncateTextColumns to truncate the length of your texts using either the character length or the number of tokens based on a tokenizer.
  • New CombineOutputs to combine the outputs of two or more steps into a single output.

Generate text embeddings using vLLM

Extra things

What's Changed

Read more

Contributors

dameikle, davidberenstein1957, and 4 other contributors
Assets 2
Loading

1.3.2

23 Aug 13:15
@gabrielmbmb gabrielmbmb
1.3.2
ed88585
This commit was signed with the committer’s verified signature.
gabrielmbmb Gabriel Martín Blázquez
GPG key ID: F13D58A2031AB1CA
Learn about vigilant mode.
Compare
Choose a tag to compare
1.3.2

What's Changed

Full Changelog: 1.3.1...1.3.2

Contributors

gabrielmbmb and plaguss
Assets 2
Loading

1.3.1

07 Aug 09:09
@gabrielmbmb gabrielmbmb
1.3.1
268358b
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
1.3.1

What's Changed

  • Create new distilabel.constants module to store constants and avoid circular imports by @plaguss in #861
  • Add OpenAI request timeout by @ashim-mahara in #858

New Contributors

Full Changelog: 1.3.0...1.3.1

Contributors

ashim-mahara and plaguss
Assets 2
Loading

1.3.0

06 Aug 14:16
@gabrielmbmb gabrielmbmb
1.3.0
63f948b
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
1.3.0

What's Changed

New Contributors

Full Changelog: 1.2.4...1.3.0

Contributors

davidberenstein1957, gabrielmbmb, and 3 other contributors
Assets 2
Loading

1.2.4

23 Jul 16:03
@gabrielmbmb gabrielmbmb
1.2.4
add2b6e
This commit was signed with the committer’s verified signature.
gabrielmbmb Gabriel Martín Blázquez
GPG key ID: F13D58A2031AB1CA
Learn about vigilant mode.
Compare
Choose a tag to compare
1.2.4

What's Changed

  • Update InferenceEndpointsLLM to use chat_completion method by @gabrielmbmb in #815

Full Changelog: 1.2.3...1.2.4

Contributors

gabrielmbmb
Assets 2
Loading

1.2.3

23 Jul 08:02
@gabrielmbmb gabrielmbmb
1.2.3
54ecc38
This commit was signed with the committer’s verified signature.
gabrielmbmb Gabriel Martín Blázquez
GPG key ID: F13D58A2031AB1CA
Learn about vigilant mode.
Compare
Choose a tag to compare
1.2.3

What's Changed

New Contributors

Full Changelog: 1.2.2...1.2.3

Contributors

dvsrepo, Hassaan-Qaisar, and sdiazlor
Assets 2
Loading

1.2.2

12 Jul 11:09
@gabrielmbmb gabrielmbmb
1.2.2
a22c7e2
This commit was signed with the committer’s verified signature.
gabrielmbmb Gabriel Martín Blázquez
GPG key ID: F13D58A2031AB1CA
Learn about vigilant mode.
Compare
Choose a tag to compare
1.2.2

What's Changed

Full Changelog: 1.2.1...1.2.2

Contributors

gabrielmbmb
Assets 2
Loading

1.2.1

01 Jul 08:58
@gabrielmbmb gabrielmbmb
1.2.1
fe615d6
This commit was signed with the committer’s verified signature.
gabrielmbmb Gabriel Martín Blázquez
GPG key ID: F13D58A2031AB1CA
Learn about vigilant mode.
Compare
Choose a tag to compare
1.2.1

What's Changed

New Contributors

Full Changelog: 1.2.0...1.2.1

Contributors

fpreiss, gabrielmbmb, and sdiazlor
Assets 2
Loading

1.2.0

18 Jun 12:40
@gabrielmbmb gabrielmbmb
1.2.0
3910aca
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
1.2.0

✨ Release highlights

Structured generation with instructor, InferenceEndpointsLLM now supports structured generation and StructuredGeneration task

  • instructor has been integrated bringing support for structured generation with OpenAILLM, AnthropicLLM, LiteLLM, MistralLLM, CohereLLM and GroqLLM:

    Structured generation with `instructor` example 
    from typing import List

from distilabel.llms import MistralLLM
from distilabel.pipeline import Pipeline
from distilabel.steps import LoadDataFromDicts
from distilabel.steps.tasks import TextGeneration
from pydantic import BaseModel, Field


class Node(BaseModel):
    id: int
    label: str
    color: str


class Edge(BaseModel):
    source: int
    target: int
    label: str
    color: str = "black"


class KnowledgeGraph(BaseModel):
    nodes: List[Node] = Field(..., default_factory=list)
    edges: List[Edge] = Field(..., default_factory=list)


with Pipeline(
    name="Knowledge-Graphs",
    description=(
        "Generate knowledge graphs to answer questions, this type of dataset can be used to "
        "steer a model to answer questions with a knowledge graph."
    ),
) as pipeline:
    sample_questions = [
        "Teach me about quantum mechanics",
        "Who is who in The Simpsons family?",
        "Tell me about the evolution of programming languages",
    ]

    load_dataset = LoadDataFromDicts(
        name="load_instructions",
        data=[
            {
                "system_prompt": "You are a knowledge graph expert generator. Help me understand by describing everything as a detailed knowledge graph.",
                "instruction": f"{question}",
            }
            for question in sample_questions
        ],
    )

    text_generation = TextGeneration(
        name="knowledge_graph_generation",
        llm=MistralLLM(
            model="open-mixtral-8x22b",
            structured_output={"schema": KnowledgeGraph}
        ),
    )
    load_dataset >> text_generation

  • InferenceEndpointsLLM now supports structured generation

  • New StructuredGeneration task that allows defining the schema of the structured generation per input row.

New tasks for generating datasets for training embedding models

sentence-transformers v3 was recently released and we couldn't resist the urge of adding a few new tasks to allow creating datasets for training embedding models!

New Steps for loading data from different sources and saving/loading Distiset to disk

We've added a few new steps allowing to load data from different sources:

  • LoadDataFromDisk allows loading a Distisetor datasets.Dataset that was previously saved using the save_to_disk method.
  • LoadDataFromFileSystem allows loading a datasets.Dataset from a file system.

Thanks to @rasdani for helping us testing this new tasks!

In addition, we have added save_to_disk method to Distiset akin to datasets.Dataset.save_to_disk, that allows saving the generated distiset to disk, along with the pipeline.yaml and pipeline.log.

`save_to_disk` example 
from distilabel.pipeline import Pipeline

with Pipeline(name="my-pipeline") as pipeline:
    ...
    
if __name__ == "__main__":
    distiset = pipeline.run(...)
    distiset.save_to_disk(dataset_path="my-distiset")

MixtureOfAgentsLLM implementation

We've added a new LLM called MixtureOfAgentsLLM derived from the paper Mixture-of-Agents Enhances Large Language Model Capabilities. This new LLM allows generating improved outputs thanks to the collective expertise of several LLMs.

`MixtureOfAgentsLLM` example 
from distilabel.llms import MixtureOfAgentsLLM, InferenceEndpointsLLM

llm = MixtureOfAgentsLLM(
    aggregator_llm=InferenceEndpointsLLM(
        model_id="meta-llama/Meta-Llama-3-70B-Instruct",
        tokenizer_id="meta-llama/Meta-Llama-3-70B-Instruct",
    ),
    proposers_llms=[
        InferenceEndpointsLLM(
            model_id="meta-llama/Meta-Llama-3-70B-Instruct",
            tokenizer_id="meta-llama/Meta-Llama-3-70B-Instruct",
        ),
        InferenceEndpointsLLM(
            model_id="NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO",
            tokenizer_id="NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO",
        ),
        InferenceEndpointsLLM(
            model_id="HuggingFaceH4/zephyr-orpo-141b-A35b-v0.1",
            tokenizer_id="HuggingFaceH4/zephyr-orpo-141b-A35b-v0.1",
        ),
    ],
    rounds=2,
)

llm.load()

output = llm.generate(
    inputs=[
        [
            {
                "role": "user",
                "content": "My favorite witty review of The Rings of Power series is this: Input:",
            }
        ]
    ]
)

Saving cache and passing batches to GlobalSteps optimizations

  • The cache logic of the _BatchManager has been improved to incrementally update the cache making the process much faster.
  • The data of the input batches of the GlobalSteps will be passed to the step using the file system, as this is faster than passing it using the queue. This is possible thanks to new integration of fsspec, which can be configured to use a file system or cloud storage as backend for passing the data of the batches.

BasePipeline and _BatchManager refactor

The logic around BasePipeline and _BatchManager has been refactored, which will make it easier to implement new pipelines in the future.

Added ArenaHard as an example of how to use distilabel to implement a benchmark

distilabel can be easily used to create an LLM benchmark. To showcase this, we decided to implement Arena Hard as an example: Benchmarking with distilabel: Arena Hard

📚 Improved documentation structure

We have updated the documentation structure to make it more clear and self-explanatory, as well as more visually appealing 😏.

image

What's Changed

Read more

Contributors

leiyre, davidberenstein1957, and 4 other contributors
Assets 2
Loading