docs/guides/modules.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "import sys; sys.path.append('/future/u/okhattab/repos/public/tmp/dspy')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"../../docs/images/DSPy8.png\" alt=\"DSPy7 Image\" height=\"150\"/>\n",
    "\n",
    "## Guide: **DSPy Modules**\n",
    "\n",
    "[<img align=\"center\" src=\"https://colab.research.google.com/assets/colab-badge.svg\" />](https://colab.research.google.com/github/stanfordnlp/dspy/blob/main/docs/guides/signatures.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Quick Recap\n",
    "\n",
    "This guide assumes you followed the [intro tutorial]() to build your first few DSPy programs.\n",
    "\n",
    "Remember that **DSPy program** is just Python code that calls one or more **DSPy modules**, like `dspy.Predict` or `dspy.ChainOfThought`, to use LMs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1) What is a DSPy Module?\n",
    "\n",
    "A **DSPy module** is a building block for programs that use LMs.\n",
    "\n",
    "- Each built-in module abstracts a **prompting technique** (like chain of thought or ReAct). Crucially, they are generalized to handle any [DSPy Signature]().\n",
    "\n",
    "- A DSPy module has **learnable parameters** (i.e., the little pieces comprising the prompt and the LM weights) and can be invoked (called) to process inputs and return outputs.\n",
    "\n",
    "- Multiple modules can be composed into bigger modules (programs). DSPy modules are inspired directly by NN modules in PyTorch, but applied to LM programs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2) Why should I use a DSPy Module?\n",
    "\n",
    "TODO. I typically take this as self-evident, but I'll spell it out here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Install `dspy-ai` if needed. Then set up a default language model.\n",
    "# TODO: Add a graceful line for OPENAI_API_KEY.\n",
    "\n",
    "try: import dspy\n",
    "except ImportError:\n",
    "    %pip install dspy-ai\n",
    "    import dspy\n",
    "\n",
    "dspy.configure(lm=dspy.OpenAI(model='gpt-3.5-turbo-1106'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3) What DSPy Modules are currently built-in?\n",
    "\n",
    "1. **`dspy.Predict`**:\n",
    "\n",
    "2. **`dspy.ChainOfThought`**: \n",
    "\n",
    "3. **`dspy.ProgramOfThought`**:\n",
    "\n",
    "4. **`dspy.ReAct`**:\n",
    "\n",
    "5. **`dspy.MultiChainComparison`**:\n",
    "\n",
    "\n",
    "We also have some function-style modules:\n",
    "\n",
    "6. **`dspy.majority`**:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4) How do I use a built-in module, like `dspy.Predict` or `dspy.ChainOfThought`?\n",
    "\n",
    "Let's start with the most fundamental one, `dspy.Predict`. Internally, all of the others are just built using it!\n",
    "\n",
    "We'll assume you are already at least a little familiar with [DSPy signatures](), which are declarative specs for defining the behavior of any module we use in DSPy.\n",
    "\n",
    "To use a module, we first **declare** it by giving it a signature. Then we **call** the module with the input arguments, and extract the output fields!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Positive\n"
     ]
    }
   ],
   "source": [
    "sentence = \"it's a charming and often affecting journey.\"  # example from the SST-2 dataset.\n",
    "\n",
    "# 1) Declare with a signature.\n",
    "classify = dspy.Predict('sentence -> sentiment')\n",
    "\n",
    "# 2) Call with input argument(s). \n",
    "response = classify(sentence=sentence)\n",
    "\n",
    "# 3) Access the output.\n",
    "print(response.sentiment)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we declare a module, we can pass configuration keys to it.\n",
    "\n",
    "Below, we'll pass `n=5` to request five completions. We can also pass `temperature` or `max_len`, etc.\n",
    "\n",
    "Let's use `dspy.ChainOfThought`. In many cases, simply swapping `dspy.ChainOfThought` in place of `dspy.Predict` improves quality."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['One great thing about the ColBERT retrieval model is its superior efficiency and effectiveness compared to other models.',\n",
       " 'Its ability to efficiently retrieve relevant information from large document collections.',\n",
       " 'One great thing about the ColBERT retrieval model is its superior performance compared to other models and its efficient use of pre-trained language models.',\n",
       " 'One great thing about the ColBERT retrieval model is its superior efficiency and accuracy compared to other models.',\n",
       " 'One great thing about the ColBERT retrieval model is its ability to incorporate user feedback and support complex queries.']"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "question = \"What's something great about the ColBERT retrieval model?\"\n",
    "\n",
    "# 1) Declare with a signature, and pass some config.\n",
    "classify = dspy.ChainOfThought('question -> answer', n=5)\n",
    "\n",
    "# 2) Call with input argument.\n",
    "response = classify(question=question)\n",
    "\n",
    "# 3) Access the outputs.\n",
    "response.completions.answer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's dicuss the output object here.\n",
    "\n",
    "The `dspy.ChainOfThought` module will generally inject a `rationale` before the output field(s) of your signature.\n",
    "\n",
    "Let's inspect the (first) rationale and answer!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Rationale: produce the answer. We can consider the fact that ColBERT has shown to outperform other state-of-the-art retrieval models in terms of efficiency and effectiveness. It uses contextualized embeddings and performs document retrieval in a way that is both accurate and scalable.\n",
      "Answer: One great thing about the ColBERT retrieval model is its superior efficiency and effectiveness compared to other models.\n"
     ]
    }
   ],
   "source": [
    "print(f\"Rationale: {response.rationale}\")\n",
    "print(f\"Answer: {response.answer}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is accessible whether we request one or many completions.\n",
    "\n",
    "We can also access the different completions as a list of `Prediction`s or as several lists, one for each field."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response.completions[3].rationale == response.completions.rationale[3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5) How do I use more complex built-in modules?\n",
    "\n",
    "The others are very similar, `dspy.ReAct` and `dspy.ProgramOfThough` etc. They mainly change the internal behavior with which your signature is implemented!\n",
    "\n",
    "More example soon!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6) How do I compose multiple modules into a bigger program?\n",
    "\n",
    "DSPy is just Python code that uses modules in any control flow you like. (There's some magic internally at `compile` time to trace your LM calls.)\n",
    "\n",
    "What this means is that, you can just call the modules freely. No weird abstractions for chaining calls.\n",
    "\n",
    "This is basically PyTorch's design approach for define-by-run / dynamic computation graphs. Refer to the intro tutorials for examples."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "py39_nov2023",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}