07_Chapter 7. Reasoning and Chain-of-Thought Prompting.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# **Chapter 7. Reasoning and Chain-of-Thought Prompting**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this chapter, we’ll explore **Chain-of-Thought (CoT) prompting**, where the main principle is to **“think first, answer later.”** This approach encourages Solar to break down problems logically and solve them step by step, rather than jumping directly to a conclusion. CoT is especially effective for multi-step math problems and complex reasoning tasks, as it guides the model to consider each part of the problem individually, ultimately building up to a coherent solution."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **Table of Contents**\n",
    "- Use `Ctrl + F` (Windows) or `Cmd + F` (Mac) to locate specific sections by title."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- **7.1 Chain-of-Thought (CoT) Prompting**\n",
    "\n",
    "\n",
    "- **7.2 Math Problem Sets - Chain of Thought**\n",
    "\n",
    "\n",
    "- **7.3 Zero-Shot Chain-of-Thought Prompting**\n",
    "\n",
    "\n",
    "- **7.4 Math Problem Sets - Zero-Shot Chain of Thought**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Set up**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Success!\n"
     ]
    }
   ],
   "source": [
    "from openai import OpenAI\n",
    "\n",
    "# Retrieve the UPSTAGE_API_KEY variable from the IPython store\n",
    "%store -r UPSTAGE_API_KEY\n",
    "\n",
    "try:\n",
    "    if UPSTAGE_API_KEY:\n",
    "        print(\"Success!\")\n",
    "except NameError as ne:\n",
    "    print(f\"Since, {ne}\")\n",
    "    print(\"Please, insert your API key.\")\n",
    "    UPSTAGE_API_KEY = input(\"UPSTAGE_API_KEY =\")\n",
    "\n",
    "# Set your API key: \n",
    "# UPSTAGE_API_KEY = \" \" ←- Insert your API key here. \n",
    "\n",
    "client = OpenAI(\n",
    "    api_key= UPSTAGE_API_KEY,\n",
    "    base_url=\"https://api.upstage.ai/v1/solar\"\n",
    ")\n",
    "\n",
    "config_model = {\n",
    "    \"model\": \"solar-pro\",\n",
    "    \"max_tokens\": 1997,\n",
    "    \"temperature\": 0.0,\n",
    "    \"top_p\": 0.9,\n",
    "}\n",
    "\n",
    "def get_completion(messages, system_prompt=\"\", config=config_model):\n",
    "    try:\n",
    "        if system_prompt:\n",
    "            messages = [{\"role\": \"system\", \"content\": system_prompt}] + messages\n",
    "\n",
    "        message = client.chat.completions.create(messages=messages, **config)\n",
    "        return message.choices[0].message.content\n",
    "    \n",
    "    except Exception as e:\n",
    "        print(f\"Error during API call: {e}\")\n",
    "        return None"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **7.1 Chain-of-Thought (CoT) Prompting**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Many studies have demonstrated the effectiveness of using CoT in Large Language Models. A representative study, titled \"*Chain-of-Thought Prompting Elicits Reasoning in Large Language Models*\" by [Jason Wei and colleagues (2022)](https://arxiv.org/pdf/2201.11903), offers valuable insights. This paper explores how generating a chain of thought—a sequence of intermediate reasoning steps—significantly enhances the ability of large language models to perform complex reasoning tasks."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Reference**\n",
    "\n",
    "Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., ... & Zhou, D. (2022). Chain-of-thought prompting elicits reasoning in large language models. *Advances in neural information processing systems, 35,* 24824-24837."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Key Features of CoT Prompt Engineering are as follows: \n",
    "\n",
    "- **Step-by-Step Reasoning**: CoT prompts encourage models to articulate their reasoning process, improving their accuracy and clarity when solving complex problems.  \n",
    "\n",
    "- **Enhanced Transparency**: Since the model outputs intermediate steps, users can trace the reasoning behind the solution, making it easier to debug or understand the model’s behavior."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Solar excels at solving complex mathematical problems using the CoT approach. In this chapter, we will demonstrate how to solve problems related to **prime numbers** and **probability** using CoT. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **7.2 Math Problem Sets - Chain of Thought**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here are some examples of math problem sets where CoT can be effectively applied:\n",
    "  \n",
    "1. Prime Numbers  \n",
    "\n",
    "2. Probability of A Dice  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Problem #1: Prime Numbers**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Find a counterexample with a proof to show that the following statement is false: 'Every positive integer can be expressed as p + a^2, where p is either a prime number or 1, and a >= 0'.\n",
    ">\n",
    "> **Answer**: $n = 25$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- [**The correct answer**] CoT Prompt: $n = 25$ "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Let's try n=25.\n",
      "\n",
      "For n=25, we check the possible combinations of p+a^2=25:\n",
      "p=1 and a^2=24: Not possible because 24 is not a perfect square.\n",
      "p=2 and a^2=23: Not possible because 23 is not a perfect square.\n",
      "p=3 and a^2=22: Not possible because 22 is not a perfect square.\n",
      "p=5 and a^2=20: Not possible because 20 is not a perfect square.\n",
      "p=7 and a^2=18: Not possible because 18 is not a perfect square.\n",
      "p=11 and a^2=14: Not possible because 14 is not a perfect square.\n",
      "p=13 and a^2=12: Not possible because 12 is not a perfect square.\n",
      "p=17 and a^2=8: Not possible because 8 is not a perfect square.\n",
      "p=19 and a^2=6: Not possible because 6 is not a perfect square.\n",
      "p=23 and a^2=2: Not possible because 2 is not a perfect square.\n",
      "\n",
      "Since none of the combinations of p and a^2 satisfy n=25, we have found a counterexample. Therefore, the statement 'Every positive integer can be expressed as p + a^2, where p is either a prime number or 1, and a >= 0' is false. \n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "message = [\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": \"\"\"Find a counterexample with a proof to show that the following statement is false: 'Every positive integer can be expressed as p + a^2, where p is either a prime number or 1, and a >= 0'.\n",
    "#Instruction\n",
    "1. Let n >= 10, since the combinations of p and a^2 satisfy n = p + a^2 for all n < 10. As n increases, the gaps between a^2 values where a^ 2 < n widen, and the density of p < n decreases.\n",
    "2. Consider n as the square number first, such as 9, 16, 25, and so on to quickly find a counterexample.\n",
    "\n",
    "##Caution\n",
    "- If you find any of possible combination of p and a^2 to express n is satisfied, then it is not a counterexample.\n",
    "- p SHOULD NOT BE NON-PRIME NUMBER without 1. p can be 1, 2, 3, 5, 7, 11, and so on.\n",
    "- a SHOULD BE A SQUAR NUMBER. a can be 0, 1, 4, 9, 16, 25, and so on.\n",
    "\n",
    "###Trying\n",
    "n=16\n",
    "For n=16, we check the possible combinations of p+a^2=16:\n",
    "p=1 and a^2=15: Not possible because 15 is not a perfect square.\n",
    "p=2 and a^2=14: Not possible because 14 is not a perfect square.\n",
    "p=3 and a^2=13: Not possible because 13 is not a perfect square.\n",
    "p=5 and a^2=11: Not possible because 11 is not a perfect square.\n",
    "p=7 and a^2=9: Possible, since a=3.\n",
    "\n",
    "Since n=16 can be written as 16=7+9, it is not a counterexample.\n",
    "\"\"\"\n",
    "    }\n",
    "]\n",
    "\n",
    "response = get_completion(messages=message)\n",
    "print(response, \"\\n\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- [**The incorrect answer**] Zero-Shot Prompt: $n = 15$\n",
    "\n",
    "    > \"Consider the positive integer n = 15. We want to find integers p and a such that n = p + a^2, where p is either a prime number or 1, and a >= 0.\n",
    "    > \n",
    "    > First, let's try to find a prime number p. The prime numbers less than 15 are 2, 3, 5, 7, 11, and 13. However, none of these primes, when subtracted from 15, result in a perfect square:\n",
    "    > \n",
    "    > - 15 - 2 = 13 (not a perfect square)\n",
    "    > - 15 - 3 = 12 (not a perfect square)\n",
    "    > - 15 - 5 = 10 (not a perfect square)\n",
    "    > - 15 - 7 = 8 (not a perfect square)\n",
    "    > - 15 - 11 = 4 (a perfect square, but 11 is not a prime number less than 15)\n",
    "    > - 15 - 13 = 2 (not a perfect square)\n",
    "    >\n",
    "    > Next, let's try p = 1. In this case, we have n = 1 + a^2, which implies a^2 = 14. However, there is no integer a such that a^2 = 14, as the square of any integer is either a multiple of 4 or one more than a multiple of 4.\n",
    "    > \n",
    "    > Therefore, we have shown that the positive integer 15 cannot be expressed as p + a^2, where p is either a prime number or 1, and a >= 0. This counterexample proves that the given statement is false.\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Problem #2: Probability of A Dice**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Suppose we have a dice that the probability of odd numbers is $\\frac{5}{2}$ times the probability of even numbers.  \n",
    "> Find the probability that dice shows 6.\n",
    "> \n",
    "> **Answer**: The probability is $\\frac{2}{21}$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- [**The correct answer**] CoT Prompt:  $\\frac{2}{21}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Let p(odd): p(1) = p(3) = p(5), p(even): p(2) = p(4) = p(6),\n",
      "-> 3p(odd) + 3p(even) = 1,\n",
      "-> 3*(5/2p(even)) + 3p(even) (because: p(odd) = 5/2p(even)),\n",
      "-> 15/2p(even) + 3p(even) = 1 (because: 3*5/2 = 15/2),\n",
      "-> 21/2p(even) = 1 (because: 3 = 6/2),\n",
      "-> p(even) = 2/21,\n",
      "-> p(even) = p(6) = 2/21,\n",
      "\n",
      "Therefore, p(6) = 2/21. \n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "message = [\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": \"\"\"\n",
    "        Suppose we have a dice that the probability of odd numbers is 4/3 times the probability of even numbers.\n",
    "        Find the probability that dice shows 6.\n",
    "        \n",
    "        A: Let p(odd): p(1) = p(3) = p(5), p(even): p(2) = p(4) = p(6),\n",
    "        -> 3p(odd) + 3p(even) = 1,\n",
    "        -> 3*(4/3p(even)) + 3p(even) (because: p(odd) = 4/3p(even), 3*4/3 = 12/3 = 4),\n",
    "        -> 4p(even) + 3p(even) = 1,\n",
    "        -> 7p(even) = 1,\n",
    "        -> p(even) = 1/7,\n",
    "        -> p(even) = p(6) = 1/7,\n",
    "\n",
    "        Therefore, p(6) = 1/7.\n",
    "        \n",
    "        Suppose we have a dice that the probability of odd numbers is 5/6 times the probability of even numbers.\n",
    "        Find the probability that dice shows 6.\n",
    "        A: Let p(odd): p(1) = p(3) = p(5), p(even): p(2) = p(4) = p(6),\n",
    "        -> 3p(odd) + 3p(even) = 1,\n",
    "        -> 3*(5/6p(even)) + 3p(even) (because: p(odd) = 5/6p(even)),\n",
    "        -> 5/2p(even) + 3p(even) = 1 (because: 3*5/6 = 15/6 = 5/2),\n",
    "        -> 11/2p(even) = 1 (because: 3 = 6/2),\n",
    "        -> p(even) = 2/11,\n",
    "        -> p(even) = p(6) = 2/11,\n",
    "\n",
    "        Therefore, p(6) = 2/11.\n",
    "        \n",
    "        Suppose we have a dice that the probability of odd numbers is 7/4 times the probability of even numbers.\n",
    "        Find the probability that dice shows 6.\n",
    "        A: Let p(odd): p(1) = p(3) = p(5), p(even): p(2) = p(4) = p(6),\n",
    "        -> 3p(odd) + 3p(even) = 1,\n",
    "        -> 3*(7/4p(even)) + 3p(even) (because: p(odd) = 7/4p(even)),\n",
    "        -> 21/4p(even) + 3p(even) = 1 (because: 3*7/4 = 21/4),\n",
    "        -> 33/4p(even) = 1 (because: 3 = 12/4),\n",
    "        -> p(even) = 4/33,\n",
    "        -> p(even) = p(6) = 4/33,\n",
    "\n",
    "        Therefore, p(6) = 4/33.\n",
    "\n",
    "        -----\n",
    "        Suppose we have a dice that the probability of odd numbers is 5/2 times the probability of even numbers.\n",
    "        Find the probability that dice shows 6.\n",
    "        A: \n",
    "        \"\"\"\n",
    "    }\n",
    "]\n",
    "\n",
    "response = get_completion(messages=message)\n",
    "print(response, \"\\n\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- [**The incorrect answer**] Zero-Shot Prompt: $\\frac{2}{7}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> \"Let's denote the probability of rolling an even number as P(even) and the probability of rolling an odd number as P(odd). According to the problem, P(odd) = 5/2 * P(even).\n",
    "> \n",
    "> Since there are only even and odd numbers on a dice, the sum of their probabilities should be equal to 1. So, we can write:\n",
    "> \n",
    "> P(even) + P(odd) = 1\n",
    "> \n",
    "> Substituting the given relationship between P(odd) and P(even), we get:\n",
    "> \n",
    "> P(even) + 5/2 * P(even) = 1\n",
    "> \n",
    "> Combining the terms, we have:\n",
    "> \n",
    "> 7/2 * P(even) = 1\n",
    "> \n",
    "> Now, solving for P(even), we get:\n",
    "> \n",
    "> P(even) = 2/7\n",
    "> \n",
    "> Since there are three even numbers (2, 4, and 6) on a dice, the probability of rolling any even number is 3 * P(even). Therefore, the probability of rolling a 6 is:\n",
    "> \n",
    "> P(6) = 3 * P(even) = 3 * 2/7 = 6/7\n",
    "> \n",
    "> However, this result is not possible because the probability of an event cannot be greater than 1. It seems there was a mistake in the calculation. Let's correct it.\n",
    "> \n",
    "> The probability of rolling a 6 is actually:\n",
    "> \n",
    "> P(6) = P(even) = 2/7\n",
    "> \n",
    "> So, the probability that the dice shows 6 is 2/7.\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **7.3 Zero-Shot Chain-of-Thought Prompting**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Zero-shot chain-of-thought(CoT)** prompt engineering is a method that enhances the reasoning capabilities of a model without providing explicit guidance or answers for problem-solving, as utilized in CoT."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- The paper [*Large Language Models are Zero-Shot Reasoners* (2022)](https://arxiv.org/abs/2205.11916) proposed the phrase \"**Let's think step by step.**\". Zero-shot CoT guides the model to think step by step, aiding in solving complex problems. Using this phrase, we will solve two problems. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Reference**\n",
    "\n",
    "Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022). Large language models are zero-shot reasoners. *Advances in neural information processing systems, 35*, 22199-22213."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **7.4 Math Problem Sets - Zero Shot Chain of Thought**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3. The Differential Equation  \n",
    "\n",
    "4. The Cryptarithms  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From now on, we will demonstrate how to solve invovling **differential equations, and cryptarithms** problems using Zero-Shot CoT."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Problem #3: The Differential Equation**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Solve the following equation: $dx + e^{3x}dy = 0$\n",
    ">\n",
    "> **Answer**: $y = \\frac{1}{3} e^{-3x} + C$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- [**The correct answer**] Zero-Shot CoT Prompt: $y = \\frac{1}{3} e^{-3x} + C$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "To solve the given differential equation, we can use the method of separation of variables. The equation is given as:\n",
      "\n",
      "dx + e^(3x)dy = 0\n",
      "\n",
      "We can rewrite this as:\n",
      "\n",
      "dx = -e^(3x)dy\n",
      "\n",
      "Now, we can separate the variables by dividing both sides by e^(3x) and dx:\n",
      "\n",
      "1/e^(3x) dx = -dy\n",
      "\n",
      "Next, we integrate both sides:\n",
      "\n",
      "∫ 1/e^(3x) dx = -∫ dy\n",
      "\n",
      "The integral on the left side can be solved by using the substitution method. Let u = 3x, then du = 3dx, and dx = du/3. Substituting this into the integral, we get:\n",
      "\n",
      "∫ 1/e^u * (1/3) du = -∫ dy\n",
      "\n",
      "This simplifies to:\n",
      "\n",
      "-(1/3) ∫ e^(-u) du = -∫ dy\n",
      "\n",
      "The integral of e^(-u) with respect to u is -e^(-u), so we have:\n",
      "\n",
      "-(1/3) * -e^(-u) = -y + C\n",
      "\n",
      "Substituting back u = 3x, we get:\n",
      "\n",
      "(1/3) * e^(-3x) = y + C\n",
      "\n",
      "Finally, we can solve for y:\n",
      "\n",
      "y = (1/3) * e^(-3x) + C\n",
      "\n",
      "This is the general solution to the given differential equation. \n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "message = [\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": \"\"\"Solve the differential equation: dx + e^(3x)dy = 0.\n",
    "        \n",
    "        ## Instruction\n",
    "        dx + x dy = 0 -> x dy = - dx -> dy = - 1/x dx.\n",
    "        \n",
    "        Let's think step by step.\"\"\"\n",
    "    }\n",
    "]\n",
    "\n",
    "response = get_completion(messages=message)\n",
    "print(response, \"\\n\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- [**The incorrect answer**] Zero-Shot Prompt: $x + \\frac{1}{3}e^{3x} = C$\n",
    "\n",
    "    > \"The given differential equation is:\n",
    "    > \n",
    "    > dx + e^(3x)dy = 0\n",
    "    > \n",
    "    > This is a first-order linear differential equation. To solve it, we can rearrange the terms to separate the variables:\n",
    "    > \n",
    "    > dx = -e^(3x)dy\n",
    "    >\n",
    "    > Now, we can integrate both sides with respect to their respective variables:\n",
    "    > \n",
    "    > ∫ dx = -∫ e^(3x) dy\n",
    "    > \n",
    "    > x + C1 = - (1/3)e^(3x) + C2\n",
    "    >\n",
    "    > where C1 and C2 are constants of integration. To simplify, we can combine the constants:\n",
    "    >\n",
    "    > x + (1/3)e^(3x) = C\n",
    "    >\n",
    "    > where C = C - C1.\n",
    "    > \n",
    "    > So, the general solution to the given differential equation is:\n",
    "    >\n",
    "    > x + (1/3)e^(3x) = C \"\n",
    "\n",
    "<br>  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Problem #4: The Cryptarithms**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Solve for distinct single-digit natural numbers A, B, and C in the equation: $ABC + BAC = CACA$.  \n",
    "> \n",
    "> **Answer**: $A = 2, B = 9, C = 1$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- [**The correct answer**] Zero-Shot CoT Prompt: $A = 2, B = 9, C = 1$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Let's start with step 1.\n",
      "\n",
      "1. Estimate the range of CACA to satisfy the Constraints.\n",
      "\n",
      "Given that 1000 < CACA < the maximum of ABC + BAC, we can estimate the range of CACA. Since A, B, and C are distinct single-digit natural numbers, the maximum value for ABC + BAC is 999 + 999 = 1998. Therefore, the range of CACA is 1000 < CACA < 1998.\n",
      "\n",
      "Now, let's move on to step 2.\n",
      "\n",
      "2. Identify the first digit, C, of CACA through this range of CACA. It MUST be one number.\n",
      "\n",
      "Since C is a single-digit natural number, the possible values for C are 1, 2, 3, 4, 5, 6, 7, 8, and 9. However, given the range of CACA, the only possible value for C is 1.\n",
      "\n",
      "Now, let's proceed to step 3.\n",
      "\n",
      "3. Calculate units places of LHS using the result of step2. Then, estimate units place of RHS, A.\n",
      "\n",
      "Given that C = 1, the units place of the left-hand side (LHS) of the equation is 2. Therefore, the units place of the right-hand side (RHS) must also be 2. Since A is a single-digit natural number, the possible values for A are 2, 3, 4, 5, 6, 7, 8, and 9. However, given the range of CACA, the only possible value for A is 2.\n",
      "\n",
      "Now, let's move on to step 4.\n",
      "\n",
      "4. Determine the value of B given the values of A and C in the equation ABC + BAC = CACA.\n",
      "\n",
      "Given that A = 2 and C = 1, we can substitute these values into the equation and solve for B.\n",
      "\n",
      "The left-hand side (LHS) of the equation is:\n",
      "\n",
      "LHS = 110B + 110A + 2C\n",
      "    = 110B + 110(2) + 2(1)\n",
      "    = 110B + 222\n",
      "\n",
      "The right-hand side (RHS) of the equation is:\n",
      "\n",
      "RHS = 1010C + 101A\n",
      "    = 1010(1) + 101(2)\n",
      "    = 1010 + 202\n",
      "    = 1212\n",
      "\n",
      "Setting LHS equal to RHS, we get:\n",
      "\n",
      "110B + 222 = 1212\n",
      "\n",
      "Solving for B, we get:\n",
      "\n",
      "110B = 1212 - 222\n",
      "    = 990\n",
      "\n",
      "B = 990 / 110\n",
      "    = 9\n",
      "\n",
      "Therefore, the values of A, B, and C that satisfy the given equation and constraints are A = 2, B = 9, and C = 1. \n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "message = [\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": \"\"\"Solve for distinct single-digit natural numbers A, B, and C in the equation: ABC + BAC = CACA based on the following steps.\n",
    "<Constraints>\n",
    "- A, B, and C MUST be different natural numbers.\n",
    "- ABC, BAC are 3-digit numbers: ABC = 100*A + 10*B + 1*C, BAC = 100*B + 10*A + 1*C.\n",
    "- 1000 < CACA < the maximum of ABC+BAC.\n",
    "- ABC + BAC = 110*A + 110*B + 2*C = 100*A + 10*B + 1*C + 100*B + 10*A + 1*C.\n",
    "- CACA is a 4-digit number: CACA = 1010*C + 100*A + 1*A = 1000*C + 100*A + 10*C + 1*A.\n",
    "- 3-digit numbers cannot be greater than 1000.\n",
    "</Constraints>\n",
    "\n",
    "<STEPs>\n",
    "1. Estimate the range of CACA to satisfy the Constraints.\n",
    "2. Identify the first digit, C, of CACA through this range of CACA. It MUST be one number.\n",
    "3. Calculate units places of LHS using the result of step2. Then, estimate units place of RHS, A.\n",
    "4. Determine the value of B given the values of A and C in the equation ABC + BAC = CACA.\n",
    "- 1) LHS: 110*B + 110*A + 2*C.\n",
    "- 2) RHS: 1010*C + 101*A.\n",
    "- 3) Express the left side of the equation in terms of B : 110B = 1010C + 101A - 110A - 2C.\n",
    "</STEPs>\n",
    "\n",
    "<<Cautions>>\n",
    "- Before move to next step, you should do self-critic on your think.\n",
    "- Each step should be connected, do not ignore previous result.\n",
    "\n",
    "Let's think step by step.\"\"\"\n",
    "    }\n",
    "]\n",
    "\n",
    "response = get_completion(messages=message)\n",
    "print(response, \"\\n\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- [**The incorrect answer**] Zero-Shot Prompt: $A = 2, B = 1, C = 1$\n",
    "\n",
    "    > \"Let's solve this step by step.\n",
    "    > \n",
    "    > First, let's write the equation in a more understandable way:\n",
    "    >\n",
    "    > 100A + 10B + C + 100B + 10A + C = 1000C + 100A + 10C + A\n",
    "    >\n",
    "    > Now, let's simplify and rearrange the equation:\n",
    "    > \n",
    "    > 110A + 110B + 2C = 1000C + 100A + 10C + A\n",
    "    > \n",
    "    > Subtract 110A from both sides:\n",
    "    >\n",
    "    > 110B + 2C = 1000C + 10C + A\n",
    "    >\n",
    "    > Subtract 2C from both sides:\n",
    "    > \n",
    "    > 110B = 998C + 10C + A\n",
    "    > \n",
    "    > Subtract 10C from both sides:\n",
    "    > \n",
    "    > 110B = 988C + A\n",
    "    >\n",
    "    > Now, we know that A, B, and C are single-digit natural numbers, so the left side of the equation must be a multiple of 110. The only single-digit natural number that can be added to a multiple of 988 to get a multiple of 110 is 2. So, A = 2.\n",
    "    > \n",
    "    > Substitute A = 2 into the equation:\n",
    "    >\n",
    "    > 110B = 988C + 2\n",
    "    >\n",
    "    > Now, we need to find B and C such that 110B is a multiple of 110 and 988C + 2 is a single-digit number. The only possible value for B is 1, and the only possible value for C is 1.\n",
    "    > \n",
    "    > So, the solution is A = 2, B = 1, and C = 1.\"\n",
    "\n",
    "<br>  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*As shown above, when the same problem is solved using zero-shot prompting instead of CoT, Solar fails to arrive at the correct answer.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Next: [Chapter 8. Resolving Hallucinations](./08_Chapter%208.%20Resolving%20Hallucinations.ipynb)*"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}