rwkv.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/Caskroom/miniconda/base/envs/llm310/lib/python3.10/site-packages/transformers/generation/utils.py:1411: UserWarning: You have modified the pretrained model configuration to control generation. This is a deprecated strategy to control generation and will be removed soon, in a future version. Please use a generation configuration file (see https://huggingface.co/docs/transformers/main_classes/text_generation )\n",
      "  warnings.warn(\n",
      "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[{'generated_text': '\\nIn a shocking finding, scientists discovered a herd of dragons living in a remote, previously unexplored valley, in Tibet. Even more surprising to the researchers was the fact that the dragons spoke perfect Chinese.\\n\\nDr. Paul Lassatt, professor of botany at Stanford University, was the lead'}]\n"
     ]
    }
   ],
   "source": [
    "from transformers import pipeline\n",
    "\n",
    "model_id = \"RWKV/rwkv-4-169m-pile\"\n",
    "\n",
    "prompt = \"\\nIn a shocking finding, scientists discovered a herd of dragons living in a remote, previously unexplored valley, in Tibet. Even more surprising to the researchers was the fact that the dragons spoke perfect Chinese.\"\n",
    "\n",
    "pipe = pipeline(\"text-generation\", model=model_id, top_k=50, do_sample=True)\n",
    "print(pipe(prompt, max_new_tokens=20))\n",
    "# [{'generated_text': '\\nIn a shocking finding, scientist discovered a herd of dragons living in a remote, previously unexplored valley, in Tibet. Even more surprising to the researchers was the fact that the dragons spoke perfect Chinese.\\n\\nThe researchers found that the dragons were able to communicate with each other, and that they were'}]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n",
      "This is a friendly reminder - the current text generation call will exceed the model's predefined maximum length (1024). Depending on the model, you may observe exceptions, performance degradation, or nothing at all.\n"
     ]
    }
   ],
   "source": [
    "out = pipe(prompt, max_new_tokens=1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "In a shocking finding, scientists discovered a herd of dragons living in a remote, previously unexplored valley, in Tibet. Even more surprising to the researchers was the fact that the dragons spoke perfect Chinese. Scientists say they were from a language called the Brahmic, not a dragon—which may mean “language.”\n",
      "\n",
      "“One of the Chinese characters is used to represent the Brahmic meaning of the name,” said study leader Hai Jia, who led the team. “The language also has two different names for their own language, Chinese and Tibetan.”\n",
      "\n",
      "\n",
      "To determine if a certain language, as they claim it was, was a direct ancestor of Brahmic, they took the Himalayan mountain (which contains about 300,000 Buddhas) and transformed it into a snake. They found that the animals have the characteristics of an animal-like creature with the characteristic of having a long tail and little claws.\n",
      "\n",
      "The researchers discovered that the language is derived from a dialect of the Lixi people, the tribe called “He’s from the mountains.” In Tibetan, these words are often translated into English as “you’ve reached the Tibet mountain.” They said those are the Tibetan word for dragon.\n",
      "\n",
      "“It is a very strong word,” said Jian Ding, a paleontologist at Georgia State University in Seattle. “It’s very strong.”\n",
      "\n",
      "The scientists said that if they had tested the findings in the Himalayan region on Tibetan mountain goats and chimpanzees, they would have found the word “Beauty.”\n",
      "\n",
      "\n",
      "“The dragon is the ultimate dragon,” they said.\n",
      "\n",
      "After examining the words, they concluded that the word “He” means “to make an appearance,” and “he” means “someone.”\n",
      "\n",
      "The Tibetan language is much easier to understand, Jian said, than what one might think of as a language from “Eastern Europe.” For instance, a person may come to the mountain with a sense of “good and evil,” or a person may come across that person’s name through bonafide translations as if it were English “Eternal Life.”\n",
      "\n",
      "For example, one group of geese came to Tibet with the idea of entering the Himalaya. They could travel from the mountains to Tibet. They could bring their children with them.\n",
      "\n",
      "“The word ‘Beauty’ means ‘fruhya,’ which means ‘beautiful,’” said M.J. Weng, a geologist at Cambridge University. “The Tibetan people have no idea that they have the word ‘beautiful.’ They just see the mountain as the place they go to find the mountain.”\n",
      "\n",
      "A number of geographers have tried to connect the words “Beauty” with the language of Tibetans. They say that “Beauchin” is part Tibetan, part Chinese. It might be called, as the researchers were able to learn, “Beauty-Beautified.” The same is true of some, especially Chinese, who study the Bearded Tibetan word for “beautiful”—the word “Beautiful.”\n",
      "\n",
      "\n",
      "This is a particularly telling sign of cultural change, M.J. Weng, an associate professor of forestry at the University of Houston. He was part of a team at the US Geological Survey in the last two decades, documenting how many people lived on a mountain, sometimes up to 200 people from a village, and then calling the locals, often by name, to take their places.\n",
      "\n",
      "The study, published in the March 2009 online journal, “Beautiful Tibet,” looked at more than 1,000 people on two expeditions to two distinct valleys in southern Tibet, a region that has for centuries been considered one of the best regions for exploring Tibetan culture and history.\n",
      "\n",
      "The team looked at the language and cultural groups that the scientists studied. Even though the geologists and linguists are experts in the language, they were not sure what to do about a mountain on the Tibetan plateau that has changed little since the last Ice Age in the 13th century. They began wondering how deep it would be and began to question whether it would remain a mountain for the next 2,800 years.\n",
      "\n",
      "In the days before the scientists decided that it was time to add the word “Behest” to the words, they began researching how people used and studied some dialects of Tsimshian.\n",
      "\n",
      "\n",
      "“It is very difficult to explain all of the language use to people without going through the language,” said M.J.W. Meng, a geologist at The University of Kansas and a co-author on the study. He said his team found very small differences among the different dialects of the language.\n",
      "\n",
      "In Tibetan, “Beautiful” probably should have been written on top of the name as a word for a mountain range that lies in the middle of a Tibetan plateau, but it was also quite important. People in the mountains often learned more than the words did, Meng said. He speculated that some people did not even learn that\n"
     ]
    }
   ],
   "source": [
    "print(out[0]['generated_text'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "os.environ[\"TOKENIZERS_PARALLELISM\"] = \"false\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/Caskroom/miniconda/base/envs/llm310/lib/python3.10/site-packages/transformers/generation/utils.py:1411: UserWarning: You have modified the pretrained model configuration to control generation. This is a deprecated strategy to control generation and will be removed soon, in a future version. Please use a generation configuration file (see https://huggingface.co/docs/transformers/main_classes/text_generation )\n",
      "  warnings.warn(\n",
      "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n",
      "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "In a shocking finding, scientist discovered a herd of dragons living in a remote, previously unexplored valley, in Tibet. Even more surprising to the researchers was the fact that the dragons spoke perfect Chinese.\n",
      "\n",
      "The researchers found that the dragons were able to communicate with each other, and that they were able to communicate with each other.\n",
      "\n",
      "The researchers also found that the dragons were able to communicate with each other, and that they were able to communicate with each other.\n",
      "\n",
      "The researchers also found that the dragons were able to communicate with each other, and that they were able to communicate with each other.\n",
      "\n",
      "The researchers also found that the dragons were able to communicate with each other,\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "from transformers import AutoModelForCausalLM, AutoTokenizer\n",
    "\n",
    "model = AutoModelForCausalLM.from_pretrained(\"RWKV/rwkv-4-169m-pile\")\n",
    "tokenizer = AutoTokenizer.from_pretrained(\"RWKV/rwkv-4-169m-pile\")\n",
    "\n",
    "prompt = \"\\nIn a shocking finding, scientist discovered a herd of dragons living in a remote, previously unexplored valley, in Tibet. Even more surprising to the researchers was the fact that the dragons spoke perfect Chinese.\"\n",
    "\n",
    "inputs = tokenizer(prompt, return_tensors=\"pt\")\n",
    "output = model.generate(inputs[\"input_ids\"], max_new_tokens=100)\n",
    "print(tokenizer.decode(output[0].tolist()))\n",
    "# In a shocking finding, scientist discovered a herd of dragons living in a remote, previously unexplored valley, in Tibet. Even more surprising to the researchers was the fact that the dragons spoke perfect Chinese.\\n\\nThe researchers found that the dragons were able to communicate with each other, and that they were"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b0cf46f4fd8041fb96bf5d02ada998c4",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading (…)lve/main/config.json:   0%|          | 0.00/523 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a2d63e5c92fc49c39e47c44622e2949f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading (…)model.bin.index.json:   0%|          | 0.00/34.8k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a0b7a50c252c45ab8869dbc5fa0dc124",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading shards:   0%|          | 0/4 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "46d8717f3d7d474bb314a3180576dee3",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading (…)l-00001-of-00004.bin:   0%|          | 0.00/1.99G [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "4bf763a8c83140ba882e353773b66e0e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading (…)l-00002-of-00004.bin:   0%|          | 0.00/1.98G [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c8ce3f461ddd4676a69acddc17ef03eb",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading (…)l-00003-of-00004.bin:   0%|          | 0.00/1.68G [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "226fb5eb7cc54a818683bd8602c3e3a1",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading (…)l-00004-of-00004.bin:   0%|          | 0.00/412M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6fb4e37225ea4e8ab3342cc45b26c0a0",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "821b136d79dd4f9c83b57a8c761f5811",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading (…)neration_config.json:   0%|          | 0.00/116 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #800000; text-decoration-color: #800000\">╭─────────────────────────────── </span><span style=\"color: #800000; text-decoration-color: #800000; font-weight: bold\">Traceback </span><span style=\"color: #bf7f7f; text-decoration-color: #bf7f7f; font-weight: bold\">(most recent call last)</span><span style=\"color: #800000; text-decoration-color: #800000\"> ────────────────────────────────╮</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> in <span style=\"color: #00ff00; text-decoration-color: #00ff00\">&lt;module&gt;</span>:<span style=\"color: #0000ff; text-decoration-color: #0000ff\">5</span>                                                                                    <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 2 </span>                                                                                            <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 3 </span>model_id = <span style=\"color: #808000; text-decoration-color: #808000\">\"RWKV/rwkv-raven-1b5\"</span>                                                            <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 4 </span>                                                                                            <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #800000; text-decoration-color: #800000\">❱ </span> 5 model = AutoModelForCausalLM.from_pretrained(model_id).to(<span style=\"color: #0000ff; text-decoration-color: #0000ff\">0</span>)                                <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 6 </span>tokenizer = AutoTokenizer.from_pretrained(model_id)                                         <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 7 </span>                                                                                            <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 8 </span>question = <span style=\"color: #808000; text-decoration-color: #808000\">\"Tell me about ravens\"</span>                                                           <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #bfbf7f; text-decoration-color: #bfbf7f\">/usr/local/Caskroom/miniconda/base/envs/llm310/lib/python3.10/site-packages/transformers/</span><span style=\"color: #808000; text-decoration-color: #808000; font-weight: bold\">modelin</span> <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #808000; text-decoration-color: #808000; font-weight: bold\">g_utils.py</span>:<span style=\"color: #0000ff; text-decoration-color: #0000ff\">2014</span> in <span style=\"color: #00ff00; text-decoration-color: #00ff00\">to</span>                                                                            <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">2011 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">\" model has already been set to the correct devices and casted to the co</span>  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">2012 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   </span>)                                                                             <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">2013 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">else</span>:                                                                             <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #800000; text-decoration-color: #800000\">❱ </span>2014 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">return</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff\">super</span>().to(*args, **kwargs)                                            <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">2015 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   </span>                                                                                      <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">2016 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">def</span> <span style=\"color: #00ff00; text-decoration-color: #00ff00\">half</span>(<span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>, *args):                                                                <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">2017 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"># Checks if the model is quantized</span>                                                <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #bfbf7f; text-decoration-color: #bfbf7f\">/usr/local/Caskroom/miniconda/base/envs/llm310/lib/python3.10/site-packages/torch/nn/modules/</span><span style=\"color: #808000; text-decoration-color: #808000; font-weight: bold\">mod</span> <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #808000; text-decoration-color: #808000; font-weight: bold\">ule.py</span>:<span style=\"color: #0000ff; text-decoration-color: #0000ff\">1145</span> in <span style=\"color: #00ff00; text-decoration-color: #00ff00\">to</span>                                                                                <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1142 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   │   │   │   │   </span>non_blocking, memory_format=convert_to_format)                <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1143 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">return</span> t.to(device, dtype <span style=\"color: #0000ff; text-decoration-color: #0000ff\">if</span> t.is_floating_point() <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">or</span> t.is_complex() <span style=\"color: #0000ff; text-decoration-color: #0000ff\">else</span> <span style=\"color: #0000ff; text-decoration-color: #0000ff\">No</span>  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1144 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span>                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #800000; text-decoration-color: #800000\">❱ </span>1145 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">return</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>._apply(convert)                                                       <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1146 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   </span>                                                                                      <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1147 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">def</span> <span style=\"color: #00ff00; text-decoration-color: #00ff00\">register_full_backward_pre_hook</span>(                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1148 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>,                                                                             <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #bfbf7f; text-decoration-color: #bfbf7f\">/usr/local/Caskroom/miniconda/base/envs/llm310/lib/python3.10/site-packages/torch/nn/modules/</span><span style=\"color: #808000; text-decoration-color: #808000; font-weight: bold\">mod</span> <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #808000; text-decoration-color: #808000; font-weight: bold\">ule.py</span>:<span style=\"color: #0000ff; text-decoration-color: #0000ff\">797</span> in <span style=\"color: #00ff00; text-decoration-color: #00ff00\">_apply</span>                                                                             <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 794 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   </span>                                                                                      <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 795 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">def</span> <span style=\"color: #00ff00; text-decoration-color: #00ff00\">_apply</span>(<span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>, fn):                                                                 <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 796 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">for</span> module <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">in</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>.children():                                                    <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #800000; text-decoration-color: #800000\">❱ </span> 797 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   </span>module._apply(fn)                                                             <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 798 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span>                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 799 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">def</span> <span style=\"color: #00ff00; text-decoration-color: #00ff00\">compute_should_use_set_data</span>(tensor, tensor_applied):                          <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 800 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">if</span> torch._has_compatible_shallow_copy_type(tensor, tensor_applied):           <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #bfbf7f; text-decoration-color: #bfbf7f\">/usr/local/Caskroom/miniconda/base/envs/llm310/lib/python3.10/site-packages/torch/nn/modules/</span><span style=\"color: #808000; text-decoration-color: #808000; font-weight: bold\">mod</span> <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #808000; text-decoration-color: #808000; font-weight: bold\">ule.py</span>:<span style=\"color: #0000ff; text-decoration-color: #0000ff\">797</span> in <span style=\"color: #00ff00; text-decoration-color: #00ff00\">_apply</span>                                                                             <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 794 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   </span>                                                                                      <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 795 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">def</span> <span style=\"color: #00ff00; text-decoration-color: #00ff00\">_apply</span>(<span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>, fn):                                                                 <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 796 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">for</span> module <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">in</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>.children():                                                    <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #800000; text-decoration-color: #800000\">❱ </span> 797 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   </span>module._apply(fn)                                                             <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 798 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span>                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 799 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">def</span> <span style=\"color: #00ff00; text-decoration-color: #00ff00\">compute_should_use_set_data</span>(tensor, tensor_applied):                          <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 800 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">if</span> torch._has_compatible_shallow_copy_type(tensor, tensor_applied):           <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #bfbf7f; text-decoration-color: #bfbf7f\">/usr/local/Caskroom/miniconda/base/envs/llm310/lib/python3.10/site-packages/torch/nn/modules/</span><span style=\"color: #808000; text-decoration-color: #808000; font-weight: bold\">mod</span> <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #808000; text-decoration-color: #808000; font-weight: bold\">ule.py</span>:<span style=\"color: #0000ff; text-decoration-color: #0000ff\">820</span> in <span style=\"color: #00ff00; text-decoration-color: #00ff00\">_apply</span>                                                                             <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 817 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"># track autograd history of `param_applied`, so we have to use</span>                <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 818 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"># `with torch.no_grad():`</span>                                                     <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 819 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">with</span> torch.no_grad():                                                         <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #800000; text-decoration-color: #800000\">❱ </span> 820 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   │   </span>param_applied = fn(param)                                                 <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 821 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   </span>should_use_set_data = compute_should_use_set_data(param, param_applied)       <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 822 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">if</span> should_use_set_data:                                                       <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 823 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   │   </span>param.data = param_applied                                                <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #bfbf7f; text-decoration-color: #bfbf7f\">/usr/local/Caskroom/miniconda/base/envs/llm310/lib/python3.10/site-packages/torch/nn/modules/</span><span style=\"color: #808000; text-decoration-color: #808000; font-weight: bold\">mod</span> <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #808000; text-decoration-color: #808000; font-weight: bold\">ule.py</span>:<span style=\"color: #0000ff; text-decoration-color: #0000ff\">1143</span> in <span style=\"color: #00ff00; text-decoration-color: #00ff00\">convert</span>                                                                           <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1140 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">if</span> convert_to_format <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">is</span> <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">not</span> <span style=\"color: #0000ff; text-decoration-color: #0000ff\">None</span> <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">and</span> t.dim() <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">in</span> (<span style=\"color: #0000ff; text-decoration-color: #0000ff\">4</span>, <span style=\"color: #0000ff; text-decoration-color: #0000ff\">5</span>):                       <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1141 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">return</span> t.to(device, dtype <span style=\"color: #0000ff; text-decoration-color: #0000ff\">if</span> t.is_floating_point() <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">or</span> t.is_complex() <span style=\"color: #0000ff; text-decoration-color: #0000ff\">els</span>  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1142 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   │   │   │   │   </span>non_blocking, memory_format=convert_to_format)                <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #800000; text-decoration-color: #800000\">❱ </span>1143 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">return</span> t.to(device, dtype <span style=\"color: #0000ff; text-decoration-color: #0000ff\">if</span> t.is_floating_point() <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">or</span> t.is_complex() <span style=\"color: #0000ff; text-decoration-color: #0000ff\">else</span> <span style=\"color: #0000ff; text-decoration-color: #0000ff\">No</span>  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1144 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span>                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1145 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">return</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>._apply(convert)                                                       <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1146 </span>                                                                                          <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #bfbf7f; text-decoration-color: #bfbf7f\">/usr/local/Caskroom/miniconda/base/envs/llm310/lib/python3.10/site-packages/torch/cuda/</span><span style=\"color: #808000; text-decoration-color: #808000; font-weight: bold\">__init__.</span> <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #808000; text-decoration-color: #808000; font-weight: bold\">py</span>:<span style=\"color: #0000ff; text-decoration-color: #0000ff\">239</span> in <span style=\"color: #00ff00; text-decoration-color: #00ff00\">_lazy_init</span>                                                                             <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 236 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">\"Cannot re-initialize CUDA in forked subprocess. To use CUDA with \"</span>       <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 237 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">\"multiprocessing, you must use the 'spawn' start method\"</span>)                 <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 238 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">if</span> <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">not</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff\">hasattr</span>(torch._C, <span style=\"color: #808000; text-decoration-color: #808000\">'_cuda_getDeviceCount'</span>):                                 <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #800000; text-decoration-color: #800000\">❱ </span> 239 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">raise</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff\">AssertionError</span>(<span style=\"color: #808000; text-decoration-color: #808000\">\"Torch not compiled with CUDA enabled\"</span>)                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 240 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">if</span> _cudart <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">is</span> <span style=\"color: #0000ff; text-decoration-color: #0000ff\">None</span>:                                                               <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 241 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">raise</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff\">AssertionError</span>(                                                         <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 242 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">\"libcudart functions unavailable. It looks like you have a broken build?</span>  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">╰──────────────────────────────────────────────────────────────────────────────────────────────────╯</span>\n",
       "<span style=\"color: #ff0000; text-decoration-color: #ff0000; font-weight: bold\">AssertionError: </span>Torch not compiled with CUDA enabled\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[31m╭─\u001b[0m\u001b[31m──────────────────────────────\u001b[0m\u001b[31m \u001b[0m\u001b[1;31mTraceback \u001b[0m\u001b[1;2;31m(most recent call last)\u001b[0m\u001b[31m \u001b[0m\u001b[31m───────────────────────────────\u001b[0m\u001b[31m─╮\u001b[0m\n",
       "\u001b[31m│\u001b[0m in \u001b[92m<module>\u001b[0m:\u001b[94m5\u001b[0m                                                                                    \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 2 \u001b[0m                                                                                            \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 3 \u001b[0mmodel_id = \u001b[33m\"\u001b[0m\u001b[33mRWKV/rwkv-raven-1b5\u001b[0m\u001b[33m\"\u001b[0m                                                            \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 4 \u001b[0m                                                                                            \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[31m❱ \u001b[0m 5 model = AutoModelForCausalLM.from_pretrained(model_id).to(\u001b[94m0\u001b[0m)                                \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 6 \u001b[0mtokenizer = AutoTokenizer.from_pretrained(model_id)                                         \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 7 \u001b[0m                                                                                            \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 8 \u001b[0mquestion = \u001b[33m\"\u001b[0m\u001b[33mTell me about ravens\u001b[0m\u001b[33m\"\u001b[0m                                                           \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[2;33m/usr/local/Caskroom/miniconda/base/envs/llm310/lib/python3.10/site-packages/transformers/\u001b[0m\u001b[1;33mmodelin\u001b[0m \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[1;33mg_utils.py\u001b[0m:\u001b[94m2014\u001b[0m in \u001b[92mto\u001b[0m                                                                            \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m2011 \u001b[0m\u001b[2m│   │   │   │   \u001b[0m\u001b[33m\"\u001b[0m\u001b[33m model has already been set to the correct devices and casted to the co\u001b[0m  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m2012 \u001b[0m\u001b[2m│   │   │   \u001b[0m)                                                                             \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m2013 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[94melse\u001b[0m:                                                                             \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[31m❱ \u001b[0m2014 \u001b[2m│   │   │   \u001b[0m\u001b[94mreturn\u001b[0m \u001b[96msuper\u001b[0m().to(*args, **kwargs)                                            \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m2015 \u001b[0m\u001b[2m│   \u001b[0m                                                                                      \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m2016 \u001b[0m\u001b[2m│   \u001b[0m\u001b[94mdef\u001b[0m \u001b[92mhalf\u001b[0m(\u001b[96mself\u001b[0m, *args):                                                                \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m2017 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[2m# Checks if the model is quantized\u001b[0m                                                \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[2;33m/usr/local/Caskroom/miniconda/base/envs/llm310/lib/python3.10/site-packages/torch/nn/modules/\u001b[0m\u001b[1;33mmod\u001b[0m \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[1;33mule.py\u001b[0m:\u001b[94m1145\u001b[0m in \u001b[92mto\u001b[0m                                                                                \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m1142 \u001b[0m\u001b[2m│   │   │   │   │   │   │   \u001b[0mnon_blocking, memory_format=convert_to_format)                \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m1143 \u001b[0m\u001b[2m│   │   │   \u001b[0m\u001b[94mreturn\u001b[0m t.to(device, dtype \u001b[94mif\u001b[0m t.is_floating_point() \u001b[95mor\u001b[0m t.is_complex() \u001b[94melse\u001b[0m \u001b[94mNo\u001b[0m  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m1144 \u001b[0m\u001b[2m│   │   \u001b[0m                                                                                  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[31m❱ \u001b[0m1145 \u001b[2m│   │   \u001b[0m\u001b[94mreturn\u001b[0m \u001b[96mself\u001b[0m._apply(convert)                                                       \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m1146 \u001b[0m\u001b[2m│   \u001b[0m                                                                                      \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m1147 \u001b[0m\u001b[2m│   \u001b[0m\u001b[94mdef\u001b[0m \u001b[92mregister_full_backward_pre_hook\u001b[0m(                                                  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m1148 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[96mself\u001b[0m,                                                                             \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[2;33m/usr/local/Caskroom/miniconda/base/envs/llm310/lib/python3.10/site-packages/torch/nn/modules/\u001b[0m\u001b[1;33mmod\u001b[0m \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[1;33mule.py\u001b[0m:\u001b[94m797\u001b[0m in \u001b[92m_apply\u001b[0m                                                                             \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 794 \u001b[0m\u001b[2m│   \u001b[0m                                                                                      \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 795 \u001b[0m\u001b[2m│   \u001b[0m\u001b[94mdef\u001b[0m \u001b[92m_apply\u001b[0m(\u001b[96mself\u001b[0m, fn):                                                                 \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 796 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[94mfor\u001b[0m module \u001b[95min\u001b[0m \u001b[96mself\u001b[0m.children():                                                    \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[31m❱ \u001b[0m 797 \u001b[2m│   │   │   \u001b[0mmodule._apply(fn)                                                             \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 798 \u001b[0m\u001b[2m│   │   \u001b[0m                                                                                  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 799 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[94mdef\u001b[0m \u001b[92mcompute_should_use_set_data\u001b[0m(tensor, tensor_applied):                          \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 800 \u001b[0m\u001b[2m│   │   │   \u001b[0m\u001b[94mif\u001b[0m torch._has_compatible_shallow_copy_type(tensor, tensor_applied):           \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[2;33m/usr/local/Caskroom/miniconda/base/envs/llm310/lib/python3.10/site-packages/torch/nn/modules/\u001b[0m\u001b[1;33mmod\u001b[0m \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[1;33mule.py\u001b[0m:\u001b[94m797\u001b[0m in \u001b[92m_apply\u001b[0m                                                                             \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 794 \u001b[0m\u001b[2m│   \u001b[0m                                                                                      \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 795 \u001b[0m\u001b[2m│   \u001b[0m\u001b[94mdef\u001b[0m \u001b[92m_apply\u001b[0m(\u001b[96mself\u001b[0m, fn):                                                                 \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 796 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[94mfor\u001b[0m module \u001b[95min\u001b[0m \u001b[96mself\u001b[0m.children():                                                    \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[31m❱ \u001b[0m 797 \u001b[2m│   │   │   \u001b[0mmodule._apply(fn)                                                             \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 798 \u001b[0m\u001b[2m│   │   \u001b[0m                                                                                  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 799 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[94mdef\u001b[0m \u001b[92mcompute_should_use_set_data\u001b[0m(tensor, tensor_applied):                          \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 800 \u001b[0m\u001b[2m│   │   │   \u001b[0m\u001b[94mif\u001b[0m torch._has_compatible_shallow_copy_type(tensor, tensor_applied):           \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[2;33m/usr/local/Caskroom/miniconda/base/envs/llm310/lib/python3.10/site-packages/torch/nn/modules/\u001b[0m\u001b[1;33mmod\u001b[0m \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[1;33mule.py\u001b[0m:\u001b[94m820\u001b[0m in \u001b[92m_apply\u001b[0m                                                                             \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 817 \u001b[0m\u001b[2m│   │   │   \u001b[0m\u001b[2m# track autograd history of `param_applied`, so we have to use\u001b[0m                \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 818 \u001b[0m\u001b[2m│   │   │   \u001b[0m\u001b[2m# `with torch.no_grad():`\u001b[0m                                                     \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 819 \u001b[0m\u001b[2m│   │   │   \u001b[0m\u001b[94mwith\u001b[0m torch.no_grad():                                                         \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[31m❱ \u001b[0m 820 \u001b[2m│   │   │   │   \u001b[0mparam_applied = fn(param)                                                 \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 821 \u001b[0m\u001b[2m│   │   │   \u001b[0mshould_use_set_data = compute_should_use_set_data(param, param_applied)       \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 822 \u001b[0m\u001b[2m│   │   │   \u001b[0m\u001b[94mif\u001b[0m should_use_set_data:                                                       \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 823 \u001b[0m\u001b[2m│   │   │   │   \u001b[0mparam.data = param_applied                                                \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[2;33m/usr/local/Caskroom/miniconda/base/envs/llm310/lib/python3.10/site-packages/torch/nn/modules/\u001b[0m\u001b[1;33mmod\u001b[0m \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[1;33mule.py\u001b[0m:\u001b[94m1143\u001b[0m in \u001b[92mconvert\u001b[0m                                                                           \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m1140 \u001b[0m\u001b[2m│   │   │   \u001b[0m\u001b[94mif\u001b[0m convert_to_format \u001b[95mis\u001b[0m \u001b[95mnot\u001b[0m \u001b[94mNone\u001b[0m \u001b[95mand\u001b[0m t.dim() \u001b[95min\u001b[0m (\u001b[94m4\u001b[0m, \u001b[94m5\u001b[0m):                       \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m1141 \u001b[0m\u001b[2m│   │   │   │   \u001b[0m\u001b[94mreturn\u001b[0m t.to(device, dtype \u001b[94mif\u001b[0m t.is_floating_point() \u001b[95mor\u001b[0m t.is_complex() \u001b[94mels\u001b[0m  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m1142 \u001b[0m\u001b[2m│   │   │   │   │   │   │   \u001b[0mnon_blocking, memory_format=convert_to_format)                \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[31m❱ \u001b[0m1143 \u001b[2m│   │   │   \u001b[0m\u001b[94mreturn\u001b[0m t.to(device, dtype \u001b[94mif\u001b[0m t.is_floating_point() \u001b[95mor\u001b[0m t.is_complex() \u001b[94melse\u001b[0m \u001b[94mNo\u001b[0m  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m1144 \u001b[0m\u001b[2m│   │   \u001b[0m                                                                                  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m1145 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[94mreturn\u001b[0m \u001b[96mself\u001b[0m._apply(convert)                                                       \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m1146 \u001b[0m                                                                                          \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[2;33m/usr/local/Caskroom/miniconda/base/envs/llm310/lib/python3.10/site-packages/torch/cuda/\u001b[0m\u001b[1;33m__init__.\u001b[0m \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[1;33mpy\u001b[0m:\u001b[94m239\u001b[0m in \u001b[92m_lazy_init\u001b[0m                                                                             \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 236 \u001b[0m\u001b[2m│   │   │   │   \u001b[0m\u001b[33m\"\u001b[0m\u001b[33mCannot re-initialize CUDA in forked subprocess. To use CUDA with \u001b[0m\u001b[33m\"\u001b[0m       \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 237 \u001b[0m\u001b[2m│   │   │   │   \u001b[0m\u001b[33m\"\u001b[0m\u001b[33mmultiprocessing, you must use the \u001b[0m\u001b[33m'\u001b[0m\u001b[33mspawn\u001b[0m\u001b[33m'\u001b[0m\u001b[33m start method\u001b[0m\u001b[33m\"\u001b[0m)                 \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 238 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[94mif\u001b[0m \u001b[95mnot\u001b[0m \u001b[96mhasattr\u001b[0m(torch._C, \u001b[33m'\u001b[0m\u001b[33m_cuda_getDeviceCount\u001b[0m\u001b[33m'\u001b[0m):                                 \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[31m❱ \u001b[0m 239 \u001b[2m│   │   │   \u001b[0m\u001b[94mraise\u001b[0m \u001b[96mAssertionError\u001b[0m(\u001b[33m\"\u001b[0m\u001b[33mTorch not compiled with CUDA enabled\u001b[0m\u001b[33m\"\u001b[0m)                  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 240 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[94mif\u001b[0m _cudart \u001b[95mis\u001b[0m \u001b[94mNone\u001b[0m:                                                               \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 241 \u001b[0m\u001b[2m│   │   │   \u001b[0m\u001b[94mraise\u001b[0m \u001b[96mAssertionError\u001b[0m(                                                         \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 242 \u001b[0m\u001b[2m│   │   │   │   \u001b[0m\u001b[33m\"\u001b[0m\u001b[33mlibcudart functions unavailable. It looks like you have a broken build?\u001b[0m  \u001b[31m│\u001b[0m\n",
       "\u001b[31m╰──────────────────────────────────────────────────────────────────────────────────────────────────╯\u001b[0m\n",
       "\u001b[1;91mAssertionError: \u001b[0mTorch not compiled with CUDA enabled\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from transformers import AutoTokenizer, AutoModelForCausalLM\n",
    "\n",
    "model_id = \"RWKV/rwkv-raven-1b5\"\n",
    "\n",
    "model = AutoModelForCausalLM.from_pretrained(model_id).to(0)\n",
    "tokenizer = AutoTokenizer.from_pretrained(model_id)\n",
    "\n",
    "question = \"Tell me about ravens\"\n",
    "prompt = f\"### Instruction: {question}\\n### Response:\"\n",
    "\n",
    "inputs = tokenizer(prompt, return_tensors=\"pt\").to(0)\n",
    "output = model.generate(inputs[\"input_ids\"], max_new_tokens=100)\n",
    "\n",
    "print(tokenizer.decode(output[0].tolist(), skip_special_tokens=True))\n",
    "# >>> ### Instruction: Tell me about ravens\n",
    "### Response: RAVENS are a type of bird that is native to the Middle East and North Africa. They are known for their intelligence, adaptability, and their ability to live in a variety of environments. RAVENS are known for their intelligence, adaptability, and their ability to live in a variety of environments. They are known for their intelligence, adaptability, and their ability to live in a variety of environments.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #800000; text-decoration-color: #800000\">╭─────────────────────────────── </span><span style=\"color: #800000; text-decoration-color: #800000; font-weight: bold\">Traceback </span><span style=\"color: #bf7f7f; text-decoration-color: #bf7f7f; font-weight: bold\">(most recent call last)</span><span style=\"color: #800000; text-decoration-color: #800000\"> ────────────────────────────────╮</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> in <span style=\"color: #00ff00; text-decoration-color: #00ff00\">&lt;module&gt;</span>:<span style=\"color: #0000ff; text-decoration-color: #0000ff\">4</span>                                                                                    <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">1 </span>question = <span style=\"color: #808000; text-decoration-color: #808000\">\"\\nContinue the following story: In a shocking finding, scientist discovered </span>     <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">2 </span>prompt = <span style=\"color: #808000; text-decoration-color: #808000\">f\"### Instruction: {</span>question<span style=\"color: #808000; text-decoration-color: #808000\">}\\n### Response:\"</span>                                       <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">3 </span>                                                                                             <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #800000; text-decoration-color: #800000\">❱ </span>4 inputs = tokenizer(prompt, return_tensors=<span style=\"color: #808000; text-decoration-color: #808000\">\"pt\"</span>).to(<span style=\"color: #0000ff; text-decoration-color: #0000ff\">0</span>)                                        <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">5 </span>output = model.generate(inputs[<span style=\"color: #808000; text-decoration-color: #808000\">\"input_ids\"</span>], max_new_tokens=<span style=\"color: #0000ff; text-decoration-color: #0000ff\">100</span>)                             <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">6 </span>                                                                                             <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">7 </span><span style=\"color: #00ffff; text-decoration-color: #00ffff\">print</span>(tokenizer.decode(output[<span style=\"color: #0000ff; text-decoration-color: #0000ff\">0</span>].tolist(), skip_special_tokens=<span style=\"color: #0000ff; text-decoration-color: #0000ff\">True</span>))                        <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #bfbf7f; text-decoration-color: #bfbf7f\">/usr/local/Caskroom/miniconda/base/envs/llm310/lib/python3.10/site-packages/transformers/</span><span style=\"color: #808000; text-decoration-color: #808000; font-weight: bold\">tokeniz</span> <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #808000; text-decoration-color: #808000; font-weight: bold\">ation_utils_base.py</span>:<span style=\"color: #0000ff; text-decoration-color: #0000ff\">777</span> in <span style=\"color: #00ff00; text-decoration-color: #00ff00\">to</span>                                                                    <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 774 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"># Otherwise it passes the casts down and casts the LongTensor containing the tok</span>  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 775 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"># into a HalfTensor</span>                                                               <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 776 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">if</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff\">isinstance</span>(device, <span style=\"color: #00ffff; text-decoration-color: #00ffff\">str</span>) <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">or</span> is_torch_device(device) <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">or</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff\">isinstance</span>(device, <span style=\"color: #00ffff; text-decoration-color: #00ffff\">int</span>)  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #800000; text-decoration-color: #800000\">❱ </span> 777 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   </span><span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>.data = {k: v.to(device=device) <span style=\"color: #0000ff; text-decoration-color: #0000ff\">for</span> k, v <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">in</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>.data.items()}            <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 778 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">else</span>:                                                                             <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 779 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   </span>logger.warning(<span style=\"color: #808000; text-decoration-color: #808000\">f\"Attempting to cast a BatchEncoding to type {</span><span style=\"color: #00ffff; text-decoration-color: #00ffff\">str</span>(device)<span style=\"color: #808000; text-decoration-color: #808000\">}. T</span>  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 780 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">return</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>                                                                       <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #bfbf7f; text-decoration-color: #bfbf7f\">/usr/local/Caskroom/miniconda/base/envs/llm310/lib/python3.10/site-packages/transformers/</span><span style=\"color: #808000; text-decoration-color: #808000; font-weight: bold\">tokeniz</span> <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #808000; text-decoration-color: #808000; font-weight: bold\">ation_utils_base.py</span>:<span style=\"color: #0000ff; text-decoration-color: #0000ff\">777</span> in <span style=\"color: #00ff00; text-decoration-color: #00ff00\">&lt;dictcomp&gt;</span>                                                            <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 774 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"># Otherwise it passes the casts down and casts the LongTensor containing the tok</span>  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 775 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"># into a HalfTensor</span>                                                               <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 776 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">if</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff\">isinstance</span>(device, <span style=\"color: #00ffff; text-decoration-color: #00ffff\">str</span>) <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">or</span> is_torch_device(device) <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">or</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff\">isinstance</span>(device, <span style=\"color: #00ffff; text-decoration-color: #00ffff\">int</span>)  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #800000; text-decoration-color: #800000\">❱ </span> 777 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   </span><span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>.data = {k: v.to(device=device) <span style=\"color: #0000ff; text-decoration-color: #0000ff\">for</span> k, v <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">in</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>.data.items()}            <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 778 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">else</span>:                                                                             <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 779 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   </span>logger.warning(<span style=\"color: #808000; text-decoration-color: #808000\">f\"Attempting to cast a BatchEncoding to type {</span><span style=\"color: #00ffff; text-decoration-color: #00ffff\">str</span>(device)<span style=\"color: #808000; text-decoration-color: #808000\">}. T</span>  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 780 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">return</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff\">self</span>                                                                       <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #bfbf7f; text-decoration-color: #bfbf7f\">/usr/local/Caskroom/miniconda/base/envs/llm310/lib/python3.10/site-packages/torch/cuda/</span><span style=\"color: #808000; text-decoration-color: #808000; font-weight: bold\">__init__.</span> <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #808000; text-decoration-color: #808000; font-weight: bold\">py</span>:<span style=\"color: #0000ff; text-decoration-color: #0000ff\">239</span> in <span style=\"color: #00ff00; text-decoration-color: #00ff00\">_lazy_init</span>                                                                             <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 236 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">\"Cannot re-initialize CUDA in forked subprocess. To use CUDA with \"</span>       <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 237 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">\"multiprocessing, you must use the 'spawn' start method\"</span>)                 <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 238 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">if</span> <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">not</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff\">hasattr</span>(torch._C, <span style=\"color: #808000; text-decoration-color: #808000\">'_cuda_getDeviceCount'</span>):                                 <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #800000; text-decoration-color: #800000\">❱ </span> 239 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">raise</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff\">AssertionError</span>(<span style=\"color: #808000; text-decoration-color: #808000\">\"Torch not compiled with CUDA enabled\"</span>)                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 240 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">if</span> _cudart <span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">is</span> <span style=\"color: #0000ff; text-decoration-color: #0000ff\">None</span>:                                                               <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 241 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">raise</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff\">AssertionError</span>(                                                         <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\"> 242 </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">\"libcudart functions unavailable. It looks like you have a broken build?</span>  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n",
       "<span style=\"color: #800000; text-decoration-color: #800000\">╰──────────────────────────────────────────────────────────────────────────────────────────────────╯</span>\n",
       "<span style=\"color: #ff0000; text-decoration-color: #ff0000; font-weight: bold\">AssertionError: </span>Torch not compiled with CUDA enabled\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[31m╭─\u001b[0m\u001b[31m──────────────────────────────\u001b[0m\u001b[31m \u001b[0m\u001b[1;31mTraceback \u001b[0m\u001b[1;2;31m(most recent call last)\u001b[0m\u001b[31m \u001b[0m\u001b[31m───────────────────────────────\u001b[0m\u001b[31m─╮\u001b[0m\n",
       "\u001b[31m│\u001b[0m in \u001b[92m<module>\u001b[0m:\u001b[94m4\u001b[0m                                                                                    \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m1 \u001b[0mquestion = \u001b[33m\"\u001b[0m\u001b[33m\\n\u001b[0m\u001b[33mContinue the following story: In a shocking finding, scientist discovered \u001b[0m     \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m2 \u001b[0mprompt = \u001b[33mf\u001b[0m\u001b[33m\"\u001b[0m\u001b[33m### Instruction: \u001b[0m\u001b[33m{\u001b[0mquestion\u001b[33m}\u001b[0m\u001b[33m\\n\u001b[0m\u001b[33m### Response:\u001b[0m\u001b[33m\"\u001b[0m                                       \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m3 \u001b[0m                                                                                             \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[31m❱ \u001b[0m4 inputs = tokenizer(prompt, return_tensors=\u001b[33m\"\u001b[0m\u001b[33mpt\u001b[0m\u001b[33m\"\u001b[0m).to(\u001b[94m0\u001b[0m)                                        \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m5 \u001b[0moutput = model.generate(inputs[\u001b[33m\"\u001b[0m\u001b[33minput_ids\u001b[0m\u001b[33m\"\u001b[0m], max_new_tokens=\u001b[94m100\u001b[0m)                             \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m6 \u001b[0m                                                                                             \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m7 \u001b[0m\u001b[96mprint\u001b[0m(tokenizer.decode(output[\u001b[94m0\u001b[0m].tolist(), skip_special_tokens=\u001b[94mTrue\u001b[0m))                        \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[2;33m/usr/local/Caskroom/miniconda/base/envs/llm310/lib/python3.10/site-packages/transformers/\u001b[0m\u001b[1;33mtokeniz\u001b[0m \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[1;33mation_utils_base.py\u001b[0m:\u001b[94m777\u001b[0m in \u001b[92mto\u001b[0m                                                                    \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 774 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[2m# Otherwise it passes the casts down and casts the LongTensor containing the tok\u001b[0m  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 775 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[2m# into a HalfTensor\u001b[0m                                                               \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 776 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[94mif\u001b[0m \u001b[96misinstance\u001b[0m(device, \u001b[96mstr\u001b[0m) \u001b[95mor\u001b[0m is_torch_device(device) \u001b[95mor\u001b[0m \u001b[96misinstance\u001b[0m(device, \u001b[96mint\u001b[0m)  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[31m❱ \u001b[0m 777 \u001b[2m│   │   │   \u001b[0m\u001b[96mself\u001b[0m.data = {k: v.to(device=device) \u001b[94mfor\u001b[0m k, v \u001b[95min\u001b[0m \u001b[96mself\u001b[0m.data.items()}            \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 778 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[94melse\u001b[0m:                                                                             \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 779 \u001b[0m\u001b[2m│   │   │   \u001b[0mlogger.warning(\u001b[33mf\u001b[0m\u001b[33m\"\u001b[0m\u001b[33mAttempting to cast a BatchEncoding to type \u001b[0m\u001b[33m{\u001b[0m\u001b[96mstr\u001b[0m(device)\u001b[33m}\u001b[0m\u001b[33m. T\u001b[0m  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 780 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[94mreturn\u001b[0m \u001b[96mself\u001b[0m                                                                       \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[2;33m/usr/local/Caskroom/miniconda/base/envs/llm310/lib/python3.10/site-packages/transformers/\u001b[0m\u001b[1;33mtokeniz\u001b[0m \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[1;33mation_utils_base.py\u001b[0m:\u001b[94m777\u001b[0m in \u001b[92m<dictcomp>\u001b[0m                                                            \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 774 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[2m# Otherwise it passes the casts down and casts the LongTensor containing the tok\u001b[0m  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 775 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[2m# into a HalfTensor\u001b[0m                                                               \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 776 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[94mif\u001b[0m \u001b[96misinstance\u001b[0m(device, \u001b[96mstr\u001b[0m) \u001b[95mor\u001b[0m is_torch_device(device) \u001b[95mor\u001b[0m \u001b[96misinstance\u001b[0m(device, \u001b[96mint\u001b[0m)  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[31m❱ \u001b[0m 777 \u001b[2m│   │   │   \u001b[0m\u001b[96mself\u001b[0m.data = {k: v.to(device=device) \u001b[94mfor\u001b[0m k, v \u001b[95min\u001b[0m \u001b[96mself\u001b[0m.data.items()}            \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 778 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[94melse\u001b[0m:                                                                             \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 779 \u001b[0m\u001b[2m│   │   │   \u001b[0mlogger.warning(\u001b[33mf\u001b[0m\u001b[33m\"\u001b[0m\u001b[33mAttempting to cast a BatchEncoding to type \u001b[0m\u001b[33m{\u001b[0m\u001b[96mstr\u001b[0m(device)\u001b[33m}\u001b[0m\u001b[33m. T\u001b[0m  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 780 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[94mreturn\u001b[0m \u001b[96mself\u001b[0m                                                                       \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[2;33m/usr/local/Caskroom/miniconda/base/envs/llm310/lib/python3.10/site-packages/torch/cuda/\u001b[0m\u001b[1;33m__init__.\u001b[0m \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[1;33mpy\u001b[0m:\u001b[94m239\u001b[0m in \u001b[92m_lazy_init\u001b[0m                                                                             \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 236 \u001b[0m\u001b[2m│   │   │   │   \u001b[0m\u001b[33m\"\u001b[0m\u001b[33mCannot re-initialize CUDA in forked subprocess. To use CUDA with \u001b[0m\u001b[33m\"\u001b[0m       \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 237 \u001b[0m\u001b[2m│   │   │   │   \u001b[0m\u001b[33m\"\u001b[0m\u001b[33mmultiprocessing, you must use the \u001b[0m\u001b[33m'\u001b[0m\u001b[33mspawn\u001b[0m\u001b[33m'\u001b[0m\u001b[33m start method\u001b[0m\u001b[33m\"\u001b[0m)                 \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 238 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[94mif\u001b[0m \u001b[95mnot\u001b[0m \u001b[96mhasattr\u001b[0m(torch._C, \u001b[33m'\u001b[0m\u001b[33m_cuda_getDeviceCount\u001b[0m\u001b[33m'\u001b[0m):                                 \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m \u001b[31m❱ \u001b[0m 239 \u001b[2m│   │   │   \u001b[0m\u001b[94mraise\u001b[0m \u001b[96mAssertionError\u001b[0m(\u001b[33m\"\u001b[0m\u001b[33mTorch not compiled with CUDA enabled\u001b[0m\u001b[33m\"\u001b[0m)                  \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 240 \u001b[0m\u001b[2m│   │   \u001b[0m\u001b[94mif\u001b[0m _cudart \u001b[95mis\u001b[0m \u001b[94mNone\u001b[0m:                                                               \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 241 \u001b[0m\u001b[2m│   │   │   \u001b[0m\u001b[94mraise\u001b[0m \u001b[96mAssertionError\u001b[0m(                                                         \u001b[31m│\u001b[0m\n",
       "\u001b[31m│\u001b[0m   \u001b[2m 242 \u001b[0m\u001b[2m│   │   │   │   \u001b[0m\u001b[33m\"\u001b[0m\u001b[33mlibcudart functions unavailable. It looks like you have a broken build?\u001b[0m  \u001b[31m│\u001b[0m\n",
       "\u001b[31m╰──────────────────────────────────────────────────────────────────────────────────────────────────╯\u001b[0m\n",
       "\u001b[1;91mAssertionError: \u001b[0mTorch not compiled with CUDA enabled\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "question = \"\\nContinue the following story: In a shocking finding, scientist discovered a herd of dragons living in a remote, previously unexplored valley, in Tibet. Even more surprising to the researchers was the fact that the dragons spoke perfect Chinese.\"\n",
    "prompt = f\"### Instruction: {question}\\n### Response:\"\n",
    "\n",
    "inputs = tokenizer(prompt, return_tensors=\"pt\").to(0)\n",
    "output = model.generate(inputs[\"input_ids\"], max_new_tokens=100)\n",
    "\n",
    "print(tokenizer.decode(output[0].tolist(), skip_special_tokens=True))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "llm310",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}