Demos/10. Inseq：可视化解释 LLM 的输出.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c48d6b26-1254-4e0a-bd10-d98284691c94",
   "metadata": {},
   "source": [
    "# Inseq：可视化解释LLM的输出\n",
    "\n",
    "> [HW7: Understand what Generative AI is thinking](https://colab.research.google.com/drive/1Xnz0GHC0yWO2Do0aAYBCq9zL45lbiRjM?usp=sharing#scrollTo=UFOUfh2k1jFN) 中文镜像版\n",
    ">\n",
    "> 指导文章：[12. Inseq 特征归因：可视化解释 LLM 的输出](https://github.com/Hoper-J/LLM-Guide-and-Demos-zh_CN/blob/master/Guide/12.%20Inseq%20特征归因：可视化解释%20LLM%20的输出.md)\n",
    "\n",
    "Feature attribution 特征归因，你可以将其当做对输出的解释，就像图像分类模型可以可视化关注区域一样，LLM 一样也可以。\n",
    "\n",
    "在线链接：[Kaggle](https://www.kaggle.com/code/aidemos/10-inseq-llm) | [Colab](https://colab.research.google.com/drive/1bWqGtRaG3aO7Vo149wIPHaz_XKnbJqlE?usp=sharing)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "46f87509-4dd8-4f1e-97b9-32f6c7c8c447",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "!pip install inseq\n",
    "!pip install transformers\n",
    "!pip install bitsandbytes\n",
    "!pip install accelerate\n",
    "!pip install sacremoses"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e0b2c5b0-3020-4995-8ac8-15181adb5cd3",
   "metadata": {
    "tags": []
   },
   "source": [
    "## 可视化翻译任务"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f11fd74d-d5ba-4f20-a120-7a41e0c56686",
   "metadata": {
    "tags": []
   },
   "source": [
    "### 下载中译英模型到本地\n",
    "\n",
    "我们这里使用多线程的方法进行快速下载。\n",
    "\n",
    "如果直接运行以下命令报错，根据[a. 使用 HFD 加快 Hugging Face 模型和数据集的下载](https://github.com/Hoper-J/LLM-Guide-and-Demos-zh_CN/blob/master/Guide/a.%20使用%20HFD%20加快%20Hugging%20Face%20模型和数据集的下载.md)进行前置安装。\n",
    "\n",
    "当然，你也可以取消我注释的部分，使用官方的命令进行安装，但是会很慢。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "16260d08-5e22-475e-8a35-d75e5aafd827",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "!wget https://hf-mirror.com/hfd/hfd.sh\n",
    "!chmod a+x hfd.sh"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d8658e4c-4a12-4ae2-ac7d-45062d53004f",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "!export HF_ENDPOINT=https://hf-mirror.com\n",
    "!./hfd.sh 'Helsinki-NLP/opus-mt-zh-en' --tool aria2c -x 16"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "d1c4335c-b975-4e67-b03d-54359dcbeb73",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "======= 归因方法: saliency =======\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Attributing with saliency...: 100%|██████████████████████████████████████| 10/10 [00:00<00:00, 69.63it/s]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<br/><b>0th instance:</b><br/>\n",
       "<html>\n",
       "<div id=\"bqavwrdohopsxcuckauy_viz_container\">\n",
       "    <div id=\"bqavwrdohopsxcuckauy_content\" style=\"padding:15px;border-style:solid;margin:5px;\">\n",
       "        <div id = \"bqavwrdohopsxcuckauy_saliency_plot_container\" class=\"bqavwrdohopsxcuckauy_viz_container\" style=\"display:block\">\n",
       "            \n",
       "<div id=\"nespnjewghrvsnorhwxx_saliency_plot\" class=\"nespnjewghrvsnorhwxx_viz_content\">\n",
       "    <div style=\"margin:5px;font-family:sans-serif;font-weight:bold;\">\n",
       "        <span style=\"font-size: 20px;\">Source Saliency Heatmap</span>\n",
       "        <br>\n",
       "        x: Generated tokens, y: Attributed tokens\n",
       "    </div>\n",
       "    \n",
       "<table border=\"1\" cellpadding=\"5\" cellspacing=\"5\"\n",
       "    style=\"overflow-x:scroll;display:block;\">\n",
       "    <tr><th></th>\n",
       "<th>I</th><th>like</th><th>machine</th><th>learning</th><th>and</th><th>artificial</th><th>intelligence</th><th>.</th><th>&lt;/s&gt;</th></tr><tr><th>我喜歡</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.46397306397306415)\">0.23</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.8975242622301447)\">0.438</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.16442859972271742)\">0.084</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.12501485442661908)\">0.064</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.3693800752624282)\">0.183</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.10136660724896006)\">0.052</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.1171321053673995)\">0.058</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.5112695583283818)\">0.25</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.39302832244008706)\">0.193</th></tr><tr><th>機器</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.3299663299663301)\">0.162</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.2747870865517925)\">0.135</th><th style=\"background:rgba(255.0, 13.0, 87.0, 1.0)\">0.491</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.4560903149138443)\">0.225</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.26690433749257286)\">0.13</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.13289760348583876)\">0.069</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.14078035254505847)\">0.07</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.2826698356110118)\">0.141</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.306318082788671)\">0.151</th></tr><tr><th>學</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.14866310160427795)\">0.074</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.10924935630817992)\">0.054</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.306318082788671)\">0.151</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.3772628243216479)\">0.188</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.14078035254505847)\">0.07</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.03</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.032</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.13289760348583876)\">0.069</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.1171321053673995)\">0.061</th></tr><tr><th>習</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.14078035254505847)\">0.07</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.15654585066349747)\">0.078</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.2117250940780353)\">0.106</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.5428005545652606)\">0.267</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.23537334125569415)\">0.118</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.10136660724896006)\">0.052</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.10924935630817992)\">0.055</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.16442859972271742)\">0.083</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.12501485442661908)\">0.064</th></tr><tr><th>和</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.13289760348583876)\">0.067</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.09348385818974037)\">0.047</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.04618736383442265)\">0.026</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.10136660724896006)\">0.05</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.32208358090711037)\">0.158</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06983561101208159)\">0.036</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.03</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.14078035254505847)\">0.073</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.14078035254505847)\">0.073</th></tr><tr><th>人工</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.1959595959595959)\">0.099</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.10136660724896006)\">0.052</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.07771836007130124)\">0.039</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.12501485442661908)\">0.063</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.2117250940780353)\">0.104</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.8186967716379481)\">0.399</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.41667656961774624)\">0.203</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.18019409784115661)\">0.091</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.22749059219647458)\">0.114</th></tr><tr><th>智慧</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.22749059219647458)\">0.113</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.13289760348583876)\">0.068</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.07771836007130124)\">0.039</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.14866310160427795)\">0.073</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.219607843137255)\">0.109</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.5585660526836996)\">0.274</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.889641513170925)\">0.437</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.219607843137255)\">0.111</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.306318082788671)\">0.153</th></tr><tr><th>。</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.23537334125569415)\">0.119</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.13289760348583876)\">0.066</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.02</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.04618736383442265)\">0.027</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.16442859972271742)\">0.083</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.08560110913052081)\">0.045</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.10924935630817992)\">0.057</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.23537334125569415)\">0.118</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.2511388393741335)\">0.124</th></tr><tr><th>&lt;/s&gt;</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.13289760348583876)\">0.066</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.12501485442661908)\">0.063</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.08560110913052081)\">0.044</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.08560110913052081)\">0.043</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.09348385818974037)\">0.046</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.08560110913052081)\">0.043</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.1171321053673995)\">0.058</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.12501485442661908)\">0.064</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.13289760348583876)\">0.068</th></tr><tr style=\"outline: thin solid\"><th><b>probability</b></th><th><b>0.767</b></th><th><b>0.589</b></th><th>0.425</th><th><b>0.671</b></th><th><b>0.856</b></th><th><b>0.647</b></th><th><b>0.866</b></th><th><b>0.891</b></th><th><b>0.896</b></th></table>\n",
       "</div>\n",
       "\n",
       "        </div>\n",
       "    </div>\n",
       "</div>\n",
       "</html>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "======= 归因方法: attention =======\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Attributing with attention...: 100%|████████████████████████████████████| 10/10 [00:00<00:00, 233.77it/s]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<br/><b>0th instance:</b><br/>\n",
       "<html>\n",
       "<div id=\"tmjvtvgtxirxuqbuyexi_viz_container\">\n",
       "    <div id=\"tmjvtvgtxirxuqbuyexi_content\" style=\"padding:15px;border-style:solid;margin:5px;\">\n",
       "        <div id = \"tmjvtvgtxirxuqbuyexi_saliency_plot_container\" class=\"tmjvtvgtxirxuqbuyexi_viz_container\" style=\"display:block\">\n",
       "            \n",
       "<div id=\"vnrirtxjxlhvuoiljqgq_saliency_plot\" class=\"vnrirtxjxlhvuoiljqgq_viz_content\">\n",
       "    <div style=\"margin:5px;font-family:sans-serif;font-weight:bold;\">\n",
       "        <span style=\"font-size: 20px;\">Source Saliency Heatmap</span>\n",
       "        <br>\n",
       "        x: Generated tokens, y: Attributed tokens\n",
       "    </div>\n",
       "    \n",
       "<table border=\"1\" cellpadding=\"5\" cellspacing=\"5\"\n",
       "    style=\"overflow-x:scroll;display:block;\">\n",
       "    <tr><th></th>\n",
       "<th>I</th><th>like</th><th>machine</th><th>learning</th><th>and</th><th>artificial</th><th>intelligence</th><th>.</th><th>&lt;/s&gt;</th></tr><tr><th>我喜歡</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.3693800752624282)\">0.19</th><th style=\"background:rgba(255.0, 13.0, 87.0, 1.0)\">0.517</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.5428005545652606)\">0.28</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.018</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.08560110913052081)\">0.048</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.032</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.02253911665676371)\">0.014</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.09348385818974037)\">0.049</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.023</th></tr><tr><th>機器</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.15654585066349747)\">0.084</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06983561101208159)\">0.037</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.44820756585462457)\">0.233</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.4009110714993067)\">0.208</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06983561101208159)\">0.038</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06983561101208159)\">0.036</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.024</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.04618736383442265)\">0.028</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.02253911665676371)\">0.015</th></tr><tr><th>學</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.02253911665676371)\">0.012</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.01</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.07771836007130124)\">0.044</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.24325609031491383)\">0.129</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.20384234501881549)\">0.105</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.018</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.00677361853832443)\">0.006</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.017</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.012</th></tr><tr><th>習</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.021</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.02253911665676371)\">0.015</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.12501485442661908)\">0.066</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.29843533372945136)\">0.155</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.20384234501881549)\">0.109</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.024</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.009</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.024</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.00677361853832443)\">0.007</th></tr><tr><th>和</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.10924935630817992)\">0.058</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.07771836007130124)\">0.042</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.1171321053673995)\">0.063</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06983561101208159)\">0.039</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.29055258467023165)\">0.152</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.3457318280847692)\">0.178</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.02253911665676371)\">0.015</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.034</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.036</th></tr><tr><th>人工</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.023</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.02253911665676371)\">0.015</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.022</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.04618736383442265)\">0.027</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.04618736383442265)\">0.025</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.5033868092691621)\">0.26</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.3536145771439889)\">0.183</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.033</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.02253911665676371)\">0.016</th></tr><tr><th>智慧</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.029</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.017</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.033</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.08560110913052081)\">0.046</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06983561101208159)\">0.04</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.2511388393741335)\">0.132</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.7083382848088731)\">0.365</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.29843533372945136)\">0.155</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.02253911665676371)\">0.013</th></tr><tr><th>。</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.29055258467023165)\">0.153</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.16442859972271742)\">0.087</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.08560110913052081)\">0.047</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.019</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.10924935630817992)\">0.059</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.14078035254505847)\">0.076</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06983561101208159)\">0.037</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.3142008318478907)\">0.163</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.7004555357496535)\">0.363</th></tr><tr><th>&lt;/s&gt;</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.8344622697563875)\">0.43</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.5033868092691621)\">0.262</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.4087938205585263)\">0.212</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.6925727866904339)\">0.359</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.8265795206971679)\">0.425</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.4718558130322837)\">0.245</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.6768072885719946)\">0.348</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.9605862547039017)\">0.496</th><th style=\"background:rgba(255.0, 13.0, 87.0, 1.0)\">0.516</th></tr><tr style=\"outline: thin solid\"><th><b>probability</b></th><th><b>0.767</b></th><th><b>0.589</b></th><th>0.425</th><th><b>0.671</b></th><th><b>0.856</b></th><th><b>0.647</b></th><th><b>0.866</b></th><th><b>0.891</b></th><th><b>0.896</b></th></table>\n",
       "</div>\n",
       "\n",
       "        </div>\n",
       "    </div>\n",
       "</div>\n",
       "</html>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import inseq\n",
    "\n",
    "# 定义要使用的归因方法列表\n",
    "attribution_methods = ['saliency', 'attention']\n",
    "\n",
    "for method in attribution_methods:\n",
    "    print(f\"======= 归因方法: {method} =======\")\n",
    "    # 加载中译英模型并设置归因方法\n",
    "    # model = inseq.load_model(\"Helsinki-NLP/opus-mt-zh-en\", method)\n",
    "    model = inseq.load_model(\"opus-mt-zh-en\", method)  # 导入之前下载到本地的模型\n",
    "\n",
    "    # 使用指定的归因方法对输入文本进行归因\n",
    "    attribution_result = model.attribute(\n",
    "        input_texts=\"我喜歡機器學習和人工智慧。\",\n",
    "    )\n",
    "\n",
    "    # 从tokenizer中去除 '▁' 前缀以避免混淆（可以忽略这段代码）\n",
    "    for attr in attribution_result.sequence_attributions:\n",
    "        for item in attr.source:\n",
    "            item.token = item.token.replace('▁', '')\n",
    "        for item in attr.target:\n",
    "            item.token = item.token.replace('▁', '')\n",
    "\n",
    "    # 显示归因结果\n",
    "    attribution_result.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9a7943ba-3941-46db-b327-f31d12433a2e",
   "metadata": {
    "tags": []
   },
   "source": [
    "## 可视化文本生成任务"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a774c635-196d-4001-aa78-d71cf7beb201",
   "metadata": {
    "tags": []
   },
   "source": [
    "### 下载 GPT-2 XL 模型\n",
    "\n",
    "如果只是为了查看的话，可以取消注释部分下载 GPT-2，因为 GPT-2 XL 下载完需要占用 30G，记得注释原来的部分。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bb94fb14-4300-48b5-a419-76d21dc3a1ad",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "!export HF_ENDPOINT=https://hf-mirror.com\n",
    "!./hfd.sh 'gpt2-xl' --tool aria2c -x 16\n",
    "#!./hfd.sh 'gpt2' --tool aria2c -x 16"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "9fdcb44e-9605-40b5-afa2-e32cd580b836",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from transformers import AutoModelForCausalLM, BitsAndBytesConfig\n",
    "\n",
    "# 创建一个 BitsAndBytesConfig 对象，用于配置量化选项\n",
    "bnb_config = BitsAndBytesConfig(load_in_8bit=True)\n",
    "model = AutoModelForCausalLM.from_pretrained(\n",
    "    \"gpt2-xl\",  # gpt2\n",
    "    quantization_config=bnb_config,\n",
    "    device_map={\"\": 0}\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "a7d80a80-641a-46f3-baf6-9a4e7c624a5c",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "The model is loaded in 8bit mode. The device cannot be changed after loading the model.\n",
      "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n",
      "Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "======= 归因方法: saliency =======\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Attributing with saliency...: 100%|██████████████████████████████████████| 20/20 [00:04<00:00,  3.89it/s]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<br/><b>0th instance:</b><br/>\n",
       "<html>\n",
       "<div id=\"kvnkstxjnlltfqypzzwg_viz_container\">\n",
       "    <div id=\"kvnkstxjnlltfqypzzwg_content\" style=\"padding:15px;border-style:solid;margin:5px;\">\n",
       "        <div id = \"kvnkstxjnlltfqypzzwg_saliency_plot_container\" class=\"kvnkstxjnlltfqypzzwg_viz_container\" style=\"display:block\">\n",
       "            \n",
       "<div id=\"lzgxoakewpcefkxhkbtv_saliency_plot\" class=\"lzgxoakewpcefkxhkbtv_viz_content\">\n",
       "    <div style=\"margin:5px;font-family:sans-serif;font-weight:bold;\">\n",
       "        <span style=\"font-size: 20px;\">Target Saliency Heatmap</span>\n",
       "        <br>\n",
       "        x: Generated tokens, y: Attributed tokens\n",
       "    </div>\n",
       "    \n",
       "<table border=\"1\" cellpadding=\"5\" cellspacing=\"5\"\n",
       "    style=\"overflow-x:scroll;display:block;\">\n",
       "    <tr><th></th>\n",
       "<th>\"</th><th>Ċ</th><th>Ċ</th><th>The</th><th>first</th><th>thing</th><th>you</th><th>'ll</th><th>notice</th><th>is</th><th>that</th><th>the</th><th>code</th><th>is</th><th>a</th><th>bit</th><th>more</th><th>verb</th></tr><tr><th>Hello</th><th style=\"background:rgba(255.0, 13.0, 87.0, 1.0)\">0.647</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.7871657754010695)\">0.505</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.5585660526836996)\">0.359</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.621628045157457)\">0.401</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.5979797979797981)\">0.388</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.5349178055060405)\">0.344</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.5428005545652606)\">0.35</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.4009110714993067)\">0.261</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.4009110714993067)\">0.261</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.46397306397306415)\">0.302</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.4718558130322837)\">0.305</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.3536145771439889)\">0.232</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.5664488017429193)\">0.368</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.3457318280847692)\">0.224</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.44032481679540497)\">0.283</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.36149732620320857)\">0.236</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.29843533372945136)\">0.194</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.3693800752624282)\">0.241</th></tr><tr><th>world</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.5428005545652606)\">0.353</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.48762131115072294)\">0.315</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.3457318280847692)\">0.223</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.3693800752624282)\">0.238</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.3851455733808676)\">0.249</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.306318082788671)\">0.2</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.3693800752624282)\">0.242</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.26690433749257286)\">0.176</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.2590215884333532)\">0.169</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.32208358090711037)\">0.212</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.306318082788671)\">0.197</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.2511388393741335)\">0.165</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.3772628243216479)\">0.246</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.22749059219647458)\">0.149</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.2511388393741335)\">0.164</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.23537334125569415)\">0.153</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.1880768469003763)\">0.122</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.23537334125569415)\">0.152</th></tr><tr><th>\"</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.2747870865517925)\">0.18</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.2826698356110118)\">0.183</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.24325609031491383)\">0.159</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.20384234501881549)\">0.133</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.1959595959595959)\">0.128</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.15654585066349747)\">0.103</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.14078035254505847)\">0.093</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.12501485442661908)\">0.084</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.14078035254505847)\">0.096</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.10924935630817992)\">0.073</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.10924935630817992)\">0.071</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.043</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.08560110913052081)\">0.058</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.09348385818974037)\">0.064</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.10136660724896006)\">0.068</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06983561101208159)\">0.047</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.07771836007130124)\">0.052</th></tr><tr><th>Ċ</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.36149732620320857)\">0.235</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.14078035254505847)\">0.092</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.08560110913052081)\">0.058</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.08560110913052081)\">0.056</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.044</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.045</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.044</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.042</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.039</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.09348385818974037)\">0.065</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.036</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.042</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.043</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.04618736383442265)\">0.033</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.025</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.027</th></tr><tr><th>Ċ</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.16442859972271742)\">0.11</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.08560110913052081)\">0.058</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.09348385818974037)\">0.062</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06983561101208159)\">0.046</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.07771836007130124)\">0.053</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.07771836007130124)\">0.052</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.041</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.04618736383442265)\">0.035</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.04</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.025</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.037</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.027</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.028</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.025</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.024</th></tr><tr><th>The</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.17231134878193693)\">0.114</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.10924935630817992)\">0.072</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.10136660724896006)\">0.066</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.08560110913052081)\">0.059</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.045</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.04</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06983561101208159)\">0.045</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.041</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.02253911665676371)\">0.02</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.041</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.03</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.037</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.025</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.027</th></tr><tr><th>first</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.2117250940780353)\">0.138</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06983561101208159)\">0.048</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.10924935630817992)\">0.074</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.08560110913052081)\">0.06</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06983561101208159)\">0.046</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06983561101208159)\">0.047</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.04</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.021</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.07771836007130124)\">0.051</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.027</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.039</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.037</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.027</th></tr><tr><th>thing</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.15654585066349747)\">0.101</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.15654585066349747)\">0.101</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.13289760348583876)\">0.087</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06983561101208159)\">0.048</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06983561101208159)\">0.048</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.07771836007130124)\">0.053</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.028</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.043</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.037</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06983561101208159)\">0.048</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.037</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.04618736383442265)\">0.032</th></tr><tr><th>you</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.2117250940780353)\">0.137</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.12501485442661908)\">0.082</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06983561101208159)\">0.047</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06983561101208159)\">0.048</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06983561101208159)\">0.049</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.028</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.043</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.044</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.07771836007130124)\">0.054</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.036</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.04618736383442265)\">0.035</th></tr><tr><th>'ll</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.18019409784115661)\">0.117</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06983561101208159)\">0.047</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.07771836007130124)\">0.051</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.09348385818974037)\">0.064</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.04618736383442265)\">0.032</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.07771836007130124)\">0.052</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06983561101208159)\">0.049</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.09348385818974037)\">0.063</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.043</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.042</th></tr><tr><th>notice</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.1171321053673995)\">0.079</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.10924935630817992)\">0.074</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.12501485442661908)\">0.084</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.07771836007130124)\">0.055</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.10136660724896006)\">0.069</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.08560110913052081)\">0.06</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.09348385818974037)\">0.063</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.09348385818974037)\">0.065</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06983561101208159)\">0.048</th></tr><tr><th>is</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.037</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.036</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.02253911665676371)\">0.02</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.022</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.026</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.02253911665676371)\">0.018</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.028</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.022</th></tr><tr><th>that</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.09348385818974037)\">0.061</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.022</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.04618736383442265)\">0.031</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.025</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.021</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.04618736383442265)\">0.031</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.02</th></tr><tr><th>the</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.08560110913052081)\">0.056</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.029</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.027</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.023</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.026</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.027</th></tr><tr><th>code</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.16442859972271742)\">0.111</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.08560110913052081)\">0.058</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.07771836007130124)\">0.055</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.12501485442661908)\">0.082</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.13289760348583876)\">0.088</th></tr><tr><th>is</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.036</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.02253911665676371)\">0.016</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.025</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.014</th></tr><tr><th>a</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.044</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.027</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.013</th></tr><tr><th>bit</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.1880768469003763)\">0.125</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06983561101208159)\">0.048</th></tr><tr><th>more</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.08560110913052081)\">0.059</th></tr><tr><th>verb</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th></tr><tr style=\"outline: thin solid\"><th><b>probability</b></th><th>0.197</th><th>0.097</th><th><b>0.988</b></th><th>0.043</th><th>0.046</th><th>0.211</th><th>0.25</th><th>0.248</th><th><b>0.52</b></th><th><b>0.557</b></th><th><b>0.662</b></th><th>0.257</th><th>0.046</th><th>0.483</th><th>0.106</th><th>0.438</th><th>0.207</th><th>0.259</th></table>\n",
       "</div>\n",
       "\n",
       "        </div>\n",
       "    </div>\n",
       "</div>\n",
       "</html>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "The model is loaded in 8bit mode. The device cannot be changed after loading the model.\n",
      "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n",
      "Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "======= 归因方法: attention =======\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Attributing with attention...: 100%|█████████████████████████████████████| 20/20 [00:01<00:00, 14.14it/s]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<br/><b>0th instance:</b><br/>\n",
       "<html>\n",
       "<div id=\"sdcxufjthkesmkfnrhkg_viz_container\">\n",
       "    <div id=\"sdcxufjthkesmkfnrhkg_content\" style=\"padding:15px;border-style:solid;margin:5px;\">\n",
       "        <div id = \"sdcxufjthkesmkfnrhkg_saliency_plot_container\" class=\"sdcxufjthkesmkfnrhkg_viz_container\" style=\"display:block\">\n",
       "            \n",
       "<div id=\"hfqjgynzxlspvhfngeum_saliency_plot\" class=\"hfqjgynzxlspvhfngeum_viz_content\">\n",
       "    <div style=\"margin:5px;font-family:sans-serif;font-weight:bold;\">\n",
       "        <span style=\"font-size: 20px;\">Target Saliency Heatmap</span>\n",
       "        <br>\n",
       "        x: Generated tokens, y: Attributed tokens\n",
       "    </div>\n",
       "    \n",
       "<table border=\"1\" cellpadding=\"5\" cellspacing=\"5\"\n",
       "    style=\"overflow-x:scroll;display:block;\">\n",
       "    <tr><th></th>\n",
       "<th>\"</th><th>Ċ</th><th>Ċ</th><th>The</th><th>first</th><th>thing</th><th>you</th><th>'ll</th><th>notice</th><th>is</th><th>that</th><th>the</th><th>code</th><th>is</th><th>a</th><th>bit</th><th>more</th><th>verb</th></tr><tr><th>Hello</th><th style=\"background:rgba(255.0, 13.0, 87.0, 1.0)\">0.867</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.8738760150524857)\">0.758</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.77928302634185)\">0.675</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.7162210338680925)\">0.619</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.7162210338680925)\">0.619</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.7319865319865321)\">0.636</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.7083382848088731)\">0.611</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.7241037829273124)\">0.626</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.7162210338680925)\">0.623</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.6768072885719946)\">0.586</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.5979797979797981)\">0.518</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.6058625470390177)\">0.527</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.6373935432758963)\">0.551</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.6610417904535549)\">0.571</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.6373935432758963)\">0.555</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.6689245395127748)\">0.577</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.6531590413943356)\">0.563</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.6452762923351159)\">0.556</th></tr><tr><th>world</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.14866310160427795)\">0.133</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.12501485442661908)\">0.109</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.054</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.07771836007130124)\">0.068</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.055</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.038</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.04618736383442265)\">0.041</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.03</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.02253911665676371)\">0.024</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.028</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.037</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.04</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.034</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.02253911665676371)\">0.024</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.015</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.015</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.00677361853832443)\">0.013</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.00677361853832443)\">0.012</th></tr><tr><th>\"</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.14866310160427795)\">0.133</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.09348385818974037)\">0.086</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.09348385818974037)\">0.084</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06983561101208159)\">0.062</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.048</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.04618736383442265)\">0.043</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.037</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.031</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.03</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.035</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.037</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.035</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.02253911665676371)\">0.025</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.019</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.018</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.016</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.016</th></tr><tr><th>Ċ</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.2117250940780353)\">0.185</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.12501485442661908)\">0.114</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.10136660724896006)\">0.092</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.08560110913052081)\">0.075</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.07771836007130124)\">0.068</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.061</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.054</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.059</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.058</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.052</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.04618736383442265)\">0.046</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.04618736383442265)\">0.042</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.038</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.034</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.032</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.033</th></tr><tr><th>Ċ</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.13289760348583876)\">0.116</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.10136660724896006)\">0.093</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.07771836007130124)\">0.072</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06983561101208159)\">0.065</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.049</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.04618736383442265)\">0.041</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.04618736383442265)\">0.043</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.056</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.054</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.041</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.03</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.02253911665676371)\">0.024</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.02253911665676371)\">0.021</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.02</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.02</th></tr><tr><th>The</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.08560110913052081)\">0.08</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.08560110913052081)\">0.078</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.061</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.04618736383442265)\">0.041</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.034</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.033</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.04618736383442265)\">0.043</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.036</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.031</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.02253911665676371)\">0.025</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.019</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.017</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.016</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.016</th></tr><tr><th>first</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.055</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.059</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.037</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.031</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.032</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.02253911665676371)\">0.026</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.02253911665676371)\">0.022</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.02</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.014</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.00677361853832443)\">0.013</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.00677361853832443)\">0.012</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.00677361853832443)\">0.012</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.00677361853832443)\">0.011</th></tr><tr><th>thing</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.052</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.058</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.04618736383442265)\">0.045</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.04618736383442265)\">0.041</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.02253911665676371)\">0.027</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.02</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.017</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.014</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.00677361853832443)\">0.013</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.00677361853832443)\">0.011</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.00677361853832443)\">0.011</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.00677361853832443)\">0.009</th></tr><tr><th>you</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06983561101208159)\">0.062</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.06</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.04</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.03</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.02253911665676371)\">0.022</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.018</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.018</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.00677361853832443)\">0.013</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.00677361853832443)\">0.011</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.00677361853832443)\">0.01</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.00677361853832443)\">0.009</th></tr><tr><th>'ll</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.057</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.048</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.02253911665676371)\">0.027</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.02253911665676371)\">0.02</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.015</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.017</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.014</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.00677361853832443)\">0.011</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.00677361853832443)\">0.01</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.00677361853832443)\">0.009</th></tr><tr><th>notice</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.059</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06983561101208159)\">0.065</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.04</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.028</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.02253911665676371)\">0.025</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.02253911665676371)\">0.025</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.017</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.018</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.016</th></tr><tr><th>is</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.08560110913052081)\">0.08</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.054</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.031</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.02253911665676371)\">0.026</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.03</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.019</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.02</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.014656367597544035)\">0.018</th></tr><tr><th>that</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.08560110913052081)\">0.075</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06983561101208159)\">0.063</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.048</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.04618736383442265)\">0.041</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.03</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.032</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.028</th></tr><tr><th>the</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.07771836007130124)\">0.071</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.059</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.039</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.03</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.030421865715983164)\">0.029</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.02253911665676371)\">0.026</th></tr><tr><th>code</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06983561101208159)\">0.062</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.07771836007130124)\">0.071</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.052</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.048</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.04618736383442265)\">0.045</th></tr><tr><th>is</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.07771836007130124)\">0.069</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.059</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06195286195286207)\">0.058</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.048</th></tr><tr><th>a</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.06983561101208159)\">0.067</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.049</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.03830461477520289)\">0.038</th></tr><tr><th>bit</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.04618736383442265)\">0.044</th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.04618736383442265)\">0.042</th></tr><tr><th>more</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(255.0, 13.0, 87.0, 0.05407011289364243)\">0.049</th></tr><tr><th>verb</th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th><th style=\"background:rgba(0.0, 0.0, 0.0, 0.0)\"></th></tr><tr style=\"outline: thin solid\"><th><b>probability</b></th><th>0.197</th><th>0.097</th><th><b>0.988</b></th><th>0.043</th><th>0.046</th><th>0.211</th><th>0.25</th><th>0.248</th><th><b>0.52</b></th><th><b>0.557</b></th><th><b>0.662</b></th><th>0.257</th><th>0.046</th><th>0.483</th><th>0.106</th><th>0.438</th><th>0.207</th><th>0.259</th></table>\n",
       "</div>\n",
       "\n",
       "        </div>\n",
       "    </div>\n",
       "</div>\n",
       "</html>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import inseq\n",
    "\n",
    "# 定义要使用的归因方法列表\n",
    "attribution_methods = ['saliency', 'attention']\n",
    "\n",
    "for method in attribution_methods:\n",
    "    print(f\"======= 归因方法: {method} =======\")\n",
    "    # 使用指定的归因方法加载模型\n",
    "    inseq_model = inseq.load_model(model, method)\n",
    "\n",
    "    # 对输入文本进行归因分析\n",
    "    attribution_result = inseq_model.attribute(\n",
    "        input_texts=\"Hello world\",\n",
    "        step_scores=[\"probability\"],\n",
    "    )\n",
    "\n",
    "    # 清理 tokenizer 中的特殊字符（可选）\n",
    "    for attr in attribution_result.sequence_attributions:\n",
    "        for item in attr.source:\n",
    "            item.token = item.token.replace('Ġ', '')\n",
    "        for item in attr.target:\n",
    "            item.token = item.token.replace('Ġ', '')\n",
    "\n",
    "    # 显示归因结果\n",
    "    attribution_result.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2bd7a155-5776-44e9-b0ec-a6e1220c3ee7",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}