Training_You.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Training You",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyOvPZAwQypoWmGilHbvmYBu",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/nuwandavek/you/blob/master/Training_You.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Rm5EzwNPr8w0"
      },
      "source": [
        "## Setup"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lTGlWqio-jFO"
      },
      "source": [
        "First, connect to a GPU runtime via Edit->Notebook Settings and select GPU as the hardare accelerator. Then, run the block below to install the libraries required to fine-tune DistilGPT2 on your WhatsApp history."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "cellView": "form",
        "id": "vTVbuOL6Xsx3"
      },
      "source": [
        "#@title Install libraries\n",
        "!pip install transformers\n",
        "!git clone https://github.com/huggingface/transformers.git\n",
        "!pip install ./transformers\n",
        "!pip install -r ./transformers/examples/language-modeling/requirements.txt\n",
        "!mkdir output"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Rb6HmORjsKtw"
      },
      "source": [
        "## Upload files for training"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V8zKaOmC_M2x"
      },
      "source": [
        "In order to train the model on your chat history, first export your chat history in the form of txt files (instructions [here](https://faq.whatsapp.com/android/chats/how-to-save-your-chat-history/?lang=en)). Then, run the following block and upload all your txt files."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Vfrb4h88z_mf",
        "cellView": "form"
      },
      "source": [
        "#@title Upload WhatsApp history files\n",
        "import re\n",
        "\n",
        "def RemoveTimestamps(text):\n",
        "  return re.sub(b'\\d+/\\d+/\\d+.*-\\ ', b'', text)\n",
        "\n",
        "def UnicodeString(bytes_string):\n",
        "  return bytes_string.decode('utf-8')\n",
        "\n",
        "def AddSeparators(file_text):\n",
        "  return b'#\\n'.join(file_text.split(b'\\n'))\n",
        "\n",
        "CHUNK_LENGTH = 500\n",
        "def ChunkFile(file_text):\n",
        "  lines = file_text.split(b'\\n')\n",
        "  chunks = []\n",
        "  for line_index in range(0, len(lines), CHUNK_LENGTH):\n",
        "    chunk = b'\\n'.join(lines[line_index:line_index+CHUNK_LENGTH])\n",
        "    chunk += b'<|endoftext|>'\n",
        "    chunks.append(chunk)\n",
        "  return chunks\n",
        "\n",
        "from itertools import chain\n",
        "import random\n",
        "def MixChunks(chunked_files):\n",
        "  all_chunks = [chunk for chunked_file in chunked_files for chunk in chunked_file]\n",
        "  random.shuffle(all_chunks)\n",
        "  return all_chunks\n",
        "\n",
        "def ConvertChunksToString(chunks):\n",
        "  return b'\\n'.join(chunks)\n",
        "\n",
        "def GetShuffledAndCleanedTextFromFiles(file_contents):\n",
        "  file_chunks = []\n",
        "  for file_content in file_contents:\n",
        "    file_chunks.append(ChunkFile(AddSeparators(RemoveTimestamps(file_content))))\n",
        "  return ConvertChunksToString(MixChunks(file_chunks))\n",
        "\n",
        "import random\n",
        "\n",
        "def SampleTextFromFile(file):\n",
        "  file_contents = open(file).readlines()\n",
        "  begin = random.randint(0, len(file_contents) - 50)\n",
        "  for line in file_contents[begin:begin+50]:\n",
        "    print(line, end='')\n",
        "\n",
        "from google.colab import files\n",
        "uploaded_files = files.upload()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Aw8e-Hr_gWB_"
      },
      "source": [
        "## Construct training data"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iOh8A-VNgh2P"
      },
      "source": [
        "Next, we clean up the data and prep it for training."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cDPYjntwb2US",
        "cellView": "form"
      },
      "source": [
        "#@title Clean data and create train and test splits.\n",
        "cleaned_text = GetShuffledAndCleanedTextFromFiles(uploaded_files.values())\n",
        "data_file = open('data.txt', 'wb')\n",
        "data_file.write(cleaned_text)\n",
        "data_file.close()\n",
        "num_lines = cleaned_text.count(b'\\n')\n",
        "test_size = int(0.1 * num_lines)\n",
        "train_size = num_lines - test_size\n",
        "data_file.close()\n",
        "!tail -n {test_size} data.txt > test.txt\n",
        "!head -n {train_size} data.txt > train.txt"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4QyQGDQ8gmlR"
      },
      "source": [
        "We can sample chunks from the training data file to inspect it. Note that a '#' token has been added at the ends of messages, and a <|endoftext|> token delineates different chat files."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ejxPLIfHIMuy",
        "outputId": "1c86cef4-498d-4b45-8d69-1fdb3dc3d143"
      },
      "source": [
        "SampleTextFromFile('train.txt')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Vivek: Can transfer after one hour of adding#\n",
            "Vivek: What verification thing?#\n",
            "Sreejith2: The bank account should have 40 lakhs thing#\n",
            "Sreejith2: Keep it and transfer after no?#\n",
            "Vivek: Yoyo all that is over \\m/#\n",
            "Sreejith2: Wooh!#\n",
            "Sreejith2: Peace peace#\n",
            "Vivek: That was required before visa#\n",
            "Sreejith2: Transfer off then#\n",
            "Vivek: Now peacemax#\n",
            "Vivek: 😅#\n",
            "Vivek: Yoyoyo#\n",
            "Sreejith2: Hahaha nice nice!#\n",
            "Vivek: What plans today?#\n",
            "Vivek: Free for a call?#\n",
            "Sreejith2: Hey, no plans as such#\n",
            "Sreejith2: Yo in 5 mins#\n",
            "Vivek: Yoyo#\n",
            "Vivek: Ping me#\n",
            "Sreejith2: Haan#\n",
            "Vivek: Eyo#\n",
            "Vivek: I sent 1000 rs#\n",
            "Vivek: Got that?#\n",
            "Vivek: Once you confirm I'll transfer the rest#\n",
            "Sreejith2: Hey got#\n",
            "Sreejith2: Got 1k#\n",
            "Vivek: Yoyoyo#<|endoftext|>\n",
            "Himaya: hope I have a good day#\n",
            "Mihir London: https://player.vimeo.com/video/427943452#\n",
            "Mihir London: Wtf#\n",
            "Sreejith2: Wow that's amazing 🤯#\n",
            "Rishi Amreeka: https://youtu.be/fZSFNUT6iY8#\n",
            "Rishi Amreeka: Have you guys seen this one?#\n",
            "Rishi Amreeka: Pretty insane#\n",
            "Sreejith2: Wow, wtf!#\n",
            "Sreejith2: Bet this would win pioneer easy 😅#\n",
            "Vivek: Are we playing tomorrow?#\n",
            "Vivek: :)#\n",
            "Vivek: Wtfffff#\n",
            "Vikrant London: They also released an API yesterday https://beta.openai.com/#\n",
            "Vivek: Yeah, I saw this.#\n",
            "Vikrant London: I'm down 👻#\n",
            "Rishi Amreeka: I won't be able to make it in the for the first 4 hours#\n",
            "Vivek: Whats the start time?#\n",
            "Vivek: Also I've to do some work in the morning. Shall we start at 11am ish? Like last week?#\n",
            "Vivek: Sreejith wakes up at that time too I think#\n",
            "Sreejith2: Works#\n",
            "Sreejith2: 11:30 pm indian time iiuc#\n",
            "Sreejith2: What time can you make it?#\n",
            "Vikrant London: I dreamt about you, and in my dream you said the same thing#\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xQYhEC7-uRZ6"
      },
      "source": [
        "## Train model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CrWK0ONLhcqT"
      },
      "source": [
        "Next, we fine-tune the DistilGPT2 model on our training data. Depending on how many files you uploaded, this could take between 5-30 minutes."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "x8KUxgU6_mjl",
        "outputId": "b8392a3a-423b-4a90-f380-bd2f0bc2807d"
      },
      "source": [
        "!python ./transformers/examples/language-modeling/run_clm.py --model_name_or_path distilgpt2 --train_file train.txt --validation_file test.txt --do_train --do_eval --output_dir ./output --per_gpu_train_batch_size 1 --per_gpu_eval_batch_size 1 --save_steps 800 --eval_steps 800 --logging_steps 800 --evaluation_strategy steps --overwrite_output_dir --block_size 256"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "2021-01-05 07:35:25.320305: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.1\n",
            "01/05/2021 07:35:27 - WARNING - __main__ -   Process rank: -1, device: cuda:0, n_gpu: 1distributed training: False, 16-bits training: False\n",
            "01/05/2021 07:35:27 - INFO - __main__ -   Training/evaluation parameters TrainingArguments(output_dir=./output, overwrite_output_dir=True, do_train=True, do_eval=True, do_predict=False, model_parallel=False, evaluation_strategy=EvaluationStrategy.STEPS, prediction_loss_only=False, per_device_train_batch_size=8, per_device_eval_batch_size=8, gradient_accumulation_steps=1, eval_accumulation_steps=None, learning_rate=5e-05, weight_decay=0.0, adam_beta1=0.9, adam_beta2=0.999, adam_epsilon=1e-08, max_grad_norm=1.0, num_train_epochs=3.0, max_steps=-1, lr_scheduler_type=SchedulerType.LINEAR, warmup_steps=0, logging_dir=runs/Jan05_07-35-27_6201cf33901f, logging_first_step=False, logging_steps=800, save_steps=800, save_total_limit=None, no_cuda=False, seed=42, fp16=False, fp16_opt_level=O1, local_rank=-1, tpu_num_cores=None, tpu_metrics_debug=False, debug=False, dataloader_drop_last=False, eval_steps=800, dataloader_num_workers=0, past_index=-1, run_name=./output, disable_tqdm=False, remove_unused_columns=True, label_names=None, load_best_model_at_end=False, metric_for_best_model=None, greater_is_better=None, ignore_data_skip=False, fp16_backend=auto, sharded_ddp=False, label_smoothing_factor=0.0, adafactor=False)\n",
            "Downloading: 2.57kB [00:00, 3.17MB/s]       \n",
            "Using custom data configuration default\n",
            "Downloading and preparing dataset text/default-ead5d6d2afdaed66 (download: Unknown size, generated: Unknown size, post-processed: Unknown size, total: Unknown size) to /root/.cache/huggingface/datasets/text/default-ead5d6d2afdaed66/0.0.0/daf90a707a433ac193b369c8cc1772139bb6cca21a9c7fe83bdd16aad9b9b6ab...\n",
            "Dataset text downloaded and prepared to /root/.cache/huggingface/datasets/text/default-ead5d6d2afdaed66/0.0.0/daf90a707a433ac193b369c8cc1772139bb6cca21a9c7fe83bdd16aad9b9b6ab. Subsequent calls will reuse this data.\n",
            "01/05/2021 07:35:28 - INFO - filelock -   Lock 139827271320912 acquired on /root/.cache/huggingface/transformers/f985248d2791fcff97732e4ee263617adec1edb5429a2b8421734c6d14e39bee.422318838d1ec4e061efb4ea29671cb2a044e244dc69229682bebd7cacc81631.lock\n",
            "[INFO|file_utils.py:1334] 2021-01-05 07:35:28,948 >> https://huggingface.co/distilgpt2/resolve/main/config.json not found in cache or force_download set to True, downloading to /root/.cache/huggingface/transformers/tmp55o8lq_4\n",
            "Downloading: 100% 762/762 [00:00<00:00, 636kB/s]\n",
            "[INFO|file_utils.py:1338] 2021-01-05 07:35:29,214 >> storing https://huggingface.co/distilgpt2/resolve/main/config.json in cache at /root/.cache/huggingface/transformers/f985248d2791fcff97732e4ee263617adec1edb5429a2b8421734c6d14e39bee.422318838d1ec4e061efb4ea29671cb2a044e244dc69229682bebd7cacc81631\n",
            "[INFO|file_utils.py:1341] 2021-01-05 07:35:29,214 >> creating metadata file for /root/.cache/huggingface/transformers/f985248d2791fcff97732e4ee263617adec1edb5429a2b8421734c6d14e39bee.422318838d1ec4e061efb4ea29671cb2a044e244dc69229682bebd7cacc81631\n",
            "01/05/2021 07:35:29 - INFO - filelock -   Lock 139827271320912 released on /root/.cache/huggingface/transformers/f985248d2791fcff97732e4ee263617adec1edb5429a2b8421734c6d14e39bee.422318838d1ec4e061efb4ea29671cb2a044e244dc69229682bebd7cacc81631.lock\n",
            "[INFO|configuration_utils.py:431] 2021-01-05 07:35:29,215 >> loading configuration file https://huggingface.co/distilgpt2/resolve/main/config.json from cache at /root/.cache/huggingface/transformers/f985248d2791fcff97732e4ee263617adec1edb5429a2b8421734c6d14e39bee.422318838d1ec4e061efb4ea29671cb2a044e244dc69229682bebd7cacc81631\n",
            "[INFO|configuration_utils.py:467] 2021-01-05 07:35:29,216 >> Model config GPT2Config {\n",
            "  \"_num_labels\": 1,\n",
            "  \"activation_function\": \"gelu_new\",\n",
            "  \"architectures\": [\n",
            "    \"GPT2LMHeadModel\"\n",
            "  ],\n",
            "  \"attn_pdrop\": 0.1,\n",
            "  \"bos_token_id\": 50256,\n",
            "  \"embd_pdrop\": 0.1,\n",
            "  \"eos_token_id\": 50256,\n",
            "  \"gradient_checkpointing\": false,\n",
            "  \"id2label\": {\n",
            "    \"0\": \"LABEL_0\"\n",
            "  },\n",
            "  \"initializer_range\": 0.02,\n",
            "  \"label2id\": {\n",
            "    \"LABEL_0\": 0\n",
            "  },\n",
            "  \"layer_norm_epsilon\": 1e-05,\n",
            "  \"model_type\": \"gpt2\",\n",
            "  \"n_ctx\": 1024,\n",
            "  \"n_embd\": 768,\n",
            "  \"n_head\": 12,\n",
            "  \"n_inner\": null,\n",
            "  \"n_layer\": 6,\n",
            "  \"n_positions\": 1024,\n",
            "  \"resid_pdrop\": 0.1,\n",
            "  \"summary_activation\": null,\n",
            "  \"summary_first_dropout\": 0.1,\n",
            "  \"summary_proj_to_labels\": true,\n",
            "  \"summary_type\": \"cls_index\",\n",
            "  \"summary_use_proj\": true,\n",
            "  \"task_specific_params\": {\n",
            "    \"text-generation\": {\n",
            "      \"do_sample\": true,\n",
            "      \"max_length\": 50\n",
            "    }\n",
            "  },\n",
            "  \"use_cache\": true,\n",
            "  \"vocab_size\": 50257\n",
            "}\n",
            "\n",
            "[INFO|configuration_utils.py:431] 2021-01-05 07:35:29,481 >> loading configuration file https://huggingface.co/distilgpt2/resolve/main/config.json from cache at /root/.cache/huggingface/transformers/f985248d2791fcff97732e4ee263617adec1edb5429a2b8421734c6d14e39bee.422318838d1ec4e061efb4ea29671cb2a044e244dc69229682bebd7cacc81631\n",
            "[INFO|configuration_utils.py:467] 2021-01-05 07:35:29,482 >> Model config GPT2Config {\n",
            "  \"_num_labels\": 1,\n",
            "  \"activation_function\": \"gelu_new\",\n",
            "  \"architectures\": [\n",
            "    \"GPT2LMHeadModel\"\n",
            "  ],\n",
            "  \"attn_pdrop\": 0.1,\n",
            "  \"bos_token_id\": 50256,\n",
            "  \"embd_pdrop\": 0.1,\n",
            "  \"eos_token_id\": 50256,\n",
            "  \"gradient_checkpointing\": false,\n",
            "  \"id2label\": {\n",
            "    \"0\": \"LABEL_0\"\n",
            "  },\n",
            "  \"initializer_range\": 0.02,\n",
            "  \"label2id\": {\n",
            "    \"LABEL_0\": 0\n",
            "  },\n",
            "  \"layer_norm_epsilon\": 1e-05,\n",
            "  \"model_type\": \"gpt2\",\n",
            "  \"n_ctx\": 1024,\n",
            "  \"n_embd\": 768,\n",
            "  \"n_head\": 12,\n",
            "  \"n_inner\": null,\n",
            "  \"n_layer\": 6,\n",
            "  \"n_positions\": 1024,\n",
            "  \"resid_pdrop\": 0.1,\n",
            "  \"summary_activation\": null,\n",
            "  \"summary_first_dropout\": 0.1,\n",
            "  \"summary_proj_to_labels\": true,\n",
            "  \"summary_type\": \"cls_index\",\n",
            "  \"summary_use_proj\": true,\n",
            "  \"task_specific_params\": {\n",
            "    \"text-generation\": {\n",
            "      \"do_sample\": true,\n",
            "      \"max_length\": 50\n",
            "    }\n",
            "  },\n",
            "  \"use_cache\": true,\n",
            "  \"vocab_size\": 50257\n",
            "}\n",
            "\n",
            "01/05/2021 07:35:29 - INFO - filelock -   Lock 139827134885224 acquired on /root/.cache/huggingface/transformers/55051ac97dcc32f0a736d21a32a4d42b0d9b90f117ca7c38e65038b04bd5c3f5.c7ed1f96aac49e745788faa77ba0a26a392643a50bb388b9c04ff469e555241f.lock\n",
            "[INFO|file_utils.py:1334] 2021-01-05 07:35:29,756 >> https://huggingface.co/distilgpt2/resolve/main/vocab.json not found in cache or force_download set to True, downloading to /root/.cache/huggingface/transformers/tmp0dfokwoz\n",
            "Downloading: 100% 1.04M/1.04M [00:00<00:00, 2.04MB/s]\n",
            "[INFO|file_utils.py:1338] 2021-01-05 07:35:30,540 >> storing https://huggingface.co/distilgpt2/resolve/main/vocab.json in cache at /root/.cache/huggingface/transformers/55051ac97dcc32f0a736d21a32a4d42b0d9b90f117ca7c38e65038b04bd5c3f5.c7ed1f96aac49e745788faa77ba0a26a392643a50bb388b9c04ff469e555241f\n",
            "[INFO|file_utils.py:1341] 2021-01-05 07:35:30,540 >> creating metadata file for /root/.cache/huggingface/transformers/55051ac97dcc32f0a736d21a32a4d42b0d9b90f117ca7c38e65038b04bd5c3f5.c7ed1f96aac49e745788faa77ba0a26a392643a50bb388b9c04ff469e555241f\n",
            "01/05/2021 07:35:30 - INFO - filelock -   Lock 139827134885224 released on /root/.cache/huggingface/transformers/55051ac97dcc32f0a736d21a32a4d42b0d9b90f117ca7c38e65038b04bd5c3f5.c7ed1f96aac49e745788faa77ba0a26a392643a50bb388b9c04ff469e555241f.lock\n",
            "01/05/2021 07:35:30 - INFO - filelock -   Lock 139827134885112 acquired on /root/.cache/huggingface/transformers/9dfb299b74cdf7601ba7cd3a8073dbdac351caec0ed7ab5849b098b3c8ae3d57.5d12962c5ee615a4c803841266e9c3be9a691a924f72d395d3a6c6c81157788b.lock\n",
            "[INFO|file_utils.py:1334] 2021-01-05 07:35:30,812 >> https://huggingface.co/distilgpt2/resolve/main/merges.txt not found in cache or force_download set to True, downloading to /root/.cache/huggingface/transformers/tmphmn8mzvm\n",
            "Downloading: 100% 456k/456k [00:00<00:00, 1.10MB/s]\n",
            "[INFO|file_utils.py:1338] 2021-01-05 07:35:31,501 >> storing https://huggingface.co/distilgpt2/resolve/main/merges.txt in cache at /root/.cache/huggingface/transformers/9dfb299b74cdf7601ba7cd3a8073dbdac351caec0ed7ab5849b098b3c8ae3d57.5d12962c5ee615a4c803841266e9c3be9a691a924f72d395d3a6c6c81157788b\n",
            "[INFO|file_utils.py:1341] 2021-01-05 07:35:31,501 >> creating metadata file for /root/.cache/huggingface/transformers/9dfb299b74cdf7601ba7cd3a8073dbdac351caec0ed7ab5849b098b3c8ae3d57.5d12962c5ee615a4c803841266e9c3be9a691a924f72d395d3a6c6c81157788b\n",
            "01/05/2021 07:35:31 - INFO - filelock -   Lock 139827134885112 released on /root/.cache/huggingface/transformers/9dfb299b74cdf7601ba7cd3a8073dbdac351caec0ed7ab5849b098b3c8ae3d57.5d12962c5ee615a4c803841266e9c3be9a691a924f72d395d3a6c6c81157788b.lock\n",
            "01/05/2021 07:35:31 - INFO - filelock -   Lock 139827134885840 acquired on /root/.cache/huggingface/transformers/accb287b5a5396b2597382916b6cc939fdab1366e89475a92338d3971b3d02b7.cf2d0ecb83b6df91b3dbb53f1d1e4c311578bfd3aa0e04934215a49bf9898df0.lock\n",
            "[INFO|file_utils.py:1334] 2021-01-05 07:35:31,779 >> https://huggingface.co/distilgpt2/resolve/main/tokenizer.json not found in cache or force_download set to True, downloading to /root/.cache/huggingface/transformers/tmphm0j8_48\n",
            "Downloading: 100% 1.36M/1.36M [00:00<00:00, 2.64MB/s]\n",
            "[INFO|file_utils.py:1338] 2021-01-05 07:35:32,571 >> storing https://huggingface.co/distilgpt2/resolve/main/tokenizer.json in cache at /root/.cache/huggingface/transformers/accb287b5a5396b2597382916b6cc939fdab1366e89475a92338d3971b3d02b7.cf2d0ecb83b6df91b3dbb53f1d1e4c311578bfd3aa0e04934215a49bf9898df0\n",
            "[INFO|file_utils.py:1341] 2021-01-05 07:35:32,571 >> creating metadata file for /root/.cache/huggingface/transformers/accb287b5a5396b2597382916b6cc939fdab1366e89475a92338d3971b3d02b7.cf2d0ecb83b6df91b3dbb53f1d1e4c311578bfd3aa0e04934215a49bf9898df0\n",
            "01/05/2021 07:35:32 - INFO - filelock -   Lock 139827134885840 released on /root/.cache/huggingface/transformers/accb287b5a5396b2597382916b6cc939fdab1366e89475a92338d3971b3d02b7.cf2d0ecb83b6df91b3dbb53f1d1e4c311578bfd3aa0e04934215a49bf9898df0.lock\n",
            "[INFO|tokenization_utils_base.py:1802] 2021-01-05 07:35:32,571 >> loading file https://huggingface.co/distilgpt2/resolve/main/vocab.json from cache at /root/.cache/huggingface/transformers/55051ac97dcc32f0a736d21a32a4d42b0d9b90f117ca7c38e65038b04bd5c3f5.c7ed1f96aac49e745788faa77ba0a26a392643a50bb388b9c04ff469e555241f\n",
            "[INFO|tokenization_utils_base.py:1802] 2021-01-05 07:35:32,571 >> loading file https://huggingface.co/distilgpt2/resolve/main/merges.txt from cache at /root/.cache/huggingface/transformers/9dfb299b74cdf7601ba7cd3a8073dbdac351caec0ed7ab5849b098b3c8ae3d57.5d12962c5ee615a4c803841266e9c3be9a691a924f72d395d3a6c6c81157788b\n",
            "[INFO|tokenization_utils_base.py:1802] 2021-01-05 07:35:32,571 >> loading file https://huggingface.co/distilgpt2/resolve/main/tokenizer.json from cache at /root/.cache/huggingface/transformers/accb287b5a5396b2597382916b6cc939fdab1366e89475a92338d3971b3d02b7.cf2d0ecb83b6df91b3dbb53f1d1e4c311578bfd3aa0e04934215a49bf9898df0\n",
            "01/05/2021 07:35:32 - INFO - filelock -   Lock 139827117379992 acquired on /root/.cache/huggingface/transformers/43a212e83e76bcb07f45be584cf100676bdbbbe9c13f9e5c1c050049143a832f.a83d881ec4d624fd4b5826dd026e315246c48c67504ff91c0500570e291a54ba.lock\n",
            "[INFO|file_utils.py:1334] 2021-01-05 07:35:32,896 >> https://huggingface.co/distilgpt2/resolve/main/pytorch_model.bin not found in cache or force_download set to True, downloading to /root/.cache/huggingface/transformers/tmpkzcur190\n",
            "Downloading: 100% 353M/353M [00:03<00:00, 90.8MB/s]\n",
            "[INFO|file_utils.py:1338] 2021-01-05 07:35:36,864 >> storing https://huggingface.co/distilgpt2/resolve/main/pytorch_model.bin in cache at /root/.cache/huggingface/transformers/43a212e83e76bcb07f45be584cf100676bdbbbe9c13f9e5c1c050049143a832f.a83d881ec4d624fd4b5826dd026e315246c48c67504ff91c0500570e291a54ba\n",
            "[INFO|file_utils.py:1341] 2021-01-05 07:35:36,864 >> creating metadata file for /root/.cache/huggingface/transformers/43a212e83e76bcb07f45be584cf100676bdbbbe9c13f9e5c1c050049143a832f.a83d881ec4d624fd4b5826dd026e315246c48c67504ff91c0500570e291a54ba\n",
            "01/05/2021 07:35:36 - INFO - filelock -   Lock 139827117379992 released on /root/.cache/huggingface/transformers/43a212e83e76bcb07f45be584cf100676bdbbbe9c13f9e5c1c050049143a832f.a83d881ec4d624fd4b5826dd026e315246c48c67504ff91c0500570e291a54ba.lock\n",
            "[INFO|modeling_utils.py:1024] 2021-01-05 07:35:36,864 >> loading weights file https://huggingface.co/distilgpt2/resolve/main/pytorch_model.bin from cache at /root/.cache/huggingface/transformers/43a212e83e76bcb07f45be584cf100676bdbbbe9c13f9e5c1c050049143a832f.a83d881ec4d624fd4b5826dd026e315246c48c67504ff91c0500570e291a54ba\n",
            "[INFO|modeling_utils.py:1140] 2021-01-05 07:35:40,144 >> All model checkpoint weights were used when initializing GPT2LMHeadModel.\n",
            "\n",
            "[INFO|modeling_utils.py:1149] 2021-01-05 07:35:40,144 >> All the weights of GPT2LMHeadModel were initialized from the model checkpoint at distilgpt2.\n",
            "If your task is similar to the task the model of the checkpoint was trained on, you can already use GPT2LMHeadModel for predictions without further training.\n",
            "100% 57/57 [00:01<00:00, 35.60ba/s]\n",
            "100% 7/7 [00:00<00:00, 44.50ba/s]\n",
            "100% 57/57 [00:03<00:00, 18.73ba/s]\n",
            "100% 7/7 [00:00<00:00, 21.16ba/s]\n",
            "[INFO|trainer.py:396] 2021-01-05 07:35:59,491 >> The following columns in the training set don't have a corresponding argument in `GPT2LMHeadModel.forward` and have been ignored: .\n",
            "[INFO|trainer.py:396] 2021-01-05 07:35:59,491 >> The following columns in the evaluation set don't have a corresponding argument in `GPT2LMHeadModel.forward` and have been ignored: .\n",
            "[WARNING|training_args.py:450] 2021-01-05 07:35:59,492 >> Using deprecated `--per_gpu_train_batch_size` argument which will be removed in a future version. Using `--per_device_train_batch_size` is preferred.\n",
            "[WARNING|training_args.py:450] 2021-01-05 07:35:59,493 >> Using deprecated `--per_gpu_train_batch_size` argument which will be removed in a future version. Using `--per_device_train_batch_size` is preferred.\n",
            "[INFO|trainer.py:719] 2021-01-05 07:35:59,493 >> ***** Running training *****\n",
            "[INFO|trainer.py:720] 2021-01-05 07:35:59,493 >>   Num examples = 3015\n",
            "[INFO|trainer.py:721] 2021-01-05 07:35:59,493 >>   Num Epochs = 3\n",
            "[INFO|trainer.py:722] 2021-01-05 07:35:59,493 >>   Instantaneous batch size per device = 8\n",
            "[INFO|trainer.py:723] 2021-01-05 07:35:59,493 >>   Total train batch size (w. parallel, distributed & accumulation) = 1\n",
            "[INFO|trainer.py:724] 2021-01-05 07:35:59,493 >>   Gradient Accumulation steps = 1\n",
            "[INFO|trainer.py:725] 2021-01-05 07:35:59,493 >>   Total optimization steps = 9045\n",
            "[WARNING|training_args.py:450] 2021-01-05 07:35:59,497 >> Using deprecated `--per_gpu_train_batch_size` argument which will be removed in a future version. Using `--per_device_train_batch_size` is preferred.\n",
            "[WARNING|training_args.py:467] 2021-01-05 07:35:59,497 >> Using deprecated `--per_gpu_eval_batch_size` argument which will be removed in a future version. Using `--per_device_eval_batch_size` is preferred.\n",
            "{'loss': 2.755792236328125, 'learning_rate': 4.5577667219458266e-05, 'epoch': 0.26533996683250416}\n",
            "  9% 800/9045 [00:39<06:45, 20.33it/s][WARNING|training_args.py:467] 2021-01-05 07:36:39,467 >> Using deprecated `--per_gpu_eval_batch_size` argument which will be removed in a future version. Using `--per_device_eval_batch_size` is preferred.\n",
            "[INFO|trainer.py:1441] 2021-01-05 07:36:39,467 >> ***** Running Evaluation *****\n",
            "[INFO|trainer.py:1442] 2021-01-05 07:36:39,467 >>   Num examples = 338\n",
            "[INFO|trainer.py:1443] 2021-01-05 07:36:39,468 >>   Batch size = 1\n",
            "\n",
            "  0% 0/338 [00:00<?, ?it/s]\u001b[A\n",
            "  4% 12/338 [00:00<00:02, 112.92it/s]\u001b[A\n",
            "  7% 23/338 [00:00<00:02, 109.06it/s]\u001b[A\n",
            " 10% 34/338 [00:00<00:02, 106.46it/s]\u001b[A\n",
            " 13% 45/338 [00:00<00:02, 105.18it/s]\u001b[A\n",
            " 17% 56/338 [00:00<00:02, 103.92it/s]\u001b[A\n",
            " 20% 66/338 [00:00<00:02, 101.38it/s]\u001b[A\n",
            " 23% 77/338 [00:00<00:02, 101.73it/s]\u001b[A\n",
            " 26% 88/338 [00:00<00:02, 101.86it/s]\u001b[A\n",
            " 29% 98/338 [00:00<00:02, 101.08it/s]\u001b[A\n",
            " 32% 109/338 [00:01<00:02, 101.47it/s]\u001b[A\n",
            " 36% 120/338 [00:01<00:02, 101.77it/s]\u001b[A\n",
            " 39% 131/338 [00:01<00:02, 101.36it/s]\u001b[A\n",
            " 42% 141/338 [00:01<00:01, 100.59it/s]\u001b[A\n",
            " 45% 152/338 [00:01<00:01, 100.74it/s]\u001b[A\n",
            " 48% 163/338 [00:01<00:01, 100.95it/s]\u001b[A\n",
            " 51% 174/338 [00:01<00:01, 99.29it/s] \u001b[A\n",
            " 55% 185/338 [00:01<00:01, 99.90it/s]\u001b[A\n",
            " 58% 196/338 [00:01<00:01, 100.62it/s]\u001b[A\n",
            " 61% 207/338 [00:02<00:01, 101.10it/s]\u001b[A\n",
            " 64% 218/338 [00:02<00:01, 100.40it/s]\u001b[A\n",
            " 68% 229/338 [00:02<00:01, 99.76it/s] \u001b[A\n",
            " 71% 240/338 [00:02<00:00, 99.95it/s]\u001b[A\n",
            " 74% 250/338 [00:02<00:00, 99.46it/s]\u001b[A\n",
            " 77% 261/338 [00:02<00:00, 99.81it/s]\u001b[A\n",
            " 80% 271/338 [00:02<00:00, 99.79it/s]\u001b[A\n",
            " 83% 281/338 [00:02<00:00, 99.67it/s]\u001b[A\n",
            " 86% 292/338 [00:02<00:00, 100.37it/s]\u001b[A\n",
            " 90% 303/338 [00:03<00:00, 100.36it/s]\u001b[A\n",
            " 93% 314/338 [00:03<00:00, 100.86it/s]\u001b[A\n",
            " 96% 325/338 [00:03<00:00, 100.34it/s]\u001b[A\n",
            "                                      \n",
            "\u001b[A{'eval_loss': 2.6152448654174805, 'eval_runtime': 3.3662, 'eval_samples_per_second': 100.409, 'epoch': 0.26533996683250416}\n",
            "  9% 800/9045 [00:43<06:45, 20.33it/s]\n",
            "100% 338/338 [00:03<00:00, 101.05it/s]\u001b[A\n",
            "                                      \u001b[A[INFO|trainer.py:1249] 2021-01-05 07:36:42,835 >> Saving model checkpoint to ./output/checkpoint-800\n",
            "[INFO|configuration_utils.py:289] 2021-01-05 07:36:42,836 >> Configuration saved in ./output/checkpoint-800/config.json\n",
            "[INFO|modeling_utils.py:814] 2021-01-05 07:36:43,819 >> Model weights saved in ./output/checkpoint-800/pytorch_model.bin\n",
            " 11% 1022/9045 [00:58<06:39, 20.10it/s]"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vj_aJ6myvpZR"
      },
      "source": [
        "## Play with model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6riV2LC7vqPQ"
      },
      "source": [
        "from transformers import pipeline"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0CV12Xz0wiI5"
      },
      "source": [
        "ft_generator = pipeline('text-generation', model='./output')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "U7oCnZNhPCEZ"
      },
      "source": [
        "def PrettyPrintPrediction(text):\n",
        "  print()\n",
        "  text = text.replace('#', '\\n')\n",
        "  print(text)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XBnHXEjvf0HX"
      },
      "source": [
        "ft_generator( )"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IMfMQy-o2Q__",
        "outputId": "47c392ca-659d-4e17-d04f-1a0598780ba1"
      },
      "source": [
        "for text in ft_generator(\"Vivek: Mihir sucks #Sreejith2: I agree! Tell me more#Vivek: Dude he always makes fun of me#Vivek:\", max_length=256, num_return_sequences=3):\n",
        "  PrettyPrintPrediction(text['generated_text'])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "\n",
            "Vivek: Mihir sucks \\m/\n",
            "Sreejith2: I agree! Tell me more\n",
            "Vivek: Dude he always makes fun of me\n",
            "Vivek: 🤣\n",
            "Sreejith2: Hey thanks man. There is just one guy in the building who says \"hey fuck you bitch\"\n",
            "Sreejith2: https://youtu.be/z6YQtJd8sq\n",
            "Vivek: He has more jokes than Hitler\n",
            "Vivek: 🤣\n",
            "Sreejith2: Oho\n",
            "Sreejith2: Hey, all peace is here man\n",
            "Vivek: Peace, will probably find peace here soon\n",
            "Sreejith2: I think only if you actually feel safe.\n",
            "Vivek: Yup.\n",
            "Sreejith2: Can stay in your car next morning\n",
            "Vivek: There?\n",
            "Sreejith2: Come to the police station\n",
            "Sreejith2: Wassup man\n",
            "Vivek: Hey!\n",
            "Sreejith2: Whose name you're working on?\n",
            "Vivek: I want to go to your place\n",
            "Sreejith2: What time\n",
            "\n",
            "Vivek: Mihir sucks \\m/\n",
            "Sreejith2: I agree! Tell me more\n",
            "Vivek: Dude he always makes fun of me\n",
            "Vivek: He's the same person\n",
            "Vivek: 🙄\n",
            "Sreejith2: Hey how did you get your visa?\n",
            "Vivek: To the US?\n",
            "Sreejith2: Hahaha\n",
            "Vivek: I wanted to change the visa\n",
            "Vivek: I decided to apply for it\n",
            "Sreejith2: In Bangalore for the last 2 days\n",
            "Vivek: And you've come across this\n",
            "Vivek: 🙈\n",
            "Vivek: Also I'm going to go to London for the last 3 days\n",
            "Sreejith2: Hey y'all have you gotten the visa?\n",
            "Vivek: And you'll need to submit it to the new US mail\n",
            "Vivek: I have no visa\n",
            "Vivek: And there is no visa\n",
            "Sreejith2: Ah ok okay😋\n",
            "Vivek: 👻\n",
            "Sreejith2: Also, don't you have one that will do it\n",
            "Sreejith2\n",
            "\n",
            "Vivek: Mihir sucks \\m/\n",
            "Sreejith2: I agree! Tell me more\n",
            "Vivek: Dude he always makes fun of me\n",
            "Vivek: Have you met?\n",
            "Sreejith2: Haan yeah\n",
            "Vivek: I'm meeting him tomorrow\n",
            "Sreejith2: <Media omitted>\n",
            "Sreejith2: <Media omitted>\n",
            "Vivek: He's being called\n",
            "Vivek: 😂😂\n",
            "Vivek: Hahahahahahaha\n",
            "Vivek: 😬\n",
            "Sreejith2: I think he's a weird character\n",
            "Vivek: In SF and tech\n",
            "Sreejith2: Also\n",
            "Sreejith2: I don't know if she's in SF now\n",
            "Vivek: 🙈\n",
            "Sreejith2: Yo what location?\n",
            "Vivek: Eyo\n",
            "Sreejith2: Hey\n",
            "Sreejith2: Yoyo\n",
            "Vivek: There\n",
            "Vivek: You\n",
            "Vivek: Free for a call?\n",
            "Vivek: For a call?\n",
            "Sreejith2: Yo\n",
            "Vivek\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RDRDEqH2vGMV"
      },
      "source": [
        "## Download model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8gp4JYnzjMRf"
      },
      "source": [
        "Download the model so you can use it with the Chrome extension."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fac6_VtIvHQ_",
        "outputId": "80f41b42-e58a-4228-cdde-a38945fe1b37"
      },
      "source": [
        "!zip model.zip ./output/*"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "  adding: output/checkpoint-1600/ (stored 0%)\n",
            "  adding: output/checkpoint-2400/ (stored 0%)\n",
            "  adding: output/checkpoint-3200/ (stored 0%)\n",
            "  adding: output/checkpoint-4000/ (stored 0%)\n",
            "  adding: output/checkpoint-800/ (stored 0%)\n",
            "  adding: output/config.json (deflated 51%)\n",
            "  adding: output/eval_results_clm.txt (stored 0%)\n",
            "  adding: output/merges.txt (deflated 53%)\n",
            "  adding: output/pytorch_model.bin (deflated 9%)\n",
            "  adding: output/special_tokens_map.json (deflated 52%)\n",
            "  adding: output/tokenizer_config.json (deflated 38%)\n",
            "  adding: output/trainer_state.json (deflated 70%)\n",
            "  adding: output/training_args.bin (deflated 46%)\n",
            "  adding: output/train_results.txt (deflated 10%)\n",
            "  adding: output/vocab.json (deflated 59%)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1Cvky9WHxLSd",
        "outputId": "c091e24f-bb09-4677-baa8-abcfb197dbcb"
      },
      "source": [
        "ls -l"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "total 302268\n",
            "-rw-r--r--  1 root root   1276333 Jan  4 07:30  data.txt\n",
            "-rw-r--r--  1 root root 305021348 Jan  4 07:58  model.zip\n",
            "drwxr-xr-x  7 root root      4096 Jan  4 07:38  \u001b[0m\u001b[01;34moutput\u001b[0m/\n",
            "drwxr-xr-x  3 root root      4096 Jan  4 07:31  \u001b[01;34mruns\u001b[0m/\n",
            "drwxr-xr-x  1 root root      4096 Dec 21 17:29  \u001b[01;34msample_data\u001b[0m/\n",
            "-rw-r--r--  1 root root    127482 Jan  4 07:30  test.txt\n",
            "-rw-r--r--  1 root root   1148810 Jan  4 07:30  train.txt\n",
            "drwxr-xr-x 15 root root      4096 Jan  4 07:28  \u001b[01;34mtransformers\u001b[0m/\n",
            "-rw-r--r--  1 root root    188024 Jan  4 07:29 'WhatsApp Chat with 5 Years Time 🌞.txt'\n",
            "-rw-r--r--  1 root root     96072 Jan  4 07:29 'WhatsApp Chat with Mihir London.txt'\n",
            "-rw-r--r--  1 root root    493383 Jan  4 07:30 'WhatsApp Chat with Rishi Amreeka.txt'\n",
            "-rw-r--r--  1 root root    271150 Jan  4 07:29 'WhatsApp Chat with Sreejith2.txt'\n",
            "-rw-r--r--  1 root root    351486 Jan  4 07:29 'WhatsApp Chat with Sreejith.txt'\n",
            "-rw-r--r--  1 root root    509144 Jan  4 07:30 'WhatsApp Chat with Vikrant London.txt'\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 17
        },
        "id": "IF1Qh8X6xrt1",
        "outputId": "f4c9ad0f-b020-47ff-8eee-0fe0ac1f8e1b"
      },
      "source": [
        "files.download('model.zip')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "application/javascript": [
              "\n",
              "    async function download(id, filename, size) {\n",
              "      if (!google.colab.kernel.accessAllowed) {\n",
              "        return;\n",
              "      }\n",
              "      const div = document.createElement('div');\n",
              "      const label = document.createElement('label');\n",
              "      label.textContent = `Downloading \"${filename}\": `;\n",
              "      div.appendChild(label);\n",
              "      const progress = document.createElement('progress');\n",
              "      progress.max = size;\n",
              "      div.appendChild(progress);\n",
              "      document.body.appendChild(div);\n",
              "\n",
              "      const buffers = [];\n",
              "      let downloaded = 0;\n",
              "\n",
              "      const channel = await google.colab.kernel.comms.open(id);\n",
              "      // Send a message to notify the kernel that we're ready.\n",
              "      channel.send({})\n",
              "\n",
              "      for await (const message of channel.messages) {\n",
              "        // Send a message to notify the kernel that we're ready.\n",
              "        channel.send({})\n",
              "        if (message.buffers) {\n",
              "          for (const buffer of message.buffers) {\n",
              "            buffers.push(buffer);\n",
              "            downloaded += buffer.byteLength;\n",
              "            progress.value = downloaded;\n",
              "          }\n",
              "        }\n",
              "      }\n",
              "      const blob = new Blob(buffers, {type: 'application/binary'});\n",
              "      const a = document.createElement('a');\n",
              "      a.href = window.URL.createObjectURL(blob);\n",
              "      a.download = filename;\n",
              "      div.appendChild(a);\n",
              "      a.click();\n",
              "      div.remove();\n",
              "    }\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.Javascript object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "application/javascript": [
              "download(\"download_8df03770-4b81-4eec-8bb7-342f78da7a4a\", \"model.zip\", 305021348)"
            ],
            "text/plain": [
              "<IPython.core.display.Javascript object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Hk9baYSr62rD"
      },
      "source": [
        "## Load a saved model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "l_4X0yePiqv2"
      },
      "source": [
        "Use this to play with a model you've previously downloaded. You will need to connect colab to a locally running Jupyter runtime."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gfxgcKlb6_Uf"
      },
      "source": [
        "from transformers import pipeline"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PJT7FdFG7M8q"
      },
      "source": [
        "ft_generator = pipeline('text-generation', model='../../Downloads/output_2')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HscWDyJgz0fy",
        "outputId": "ddf36a2c-8e73-4943-8ca2-9a3188286874"
      },
      "source": [
        "for text in ft_generator(\"Vivek: Mihir sucks :(#Sreejith2: I agree! Tell me more#Vivek: Dude he always makes fun of me#Vivek:\", max_length=100, num_return_sequences=3, do_sample=True, eos_token_id=2, pad_token_id=0, skip_special_tokens=True, top_k=50, top_p=0.95):\n",
        "  PrettyPrintPrediction(text['generated_text'])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\n",
            "Vivek: Mihir sucks :(\n",
            "Sreejith2: I agree! Tell me more\n",
            "Vivek: Dude he always makes fun of me\n",
            "Vivek: I just wanted to know if there is any one thing that he does in life for me no?\n",
            "\n",
            "\n",
            "Vivek: Mihir sucks :(\n",
            "Sreejith2: I agree! Tell me more\n",
            "Vivek: Dude he always makes fun of me\n",
            "Vivek: And what did you mean?\n",
            "!!!!!!!!!!!!!!\n",
            "\n",
            "Vivek: Mihir sucks :(\n",
            "Sreejith2: I agree! Tell me more\n",
            "Vivek: Dude he always makes fun of me\n",
            "Vivek: That was a big problem for me when I was younger 🤣\n",
            "!!!!!!\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}