GPT_J_6B_Inference_Demo_(Fixed).ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "GPT-J-6B Inference Demo (Fixed).ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true,
      "machine_shape": "hm",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "TPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/dvschultz/ml-art-colabs/blob/master/GPT_J_6B_Inference_Demo_(Fixed).ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pHIJVqHsh4An"
      },
      "source": [
        "# GPT-J-6B Inference Demo\n",
        "\n",
        "<a href=\"http://colab.research.google.com/github/kingoflolz/mesh-transformer-jax/blob/master/colab_demo.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
        "\n",
        "This notebook demonstrates how to run the [GPT-J-6B model](https://github.com/kingoflolz/mesh-transformer-jax/#GPT-J-6B). See the link for more details about the model, including evaluation metrics and credits.\n",
        "\n",
        "Looking for a dead simple version without configuration? Try the [Huggingface demo](https://huggingface.co/EleutherAI/gpt-j-6B?text=Do+I+contradict+myself%3F%0AVery+well+then+I+contradict+myself%2C%0A%28I+am+large%2C+I+contain+multitudes.%29)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8CMw_dSQKfhT"
      },
      "source": [
        "## Install Dependencies\n",
        "\n",
        "First we download the model and install some dependencies. This step takes at least 5 minutes (possibly longer depending on server load).\n",
        "\n",
        "!!! **Make sure you are using a TPU runtime!** !!!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "n7xAFw-LOYfe",
        "outputId": "3870a92b-5dd7-4a8e-f83d-ba9d2c1806b4"
      },
      "source": [
        "!apt install zstd.  \n",
        "\n",
        "# the \"slim\" version contain only bf16 weights and no optimizer parameters, which minimizes bandwidth and memory\n",
        "!time wget -c https://the-eye.eu/public/AI/GPT-J-6B/step_383500_slim.tar.zstd\n",
        "\n",
        "!time tar -I zstd -xf step_383500_slim.tar.zstd\n",
        "\n",
        "!git clone https://github.com/kingoflolz/mesh-transformer-jax.git\n",
        "#!pip install -r /content/mesh-transformer-jax/requirements.txt\n",
        "\n",
        "# jax 0.2.12 is required due to a regression with xmap in 0.2.13\n",
        "!pip install mesh-transformer-jax/ jax==0.2.12 #tensorflow==2.5.0\n",
        "!pip install git+git://github.com/deepmind/optax.git\n",
        "!pip install git+https://github.com/EleutherAI/lm-evaluation-harness@c406a62047\n",
        "!pip install transformers fabric wandb google-cloud-storage cloudpickle Flask ray[default] einops smart_open[gcs] func_timeout ftfy fastapi uvicorn lm_dataformat pathy\n",
        "!pip install git+https://github.com/deepmind/dm-haiku\n",
        "!pip install git+https://github.com/EleutherAI/lm-evaluation-harness/"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Reading package lists... Done\n",
            "Building dependency tree       \n",
            "Reading state information... Done\n",
            "Note, selecting 'libzstd-dev' for regex 'zstd.'\n",
            "Note, selecting 'libzstd1' for regex 'zstd.'\n",
            "Note, selecting 'libzstd1-dev' for regex 'zstd.'\n",
            "The following NEW packages will be installed:\n",
            "  libzstd-dev libzstd1-dev\n",
            "The following packages will be upgraded:\n",
            "  libzstd1\n",
            "1 upgraded, 2 newly installed, 0 to remove and 36 not upgraded.\n",
            "Need to get 423 kB of archives.\n",
            "After this operation, 844 kB of additional disk space will be used.\n",
            "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libzstd1 amd64 1.3.3+dfsg-2ubuntu1.2 [189 kB]\n",
            "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libzstd-dev amd64 1.3.3+dfsg-2ubuntu1.2 [230 kB]\n",
            "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libzstd1-dev amd64 1.3.3+dfsg-2ubuntu1.2 [4,492 B]\n",
            "Fetched 423 kB in 1s (458 kB/s)\n",
            "(Reading database ... 155219 files and directories currently installed.)\n",
            "Preparing to unpack .../libzstd1_1.3.3+dfsg-2ubuntu1.2_amd64.deb ...\n",
            "Unpacking libzstd1:amd64 (1.3.3+dfsg-2ubuntu1.2) over (1.3.3+dfsg-2ubuntu1.1) ...\n",
            "Setting up libzstd1:amd64 (1.3.3+dfsg-2ubuntu1.2) ...\n",
            "Selecting previously unselected package libzstd-dev:amd64.\n",
            "(Reading database ... 155219 files and directories currently installed.)\n",
            "Preparing to unpack .../libzstd-dev_1.3.3+dfsg-2ubuntu1.2_amd64.deb ...\n",
            "Unpacking libzstd-dev:amd64 (1.3.3+dfsg-2ubuntu1.2) ...\n",
            "Selecting previously unselected package libzstd1-dev.\n",
            "Preparing to unpack .../libzstd1-dev_1.3.3+dfsg-2ubuntu1.2_amd64.deb ...\n",
            "Unpacking libzstd1-dev (1.3.3+dfsg-2ubuntu1.2) ...\n",
            "Setting up libzstd-dev:amd64 (1.3.3+dfsg-2ubuntu1.2) ...\n",
            "Setting up libzstd1-dev (1.3.3+dfsg-2ubuntu1.2) ...\n",
            "Processing triggers for libc-bin (2.27-3ubuntu1.3) ...\n",
            "/sbin/ldconfig.real: /usr/local/lib/python3.7/dist-packages/ideep4py/lib/libmkldnn.so.0 is not a symbolic link\n",
            "\n",
            "--2021-11-12 01:52:51--  https://the-eye.eu/public/AI/GPT-J-6B/step_383500_slim.tar.zstd\n",
            "Resolving the-eye.eu (the-eye.eu)... 162.213.130.244\n",
            "Connecting to the-eye.eu (the-eye.eu)|162.213.130.244|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 9414712325 (8.8G) [application/octet-stream]\n",
            "Saving to: ‘step_383500_slim.tar.zstd’\n",
            "\n",
            "step_383500_slim.ta 100%[===================>]   8.77G  30.9MB/s    in 4m 21s  \n",
            "\n",
            "2021-11-12 01:57:12 (34.4 MB/s) - ‘step_383500_slim.tar.zstd’ saved [9414712325/9414712325]\n",
            "\n",
            "\n",
            "real\t4m21.000s\n",
            "user\t0m10.185s\n",
            "sys\t0m53.202s\n",
            "tar (child): zstd: Cannot exec: No such file or directory\n",
            "tar (child): Error is not recoverable: exiting now\n",
            "tar: Child returned status 2\n",
            "tar: Error is not recoverable: exiting now\n",
            "\n",
            "real\t0m0.106s\n",
            "user\t0m0.005s\n",
            "sys\t0m0.010s\n",
            "Cloning into 'mesh-transformer-jax'...\n",
            "remote: Enumerating objects: 766, done.\u001b[K\n",
            "remote: Counting objects: 100% (403/403), done.\u001b[K\n",
            "remote: Compressing objects: 100% (124/124), done.\u001b[K\n",
            "remote: Total 766 (delta 344), reused 286 (delta 279), pack-reused 363\u001b[K\n",
            "Receiving objects: 100% (766/766), 243.27 KiB | 3.29 MiB/s, done.\n",
            "Resolving deltas: 100% (506/506), done.\n",
            "Processing ./mesh-transformer-jax\n",
            "\u001b[33m  DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default.\n",
            "   pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555.\u001b[0m\n",
            "Collecting jax==0.2.12\n",
            "  Downloading jax-0.2.12.tar.gz (590 kB)\n",
            "\u001b[K     |████████████████████████████████| 590 kB 5.2 MB/s \n",
            "\u001b[?25hRequirement already satisfied: numpy>=1.12 in /usr/local/lib/python3.7/dist-packages (from jax==0.2.12) (1.19.5)\n",
            "Requirement already satisfied: absl-py in /usr/local/lib/python3.7/dist-packages (from jax==0.2.12) (0.12.0)\n",
            "Requirement already satisfied: opt_einsum in /usr/local/lib/python3.7/dist-packages (from jax==0.2.12) (3.3.0)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from absl-py->jax==0.2.12) (1.15.0)\n",
            "Building wheels for collected packages: mesh-transformer, jax\n",
            "  Building wheel for mesh-transformer (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for mesh-transformer: filename=mesh_transformer-0.0.0-py3-none-any.whl size=26330 sha256=3fe122f051bc4a6b9bf8d230bc27f7640718a9bb69a189954b8c6fbeb3ced25a\n",
            "  Stored in directory: /root/.cache/pip/wheels/56/bd/89/b1f6b2f3d6b938d0c5812ee97756a1afd32521bea293543863\n",
            "  Building wheel for jax (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for jax: filename=jax-0.2.12-py3-none-any.whl size=682487 sha256=db961b56d4d6149461379977ec8d183aefc228e67074ccd76d648af47c8c7951\n",
            "  Stored in directory: /root/.cache/pip/wheels/f6/4d/e5/73eec5070b77f25664c67bd793d4eb97f41bbf9be7afafd15e\n",
            "Successfully built mesh-transformer jax\n",
            "Installing collected packages: mesh-transformer, jax\n",
            "  Attempting uninstall: jax\n",
            "    Found existing installation: jax 0.2.21\n",
            "    Uninstalling jax-0.2.21:\n",
            "      Successfully uninstalled jax-0.2.21\n",
            "Successfully installed jax-0.2.12 mesh-transformer-0.0.0\n",
            "Collecting git+git://github.com/deepmind/optax.git\n",
            "  Cloning git://github.com/deepmind/optax.git to /tmp/pip-req-build-k9bte5gp\n",
            "  Running command git clone -q git://github.com/deepmind/optax.git /tmp/pip-req-build-k9bte5gp\n",
            "Requirement already satisfied: absl-py>=0.7.1 in /usr/local/lib/python3.7/dist-packages (from optax==0.0.9) (0.12.0)\n",
            "Collecting chex>=0.0.4\n",
            "  Downloading chex-0.0.8-py3-none-any.whl (57 kB)\n",
            "\u001b[K     |████████████████████████████████| 57 kB 2.9 MB/s \n",
            "\u001b[?25hRequirement already satisfied: jax>=0.1.55 in /usr/local/lib/python3.7/dist-packages (from optax==0.0.9) (0.2.12)\n",
            "Requirement already satisfied: jaxlib>=0.1.37 in /usr/local/lib/python3.7/dist-packages (from optax==0.0.9) (0.1.71+cuda111)\n",
            "Requirement already satisfied: numpy>=1.18.0 in /usr/local/lib/python3.7/dist-packages (from optax==0.0.9) (1.19.5)\n",
            "Requirement already satisfied: typing_extensions~=3.10.0 in /usr/local/lib/python3.7/dist-packages (from optax==0.0.9) (3.10.0.2)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from absl-py>=0.7.1->optax==0.0.9) (1.15.0)\n",
            "Requirement already satisfied: toolz>=0.9.0 in /usr/local/lib/python3.7/dist-packages (from chex>=0.0.4->optax==0.0.9) (0.11.2)\n",
            "Requirement already satisfied: dm-tree>=0.1.5 in /usr/local/lib/python3.7/dist-packages (from chex>=0.0.4->optax==0.0.9) (0.1.6)\n",
            "Requirement already satisfied: opt-einsum in /usr/local/lib/python3.7/dist-packages (from jax>=0.1.55->optax==0.0.9) (3.3.0)\n",
            "Requirement already satisfied: flatbuffers<3.0,>=1.12 in /usr/local/lib/python3.7/dist-packages (from jaxlib>=0.1.37->optax==0.0.9) (2.0)\n",
            "Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from jaxlib>=0.1.37->optax==0.0.9) (1.4.1)\n",
            "Building wheels for collected packages: optax\n",
            "  Building wheel for optax (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for optax: filename=optax-0.0.9-py3-none-any.whl size=126766 sha256=5ef9306d2c84c4f2ecf23e679ab2fe4da029fb8302b1cd3aa073551a4399ced7\n",
            "  Stored in directory: /tmp/pip-ephem-wheel-cache-hgdx4snb/wheels/99/a8/ea/844576cb39aaa6929daf91bd459d92d5d2e0c52abcc31d5b8c\n",
            "Successfully built optax\n",
            "Installing collected packages: chex, optax\n",
            "Successfully installed chex-0.0.8 optax-0.0.9\n",
            "Collecting git+https://github.com/EleutherAI/lm-evaluation-harness@c406a62047\n",
            "  Cloning https://github.com/EleutherAI/lm-evaluation-harness (to revision c406a62047) to /tmp/pip-req-build-sbma4nrc\n",
            "  Running command git clone -q https://github.com/EleutherAI/lm-evaluation-harness /tmp/pip-req-build-sbma4nrc\n",
            "\u001b[33m  WARNING: Did not find branch or tag 'c406a62047', assuming revision or ref.\u001b[0m\n",
            "  Running command git checkout -q c406a62047\n",
            "Collecting bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt\n",
            "  Downloading https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip\n",
            "\u001b[K     - 16.5 MB 79.5 MB/s\n",
            "\u001b[?25hCollecting black==20.8b1\n",
            "  Downloading black-20.8b1.tar.gz (1.1 MB)\n",
            "\u001b[K     |████████████████████████████████| 1.1 MB 5.5 MB/s \n",
            "\u001b[?25h  Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",
            "  Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",
            "    Preparing wheel metadata ... \u001b[?25l\u001b[?25hdone\n",
            "Collecting best_download>=0.0.6\n",
            "  Downloading best_download-0.0.7-py3-none-any.whl (4.5 kB)\n",
            "Collecting datasets>=1.2.1\n",
            "  Downloading datasets-1.15.1-py3-none-any.whl (290 kB)\n",
            "\u001b[K     |████████████████████████████████| 290 kB 41.8 MB/s \n",
            "\u001b[?25hRequirement already satisfied: click>=7.1 in /usr/local/lib/python3.7/dist-packages (from lm-eval==0.0.1) (7.1.2)\n",
            "Collecting scikit-learn>=0.24.1\n",
            "  Downloading scikit_learn-1.0.1-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (23.2 MB)\n",
            "\u001b[K     |████████████████████████████████| 23.2 MB 1.8 MB/s \n",
            "\u001b[?25hRequirement already satisfied: torch>=1.7 in /usr/local/lib/python3.7/dist-packages (from lm-eval==0.0.1) (1.10.0+cu111)\n",
            "Collecting transformers>=4.1\n",
            "  Downloading transformers-4.12.3-py3-none-any.whl (3.1 MB)\n",
            "\u001b[K     |████████████████████████████████| 3.1 MB 42.9 MB/s \n",
            "\u001b[?25hCollecting sqlitedict==1.6.0\n",
            "  Downloading sqlitedict-1.6.0.tar.gz (29 kB)\n",
            "Collecting pytablewriter==0.58.0\n",
            "  Downloading pytablewriter-0.58.0-py3-none-any.whl (96 kB)\n",
            "\u001b[K     |████████████████████████████████| 96 kB 5.8 MB/s \n",
            "\u001b[?25hCollecting sacrebleu==1.5.0\n",
            "  Downloading sacrebleu-1.5.0-py3-none-any.whl (65 kB)\n",
            "\u001b[K     |████████████████████████████████| 65 kB 3.5 MB/s \n",
            "\u001b[?25hCollecting pycountry==20.7.3\n",
            "  Downloading pycountry-20.7.3.tar.gz (10.1 MB)\n",
            "\u001b[K     |████████████████████████████████| 10.1 MB 43.2 MB/s \n",
            "\u001b[?25hCollecting numexpr==2.7.2\n",
            "  Downloading numexpr-2.7.2-cp37-cp37m-manylinux2010_x86_64.whl (471 kB)\n",
            "\u001b[K     |████████████████████████████████| 471 kB 68.8 MB/s \n",
            "\u001b[?25hCollecting lm_dataformat==0.0.19\n",
            "  Downloading lm_dataformat-0.0.19-py3-none-any.whl (5.4 kB)\n",
            "Collecting pytest==6.2.3\n",
            "  Downloading pytest-6.2.3-py3-none-any.whl (280 kB)\n",
            "\u001b[K     |████████████████████████████████| 280 kB 36.9 MB/s \n",
            "\u001b[?25hCollecting pybind11==2.6.2\n",
            "  Downloading pybind11-2.6.2-py2.py3-none-any.whl (191 kB)\n",
            "\u001b[K     |████████████████████████████████| 191 kB 58.8 MB/s \n",
            "\u001b[?25hCollecting tqdm-multiprocess==0.0.11\n",
            "  Downloading tqdm_multiprocess-0.0.11-py3-none-any.whl (9.8 kB)\n",
            "Collecting zstandard==0.15.2\n",
            "  Downloading zstandard-0.15.2-cp37-cp37m-manylinux2014_x86_64.whl (2.2 MB)\n",
            "\u001b[K     |████████████████████████████████| 2.2 MB 54.7 MB/s \n",
            "\u001b[?25hCollecting jsonlines==2.0.0\n",
            "  Downloading jsonlines-2.0.0-py3-none-any.whl (6.3 kB)\n",
            "Collecting mock==4.0.3\n",
            "  Downloading mock-4.0.3-py3-none-any.whl (28 kB)\n",
            "Collecting openai==0.6.4\n",
            "  Downloading openai-0.6.4.tar.gz (159 kB)\n",
            "\u001b[K     |████████████████████████████████| 159 kB 56.9 MB/s \n",
            "\u001b[?25hRequirement already satisfied: jieba==0.42.1 in /usr/local/lib/python3.7/dist-packages (from lm-eval==0.0.1) (0.42.1)\n",
            "Collecting nagisa==0.2.7\n",
            "  Downloading nagisa-0.2.7-cp37-cp37m-manylinux1_x86_64.whl (21.5 MB)\n",
            "\u001b[K     |████████████████████████████████| 21.5 MB 1.8 MB/s \n",
            "\u001b[?25hCollecting t5==0.7.1\n",
            "  Downloading t5-0.7.1-py3-none-any.whl (172 kB)\n",
            "\u001b[K     |████████████████████████████████| 172 kB 53.5 MB/s \n",
            "\u001b[?25hRequirement already satisfied: pandas in /usr/local/lib/python3.7/dist-packages (from bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (1.1.5)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (1.19.5)\n",
            "Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (1.4.1)\n",
            "Requirement already satisfied: tensorflow in /usr/local/lib/python3.7/dist-packages (from bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (2.7.0)\n",
            "Collecting tf-slim>=1.1\n",
            "  Downloading tf_slim-1.1.0-py2.py3-none-any.whl (352 kB)\n",
            "\u001b[K     |████████████████████████████████| 352 kB 70.7 MB/s \n",
            "\u001b[?25hCollecting sentencepiece\n",
            "  Downloading sentencepiece-0.1.96-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)\n",
            "\u001b[K     |████████████████████████████████| 1.2 MB 48.1 MB/s \n",
            "\u001b[?25hRequirement already satisfied: toml>=0.10.1 in /usr/local/lib/python3.7/dist-packages (from black==20.8b1->lm-eval==0.0.1) (0.10.2)\n",
            "Collecting mypy-extensions>=0.4.3\n",
            "  Downloading mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)\n",
            "Collecting pathspec<1,>=0.6\n",
            "  Downloading pathspec-0.9.0-py2.py3-none-any.whl (31 kB)\n",
            "Requirement already satisfied: appdirs in /usr/local/lib/python3.7/dist-packages (from black==20.8b1->lm-eval==0.0.1) (1.4.4)\n",
            "Collecting regex>=2020.1.8\n",
            "  Downloading regex-2021.11.10-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (749 kB)\n",
            "\u001b[K     |████████████████████████████████| 749 kB 53.8 MB/s \n",
            "\u001b[?25hRequirement already satisfied: typing-extensions>=3.7.4 in /usr/local/lib/python3.7/dist-packages (from black==20.8b1->lm-eval==0.0.1) (3.10.0.2)\n",
            "Collecting typed-ast>=1.4.0\n",
            "  Downloading typed_ast-1.5.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (843 kB)\n",
            "\u001b[K     |████████████████████████████████| 843 kB 55.8 MB/s \n",
            "\u001b[?25hCollecting ujson\n",
            "  Downloading ujson-4.2.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (214 kB)\n",
            "\u001b[K     |████████████████████████████████| 214 kB 43.1 MB/s \n",
            "\u001b[?25hRequirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from nagisa==0.2.7->lm-eval==0.0.1) (1.15.0)\n",
            "Collecting DyNet\n",
            "  Downloading dyNET-2.1.2-cp37-cp37m-manylinux1_x86_64.whl (4.4 MB)\n",
            "\u001b[K     |████████████████████████████████| 4.4 MB 43.6 MB/s \n",
            "\u001b[?25hRequirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from openai==0.6.4->lm-eval==0.0.1) (4.62.3)\n",
            "Requirement already satisfied: requests>=2.20 in /usr/local/lib/python3.7/dist-packages (from openai==0.6.4->lm-eval==0.0.1) (2.23.0)\n",
            "Collecting typepy[datetime]<2,>=1.1.1\n",
            "  Downloading typepy-1.3.0-py3-none-any.whl (31 kB)\n",
            "Collecting tcolorpy<1,>=0.0.5\n",
            "  Downloading tcolorpy-0.1.1-py3-none-any.whl (8.0 kB)\n",
            "Collecting pathvalidate<3,>=2.3.0\n",
            "  Downloading pathvalidate-2.5.0-py3-none-any.whl (19 kB)\n",
            "Collecting msgfy<1,>=0.1.0\n",
            "  Downloading msgfy-0.2.0-py3-none-any.whl (4.3 kB)\n",
            "Collecting mbstrdecoder<2,>=1.0.0\n",
            "  Downloading mbstrdecoder-1.1.0-py3-none-any.whl (7.8 kB)\n",
            "Collecting tabledata<2,>=1.1.3\n",
            "  Downloading tabledata-1.3.0-py3-none-any.whl (11 kB)\n",
            "Requirement already satisfied: setuptools>=38.3.0 in /usr/local/lib/python3.7/dist-packages (from pytablewriter==0.58.0->lm-eval==0.0.1) (57.4.0)\n",
            "Collecting DataProperty<2,>=0.50.0\n",
            "  Downloading DataProperty-0.54.2-py3-none-any.whl (26 kB)\n",
            "Requirement already satisfied: attrs>=19.2.0 in /usr/local/lib/python3.7/dist-packages (from pytest==6.2.3->lm-eval==0.0.1) (21.2.0)\n",
            "Requirement already satisfied: iniconfig in /usr/local/lib/python3.7/dist-packages (from pytest==6.2.3->lm-eval==0.0.1) (1.1.1)\n",
            "Collecting pluggy<1.0.0a1,>=0.12\n",
            "  Downloading pluggy-0.13.1-py2.py3-none-any.whl (18 kB)\n",
            "Requirement already satisfied: py>=1.8.2 in /usr/local/lib/python3.7/dist-packages (from pytest==6.2.3->lm-eval==0.0.1) (1.11.0)\n",
            "Requirement already satisfied: packaging in /usr/local/lib/python3.7/dist-packages (from pytest==6.2.3->lm-eval==0.0.1) (21.2)\n",
            "Requirement already satisfied: importlib-metadata>=0.12 in /usr/local/lib/python3.7/dist-packages (from pytest==6.2.3->lm-eval==0.0.1) (4.8.2)\n",
            "Collecting portalocker\n",
            "  Downloading portalocker-2.3.2-py2.py3-none-any.whl (15 kB)\n",
            "Collecting tensorflow-text\n",
            "  Downloading tensorflow_text-2.6.0-cp37-cp37m-manylinux1_x86_64.whl (4.4 MB)\n",
            "\u001b[K     |████████████████████████████████| 4.4 MB 41.3 MB/s \n",
            "\u001b[?25hCollecting tfds-nightly\n",
            "  Downloading tfds_nightly-4.4.0.dev202111110107-py3-none-any.whl (4.0 MB)\n",
            "\u001b[K     |████████████████████████████████| 4.0 MB 44.5 MB/s \n",
            "\u001b[?25hRequirement already satisfied: babel in /usr/local/lib/python3.7/dist-packages (from t5==0.7.1->lm-eval==0.0.1) (2.9.1)\n",
            "Collecting rouge-score\n",
            "  Downloading rouge_score-0.0.4-py2.py3-none-any.whl (22 kB)\n",
            "Requirement already satisfied: absl-py in /usr/local/lib/python3.7/dist-packages (from t5==0.7.1->lm-eval==0.0.1) (0.12.0)\n",
            "Requirement already satisfied: nltk in /usr/local/lib/python3.7/dist-packages (from t5==0.7.1->lm-eval==0.0.1) (3.2.5)\n",
            "Collecting mesh-tensorflow[transformer]>=0.1.13\n",
            "  Downloading mesh_tensorflow-0.1.19-py3-none-any.whl (366 kB)\n",
            "\u001b[K     |████████████████████████████████| 366 kB 61.4 MB/s \n",
            "\u001b[?25hRequirement already satisfied: gin-config in /usr/local/lib/python3.7/dist-packages (from t5==0.7.1->lm-eval==0.0.1) (0.5.0)\n",
            "Collecting colorama\n",
            "  Downloading colorama-0.4.4-py2.py3-none-any.whl (16 kB)\n",
            "Collecting rehash\n",
            "  Downloading rehash-1.0.0-py2.py3-none-any.whl (8.7 kB)\n",
            "Requirement already satisfied: multiprocess in /usr/local/lib/python3.7/dist-packages (from datasets>=1.2.1->lm-eval==0.0.1) (0.70.12.2)\n",
            "Collecting huggingface-hub<1.0.0,>=0.1.0\n",
            "  Downloading huggingface_hub-0.1.2-py3-none-any.whl (59 kB)\n",
            "\u001b[K     |████████████████████████████████| 59 kB 5.8 MB/s \n",
            "\u001b[?25hCollecting fsspec[http]>=2021.05.0\n",
            "  Downloading fsspec-2021.11.0-py3-none-any.whl (132 kB)\n",
            "\u001b[K     |████████████████████████████████| 132 kB 46.7 MB/s \n",
            "\u001b[?25hCollecting aiohttp\n",
            "  Downloading aiohttp-3.8.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.1 MB)\n",
            "\u001b[K     |████████████████████████████████| 1.1 MB 56.1 MB/s \n",
            "\u001b[?25hCollecting xxhash\n",
            "  Downloading xxhash-2.0.2-cp37-cp37m-manylinux2010_x86_64.whl (243 kB)\n",
            "\u001b[K     |████████████████████████████████| 243 kB 55.5 MB/s \n",
            "\u001b[?25hRequirement already satisfied: dill in /usr/local/lib/python3.7/dist-packages (from datasets>=1.2.1->lm-eval==0.0.1) (0.3.4)\n",
            "Requirement already satisfied: pyarrow!=4.0.0,>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from datasets>=1.2.1->lm-eval==0.0.1) (3.0.0)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.7/dist-packages (from huggingface-hub<1.0.0,>=0.1.0->datasets>=1.2.1->lm-eval==0.0.1) (3.3.2)\n",
            "Requirement already satisfied: pyyaml in /usr/local/lib/python3.7/dist-packages (from huggingface-hub<1.0.0,>=0.1.0->datasets>=1.2.1->lm-eval==0.0.1) (3.13)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=0.12->pytest==6.2.3->lm-eval==0.0.1) (3.6.0)\n",
            "Requirement already satisfied: chardet<5,>=3.0.4 in /usr/local/lib/python3.7/dist-packages (from mbstrdecoder<2,>=1.0.0->pytablewriter==0.58.0->lm-eval==0.0.1) (3.0.4)\n",
            "Requirement already satisfied: future in /usr/local/lib/python3.7/dist-packages (from mesh-tensorflow[transformer]>=0.1.13->t5==0.7.1->lm-eval==0.0.1) (0.16.0)\n",
            "Requirement already satisfied: tensorflow-datasets in /usr/local/lib/python3.7/dist-packages (from mesh-tensorflow[transformer]>=0.1.13->t5==0.7.1->lm-eval==0.0.1) (4.0.1)\n",
            "Requirement already satisfied: pyparsing<3,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging->pytest==6.2.3->lm-eval==0.0.1) (2.4.7)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests>=2.20->openai==0.6.4->lm-eval==0.0.1) (1.24.3)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests>=2.20->openai==0.6.4->lm-eval==0.0.1) (2.10)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests>=2.20->openai==0.6.4->lm-eval==0.0.1) (2021.10.8)\n",
            "Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=0.24.1->lm-eval==0.0.1) (1.1.0)\n",
            "Collecting threadpoolctl>=2.0.0\n",
            "  Downloading threadpoolctl-3.0.0-py3-none-any.whl (14 kB)\n",
            "Collecting pyyaml\n",
            "  Downloading PyYAML-6.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (596 kB)\n",
            "\u001b[K     |████████████████████████████████| 596 kB 59.9 MB/s \n",
            "\u001b[?25hCollecting sacremoses\n",
            "  Downloading sacremoses-0.0.46-py3-none-any.whl (895 kB)\n",
            "\u001b[K     |████████████████████████████████| 895 kB 56.8 MB/s \n",
            "\u001b[?25hCollecting tokenizers<0.11,>=0.10.1\n",
            "  Downloading tokenizers-0.10.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (3.3 MB)\n",
            "\u001b[K     |████████████████████████████████| 3.3 MB 46.6 MB/s \n",
            "\u001b[?25hRequirement already satisfied: python-dateutil<3.0.0,>=2.8.0 in /usr/local/lib/python3.7/dist-packages (from typepy[datetime]<2,>=1.1.1->pytablewriter==0.58.0->lm-eval==0.0.1) (2.8.2)\n",
            "Requirement already satisfied: pytz>=2018.9 in /usr/local/lib/python3.7/dist-packages (from typepy[datetime]<2,>=1.1.1->pytablewriter==0.58.0->lm-eval==0.0.1) (2018.9)\n",
            "Collecting frozenlist>=1.1.1\n",
            "  Downloading frozenlist-1.2.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (192 kB)\n",
            "\u001b[K     |████████████████████████████████| 192 kB 70.5 MB/s \n",
            "\u001b[?25hRequirement already satisfied: charset-normalizer<3.0,>=2.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets>=1.2.1->lm-eval==0.0.1) (2.0.7)\n",
            "Collecting async-timeout<5.0,>=4.0.0a3\n",
            "  Downloading async_timeout-4.0.1-py3-none-any.whl (5.7 kB)\n",
            "Collecting multidict<7.0,>=4.5\n",
            "  Downloading multidict-5.2.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (160 kB)\n",
            "\u001b[K     |████████████████████████████████| 160 kB 55.6 MB/s \n",
            "\u001b[?25hCollecting aiosignal>=1.1.2\n",
            "  Downloading aiosignal-1.2.0-py3-none-any.whl (8.2 kB)\n",
            "Collecting yarl<2.0,>=1.0\n",
            "  Downloading yarl-1.7.2-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (271 kB)\n",
            "\u001b[K     |████████████████████████████████| 271 kB 59.8 MB/s \n",
            "\u001b[?25hCollecting asynctest==0.13.0\n",
            "  Downloading asynctest-0.13.0-py3-none-any.whl (26 kB)\n",
            "Requirement already satisfied: cython in /usr/local/lib/python3.7/dist-packages (from DyNet->nagisa==0.2.7->lm-eval==0.0.1) (0.29.24)\n",
            "Requirement already satisfied: grpcio<2.0,>=1.24.3 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (1.41.1)\n",
            "Requirement already satisfied: keras<2.8,>=2.7.0rc0 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (2.7.0)\n",
            "Requirement already satisfied: keras-preprocessing>=1.1.1 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (1.1.2)\n",
            "Requirement already satisfied: tensorflow-estimator<2.8,~=2.7.0rc0 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (2.7.0)\n",
            "Requirement already satisfied: libclang>=9.0.1 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (12.0.0)\n",
            "Requirement already satisfied: google-pasta>=0.1.1 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (0.2.0)\n",
            "Requirement already satisfied: protobuf>=3.9.2 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (3.17.3)\n",
            "Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.21.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (0.21.0)\n",
            "Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (3.3.0)\n",
            "Requirement already satisfied: h5py>=2.9.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (3.1.0)\n",
            "Requirement already satisfied: tensorboard~=2.6 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (2.7.0)\n",
            "Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (1.1.0)\n",
            "Requirement already satisfied: flatbuffers<3.0,>=1.12 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (2.0)\n",
            "Requirement already satisfied: astunparse>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (1.6.3)\n",
            "Requirement already satisfied: gast<0.5.0,>=0.2.1 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (0.4.0)\n",
            "Requirement already satisfied: wheel<1.0,>=0.32.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (0.37.0)\n",
            "Requirement already satisfied: wrapt>=1.11.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (1.13.3)\n",
            "Requirement already satisfied: cached-property in /usr/local/lib/python3.7/dist-packages (from h5py>=2.9.0->tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (1.5.2)\n",
            "Requirement already satisfied: google-auth<3,>=1.6.3 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (1.35.0)\n",
            "Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (0.4.6)\n",
            "Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (1.0.1)\n",
            "Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (1.8.0)\n",
            "Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (0.6.1)\n",
            "Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (3.3.4)\n",
            "Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tensorboard~=2.6->tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (4.2.4)\n",
            "Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tensorboard~=2.6->tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (4.7.2)\n",
            "Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tensorboard~=2.6->tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (0.2.8)\n",
            "Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.7/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.6->tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (1.3.0)\n",
            "Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.7/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard~=2.6->tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (0.4.8)\n",
            "Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.6->tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (3.1.1)\n",
            "Requirement already satisfied: dm-tree in /usr/local/lib/python3.7/dist-packages (from tensorflow-datasets->mesh-tensorflow[transformer]>=0.1.13->t5==0.7.1->lm-eval==0.0.1) (0.1.6)\n",
            "Requirement already satisfied: importlib-resources in /usr/local/lib/python3.7/dist-packages (from tensorflow-datasets->mesh-tensorflow[transformer]>=0.1.13->t5==0.7.1->lm-eval==0.0.1) (5.4.0)\n",
            "Requirement already satisfied: promise in /usr/local/lib/python3.7/dist-packages (from tensorflow-datasets->mesh-tensorflow[transformer]>=0.1.13->t5==0.7.1->lm-eval==0.0.1) (2.3)\n",
            "Requirement already satisfied: tensorflow-metadata in /usr/local/lib/python3.7/dist-packages (from tensorflow-datasets->mesh-tensorflow[transformer]>=0.1.13->t5==0.7.1->lm-eval==0.0.1) (1.4.0)\n",
            "Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow-metadata->tensorflow-datasets->mesh-tensorflow[transformer]>=0.1.13->t5==0.7.1->lm-eval==0.0.1) (1.53.0)\n",
            "Requirement already satisfied: tensorflow-hub>=0.8.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow-text->t5==0.7.1->lm-eval==0.0.1) (0.12.0)\n",
            "Collecting tensorflow\n",
            "  Downloading tensorflow-2.6.2-cp37-cp37m-manylinux2010_x86_64.whl (458.3 MB)\n",
            "\u001b[K     |████████████████████████████████| 458.3 MB 12 kB/s \n",
            "\u001b[?25hCollecting typing-extensions>=3.7.4\n",
            "  Downloading typing_extensions-3.7.4.3-py3-none-any.whl (22 kB)\n",
            "Collecting tensorflow\n",
            "  Downloading tensorflow-2.6.1-cp37-cp37m-manylinux2010_x86_64.whl (458.3 MB)\n",
            "\u001b[K     |████████████████████████████████| 458.3 MB 12 kB/s \n",
            "\u001b[?25h  Downloading tensorflow-2.6.0-cp37-cp37m-manylinux2010_x86_64.whl (458.3 MB)\n",
            "\u001b[K     |████████████████████████████████| 458.3 MB 11 kB/s \n",
            "\u001b[?25hCollecting flatbuffers<3.0,>=1.12\n",
            "  Downloading flatbuffers-1.12-py2.py3-none-any.whl (15 kB)\n",
            "Collecting wrapt>=1.11.0\n",
            "  Downloading wrapt-1.12.1.tar.gz (27 kB)\n",
            "Collecting clang~=5.0\n",
            "  Downloading clang-5.0.tar.gz (30 kB)\n",
            "Building wheels for collected packages: lm-eval, bleurt, black, openai, pycountry, sqlitedict, clang, wrapt\n",
            "  Building wheel for lm-eval (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for lm-eval: filename=lm_eval-0.0.1-py3-none-any.whl size=108310 sha256=5c8fc01b7c63a0eb8276badbfd3982c421f89155df811bc4661ed823eedd0d71\n",
            "  Stored in directory: /tmp/pip-ephem-wheel-cache-cyd_4nyq/wheels/6f/98/3f/371e76331d3d53f7bc480c85c56423faadda8d21e848cd7abc\n",
            "  Building wheel for bleurt (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for bleurt: filename=BLEURT-0.0.2-py3-none-any.whl size=16454023 sha256=38a1c6efbdeb96baa21d163cbfb5249eb49027e8367e872d94b86fcdbf3018c4\n",
            "  Stored in directory: /tmp/pip-ephem-wheel-cache-cyd_4nyq/wheels/e4/25/b5/d3b124fc2717a794a8dcb176fac9a01bec46e9f3ff0dbf5d37\n",
            "  Building wheel for black (PEP 517) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for black: filename=black-20.8b1-py3-none-any.whl size=124196 sha256=38a82791608168206858c7f57fd72ceee6d1e31327e0715b97c7ab97ffd15fa0\n",
            "  Stored in directory: /root/.cache/pip/wheels/c5/85/79/f3af8daaf8037c0bf14beb3b7a1511a39b6e6902ca2aaf494e\n",
            "  Building wheel for openai (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for openai: filename=openai-0.6.4-py3-none-any.whl size=172215 sha256=e62a724ae4e3b972a8b5f7a5e1bedfc0fb734bc6599c17d06d538af5b0a5f43a\n",
            "  Stored in directory: /root/.cache/pip/wheels/54/c4/02/aa519fe2aaf97a9bba197a8585182c8c07802760351afdb64a\n",
            "  Building wheel for pycountry (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for pycountry: filename=pycountry-20.7.3-py2.py3-none-any.whl size=10746883 sha256=b91fdc4bf0823fe6d76548fc4633370507c5f05e76c63dc6086d00d0ed755bb6\n",
            "  Stored in directory: /root/.cache/pip/wheels/57/e8/3f/120ccc1ff7541c108bc5d656e2a14c39da0d824653b62284c6\n",
            "  Building wheel for sqlitedict (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for sqlitedict: filename=sqlitedict-1.6.0-py3-none-any.whl size=14712 sha256=526987c784db1cfced59c97d786b10b9356fcae70597ca74d3a73f5e8b78dbf8\n",
            "  Stored in directory: /root/.cache/pip/wheels/58/dd/2c/0a57aadf6a7f26bec0af66d742c50af74d11967780f0bb7a7d\n",
            "  Building wheel for clang (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for clang: filename=clang-5.0-py3-none-any.whl size=30692 sha256=abc56cd62ac4248b8eb93b1a9d05c202e120a29511ad96c7954313f52d191dd2\n",
            "  Stored in directory: /root/.cache/pip/wheels/98/91/04/971b4c587cf47ae952b108949b46926f426c02832d120a082a\n",
            "  Building wheel for wrapt (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for wrapt: filename=wrapt-1.12.1-cp37-cp37m-linux_x86_64.whl size=68720 sha256=1304a7f4e36618024df0ea93bc04eb2cd1a038d46479b73da02fa971ea6ae4f1\n",
            "  Stored in directory: /root/.cache/pip/wheels/62/76/4c/aa25851149f3f6d9785f6c869387ad82b3fd37582fa8147ac6\n",
            "Successfully built lm-eval bleurt black openai pycountry sqlitedict clang wrapt\n",
            "Installing collected packages: typing-extensions, mbstrdecoder, typepy, multidict, frozenlist, yarl, wrapt, regex, pyyaml, flatbuffers, clang, asynctest, async-timeout, aiosignal, tokenizers, threadpoolctl, tensorflow, sacremoses, portalocker, mesh-tensorflow, huggingface-hub, fsspec, DataProperty, aiohttp, zstandard, xxhash, ujson, typed-ast, transformers, tfds-nightly, tf-slim, tensorflow-text, tcolorpy, tabledata, sentencepiece, scikit-learn, sacrebleu, rouge-score, rehash, pluggy, pathvalidate, pathspec, mypy-extensions, msgfy, jsonlines, DyNet, colorama, tqdm-multiprocess, t5, sqlitedict, pytest, pytablewriter, pycountry, pybind11, openai, numexpr, nagisa, mock, lm-dataformat, datasets, bleurt, black, best-download, lm-eval\n",
            "  Attempting uninstall: typing-extensions\n",
            "    Found existing installation: typing-extensions 3.10.0.2\n",
            "    Uninstalling typing-extensions-3.10.0.2:\n",
            "      Successfully uninstalled typing-extensions-3.10.0.2\n",
            "  Attempting uninstall: wrapt\n",
            "    Found existing installation: wrapt 1.13.3\n",
            "    Uninstalling wrapt-1.13.3:\n",
            "      Successfully uninstalled wrapt-1.13.3\n",
            "  Attempting uninstall: regex\n",
            "    Found existing installation: regex 2019.12.20\n",
            "    Uninstalling regex-2019.12.20:\n",
            "      Successfully uninstalled regex-2019.12.20\n",
            "  Attempting uninstall: pyyaml\n",
            "    Found existing installation: PyYAML 3.13\n",
            "    Uninstalling PyYAML-3.13:\n",
            "      Successfully uninstalled PyYAML-3.13\n",
            "  Attempting uninstall: flatbuffers\n",
            "    Found existing installation: flatbuffers 2.0\n",
            "    Uninstalling flatbuffers-2.0:\n",
            "      Successfully uninstalled flatbuffers-2.0\n",
            "  Attempting uninstall: tensorflow\n",
            "    Found existing installation: tensorflow 2.7.0\n",
            "    Uninstalling tensorflow-2.7.0:\n",
            "      Successfully uninstalled tensorflow-2.7.0\n",
            "  Attempting uninstall: scikit-learn\n",
            "    Found existing installation: scikit-learn 0.22.2.post1\n",
            "    Uninstalling scikit-learn-0.22.2.post1:\n",
            "      Successfully uninstalled scikit-learn-0.22.2.post1\n",
            "  Attempting uninstall: pluggy\n",
            "    Found existing installation: pluggy 0.7.1\n",
            "    Uninstalling pluggy-0.7.1:\n",
            "      Successfully uninstalled pluggy-0.7.1\n",
            "  Attempting uninstall: pytest\n",
            "    Found existing installation: pytest 3.6.4\n",
            "    Uninstalling pytest-3.6.4:\n",
            "      Successfully uninstalled pytest-3.6.4\n",
            "  Attempting uninstall: numexpr\n",
            "    Found existing installation: numexpr 2.7.3\n",
            "    Uninstalling numexpr-2.7.3:\n",
            "      Successfully uninstalled numexpr-2.7.3\n",
            "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "optax 0.0.9 requires typing-extensions~=3.10.0, but you have typing-extensions 3.7.4.3 which is incompatible.\n",
            "datascience 0.10.6 requires folium==0.2.1, but you have folium 0.8.3 which is incompatible.\u001b[0m\n",
            "Successfully installed DataProperty-0.54.2 DyNet-2.1.2 aiohttp-3.8.0 aiosignal-1.2.0 async-timeout-4.0.1 asynctest-0.13.0 best-download-0.0.7 black-20.8b1 bleurt-0.0.2 clang-5.0 colorama-0.4.4 datasets-1.15.1 flatbuffers-1.12 frozenlist-1.2.0 fsspec-2021.11.0 huggingface-hub-0.1.2 jsonlines-2.0.0 lm-dataformat-0.0.19 lm-eval-0.0.1 mbstrdecoder-1.1.0 mesh-tensorflow-0.1.19 mock-4.0.3 msgfy-0.2.0 multidict-5.2.0 mypy-extensions-0.4.3 nagisa-0.2.7 numexpr-2.7.2 openai-0.6.4 pathspec-0.9.0 pathvalidate-2.5.0 pluggy-0.13.1 portalocker-2.3.2 pybind11-2.6.2 pycountry-20.7.3 pytablewriter-0.58.0 pytest-6.2.3 pyyaml-6.0 regex-2021.11.10 rehash-1.0.0 rouge-score-0.0.4 sacrebleu-1.5.0 sacremoses-0.0.46 scikit-learn-1.0.1 sentencepiece-0.1.96 sqlitedict-1.6.0 t5-0.7.1 tabledata-1.3.0 tcolorpy-0.1.1 tensorflow-2.6.0 tensorflow-text-2.6.0 tf-slim-1.1.0 tfds-nightly-4.4.0.dev202111110107 threadpoolctl-3.0.0 tokenizers-0.10.3 tqdm-multiprocess-0.0.11 transformers-4.12.3 typed-ast-1.5.0 typepy-1.3.0 typing-extensions-3.7.4.3 ujson-4.2.0 wrapt-1.12.1 xxhash-2.0.2 yarl-1.7.2 zstandard-0.15.2\n",
            "Requirement already satisfied: transformers in /usr/local/lib/python3.7/dist-packages (4.12.3)\n",
            "Collecting fabric\n",
            "  Downloading fabric-2.6.0-py2.py3-none-any.whl (53 kB)\n",
            "\u001b[K     |████████████████████████████████| 53 kB 1.2 MB/s \n",
            "\u001b[?25hCollecting wandb\n",
            "  Downloading wandb-0.12.6-py2.py3-none-any.whl (1.7 MB)\n",
            "\u001b[K     |████████████████████████████████| 1.7 MB 9.6 MB/s \n",
            "\u001b[?25hRequirement already satisfied: google-cloud-storage in /usr/local/lib/python3.7/dist-packages (1.18.1)\n",
            "Requirement already satisfied: cloudpickle in /usr/local/lib/python3.7/dist-packages (1.3.0)\n",
            "Requirement already satisfied: Flask in /usr/local/lib/python3.7/dist-packages (1.1.4)\n",
            "Collecting ray[default]\n",
            "  Downloading ray-1.8.0-cp37-cp37m-manylinux2014_x86_64.whl (54.7 MB)\n",
            "\u001b[K     |████████████████████████████████| 54.7 MB 72 kB/s \n",
            "\u001b[?25hCollecting einops\n",
            "  Downloading einops-0.3.2-py3-none-any.whl (25 kB)\n",
            "Requirement already satisfied: smart_open[gcs] in /usr/local/lib/python3.7/dist-packages (5.2.1)\n",
            "Collecting func_timeout\n",
            "  Downloading func_timeout-4.3.5.tar.gz (44 kB)\n",
            "\u001b[K     |████████████████████████████████| 44 kB 2.0 MB/s \n",
            "\u001b[?25hCollecting ftfy\n",
            "  Downloading ftfy-6.0.3.tar.gz (64 kB)\n",
            "\u001b[K     |████████████████████████████████| 64 kB 2.5 MB/s \n",
            "\u001b[?25hCollecting fastapi\n",
            "  Downloading fastapi-0.70.0-py3-none-any.whl (51 kB)\n",
            "\u001b[K     |████████████████████████████████| 51 kB 509 kB/s \n",
            "\u001b[?25hCollecting uvicorn\n",
            "  Downloading uvicorn-0.15.0-py3-none-any.whl (54 kB)\n",
            "\u001b[K     |████████████████████████████████| 54 kB 2.4 MB/s \n",
            "\u001b[?25hRequirement already satisfied: lm_dataformat in /usr/local/lib/python3.7/dist-packages (0.0.19)\n",
            "Collecting pathy\n",
            "  Downloading pathy-0.6.1-py3-none-any.whl (42 kB)\n",
            "\u001b[K     |████████████████████████████████| 42 kB 1.1 MB/s \n",
            "\u001b[?25hRequirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.7/dist-packages (from transformers) (21.2)\n",
            "Requirement already satisfied: sacremoses in /usr/local/lib/python3.7/dist-packages (from transformers) (0.0.46)\n",
            "Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/dist-packages (from transformers) (4.8.2)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from transformers) (2.23.0)\n",
            "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.7/dist-packages (from transformers) (1.19.5)\n",
            "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.7/dist-packages (from transformers) (2021.11.10)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.7/dist-packages (from transformers) (3.3.2)\n",
            "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.7/dist-packages (from transformers) (6.0)\n",
            "Requirement already satisfied: huggingface-hub<1.0,>=0.1.0 in /usr/local/lib/python3.7/dist-packages (from transformers) (0.1.2)\n",
            "Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.7/dist-packages (from transformers) (4.62.3)\n",
            "Requirement already satisfied: tokenizers<0.11,>=0.10.1 in /usr/local/lib/python3.7/dist-packages (from transformers) (0.10.3)\n",
            "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.7/dist-packages (from huggingface-hub<1.0,>=0.1.0->transformers) (3.7.4.3)\n",
            "Requirement already satisfied: pyparsing<3,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging>=20.0->transformers) (2.4.7)\n",
            "Collecting paramiko>=2.4\n",
            "  Downloading paramiko-2.8.0-py2.py3-none-any.whl (206 kB)\n",
            "\u001b[K     |████████████████████████████████| 206 kB 47.8 MB/s \n",
            "\u001b[?25hCollecting invoke<2.0,>=1.3\n",
            "  Downloading invoke-1.6.0-py3-none-any.whl (212 kB)\n",
            "\u001b[K     |████████████████████████████████| 212 kB 47.7 MB/s \n",
            "\u001b[?25hCollecting pathlib2\n",
            "  Downloading pathlib2-2.3.6-py2.py3-none-any.whl (17 kB)\n",
            "Collecting cryptography>=2.5\n",
            "  Downloading cryptography-35.0.0-cp36-abi3-manylinux_2_24_x86_64.whl (3.5 MB)\n",
            "\u001b[K     |████████████████████████████████| 3.5 MB 56.1 MB/s \n",
            "\u001b[?25hCollecting pynacl>=1.0.1\n",
            "  Downloading PyNaCl-1.4.0-cp35-abi3-manylinux1_x86_64.whl (961 kB)\n",
            "\u001b[K     |████████████████████████████████| 961 kB 33.0 MB/s \n",
            "\u001b[?25hCollecting bcrypt>=3.1.3\n",
            "  Downloading bcrypt-3.2.0-cp36-abi3-manylinux2010_x86_64.whl (63 kB)\n",
            "\u001b[K     |████████████████████████████████| 63 kB 2.0 MB/s \n",
            "\u001b[?25hRequirement already satisfied: cffi>=1.1 in /usr/local/lib/python3.7/dist-packages (from bcrypt>=3.1.3->paramiko>=2.4->fabric) (1.15.0)\n",
            "Requirement already satisfied: six>=1.4.1 in /usr/local/lib/python3.7/dist-packages (from bcrypt>=3.1.3->paramiko>=2.4->fabric) (1.15.0)\n",
            "Requirement already satisfied: pycparser in /usr/local/lib/python3.7/dist-packages (from cffi>=1.1->bcrypt>=3.1.3->paramiko>=2.4->fabric) (2.21)\n",
            "Requirement already satisfied: promise<3,>=2.0 in /usr/local/lib/python3.7/dist-packages (from wandb) (2.3)\n",
            "Requirement already satisfied: psutil>=5.0.0 in /usr/local/lib/python3.7/dist-packages (from wandb) (5.4.8)\n",
            "Collecting pathtools\n",
            "  Downloading pathtools-0.1.2.tar.gz (11 kB)\n",
            "Collecting yaspin>=1.0.0\n",
            "  Downloading yaspin-2.1.0-py3-none-any.whl (18 kB)\n",
            "Requirement already satisfied: python-dateutil>=2.6.1 in /usr/local/lib/python3.7/dist-packages (from wandb) (2.8.2)\n",
            "Collecting shortuuid>=0.5.0\n",
            "  Downloading shortuuid-1.0.8-py3-none-any.whl (9.5 kB)\n",
            "Collecting GitPython>=1.0.0\n",
            "  Downloading GitPython-3.1.24-py3-none-any.whl (180 kB)\n",
            "\u001b[K     |████████████████████████████████| 180 kB 39.1 MB/s \n",
            "\u001b[?25hCollecting subprocess32>=3.5.3\n",
            "  Downloading subprocess32-3.5.4.tar.gz (97 kB)\n",
            "\u001b[K     |████████████████████████████████| 97 kB 5.3 MB/s \n",
            "\u001b[?25hCollecting sentry-sdk>=1.0.0\n",
            "  Downloading sentry_sdk-1.4.3-py2.py3-none-any.whl (139 kB)\n",
            "\u001b[K     |████████████████████████████████| 139 kB 35.4 MB/s \n",
            "\u001b[?25hCollecting configparser>=3.8.1\n",
            "  Downloading configparser-5.1.0-py3-none-any.whl (19 kB)\n",
            "Collecting docker-pycreds>=0.4.0\n",
            "  Downloading docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB)\n",
            "Requirement already satisfied: protobuf>=3.12.0 in /usr/local/lib/python3.7/dist-packages (from wandb) (3.17.3)\n",
            "Requirement already satisfied: Click!=8.0.0,>=7.0 in /usr/local/lib/python3.7/dist-packages (from wandb) (7.1.2)\n",
            "Collecting gitdb<5,>=4.0.1\n",
            "  Downloading gitdb-4.0.9-py3-none-any.whl (63 kB)\n",
            "\u001b[K     |████████████████████████████████| 63 kB 1.3 MB/s \n",
            "\u001b[?25hCollecting smmap<6,>=3.0.1\n",
            "  Downloading smmap-5.0.0-py3-none-any.whl (24 kB)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (3.0.4)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (1.24.3)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (2.10)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (2021.10.8)\n",
            "Requirement already satisfied: termcolor<2.0.0,>=1.1.0 in /usr/local/lib/python3.7/dist-packages (from yaspin>=1.0.0->wandb) (1.1.0)\n",
            "Requirement already satisfied: google-auth>=1.2.0 in /usr/local/lib/python3.7/dist-packages (from google-cloud-storage) (1.35.0)\n",
            "Requirement already satisfied: google-cloud-core<2.0dev,>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from google-cloud-storage) (1.0.3)\n",
            "Requirement already satisfied: google-resumable-media<0.5.0dev,>=0.3.1 in /usr/local/lib/python3.7/dist-packages (from google-cloud-storage) (0.4.1)\n",
            "Requirement already satisfied: setuptools>=40.3.0 in /usr/local/lib/python3.7/dist-packages (from google-auth>=1.2.0->google-cloud-storage) (57.4.0)\n",
            "Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.7/dist-packages (from google-auth>=1.2.0->google-cloud-storage) (4.7.2)\n",
            "Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from google-auth>=1.2.0->google-cloud-storage) (4.2.4)\n",
            "Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.7/dist-packages (from google-auth>=1.2.0->google-cloud-storage) (0.2.8)\n",
            "Requirement already satisfied: google-api-core<2.0.0dev,>=1.14.0 in /usr/local/lib/python3.7/dist-packages (from google-cloud-core<2.0dev,>=1.0.0->google-cloud-storage) (1.26.3)\n",
            "Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from google-api-core<2.0.0dev,>=1.14.0->google-cloud-core<2.0dev,>=1.0.0->google-cloud-storage) (1.53.0)\n",
            "Requirement already satisfied: pytz in /usr/local/lib/python3.7/dist-packages (from google-api-core<2.0.0dev,>=1.14.0->google-cloud-core<2.0dev,>=1.0.0->google-cloud-storage) (2018.9)\n",
            "Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.7/dist-packages (from pyasn1-modules>=0.2.1->google-auth>=1.2.0->google-cloud-storage) (0.4.8)\n",
            "Requirement already satisfied: Jinja2<3.0,>=2.10.1 in /usr/local/lib/python3.7/dist-packages (from Flask) (2.11.3)\n",
            "Requirement already satisfied: itsdangerous<2.0,>=0.24 in /usr/local/lib/python3.7/dist-packages (from Flask) (1.1.0)\n",
            "Requirement already satisfied: Werkzeug<2.0,>=0.15 in /usr/local/lib/python3.7/dist-packages (from Flask) (1.0.1)\n",
            "Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.7/dist-packages (from Jinja2<3.0,>=2.10.1->Flask) (2.0.1)\n",
            "Requirement already satisfied: wcwidth in /usr/local/lib/python3.7/dist-packages (from ftfy) (0.2.5)\n",
            "Collecting pydantic!=1.7,!=1.7.1,!=1.7.2,!=1.7.3,!=1.8,!=1.8.1,<2.0.0,>=1.6.2\n",
            "  Downloading pydantic-1.8.2-cp37-cp37m-manylinux2014_x86_64.whl (10.1 MB)\n",
            "\u001b[K     |████████████████████████████████| 10.1 MB 53.0 MB/s \n",
            "\u001b[?25hCollecting starlette==0.16.0\n",
            "  Downloading starlette-0.16.0-py3-none-any.whl (61 kB)\n",
            "\u001b[K     |████████████████████████████████| 61 kB 231 kB/s \n",
            "\u001b[?25hCollecting anyio<4,>=3.0.0\n",
            "  Downloading anyio-3.3.4-py3-none-any.whl (78 kB)\n",
            "\u001b[K     |████████████████████████████████| 78 kB 1.5 MB/s \n",
            "\u001b[?25hCollecting sniffio>=1.1\n",
            "  Downloading sniffio-1.2.0-py3-none-any.whl (10 kB)\n",
            "Collecting asgiref>=3.4.0\n",
            "  Downloading asgiref-3.4.1-py3-none-any.whl (25 kB)\n",
            "Collecting h11>=0.8\n",
            "  Downloading h11-0.12.0-py3-none-any.whl (54 kB)\n",
            "\u001b[K     |████████████████████████████████| 54 kB 2.4 MB/s \n",
            "\u001b[?25hRequirement already satisfied: ujson in /usr/local/lib/python3.7/dist-packages (from lm_dataformat) (4.2.0)\n",
            "Requirement already satisfied: zstandard in /usr/local/lib/python3.7/dist-packages (from lm_dataformat) (0.15.2)\n",
            "Requirement already satisfied: jsonlines in /usr/local/lib/python3.7/dist-packages (from lm_dataformat) (2.0.0)\n",
            "Collecting typer<1.0.0,>=0.3.0\n",
            "  Downloading typer-0.4.0-py3-none-any.whl (27 kB)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata->transformers) (3.6.0)\n",
            "Requirement already satisfied: msgpack<2.0.0,>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from ray[default]) (1.0.2)\n",
            "Collecting redis>=3.5.0\n",
            "  Downloading redis-3.5.3-py2.py3-none-any.whl (72 kB)\n",
            "\u001b[K     |████████████████████████████████| 72 kB 439 kB/s \n",
            "\u001b[?25hRequirement already satisfied: attrs in /usr/local/lib/python3.7/dist-packages (from ray[default]) (21.2.0)\n",
            "Requirement already satisfied: grpcio>=1.28.1 in /usr/local/lib/python3.7/dist-packages (from ray[default]) (1.41.1)\n",
            "Requirement already satisfied: jsonschema in /usr/local/lib/python3.7/dist-packages (from ray[default]) (2.6.0)\n",
            "Collecting aiohttp-cors\n",
            "  Downloading aiohttp_cors-0.7.0-py3-none-any.whl (27 kB)\n",
            "Collecting opencensus\n",
            "  Downloading opencensus-0.8.0-py2.py3-none-any.whl (128 kB)\n",
            "\u001b[K     |████████████████████████████████| 128 kB 46.4 MB/s \n",
            "\u001b[?25hCollecting aioredis<2\n",
            "  Downloading aioredis-1.3.1-py3-none-any.whl (65 kB)\n",
            "\u001b[K     |████████████████████████████████| 65 kB 3.5 MB/s \n",
            "\u001b[?25hRequirement already satisfied: aiohttp>=3.7 in /usr/local/lib/python3.7/dist-packages (from ray[default]) (3.8.0)\n",
            "Collecting gpustat>=1.0.0b1\n",
            "  Downloading gpustat-1.0.0b1.tar.gz (82 kB)\n",
            "\u001b[K     |████████████████████████████████| 82 kB 207 kB/s \n",
            "\u001b[?25hCollecting colorful\n",
            "  Downloading colorful-0.5.4-py2.py3-none-any.whl (201 kB)\n",
            "\u001b[K     |████████████████████████████████| 201 kB 34.5 MB/s \n",
            "\u001b[?25hRequirement already satisfied: prometheus-client>=0.7.1 in /usr/local/lib/python3.7/dist-packages (from ray[default]) (0.12.0)\n",
            "Collecting py-spy>=0.2.0\n",
            "  Downloading py_spy-0.3.10-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (3.2 MB)\n",
            "\u001b[K     |████████████████████████████████| 3.2 MB 45.9 MB/s \n",
            "\u001b[?25hRequirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.7/dist-packages (from aiohttp>=3.7->ray[default]) (4.0.1)\n",
            "Requirement already satisfied: asynctest==0.13.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp>=3.7->ray[default]) (0.13.0)\n",
            "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp>=3.7->ray[default]) (1.7.2)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.7/dist-packages (from aiohttp>=3.7->ray[default]) (1.2.0)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.7/dist-packages (from aiohttp>=3.7->ray[default]) (1.2.0)\n",
            "Requirement already satisfied: charset-normalizer<3.0,>=2.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp>=3.7->ray[default]) (2.0.7)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.7/dist-packages (from aiohttp>=3.7->ray[default]) (5.2.0)\n",
            "Collecting hiredis\n",
            "  Downloading hiredis-2.0.0-cp37-cp37m-manylinux2010_x86_64.whl (85 kB)\n",
            "\u001b[K     |████████████████████████████████| 85 kB 3.5 MB/s \n",
            "\u001b[?25hRequirement already satisfied: nvidia-ml-py3>=7.352.0 in /usr/local/lib/python3.7/dist-packages (from gpustat>=1.0.0b1->ray[default]) (7.352.0)\n",
            "Collecting blessed>=1.17.1\n",
            "  Downloading blessed-1.19.0-py2.py3-none-any.whl (57 kB)\n",
            "\u001b[K     |████████████████████████████████| 57 kB 4.9 MB/s \n",
            "\u001b[?25hCollecting opencensus-context==0.1.2\n",
            "  Downloading opencensus_context-0.1.2-py2.py3-none-any.whl (4.4 kB)\n",
            "Requirement already satisfied: joblib in /usr/local/lib/python3.7/dist-packages (from sacremoses->transformers) (1.1.0)\n",
            "Building wheels for collected packages: subprocess32, func-timeout, ftfy, pathtools, gpustat\n",
            "  Building wheel for subprocess32 (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for subprocess32: filename=subprocess32-3.5.4-py3-none-any.whl size=6502 sha256=355942b9c226a274f7dad0518c95569b0af6e30616930bcd404766676597350a\n",
            "  Stored in directory: /root/.cache/pip/wheels/50/ca/fa/8fca8d246e64f19488d07567547ddec8eb084e8c0d7a59226a\n",
            "  Building wheel for func-timeout (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for func-timeout: filename=func_timeout-4.3.5-py3-none-any.whl size=15095 sha256=1e4f20e1cd6f140537e4694777f337afa342eb5e4cc17701bd480cf83ae5a12a\n",
            "  Stored in directory: /root/.cache/pip/wheels/68/b5/a5/67c4364c354e141f5a1bd3ec568126f77877ab7554cf5af8cb\n",
            "  Building wheel for ftfy (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for ftfy: filename=ftfy-6.0.3-py3-none-any.whl size=41933 sha256=b05fa7fa12ca9211c8733b8f27b3d02c19aa5e9b861c59d3c105087f9d2f9e25\n",
            "  Stored in directory: /root/.cache/pip/wheels/19/f5/38/273eb3b5e76dfd850619312f693716ac4518b498f5ffb6f56d\n",
            "  Building wheel for pathtools (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for pathtools: filename=pathtools-0.1.2-py3-none-any.whl size=8807 sha256=dc0a6dbacf60bd4c0a66b72add9d58e1339eae64e5fd1846ad77bf67f36aebe1\n",
            "  Stored in directory: /root/.cache/pip/wheels/3e/31/09/fa59cef12cdcfecc627b3d24273699f390e71828921b2cbba2\n",
            "  Building wheel for gpustat (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for gpustat: filename=gpustat-1.0.0b1-py3-none-any.whl size=15979 sha256=711db26002b23c50aed72df43b2bea1690701a209286008b085ca68907fe93f8\n",
            "  Stored in directory: /root/.cache/pip/wheels/1a/16/e2/3e2437fba4c4b6a97a97bd96fce5d14e66cff5c4966fb1cc8c\n",
            "Successfully built subprocess32 func-timeout ftfy pathtools gpustat\n",
            "Installing collected packages: sniffio, smmap, redis, pynacl, opencensus-context, hiredis, gitdb, cryptography, blessed, bcrypt, anyio, yaspin, typer, subprocess32, starlette, shortuuid, sentry-sdk, ray, pydantic, py-spy, pathtools, pathlib2, paramiko, opencensus, invoke, h11, gpustat, GitPython, docker-pycreds, configparser, colorful, asgiref, aioredis, aiohttp-cors, wandb, uvicorn, pathy, func-timeout, ftfy, fastapi, fabric, einops\n",
            "Successfully installed GitPython-3.1.24 aiohttp-cors-0.7.0 aioredis-1.3.1 anyio-3.3.4 asgiref-3.4.1 bcrypt-3.2.0 blessed-1.19.0 colorful-0.5.4 configparser-5.1.0 cryptography-35.0.0 docker-pycreds-0.4.0 einops-0.3.2 fabric-2.6.0 fastapi-0.70.0 ftfy-6.0.3 func-timeout-4.3.5 gitdb-4.0.9 gpustat-1.0.0b1 h11-0.12.0 hiredis-2.0.0 invoke-1.6.0 opencensus-0.8.0 opencensus-context-0.1.2 paramiko-2.8.0 pathlib2-2.3.6 pathtools-0.1.2 pathy-0.6.1 py-spy-0.3.10 pydantic-1.8.2 pynacl-1.4.0 ray-1.8.0 redis-3.5.3 sentry-sdk-1.4.3 shortuuid-1.0.8 smmap-5.0.0 sniffio-1.2.0 starlette-0.16.0 subprocess32-3.5.4 typer-0.4.0 uvicorn-0.15.0 wandb-0.12.6 yaspin-2.1.0\n",
            "Collecting git+https://github.com/deepmind/dm-haiku\n",
            "  Cloning https://github.com/deepmind/dm-haiku to /tmp/pip-req-build-2wziqdmx\n",
            "  Running command git clone -q https://github.com/deepmind/dm-haiku /tmp/pip-req-build-2wziqdmx\n",
            "Requirement already satisfied: absl-py>=0.7.1 in /usr/local/lib/python3.7/dist-packages (from dm-haiku==0.0.6.dev0) (0.12.0)\n",
            "Collecting jmp>=0.0.2\n",
            "  Downloading jmp-0.0.2-py3-none-any.whl (16 kB)\n",
            "Requirement already satisfied: numpy>=1.18.0 in /usr/local/lib/python3.7/dist-packages (from dm-haiku==0.0.6.dev0) (1.19.5)\n",
            "Requirement already satisfied: tabulate>=0.8.9 in /usr/local/lib/python3.7/dist-packages (from dm-haiku==0.0.6.dev0) (0.8.9)\n",
            "Requirement already satisfied: typing_extensions in /usr/local/lib/python3.7/dist-packages (from dm-haiku==0.0.6.dev0) (3.7.4.3)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from absl-py>=0.7.1->dm-haiku==0.0.6.dev0) (1.15.0)\n",
            "Building wheels for collected packages: dm-haiku\n",
            "  Building wheel for dm-haiku (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for dm-haiku: filename=dm_haiku-0.0.6.dev0-py3-none-any.whl size=537934 sha256=263c2ac7a2c75b8fa5acd110ebacb2c02584ab91e511f6b55432b184194125f2\n",
            "  Stored in directory: /tmp/pip-ephem-wheel-cache-o668gx72/wheels/06/28/69/ebaac5b2435641427299f29d88d005fb4e2627f4a108f0bdbc\n",
            "Successfully built dm-haiku\n",
            "Installing collected packages: jmp, dm-haiku\n",
            "Successfully installed dm-haiku-0.0.6.dev0 jmp-0.0.2\n",
            "Collecting git+https://github.com/EleutherAI/lm-evaluation-harness/\n",
            "  Cloning https://github.com/EleutherAI/lm-evaluation-harness/ to /tmp/pip-req-build-lvnftijn\n",
            "  Running command git clone -q https://github.com/EleutherAI/lm-evaluation-harness/ /tmp/pip-req-build-lvnftijn\n",
            "Collecting bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt\n",
            "  Using cached https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip\n",
            "Requirement already satisfied: black==20.8b1 in /usr/local/lib/python3.7/dist-packages (from lm-eval==0.0.1) (20.8b1)\n",
            "Requirement already satisfied: best_download>=0.0.6 in /usr/local/lib/python3.7/dist-packages (from lm-eval==0.0.1) (0.0.7)\n",
            "Requirement already satisfied: datasets>=1.2.1 in /usr/local/lib/python3.7/dist-packages (from lm-eval==0.0.1) (1.15.1)\n",
            "Requirement already satisfied: click>=7.1 in /usr/local/lib/python3.7/dist-packages (from lm-eval==0.0.1) (7.1.2)\n",
            "Requirement already satisfied: scikit-learn>=0.24.1 in /usr/local/lib/python3.7/dist-packages (from lm-eval==0.0.1) (1.0.1)\n",
            "Requirement already satisfied: torch>=1.7 in /usr/local/lib/python3.7/dist-packages (from lm-eval==0.0.1) (1.10.0+cu111)\n",
            "Requirement already satisfied: transformers>=4.1 in /usr/local/lib/python3.7/dist-packages (from lm-eval==0.0.1) (4.12.3)\n",
            "Requirement already satisfied: sqlitedict==1.6.0 in /usr/local/lib/python3.7/dist-packages (from lm-eval==0.0.1) (1.6.0)\n",
            "Requirement already satisfied: pytablewriter==0.58.0 in /usr/local/lib/python3.7/dist-packages (from lm-eval==0.0.1) (0.58.0)\n",
            "Requirement already satisfied: sacrebleu==1.5.0 in /usr/local/lib/python3.7/dist-packages (from lm-eval==0.0.1) (1.5.0)\n",
            "Requirement already satisfied: pycountry==20.7.3 in /usr/local/lib/python3.7/dist-packages (from lm-eval==0.0.1) (20.7.3)\n",
            "Requirement already satisfied: numexpr==2.7.2 in /usr/local/lib/python3.7/dist-packages (from lm-eval==0.0.1) (2.7.2)\n",
            "Requirement already satisfied: lm_dataformat==0.0.19 in /usr/local/lib/python3.7/dist-packages (from lm-eval==0.0.1) (0.0.19)\n",
            "Requirement already satisfied: pytest==6.2.3 in /usr/local/lib/python3.7/dist-packages (from lm-eval==0.0.1) (6.2.3)\n",
            "Requirement already satisfied: pybind11==2.6.2 in /usr/local/lib/python3.7/dist-packages (from lm-eval==0.0.1) (2.6.2)\n",
            "Requirement already satisfied: tqdm-multiprocess==0.0.11 in /usr/local/lib/python3.7/dist-packages (from lm-eval==0.0.1) (0.0.11)\n",
            "Requirement already satisfied: zstandard==0.15.2 in /usr/local/lib/python3.7/dist-packages (from lm-eval==0.0.1) (0.15.2)\n",
            "Requirement already satisfied: jsonlines==2.0.0 in /usr/local/lib/python3.7/dist-packages (from lm-eval==0.0.1) (2.0.0)\n",
            "Requirement already satisfied: mock==4.0.3 in /usr/local/lib/python3.7/dist-packages (from lm-eval==0.0.1) (4.0.3)\n",
            "Requirement already satisfied: openai==0.6.4 in /usr/local/lib/python3.7/dist-packages (from lm-eval==0.0.1) (0.6.4)\n",
            "Requirement already satisfied: jieba==0.42.1 in /usr/local/lib/python3.7/dist-packages (from lm-eval==0.0.1) (0.42.1)\n",
            "Requirement already satisfied: nagisa==0.2.7 in /usr/local/lib/python3.7/dist-packages (from lm-eval==0.0.1) (0.2.7)\n",
            "Requirement already satisfied: t5==0.7.1 in /usr/local/lib/python3.7/dist-packages (from lm-eval==0.0.1) (0.7.1)\n",
            "Collecting tensorflow-estimator==2.6.0\n",
            "  Downloading tensorflow_estimator-2.6.0-py2.py3-none-any.whl (462 kB)\n",
            "\u001b[K     |████████████████████████████████| 462 kB 5.1 MB/s \n",
            "\u001b[?25hRequirement already satisfied: pandas in /usr/local/lib/python3.7/dist-packages (from bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (1.1.5)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (1.19.5)\n",
            "Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (1.4.1)\n",
            "Requirement already satisfied: tensorflow in /usr/local/lib/python3.7/dist-packages (from bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (2.6.0)\n",
            "Requirement already satisfied: tf-slim>=1.1 in /usr/local/lib/python3.7/dist-packages (from bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (1.1.0)\n",
            "Requirement already satisfied: sentencepiece in /usr/local/lib/python3.7/dist-packages (from bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (0.1.96)\n",
            "Requirement already satisfied: typed-ast>=1.4.0 in /usr/local/lib/python3.7/dist-packages (from black==20.8b1->lm-eval==0.0.1) (1.5.0)\n",
            "Requirement already satisfied: toml>=0.10.1 in /usr/local/lib/python3.7/dist-packages (from black==20.8b1->lm-eval==0.0.1) (0.10.2)\n",
            "Requirement already satisfied: typing-extensions>=3.7.4 in /usr/local/lib/python3.7/dist-packages (from black==20.8b1->lm-eval==0.0.1) (3.7.4.3)\n",
            "Requirement already satisfied: regex>=2020.1.8 in /usr/local/lib/python3.7/dist-packages (from black==20.8b1->lm-eval==0.0.1) (2021.11.10)\n",
            "Requirement already satisfied: appdirs in /usr/local/lib/python3.7/dist-packages (from black==20.8b1->lm-eval==0.0.1) (1.4.4)\n",
            "Requirement already satisfied: pathspec<1,>=0.6 in /usr/local/lib/python3.7/dist-packages (from black==20.8b1->lm-eval==0.0.1) (0.9.0)\n",
            "Requirement already satisfied: mypy-extensions>=0.4.3 in /usr/local/lib/python3.7/dist-packages (from black==20.8b1->lm-eval==0.0.1) (0.4.3)\n",
            "Requirement already satisfied: ujson in /usr/local/lib/python3.7/dist-packages (from lm_dataformat==0.0.19->lm-eval==0.0.1) (4.2.0)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from nagisa==0.2.7->lm-eval==0.0.1) (1.15.0)\n",
            "Requirement already satisfied: DyNet in /usr/local/lib/python3.7/dist-packages (from nagisa==0.2.7->lm-eval==0.0.1) (2.1.2)\n",
            "Requirement already satisfied: requests>=2.20 in /usr/local/lib/python3.7/dist-packages (from openai==0.6.4->lm-eval==0.0.1) (2.23.0)\n",
            "Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from openai==0.6.4->lm-eval==0.0.1) (4.62.3)\n",
            "Requirement already satisfied: pathvalidate<3,>=2.3.0 in /usr/local/lib/python3.7/dist-packages (from pytablewriter==0.58.0->lm-eval==0.0.1) (2.5.0)\n",
            "Requirement already satisfied: msgfy<1,>=0.1.0 in /usr/local/lib/python3.7/dist-packages (from pytablewriter==0.58.0->lm-eval==0.0.1) (0.2.0)\n",
            "Requirement already satisfied: mbstrdecoder<2,>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from pytablewriter==0.58.0->lm-eval==0.0.1) (1.1.0)\n",
            "Requirement already satisfied: setuptools>=38.3.0 in /usr/local/lib/python3.7/dist-packages (from pytablewriter==0.58.0->lm-eval==0.0.1) (57.4.0)\n",
            "Requirement already satisfied: DataProperty<2,>=0.50.0 in /usr/local/lib/python3.7/dist-packages (from pytablewriter==0.58.0->lm-eval==0.0.1) (0.54.2)\n",
            "Requirement already satisfied: tabledata<2,>=1.1.3 in /usr/local/lib/python3.7/dist-packages (from pytablewriter==0.58.0->lm-eval==0.0.1) (1.3.0)\n",
            "Requirement already satisfied: typepy[datetime]<2,>=1.1.1 in /usr/local/lib/python3.7/dist-packages (from pytablewriter==0.58.0->lm-eval==0.0.1) (1.3.0)\n",
            "Requirement already satisfied: tcolorpy<1,>=0.0.5 in /usr/local/lib/python3.7/dist-packages (from pytablewriter==0.58.0->lm-eval==0.0.1) (0.1.1)\n",
            "Requirement already satisfied: iniconfig in /usr/local/lib/python3.7/dist-packages (from pytest==6.2.3->lm-eval==0.0.1) (1.1.1)\n",
            "Requirement already satisfied: importlib-metadata>=0.12 in /usr/local/lib/python3.7/dist-packages (from pytest==6.2.3->lm-eval==0.0.1) (4.8.2)\n",
            "Requirement already satisfied: py>=1.8.2 in /usr/local/lib/python3.7/dist-packages (from pytest==6.2.3->lm-eval==0.0.1) (1.11.0)\n",
            "Requirement already satisfied: packaging in /usr/local/lib/python3.7/dist-packages (from pytest==6.2.3->lm-eval==0.0.1) (21.2)\n",
            "Requirement already satisfied: pluggy<1.0.0a1,>=0.12 in /usr/local/lib/python3.7/dist-packages (from pytest==6.2.3->lm-eval==0.0.1) (0.13.1)\n",
            "Requirement already satisfied: attrs>=19.2.0 in /usr/local/lib/python3.7/dist-packages (from pytest==6.2.3->lm-eval==0.0.1) (21.2.0)\n",
            "Requirement already satisfied: portalocker in /usr/local/lib/python3.7/dist-packages (from sacrebleu==1.5.0->lm-eval==0.0.1) (2.3.2)\n",
            "Requirement already satisfied: mesh-tensorflow[transformer]>=0.1.13 in /usr/local/lib/python3.7/dist-packages (from t5==0.7.1->lm-eval==0.0.1) (0.1.19)\n",
            "Requirement already satisfied: nltk in /usr/local/lib/python3.7/dist-packages (from t5==0.7.1->lm-eval==0.0.1) (3.2.5)\n",
            "Requirement already satisfied: tensorflow-text in /usr/local/lib/python3.7/dist-packages (from t5==0.7.1->lm-eval==0.0.1) (2.6.0)\n",
            "Requirement already satisfied: tfds-nightly in /usr/local/lib/python3.7/dist-packages (from t5==0.7.1->lm-eval==0.0.1) (4.4.0.dev202111110107)\n",
            "Requirement already satisfied: babel in /usr/local/lib/python3.7/dist-packages (from t5==0.7.1->lm-eval==0.0.1) (2.9.1)\n",
            "Requirement already satisfied: rouge-score in /usr/local/lib/python3.7/dist-packages (from t5==0.7.1->lm-eval==0.0.1) (0.0.4)\n",
            "Requirement already satisfied: absl-py in /usr/local/lib/python3.7/dist-packages (from t5==0.7.1->lm-eval==0.0.1) (0.12.0)\n",
            "Requirement already satisfied: gin-config in /usr/local/lib/python3.7/dist-packages (from t5==0.7.1->lm-eval==0.0.1) (0.5.0)\n",
            "Requirement already satisfied: colorama in /usr/local/lib/python3.7/dist-packages (from tqdm-multiprocess==0.0.11->lm-eval==0.0.1) (0.4.4)\n",
            "Requirement already satisfied: rehash in /usr/local/lib/python3.7/dist-packages (from best_download>=0.0.6->lm-eval==0.0.1) (1.0.0)\n",
            "Requirement already satisfied: xxhash in /usr/local/lib/python3.7/dist-packages (from datasets>=1.2.1->lm-eval==0.0.1) (2.0.2)\n",
            "Requirement already satisfied: dill in /usr/local/lib/python3.7/dist-packages (from datasets>=1.2.1->lm-eval==0.0.1) (0.3.4)\n",
            "Requirement already satisfied: pyarrow!=4.0.0,>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from datasets>=1.2.1->lm-eval==0.0.1) (3.0.0)\n",
            "Requirement already satisfied: huggingface-hub<1.0.0,>=0.1.0 in /usr/local/lib/python3.7/dist-packages (from datasets>=1.2.1->lm-eval==0.0.1) (0.1.2)\n",
            "Requirement already satisfied: multiprocess in /usr/local/lib/python3.7/dist-packages (from datasets>=1.2.1->lm-eval==0.0.1) (0.70.12.2)\n",
            "Requirement already satisfied: aiohttp in /usr/local/lib/python3.7/dist-packages (from datasets>=1.2.1->lm-eval==0.0.1) (3.8.0)\n",
            "Requirement already satisfied: fsspec[http]>=2021.05.0 in /usr/local/lib/python3.7/dist-packages (from datasets>=1.2.1->lm-eval==0.0.1) (2021.11.0)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.7/dist-packages (from huggingface-hub<1.0.0,>=0.1.0->datasets>=1.2.1->lm-eval==0.0.1) (3.3.2)\n",
            "Requirement already satisfied: pyyaml in /usr/local/lib/python3.7/dist-packages (from huggingface-hub<1.0.0,>=0.1.0->datasets>=1.2.1->lm-eval==0.0.1) (6.0)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=0.12->pytest==6.2.3->lm-eval==0.0.1) (3.6.0)\n",
            "Requirement already satisfied: chardet<5,>=3.0.4 in /usr/local/lib/python3.7/dist-packages (from mbstrdecoder<2,>=1.0.0->pytablewriter==0.58.0->lm-eval==0.0.1) (3.0.4)\n",
            "Requirement already satisfied: future in /usr/local/lib/python3.7/dist-packages (from mesh-tensorflow[transformer]>=0.1.13->t5==0.7.1->lm-eval==0.0.1) (0.16.0)\n",
            "Requirement already satisfied: tensorflow-datasets in /usr/local/lib/python3.7/dist-packages (from mesh-tensorflow[transformer]>=0.1.13->t5==0.7.1->lm-eval==0.0.1) (4.0.1)\n",
            "Requirement already satisfied: pyparsing<3,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging->pytest==6.2.3->lm-eval==0.0.1) (2.4.7)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests>=2.20->openai==0.6.4->lm-eval==0.0.1) (2.10)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests>=2.20->openai==0.6.4->lm-eval==0.0.1) (2021.10.8)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests>=2.20->openai==0.6.4->lm-eval==0.0.1) (1.24.3)\n",
            "Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=0.24.1->lm-eval==0.0.1) (1.1.0)\n",
            "Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=0.24.1->lm-eval==0.0.1) (3.0.0)\n",
            "Requirement already satisfied: tokenizers<0.11,>=0.10.1 in /usr/local/lib/python3.7/dist-packages (from transformers>=4.1->lm-eval==0.0.1) (0.10.3)\n",
            "Requirement already satisfied: sacremoses in /usr/local/lib/python3.7/dist-packages (from transformers>=4.1->lm-eval==0.0.1) (0.0.46)\n",
            "Requirement already satisfied: pytz>=2018.9 in /usr/local/lib/python3.7/dist-packages (from typepy[datetime]<2,>=1.1.1->pytablewriter==0.58.0->lm-eval==0.0.1) (2018.9)\n",
            "Requirement already satisfied: python-dateutil<3.0.0,>=2.8.0 in /usr/local/lib/python3.7/dist-packages (from typepy[datetime]<2,>=1.1.1->pytablewriter==0.58.0->lm-eval==0.0.1) (2.8.2)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets>=1.2.1->lm-eval==0.0.1) (1.2.0)\n",
            "Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets>=1.2.1->lm-eval==0.0.1) (4.0.1)\n",
            "Requirement already satisfied: charset-normalizer<3.0,>=2.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets>=1.2.1->lm-eval==0.0.1) (2.0.7)\n",
            "Requirement already satisfied: asynctest==0.13.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets>=1.2.1->lm-eval==0.0.1) (0.13.0)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets>=1.2.1->lm-eval==0.0.1) (1.2.0)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets>=1.2.1->lm-eval==0.0.1) (5.2.0)\n",
            "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets>=1.2.1->lm-eval==0.0.1) (1.7.2)\n",
            "Requirement already satisfied: cython in /usr/local/lib/python3.7/dist-packages (from DyNet->nagisa==0.2.7->lm-eval==0.0.1) (0.29.24)\n",
            "Requirement already satisfied: flatbuffers~=1.12.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (1.12)\n",
            "Requirement already satisfied: termcolor~=1.1.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (1.1.0)\n",
            "Requirement already satisfied: clang~=5.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (5.0)\n",
            "Requirement already satisfied: keras~=2.6 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (2.7.0)\n",
            "Requirement already satisfied: grpcio<2.0,>=1.37.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (1.41.1)\n",
            "Requirement already satisfied: opt-einsum~=3.3.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (3.3.0)\n",
            "Requirement already satisfied: google-pasta~=0.2 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (0.2.0)\n",
            "Requirement already satisfied: wheel~=0.35 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (0.37.0)\n",
            "Requirement already satisfied: keras-preprocessing~=1.1.2 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (1.1.2)\n",
            "Requirement already satisfied: wrapt~=1.12.1 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (1.12.1)\n",
            "Requirement already satisfied: h5py~=3.1.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (3.1.0)\n",
            "Requirement already satisfied: protobuf>=3.9.2 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (3.17.3)\n",
            "Requirement already satisfied: tensorboard~=2.6 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (2.7.0)\n",
            "Requirement already satisfied: astunparse~=1.6.3 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (1.6.3)\n",
            "Requirement already satisfied: gast==0.4.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (0.4.0)\n",
            "Requirement already satisfied: cached-property in /usr/local/lib/python3.7/dist-packages (from h5py~=3.1.0->tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (1.5.2)\n",
            "Requirement already satisfied: google-auth<3,>=1.6.3 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (1.35.0)\n",
            "Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (0.4.6)\n",
            "Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (1.0.1)\n",
            "Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (1.8.0)\n",
            "Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (3.3.4)\n",
            "Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (0.6.1)\n",
            "Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tensorboard~=2.6->tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (4.7.2)\n",
            "Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tensorboard~=2.6->tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (0.2.8)\n",
            "Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tensorboard~=2.6->tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (4.2.4)\n",
            "Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.7/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.6->tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (1.3.0)\n",
            "Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.7/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard~=2.6->tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (0.4.8)\n",
            "Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.6->tensorflow->bleurt@ https://github.com/google-research/bleurt/archive/b610120347ef22b494b6d69b4316e303f5932516.zip#egg=bleurt->lm-eval==0.0.1) (3.1.1)\n",
            "Requirement already satisfied: tensorflow-metadata in /usr/local/lib/python3.7/dist-packages (from tensorflow-datasets->mesh-tensorflow[transformer]>=0.1.13->t5==0.7.1->lm-eval==0.0.1) (1.4.0)\n",
            "Requirement already satisfied: importlib-resources in /usr/local/lib/python3.7/dist-packages (from tensorflow-datasets->mesh-tensorflow[transformer]>=0.1.13->t5==0.7.1->lm-eval==0.0.1) (5.4.0)\n",
            "Requirement already satisfied: promise in /usr/local/lib/python3.7/dist-packages (from tensorflow-datasets->mesh-tensorflow[transformer]>=0.1.13->t5==0.7.1->lm-eval==0.0.1) (2.3)\n",
            "Requirement already satisfied: dm-tree in /usr/local/lib/python3.7/dist-packages (from tensorflow-datasets->mesh-tensorflow[transformer]>=0.1.13->t5==0.7.1->lm-eval==0.0.1) (0.1.6)\n",
            "Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow-metadata->tensorflow-datasets->mesh-tensorflow[transformer]>=0.1.13->t5==0.7.1->lm-eval==0.0.1) (1.53.0)\n",
            "Requirement already satisfied: tensorflow-hub>=0.8.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow-text->t5==0.7.1->lm-eval==0.0.1) (0.12.0)\n",
            "Installing collected packages: tensorflow-estimator\n",
            "  Attempting uninstall: tensorflow-estimator\n",
            "    Found existing installation: tensorflow-estimator 2.7.0\n",
            "    Uninstalling tensorflow-estimator-2.7.0:\n",
            "      Successfully uninstalled tensorflow-estimator-2.7.0\n",
            "Successfully installed tensorflow-estimator-2.6.0\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aO1UXepF-0Uq"
      },
      "source": [
        "## Setup Model\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ex0qJgaueZtJ"
      },
      "source": [
        "import os\n",
        "import requests \n",
        "from jax.config import config\n",
        "\n",
        "colab_tpu_addr = os.environ['COLAB_TPU_ADDR'].split(':')[0]\n",
        "url = f'http://{colab_tpu_addr}:8475/requestversion/tpu_driver0.1_dev20210607'\n",
        "requests.post(url)\n",
        "\n",
        "# The following is required to use TPU Driver as JAX's backend.\n",
        "config.FLAGS.jax_xla_backend = \"tpu_driver\"\n",
        "config.FLAGS.jax_backend_target = \"grpc://\" + os.environ['COLAB_TPU_ADDR']"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NIgUVdFLe4A8"
      },
      "source": [
        "Sometimes the next step errors for some reason, just run it again ¯\\\\\\_(ツ)\\_/¯"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-A5IGYSaeze3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 394
        },
        "outputId": "0d73007b-4ee8-49a9-acdd-6bdf0e213853"
      },
      "source": [
        "import time\n",
        "\n",
        "import jax\n",
        "from jax.experimental import maps\n",
        "import numpy as np\n",
        "import optax\n",
        "import transformers\n",
        "\n",
        "from mesh_transformer.checkpoint import read_ckpt_lowmem\n",
        "from mesh_transformer.sampling import nucleaus_sample\n",
        "from mesh_transformer.transformer_shard import CausalTransformer"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "error",
          "ename": "TypeError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-3-a22d9a83aa66>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      7\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mtransformers\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      8\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 9\u001b[0;31m \u001b[0;32mfrom\u001b[0m \u001b[0mmesh_transformer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcheckpoint\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mread_ckpt_lowmem\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     10\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mmesh_transformer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msampling\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mnucleaus_sample\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     11\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mmesh_transformer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtransformer_shard\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mCausalTransformer\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/mesh_transformer/checkpoint.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     12\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0msmart_open\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mopen\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     13\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 14\u001b[0;31m \u001b[0;32mfrom\u001b[0m \u001b[0mmesh_transformer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mutil\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mhead_print\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     15\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     16\u001b[0m \u001b[0mpieces\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m16\u001b[0m  \u001b[0;31m# how many files to split each shard across\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/mesh_transformer/util.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     34\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     35\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 36\u001b[0;31m \u001b[0;32mclass\u001b[0m \u001b[0mClipByGlobalNormState\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mOptState\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     37\u001b[0m     \u001b[0;34m\"\"\"The `clip_by_global_norm` transformation is stateless.\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     38\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/lib/python3.7/typing.py\u001b[0m in \u001b[0;36m__new__\u001b[0;34m(cls, *args, **kwds)\u001b[0m\n\u001b[1;32m    308\u001b[0m                 isinstance(args[1], tuple)):\n\u001b[1;32m    309\u001b[0m             \u001b[0;31m# Close enough.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 310\u001b[0;31m             \u001b[0;32mraise\u001b[0m \u001b[0mTypeError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34mf\"Cannot subclass {cls!r}\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    311\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0msuper\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__new__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcls\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    312\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mTypeError\u001b[0m: Cannot subclass <class 'typing._SpecialForm'>"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QAgKq-X2kmba"
      },
      "source": [
        "params = {\n",
        "  \"layers\": 28,\n",
        "  \"d_model\": 4096,\n",
        "  \"n_heads\": 16,\n",
        "  \"n_vocab\": 50400,\n",
        "  \"norm\": \"layernorm\",\n",
        "  \"pe\": \"rotary\",\n",
        "  \"pe_rotary_dims\": 64,\n",
        "\n",
        "  \"seq\": 2048,\n",
        "  \"cores_per_replica\": 8,\n",
        "  \"per_replica_batch\": 1,\n",
        "}\n",
        "\n",
        "per_replica_batch = params[\"per_replica_batch\"]\n",
        "cores_per_replica = params[\"cores_per_replica\"]\n",
        "seq = params[\"seq\"]\n",
        "\n",
        "\n",
        "params[\"sampler\"] = nucleaus_sample\n",
        "\n",
        "# here we \"remove\" the optimizer parameters from the model (as we don't need them for inference)\n",
        "params[\"optimizer\"] = optax.scale(0)\n",
        "\n",
        "mesh_shape = (jax.device_count() // cores_per_replica, cores_per_replica)\n",
        "devices = np.array(jax.devices()).reshape(mesh_shape)\n",
        "\n",
        "maps.thread_resources.env = maps.ResourceEnv(maps.Mesh(devices, ('dp', 'mp')))\n",
        "\n",
        "tokenizer = transformers.GPT2TokenizerFast.from_pretrained('gpt2')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yFgRkUgfiNdA"
      },
      "source": [
        "Here we create the network and load the parameters from the downloaded files. Expect this to take around 5 minutes."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lwNETD2Uk8nu"
      },
      "source": [
        "total_batch = per_replica_batch * jax.device_count() // cores_per_replica\n",
        "\n",
        "network = CausalTransformer(params)\n",
        "\n",
        "network.state = read_ckpt_lowmem(network.state, \"step_383500/\", devices.shape[1])\n",
        "\n",
        "network.state = network.move_xmap(network.state, np.zeros(cores_per_replica))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "A-eT7Sw6if4J"
      },
      "source": [
        "## Run Model\n",
        "\n",
        "Finally, we are ready to infer with the model! The first sample takes around a minute due to compilation, but after that it should only take about 10 seconds per sample.\n",
        "\n",
        "Feel free to mess with the different sampling parameters (top_p and temp), as well as the length of the generations (gen_len, causes a recompile when changed).\n",
        "\n",
        "You can also change other things like per_replica_batch in the previous cells to change how many generations are done in parallel. A larger batch has higher latency but higher throughput when measured in tokens generated/s. This is useful for doing things like best-of-n cherry picking.\n",
        "\n",
        "*Tip for best results: Make sure your prompt does not have any trailing spaces, which tend to confuse the model due to the BPE tokenization used during training.*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "t7i5Xf10nSVJ"
      },
      "source": [
        "# allow text wrapping in generated output: https://stackoverflow.com/a/61401455\n",
        "from IPython.display import HTML, display\n",
        "\n",
        "def set_css():\n",
        "  display(HTML('''\n",
        "  <style>\n",
        "    pre {\n",
        "        white-space: pre-wrap;\n",
        "    }\n",
        "  </style>\n",
        "  '''))\n",
        "get_ipython().events.register('pre_run_cell', set_css)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 989
        },
        "id": "ZVzs2TYlvYeX",
        "outputId": "10955daf-c0af-41fd-8432-654c18c778d8"
      },
      "source": [
        "def infer(context, top_p=0.9, temp=1.0, gen_len=512):\n",
        "    tokens = tokenizer.encode(context)\n",
        "\n",
        "    provided_ctx = len(tokens)\n",
        "    pad_amount = seq - provided_ctx\n",
        "\n",
        "    padded_tokens = np.pad(tokens, ((pad_amount, 0),)).astype(np.uint32)\n",
        "    batched_tokens = np.array([padded_tokens] * total_batch)\n",
        "    length = np.ones(total_batch, dtype=np.uint32) * len(tokens)\n",
        "\n",
        "    start = time.time()\n",
        "    output = network.generate(batched_tokens, length, gen_len, {\"top_p\": np.ones(total_batch) * top_p, \"temp\": np.ones(total_batch) * temp})\n",
        "\n",
        "    samples = []\n",
        "    decoded_tokens = output[1][0]\n",
        "\n",
        "    for o in decoded_tokens[:, :, 0]:\n",
        "      samples.append(f\"\\033[1m{context}\\033[0m{tokenizer.decode(o)}\")\n",
        "\n",
        "    print(f\"completion done in {time.time() - start:06}s\")\n",
        "    return samples\n",
        "\n",
        "print(infer(\"EleutherAI is\")[0])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "completion done in 52.59220790863037s\n",
            "\u001b[1mEleutherAI is\u001b[0m a creative company and engine integrator\n",
            "\n",
            "If you are looking for a highly skilled audio visual production company with award winning creative team, looking for a reliable audio visual company that has over 12 years of experience and has built a reputation for creating great creative work with the following skills:\n",
            "\n",
            "1. Videographer\n",
            "\n",
            "2. Creative Services\n",
            "\n",
            "3. Graphics/motion graphics\n",
            "\n",
            "4. Audiovisual/Cinematography\n",
            "\n",
            "5. Photographer\n",
            "\n",
            "6. Title Design\n",
            "\n",
            "7. Animation and Technical Illustration\n",
            "\n",
            "We at EuKai believe you need not just a partner but a creative company that will provide all your company's video needs that needs to be done, such as:\n",
            "\n",
            "Video Proposal, Storyboards, Pre-Production, Production, Post-Production, VR and 360° Videos, Broadcast, Web, Corporate Videos, Packaging Videos, Documentaries, Television, Online Ads, Promotional Videos, Broadcast News, International Channel, Multimedia, Music Videos, Musicals, Music Production, Corporate Events, Banners, Commercials, Creative Film, Advertising Films, Events, Social Media Videos, Animations, Video Editing, Presentations, Short films, Documentaries, Motion Graphic Designs, Light Animation and much more.\n",
            "\n",
            "Our well-designed Videos\n",
            "\n",
            "No matter you need TV commercials, corporate videos or motion graphic designs, we always make sure you get the perfect creative solutions to make your video become more engaging and appealing.\n",
            "\n",
            "We have been helping businesses to create better video content for their purpose. Our clients are able to meet their objectives and maximize their visual communication. It is critical to your success to have a well-thought out plan for your video. We are dedicated to helping you achieve your goal. If you have any questions, please don't hesitate to contact us!\n",
            "\n",
            "Working with EuKai\n",
            "\n",
            "We know your business, and have experience in being your partner, client, and friend. Your success is our success.\n",
            "\n",
            "Communication\n",
            "\n",
            "Great communication is the key to great business and relationships. We believe that feedback is crucial to us and each other. Our communication is efficient and effective. We do not judge or sugarcoat anything. We embrace everyone and everything!\n",
            "\n",
            "Focus on the quality\n",
            "\n",
            "We have a high quality standard and a realistic approach in every aspect of our business. Our time is valuable and we put our clients first. We put in the hours to get the job done right.\n",
            "\n",
            "Creativity\n",
            "\n",
            "Creat\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ux5aWaT432QM"
      },
      "source": [
        "`temperature` - What sampling temperature to use. Higher values means the model will take more risks. Try 0.9 for more creative applications, and 0 (argmax sampling) for ones with a well-defined answer. We generally recommend altering this or top_p but not both.\n",
        "  \n",
        "`top_p` - An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered.\n",
        "\n",
        "We generally recommend altering this or temperature but not both.\n",
        "\n",
        "(Definitions taken from https://docs.zeroqode.com/plugins/artifical-intelligence-gpt-3-by-openai)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 575
        },
        "id": "nvlAK6RbCJYg",
        "outputId": "c593cf19-753d-45c5-8fe9-9234ad824436"
      },
      "source": [
        "#@title  { form-width: \"300px\" }\n",
        "top_p = 0.9 #@param {type:\"slider\", min:0, max:1, step:0.1}\n",
        "temp = 1 #@param {type:\"slider\", min:0, max:1, step:0.1}\n",
        "\n",
        "context = \"\"\"Do I contradict myself?\n",
        "Very well then I contradict myself,\n",
        "(I am large, I contain multitudes.)\"\"\"\n",
        "\n",
        "print(infer(top_p=top_p, temp=temp, gen_len=512, context=context)[0])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "completion done in 13.52977204322815s\n",
            "\u001b[1mDo I contradict myself?\n",
            "Very well then I contradict myself,\n",
            "(I am large, I contain multitudes.)\u001b[0m\n",
            "\n",
            "St. Paul\n",
            "\n",
            "28.8.08\n",
            "\n",
            "I'm reading Barbara Ehrenreich's Nickle and Dimed. The most striking story she tells (and it's the lead off story) is about making her house a homestay and getting a teaching job as a way to pay for some of the bills. Here is the last line of her book.\n",
            "\n",
            "It's like waking up in a strange house that's starting to fall apart, to have to figure out how to fix everything at once. And then when you do, you realize that there's more out there than you can imagine. In some ways it feels like winning the lottery. You're suddenly rich, and everything's fresh. Your laundry might even smell good for the first time in years.\n",
            "\n",
            "24.7.08\n",
            "\n",
            "I got back to LA on Saturday and as soon as I got to the house, I called work. I spoke to one of my co-workers and she said that our new chief, Sandeep, would call me this morning. He had a big lunch so I don't know what was going on. Well, in the meantime, the other co-worker (whom I really like a lot) was wondering if I would stay at the house, and also if she could borrow my vibrator. I told her no, that it belonged to my girlfriend and that I couldn't lend it to her.\n",
            "\n",
            "Anyways, after calling work, I went out for lunch with Frank. On the way there, we took a side road to get off the freeway and I saw another freight train, one of the L.A. Metrolink trains, on the right. We were going at about 50 miles an hour, so it was kind of scary. If I wanted to, I could make out the letters, \"L.A.\" on the side of the train.\n",
            "\n",
            "It was really pretty cool and it was kinda bugging me that the train was headed towards us. There was a woman on the side of the train waving to us and asking where we were going. Frank just waved back and kept on driving. Eventually, we saw the signs for Echo Park, but before we got there, we turned off the road and made our way to the train tracks. There were so many things to look at and so much to take in. A couple of ways over from the tracks, there is a huge red-brick house with some trees. There were leaves on the\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "L9REQIXax1PT"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}