SMM4H_2020_ADR_classification.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "SMM4H_2020_ADR_classification.ipynb",
      "provenance": [],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "C862xYe9FGqz"
      },
      "source": [
        "## Read me\n",
        "\n",
        "This notebook is an example of a binary ADR sentence classification for the SMM4H 2020 Shared task.\n",
        "\n",
        "We use the EnRuDR-BERT model from:\n",
        "\n",
        "https://github.com/cimm-kzn/RuDReC\n",
        "\n",
        "In this notebook, we use the Google's Tensorflow implementation of BERT.\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zKN6hs1vdhRR",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "963bafd9-9250-421c-9ae8-f0bb5740b22d"
      },
      "source": [
        "%tensorflow_version 1.x"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "TensorFlow 1.x selected.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kh4lTHpvihZQ",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a0745efd-859e-4b1e-dfa6-dbf725488be6"
      },
      "source": [
        "!nvidia-smi"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Thu Dec  3 15:41:59 2020       \n",
            "+-----------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 455.38       Driver Version: 418.67       CUDA Version: 10.1     |\n",
            "|-------------------------------+----------------------+----------------------+\n",
            "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                               |                      |               MIG M. |\n",
            "|===============================+======================+======================|\n",
            "|   0  Tesla T4            Off  | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   48C    P8    10W /  70W |      0MiB / 15079MiB |      0%      Default |\n",
            "|                               |                      |                 ERR! |\n",
            "+-------------------------------+----------------------+----------------------+\n",
            "                                                                               \n",
            "+-----------------------------------------------------------------------------+\n",
            "| Processes:                                                                  |\n",
            "|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |\n",
            "|        ID   ID                                                   Usage      |\n",
            "|=============================================================================|\n",
            "|  No running processes found                                                 |\n",
            "+-----------------------------------------------------------------------------+\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7-bGoNSY06SM"
      },
      "source": [
        "For a batch size of 32 you need a GPU with 15+ Gb memory. If you get less memory, you can reset runtime by clicking \"Factory reset runtime\" under \"Runtime\" section of Colab menu."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "90jvcmcfGQX4"
      },
      "source": [
        "### Pulling necessary code"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7VHKWxYmdzdh",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "30fff698-31cf-4ca0-a2e0-2dce6628ea41"
      },
      "source": [
        "!git init\n",
        "!git clone https://github.com/google-research/bert.git\n",
        "!git clone https://github.com/Andoree/med_project.git\n",
        "!git clone https://github.com/Andoree/smm4h_classification.git\n",
        "!cp med_project/multilabel_code/bert_preprocessing.py ./bert\n",
        "!cp med_project/multilabel_code/multilabel_bert.py ./bert\n",
        "!cp /content/smm4h_classification/training/run_binary_classification.py ./bert\n",
        "!cp /content/smm4h_classification/training/multilabel_pretraining.py ./bert"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Initialized empty Git repository in /content/.git/\n",
            "Cloning into 'bert'...\n",
            "remote: Enumerating objects: 340, done.\u001b[K\n",
            "remote: Total 340 (delta 0), reused 0 (delta 0), pack-reused 340\u001b[K\n",
            "Receiving objects: 100% (340/340), 317.85 KiB | 17.66 MiB/s, done.\n",
            "Resolving deltas: 100% (185/185), done.\n",
            "Cloning into 'med_project'...\n",
            "remote: Enumerating objects: 395, done.\u001b[K\n",
            "remote: Counting objects: 100% (395/395), done.\u001b[K\n",
            "remote: Compressing objects: 100% (253/253), done.\u001b[K\n",
            "remote: Total 2488 (delta 251), reused 267 (delta 124), pack-reused 2093\u001b[K\n",
            "Receiving objects: 100% (2488/2488), 21.22 MiB | 20.89 MiB/s, done.\n",
            "Resolving deltas: 100% (961/961), done.\n",
            "Cloning into 'smm4h_classification'...\n",
            "remote: Enumerating objects: 118, done.\u001b[K\n",
            "remote: Counting objects: 100% (118/118), done.\u001b[K\n",
            "remote: Compressing objects: 100% (87/87), done.\u001b[K\n",
            "remote: Total 118 (delta 55), reused 78 (delta 26), pack-reused 0\u001b[K\n",
            "Receiving objects: 100% (118/118), 2.80 MiB | 22.61 MiB/s, done.\n",
            "Resolving deltas: 100% (55/55), done.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DX4rRkeo1STJ"
      },
      "source": [
        "#### Downloading EnRuDR-BERT model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4sD9FPSZViXx",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "2f9d1f56-6a2a-45d1-8544-e99940636b1e"
      },
      "source": [
        "!mkdir bert_models/\n",
        "!wget --load-cookies /tmp/cookies.txt \"https://docs.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate 'https://docs.google.com/uc?export=download&id=1jCCcafZO6RMl324W8WinXTHBCX86lXnS' -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\\1\\n/p')&id=1jCCcafZO6RMl324W8WinXTHBCX86lXnS\" -O bert_models/EnRuDR_BERT.tar.gz && rm -rf /tmp/cookies.txt \n",
        "!tar -xvf bert_models/EnRuDR_BERT.tar.gz -C bert_models\n",
        "!ls bert_models/multilingual_eng_rus_reviews_finetuned"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "--2020-12-03 15:42:05--  https://docs.google.com/uc?export=download&confirm=isLe&id=1jCCcafZO6RMl324W8WinXTHBCX86lXnS\n",
            "Resolving docs.google.com (docs.google.com)... 172.217.12.238, 2607:f8b0:4004:809::200e\n",
            "Connecting to docs.google.com (docs.google.com)|172.217.12.238|:443... connected.\n",
            "HTTP request sent, awaiting response... 302 Moved Temporarily\n",
            "Location: https://doc-0g-2k-docs.googleusercontent.com/docs/securesc/6j9t4lec85dbv73nrnltij8454k62ajr/7u9krb1qn5bmorc1ia24gnrivar0mstn/1607010075000/06930042168325031160/14649094590169478743Z/1jCCcafZO6RMl324W8WinXTHBCX86lXnS?e=download [following]\n",
            "--2020-12-03 15:42:05--  https://doc-0g-2k-docs.googleusercontent.com/docs/securesc/6j9t4lec85dbv73nrnltij8454k62ajr/7u9krb1qn5bmorc1ia24gnrivar0mstn/1607010075000/06930042168325031160/14649094590169478743Z/1jCCcafZO6RMl324W8WinXTHBCX86lXnS?e=download\n",
            "Resolving doc-0g-2k-docs.googleusercontent.com (doc-0g-2k-docs.googleusercontent.com)... 172.217.15.97, 2607:f8b0:4004:811::2001\n",
            "Connecting to doc-0g-2k-docs.googleusercontent.com (doc-0g-2k-docs.googleusercontent.com)|172.217.15.97|:443... connected.\n",
            "HTTP request sent, awaiting response... 302 Found\n",
            "Location: https://docs.google.com/nonceSigner?nonce=csq9tl9916nae&continue=https://doc-0g-2k-docs.googleusercontent.com/docs/securesc/6j9t4lec85dbv73nrnltij8454k62ajr/7u9krb1qn5bmorc1ia24gnrivar0mstn/1607010075000/06930042168325031160/14649094590169478743Z/1jCCcafZO6RMl324W8WinXTHBCX86lXnS?e%3Ddownload&hash=5hr3aenqqj7rji40n8eo78vkrfk0gbpa [following]\n",
            "--2020-12-03 15:42:05--  https://docs.google.com/nonceSigner?nonce=csq9tl9916nae&continue=https://doc-0g-2k-docs.googleusercontent.com/docs/securesc/6j9t4lec85dbv73nrnltij8454k62ajr/7u9krb1qn5bmorc1ia24gnrivar0mstn/1607010075000/06930042168325031160/14649094590169478743Z/1jCCcafZO6RMl324W8WinXTHBCX86lXnS?e%3Ddownload&hash=5hr3aenqqj7rji40n8eo78vkrfk0gbpa\n",
            "Connecting to docs.google.com (docs.google.com)|172.217.12.238|:443... connected.\n",
            "HTTP request sent, awaiting response... 302 Found\n",
            "Location: https://doc-0g-2k-docs.googleusercontent.com/docs/securesc/6j9t4lec85dbv73nrnltij8454k62ajr/7u9krb1qn5bmorc1ia24gnrivar0mstn/1607010075000/06930042168325031160/14649094590169478743Z/1jCCcafZO6RMl324W8WinXTHBCX86lXnS?e=download&nonce=csq9tl9916nae&user=14649094590169478743Z&hash=vjt3ddel5eilma0o842gfdicfbgk9b5k [following]\n",
            "--2020-12-03 15:42:05--  https://doc-0g-2k-docs.googleusercontent.com/docs/securesc/6j9t4lec85dbv73nrnltij8454k62ajr/7u9krb1qn5bmorc1ia24gnrivar0mstn/1607010075000/06930042168325031160/14649094590169478743Z/1jCCcafZO6RMl324W8WinXTHBCX86lXnS?e=download&nonce=csq9tl9916nae&user=14649094590169478743Z&hash=vjt3ddel5eilma0o842gfdicfbgk9b5k\n",
            "Connecting to doc-0g-2k-docs.googleusercontent.com (doc-0g-2k-docs.googleusercontent.com)|172.217.15.97|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: unspecified [application/gzip]\n",
            "Saving to: ‘bert_models/EnRuDR_BERT.tar.gz’\n",
            "\n",
            "bert_models/EnRuDR_     [   <=>              ]   1.84G  43.1MB/s    in 17s     \n",
            "\n",
            "2020-12-03 15:42:22 (112 MB/s) - ‘bert_models/EnRuDR_BERT.tar.gz’ saved [1981774991]\n",
            "\n",
            "multilingual_eng_rus_reviews_finetuned/\n",
            "multilingual_eng_rus_reviews_finetuned/bert_model.ckpt.data-00000-of-00001\n",
            "multilingual_eng_rus_reviews_finetuned/vocab.txt\n",
            "multilingual_eng_rus_reviews_finetuned/bert_model.ckpt.index\n",
            "multilingual_eng_rus_reviews_finetuned/bert_model.ckpt.meta\n",
            "multilingual_eng_rus_reviews_finetuned/bert_config.json\n",
            "bert_config.json\t\t     bert_model.ckpt.index  vocab.txt\n",
            "bert_model.ckpt.data-00000-of-00001  bert_model.ckpt.meta\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kqtz3KoW1Xsw"
      },
      "source": [
        "#### Downloading the annotated part of the RuDReC corpus and splitting it into senteces"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-keuQK3ooJsi",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a2bffca0-bc1c-408c-8ae6-b4c388c516fa"
      },
      "source": [
        "!mkdir -p data/rudrec_annotated\n",
        "!wget --load-cookies /tmp/cookies.txt \"https://docs.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate 'https://docs.google.com/uc?export=download&id=1plApL6qmdHtNNP3OXgJQEmo7Lfp6MVeO' -O- > /dev/null | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\\1\\n/p')&id=1plApL6qmdHtNNP3OXgJQEmo7Lfp6MVeO\" -O data/rudrec_annotated/rudrec.zip && rm -rf /tmp/cookies.txt"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "--2020-12-03 15:43:40--  https://docs.google.com/uc?export=download&confirm=&id=1plApL6qmdHtNNP3OXgJQEmo7Lfp6MVeO\n",
            "Resolving docs.google.com (docs.google.com)... 172.217.12.238, 2607:f8b0:4004:809::200e\n",
            "Connecting to docs.google.com (docs.google.com)|172.217.12.238|:443... connected.\n",
            "HTTP request sent, awaiting response... 302 Moved Temporarily\n",
            "Location: https://doc-00-7o-docs.googleusercontent.com/docs/securesc/fi17g1ceqnjp8e2ecekcq9a0b9eeakj8/ulblg4pthv8lfuo63o6b66b893r3cfue/1607010225000/06930042168325031160/06409174565933036453Z/1plApL6qmdHtNNP3OXgJQEmo7Lfp6MVeO?e=download [following]\n",
            "--2020-12-03 15:43:57--  https://doc-00-7o-docs.googleusercontent.com/docs/securesc/fi17g1ceqnjp8e2ecekcq9a0b9eeakj8/ulblg4pthv8lfuo63o6b66b893r3cfue/1607010225000/06930042168325031160/06409174565933036453Z/1plApL6qmdHtNNP3OXgJQEmo7Lfp6MVeO?e=download\n",
            "Resolving doc-00-7o-docs.googleusercontent.com (doc-00-7o-docs.googleusercontent.com)... 172.217.15.97, 2607:f8b0:4004:811::2001\n",
            "Connecting to doc-00-7o-docs.googleusercontent.com (doc-00-7o-docs.googleusercontent.com)|172.217.15.97|:443... connected.\n",
            "HTTP request sent, awaiting response... 302 Found\n",
            "Location: https://docs.google.com/nonceSigner?nonce=1imneb4h9dom8&continue=https://doc-00-7o-docs.googleusercontent.com/docs/securesc/fi17g1ceqnjp8e2ecekcq9a0b9eeakj8/ulblg4pthv8lfuo63o6b66b893r3cfue/1607010225000/06930042168325031160/06409174565933036453Z/1plApL6qmdHtNNP3OXgJQEmo7Lfp6MVeO?e%3Ddownload&hash=ep6a67jvqcc64ujptqripjt3g0dpnl1s [following]\n",
            "--2020-12-03 15:43:57--  https://docs.google.com/nonceSigner?nonce=1imneb4h9dom8&continue=https://doc-00-7o-docs.googleusercontent.com/docs/securesc/fi17g1ceqnjp8e2ecekcq9a0b9eeakj8/ulblg4pthv8lfuo63o6b66b893r3cfue/1607010225000/06930042168325031160/06409174565933036453Z/1plApL6qmdHtNNP3OXgJQEmo7Lfp6MVeO?e%3Ddownload&hash=ep6a67jvqcc64ujptqripjt3g0dpnl1s\n",
            "Connecting to docs.google.com (docs.google.com)|172.217.12.238|:443... connected.\n",
            "HTTP request sent, awaiting response... 302 Found\n",
            "Location: https://doc-00-7o-docs.googleusercontent.com/docs/securesc/fi17g1ceqnjp8e2ecekcq9a0b9eeakj8/ulblg4pthv8lfuo63o6b66b893r3cfue/1607010225000/06930042168325031160/06409174565933036453Z/1plApL6qmdHtNNP3OXgJQEmo7Lfp6MVeO?e=download&nonce=1imneb4h9dom8&user=06409174565933036453Z&hash=d6smg9i5ecmadfe8n226mn02tdn3cvqt [following]\n",
            "--2020-12-03 15:43:57--  https://doc-00-7o-docs.googleusercontent.com/docs/securesc/fi17g1ceqnjp8e2ecekcq9a0b9eeakj8/ulblg4pthv8lfuo63o6b66b893r3cfue/1607010225000/06930042168325031160/06409174565933036453Z/1plApL6qmdHtNNP3OXgJQEmo7Lfp6MVeO?e=download&nonce=1imneb4h9dom8&user=06409174565933036453Z&hash=d6smg9i5ecmadfe8n226mn02tdn3cvqt\n",
            "Connecting to doc-00-7o-docs.googleusercontent.com (doc-00-7o-docs.googleusercontent.com)|172.217.15.97|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: unspecified [application/zip]\n",
            "Saving to: ‘data/rudrec_annotated/rudrec.zip’\n",
            "\n",
            "data/rudrec_annotat     [ <=>                ]  14.03M  --.-KB/s    in 0.05s   \n",
            "\n",
            "2020-12-03 15:43:57 (291 MB/s) - ‘data/rudrec_annotated/rudrec.zip’ saved [14712824]\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bbzru4qOolG6"
      },
      "source": [
        "!unzip -q data/rudrec_annotated/rudrec.zip -d data/rudrec_annotated/\n",
        "!rm data/rudrec_annotated/rudrec.zip"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YajRQl2q1qPw"
      },
      "source": [
        "### Downloading PsyTAR sentences"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Qc7Waeid8ucb",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "88b7f3c0-f610-4832-f197-e6451fa5c53a"
      },
      "source": [
        "!wget --load-cookies /tmp/cookies.txt \"https://docs.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate 'https://docs.google.com/uc?export=download&id=1zBFgdsiWlvb3LsNp0s65MpDYyBGC-jih' -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\\1\\n/p')&id=1zBFgdsiWlvb3LsNp0s65MpDYyBGC-jih\" -O data/psytar_csvs.zip && rm -rf /tmp/cookies.txt"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "--2020-12-03 15:44:00--  https://docs.google.com/uc?export=download&confirm=&id=1zBFgdsiWlvb3LsNp0s65MpDYyBGC-jih\n",
            "Resolving docs.google.com (docs.google.com)... 172.217.12.238, 2607:f8b0:4004:807::200e\n",
            "Connecting to docs.google.com (docs.google.com)|172.217.12.238|:443... connected.\n",
            "HTTP request sent, awaiting response... 302 Moved Temporarily\n",
            "Location: https://doc-00-bo-docs.googleusercontent.com/docs/securesc/79ninmtchkemtrvudqdfal1nepfgiij2/7dvjhk744osokg22upaepeo9uetpq70k/1607010225000/03082508243073582503/17384356599220833547Z/1zBFgdsiWlvb3LsNp0s65MpDYyBGC-jih?e=download [following]\n",
            "--2020-12-03 15:44:00--  https://doc-00-bo-docs.googleusercontent.com/docs/securesc/79ninmtchkemtrvudqdfal1nepfgiij2/7dvjhk744osokg22upaepeo9uetpq70k/1607010225000/03082508243073582503/17384356599220833547Z/1zBFgdsiWlvb3LsNp0s65MpDYyBGC-jih?e=download\n",
            "Resolving doc-00-bo-docs.googleusercontent.com (doc-00-bo-docs.googleusercontent.com)... 172.217.15.97, 2607:f8b0:4004:811::2001\n",
            "Connecting to doc-00-bo-docs.googleusercontent.com (doc-00-bo-docs.googleusercontent.com)|172.217.15.97|:443... connected.\n",
            "HTTP request sent, awaiting response... 302 Found\n",
            "Location: https://docs.google.com/nonceSigner?nonce=leg8knde4ed4g&continue=https://doc-00-bo-docs.googleusercontent.com/docs/securesc/79ninmtchkemtrvudqdfal1nepfgiij2/7dvjhk744osokg22upaepeo9uetpq70k/1607010225000/03082508243073582503/17384356599220833547Z/1zBFgdsiWlvb3LsNp0s65MpDYyBGC-jih?e%3Ddownload&hash=ijdke0grp8h987lb0b91ns5h0h48eno8 [following]\n",
            "--2020-12-03 15:44:01--  https://docs.google.com/nonceSigner?nonce=leg8knde4ed4g&continue=https://doc-00-bo-docs.googleusercontent.com/docs/securesc/79ninmtchkemtrvudqdfal1nepfgiij2/7dvjhk744osokg22upaepeo9uetpq70k/1607010225000/03082508243073582503/17384356599220833547Z/1zBFgdsiWlvb3LsNp0s65MpDYyBGC-jih?e%3Ddownload&hash=ijdke0grp8h987lb0b91ns5h0h48eno8\n",
            "Connecting to docs.google.com (docs.google.com)|172.217.12.238|:443... connected.\n",
            "HTTP request sent, awaiting response... 302 Found\n",
            "Location: https://doc-00-bo-docs.googleusercontent.com/docs/securesc/79ninmtchkemtrvudqdfal1nepfgiij2/7dvjhk744osokg22upaepeo9uetpq70k/1607010225000/03082508243073582503/17384356599220833547Z/1zBFgdsiWlvb3LsNp0s65MpDYyBGC-jih?e=download&nonce=leg8knde4ed4g&user=17384356599220833547Z&hash=rep5c3dfib4t58fp8n2s8l46rped8uu5 [following]\n",
            "--2020-12-03 15:44:01--  https://doc-00-bo-docs.googleusercontent.com/docs/securesc/79ninmtchkemtrvudqdfal1nepfgiij2/7dvjhk744osokg22upaepeo9uetpq70k/1607010225000/03082508243073582503/17384356599220833547Z/1zBFgdsiWlvb3LsNp0s65MpDYyBGC-jih?e=download&nonce=leg8knde4ed4g&user=17384356599220833547Z&hash=rep5c3dfib4t58fp8n2s8l46rped8uu5\n",
            "Connecting to doc-00-bo-docs.googleusercontent.com (doc-00-bo-docs.googleusercontent.com)|172.217.15.97|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 235822 (230K) [application/zip]\n",
            "Saving to: ‘data/psytar_csvs.zip’\n",
            "\n",
            "data/psytar_csvs.zi 100%[===================>] 230.29K  --.-KB/s    in 0.002s  \n",
            "\n",
            "2020-12-03 15:44:01 (125 MB/s) - ‘data/psytar_csvs.zip’ saved [235822/235822]\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fvByAcLh8zxa"
      },
      "source": [
        "!unzip -q data/psytar_csvs.zip -d data/\n",
        "!rm data/psytar_csvs.zip"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9IRA7xUyGZN8"
      },
      "source": [
        "\"otzovik_reviews_formatting.py\" script tokenizes reviews into sentences and splits the data into train, test, and dev sets.\n",
        "\n",
        "**n_splits=1** corresponds to a simple splitting of data into training, validation, and test sets. **n_splits > 1** is cross-validation."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JQJLYKJGp6sH",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "2c106a36-aec6-4195-e273-a10efa275614"
      },
      "source": [
        "%cd /content/med_project/bert_multilabel/formatting/\n",
        "!python otzovik_reviews_formatting.py \\\n",
        "--reviews_dir=/content/data/rudrec_annotated/annotation \\\n",
        "--output_dir=/content/data/rudrec_annotated/sentences \\\n",
        "--n_splits=1\n",
        "!ls /content/data/rudrec_annotated/sentences"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content/med_project/bert_multilabel/formatting\n",
            "dev.csv  test.csv  train.csv\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "na8yebQq-11y"
      },
      "source": [
        "!mv /content/data/rudrec_annotated/sentences /content/data/rudrec"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RDC2vi6cmiob",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "361f84cb-af64-4f41-93c2-c827a825231c"
      },
      "source": [
        "!pip install emoji"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Collecting emoji\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/ff/1c/1f1457fe52d0b30cbeebfd578483cedb3e3619108d2d5a21380dfecf8ffd/emoji-0.6.0.tar.gz (51kB)\n",
            "\r\u001b[K     |██████▍                         | 10kB 21.6MB/s eta 0:00:01\r\u001b[K     |████████████▉                   | 20kB 15.5MB/s eta 0:00:01\r\u001b[K     |███████████████████▎            | 30kB 14.0MB/s eta 0:00:01\r\u001b[K     |█████████████████████████▊      | 40kB 14.0MB/s eta 0:00:01\r\u001b[K     |████████████████████████████████| 51kB 5.6MB/s \n",
            "\u001b[?25hBuilding wheels for collected packages: emoji\n",
            "  Building wheel for emoji (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for emoji: filename=emoji-0.6.0-cp36-none-any.whl size=49716 sha256=d8da26a2c8fcacf35dade71d39f5d94d1a3f085a4ee0c509fe29fac7c9ab1221\n",
            "  Stored in directory: /root/.cache/pip/wheels/46/2c/8b/9dcf5216ca68e14e0320e283692dce8ae321cdc01e73e17796\n",
            "Successfully built emoji\n",
            "Installing collected packages: emoji\n",
            "Successfully installed emoji-0.6.0\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OrqaIzgIzW_A"
      },
      "source": [
        "Merging PsyTAR and RuDReC sentences."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oPriOWfX_5GT",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "68a69ccb-96ae-4508-b94d-164207fb6f38"
      },
      "source": [
        "%cd /content/\n",
        "!ls smm4h_classification/preprocessing/\n",
        "!python smm4h_classification/preprocessing/create_merged_reviews_corpus.py \\\n",
        "--psytar_dir /content/data/psytar_csvs/ \\\n",
        "--rudrec_dir /content/data/rudrec/ \\\n",
        "--output_dir /content/data/combined_psytar_rudrec"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content\n",
            "create_merged_reviews_corpus.py  preprocessing_parameters.py\n",
            "__init__.py\t\t\t preprocessing_utils.py\n",
            "merge_tweets_sets.py\t\t preprocess_tweet_texts.py\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8pJnNj4i0WTY"
      },
      "source": [
        "## Tweets preprocessing\n",
        "\n",
        "During preprocessing, we mask user mentions and urls; replace emojis with words; fix &'s representations: \"\\&amp;\" "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jRzceagam7Kr"
      },
      "source": [
        "!python /content/smm4h_classification/preprocessing/preprocess_tweet_texts.py  --input_dir /content/smm4h_classification/data/raw/ru/ --lang ru --output_dir /content/smm4h_classification/data/preprocessed/ru/"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UGhausA5YA5J"
      },
      "source": [
        "!python /content/smm4h_classification/preprocessing/preprocess_tweet_texts.py  --input_dir /content/smm4h_classification/data/raw/en/ --lang en --output_dir /content/smm4h_classification/data/preprocessed/en/"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HX9K7JezS-A1"
      },
      "source": [
        "## Multilabel classification pretraining"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aerOuRvOQqws",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "bd65e052-2d8d-4084-f1ab-3610745c1c9a"
      },
      "source": [
        "%cd /content/bert/\n",
        "!python multilabel_pretraining.py \\\n",
        "--corpus_dir=/content/data/combined_psytar_rudrec \\\n",
        "--bert_vocab=/content/bert_models/multilingual_eng_rus_reviews_finetuned/vocab.txt \\\n",
        "--bert_checkpoint=/content/bert_models/multilingual_eng_rus_reviews_finetuned/bert_model.ckpt \\\n",
        "--bert_config=/content/bert_models/multilingual_eng_rus_reviews_finetuned/bert_config.json \\\n",
        "--batch_size=32 \\\n",
        "--epochs=5 \\\n",
        "--warmup_proportion=0.1 \\\n",
        "--max_seq_length=128 \\\n",
        "--learning_rate=2e-5 \\\n",
        "--save_summary_steps=500 \\\n",
        "--text_column=sentences \\\n",
        "--num_labels=5 \\\n",
        "--threshold=0.5 \\\n",
        "--output_dir=/content/pretrained_model/ \\\n",
        "--prediction_filename=predicted_labels.csv"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content/bert\n",
            "WARNING:tensorflow:From /content/bert/optimization.py:87: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.\n",
            "\n",
            "WARNING:tensorflow:From /content/bert/tokenization.py:125: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.\n",
            "\n",
            "WARNING:tensorflow:From multilabel_pretraining.py:89: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.\n",
            "\n",
            "WARNING:tensorflow:From multilabel_pretraining.py:89: The name tf.logging.INFO is deprecated. Please use tf.compat.v1.logging.INFO instead.\n",
            "\n",
            "WARNING:tensorflow:From /content/bert/bert_preprocessing.py:257: The name tf.python_io.TFRecordWriter is deprecated. Please use tf.io.TFRecordWriter instead.\n",
            "\n",
            "WARNING:tensorflow:From /content/bert/multilabel_bert.py:286: The name tf.logging.info is deprecated. Please use tf.compat.v1.logging.info instead.\n",
            "\n",
            "INFO:tensorflow:***** Running training *****\n",
            "INFO:tensorflow:  Num examples = 8088\n",
            "INFO:tensorflow:  Batch size = 32\n",
            "INFO:tensorflow:  Num steps = 1263\n",
            "WARNING:tensorflow:From /content/bert/multilabel_bert.py:21: The name tf.FixedLenFeature is deprecated. Please use tf.io.FixedLenFeature instead.\n",
            "\n",
            "Beginning Training!\n",
            "INFO:tensorflow:Not using Distribute Coordinator.\n",
            "INFO:tensorflow:Running training and evaluation locally (non-distributed).\n",
            "INFO:tensorflow:Start train and evaluate loop. The evaluate will happen after every checkpoint. Checkpoint frequency is determined based on RunConfig arguments: save_checkpoints_steps 252 or save_checkpoints_secs None.\n",
            "WARNING:tensorflow:From /tensorflow-1.15.2/python3.6/tensorflow_core/python/training/training_util.py:236: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.\n",
            "WARNING:tensorflow:\n",
            "The TensorFlow contrib module will not be included in TensorFlow 2.0.\n",
            "For more information, please see:\n",
            "  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md\n",
            "  * https://github.com/tensorflow/addons\n",
            "  * https://github.com/tensorflow/io (for I/O related ops)\n",
            "If you depend on functionality not listed there, please file an issue.\n",
            "\n",
            "WARNING:tensorflow:From /content/bert/multilabel_bert.py:57: map_and_batch (from tensorflow.contrib.data.python.ops.batching) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use `tf.data.experimental.map_and_batch(...)`.\n",
            "WARNING:tensorflow:From /tensorflow-1.15.2/python3.6/tensorflow_core/contrib/data/python/ops/batching.py:276: map_and_batch (from tensorflow.python.data.experimental.ops.batching) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use `tf.data.Dataset.map(map_func, num_parallel_calls)` followed by `tf.data.Dataset.batch(batch_size, drop_remainder)`. Static tf.data optimizations will take care of using the fused implementation.\n",
            "WARNING:tensorflow:Entity <function file_based_input_fn_builder.<locals>.input_fn.<locals>.<lambda> at 0x7fb0ff6ad840> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: module 'gast' has no attribute 'Str'\n",
            "WARNING:tensorflow:From /content/bert/multilabel_bert.py:30: The name tf.parse_single_example is deprecated. Please use tf.io.parse_single_example instead.\n",
            "\n",
            "WARNING:tensorflow:From /content/bert/multilabel_bert.py:37: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use `tf.cast` instead.\n",
            "INFO:tensorflow:Calling model_fn.\n",
            "WARNING:tensorflow:From /content/bert/modeling.py:171: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.\n",
            "\n",
            "WARNING:tensorflow:From /content/bert/modeling.py:409: The name tf.get_variable is deprecated. Please use tf.compat.v1.get_variable instead.\n",
            "\n",
            "WARNING:tensorflow:From /content/bert/modeling.py:490: The name tf.assert_less_equal is deprecated. Please use tf.compat.v1.assert_less_equal instead.\n",
            "\n",
            "WARNING:tensorflow:From /content/bert/modeling.py:358: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.\n",
            "WARNING:tensorflow:From /content/bert/modeling.py:671: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use keras.layers.Dense instead.\n",
            "WARNING:tensorflow:From /tensorflow-1.15.2/python3.6/tensorflow_core/python/layers/core.py:187: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Please use `layer.__call__` method instead.\n",
            "INFO:tensorflow:num_labels:5;logits:Tensor(\"loss/BiasAdd:0\", shape=(32, 5), dtype=float32);labels:Tensor(\"loss/Cast:0\", shape=(32, 5), dtype=float32)\n",
            "WARNING:tensorflow:From /tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/nn_impl.py:183: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use tf.where in 2.0, which has the same broadcast rule as np.where\n",
            "WARNING:tensorflow:From /content/bert/multilabel_bert.py:133: The name tf.metrics.accuracy is deprecated. Please use tf.compat.v1.metrics.accuracy instead.\n",
            "\n",
            "WARNING:tensorflow:From /content/bert/multilabel_bert.py:136: The name tf.summary.scalar is deprecated. Please use tf.compat.v1.summary.scalar instead.\n",
            "\n",
            "WARNING:tensorflow:From /content/bert/multilabel_bert.py:139: The name tf.trainable_variables is deprecated. Please use tf.compat.v1.trainable_variables instead.\n",
            "\n",
            "WARNING:tensorflow:From /content/bert/multilabel_bert.py:154: The name tf.train.init_from_checkpoint is deprecated. Please use tf.compat.v1.train.init_from_checkpoint instead.\n",
            "\n",
            "INFO:tensorflow:**** Trainable Variables ****\n",
            "WARNING:tensorflow:From /content/bert/optimization.py:27: The name tf.train.get_or_create_global_step is deprecated. Please use tf.compat.v1.train.get_or_create_global_step instead.\n",
            "\n",
            "WARNING:tensorflow:From /content/bert/optimization.py:32: The name tf.train.polynomial_decay is deprecated. Please use tf.compat.v1.train.polynomial_decay instead.\n",
            "\n",
            "WARNING:tensorflow:From /content/bert/multilabel_bert.py:172: The name tf.train.LoggingTensorHook is deprecated. Please use tf.estimator.LoggingTensorHook instead.\n",
            "\n",
            "INFO:tensorflow:Done calling model_fn.\n",
            "INFO:tensorflow:Create CheckpointSaverHook.\n",
            "INFO:tensorflow:Graph was finalized.\n",
            "2020-12-03 15:44:43.126357: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2200000000 Hz\n",
            "2020-12-03 15:44:43.126625: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1597100 initialized for platform Host (this does not guarantee that XLA will be used). Devices:\n",
            "2020-12-03 15:44:43.126661: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version\n",
            "2020-12-03 15:44:43.130260: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1\n",
            "2020-12-03 15:44:43.323540: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 15:44:43.324302: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1596bc0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:\n",
            "2020-12-03 15:44:43.324331: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Tesla T4, Compute Capability 7.5\n",
            "2020-12-03 15:44:43.325502: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 15:44:43.326130: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 0 with properties: \n",
            "name: Tesla T4 major: 7 minor: 5 memoryClockRate(GHz): 1.59\n",
            "pciBusID: 0000:00:04.0\n",
            "2020-12-03 15:44:43.326574: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1\n",
            "2020-12-03 15:44:43.563584: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10\n",
            "2020-12-03 15:44:43.683236: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10\n",
            "2020-12-03 15:44:43.725082: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10\n",
            "2020-12-03 15:44:43.978665: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10\n",
            "2020-12-03 15:44:44.023859: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10\n",
            "2020-12-03 15:44:44.495972: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7\n",
            "2020-12-03 15:44:44.496178: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 15:44:44.496935: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 15:44:44.497478: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1767] Adding visible gpu devices: 0\n",
            "2020-12-03 15:44:44.499625: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1\n",
            "2020-12-03 15:44:44.500958: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1180] Device interconnect StreamExecutor with strength 1 edge matrix:\n",
            "2020-12-03 15:44:44.500987: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1186]      0 \n",
            "2020-12-03 15:44:44.500999: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1199] 0:   N \n",
            "2020-12-03 15:44:44.519693: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 15:44:44.520364: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 15:44:44.520989: W tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:39] Overriding allow_growth setting because the TF_FORCE_GPU_ALLOW_GROWTH environment variable is set. Original config value was 0.\n",
            "2020-12-03 15:44:44.521033: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1325] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14221 MB memory) -> physical GPU (device: 0, name: Tesla T4, pci bus id: 0000:00:04.0, compute capability: 7.5)\n",
            "2020-12-03 15:44:45.087970: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 367248384 exceeds 10% of system memory.\n",
            "2020-12-03 15:44:45.273997: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 367248384 exceeds 10% of system memory.\n",
            "2020-12-03 15:44:46.320238: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 367248384 exceeds 10% of system memory.\n",
            "2020-12-03 15:44:46.392048: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 367248384 exceeds 10% of system memory.\n",
            "2020-12-03 15:44:46.707336: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 367248384 exceeds 10% of system memory.\n",
            "INFO:tensorflow:Running local_init_op.\n",
            "INFO:tensorflow:Done running local_init_op.\n",
            "INFO:tensorflow:Saving checkpoints for 0 into /content/pretrained_model/model.ckpt.\n",
            "2020-12-03 15:45:15.020178: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10\n",
            "INFO:tensorflow:loss = 0.72989404, step = 0\n",
            "INFO:tensorflow:accuracy = 0.21875, loss = 0.72989404\n",
            "INFO:tensorflow:accuracy = 0.346875, loss = 0.69831026 (15.725 sec)\n",
            "INFO:tensorflow:accuracy = 0.50416666, loss = 0.5735383 (8.435 sec)\n",
            "INFO:tensorflow:accuracy = 0.5921875, loss = 0.45499557 (8.496 sec)\n",
            "INFO:tensorflow:accuracy = 0.64, loss = 0.40316144 (8.537 sec)\n",
            "INFO:tensorflow:accuracy = 0.678125, loss = 0.32355016 (8.591 sec)\n",
            "INFO:tensorflow:accuracy = 0.70535713, loss = 0.35792208 (8.641 sec)\n",
            "INFO:tensorflow:accuracy = 0.72265625, loss = 0.37814635 (8.717 sec)\n",
            "INFO:tensorflow:accuracy = 0.74583334, loss = 0.23566036 (8.762 sec)\n",
            "INFO:tensorflow:accuracy = 0.765625, loss = 0.21114819 (8.766 sec)\n",
            "INFO:tensorflow:global_step/sec: 1.02967\n",
            "INFO:tensorflow:loss = 0.28240532, step = 100 (97.118 sec)\n",
            "INFO:tensorflow:accuracy = 0.77670455, loss = 0.28240532 (12.448 sec)\n",
            "INFO:tensorflow:accuracy = 0.78489584, loss = 0.25937662 (8.748 sec)\n",
            "INFO:tensorflow:accuracy = 0.79759616, loss = 0.16967592 (8.771 sec)\n",
            "INFO:tensorflow:accuracy = 0.8058036, loss = 0.24213341 (8.810 sec)\n",
            "INFO:tensorflow:accuracy = 0.8154167, loss = 0.19473368 (8.865 sec)\n",
            "INFO:tensorflow:accuracy = 0.82304686, loss = 0.18654087 (8.930 sec)\n",
            "INFO:tensorflow:accuracy = 0.8301471, loss = 0.15672414 (8.934 sec)\n",
            "INFO:tensorflow:accuracy = 0.8350694, loss = 0.23476753 (8.995 sec)\n",
            "INFO:tensorflow:accuracy = 0.8401316, loss = 0.18448427 (8.989 sec)\n",
            "INFO:tensorflow:accuracy = 0.845, loss = 0.1405339 (9.026 sec)\n",
            "INFO:tensorflow:global_step/sec: 1.12246\n",
            "INFO:tensorflow:loss = 0.1626044, step = 200 (89.090 sec)\n",
            "INFO:tensorflow:accuracy = 0.84910715, loss = 0.1626044 (9.022 sec)\n",
            "INFO:tensorflow:accuracy = 0.85426134, loss = 0.15429278 (9.042 sec)\n",
            "INFO:tensorflow:accuracy = 0.85625, loss = 0.2544128 (9.049 sec)\n",
            "INFO:tensorflow:accuracy = 0.8588542, loss = 0.19428882 (9.053 sec)\n",
            "INFO:tensorflow:accuracy = 0.86225, loss = 0.16314824 (9.080 sec)\n",
            "INFO:tensorflow:accuracy = 0.8663462, loss = 0.13871428 (9.108 sec)\n",
            "INFO:tensorflow:Saving checkpoints for 252 into /content/pretrained_model/model.ckpt.\n",
            "WARNING:tensorflow:From /tensorflow-1.15.2/python3.6/tensorflow_core/python/training/saver.py:963: remove_checkpoint (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use standard file APIs to delete files with this prefix.\n",
            "WARNING:tensorflow:Entity <function file_based_input_fn_builder.<locals>.input_fn.<locals>.<lambda> at 0x7fb0f57abd90> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: module 'gast' has no attribute 'Str'\n",
            "INFO:tensorflow:Calling model_fn.\n",
            "INFO:tensorflow:num_labels:5;logits:Tensor(\"loss/BiasAdd:0\", shape=(?, 5), dtype=float32);labels:Tensor(\"loss/Cast:0\", shape=(?, 5), dtype=float32)\n",
            "INFO:tensorflow:**** Trainable Variables ****\n",
            "WARNING:tensorflow:From /content/bert/multilabel_bert.py:191: The name tf.metrics.auc is deprecated. Please use tf.compat.v1.metrics.auc instead.\n",
            "\n",
            "WARNING:tensorflow:From /tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/metrics_impl.py:808: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Deprecated in favor of operator or tf.math.divide.\n",
            "WARNING:tensorflow:From /content/bert/multilabel_bert.py:193: The name tf.metrics.mean is deprecated. Please use tf.compat.v1.metrics.mean instead.\n",
            "\n",
            "INFO:tensorflow:Done calling model_fn.\n",
            "INFO:tensorflow:Starting evaluation at 2020-12-03T15:49:34Z\n",
            "INFO:tensorflow:Graph was finalized.\n",
            "2020-12-03 15:49:35.250559: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 15:49:35.250898: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 0 with properties: \n",
            "name: Tesla T4 major: 7 minor: 5 memoryClockRate(GHz): 1.59\n",
            "pciBusID: 0000:00:04.0\n",
            "2020-12-03 15:49:35.250994: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1\n",
            "2020-12-03 15:49:35.251018: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10\n",
            "2020-12-03 15:49:35.251039: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10\n",
            "2020-12-03 15:49:35.251061: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10\n",
            "2020-12-03 15:49:35.251082: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10\n",
            "2020-12-03 15:49:35.251101: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10\n",
            "2020-12-03 15:49:35.251121: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7\n",
            "2020-12-03 15:49:35.251207: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 15:49:35.251568: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 15:49:35.251829: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1767] Adding visible gpu devices: 0\n",
            "2020-12-03 15:49:35.251879: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1180] Device interconnect StreamExecutor with strength 1 edge matrix:\n",
            "2020-12-03 15:49:35.251892: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1186]      0 \n",
            "2020-12-03 15:49:35.251901: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1199] 0:   N \n",
            "2020-12-03 15:49:35.251991: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 15:49:35.252276: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 15:49:35.252564: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1325] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14221 MB memory) -> physical GPU (device: 0, name: Tesla T4, pci bus id: 0000:00:04.0, compute capability: 7.5)\n",
            "INFO:tensorflow:Restoring parameters from /content/pretrained_model/model.ckpt-252\n",
            "INFO:tensorflow:Running local_init_op.\n",
            "INFO:tensorflow:Done running local_init_op.\n",
            "INFO:tensorflow:accuracy = 0.94375, loss = 0.16259398\n",
            "INFO:tensorflow:accuracy = 0.94375, loss = 0.16630259 (1.063 sec)\n",
            "INFO:tensorflow:accuracy = 0.9458333, loss = 0.1354974 (0.581 sec)\n",
            "INFO:tensorflow:accuracy = 0.9359375, loss = 0.22901575 (0.547 sec)\n",
            "INFO:tensorflow:accuracy = 0.9325, loss = 0.18552168 (0.576 sec)\n",
            "INFO:tensorflow:accuracy = 0.93541664, loss = 0.1632175 (0.559 sec)\n",
            "INFO:tensorflow:accuracy = 0.93125, loss = 0.22930269 (0.573 sec)\n",
            "INFO:tensorflow:accuracy = 0.9351562, loss = 0.13651103 (0.566 sec)\n",
            "INFO:tensorflow:accuracy = 0.93819445, loss = 0.12907901 (0.564 sec)\n",
            "INFO:tensorflow:accuracy = 0.939375, loss = 0.16651061 (0.574 sec)\n",
            "INFO:tensorflow:accuracy = 0.9380682, loss = 0.18347353 (0.568 sec)\n",
            "INFO:tensorflow:accuracy = 0.9401042, loss = 0.12466067 (0.570 sec)\n",
            "INFO:tensorflow:accuracy = 0.940201, loss = 0.14613903 (0.407 sec)\n",
            "INFO:tensorflow:Finished evaluation at 2020-12-03-15:49:44\n",
            "INFO:tensorflow:Saving dict for global step 252: 0 = 0.9736801, 1 = 0.9371994, 2 = 0.96466875, 3 = 0.9717435, 4 = 0.8787878, eval_loss = 0.14789742, global_step = 252, loss = 0.14785787\n",
            "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 252: /content/pretrained_model/model.ckpt-252\n",
            "INFO:tensorflow:accuracy = 0.8699074, loss = 0.13621289 (44.534 sec)\n",
            "INFO:tensorflow:accuracy = 0.87254465, loss = 0.15677953 (9.003 sec)\n",
            "INFO:tensorflow:accuracy = 0.875, loss = 0.17001167 (9.034 sec)\n",
            "INFO:tensorflow:accuracy = 0.876875, loss = 0.15297952 (9.035 sec)\n",
            "INFO:tensorflow:global_step/sec: 0.793642\n",
            "INFO:tensorflow:loss = 0.14828858, step = 300 (126.001 sec)\n",
            "INFO:tensorflow:accuracy = 0.8796371, loss = 0.14828858 (9.063 sec)\n",
            "INFO:tensorflow:accuracy = 0.8816406, loss = 0.179762 (9.058 sec)\n",
            "INFO:tensorflow:accuracy = 0.8837121, loss = 0.13247964 (9.069 sec)\n",
            "INFO:tensorflow:accuracy = 0.88400733, loss = 0.23523831 (9.086 sec)\n",
            "INFO:tensorflow:accuracy = 0.88535714, loss = 0.16421734 (9.101 sec)\n",
            "INFO:tensorflow:accuracy = 0.88697916, loss = 0.15176268 (9.127 sec)\n",
            "INFO:tensorflow:accuracy = 0.8878378, loss = 0.16308865 (9.146 sec)\n",
            "INFO:tensorflow:accuracy = 0.8893092, loss = 0.14730516 (9.161 sec)\n",
            "INFO:tensorflow:accuracy = 0.8908654, loss = 0.13392124 (9.169 sec)\n",
            "INFO:tensorflow:accuracy = 0.8921875, loss = 0.14753923 (9.215 sec)\n",
            "INFO:tensorflow:global_step/sec: 1.09452\n",
            "INFO:tensorflow:loss = 0.12888154, step = 400 (91.364 sec)\n",
            "INFO:tensorflow:accuracy = 0.89359754, loss = 0.12888154 (9.232 sec)\n",
            "INFO:tensorflow:accuracy = 0.8949405, loss = 0.11568947 (9.240 sec)\n",
            "INFO:tensorflow:accuracy = 0.89593023, loss = 0.15871711 (9.268 sec)\n",
            "INFO:tensorflow:accuracy = 0.89772725, loss = 0.0863826 (9.269 sec)\n",
            "INFO:tensorflow:accuracy = 0.89916664, loss = 0.11092188 (9.292 sec)\n",
            "INFO:tensorflow:accuracy = 0.9009511, loss = 0.085421756 (9.260 sec)\n",
            "INFO:tensorflow:accuracy = 0.90226066, loss = 0.13207924 (9.270 sec)\n",
            "INFO:tensorflow:accuracy = 0.903125, loss = 0.15788932 (9.276 sec)\n",
            "INFO:tensorflow:accuracy = 0.90459186, loss = 0.09883602 (9.260 sec)\n",
            "INFO:tensorflow:accuracy = 0.906, loss = 0.08108566 (9.286 sec)\n",
            "INFO:tensorflow:global_step/sec: 1.03952\n",
            "INFO:tensorflow:loss = 0.09199138, step = 500 (96.198 sec)\n",
            "INFO:tensorflow:accuracy = 0.9074755, loss = 0.09199138 (12.775 sec)\n",
            "INFO:tensorflow:Saving checkpoints for 504 into /content/pretrained_model/model.ckpt.\n",
            "WARNING:tensorflow:Entity <function file_based_input_fn_builder.<locals>.input_fn.<locals>.<lambda> at 0x7fb09077ff28> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: module 'gast' has no attribute 'Str'\n",
            "INFO:tensorflow:Calling model_fn.\n",
            "INFO:tensorflow:num_labels:5;logits:Tensor(\"loss/BiasAdd:0\", shape=(?, 5), dtype=float32);labels:Tensor(\"loss/Cast:0\", shape=(?, 5), dtype=float32)\n",
            "INFO:tensorflow:**** Trainable Variables ****\n",
            "INFO:tensorflow:Done calling model_fn.\n",
            "INFO:tensorflow:Starting evaluation at 2020-12-03T15:54:04Z\n",
            "INFO:tensorflow:Graph was finalized.\n",
            "2020-12-03 15:54:05.017670: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 15:54:05.017998: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 0 with properties: \n",
            "name: Tesla T4 major: 7 minor: 5 memoryClockRate(GHz): 1.59\n",
            "pciBusID: 0000:00:04.0\n",
            "2020-12-03 15:54:05.019840: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1\n",
            "2020-12-03 15:54:05.019907: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10\n",
            "2020-12-03 15:54:05.019930: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10\n",
            "2020-12-03 15:54:05.019951: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10\n",
            "2020-12-03 15:54:05.019974: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10\n",
            "2020-12-03 15:54:05.019992: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10\n",
            "2020-12-03 15:54:05.020012: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7\n",
            "2020-12-03 15:54:05.020093: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 15:54:05.020467: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 15:54:05.020737: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1767] Adding visible gpu devices: 0\n",
            "2020-12-03 15:54:05.020838: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1180] Device interconnect StreamExecutor with strength 1 edge matrix:\n",
            "2020-12-03 15:54:05.020853: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1186]      0 \n",
            "2020-12-03 15:54:05.020862: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1199] 0:   N \n",
            "2020-12-03 15:54:05.020993: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 15:54:05.021299: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 15:54:05.021565: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1325] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14221 MB memory) -> physical GPU (device: 0, name: Tesla T4, pci bus id: 0000:00:04.0, compute capability: 7.5)\n",
            "INFO:tensorflow:Restoring parameters from /content/pretrained_model/model.ckpt-504\n",
            "INFO:tensorflow:Running local_init_op.\n",
            "INFO:tensorflow:Done running local_init_op.\n",
            "INFO:tensorflow:accuracy = 0.9625, loss = 0.122648045\n",
            "INFO:tensorflow:accuracy = 0.946875, loss = 0.1646543 (1.074 sec)\n",
            "INFO:tensorflow:accuracy = 0.95416665, loss = 0.09758177 (0.586 sec)\n",
            "INFO:tensorflow:accuracy = 0.9484375, loss = 0.21760337 (0.555 sec)\n",
            "INFO:tensorflow:accuracy = 0.95125, loss = 0.12587364 (0.583 sec)\n",
            "INFO:tensorflow:accuracy = 0.95104164, loss = 0.14385441 (0.567 sec)\n",
            "INFO:tensorflow:accuracy = 0.95357144, loss = 0.10906271 (0.572 sec)\n",
            "INFO:tensorflow:accuracy = 0.9554688, loss = 0.077872366 (0.575 sec)\n",
            "INFO:tensorflow:accuracy = 0.95763886, loss = 0.08270786 (0.567 sec)\n",
            "INFO:tensorflow:accuracy = 0.958125, loss = 0.100899324 (0.581 sec)\n",
            "INFO:tensorflow:accuracy = 0.9590909, loss = 0.11050081 (0.563 sec)\n",
            "INFO:tensorflow:accuracy = 0.9609375, loss = 0.069298096 (0.578 sec)\n",
            "INFO:tensorflow:accuracy = 0.96231157, loss = 0.049957577 (0.413 sec)\n",
            "INFO:tensorflow:Finished evaluation at 2020-12-03-15:54:14\n",
            "INFO:tensorflow:Saving dict for global step 504: 0 = 0.98670393, 1 = 0.9607651, 2 = 0.98276097, 3 = 0.9818303, 4 = 0.9160173, eval_loss = 0.104513876, global_step = 504, loss = 0.10328634\n",
            "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 504: /content/pretrained_model/model.ckpt-504\n",
            "INFO:tensorflow:accuracy = 0.908774, loss = 0.11442802 (43.661 sec)\n",
            "INFO:tensorflow:accuracy = 0.9096698, loss = 0.1351094 (9.051 sec)\n",
            "INFO:tensorflow:accuracy = 0.91064817, loss = 0.119114354 (9.061 sec)\n",
            "INFO:tensorflow:accuracy = 0.91125, loss = 0.14974634 (9.081 sec)\n",
            "INFO:tensorflow:accuracy = 0.9122768, loss = 0.08935064 (9.093 sec)\n",
            "INFO:tensorflow:accuracy = 0.9135965, loss = 0.08354577 (9.118 sec)\n",
            "INFO:tensorflow:accuracy = 0.9144397, loss = 0.095749214 (9.138 sec)\n",
            "INFO:tensorflow:accuracy = 0.91525424, loss = 0.114392206 (9.150 sec)\n",
            "INFO:tensorflow:accuracy = 0.9164583, loss = 0.04401397 (9.164 sec)\n",
            "INFO:tensorflow:global_step/sec: 0.795571\n",
            "INFO:tensorflow:loss = 0.08932591, step = 600 (125.696 sec)\n",
            "INFO:tensorflow:accuracy = 0.9171107, loss = 0.08932591 (9.179 sec)\n",
            "INFO:tensorflow:accuracy = 0.91844755, loss = 0.02679593 (9.197 sec)\n",
            "INFO:tensorflow:accuracy = 0.91974205, loss = 0.026652819 (9.213 sec)\n",
            "INFO:tensorflow:accuracy = 0.92099607, loss = 0.025177214 (9.210 sec)\n",
            "INFO:tensorflow:accuracy = 0.92173076, loss = 0.09771876 (9.244 sec)\n",
            "INFO:tensorflow:accuracy = 0.9227273, loss = 0.06757225 (9.237 sec)\n",
            "INFO:tensorflow:accuracy = 0.9234142, loss = 0.096180454 (9.250 sec)\n",
            "INFO:tensorflow:accuracy = 0.9237132, loss = 0.10637574 (9.264 sec)\n",
            "INFO:tensorflow:accuracy = 0.9247283, loss = 0.049413245 (9.281 sec)\n",
            "INFO:tensorflow:accuracy = 0.925625, loss = 0.04572497 (9.294 sec)\n",
            "INFO:tensorflow:global_step/sec: 1.08102\n",
            "INFO:tensorflow:loss = 0.081647694, step = 700 (92.505 sec)\n",
            "INFO:tensorflow:accuracy = 0.92640847, loss = 0.081647694 (9.315 sec)\n",
            "INFO:tensorflow:accuracy = 0.92690974, loss = 0.08723676 (9.293 sec)\n",
            "INFO:tensorflow:accuracy = 0.92739725, loss = 0.13107117 (9.317 sec)\n",
            "INFO:tensorflow:accuracy = 0.9277027, loss = 0.11365805 (9.315 sec)\n",
            "INFO:tensorflow:accuracy = 0.9285, loss = 0.06353283 (9.320 sec)\n",
            "INFO:tensorflow:accuracy = 0.9290296, loss = 0.075896 (9.333 sec)\n",
            "INFO:tensorflow:Saving checkpoints for 756 into /content/pretrained_model/model.ckpt.\n",
            "WARNING:tensorflow:Entity <function file_based_input_fn_builder.<locals>.input_fn.<locals>.<lambda> at 0x7fb0905e9268> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: module 'gast' has no attribute 'Str'\n",
            "INFO:tensorflow:Calling model_fn.\n",
            "INFO:tensorflow:num_labels:5;logits:Tensor(\"loss/BiasAdd:0\", shape=(?, 5), dtype=float32);labels:Tensor(\"loss/Cast:0\", shape=(?, 5), dtype=float32)\n",
            "INFO:tensorflow:**** Trainable Variables ****\n",
            "INFO:tensorflow:Done calling model_fn.\n",
            "INFO:tensorflow:Starting evaluation at 2020-12-03T15:58:30Z\n",
            "INFO:tensorflow:Graph was finalized.\n",
            "2020-12-03 15:58:31.277950: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 15:58:31.278326: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 0 with properties: \n",
            "name: Tesla T4 major: 7 minor: 5 memoryClockRate(GHz): 1.59\n",
            "pciBusID: 0000:00:04.0\n",
            "2020-12-03 15:58:31.278445: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1\n",
            "2020-12-03 15:58:31.278490: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10\n",
            "2020-12-03 15:58:31.278515: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10\n",
            "2020-12-03 15:58:31.278550: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10\n",
            "2020-12-03 15:58:31.278569: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10\n",
            "2020-12-03 15:58:31.278587: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10\n",
            "2020-12-03 15:58:31.278608: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7\n",
            "2020-12-03 15:58:31.278699: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 15:58:31.279164: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 15:58:31.279455: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1767] Adding visible gpu devices: 0\n",
            "2020-12-03 15:58:31.279501: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1180] Device interconnect StreamExecutor with strength 1 edge matrix:\n",
            "2020-12-03 15:58:31.279516: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1186]      0 \n",
            "2020-12-03 15:58:31.279525: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1199] 0:   N \n",
            "2020-12-03 15:58:31.279618: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 15:58:31.279906: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 15:58:31.280187: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1325] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14221 MB memory) -> physical GPU (device: 0, name: Tesla T4, pci bus id: 0000:00:04.0, compute capability: 7.5)\n",
            "INFO:tensorflow:Restoring parameters from /content/pretrained_model/model.ckpt-756\n",
            "INFO:tensorflow:Running local_init_op.\n",
            "INFO:tensorflow:Done running local_init_op.\n",
            "INFO:tensorflow:accuracy = 0.96875, loss = 0.101344444\n",
            "INFO:tensorflow:accuracy = 0.959375, loss = 0.17922378 (1.091 sec)\n",
            "INFO:tensorflow:accuracy = 0.96666664, loss = 0.089656934 (0.586 sec)\n",
            "INFO:tensorflow:accuracy = 0.9578125, loss = 0.22628915 (0.554 sec)\n",
            "INFO:tensorflow:accuracy = 0.95875, loss = 0.1056685 (0.580 sec)\n",
            "INFO:tensorflow:accuracy = 0.9614583, loss = 0.1126703 (0.565 sec)\n",
            "INFO:tensorflow:accuracy = 0.96428573, loss = 0.090998925 (0.568 sec)\n",
            "INFO:tensorflow:accuracy = 0.96484375, loss = 0.06272755 (0.572 sec)\n",
            "INFO:tensorflow:accuracy = 0.96597224, loss = 0.08058165 (0.563 sec)\n",
            "INFO:tensorflow:accuracy = 0.966875, loss = 0.086993895 (0.575 sec)\n",
            "INFO:tensorflow:accuracy = 0.9664773, loss = 0.09302237 (0.559 sec)\n",
            "INFO:tensorflow:accuracy = 0.9682292, loss = 0.05312265 (0.577 sec)\n",
            "INFO:tensorflow:accuracy = 0.96934676, loss = 0.03228813 (0.405 sec)\n",
            "INFO:tensorflow:Finished evaluation at 2020-12-03-15:58:40\n",
            "INFO:tensorflow:Saving dict for global step 756: 0 = 0.99207556, 1 = 0.9633744, 2 = 0.9805637, 3 = 0.9789011, 4 = 0.9246754, eval_loss = 0.09432583, global_step = 756, loss = 0.092929974\n",
            "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 756: /content/pretrained_model/model.ckpt-756\n",
            "INFO:tensorflow:accuracy = 0.92978895, loss = 0.05364446 (43.388 sec)\n",
            "INFO:tensorflow:accuracy = 0.930609, loss = 0.042765684 (9.061 sec)\n",
            "INFO:tensorflow:accuracy = 0.93125, loss = 0.05273257 (9.092 sec)\n",
            "INFO:tensorflow:accuracy = 0.93195313, loss = 0.058520608 (9.103 sec)\n",
            "INFO:tensorflow:global_step/sec: 0.791575\n",
            "INFO:tensorflow:loss = 0.03963072, step = 800 (126.330 sec)\n",
            "INFO:tensorflow:accuracy = 0.9327161, loss = 0.03963072 (9.110 sec)\n",
            "INFO:tensorflow:accuracy = 0.93330795, loss = 0.06330124 (9.146 sec)\n",
            "INFO:tensorflow:accuracy = 0.9338855, loss = 0.06194984 (9.160 sec)\n",
            "INFO:tensorflow:accuracy = 0.934375, loss = 0.08265408 (9.180 sec)\n",
            "INFO:tensorflow:accuracy = 0.93485296, loss = 0.061651193 (9.193 sec)\n",
            "INFO:tensorflow:accuracy = 0.93539244, loss = 0.09329298 (9.212 sec)\n",
            "INFO:tensorflow:accuracy = 0.93613505, loss = 0.036250323 (9.222 sec)\n",
            "INFO:tensorflow:accuracy = 0.93671876, loss = 0.05138553 (9.220 sec)\n",
            "INFO:tensorflow:accuracy = 0.9372893, loss = 0.05397929 (9.237 sec)\n",
            "INFO:tensorflow:accuracy = 0.9378472, loss = 0.06364551 (9.235 sec)\n",
            "INFO:tensorflow:global_step/sec: 1.08611\n",
            "INFO:tensorflow:loss = 0.037404668, step = 900 (92.071 sec)\n",
            "INFO:tensorflow:accuracy = 0.93846154, loss = 0.037404668 (9.266 sec)\n",
            "INFO:tensorflow:accuracy = 0.9389946, loss = 0.061928958 (9.265 sec)\n",
            "INFO:tensorflow:accuracy = 0.9395161, loss = 0.054958068 (9.313 sec)\n",
            "INFO:tensorflow:accuracy = 0.9400266, loss = 0.056333613 (9.357 sec)\n",
            "INFO:tensorflow:accuracy = 0.9405921, loss = 0.044158306 (9.371 sec)\n",
            "INFO:tensorflow:accuracy = 0.94114584, loss = 0.05323438 (9.395 sec)\n",
            "INFO:tensorflow:accuracy = 0.9416881, loss = 0.02851173 (9.377 sec)\n",
            "INFO:tensorflow:accuracy = 0.94196427, loss = 0.06688686 (9.361 sec)\n",
            "INFO:tensorflow:accuracy = 0.9425505, loss = 0.030130481 (9.324 sec)\n",
            "INFO:tensorflow:accuracy = 0.9430625, loss = 0.03984626 (9.315 sec)\n",
            "INFO:tensorflow:global_step/sec: 1.07063\n",
            "INFO:tensorflow:loss = 0.03045258, step = 1000 (93.403 sec)\n",
            "INFO:tensorflow:accuracy = 0.9436262, loss = 0.03045258 (9.325 sec)\n",
            "INFO:tensorflow:Saving checkpoints for 1008 into /content/pretrained_model/model.ckpt.\n",
            "WARNING:tensorflow:Entity <function file_based_input_fn_builder.<locals>.input_fn.<locals>.<lambda> at 0x7fafecbc8950> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: module 'gast' has no attribute 'Str'\n",
            "INFO:tensorflow:Calling model_fn.\n",
            "INFO:tensorflow:num_labels:5;logits:Tensor(\"loss/BiasAdd:0\", shape=(?, 5), dtype=float32);labels:Tensor(\"loss/Cast:0\", shape=(?, 5), dtype=float32)\n",
            "INFO:tensorflow:**** Trainable Variables ****\n",
            "INFO:tensorflow:Done calling model_fn.\n",
            "INFO:tensorflow:Starting evaluation at 2020-12-03T16:02:57Z\n",
            "INFO:tensorflow:Graph was finalized.\n",
            "2020-12-03 16:02:58.026375: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 16:02:58.026732: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 0 with properties: \n",
            "name: Tesla T4 major: 7 minor: 5 memoryClockRate(GHz): 1.59\n",
            "pciBusID: 0000:00:04.0\n",
            "2020-12-03 16:02:58.026829: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1\n",
            "2020-12-03 16:02:58.026858: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10\n",
            "2020-12-03 16:02:58.026881: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10\n",
            "2020-12-03 16:02:58.026906: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10\n",
            "2020-12-03 16:02:58.026926: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10\n",
            "2020-12-03 16:02:58.026946: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10\n",
            "2020-12-03 16:02:58.026967: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7\n",
            "2020-12-03 16:02:58.027048: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 16:02:58.027369: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 16:02:58.027623: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1767] Adding visible gpu devices: 0\n",
            "2020-12-03 16:02:58.027668: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1180] Device interconnect StreamExecutor with strength 1 edge matrix:\n",
            "2020-12-03 16:02:58.027687: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1186]      0 \n",
            "2020-12-03 16:02:58.027696: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1199] 0:   N \n",
            "2020-12-03 16:02:58.027791: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 16:02:58.028080: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 16:02:58.028339: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1325] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14221 MB memory) -> physical GPU (device: 0, name: Tesla T4, pci bus id: 0000:00:04.0, compute capability: 7.5)\n",
            "INFO:tensorflow:Restoring parameters from /content/pretrained_model/model.ckpt-1008\n",
            "INFO:tensorflow:Running local_init_op.\n",
            "INFO:tensorflow:Done running local_init_op.\n",
            "INFO:tensorflow:accuracy = 0.96875, loss = 0.08990125\n",
            "INFO:tensorflow:accuracy = 0.95625, loss = 0.17121807 (1.093 sec)\n",
            "INFO:tensorflow:accuracy = 0.96666664, loss = 0.07865625 (0.597 sec)\n",
            "INFO:tensorflow:accuracy = 0.95625, loss = 0.23752913 (0.567 sec)\n",
            "INFO:tensorflow:accuracy = 0.96, loss = 0.09010588 (0.598 sec)\n",
            "INFO:tensorflow:accuracy = 0.9604167, loss = 0.11548426 (0.587 sec)\n",
            "INFO:tensorflow:accuracy = 0.96160716, loss = 0.098019615 (0.585 sec)\n",
            "INFO:tensorflow:accuracy = 0.9640625, loss = 0.040782303 (0.596 sec)\n",
            "INFO:tensorflow:accuracy = 0.96458334, loss = 0.07837526 (0.580 sec)\n",
            "INFO:tensorflow:accuracy = 0.965, loss = 0.09258925 (0.601 sec)\n",
            "INFO:tensorflow:accuracy = 0.9653409, loss = 0.092903316 (0.592 sec)\n",
            "INFO:tensorflow:accuracy = 0.9671875, loss = 0.045074232 (0.593 sec)\n",
            "INFO:tensorflow:accuracy = 0.9683417, loss = 0.028002629 (0.432 sec)\n",
            "INFO:tensorflow:Finished evaluation at 2020-12-03-16:03:07\n",
            "INFO:tensorflow:Saving dict for global step 1008: 0 = 0.9930011, 1 = 0.963306, 2 = 0.98372746, 3 = 0.98100305, 4 = 0.94339824, eval_loss = 0.08718278, global_step = 1008, loss = 0.08585123\n",
            "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 1008: /content/pretrained_model/model.ckpt-1008\n",
            "INFO:tensorflow:accuracy = 0.94417894, loss = 0.023087408 (43.345 sec)\n",
            "INFO:tensorflow:accuracy = 0.9446602, loss = 0.022959735 (9.442 sec)\n",
            "INFO:tensorflow:accuracy = 0.9451322, loss = 0.045927074 (9.629 sec)\n",
            "INFO:tensorflow:accuracy = 0.9455357, loss = 0.067591354 (9.608 sec)\n",
            "INFO:tensorflow:accuracy = 0.94587266, loss = 0.05170436 (9.418 sec)\n",
            "INFO:tensorflow:accuracy = 0.9462617, loss = 0.06323792 (9.302 sec)\n",
            "INFO:tensorflow:accuracy = 0.9465278, loss = 0.06024913 (9.278 sec)\n",
            "INFO:tensorflow:accuracy = 0.9470183, loss = 0.021092158 (9.260 sec)\n",
            "INFO:tensorflow:accuracy = 0.9473864, loss = 0.036240395 (9.299 sec)\n",
            "INFO:tensorflow:global_step/sec: 0.781571\n",
            "INFO:tensorflow:loss = 0.06906842, step = 1100 (127.947 sec)\n",
            "INFO:tensorflow:accuracy = 0.94769144, loss = 0.06906842 (9.364 sec)\n",
            "INFO:tensorflow:accuracy = 0.9481585, loss = 0.016415436 (9.423 sec)\n",
            "INFO:tensorflow:accuracy = 0.94856197, loss = 0.031713076 (9.416 sec)\n",
            "INFO:tensorflow:accuracy = 0.9490132, loss = 0.017752543 (9.386 sec)\n",
            "INFO:tensorflow:accuracy = 0.94934785, loss = 0.073002994 (9.349 sec)\n",
            "INFO:tensorflow:accuracy = 0.94978446, loss = 0.020255161 (9.334 sec)\n",
            "INFO:tensorflow:accuracy = 0.9502137, loss = 0.03017421 (9.326 sec)\n",
            "INFO:tensorflow:accuracy = 0.95052963, loss = 0.07215066 (9.326 sec)\n",
            "INFO:tensorflow:accuracy = 0.95084035, loss = 0.051851057 (9.351 sec)\n",
            "INFO:tensorflow:accuracy = 0.95109373, loss = 0.08387108 (9.367 sec)\n",
            "INFO:tensorflow:global_step/sec: 1.06753\n",
            "INFO:tensorflow:loss = 0.0212176, step = 1200 (93.674 sec)\n",
            "INFO:tensorflow:accuracy = 0.9514979, loss = 0.0212176 (9.397 sec)\n",
            "INFO:tensorflow:accuracy = 0.9518955, loss = 0.011481852 (9.386 sec)\n",
            "INFO:tensorflow:accuracy = 0.95223576, loss = 0.04336855 (9.360 sec)\n",
            "INFO:tensorflow:accuracy = 0.95257056, loss = 0.032237172 (9.352 sec)\n",
            "INFO:tensorflow:accuracy = 0.9528, loss = 0.089674465 (9.335 sec)\n",
            "INFO:tensorflow:accuracy = 0.9531746, loss = 0.022480339 (9.324 sec)\n",
            "INFO:tensorflow:Saving checkpoints for 1260 into /content/pretrained_model/model.ckpt.\n",
            "WARNING:tensorflow:Entity <function file_based_input_fn_builder.<locals>.input_fn.<locals>.<lambda> at 0x7fafeb63c488> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: module 'gast' has no attribute 'Str'\n",
            "INFO:tensorflow:Calling model_fn.\n",
            "INFO:tensorflow:num_labels:5;logits:Tensor(\"loss/BiasAdd:0\", shape=(?, 5), dtype=float32);labels:Tensor(\"loss/Cast:0\", shape=(?, 5), dtype=float32)\n",
            "INFO:tensorflow:**** Trainable Variables ****\n",
            "INFO:tensorflow:Done calling model_fn.\n",
            "INFO:tensorflow:Starting evaluation at 2020-12-03T16:07:28Z\n",
            "INFO:tensorflow:Graph was finalized.\n",
            "2020-12-03 16:07:28.515023: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 16:07:28.515371: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 0 with properties: \n",
            "name: Tesla T4 major: 7 minor: 5 memoryClockRate(GHz): 1.59\n",
            "pciBusID: 0000:00:04.0\n",
            "2020-12-03 16:07:28.515471: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1\n",
            "2020-12-03 16:07:28.515503: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10\n",
            "2020-12-03 16:07:28.515527: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10\n",
            "2020-12-03 16:07:28.515548: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10\n",
            "2020-12-03 16:07:28.515569: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10\n",
            "2020-12-03 16:07:28.515588: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10\n",
            "2020-12-03 16:07:28.515608: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7\n",
            "2020-12-03 16:07:28.515688: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 16:07:28.516002: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 16:07:28.516251: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1767] Adding visible gpu devices: 0\n",
            "2020-12-03 16:07:28.516300: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1180] Device interconnect StreamExecutor with strength 1 edge matrix:\n",
            "2020-12-03 16:07:28.516314: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1186]      0 \n",
            "2020-12-03 16:07:28.516323: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1199] 0:   N \n",
            "2020-12-03 16:07:28.516429: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 16:07:28.516735: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 16:07:28.517003: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1325] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14221 MB memory) -> physical GPU (device: 0, name: Tesla T4, pci bus id: 0000:00:04.0, compute capability: 7.5)\n",
            "INFO:tensorflow:Restoring parameters from /content/pretrained_model/model.ckpt-1260\n",
            "INFO:tensorflow:Running local_init_op.\n",
            "INFO:tensorflow:Done running local_init_op.\n",
            "INFO:tensorflow:accuracy = 0.96875, loss = 0.09218173\n",
            "INFO:tensorflow:accuracy = 0.959375, loss = 0.17201835 (1.118 sec)\n",
            "INFO:tensorflow:accuracy = 0.96875, loss = 0.06815762 (0.591 sec)\n",
            "INFO:tensorflow:accuracy = 0.9609375, loss = 0.21383107 (0.567 sec)\n",
            "INFO:tensorflow:accuracy = 0.96375, loss = 0.09057747 (0.595 sec)\n",
            "INFO:tensorflow:accuracy = 0.96458334, loss = 0.11571033 (0.581 sec)\n",
            "INFO:tensorflow:accuracy = 0.96428573, loss = 0.10424285 (0.580 sec)\n",
            "INFO:tensorflow:accuracy = 0.96796876, loss = 0.026771927 (0.588 sec)\n",
            "INFO:tensorflow:accuracy = 0.96875, loss = 0.06889564 (0.574 sec)\n",
            "INFO:tensorflow:accuracy = 0.96875, loss = 0.09828354 (0.597 sec)\n",
            "INFO:tensorflow:accuracy = 0.9693182, loss = 0.08152844 (0.578 sec)\n",
            "INFO:tensorflow:accuracy = 0.97083336, loss = 0.03693514 (0.594 sec)\n",
            "INFO:tensorflow:accuracy = 0.9718593, loss = 0.020572778 (0.421 sec)\n",
            "INFO:tensorflow:Finished evaluation at 2020-12-03-16:07:38\n",
            "INFO:tensorflow:Saving dict for global step 1260: 0 = 0.9926502, 1 = 0.9661885, 2 = 0.98651004, 3 = 0.9771117, 4 = 0.9411255, eval_loss = 0.083129354, global_step = 1260, loss = 0.081721805\n",
            "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 1260: /content/pretrained_model/model.ckpt-1260\n",
            "INFO:tensorflow:accuracy = 0.9534449, loss = 0.033377323 (43.593 sec)\n",
            "INFO:tensorflow:Saving checkpoints for 1263 into /content/pretrained_model/model.ckpt.\n",
            "WARNING:tensorflow:Entity <function file_based_input_fn_builder.<locals>.input_fn.<locals>.<lambda> at 0x7fb09077ff28> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: module 'gast' has no attribute 'Str'\n",
            "INFO:tensorflow:Calling model_fn.\n",
            "INFO:tensorflow:num_labels:5;logits:Tensor(\"loss/BiasAdd:0\", shape=(?, 5), dtype=float32);labels:Tensor(\"loss/Cast:0\", shape=(?, 5), dtype=float32)\n",
            "INFO:tensorflow:**** Trainable Variables ****\n",
            "INFO:tensorflow:Done calling model_fn.\n",
            "INFO:tensorflow:Starting evaluation at 2020-12-03T16:08:17Z\n",
            "INFO:tensorflow:Graph was finalized.\n",
            "2020-12-03 16:08:18.083203: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 16:08:18.083582: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 0 with properties: \n",
            "name: Tesla T4 major: 7 minor: 5 memoryClockRate(GHz): 1.59\n",
            "pciBusID: 0000:00:04.0\n",
            "2020-12-03 16:08:18.083659: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1\n",
            "2020-12-03 16:08:18.083687: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10\n",
            "2020-12-03 16:08:18.083708: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10\n",
            "2020-12-03 16:08:18.083729: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10\n",
            "2020-12-03 16:08:18.083747: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10\n",
            "2020-12-03 16:08:18.083765: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10\n",
            "2020-12-03 16:08:18.083783: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7\n",
            "2020-12-03 16:08:18.083861: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 16:08:18.084156: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 16:08:18.084409: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1767] Adding visible gpu devices: 0\n",
            "2020-12-03 16:08:18.084446: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1180] Device interconnect StreamExecutor with strength 1 edge matrix:\n",
            "2020-12-03 16:08:18.084459: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1186]      0 \n",
            "2020-12-03 16:08:18.084468: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1199] 0:   N \n",
            "2020-12-03 16:08:18.084561: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 16:08:18.084845: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 16:08:18.085091: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1325] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14221 MB memory) -> physical GPU (device: 0, name: Tesla T4, pci bus id: 0000:00:04.0, compute capability: 7.5)\n",
            "INFO:tensorflow:Restoring parameters from /content/pretrained_model/model.ckpt-1263\n",
            "INFO:tensorflow:Running local_init_op.\n",
            "INFO:tensorflow:Done running local_init_op.\n",
            "INFO:tensorflow:accuracy = 0.96875, loss = 0.09217294\n",
            "INFO:tensorflow:accuracy = 0.959375, loss = 0.1720174 (1.116 sec)\n",
            "INFO:tensorflow:accuracy = 0.96875, loss = 0.068158425 (0.600 sec)\n",
            "INFO:tensorflow:accuracy = 0.9609375, loss = 0.21383433 (0.570 sec)\n",
            "INFO:tensorflow:accuracy = 0.96375, loss = 0.09056044 (0.602 sec)\n",
            "INFO:tensorflow:accuracy = 0.96458334, loss = 0.115706846 (0.594 sec)\n",
            "INFO:tensorflow:accuracy = 0.96428573, loss = 0.10424234 (0.586 sec)\n",
            "INFO:tensorflow:accuracy = 0.96796876, loss = 0.026762014 (0.604 sec)\n",
            "INFO:tensorflow:accuracy = 0.96875, loss = 0.06890913 (0.584 sec)\n",
            "INFO:tensorflow:accuracy = 0.96875, loss = 0.098286316 (0.606 sec)\n",
            "INFO:tensorflow:accuracy = 0.9693182, loss = 0.08152 (0.592 sec)\n",
            "INFO:tensorflow:accuracy = 0.97083336, loss = 0.03694243 (0.598 sec)\n",
            "INFO:tensorflow:accuracy = 0.9718593, loss = 0.020572498 (0.431 sec)\n",
            "INFO:tensorflow:Finished evaluation at 2020-12-03-16:08:28\n",
            "INFO:tensorflow:Saving dict for global step 1263: 0 = 0.99265623, 1 = 0.9661885, 2 = 0.98651004, 3 = 0.977124, 4 = 0.94117963, eval_loss = 0.083128154, global_step = 1263, loss = 0.081720665\n",
            "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 1263: /content/pretrained_model/model.ckpt-1263\n",
            "INFO:tensorflow:Loss for final step: 0.033144973.\n",
            "Training took time  0:23:56.839992\n",
            "WARNING:tensorflow:Entity <function file_based_input_fn_builder.<locals>.input_fn.<locals>.<lambda> at 0x7fb100884488> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: module 'gast' has no attribute 'Str'\n",
            "INFO:tensorflow:Calling model_fn.\n",
            "INFO:tensorflow:num_labels:5;logits:Tensor(\"loss/BiasAdd:0\", shape=(?, 5), dtype=float32);labels:Tensor(\"loss/Cast:0\", shape=(?, 5), dtype=float32)\n",
            "INFO:tensorflow:**** Trainable Variables ****\n",
            "INFO:tensorflow:Done calling model_fn.\n",
            "INFO:tensorflow:Starting evaluation at 2020-12-03T16:08:31Z\n",
            "INFO:tensorflow:Graph was finalized.\n",
            "2020-12-03 16:08:32.297619: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 16:08:32.297954: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 0 with properties: \n",
            "name: Tesla T4 major: 7 minor: 5 memoryClockRate(GHz): 1.59\n",
            "pciBusID: 0000:00:04.0\n",
            "2020-12-03 16:08:32.298037: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1\n",
            "2020-12-03 16:08:32.298066: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10\n",
            "2020-12-03 16:08:32.298084: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10\n",
            "2020-12-03 16:08:32.298104: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10\n",
            "2020-12-03 16:08:32.298137: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10\n",
            "2020-12-03 16:08:32.298154: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10\n",
            "2020-12-03 16:08:32.298175: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7\n",
            "2020-12-03 16:08:32.298260: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 16:08:32.298653: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 16:08:32.298922: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1767] Adding visible gpu devices: 0\n",
            "2020-12-03 16:08:32.298963: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1180] Device interconnect StreamExecutor with strength 1 edge matrix:\n",
            "2020-12-03 16:08:32.298976: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1186]      0 \n",
            "2020-12-03 16:08:32.298999: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1199] 0:   N \n",
            "2020-12-03 16:08:32.299092: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 16:08:32.299430: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 16:08:32.299732: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1325] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14221 MB memory) -> physical GPU (device: 0, name: Tesla T4, pci bus id: 0000:00:04.0, compute capability: 7.5)\n",
            "INFO:tensorflow:Restoring parameters from /content/pretrained_model/model.ckpt-1263\n",
            "INFO:tensorflow:Running local_init_op.\n",
            "INFO:tensorflow:Done running local_init_op.\n",
            "INFO:tensorflow:accuracy = 0.96875, loss = 0.09217294\n",
            "INFO:tensorflow:accuracy = 0.959375, loss = 0.1720174 (1.107 sec)\n",
            "INFO:tensorflow:accuracy = 0.96875, loss = 0.068158425 (0.601 sec)\n",
            "INFO:tensorflow:accuracy = 0.9609375, loss = 0.21383433 (0.574 sec)\n",
            "INFO:tensorflow:accuracy = 0.96375, loss = 0.09056044 (0.607 sec)\n",
            "INFO:tensorflow:accuracy = 0.96458334, loss = 0.115706846 (0.592 sec)\n",
            "INFO:tensorflow:accuracy = 0.96428573, loss = 0.10424234 (0.589 sec)\n",
            "INFO:tensorflow:accuracy = 0.96796876, loss = 0.026762014 (0.603 sec)\n",
            "INFO:tensorflow:accuracy = 0.96875, loss = 0.06890913 (0.587 sec)\n",
            "INFO:tensorflow:accuracy = 0.96875, loss = 0.098286316 (0.606 sec)\n",
            "INFO:tensorflow:accuracy = 0.9693182, loss = 0.08152 (0.594 sec)\n",
            "INFO:tensorflow:accuracy = 0.97083336, loss = 0.03694243 (0.604 sec)\n",
            "INFO:tensorflow:accuracy = 0.9718593, loss = 0.020572498 (0.433 sec)\n",
            "INFO:tensorflow:Finished evaluation at 2020-12-03-16:08:42\n",
            "INFO:tensorflow:Saving dict for global step 1263: 0 = 0.99265623, 1 = 0.9661885, 2 = 0.98651004, 3 = 0.977124, 4 = 0.94117963, eval_loss = 0.083128154, global_step = 1263, loss = 0.081720665\n",
            "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 1263: /content/pretrained_model/model.ckpt-1263\n",
            "INFO:tensorflow:***** Eval results *****\n",
            "INFO:tensorflow:  0 = 0.99265623\n",
            "INFO:tensorflow:  1 = 0.9661885\n",
            "INFO:tensorflow:  2 = 0.98651004\n",
            "INFO:tensorflow:  3 = 0.977124\n",
            "INFO:tensorflow:  4 = 0.94117963\n",
            "INFO:tensorflow:  eval_loss = 0.083128154\n",
            "INFO:tensorflow:  global_step = 1263\n",
            "INFO:tensorflow:  loss = 0.081720665\n",
            "INFO:tensorflow:Using config: {'_model_dir': '/content/pretrained_model/', '_tf_random_seed': None, '_save_summary_steps': 500, '_save_checkpoints_steps': 1000, '_save_checkpoints_secs': None, '_session_config': allow_soft_placement: true\n",
            "graph_options {\n",
            "  rewrite_options {\n",
            "    meta_optimizer_iterations: ONE\n",
            "  }\n",
            "}\n",
            ", '_keep_checkpoint_max': 1, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7fafecc88550>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}\n",
            "Beginning Predictions!\n",
            "Prediction took time  0:00:00.000298\n",
            "INFO:tensorflow:Calling model_fn.\n",
            "INFO:tensorflow:num_labels:5;logits:Tensor(\"loss/BiasAdd:0\", shape=(?, 5), dtype=float32);labels:Tensor(\"loss/Cast:0\", shape=(?, 5), dtype=float32)\n",
            "INFO:tensorflow:**** Trainable Variables ****\n",
            "mode: infer probabilities: Tensor(\"loss/Sigmoid:0\", shape=(?, 5), dtype=float32)\n",
            "INFO:tensorflow:Done calling model_fn.\n",
            "INFO:tensorflow:Graph was finalized.\n",
            "2020-12-03 16:08:45.455720: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 16:08:45.456061: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 0 with properties: \n",
            "name: Tesla T4 major: 7 minor: 5 memoryClockRate(GHz): 1.59\n",
            "pciBusID: 0000:00:04.0\n",
            "2020-12-03 16:08:45.456151: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1\n",
            "2020-12-03 16:08:45.456178: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10\n",
            "2020-12-03 16:08:45.456203: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10\n",
            "2020-12-03 16:08:45.456235: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10\n",
            "2020-12-03 16:08:45.456257: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10\n",
            "2020-12-03 16:08:45.456280: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10\n",
            "2020-12-03 16:08:45.456314: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7\n",
            "2020-12-03 16:08:45.456410: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 16:08:45.456717: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 16:08:45.456957: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1767] Adding visible gpu devices: 0\n",
            "2020-12-03 16:08:45.456996: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1180] Device interconnect StreamExecutor with strength 1 edge matrix:\n",
            "2020-12-03 16:08:45.457008: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1186]      0 \n",
            "2020-12-03 16:08:45.457024: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1199] 0:   N \n",
            "2020-12-03 16:08:45.457116: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 16:08:45.457428: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 16:08:45.457681: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1325] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14221 MB memory) -> physical GPU (device: 0, name: Tesla T4, pci bus id: 0000:00:04.0, compute capability: 7.5)\n",
            "INFO:tensorflow:Restoring parameters from /content/pretrained_model/model.ckpt-1263\n",
            "INFO:tensorflow:Running local_init_op.\n",
            "INFO:tensorflow:Done running local_init_op.\n",
            "Pretraining validation (0.5 decision threshold):\n",
            "0 EF\n",
            "\tPrecision : 0.926829268292683\n",
            "\tRecall : 0.9047619047619048\n",
            "\tF-score : 0.9156626506024096\n",
            "1 INF\n",
            "\tPrecision : 0.9148936170212766\n",
            "\tRecall : 0.86\n",
            "\tF-score : 0.8865979381443299\n",
            "2 ADR\n",
            "\tPrecision : 0.9710743801652892\n",
            "\tRecall : 0.9475806451612904\n",
            "\tF-score : 0.9591836734693877\n",
            "3 DI\n",
            "\tPrecision : 0.8444444444444444\n",
            "\tRecall : 0.9193548387096774\n",
            "\tF-score : 0.8803088803088802\n",
            "4 Finding\n",
            "\tPrecision : 0.36363636363636365\n",
            "\tRecall : 0.3333333333333333\n",
            "\tF-score : 0.34782608695652173\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "--FpTpX05PMG"
      },
      "source": [
        "## Training"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "t7FNJdS8wFBK",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "434e0562-d75e-40f9-ba7d-da0e7dc2e9df"
      },
      "source": [
        "!ls /content/pretrained_model/\n",
        "!mkdir /content/classification_results"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "checkpoint\n",
            "eval\n",
            "eval_results.txt\n",
            "eval.tf_record\n",
            "events.out.tfevents.1607010281.51ffb976f9f6\n",
            "graph.pbtxt\n",
            "model.ckpt-1263.data-00000-of-00001\n",
            "model.ckpt-1263.index\n",
            "model.ckpt-1263.meta\n",
            "predicted_labels.csv\n",
            "train.tf_record\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lGLBXQqpRJfG",
        "outputId": "fd08dbb4-8813-4193-8d72-eae994a85dd5"
      },
      "source": [
        "%cd /content\n",
        "!python smm4h_classification/preprocessing/merge_tweets_sets.py \\\n",
        "--input_files smm4h_classification/data/preprocessed/ru/train.tsv smm4h_classification/data/preprocessed/en/train.tsv \\\n",
        "--output_path smm4h_classification/data/preprocessed/ruen/train.tsv\n",
        "!cp smm4h_classification/data/preprocessed/ru/dev.tsv smm4h_classification/data/preprocessed/ruen/dev.tsv\n",
        "!cp smm4h_classification/data/preprocessed/ru/test.tsv smm4h_classification/data/preprocessed/ruen/test.tsv"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content\n",
            "smm4h_classification/data/preprocessed/ru/train.tsv\n",
            "(6090, 2)\n",
            "smm4h_classification/data/preprocessed/en/train.tsv\n",
            "(20544, 2)\n",
            "Result shape (26634, 2)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wNcpkLUIxnb_",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "deb98940-12dd-4a3d-fcde-ade8e4e20563"
      },
      "source": [
        "%cd /content/bert\n",
        "!python run_binary_classification.py \\\n",
        "--task_name=cola \\\n",
        "--do_lower_case=false \\\n",
        "--do_predict=true \\\n",
        "--do_train=true \\\n",
        "--do_eval=true \\\n",
        "--data_dir=/content/smm4h_classification/data/preprocessed/ruen \\\n",
        "--vocab_file=/content/bert_models/multilingual_eng_rus_reviews_finetuned/vocab.txt \\\n",
        "--bert_config_file=/content/bert_models/multilingual_eng_rus_reviews_finetuned/bert_config.json \\\n",
        "--init_checkpoint=/content/pretrained_model/model.ckpt-1263 \\\n",
        "--max_seq_length=128 \\\n",
        "--train_batch_size=32 \\\n",
        "--learning_rate=3e-5 \\\n",
        "--num_train_epochs=5 \\\n",
        "--save_checkpoints_steps=1000 \\\n",
        "--output_dir=/content/classification_results/1"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\u001b[1;30;43mStreaming output truncated to the last 5000 lines.\u001b[0m\n",
            "INFO:tensorflow:global_step/sec: 1.06272\n",
            "I1203 18:22:52.560968 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06272\n",
            "INFO:tensorflow:examples/sec: 34.0071\n",
            "I1203 18:22:52.561199 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0071\n",
            "INFO:tensorflow:global_step/sec: 1.06325\n",
            "I1203 18:22:53.501524 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06325\n",
            "INFO:tensorflow:examples/sec: 34.024\n",
            "I1203 18:22:53.501772 140409631090560 tpu_estimator.py:2308] examples/sec: 34.024\n",
            "INFO:tensorflow:global_step/sec: 1.06642\n",
            "I1203 18:22:54.439234 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06642\n",
            "INFO:tensorflow:examples/sec: 34.1256\n",
            "I1203 18:22:54.439508 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1256\n",
            "INFO:tensorflow:global_step/sec: 1.06753\n",
            "I1203 18:22:55.375972 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06753\n",
            "INFO:tensorflow:examples/sec: 34.1608\n",
            "I1203 18:22:55.376255 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1608\n",
            "INFO:tensorflow:global_step/sec: 1.06505\n",
            "I1203 18:22:56.314895 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06505\n",
            "INFO:tensorflow:examples/sec: 34.0816\n",
            "I1203 18:22:56.315152 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0816\n",
            "INFO:tensorflow:global_step/sec: 1.07231\n",
            "I1203 18:22:57.247441 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07231\n",
            "INFO:tensorflow:examples/sec: 34.314\n",
            "I1203 18:22:57.247808 140409631090560 tpu_estimator.py:2308] examples/sec: 34.314\n",
            "INFO:tensorflow:global_step/sec: 1.06049\n",
            "I1203 18:22:58.190416 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06049\n",
            "INFO:tensorflow:examples/sec: 33.9356\n",
            "I1203 18:22:58.190654 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9356\n",
            "INFO:tensorflow:global_step/sec: 1.06489\n",
            "I1203 18:22:59.129476 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06489\n",
            "INFO:tensorflow:examples/sec: 34.0765\n",
            "I1203 18:22:59.129727 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0765\n",
            "INFO:tensorflow:global_step/sec: 1.07154\n",
            "I1203 18:23:00.062721 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07154\n",
            "INFO:tensorflow:examples/sec: 34.2894\n",
            "I1203 18:23:00.063150 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2894\n",
            "INFO:tensorflow:global_step/sec: 1.06743\n",
            "I1203 18:23:00.999559 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06743\n",
            "INFO:tensorflow:examples/sec: 34.1577\n",
            "I1203 18:23:00.999796 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1577\n",
            "INFO:tensorflow:global_step/sec: 1.0684\n",
            "I1203 18:23:01.935534 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0684\n",
            "INFO:tensorflow:examples/sec: 34.1886\n",
            "I1203 18:23:01.935778 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1886\n",
            "INFO:tensorflow:global_step/sec: 1.06296\n",
            "I1203 18:23:02.876279 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06296\n",
            "INFO:tensorflow:examples/sec: 34.0147\n",
            "I1203 18:23:02.876687 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0147\n",
            "INFO:tensorflow:global_step/sec: 1.06172\n",
            "I1203 18:23:03.818202 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06172\n",
            "INFO:tensorflow:examples/sec: 33.9751\n",
            "I1203 18:23:03.818646 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9751\n",
            "INFO:tensorflow:global_step/sec: 1.06051\n",
            "I1203 18:23:04.761099 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06051\n",
            "INFO:tensorflow:examples/sec: 33.9365\n",
            "I1203 18:23:04.761343 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9365\n",
            "INFO:tensorflow:global_step/sec: 1.06648\n",
            "I1203 18:23:05.698748 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06648\n",
            "INFO:tensorflow:examples/sec: 34.1274\n",
            "I1203 18:23:05.698971 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1274\n",
            "INFO:tensorflow:global_step/sec: 1.06992\n",
            "I1203 18:23:06.633433 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06992\n",
            "INFO:tensorflow:examples/sec: 34.2375\n",
            "I1203 18:23:06.633675 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2375\n",
            "INFO:tensorflow:global_step/sec: 1.06724\n",
            "I1203 18:23:07.570415 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06724\n",
            "INFO:tensorflow:examples/sec: 34.1515\n",
            "I1203 18:23:07.570656 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1515\n",
            "INFO:tensorflow:global_step/sec: 1.06646\n",
            "I1203 18:23:08.508091 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06646\n",
            "INFO:tensorflow:examples/sec: 34.1269\n",
            "I1203 18:23:08.508324 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1269\n",
            "INFO:tensorflow:global_step/sec: 1.06492\n",
            "I1203 18:23:09.447210 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06492\n",
            "INFO:tensorflow:examples/sec: 34.0773\n",
            "I1203 18:23:09.447560 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0773\n",
            "INFO:tensorflow:global_step/sec: 1.06488\n",
            "I1203 18:23:10.386265 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06488\n",
            "INFO:tensorflow:examples/sec: 34.0762\n",
            "I1203 18:23:10.386597 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0762\n",
            "INFO:tensorflow:global_step/sec: 1.06571\n",
            "I1203 18:23:11.324641 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06571\n",
            "INFO:tensorflow:examples/sec: 34.1027\n",
            "I1203 18:23:11.324916 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1027\n",
            "INFO:tensorflow:global_step/sec: 1.0621\n",
            "I1203 18:23:12.266149 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0621\n",
            "INFO:tensorflow:examples/sec: 33.9871\n",
            "I1203 18:23:12.266555 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9871\n",
            "INFO:tensorflow:global_step/sec: 1.06447\n",
            "I1203 18:23:13.205627 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06447\n",
            "INFO:tensorflow:examples/sec: 34.0631\n",
            "I1203 18:23:13.205976 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0631\n",
            "INFO:tensorflow:global_step/sec: 1.0659\n",
            "I1203 18:23:14.143724 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0659\n",
            "INFO:tensorflow:examples/sec: 34.1089\n",
            "I1203 18:23:14.144054 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1089\n",
            "INFO:tensorflow:global_step/sec: 1.0663\n",
            "I1203 18:23:15.081519 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0663\n",
            "INFO:tensorflow:examples/sec: 34.1216\n",
            "I1203 18:23:15.081979 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1216\n",
            "INFO:tensorflow:global_step/sec: 1.06962\n",
            "I1203 18:23:16.016421 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06962\n",
            "INFO:tensorflow:examples/sec: 34.2277\n",
            "I1203 18:23:16.016681 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2277\n",
            "INFO:tensorflow:global_step/sec: 1.06244\n",
            "I1203 18:23:16.957671 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06244\n",
            "INFO:tensorflow:examples/sec: 33.998\n",
            "I1203 18:23:16.958063 140409631090560 tpu_estimator.py:2308] examples/sec: 33.998\n",
            "INFO:tensorflow:global_step/sec: 1.06165\n",
            "I1203 18:23:17.899610 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06165\n",
            "INFO:tensorflow:examples/sec: 33.9728\n",
            "I1203 18:23:17.899861 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9728\n",
            "INFO:tensorflow:global_step/sec: 1.06298\n",
            "I1203 18:23:18.840309 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06298\n",
            "INFO:tensorflow:examples/sec: 34.0155\n",
            "I1203 18:23:18.840576 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0155\n",
            "INFO:tensorflow:global_step/sec: 1.05991\n",
            "I1203 18:23:19.783807 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05991\n",
            "INFO:tensorflow:examples/sec: 33.9173\n",
            "I1203 18:23:19.784065 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9173\n",
            "INFO:tensorflow:global_step/sec: 1.06924\n",
            "I1203 18:23:20.718988 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06924\n",
            "INFO:tensorflow:examples/sec: 34.2156\n",
            "I1203 18:23:20.719258 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2156\n",
            "INFO:tensorflow:global_step/sec: 1.06594\n",
            "I1203 18:23:21.657236 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06594\n",
            "INFO:tensorflow:examples/sec: 34.1101\n",
            "I1203 18:23:21.657532 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1101\n",
            "INFO:tensorflow:global_step/sec: 1.05867\n",
            "I1203 18:23:22.601755 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05867\n",
            "INFO:tensorflow:examples/sec: 33.8776\n",
            "I1203 18:23:22.602008 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8776\n",
            "INFO:tensorflow:global_step/sec: 1.05906\n",
            "I1203 18:23:23.545983 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05906\n",
            "INFO:tensorflow:examples/sec: 33.8901\n",
            "I1203 18:23:23.546236 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8901\n",
            "INFO:tensorflow:global_step/sec: 1.06382\n",
            "I1203 18:23:24.485989 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06382\n",
            "INFO:tensorflow:examples/sec: 34.0422\n",
            "I1203 18:23:24.486253 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0422\n",
            "INFO:tensorflow:global_step/sec: 1.06574\n",
            "I1203 18:23:25.424315 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06574\n",
            "INFO:tensorflow:examples/sec: 34.1038\n",
            "I1203 18:23:25.424747 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1038\n",
            "INFO:tensorflow:global_step/sec: 1.05913\n",
            "I1203 18:23:26.368525 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05913\n",
            "INFO:tensorflow:examples/sec: 33.892\n",
            "I1203 18:23:26.368788 140409631090560 tpu_estimator.py:2308] examples/sec: 33.892\n",
            "INFO:tensorflow:global_step/sec: 1.06279\n",
            "I1203 18:23:27.309421 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06279\n",
            "INFO:tensorflow:examples/sec: 34.0093\n",
            "I1203 18:23:27.309675 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0093\n",
            "INFO:tensorflow:global_step/sec: 1.06645\n",
            "I1203 18:23:28.247091 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06645\n",
            "INFO:tensorflow:examples/sec: 34.1265\n",
            "I1203 18:23:28.247314 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1265\n",
            "INFO:tensorflow:global_step/sec: 1.0637\n",
            "I1203 18:23:29.187211 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0637\n",
            "INFO:tensorflow:examples/sec: 34.0385\n",
            "I1203 18:23:29.187473 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0385\n",
            "INFO:tensorflow:global_step/sec: 1.05699\n",
            "I1203 18:23:30.133277 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05699\n",
            "INFO:tensorflow:examples/sec: 33.8237\n",
            "I1203 18:23:30.133537 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8237\n",
            "INFO:tensorflow:global_step/sec: 1.05885\n",
            "I1203 18:23:31.077713 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05885\n",
            "INFO:tensorflow:examples/sec: 33.8833\n",
            "I1203 18:23:31.077948 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8833\n",
            "INFO:tensorflow:global_step/sec: 1.06744\n",
            "I1203 18:23:32.014564 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06744\n",
            "INFO:tensorflow:examples/sec: 34.158\n",
            "I1203 18:23:32.014805 140409631090560 tpu_estimator.py:2308] examples/sec: 34.158\n",
            "INFO:tensorflow:global_step/sec: 1.06112\n",
            "I1203 18:23:32.956914 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06112\n",
            "INFO:tensorflow:examples/sec: 33.9557\n",
            "I1203 18:23:32.957394 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9557\n",
            "INFO:tensorflow:global_step/sec: 1.0605\n",
            "I1203 18:23:33.899895 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0605\n",
            "INFO:tensorflow:examples/sec: 33.936\n",
            "I1203 18:23:33.900297 140409631090560 tpu_estimator.py:2308] examples/sec: 33.936\n",
            "INFO:tensorflow:global_step/sec: 1.0654\n",
            "I1203 18:23:34.838501 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0654\n",
            "INFO:tensorflow:examples/sec: 34.0929\n",
            "I1203 18:23:34.838907 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0929\n",
            "INFO:tensorflow:global_step/sec: 1.05551\n",
            "I1203 18:23:35.785929 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05551\n",
            "INFO:tensorflow:examples/sec: 33.7764\n",
            "I1203 18:23:35.786329 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7764\n",
            "INFO:tensorflow:global_step/sec: 1.06419\n",
            "I1203 18:23:36.725608 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06419\n",
            "INFO:tensorflow:examples/sec: 34.0542\n",
            "I1203 18:23:36.725980 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0542\n",
            "INFO:tensorflow:global_step/sec: 1.06393\n",
            "I1203 18:23:37.665503 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06393\n",
            "INFO:tensorflow:examples/sec: 34.0459\n",
            "I1203 18:23:37.665874 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0459\n",
            "INFO:tensorflow:global_step/sec: 1.06226\n",
            "I1203 18:23:38.606859 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06226\n",
            "INFO:tensorflow:examples/sec: 33.9924\n",
            "I1203 18:23:38.607234 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9924\n",
            "INFO:tensorflow:global_step/sec: 1.06484\n",
            "I1203 18:23:39.545963 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06484\n",
            "INFO:tensorflow:examples/sec: 34.075\n",
            "I1203 18:23:39.546205 140409631090560 tpu_estimator.py:2308] examples/sec: 34.075\n",
            "INFO:tensorflow:global_step/sec: 1.06355\n",
            "I1203 18:23:40.486225 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06355\n",
            "INFO:tensorflow:examples/sec: 34.0337\n",
            "I1203 18:23:40.486595 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0337\n",
            "INFO:tensorflow:global_step/sec: 1.04838\n",
            "I1203 18:23:41.440074 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.04838\n",
            "INFO:tensorflow:examples/sec: 33.5481\n",
            "I1203 18:23:41.440461 140409631090560 tpu_estimator.py:2308] examples/sec: 33.5481\n",
            "INFO:tensorflow:global_step/sec: 1.05935\n",
            "I1203 18:23:42.384045 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05935\n",
            "INFO:tensorflow:examples/sec: 33.8992\n",
            "I1203 18:23:42.384454 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8992\n",
            "INFO:tensorflow:global_step/sec: 1.06116\n",
            "I1203 18:23:43.326428 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06116\n",
            "INFO:tensorflow:examples/sec: 33.9572\n",
            "I1203 18:23:43.326675 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9572\n",
            "INFO:tensorflow:global_step/sec: 1.06404\n",
            "I1203 18:23:44.266229 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06404\n",
            "INFO:tensorflow:examples/sec: 34.0493\n",
            "I1203 18:23:44.266634 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0493\n",
            "INFO:tensorflow:global_step/sec: 1.06047\n",
            "I1203 18:23:45.209224 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06047\n",
            "INFO:tensorflow:examples/sec: 33.935\n",
            "I1203 18:23:45.209620 140409631090560 tpu_estimator.py:2308] examples/sec: 33.935\n",
            "INFO:tensorflow:global_step/sec: 1.06649\n",
            "I1203 18:23:46.146863 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06649\n",
            "INFO:tensorflow:examples/sec: 34.1276\n",
            "I1203 18:23:46.147138 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1276\n",
            "INFO:tensorflow:global_step/sec: 1.06162\n",
            "I1203 18:23:47.088828 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06162\n",
            "INFO:tensorflow:examples/sec: 33.9719\n",
            "I1203 18:23:47.089229 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9719\n",
            "INFO:tensorflow:global_step/sec: 1.06008\n",
            "I1203 18:23:48.032187 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06008\n",
            "INFO:tensorflow:examples/sec: 33.9224\n",
            "I1203 18:23:48.032564 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9224\n",
            "INFO:tensorflow:global_step/sec: 1.06382\n",
            "I1203 18:23:48.972166 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06382\n",
            "INFO:tensorflow:examples/sec: 34.0423\n",
            "I1203 18:23:48.972570 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0423\n",
            "INFO:tensorflow:global_step/sec: 1.05797\n",
            "I1203 18:23:49.917362 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05797\n",
            "INFO:tensorflow:examples/sec: 33.855\n",
            "I1203 18:23:49.917759 140409631090560 tpu_estimator.py:2308] examples/sec: 33.855\n",
            "INFO:tensorflow:global_step/sec: 1.05965\n",
            "I1203 18:23:50.861079 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05965\n",
            "INFO:tensorflow:examples/sec: 33.9088\n",
            "I1203 18:23:50.861487 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9088\n",
            "INFO:tensorflow:global_step/sec: 1.05633\n",
            "I1203 18:23:51.807737 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05633\n",
            "INFO:tensorflow:examples/sec: 33.8027\n",
            "I1203 18:23:51.808124 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8027\n",
            "INFO:tensorflow:global_step/sec: 1.05657\n",
            "I1203 18:23:52.754224 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05657\n",
            "INFO:tensorflow:examples/sec: 33.8103\n",
            "I1203 18:23:52.754698 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8103\n",
            "INFO:tensorflow:global_step/sec: 1.05983\n",
            "I1203 18:23:53.697749 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05983\n",
            "INFO:tensorflow:examples/sec: 33.9146\n",
            "I1203 18:23:53.698006 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9146\n",
            "INFO:tensorflow:global_step/sec: 1.06382\n",
            "I1203 18:23:54.637754 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06382\n",
            "INFO:tensorflow:examples/sec: 34.0423\n",
            "I1203 18:23:54.638006 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0423\n",
            "INFO:tensorflow:global_step/sec: 1.057\n",
            "I1203 18:23:55.583830 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.057\n",
            "INFO:tensorflow:examples/sec: 33.8239\n",
            "I1203 18:23:55.584078 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8239\n",
            "INFO:tensorflow:global_step/sec: 1.05702\n",
            "I1203 18:23:56.529906 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05702\n",
            "INFO:tensorflow:examples/sec: 33.8248\n",
            "I1203 18:23:56.530181 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8248\n",
            "INFO:tensorflow:global_step/sec: 1.06049\n",
            "I1203 18:23:57.472851 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06049\n",
            "INFO:tensorflow:examples/sec: 33.9356\n",
            "I1203 18:23:57.473086 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9356\n",
            "INFO:tensorflow:global_step/sec: 1.06786\n",
            "I1203 18:23:58.409311 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06786\n",
            "INFO:tensorflow:examples/sec: 34.1716\n",
            "I1203 18:23:58.409584 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1716\n",
            "INFO:tensorflow:global_step/sec: 1.04952\n",
            "I1203 18:23:59.362125 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.04952\n",
            "INFO:tensorflow:examples/sec: 33.5845\n",
            "I1203 18:23:59.362636 140409631090560 tpu_estimator.py:2308] examples/sec: 33.5845\n",
            "INFO:tensorflow:global_step/sec: 1.0588\n",
            "I1203 18:24:00.306629 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0588\n",
            "INFO:tensorflow:examples/sec: 33.8815\n",
            "I1203 18:24:00.307011 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8815\n",
            "INFO:tensorflow:global_step/sec: 1.05805\n",
            "I1203 18:24:01.251715 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05805\n",
            "INFO:tensorflow:examples/sec: 33.8576\n",
            "I1203 18:24:01.252089 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8576\n",
            "INFO:tensorflow:global_step/sec: 1.05668\n",
            "I1203 18:24:02.198091 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05668\n",
            "INFO:tensorflow:examples/sec: 33.8137\n",
            "I1203 18:24:02.198455 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8137\n",
            "INFO:tensorflow:global_step/sec: 1.06108\n",
            "I1203 18:24:03.140548 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06108\n",
            "INFO:tensorflow:examples/sec: 33.9547\n",
            "I1203 18:24:03.140948 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9547\n",
            "INFO:tensorflow:global_step/sec: 1.06307\n",
            "I1203 18:24:04.081171 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06307\n",
            "INFO:tensorflow:examples/sec: 34.0184\n",
            "I1203 18:24:04.081544 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0184\n",
            "INFO:tensorflow:global_step/sec: 1.05202\n",
            "I1203 18:24:05.031748 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05202\n",
            "INFO:tensorflow:examples/sec: 33.6647\n",
            "I1203 18:24:05.032142 140409631090560 tpu_estimator.py:2308] examples/sec: 33.6647\n",
            "INFO:tensorflow:global_step/sec: 1.05582\n",
            "I1203 18:24:05.978884 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05582\n",
            "INFO:tensorflow:examples/sec: 33.7864\n",
            "I1203 18:24:05.979177 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7864\n",
            "INFO:tensorflow:global_step/sec: 1.06372\n",
            "I1203 18:24:06.918958 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06372\n",
            "INFO:tensorflow:examples/sec: 34.039\n",
            "I1203 18:24:06.919201 140409631090560 tpu_estimator.py:2308] examples/sec: 34.039\n",
            "INFO:tensorflow:global_step/sec: 1.06181\n",
            "I1203 18:24:07.860734 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06181\n",
            "INFO:tensorflow:examples/sec: 33.9778\n",
            "I1203 18:24:07.860967 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9778\n",
            "INFO:tensorflow:global_step/sec: 1.05868\n",
            "I1203 18:24:08.805377 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05868\n",
            "INFO:tensorflow:examples/sec: 33.8777\n",
            "I1203 18:24:08.805664 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8777\n",
            "INFO:tensorflow:global_step/sec: 1.06702\n",
            "I1203 18:24:09.742535 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06702\n",
            "INFO:tensorflow:examples/sec: 34.1448\n",
            "I1203 18:24:09.742901 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1448\n",
            "INFO:tensorflow:global_step/sec: 1.06279\n",
            "I1203 18:24:10.683458 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06279\n",
            "INFO:tensorflow:examples/sec: 34.0092\n",
            "I1203 18:24:10.683828 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0092\n",
            "INFO:tensorflow:global_step/sec: 1.05814\n",
            "I1203 18:24:11.628506 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05814\n",
            "INFO:tensorflow:examples/sec: 33.8606\n",
            "I1203 18:24:11.628884 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8606\n",
            "INFO:tensorflow:global_step/sec: 1.05845\n",
            "I1203 18:24:12.573249 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05845\n",
            "INFO:tensorflow:examples/sec: 33.8705\n",
            "I1203 18:24:12.573620 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8705\n",
            "INFO:tensorflow:global_step/sec: 1.05898\n",
            "I1203 18:24:13.517601 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05898\n",
            "INFO:tensorflow:examples/sec: 33.8875\n",
            "I1203 18:24:13.517972 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8875\n",
            "INFO:tensorflow:global_step/sec: 1.06055\n",
            "I1203 18:24:14.460552 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06055\n",
            "INFO:tensorflow:examples/sec: 33.9377\n",
            "I1203 18:24:14.460937 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9377\n",
            "INFO:tensorflow:global_step/sec: 1.06525\n",
            "I1203 18:24:15.399232 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06525\n",
            "INFO:tensorflow:examples/sec: 34.0881\n",
            "I1203 18:24:15.399629 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0881\n",
            "INFO:tensorflow:global_step/sec: 1.05611\n",
            "I1203 18:24:16.346096 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05611\n",
            "INFO:tensorflow:examples/sec: 33.7956\n",
            "I1203 18:24:16.346469 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7956\n",
            "INFO:tensorflow:global_step/sec: 1.06614\n",
            "I1203 18:24:17.284032 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06614\n",
            "INFO:tensorflow:examples/sec: 34.1163\n",
            "I1203 18:24:17.284410 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1163\n",
            "INFO:tensorflow:global_step/sec: 1.06153\n",
            "I1203 18:24:18.226116 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06153\n",
            "INFO:tensorflow:examples/sec: 33.969\n",
            "I1203 18:24:18.226532 140409631090560 tpu_estimator.py:2308] examples/sec: 33.969\n",
            "INFO:tensorflow:global_step/sec: 1.06539\n",
            "I1203 18:24:19.164695 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06539\n",
            "INFO:tensorflow:examples/sec: 34.0926\n",
            "I1203 18:24:19.165067 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0926\n",
            "INFO:tensorflow:global_step/sec: 1.06174\n",
            "I1203 18:24:20.106584 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06174\n",
            "INFO:tensorflow:examples/sec: 33.9757\n",
            "I1203 18:24:20.106976 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9757\n",
            "INFO:tensorflow:global_step/sec: 1.06382\n",
            "I1203 18:24:21.046580 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06382\n",
            "INFO:tensorflow:examples/sec: 34.0421\n",
            "I1203 18:24:21.046953 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0421\n",
            "INFO:tensorflow:global_step/sec: 1.06207\n",
            "I1203 18:24:21.988148 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06207\n",
            "INFO:tensorflow:examples/sec: 33.9863\n",
            "I1203 18:24:21.988558 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9863\n",
            "INFO:tensorflow:global_step/sec: 1.06408\n",
            "I1203 18:24:22.927891 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06408\n",
            "INFO:tensorflow:examples/sec: 34.0505\n",
            "I1203 18:24:22.928152 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0505\n",
            "INFO:tensorflow:global_step/sec: 1.0626\n",
            "I1203 18:24:23.868984 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0626\n",
            "INFO:tensorflow:examples/sec: 34.0034\n",
            "I1203 18:24:23.869356 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0034\n",
            "INFO:tensorflow:global_step/sec: 1.06389\n",
            "I1203 18:24:24.809270 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06389\n",
            "INFO:tensorflow:examples/sec: 34.0444\n",
            "I1203 18:24:24.809675 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0444\n",
            "INFO:tensorflow:global_step/sec: 1.06024\n",
            "I1203 18:24:25.752120 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06024\n",
            "INFO:tensorflow:examples/sec: 33.9275\n",
            "I1203 18:24:25.752507 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9275\n",
            "INFO:tensorflow:global_step/sec: 1.06343\n",
            "I1203 18:24:26.692465 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06343\n",
            "INFO:tensorflow:examples/sec: 34.0299\n",
            "I1203 18:24:26.692837 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0299\n",
            "INFO:tensorflow:global_step/sec: 1.06079\n",
            "I1203 18:24:27.635154 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06079\n",
            "INFO:tensorflow:examples/sec: 33.9453\n",
            "I1203 18:24:27.635563 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9453\n",
            "INFO:tensorflow:global_step/sec: 1.06372\n",
            "I1203 18:24:28.575254 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06372\n",
            "INFO:tensorflow:examples/sec: 34.039\n",
            "I1203 18:24:28.575657 140409631090560 tpu_estimator.py:2308] examples/sec: 34.039\n",
            "INFO:tensorflow:global_step/sec: 1.05856\n",
            "I1203 18:24:29.519940 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05856\n",
            "INFO:tensorflow:examples/sec: 33.874\n",
            "I1203 18:24:29.520343 140409631090560 tpu_estimator.py:2308] examples/sec: 33.874\n",
            "INFO:tensorflow:global_step/sec: 1.0613\n",
            "I1203 18:24:30.462185 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0613\n",
            "INFO:tensorflow:examples/sec: 33.9617\n",
            "I1203 18:24:30.462593 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9617\n",
            "INFO:tensorflow:global_step/sec: 1.06524\n",
            "I1203 18:24:31.400915 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06524\n",
            "INFO:tensorflow:examples/sec: 34.0877\n",
            "I1203 18:24:31.401266 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0877\n",
            "INFO:tensorflow:global_step/sec: 1.06369\n",
            "I1203 18:24:32.341057 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06369\n",
            "INFO:tensorflow:examples/sec: 34.0381\n",
            "I1203 18:24:32.341449 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0381\n",
            "INFO:tensorflow:global_step/sec: 1.06091\n",
            "I1203 18:24:33.283630 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06091\n",
            "INFO:tensorflow:examples/sec: 33.9492\n",
            "I1203 18:24:33.283992 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9492\n",
            "INFO:tensorflow:global_step/sec: 1.06702\n",
            "I1203 18:24:34.220827 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06702\n",
            "INFO:tensorflow:examples/sec: 34.1445\n",
            "I1203 18:24:34.221374 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1445\n",
            "INFO:tensorflow:global_step/sec: 1.06456\n",
            "I1203 18:24:35.160175 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06456\n",
            "INFO:tensorflow:examples/sec: 34.0661\n",
            "I1203 18:24:35.160540 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0661\n",
            "INFO:tensorflow:global_step/sec: 1.07048\n",
            "I1203 18:24:36.094345 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07048\n",
            "INFO:tensorflow:examples/sec: 34.2552\n",
            "I1203 18:24:36.094604 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2552\n",
            "INFO:tensorflow:global_step/sec: 1.06326\n",
            "I1203 18:24:37.034839 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06326\n",
            "INFO:tensorflow:examples/sec: 34.0243\n",
            "I1203 18:24:37.035241 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0243\n",
            "INFO:tensorflow:global_step/sec: 1.05996\n",
            "I1203 18:24:37.978295 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05996\n",
            "INFO:tensorflow:examples/sec: 33.9188\n",
            "I1203 18:24:37.978677 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9188\n",
            "INFO:tensorflow:global_step/sec: 1.06178\n",
            "I1203 18:24:38.920110 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06178\n",
            "INFO:tensorflow:examples/sec: 33.9771\n",
            "I1203 18:24:38.920361 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9771\n",
            "INFO:tensorflow:global_step/sec: 1.05977\n",
            "I1203 18:24:39.863683 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05977\n",
            "INFO:tensorflow:examples/sec: 33.9126\n",
            "I1203 18:24:39.864074 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9126\n",
            "INFO:tensorflow:global_step/sec: 1.06539\n",
            "I1203 18:24:40.802302 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06539\n",
            "INFO:tensorflow:examples/sec: 34.0926\n",
            "I1203 18:24:40.802739 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0926\n",
            "INFO:tensorflow:global_step/sec: 1.06116\n",
            "I1203 18:24:41.744684 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06116\n",
            "INFO:tensorflow:examples/sec: 33.9572\n",
            "I1203 18:24:41.744915 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9572\n",
            "INFO:tensorflow:global_step/sec: 1.06842\n",
            "I1203 18:24:42.680657 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06842\n",
            "INFO:tensorflow:examples/sec: 34.1894\n",
            "I1203 18:24:42.681077 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1894\n",
            "INFO:tensorflow:global_step/sec: 1.06582\n",
            "I1203 18:24:43.618882 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06582\n",
            "INFO:tensorflow:examples/sec: 34.1063\n",
            "I1203 18:24:43.619136 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1063\n",
            "INFO:tensorflow:global_step/sec: 1.06234\n",
            "I1203 18:24:44.560190 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06234\n",
            "INFO:tensorflow:examples/sec: 33.9949\n",
            "I1203 18:24:44.560658 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9949\n",
            "INFO:tensorflow:global_step/sec: 1.06289\n",
            "I1203 18:24:45.501034 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06289\n",
            "INFO:tensorflow:examples/sec: 34.0126\n",
            "I1203 18:24:45.501459 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0126\n",
            "INFO:tensorflow:global_step/sec: 1.05957\n",
            "I1203 18:24:46.444801 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05957\n",
            "INFO:tensorflow:examples/sec: 33.9063\n",
            "I1203 18:24:46.445229 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9063\n",
            "INFO:tensorflow:global_step/sec: 1.06191\n",
            "I1203 18:24:47.386543 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06191\n",
            "INFO:tensorflow:examples/sec: 33.9811\n",
            "I1203 18:24:47.386930 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9811\n",
            "INFO:tensorflow:global_step/sec: 1.05879\n",
            "I1203 18:24:48.330994 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05879\n",
            "INFO:tensorflow:examples/sec: 33.8812\n",
            "I1203 18:24:48.331229 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8812\n",
            "INFO:tensorflow:global_step/sec: 1.06449\n",
            "I1203 18:24:49.270435 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06449\n",
            "INFO:tensorflow:examples/sec: 34.0638\n",
            "I1203 18:24:49.270829 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0638\n",
            "INFO:tensorflow:global_step/sec: 1.06412\n",
            "I1203 18:24:50.210165 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06412\n",
            "INFO:tensorflow:examples/sec: 34.0518\n",
            "I1203 18:24:50.210561 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0518\n",
            "INFO:tensorflow:global_step/sec: 1.06393\n",
            "I1203 18:24:51.150045 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06393\n",
            "INFO:tensorflow:examples/sec: 34.0458\n",
            "I1203 18:24:51.150423 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0458\n",
            "INFO:tensorflow:global_step/sec: 1.06983\n",
            "I1203 18:24:52.084774 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06983\n",
            "INFO:tensorflow:examples/sec: 34.2344\n",
            "I1203 18:24:52.085161 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2344\n",
            "INFO:tensorflow:global_step/sec: 1.06463\n",
            "I1203 18:24:53.024095 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06463\n",
            "INFO:tensorflow:examples/sec: 34.0683\n",
            "I1203 18:24:53.024493 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0683\n",
            "INFO:tensorflow:global_step/sec: 1.063\n",
            "I1203 18:24:53.964753 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.063\n",
            "INFO:tensorflow:examples/sec: 34.016\n",
            "I1203 18:24:53.965101 140409631090560 tpu_estimator.py:2308] examples/sec: 34.016\n",
            "INFO:tensorflow:global_step/sec: 1.06119\n",
            "I1203 18:24:54.907149 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06119\n",
            "INFO:tensorflow:examples/sec: 33.9581\n",
            "I1203 18:24:54.907674 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9581\n",
            "INFO:tensorflow:global_step/sec: 1.06587\n",
            "I1203 18:24:55.845361 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06587\n",
            "INFO:tensorflow:examples/sec: 34.1078\n",
            "I1203 18:24:55.845627 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1078\n",
            "INFO:tensorflow:global_step/sec: 1.06388\n",
            "I1203 18:24:56.785321 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06388\n",
            "INFO:tensorflow:examples/sec: 34.0441\n",
            "I1203 18:24:56.785723 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0441\n",
            "INFO:tensorflow:global_step/sec: 1.06787\n",
            "I1203 18:24:57.721746 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06787\n",
            "INFO:tensorflow:examples/sec: 34.172\n",
            "I1203 18:24:57.722182 140409631090560 tpu_estimator.py:2308] examples/sec: 34.172\n",
            "INFO:tensorflow:global_step/sec: 1.06586\n",
            "I1203 18:24:58.659961 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06586\n",
            "INFO:tensorflow:examples/sec: 34.1075\n",
            "I1203 18:24:58.660378 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1075\n",
            "INFO:tensorflow:global_step/sec: 1.06787\n",
            "I1203 18:24:59.596461 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06787\n",
            "INFO:tensorflow:examples/sec: 34.1719\n",
            "I1203 18:24:59.596692 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1719\n",
            "INFO:tensorflow:global_step/sec: 1.06254\n",
            "I1203 18:25:00.537573 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06254\n",
            "INFO:tensorflow:examples/sec: 34.0014\n",
            "I1203 18:25:00.537814 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0014\n",
            "INFO:tensorflow:global_step/sec: 1.06464\n",
            "I1203 18:25:01.476831 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06464\n",
            "INFO:tensorflow:examples/sec: 34.0686\n",
            "I1203 18:25:01.477250 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0686\n",
            "INFO:tensorflow:global_step/sec: 1.06471\n",
            "I1203 18:25:02.416064 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06471\n",
            "INFO:tensorflow:examples/sec: 34.0707\n",
            "I1203 18:25:02.416324 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0707\n",
            "INFO:tensorflow:global_step/sec: 1.06549\n",
            "I1203 18:25:03.354636 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06549\n",
            "INFO:tensorflow:examples/sec: 34.0955\n",
            "I1203 18:25:03.355058 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0955\n",
            "INFO:tensorflow:global_step/sec: 1.06234\n",
            "I1203 18:25:04.295894 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06234\n",
            "INFO:tensorflow:examples/sec: 33.9948\n",
            "I1203 18:25:04.296267 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9948\n",
            "INFO:tensorflow:global_step/sec: 1.06953\n",
            "I1203 18:25:05.230890 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06953\n",
            "INFO:tensorflow:examples/sec: 34.2249\n",
            "I1203 18:25:05.231192 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2249\n",
            "INFO:tensorflow:global_step/sec: 1.06059\n",
            "I1203 18:25:06.173760 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06059\n",
            "INFO:tensorflow:examples/sec: 33.9388\n",
            "I1203 18:25:06.174130 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9388\n",
            "INFO:tensorflow:global_step/sec: 1.06353\n",
            "I1203 18:25:07.114051 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06353\n",
            "INFO:tensorflow:examples/sec: 34.033\n",
            "I1203 18:25:07.114674 140409631090560 tpu_estimator.py:2308] examples/sec: 34.033\n",
            "INFO:tensorflow:global_step/sec: 1.06144\n",
            "I1203 18:25:08.056159 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06144\n",
            "INFO:tensorflow:examples/sec: 33.966\n",
            "I1203 18:25:08.056542 140409631090560 tpu_estimator.py:2308] examples/sec: 33.966\n",
            "INFO:tensorflow:global_step/sec: 1.06677\n",
            "I1203 18:25:08.993592 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06677\n",
            "INFO:tensorflow:examples/sec: 34.1365\n",
            "I1203 18:25:08.993967 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1365\n",
            "INFO:tensorflow:global_step/sec: 1.06407\n",
            "I1203 18:25:09.933345 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06407\n",
            "INFO:tensorflow:examples/sec: 34.0502\n",
            "I1203 18:25:09.933717 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0502\n",
            "INFO:tensorflow:global_step/sec: 1.06374\n",
            "I1203 18:25:10.873445 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06374\n",
            "INFO:tensorflow:examples/sec: 34.0398\n",
            "I1203 18:25:10.873711 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0398\n",
            "INFO:tensorflow:global_step/sec: 1.06378\n",
            "I1203 18:25:11.813510 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06378\n",
            "INFO:tensorflow:examples/sec: 34.0409\n",
            "I1203 18:25:11.813770 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0409\n",
            "INFO:tensorflow:global_step/sec: 1.05778\n",
            "I1203 18:25:12.758864 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05778\n",
            "INFO:tensorflow:examples/sec: 33.8491\n",
            "I1203 18:25:12.759142 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8491\n",
            "INFO:tensorflow:global_step/sec: 1.06513\n",
            "I1203 18:25:13.697705 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06513\n",
            "INFO:tensorflow:examples/sec: 34.0843\n",
            "I1203 18:25:13.698067 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0843\n",
            "INFO:tensorflow:global_step/sec: 1.06418\n",
            "I1203 18:25:14.637374 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06418\n",
            "INFO:tensorflow:examples/sec: 34.0539\n",
            "I1203 18:25:14.637758 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0539\n",
            "INFO:tensorflow:global_step/sec: 1.06195\n",
            "I1203 18:25:15.579076 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06195\n",
            "INFO:tensorflow:examples/sec: 33.9825\n",
            "I1203 18:25:15.579555 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9825\n",
            "INFO:tensorflow:global_step/sec: 1.0578\n",
            "I1203 18:25:16.524411 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0578\n",
            "INFO:tensorflow:examples/sec: 33.8497\n",
            "I1203 18:25:16.524789 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8497\n",
            "INFO:tensorflow:global_step/sec: 1.06641\n",
            "I1203 18:25:17.462126 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06641\n",
            "INFO:tensorflow:examples/sec: 34.1252\n",
            "I1203 18:25:17.462523 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1252\n",
            "INFO:tensorflow:global_step/sec: 1.0622\n",
            "I1203 18:25:18.403615 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0622\n",
            "INFO:tensorflow:examples/sec: 33.9905\n",
            "I1203 18:25:18.404013 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9905\n",
            "INFO:tensorflow:global_step/sec: 1.06213\n",
            "I1203 18:25:19.345069 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06213\n",
            "INFO:tensorflow:examples/sec: 33.9883\n",
            "I1203 18:25:19.345474 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9883\n",
            "INFO:tensorflow:global_step/sec: 1.06085\n",
            "I1203 18:25:20.287714 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06085\n",
            "INFO:tensorflow:examples/sec: 33.9473\n",
            "I1203 18:25:20.288074 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9473\n",
            "INFO:tensorflow:global_step/sec: 1.06105\n",
            "I1203 18:25:21.230167 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06105\n",
            "INFO:tensorflow:examples/sec: 33.9536\n",
            "I1203 18:25:21.230551 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9536\n",
            "INFO:tensorflow:global_step/sec: 1.06246\n",
            "I1203 18:25:22.171363 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06246\n",
            "INFO:tensorflow:examples/sec: 33.9988\n",
            "I1203 18:25:22.171621 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9988\n",
            "INFO:tensorflow:global_step/sec: 1.06632\n",
            "I1203 18:25:23.109188 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06632\n",
            "INFO:tensorflow:examples/sec: 34.1222\n",
            "I1203 18:25:23.109559 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1222\n",
            "INFO:tensorflow:global_step/sec: 1.05628\n",
            "I1203 18:25:24.055906 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05628\n",
            "INFO:tensorflow:examples/sec: 33.801\n",
            "I1203 18:25:24.056280 140409631090560 tpu_estimator.py:2308] examples/sec: 33.801\n",
            "INFO:tensorflow:global_step/sec: 1.06763\n",
            "I1203 18:25:24.992573 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06763\n",
            "INFO:tensorflow:examples/sec: 34.1642\n",
            "I1203 18:25:24.992803 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1642\n",
            "INFO:tensorflow:global_step/sec: 1.06109\n",
            "I1203 18:25:25.934977 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06109\n",
            "INFO:tensorflow:examples/sec: 33.9547\n",
            "I1203 18:25:25.935204 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9547\n",
            "INFO:tensorflow:global_step/sec: 1.06353\n",
            "I1203 18:25:26.875251 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06353\n",
            "INFO:tensorflow:examples/sec: 34.0329\n",
            "I1203 18:25:26.875533 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0329\n",
            "INFO:tensorflow:global_step/sec: 1.05558\n",
            "I1203 18:25:27.822649 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05558\n",
            "INFO:tensorflow:examples/sec: 33.7785\n",
            "I1203 18:25:27.823043 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7785\n",
            "INFO:tensorflow:global_step/sec: 1.06661\n",
            "I1203 18:25:28.760153 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06661\n",
            "INFO:tensorflow:examples/sec: 34.1315\n",
            "I1203 18:25:28.760538 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1315\n",
            "INFO:tensorflow:global_step/sec: 1.06613\n",
            "I1203 18:25:29.698130 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06613\n",
            "INFO:tensorflow:examples/sec: 34.1161\n",
            "I1203 18:25:29.698518 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1161\n",
            "INFO:tensorflow:global_step/sec: 1.0669\n",
            "I1203 18:25:30.635423 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0669\n",
            "INFO:tensorflow:examples/sec: 34.1409\n",
            "I1203 18:25:30.635837 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1409\n",
            "INFO:tensorflow:global_step/sec: 1.06544\n",
            "I1203 18:25:31.573988 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06544\n",
            "INFO:tensorflow:examples/sec: 34.094\n",
            "I1203 18:25:31.574421 140409631090560 tpu_estimator.py:2308] examples/sec: 34.094\n",
            "INFO:tensorflow:global_step/sec: 1.05153\n",
            "I1203 18:25:32.525003 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05153\n",
            "INFO:tensorflow:examples/sec: 33.6491\n",
            "I1203 18:25:32.525379 140409631090560 tpu_estimator.py:2308] examples/sec: 33.6491\n",
            "INFO:tensorflow:global_step/sec: 1.06332\n",
            "I1203 18:25:33.465453 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06332\n",
            "INFO:tensorflow:examples/sec: 34.0264\n",
            "I1203 18:25:33.465947 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0264\n",
            "INFO:tensorflow:global_step/sec: 1.0702\n",
            "I1203 18:25:34.399842 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0702\n",
            "INFO:tensorflow:examples/sec: 34.2465\n",
            "I1203 18:25:34.400257 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2465\n",
            "INFO:tensorflow:global_step/sec: 1.06397\n",
            "I1203 18:25:35.339716 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06397\n",
            "INFO:tensorflow:examples/sec: 34.0471\n",
            "I1203 18:25:35.340112 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0471\n",
            "INFO:tensorflow:global_step/sec: 1.05782\n",
            "I1203 18:25:36.285053 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05782\n",
            "INFO:tensorflow:examples/sec: 33.8501\n",
            "I1203 18:25:36.285505 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8501\n",
            "INFO:tensorflow:global_step/sec: 1.06134\n",
            "I1203 18:25:37.227265 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06134\n",
            "INFO:tensorflow:examples/sec: 33.9628\n",
            "I1203 18:25:37.227643 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9628\n",
            "INFO:tensorflow:global_step/sec: 1.06316\n",
            "I1203 18:25:38.167851 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06316\n",
            "INFO:tensorflow:examples/sec: 34.021\n",
            "I1203 18:25:38.168082 140409631090560 tpu_estimator.py:2308] examples/sec: 34.021\n",
            "INFO:tensorflow:global_step/sec: 1.0664\n",
            "I1203 18:25:39.105637 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0664\n",
            "INFO:tensorflow:examples/sec: 34.1248\n",
            "I1203 18:25:39.105990 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1248\n",
            "INFO:tensorflow:global_step/sec: 1.05981\n",
            "I1203 18:25:40.049151 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05981\n",
            "INFO:tensorflow:examples/sec: 33.9141\n",
            "I1203 18:25:40.049534 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9141\n",
            "INFO:tensorflow:global_step/sec: 1.06033\n",
            "I1203 18:25:40.992244 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06033\n",
            "INFO:tensorflow:examples/sec: 33.9306\n",
            "I1203 18:25:40.992485 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9306\n",
            "INFO:tensorflow:global_step/sec: 1.0647\n",
            "I1203 18:25:41.931511 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0647\n",
            "INFO:tensorflow:examples/sec: 34.0704\n",
            "I1203 18:25:41.931880 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0704\n",
            "INFO:tensorflow:global_step/sec: 1.06486\n",
            "I1203 18:25:42.870616 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06486\n",
            "INFO:tensorflow:examples/sec: 34.0757\n",
            "I1203 18:25:42.870999 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0757\n",
            "INFO:tensorflow:global_step/sec: 1.06167\n",
            "I1203 18:25:43.812503 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06167\n",
            "INFO:tensorflow:examples/sec: 33.9734\n",
            "I1203 18:25:43.812894 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9734\n",
            "INFO:tensorflow:global_step/sec: 1.06239\n",
            "I1203 18:25:44.753762 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06239\n",
            "INFO:tensorflow:examples/sec: 33.9966\n",
            "I1203 18:25:44.754188 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9966\n",
            "INFO:tensorflow:global_step/sec: 1.06423\n",
            "I1203 18:25:45.693412 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06423\n",
            "INFO:tensorflow:examples/sec: 34.0553\n",
            "I1203 18:25:45.694106 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0553\n",
            "INFO:tensorflow:global_step/sec: 1.06407\n",
            "I1203 18:25:46.633193 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06407\n",
            "INFO:tensorflow:examples/sec: 34.0501\n",
            "I1203 18:25:46.633897 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0501\n",
            "INFO:tensorflow:global_step/sec: 1.06298\n",
            "I1203 18:25:47.573945 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06298\n",
            "INFO:tensorflow:examples/sec: 34.0155\n",
            "I1203 18:25:47.574299 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0155\n",
            "INFO:tensorflow:global_step/sec: 1.07158\n",
            "I1203 18:25:48.507143 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07158\n",
            "INFO:tensorflow:examples/sec: 34.2907\n",
            "I1203 18:25:48.507377 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2907\n",
            "INFO:tensorflow:global_step/sec: 1.06308\n",
            "I1203 18:25:49.447804 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06308\n",
            "INFO:tensorflow:examples/sec: 34.0184\n",
            "I1203 18:25:49.448059 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0184\n",
            "INFO:tensorflow:global_step/sec: 1.06275\n",
            "I1203 18:25:50.388768 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06275\n",
            "INFO:tensorflow:examples/sec: 34.0079\n",
            "I1203 18:25:50.389243 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0079\n",
            "INFO:tensorflow:global_step/sec: 1.06294\n",
            "I1203 18:25:51.329593 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06294\n",
            "INFO:tensorflow:examples/sec: 34.014\n",
            "I1203 18:25:51.330002 140409631090560 tpu_estimator.py:2308] examples/sec: 34.014\n",
            "INFO:tensorflow:global_step/sec: 1.06268\n",
            "I1203 18:25:52.270602 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06268\n",
            "INFO:tensorflow:examples/sec: 34.0057\n",
            "I1203 18:25:52.270980 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0057\n",
            "INFO:tensorflow:global_step/sec: 1.0646\n",
            "I1203 18:25:53.209897 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0646\n",
            "INFO:tensorflow:examples/sec: 34.067\n",
            "I1203 18:25:53.210126 140409631090560 tpu_estimator.py:2308] examples/sec: 34.067\n",
            "INFO:tensorflow:global_step/sec: 1.06638\n",
            "I1203 18:25:54.147645 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06638\n",
            "INFO:tensorflow:examples/sec: 34.1242\n",
            "I1203 18:25:54.147877 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1242\n",
            "INFO:tensorflow:global_step/sec: 1.05244\n",
            "I1203 18:25:55.097850 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05244\n",
            "INFO:tensorflow:examples/sec: 33.678\n",
            "I1203 18:25:55.098537 140409631090560 tpu_estimator.py:2308] examples/sec: 33.678\n",
            "INFO:tensorflow:global_step/sec: 1.05494\n",
            "I1203 18:25:56.045742 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05494\n",
            "INFO:tensorflow:examples/sec: 33.758\n",
            "I1203 18:25:56.045983 140409631090560 tpu_estimator.py:2308] examples/sec: 33.758\n",
            "INFO:tensorflow:global_step/sec: 1.06664\n",
            "I1203 18:25:56.983326 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06664\n",
            "INFO:tensorflow:examples/sec: 34.1324\n",
            "I1203 18:25:56.983725 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1324\n",
            "INFO:tensorflow:global_step/sec: 1.06235\n",
            "I1203 18:25:57.924600 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06235\n",
            "INFO:tensorflow:examples/sec: 33.9951\n",
            "I1203 18:25:57.924982 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9951\n",
            "INFO:tensorflow:global_step/sec: 1.06182\n",
            "I1203 18:25:58.866367 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06182\n",
            "INFO:tensorflow:examples/sec: 33.9783\n",
            "I1203 18:25:58.866632 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9783\n",
            "INFO:tensorflow:global_step/sec: 1.06087\n",
            "I1203 18:25:59.808990 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06087\n",
            "INFO:tensorflow:examples/sec: 33.948\n",
            "I1203 18:25:59.809369 140409631090560 tpu_estimator.py:2308] examples/sec: 33.948\n",
            "INFO:tensorflow:global_step/sec: 1.06266\n",
            "I1203 18:26:00.750036 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06266\n",
            "INFO:tensorflow:examples/sec: 34.0051\n",
            "I1203 18:26:00.750916 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0051\n",
            "INFO:tensorflow:global_step/sec: 1.05938\n",
            "I1203 18:26:01.693955 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05938\n",
            "INFO:tensorflow:examples/sec: 33.9003\n",
            "I1203 18:26:01.694316 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9003\n",
            "INFO:tensorflow:global_step/sec: 1.06364\n",
            "I1203 18:26:02.634123 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06364\n",
            "INFO:tensorflow:examples/sec: 34.0363\n",
            "I1203 18:26:02.634668 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0363\n",
            "INFO:tensorflow:global_step/sec: 1.0637\n",
            "I1203 18:26:03.574252 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0637\n",
            "INFO:tensorflow:examples/sec: 34.0383\n",
            "I1203 18:26:03.574522 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0383\n",
            "INFO:tensorflow:global_step/sec: 1.06526\n",
            "I1203 18:26:04.512994 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06526\n",
            "INFO:tensorflow:examples/sec: 34.0883\n",
            "I1203 18:26:04.513218 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0883\n",
            "INFO:tensorflow:global_step/sec: 1.06114\n",
            "I1203 18:26:05.455370 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06114\n",
            "INFO:tensorflow:examples/sec: 33.9564\n",
            "I1203 18:26:05.455744 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9564\n",
            "INFO:tensorflow:global_step/sec: 1.05957\n",
            "I1203 18:26:06.399182 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05957\n",
            "INFO:tensorflow:examples/sec: 33.9063\n",
            "I1203 18:26:06.399552 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9063\n",
            "INFO:tensorflow:global_step/sec: 1.06091\n",
            "I1203 18:26:07.341756 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06091\n",
            "INFO:tensorflow:examples/sec: 33.949\n",
            "I1203 18:26:07.342003 140409631090560 tpu_estimator.py:2308] examples/sec: 33.949\n",
            "INFO:tensorflow:global_step/sec: 1.07219\n",
            "I1203 18:26:08.274432 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07219\n",
            "INFO:tensorflow:examples/sec: 34.3101\n",
            "I1203 18:26:08.274667 140409631090560 tpu_estimator.py:2308] examples/sec: 34.3101\n",
            "INFO:tensorflow:global_step/sec: 1.06408\n",
            "I1203 18:26:09.214227 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06408\n",
            "INFO:tensorflow:examples/sec: 34.0505\n",
            "I1203 18:26:09.214624 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0505\n",
            "INFO:tensorflow:global_step/sec: 1.05937\n",
            "I1203 18:26:10.158167 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05937\n",
            "INFO:tensorflow:examples/sec: 33.8997\n",
            "I1203 18:26:10.158413 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8997\n",
            "INFO:tensorflow:global_step/sec: 1.06036\n",
            "I1203 18:26:11.101238 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06036\n",
            "INFO:tensorflow:examples/sec: 33.9314\n",
            "I1203 18:26:11.101491 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9314\n",
            "INFO:tensorflow:global_step/sec: 1.06575\n",
            "I1203 18:26:12.039550 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06575\n",
            "INFO:tensorflow:examples/sec: 34.104\n",
            "I1203 18:26:12.039805 140409631090560 tpu_estimator.py:2308] examples/sec: 34.104\n",
            "INFO:tensorflow:global_step/sec: 1.0627\n",
            "I1203 18:26:12.980564 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0627\n",
            "INFO:tensorflow:examples/sec: 34.0065\n",
            "I1203 18:26:12.980948 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0065\n",
            "INFO:tensorflow:global_step/sec: 1.06199\n",
            "I1203 18:26:13.922166 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06199\n",
            "INFO:tensorflow:examples/sec: 33.9837\n",
            "I1203 18:26:13.922478 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9837\n",
            "INFO:tensorflow:global_step/sec: 1.06078\n",
            "I1203 18:26:14.864862 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06078\n",
            "INFO:tensorflow:examples/sec: 33.945\n",
            "I1203 18:26:14.865100 140409631090560 tpu_estimator.py:2308] examples/sec: 33.945\n",
            "INFO:tensorflow:global_step/sec: 1.06602\n",
            "I1203 18:26:15.802940 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06602\n",
            "INFO:tensorflow:examples/sec: 34.1125\n",
            "I1203 18:26:15.803304 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1125\n",
            "INFO:tensorflow:global_step/sec: 1.07023\n",
            "I1203 18:26:16.737313 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07023\n",
            "INFO:tensorflow:examples/sec: 34.2472\n",
            "I1203 18:26:16.737688 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2472\n",
            "INFO:tensorflow:global_step/sec: 1.06147\n",
            "I1203 18:26:17.679451 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06147\n",
            "INFO:tensorflow:examples/sec: 33.967\n",
            "I1203 18:26:17.679857 140409631090560 tpu_estimator.py:2308] examples/sec: 33.967\n",
            "INFO:tensorflow:global_step/sec: 1.0604\n",
            "I1203 18:26:18.622457 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0604\n",
            "INFO:tensorflow:examples/sec: 33.933\n",
            "I1203 18:26:18.622817 140409631090560 tpu_estimator.py:2308] examples/sec: 33.933\n",
            "INFO:tensorflow:global_step/sec: 1.05942\n",
            "I1203 18:26:19.566371 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05942\n",
            "INFO:tensorflow:examples/sec: 33.9014\n",
            "I1203 18:26:19.566645 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9014\n",
            "INFO:tensorflow:global_step/sec: 1.05821\n",
            "I1203 18:26:20.511351 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05821\n",
            "INFO:tensorflow:examples/sec: 33.8629\n",
            "I1203 18:26:20.511611 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8629\n",
            "INFO:tensorflow:global_step/sec: 1.06557\n",
            "I1203 18:26:21.449819 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06557\n",
            "INFO:tensorflow:examples/sec: 34.0983\n",
            "I1203 18:26:21.450050 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0983\n",
            "INFO:tensorflow:global_step/sec: 1.05893\n",
            "I1203 18:26:22.394147 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05893\n",
            "INFO:tensorflow:examples/sec: 33.8859\n",
            "I1203 18:26:22.394371 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8859\n",
            "INFO:tensorflow:global_step/sec: 1.06347\n",
            "I1203 18:26:23.334500 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06347\n",
            "INFO:tensorflow:examples/sec: 34.0309\n",
            "I1203 18:26:23.334914 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0309\n",
            "INFO:tensorflow:global_step/sec: 1.06188\n",
            "I1203 18:26:24.276210 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06188\n",
            "INFO:tensorflow:examples/sec: 33.9801\n",
            "I1203 18:26:24.276484 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9801\n",
            "INFO:tensorflow:global_step/sec: 1.06439\n",
            "I1203 18:26:25.215708 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06439\n",
            "INFO:tensorflow:examples/sec: 34.0606\n",
            "I1203 18:26:25.216152 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0606\n",
            "INFO:tensorflow:global_step/sec: 1.0567\n",
            "I1203 18:26:26.162053 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0567\n",
            "INFO:tensorflow:examples/sec: 33.8144\n",
            "I1203 18:26:26.162295 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8144\n",
            "INFO:tensorflow:global_step/sec: 1.06678\n",
            "I1203 18:26:27.099390 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06678\n",
            "INFO:tensorflow:examples/sec: 34.1368\n",
            "I1203 18:26:27.099624 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1368\n",
            "INFO:tensorflow:global_step/sec: 1.06463\n",
            "I1203 18:26:28.038853 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06463\n",
            "INFO:tensorflow:examples/sec: 34.0681\n",
            "I1203 18:26:28.039132 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0681\n",
            "INFO:tensorflow:global_step/sec: 1.0616\n",
            "I1203 18:26:28.980725 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0616\n",
            "INFO:tensorflow:examples/sec: 33.9714\n",
            "I1203 18:26:28.981100 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9714\n",
            "INFO:tensorflow:global_step/sec: 1.06436\n",
            "I1203 18:26:29.920264 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06436\n",
            "INFO:tensorflow:examples/sec: 34.0595\n",
            "I1203 18:26:29.920650 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0595\n",
            "INFO:tensorflow:global_step/sec: 1.06171\n",
            "I1203 18:26:30.862164 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06171\n",
            "INFO:tensorflow:examples/sec: 33.9748\n",
            "I1203 18:26:30.862428 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9748\n",
            "INFO:tensorflow:global_step/sec: 1.05745\n",
            "I1203 18:26:31.807812 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05745\n",
            "INFO:tensorflow:examples/sec: 33.8384\n",
            "I1203 18:26:31.808212 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8384\n",
            "INFO:tensorflow:global_step/sec: 1.06593\n",
            "I1203 18:26:32.745942 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06593\n",
            "INFO:tensorflow:examples/sec: 34.1097\n",
            "I1203 18:26:32.746362 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1097\n",
            "INFO:tensorflow:global_step/sec: 1.06084\n",
            "I1203 18:26:33.688616 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06084\n",
            "INFO:tensorflow:examples/sec: 33.947\n",
            "I1203 18:26:33.689038 140409631090560 tpu_estimator.py:2308] examples/sec: 33.947\n",
            "INFO:tensorflow:global_step/sec: 1.06455\n",
            "I1203 18:26:34.627966 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06455\n",
            "INFO:tensorflow:examples/sec: 34.0655\n",
            "I1203 18:26:34.628196 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0655\n",
            "INFO:tensorflow:global_step/sec: 1.05694\n",
            "I1203 18:26:35.574098 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05694\n",
            "INFO:tensorflow:examples/sec: 33.822\n",
            "I1203 18:26:35.574322 140409631090560 tpu_estimator.py:2308] examples/sec: 33.822\n",
            "INFO:tensorflow:global_step/sec: 1.06844\n",
            "I1203 18:26:36.510040 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06844\n",
            "INFO:tensorflow:examples/sec: 34.19\n",
            "I1203 18:26:36.510504 140409631090560 tpu_estimator.py:2308] examples/sec: 34.19\n",
            "INFO:tensorflow:global_step/sec: 1.06227\n",
            "I1203 18:26:37.451445 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06227\n",
            "INFO:tensorflow:examples/sec: 33.9926\n",
            "I1203 18:26:37.451807 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9926\n",
            "INFO:tensorflow:global_step/sec: 1.06388\n",
            "I1203 18:26:38.391386 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06388\n",
            "INFO:tensorflow:examples/sec: 34.0442\n",
            "I1203 18:26:38.391639 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0442\n",
            "INFO:tensorflow:global_step/sec: 1.06088\n",
            "I1203 18:26:39.333998 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06088\n",
            "INFO:tensorflow:examples/sec: 33.9483\n",
            "I1203 18:26:39.334244 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9483\n",
            "INFO:tensorflow:global_step/sec: 1.06147\n",
            "I1203 18:26:40.276100 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06147\n",
            "INFO:tensorflow:examples/sec: 33.967\n",
            "I1203 18:26:40.276487 140409631090560 tpu_estimator.py:2308] examples/sec: 33.967\n",
            "INFO:tensorflow:global_step/sec: 1.06302\n",
            "I1203 18:26:41.216795 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06302\n",
            "INFO:tensorflow:examples/sec: 34.0166\n",
            "I1203 18:26:41.217164 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0166\n",
            "INFO:tensorflow:global_step/sec: 1.06654\n",
            "I1203 18:26:42.154432 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06654\n",
            "INFO:tensorflow:examples/sec: 34.1294\n",
            "I1203 18:26:42.154827 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1294\n",
            "INFO:tensorflow:global_step/sec: 1.05773\n",
            "I1203 18:26:43.099841 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05773\n",
            "INFO:tensorflow:examples/sec: 33.8473\n",
            "I1203 18:26:43.100228 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8473\n",
            "INFO:tensorflow:global_step/sec: 1.06699\n",
            "I1203 18:26:44.037061 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06699\n",
            "INFO:tensorflow:examples/sec: 34.1436\n",
            "I1203 18:26:44.037309 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1436\n",
            "INFO:tensorflow:global_step/sec: 1.0567\n",
            "I1203 18:26:44.983378 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0567\n",
            "INFO:tensorflow:examples/sec: 33.8142\n",
            "I1203 18:26:44.983623 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8142\n",
            "INFO:tensorflow:global_step/sec: 1.06942\n",
            "I1203 18:26:45.918495 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06942\n",
            "INFO:tensorflow:examples/sec: 34.2216\n",
            "I1203 18:26:45.918729 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2216\n",
            "INFO:tensorflow:global_step/sec: 1.05615\n",
            "I1203 18:26:46.865312 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05615\n",
            "INFO:tensorflow:examples/sec: 33.7967\n",
            "I1203 18:26:46.865580 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7967\n",
            "INFO:tensorflow:global_step/sec: 1.06965\n",
            "I1203 18:26:47.800182 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06965\n",
            "INFO:tensorflow:examples/sec: 34.2289\n",
            "I1203 18:26:47.800429 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2289\n",
            "INFO:tensorflow:global_step/sec: 1.05872\n",
            "I1203 18:26:48.744737 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05872\n",
            "INFO:tensorflow:examples/sec: 33.8789\n",
            "I1203 18:26:48.744975 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8789\n",
            "INFO:tensorflow:global_step/sec: 1.06532\n",
            "I1203 18:26:49.683450 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06532\n",
            "INFO:tensorflow:examples/sec: 34.0903\n",
            "I1203 18:26:49.683683 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0903\n",
            "INFO:tensorflow:global_step/sec: 1.0527\n",
            "I1203 18:26:50.633357 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0527\n",
            "INFO:tensorflow:examples/sec: 33.6863\n",
            "I1203 18:26:50.633602 140409631090560 tpu_estimator.py:2308] examples/sec: 33.6863\n",
            "INFO:tensorflow:global_step/sec: 1.06506\n",
            "I1203 18:26:51.572278 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06506\n",
            "INFO:tensorflow:examples/sec: 34.0819\n",
            "I1203 18:26:51.572530 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0819\n",
            "INFO:tensorflow:global_step/sec: 1.05624\n",
            "I1203 18:26:52.519044 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05624\n",
            "INFO:tensorflow:examples/sec: 33.7998\n",
            "I1203 18:26:52.519277 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7998\n",
            "INFO:tensorflow:global_step/sec: 1.06283\n",
            "I1203 18:26:53.459934 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06283\n",
            "INFO:tensorflow:examples/sec: 34.0105\n",
            "I1203 18:26:53.460321 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0105\n",
            "INFO:tensorflow:global_step/sec: 1.06192\n",
            "I1203 18:26:54.401645 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06192\n",
            "INFO:tensorflow:examples/sec: 33.9813\n",
            "I1203 18:26:54.402033 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9813\n",
            "INFO:tensorflow:global_step/sec: 1.06093\n",
            "I1203 18:26:55.344188 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06093\n",
            "INFO:tensorflow:examples/sec: 33.9498\n",
            "I1203 18:26:55.344747 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9498\n",
            "INFO:tensorflow:global_step/sec: 1.0628\n",
            "I1203 18:26:56.285085 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0628\n",
            "INFO:tensorflow:examples/sec: 34.0096\n",
            "I1203 18:26:56.285320 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0096\n",
            "INFO:tensorflow:global_step/sec: 1.06603\n",
            "I1203 18:26:57.223134 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06603\n",
            "INFO:tensorflow:examples/sec: 34.1131\n",
            "I1203 18:26:57.223367 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1131\n",
            "INFO:tensorflow:global_step/sec: 1.05728\n",
            "I1203 18:26:58.168956 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05728\n",
            "INFO:tensorflow:examples/sec: 33.8331\n",
            "I1203 18:26:58.169196 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8331\n",
            "INFO:tensorflow:global_step/sec: 1.06013\n",
            "I1203 18:26:59.112229 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06013\n",
            "INFO:tensorflow:examples/sec: 33.924\n",
            "I1203 18:26:59.112501 140409631090560 tpu_estimator.py:2308] examples/sec: 33.924\n",
            "INFO:tensorflow:global_step/sec: 1.06284\n",
            "I1203 18:27:00.053112 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06284\n",
            "INFO:tensorflow:examples/sec: 34.011\n",
            "I1203 18:27:00.053500 140409631090560 tpu_estimator.py:2308] examples/sec: 34.011\n",
            "INFO:tensorflow:global_step/sec: 1.06419\n",
            "I1203 18:27:00.992794 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06419\n",
            "INFO:tensorflow:examples/sec: 34.0539\n",
            "I1203 18:27:00.993030 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0539\n",
            "INFO:tensorflow:global_step/sec: 1.05933\n",
            "I1203 18:27:01.936824 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05933\n",
            "INFO:tensorflow:examples/sec: 33.8985\n",
            "I1203 18:27:01.937071 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8985\n",
            "INFO:tensorflow:global_step/sec: 1.05954\n",
            "I1203 18:27:02.880607 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05954\n",
            "INFO:tensorflow:examples/sec: 33.9053\n",
            "I1203 18:27:02.880968 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9053\n",
            "INFO:tensorflow:global_step/sec: 1.06323\n",
            "I1203 18:27:03.821160 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06323\n",
            "INFO:tensorflow:examples/sec: 34.0234\n",
            "I1203 18:27:03.821430 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0234\n",
            "INFO:tensorflow:global_step/sec: 1.06449\n",
            "I1203 18:27:04.760589 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06449\n",
            "INFO:tensorflow:examples/sec: 34.0636\n",
            "I1203 18:27:04.760843 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0636\n",
            "INFO:tensorflow:global_step/sec: 1.06825\n",
            "I1203 18:27:05.696664 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06825\n",
            "INFO:tensorflow:examples/sec: 34.1839\n",
            "I1203 18:27:05.696905 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1839\n",
            "INFO:tensorflow:global_step/sec: 1.06139\n",
            "I1203 18:27:06.638831 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06139\n",
            "INFO:tensorflow:examples/sec: 33.9643\n",
            "I1203 18:27:06.639196 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9643\n",
            "INFO:tensorflow:global_step/sec: 1.06212\n",
            "I1203 18:27:07.580346 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06212\n",
            "INFO:tensorflow:examples/sec: 33.9878\n",
            "I1203 18:27:07.580759 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9878\n",
            "INFO:tensorflow:global_step/sec: 1.05348\n",
            "I1203 18:27:08.529597 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05348\n",
            "INFO:tensorflow:examples/sec: 33.7112\n",
            "I1203 18:27:08.529827 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7112\n",
            "INFO:tensorflow:global_step/sec: 1.0654\n",
            "I1203 18:27:09.468212 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0654\n",
            "INFO:tensorflow:examples/sec: 34.0928\n",
            "I1203 18:27:09.468477 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0928\n",
            "INFO:tensorflow:global_step/sec: 1.06721\n",
            "I1203 18:27:10.405228 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06721\n",
            "INFO:tensorflow:examples/sec: 34.1509\n",
            "I1203 18:27:10.405648 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1509\n",
            "INFO:tensorflow:global_step/sec: 1.0638\n",
            "I1203 18:27:11.345241 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0638\n",
            "INFO:tensorflow:examples/sec: 34.0414\n",
            "I1203 18:27:11.345484 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0414\n",
            "INFO:tensorflow:global_step/sec: 1.06099\n",
            "I1203 18:27:12.287751 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06099\n",
            "INFO:tensorflow:examples/sec: 33.9517\n",
            "I1203 18:27:12.287989 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9517\n",
            "INFO:tensorflow:global_step/sec: 1.05611\n",
            "I1203 18:27:13.234643 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05611\n",
            "INFO:tensorflow:examples/sec: 33.7955\n",
            "I1203 18:27:13.234877 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7955\n",
            "INFO:tensorflow:global_step/sec: 1.05834\n",
            "I1203 18:27:14.179550 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05834\n",
            "INFO:tensorflow:examples/sec: 33.8669\n",
            "I1203 18:27:14.179807 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8669\n",
            "INFO:tensorflow:global_step/sec: 1.05855\n",
            "I1203 18:27:15.124200 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05855\n",
            "INFO:tensorflow:examples/sec: 33.8737\n",
            "I1203 18:27:15.124600 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8737\n",
            "INFO:tensorflow:global_step/sec: 1.06724\n",
            "I1203 18:27:16.061192 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06724\n",
            "INFO:tensorflow:examples/sec: 34.1516\n",
            "I1203 18:27:16.061455 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1516\n",
            "INFO:tensorflow:global_step/sec: 1.06099\n",
            "I1203 18:27:17.003689 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06099\n",
            "INFO:tensorflow:examples/sec: 33.9518\n",
            "I1203 18:27:17.004088 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9518\n",
            "INFO:tensorflow:global_step/sec: 1.06275\n",
            "I1203 18:27:17.944652 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06275\n",
            "INFO:tensorflow:examples/sec: 34.008\n",
            "I1203 18:27:17.944880 140409631090560 tpu_estimator.py:2308] examples/sec: 34.008\n",
            "INFO:tensorflow:global_step/sec: 1.05838\n",
            "I1203 18:27:18.889527 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05838\n",
            "INFO:tensorflow:examples/sec: 33.8681\n",
            "I1203 18:27:18.889750 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8681\n",
            "INFO:tensorflow:global_step/sec: 1.06464\n",
            "I1203 18:27:19.828790 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06464\n",
            "INFO:tensorflow:examples/sec: 34.0684\n",
            "I1203 18:27:19.829049 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0684\n",
            "INFO:tensorflow:global_step/sec: 1.06218\n",
            "I1203 18:27:20.770238 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06218\n",
            "INFO:tensorflow:examples/sec: 33.9899\n",
            "I1203 18:27:20.770522 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9899\n",
            "INFO:tensorflow:global_step/sec: 1.06426\n",
            "I1203 18:27:21.709865 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06426\n",
            "INFO:tensorflow:examples/sec: 34.0563\n",
            "I1203 18:27:21.710105 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0563\n",
            "INFO:tensorflow:global_step/sec: 1.05018\n",
            "I1203 18:27:22.662063 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05018\n",
            "INFO:tensorflow:examples/sec: 33.6058\n",
            "I1203 18:27:22.662438 140409631090560 tpu_estimator.py:2308] examples/sec: 33.6058\n",
            "INFO:tensorflow:global_step/sec: 1.06171\n",
            "I1203 18:27:23.603966 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06171\n",
            "INFO:tensorflow:examples/sec: 33.9749\n",
            "I1203 18:27:23.604215 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9749\n",
            "INFO:tensorflow:global_step/sec: 1.06215\n",
            "I1203 18:27:24.545441 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06215\n",
            "INFO:tensorflow:examples/sec: 33.9889\n",
            "I1203 18:27:24.545820 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9889\n",
            "INFO:tensorflow:global_step/sec: 1.06148\n",
            "I1203 18:27:25.487550 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06148\n",
            "INFO:tensorflow:examples/sec: 33.9673\n",
            "I1203 18:27:25.487791 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9673\n",
            "INFO:tensorflow:global_step/sec: 1.06399\n",
            "I1203 18:27:26.427377 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06399\n",
            "INFO:tensorflow:examples/sec: 34.0477\n",
            "I1203 18:27:26.427765 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0477\n",
            "INFO:tensorflow:global_step/sec: 1.06079\n",
            "I1203 18:27:27.370076 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06079\n",
            "INFO:tensorflow:examples/sec: 33.9453\n",
            "I1203 18:27:27.370310 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9453\n",
            "INFO:tensorflow:global_step/sec: 1.06156\n",
            "I1203 18:27:28.312122 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06156\n",
            "INFO:tensorflow:examples/sec: 33.9699\n",
            "I1203 18:27:28.312418 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9699\n",
            "INFO:tensorflow:global_step/sec: 1.0607\n",
            "I1203 18:27:29.254850 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0607\n",
            "INFO:tensorflow:examples/sec: 33.9424\n",
            "I1203 18:27:29.255285 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9424\n",
            "INFO:tensorflow:global_step/sec: 1.06426\n",
            "I1203 18:27:30.194485 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06426\n",
            "INFO:tensorflow:examples/sec: 34.0563\n",
            "I1203 18:27:30.194721 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0563\n",
            "INFO:tensorflow:global_step/sec: 1.0631\n",
            "I1203 18:27:31.135117 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0631\n",
            "INFO:tensorflow:examples/sec: 34.0192\n",
            "I1203 18:27:31.135349 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0192\n",
            "INFO:tensorflow:global_step/sec: 1.06002\n",
            "I1203 18:27:32.078525 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06002\n",
            "INFO:tensorflow:examples/sec: 33.9207\n",
            "I1203 18:27:32.078762 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9207\n",
            "INFO:tensorflow:global_step/sec: 1.0598\n",
            "I1203 18:27:33.022066 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0598\n",
            "INFO:tensorflow:examples/sec: 33.9137\n",
            "I1203 18:27:33.022294 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9137\n",
            "INFO:tensorflow:global_step/sec: 1.06574\n",
            "I1203 18:27:33.960386 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06574\n",
            "INFO:tensorflow:examples/sec: 34.1037\n",
            "I1203 18:27:33.960638 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1037\n",
            "INFO:tensorflow:global_step/sec: 1.06128\n",
            "I1203 18:27:34.902645 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06128\n",
            "INFO:tensorflow:examples/sec: 33.9609\n",
            "I1203 18:27:34.902878 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9609\n",
            "INFO:tensorflow:global_step/sec: 1.0564\n",
            "I1203 18:27:35.849250 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0564\n",
            "INFO:tensorflow:examples/sec: 33.8048\n",
            "I1203 18:27:35.849627 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8048\n",
            "INFO:tensorflow:global_step/sec: 1.06302\n",
            "I1203 18:27:36.789960 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06302\n",
            "INFO:tensorflow:examples/sec: 34.0167\n",
            "I1203 18:27:36.790195 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0167\n",
            "INFO:tensorflow:global_step/sec: 1.0704\n",
            "I1203 18:27:37.724202 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0704\n",
            "INFO:tensorflow:examples/sec: 34.2528\n",
            "I1203 18:27:37.724448 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2528\n",
            "INFO:tensorflow:global_step/sec: 1.05807\n",
            "I1203 18:27:38.669311 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05807\n",
            "INFO:tensorflow:examples/sec: 33.8582\n",
            "I1203 18:27:38.669563 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8582\n",
            "INFO:tensorflow:global_step/sec: 1.05403\n",
            "I1203 18:27:39.618059 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05403\n",
            "INFO:tensorflow:examples/sec: 33.7291\n",
            "I1203 18:27:39.618288 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7291\n",
            "INFO:tensorflow:global_step/sec: 1.0611\n",
            "I1203 18:27:40.560498 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0611\n",
            "INFO:tensorflow:examples/sec: 33.9552\n",
            "I1203 18:27:40.560872 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9552\n",
            "INFO:tensorflow:global_step/sec: 1.05553\n",
            "I1203 18:27:41.507868 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05553\n",
            "INFO:tensorflow:examples/sec: 33.7769\n",
            "I1203 18:27:41.508088 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7769\n",
            "INFO:tensorflow:global_step/sec: 1.06382\n",
            "I1203 18:27:42.447870 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06382\n",
            "INFO:tensorflow:examples/sec: 34.0424\n",
            "I1203 18:27:42.448253 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0424\n",
            "INFO:tensorflow:global_step/sec: 1.06912\n",
            "I1203 18:27:43.383202 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06912\n",
            "INFO:tensorflow:examples/sec: 34.2118\n",
            "I1203 18:27:43.383449 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2118\n",
            "INFO:tensorflow:global_step/sec: 1.06156\n",
            "I1203 18:27:44.325227 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06156\n",
            "INFO:tensorflow:examples/sec: 33.9698\n",
            "I1203 18:27:44.325504 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9698\n",
            "INFO:tensorflow:global_step/sec: 1.06037\n",
            "I1203 18:27:45.268288 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06037\n",
            "INFO:tensorflow:examples/sec: 33.9317\n",
            "I1203 18:27:45.268665 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9317\n",
            "INFO:tensorflow:global_step/sec: 1.06261\n",
            "I1203 18:27:46.209414 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06261\n",
            "INFO:tensorflow:examples/sec: 34.0036\n",
            "I1203 18:27:46.209817 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0036\n",
            "INFO:tensorflow:global_step/sec: 1.05693\n",
            "I1203 18:27:47.155544 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05693\n",
            "INFO:tensorflow:examples/sec: 33.8217\n",
            "I1203 18:27:47.155807 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8217\n",
            "INFO:tensorflow:global_step/sec: 1.06309\n",
            "I1203 18:27:48.096176 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06309\n",
            "INFO:tensorflow:examples/sec: 34.0188\n",
            "I1203 18:27:48.096417 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0188\n",
            "INFO:tensorflow:global_step/sec: 1.06609\n",
            "I1203 18:27:49.034185 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06609\n",
            "INFO:tensorflow:examples/sec: 34.1148\n",
            "I1203 18:27:49.034565 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1148\n",
            "INFO:tensorflow:global_step/sec: 1.0656\n",
            "I1203 18:27:49.972625 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0656\n",
            "INFO:tensorflow:examples/sec: 34.0993\n",
            "I1203 18:27:49.972852 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0993\n",
            "INFO:tensorflow:global_step/sec: 1.06081\n",
            "I1203 18:27:50.915282 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06081\n",
            "INFO:tensorflow:examples/sec: 33.9459\n",
            "I1203 18:27:50.915528 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9459\n",
            "INFO:tensorflow:global_step/sec: 1.06572\n",
            "I1203 18:27:51.853639 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06572\n",
            "INFO:tensorflow:examples/sec: 34.1029\n",
            "I1203 18:27:51.853869 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1029\n",
            "INFO:tensorflow:global_step/sec: 1.06324\n",
            "I1203 18:27:52.794159 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06324\n",
            "INFO:tensorflow:examples/sec: 34.0238\n",
            "I1203 18:27:52.794549 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0238\n",
            "INFO:tensorflow:global_step/sec: 1.06262\n",
            "I1203 18:27:53.735234 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06262\n",
            "INFO:tensorflow:examples/sec: 34.0037\n",
            "I1203 18:27:53.735479 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0037\n",
            "INFO:tensorflow:global_step/sec: 1.06212\n",
            "I1203 18:27:54.676875 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06212\n",
            "INFO:tensorflow:examples/sec: 33.9878\n",
            "I1203 18:27:54.677132 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9878\n",
            "INFO:tensorflow:global_step/sec: 1.06396\n",
            "I1203 18:27:55.616628 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06396\n",
            "INFO:tensorflow:examples/sec: 34.0468\n",
            "I1203 18:27:55.617008 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0468\n",
            "INFO:tensorflow:global_step/sec: 1.06147\n",
            "I1203 18:27:56.558709 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06147\n",
            "INFO:tensorflow:examples/sec: 33.9671\n",
            "I1203 18:27:56.558943 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9671\n",
            "INFO:tensorflow:global_step/sec: 1.05758\n",
            "I1203 18:27:57.504275 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05758\n",
            "INFO:tensorflow:examples/sec: 33.8426\n",
            "I1203 18:27:57.504678 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8426\n",
            "INFO:tensorflow:global_step/sec: 1.06978\n",
            "I1203 18:27:58.439030 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06978\n",
            "INFO:tensorflow:examples/sec: 34.233\n",
            "I1203 18:27:58.439447 140409631090560 tpu_estimator.py:2308] examples/sec: 34.233\n",
            "INFO:tensorflow:global_step/sec: 1.06325\n",
            "I1203 18:27:59.379585 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06325\n",
            "INFO:tensorflow:examples/sec: 34.0239\n",
            "I1203 18:27:59.379839 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0239\n",
            "INFO:tensorflow:global_step/sec: 1.05732\n",
            "I1203 18:28:00.325295 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05732\n",
            "INFO:tensorflow:examples/sec: 33.8343\n",
            "I1203 18:28:00.325670 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8343\n",
            "INFO:tensorflow:global_step/sec: 1.06236\n",
            "I1203 18:28:01.266644 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06236\n",
            "INFO:tensorflow:examples/sec: 33.9954\n",
            "I1203 18:28:01.266874 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9954\n",
            "INFO:tensorflow:global_step/sec: 1.05844\n",
            "I1203 18:28:02.211429 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05844\n",
            "INFO:tensorflow:examples/sec: 33.8702\n",
            "I1203 18:28:02.211654 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8702\n",
            "INFO:tensorflow:global_step/sec: 1.05995\n",
            "I1203 18:28:03.154863 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05995\n",
            "INFO:tensorflow:examples/sec: 33.9183\n",
            "I1203 18:28:03.155277 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9183\n",
            "INFO:tensorflow:global_step/sec: 1.06405\n",
            "I1203 18:28:04.094676 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06405\n",
            "INFO:tensorflow:examples/sec: 34.0497\n",
            "I1203 18:28:04.094902 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0497\n",
            "INFO:tensorflow:global_step/sec: 1.06138\n",
            "I1203 18:28:05.036848 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06138\n",
            "INFO:tensorflow:examples/sec: 33.9641\n",
            "I1203 18:28:05.037257 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9641\n",
            "INFO:tensorflow:global_step/sec: 1.06258\n",
            "I1203 18:28:05.977979 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06258\n",
            "INFO:tensorflow:examples/sec: 34.0024\n",
            "I1203 18:28:05.978234 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0024\n",
            "INFO:tensorflow:global_step/sec: 1.06538\n",
            "I1203 18:28:06.916610 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06538\n",
            "INFO:tensorflow:examples/sec: 34.0922\n",
            "I1203 18:28:06.917000 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0922\n",
            "INFO:tensorflow:global_step/sec: 1.06112\n",
            "I1203 18:28:07.858977 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06112\n",
            "INFO:tensorflow:examples/sec: 33.9559\n",
            "I1203 18:28:07.859240 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9559\n",
            "INFO:tensorflow:global_step/sec: 1.0632\n",
            "I1203 18:28:08.799580 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0632\n",
            "INFO:tensorflow:examples/sec: 34.0225\n",
            "I1203 18:28:08.800070 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0225\n",
            "INFO:tensorflow:global_step/sec: 1.0627\n",
            "I1203 18:28:09.740533 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0627\n",
            "INFO:tensorflow:examples/sec: 34.0064\n",
            "I1203 18:28:09.740756 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0064\n",
            "INFO:tensorflow:global_step/sec: 1.06222\n",
            "I1203 18:28:10.681946 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06222\n",
            "INFO:tensorflow:examples/sec: 33.991\n",
            "I1203 18:28:10.682177 140409631090560 tpu_estimator.py:2308] examples/sec: 33.991\n",
            "INFO:tensorflow:global_step/sec: 1.06447\n",
            "I1203 18:28:11.621384 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06447\n",
            "INFO:tensorflow:examples/sec: 34.0629\n",
            "I1203 18:28:11.621623 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0629\n",
            "INFO:tensorflow:global_step/sec: 1.0645\n",
            "I1203 18:28:12.560814 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0645\n",
            "INFO:tensorflow:examples/sec: 34.0639\n",
            "I1203 18:28:12.561182 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0639\n",
            "INFO:tensorflow:global_step/sec: 1.06271\n",
            "I1203 18:28:13.501786 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06271\n",
            "INFO:tensorflow:examples/sec: 34.0068\n",
            "I1203 18:28:13.502014 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0068\n",
            "INFO:tensorflow:global_step/sec: 1.06266\n",
            "I1203 18:28:14.442826 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06266\n",
            "INFO:tensorflow:examples/sec: 34.005\n",
            "I1203 18:28:14.443061 140409631090560 tpu_estimator.py:2308] examples/sec: 34.005\n",
            "INFO:tensorflow:global_step/sec: 1.06455\n",
            "I1203 18:28:15.382186 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06455\n",
            "INFO:tensorflow:examples/sec: 34.0656\n",
            "I1203 18:28:15.382426 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0656\n",
            "INFO:tensorflow:global_step/sec: 1.05892\n",
            "I1203 18:28:16.326606 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05892\n",
            "INFO:tensorflow:examples/sec: 33.8855\n",
            "I1203 18:28:16.326980 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8855\n",
            "INFO:tensorflow:global_step/sec: 1.06143\n",
            "I1203 18:28:17.268689 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06143\n",
            "INFO:tensorflow:examples/sec: 33.9656\n",
            "I1203 18:28:17.269113 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9656\n",
            "INFO:tensorflow:global_step/sec: 1.0595\n",
            "I1203 18:28:18.212593 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0595\n",
            "INFO:tensorflow:examples/sec: 33.9039\n",
            "I1203 18:28:18.212970 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9039\n",
            "INFO:tensorflow:global_step/sec: 1.05728\n",
            "I1203 18:28:19.158343 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05728\n",
            "INFO:tensorflow:examples/sec: 33.8331\n",
            "I1203 18:28:19.158720 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8331\n",
            "INFO:tensorflow:global_step/sec: 1.06875\n",
            "I1203 18:28:20.094012 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06875\n",
            "INFO:tensorflow:examples/sec: 34.2001\n",
            "I1203 18:28:20.094263 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2001\n",
            "INFO:tensorflow:global_step/sec: 1.0629\n",
            "I1203 18:28:21.034836 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0629\n",
            "INFO:tensorflow:examples/sec: 34.0129\n",
            "I1203 18:28:21.035084 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0129\n",
            "INFO:tensorflow:global_step/sec: 1.06347\n",
            "I1203 18:28:21.975150 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06347\n",
            "INFO:tensorflow:examples/sec: 34.0312\n",
            "I1203 18:28:21.975417 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0312\n",
            "INFO:tensorflow:global_step/sec: 1.06127\n",
            "I1203 18:28:22.917466 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06127\n",
            "INFO:tensorflow:examples/sec: 33.9606\n",
            "I1203 18:28:22.917740 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9606\n",
            "INFO:tensorflow:global_step/sec: 1.0626\n",
            "I1203 18:28:23.858567 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0626\n",
            "INFO:tensorflow:examples/sec: 34.0033\n",
            "I1203 18:28:23.858844 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0033\n",
            "INFO:tensorflow:global_step/sec: 1.05442\n",
            "I1203 18:28:24.806921 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05442\n",
            "INFO:tensorflow:examples/sec: 33.7414\n",
            "I1203 18:28:24.807186 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7414\n",
            "INFO:tensorflow:global_step/sec: 1.06398\n",
            "I1203 18:28:25.746783 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06398\n",
            "INFO:tensorflow:examples/sec: 34.0473\n",
            "I1203 18:28:25.747056 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0473\n",
            "INFO:tensorflow:global_step/sec: 1.06376\n",
            "I1203 18:28:26.686832 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06376\n",
            "INFO:tensorflow:examples/sec: 34.0403\n",
            "I1203 18:28:26.687103 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0403\n",
            "INFO:tensorflow:global_step/sec: 1.06481\n",
            "I1203 18:28:27.626000 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06481\n",
            "INFO:tensorflow:examples/sec: 34.0739\n",
            "I1203 18:28:27.626269 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0739\n",
            "INFO:tensorflow:global_step/sec: 1.05847\n",
            "I1203 18:28:28.570743 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05847\n",
            "INFO:tensorflow:examples/sec: 33.8711\n",
            "I1203 18:28:28.571046 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8711\n",
            "INFO:tensorflow:global_step/sec: 1.06506\n",
            "I1203 18:28:29.509665 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06506\n",
            "INFO:tensorflow:examples/sec: 34.0819\n",
            "I1203 18:28:29.509946 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0819\n",
            "INFO:tensorflow:global_step/sec: 1.05244\n",
            "I1203 18:28:30.459826 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05244\n",
            "INFO:tensorflow:examples/sec: 33.6781\n",
            "I1203 18:28:30.460288 140409631090560 tpu_estimator.py:2308] examples/sec: 33.6781\n",
            "INFO:tensorflow:global_step/sec: 1.0527\n",
            "I1203 18:28:31.409745 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0527\n",
            "INFO:tensorflow:examples/sec: 33.6864\n",
            "I1203 18:28:31.410255 140409631090560 tpu_estimator.py:2308] examples/sec: 33.6864\n",
            "INFO:tensorflow:global_step/sec: 1.06353\n",
            "I1203 18:28:32.350047 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06353\n",
            "INFO:tensorflow:examples/sec: 34.033\n",
            "I1203 18:28:32.350465 140409631090560 tpu_estimator.py:2308] examples/sec: 34.033\n",
            "INFO:tensorflow:global_step/sec: 1.06933\n",
            "I1203 18:28:33.285169 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06933\n",
            "INFO:tensorflow:examples/sec: 34.2185\n",
            "I1203 18:28:33.285615 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2185\n",
            "INFO:tensorflow:global_step/sec: 1.05886\n",
            "I1203 18:28:34.229650 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05886\n",
            "INFO:tensorflow:examples/sec: 33.8835\n",
            "I1203 18:28:34.230020 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8835\n",
            "INFO:tensorflow:global_step/sec: 1.06536\n",
            "I1203 18:28:35.168250 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06536\n",
            "INFO:tensorflow:examples/sec: 34.0916\n",
            "I1203 18:28:35.168548 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0916\n",
            "INFO:tensorflow:global_step/sec: 1.05882\n",
            "I1203 18:28:36.112694 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05882\n",
            "INFO:tensorflow:examples/sec: 33.8822\n",
            "I1203 18:28:36.113125 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8822\n",
            "INFO:tensorflow:global_step/sec: 1.06018\n",
            "I1203 18:28:37.055920 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06018\n",
            "INFO:tensorflow:examples/sec: 33.9259\n",
            "I1203 18:28:37.056293 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9259\n",
            "INFO:tensorflow:global_step/sec: 1.0611\n",
            "I1203 18:28:37.998361 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0611\n",
            "INFO:tensorflow:examples/sec: 33.9551\n",
            "I1203 18:28:37.998618 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9551\n",
            "INFO:tensorflow:global_step/sec: 1.06753\n",
            "I1203 18:28:38.935087 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06753\n",
            "INFO:tensorflow:examples/sec: 34.1611\n",
            "I1203 18:28:38.935318 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1611\n",
            "INFO:tensorflow:global_step/sec: 1.06074\n",
            "I1203 18:28:39.877825 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06074\n",
            "INFO:tensorflow:examples/sec: 33.9437\n",
            "I1203 18:28:39.878095 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9437\n",
            "INFO:tensorflow:global_step/sec: 1.05768\n",
            "I1203 18:28:40.823294 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05768\n",
            "INFO:tensorflow:examples/sec: 33.8457\n",
            "I1203 18:28:40.823656 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8457\n",
            "INFO:tensorflow:global_step/sec: 1.06778\n",
            "I1203 18:28:41.759798 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06778\n",
            "INFO:tensorflow:examples/sec: 34.1691\n",
            "I1203 18:28:41.760182 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1691\n",
            "INFO:tensorflow:global_step/sec: 1.06487\n",
            "I1203 18:28:42.698880 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06487\n",
            "INFO:tensorflow:examples/sec: 34.0759\n",
            "I1203 18:28:42.699279 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0759\n",
            "INFO:tensorflow:global_step/sec: 1.06634\n",
            "I1203 18:28:43.636669 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06634\n",
            "INFO:tensorflow:examples/sec: 34.1228\n",
            "I1203 18:28:43.637059 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1228\n",
            "INFO:tensorflow:global_step/sec: 1.05686\n",
            "I1203 18:28:44.582872 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05686\n",
            "INFO:tensorflow:examples/sec: 33.8195\n",
            "I1203 18:28:44.583308 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8195\n",
            "INFO:tensorflow:global_step/sec: 1.05568\n",
            "I1203 18:28:45.530125 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05568\n",
            "INFO:tensorflow:examples/sec: 33.7816\n",
            "I1203 18:28:45.530539 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7816\n",
            "INFO:tensorflow:global_step/sec: 1.06507\n",
            "I1203 18:28:46.469039 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06507\n",
            "INFO:tensorflow:examples/sec: 34.0822\n",
            "I1203 18:28:46.469435 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0822\n",
            "INFO:tensorflow:global_step/sec: 1.06236\n",
            "I1203 18:28:47.410350 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06236\n",
            "INFO:tensorflow:examples/sec: 33.9955\n",
            "I1203 18:28:47.410604 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9955\n",
            "INFO:tensorflow:global_step/sec: 1.06083\n",
            "I1203 18:28:48.353050 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06083\n",
            "INFO:tensorflow:examples/sec: 33.9465\n",
            "I1203 18:28:48.353376 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9465\n",
            "INFO:tensorflow:global_step/sec: 1.06364\n",
            "I1203 18:28:49.293187 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06364\n",
            "INFO:tensorflow:examples/sec: 34.0366\n",
            "I1203 18:28:49.293473 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0366\n",
            "INFO:tensorflow:global_step/sec: 1.06048\n",
            "I1203 18:28:50.236137 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06048\n",
            "INFO:tensorflow:examples/sec: 33.9355\n",
            "I1203 18:28:50.236369 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9355\n",
            "INFO:tensorflow:global_step/sec: 1.06605\n",
            "I1203 18:28:51.174206 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06605\n",
            "INFO:tensorflow:examples/sec: 34.1135\n",
            "I1203 18:28:51.174616 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1135\n",
            "INFO:tensorflow:global_step/sec: 1.05937\n",
            "I1203 18:28:52.118132 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05937\n",
            "INFO:tensorflow:examples/sec: 33.8997\n",
            "I1203 18:28:52.118355 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8997\n",
            "INFO:tensorflow:global_step/sec: 1.06133\n",
            "I1203 18:28:53.060356 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06133\n",
            "INFO:tensorflow:examples/sec: 33.9625\n",
            "I1203 18:28:53.060608 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9625\n",
            "INFO:tensorflow:global_step/sec: 1.06232\n",
            "I1203 18:28:54.001677 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06232\n",
            "INFO:tensorflow:examples/sec: 33.9941\n",
            "I1203 18:28:54.001901 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9941\n",
            "INFO:tensorflow:global_step/sec: 1.06812\n",
            "I1203 18:28:54.937917 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06812\n",
            "INFO:tensorflow:examples/sec: 34.1798\n",
            "I1203 18:28:54.938151 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1798\n",
            "INFO:tensorflow:global_step/sec: 1.06371\n",
            "I1203 18:28:55.878022 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06371\n",
            "INFO:tensorflow:examples/sec: 34.0389\n",
            "I1203 18:28:55.878262 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0389\n",
            "INFO:tensorflow:global_step/sec: 1.06264\n",
            "I1203 18:28:56.819075 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06264\n",
            "INFO:tensorflow:examples/sec: 34.0046\n",
            "I1203 18:28:56.819313 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0046\n",
            "INFO:tensorflow:global_step/sec: 1.06405\n",
            "I1203 18:28:57.758890 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06405\n",
            "INFO:tensorflow:examples/sec: 34.0495\n",
            "I1203 18:28:57.759136 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0495\n",
            "INFO:tensorflow:global_step/sec: 1.06113\n",
            "I1203 18:28:58.701326 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06113\n",
            "INFO:tensorflow:examples/sec: 33.9562\n",
            "I1203 18:28:58.701648 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9562\n",
            "INFO:tensorflow:global_step/sec: 1.06214\n",
            "I1203 18:28:59.642761 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06214\n",
            "INFO:tensorflow:examples/sec: 33.9884\n",
            "I1203 18:28:59.643136 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9884\n",
            "INFO:tensorflow:global_step/sec: 1.05859\n",
            "I1203 18:29:00.587432 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05859\n",
            "INFO:tensorflow:examples/sec: 33.875\n",
            "I1203 18:29:00.587800 140409631090560 tpu_estimator.py:2308] examples/sec: 33.875\n",
            "INFO:tensorflow:global_step/sec: 1.06834\n",
            "I1203 18:29:01.523468 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06834\n",
            "INFO:tensorflow:examples/sec: 34.1868\n",
            "I1203 18:29:01.524118 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1868\n",
            "INFO:tensorflow:global_step/sec: 1.05598\n",
            "I1203 18:29:02.470507 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05598\n",
            "INFO:tensorflow:examples/sec: 33.7913\n",
            "I1203 18:29:02.470893 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7913\n",
            "INFO:tensorflow:global_step/sec: 1.06929\n",
            "I1203 18:29:03.405645 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06929\n",
            "INFO:tensorflow:examples/sec: 34.2174\n",
            "I1203 18:29:03.406043 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2174\n",
            "INFO:tensorflow:global_step/sec: 1.06061\n",
            "I1203 18:29:04.348526 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06061\n",
            "INFO:tensorflow:examples/sec: 33.9394\n",
            "I1203 18:29:04.348754 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9394\n",
            "INFO:tensorflow:global_step/sec: 1.06358\n",
            "I1203 18:29:05.288701 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06358\n",
            "INFO:tensorflow:examples/sec: 34.0347\n",
            "I1203 18:29:05.288935 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0347\n",
            "INFO:tensorflow:global_step/sec: 1.06414\n",
            "I1203 18:29:06.228457 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06414\n",
            "INFO:tensorflow:examples/sec: 34.0525\n",
            "I1203 18:29:06.228855 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0525\n",
            "INFO:tensorflow:global_step/sec: 1.06331\n",
            "I1203 18:29:07.168892 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06331\n",
            "INFO:tensorflow:examples/sec: 34.026\n",
            "I1203 18:29:07.169281 140409631090560 tpu_estimator.py:2308] examples/sec: 34.026\n",
            "INFO:tensorflow:global_step/sec: 1.06997\n",
            "I1203 18:29:08.103511 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06997\n",
            "INFO:tensorflow:examples/sec: 34.2391\n",
            "I1203 18:29:08.103929 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2391\n",
            "INFO:tensorflow:global_step/sec: 1.05959\n",
            "I1203 18:29:09.047264 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05959\n",
            "INFO:tensorflow:examples/sec: 33.9068\n",
            "I1203 18:29:09.047649 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9068\n",
            "INFO:tensorflow:global_step/sec: 1.06223\n",
            "I1203 18:29:09.988705 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06223\n",
            "INFO:tensorflow:examples/sec: 33.9913\n",
            "I1203 18:29:09.989165 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9913\n",
            "INFO:tensorflow:global_step/sec: 1.05746\n",
            "I1203 18:29:10.934328 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05746\n",
            "INFO:tensorflow:examples/sec: 33.8387\n",
            "I1203 18:29:10.934717 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8387\n",
            "INFO:tensorflow:global_step/sec: 1.06916\n",
            "I1203 18:29:11.869646 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06916\n",
            "INFO:tensorflow:examples/sec: 34.2132\n",
            "I1203 18:29:11.870019 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2132\n",
            "INFO:tensorflow:global_step/sec: 1.06263\n",
            "I1203 18:29:12.810698 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06263\n",
            "INFO:tensorflow:examples/sec: 34.0043\n",
            "I1203 18:29:12.811061 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0043\n",
            "INFO:tensorflow:global_step/sec: 1.06748\n",
            "I1203 18:29:13.747512 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06748\n",
            "INFO:tensorflow:examples/sec: 34.1592\n",
            "I1203 18:29:13.747758 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1592\n",
            "INFO:tensorflow:global_step/sec: 1.06155\n",
            "I1203 18:29:14.689542 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06155\n",
            "INFO:tensorflow:examples/sec: 33.9697\n",
            "I1203 18:29:14.689908 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9697\n",
            "INFO:tensorflow:global_step/sec: 1.05647\n",
            "I1203 18:29:15.636124 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05647\n",
            "INFO:tensorflow:examples/sec: 33.807\n",
            "I1203 18:29:15.636520 140409631090560 tpu_estimator.py:2308] examples/sec: 33.807\n",
            "INFO:tensorflow:global_step/sec: 1.06485\n",
            "I1203 18:29:16.575149 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06485\n",
            "INFO:tensorflow:examples/sec: 34.0752\n",
            "I1203 18:29:16.575504 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0752\n",
            "INFO:tensorflow:global_step/sec: 1.06606\n",
            "I1203 18:29:17.513207 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06606\n",
            "INFO:tensorflow:examples/sec: 34.114\n",
            "I1203 18:29:17.513600 140409631090560 tpu_estimator.py:2308] examples/sec: 34.114\n",
            "INFO:tensorflow:global_step/sec: 1.06558\n",
            "I1203 18:29:18.451641 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06558\n",
            "INFO:tensorflow:examples/sec: 34.0986\n",
            "I1203 18:29:18.451891 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0986\n",
            "INFO:tensorflow:global_step/sec: 1.06542\n",
            "I1203 18:29:19.390243 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06542\n",
            "INFO:tensorflow:examples/sec: 34.0934\n",
            "I1203 18:29:19.390633 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0934\n",
            "INFO:tensorflow:global_step/sec: 1.06598\n",
            "I1203 18:29:20.328370 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06598\n",
            "INFO:tensorflow:examples/sec: 34.1112\n",
            "I1203 18:29:20.328802 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1112\n",
            "INFO:tensorflow:global_step/sec: 1.05975\n",
            "I1203 18:29:21.272062 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05975\n",
            "INFO:tensorflow:examples/sec: 33.912\n",
            "I1203 18:29:21.272517 140409631090560 tpu_estimator.py:2308] examples/sec: 33.912\n",
            "INFO:tensorflow:global_step/sec: 1.06153\n",
            "I1203 18:29:22.214019 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06153\n",
            "INFO:tensorflow:examples/sec: 33.9689\n",
            "I1203 18:29:22.214384 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9689\n",
            "INFO:tensorflow:global_step/sec: 1.06265\n",
            "I1203 18:29:23.155050 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06265\n",
            "INFO:tensorflow:examples/sec: 34.0047\n",
            "I1203 18:29:23.155441 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0047\n",
            "INFO:tensorflow:global_step/sec: 1.06011\n",
            "I1203 18:29:24.098340 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06011\n",
            "INFO:tensorflow:examples/sec: 33.9236\n",
            "I1203 18:29:24.098719 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9236\n",
            "INFO:tensorflow:global_step/sec: 1.06592\n",
            "I1203 18:29:25.036548 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06592\n",
            "INFO:tensorflow:examples/sec: 34.1093\n",
            "I1203 18:29:25.036921 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1093\n",
            "INFO:tensorflow:global_step/sec: 1.06591\n",
            "I1203 18:29:25.974667 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06591\n",
            "INFO:tensorflow:examples/sec: 34.1091\n",
            "I1203 18:29:25.975026 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1091\n",
            "INFO:tensorflow:global_step/sec: 1.05989\n",
            "I1203 18:29:26.918157 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05989\n",
            "INFO:tensorflow:examples/sec: 33.9165\n",
            "I1203 18:29:26.918537 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9165\n",
            "INFO:tensorflow:global_step/sec: 1.06218\n",
            "I1203 18:29:27.859635 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06218\n",
            "INFO:tensorflow:examples/sec: 33.9898\n",
            "I1203 18:29:27.859874 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9898\n",
            "INFO:tensorflow:global_step/sec: 1.06262\n",
            "I1203 18:29:28.800703 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06262\n",
            "INFO:tensorflow:examples/sec: 34.0039\n",
            "I1203 18:29:28.801051 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0039\n",
            "INFO:tensorflow:global_step/sec: 1.06185\n",
            "I1203 18:29:29.742490 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06185\n",
            "INFO:tensorflow:examples/sec: 33.9791\n",
            "I1203 18:29:29.742737 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9791\n",
            "INFO:tensorflow:global_step/sec: 1.06203\n",
            "I1203 18:29:30.684062 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06203\n",
            "INFO:tensorflow:examples/sec: 33.985\n",
            "I1203 18:29:30.684495 140409631090560 tpu_estimator.py:2308] examples/sec: 33.985\n",
            "INFO:tensorflow:global_step/sec: 1.0634\n",
            "I1203 18:29:31.624447 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0634\n",
            "INFO:tensorflow:examples/sec: 34.0288\n",
            "I1203 18:29:31.624818 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0288\n",
            "INFO:tensorflow:global_step/sec: 1.06221\n",
            "I1203 18:29:32.565857 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06221\n",
            "INFO:tensorflow:examples/sec: 33.9907\n",
            "I1203 18:29:32.566282 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9907\n",
            "INFO:tensorflow:global_step/sec: 1.06631\n",
            "I1203 18:29:33.503618 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06631\n",
            "INFO:tensorflow:examples/sec: 34.1221\n",
            "I1203 18:29:33.503871 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1221\n",
            "INFO:tensorflow:global_step/sec: 1.06757\n",
            "I1203 18:29:34.440364 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06757\n",
            "INFO:tensorflow:examples/sec: 34.1624\n",
            "I1203 18:29:34.440742 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1624\n",
            "INFO:tensorflow:global_step/sec: 1.05779\n",
            "I1203 18:29:35.385734 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05779\n",
            "INFO:tensorflow:examples/sec: 33.8492\n",
            "I1203 18:29:35.385965 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8492\n",
            "INFO:tensorflow:global_step/sec: 1.06697\n",
            "I1203 18:29:36.323002 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06697\n",
            "INFO:tensorflow:examples/sec: 34.143\n",
            "I1203 18:29:36.323362 140409631090560 tpu_estimator.py:2308] examples/sec: 34.143\n",
            "INFO:tensorflow:global_step/sec: 1.06117\n",
            "I1203 18:29:37.265316 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06117\n",
            "INFO:tensorflow:examples/sec: 33.9576\n",
            "I1203 18:29:37.265565 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9576\n",
            "INFO:tensorflow:global_step/sec: 1.06545\n",
            "I1203 18:29:38.203898 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06545\n",
            "INFO:tensorflow:examples/sec: 34.0944\n",
            "I1203 18:29:38.204338 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0944\n",
            "INFO:tensorflow:global_step/sec: 1.05528\n",
            "I1203 18:29:39.151530 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05528\n",
            "INFO:tensorflow:examples/sec: 33.769\n",
            "I1203 18:29:39.151890 140409631090560 tpu_estimator.py:2308] examples/sec: 33.769\n",
            "INFO:tensorflow:global_step/sec: 1.0632\n",
            "I1203 18:29:40.092072 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0632\n",
            "INFO:tensorflow:examples/sec: 34.0223\n",
            "I1203 18:29:40.092455 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0223\n",
            "INFO:tensorflow:global_step/sec: 1.06475\n",
            "I1203 18:29:41.031268 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06475\n",
            "INFO:tensorflow:examples/sec: 34.072\n",
            "I1203 18:29:41.031518 140409631090560 tpu_estimator.py:2308] examples/sec: 34.072\n",
            "INFO:tensorflow:global_step/sec: 1.0554\n",
            "I1203 18:29:41.978770 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0554\n",
            "INFO:tensorflow:examples/sec: 33.7729\n",
            "I1203 18:29:41.979159 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7729\n",
            "INFO:tensorflow:global_step/sec: 1.06771\n",
            "I1203 18:29:42.915363 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06771\n",
            "INFO:tensorflow:examples/sec: 34.1666\n",
            "I1203 18:29:42.915743 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1666\n",
            "INFO:tensorflow:global_step/sec: 1.06157\n",
            "I1203 18:29:43.857384 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06157\n",
            "INFO:tensorflow:examples/sec: 33.9703\n",
            "I1203 18:29:43.857814 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9703\n",
            "INFO:tensorflow:global_step/sec: 1.06708\n",
            "I1203 18:29:44.794522 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06708\n",
            "INFO:tensorflow:examples/sec: 34.1465\n",
            "I1203 18:29:44.794875 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1465\n",
            "INFO:tensorflow:global_step/sec: 1.05992\n",
            "I1203 18:29:45.737958 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05992\n",
            "INFO:tensorflow:examples/sec: 33.9173\n",
            "I1203 18:29:45.738329 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9173\n",
            "INFO:tensorflow:global_step/sec: 1.06657\n",
            "I1203 18:29:46.675558 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06657\n",
            "INFO:tensorflow:examples/sec: 34.1303\n",
            "I1203 18:29:46.675914 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1303\n",
            "INFO:tensorflow:global_step/sec: 1.06083\n",
            "I1203 18:29:47.618201 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06083\n",
            "INFO:tensorflow:examples/sec: 33.9465\n",
            "I1203 18:29:47.618499 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9465\n",
            "INFO:tensorflow:global_step/sec: 1.06642\n",
            "I1203 18:29:48.555939 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06642\n",
            "INFO:tensorflow:examples/sec: 34.1253\n",
            "I1203 18:29:48.556169 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1253\n",
            "INFO:tensorflow:global_step/sec: 1.06082\n",
            "I1203 18:29:49.498616 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06082\n",
            "INFO:tensorflow:examples/sec: 33.9461\n",
            "I1203 18:29:49.498849 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9461\n",
            "INFO:tensorflow:global_step/sec: 1.06223\n",
            "I1203 18:29:50.440054 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06223\n",
            "INFO:tensorflow:examples/sec: 33.9913\n",
            "I1203 18:29:50.440472 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9913\n",
            "INFO:tensorflow:global_step/sec: 1.06074\n",
            "I1203 18:29:51.382761 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06074\n",
            "INFO:tensorflow:examples/sec: 33.9436\n",
            "I1203 18:29:51.383167 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9436\n",
            "INFO:tensorflow:global_step/sec: 1.06416\n",
            "I1203 18:29:52.322487 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06416\n",
            "INFO:tensorflow:examples/sec: 34.0533\n",
            "I1203 18:29:52.322869 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0533\n",
            "INFO:tensorflow:global_step/sec: 1.06068\n",
            "I1203 18:29:53.265239 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06068\n",
            "INFO:tensorflow:examples/sec: 33.9417\n",
            "I1203 18:29:53.265527 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9417\n",
            "INFO:tensorflow:global_step/sec: 1.05422\n",
            "I1203 18:29:54.213862 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05422\n",
            "INFO:tensorflow:examples/sec: 33.735\n",
            "I1203 18:29:54.214650 140409631090560 tpu_estimator.py:2308] examples/sec: 33.735\n",
            "INFO:tensorflow:global_step/sec: 1.05814\n",
            "I1203 18:29:55.158882 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05814\n",
            "INFO:tensorflow:examples/sec: 33.8604\n",
            "I1203 18:29:55.159128 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8604\n",
            "INFO:tensorflow:global_step/sec: 1.05916\n",
            "I1203 18:29:56.103022 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05916\n",
            "INFO:tensorflow:examples/sec: 33.8933\n",
            "I1203 18:29:56.103273 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8933\n",
            "INFO:tensorflow:global_step/sec: 1.06023\n",
            "I1203 18:29:57.046223 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06023\n",
            "INFO:tensorflow:examples/sec: 33.9275\n",
            "I1203 18:29:57.046487 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9275\n",
            "INFO:tensorflow:global_step/sec: 1.06701\n",
            "I1203 18:29:57.983414 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06701\n",
            "INFO:tensorflow:examples/sec: 34.1443\n",
            "I1203 18:29:57.983676 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1443\n",
            "INFO:tensorflow:global_step/sec: 1.06434\n",
            "I1203 18:29:58.922965 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06434\n",
            "INFO:tensorflow:examples/sec: 34.0589\n",
            "I1203 18:29:58.923198 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0589\n",
            "INFO:tensorflow:global_step/sec: 1.06367\n",
            "I1203 18:29:59.863127 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06367\n",
            "INFO:tensorflow:examples/sec: 34.0374\n",
            "I1203 18:29:59.863365 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0374\n",
            "INFO:tensorflow:global_step/sec: 1.06082\n",
            "I1203 18:30:00.805774 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06082\n",
            "INFO:tensorflow:examples/sec: 33.9463\n",
            "I1203 18:30:00.806020 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9463\n",
            "INFO:tensorflow:global_step/sec: 1.05844\n",
            "I1203 18:30:01.750594 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05844\n",
            "INFO:tensorflow:examples/sec: 33.87\n",
            "I1203 18:30:01.750844 140409631090560 tpu_estimator.py:2308] examples/sec: 33.87\n",
            "INFO:tensorflow:global_step/sec: 1.06079\n",
            "I1203 18:30:02.693248 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06079\n",
            "INFO:tensorflow:examples/sec: 33.9454\n",
            "I1203 18:30:02.693678 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9454\n",
            "INFO:tensorflow:global_step/sec: 1.0613\n",
            "I1203 18:30:03.635534 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0613\n",
            "INFO:tensorflow:examples/sec: 33.9616\n",
            "I1203 18:30:03.635935 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9616\n",
            "INFO:tensorflow:global_step/sec: 1.06444\n",
            "I1203 18:30:04.574949 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06444\n",
            "INFO:tensorflow:examples/sec: 34.0621\n",
            "I1203 18:30:04.575374 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0621\n",
            "INFO:tensorflow:global_step/sec: 1.06638\n",
            "I1203 18:30:05.512691 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06638\n",
            "INFO:tensorflow:examples/sec: 34.1241\n",
            "I1203 18:30:05.512924 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1241\n",
            "INFO:tensorflow:global_step/sec: 1.06362\n",
            "I1203 18:30:06.452884 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06362\n",
            "INFO:tensorflow:examples/sec: 34.036\n",
            "I1203 18:30:06.453113 140409631090560 tpu_estimator.py:2308] examples/sec: 34.036\n",
            "INFO:tensorflow:global_step/sec: 1.06438\n",
            "I1203 18:30:07.392419 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06438\n",
            "INFO:tensorflow:examples/sec: 34.0602\n",
            "I1203 18:30:07.392806 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0602\n",
            "INFO:tensorflow:global_step/sec: 1.06274\n",
            "I1203 18:30:08.333352 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06274\n",
            "INFO:tensorflow:examples/sec: 34.0076\n",
            "I1203 18:30:08.333606 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0076\n",
            "INFO:tensorflow:global_step/sec: 1.06484\n",
            "I1203 18:30:09.272504 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06484\n",
            "INFO:tensorflow:examples/sec: 34.0748\n",
            "I1203 18:30:09.272746 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0748\n",
            "INFO:tensorflow:global_step/sec: 1.06148\n",
            "I1203 18:30:10.214564 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06148\n",
            "INFO:tensorflow:examples/sec: 33.9673\n",
            "I1203 18:30:10.214790 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9673\n",
            "INFO:tensorflow:global_step/sec: 1.06773\n",
            "I1203 18:30:11.151133 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06773\n",
            "INFO:tensorflow:examples/sec: 34.1674\n",
            "I1203 18:30:11.151361 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1674\n",
            "INFO:tensorflow:global_step/sec: 1.06731\n",
            "I1203 18:30:12.088044 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06731\n",
            "INFO:tensorflow:examples/sec: 34.1539\n",
            "I1203 18:30:12.088419 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1539\n",
            "INFO:tensorflow:global_step/sec: 1.06696\n",
            "I1203 18:30:13.025300 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06696\n",
            "INFO:tensorflow:examples/sec: 34.1427\n",
            "I1203 18:30:13.025665 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1427\n",
            "INFO:tensorflow:global_step/sec: 1.06001\n",
            "I1203 18:30:13.968675 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06001\n",
            "INFO:tensorflow:examples/sec: 33.9205\n",
            "I1203 18:30:13.969060 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9205\n",
            "INFO:tensorflow:global_step/sec: 1.05653\n",
            "I1203 18:30:14.915176 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05653\n",
            "INFO:tensorflow:examples/sec: 33.809\n",
            "I1203 18:30:14.915437 140409631090560 tpu_estimator.py:2308] examples/sec: 33.809\n",
            "INFO:tensorflow:global_step/sec: 1.04758\n",
            "I1203 18:30:15.869800 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.04758\n",
            "INFO:tensorflow:examples/sec: 33.5225\n",
            "I1203 18:30:15.870043 140409631090560 tpu_estimator.py:2308] examples/sec: 33.5225\n",
            "INFO:tensorflow:global_step/sec: 1.07179\n",
            "I1203 18:30:16.802800 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07179\n",
            "INFO:tensorflow:examples/sec: 34.2972\n",
            "I1203 18:30:16.803074 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2972\n",
            "INFO:tensorflow:global_step/sec: 1.06541\n",
            "I1203 18:30:17.741374 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06541\n",
            "INFO:tensorflow:examples/sec: 34.0931\n",
            "I1203 18:30:17.741628 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0931\n",
            "INFO:tensorflow:global_step/sec: 1.06531\n",
            "I1203 18:30:18.680072 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06531\n",
            "INFO:tensorflow:examples/sec: 34.0899\n",
            "I1203 18:30:18.680315 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0899\n",
            "INFO:tensorflow:global_step/sec: 1.06092\n",
            "I1203 18:30:19.622653 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06092\n",
            "INFO:tensorflow:examples/sec: 33.9495\n",
            "I1203 18:30:19.623009 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9495\n",
            "INFO:tensorflow:global_step/sec: 1.06478\n",
            "I1203 18:30:20.561808 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06478\n",
            "INFO:tensorflow:examples/sec: 34.0731\n",
            "I1203 18:30:20.562217 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0731\n",
            "INFO:tensorflow:global_step/sec: 1.06433\n",
            "I1203 18:30:21.501410 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06433\n",
            "INFO:tensorflow:examples/sec: 34.0586\n",
            "I1203 18:30:21.501641 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0586\n",
            "INFO:tensorflow:global_step/sec: 1.06754\n",
            "I1203 18:30:22.438102 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06754\n",
            "INFO:tensorflow:examples/sec: 34.1612\n",
            "I1203 18:30:22.438319 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1612\n",
            "INFO:tensorflow:global_step/sec: 1.05976\n",
            "I1203 18:30:23.381721 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05976\n",
            "INFO:tensorflow:examples/sec: 33.9123\n",
            "I1203 18:30:23.382135 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9123\n",
            "INFO:tensorflow:global_step/sec: 1.06455\n",
            "I1203 18:30:24.321066 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06455\n",
            "INFO:tensorflow:examples/sec: 34.0657\n",
            "I1203 18:30:24.321442 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0657\n",
            "INFO:tensorflow:global_step/sec: 1.06668\n",
            "I1203 18:30:25.258588 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06668\n",
            "INFO:tensorflow:examples/sec: 34.1339\n",
            "I1203 18:30:25.258967 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1339\n",
            "INFO:tensorflow:global_step/sec: 1.0665\n",
            "I1203 18:30:26.196522 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0665\n",
            "INFO:tensorflow:examples/sec: 34.1281\n",
            "I1203 18:30:26.196884 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1281\n",
            "INFO:tensorflow:global_step/sec: 1.06217\n",
            "I1203 18:30:27.137688 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06217\n",
            "INFO:tensorflow:examples/sec: 33.9894\n",
            "I1203 18:30:27.138043 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9894\n",
            "INFO:tensorflow:global_step/sec: 1.06538\n",
            "I1203 18:30:28.076311 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06538\n",
            "INFO:tensorflow:examples/sec: 34.0922\n",
            "I1203 18:30:28.076676 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0922\n",
            "INFO:tensorflow:global_step/sec: 1.06797\n",
            "I1203 18:30:29.012675 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06797\n",
            "INFO:tensorflow:examples/sec: 34.1749\n",
            "I1203 18:30:29.013047 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1749\n",
            "INFO:tensorflow:global_step/sec: 1.06833\n",
            "I1203 18:30:29.948701 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06833\n",
            "INFO:tensorflow:examples/sec: 34.1867\n",
            "I1203 18:30:29.949087 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1867\n",
            "INFO:tensorflow:global_step/sec: 1.06924\n",
            "I1203 18:30:30.883947 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06924\n",
            "INFO:tensorflow:examples/sec: 34.2158\n",
            "I1203 18:30:30.884170 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2158\n",
            "INFO:tensorflow:global_step/sec: 1.06253\n",
            "I1203 18:30:31.825115 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06253\n",
            "INFO:tensorflow:examples/sec: 34.0009\n",
            "I1203 18:30:31.825551 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0009\n",
            "INFO:tensorflow:global_step/sec: 1.06301\n",
            "I1203 18:30:32.765821 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06301\n",
            "INFO:tensorflow:examples/sec: 34.0162\n",
            "I1203 18:30:32.766062 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0162\n",
            "INFO:tensorflow:global_step/sec: 1.05801\n",
            "I1203 18:30:33.711022 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05801\n",
            "INFO:tensorflow:examples/sec: 33.8564\n",
            "I1203 18:30:33.711314 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8564\n",
            "INFO:tensorflow:global_step/sec: 1.06082\n",
            "I1203 18:30:34.653659 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06082\n",
            "INFO:tensorflow:examples/sec: 33.9462\n",
            "I1203 18:30:34.654007 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9462\n",
            "INFO:tensorflow:global_step/sec: 1.06518\n",
            "I1203 18:30:35.592478 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06518\n",
            "INFO:tensorflow:examples/sec: 34.0857\n",
            "I1203 18:30:35.592842 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0857\n",
            "INFO:tensorflow:global_step/sec: 1.06613\n",
            "I1203 18:30:36.530465 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06613\n",
            "INFO:tensorflow:examples/sec: 34.1162\n",
            "I1203 18:30:36.530833 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1162\n",
            "INFO:tensorflow:global_step/sec: 1.06714\n",
            "I1203 18:30:37.467546 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06714\n",
            "INFO:tensorflow:examples/sec: 34.1486\n",
            "I1203 18:30:37.467819 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1486\n",
            "INFO:tensorflow:global_step/sec: 1.06439\n",
            "I1203 18:30:38.407014 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06439\n",
            "INFO:tensorflow:examples/sec: 34.0604\n",
            "I1203 18:30:38.407373 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0604\n",
            "INFO:tensorflow:global_step/sec: 1.06505\n",
            "I1203 18:30:39.345935 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06505\n",
            "INFO:tensorflow:examples/sec: 34.0816\n",
            "I1203 18:30:39.346289 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0816\n",
            "INFO:tensorflow:global_step/sec: 1.06716\n",
            "I1203 18:30:40.283022 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06716\n",
            "INFO:tensorflow:examples/sec: 34.1493\n",
            "I1203 18:30:40.283392 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1493\n",
            "INFO:tensorflow:global_step/sec: 1.06479\n",
            "I1203 18:30:41.222163 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06479\n",
            "INFO:tensorflow:examples/sec: 34.0731\n",
            "I1203 18:30:41.222642 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0731\n",
            "INFO:tensorflow:global_step/sec: 1.06498\n",
            "I1203 18:30:42.161141 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06498\n",
            "INFO:tensorflow:examples/sec: 34.0795\n",
            "I1203 18:30:42.161514 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0795\n",
            "INFO:tensorflow:global_step/sec: 1.06463\n",
            "I1203 18:30:43.100452 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06463\n",
            "INFO:tensorflow:examples/sec: 34.0682\n",
            "I1203 18:30:43.100803 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0682\n",
            "INFO:tensorflow:global_step/sec: 1.06654\n",
            "I1203 18:30:44.038069 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06654\n",
            "INFO:tensorflow:examples/sec: 34.1293\n",
            "I1203 18:30:44.038451 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1293\n",
            "INFO:tensorflow:global_step/sec: 1.06458\n",
            "I1203 18:30:44.977380 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06458\n",
            "INFO:tensorflow:examples/sec: 34.0665\n",
            "I1203 18:30:44.977771 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0665\n",
            "INFO:tensorflow:global_step/sec: 1.06512\n",
            "I1203 18:30:45.916302 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06512\n",
            "INFO:tensorflow:examples/sec: 34.0838\n",
            "I1203 18:30:45.917915 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0838\n",
            "INFO:tensorflow:global_step/sec: 1.06145\n",
            "I1203 18:30:46.858359 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06145\n",
            "INFO:tensorflow:examples/sec: 33.9665\n",
            "I1203 18:30:46.858617 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9665\n",
            "INFO:tensorflow:global_step/sec: 1.06353\n",
            "I1203 18:30:47.798627 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06353\n",
            "INFO:tensorflow:examples/sec: 34.033\n",
            "I1203 18:30:47.799006 140409631090560 tpu_estimator.py:2308] examples/sec: 34.033\n",
            "INFO:tensorflow:global_step/sec: 1.06573\n",
            "I1203 18:30:48.736952 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06573\n",
            "INFO:tensorflow:examples/sec: 34.1034\n",
            "I1203 18:30:48.737182 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1034\n",
            "INFO:tensorflow:global_step/sec: 1.06547\n",
            "I1203 18:30:49.675517 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06547\n",
            "INFO:tensorflow:examples/sec: 34.0949\n",
            "I1203 18:30:49.675891 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0949\n",
            "INFO:tensorflow:global_step/sec: 1.06528\n",
            "I1203 18:30:50.614218 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06528\n",
            "INFO:tensorflow:examples/sec: 34.0889\n",
            "I1203 18:30:50.614611 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0889\n",
            "INFO:tensorflow:global_step/sec: 1.06492\n",
            "I1203 18:30:51.553285 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06492\n",
            "INFO:tensorflow:examples/sec: 34.0773\n",
            "I1203 18:30:51.553704 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0773\n",
            "INFO:tensorflow:global_step/sec: 1.06337\n",
            "I1203 18:30:52.493674 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06337\n",
            "INFO:tensorflow:examples/sec: 34.0277\n",
            "I1203 18:30:52.494085 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0277\n",
            "INFO:tensorflow:global_step/sec: 1.06057\n",
            "I1203 18:30:53.436574 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06057\n",
            "INFO:tensorflow:examples/sec: 33.9382\n",
            "I1203 18:30:53.436942 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9382\n",
            "INFO:tensorflow:global_step/sec: 1.06125\n",
            "I1203 18:30:54.378844 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06125\n",
            "INFO:tensorflow:examples/sec: 33.9602\n",
            "I1203 18:30:54.379216 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9602\n",
            "INFO:tensorflow:global_step/sec: 1.0657\n",
            "I1203 18:30:55.317207 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0657\n",
            "INFO:tensorflow:examples/sec: 34.1025\n",
            "I1203 18:30:55.317652 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1025\n",
            "INFO:tensorflow:global_step/sec: 1.06827\n",
            "I1203 18:30:56.253295 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06827\n",
            "INFO:tensorflow:examples/sec: 34.1846\n",
            "I1203 18:30:56.253791 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1846\n",
            "INFO:tensorflow:global_step/sec: 1.06448\n",
            "I1203 18:30:57.192697 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06448\n",
            "INFO:tensorflow:examples/sec: 34.0633\n",
            "I1203 18:30:57.193073 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0633\n",
            "INFO:tensorflow:global_step/sec: 1.06632\n",
            "I1203 18:30:58.130546 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06632\n",
            "INFO:tensorflow:examples/sec: 34.1222\n",
            "I1203 18:30:58.130923 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1222\n",
            "INFO:tensorflow:global_step/sec: 1.06485\n",
            "I1203 18:30:59.069633 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06485\n",
            "INFO:tensorflow:examples/sec: 34.0751\n",
            "I1203 18:30:59.070041 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0751\n",
            "INFO:tensorflow:global_step/sec: 1.06716\n",
            "I1203 18:31:00.006679 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06716\n",
            "INFO:tensorflow:examples/sec: 34.1491\n",
            "I1203 18:31:00.007038 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1491\n",
            "INFO:tensorflow:global_step/sec: 1.06416\n",
            "I1203 18:31:00.946444 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06416\n",
            "INFO:tensorflow:examples/sec: 34.0531\n",
            "I1203 18:31:00.946830 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0531\n",
            "INFO:tensorflow:global_step/sec: 1.06522\n",
            "I1203 18:31:01.885198 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06522\n",
            "INFO:tensorflow:examples/sec: 34.087\n",
            "I1203 18:31:01.885598 140409631090560 tpu_estimator.py:2308] examples/sec: 34.087\n",
            "INFO:tensorflow:global_step/sec: 1.06845\n",
            "I1203 18:31:02.821114 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06845\n",
            "INFO:tensorflow:examples/sec: 34.1904\n",
            "I1203 18:31:02.821348 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1904\n",
            "INFO:tensorflow:global_step/sec: 1.06286\n",
            "I1203 18:31:03.761956 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06286\n",
            "INFO:tensorflow:examples/sec: 34.0117\n",
            "I1203 18:31:03.762190 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0117\n",
            "INFO:tensorflow:global_step/sec: 1.06447\n",
            "I1203 18:31:04.701382 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06447\n",
            "INFO:tensorflow:examples/sec: 34.0631\n",
            "I1203 18:31:04.701747 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0631\n",
            "INFO:tensorflow:global_step/sec: 1.0654\n",
            "I1203 18:31:05.640039 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0654\n",
            "INFO:tensorflow:examples/sec: 34.0929\n",
            "I1203 18:31:05.640274 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0929\n",
            "INFO:tensorflow:global_step/sec: 1.06793\n",
            "I1203 18:31:06.576350 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06793\n",
            "INFO:tensorflow:examples/sec: 34.1736\n",
            "I1203 18:31:06.576607 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1736\n",
            "INFO:tensorflow:global_step/sec: 1.0632\n",
            "I1203 18:31:07.516978 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0632\n",
            "INFO:tensorflow:examples/sec: 34.0223\n",
            "I1203 18:31:07.517388 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0223\n",
            "INFO:tensorflow:global_step/sec: 1.06299\n",
            "I1203 18:31:08.457688 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06299\n",
            "INFO:tensorflow:examples/sec: 34.0157\n",
            "I1203 18:31:08.458091 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0157\n",
            "INFO:tensorflow:global_step/sec: 1.05571\n",
            "I1203 18:31:09.404918 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05571\n",
            "INFO:tensorflow:examples/sec: 33.7827\n",
            "I1203 18:31:09.405144 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7827\n",
            "INFO:tensorflow:global_step/sec: 1.06913\n",
            "I1203 18:31:10.340319 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06913\n",
            "INFO:tensorflow:examples/sec: 34.2123\n",
            "I1203 18:31:10.340751 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2123\n",
            "INFO:tensorflow:global_step/sec: 1.06038\n",
            "I1203 18:31:11.283320 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06038\n",
            "INFO:tensorflow:examples/sec: 33.9322\n",
            "I1203 18:31:11.283785 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9322\n",
            "INFO:tensorflow:global_step/sec: 1.06314\n",
            "I1203 18:31:12.223973 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06314\n",
            "INFO:tensorflow:examples/sec: 34.0204\n",
            "I1203 18:31:12.224593 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0204\n",
            "INFO:tensorflow:global_step/sec: 1.05835\n",
            "I1203 18:31:13.168792 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05835\n",
            "INFO:tensorflow:examples/sec: 33.8671\n",
            "I1203 18:31:13.169048 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8671\n",
            "INFO:tensorflow:global_step/sec: 1.06516\n",
            "I1203 18:31:14.107623 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06516\n",
            "INFO:tensorflow:examples/sec: 34.0851\n",
            "I1203 18:31:14.107844 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0851\n",
            "INFO:tensorflow:global_step/sec: 1.05889\n",
            "I1203 18:31:15.052053 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05889\n",
            "INFO:tensorflow:examples/sec: 33.8846\n",
            "I1203 18:31:15.052301 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8846\n",
            "INFO:tensorflow:global_step/sec: 1.06636\n",
            "I1203 18:31:15.989780 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06636\n",
            "INFO:tensorflow:examples/sec: 34.1234\n",
            "I1203 18:31:15.990008 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1234\n",
            "INFO:tensorflow:global_step/sec: 1.05862\n",
            "I1203 18:31:16.934473 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05862\n",
            "INFO:tensorflow:examples/sec: 33.8757\n",
            "I1203 18:31:16.934846 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8757\n",
            "INFO:tensorflow:global_step/sec: 1.06691\n",
            "I1203 18:31:17.871695 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06691\n",
            "INFO:tensorflow:examples/sec: 34.141\n",
            "I1203 18:31:17.872110 140409631090560 tpu_estimator.py:2308] examples/sec: 34.141\n",
            "INFO:tensorflow:global_step/sec: 1.05906\n",
            "I1203 18:31:18.815947 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05906\n",
            "INFO:tensorflow:examples/sec: 33.8901\n",
            "I1203 18:31:18.816358 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8901\n",
            "INFO:tensorflow:global_step/sec: 1.06609\n",
            "I1203 18:31:19.753939 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06609\n",
            "INFO:tensorflow:examples/sec: 34.1149\n",
            "I1203 18:31:19.754340 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1149\n",
            "INFO:tensorflow:global_step/sec: 1.06327\n",
            "I1203 18:31:20.694453 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06327\n",
            "INFO:tensorflow:examples/sec: 34.0248\n",
            "I1203 18:31:20.694950 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0248\n",
            "INFO:tensorflow:global_step/sec: 1.06496\n",
            "I1203 18:31:21.633439 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06496\n",
            "INFO:tensorflow:examples/sec: 34.0788\n",
            "I1203 18:31:21.633697 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0788\n",
            "INFO:tensorflow:global_step/sec: 1.07084\n",
            "I1203 18:31:22.567277 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07084\n",
            "INFO:tensorflow:examples/sec: 34.2669\n",
            "I1203 18:31:22.567677 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2669\n",
            "INFO:tensorflow:global_step/sec: 1.05825\n",
            "I1203 18:31:23.512273 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05825\n",
            "INFO:tensorflow:examples/sec: 33.8641\n",
            "I1203 18:31:23.512550 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8641\n",
            "INFO:tensorflow:global_step/sec: 1.05716\n",
            "I1203 18:31:24.458182 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05716\n",
            "INFO:tensorflow:examples/sec: 33.829\n",
            "I1203 18:31:24.458436 140409631090560 tpu_estimator.py:2308] examples/sec: 33.829\n",
            "INFO:tensorflow:global_step/sec: 1.07232\n",
            "I1203 18:31:25.390730 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07232\n",
            "INFO:tensorflow:examples/sec: 34.3143\n",
            "I1203 18:31:25.391101 140409631090560 tpu_estimator.py:2308] examples/sec: 34.3143\n",
            "INFO:tensorflow:global_step/sec: 1.06041\n",
            "I1203 18:31:26.333745 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06041\n",
            "INFO:tensorflow:examples/sec: 33.933\n",
            "I1203 18:31:26.334149 140409631090560 tpu_estimator.py:2308] examples/sec: 33.933\n",
            "INFO:tensorflow:global_step/sec: 1.05778\n",
            "I1203 18:31:27.279131 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05778\n",
            "INFO:tensorflow:examples/sec: 33.8489\n",
            "I1203 18:31:27.279512 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8489\n",
            "INFO:tensorflow:global_step/sec: 1.06432\n",
            "I1203 18:31:28.218708 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06432\n",
            "INFO:tensorflow:examples/sec: 34.0584\n",
            "I1203 18:31:28.218945 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0584\n",
            "INFO:tensorflow:global_step/sec: 1.0612\n",
            "I1203 18:31:29.161029 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0612\n",
            "INFO:tensorflow:examples/sec: 33.9585\n",
            "I1203 18:31:29.161499 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9585\n",
            "INFO:tensorflow:global_step/sec: 1.06361\n",
            "I1203 18:31:30.101232 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06361\n",
            "INFO:tensorflow:examples/sec: 34.0354\n",
            "I1203 18:31:30.101650 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0354\n",
            "INFO:tensorflow:global_step/sec: 1.0556\n",
            "I1203 18:31:31.048573 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0556\n",
            "INFO:tensorflow:examples/sec: 33.7793\n",
            "I1203 18:31:31.048984 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7793\n",
            "INFO:tensorflow:global_step/sec: 1.06258\n",
            "I1203 18:31:31.989650 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06258\n",
            "INFO:tensorflow:examples/sec: 34.0026\n",
            "I1203 18:31:31.990030 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0026\n",
            "INFO:tensorflow:global_step/sec: 1.06572\n",
            "I1203 18:31:32.927982 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06572\n",
            "INFO:tensorflow:examples/sec: 34.103\n",
            "I1203 18:31:32.928463 140409631090560 tpu_estimator.py:2308] examples/sec: 34.103\n",
            "INFO:tensorflow:global_step/sec: 1.0617\n",
            "I1203 18:31:33.869889 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0617\n",
            "INFO:tensorflow:examples/sec: 33.9743\n",
            "I1203 18:31:33.870634 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9743\n",
            "INFO:tensorflow:global_step/sec: 1.06414\n",
            "I1203 18:31:34.809684 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06414\n",
            "INFO:tensorflow:examples/sec: 34.0525\n",
            "I1203 18:31:34.810055 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0525\n",
            "INFO:tensorflow:global_step/sec: 1.06227\n",
            "I1203 18:31:35.750969 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06227\n",
            "INFO:tensorflow:examples/sec: 33.9926\n",
            "I1203 18:31:35.751348 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9926\n",
            "INFO:tensorflow:global_step/sec: 1.06036\n",
            "I1203 18:31:36.694046 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06036\n",
            "INFO:tensorflow:examples/sec: 33.9316\n",
            "I1203 18:31:36.694429 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9316\n",
            "INFO:tensorflow:global_step/sec: 1.06779\n",
            "I1203 18:31:37.630587 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06779\n",
            "INFO:tensorflow:examples/sec: 34.1694\n",
            "I1203 18:31:37.630942 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1694\n",
            "INFO:tensorflow:global_step/sec: 1.06661\n",
            "I1203 18:31:38.568136 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06661\n",
            "INFO:tensorflow:examples/sec: 34.1315\n",
            "I1203 18:31:38.568518 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1315\n",
            "INFO:tensorflow:global_step/sec: 1.0657\n",
            "I1203 18:31:39.506511 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0657\n",
            "INFO:tensorflow:examples/sec: 34.1024\n",
            "I1203 18:31:39.506884 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1024\n",
            "INFO:tensorflow:global_step/sec: 1.05795\n",
            "I1203 18:31:40.451748 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05795\n",
            "INFO:tensorflow:examples/sec: 33.8545\n",
            "I1203 18:31:40.452108 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8545\n",
            "INFO:tensorflow:global_step/sec: 1.06074\n",
            "I1203 18:31:41.394457 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06074\n",
            "INFO:tensorflow:examples/sec: 33.9436\n",
            "I1203 18:31:41.394714 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9436\n",
            "INFO:tensorflow:global_step/sec: 1.06653\n",
            "I1203 18:31:42.332066 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06653\n",
            "INFO:tensorflow:examples/sec: 34.1288\n",
            "I1203 18:31:42.332451 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1288\n",
            "INFO:tensorflow:global_step/sec: 1.06327\n",
            "I1203 18:31:43.272586 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06327\n",
            "INFO:tensorflow:examples/sec: 34.0246\n",
            "I1203 18:31:43.272963 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0246\n",
            "INFO:tensorflow:global_step/sec: 1.05445\n",
            "I1203 18:31:44.220930 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05445\n",
            "INFO:tensorflow:examples/sec: 33.7425\n",
            "I1203 18:31:44.221296 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7425\n",
            "INFO:tensorflow:global_step/sec: 1.07116\n",
            "I1203 18:31:45.154475 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07116\n",
            "INFO:tensorflow:examples/sec: 34.277\n",
            "I1203 18:31:45.154702 140409631090560 tpu_estimator.py:2308] examples/sec: 34.277\n",
            "INFO:tensorflow:global_step/sec: 1.06271\n",
            "I1203 18:31:46.095471 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06271\n",
            "INFO:tensorflow:examples/sec: 34.0069\n",
            "I1203 18:31:46.095703 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0069\n",
            "INFO:tensorflow:global_step/sec: 1.06125\n",
            "I1203 18:31:47.037737 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06125\n",
            "INFO:tensorflow:examples/sec: 33.9601\n",
            "I1203 18:31:47.037968 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9601\n",
            "INFO:tensorflow:global_step/sec: 1.0658\n",
            "I1203 18:31:47.976001 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0658\n",
            "INFO:tensorflow:examples/sec: 34.1055\n",
            "I1203 18:31:47.976246 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1055\n",
            "INFO:tensorflow:global_step/sec: 1.06224\n",
            "I1203 18:31:48.917398 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06224\n",
            "INFO:tensorflow:examples/sec: 33.9918\n",
            "I1203 18:31:48.917771 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9918\n",
            "INFO:tensorflow:global_step/sec: 1.06462\n",
            "I1203 18:31:49.856718 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06462\n",
            "INFO:tensorflow:examples/sec: 34.0678\n",
            "I1203 18:31:49.857095 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0678\n",
            "INFO:tensorflow:global_step/sec: 1.06369\n",
            "I1203 18:31:50.796830 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06369\n",
            "INFO:tensorflow:examples/sec: 34.038\n",
            "I1203 18:31:50.797076 140409631090560 tpu_estimator.py:2308] examples/sec: 34.038\n",
            "INFO:tensorflow:global_step/sec: 1.06399\n",
            "I1203 18:31:51.736704 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06399\n",
            "INFO:tensorflow:examples/sec: 34.0476\n",
            "I1203 18:31:51.736931 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0476\n",
            "INFO:tensorflow:global_step/sec: 1.05911\n",
            "I1203 18:31:52.680895 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05911\n",
            "INFO:tensorflow:examples/sec: 33.8916\n",
            "I1203 18:31:52.681130 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8916\n",
            "INFO:tensorflow:global_step/sec: 1.06315\n",
            "I1203 18:31:53.621508 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06315\n",
            "INFO:tensorflow:examples/sec: 34.0208\n",
            "I1203 18:31:53.621886 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0208\n",
            "INFO:tensorflow:global_step/sec: 1.06936\n",
            "I1203 18:31:54.556622 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06936\n",
            "INFO:tensorflow:examples/sec: 34.2195\n",
            "I1203 18:31:54.556981 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2195\n",
            "INFO:tensorflow:global_step/sec: 1.06128\n",
            "I1203 18:31:55.498879 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06128\n",
            "INFO:tensorflow:examples/sec: 33.9611\n",
            "I1203 18:31:55.499115 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9611\n",
            "INFO:tensorflow:global_step/sec: 1.0626\n",
            "I1203 18:31:56.439969 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0626\n",
            "INFO:tensorflow:examples/sec: 34.0031\n",
            "I1203 18:31:56.440343 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0031\n",
            "INFO:tensorflow:global_step/sec: 1.06285\n",
            "I1203 18:31:57.380847 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06285\n",
            "INFO:tensorflow:examples/sec: 34.0113\n",
            "I1203 18:31:57.381082 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0113\n",
            "INFO:tensorflow:global_step/sec: 1.06206\n",
            "I1203 18:31:58.322437 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06206\n",
            "INFO:tensorflow:examples/sec: 33.986\n",
            "I1203 18:31:58.322673 140409631090560 tpu_estimator.py:2308] examples/sec: 33.986\n",
            "INFO:tensorflow:global_step/sec: 1.06489\n",
            "I1203 18:31:59.261469 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06489\n",
            "INFO:tensorflow:examples/sec: 34.0764\n",
            "I1203 18:31:59.261703 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0764\n",
            "INFO:tensorflow:global_step/sec: 1.06469\n",
            "I1203 18:32:00.200717 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06469\n",
            "INFO:tensorflow:examples/sec: 34.07\n",
            "I1203 18:32:00.200958 140409631090560 tpu_estimator.py:2308] examples/sec: 34.07\n",
            "INFO:tensorflow:global_step/sec: 1.06134\n",
            "I1203 18:32:01.142905 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06134\n",
            "INFO:tensorflow:examples/sec: 33.9628\n",
            "I1203 18:32:01.143177 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9628\n",
            "INFO:tensorflow:global_step/sec: 1.06735\n",
            "I1203 18:32:02.079823 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06735\n",
            "INFO:tensorflow:examples/sec: 34.1552\n",
            "I1203 18:32:02.080227 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1552\n",
            "INFO:tensorflow:global_step/sec: 1.0638\n",
            "I1203 18:32:03.019847 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0638\n",
            "INFO:tensorflow:examples/sec: 34.0417\n",
            "I1203 18:32:03.020101 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0417\n",
            "INFO:tensorflow:global_step/sec: 1.06179\n",
            "I1203 18:32:03.961656 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06179\n",
            "INFO:tensorflow:examples/sec: 33.9772\n",
            "I1203 18:32:03.961885 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9772\n",
            "INFO:tensorflow:global_step/sec: 1.06578\n",
            "I1203 18:32:04.899927 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06578\n",
            "INFO:tensorflow:examples/sec: 34.1048\n",
            "I1203 18:32:04.900300 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1048\n",
            "INFO:tensorflow:global_step/sec: 1.06706\n",
            "I1203 18:32:05.837099 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06706\n",
            "INFO:tensorflow:examples/sec: 34.1458\n",
            "I1203 18:32:05.837465 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1458\n",
            "INFO:tensorflow:global_step/sec: 1.06405\n",
            "I1203 18:32:06.776891 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06405\n",
            "INFO:tensorflow:examples/sec: 34.0496\n",
            "I1203 18:32:06.777181 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0496\n",
            "INFO:tensorflow:global_step/sec: 1.06585\n",
            "I1203 18:32:07.715113 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06585\n",
            "INFO:tensorflow:examples/sec: 34.1072\n",
            "I1203 18:32:07.715546 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1072\n",
            "INFO:tensorflow:global_step/sec: 1.06631\n",
            "I1203 18:32:08.652945 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06631\n",
            "INFO:tensorflow:examples/sec: 34.122\n",
            "I1203 18:32:08.653380 140409631090560 tpu_estimator.py:2308] examples/sec: 34.122\n",
            "INFO:tensorflow:global_step/sec: 1.06649\n",
            "I1203 18:32:09.590590 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06649\n",
            "INFO:tensorflow:examples/sec: 34.1277\n",
            "I1203 18:32:09.590828 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1277\n",
            "INFO:tensorflow:global_step/sec: 1.06298\n",
            "I1203 18:32:10.531330 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06298\n",
            "INFO:tensorflow:examples/sec: 34.0152\n",
            "I1203 18:32:10.531716 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0152\n",
            "INFO:tensorflow:global_step/sec: 1.0638\n",
            "I1203 18:32:11.471364 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0638\n",
            "INFO:tensorflow:examples/sec: 34.0417\n",
            "I1203 18:32:11.471622 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0417\n",
            "INFO:tensorflow:global_step/sec: 1.06073\n",
            "I1203 18:32:12.414149 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06073\n",
            "INFO:tensorflow:examples/sec: 33.9434\n",
            "I1203 18:32:12.414388 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9434\n",
            "INFO:tensorflow:global_step/sec: 1.06016\n",
            "I1203 18:32:13.357360 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06016\n",
            "INFO:tensorflow:examples/sec: 33.925\n",
            "I1203 18:32:13.357622 140409631090560 tpu_estimator.py:2308] examples/sec: 33.925\n",
            "INFO:tensorflow:global_step/sec: 1.06828\n",
            "I1203 18:32:14.293450 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06828\n",
            "INFO:tensorflow:examples/sec: 34.1849\n",
            "I1203 18:32:14.293683 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1849\n",
            "INFO:tensorflow:global_step/sec: 1.06972\n",
            "I1203 18:32:15.228271 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06972\n",
            "INFO:tensorflow:examples/sec: 34.2312\n",
            "I1203 18:32:15.228519 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2312\n",
            "INFO:tensorflow:global_step/sec: 1.06431\n",
            "I1203 18:32:16.167838 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06431\n",
            "INFO:tensorflow:examples/sec: 34.0578\n",
            "I1203 18:32:16.168066 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0578\n",
            "INFO:tensorflow:global_step/sec: 1.06511\n",
            "I1203 18:32:17.106719 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06511\n",
            "INFO:tensorflow:examples/sec: 34.0836\n",
            "I1203 18:32:17.106956 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0836\n",
            "INFO:tensorflow:global_step/sec: 1.0687\n",
            "I1203 18:32:18.042439 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0687\n",
            "INFO:tensorflow:examples/sec: 34.1984\n",
            "I1203 18:32:18.042671 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1984\n",
            "INFO:tensorflow:global_step/sec: 1.06278\n",
            "I1203 18:32:18.983347 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06278\n",
            "INFO:tensorflow:examples/sec: 34.0091\n",
            "I1203 18:32:18.983636 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0091\n",
            "INFO:tensorflow:global_step/sec: 1.06445\n",
            "I1203 18:32:19.922806 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06445\n",
            "INFO:tensorflow:examples/sec: 34.0624\n",
            "I1203 18:32:19.923161 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0624\n",
            "INFO:tensorflow:global_step/sec: 1.06487\n",
            "I1203 18:32:20.861889 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06487\n",
            "INFO:tensorflow:examples/sec: 34.076\n",
            "I1203 18:32:20.862129 140409631090560 tpu_estimator.py:2308] examples/sec: 34.076\n",
            "INFO:tensorflow:global_step/sec: 1.0638\n",
            "I1203 18:32:21.801923 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0638\n",
            "INFO:tensorflow:examples/sec: 34.0417\n",
            "I1203 18:32:21.802337 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0417\n",
            "INFO:tensorflow:global_step/sec: 1.06012\n",
            "I1203 18:32:22.745213 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06012\n",
            "INFO:tensorflow:examples/sec: 33.9239\n",
            "I1203 18:32:22.745479 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9239\n",
            "INFO:tensorflow:global_step/sec: 1.06574\n",
            "I1203 18:32:23.683534 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06574\n",
            "INFO:tensorflow:examples/sec: 34.1038\n",
            "I1203 18:32:23.683781 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1038\n",
            "INFO:tensorflow:global_step/sec: 1.06418\n",
            "I1203 18:32:24.623244 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06418\n",
            "INFO:tensorflow:examples/sec: 34.0536\n",
            "I1203 18:32:24.623651 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0536\n",
            "INFO:tensorflow:global_step/sec: 1.06119\n",
            "I1203 18:32:25.565569 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06119\n",
            "INFO:tensorflow:examples/sec: 33.9581\n",
            "I1203 18:32:25.565938 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9581\n",
            "INFO:tensorflow:global_step/sec: 1.05379\n",
            "I1203 18:32:26.514521 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05379\n",
            "INFO:tensorflow:examples/sec: 33.7211\n",
            "I1203 18:32:26.514756 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7211\n",
            "INFO:tensorflow:global_step/sec: 1.07043\n",
            "I1203 18:32:27.448705 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07043\n",
            "INFO:tensorflow:examples/sec: 34.2536\n",
            "I1203 18:32:27.448940 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2536\n",
            "INFO:tensorflow:global_step/sec: 1.06405\n",
            "I1203 18:32:28.388528 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06405\n",
            "INFO:tensorflow:examples/sec: 34.0497\n",
            "I1203 18:32:28.388923 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0497\n",
            "INFO:tensorflow:global_step/sec: 1.0625\n",
            "I1203 18:32:29.329693 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0625\n",
            "INFO:tensorflow:examples/sec: 34.0001\n",
            "I1203 18:32:29.329924 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0001\n",
            "INFO:tensorflow:global_step/sec: 1.06265\n",
            "I1203 18:32:30.270743 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06265\n",
            "INFO:tensorflow:examples/sec: 34.005\n",
            "I1203 18:32:30.271026 140409631090560 tpu_estimator.py:2308] examples/sec: 34.005\n",
            "INFO:tensorflow:global_step/sec: 1.07076\n",
            "I1203 18:32:31.204697 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07076\n",
            "INFO:tensorflow:examples/sec: 34.2642\n",
            "I1203 18:32:31.205099 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2642\n",
            "INFO:tensorflow:global_step/sec: 1.06243\n",
            "I1203 18:32:32.145871 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06243\n",
            "INFO:tensorflow:examples/sec: 33.9977\n",
            "I1203 18:32:32.146093 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9977\n",
            "INFO:tensorflow:global_step/sec: 1.05006\n",
            "I1203 18:32:33.098227 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05006\n",
            "INFO:tensorflow:examples/sec: 33.6019\n",
            "I1203 18:32:33.098482 140409631090560 tpu_estimator.py:2308] examples/sec: 33.6019\n",
            "INFO:tensorflow:global_step/sec: 1.06066\n",
            "I1203 18:32:34.041038 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06066\n",
            "INFO:tensorflow:examples/sec: 33.9412\n",
            "I1203 18:32:34.041301 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9412\n",
            "INFO:tensorflow:global_step/sec: 1.06381\n",
            "I1203 18:32:34.981016 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06381\n",
            "INFO:tensorflow:examples/sec: 34.042\n",
            "I1203 18:32:34.981256 140409631090560 tpu_estimator.py:2308] examples/sec: 34.042\n",
            "INFO:tensorflow:global_step/sec: 1.06113\n",
            "I1203 18:32:35.923429 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06113\n",
            "INFO:tensorflow:examples/sec: 33.9561\n",
            "I1203 18:32:35.923666 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9561\n",
            "INFO:tensorflow:global_step/sec: 1.06637\n",
            "I1203 18:32:36.861194 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06637\n",
            "INFO:tensorflow:examples/sec: 34.1238\n",
            "I1203 18:32:36.861624 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1238\n",
            "INFO:tensorflow:global_step/sec: 1.06902\n",
            "I1203 18:32:37.796626 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06902\n",
            "INFO:tensorflow:examples/sec: 34.2086\n",
            "I1203 18:32:37.796856 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2086\n",
            "INFO:tensorflow:global_step/sec: 1.05899\n",
            "I1203 18:32:38.740943 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05899\n",
            "INFO:tensorflow:examples/sec: 33.8876\n",
            "I1203 18:32:38.741176 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8876\n",
            "INFO:tensorflow:global_step/sec: 1.0645\n",
            "I1203 18:32:39.680292 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0645\n",
            "INFO:tensorflow:examples/sec: 34.064\n",
            "I1203 18:32:39.680666 140409631090560 tpu_estimator.py:2308] examples/sec: 34.064\n",
            "INFO:tensorflow:global_step/sec: 1.05799\n",
            "I1203 18:32:40.625531 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05799\n",
            "INFO:tensorflow:examples/sec: 33.8558\n",
            "I1203 18:32:40.625807 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8558\n",
            "INFO:tensorflow:global_step/sec: 1.06932\n",
            "I1203 18:32:41.560749 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06932\n",
            "INFO:tensorflow:examples/sec: 34.2181\n",
            "I1203 18:32:41.561129 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2181\n",
            "INFO:tensorflow:global_step/sec: 1.05824\n",
            "I1203 18:32:42.505662 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05824\n",
            "INFO:tensorflow:examples/sec: 33.8636\n",
            "I1203 18:32:42.505888 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8636\n",
            "INFO:tensorflow:global_step/sec: 1.06955\n",
            "I1203 18:32:43.440671 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06955\n",
            "INFO:tensorflow:examples/sec: 34.2255\n",
            "I1203 18:32:43.441635 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2255\n",
            "INFO:tensorflow:global_step/sec: 1.06531\n",
            "I1203 18:32:44.379354 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06531\n",
            "INFO:tensorflow:examples/sec: 34.09\n",
            "I1203 18:32:44.379627 140409631090560 tpu_estimator.py:2308] examples/sec: 34.09\n",
            "INFO:tensorflow:global_step/sec: 1.0628\n",
            "I1203 18:32:45.320235 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0628\n",
            "INFO:tensorflow:examples/sec: 34.0097\n",
            "I1203 18:32:45.320675 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0097\n",
            "INFO:tensorflow:global_step/sec: 1.06486\n",
            "I1203 18:32:46.259673 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06486\n",
            "INFO:tensorflow:examples/sec: 34.0754\n",
            "I1203 18:32:46.260086 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0754\n",
            "INFO:tensorflow:global_step/sec: 1.05785\n",
            "I1203 18:32:47.204643 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05785\n",
            "INFO:tensorflow:examples/sec: 33.8514\n",
            "I1203 18:32:47.204875 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8514\n",
            "INFO:tensorflow:global_step/sec: 1.06515\n",
            "I1203 18:32:48.143505 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06515\n",
            "INFO:tensorflow:examples/sec: 34.0848\n",
            "I1203 18:32:48.143737 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0848\n",
            "INFO:tensorflow:global_step/sec: 1.06719\n",
            "I1203 18:32:49.080541 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06719\n",
            "INFO:tensorflow:examples/sec: 34.15\n",
            "I1203 18:32:49.080775 140409631090560 tpu_estimator.py:2308] examples/sec: 34.15\n",
            "INFO:tensorflow:global_step/sec: 1.07263\n",
            "I1203 18:32:50.012834 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07263\n",
            "INFO:tensorflow:examples/sec: 34.324\n",
            "I1203 18:32:50.013208 140409631090560 tpu_estimator.py:2308] examples/sec: 34.324\n",
            "INFO:tensorflow:global_step/sec: 1.05732\n",
            "I1203 18:32:50.958616 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05732\n",
            "INFO:tensorflow:examples/sec: 33.8342\n",
            "I1203 18:32:50.959002 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8342\n",
            "INFO:tensorflow:global_step/sec: 1.05583\n",
            "I1203 18:32:51.905744 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05583\n",
            "INFO:tensorflow:examples/sec: 33.7865\n",
            "I1203 18:32:51.906216 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7865\n",
            "INFO:tensorflow:global_step/sec: 1.06384\n",
            "I1203 18:32:52.845716 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06384\n",
            "INFO:tensorflow:examples/sec: 34.043\n",
            "I1203 18:32:52.845938 140409631090560 tpu_estimator.py:2308] examples/sec: 34.043\n",
            "INFO:tensorflow:global_step/sec: 1.06946\n",
            "I1203 18:32:53.780758 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06946\n",
            "INFO:tensorflow:examples/sec: 34.2228\n",
            "I1203 18:32:53.780989 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2228\n",
            "INFO:tensorflow:global_step/sec: 1.05931\n",
            "I1203 18:32:54.724765 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05931\n",
            "INFO:tensorflow:examples/sec: 33.8979\n",
            "I1203 18:32:54.724999 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8979\n",
            "INFO:tensorflow:global_step/sec: 1.05656\n",
            "I1203 18:32:55.671271 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05656\n",
            "INFO:tensorflow:examples/sec: 33.81\n",
            "I1203 18:32:55.671730 140409631090560 tpu_estimator.py:2308] examples/sec: 33.81\n",
            "INFO:tensorflow:global_step/sec: 1.06311\n",
            "I1203 18:32:56.611907 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06311\n",
            "INFO:tensorflow:examples/sec: 34.0195\n",
            "I1203 18:32:56.612205 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0195\n",
            "INFO:tensorflow:global_step/sec: 1.05794\n",
            "I1203 18:32:57.557115 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05794\n",
            "INFO:tensorflow:examples/sec: 33.854\n",
            "I1203 18:32:57.557417 140409631090560 tpu_estimator.py:2308] examples/sec: 33.854\n",
            "INFO:tensorflow:global_step/sec: 1.06234\n",
            "I1203 18:32:58.498455 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06234\n",
            "INFO:tensorflow:examples/sec: 33.9949\n",
            "I1203 18:32:58.498704 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9949\n",
            "INFO:tensorflow:global_step/sec: 1.06293\n",
            "I1203 18:32:59.439240 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06293\n",
            "INFO:tensorflow:examples/sec: 34.0136\n",
            "I1203 18:32:59.439513 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0136\n",
            "INFO:tensorflow:global_step/sec: 1.06556\n",
            "I1203 18:33:00.377699 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06556\n",
            "INFO:tensorflow:examples/sec: 34.098\n",
            "I1203 18:33:00.377953 140409631090560 tpu_estimator.py:2308] examples/sec: 34.098\n",
            "INFO:tensorflow:global_step/sec: 1.06191\n",
            "I1203 18:33:01.319417 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06191\n",
            "INFO:tensorflow:examples/sec: 33.981\n",
            "I1203 18:33:01.319645 140409631090560 tpu_estimator.py:2308] examples/sec: 33.981\n",
            "INFO:tensorflow:global_step/sec: 1.06931\n",
            "I1203 18:33:02.254615 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06931\n",
            "INFO:tensorflow:examples/sec: 34.218\n",
            "I1203 18:33:02.254866 140409631090560 tpu_estimator.py:2308] examples/sec: 34.218\n",
            "INFO:tensorflow:global_step/sec: 1.06215\n",
            "I1203 18:33:03.196054 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06215\n",
            "INFO:tensorflow:examples/sec: 33.9889\n",
            "I1203 18:33:03.196286 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9889\n",
            "INFO:tensorflow:global_step/sec: 1.06173\n",
            "I1203 18:33:04.137930 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06173\n",
            "INFO:tensorflow:examples/sec: 33.9754\n",
            "I1203 18:33:04.138388 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9754\n",
            "INFO:tensorflow:global_step/sec: 1.05793\n",
            "I1203 18:33:05.083194 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05793\n",
            "INFO:tensorflow:examples/sec: 33.8538\n",
            "I1203 18:33:05.083473 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8538\n",
            "INFO:tensorflow:global_step/sec: 1.06637\n",
            "I1203 18:33:06.020925 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06637\n",
            "INFO:tensorflow:examples/sec: 34.124\n",
            "I1203 18:33:06.021332 140409631090560 tpu_estimator.py:2308] examples/sec: 34.124\n",
            "INFO:tensorflow:global_step/sec: 1.06412\n",
            "I1203 18:33:06.960674 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06412\n",
            "INFO:tensorflow:examples/sec: 34.0517\n",
            "I1203 18:33:06.961069 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0517\n",
            "INFO:tensorflow:global_step/sec: 1.05919\n",
            "I1203 18:33:07.904805 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05919\n",
            "INFO:tensorflow:examples/sec: 33.8942\n",
            "I1203 18:33:07.905166 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8942\n",
            "INFO:tensorflow:global_step/sec: 1.05744\n",
            "I1203 18:33:08.850478 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05744\n",
            "INFO:tensorflow:examples/sec: 33.8379\n",
            "I1203 18:33:08.850851 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8379\n",
            "INFO:tensorflow:global_step/sec: 1.06592\n",
            "I1203 18:33:09.788668 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06592\n",
            "INFO:tensorflow:examples/sec: 34.1096\n",
            "I1203 18:33:09.789074 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1096\n",
            "INFO:tensorflow:global_step/sec: 1.0625\n",
            "I1203 18:33:10.729790 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0625\n",
            "INFO:tensorflow:examples/sec: 34\n",
            "I1203 18:33:10.730220 140409631090560 tpu_estimator.py:2308] examples/sec: 34\n",
            "INFO:tensorflow:global_step/sec: 1.06156\n",
            "I1203 18:33:11.671824 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06156\n",
            "INFO:tensorflow:examples/sec: 33.9698\n",
            "I1203 18:33:11.672214 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9698\n",
            "INFO:tensorflow:global_step/sec: 1.06361\n",
            "I1203 18:33:12.612008 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06361\n",
            "INFO:tensorflow:examples/sec: 34.0354\n",
            "I1203 18:33:12.612239 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0354\n",
            "INFO:tensorflow:global_step/sec: 1.0682\n",
            "I1203 18:33:13.548175 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0682\n",
            "INFO:tensorflow:examples/sec: 34.1824\n",
            "I1203 18:33:13.548418 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1824\n",
            "INFO:tensorflow:global_step/sec: 1.06113\n",
            "I1203 18:33:14.490589 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06113\n",
            "INFO:tensorflow:examples/sec: 33.9563\n",
            "I1203 18:33:14.490993 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9563\n",
            "INFO:tensorflow:global_step/sec: 1.06983\n",
            "I1203 18:33:15.425289 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06983\n",
            "INFO:tensorflow:examples/sec: 34.2344\n",
            "I1203 18:33:15.425695 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2344\n",
            "INFO:tensorflow:global_step/sec: 1.06157\n",
            "I1203 18:33:16.367320 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06157\n",
            "INFO:tensorflow:examples/sec: 33.9702\n",
            "I1203 18:33:16.367738 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9702\n",
            "INFO:tensorflow:global_step/sec: 1.06078\n",
            "I1203 18:33:17.309987 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06078\n",
            "INFO:tensorflow:examples/sec: 33.9451\n",
            "I1203 18:33:17.310225 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9451\n",
            "INFO:tensorflow:global_step/sec: 1.06352\n",
            "I1203 18:33:18.250264 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06352\n",
            "INFO:tensorflow:examples/sec: 34.0326\n",
            "I1203 18:33:18.250525 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0326\n",
            "INFO:tensorflow:global_step/sec: 1.05811\n",
            "I1203 18:33:19.195354 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05811\n",
            "INFO:tensorflow:examples/sec: 33.8596\n",
            "I1203 18:33:19.195596 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8596\n",
            "INFO:tensorflow:global_step/sec: 1.06271\n",
            "I1203 18:33:20.136328 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06271\n",
            "INFO:tensorflow:examples/sec: 34.0068\n",
            "I1203 18:33:20.136568 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0068\n",
            "INFO:tensorflow:global_step/sec: 1.06572\n",
            "I1203 18:33:21.074656 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06572\n",
            "INFO:tensorflow:examples/sec: 34.103\n",
            "I1203 18:33:21.074884 140409631090560 tpu_estimator.py:2308] examples/sec: 34.103\n",
            "INFO:tensorflow:global_step/sec: 1.06294\n",
            "I1203 18:33:22.015470 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06294\n",
            "INFO:tensorflow:examples/sec: 34.014\n",
            "I1203 18:33:22.015707 140409631090560 tpu_estimator.py:2308] examples/sec: 34.014\n",
            "INFO:tensorflow:global_step/sec: 1.05788\n",
            "I1203 18:33:22.960765 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05788\n",
            "INFO:tensorflow:examples/sec: 33.8523\n",
            "I1203 18:33:22.961142 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8523\n",
            "INFO:tensorflow:global_step/sec: 1.06349\n",
            "I1203 18:33:23.901020 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06349\n",
            "INFO:tensorflow:examples/sec: 34.0318\n",
            "I1203 18:33:23.901378 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0318\n",
            "INFO:tensorflow:global_step/sec: 1.06846\n",
            "I1203 18:33:24.836951 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06846\n",
            "INFO:tensorflow:examples/sec: 34.1907\n",
            "I1203 18:33:24.837193 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1907\n",
            "INFO:tensorflow:global_step/sec: 1.05943\n",
            "I1203 18:33:25.780868 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05943\n",
            "INFO:tensorflow:examples/sec: 33.9016\n",
            "I1203 18:33:25.781097 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9016\n",
            "INFO:tensorflow:global_step/sec: 1.0604\n",
            "I1203 18:33:26.723919 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0604\n",
            "INFO:tensorflow:examples/sec: 33.9329\n",
            "I1203 18:33:26.724152 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9329\n",
            "INFO:tensorflow:global_step/sec: 1.05669\n",
            "I1203 18:33:27.670268 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05669\n",
            "INFO:tensorflow:examples/sec: 33.8139\n",
            "I1203 18:33:27.670660 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8139\n",
            "INFO:tensorflow:global_step/sec: 1.06589\n",
            "I1203 18:33:28.608461 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06589\n",
            "INFO:tensorflow:examples/sec: 34.1085\n",
            "I1203 18:33:28.608690 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1085\n",
            "INFO:tensorflow:global_step/sec: 1.06057\n",
            "I1203 18:33:29.551336 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06057\n",
            "INFO:tensorflow:examples/sec: 33.9382\n",
            "I1203 18:33:29.551577 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9382\n",
            "INFO:tensorflow:global_step/sec: 1.06928\n",
            "I1203 18:33:30.486578 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06928\n",
            "INFO:tensorflow:examples/sec: 34.2169\n",
            "I1203 18:33:30.486828 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2169\n",
            "INFO:tensorflow:global_step/sec: 1.05997\n",
            "I1203 18:33:31.429970 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05997\n",
            "INFO:tensorflow:examples/sec: 33.9192\n",
            "I1203 18:33:31.430240 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9192\n",
            "INFO:tensorflow:global_step/sec: 1.05536\n",
            "I1203 18:33:32.377527 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05536\n",
            "INFO:tensorflow:examples/sec: 33.7716\n",
            "I1203 18:33:32.377802 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7716\n",
            "INFO:tensorflow:global_step/sec: 1.06009\n",
            "I1203 18:33:33.320829 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06009\n",
            "INFO:tensorflow:examples/sec: 33.9228\n",
            "I1203 18:33:33.321119 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9228\n",
            "INFO:tensorflow:global_step/sec: 1.06222\n",
            "I1203 18:33:34.262249 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06222\n",
            "INFO:tensorflow:examples/sec: 33.9909\n",
            "I1203 18:33:34.262519 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9909\n",
            "INFO:tensorflow:global_step/sec: 1.05924\n",
            "I1203 18:33:35.206334 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05924\n",
            "INFO:tensorflow:examples/sec: 33.8958\n",
            "I1203 18:33:35.206614 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8958\n",
            "INFO:tensorflow:global_step/sec: 1.06445\n",
            "I1203 18:33:36.145811 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06445\n",
            "INFO:tensorflow:examples/sec: 34.0626\n",
            "I1203 18:33:36.146092 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0626\n",
            "INFO:tensorflow:global_step/sec: 1.0638\n",
            "I1203 18:33:37.085796 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0638\n",
            "INFO:tensorflow:examples/sec: 34.0417\n",
            "I1203 18:33:37.086054 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0417\n",
            "INFO:tensorflow:global_step/sec: 1.05602\n",
            "I1203 18:33:38.032784 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05602\n",
            "INFO:tensorflow:examples/sec: 33.7927\n",
            "I1203 18:33:38.033035 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7927\n",
            "INFO:tensorflow:global_step/sec: 1.05717\n",
            "I1203 18:33:38.978672 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05717\n",
            "INFO:tensorflow:examples/sec: 33.8294\n",
            "I1203 18:33:38.978942 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8294\n",
            "INFO:tensorflow:global_step/sec: 1.06465\n",
            "I1203 18:33:39.917954 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06465\n",
            "INFO:tensorflow:examples/sec: 34.0688\n",
            "I1203 18:33:39.918215 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0688\n",
            "INFO:tensorflow:global_step/sec: 1.06124\n",
            "I1203 18:33:40.860230 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06124\n",
            "INFO:tensorflow:examples/sec: 33.9598\n",
            "I1203 18:33:40.860498 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9598\n",
            "INFO:tensorflow:global_step/sec: 1.06505\n",
            "I1203 18:33:41.799153 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06505\n",
            "INFO:tensorflow:examples/sec: 34.0816\n",
            "I1203 18:33:41.799455 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0816\n",
            "INFO:tensorflow:global_step/sec: 1.06353\n",
            "I1203 18:33:42.739432 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06353\n",
            "INFO:tensorflow:examples/sec: 34.0328\n",
            "I1203 18:33:42.739670 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0328\n",
            "INFO:tensorflow:global_step/sec: 1.05805\n",
            "I1203 18:33:43.684599 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05805\n",
            "INFO:tensorflow:examples/sec: 33.8577\n",
            "I1203 18:33:43.684847 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8577\n",
            "INFO:tensorflow:global_step/sec: 1.06072\n",
            "I1203 18:33:44.627347 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06072\n",
            "INFO:tensorflow:examples/sec: 33.943\n",
            "I1203 18:33:44.627590 140409631090560 tpu_estimator.py:2308] examples/sec: 33.943\n",
            "INFO:tensorflow:global_step/sec: 1.06258\n",
            "I1203 18:33:45.568449 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06258\n",
            "INFO:tensorflow:examples/sec: 34.0024\n",
            "I1203 18:33:45.568813 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0024\n",
            "INFO:tensorflow:global_step/sec: 1.05886\n",
            "I1203 18:33:46.512845 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05886\n",
            "INFO:tensorflow:examples/sec: 33.8835\n",
            "I1203 18:33:46.513216 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8835\n",
            "INFO:tensorflow:global_step/sec: 1.06315\n",
            "I1203 18:33:47.453454 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06315\n",
            "INFO:tensorflow:examples/sec: 34.0207\n",
            "I1203 18:33:47.453700 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0207\n",
            "INFO:tensorflow:global_step/sec: 1.07036\n",
            "I1203 18:33:48.387711 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07036\n",
            "INFO:tensorflow:examples/sec: 34.2514\n",
            "I1203 18:33:48.387956 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2514\n",
            "INFO:tensorflow:global_step/sec: 1.06141\n",
            "I1203 18:33:49.329879 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06141\n",
            "INFO:tensorflow:examples/sec: 33.9653\n",
            "I1203 18:33:49.330255 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9653\n",
            "INFO:tensorflow:global_step/sec: 1.06343\n",
            "I1203 18:33:50.270206 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06343\n",
            "INFO:tensorflow:examples/sec: 34.0298\n",
            "I1203 18:33:50.270510 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0298\n",
            "INFO:tensorflow:global_step/sec: 1.05738\n",
            "I1203 18:33:51.215948 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05738\n",
            "INFO:tensorflow:examples/sec: 33.8363\n",
            "I1203 18:33:51.216210 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8363\n",
            "INFO:tensorflow:global_step/sec: 1.06495\n",
            "I1203 18:33:52.154934 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06495\n",
            "INFO:tensorflow:examples/sec: 34.0785\n",
            "I1203 18:33:52.155175 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0785\n",
            "INFO:tensorflow:global_step/sec: 1.05521\n",
            "I1203 18:33:53.102646 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05521\n",
            "INFO:tensorflow:examples/sec: 33.7666\n",
            "I1203 18:33:53.103025 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7666\n",
            "INFO:tensorflow:global_step/sec: 1.06355\n",
            "I1203 18:33:54.042876 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06355\n",
            "INFO:tensorflow:examples/sec: 34.0337\n",
            "I1203 18:33:54.043112 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0337\n",
            "INFO:tensorflow:global_step/sec: 1.06559\n",
            "I1203 18:33:54.981332 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06559\n",
            "INFO:tensorflow:examples/sec: 34.0989\n",
            "I1203 18:33:54.981599 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0989\n",
            "INFO:tensorflow:global_step/sec: 1.06332\n",
            "I1203 18:33:55.921772 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06332\n",
            "INFO:tensorflow:examples/sec: 34.0263\n",
            "I1203 18:33:55.922019 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0263\n",
            "INFO:tensorflow:global_step/sec: 1.06192\n",
            "I1203 18:33:56.863473 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06192\n",
            "INFO:tensorflow:examples/sec: 33.9815\n",
            "I1203 18:33:56.863708 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9815\n",
            "INFO:tensorflow:global_step/sec: 1.06316\n",
            "I1203 18:33:57.804057 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06316\n",
            "INFO:tensorflow:examples/sec: 34.0212\n",
            "I1203 18:33:57.804456 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0212\n",
            "INFO:tensorflow:global_step/sec: 1.06187\n",
            "I1203 18:33:58.745824 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06187\n",
            "INFO:tensorflow:examples/sec: 33.9799\n",
            "I1203 18:33:58.746058 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9799\n",
            "INFO:tensorflow:global_step/sec: 1.06391\n",
            "I1203 18:33:59.685719 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06391\n",
            "INFO:tensorflow:examples/sec: 34.0452\n",
            "I1203 18:33:59.686001 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0452\n",
            "INFO:tensorflow:global_step/sec: 1.06078\n",
            "I1203 18:34:00.628393 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06078\n",
            "INFO:tensorflow:examples/sec: 33.9451\n",
            "I1203 18:34:00.628643 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9451\n",
            "INFO:tensorflow:global_step/sec: 1.06443\n",
            "I1203 18:34:01.567883 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06443\n",
            "INFO:tensorflow:examples/sec: 34.0617\n",
            "I1203 18:34:01.568109 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0617\n",
            "INFO:tensorflow:global_step/sec: 1.0626\n",
            "I1203 18:34:02.508969 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0626\n",
            "INFO:tensorflow:examples/sec: 34.0032\n",
            "I1203 18:34:02.509208 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0032\n",
            "INFO:tensorflow:global_step/sec: 1.06449\n",
            "I1203 18:34:03.448409 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06449\n",
            "INFO:tensorflow:examples/sec: 34.0636\n",
            "I1203 18:34:03.448646 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0636\n",
            "INFO:tensorflow:global_step/sec: 1.06327\n",
            "I1203 18:34:04.388919 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06327\n",
            "INFO:tensorflow:examples/sec: 34.0245\n",
            "I1203 18:34:04.389317 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0245\n",
            "INFO:tensorflow:global_step/sec: 1.06456\n",
            "I1203 18:34:05.328236 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06456\n",
            "INFO:tensorflow:examples/sec: 34.0661\n",
            "I1203 18:34:05.328505 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0661\n",
            "INFO:tensorflow:global_step/sec: 1.06445\n",
            "I1203 18:34:06.267678 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06445\n",
            "INFO:tensorflow:examples/sec: 34.0623\n",
            "I1203 18:34:06.267914 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0623\n",
            "INFO:tensorflow:global_step/sec: 1.06327\n",
            "I1203 18:34:07.208181 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06327\n",
            "INFO:tensorflow:examples/sec: 34.0245\n",
            "I1203 18:34:07.208421 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0245\n",
            "INFO:tensorflow:global_step/sec: 1.05863\n",
            "I1203 18:34:08.152802 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05863\n",
            "INFO:tensorflow:examples/sec: 33.8762\n",
            "I1203 18:34:08.153040 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8762\n",
            "INFO:tensorflow:global_step/sec: 1.06921\n",
            "I1203 18:34:09.088064 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06921\n",
            "INFO:tensorflow:examples/sec: 34.2146\n",
            "I1203 18:34:09.088292 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2146\n",
            "INFO:tensorflow:global_step/sec: 1.06037\n",
            "I1203 18:34:10.031124 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06037\n",
            "INFO:tensorflow:examples/sec: 33.9318\n",
            "I1203 18:34:10.031351 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9318\n",
            "INFO:tensorflow:global_step/sec: 1.05625\n",
            "I1203 18:34:10.977887 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05625\n",
            "INFO:tensorflow:examples/sec: 33.7999\n",
            "I1203 18:34:10.978115 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7999\n",
            "INFO:tensorflow:global_step/sec: 1.06544\n",
            "I1203 18:34:11.916482 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06544\n",
            "INFO:tensorflow:examples/sec: 34.0942\n",
            "I1203 18:34:11.916718 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0942\n",
            "INFO:tensorflow:global_step/sec: 1.06796\n",
            "I1203 18:34:12.852775 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06796\n",
            "INFO:tensorflow:examples/sec: 34.1748\n",
            "I1203 18:34:12.853005 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1748\n",
            "INFO:tensorflow:global_step/sec: 1.06093\n",
            "I1203 18:34:13.795453 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06093\n",
            "INFO:tensorflow:examples/sec: 33.9496\n",
            "I1203 18:34:13.795709 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9496\n",
            "INFO:tensorflow:global_step/sec: 1.05818\n",
            "I1203 18:34:14.740436 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05818\n",
            "INFO:tensorflow:examples/sec: 33.8618\n",
            "I1203 18:34:14.740669 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8618\n",
            "INFO:tensorflow:global_step/sec: 1.0643\n",
            "I1203 18:34:15.679991 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0643\n",
            "INFO:tensorflow:examples/sec: 34.0577\n",
            "I1203 18:34:15.680372 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0577\n",
            "INFO:tensorflow:global_step/sec: 1.05792\n",
            "I1203 18:34:16.625249 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05792\n",
            "INFO:tensorflow:examples/sec: 33.8533\n",
            "I1203 18:34:16.625507 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8533\n",
            "INFO:tensorflow:global_step/sec: 1.06548\n",
            "I1203 18:34:17.563795 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06548\n",
            "INFO:tensorflow:examples/sec: 34.0953\n",
            "I1203 18:34:17.564026 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0953\n",
            "INFO:tensorflow:global_step/sec: 1.05996\n",
            "I1203 18:34:18.507228 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05996\n",
            "INFO:tensorflow:examples/sec: 33.9187\n",
            "I1203 18:34:18.507485 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9187\n",
            "INFO:tensorflow:global_step/sec: 1.06029\n",
            "I1203 18:34:19.450362 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06029\n",
            "INFO:tensorflow:examples/sec: 33.9294\n",
            "I1203 18:34:19.450627 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9294\n",
            "INFO:tensorflow:global_step/sec: 1.06676\n",
            "I1203 18:34:20.387764 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06676\n",
            "INFO:tensorflow:examples/sec: 34.1364\n",
            "I1203 18:34:20.388105 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1364\n",
            "INFO:tensorflow:global_step/sec: 1.05782\n",
            "I1203 18:34:21.333134 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05782\n",
            "INFO:tensorflow:examples/sec: 33.8503\n",
            "I1203 18:34:21.333369 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8503\n",
            "INFO:tensorflow:global_step/sec: 1.06629\n",
            "I1203 18:34:22.270950 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06629\n",
            "INFO:tensorflow:examples/sec: 34.1213\n",
            "I1203 18:34:22.271188 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1213\n",
            "INFO:tensorflow:global_step/sec: 1.06379\n",
            "I1203 18:34:23.210992 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06379\n",
            "INFO:tensorflow:examples/sec: 34.0411\n",
            "I1203 18:34:23.211392 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0411\n",
            "INFO:tensorflow:global_step/sec: 1.06014\n",
            "I1203 18:34:24.154245 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06014\n",
            "INFO:tensorflow:examples/sec: 33.9246\n",
            "I1203 18:34:24.154637 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9246\n",
            "INFO:tensorflow:global_step/sec: 1.06348\n",
            "I1203 18:34:25.094613 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06348\n",
            "INFO:tensorflow:examples/sec: 34.0313\n",
            "I1203 18:34:25.094984 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0313\n",
            "INFO:tensorflow:global_step/sec: 1.05742\n",
            "I1203 18:34:26.040261 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05742\n",
            "INFO:tensorflow:examples/sec: 33.8375\n",
            "I1203 18:34:26.040515 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8375\n",
            "INFO:tensorflow:global_step/sec: 1.06522\n",
            "I1203 18:34:26.979038 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06522\n",
            "INFO:tensorflow:examples/sec: 34.087\n",
            "I1203 18:34:26.979292 140409631090560 tpu_estimator.py:2308] examples/sec: 34.087\n",
            "INFO:tensorflow:global_step/sec: 1.06834\n",
            "I1203 18:34:27.915084 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06834\n",
            "INFO:tensorflow:examples/sec: 34.1869\n",
            "I1203 18:34:27.915313 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1869\n",
            "INFO:tensorflow:global_step/sec: 1.06151\n",
            "I1203 18:34:28.857114 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06151\n",
            "INFO:tensorflow:examples/sec: 33.9683\n",
            "I1203 18:34:28.857357 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9683\n",
            "INFO:tensorflow:global_step/sec: 1.0607\n",
            "I1203 18:34:29.799926 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0607\n",
            "INFO:tensorflow:examples/sec: 33.9424\n",
            "I1203 18:34:29.800158 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9424\n",
            "INFO:tensorflow:global_step/sec: 1.06169\n",
            "I1203 18:34:30.741781 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06169\n",
            "INFO:tensorflow:examples/sec: 33.974\n",
            "I1203 18:34:30.742006 140409631090560 tpu_estimator.py:2308] examples/sec: 33.974\n",
            "INFO:tensorflow:global_step/sec: 1.05682\n",
            "I1203 18:34:31.688050 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05682\n",
            "INFO:tensorflow:examples/sec: 33.8182\n",
            "I1203 18:34:31.688290 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8182\n",
            "INFO:tensorflow:global_step/sec: 1.06542\n",
            "I1203 18:34:32.626639 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06542\n",
            "INFO:tensorflow:examples/sec: 34.0934\n",
            "I1203 18:34:32.626882 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0934\n",
            "INFO:tensorflow:global_step/sec: 1.06498\n",
            "I1203 18:34:33.565605 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06498\n",
            "INFO:tensorflow:examples/sec: 34.0793\n",
            "I1203 18:34:33.565828 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0793\n",
            "INFO:tensorflow:global_step/sec: 1.0652\n",
            "I1203 18:34:34.504390 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0652\n",
            "INFO:tensorflow:examples/sec: 34.0864\n",
            "I1203 18:34:34.505034 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0864\n",
            "INFO:tensorflow:global_step/sec: 1.06092\n",
            "I1203 18:34:35.447003 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06092\n",
            "INFO:tensorflow:examples/sec: 33.9494\n",
            "I1203 18:34:35.447408 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9494\n",
            "INFO:tensorflow:global_step/sec: 1.07091\n",
            "I1203 18:34:36.380766 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07091\n",
            "INFO:tensorflow:examples/sec: 34.269\n",
            "I1203 18:34:36.381136 140409631090560 tpu_estimator.py:2308] examples/sec: 34.269\n",
            "INFO:tensorflow:global_step/sec: 1.06261\n",
            "I1203 18:34:37.321861 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06261\n",
            "INFO:tensorflow:examples/sec: 34.0034\n",
            "I1203 18:34:37.322231 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0034\n",
            "INFO:tensorflow:global_step/sec: 1.05393\n",
            "I1203 18:34:38.270681 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05393\n",
            "INFO:tensorflow:examples/sec: 33.7257\n",
            "I1203 18:34:38.270917 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7257\n",
            "INFO:tensorflow:global_step/sec: 1.06936\n",
            "I1203 18:34:39.205837 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06936\n",
            "INFO:tensorflow:examples/sec: 34.2196\n",
            "I1203 18:34:39.206084 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2196\n",
            "INFO:tensorflow:global_step/sec: 1.06124\n",
            "I1203 18:34:40.148129 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06124\n",
            "INFO:tensorflow:examples/sec: 33.9596\n",
            "I1203 18:34:40.148378 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9596\n",
            "INFO:tensorflow:global_step/sec: 1.06442\n",
            "I1203 18:34:41.087630 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06442\n",
            "INFO:tensorflow:examples/sec: 34.0615\n",
            "I1203 18:34:41.088036 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0615\n",
            "INFO:tensorflow:global_step/sec: 1.05941\n",
            "I1203 18:34:42.031530 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05941\n",
            "INFO:tensorflow:examples/sec: 33.9012\n",
            "I1203 18:34:42.031772 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9012\n",
            "INFO:tensorflow:global_step/sec: 1.06514\n",
            "I1203 18:34:42.970348 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06514\n",
            "INFO:tensorflow:examples/sec: 34.0845\n",
            "I1203 18:34:42.970795 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0845\n",
            "INFO:tensorflow:global_step/sec: 1.0588\n",
            "I1203 18:34:43.914832 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0588\n",
            "INFO:tensorflow:examples/sec: 33.8817\n",
            "I1203 18:34:43.915232 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8817\n",
            "INFO:tensorflow:global_step/sec: 1.06364\n",
            "I1203 18:34:44.854989 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06364\n",
            "INFO:tensorflow:examples/sec: 34.0365\n",
            "I1203 18:34:44.855363 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0365\n",
            "INFO:tensorflow:global_step/sec: 1.06315\n",
            "I1203 18:34:45.795623 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06315\n",
            "INFO:tensorflow:examples/sec: 34.0207\n",
            "I1203 18:34:45.795872 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0207\n",
            "INFO:tensorflow:global_step/sec: 1.06395\n",
            "I1203 18:34:46.735499 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06395\n",
            "INFO:tensorflow:examples/sec: 34.0465\n",
            "I1203 18:34:46.736004 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0465\n",
            "INFO:tensorflow:global_step/sec: 1.0483\n",
            "I1203 18:34:47.689434 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0483\n",
            "INFO:tensorflow:examples/sec: 33.5455\n",
            "I1203 18:34:47.689804 140409631090560 tpu_estimator.py:2308] examples/sec: 33.5455\n",
            "INFO:tensorflow:global_step/sec: 1.06305\n",
            "I1203 18:34:48.630121 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06305\n",
            "INFO:tensorflow:examples/sec: 34.0175\n",
            "I1203 18:34:48.630364 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0175\n",
            "INFO:tensorflow:global_step/sec: 1.06823\n",
            "I1203 18:34:49.566227 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06823\n",
            "INFO:tensorflow:examples/sec: 34.1833\n",
            "I1203 18:34:49.566614 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1833\n",
            "INFO:tensorflow:global_step/sec: 1.0627\n",
            "I1203 18:34:50.507247 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0627\n",
            "INFO:tensorflow:examples/sec: 34.0065\n",
            "I1203 18:34:50.507505 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0065\n",
            "INFO:tensorflow:global_step/sec: 1.06361\n",
            "I1203 18:34:51.447515 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06361\n",
            "INFO:tensorflow:examples/sec: 34.0355\n",
            "I1203 18:34:51.448207 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0355\n",
            "INFO:tensorflow:global_step/sec: 1.0588\n",
            "I1203 18:34:52.391903 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0588\n",
            "INFO:tensorflow:examples/sec: 33.8814\n",
            "I1203 18:34:52.392144 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8814\n",
            "INFO:tensorflow:global_step/sec: 1.06181\n",
            "I1203 18:34:53.333679 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06181\n",
            "INFO:tensorflow:examples/sec: 33.9779\n",
            "I1203 18:34:53.334100 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9779\n",
            "INFO:tensorflow:global_step/sec: 1.06194\n",
            "I1203 18:34:54.275384 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06194\n",
            "INFO:tensorflow:examples/sec: 33.982\n",
            "I1203 18:34:54.275674 140409631090560 tpu_estimator.py:2308] examples/sec: 33.982\n",
            "INFO:tensorflow:global_step/sec: 1.06373\n",
            "I1203 18:34:55.215477 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06373\n",
            "INFO:tensorflow:examples/sec: 34.0395\n",
            "I1203 18:34:55.215785 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0395\n",
            "INFO:tensorflow:global_step/sec: 1.05497\n",
            "I1203 18:34:56.163340 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05497\n",
            "INFO:tensorflow:examples/sec: 33.7591\n",
            "I1203 18:34:56.163602 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7591\n",
            "INFO:tensorflow:global_step/sec: 1.06417\n",
            "I1203 18:34:57.103022 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06417\n",
            "INFO:tensorflow:examples/sec: 34.0535\n",
            "I1203 18:34:57.103242 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0535\n",
            "INFO:tensorflow:global_step/sec: 1.06487\n",
            "I1203 18:34:58.042116 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06487\n",
            "INFO:tensorflow:examples/sec: 34.0757\n",
            "I1203 18:34:58.042356 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0757\n",
            "INFO:tensorflow:global_step/sec: 1.06392\n",
            "I1203 18:34:58.982052 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06392\n",
            "INFO:tensorflow:examples/sec: 34.0454\n",
            "I1203 18:34:58.982418 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0454\n",
            "INFO:tensorflow:global_step/sec: 1.06101\n",
            "I1203 18:34:59.924554 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06101\n",
            "INFO:tensorflow:examples/sec: 33.9522\n",
            "I1203 18:34:59.924788 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9522\n",
            "INFO:tensorflow:global_step/sec: 1.06845\n",
            "I1203 18:35:00.860540 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06845\n",
            "INFO:tensorflow:examples/sec: 34.1903\n",
            "I1203 18:35:00.860926 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1903\n",
            "INFO:tensorflow:global_step/sec: 1.06007\n",
            "I1203 18:35:01.803851 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06007\n",
            "INFO:tensorflow:examples/sec: 33.9222\n",
            "I1203 18:35:01.804469 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9222\n",
            "INFO:tensorflow:global_step/sec: 1.05918\n",
            "I1203 18:35:02.747946 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05918\n",
            "INFO:tensorflow:examples/sec: 33.8938\n",
            "I1203 18:35:02.748375 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8938\n",
            "INFO:tensorflow:global_step/sec: 1.05906\n",
            "I1203 18:35:03.692172 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05906\n",
            "INFO:tensorflow:examples/sec: 33.8901\n",
            "I1203 18:35:03.692605 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8901\n",
            "INFO:tensorflow:global_step/sec: 1.06493\n",
            "I1203 18:35:04.631219 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06493\n",
            "INFO:tensorflow:examples/sec: 34.0778\n",
            "I1203 18:35:04.631612 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0778\n",
            "INFO:tensorflow:global_step/sec: 1.06561\n",
            "I1203 18:35:05.569619 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06561\n",
            "INFO:tensorflow:examples/sec: 34.0997\n",
            "I1203 18:35:05.569845 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0997\n",
            "INFO:tensorflow:global_step/sec: 1.05854\n",
            "I1203 18:35:06.514338 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05854\n",
            "INFO:tensorflow:examples/sec: 33.8731\n",
            "I1203 18:35:06.514591 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8731\n",
            "INFO:tensorflow:global_step/sec: 1.06698\n",
            "I1203 18:35:07.451570 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06698\n",
            "INFO:tensorflow:examples/sec: 34.1435\n",
            "I1203 18:35:07.451822 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1435\n",
            "INFO:tensorflow:global_step/sec: 1.06235\n",
            "I1203 18:35:08.392853 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06235\n",
            "INFO:tensorflow:examples/sec: 33.9953\n",
            "I1203 18:35:08.393095 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9953\n",
            "INFO:tensorflow:global_step/sec: 1.06359\n",
            "I1203 18:35:09.333072 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06359\n",
            "INFO:tensorflow:examples/sec: 34.0348\n",
            "I1203 18:35:09.333496 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0348\n",
            "INFO:tensorflow:global_step/sec: 1.05775\n",
            "I1203 18:35:10.278490 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05775\n",
            "INFO:tensorflow:examples/sec: 33.8481\n",
            "I1203 18:35:10.278869 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8481\n",
            "INFO:tensorflow:global_step/sec: 1.06802\n",
            "I1203 18:35:11.214776 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06802\n",
            "INFO:tensorflow:examples/sec: 34.1765\n",
            "I1203 18:35:11.215001 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1765\n",
            "INFO:tensorflow:global_step/sec: 1.05918\n",
            "I1203 18:35:12.158912 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05918\n",
            "INFO:tensorflow:examples/sec: 33.8939\n",
            "I1203 18:35:12.159284 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8939\n",
            "INFO:tensorflow:global_step/sec: 1.06243\n",
            "I1203 18:35:13.100141 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06243\n",
            "INFO:tensorflow:examples/sec: 33.9977\n",
            "I1203 18:35:13.100366 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9977\n",
            "INFO:tensorflow:global_step/sec: 1.06114\n",
            "I1203 18:35:14.042531 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06114\n",
            "INFO:tensorflow:examples/sec: 33.9565\n",
            "I1203 18:35:14.042758 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9565\n",
            "INFO:tensorflow:global_step/sec: 1.06567\n",
            "I1203 18:35:14.980927 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06567\n",
            "INFO:tensorflow:examples/sec: 34.1015\n",
            "I1203 18:35:14.981201 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1015\n",
            "INFO:tensorflow:global_step/sec: 1.06212\n",
            "I1203 18:35:15.922445 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06212\n",
            "INFO:tensorflow:examples/sec: 33.9878\n",
            "I1203 18:35:15.922674 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9878\n",
            "INFO:tensorflow:global_step/sec: 1.06161\n",
            "I1203 18:35:16.864392 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06161\n",
            "INFO:tensorflow:examples/sec: 33.9716\n",
            "I1203 18:35:16.864649 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9716\n",
            "INFO:tensorflow:global_step/sec: 1.06085\n",
            "I1203 18:35:17.807028 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06085\n",
            "INFO:tensorflow:examples/sec: 33.9471\n",
            "I1203 18:35:17.807263 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9471\n",
            "INFO:tensorflow:global_step/sec: 1.06352\n",
            "I1203 18:35:18.747310 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06352\n",
            "INFO:tensorflow:examples/sec: 34.0326\n",
            "I1203 18:35:18.747774 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0326\n",
            "INFO:tensorflow:global_step/sec: 1.06421\n",
            "I1203 18:35:19.687002 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06421\n",
            "INFO:tensorflow:examples/sec: 34.0548\n",
            "I1203 18:35:19.687235 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0548\n",
            "INFO:tensorflow:global_step/sec: 1.06199\n",
            "I1203 18:35:20.628612 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06199\n",
            "INFO:tensorflow:examples/sec: 33.9836\n",
            "I1203 18:35:20.628848 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9836\n",
            "INFO:tensorflow:global_step/sec: 1.06478\n",
            "I1203 18:35:21.567755 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06478\n",
            "INFO:tensorflow:examples/sec: 34.0731\n",
            "I1203 18:35:21.567988 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0731\n",
            "INFO:tensorflow:global_step/sec: 1.06458\n",
            "I1203 18:35:22.507096 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06458\n",
            "INFO:tensorflow:examples/sec: 34.0664\n",
            "I1203 18:35:22.507519 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0664\n",
            "INFO:tensorflow:global_step/sec: 1.06579\n",
            "I1203 18:35:23.445366 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06579\n",
            "INFO:tensorflow:examples/sec: 34.1054\n",
            "I1203 18:35:23.445762 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1054\n",
            "INFO:tensorflow:global_step/sec: 1.06107\n",
            "I1203 18:35:24.387803 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06107\n",
            "INFO:tensorflow:examples/sec: 33.9542\n",
            "I1203 18:35:24.388035 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9542\n",
            "INFO:tensorflow:global_step/sec: 1.06421\n",
            "I1203 18:35:25.327484 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06421\n",
            "INFO:tensorflow:examples/sec: 34.0546\n",
            "I1203 18:35:25.327712 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0546\n",
            "INFO:tensorflow:global_step/sec: 1.05895\n",
            "I1203 18:35:26.271807 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05895\n",
            "INFO:tensorflow:examples/sec: 33.8863\n",
            "I1203 18:35:26.272256 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8863\n",
            "INFO:tensorflow:global_step/sec: 1.06411\n",
            "I1203 18:35:27.211601 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06411\n",
            "INFO:tensorflow:examples/sec: 34.0515\n",
            "I1203 18:35:27.211970 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0515\n",
            "INFO:tensorflow:global_step/sec: 1.05081\n",
            "I1203 18:35:28.163221 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05081\n",
            "INFO:tensorflow:examples/sec: 33.626\n",
            "I1203 18:35:28.163480 140409631090560 tpu_estimator.py:2308] examples/sec: 33.626\n",
            "INFO:tensorflow:global_step/sec: 1.06524\n",
            "I1203 18:35:29.101986 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06524\n",
            "INFO:tensorflow:examples/sec: 34.0878\n",
            "I1203 18:35:29.102411 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0878\n",
            "INFO:tensorflow:global_step/sec: 1.0698\n",
            "I1203 18:35:30.036728 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0698\n",
            "INFO:tensorflow:examples/sec: 34.2335\n",
            "I1203 18:35:30.036968 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2335\n",
            "INFO:tensorflow:global_step/sec: 1.06219\n",
            "I1203 18:35:30.978187 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06219\n",
            "INFO:tensorflow:examples/sec: 33.99\n",
            "I1203 18:35:30.978639 140409631090560 tpu_estimator.py:2308] examples/sec: 33.99\n",
            "INFO:tensorflow:global_step/sec: 1.06087\n",
            "I1203 18:35:31.920818 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06087\n",
            "INFO:tensorflow:examples/sec: 33.9477\n",
            "I1203 18:35:31.921212 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9477\n",
            "INFO:tensorflow:global_step/sec: 1.06553\n",
            "I1203 18:35:32.859316 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06553\n",
            "INFO:tensorflow:examples/sec: 34.0969\n",
            "I1203 18:35:32.859572 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0969\n",
            "INFO:tensorflow:global_step/sec: 1.06126\n",
            "I1203 18:35:33.801598 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06126\n",
            "INFO:tensorflow:examples/sec: 33.9603\n",
            "I1203 18:35:33.802088 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9603\n",
            "INFO:tensorflow:global_step/sec: 1.05784\n",
            "I1203 18:35:34.746927 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05784\n",
            "INFO:tensorflow:examples/sec: 33.8508\n",
            "I1203 18:35:34.747162 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8508\n",
            "INFO:tensorflow:global_step/sec: 1.07031\n",
            "I1203 18:35:35.681219 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07031\n",
            "INFO:tensorflow:examples/sec: 34.2499\n",
            "I1203 18:35:35.681496 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2499\n",
            "INFO:tensorflow:global_step/sec: 1.06085\n",
            "I1203 18:35:36.623844 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06085\n",
            "INFO:tensorflow:examples/sec: 33.9473\n",
            "I1203 18:35:36.624083 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9473\n",
            "INFO:tensorflow:global_step/sec: 1.06337\n",
            "I1203 18:35:37.564248 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06337\n",
            "INFO:tensorflow:examples/sec: 34.0277\n",
            "I1203 18:35:37.564491 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0277\n",
            "INFO:tensorflow:global_step/sec: 1.0595\n",
            "I1203 18:35:38.508097 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0595\n",
            "INFO:tensorflow:examples/sec: 33.9039\n",
            "I1203 18:35:38.508480 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9039\n",
            "INFO:tensorflow:global_step/sec: 1.05705\n",
            "I1203 18:35:39.454137 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05705\n",
            "INFO:tensorflow:examples/sec: 33.8257\n",
            "I1203 18:35:39.454540 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8257\n",
            "INFO:tensorflow:global_step/sec: 1.06868\n",
            "I1203 18:35:40.389860 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06868\n",
            "INFO:tensorflow:examples/sec: 34.1977\n",
            "I1203 18:35:40.390093 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1977\n",
            "INFO:tensorflow:global_step/sec: 1.06056\n",
            "I1203 18:35:41.332766 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06056\n",
            "INFO:tensorflow:examples/sec: 33.9379\n",
            "I1203 18:35:41.333025 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9379\n",
            "INFO:tensorflow:global_step/sec: 1.0629\n",
            "I1203 18:35:42.273590 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0629\n",
            "INFO:tensorflow:examples/sec: 34.0127\n",
            "I1203 18:35:42.273813 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0127\n",
            "INFO:tensorflow:global_step/sec: 1.06032\n",
            "I1203 18:35:43.216702 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06032\n",
            "INFO:tensorflow:examples/sec: 33.9302\n",
            "I1203 18:35:43.217061 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9302\n",
            "INFO:tensorflow:global_step/sec: 1.05891\n",
            "I1203 18:35:44.161072 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05891\n",
            "INFO:tensorflow:examples/sec: 33.885\n",
            "I1203 18:35:44.161295 140409631090560 tpu_estimator.py:2308] examples/sec: 33.885\n",
            "INFO:tensorflow:global_step/sec: 1.0672\n",
            "I1203 18:35:45.098138 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0672\n",
            "INFO:tensorflow:examples/sec: 34.1505\n",
            "I1203 18:35:45.098416 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1505\n",
            "INFO:tensorflow:global_step/sec: 1.07135\n",
            "I1203 18:35:46.031470 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07135\n",
            "INFO:tensorflow:examples/sec: 34.2832\n",
            "I1203 18:35:46.031819 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2832\n",
            "INFO:tensorflow:global_step/sec: 1.0636\n",
            "I1203 18:35:46.971693 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0636\n",
            "INFO:tensorflow:examples/sec: 34.0353\n",
            "I1203 18:35:46.971978 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0353\n",
            "INFO:tensorflow:global_step/sec: 1.06237\n",
            "I1203 18:35:47.913001 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06237\n",
            "INFO:tensorflow:examples/sec: 33.9958\n",
            "I1203 18:35:47.913243 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9958\n",
            "INFO:tensorflow:global_step/sec: 1.06232\n",
            "I1203 18:35:48.854318 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06232\n",
            "INFO:tensorflow:examples/sec: 33.9942\n",
            "I1203 18:35:48.854695 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9942\n",
            "INFO:tensorflow:global_step/sec: 1.05542\n",
            "I1203 18:35:49.801825 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05542\n",
            "INFO:tensorflow:examples/sec: 33.7733\n",
            "I1203 18:35:49.802083 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7733\n",
            "INFO:tensorflow:global_step/sec: 1.06188\n",
            "I1203 18:35:50.743564 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06188\n",
            "INFO:tensorflow:examples/sec: 33.9801\n",
            "I1203 18:35:50.743783 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9801\n",
            "INFO:tensorflow:global_step/sec: 1.0584\n",
            "I1203 18:35:51.688374 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0584\n",
            "INFO:tensorflow:examples/sec: 33.8689\n",
            "I1203 18:35:51.688634 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8689\n",
            "INFO:tensorflow:global_step/sec: 1.05693\n",
            "I1203 18:35:52.634510 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05693\n",
            "INFO:tensorflow:examples/sec: 33.8219\n",
            "I1203 18:35:52.634870 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8219\n",
            "INFO:tensorflow:global_step/sec: 1.067\n",
            "I1203 18:35:53.571714 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.067\n",
            "INFO:tensorflow:examples/sec: 34.1439\n",
            "I1203 18:35:53.572111 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1439\n",
            "INFO:tensorflow:global_step/sec: 1.06071\n",
            "I1203 18:35:54.514504 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06071\n",
            "INFO:tensorflow:examples/sec: 33.9427\n",
            "I1203 18:35:54.514738 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9427\n",
            "INFO:tensorflow:global_step/sec: 1.06435\n",
            "I1203 18:35:55.454027 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06435\n",
            "INFO:tensorflow:examples/sec: 34.0591\n",
            "I1203 18:35:55.454381 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0591\n",
            "INFO:tensorflow:global_step/sec: 1.0555\n",
            "I1203 18:35:56.401440 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0555\n",
            "INFO:tensorflow:examples/sec: 33.7761\n",
            "I1203 18:35:56.401671 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7761\n",
            "INFO:tensorflow:global_step/sec: 1.06572\n",
            "I1203 18:35:57.339760 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06572\n",
            "INFO:tensorflow:examples/sec: 34.1029\n",
            "I1203 18:35:57.340036 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1029\n",
            "INFO:tensorflow:global_step/sec: 1.06029\n",
            "I1203 18:35:58.282964 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06029\n",
            "INFO:tensorflow:examples/sec: 33.9292\n",
            "I1203 18:35:58.283353 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9292\n",
            "INFO:tensorflow:global_step/sec: 1.06235\n",
            "I1203 18:35:59.224225 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06235\n",
            "INFO:tensorflow:examples/sec: 33.9952\n",
            "I1203 18:35:59.224470 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9952\n",
            "INFO:tensorflow:global_step/sec: 1.05826\n",
            "I1203 18:36:00.169178 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05826\n",
            "INFO:tensorflow:examples/sec: 33.8643\n",
            "I1203 18:36:00.169603 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8643\n",
            "INFO:tensorflow:global_step/sec: 1.06052\n",
            "I1203 18:36:01.112284 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06052\n",
            "INFO:tensorflow:examples/sec: 33.9368\n",
            "I1203 18:36:01.112704 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9368\n",
            "INFO:tensorflow:global_step/sec: 1.06848\n",
            "I1203 18:36:02.048007 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06848\n",
            "INFO:tensorflow:examples/sec: 34.1915\n",
            "I1203 18:36:02.048437 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1915\n",
            "INFO:tensorflow:global_step/sec: 1.06709\n",
            "I1203 18:36:02.985121 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06709\n",
            "INFO:tensorflow:examples/sec: 34.147\n",
            "I1203 18:36:02.985522 140409631090560 tpu_estimator.py:2308] examples/sec: 34.147\n",
            "INFO:tensorflow:global_step/sec: 1.06256\n",
            "I1203 18:36:03.926253 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06256\n",
            "INFO:tensorflow:examples/sec: 34.0018\n",
            "I1203 18:36:03.926530 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0018\n",
            "INFO:tensorflow:global_step/sec: 1.05527\n",
            "I1203 18:36:04.873871 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05527\n",
            "INFO:tensorflow:examples/sec: 33.7686\n",
            "I1203 18:36:04.874096 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7686\n",
            "INFO:tensorflow:global_step/sec: 1.06025\n",
            "I1203 18:36:05.817066 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06025\n",
            "INFO:tensorflow:examples/sec: 33.9279\n",
            "I1203 18:36:05.817308 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9279\n",
            "INFO:tensorflow:global_step/sec: 1.06521\n",
            "I1203 18:36:06.755838 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06521\n",
            "INFO:tensorflow:examples/sec: 34.0867\n",
            "I1203 18:36:06.756078 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0867\n",
            "INFO:tensorflow:global_step/sec: 1.06834\n",
            "I1203 18:36:07.691882 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06834\n",
            "INFO:tensorflow:examples/sec: 34.1869\n",
            "I1203 18:36:07.692168 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1869\n",
            "INFO:tensorflow:global_step/sec: 1.05875\n",
            "I1203 18:36:08.636381 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05875\n",
            "INFO:tensorflow:examples/sec: 33.8799\n",
            "I1203 18:36:08.636778 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8799\n",
            "INFO:tensorflow:global_step/sec: 1.0605\n",
            "I1203 18:36:09.579323 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0605\n",
            "INFO:tensorflow:examples/sec: 33.9361\n",
            "I1203 18:36:09.579768 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9361\n",
            "INFO:tensorflow:global_step/sec: 1.06663\n",
            "I1203 18:36:10.516853 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06663\n",
            "INFO:tensorflow:examples/sec: 34.1322\n",
            "I1203 18:36:10.517094 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1322\n",
            "INFO:tensorflow:global_step/sec: 1.0613\n",
            "I1203 18:36:11.459119 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0613\n",
            "INFO:tensorflow:examples/sec: 33.9615\n",
            "I1203 18:36:11.459535 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9615\n",
            "INFO:tensorflow:global_step/sec: 1.05911\n",
            "I1203 18:36:12.403298 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05911\n",
            "INFO:tensorflow:examples/sec: 33.8914\n",
            "I1203 18:36:12.403672 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8914\n",
            "INFO:tensorflow:global_step/sec: 1.06155\n",
            "I1203 18:36:13.345328 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06155\n",
            "INFO:tensorflow:examples/sec: 33.9697\n",
            "I1203 18:36:13.345592 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9697\n",
            "INFO:tensorflow:global_step/sec: 1.06518\n",
            "I1203 18:36:14.284146 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06518\n",
            "INFO:tensorflow:examples/sec: 34.0858\n",
            "I1203 18:36:14.284536 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0858\n",
            "INFO:tensorflow:global_step/sec: 1.0618\n",
            "I1203 18:36:15.225914 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0618\n",
            "INFO:tensorflow:examples/sec: 33.9775\n",
            "I1203 18:36:15.226291 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9775\n",
            "INFO:tensorflow:global_step/sec: 1.06149\n",
            "I1203 18:36:16.168025 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06149\n",
            "INFO:tensorflow:examples/sec: 33.9676\n",
            "I1203 18:36:16.168284 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9676\n",
            "INFO:tensorflow:global_step/sec: 1.06651\n",
            "I1203 18:36:17.105640 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06651\n",
            "INFO:tensorflow:examples/sec: 34.1283\n",
            "I1203 18:36:17.105885 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1283\n",
            "INFO:tensorflow:global_step/sec: 1.05774\n",
            "I1203 18:36:18.051060 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05774\n",
            "INFO:tensorflow:examples/sec: 33.8478\n",
            "I1203 18:36:18.051292 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8478\n",
            "INFO:tensorflow:global_step/sec: 1.06552\n",
            "I1203 18:36:18.989584 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06552\n",
            "INFO:tensorflow:examples/sec: 34.0965\n",
            "I1203 18:36:18.989829 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0965\n",
            "INFO:tensorflow:global_step/sec: 1.06071\n",
            "I1203 18:36:19.932316 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06071\n",
            "INFO:tensorflow:examples/sec: 33.9428\n",
            "I1203 18:36:19.932703 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9428\n",
            "INFO:tensorflow:global_step/sec: 1.06557\n",
            "I1203 18:36:20.870776 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06557\n",
            "INFO:tensorflow:examples/sec: 34.0983\n",
            "I1203 18:36:20.871012 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0983\n",
            "INFO:tensorflow:global_step/sec: 1.05804\n",
            "I1203 18:36:21.815945 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05804\n",
            "INFO:tensorflow:examples/sec: 33.8572\n",
            "I1203 18:36:21.816193 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8572\n",
            "INFO:tensorflow:Saving checkpoints for 4000 into /content/classification_results/1/model.ckpt.\n",
            "I1203 18:36:22.752021 140409631090560 basic_session_run_hooks.py:606] Saving checkpoints for 4000 into /content/classification_results/1/model.ckpt.\n",
            "WARNING:tensorflow:Entity <function file_based_input_fn_builder.<locals>.input_fn.<locals>.<lambda> at 0x7fb23580f378> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: module 'gast' has no attribute 'Str'\n",
            "W1203 18:36:46.589898 140409631090560 ag_logging.py:146] Entity <function file_based_input_fn_builder.<locals>.input_fn.<locals>.<lambda> at 0x7fb23580f378> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: module 'gast' has no attribute 'Str'\n",
            "INFO:tensorflow:Calling model_fn.\n",
            "I1203 18:36:46.605033 140409631090560 estimator.py:1148] Calling model_fn.\n",
            "INFO:tensorflow:Running eval on CPU\n",
            "I1203 18:36:46.605290 140409631090560 tpu_estimator.py:3124] Running eval on CPU\n",
            "INFO:tensorflow:*** Features ***\n",
            "I1203 18:36:46.605681 140409631090560 run_binary_classification.py:634] *** Features ***\n",
            "INFO:tensorflow:  name = input_ids, shape = (?, 128)\n",
            "I1203 18:36:46.605832 140409631090560 run_binary_classification.py:636]   name = input_ids, shape = (?, 128)\n",
            "INFO:tensorflow:  name = input_mask, shape = (?, 128)\n",
            "I1203 18:36:46.605941 140409631090560 run_binary_classification.py:636]   name = input_mask, shape = (?, 128)\n",
            "INFO:tensorflow:  name = is_real_example, shape = (?,)\n",
            "I1203 18:36:46.606043 140409631090560 run_binary_classification.py:636]   name = is_real_example, shape = (?,)\n",
            "INFO:tensorflow:  name = label_ids, shape = (?,)\n",
            "I1203 18:36:46.606141 140409631090560 run_binary_classification.py:636]   name = label_ids, shape = (?,)\n",
            "INFO:tensorflow:  name = segment_ids, shape = (?, 128)\n",
            "I1203 18:36:46.606242 140409631090560 run_binary_classification.py:636]   name = segment_ids, shape = (?, 128)\n",
            "INFO:tensorflow:**** Trainable Variables ****\n",
            "I1203 18:36:48.995954 140409631090560 run_binary_classification.py:670] **** Trainable Variables ****\n",
            "INFO:tensorflow:Done calling model_fn.\n",
            "I1203 18:36:49.132832 140409631090560 estimator.py:1150] Done calling model_fn.\n",
            "INFO:tensorflow:Starting evaluation at 2020-12-03T18:36:49Z\n",
            "I1203 18:36:49.148274 140409631090560 evaluation.py:255] Starting evaluation at 2020-12-03T18:36:49Z\n",
            "INFO:tensorflow:Graph was finalized.\n",
            "I1203 18:36:49.566583 140409631090560 monitored_session.py:240] Graph was finalized.\n",
            "2020-12-03 18:36:49.567206: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 18:36:49.567587: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 0 with properties: \n",
            "name: Tesla T4 major: 7 minor: 5 memoryClockRate(GHz): 1.59\n",
            "pciBusID: 0000:00:04.0\n",
            "2020-12-03 18:36:49.567686: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1\n",
            "2020-12-03 18:36:49.567712: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10\n",
            "2020-12-03 18:36:49.567733: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10\n",
            "2020-12-03 18:36:49.567753: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10\n",
            "2020-12-03 18:36:49.567773: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10\n",
            "2020-12-03 18:36:49.567792: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10\n",
            "2020-12-03 18:36:49.567812: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7\n",
            "2020-12-03 18:36:49.567902: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 18:36:49.568206: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 18:36:49.568460: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1767] Adding visible gpu devices: 0\n",
            "2020-12-03 18:36:49.568514: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1180] Device interconnect StreamExecutor with strength 1 edge matrix:\n",
            "2020-12-03 18:36:49.568536: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1186]      0 \n",
            "2020-12-03 18:36:49.568546: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1199] 0:   N \n",
            "2020-12-03 18:36:49.568652: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 18:36:49.568952: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 18:36:49.569205: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1325] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14221 MB memory) -> physical GPU (device: 0, name: Tesla T4, pci bus id: 0000:00:04.0, compute capability: 7.5)\n",
            "INFO:tensorflow:Restoring parameters from /content/classification_results/1/model.ckpt-4000\n",
            "I1203 18:36:49.570348 140409631090560 saver.py:1284] Restoring parameters from /content/classification_results/1/model.ckpt-4000\n",
            "INFO:tensorflow:Running local_init_op.\n",
            "I1203 18:36:50.379458 140409631090560 session_manager.py:500] Running local_init_op.\n",
            "INFO:tensorflow:Done running local_init_op.\n",
            "I1203 18:36:50.454190 140409631090560 session_manager.py:502] Done running local_init_op.\n",
            "INFO:tensorflow:accuracy = 0.875, loss = 0.6732242\n",
            "I1203 18:36:51.295313 140409631090560 basic_session_run_hooks.py:262] accuracy = 0.875, loss = 0.6732242\n",
            "INFO:tensorflow:accuracy = 0.9375, loss = 0.00064904906 (0.590 sec)\n",
            "I1203 18:36:51.885780 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9375, loss = 0.00064904906 (0.590 sec)\n",
            "INFO:tensorflow:accuracy = 0.9583333, loss = 0.0005075184 (0.161 sec)\n",
            "I1203 18:36:52.046351 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9583333, loss = 0.0005075184 (0.161 sec)\n",
            "INFO:tensorflow:accuracy = 0.96875, loss = 0.016284857 (0.143 sec)\n",
            "I1203 18:36:52.189810 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.96875, loss = 0.016284857 (0.143 sec)\n",
            "INFO:tensorflow:accuracy = 0.975, loss = 0.0009241786 (0.144 sec)\n",
            "I1203 18:36:52.334253 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.975, loss = 0.0009241786 (0.144 sec)\n",
            "INFO:tensorflow:accuracy = 0.9791667, loss = 0.0017422094 (0.143 sec)\n",
            "I1203 18:36:52.477501 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9791667, loss = 0.0017422094 (0.143 sec)\n",
            "INFO:tensorflow:accuracy = 0.96428573, loss = 0.97133476 (0.144 sec)\n",
            "I1203 18:36:52.621630 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.96428573, loss = 0.97133476 (0.144 sec)\n",
            "INFO:tensorflow:accuracy = 0.9375, loss = 1.5467052 (0.145 sec)\n",
            "I1203 18:36:52.766461 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9375, loss = 1.5467052 (0.145 sec)\n",
            "INFO:tensorflow:accuracy = 0.9444444, loss = 0.008652742 (0.153 sec)\n",
            "I1203 18:36:52.919229 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9444444, loss = 0.008652742 (0.153 sec)\n",
            "INFO:tensorflow:accuracy = 0.9375, loss = 0.65467876 (0.147 sec)\n",
            "I1203 18:36:53.066157 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9375, loss = 0.65467876 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.9318182, loss = 0.65335464 (0.144 sec)\n",
            "I1203 18:36:53.210515 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9318182, loss = 0.65335464 (0.144 sec)\n",
            "INFO:tensorflow:accuracy = 0.9270833, loss = 0.27429944 (0.143 sec)\n",
            "I1203 18:36:53.353893 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9270833, loss = 0.27429944 (0.143 sec)\n",
            "INFO:tensorflow:accuracy = 0.9326923, loss = 0.00061718584 (0.143 sec)\n",
            "I1203 18:36:53.497319 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9326923, loss = 0.00061718584 (0.143 sec)\n",
            "INFO:tensorflow:accuracy = 0.9285714, loss = 0.6716489 (0.144 sec)\n",
            "I1203 18:36:53.641697 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9285714, loss = 0.6716489 (0.144 sec)\n",
            "INFO:tensorflow:accuracy = 0.925, loss = 0.6590882 (0.146 sec)\n",
            "I1203 18:36:53.787590 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.925, loss = 0.6590882 (0.146 sec)\n",
            "INFO:tensorflow:accuracy = 0.9296875, loss = 0.0009887607 (0.147 sec)\n",
            "I1203 18:36:53.934813 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9296875, loss = 0.0009887607 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.9338235, loss = 0.0012848617 (0.149 sec)\n",
            "I1203 18:36:54.083677 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9338235, loss = 0.0012848617 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.9305556, loss = 0.2506947 (0.147 sec)\n",
            "I1203 18:36:54.230432 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9305556, loss = 0.2506947 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.93421054, loss = 0.0039076903 (0.148 sec)\n",
            "I1203 18:36:54.378105 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.93421054, loss = 0.0039076903 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.93125, loss = 0.3869517 (0.144 sec)\n",
            "I1203 18:36:54.522312 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.93125, loss = 0.3869517 (0.144 sec)\n",
            "INFO:tensorflow:accuracy = 0.9345238, loss = 0.00096157676 (0.144 sec)\n",
            "I1203 18:36:54.666238 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9345238, loss = 0.00096157676 (0.144 sec)\n",
            "INFO:tensorflow:accuracy = 0.9375, loss = 0.0010192535 (0.151 sec)\n",
            "I1203 18:36:54.817168 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9375, loss = 0.0010192535 (0.151 sec)\n",
            "INFO:tensorflow:accuracy = 0.9293478, loss = 0.7038614 (0.149 sec)\n",
            "I1203 18:36:54.965870 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9293478, loss = 0.7038614 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.9270833, loss = 0.6755592 (0.147 sec)\n",
            "I1203 18:36:55.112887 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9270833, loss = 0.6755592 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.93, loss = 0.0005353385 (0.147 sec)\n",
            "I1203 18:36:55.259665 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.93, loss = 0.0005353385 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.9326923, loss = 0.00045275857 (0.144 sec)\n",
            "I1203 18:36:55.403458 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9326923, loss = 0.00045275857 (0.144 sec)\n",
            "INFO:tensorflow:accuracy = 0.9351852, loss = 0.0012250966 (0.147 sec)\n",
            "I1203 18:36:55.550914 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9351852, loss = 0.0012250966 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.9375, loss = 0.013370832 (0.143 sec)\n",
            "I1203 18:36:55.694215 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9375, loss = 0.013370832 (0.143 sec)\n",
            "INFO:tensorflow:accuracy = 0.9310345, loss = 1.6111926 (0.147 sec)\n",
            "I1203 18:36:55.841519 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9310345, loss = 1.6111926 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.93333334, loss = 0.002384677 (0.148 sec)\n",
            "I1203 18:36:55.989371 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.93333334, loss = 0.002384677 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.9314516, loss = 0.68087 (0.148 sec)\n",
            "I1203 18:36:56.137177 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9314516, loss = 0.68087 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.93359375, loss = 0.012661007 (0.146 sec)\n",
            "I1203 18:36:56.283425 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.93359375, loss = 0.012661007 (0.146 sec)\n",
            "INFO:tensorflow:accuracy = 0.93560606, loss = 0.0011385097 (0.145 sec)\n",
            "I1203 18:36:56.428731 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.93560606, loss = 0.0011385097 (0.145 sec)\n",
            "INFO:tensorflow:accuracy = 0.9375, loss = 0.0035805244 (0.145 sec)\n",
            "I1203 18:36:56.573317 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9375, loss = 0.0035805244 (0.145 sec)\n",
            "INFO:tensorflow:accuracy = 0.9357143, loss = 0.85575813 (0.147 sec)\n",
            "I1203 18:36:56.719936 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9357143, loss = 0.85575813 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.9375, loss = 0.0346443 (0.147 sec)\n",
            "I1203 18:36:56.867368 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9375, loss = 0.0346443 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.9358108, loss = 0.5951911 (0.147 sec)\n",
            "I1203 18:36:57.014529 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9358108, loss = 0.5951911 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.9375, loss = 0.0015235123 (0.147 sec)\n",
            "I1203 18:36:57.162025 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9375, loss = 0.0015235123 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.9391026, loss = 0.0010765 (0.146 sec)\n",
            "I1203 18:36:57.308479 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9391026, loss = 0.0010765 (0.146 sec)\n",
            "INFO:tensorflow:accuracy = 0.940625, loss = 0.0005349083 (0.144 sec)\n",
            "I1203 18:36:57.452454 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.940625, loss = 0.0005349083 (0.144 sec)\n",
            "INFO:tensorflow:accuracy = 0.94207317, loss = 0.0005070385 (0.148 sec)\n",
            "I1203 18:36:57.599991 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.94207317, loss = 0.0005070385 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.94345236, loss = 0.0008010493 (0.149 sec)\n",
            "I1203 18:36:57.748668 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.94345236, loss = 0.0008010493 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.94186044, loss = 0.6733712 (0.147 sec)\n",
            "I1203 18:36:57.895727 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.94186044, loss = 0.6733712 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.9431818, loss = 0.00047975976 (0.147 sec)\n",
            "I1203 18:36:58.042823 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9431818, loss = 0.00047975976 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.9388889, loss = 1.7758052 (0.146 sec)\n",
            "I1203 18:36:58.189246 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9388889, loss = 1.7758052 (0.146 sec)\n",
            "INFO:tensorflow:accuracy = 0.9375, loss = 0.9622218 (0.152 sec)\n",
            "I1203 18:36:58.340996 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9375, loss = 0.9622218 (0.152 sec)\n",
            "INFO:tensorflow:accuracy = 0.93351066, loss = 0.9239096 (0.145 sec)\n",
            "I1203 18:36:58.486431 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.93351066, loss = 0.9239096 (0.145 sec)\n",
            "INFO:tensorflow:accuracy = 0.9348958, loss = 0.0004434794 (0.148 sec)\n",
            "I1203 18:36:58.633966 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9348958, loss = 0.0004434794 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.93622446, loss = 0.009253921 (0.147 sec)\n",
            "I1203 18:36:58.781379 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.93622446, loss = 0.009253921 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.9375, loss = 0.002577452 (0.148 sec)\n",
            "I1203 18:36:58.929608 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9375, loss = 0.002577452 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.9362745, loss = 0.45472813 (0.147 sec)\n",
            "I1203 18:36:59.076525 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9362745, loss = 0.45472813 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.93028843, loss = 1.4699527 (0.149 sec)\n",
            "I1203 18:36:59.225621 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.93028843, loss = 1.4699527 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.9316038, loss = 0.001700518 (0.148 sec)\n",
            "I1203 18:36:59.373212 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9316038, loss = 0.001700518 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.9282407, loss = 1.1969205 (0.147 sec)\n",
            "I1203 18:36:59.520199 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9282407, loss = 1.1969205 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.92045456, loss = 1.9663415 (0.145 sec)\n",
            "I1203 18:36:59.665553 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.92045456, loss = 1.9663415 (0.145 sec)\n",
            "INFO:tensorflow:accuracy = 0.921875, loss = 0.0019711908 (0.149 sec)\n",
            "I1203 18:36:59.814286 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.921875, loss = 0.0019711908 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.9232456, loss = 0.0016084126 (0.148 sec)\n",
            "I1203 18:36:59.962159 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9232456, loss = 0.0016084126 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.91810346, loss = 1.3971813 (0.148 sec)\n",
            "I1203 18:37:00.110613 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91810346, loss = 1.3971813 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.9194915, loss = 0.008770492 (0.150 sec)\n",
            "I1203 18:37:00.260203 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9194915, loss = 0.008770492 (0.150 sec)\n",
            "INFO:tensorflow:accuracy = 0.9166667, loss = 1.8782262 (0.147 sec)\n",
            "I1203 18:37:00.406989 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9166667, loss = 1.8782262 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.91803277, loss = 0.00059359777 (0.149 sec)\n",
            "I1203 18:37:00.556115 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91803277, loss = 0.00059359777 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.9173387, loss = 0.9640391 (0.147 sec)\n",
            "I1203 18:37:00.703180 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9173387, loss = 0.9640391 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.9166667, loss = 0.9676862 (0.148 sec)\n",
            "I1203 18:37:00.850724 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9166667, loss = 0.9676862 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.9160156, loss = 0.6754817 (0.147 sec)\n",
            "I1203 18:37:00.997788 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9160156, loss = 0.6754817 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.9153846, loss = 0.65604466 (0.148 sec)\n",
            "I1203 18:37:01.145694 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9153846, loss = 0.65604466 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.91477275, loss = 0.89816916 (0.149 sec)\n",
            "I1203 18:37:01.294667 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91477275, loss = 0.89816916 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.9160448, loss = 0.0021912102 (0.146 sec)\n",
            "I1203 18:37:01.440963 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9160448, loss = 0.0021912102 (0.146 sec)\n",
            "INFO:tensorflow:accuracy = 0.91360295, loss = 1.2467608 (0.147 sec)\n",
            "I1203 18:37:01.588126 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91360295, loss = 1.2467608 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.9130435, loss = 0.4087244 (0.148 sec)\n",
            "I1203 18:37:01.736271 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9130435, loss = 0.4087244 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.9142857, loss = 0.026179641 (0.151 sec)\n",
            "I1203 18:37:01.886783 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9142857, loss = 0.026179641 (0.151 sec)\n",
            "INFO:tensorflow:accuracy = 0.91549295, loss = 0.0010499756 (0.147 sec)\n",
            "I1203 18:37:02.033694 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91549295, loss = 0.0010499756 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.9166667, loss = 0.00065497786 (0.154 sec)\n",
            "I1203 18:37:02.187469 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9166667, loss = 0.00065497786 (0.154 sec)\n",
            "INFO:tensorflow:accuracy = 0.9160959, loss = 0.4231734 (0.146 sec)\n",
            "I1203 18:37:02.333333 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9160959, loss = 0.4231734 (0.146 sec)\n",
            "INFO:tensorflow:accuracy = 0.9155405, loss = 0.68154 (0.147 sec)\n",
            "I1203 18:37:02.480378 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9155405, loss = 0.68154 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.915, loss = 0.84988546 (0.152 sec)\n",
            "I1203 18:37:02.632537 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.915, loss = 0.84988546 (0.152 sec)\n",
            "INFO:tensorflow:accuracy = 0.91611844, loss = 0.0028341045 (0.147 sec)\n",
            "I1203 18:37:02.779336 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91611844, loss = 0.0028341045 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.91558444, loss = 0.67867845 (0.149 sec)\n",
            "I1203 18:37:02.928184 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91558444, loss = 0.67867845 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.9102564, loss = 2.7329676 (0.150 sec)\n",
            "I1203 18:37:03.078089 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9102564, loss = 2.7329676 (0.150 sec)\n",
            "INFO:tensorflow:accuracy = 0.9113924, loss = 0.0018221828 (0.149 sec)\n",
            "I1203 18:37:03.227158 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9113924, loss = 0.0018221828 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.9125, loss = 0.00050355966 (0.148 sec)\n",
            "I1203 18:37:03.375231 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9125, loss = 0.00050355966 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.91358024, loss = 0.008959524 (0.149 sec)\n",
            "I1203 18:37:03.524056 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91358024, loss = 0.008959524 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.91463417, loss = 0.00048938056 (0.147 sec)\n",
            "I1203 18:37:03.670685 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91463417, loss = 0.00048938056 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.91566265, loss = 0.0004887715 (0.152 sec)\n",
            "I1203 18:37:03.822852 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91566265, loss = 0.0004887715 (0.152 sec)\n",
            "INFO:tensorflow:accuracy = 0.9151786, loss = 0.91329384 (0.146 sec)\n",
            "I1203 18:37:03.969304 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9151786, loss = 0.91329384 (0.146 sec)\n",
            "INFO:tensorflow:accuracy = 0.9132353, loss = 1.4770515 (0.149 sec)\n",
            "I1203 18:37:04.118457 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9132353, loss = 1.4770515 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.9127907, loss = 0.65909785 (0.152 sec)\n",
            "I1203 18:37:04.269981 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9127907, loss = 0.65909785 (0.152 sec)\n",
            "INFO:tensorflow:accuracy = 0.9123563, loss = 0.70522475 (0.149 sec)\n",
            "I1203 18:37:04.418635 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9123563, loss = 0.70522475 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.91335225, loss = 0.0042769946 (0.149 sec)\n",
            "I1203 18:37:04.567851 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91335225, loss = 0.0042769946 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.91292137, loss = 0.368994 (0.149 sec)\n",
            "I1203 18:37:04.717149 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91292137, loss = 0.368994 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.9125, loss = 0.6209937 (0.149 sec)\n",
            "I1203 18:37:04.866290 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9125, loss = 0.6209937 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.91346157, loss = 0.0005321362 (0.148 sec)\n",
            "I1203 18:37:05.014601 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91346157, loss = 0.0005321362 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.9144022, loss = 0.0010874459 (0.149 sec)\n",
            "I1203 18:37:05.163769 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9144022, loss = 0.0010874459 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.9153226, loss = 0.00045044994 (0.149 sec)\n",
            "I1203 18:37:05.312659 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9153226, loss = 0.00045044994 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.9148936, loss = 0.70081055 (0.149 sec)\n",
            "I1203 18:37:05.461624 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9148936, loss = 0.70081055 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.9131579, loss = 1.5480695 (0.148 sec)\n",
            "I1203 18:37:05.609955 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9131579, loss = 1.5480695 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.9133858, loss = 0.0005246216 (0.099 sec)\n",
            "I1203 18:37:05.709434 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9133858, loss = 0.0005246216 (0.099 sec)\n",
            "INFO:tensorflow:Finished evaluation at 2020-12-03-18:37:05\n",
            "I1203 18:37:05.836606 140409631090560 evaluation.py:275] Finished evaluation at 2020-12-03-18:37:05\n",
            "INFO:tensorflow:Saving dict for global step 4000: eval_accuracy = 0.89881736, eval_loss = 0.5394358, global_step = 4000, loss = 0.53731966\n",
            "I1203 18:37:05.836868 140409631090560 estimator.py:2049] Saving dict for global step 4000: eval_accuracy = 0.89881736, eval_loss = 0.5394358, global_step = 4000, loss = 0.53731966\n",
            "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 4000: /content/classification_results/1/model.ckpt-4000\n",
            "I1203 18:37:05.837366 140409631090560 estimator.py:2109] Saving 'checkpoint_path' summary for global step 4000: /content/classification_results/1/model.ckpt-4000\n",
            "INFO:tensorflow:evaluation_loop marked as finished\n",
            "I1203 18:37:05.838148 140409631090560 error_handling.py:101] evaluation_loop marked as finished\n",
            "INFO:tensorflow:global_step/sec: 0.0227138\n",
            "I1203 18:37:05.842067 140409631090560 tpu_estimator.py:2307] global_step/sec: 0.0227138\n",
            "INFO:tensorflow:examples/sec: 0.72684\n",
            "I1203 18:37:05.842227 140409631090560 tpu_estimator.py:2308] examples/sec: 0.72684\n",
            "INFO:tensorflow:global_step/sec: 1.0787\n",
            "I1203 18:37:06.769145 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0787\n",
            "INFO:tensorflow:examples/sec: 34.5183\n",
            "I1203 18:37:06.769379 140409631090560 tpu_estimator.py:2308] examples/sec: 34.5183\n",
            "INFO:tensorflow:global_step/sec: 1.06734\n",
            "I1203 18:37:07.706066 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06734\n",
            "INFO:tensorflow:examples/sec: 34.1548\n",
            "I1203 18:37:07.706446 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1548\n",
            "INFO:tensorflow:global_step/sec: 1.04989\n",
            "I1203 18:37:08.658557 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.04989\n",
            "INFO:tensorflow:examples/sec: 33.5965\n",
            "I1203 18:37:08.658782 140409631090560 tpu_estimator.py:2308] examples/sec: 33.5965\n",
            "INFO:tensorflow:global_step/sec: 1.04987\n",
            "I1203 18:37:09.611052 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.04987\n",
            "INFO:tensorflow:examples/sec: 33.5959\n",
            "I1203 18:37:09.611288 140409631090560 tpu_estimator.py:2308] examples/sec: 33.5959\n",
            "INFO:tensorflow:global_step/sec: 1.04641\n",
            "I1203 18:37:10.566689 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.04641\n",
            "INFO:tensorflow:examples/sec: 33.4851\n",
            "I1203 18:37:10.566925 140409631090560 tpu_estimator.py:2308] examples/sec: 33.4851\n",
            "INFO:tensorflow:global_step/sec: 1.04959\n",
            "I1203 18:37:11.519452 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.04959\n",
            "INFO:tensorflow:examples/sec: 33.587\n",
            "I1203 18:37:11.519846 140409631090560 tpu_estimator.py:2308] examples/sec: 33.587\n",
            "INFO:tensorflow:global_step/sec: 1.04689\n",
            "I1203 18:37:12.474651 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.04689\n",
            "INFO:tensorflow:examples/sec: 33.5005\n",
            "I1203 18:37:12.475008 140409631090560 tpu_estimator.py:2308] examples/sec: 33.5005\n",
            "INFO:tensorflow:global_step/sec: 1.03789\n",
            "I1203 18:37:13.438150 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.03789\n",
            "INFO:tensorflow:examples/sec: 33.2126\n",
            "I1203 18:37:13.438532 140409631090560 tpu_estimator.py:2308] examples/sec: 33.2126\n",
            "INFO:tensorflow:global_step/sec: 1.04662\n",
            "I1203 18:37:14.393636 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.04662\n",
            "INFO:tensorflow:examples/sec: 33.4919\n",
            "I1203 18:37:14.394024 140409631090560 tpu_estimator.py:2308] examples/sec: 33.4919\n",
            "INFO:tensorflow:global_step/sec: 1.0457\n",
            "I1203 18:37:15.349882 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0457\n",
            "INFO:tensorflow:examples/sec: 33.4623\n",
            "I1203 18:37:15.350121 140409631090560 tpu_estimator.py:2308] examples/sec: 33.4623\n",
            "INFO:tensorflow:global_step/sec: 1.03435\n",
            "I1203 18:37:16.316689 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.03435\n",
            "INFO:tensorflow:examples/sec: 33.0992\n",
            "I1203 18:37:16.316933 140409631090560 tpu_estimator.py:2308] examples/sec: 33.0992\n",
            "INFO:tensorflow:global_step/sec: 1.04048\n",
            "I1203 18:37:17.277788 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.04048\n",
            "INFO:tensorflow:examples/sec: 33.2952\n",
            "I1203 18:37:17.278033 140409631090560 tpu_estimator.py:2308] examples/sec: 33.2952\n",
            "INFO:tensorflow:global_step/sec: 1.03266\n",
            "I1203 18:37:18.246193 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.03266\n",
            "INFO:tensorflow:examples/sec: 33.045\n",
            "I1203 18:37:18.246463 140409631090560 tpu_estimator.py:2308] examples/sec: 33.045\n",
            "INFO:tensorflow:global_step/sec: 1.03212\n",
            "I1203 18:37:19.215036 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.03212\n",
            "INFO:tensorflow:examples/sec: 33.0279\n",
            "I1203 18:37:19.215278 140409631090560 tpu_estimator.py:2308] examples/sec: 33.0279\n",
            "INFO:tensorflow:global_step/sec: 1.03214\n",
            "I1203 18:37:20.183883 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.03214\n",
            "INFO:tensorflow:examples/sec: 33.0286\n",
            "I1203 18:37:20.184106 140409631090560 tpu_estimator.py:2308] examples/sec: 33.0286\n",
            "INFO:tensorflow:global_step/sec: 1.02804\n",
            "I1203 18:37:21.156631 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.02804\n",
            "INFO:tensorflow:examples/sec: 32.8972\n",
            "I1203 18:37:21.156894 140409631090560 tpu_estimator.py:2308] examples/sec: 32.8972\n",
            "INFO:tensorflow:global_step/sec: 1.02901\n",
            "I1203 18:37:22.128471 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.02901\n",
            "INFO:tensorflow:examples/sec: 32.9284\n",
            "I1203 18:37:22.128728 140409631090560 tpu_estimator.py:2308] examples/sec: 32.9284\n",
            "INFO:tensorflow:global_step/sec: 1.0271\n",
            "I1203 18:37:23.102045 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0271\n",
            "INFO:tensorflow:examples/sec: 32.8671\n",
            "I1203 18:37:23.102323 140409631090560 tpu_estimator.py:2308] examples/sec: 32.8671\n",
            "INFO:tensorflow:global_step/sec: 1.02827\n",
            "I1203 18:37:24.074556 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.02827\n",
            "INFO:tensorflow:examples/sec: 32.9047\n",
            "I1203 18:37:24.074908 140409631090560 tpu_estimator.py:2308] examples/sec: 32.9047\n",
            "INFO:tensorflow:global_step/sec: 1.02989\n",
            "I1203 18:37:25.045565 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.02989\n",
            "INFO:tensorflow:examples/sec: 32.9565\n",
            "I1203 18:37:25.046010 140409631090560 tpu_estimator.py:2308] examples/sec: 32.9565\n",
            "INFO:tensorflow:global_step/sec: 1.03112\n",
            "I1203 18:37:26.015351 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.03112\n",
            "INFO:tensorflow:examples/sec: 32.996\n",
            "I1203 18:37:26.015744 140409631090560 tpu_estimator.py:2308] examples/sec: 32.996\n",
            "INFO:tensorflow:global_step/sec: 1.03618\n",
            "I1203 18:37:26.980456 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.03618\n",
            "INFO:tensorflow:examples/sec: 33.1576\n",
            "I1203 18:37:26.980697 140409631090560 tpu_estimator.py:2308] examples/sec: 33.1576\n",
            "INFO:tensorflow:global_step/sec: 1.04162\n",
            "I1203 18:37:27.940483 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.04162\n",
            "INFO:tensorflow:examples/sec: 33.3317\n",
            "I1203 18:37:27.940843 140409631090560 tpu_estimator.py:2308] examples/sec: 33.3317\n",
            "INFO:tensorflow:global_step/sec: 1.04763\n",
            "I1203 18:37:28.895022 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.04763\n",
            "INFO:tensorflow:examples/sec: 33.5241\n",
            "I1203 18:37:28.895410 140409631090560 tpu_estimator.py:2308] examples/sec: 33.5241\n",
            "INFO:tensorflow:global_step/sec: 1.03222\n",
            "I1203 18:37:29.863796 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.03222\n",
            "INFO:tensorflow:examples/sec: 33.0311\n",
            "I1203 18:37:29.864155 140409631090560 tpu_estimator.py:2308] examples/sec: 33.0311\n",
            "INFO:tensorflow:global_step/sec: 1.05111\n",
            "I1203 18:37:30.815166 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05111\n",
            "INFO:tensorflow:examples/sec: 33.6357\n",
            "I1203 18:37:30.815388 140409631090560 tpu_estimator.py:2308] examples/sec: 33.6357\n",
            "INFO:tensorflow:global_step/sec: 1.05286\n",
            "I1203 18:37:31.764985 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05286\n",
            "INFO:tensorflow:examples/sec: 33.6915\n",
            "I1203 18:37:31.765217 140409631090560 tpu_estimator.py:2308] examples/sec: 33.6915\n",
            "INFO:tensorflow:global_step/sec: 1.05176\n",
            "I1203 18:37:32.715746 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05176\n",
            "INFO:tensorflow:examples/sec: 33.6564\n",
            "I1203 18:37:32.716135 140409631090560 tpu_estimator.py:2308] examples/sec: 33.6564\n",
            "INFO:tensorflow:global_step/sec: 1.05441\n",
            "I1203 18:37:33.664160 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05441\n",
            "INFO:tensorflow:examples/sec: 33.741\n",
            "I1203 18:37:33.664525 140409631090560 tpu_estimator.py:2308] examples/sec: 33.741\n",
            "INFO:tensorflow:global_step/sec: 1.05221\n",
            "I1203 18:37:34.614564 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05221\n",
            "INFO:tensorflow:examples/sec: 33.6706\n",
            "I1203 18:37:34.614802 140409631090560 tpu_estimator.py:2308] examples/sec: 33.6706\n",
            "INFO:tensorflow:global_step/sec: 1.05431\n",
            "I1203 18:37:35.563065 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05431\n",
            "INFO:tensorflow:examples/sec: 33.7379\n",
            "I1203 18:37:35.563485 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7379\n",
            "INFO:tensorflow:global_step/sec: 1.05383\n",
            "I1203 18:37:36.511949 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05383\n",
            "INFO:tensorflow:examples/sec: 33.7226\n",
            "I1203 18:37:36.512319 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7226\n",
            "INFO:tensorflow:global_step/sec: 1.05467\n",
            "I1203 18:37:37.460098 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05467\n",
            "INFO:tensorflow:examples/sec: 33.7495\n",
            "I1203 18:37:37.460561 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7495\n",
            "INFO:tensorflow:global_step/sec: 1.06462\n",
            "I1203 18:37:38.399420 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06462\n",
            "INFO:tensorflow:examples/sec: 34.0678\n",
            "I1203 18:37:38.399655 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0678\n",
            "INFO:tensorflow:global_step/sec: 1.05086\n",
            "I1203 18:37:39.351023 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05086\n",
            "INFO:tensorflow:examples/sec: 33.6276\n",
            "I1203 18:37:39.351439 140409631090560 tpu_estimator.py:2308] examples/sec: 33.6276\n",
            "INFO:tensorflow:global_step/sec: 1.06495\n",
            "I1203 18:37:40.290033 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06495\n",
            "INFO:tensorflow:examples/sec: 34.0783\n",
            "I1203 18:37:40.290378 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0783\n",
            "INFO:tensorflow:global_step/sec: 1.06914\n",
            "I1203 18:37:41.225347 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06914\n",
            "INFO:tensorflow:examples/sec: 34.2125\n",
            "I1203 18:37:41.225612 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2125\n",
            "INFO:tensorflow:global_step/sec: 1.07183\n",
            "I1203 18:37:42.158345 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07183\n",
            "INFO:tensorflow:examples/sec: 34.2987\n",
            "I1203 18:37:42.158601 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2987\n",
            "INFO:tensorflow:global_step/sec: 1.06731\n",
            "I1203 18:37:43.095261 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06731\n",
            "INFO:tensorflow:examples/sec: 34.1539\n",
            "I1203 18:37:43.095961 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1539\n",
            "INFO:tensorflow:global_step/sec: 1.06141\n",
            "I1203 18:37:44.037436 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06141\n",
            "INFO:tensorflow:examples/sec: 33.965\n",
            "I1203 18:37:44.037706 140409631090560 tpu_estimator.py:2308] examples/sec: 33.965\n",
            "INFO:tensorflow:global_step/sec: 1.06205\n",
            "I1203 18:37:44.978990 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06205\n",
            "INFO:tensorflow:examples/sec: 33.9855\n",
            "I1203 18:37:44.979242 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9855\n",
            "INFO:tensorflow:global_step/sec: 1.07004\n",
            "I1203 18:37:45.913549 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07004\n",
            "INFO:tensorflow:examples/sec: 34.2414\n",
            "I1203 18:37:45.913787 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2414\n",
            "INFO:tensorflow:global_step/sec: 1.06444\n",
            "I1203 18:37:46.852985 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06444\n",
            "INFO:tensorflow:examples/sec: 34.062\n",
            "I1203 18:37:46.853207 140409631090560 tpu_estimator.py:2308] examples/sec: 34.062\n",
            "INFO:tensorflow:global_step/sec: 1.06603\n",
            "I1203 18:37:47.791045 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06603\n",
            "INFO:tensorflow:examples/sec: 34.1129\n",
            "I1203 18:37:47.791434 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1129\n",
            "INFO:tensorflow:global_step/sec: 1.06243\n",
            "I1203 18:37:48.732308 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06243\n",
            "INFO:tensorflow:examples/sec: 33.9979\n",
            "I1203 18:37:48.732576 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9979\n",
            "INFO:tensorflow:global_step/sec: 1.0733\n",
            "I1203 18:37:49.664003 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0733\n",
            "INFO:tensorflow:examples/sec: 34.3457\n",
            "I1203 18:37:49.664384 140409631090560 tpu_estimator.py:2308] examples/sec: 34.3457\n",
            "INFO:tensorflow:global_step/sec: 1.07504\n",
            "I1203 18:37:50.594208 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07504\n",
            "INFO:tensorflow:examples/sec: 34.4014\n",
            "I1203 18:37:50.594638 140409631090560 tpu_estimator.py:2308] examples/sec: 34.4014\n",
            "INFO:tensorflow:global_step/sec: 1.07617\n",
            "I1203 18:37:51.523422 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07617\n",
            "INFO:tensorflow:examples/sec: 34.4373\n",
            "I1203 18:37:51.523669 140409631090560 tpu_estimator.py:2308] examples/sec: 34.4373\n",
            "INFO:tensorflow:global_step/sec: 1.07432\n",
            "I1203 18:37:52.454271 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07432\n",
            "INFO:tensorflow:examples/sec: 34.3783\n",
            "I1203 18:37:52.454525 140409631090560 tpu_estimator.py:2308] examples/sec: 34.3783\n",
            "INFO:tensorflow:global_step/sec: 1.07358\n",
            "I1203 18:37:53.385720 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07358\n",
            "INFO:tensorflow:examples/sec: 34.3545\n",
            "I1203 18:37:53.385959 140409631090560 tpu_estimator.py:2308] examples/sec: 34.3545\n",
            "INFO:tensorflow:global_step/sec: 1.07456\n",
            "I1203 18:37:54.316309 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07456\n",
            "INFO:tensorflow:examples/sec: 34.386\n",
            "I1203 18:37:54.316687 140409631090560 tpu_estimator.py:2308] examples/sec: 34.386\n",
            "INFO:tensorflow:global_step/sec: 1.0725\n",
            "I1203 18:37:55.248701 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0725\n",
            "INFO:tensorflow:examples/sec: 34.3201\n",
            "I1203 18:37:55.248924 140409631090560 tpu_estimator.py:2308] examples/sec: 34.3201\n",
            "INFO:tensorflow:global_step/sec: 1.07251\n",
            "I1203 18:37:56.181115 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07251\n",
            "INFO:tensorflow:examples/sec: 34.3204\n",
            "I1203 18:37:56.181374 140409631090560 tpu_estimator.py:2308] examples/sec: 34.3204\n",
            "INFO:tensorflow:global_step/sec: 1.07072\n",
            "I1203 18:37:57.115064 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07072\n",
            "INFO:tensorflow:examples/sec: 34.2631\n",
            "I1203 18:37:57.115306 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2631\n",
            "INFO:tensorflow:global_step/sec: 1.0777\n",
            "I1203 18:37:58.042951 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0777\n",
            "INFO:tensorflow:examples/sec: 34.4863\n",
            "I1203 18:37:58.043200 140409631090560 tpu_estimator.py:2308] examples/sec: 34.4863\n",
            "INFO:tensorflow:global_step/sec: 1.07582\n",
            "I1203 18:37:58.972491 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07582\n",
            "INFO:tensorflow:examples/sec: 34.4262\n",
            "I1203 18:37:58.972743 140409631090560 tpu_estimator.py:2308] examples/sec: 34.4262\n",
            "INFO:tensorflow:global_step/sec: 1.07483\n",
            "I1203 18:37:59.902850 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07483\n",
            "INFO:tensorflow:examples/sec: 34.3945\n",
            "I1203 18:37:59.903089 140409631090560 tpu_estimator.py:2308] examples/sec: 34.3945\n",
            "INFO:tensorflow:global_step/sec: 1.07672\n",
            "I1203 18:38:00.831612 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07672\n",
            "INFO:tensorflow:examples/sec: 34.4551\n",
            "I1203 18:38:00.831853 140409631090560 tpu_estimator.py:2308] examples/sec: 34.4551\n",
            "INFO:tensorflow:global_step/sec: 1.07489\n",
            "I1203 18:38:01.761927 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07489\n",
            "INFO:tensorflow:examples/sec: 34.3966\n",
            "I1203 18:38:01.762309 140409631090560 tpu_estimator.py:2308] examples/sec: 34.3966\n",
            "INFO:tensorflow:global_step/sec: 1.06522\n",
            "I1203 18:38:02.700653 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06522\n",
            "INFO:tensorflow:examples/sec: 34.0871\n",
            "I1203 18:38:02.701010 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0871\n",
            "INFO:tensorflow:global_step/sec: 1.07986\n",
            "I1203 18:38:03.626746 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07986\n",
            "INFO:tensorflow:examples/sec: 34.5555\n",
            "I1203 18:38:03.627037 140409631090560 tpu_estimator.py:2308] examples/sec: 34.5555\n",
            "INFO:tensorflow:global_step/sec: 1.07793\n",
            "I1203 18:38:04.554471 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07793\n",
            "INFO:tensorflow:examples/sec: 34.4937\n",
            "I1203 18:38:04.554902 140409631090560 tpu_estimator.py:2308] examples/sec: 34.4937\n",
            "INFO:tensorflow:global_step/sec: 1.07337\n",
            "I1203 18:38:05.486117 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07337\n",
            "INFO:tensorflow:examples/sec: 34.3478\n",
            "I1203 18:38:05.486356 140409631090560 tpu_estimator.py:2308] examples/sec: 34.3478\n",
            "INFO:tensorflow:global_step/sec: 1.07196\n",
            "I1203 18:38:06.418997 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07196\n",
            "INFO:tensorflow:examples/sec: 34.3026\n",
            "I1203 18:38:06.419265 140409631090560 tpu_estimator.py:2308] examples/sec: 34.3026\n",
            "INFO:tensorflow:global_step/sec: 1.07884\n",
            "I1203 18:38:07.345905 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07884\n",
            "INFO:tensorflow:examples/sec: 34.5229\n",
            "I1203 18:38:07.346155 140409631090560 tpu_estimator.py:2308] examples/sec: 34.5229\n",
            "INFO:tensorflow:global_step/sec: 1.07598\n",
            "I1203 18:38:08.275277 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07598\n",
            "INFO:tensorflow:examples/sec: 34.4314\n",
            "I1203 18:38:08.275686 140409631090560 tpu_estimator.py:2308] examples/sec: 34.4314\n",
            "INFO:tensorflow:global_step/sec: 1.0735\n",
            "I1203 18:38:09.206811 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0735\n",
            "INFO:tensorflow:examples/sec: 34.352\n",
            "I1203 18:38:09.207065 140409631090560 tpu_estimator.py:2308] examples/sec: 34.352\n",
            "INFO:tensorflow:global_step/sec: 1.0718\n",
            "I1203 18:38:10.139821 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0718\n",
            "INFO:tensorflow:examples/sec: 34.2976\n",
            "I1203 18:38:10.140084 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2976\n",
            "INFO:tensorflow:global_step/sec: 1.07139\n",
            "I1203 18:38:11.073190 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07139\n",
            "INFO:tensorflow:examples/sec: 34.2845\n",
            "I1203 18:38:11.073597 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2845\n",
            "INFO:tensorflow:global_step/sec: 1.0732\n",
            "I1203 18:38:12.004986 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0732\n",
            "INFO:tensorflow:examples/sec: 34.3423\n",
            "I1203 18:38:12.005456 140409631090560 tpu_estimator.py:2308] examples/sec: 34.3423\n",
            "INFO:tensorflow:global_step/sec: 1.07209\n",
            "I1203 18:38:12.937754 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07209\n",
            "INFO:tensorflow:examples/sec: 34.3069\n",
            "I1203 18:38:12.938112 140409631090560 tpu_estimator.py:2308] examples/sec: 34.3069\n",
            "INFO:tensorflow:global_step/sec: 1.06608\n",
            "I1203 18:38:13.875742 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06608\n",
            "INFO:tensorflow:examples/sec: 34.1146\n",
            "I1203 18:38:13.876108 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1146\n",
            "INFO:tensorflow:global_step/sec: 1.06998\n",
            "I1203 18:38:14.810339 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06998\n",
            "INFO:tensorflow:examples/sec: 34.2394\n",
            "I1203 18:38:14.810752 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2394\n",
            "INFO:tensorflow:global_step/sec: 1.0725\n",
            "I1203 18:38:15.742763 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0725\n",
            "INFO:tensorflow:examples/sec: 34.3199\n",
            "I1203 18:38:15.743161 140409631090560 tpu_estimator.py:2308] examples/sec: 34.3199\n",
            "INFO:tensorflow:global_step/sec: 1.06337\n",
            "I1203 18:38:16.683162 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06337\n",
            "INFO:tensorflow:examples/sec: 34.0279\n",
            "I1203 18:38:16.683411 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0279\n",
            "INFO:tensorflow:global_step/sec: 1.06505\n",
            "I1203 18:38:17.622066 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06505\n",
            "INFO:tensorflow:examples/sec: 34.0816\n",
            "I1203 18:38:17.622309 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0816\n",
            "INFO:tensorflow:global_step/sec: 1.07107\n",
            "I1203 18:38:18.555723 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07107\n",
            "INFO:tensorflow:examples/sec: 34.2742\n",
            "I1203 18:38:18.555958 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2742\n",
            "INFO:tensorflow:global_step/sec: 1.06656\n",
            "I1203 18:38:19.493317 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06656\n",
            "INFO:tensorflow:examples/sec: 34.1299\n",
            "I1203 18:38:19.493581 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1299\n",
            "INFO:tensorflow:global_step/sec: 1.06804\n",
            "I1203 18:38:20.429620 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06804\n",
            "INFO:tensorflow:examples/sec: 34.1771\n",
            "I1203 18:38:20.430006 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1771\n",
            "INFO:tensorflow:global_step/sec: 1.06763\n",
            "I1203 18:38:21.366277 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06763\n",
            "INFO:tensorflow:examples/sec: 34.1643\n",
            "I1203 18:38:21.366536 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1643\n",
            "INFO:tensorflow:global_step/sec: 1.06378\n",
            "I1203 18:38:22.306309 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06378\n",
            "INFO:tensorflow:examples/sec: 34.041\n",
            "I1203 18:38:22.306705 140409631090560 tpu_estimator.py:2308] examples/sec: 34.041\n",
            "INFO:tensorflow:global_step/sec: 1.06075\n",
            "I1203 18:38:23.249044 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06075\n",
            "INFO:tensorflow:examples/sec: 33.9439\n",
            "I1203 18:38:23.249440 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9439\n",
            "INFO:tensorflow:global_step/sec: 1.06735\n",
            "I1203 18:38:24.185930 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06735\n",
            "INFO:tensorflow:examples/sec: 34.1552\n",
            "I1203 18:38:24.186161 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1552\n",
            "INFO:tensorflow:global_step/sec: 1.05783\n",
            "I1203 18:38:25.131274 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05783\n",
            "INFO:tensorflow:examples/sec: 33.8505\n",
            "I1203 18:38:25.131666 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8505\n",
            "INFO:tensorflow:global_step/sec: 1.05991\n",
            "I1203 18:38:26.075050 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05991\n",
            "INFO:tensorflow:examples/sec: 33.9171\n",
            "I1203 18:38:26.075281 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9171\n",
            "INFO:tensorflow:global_step/sec: 1.05942\n",
            "I1203 18:38:27.018705 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05942\n",
            "INFO:tensorflow:examples/sec: 33.9014\n",
            "I1203 18:38:27.018943 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9014\n",
            "INFO:tensorflow:global_step/sec: 1.05885\n",
            "I1203 18:38:27.963094 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05885\n",
            "INFO:tensorflow:examples/sec: 33.8834\n",
            "I1203 18:38:27.963332 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8834\n",
            "INFO:tensorflow:global_step/sec: 1.05944\n",
            "I1203 18:38:28.906977 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05944\n",
            "INFO:tensorflow:examples/sec: 33.9021\n",
            "I1203 18:38:28.907238 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9021\n",
            "INFO:tensorflow:global_step/sec: 1.05818\n",
            "I1203 18:38:29.851987 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05818\n",
            "INFO:tensorflow:examples/sec: 33.8617\n",
            "I1203 18:38:29.852346 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8617\n",
            "INFO:tensorflow:global_step/sec: 1.05209\n",
            "I1203 18:38:30.802499 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05209\n",
            "INFO:tensorflow:examples/sec: 33.667\n",
            "I1203 18:38:30.802726 140409631090560 tpu_estimator.py:2308] examples/sec: 33.667\n",
            "INFO:tensorflow:global_step/sec: 1.06109\n",
            "I1203 18:38:31.744910 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06109\n",
            "INFO:tensorflow:examples/sec: 33.9548\n",
            "I1203 18:38:31.745280 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9548\n",
            "INFO:tensorflow:global_step/sec: 1.0593\n",
            "I1203 18:38:32.688921 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0593\n",
            "INFO:tensorflow:examples/sec: 33.8976\n",
            "I1203 18:38:32.689143 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8976\n",
            "INFO:tensorflow:global_step/sec: 1.05456\n",
            "I1203 18:38:33.637197 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05456\n",
            "INFO:tensorflow:examples/sec: 33.7459\n",
            "I1203 18:38:33.637461 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7459\n",
            "INFO:tensorflow:global_step/sec: 1.05089\n",
            "I1203 18:38:34.588785 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05089\n",
            "INFO:tensorflow:examples/sec: 33.6284\n",
            "I1203 18:38:34.589013 140409631090560 tpu_estimator.py:2308] examples/sec: 33.6284\n",
            "INFO:tensorflow:global_step/sec: 1.05545\n",
            "I1203 18:38:35.536246 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05545\n",
            "INFO:tensorflow:examples/sec: 33.7745\n",
            "I1203 18:38:35.536533 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7745\n",
            "INFO:tensorflow:global_step/sec: 1.05763\n",
            "I1203 18:38:36.481745 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05763\n",
            "INFO:tensorflow:examples/sec: 33.8442\n",
            "I1203 18:38:36.481976 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8442\n",
            "INFO:tensorflow:global_step/sec: 1.06185\n",
            "I1203 18:38:37.423520 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06185\n",
            "INFO:tensorflow:examples/sec: 33.9792\n",
            "I1203 18:38:37.423793 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9792\n",
            "INFO:tensorflow:global_step/sec: 1.04938\n",
            "I1203 18:38:38.376461 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.04938\n",
            "INFO:tensorflow:examples/sec: 33.58\n",
            "I1203 18:38:38.376698 140409631090560 tpu_estimator.py:2308] examples/sec: 33.58\n",
            "INFO:tensorflow:global_step/sec: 1.05929\n",
            "I1203 18:38:39.320497 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05929\n",
            "INFO:tensorflow:examples/sec: 33.8973\n",
            "I1203 18:38:39.320731 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8973\n",
            "INFO:tensorflow:global_step/sec: 1.06027\n",
            "I1203 18:38:40.263620 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06027\n",
            "INFO:tensorflow:examples/sec: 33.9288\n",
            "I1203 18:38:40.263869 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9288\n",
            "INFO:tensorflow:global_step/sec: 1.06042\n",
            "I1203 18:38:41.206648 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06042\n",
            "INFO:tensorflow:examples/sec: 33.9333\n",
            "I1203 18:38:41.206909 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9333\n",
            "INFO:tensorflow:global_step/sec: 1.05005\n",
            "I1203 18:38:42.158996 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05005\n",
            "INFO:tensorflow:examples/sec: 33.6016\n",
            "I1203 18:38:42.159263 140409631090560 tpu_estimator.py:2308] examples/sec: 33.6016\n",
            "INFO:tensorflow:global_step/sec: 1.04978\n",
            "I1203 18:38:43.111609 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.04978\n",
            "INFO:tensorflow:examples/sec: 33.593\n",
            "I1203 18:38:43.111878 140409631090560 tpu_estimator.py:2308] examples/sec: 33.593\n",
            "INFO:tensorflow:global_step/sec: 1.0513\n",
            "I1203 18:38:44.062771 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0513\n",
            "INFO:tensorflow:examples/sec: 33.6416\n",
            "I1203 18:38:44.063029 140409631090560 tpu_estimator.py:2308] examples/sec: 33.6416\n",
            "INFO:tensorflow:global_step/sec: 1.05057\n",
            "I1203 18:38:45.014632 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05057\n",
            "INFO:tensorflow:examples/sec: 33.6183\n",
            "I1203 18:38:45.014891 140409631090560 tpu_estimator.py:2308] examples/sec: 33.6183\n",
            "INFO:tensorflow:global_step/sec: 1.05207\n",
            "I1203 18:38:45.965158 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05207\n",
            "INFO:tensorflow:examples/sec: 33.6661\n",
            "I1203 18:38:45.965426 140409631090560 tpu_estimator.py:2308] examples/sec: 33.6661\n",
            "INFO:tensorflow:global_step/sec: 1.05923\n",
            "I1203 18:38:46.909229 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05923\n",
            "INFO:tensorflow:examples/sec: 33.8953\n",
            "I1203 18:38:46.909485 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8953\n",
            "INFO:tensorflow:global_step/sec: 1.06056\n",
            "I1203 18:38:47.852138 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06056\n",
            "INFO:tensorflow:examples/sec: 33.9379\n",
            "I1203 18:38:47.852381 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9379\n",
            "INFO:tensorflow:global_step/sec: 1.06169\n",
            "I1203 18:38:48.794032 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06169\n",
            "INFO:tensorflow:examples/sec: 33.974\n",
            "I1203 18:38:48.794276 140409631090560 tpu_estimator.py:2308] examples/sec: 33.974\n",
            "INFO:tensorflow:global_step/sec: 1.06624\n",
            "I1203 18:38:49.731895 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06624\n",
            "INFO:tensorflow:examples/sec: 34.1198\n",
            "I1203 18:38:49.732171 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1198\n",
            "INFO:tensorflow:global_step/sec: 1.05413\n",
            "I1203 18:38:50.680578 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05413\n",
            "INFO:tensorflow:examples/sec: 33.7323\n",
            "I1203 18:38:50.680839 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7323\n",
            "INFO:tensorflow:global_step/sec: 1.0559\n",
            "I1203 18:38:51.627609 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0559\n",
            "INFO:tensorflow:examples/sec: 33.7889\n",
            "I1203 18:38:51.627831 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7889\n",
            "INFO:tensorflow:global_step/sec: 1.06023\n",
            "I1203 18:38:52.570799 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06023\n",
            "INFO:tensorflow:examples/sec: 33.9274\n",
            "I1203 18:38:52.571040 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9274\n",
            "INFO:tensorflow:global_step/sec: 1.06361\n",
            "I1203 18:38:53.510970 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06361\n",
            "INFO:tensorflow:examples/sec: 34.0356\n",
            "I1203 18:38:53.511207 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0356\n",
            "INFO:tensorflow:global_step/sec: 1.06168\n",
            "I1203 18:38:54.452887 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06168\n",
            "INFO:tensorflow:examples/sec: 33.9738\n",
            "I1203 18:38:54.453266 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9738\n",
            "INFO:tensorflow:global_step/sec: 1.05501\n",
            "I1203 18:38:55.400753 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05501\n",
            "INFO:tensorflow:examples/sec: 33.7602\n",
            "I1203 18:38:55.401144 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7602\n",
            "INFO:tensorflow:global_step/sec: 1.0623\n",
            "I1203 18:38:56.342110 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0623\n",
            "INFO:tensorflow:examples/sec: 33.9936\n",
            "I1203 18:38:56.342373 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9936\n",
            "INFO:tensorflow:global_step/sec: 1.06345\n",
            "I1203 18:38:57.282437 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06345\n",
            "INFO:tensorflow:examples/sec: 34.0303\n",
            "I1203 18:38:57.282803 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0303\n",
            "INFO:tensorflow:global_step/sec: 1.05762\n",
            "I1203 18:38:58.227966 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05762\n",
            "INFO:tensorflow:examples/sec: 33.8439\n",
            "I1203 18:38:58.228225 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8439\n",
            "INFO:tensorflow:global_step/sec: 1.06305\n",
            "I1203 18:38:59.168637 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06305\n",
            "INFO:tensorflow:examples/sec: 34.0174\n",
            "I1203 18:38:59.169197 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0174\n",
            "INFO:tensorflow:global_step/sec: 1.05618\n",
            "I1203 18:39:00.115467 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05618\n",
            "INFO:tensorflow:examples/sec: 33.7978\n",
            "I1203 18:39:00.115706 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7978\n",
            "INFO:tensorflow:global_step/sec: 1.06653\n",
            "I1203 18:39:01.053088 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06653\n",
            "INFO:tensorflow:examples/sec: 34.1289\n",
            "I1203 18:39:01.053524 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1289\n",
            "INFO:tensorflow:global_step/sec: 1.06295\n",
            "I1203 18:39:01.993852 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06295\n",
            "INFO:tensorflow:examples/sec: 34.0144\n",
            "I1203 18:39:01.994080 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0144\n",
            "INFO:tensorflow:global_step/sec: 1.0654\n",
            "I1203 18:39:02.932488 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0654\n",
            "INFO:tensorflow:examples/sec: 34.0928\n",
            "I1203 18:39:02.932868 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0928\n",
            "INFO:tensorflow:global_step/sec: 1.06693\n",
            "I1203 18:39:03.869743 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06693\n",
            "INFO:tensorflow:examples/sec: 34.1417\n",
            "I1203 18:39:03.869982 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1417\n",
            "INFO:tensorflow:global_step/sec: 1.05663\n",
            "I1203 18:39:04.816172 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05663\n",
            "INFO:tensorflow:examples/sec: 33.8121\n",
            "I1203 18:39:04.816568 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8121\n",
            "INFO:tensorflow:global_step/sec: 1.0558\n",
            "I1203 18:39:05.763303 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0558\n",
            "INFO:tensorflow:examples/sec: 33.7856\n",
            "I1203 18:39:05.763551 140409631090560 tpu_estimator.py:2308] examples/sec: 33.7856\n",
            "INFO:tensorflow:global_step/sec: 1.06826\n",
            "I1203 18:39:06.699417 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06826\n",
            "INFO:tensorflow:examples/sec: 34.1845\n",
            "I1203 18:39:06.699701 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1845\n",
            "INFO:tensorflow:global_step/sec: 1.06681\n",
            "I1203 18:39:07.636771 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06681\n",
            "INFO:tensorflow:examples/sec: 34.138\n",
            "I1203 18:39:07.637172 140409631090560 tpu_estimator.py:2308] examples/sec: 34.138\n",
            "INFO:tensorflow:global_step/sec: 1.06555\n",
            "I1203 18:39:08.575238 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06555\n",
            "INFO:tensorflow:examples/sec: 34.0977\n",
            "I1203 18:39:08.575494 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0977\n",
            "INFO:tensorflow:global_step/sec: 1.06623\n",
            "I1203 18:39:09.513123 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06623\n",
            "INFO:tensorflow:examples/sec: 34.1195\n",
            "I1203 18:39:09.513498 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1195\n",
            "INFO:tensorflow:global_step/sec: 1.0633\n",
            "I1203 18:39:10.453642 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0633\n",
            "INFO:tensorflow:examples/sec: 34.0257\n",
            "I1203 18:39:10.454086 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0257\n",
            "INFO:tensorflow:global_step/sec: 1.06648\n",
            "I1203 18:39:11.391246 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06648\n",
            "INFO:tensorflow:examples/sec: 34.1274\n",
            "I1203 18:39:11.391489 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1274\n",
            "INFO:tensorflow:global_step/sec: 1.0665\n",
            "I1203 18:39:12.328888 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0665\n",
            "INFO:tensorflow:examples/sec: 34.1281\n",
            "I1203 18:39:12.329107 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1281\n",
            "INFO:tensorflow:global_step/sec: 1.06559\n",
            "I1203 18:39:13.267362 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06559\n",
            "INFO:tensorflow:examples/sec: 34.0988\n",
            "I1203 18:39:13.267603 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0988\n",
            "INFO:tensorflow:global_step/sec: 1.06691\n",
            "I1203 18:39:14.204640 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06691\n",
            "INFO:tensorflow:examples/sec: 34.141\n",
            "I1203 18:39:14.204901 140409631090560 tpu_estimator.py:2308] examples/sec: 34.141\n",
            "INFO:tensorflow:global_step/sec: 1.06341\n",
            "I1203 18:39:15.145031 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06341\n",
            "INFO:tensorflow:examples/sec: 34.029\n",
            "I1203 18:39:15.145431 140409631090560 tpu_estimator.py:2308] examples/sec: 34.029\n",
            "INFO:tensorflow:global_step/sec: 1.06639\n",
            "I1203 18:39:16.082774 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06639\n",
            "INFO:tensorflow:examples/sec: 34.1245\n",
            "I1203 18:39:16.083231 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1245\n",
            "INFO:tensorflow:global_step/sec: 1.07041\n",
            "I1203 18:39:17.016986 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07041\n",
            "INFO:tensorflow:examples/sec: 34.253\n",
            "I1203 18:39:17.017365 140409631090560 tpu_estimator.py:2308] examples/sec: 34.253\n",
            "INFO:tensorflow:global_step/sec: 1.06904\n",
            "I1203 18:39:17.952384 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06904\n",
            "INFO:tensorflow:examples/sec: 34.2094\n",
            "I1203 18:39:17.952686 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2094\n",
            "INFO:tensorflow:global_step/sec: 1.06464\n",
            "I1203 18:39:18.891676 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06464\n",
            "INFO:tensorflow:examples/sec: 34.0685\n",
            "I1203 18:39:18.892035 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0685\n",
            "INFO:tensorflow:global_step/sec: 1.06884\n",
            "I1203 18:39:19.827286 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06884\n",
            "INFO:tensorflow:examples/sec: 34.2028\n",
            "I1203 18:39:19.827700 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2028\n",
            "INFO:tensorflow:global_step/sec: 1.06379\n",
            "I1203 18:39:20.767322 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06379\n",
            "INFO:tensorflow:examples/sec: 34.0413\n",
            "I1203 18:39:20.767580 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0413\n",
            "INFO:tensorflow:global_step/sec: 1.0662\n",
            "I1203 18:39:21.705235 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0662\n",
            "INFO:tensorflow:examples/sec: 34.1184\n",
            "I1203 18:39:21.705729 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1184\n",
            "INFO:tensorflow:global_step/sec: 1.05966\n",
            "I1203 18:39:22.648913 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05966\n",
            "INFO:tensorflow:examples/sec: 33.9092\n",
            "I1203 18:39:22.649154 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9092\n",
            "INFO:tensorflow:global_step/sec: 1.07719\n",
            "I1203 18:39:23.577260 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07719\n",
            "INFO:tensorflow:examples/sec: 34.47\n",
            "I1203 18:39:23.577665 140409631090560 tpu_estimator.py:2308] examples/sec: 34.47\n",
            "INFO:tensorflow:global_step/sec: 1.07137\n",
            "I1203 18:39:24.510651 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07137\n",
            "INFO:tensorflow:examples/sec: 34.284\n",
            "I1203 18:39:24.510895 140409631090560 tpu_estimator.py:2308] examples/sec: 34.284\n",
            "INFO:tensorflow:global_step/sec: 1.06433\n",
            "I1203 18:39:25.450209 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06433\n",
            "INFO:tensorflow:examples/sec: 34.0585\n",
            "I1203 18:39:25.450686 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0585\n",
            "INFO:tensorflow:global_step/sec: 1.06913\n",
            "I1203 18:39:26.385559 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06913\n",
            "INFO:tensorflow:examples/sec: 34.2121\n",
            "I1203 18:39:26.385914 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2121\n",
            "INFO:tensorflow:global_step/sec: 1.06606\n",
            "I1203 18:39:27.323597 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06606\n",
            "INFO:tensorflow:examples/sec: 34.1138\n",
            "I1203 18:39:27.323978 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1138\n",
            "INFO:tensorflow:global_step/sec: 1.06968\n",
            "I1203 18:39:28.258450 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06968\n",
            "INFO:tensorflow:examples/sec: 34.2298\n",
            "I1203 18:39:28.258868 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2298\n",
            "INFO:tensorflow:global_step/sec: 1.06599\n",
            "I1203 18:39:29.196565 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06599\n",
            "INFO:tensorflow:examples/sec: 34.1116\n",
            "I1203 18:39:29.196974 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1116\n",
            "INFO:tensorflow:global_step/sec: 1.06672\n",
            "I1203 18:39:30.134008 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06672\n",
            "INFO:tensorflow:examples/sec: 34.135\n",
            "I1203 18:39:30.134237 140409631090560 tpu_estimator.py:2308] examples/sec: 34.135\n",
            "INFO:tensorflow:global_step/sec: 1.05784\n",
            "I1203 18:39:31.079307 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.05784\n",
            "INFO:tensorflow:examples/sec: 33.8507\n",
            "I1203 18:39:31.079606 140409631090560 tpu_estimator.py:2308] examples/sec: 33.8507\n",
            "INFO:tensorflow:global_step/sec: 1.06632\n",
            "I1203 18:39:32.017111 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06632\n",
            "INFO:tensorflow:examples/sec: 34.1222\n",
            "I1203 18:39:32.017513 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1222\n",
            "INFO:tensorflow:global_step/sec: 1.06431\n",
            "I1203 18:39:32.956694 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06431\n",
            "INFO:tensorflow:examples/sec: 34.0579\n",
            "I1203 18:39:32.957080 140409631090560 tpu_estimator.py:2308] examples/sec: 34.0579\n",
            "INFO:tensorflow:global_step/sec: 1.07254\n",
            "I1203 18:39:33.889047 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07254\n",
            "INFO:tensorflow:examples/sec: 34.3214\n",
            "I1203 18:39:33.889448 140409631090560 tpu_estimator.py:2308] examples/sec: 34.3214\n",
            "INFO:tensorflow:global_step/sec: 1.0686\n",
            "I1203 18:39:34.824875 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.0686\n",
            "INFO:tensorflow:examples/sec: 34.1953\n",
            "I1203 18:39:34.825246 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1953\n",
            "INFO:tensorflow:global_step/sec: 1.06572\n",
            "I1203 18:39:35.763149 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06572\n",
            "INFO:tensorflow:examples/sec: 34.1029\n",
            "I1203 18:39:35.763514 140409631090560 tpu_estimator.py:2308] examples/sec: 34.1029\n",
            "INFO:tensorflow:global_step/sec: 1.06187\n",
            "I1203 18:39:36.704932 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.06187\n",
            "INFO:tensorflow:examples/sec: 33.9799\n",
            "I1203 18:39:36.705329 140409631090560 tpu_estimator.py:2308] examples/sec: 33.9799\n",
            "INFO:tensorflow:global_step/sec: 1.07185\n",
            "I1203 18:39:37.637892 140409631090560 tpu_estimator.py:2307] global_step/sec: 1.07185\n",
            "INFO:tensorflow:examples/sec: 34.2992\n",
            "I1203 18:39:37.638296 140409631090560 tpu_estimator.py:2308] examples/sec: 34.2992\n",
            "INFO:tensorflow:Saving checkpoints for 4161 into /content/classification_results/1/model.ckpt.\n",
            "I1203 18:39:37.638949 140409631090560 basic_session_run_hooks.py:606] Saving checkpoints for 4161 into /content/classification_results/1/model.ckpt.\n",
            "WARNING:tensorflow:From /tensorflow-1.15.2/python3.6/tensorflow_core/python/training/saver.py:963: remove_checkpoint (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use standard file APIs to delete files with this prefix.\n",
            "W1203 18:40:02.165009 140409631090560 deprecation.py:323] From /tensorflow-1.15.2/python3.6/tensorflow_core/python/training/saver.py:963: remove_checkpoint (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use standard file APIs to delete files with this prefix.\n",
            "WARNING:tensorflow:Entity <function file_based_input_fn_builder.<locals>.input_fn.<locals>.<lambda> at 0x7fb2d8a30950> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: module 'gast' has no attribute 'Str'\n",
            "W1203 18:40:03.014805 140409631090560 ag_logging.py:146] Entity <function file_based_input_fn_builder.<locals>.input_fn.<locals>.<lambda> at 0x7fb2d8a30950> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: module 'gast' has no attribute 'Str'\n",
            "INFO:tensorflow:Calling model_fn.\n",
            "I1203 18:40:03.029632 140409631090560 estimator.py:1148] Calling model_fn.\n",
            "INFO:tensorflow:Running eval on CPU\n",
            "I1203 18:40:03.029903 140409631090560 tpu_estimator.py:3124] Running eval on CPU\n",
            "INFO:tensorflow:*** Features ***\n",
            "I1203 18:40:03.030238 140409631090560 run_binary_classification.py:634] *** Features ***\n",
            "INFO:tensorflow:  name = input_ids, shape = (?, 128)\n",
            "I1203 18:40:03.030392 140409631090560 run_binary_classification.py:636]   name = input_ids, shape = (?, 128)\n",
            "INFO:tensorflow:  name = input_mask, shape = (?, 128)\n",
            "I1203 18:40:03.030515 140409631090560 run_binary_classification.py:636]   name = input_mask, shape = (?, 128)\n",
            "INFO:tensorflow:  name = is_real_example, shape = (?,)\n",
            "I1203 18:40:03.030608 140409631090560 run_binary_classification.py:636]   name = is_real_example, shape = (?,)\n",
            "INFO:tensorflow:  name = label_ids, shape = (?,)\n",
            "I1203 18:40:03.030701 140409631090560 run_binary_classification.py:636]   name = label_ids, shape = (?,)\n",
            "INFO:tensorflow:  name = segment_ids, shape = (?, 128)\n",
            "I1203 18:40:03.030799 140409631090560 run_binary_classification.py:636]   name = segment_ids, shape = (?, 128)\n",
            "INFO:tensorflow:**** Trainable Variables ****\n",
            "I1203 18:40:05.145812 140409631090560 run_binary_classification.py:670] **** Trainable Variables ****\n",
            "INFO:tensorflow:Done calling model_fn.\n",
            "I1203 18:40:05.290286 140409631090560 estimator.py:1150] Done calling model_fn.\n",
            "INFO:tensorflow:Starting evaluation at 2020-12-03T18:40:05Z\n",
            "I1203 18:40:05.306893 140409631090560 evaluation.py:255] Starting evaluation at 2020-12-03T18:40:05Z\n",
            "INFO:tensorflow:Graph was finalized.\n",
            "I1203 18:40:05.741013 140409631090560 monitored_session.py:240] Graph was finalized.\n",
            "2020-12-03 18:40:05.741723: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 18:40:05.742076: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 0 with properties: \n",
            "name: Tesla T4 major: 7 minor: 5 memoryClockRate(GHz): 1.59\n",
            "pciBusID: 0000:00:04.0\n",
            "2020-12-03 18:40:05.742182: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1\n",
            "2020-12-03 18:40:05.742207: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10\n",
            "2020-12-03 18:40:05.742229: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10\n",
            "2020-12-03 18:40:05.742252: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10\n",
            "2020-12-03 18:40:05.742272: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10\n",
            "2020-12-03 18:40:05.742292: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10\n",
            "2020-12-03 18:40:05.742311: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7\n",
            "2020-12-03 18:40:05.742392: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 18:40:05.742726: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 18:40:05.742967: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1767] Adding visible gpu devices: 0\n",
            "2020-12-03 18:40:05.743071: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1180] Device interconnect StreamExecutor with strength 1 edge matrix:\n",
            "2020-12-03 18:40:05.743086: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1186]      0 \n",
            "2020-12-03 18:40:05.743096: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1199] 0:   N \n",
            "2020-12-03 18:40:05.743198: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 18:40:05.743538: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 18:40:05.743811: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1325] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14221 MB memory) -> physical GPU (device: 0, name: Tesla T4, pci bus id: 0000:00:04.0, compute capability: 7.5)\n",
            "INFO:tensorflow:Restoring parameters from /content/classification_results/1/model.ckpt-4161\n",
            "I1203 18:40:05.745069 140409631090560 saver.py:1284] Restoring parameters from /content/classification_results/1/model.ckpt-4161\n",
            "INFO:tensorflow:Running local_init_op.\n",
            "I1203 18:40:06.544677 140409631090560 session_manager.py:500] Running local_init_op.\n",
            "INFO:tensorflow:Done running local_init_op.\n",
            "I1203 18:40:06.617251 140409631090560 session_manager.py:502] Done running local_init_op.\n",
            "INFO:tensorflow:accuracy = 0.875, loss = 0.676852\n",
            "I1203 18:40:07.444648 140409631090560 basic_session_run_hooks.py:262] accuracy = 0.875, loss = 0.676852\n",
            "INFO:tensorflow:accuracy = 0.9375, loss = 0.0006474487 (0.618 sec)\n",
            "I1203 18:40:08.062318 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9375, loss = 0.0006474487 (0.618 sec)\n",
            "INFO:tensorflow:accuracy = 0.9583333, loss = 0.00051657285 (0.160 sec)\n",
            "I1203 18:40:08.222617 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9583333, loss = 0.00051657285 (0.160 sec)\n",
            "INFO:tensorflow:accuracy = 0.96875, loss = 0.016387569 (0.144 sec)\n",
            "I1203 18:40:08.366381 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.96875, loss = 0.016387569 (0.144 sec)\n",
            "INFO:tensorflow:accuracy = 0.975, loss = 0.00091130927 (0.146 sec)\n",
            "I1203 18:40:08.512135 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.975, loss = 0.00091130927 (0.146 sec)\n",
            "INFO:tensorflow:accuracy = 0.9791667, loss = 0.0017228947 (0.144 sec)\n",
            "I1203 18:40:08.655610 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9791667, loss = 0.0017228947 (0.144 sec)\n",
            "INFO:tensorflow:accuracy = 0.96428573, loss = 0.9693742 (0.143 sec)\n",
            "I1203 18:40:08.798326 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.96428573, loss = 0.9693742 (0.143 sec)\n",
            "INFO:tensorflow:accuracy = 0.9375, loss = 1.5444422 (0.143 sec)\n",
            "I1203 18:40:08.940890 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9375, loss = 1.5444422 (0.143 sec)\n",
            "INFO:tensorflow:accuracy = 0.9444444, loss = 0.0077639157 (0.149 sec)\n",
            "I1203 18:40:09.090389 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9444444, loss = 0.0077639157 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.9375, loss = 0.6549734 (0.149 sec)\n",
            "I1203 18:40:09.239449 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9375, loss = 0.6549734 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.9318182, loss = 0.6567242 (0.147 sec)\n",
            "I1203 18:40:09.386735 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9318182, loss = 0.6567242 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.9270833, loss = 0.26903707 (0.144 sec)\n",
            "I1203 18:40:09.531182 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9270833, loss = 0.26903707 (0.144 sec)\n",
            "INFO:tensorflow:accuracy = 0.9326923, loss = 0.00061859 (0.144 sec)\n",
            "I1203 18:40:09.675488 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9326923, loss = 0.00061859 (0.144 sec)\n",
            "INFO:tensorflow:accuracy = 0.9285714, loss = 0.67529494 (0.146 sec)\n",
            "I1203 18:40:09.821482 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9285714, loss = 0.67529494 (0.146 sec)\n",
            "INFO:tensorflow:accuracy = 0.925, loss = 0.66313696 (0.144 sec)\n",
            "I1203 18:40:09.965316 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.925, loss = 0.66313696 (0.144 sec)\n",
            "INFO:tensorflow:accuracy = 0.9296875, loss = 0.0009786112 (0.150 sec)\n",
            "I1203 18:40:10.115084 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9296875, loss = 0.0009786112 (0.150 sec)\n",
            "INFO:tensorflow:accuracy = 0.9338235, loss = 0.0013179578 (0.147 sec)\n",
            "I1203 18:40:10.261626 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9338235, loss = 0.0013179578 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.9305556, loss = 0.25051278 (0.146 sec)\n",
            "I1203 18:40:10.408094 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9305556, loss = 0.25051278 (0.146 sec)\n",
            "INFO:tensorflow:accuracy = 0.93421054, loss = 0.003350802 (0.144 sec)\n",
            "I1203 18:40:10.552493 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.93421054, loss = 0.003350802 (0.144 sec)\n",
            "INFO:tensorflow:accuracy = 0.93125, loss = 0.43308967 (0.145 sec)\n",
            "I1203 18:40:10.697594 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.93125, loss = 0.43308967 (0.145 sec)\n",
            "INFO:tensorflow:accuracy = 0.9345238, loss = 0.0009512 (0.144 sec)\n",
            "I1203 18:40:10.841545 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9345238, loss = 0.0009512 (0.144 sec)\n",
            "INFO:tensorflow:accuracy = 0.9375, loss = 0.001004631 (0.146 sec)\n",
            "I1203 18:40:10.987151 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9375, loss = 0.001004631 (0.146 sec)\n",
            "INFO:tensorflow:accuracy = 0.9293478, loss = 0.80104744 (0.148 sec)\n",
            "I1203 18:40:11.135634 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9293478, loss = 0.80104744 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.9270833, loss = 0.6791903 (0.147 sec)\n",
            "I1203 18:40:11.282922 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9270833, loss = 0.6791903 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.93, loss = 0.00054556923 (0.149 sec)\n",
            "I1203 18:40:11.432278 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.93, loss = 0.00054556923 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.9326923, loss = 0.0004570332 (0.144 sec)\n",
            "I1203 18:40:11.576695 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9326923, loss = 0.0004570332 (0.144 sec)\n",
            "INFO:tensorflow:accuracy = 0.9351852, loss = 0.0012161571 (0.146 sec)\n",
            "I1203 18:40:11.722477 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9351852, loss = 0.0012161571 (0.146 sec)\n",
            "INFO:tensorflow:accuracy = 0.9375, loss = 0.013286671 (0.145 sec)\n",
            "I1203 18:40:11.867203 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9375, loss = 0.013286671 (0.145 sec)\n",
            "INFO:tensorflow:accuracy = 0.9310345, loss = 1.6133962 (0.148 sec)\n",
            "I1203 18:40:12.015123 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9310345, loss = 1.6133962 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.93333334, loss = 0.0024730207 (0.147 sec)\n",
            "I1203 18:40:12.162068 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.93333334, loss = 0.0024730207 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.9314516, loss = 0.6848977 (0.147 sec)\n",
            "I1203 18:40:12.309387 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9314516, loss = 0.6848977 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.93359375, loss = 0.014226072 (0.149 sec)\n",
            "I1203 18:40:12.457907 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.93359375, loss = 0.014226072 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.93560606, loss = 0.0011857371 (0.145 sec)\n",
            "I1203 18:40:12.602554 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.93560606, loss = 0.0011857371 (0.145 sec)\n",
            "INFO:tensorflow:accuracy = 0.9375, loss = 0.0032511672 (0.147 sec)\n",
            "I1203 18:40:12.749111 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9375, loss = 0.0032511672 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.9357143, loss = 0.85814875 (0.146 sec)\n",
            "I1203 18:40:12.894640 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9357143, loss = 0.85814875 (0.146 sec)\n",
            "INFO:tensorflow:accuracy = 0.9375, loss = 0.025312962 (0.147 sec)\n",
            "I1203 18:40:13.041198 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9375, loss = 0.025312962 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.9358108, loss = 0.5829117 (0.148 sec)\n",
            "I1203 18:40:13.188816 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9358108, loss = 0.5829117 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.9375, loss = 0.0014081923 (0.148 sec)\n",
            "I1203 18:40:13.336921 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9375, loss = 0.0014081923 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.9391026, loss = 0.0010745059 (0.147 sec)\n",
            "I1203 18:40:13.483939 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9391026, loss = 0.0010745059 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.940625, loss = 0.0005433066 (0.149 sec)\n",
            "I1203 18:40:13.632703 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.940625, loss = 0.0005433066 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.94207317, loss = 0.0005127276 (0.146 sec)\n",
            "I1203 18:40:13.778783 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.94207317, loss = 0.0005127276 (0.146 sec)\n",
            "INFO:tensorflow:accuracy = 0.94345236, loss = 0.00082248513 (0.145 sec)\n",
            "I1203 18:40:13.924086 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.94345236, loss = 0.00082248513 (0.145 sec)\n",
            "INFO:tensorflow:accuracy = 0.94186044, loss = 0.677264 (0.147 sec)\n",
            "I1203 18:40:14.070801 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.94186044, loss = 0.677264 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.9431818, loss = 0.0004861341 (0.149 sec)\n",
            "I1203 18:40:14.219308 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9431818, loss = 0.0004861341 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.9388889, loss = 1.7731619 (0.149 sec)\n",
            "I1203 18:40:14.368062 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9388889, loss = 1.7731619 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.9375, loss = 0.96079373 (0.148 sec)\n",
            "I1203 18:40:14.515695 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9375, loss = 0.96079373 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.93351066, loss = 0.94169056 (0.147 sec)\n",
            "I1203 18:40:14.662651 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.93351066, loss = 0.94169056 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.9348958, loss = 0.00044572845 (0.145 sec)\n",
            "I1203 18:40:14.807734 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9348958, loss = 0.00044572845 (0.145 sec)\n",
            "INFO:tensorflow:accuracy = 0.93622446, loss = 0.010211733 (0.152 sec)\n",
            "I1203 18:40:14.959527 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.93622446, loss = 0.010211733 (0.152 sec)\n",
            "INFO:tensorflow:accuracy = 0.9375, loss = 0.0023829814 (0.145 sec)\n",
            "I1203 18:40:15.104998 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9375, loss = 0.0023829814 (0.145 sec)\n",
            "INFO:tensorflow:accuracy = 0.9362745, loss = 0.4583648 (0.150 sec)\n",
            "I1203 18:40:15.254946 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9362745, loss = 0.4583648 (0.150 sec)\n",
            "INFO:tensorflow:accuracy = 0.93028843, loss = 1.5291735 (0.148 sec)\n",
            "I1203 18:40:15.402533 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.93028843, loss = 1.5291735 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.9316038, loss = 0.0017979429 (0.152 sec)\n",
            "I1203 18:40:15.554967 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9316038, loss = 0.0017979429 (0.152 sec)\n",
            "INFO:tensorflow:accuracy = 0.9282407, loss = 1.1633711 (0.146 sec)\n",
            "I1203 18:40:15.701102 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9282407, loss = 1.1633711 (0.146 sec)\n",
            "INFO:tensorflow:accuracy = 0.9227273, loss = 1.9526772 (0.146 sec)\n",
            "I1203 18:40:15.846786 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9227273, loss = 1.9526772 (0.146 sec)\n",
            "INFO:tensorflow:accuracy = 0.92410713, loss = 0.00203904 (0.148 sec)\n",
            "I1203 18:40:15.994720 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.92410713, loss = 0.00203904 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.9254386, loss = 0.0016014895 (0.148 sec)\n",
            "I1203 18:40:16.143074 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9254386, loss = 0.0016014895 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.92025864, loss = 1.3590809 (0.148 sec)\n",
            "I1203 18:40:16.290890 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.92025864, loss = 1.3590809 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.9216102, loss = 0.009135161 (0.149 sec)\n",
            "I1203 18:40:16.439700 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9216102, loss = 0.009135161 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.91875, loss = 1.8740106 (0.150 sec)\n",
            "I1203 18:40:16.589488 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91875, loss = 1.8740106 (0.150 sec)\n",
            "INFO:tensorflow:accuracy = 0.920082, loss = 0.0005985869 (0.146 sec)\n",
            "I1203 18:40:16.735798 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.920082, loss = 0.0005985869 (0.146 sec)\n",
            "INFO:tensorflow:accuracy = 0.91935486, loss = 0.9629752 (0.148 sec)\n",
            "I1203 18:40:16.883345 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91935486, loss = 0.9629752 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.9186508, loss = 0.967149 (0.147 sec)\n",
            "I1203 18:40:17.030266 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9186508, loss = 0.967149 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.91796875, loss = 0.67897695 (0.149 sec)\n",
            "I1203 18:40:17.179201 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91796875, loss = 0.67897695 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.9173077, loss = 0.6602134 (0.147 sec)\n",
            "I1203 18:40:17.326542 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9173077, loss = 0.6602134 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.9166667, loss = 0.89682907 (0.149 sec)\n",
            "I1203 18:40:17.475742 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9166667, loss = 0.89682907 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.91791046, loss = 0.0022354503 (0.149 sec)\n",
            "I1203 18:40:17.624833 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91791046, loss = 0.0022354503 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.91544116, loss = 1.2488148 (0.146 sec)\n",
            "I1203 18:40:17.771105 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91544116, loss = 1.2488148 (0.146 sec)\n",
            "INFO:tensorflow:accuracy = 0.91485506, loss = 0.4026037 (0.148 sec)\n",
            "I1203 18:40:17.919358 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91485506, loss = 0.4026037 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.9160714, loss = 0.02449872 (0.148 sec)\n",
            "I1203 18:40:18.067519 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9160714, loss = 0.02449872 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.9172535, loss = 0.0011005874 (0.148 sec)\n",
            "I1203 18:40:18.215460 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9172535, loss = 0.0011005874 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.9184028, loss = 0.0006653926 (0.150 sec)\n",
            "I1203 18:40:18.365388 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9184028, loss = 0.0006653926 (0.150 sec)\n",
            "INFO:tensorflow:accuracy = 0.91780823, loss = 0.41666108 (0.150 sec)\n",
            "I1203 18:40:18.515594 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91780823, loss = 0.41666108 (0.150 sec)\n",
            "INFO:tensorflow:accuracy = 0.9172297, loss = 0.68499345 (0.147 sec)\n",
            "I1203 18:40:18.662277 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9172297, loss = 0.68499345 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.9166667, loss = 0.8370291 (0.148 sec)\n",
            "I1203 18:40:18.810169 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9166667, loss = 0.8370291 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.9177632, loss = 0.0027795697 (0.148 sec)\n",
            "I1203 18:40:18.958671 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9177632, loss = 0.0027795697 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.9172078, loss = 0.6821154 (0.147 sec)\n",
            "I1203 18:40:19.105722 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9172078, loss = 0.6821154 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.911859, loss = 2.743495 (0.149 sec)\n",
            "I1203 18:40:19.254392 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.911859, loss = 2.743495 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.91297466, loss = 0.0017961741 (0.150 sec)\n",
            "I1203 18:40:19.404310 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91297466, loss = 0.0017961741 (0.150 sec)\n",
            "INFO:tensorflow:accuracy = 0.9140625, loss = 0.00051511626 (0.150 sec)\n",
            "I1203 18:40:19.553981 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9140625, loss = 0.00051511626 (0.150 sec)\n",
            "INFO:tensorflow:accuracy = 0.91512346, loss = 0.009938932 (0.148 sec)\n",
            "I1203 18:40:19.701861 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91512346, loss = 0.009938932 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.91615856, loss = 0.00049618643 (0.151 sec)\n",
            "I1203 18:40:19.852620 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91615856, loss = 0.00049618643 (0.151 sec)\n",
            "INFO:tensorflow:accuracy = 0.9171687, loss = 0.0004972756 (0.149 sec)\n",
            "I1203 18:40:20.001481 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9171687, loss = 0.0004972756 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.9166667, loss = 0.9111779 (0.149 sec)\n",
            "I1203 18:40:20.150732 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9166667, loss = 0.9111779 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.9147059, loss = 1.4793191 (0.150 sec)\n",
            "I1203 18:40:20.300826 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9147059, loss = 1.4793191 (0.150 sec)\n",
            "INFO:tensorflow:accuracy = 0.9142442, loss = 0.66135645 (0.152 sec)\n",
            "I1203 18:40:20.452877 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9142442, loss = 0.66135645 (0.152 sec)\n",
            "INFO:tensorflow:accuracy = 0.9137931, loss = 0.7089546 (0.149 sec)\n",
            "I1203 18:40:20.602303 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9137931, loss = 0.7089546 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.91477275, loss = 0.0051250127 (0.147 sec)\n",
            "I1203 18:40:20.749794 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91477275, loss = 0.0051250127 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.91432583, loss = 0.2924425 (0.147 sec)\n",
            "I1203 18:40:20.897214 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91432583, loss = 0.2924425 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.9138889, loss = 0.6182866 (0.148 sec)\n",
            "I1203 18:40:21.045684 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9138889, loss = 0.6182866 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.91483516, loss = 0.0005413691 (0.146 sec)\n",
            "I1203 18:40:21.192058 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91483516, loss = 0.0005413691 (0.146 sec)\n",
            "INFO:tensorflow:accuracy = 0.9157609, loss = 0.0010757645 (0.150 sec)\n",
            "I1203 18:40:21.342371 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9157609, loss = 0.0010757645 (0.150 sec)\n",
            "INFO:tensorflow:accuracy = 0.9166667, loss = 0.00045369688 (0.150 sec)\n",
            "I1203 18:40:21.492454 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9166667, loss = 0.00045369688 (0.150 sec)\n",
            "INFO:tensorflow:accuracy = 0.9162234, loss = 0.6959347 (0.149 sec)\n",
            "I1203 18:40:21.641012 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9162234, loss = 0.6959347 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.9144737, loss = 1.5457308 (0.152 sec)\n",
            "I1203 18:40:21.793151 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9144737, loss = 1.5457308 (0.152 sec)\n",
            "INFO:tensorflow:accuracy = 0.9146982, loss = 0.0005333788 (0.096 sec)\n",
            "I1203 18:40:21.889352 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9146982, loss = 0.0005333788 (0.096 sec)\n",
            "INFO:tensorflow:Finished evaluation at 2020-12-03-18:40:22\n",
            "I1203 18:40:22.053096 140409631090560 evaluation.py:275] Finished evaluation at 2020-12-03-18:40:22\n",
            "INFO:tensorflow:Saving dict for global step 4161: eval_accuracy = 0.8994744, eval_loss = 0.53927267, global_step = 4161, loss = 0.53715724\n",
            "I1203 18:40:22.053376 140409631090560 estimator.py:2049] Saving dict for global step 4161: eval_accuracy = 0.8994744, eval_loss = 0.53927267, global_step = 4161, loss = 0.53715724\n",
            "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 4161: /content/classification_results/1/model.ckpt-4161\n",
            "I1203 18:40:22.055056 140409631090560 estimator.py:2109] Saving 'checkpoint_path' summary for global step 4161: /content/classification_results/1/model.ckpt-4161\n",
            "INFO:tensorflow:evaluation_loop marked as finished\n",
            "I1203 18:40:22.055706 140409631090560 error_handling.py:101] evaluation_loop marked as finished\n",
            "INFO:tensorflow:Loss for final step: 0.00077268714.\n",
            "I1203 18:40:22.238180 140409631090560 estimator.py:371] Loss for final step: 0.00077268714.\n",
            "INFO:tensorflow:training_loop marked as finished\n",
            "I1203 18:40:22.239108 140409631090560 error_handling.py:101] training_loop marked as finished\n",
            "INFO:tensorflow:Writing example 0 of 1522\n",
            "I1203 18:40:22.264692 140409631090560 run_binary_classification.py:493] Writing example 0 of 1522\n",
            "INFO:tensorflow:***** Running evaluation *****\n",
            "I1203 18:40:23.077945 140409631090560 run_binary_classification.py:952] ***** Running evaluation *****\n",
            "INFO:tensorflow:  Num examples = 1522 (1522 actual, 0 padding)\n",
            "I1203 18:40:23.078194 140409631090560 run_binary_classification.py:955]   Num examples = 1522 (1522 actual, 0 padding)\n",
            "INFO:tensorflow:  Batch size = 8\n",
            "I1203 18:40:23.078312 140409631090560 run_binary_classification.py:956]   Batch size = 8\n",
            "WARNING:tensorflow:Entity <function file_based_input_fn_builder.<locals>.input_fn.<locals>.<lambda> at 0x7fb34281b950> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: module 'gast' has no attribute 'Str'\n",
            "W1203 18:40:23.098321 140409631090560 ag_logging.py:146] Entity <function file_based_input_fn_builder.<locals>.input_fn.<locals>.<lambda> at 0x7fb34281b950> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: module 'gast' has no attribute 'Str'\n",
            "INFO:tensorflow:Calling model_fn.\n",
            "I1203 18:40:23.112719 140409631090560 estimator.py:1148] Calling model_fn.\n",
            "INFO:tensorflow:Running eval on CPU\n",
            "I1203 18:40:23.112915 140409631090560 tpu_estimator.py:3124] Running eval on CPU\n",
            "INFO:tensorflow:*** Features ***\n",
            "I1203 18:40:23.113217 140409631090560 run_binary_classification.py:634] *** Features ***\n",
            "INFO:tensorflow:  name = input_ids, shape = (?, 128)\n",
            "I1203 18:40:23.113367 140409631090560 run_binary_classification.py:636]   name = input_ids, shape = (?, 128)\n",
            "INFO:tensorflow:  name = input_mask, shape = (?, 128)\n",
            "I1203 18:40:23.113497 140409631090560 run_binary_classification.py:636]   name = input_mask, shape = (?, 128)\n",
            "INFO:tensorflow:  name = is_real_example, shape = (?,)\n",
            "I1203 18:40:23.113595 140409631090560 run_binary_classification.py:636]   name = is_real_example, shape = (?,)\n",
            "INFO:tensorflow:  name = label_ids, shape = (?,)\n",
            "I1203 18:40:23.113696 140409631090560 run_binary_classification.py:636]   name = label_ids, shape = (?,)\n",
            "INFO:tensorflow:  name = segment_ids, shape = (?, 128)\n",
            "I1203 18:40:23.113792 140409631090560 run_binary_classification.py:636]   name = segment_ids, shape = (?, 128)\n",
            "INFO:tensorflow:**** Trainable Variables ****\n",
            "I1203 18:40:25.565872 140409631090560 run_binary_classification.py:670] **** Trainable Variables ****\n",
            "INFO:tensorflow:Done calling model_fn.\n",
            "I1203 18:40:25.704141 140409631090560 estimator.py:1150] Done calling model_fn.\n",
            "INFO:tensorflow:Starting evaluation at 2020-12-03T18:40:25Z\n",
            "I1203 18:40:25.719773 140409631090560 evaluation.py:255] Starting evaluation at 2020-12-03T18:40:25Z\n",
            "INFO:tensorflow:Graph was finalized.\n",
            "I1203 18:40:26.121984 140409631090560 monitored_session.py:240] Graph was finalized.\n",
            "2020-12-03 18:40:26.122646: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 18:40:26.122983: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 0 with properties: \n",
            "name: Tesla T4 major: 7 minor: 5 memoryClockRate(GHz): 1.59\n",
            "pciBusID: 0000:00:04.0\n",
            "2020-12-03 18:40:26.123068: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1\n",
            "2020-12-03 18:40:26.123092: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10\n",
            "2020-12-03 18:40:26.123113: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10\n",
            "2020-12-03 18:40:26.123135: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10\n",
            "2020-12-03 18:40:26.123155: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10\n",
            "2020-12-03 18:40:26.123175: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10\n",
            "2020-12-03 18:40:26.123327: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7\n",
            "2020-12-03 18:40:26.123480: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 18:40:26.123823: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 18:40:26.124068: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1767] Adding visible gpu devices: 0\n",
            "2020-12-03 18:40:26.124108: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1180] Device interconnect StreamExecutor with strength 1 edge matrix:\n",
            "2020-12-03 18:40:26.124122: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1186]      0 \n",
            "2020-12-03 18:40:26.124131: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1199] 0:   N \n",
            "2020-12-03 18:40:26.124240: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 18:40:26.124570: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 18:40:26.124889: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1325] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14221 MB memory) -> physical GPU (device: 0, name: Tesla T4, pci bus id: 0000:00:04.0, compute capability: 7.5)\n",
            "INFO:tensorflow:Restoring parameters from /content/classification_results/1/model.ckpt-4161\n",
            "I1203 18:40:26.126296 140409631090560 saver.py:1284] Restoring parameters from /content/classification_results/1/model.ckpt-4161\n",
            "INFO:tensorflow:Running local_init_op.\n",
            "I1203 18:40:26.904555 140409631090560 session_manager.py:500] Running local_init_op.\n",
            "INFO:tensorflow:Done running local_init_op.\n",
            "I1203 18:40:26.970709 140409631090560 session_manager.py:502] Done running local_init_op.\n",
            "INFO:tensorflow:accuracy = 0.875, loss = 0.676852\n",
            "I1203 18:40:27.765644 140409631090560 basic_session_run_hooks.py:262] accuracy = 0.875, loss = 0.676852\n",
            "INFO:tensorflow:accuracy = 0.9375, loss = 0.0006474487 (0.631 sec)\n",
            "I1203 18:40:28.396987 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9375, loss = 0.0006474487 (0.631 sec)\n",
            "INFO:tensorflow:accuracy = 0.9583333, loss = 0.00051657285 (0.145 sec)\n",
            "I1203 18:40:28.542150 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9583333, loss = 0.00051657285 (0.145 sec)\n",
            "INFO:tensorflow:accuracy = 0.96875, loss = 0.016387569 (0.145 sec)\n",
            "I1203 18:40:28.687385 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.96875, loss = 0.016387569 (0.145 sec)\n",
            "INFO:tensorflow:accuracy = 0.975, loss = 0.00091130927 (0.146 sec)\n",
            "I1203 18:40:28.833097 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.975, loss = 0.00091130927 (0.146 sec)\n",
            "INFO:tensorflow:accuracy = 0.9791667, loss = 0.0017228947 (0.146 sec)\n",
            "I1203 18:40:28.979557 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9791667, loss = 0.0017228947 (0.146 sec)\n",
            "INFO:tensorflow:accuracy = 0.96428573, loss = 0.9693742 (0.146 sec)\n",
            "I1203 18:40:29.125372 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.96428573, loss = 0.9693742 (0.146 sec)\n",
            "INFO:tensorflow:accuracy = 0.9375, loss = 1.5444422 (0.157 sec)\n",
            "I1203 18:40:29.282563 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9375, loss = 1.5444422 (0.157 sec)\n",
            "INFO:tensorflow:accuracy = 0.9444444, loss = 0.0077639157 (0.154 sec)\n",
            "I1203 18:40:29.436615 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9444444, loss = 0.0077639157 (0.154 sec)\n",
            "INFO:tensorflow:accuracy = 0.9375, loss = 0.6549734 (0.148 sec)\n",
            "I1203 18:40:29.584571 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9375, loss = 0.6549734 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.9318182, loss = 0.6567242 (0.146 sec)\n",
            "I1203 18:40:29.730529 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9318182, loss = 0.6567242 (0.146 sec)\n",
            "INFO:tensorflow:accuracy = 0.9270833, loss = 0.26903707 (0.146 sec)\n",
            "I1203 18:40:29.876414 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9270833, loss = 0.26903707 (0.146 sec)\n",
            "INFO:tensorflow:accuracy = 0.9326923, loss = 0.00061859 (0.147 sec)\n",
            "I1203 18:40:30.023099 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9326923, loss = 0.00061859 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.9285714, loss = 0.67529494 (0.149 sec)\n",
            "I1203 18:40:30.171764 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9285714, loss = 0.67529494 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.925, loss = 0.66313696 (0.155 sec)\n",
            "I1203 18:40:30.327190 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.925, loss = 0.66313696 (0.155 sec)\n",
            "INFO:tensorflow:accuracy = 0.9296875, loss = 0.0009786112 (0.152 sec)\n",
            "I1203 18:40:30.478892 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9296875, loss = 0.0009786112 (0.152 sec)\n",
            "INFO:tensorflow:accuracy = 0.9338235, loss = 0.0013179578 (0.149 sec)\n",
            "I1203 18:40:30.627416 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9338235, loss = 0.0013179578 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.9305556, loss = 0.25051278 (0.147 sec)\n",
            "I1203 18:40:30.774031 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9305556, loss = 0.25051278 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.93421054, loss = 0.003350802 (0.147 sec)\n",
            "I1203 18:40:30.920572 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.93421054, loss = 0.003350802 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.93125, loss = 0.43308967 (0.149 sec)\n",
            "I1203 18:40:31.069847 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.93125, loss = 0.43308967 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.9345238, loss = 0.0009512 (0.150 sec)\n",
            "I1203 18:40:31.220005 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9345238, loss = 0.0009512 (0.150 sec)\n",
            "INFO:tensorflow:accuracy = 0.9375, loss = 0.001004631 (0.154 sec)\n",
            "I1203 18:40:31.374014 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9375, loss = 0.001004631 (0.154 sec)\n",
            "INFO:tensorflow:accuracy = 0.9293478, loss = 0.80104744 (0.149 sec)\n",
            "I1203 18:40:31.523495 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9293478, loss = 0.80104744 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.9270833, loss = 0.6791903 (0.150 sec)\n",
            "I1203 18:40:31.673358 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9270833, loss = 0.6791903 (0.150 sec)\n",
            "INFO:tensorflow:accuracy = 0.93, loss = 0.00054556923 (0.147 sec)\n",
            "I1203 18:40:31.820623 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.93, loss = 0.00054556923 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.9326923, loss = 0.0004570332 (0.147 sec)\n",
            "I1203 18:40:31.967864 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9326923, loss = 0.0004570332 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.9351852, loss = 0.0012161571 (0.151 sec)\n",
            "I1203 18:40:32.118655 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9351852, loss = 0.0012161571 (0.151 sec)\n",
            "INFO:tensorflow:accuracy = 0.9375, loss = 0.013286671 (0.151 sec)\n",
            "I1203 18:40:32.269207 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9375, loss = 0.013286671 (0.151 sec)\n",
            "INFO:tensorflow:accuracy = 0.9310345, loss = 1.6133962 (0.152 sec)\n",
            "I1203 18:40:32.420933 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9310345, loss = 1.6133962 (0.152 sec)\n",
            "INFO:tensorflow:accuracy = 0.93333334, loss = 0.0024730207 (0.151 sec)\n",
            "I1203 18:40:32.572026 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.93333334, loss = 0.0024730207 (0.151 sec)\n",
            "INFO:tensorflow:accuracy = 0.9314516, loss = 0.6848977 (0.149 sec)\n",
            "I1203 18:40:32.720572 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9314516, loss = 0.6848977 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.93359375, loss = 0.014226072 (0.147 sec)\n",
            "I1203 18:40:32.867855 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.93359375, loss = 0.014226072 (0.147 sec)\n",
            "INFO:tensorflow:accuracy = 0.93560606, loss = 0.0011857371 (0.151 sec)\n",
            "I1203 18:40:33.019306 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.93560606, loss = 0.0011857371 (0.151 sec)\n",
            "INFO:tensorflow:accuracy = 0.9375, loss = 0.0032511672 (0.149 sec)\n",
            "I1203 18:40:33.168183 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9375, loss = 0.0032511672 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.9357143, loss = 0.85814875 (0.153 sec)\n",
            "I1203 18:40:33.321385 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9357143, loss = 0.85814875 (0.153 sec)\n",
            "INFO:tensorflow:accuracy = 0.9375, loss = 0.025312962 (0.150 sec)\n",
            "I1203 18:40:33.471381 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9375, loss = 0.025312962 (0.150 sec)\n",
            "INFO:tensorflow:accuracy = 0.9358108, loss = 0.5829117 (0.151 sec)\n",
            "I1203 18:40:33.622524 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9358108, loss = 0.5829117 (0.151 sec)\n",
            "INFO:tensorflow:accuracy = 0.9375, loss = 0.0014081923 (0.148 sec)\n",
            "I1203 18:40:33.770896 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9375, loss = 0.0014081923 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.9391026, loss = 0.0010745059 (0.149 sec)\n",
            "I1203 18:40:33.919588 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9391026, loss = 0.0010745059 (0.149 sec)\n",
            "INFO:tensorflow:accuracy = 0.940625, loss = 0.0005433066 (0.150 sec)\n",
            "I1203 18:40:34.069748 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.940625, loss = 0.0005433066 (0.150 sec)\n",
            "INFO:tensorflow:accuracy = 0.94207317, loss = 0.0005127276 (0.151 sec)\n",
            "I1203 18:40:34.220853 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.94207317, loss = 0.0005127276 (0.151 sec)\n",
            "INFO:tensorflow:accuracy = 0.94345236, loss = 0.00082248513 (0.156 sec)\n",
            "I1203 18:40:34.376746 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.94345236, loss = 0.00082248513 (0.156 sec)\n",
            "INFO:tensorflow:accuracy = 0.94186044, loss = 0.677264 (0.150 sec)\n",
            "I1203 18:40:34.527238 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.94186044, loss = 0.677264 (0.150 sec)\n",
            "INFO:tensorflow:accuracy = 0.9431818, loss = 0.0004861341 (0.150 sec)\n",
            "I1203 18:40:34.677255 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9431818, loss = 0.0004861341 (0.150 sec)\n",
            "INFO:tensorflow:accuracy = 0.9388889, loss = 1.7731619 (0.148 sec)\n",
            "I1203 18:40:34.825654 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9388889, loss = 1.7731619 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.9375, loss = 0.96079373 (0.150 sec)\n",
            "I1203 18:40:34.975372 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9375, loss = 0.96079373 (0.150 sec)\n",
            "INFO:tensorflow:accuracy = 0.93351066, loss = 0.94169056 (0.151 sec)\n",
            "I1203 18:40:35.126412 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.93351066, loss = 0.94169056 (0.151 sec)\n",
            "INFO:tensorflow:accuracy = 0.9348958, loss = 0.00044572845 (0.151 sec)\n",
            "I1203 18:40:35.276997 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9348958, loss = 0.00044572845 (0.151 sec)\n",
            "INFO:tensorflow:accuracy = 0.93622446, loss = 0.010211733 (0.153 sec)\n",
            "I1203 18:40:35.430309 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.93622446, loss = 0.010211733 (0.153 sec)\n",
            "INFO:tensorflow:accuracy = 0.9375, loss = 0.0023829814 (0.151 sec)\n",
            "I1203 18:40:35.581103 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9375, loss = 0.0023829814 (0.151 sec)\n",
            "INFO:tensorflow:accuracy = 0.9362745, loss = 0.4583648 (0.152 sec)\n",
            "I1203 18:40:35.732826 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9362745, loss = 0.4583648 (0.152 sec)\n",
            "INFO:tensorflow:accuracy = 0.93028843, loss = 1.5291735 (0.150 sec)\n",
            "I1203 18:40:35.882703 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.93028843, loss = 1.5291735 (0.150 sec)\n",
            "INFO:tensorflow:accuracy = 0.9316038, loss = 0.0017979429 (0.152 sec)\n",
            "I1203 18:40:36.034721 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9316038, loss = 0.0017979429 (0.152 sec)\n",
            "INFO:tensorflow:accuracy = 0.9282407, loss = 1.1633711 (0.150 sec)\n",
            "I1203 18:40:36.184876 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9282407, loss = 1.1633711 (0.150 sec)\n",
            "INFO:tensorflow:accuracy = 0.9227273, loss = 1.9526772 (0.153 sec)\n",
            "I1203 18:40:36.337996 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9227273, loss = 1.9526772 (0.153 sec)\n",
            "INFO:tensorflow:accuracy = 0.92410713, loss = 0.00203904 (0.152 sec)\n",
            "I1203 18:40:36.489643 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.92410713, loss = 0.00203904 (0.152 sec)\n",
            "INFO:tensorflow:accuracy = 0.9254386, loss = 0.0016014895 (0.158 sec)\n",
            "I1203 18:40:36.647176 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9254386, loss = 0.0016014895 (0.158 sec)\n",
            "INFO:tensorflow:accuracy = 0.92025864, loss = 1.3590809 (0.148 sec)\n",
            "I1203 18:40:36.795561 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.92025864, loss = 1.3590809 (0.148 sec)\n",
            "INFO:tensorflow:accuracy = 0.9216102, loss = 0.009135161 (0.152 sec)\n",
            "I1203 18:40:36.947905 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9216102, loss = 0.009135161 (0.152 sec)\n",
            "INFO:tensorflow:accuracy = 0.91875, loss = 1.8740106 (0.150 sec)\n",
            "I1203 18:40:37.098197 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91875, loss = 1.8740106 (0.150 sec)\n",
            "INFO:tensorflow:accuracy = 0.920082, loss = 0.0005985869 (0.156 sec)\n",
            "I1203 18:40:37.253735 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.920082, loss = 0.0005985869 (0.156 sec)\n",
            "INFO:tensorflow:accuracy = 0.91935486, loss = 0.9629752 (0.151 sec)\n",
            "I1203 18:40:37.404892 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91935486, loss = 0.9629752 (0.151 sec)\n",
            "INFO:tensorflow:accuracy = 0.9186508, loss = 0.967149 (0.153 sec)\n",
            "I1203 18:40:37.557697 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9186508, loss = 0.967149 (0.153 sec)\n",
            "INFO:tensorflow:accuracy = 0.91796875, loss = 0.67897695 (0.154 sec)\n",
            "I1203 18:40:37.711827 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91796875, loss = 0.67897695 (0.154 sec)\n",
            "INFO:tensorflow:accuracy = 0.9173077, loss = 0.6602134 (0.152 sec)\n",
            "I1203 18:40:37.863865 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9173077, loss = 0.6602134 (0.152 sec)\n",
            "INFO:tensorflow:accuracy = 0.9166667, loss = 0.89682907 (0.152 sec)\n",
            "I1203 18:40:38.016294 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9166667, loss = 0.89682907 (0.152 sec)\n",
            "INFO:tensorflow:accuracy = 0.91791046, loss = 0.0022354503 (0.151 sec)\n",
            "I1203 18:40:38.166887 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91791046, loss = 0.0022354503 (0.151 sec)\n",
            "INFO:tensorflow:accuracy = 0.91544116, loss = 1.2488148 (0.154 sec)\n",
            "I1203 18:40:38.321295 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91544116, loss = 1.2488148 (0.154 sec)\n",
            "INFO:tensorflow:accuracy = 0.91485506, loss = 0.4026037 (0.152 sec)\n",
            "I1203 18:40:38.473378 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91485506, loss = 0.4026037 (0.152 sec)\n",
            "INFO:tensorflow:accuracy = 0.9160714, loss = 0.02449872 (0.157 sec)\n",
            "I1203 18:40:38.629991 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9160714, loss = 0.02449872 (0.157 sec)\n",
            "INFO:tensorflow:accuracy = 0.9172535, loss = 0.0011005874 (0.150 sec)\n",
            "I1203 18:40:38.780354 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9172535, loss = 0.0011005874 (0.150 sec)\n",
            "INFO:tensorflow:accuracy = 0.9184028, loss = 0.0006653926 (0.154 sec)\n",
            "I1203 18:40:38.934458 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9184028, loss = 0.0006653926 (0.154 sec)\n",
            "INFO:tensorflow:accuracy = 0.91780823, loss = 0.41666108 (0.151 sec)\n",
            "I1203 18:40:39.085268 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91780823, loss = 0.41666108 (0.151 sec)\n",
            "INFO:tensorflow:accuracy = 0.9172297, loss = 0.68499345 (0.155 sec)\n",
            "I1203 18:40:39.240501 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9172297, loss = 0.68499345 (0.155 sec)\n",
            "INFO:tensorflow:accuracy = 0.9166667, loss = 0.8370291 (0.153 sec)\n",
            "I1203 18:40:39.393348 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9166667, loss = 0.8370291 (0.153 sec)\n",
            "INFO:tensorflow:accuracy = 0.9177632, loss = 0.0027795697 (0.153 sec)\n",
            "I1203 18:40:39.546736 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9177632, loss = 0.0027795697 (0.153 sec)\n",
            "INFO:tensorflow:accuracy = 0.9172078, loss = 0.6821154 (0.155 sec)\n",
            "I1203 18:40:39.701617 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9172078, loss = 0.6821154 (0.155 sec)\n",
            "INFO:tensorflow:accuracy = 0.911859, loss = 2.743495 (0.152 sec)\n",
            "I1203 18:40:39.853562 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.911859, loss = 2.743495 (0.152 sec)\n",
            "INFO:tensorflow:accuracy = 0.91297466, loss = 0.0017961741 (0.155 sec)\n",
            "I1203 18:40:40.008075 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91297466, loss = 0.0017961741 (0.155 sec)\n",
            "INFO:tensorflow:accuracy = 0.9140625, loss = 0.00051511626 (0.151 sec)\n",
            "I1203 18:40:40.159314 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9140625, loss = 0.00051511626 (0.151 sec)\n",
            "INFO:tensorflow:accuracy = 0.91512346, loss = 0.009938932 (0.154 sec)\n",
            "I1203 18:40:40.313345 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91512346, loss = 0.009938932 (0.154 sec)\n",
            "INFO:tensorflow:accuracy = 0.91615856, loss = 0.00049618643 (0.152 sec)\n",
            "I1203 18:40:40.464986 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91615856, loss = 0.00049618643 (0.152 sec)\n",
            "INFO:tensorflow:accuracy = 0.9171687, loss = 0.0004972756 (0.154 sec)\n",
            "I1203 18:40:40.619230 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9171687, loss = 0.0004972756 (0.154 sec)\n",
            "INFO:tensorflow:accuracy = 0.9166667, loss = 0.9111779 (0.157 sec)\n",
            "I1203 18:40:40.776574 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9166667, loss = 0.9111779 (0.157 sec)\n",
            "INFO:tensorflow:accuracy = 0.9147059, loss = 1.4793191 (0.152 sec)\n",
            "I1203 18:40:40.928503 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9147059, loss = 1.4793191 (0.152 sec)\n",
            "INFO:tensorflow:accuracy = 0.9142442, loss = 0.66135645 (0.155 sec)\n",
            "I1203 18:40:41.083863 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9142442, loss = 0.66135645 (0.155 sec)\n",
            "INFO:tensorflow:accuracy = 0.9137931, loss = 0.7089546 (0.156 sec)\n",
            "I1203 18:40:41.240163 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9137931, loss = 0.7089546 (0.156 sec)\n",
            "INFO:tensorflow:accuracy = 0.91477275, loss = 0.0051250127 (0.152 sec)\n",
            "I1203 18:40:41.392141 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91477275, loss = 0.0051250127 (0.152 sec)\n",
            "INFO:tensorflow:accuracy = 0.91432583, loss = 0.2924425 (0.153 sec)\n",
            "I1203 18:40:41.545547 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91432583, loss = 0.2924425 (0.153 sec)\n",
            "INFO:tensorflow:accuracy = 0.9138889, loss = 0.6182866 (0.153 sec)\n",
            "I1203 18:40:41.698616 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9138889, loss = 0.6182866 (0.153 sec)\n",
            "INFO:tensorflow:accuracy = 0.91483516, loss = 0.0005413691 (0.154 sec)\n",
            "I1203 18:40:41.852617 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.91483516, loss = 0.0005413691 (0.154 sec)\n",
            "INFO:tensorflow:accuracy = 0.9157609, loss = 0.0010757645 (0.153 sec)\n",
            "I1203 18:40:42.006109 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9157609, loss = 0.0010757645 (0.153 sec)\n",
            "INFO:tensorflow:accuracy = 0.9166667, loss = 0.00045369688 (0.156 sec)\n",
            "I1203 18:40:42.161884 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9166667, loss = 0.00045369688 (0.156 sec)\n",
            "INFO:tensorflow:accuracy = 0.9162234, loss = 0.6959347 (0.155 sec)\n",
            "I1203 18:40:42.316706 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9162234, loss = 0.6959347 (0.155 sec)\n",
            "INFO:tensorflow:accuracy = 0.9144737, loss = 1.5457308 (0.154 sec)\n",
            "I1203 18:40:42.470787 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9144737, loss = 1.5457308 (0.154 sec)\n",
            "INFO:tensorflow:accuracy = 0.9146982, loss = 0.0005333788 (0.101 sec)\n",
            "I1203 18:40:42.571646 140409631090560 basic_session_run_hooks.py:260] accuracy = 0.9146982, loss = 0.0005333788 (0.101 sec)\n",
            "INFO:tensorflow:Finished evaluation at 2020-12-03-18:40:42\n",
            "I1203 18:40:42.726491 140409631090560 evaluation.py:275] Finished evaluation at 2020-12-03-18:40:42\n",
            "INFO:tensorflow:Saving dict for global step 4161: eval_accuracy = 0.8994744, eval_loss = 0.53927267, global_step = 4161, loss = 0.53715724\n",
            "I1203 18:40:42.726770 140409631090560 estimator.py:2049] Saving dict for global step 4161: eval_accuracy = 0.8994744, eval_loss = 0.53927267, global_step = 4161, loss = 0.53715724\n",
            "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 4161: /content/classification_results/1/model.ckpt-4161\n",
            "I1203 18:40:42.727275 140409631090560 estimator.py:2109] Saving 'checkpoint_path' summary for global step 4161: /content/classification_results/1/model.ckpt-4161\n",
            "INFO:tensorflow:evaluation_loop marked as finished\n",
            "I1203 18:40:42.727908 140409631090560 error_handling.py:101] evaluation_loop marked as finished\n",
            "INFO:tensorflow:***** Eval results *****\n",
            "I1203 18:40:42.728104 140409631090560 run_binary_classification.py:976] ***** Eval results *****\n",
            "INFO:tensorflow:  eval_accuracy = 0.8994744\n",
            "I1203 18:40:42.728200 140409631090560 run_binary_classification.py:978]   eval_accuracy = 0.8994744\n",
            "INFO:tensorflow:  eval_loss = 0.53927267\n",
            "I1203 18:40:42.728440 140409631090560 run_binary_classification.py:978]   eval_loss = 0.53927267\n",
            "INFO:tensorflow:  global_step = 4161\n",
            "I1203 18:40:42.728574 140409631090560 run_binary_classification.py:978]   global_step = 4161\n",
            "INFO:tensorflow:  loss = 0.53715724\n",
            "I1203 18:40:42.728662 140409631090560 run_binary_classification.py:978]   loss = 0.53715724\n",
            "INFO:tensorflow:***** Eval results *****\n",
            "I1203 18:40:42.728832 140409631090560 run_binary_classification.py:985] ***** Eval results *****\n",
            "WARNING:tensorflow:Entity <function file_based_input_fn_builder.<locals>.input_fn.<locals>.<lambda> at 0x7fb2d923f268> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: module 'gast' has no attribute 'Str'\n",
            "W1203 18:40:42.749618 140409631090560 ag_logging.py:146] Entity <function file_based_input_fn_builder.<locals>.input_fn.<locals>.<lambda> at 0x7fb2d923f268> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: module 'gast' has no attribute 'Str'\n",
            "INFO:tensorflow:Calling model_fn.\n",
            "I1203 18:40:42.764257 140409631090560 estimator.py:1148] Calling model_fn.\n",
            "INFO:tensorflow:Running infer on CPU\n",
            "I1203 18:40:42.764458 140409631090560 tpu_estimator.py:3124] Running infer on CPU\n",
            "INFO:tensorflow:*** Features ***\n",
            "I1203 18:40:42.764759 140409631090560 run_binary_classification.py:634] *** Features ***\n",
            "INFO:tensorflow:  name = input_ids, shape = (?, 128)\n",
            "I1203 18:40:42.764901 140409631090560 run_binary_classification.py:636]   name = input_ids, shape = (?, 128)\n",
            "INFO:tensorflow:  name = input_mask, shape = (?, 128)\n",
            "I1203 18:40:42.765007 140409631090560 run_binary_classification.py:636]   name = input_mask, shape = (?, 128)\n",
            "INFO:tensorflow:  name = is_real_example, shape = (?,)\n",
            "I1203 18:40:42.765100 140409631090560 run_binary_classification.py:636]   name = is_real_example, shape = (?,)\n",
            "INFO:tensorflow:  name = label_ids, shape = (?,)\n",
            "I1203 18:40:42.765201 140409631090560 run_binary_classification.py:636]   name = label_ids, shape = (?,)\n",
            "INFO:tensorflow:  name = segment_ids, shape = (?, 128)\n",
            "I1203 18:40:42.765298 140409631090560 run_binary_classification.py:636]   name = segment_ids, shape = (?, 128)\n",
            "INFO:tensorflow:**** Trainable Variables ****\n",
            "I1203 18:40:44.997008 140409631090560 run_binary_classification.py:670] **** Trainable Variables ****\n",
            "INFO:tensorflow:Done calling model_fn.\n",
            "I1203 18:40:44.997828 140409631090560 estimator.py:1150] Done calling model_fn.\n",
            "INFO:tensorflow:Graph was finalized.\n",
            "I1203 18:40:45.402159 140409631090560 monitored_session.py:240] Graph was finalized.\n",
            "2020-12-03 18:40:45.402843: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 18:40:45.403169: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 0 with properties: \n",
            "name: Tesla T4 major: 7 minor: 5 memoryClockRate(GHz): 1.59\n",
            "pciBusID: 0000:00:04.0\n",
            "2020-12-03 18:40:45.403258: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1\n",
            "2020-12-03 18:40:45.403285: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10\n",
            "2020-12-03 18:40:45.403309: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10\n",
            "2020-12-03 18:40:45.403331: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10\n",
            "2020-12-03 18:40:45.403352: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10\n",
            "2020-12-03 18:40:45.403375: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10\n",
            "2020-12-03 18:40:45.403415: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7\n",
            "2020-12-03 18:40:45.403527: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 18:40:45.403865: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 18:40:45.404104: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1767] Adding visible gpu devices: 0\n",
            "2020-12-03 18:40:45.404143: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1180] Device interconnect StreamExecutor with strength 1 edge matrix:\n",
            "2020-12-03 18:40:45.404156: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1186]      0 \n",
            "2020-12-03 18:40:45.404165: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1199] 0:   N \n",
            "2020-12-03 18:40:45.404261: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 18:40:45.404660: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 18:40:45.404918: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1325] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14221 MB memory) -> physical GPU (device: 0, name: Tesla T4, pci bus id: 0000:00:04.0, compute capability: 7.5)\n",
            "INFO:tensorflow:Restoring parameters from /content/classification_results/1/model.ckpt-4161\n",
            "I1203 18:40:45.405940 140409631090560 saver.py:1284] Restoring parameters from /content/classification_results/1/model.ckpt-4161\n",
            "INFO:tensorflow:Running local_init_op.\n",
            "I1203 18:40:46.215304 140409631090560 session_manager.py:500] Running local_init_op.\n",
            "INFO:tensorflow:Done running local_init_op.\n",
            "I1203 18:40:46.271452 140409631090560 session_manager.py:502] Done running local_init_op.\n",
            "INFO:tensorflow:prediction_loop marked as finished\n",
            "I1203 18:41:01.263769 140409631090560 error_handling.py:101] prediction_loop marked as finished\n",
            "INFO:tensorflow:prediction_loop marked as finished\n",
            "I1203 18:41:01.264083 140409631090560 error_handling.py:101] prediction_loop marked as finished\n",
            "INFO:tensorflow:Writing example 0 of 1902\n",
            "I1203 18:41:01.281945 140409631090560 run_binary_classification.py:493] Writing example 0 of 1902\n",
            "INFO:tensorflow:***** Running prediction*****\n",
            "I1203 18:41:02.293862 140409631090560 run_binary_classification.py:1013] ***** Running prediction*****\n",
            "INFO:tensorflow:  Num examples = 1902 (1902 actual, 0 padding)\n",
            "I1203 18:41:02.294074 140409631090560 run_binary_classification.py:1016]   Num examples = 1902 (1902 actual, 0 padding)\n",
            "INFO:tensorflow:  Batch size = 8\n",
            "I1203 18:41:02.294172 140409631090560 run_binary_classification.py:1017]   Batch size = 8\n",
            "INFO:tensorflow:***** Predict results *****\n",
            "I1203 18:41:02.294319 140409631090560 run_binary_classification.py:1031] ***** Predict results *****\n",
            "WARNING:tensorflow:Entity <function file_based_input_fn_builder.<locals>.input_fn.<locals>.<lambda> at 0x7fb2d9c10950> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: module 'gast' has no attribute 'Str'\n",
            "W1203 18:41:02.315099 140409631090560 ag_logging.py:146] Entity <function file_based_input_fn_builder.<locals>.input_fn.<locals>.<lambda> at 0x7fb2d9c10950> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: module 'gast' has no attribute 'Str'\n",
            "INFO:tensorflow:Calling model_fn.\n",
            "I1203 18:41:02.329966 140409631090560 estimator.py:1148] Calling model_fn.\n",
            "INFO:tensorflow:Running infer on CPU\n",
            "I1203 18:41:02.330168 140409631090560 tpu_estimator.py:3124] Running infer on CPU\n",
            "INFO:tensorflow:*** Features ***\n",
            "I1203 18:41:02.330473 140409631090560 run_binary_classification.py:634] *** Features ***\n",
            "INFO:tensorflow:  name = input_ids, shape = (?, 128)\n",
            "I1203 18:41:02.330628 140409631090560 run_binary_classification.py:636]   name = input_ids, shape = (?, 128)\n",
            "INFO:tensorflow:  name = input_mask, shape = (?, 128)\n",
            "I1203 18:41:02.330753 140409631090560 run_binary_classification.py:636]   name = input_mask, shape = (?, 128)\n",
            "INFO:tensorflow:  name = is_real_example, shape = (?,)\n",
            "I1203 18:41:02.330847 140409631090560 run_binary_classification.py:636]   name = is_real_example, shape = (?,)\n",
            "INFO:tensorflow:  name = label_ids, shape = (?,)\n",
            "I1203 18:41:02.330938 140409631090560 run_binary_classification.py:636]   name = label_ids, shape = (?,)\n",
            "INFO:tensorflow:  name = segment_ids, shape = (?, 128)\n",
            "I1203 18:41:02.331036 140409631090560 run_binary_classification.py:636]   name = segment_ids, shape = (?, 128)\n",
            "INFO:tensorflow:**** Trainable Variables ****\n",
            "I1203 18:41:04.588768 140409631090560 run_binary_classification.py:670] **** Trainable Variables ****\n",
            "INFO:tensorflow:Done calling model_fn.\n",
            "I1203 18:41:04.589502 140409631090560 estimator.py:1150] Done calling model_fn.\n",
            "INFO:tensorflow:Graph was finalized.\n",
            "I1203 18:41:04.984127 140409631090560 monitored_session.py:240] Graph was finalized.\n",
            "2020-12-03 18:41:04.984816: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 18:41:04.985155: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 0 with properties: \n",
            "name: Tesla T4 major: 7 minor: 5 memoryClockRate(GHz): 1.59\n",
            "pciBusID: 0000:00:04.0\n",
            "2020-12-03 18:41:04.985250: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1\n",
            "2020-12-03 18:41:04.985275: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10\n",
            "2020-12-03 18:41:04.985298: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10\n",
            "2020-12-03 18:41:04.985319: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10\n",
            "2020-12-03 18:41:04.985340: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10\n",
            "2020-12-03 18:41:04.985359: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10\n",
            "2020-12-03 18:41:04.985379: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7\n",
            "2020-12-03 18:41:04.985485: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 18:41:04.985798: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 18:41:04.986044: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1767] Adding visible gpu devices: 0\n",
            "2020-12-03 18:41:04.986089: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1180] Device interconnect StreamExecutor with strength 1 edge matrix:\n",
            "2020-12-03 18:41:04.986102: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1186]      0 \n",
            "2020-12-03 18:41:04.986112: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1199] 0:   N \n",
            "2020-12-03 18:41:04.986210: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 18:41:04.986527: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-12-03 18:41:04.986787: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1325] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14221 MB memory) -> physical GPU (device: 0, name: Tesla T4, pci bus id: 0000:00:04.0, compute capability: 7.5)\n",
            "INFO:tensorflow:Restoring parameters from /content/classification_results/1/model.ckpt-4161\n",
            "I1203 18:41:04.987894 140409631090560 saver.py:1284] Restoring parameters from /content/classification_results/1/model.ckpt-4161\n",
            "INFO:tensorflow:Running local_init_op.\n",
            "I1203 18:41:05.783426 140409631090560 session_manager.py:500] Running local_init_op.\n",
            "INFO:tensorflow:Done running local_init_op.\n",
            "I1203 18:41:05.839469 140409631090560 session_manager.py:502] Done running local_init_op.\n",
            "INFO:tensorflow:prediction_loop marked as finished\n",
            "I1203 18:41:23.812168 140409631090560 error_handling.py:101] prediction_loop marked as finished\n",
            "INFO:tensorflow:prediction_loop marked as finished\n",
            "I1203 18:41:23.812448 140409631090560 error_handling.py:101] prediction_loop marked as finished\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gtkTx91mhCZF",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "b4d7189f-15cf-4f99-aac8-692b7535de66"
      },
      "source": [
        "%cd /content\n",
        "!mkdir /content/classification_results/dev_probs\n",
        "!mv /content/classification_results/1/eval_results.txt  /content/classification_results/dev_probs/eval_results_1.txt"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content\n",
            "mkdir: cannot create directory ‘/content/classification_results/dev_probs’: File exists\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dnlYXxvV2CEF"
      },
      "source": [
        "## Evaluation\n",
        "\n",
        "The parameters of the script below are:\n",
        "\n",
        "**labels_path** is the path where the labels are saved. Label decisions are made using the threshold parameter.\n",
        "\n",
        "**output_path** is the path to file with the results of quality assesment. It contains P, R, F1 scores which are the same as in the output cell."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "H7UOvCj-EH21",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "725e6f4f-1cb2-4a6d-d672-7515a75676a3"
      },
      "source": [
        "%cd /content/smm4h_classification/\n",
        "!python evaluation/evaluate_classification.py \\\n",
        "--true_labels_path=data/preprocessed/ru/dev.tsv \\\n",
        "--predicted_labels_path=/content/classification_results/dev_probs/eval_results_1.txt \\\n",
        "--threshold=0.5 \\\n",
        "--labels_path=/content/classification_results/labels/dev_labels.tsv \\\n",
        "--output_path=/content/classification_results/classification_statistics/eval_results_1.txt"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content/smm4h_classification\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.95      0.93      0.94      1389\n",
            "           1       0.43      0.53      0.47       133\n",
            "\n",
            "    accuracy                           0.90      1522\n",
            "   macro avg       0.69      0.73      0.71      1522\n",
            "weighted avg       0.91      0.90      0.90      1522\n",
            "\n",
            "Precision 0.4268292682926829\n",
            "Recall 0.5263157894736842\n",
            "F-score 0.47138047138047134\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NQylhSrA1haw"
      },
      "source": [
        "## Ensembling\n",
        "For the ensembling, you need to train at least two models. You can do it by iteratively running all the cells between the training and this sections (Not including this one: run this section after you've done training your models).\n",
        "\n",
        "Note: To train a new model, you need to change output paths in the training section. For example, you can do the following changes in scripts parameters:\n",
        "\n",
        "content/classification_results/0 -> content/classification_results/1\n",
        "\n",
        "eval_results_0.txt -> eval_results_1.txt\n",
        "\n",
        "**predicted_probs_dir** is the directory of predicted probabilities.\n",
        "\n",
        "**data_tsv** is the path to the original dataset. The final labels will be added to this dataset. The dataset may not contain true labels. In this case, do not use **\"calculate metrics\"** parameter. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tjutDPRixFnv",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "e67623de-426c-4ca7-fc1f-2fa13ab0ae9a"
      },
      "source": [
        "%cd /content/smm4h_classification/\n",
        "!git pull\n",
        "!python evaluation/ensemble_results.py \\\n",
        "--predicted_probs_dir=/content/classification_results/dev_probs \\\n",
        "--data_tsv=data/preprocessed/ru/dev.tsv \\\n",
        "--calculate_metrics \\\n",
        "--threshold=0.5 \\\n",
        "--output_path=/content/classification_results/labels/dev_labels_ensemble.tsv"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content/smm4h_classification\n",
            "Already up to date.\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.96      0.92      0.94      1389\n",
            "           1       0.40      0.58      0.47       133\n",
            "\n",
            "    accuracy                           0.89      1522\n",
            "   macro avg       0.68      0.75      0.70      1522\n",
            "weighted avg       0.91      0.89      0.90      1522\n",
            "\n",
            "Precision 0.39896373056994816\n",
            "Recall 0.5789473684210527\n",
            "F-score 0.4723926380368098\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "j8ChoxRF7Auv"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}