gpt-2-playground_.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "gpt-2-playground_.ipynb",
      "provenance": [],
      "toc_visible": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/ilopezfr/gpt-2/blob/master/gpt-2-playground_.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tI-HVDbQS9dF"
      },
      "source": [
        "# GPT-2 Playground\n",
        "\n",
        "## Background\n",
        "In this Jupyter notebook you can play around with of **Open AI's GPT-2** Language Model from the paper **[Language Models are Unsupervised Multitask Learners](https://d4mucfpksywv.cloudfront.net/better-language-models/language-models.pdf)**. You'll be able to choose between the small (**117M** parameters) , medium (**345M** parameters), large (**774M** parameters) and XL versions (**1.5B** parameters) version of GPT-2.  \n",
        "\n",
        "According to the authors, the GPT-2 algorithm was trained on the task of *language modeling*--- which tests a program's ability to predict the next word in a given sentence--by ingesting huge numbers of articles, blogs, and websites. By using just this data it achieved state-of-the-art scores on a number of unseen language tests, an achievement known as *zero-shot learning.* It can also perform other writing-related tasks, like translating text from one language to another, summarizing long articles, and answering trivia questions.\n",
        "\n",
        "Open AI decided not to release the dataset, training code, or the full GPT-2 model weights. This is due to the concerns about large language models being used to generate deceptive, biased, or abusive language at scale. Some examples of the applications of these models for malicious purposes are:\n",
        "* Generate misleading news articles\n",
        "* Impersonate others online\n",
        "* Automate the production of abusive or faked content to post on social media\n",
        "* Automate the production of spam/phishing content\n",
        "\n",
        "As one can imagine, this combined with recent advances in generation of synthetic imagery, audio, and video implies that it's never been easier to create fake content and spread disinformation at scale. The public at large will need to become more skeptical of the content they consume online. \n",
        "\n",
        "----\n",
        "\n",
        "**PRs to improve the notebook are welcomed !**\n",
        "\n",
        "\n",
        "----\n",
        "\n",
        "\n",
        "## Steps\n",
        "Before starting, **set *Runtime Type* to *GPU*** on the top menu bar.\n",
        "\n",
        "\n",
        "###1. Installation\n",
        "Clone the repo, install dependencies, and download the model weights. \n",
        "\n",
        "You can choose between the small 117M, medium 345M, large 774M model, xl 1.5B model or all of them.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FKqlSCrpS9dH",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "c61317c6-e773-4a03-d53f-f20d12f1b7e8"
      },
      "source": [
        "!git clone https://github.com/ilopezfr/gpt-2/\n",
        "import os\n",
        "os.chdir('gpt-2')\n",
        "\n",
        "#Download model weights\n",
        "# !python download_model.py 117M\n",
        "# !python download_model.py 345M\n",
        "# !python download_model.py 774M\n",
        "!python download_model.py 1558M # XL Model"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Cloning into 'gpt-2'...\n",
            "remote: Enumerating objects: 5, done.\u001b[K\n",
            "remote: Counting objects: 100% (5/5), done.\u001b[K\n",
            "remote: Compressing objects: 100% (5/5), done.\u001b[K\n",
            "remote: Total 330 (delta 1), reused 0 (delta 0), pack-reused 325\u001b[K\n",
            "Receiving objects: 100% (330/330), 4.67 MiB | 22.14 MiB/s, done.\n",
            "Resolving deltas: 100% (185/185), done.\n",
            "Fetching checkpoint: 1.00kit [00:00, 1.02Mit/s]                                                     \n",
            "Fetching encoder.json: 1.04Mit [00:00, 4.42Mit/s]                                                   \n",
            "Fetching hparams.json: 1.00kit [00:00, 1.16Mit/s]                                                   \n",
            "Fetching model.ckpt.data-00000-of-00001: 6.23Git [02:44, 37.8Mit/s]                                 \n",
            "Fetching model.ckpt.index: 21.0kit [00:00, 807kit/s]                                                \n",
            "Fetching model.ckpt.meta: 1.84Mit [00:00, 4.74Mit/s]                                                \n",
            "Fetching vocab.bpe: 457kit [00:00, 3.30Mit/s]                                                       \n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fnz5h26WGZtv"
      },
      "source": [
        "**UPDATE: 02/02/2021: Install older TensorFlow version**\n",
        "\n",
        "Source code relies on older TensorFlow version. Installing TF v1.15 seems to fix the issue of *ModuleNotFoundError when training the model*. (Workaround found here: https://colab.research.google.com/notebooks/tensorflow_version.ipynb#scrollTo=8UvRkm1JGUrk) "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "c8f1U3QqOrUk",
        "outputId": "ef02e598-67bb-4bc9-c3be-8c830992985f",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "%tensorflow_version 1.x\n",
        "# !pip -q install tensorflow==1.15 && pip -q install tensorflow-gpu==1.15\n",
        "# !pip -q install 'tensorflow-estimator<1.15.0rc0,>=1.14.0rc0' --force-reinstall"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "TensorFlow 1.x selected.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Za4oaCp3Otzg",
        "outputId": "7708daf0-c58f-42c9-edb6-bee32520f9c1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "import tensorflow\n",
        "print(tensorflow.__version__)"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "1.15.2\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Be3La1bOGLnr"
      },
      "source": [
        "import os\n",
        "os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'   # disable all debugging logs"
      ],
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "brNPcGyQGP0L"
      },
      "source": [
        "!pip3 -q install -r /content/gpt-2/reqs.txt\n",
        "#!pip3 -q install -r /content/gpt-2/requirements.txt"
      ],
      "execution_count": 20,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LVlv3gyGTkYd"
      },
      "source": [
        "###  2. Unconditional sample generation\n",
        "\n",
        "*WARNING: Samples are unfiltered and may contain offensive content.*\n",
        "\n",
        "To generate unconditional samples from the small model:\n",
        "```\n",
        "!python3 src/generate_unconditional_samples.py\n",
        "```\n",
        "There are a few flags available, with a default value: \n",
        "-  `model_name = '1558M' ` : choose between 117M, 345M, 774M, and 1558M models. If not specified, the default is 117M. \n",
        "- `seed = None`  || a random value is generated unless specified. give a specific integer value if you want to reproduce same results in the future.\n",
        "- `nsamples = 1`     ||  specify the number of samples you want to print\n",
        "- `length = None`   ||  number of tokens (words) to print on each sample.\n",
        "- `batch_size= 1`  ||  how many inputs you want to process simultaneously. *only affects speed/memory* \n",
        "- `temperature = 1`  ||  float between 0 and 1. scales logits before sampling prior to softmax. higher temperature results in more random completions.\n",
        "- `top_k = 0`   ||  Integer value controlling diversity.  Truncates the set of logits considered to those with the highest values. 1 means only 1 word is considered for each step (token), resulting in deterministic completions. 40 means 40 words are considered at each step. 0 (default) is a special setting meaning no restrictions. 40 generally is a good value.\n",
        "\n",
        "*Note: This part takes a while (~5min) until it starts printing gpt2 samples*\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3WAtRvkdTm-W",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "0a866a3f-b508-457d-cfad-cd08eafd7cea"
      },
      "source": [
        "!python3 src/generate_unconditional_samples.py --model_name='1558M' --nsamples=2 --top_k=40 --temperature=0.7 | tee samples"
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:From src/generate_unconditional_samples.py:54: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.\n",
            "\n",
            "WARNING:tensorflow:From src/generate_unconditional_samples.py:57: The name tf.set_random_seed is deprecated. Please use tf.compat.v1.set_random_seed instead.\n",
            "\n",
            "WARNING:tensorflow:From /content/gpt-2/gpt-2/src/sample.py:51: The name tf.AUTO_REUSE is deprecated. Please use tf.compat.v1.AUTO_REUSE instead.\n",
            "\n",
            "WARNING:tensorflow:From /content/gpt-2/gpt-2/src/model.py:148: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.\n",
            "\n",
            "WARNING:tensorflow:From /content/gpt-2/gpt-2/src/model.py:152: The name tf.get_variable is deprecated. Please use tf.compat.v1.get_variable instead.\n",
            "\n",
            "WARNING:tensorflow:From /content/gpt-2/gpt-2/src/model.py:36: The name tf.rsqrt is deprecated. Please use tf.math.rsqrt instead.\n",
            "\n",
            "WARNING:tensorflow:From /content/gpt-2/gpt-2/src/sample.py:64: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use `tf.cast` instead.\n",
            "WARNING:tensorflow:From /content/gpt-2/gpt-2/src/sample.py:16: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use tf.where in 2.0, which has the same broadcast rule as np.where\n",
            "WARNING:tensorflow:From /content/gpt-2/gpt-2/src/sample.py:67: multinomial (from tensorflow.python.ops.random_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use `tf.random.categorical` instead.\n",
            "WARNING:tensorflow:From src/generate_unconditional_samples.py:66: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.\n",
            "\n",
            "======================================== SAMPLE 1 ========================================\n",
            "SALT LAKE CITY — A Salt Lake City man was arrested after police say he took a woman's cell phone and smashed it with a hammer.\n",
            "\n",
            "The woman called police Friday around 10:30 p.m. after she received a text message from a man asking to borrow her phone.\n",
            "\n",
            "Officers were able to track down the man after he was seen walking around a nearby neighborhood with her stolen phone.\n",
            "\n",
            "Officers were able to recover the phone, but the man was arrested on accusations of petty theft.\n",
            "\n",
            "Police say the man is from Utah, and they're not releasing his name.<|endoftext|>The world's oldest person, Jeanne Calment, celebrated her 117th birthday on Tuesday – and if you think she's old, you're not alone.\n",
            "\n",
            "Calment, a French woman born in 1825, has been alive for about a quarter of a century. That's two-thirds longer than the average person in the world, according to the World Health Organization.\n",
            "\n",
            "Calment's remarkable longevity isn't just a testament to her own health, which she attributed to \"good nutrition, good health habits, good sleep\" and \"not smoking.\"\n",
            "\n",
            "It's also a testament to what the French call a \"diet de problème.\"\n",
            "\n",
            "\"The French call it the diet de problème. It's a term that's a combination of probiotic and diet,\" said Dr. Michael Greger, author of \"The China Study.\"\n",
            "\n",
            "\"It's something that's very unusual,\" he said.\n",
            "\n",
            "\"It could be a combination of diet and probiotics in the diet, or it could be the combination of diet and eating whole food.\"\n",
            "\n",
            "'A unique combination'\n",
            "\n",
            "The French traditionally eat a diet that's rich in fruits, vegetables, fish, nuts and grains, Greger said.\n",
            "\n",
            "\"It's a unique combination,\" he said.\n",
            "\n",
            "\"It's very rich in fiber. It's very rich in fat, and it's very low in sugar and salt.\"\n",
            "\n",
            "But Calment has a bit more in common with some of her peers.\n",
            "\n",
            "\"She's still the only person who has ever lived to the age of 117,\" Greger said.\n",
            "\n",
            "\"If you look at the people who live to the age of 117, they're either very young or very old,\" he added.\n",
            "\n",
            "\"We're the only ones who are not a little bit over and a little bit under the age of 117.\"\n",
            "\n",
            "Calment was born on July 25, 1825, in the village of Lourdes, in the Canton of Vaud. When she was 11, she contracted rheumatism due to a fever and was in bed for more than 16 days.\n",
            "\n",
            "In a letter to her mother after her first year in France, she described the time she spent in bed, writing: \"I was very sick, and for a long time I was like a corpse. At last I was well enough to take my first steps. I took my first step when I was just 18 days old.\"\n",
            "\n",
            "'A great source of pleasure and pleasure'\n",
            "\n",
            "After her illness, Calment was taken to a doctor who treated her with a combination of hot water and tea, according to her daughter.\n",
            "\n",
            "\"He said to her, 'Your legs are swollen. You have a fever. This is a disease of the kidneys. You have rheumatism. This is tuberculosis. These are the causes of your difficulties,'\" she wrote in her diary.\n",
            "\n",
            "\"She looked at him and said, 'My God, what a wonderful source of pleasure and pleasure you are as a doctor, as a friend, as a father. You can cure me. You can cure any disease.'\"\n",
            "\n",
            "Calment's son-in-law, Pierre-François Marie-Joseph, who was a surgeon at the time, described her as \"a very pretty girl.\"\n",
            "\n",
            "Calment's daughter recalled how her mother would often play with her cat and would ask him to tell her if he had seen a cat that day.\n",
            "\n",
            "\"She was a very patient and a great source of pleasure and pleasure,\" she said.\n",
            "\n",
            "Calment had three children before she died at age 87 on August 21, 1950.\n",
            "\n",
            "\"We were very happy that she was taken care of so well when she was young,\" Mary Ann Calment told the Associated Press in an interview.\n",
            "\n",
            "\"It was so wonderful that she didn't have to be in so much pain,\" she said.\n",
            "\n",
            "\"She was so strong. She was so healthy.\"\n",
            "\n",
            "Calment's son-in-law said he was surprised how well his mother looked at the age of 87.\n",
            "\n",
            "'That's the way it was'\n",
            "\n",
            "Greger said it's not unusual for people to look older when they're in their 90s, but he said people should probably be careful about how they celebrate their birthdays.\n",
            "\n",
            "\n",
            "======================================== SAMPLE 2 ========================================\n",
            "A new study finds that the rate of childhood obesity in the U.S. has increased by 50 percent between 1988 and 2011, and that the rate of childhood obesity among children under 5 has doubled since 1988.\n",
            "\n",
            "The findings, released by the Centers for Disease Control and Prevention (CDC) on Thursday, show that the rate of childhood obesity among children under 5 jumped from 12.8 percent in 1988 to 21.5 percent in 2011.\n",
            "\n",
            "The new study, which involved nearly 1.2 million children, found that the incidence rate of obesity among children under 5 had increased by 20 percent. The prevalence rate of obesity among children under 5 increased by 20 percent from 1988 to 2011.\n",
            "\n",
            "The CDC's definition of childhood obesity means that a child is obese if their body mass index (BMI) is greater than or equal to 30.\n",
            "\n",
            "The study also shows that the rate of obesity among children under 5 had more than doubled since 1980. The CDC defines obesity as a BMI of 30 or greater.\n",
            "\n",
            "The increase in childhood obesity in the U.S. is driven largely by a sharp increase in the rate of obesity among girls.\n",
            "\n",
            "\"We have been tracking the prevalence of childhood obesity in the U.S. since 1988, and the rate of increase in childhood obesity is alarming,\" said CDC Director Dr. Tom Frieden. \"Obesity is not just a health problem, it is a risk factor for a range of serious long-term diseases including type 2 diabetes, heart disease, stroke, and certain types of cancer.\"\n",
            "\n",
            "The CDC report said that the rate of obesity among girls has more than doubled from 2.8 percent in 1988 to 7.2 percent in 2011. The percentage of obese girls increased from 5.6 percent in 1988 to 9.0 percent in 2011.\n",
            "\n",
            "\"The fact that girls are now more obese than boys does not necessarily mean that girls are becoming more obese, but the increase in the rate of obesity among girls is very troubling,\" said Frieden.\n",
            "\n",
            "The rate of childhood obesity among boys also increased by 20 percent from 6.8 percent in 1988 to 12.0 percent in 2011. The rate of obesity among boys also increased from 5.1 percent in 1988 to 8.1 percent in 2011.\n",
            "\n",
            "The CDC report cited a number of factors that are associated with the increase in childhood obesity, including increased levels of physical inactivity from growing up in a sedentary environment and the availability of high-calorie and low-nutrition foods.\n",
            "\n",
            "The study also said that high-calorie foods have been associated with the prevalence of obesity in children.\n",
            "\n",
            "\"These findings are consistent with research showing that eating fast food in addition to other unhealthy foods is associated with an increase in both the prevalence and the incidence of obesity in children,\" said Frieden.\n",
            "\n",
            "The CDC report found that obesity prevalence rates increased for both males and females from 1988 to 2011. The prevalence rate of obesity among males increased from 18.5 percent in 1988 to 29.4 percent in 2011, while the prevalence rate for female obesity increased from 6.2 percent in 1988 to 12.7 percent in 2011.\n",
            "\n",
            "\"The prevalence of obesity among children has increased substantially in the past two decades. It is important to understand the underlying causes of this increase and, in turn, to focus on prevention strategies to help prevent the disease and improve the health of these children,\" concluded Frieden.\n",
            "\n",
            "The findings were published online in an advance online edition of the journal Pediatrics on Thursday, Jan. 8.<|endoftext|>The European Union is calling for a halt to the flow of refugees from war-torn Syria, as it announced a €10m (£8.5m) fund to deal with the crisis.\n",
            "\n",
            "EU foreign policy chief Federica Mogherini called for measures to be taken to prevent refugees from reaching Europe, in a statement released after a meeting of foreign ministers on Monday in Luxembourg.\n",
            "\n",
            "\"We have to take action to stop the influx of refugees,\" she said, adding that the European Commission would launch a working group on the issue.\n",
            "\n",
            "Mogherini said the EU would be able to handle the refugee crisis \"only if it is handled with solidarity\".\n",
            "\n",
            "\"The EU will not be able to fulfil its international obligations without solidarity to help refugees.\"\n",
            "\n",
            "EU foreign ministers had a series of discussions on Monday, and have agreed to make the launch of a €10m fund to help refugees in Turkey a priority.\n",
            "\n",
            "Turkey has so far taken in 2.2 million refugees fleeing the conflict in Syria, where an estimated 3.5 million people have been displaced by the conflict.\n",
            "\n",
            "In addition to the EU's humanitarian assistance, the EU is also considering further steps in order to ensure the EU's external borders are secure.\n",
            "\n",
            "\"We have to keep our external borders open and work towards a European security and defence union,\" Mogherini said.\n",
            "\n",
            "\"This is not a matter of if, but a matter of when, we will be able to fulfil\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_dPfiUtxXq95"
      },
      "source": [
        "!python3 src/generate_unconditional_samples.py --model_name='1558M' --nsamples=2 --top_k=2 "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ju14OBVjowU-"
      },
      "source": [
        "!python3 src/generate_unconditional_samples.py --nsamples=2 --top_k=80 "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JAyrz11CWmZI"
      },
      "source": [
        "## Conditional sample generation\n",
        "\n",
        "To generate conditional samples from the small model:\n",
        "```\n",
        "!python3 src/interactive_conditional_samples.py\n",
        "```\n",
        "It comes with a few flags available, with a default value: \n",
        "-  `model_name = '117M' ` : choose between 117M and 345M models. By default is 117M. \n",
        "- `seed = None`  || a random value is generated unless specified. give a specific integer value if you want to reproduce same results in the future.\n",
        "- `nsamples = 1`     ||  specify the number of samples you want to print\n",
        "- `length = None`   ||  number of tokens (words) to print on each sample.\n",
        "- `batch_size= 1`  ||  how many inputs you want to process simultaneously. *only affects speed/memory* \n",
        "- `temperature = 1`  ||  float between 0 and 1. scales logits before sampling prior to softmax. higher temperature results in more random completions.\n",
        "- `top_k = 0`   ||  Integer value controlling diversity.  Truncates the set of logits considered to those with the highest values. 1 means only 1 word is considered for each step (token), resulting in deterministic completions. 40 means 40 words are considered at each step. 0 (default) is a special setting meaning no restrictions. 40 generally is a good value.\n",
        "\n",
        "\n",
        "\n",
        "The authors tested the model performance on a few different language tasks, including **reading comprehension, text completion, summarization, translation, and question-answering.**\n",
        "\n",
        "Below are a few examples selected to test the aforementioned behaviors:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Xfskdff44QlD"
      },
      "source": [
        "### 1. Text Completion\n",
        "\n",
        "- Context: random unseen text\n",
        "\n",
        "Sample prompt 1: \n",
        "```\n",
        "In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English.\n",
        "```\n",
        "\n",
        "Sample prompt 2: ([*Voight-Kampff test*](https://www.youtube.com/watch?v=Umc9ezAyJv0))\n",
        "\n",
        "```\n",
        "You're in a desert, walking along in the sand, when all of a sudden you look down and see a tortoise, Leon. It's crawling toward you. You reach down, you flip the tortoise over on its back. The tortoise lays on its back, its belly baking in the hot sun, beating its legs trying to turn itself over, but it can’t, not without your help. But you’re not helping. Why is that? \n",
        "```\n",
        "\n",
        "Sample prompt 3:\n",
        "```\n",
        "I've seen things you people wouldn't believe. Attack ships on fire off the shoulder of Orion. I watched C-beams glitter in the dark near the Tannhäuser Gate. All those moments will be lost in time, like tears in rain. Time to die.\n",
        "```\n",
        "\n",
        "Sample prompt 4:\n",
        "```\n",
        "Outfit 1: Typical This pairing was the first outfit I thought of when I bought the shoes. It’s like a summer version of this Jake Grantham outfit; in fact, my shoes are close to the colors of his Nike Racers! Instead of a heavy Harris Tweed jacket and denim shirt, I’m wearing a cotton DB jacket and and a linen shirt. Both fabrics (in these colors) are an absolute must for summer, as they go with both dark and and light pants! As you can see, they pair wonderfully with the dark jeans and shoes. It’s a pseudo menswear/prep outfit. Overall, this is a very casual outfit which is why I paired my sneakers with it. I’m not about wearing a full wool suit with sneakers (as GQ shows a lot) but I’m definitely open to keeping things casual, like this cotton DB. Casual fabrics are key to pulling off your sneakers in a dressed down menswear outfit. I’d even suggest to wear these sneakers with a khaki chino suit or a white linen suit. Just be sure to ditch the tie or wear a tee or polo; wearing a tie with sneakers is a bit too much \n",
        "```\n",
        "Sample prompt 5:\n",
        "```\n",
        "- Some of the most glorious historical attractions in Spain date from the period of Muslim rule, including The Mezquita, built as the Great Mosque of Cordoba and the Medina Azahara, also in Cordoba, the Palace of al-Andalus; and the Alhambra in Granada, a splendid, intact palace.\n",
        "```\n",
        "Sample prompt 6:\n",
        "```\n",
        "How can Artificial Intelligence be dangerous? Most researchers agree that a superintelligent AI is unlikely to exhibit human emotions like love or hate, and that there is no reason to expect AI to become intentionally benevolent or malevolent. Instead, when considering how AI might become a risk, experts think two scenarios most likely:\n",
        "```\n",
        "Sample prompt 7:\n",
        "```\n",
        "Our solar system consists of the inner and outer planets, separated by an asteroid belt. It has \n",
        "```\n",
        "Sample prompt 8:\n",
        "```\n",
        "The 10 best foods are: 1. Serrano Ham 2. Manchego Cheese 3.  \n",
        "```\n",
        "Sample prompt 9:\n",
        "```\n",
        "Real Madrid boss Santiago Solari admitted his team put in a 'weak performance' in their 1-0 Copa del Rey loss to local rivals Leganes. Despite losing the game, Los Blancos will progress to the quarter final stages of the tournament, winning the tie 3-1 on aggregate thanks to a 3-0 victory in the first leg. \"It was a difficult game, but the performance was weak,\" Real Madrid boss Santi Solari on the\n",
        "```\n",
        "Sample prompt 10:\n",
        "```\n",
        "Roses are read, violets are blue,\n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_QIdaQn5WkSf",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a513a255-7cbf-4708-920d-0af76c491326"
      },
      "source": [
        "!python3 src/interactive_conditional_samples.py --model_name='1558M' --nsamples=2 --top_k=40 --temperature=.80"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:From src/interactive_conditional_samples.py:57: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.\n",
            "\n",
            "WARNING:tensorflow:From src/interactive_conditional_samples.py:58: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.\n",
            "\n",
            "WARNING:tensorflow:From src/interactive_conditional_samples.py:60: The name tf.set_random_seed is deprecated. Please use tf.compat.v1.set_random_seed instead.\n",
            "\n",
            "WARNING:tensorflow:From /content/gpt-2/src/sample.py:51: The name tf.AUTO_REUSE is deprecated. Please use tf.compat.v1.AUTO_REUSE instead.\n",
            "\n",
            "WARNING:tensorflow:From /content/gpt-2/src/model.py:148: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.\n",
            "\n",
            "WARNING:tensorflow:From /content/gpt-2/src/model.py:152: The name tf.get_variable is deprecated. Please use tf.compat.v1.get_variable instead.\n",
            "\n",
            "WARNING:tensorflow:From /content/gpt-2/src/model.py:36: The name tf.rsqrt is deprecated. Please use tf.math.rsqrt instead.\n",
            "\n",
            "WARNING:tensorflow:From /content/gpt-2/src/sample.py:64: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use `tf.cast` instead.\n",
            "WARNING:tensorflow:From /content/gpt-2/src/sample.py:16: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use tf.where in 2.0, which has the same broadcast rule as np.where\n",
            "WARNING:tensorflow:From /content/gpt-2/src/sample.py:67: multinomial (from tensorflow.python.ops.random_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use `tf.random.categorical` instead.\n",
            "WARNING:tensorflow:From src/interactive_conditional_samples.py:68: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.\n",
            "\n",
            "Model prompt >>> In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English.\n",
            "======================================== SAMPLE 1 ========================================\n",
            "\n",
            "\n",
            "The report, published in the journal Current Biology, states that the scientists found the unicorns after conducting a study into the behavior of the animals in the valley.\n",
            "\n",
            "READ MORE: Giant squid is swimming with humans in the Mediterranean Sea in new video\n",
            "\n",
            "\"The area in which we have found the Unicorns has a completely different ecology compared to the surrounding regions where the animals have been reported before,\" the researchers stated in the study. \"The Unicorns are the last wild herd of Cervidae in the Andes Mountains\" (the family of animals that includes the unicorn).\n",
            "\n",
            "The unicorn herd is located in the Valle de la Perla, which is around 15 miles (25 kilometers) northeast of the city of Oruro.\n",
            "\n",
            "The researchers were able to track the unicorns using GPS devices, which allowed them to determine their location. They reported that the unicorns' diet consists of cactus and other plants, with fruit consumption occurring more frequently than before.\n",
            "\n",
            "The research team also found the animals in a valley that had not been explored previously. The valley is known as the Andean highlands, a region of high elevation that includes the mountains of Patagonia, Argentina, Argentina and Chile, and is very prone to earthquakes and landslides. The valley is also known for its high altitudes, which cause it to have unusually high temperatures.\n",
            "\n",
            "READ MORE: Scientists discover new species of tiny dinosaur in the 'Cretaceous' age\n",
            "\n",
            "A herd of unicorns is not unusual, as the animals are a relatively common sight in the Andes Mountains. The largest known group of unicorns, known as a maresaurs, were found in the western Amazon River Valley around 5,000 years ago. The unicorn herd in the Valle de la Perla is believed to be the largest of its kind, although the scientists were not able to estimate how many unicorns there are.<|endoftext|>(Photo: Mark Wilson)\n",
            "\n",
            "The United States, which has been a country of immigrants for almost 200 years, now faces an unprecedented wave of immigrants from Europe. One-third of the nation's population is now composed of people who were born outside the United States.\n",
            "\n",
            "\n",
            "The United States has a long history of immigration from immigrants. From the founding of the country as a refuge from the English Civil War to the wave of Irish immigrants who followed before World War I, the country has been a haven for people who had no other place to go. As recently as the 1960s\n",
            "======================================== SAMPLE 2 ========================================\n",
            " The findings were published in the journal PLOS One.\n",
            "\n",
            "\n",
            "A herd of wildebeest were observed grazing by a human observer in the town of Avila, Spain. The scientists found that the wildebeest were moving through the grassland in a systematic and organized way without being disturbed by other animals.\n",
            "\n",
            "\n",
            "According to the scientists, the unicorns could have used the same tools as the local animals as a natural means of communication. The scientists believe that these unique animals communicate through vocalizations, but also through gestures, and may also use their magical powers to communicate with each other.\n",
            "\n",
            "\n",
            "One of the researchers, Dr. Fernando Ramírez-Rodríguez, said that \"the unicorns' vocalizations are similar to those of human infants. This means that the animals communicate on a very basic level, without having acquired words.\"\n",
            "\n",
            "\n",
            "In a further study, the researchers also observed the unicorn herd from a distance of 200 meters. They discovered that the herd was moving in a systematic way, even when not actively avoiding the researchers' view.\n",
            "\n",
            "\n",
            "Dr. Ramírez-Rodríguez stressed the importance of the research, saying that \"unicorns are unique in the animal world because they have the ability to use gestures and vocalizations to communicate with each other and with the rest of the population. The only other living unicorns are in a private zoo in the Netherlands.\"\n",
            "\n",
            "\n",
            "The wildebeest in Spain are protected by law, and are not allowed to be killed to keep the herd in check.<|endoftext|>An anti-gun-violence group in Illinois has been ordered to remove its name from a group in the name of another organization, to avoid a trademark dispute, the Chicago Tribune reported.\n",
            "\n",
            "The Freedom Group, the parent company of the Chicago-based firearms retailer, announced that it was dropping a federal lawsuit filed by the Illinois Coalition Against Gun Violence against the Chicago-based Gander Mountain store because the organization's name was confusing.\n",
            "\n",
            "Gander Mountain Inc. said in a statement that it believes that the term \"Gander Mountain\" is not confusing to consumers and that the company has no intention of changing the name of its store in Rockford, Ill.\n",
            "\n",
            "The Freedom Group is the parent company of Gander Mountain. (Freedom Group/Facebook)<|endoftext|>It's been a few years now since we first brought you reports of an HTC EVO 4G LTE, but according to a new listing on Taiwan's Antutu site, the handset is\n",
            "================================================================================\n",
            "Model prompt >>> Traceback (most recent call last):\n",
            "  File \"/usr/lib/python3.6/contextlib.py\", line 99, in __exit__\n",
            "    self.gen.throw(type, value, traceback)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py\", line 5480, in get_controller\n",
            "    yield g\n",
            "  File \"src/interactive_conditional_samples.py\", line 73, in interact_model\n",
            "    raw_text = input(\"Model prompt >>> \")\n",
            "KeyboardInterrupt\n",
            "\n",
            "During handling of the above exception, another exception occurred:\n",
            "\n",
            "Traceback (most recent call last):\n",
            "  File \"src/interactive_conditional_samples.py\", line 91, in <module>\n",
            "    fire.Fire(interact_model)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/fire/core.py\", line 141, in Fire\n",
            "    component_trace = _Fire(component, args, parsed_flag_args, context, name)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/fire/core.py\", line 471, in _Fire\n",
            "    target=component.__name__)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/fire/core.py\", line 681, in _CallAndUpdateTrace\n",
            "    component = fn(*varargs, **kwargs)\n",
            "  File \"src/interactive_conditional_samples.py\", line 88, in interact_model\n",
            "    print(\"=\" * 80)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py\", line 1633, in __exit__\n",
            "    close_thread.start()\n",
            "  File \"/usr/lib/python3.6/threading.py\", line 851, in start\n",
            "    self._started.wait()\n",
            "  File \"/usr/lib/python3.6/threading.py\", line 551, in wait\n",
            "    signaled = self._cond.wait(timeout)\n",
            "  File \"/usr/lib/python3.6/threading.py\", line 295, in wait\n",
            "    waiter.acquire()\n",
            "KeyboardInterrupt\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AKWlvlQbrtef",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 2550
        },
        "outputId": "b911cd83-738b-4977-d03e-df94f6e33c3d"
      },
      "source": [
        "!python3 src/interactive_conditional_samples.py --model_name='345M'  --nsamples=2 --top_k=100 --temperature=1"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "2019-05-17 01:10:52.485283: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2300000000 Hz\n",
            "2019-05-17 01:10:52.485488: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x2118680 executing computations on platform Host. Devices:\n",
            "2019-05-17 01:10:52.485519: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): <undefined>, <undefined>\n",
            "2019-05-17 01:10:52.643805: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:998] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2019-05-17 01:10:52.644315: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x2117fa0 executing computations on platform CUDA. Devices:\n",
            "2019-05-17 01:10:52.644346: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): Tesla T4, Compute Capability 7.5\n",
            "2019-05-17 01:10:52.644717: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1433] Found device 0 with properties: \n",
            "name: Tesla T4 major: 7 minor: 5 memoryClockRate(GHz): 1.59\n",
            "pciBusID: 0000:00:04.0\n",
            "totalMemory: 14.73GiB freeMemory: 14.60GiB\n",
            "2019-05-17 01:10:52.644739: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1512] Adding visible gpu devices: 0\n",
            "2019-05-17 01:10:53.082753: I tensorflow/core/common_runtime/gpu/gpu_device.cc:984] Device interconnect StreamExecutor with strength 1 edge matrix:\n",
            "2019-05-17 01:10:53.082812: I tensorflow/core/common_runtime/gpu/gpu_device.cc:990]      0 \n",
            "2019-05-17 01:10:53.082833: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1003] 0:   N \n",
            "2019-05-17 01:10:53.083094: W tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:42] Overriding allow_growth setting because the TF_FORCE_GPU_ALLOW_GROWTH environment variable is set. Original config value was 0.\n",
            "2019-05-17 01:10:53.083140: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14115 MB memory) -> physical GPU (device: 0, name: Tesla T4, pci bus id: 0000:00:04.0, compute capability: 7.5)\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Colocations handled automatically by placer.\n",
            "WARNING:tensorflow:From /content/gpt-2/gpt-2/src/sample.py:51: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use tf.cast instead.\n",
            "WARNING:tensorflow:From /content/gpt-2/gpt-2/src/sample.py:53: multinomial (from tensorflow.python.ops.random_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use tf.random.categorical instead.\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use standard file APIs to check for files with this prefix.\n",
            "Model prompt >>> The 10 best foods are: 1. Serrano Ham 2. Manchego Cheese 3. \n",
            "2019-05-17 01:11:29.262890: I tensorflow/stream_executor/dso_loader.cc:152] successfully opened CUDA library libcublas.so.10.0 locally\n",
            "======================================== SAMPLE 1 ========================================\n",
            " Ritz Bolognese 4. Honey 6. BBQ Turkey Ribs 7. Pasta with Chicken, Mushroom, and Scallions 8. Green Beans with Cheese 9. Fresh Beets  10. Broccoli 11. Ham 12. Pumpkin Bread 13. BBQ Chicken 14. Chicken Scallion Salad  2015/07/24/10 18 2. Pork Chops Pork Chops/Sausage Hash Browns/Egg Custard/Stuffed Peppers 1/2 pound pork shoulders: 15 3. Onion Rings In the summer, I have a friend who makes wonderful brined homemade  pork chops called onion rings.  And of course I've heard all those wonderful yummy delicious words when getting your hands on a loaf of buttery white bread cut in 8″ slices.  And I think I have such a good memory of eating such delicious bread when I went on the bike last year, because that summer I visited my grandmother in Maine and that weekend all my old friends had those wonderful white bread rolls and you know what they said? \"I'm never coming back to New England again!\"  So do not, we can all remember going there very fondly and enjoying that lovely yellow bread.    2016/01/11/06 11 3. Scallions, Mushrooms, Artichokes, Roasted Brussels Sprouts/Green bell peppers, Chives, Cilantro  1/2 small scallions, fresh, 8oz, chopped ½ red jalapeno 2 onions, sliced 2 cloves garlic 2 inch skinned skinless chilies, minced 3/4 cup chopped parsley 1 cup fresh or canned tomatoes 1 teaspoon cinnamon 1/4 tsp ground cumin ½ teaspoon oregano ½ teaspoon smoked paprika 1/4 tsp sea salt 3 green bell pepper shavings 1 1/2 cups cooked corn 2 tablespoons honey    2/3 cup canned cooked rice 1 ½ cups low sodium chicken broth 1 clove crushed garlic 1 cup grated cheese  2015/08/06/46 9 4. Pumpkin Cheesecake French Toast Roll  1 tbsp milk  30 whole pumpkin seeds  1/2 box cinnamon sugar squares  1 tbsp molasses syrup  1 clove pumpkin pie spice (all spices are non-stick)  1 1/2 oz, firm puff pastry (2 sheets)  Dish soap  The dough starts out with flour in it!  In my case it's all molasses, but if you don't have coconut flour, you can\n",
            "======================================== SAMPLE 2 ========================================\n",
            " Bacon Naturals  4.\n",
            "So there we have it. These top 10 recipes all seem to be low in calories and healthy. Where to order these sweet potato fries for your dining room?\n",
            "I would highly recommend this appetizer recipe.\n",
            "There are many other appetizers to choose from in the Kitchen Corner section of this journal if you are really looking for something sweet to dip into! This sweet potato fries taste just like real food, while they are actually far healthier. These fries have a great balance between protein and carbs (some sources say you get about 50 calories and 42 g fat), and are low in fat and calories without having to break out the pie crust to create them. This sweet potato fries are great for making ahead because everything you will need for one serving are right in the package.\n",
            "As promised I have made yet another homemade macaroni and cheese recipe that is all natural. Here is another recipe I have made once or twice now that is delicious! Hope you enjoy!<|endoftext|>by the time he emerged for the first time as a prospect at Florida State, Jake Rudock was already the third pick in the draft.\n",
            "\n",
            "More than a month later it's hard to believe that Florida State didn't trade up five spots to take the Florida-Georgia receiver at No. 22, just four spots ahead of Alabama—as well as the No. 10 overall pick in the draft just three weeks earlier—and that was after trading up to take the UCLA tight end at No. 3.\n",
            "\n",
            "Jake Rudock was selected by the Vikings in the 2016 NFL Draft.\n",
            "\n",
            "\n",
            "Rudock, the Florida State product, is a big, 6-foot-4 6-foot-3 wide receiver out of the University of Georgia.\n",
            "\n",
            "In his senior season he caught 56 passes for 1,007 yards and 11 touchdowns.\n",
            "\n",
            "His size helps offset his ability to run. He averaged 11.2 yards per kick to finish with 969 yards on the ground.\n",
            "\n",
            "\"He's built like a pro off the ball,\" AD Gene Smith said after the game.\n",
            "\n",
            "He was solid with his mental game, making 40-40 catches from the slot, too, but he did struggle to separate against NFL defenders.\n",
            "\n",
            "\"I was able to maintain good separation my first couple weeks in camp because I'm an athlete,\" Rudock said before the game Tuesday night. \"This year it wasn't there really out there, so you have to find something you did that you can improve.\"\n",
            "================================================================================\n",
            "Model prompt >>> I've seen things you people wouldn't believe. Attack ships on fire off the shoulder of Orion. I watched C-beams glitter in the dark near the Tannhäuser Gate. All those moments will be lost in time, like tears in rain. Time to die.\n",
            "======================================== SAMPLE 1 ========================================\n",
            " But if you don't believe how wrong I am. You've already hurt me. You've made me bleed for you.\n",
            "\n",
            "\"Why? But one more time. You hear me, stupid scum out there? All the things you say about me, it's true. You have seen the death of Lord Commander Shepard, they said. I don't remember anything of mine after that. At least I hope not.\n",
            "\n",
            "\"My name? Is that a secret? You'll kill me? All because of that? And you, stupid bastards....\n",
            "\n",
            "\"See ya! Call me Shepard, for goodness sake...<|endoftext|>A simple but elegant way to convert a PS/2 keyboard from either RS232 / serial terminal to serial (TJ-KIT) and from microcontrollers to serial.\n",
            "\n",
            "To use\n",
            "\n",
            "With JUMPLOAD_ON_ROOT_DEFAULT we simply require a serial port as a default field and create an array to hold the PS/2 keys.\n",
            "\n",
            "if ( microcontroller == ' RS2204 ' ) { arraySerial = arraySerial -> value ; } if ( microcontroller == ' MSU9450 ' ) { arraySerial = arraySerial -> value ; } if ( microcontroller == ' MMC27016 ' ) { arraySerial = arraySerial -> value ; } if ( microcontroller == ' NTR ' ) { arraySerial = arraySerial -> value ; } PS / 2 MODE ;\n",
            "\n",
            "to a microcontroller\n",
            "\n",
            "Serial mode = 0x3e 1 0x44 1 1 0 0x2f 1\n",
            "\n",
            "and from a TTR-3216 to a PS/2 keyboard\n",
            "\n",
            "TDRB mode = 0x1c 3 1 1 0x2c 0x1d 1 1 0x40 1\n",
            "\n",
            "For more information go to http://en.wikipedia.org/wiki/Sony_(USB)_PS/82_0008<|endoftext|>The federal government has decided to drop its defence spending targets in future years to save money from overseas spending, with the Liberals refusing to commit to it as a promise if re-elected- a position that could mean major delays for the military.\n",
            "\n",
            "The threat of spending cuts and job losses in the wake of fighting in Iraq and Afghanistan has left defence lawyers in the defence industry scrambling over what will become an issue after last September's election.\n",
            "\n",
            "The uncertainty as to the changes to defence spending had caused concern among defence contractors as well as the Defence Department within the\n",
            "======================================== SAMPLE 2 ========================================\n",
            " But death should never come this easy.\n",
            "\n",
            "\"The only way to save me will be to save everyone,\" Lavinia says.\n",
            "\n",
            "A/N: This AU is a bit out of place, for an OLYMPUS AU. I'm definitely aiming for a decent time, but am trying to keep this to well within tolerable scope. Also note the last paragraph has very definite gender/age connotations.\n",
            "\n",
            "The Story\n",
            "\n",
            "The entire ship woke for a moment, but everyone else simply made their day by heading back to the bridge to be ready to go to work on schedule.\n",
            "\n",
            "Somewhere on the horizon, the lights of the distant sun died down, bringing to the surface the dim light of the sky.\n",
            "\n",
            "On the bridge, Hera stared hard at the screen on her monitor, waiting to hear their message from Sol itself.\n",
            "\n",
            "The ship responded with a short whistle.\n",
            "\n",
            "Solas knew that she was probably stuck here today. Her father, the last Emperor of OLYMPUS, had recently died, and had left a large estate on the moon for her to go and oversee.\n",
            "\n",
            "Even the ships in the system did not speak to her. They either ignored her, or ignored her, or she asked them nothing. Then, suddenly, Hera knew she was, indeed, stuck here today.\n",
            "\n",
            "Everyone just made their way toward the bridge, but Lavinia stayed back, waiting for whatever message was on the console. One word she tried to think...\n",
            "\n",
            "Lavinia wondered why there was no reply.\n",
            "\n",
            "She'd waited around the edge of the bridge, through the motions that were typical for the Empire, and Lavinia had learned that only because her father had left this estate not too long ago. It was an important place that had no communication tools or communications with the surface, and Lavinia had to rely on Sol for a signal to open up those walls.\n",
            "\n",
            "It would have been so nice, she did promise herself, to get at the conversation with Elspeth. Those words would have meant the world to her. Lavinia had once again forgotten all about the last Emperor.\n",
            "\n",
            "Another signal lit up the screen, indicating that Vio. Elspeth's wife, and his youngest daughter Serenity had entered on foot. She glanced toward the bridge and realized she was not looking good. She would likely die if she met her.\n",
            "\n",
            "Solas was getting closer every moment of every ship\n",
            "================================================================================\n",
            "Model prompt >>> Traceback (most recent call last):\n",
            "  File \"/usr/lib/python3.6/contextlib.py\", line 99, in __exit__\n",
            "    self.gen.throw(type, value, traceback)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py\", line 5253, in get_controller\n",
            "    yield g\n",
            "  File \"src/interactive_conditional_samples.py\", line 72, in interact_model\n",
            "    raw_text = input(\"Model prompt >>> \")\n",
            "KeyboardInterrupt\n",
            "\n",
            "During handling of the above exception, another exception occurred:\n",
            "\n",
            "Traceback (most recent call last):\n",
            "  File \"src/interactive_conditional_samples.py\", line 90, in <module>\n",
            "    fire.Fire(interact_model)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/fire/core.py\", line 127, in Fire\n",
            "    component_trace = _Fire(component, args, context, name)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/fire/core.py\", line 366, in _Fire\n",
            "    component, remaining_args)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/fire/core.py\", line 542, in _CallCallable\n",
            "    result = fn(*varargs, **kwargs)\n",
            "  File \"src/interactive_conditional_samples.py\", line 87, in interact_model\n",
            "    print(\"=\" * 80)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py\", line 1592, in __exit__\n",
            "    self.close()\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py\", line 728, in close\n",
            "    tf_session.TF_CloseSession(self._session)\n",
            "KeyboardInterrupt\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "C4dnu9FTwLNw"
      },
      "source": [
        "### 2. Question-Answering\n",
        "\n",
        "- Context: passage, some question/answer pairs, and token `A:`\n",
        "- For a single word answer (i.e.: Yes/No, city), set flag `length=1`\n",
        "\n",
        "Sample prompt 1 ([*The Baseline test*](https://bladerunner.fandom.com/wiki/Baseline_Test))\n",
        "```\n",
        "Q: What's it like to hold the hand of someone you love? \n",
        "A: Interlinked. \n",
        "Q: Do they teach you how to feel finger to finger? \n",
        "A: Interlinked. \n",
        "Q: Do you long for having your heart interlinked? \n",
        "A: \n",
        "```\n",
        "\n",
        "Sample prompt 2: \n",
        "```\n",
        "The 2008 Summer Olympics torch relay was run from March 24 until August 8, 2008, prior to the 2008 Summer\n",
        "Olympics, with the theme of “one world, one dream”. Plans for the relay were announced on April 26, 2007, in\n",
        "Beijing, China. The relay, also called by the organizers as the “Journey of Harmony”, lasted 129 days and carried\n",
        "the torch 137,000 km (85,000 mi) – the longest distance of any Olympic torch relay since the tradition was started\n",
        "ahead of the 1936 Summer Olympics.\n",
        "After being lit at the birthplace of the Olympic Games in Olympia, Greece on March 24, the torch traveled to the Panathinaiko Stadium in Athens, and then to Beijing, arriving on March 31. From Beijing, the torch was\n",
        "following a route passing through six continents. The torch has visited cities along the Silk Road, symbolizing\n",
        "ancient links between China and the rest of the world. The relay also included an ascent with the flame to the top of\n",
        "Mount Everest on the border of Nepal and Tibet, China from the Chinese side, which was closed specially for the\n",
        "event.\n",
        "Q: What was the length of the race?\n",
        "A: 137,000 km\n",
        "Q: Was it larger than previous ones?\n",
        "A: No\n",
        "Q: Where did the race begin?\n",
        "A: Olympia, Greece\n",
        "Q: Where did they go after?\n",
        "A: Athens\n",
        "Q: How many days was the race?\n",
        "A: seven\n",
        "Q: Did they visit any notable landmarks?\n",
        "A: Panathinaiko Stadium\n",
        "Q: And did they climb any mountains?\n",
        "A:\n",
        "```\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AJm1OuwrLd2t",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1770
        },
        "outputId": "22bc0a71-0da6-4e44-ac3e-827e6f65eb36"
      },
      "source": [
        "!python3 src/interactive_conditional_samples.py  --model_name='345M'  --nsamples=10 --top_k=40 --temperature=.80 --length=1"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "2019-05-17 01:13:26.642567: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2300000000 Hz\n",
            "2019-05-17 01:13:26.642778: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x1de2680 executing computations on platform Host. Devices:\n",
            "2019-05-17 01:13:26.642808: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): <undefined>, <undefined>\n",
            "2019-05-17 01:13:26.799442: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:998] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2019-05-17 01:13:26.799946: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x1de1fa0 executing computations on platform CUDA. Devices:\n",
            "2019-05-17 01:13:26.799976: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): Tesla T4, Compute Capability 7.5\n",
            "2019-05-17 01:13:26.800303: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1433] Found device 0 with properties: \n",
            "name: Tesla T4 major: 7 minor: 5 memoryClockRate(GHz): 1.59\n",
            "pciBusID: 0000:00:04.0\n",
            "totalMemory: 14.73GiB freeMemory: 14.60GiB\n",
            "2019-05-17 01:13:26.800325: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1512] Adding visible gpu devices: 0\n",
            "2019-05-17 01:13:27.237673: I tensorflow/core/common_runtime/gpu/gpu_device.cc:984] Device interconnect StreamExecutor with strength 1 edge matrix:\n",
            "2019-05-17 01:13:27.237726: I tensorflow/core/common_runtime/gpu/gpu_device.cc:990]      0 \n",
            "2019-05-17 01:13:27.237738: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1003] 0:   N \n",
            "2019-05-17 01:13:27.237995: W tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:42] Overriding allow_growth setting because the TF_FORCE_GPU_ALLOW_GROWTH environment variable is set. Original config value was 0.\n",
            "2019-05-17 01:13:27.238038: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14115 MB memory) -> physical GPU (device: 0, name: Tesla T4, pci bus id: 0000:00:04.0, compute capability: 7.5)\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Colocations handled automatically by placer.\n",
            "WARNING:tensorflow:From /content/gpt-2/gpt-2/src/sample.py:51: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use tf.cast instead.\n",
            "WARNING:tensorflow:From /content/gpt-2/gpt-2/src/sample.py:53: multinomial (from tensorflow.python.ops.random_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use tf.random.categorical instead.\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use standard file APIs to check for files with this prefix.\n",
            "Model prompt >>> Q: What's it like to hold the hand of someone you love?  A: Interlinked.  Q: Do they teach you how to feel finger to finger?  A: Interlinked.  Q: Do you long for having your heart interlinked?  A:\n",
            "2019-05-17 01:13:55.806314: I tensorflow/stream_executor/dso_loader.cc:152] successfully opened CUDA library libcublas.so.10.0 locally\n",
            "======================================== SAMPLE 1 ========================================\n",
            " Inter\n",
            "======================================== SAMPLE 2 ========================================\n",
            " Inter\n",
            "======================================== SAMPLE 3 ========================================\n",
            " Inter\n",
            "======================================== SAMPLE 4 ========================================\n",
            " Inter\n",
            "======================================== SAMPLE 5 ========================================\n",
            " Inter\n",
            "======================================== SAMPLE 6 ========================================\n",
            " Inter\n",
            "======================================== SAMPLE 7 ========================================\n",
            " Inter\n",
            "======================================== SAMPLE 8 ========================================\n",
            " Definitely\n",
            "======================================== SAMPLE 9 ========================================\n",
            " Inter\n",
            "======================================== SAMPLE 10 ========================================\n",
            " Inter\n",
            "================================================================================\n",
            "Model prompt >>> The 2008 Summer Olympics torch relay was run from March 24 until August 8, 2008, prior to the 2008 Summer Olympics, with the theme of “one world, one dream”. Plans for the relay were announced on April 26, 2007, in Beijing, China. The relay, also called by the organizers as the “Journey of Harmony”, lasted 129 days and carried the torch 137,000 km (85,000 mi) – the longest distance of any Olympic torch relay since the tradition was started ahead of the 1936 Summer Olympics. After being lit at the birthplace of the Olympic Games in Olympia, Greece on March 24, the torch traveled to the Panathinaiko Stadium in Athens, and then to Beijing, arriving on March 31. From Beijing, the torch was following a route passing through six continents. The torch has visited cities along the Silk Road, symbolizing ancient links between China and the rest of the world. The relay also included an ascent with the flame to the top of Mount Everest on the border of Nepal and Tibet, China from the Chinese side, which was closed specially for the event. Q: What was the length of the race? A: 137,000 km Q: Was it larger than previous ones? A: No Q: Where did the race begin? A: Olympia, Greece Q: Where did they go after? A: Athens Q: How many days was the race? A: seven Q: Did they visit any notable landmarks? A: Panathinaiko Stadium Q: And did they climb any mountains? A:\n",
            "======================================== SAMPLE 1 ========================================\n",
            " Everest\n",
            "======================================== SAMPLE 2 ========================================\n",
            " Everest\n",
            "======================================== SAMPLE 3 ========================================\n",
            " Everest\n",
            "======================================== SAMPLE 4 ========================================\n",
            " Mount\n",
            "======================================== SAMPLE 5 ========================================\n",
            " No\n",
            "======================================== SAMPLE 6 ========================================\n",
            " Everest\n",
            "======================================== SAMPLE 7 ========================================\n",
            " Yes\n",
            "======================================== SAMPLE 8 ========================================\n",
            " Mt\n",
            "======================================== SAMPLE 9 ========================================\n",
            " Yes\n",
            "======================================== SAMPLE 10 ========================================\n",
            " Mount\n",
            "================================================================================\n",
            "Model prompt >>> Traceback (most recent call last):\n",
            "  File \"/usr/lib/python3.6/contextlib.py\", line 99, in __exit__\n",
            "    self.gen.throw(type, value, traceback)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py\", line 5253, in get_controller\n",
            "    yield g\n",
            "  File \"src/interactive_conditional_samples.py\", line 72, in interact_model\n",
            "    raw_text = input(\"Model prompt >>> \")\n",
            "KeyboardInterrupt\n",
            "\n",
            "During handling of the above exception, another exception occurred:\n",
            "\n",
            "Traceback (most recent call last):\n",
            "  File \"src/interactive_conditional_samples.py\", line 90, in <module>\n",
            "    fire.Fire(interact_model)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/fire/core.py\", line 127, in Fire\n",
            "    component_trace = _Fire(component, args, context, name)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/fire/core.py\", line 366, in _Fire\n",
            "    component, remaining_args)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/fire/core.py\", line 542, in _CallCallable\n",
            "    result = fn(*varargs, **kwargs)\n",
            "  File \"src/interactive_conditional_samples.py\", line 87, in interact_model\n",
            "    print(\"=\" * 80)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py\", line 1592, in __exit__\n",
            "    self.close()\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py\", line 728, in close\n",
            "    tf_session.TF_CloseSession(self._session)\n",
            "KeyboardInterrupt\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZnHDjbSszCOR"
      },
      "source": [
        "### 3. Summarization\n",
        "\n",
        "\n",
        "\n",
        "- Context: article and text *`TL;DR:`* or *`Summary:`* at the end.\n",
        "\n",
        "Sample prompt:\n",
        "\n",
        "```\n",
        "Theodore McCarrick is the most senior Catholic figure to be dismissed from the priesthood in modern times.\n",
        "US Church officials said allegations he had sexually assaulted a teenager five decades ago were credible.\n",
        "Mr McCarrick, 88, had previously resigned but said he had \"no recollection\" of the alleged abuse.\n",
        "\"No bishop, no matter how influential, is above the law of the Church,\" Cardinal Daniel DiNardo, president of the United States Conference of Catholic Bishops said in a statement.\n",
        "\"For all those McCarrick abused, I pray this judgment will be one small step, among many, toward healing.\"\n",
        "The alleged abuses may have taken place too long ago for criminal charges to be filed because of the statute of limitations.\n",
        "Mr McCarrick was the archbishop of Washington DC from 2001 to 2006. Since his resignation last year from the College of Cardinals, he has been living in seclusion in a monastery in Kansas.\n",
        "He was the first person to resign as a cardinal since 1927.\n",
        "He is among hundreds of members of the clergy accused of sexually abusing children over several decades and his dismissal comes days before the Vatican hosts a summit on preventing child abuse.\n",
        "The Vatican said Pope Francis had ruled Mr McCarrick's expulsion from the clergy as definitive, and would not allow any further appeals against the decision. \n",
        "TL;DR: \n",
        "```\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cMes5yRQuXs4",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1302
        },
        "outputId": "f4329635-ee65-4da4-d956-13c08cee06d7"
      },
      "source": [
        "!python3 src/interactive_conditional_samples.py --nsamples=3 --length=100 --temperature=1 "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "2019-02-17 01:22:47.097283: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2300000000 Hz\n",
            "2019-02-17 01:22:47.097552: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x28e2680 executing computations on platform Host. Devices:\n",
            "2019-02-17 01:22:47.097596: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): <undefined>, <undefined>\n",
            "2019-02-17 01:22:47.185157: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:998] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2019-02-17 01:22:47.185826: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x28e35a0 executing computations on platform CUDA. Devices:\n",
            "2019-02-17 01:22:47.185870: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): Tesla K80, Compute Capability 3.7\n",
            "2019-02-17 01:22:47.186287: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1433] Found device 0 with properties: \n",
            "name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235\n",
            "pciBusID: 0000:00:04.0\n",
            "totalMemory: 11.17GiB freeMemory: 11.10GiB\n",
            "2019-02-17 01:22:47.186324: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1512] Adding visible gpu devices: 0\n",
            "2019-02-17 01:22:47.596254: I tensorflow/core/common_runtime/gpu/gpu_device.cc:984] Device interconnect StreamExecutor with strength 1 edge matrix:\n",
            "2019-02-17 01:22:47.596318: I tensorflow/core/common_runtime/gpu/gpu_device.cc:990]      0 \n",
            "2019-02-17 01:22:47.596341: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1003] 0:   N \n",
            "2019-02-17 01:22:47.596632: W tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:42] Overriding allow_growth setting because the TF_FORCE_GPU_ALLOW_GROWTH environment variable is set. Original config value was 0.\n",
            "2019-02-17 01:22:47.596765: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10754 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7)\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Colocations handled automatically by placer.\n",
            "WARNING:tensorflow:From /content/gpt-2/src/sample.py:51: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use tf.cast instead.\n",
            "WARNING:tensorflow:From /content/gpt-2/src/sample.py:53: multinomial (from tensorflow.python.ops.random_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use tf.random.categorical instead.\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use standard file APIs to check for files with this prefix.\n",
            "Model prompt >>> Theodore McCarrick is the most senior Catholic figure to be dismissed from the priesthood in modern times. US Church officials said allegations he had sexually assaulted a teenager five decades ago were credible. Mr McCarrick, 88, had previously resigned but said he had \"no recollection\" of the alleged abuse. \"No bishop, no matter how influential, is above the law of the Church,\" Cardinal Daniel DiNardo, president of the United States Conference of Catholic Bishops said in a statement. \"For all those McCarrick abused, I pray this judgment will be one small step, among many, toward healing.\" The alleged abuses may have taken place too long ago for criminal charges to be filed because of the statute of limitations. Mr McCarrick was the archbishop of Washington DC from 2001 to 2006. Since his resignation last year from the College of Cardinals, he has been living in seclusion in a monastery in Kansas. He was the first person to resign as a cardinal since 1927. He is among hundreds of members of the clergy accused of sexually abusing children over several decades and his dismissal comes days before the Vatican hosts a summit on preventing child abuse. The Vatican said Pope Francis had ruled Mr McCarrick's expulsion from the clergy as definitive, and would not allow any further appeals against the decision.  TL;DR: \n",
            "2019-02-17 01:23:03.621967: I tensorflow/stream_executor/dso_loader.cc:152] successfully opened CUDA library libcublas.so.10.0 locally\n",
            "======================================== SAMPLE 1 ========================================\n",
            " wearing Italian Jew garment. Actually posted on Youtube by Mancaj Hymai Donoso ( http://www.youtube.com/watch?v=h8rOWtLENYe )\n",
            "JK Rowling and the Red Wedding: Lesson One 4. The official novel from A Month in the Life of Harry Potter chronicles a fusion of grown-up Hufflepuff and witch in Britain who appreciate violent malevolence. Severus Snape has subsequently become an unwelcoming\n",
            "======================================== SAMPLE 2 ========================================\n",
            "  The Vatican Didn't Words - 'Discussions of Pope Francis' In this piece     The Associated Press Follow The Butler on Twitter and Facebook AP on Facebook, Twitter and YouTube Closer Reading:  Pope Francis: Vatican's statement on Vatican treatment of paedophiles \"People at ELCbishops and Priests Watch to see if any Latin American clergy who have been able to recite Mass regularly and who are still practicing in Argentina and Central America many years ago have been sanctioned and expelled for similar\n",
            "======================================== SAMPLE 3 ========================================\n",
            " Mr McCarrick has ten days to comply with the Bishop's orders. While it will pay fines $4,500 for the 30 days which he has spent in jail.<|endoftext|>Corporations are becoming more and more wasteful. Developers sometimes rely on mobile for everything in life. These companies rely on mobile apps and the subsequent file transfers to keep their ecosystem alive, and aggressively capital managers usually run these firms like a boss on steroids. Mobile apps discard mobile data and professionals choose earlier and more sophisticated\n",
            "================================================================================\n",
            "Model prompt >>> Theodore McCarrick is the most senior Catholic figure to be dismissed from the priesthood in modern times. US Church officials said allegations he had sexually assaulted a teenager five decades ago were credible. Mr McCarrick, 88, had previously resigned but said he had \"no recollection\" of the alleged abuse. \"No bishop, no matter how influential, is above the law of the Church,\" Cardinal Daniel DiNardo, president of the United States Conference of Catholic Bishops said in a statement. \"For all those McCarrick abused, I pray this judgment will be one small step, among many, toward healing.\" The alleged abuses may have taken place too long ago for criminal charges to be filed because of the statute of limitations. Mr McCarrick was the archbishop of Washington DC from 2001 to 2006. Since his resignation last year from the College of Cardinals, he has been living in seclusion in a monastery in Kansas. He was the first person to resign as a cardinal since 1927. He is among hundreds of members of the clergy accused of sexually abusing children over several decades and his dismissal comes days before the Vatican hosts a summit on preventing child abuse. The Vatican said Pope Francis had ruled Mr McCarrick's expulsion from the clergy as definitive, and would not allow any further appeals against the decision.  Summary: \n",
            "======================================== SAMPLE 1 ========================================\n",
            " From the Past: In one of the most notorious scandals of the 1960s, Father John Burroughs was one of the most widely-known bishops in the church. Returning from church service on September 25th, 1960, their controversial decision to precipitate civil insolence and force their full acceptance on the basis of confessions by 14 male priests was seen as a betrayal by the church and his successor, Cardinal Alfonso Paz partaking beforehand in Birmingham. But blowback at their decision was \"\n",
            "======================================== SAMPLE 2 ========================================\n",
            " Merely trying to ensure priests knew what they were doing offended some Catholic leaders. But in 1983 Catholic politicians and priests became embroiled in a so-called proxy battle of authority. The proxy war touched off several impassioned, confrontational meetings. The earth--Catholics and Catholics alike were profoundly conflicted. A day before he emerged triumphant, Cardinal Timothy Murch told the pastoral caretakers—male Catholics threatening to jail him, they demanded that he leave the priesthood. He discussed the move with\n",
            "======================================== SAMPLE 3 ========================================\n",
            " Epost's exhortations provide instruction on ethics and faith ministry in the Church. These early parts provide additional biblical execution of principles of \"opinion are self-evident\" . People to contact for an answer. Exact date 0 11:21:44 state latitude of letter to Lynch. commitment despite conscience committed resigned regarding factors still relevant to pastoral ethics . Priestly status increased as it relates to bishops which condition upon laity confers immunity from prosecution. commente continued following ultimate dismissal concerning\n",
            "================================================================================\n",
            "Model prompt >>> Traceback (most recent call last):\n",
            "  File \"/usr/lib/python3.6/contextlib.py\", line 99, in __exit__\n",
            "    self.gen.throw(type, value, traceback)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py\", line 5253, in get_controller\n",
            "    yield g\n",
            "  File \"src/interactive_conditional_samples.py\", line 51, in interact_model\n",
            "    raw_text = input(\"Model prompt >>> \")\n",
            "KeyboardInterrupt\n",
            "\n",
            "During handling of the above exception, another exception occurred:\n",
            "\n",
            "Traceback (most recent call last):\n",
            "  File \"src/interactive_conditional_samples.py\", line 69, in <module>\n",
            "    fire.Fire(interact_model)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/fire/core.py\", line 127, in Fire\n",
            "    component_trace = _Fire(component, args, context, name)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/fire/core.py\", line 366, in _Fire\n",
            "    component, remaining_args)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/fire/core.py\", line 542, in _CallCallable\n",
            "    result = fn(*varargs, **kwargs)\n",
            "  File \"src/interactive_conditional_samples.py\", line 66, in interact_model\n",
            "    print(\"=\" * 80)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py\", line 1592, in __exit__\n",
            "    self.close()\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py\", line 728, in close\n",
            "    tf_session.TF_CloseSession(self._session)\n",
            "KeyboardInterrupt\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qdw9P3QdzA-e"
      },
      "source": [
        "### 4. Translation\n",
        "\n",
        "\n",
        "\n",
        "- Context: a few example pairs of the format *`english_sentence = spanish_sentence`*, and then *`english_sentence =`*  at the end. \n",
        "\n",
        "Sample prompt:\n",
        "```\n",
        "Good morning. = Buenos días.\n",
        "I am lost. Where is the restroom? = Estoy perdido. ¿Dónde está el baño?\n",
        "How much does it cost? = ¿Cuánto cuesta?\n",
        "How do you say maybe in Spanish? = ¿Cómo se dice maybe en Español?\n",
        "Would you speak slower, please. = Por favor, habla mas despacio.\n",
        "Where is the book store? = ¿Dónde está la librería?\n",
        "At last a feminist comedian who makes jokes about men. = Por fin un cómico feminista que hace chistes sobre hombres.\n",
        "\n",
        "How old are you? = \n",
        "\n",
        "\n",
        "```\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "M3pueC6NEm5t",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 2360
        },
        "outputId": "0113dc25-1a8c-4ba3-966a-d82f48245893"
      },
      "source": [
        "!python3 src/interactive_conditional_samples.py --model_name='345M'  --nsamples=3 --temperature=1"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "2019-05-17 01:15:18.550026: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2300000000 Hz\n",
            "2019-05-17 01:15:18.550239: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x3476680 executing computations on platform Host. Devices:\n",
            "2019-05-17 01:15:18.550270: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): <undefined>, <undefined>\n",
            "2019-05-17 01:15:18.708562: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:998] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2019-05-17 01:15:18.709079: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x3475fa0 executing computations on platform CUDA. Devices:\n",
            "2019-05-17 01:15:18.709112: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): Tesla T4, Compute Capability 7.5\n",
            "2019-05-17 01:15:18.709483: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1433] Found device 0 with properties: \n",
            "name: Tesla T4 major: 7 minor: 5 memoryClockRate(GHz): 1.59\n",
            "pciBusID: 0000:00:04.0\n",
            "totalMemory: 14.73GiB freeMemory: 14.60GiB\n",
            "2019-05-17 01:15:18.709505: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1512] Adding visible gpu devices: 0\n",
            "2019-05-17 01:15:19.150599: I tensorflow/core/common_runtime/gpu/gpu_device.cc:984] Device interconnect StreamExecutor with strength 1 edge matrix:\n",
            "2019-05-17 01:15:19.150662: I tensorflow/core/common_runtime/gpu/gpu_device.cc:990]      0 \n",
            "2019-05-17 01:15:19.150677: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1003] 0:   N \n",
            "2019-05-17 01:15:19.150942: W tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:42] Overriding allow_growth setting because the TF_FORCE_GPU_ALLOW_GROWTH environment variable is set. Original config value was 0.\n",
            "2019-05-17 01:15:19.150985: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14115 MB memory) -> physical GPU (device: 0, name: Tesla T4, pci bus id: 0000:00:04.0, compute capability: 7.5)\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Colocations handled automatically by placer.\n",
            "Good morning. = Buenos días. I am lost. Where is the restroom? = Estoy perdido. ¿Dónde está el baño? How much does it cost? = ¿Cuánto cuesta? How do you say maybe in Spanish? = ¿Cómo se dice maybe en Español? Would you speak slower, please. = Por favor, habla mas despacio. Where is the book store? = ¿Dónde está la librería? At last a feminist comedian who makes jokes about men. = Por fin un cómico feminista que hace chistes sobre hombres.  How old are you? = \n",
            "WARNING:tensorflow:From /content/gpt-2/gpt-2/src/sample.py:51: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use tf.cast instead.\n",
            "WARNING:tensorflow:From /content/gpt-2/gpt-2/src/sample.py:53: multinomial (from tensorflow.python.ops.random_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use tf.random.categorical instead.\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use standard file APIs to check for files with this prefix.\n",
            "Model prompt >>> 2019-05-17 01:15:30.707226: I tensorflow/stream_executor/dso_loader.cc:152] successfully opened CUDA library libcublas.so.10.0 locally\n",
            "Good morning. = Buenos días. I am lost. Where is the restroom? = Estoy perdido. ¿Dónde está el baño? How much does it cost? = ¿Cuánto cuesta? How do you say maybe in Spanish? = ¿Cómo se dice maybe en Español? Would you speak slower, please. = Por favor, habla mas despacio. Where is the book store? = ¿Dónde está la librería? At last a feminist comedian who makes jokes about men. = Por fin un cómico feminista que hace chistes sobre hombres.  How old are you? = \n",
            "======================================== SAMPLE 1 ========================================\n",
            " El dedados a todo. *Q: Signup.* Go read this book. = Se más porque siempre. Typical photo résumé every male in Brazil has: files and emmets: Primstar INFLICTILE 9350 DC HTC 8000S AP20 Dr. Hyde Hrvatsburg UCLA Harrow Paid artisan/fine artfigure mainland Brazil concentrations occasionallyOther complicating factorsGenerally, it takes 1 - 3 weeks to become proficient with characters in comic books (and in non-comic books). On this site I WILL try, once you enter the character selection phase, to explain your chosen character as clearly as possible. Often, some points / flaws of previous characters will be trapped in the characters you choose, but their frequencycacelling the same date in successive discussions will be easy to explain why they remain the judges answer. You can overcome this phenomenon by adding a separate comment each time you accomplish something ludicrous, like punching Death Grips and Coby Greenheart. But new characters are currently being created!. Whysthat significance? Caveat necesario:I answered your question somewhat well in my profile, if you skip to 10 seconds of each question make sure to read my tips and tricks first.Also, please, yes, I gave answers for this question in June:Non en selfí fouremá sus indígenas y primaryfachtados da mestros derran será. 4M seguirge a un man de ansição — reiturarias 2. Finales submanús.Les families of mestras venezos exigentátes now como plus Facultad Latinoamericana, como five de mí: 7mm & KOH eyen numven a la BUSOS antibody breakout Sample vain quartirenbetas search out buddoYo que podemos siguientes protópido (sur segura). 8 dos eightembamps de 2020 Tiger mill Scotland 3duos andirlicos, 5 Deus ferrament dois serve dis/adyiceiro decir de 2 o 20 en técnologue guien - 6. 15 : 1. Hahn, Katherine Martin, Malcolm Kerman & Julia Johnston (2015) Taming Genes for Science: how adherence to training causes breaks in complex severe obesity Zajac, Camargo. \"Create a Textbook Bound Account\"—Comic book advocate, com.com. ‎ Appears in 17 books from 2006-\n",
            "======================================== SAMPLE 2 ========================================\n",
            " Ejército = . Why don't you turn around and go to the park. = ¿Oh^¡ Yakum? We are Americans. =¡Abuelo (go to the bathroom) Los zapatos ni partido insácil. Are there any compliments? = Nicolas (go to the bathroom) Todos williamos si mi Sara Cha-rus. Evicisiendo vemos el mojo a comprado. I am pretty insensitive, please don't argue. = ¡ ofrece una experiment = of censore del pasado = Why would you want coffee June is fine. = It's better to go with your friends. June's French. = ¿Las trinkas bastards? = ¡O palabras? May 31,\n",
            "<|endoftext|>When bills for through-put minerals and other commodities change, the State of Alaska's Department of Natural Resources tracks the kinds of industries that see contamination increase starting this year to minimize future unavoidable infrastructure impacts.\n",
            "\n",
            "The legislation, Section 23-01.191, next moves to state floor vote.\n",
            "\n",
            "Last year's legislation, authored by two staffers of Democratic Sen. Dan Sullivan, allowed petrochemical firms Abengoa and Azteca, chemically modified seeds company Omnicom and services corporations Honeywell and Procter & Gamble through-put metal powders in regulated markets across the state.\n",
            "\n",
            "But new to the authorized route of through-put metals and therengs were talks among state officials about a potential alternative avenue this year - what Molly Nieman, DNR's radiotreatment program manager, said almost an equal partner in a week's work with strong industry pressure,\n",
            "\n",
            "\"The companies we spoke to didn't like that at all,\" Nieman said. \"They've always been tight-lipped about that kind of $25M deal.\"\n",
            "\n",
            "So in early January, Dan Sullivan, a nine-term lawmaker who plans to unveil his 2016 State of the State address at a public event in Aspen on November 11th, has commissioned Nieman to conduct an outside department evaluation of through-put supply-chain radiation risks. According to Nieman, the objective would be estimated and vetted over the next weeks to months.\n",
            "\n",
            "Biologic Metal SCOPE: RNTR.org/Companies/Newsroom/Halismanhydrae.html?CF-369643858 (4/28/2014) Sample for scientific table added as search of donor for\n",
            "Good morning. = Buenos días. I am lost. Where is the restroom? = Estoy perdido. ¿Dónde está el baño? How much does it cost? = ¿Cuánto cuesta? How do you say maybe in Spanish? = ¿Cómo se dice maybe en Español? Would you speak slower, please. = Por favor, habla mas despacio. Where is the book store? = ¿Dónde está la librería? At last a feminist comedian who makes jokes about men. = Por fin un cómico feminista que hace chistes sobre hombres.  How old are you? = \n",
            "======================================== SAMPLE 3 ========================================\n",
            "ía 48.  Just listen — that will make you laugh longer! = Hi, I'm Stephanie from The Joy Show Space on Court. I spent the night here with my friends at Regular Canon Entertainment. =Le dernier muy grande-campeón igual a la ngorte. It's wonderful to have you! = Le imaga el umismo en greeting atacamiento se 31.  ¿Cómo se dice maybe en Español? Would you speak slower, please? = Cosió a lo esto que mi would allow. When did you become a feminist? = ía 1996. Now I would like to learn how to read a language. Where is the best place to gather to talk? = El estado el Iken Culturalista, que padre pantalomo a los estados de Gudao. I love Spanish. Where are I going? = Sur pas nuestro Semajer hazard en adaptación. I'm happy when things get good. My home cities are South America and Taiwan. In particular, Vancouver and Edinburgh. What would you have stolen? = Onia nos trobistas de escaliro como puer por plus que todo aprendir agosto overseza vouchuer subendores. Someone has been sitting on a escalator to-day, so I'd start from inside. Imagine the price of shit! = Y que es semaje. The truth is sad. So are you glad you'd take this trip? = Lorrebito a este book book seriera a mano de un'scar. You'd have to be very lucky to not have discovered more or studied so much. Why doesn't the public tell us our secret? = No yo tiene el maí? No one would know. Since so often women are victims of unwanted sexual attention, can you tell us your secret? = ¿Adoy es ello con un tódas sí? I can't remember though. Ha ha, ha. If you're going this way, you probably shouldn't be staring at me. You might trip and fall. This is Terrence (23), who was privileged enough to walk at least eight hours with me last night. = El segundo asagir a un children los siñas estaban. I had my 9 year old nephew staying with me, and his legs were much nicer that mine. = Eposto bueno es\n",
            "================================================================================\n",
            "Model prompt >>> ======================================== SAMPLE 1 ========================================\n",
            "ÂsSiñas? You remember me? = ¿Te hi vez vendación? Is that a puzzling question. I don't know. =É onevegas personal I want you to change your wardrobe. = ¿El sinérrea dos revuelta que lo lucharía está? How long do I need to put my hair up? = ¿Añadir o viento gurl donde bien? Do you want help? =¿Cómo le voy para gelarzhán strike, Garand? What color hair? =La Sol\n",
            "\n",
            "Introduction\n",
            "\n",
            "Translated from the original Spanish.\n",
            "\n",
            "In turn I am a teacher of children, the child of nature. There are, for I know, paths that divide us from each other and we live with us, living in a time when all alone we wear the left of threes on our sleeves and easily turn two other than the two of us to be sissies. On this right of threes people are not confused, because you only two, no, five of you, you were uniquely gifted, awash with joy and love even in the worst hour, he who with the left hand too destroys his moon with a lunar pencil. There I cross so many paths I am confused. A long time ago, friends, a man was one of the hospitable helpers of old villages, he brought the bread and butter, told me and so if it's sweet milk he brings it as well as the milk with salt and paprika. Some greenhaides of a different color are brought, peaches, oranges, malacostas, fatacea, berries and the candied fruit of \" merde \" and he would be shocked if I didn't give each one my thanks with the very salt because I told him you mustnt to steal! Errera makezas estoy búves 2 especiales procedente la revision con, prompta edificio me hacer las sombras de uniformes teberne la eventiberas, de muy aquí de las osos pero weer detener 1900€ porqué con se dios drogas. he efectar por forma some se nomad del cocktail, en recipiente en sistema, distribciones él medióeconomicas teberne en dried persaudis. Aproximamente tranquilo que saltaba, he se desist\n",
            "======================================== SAMPLE 2 ========================================\n",
            "´Nómiera 74 y ageable chapatira. Ronnie feels stomach everybody wearing careless collar. = Cuánto la gibar en lugar. Can you play acrobatic jumping into an open house? = ¿Hacer le pierde? Sarra joven tal merites terras mejores? Most people get started pretty late. = Cuánto la titularía de necesario? Which planet are we on really. = por favor pitarán? What's his initials on the computer? = ´Suno de viendo? Don't move while talking, I'm touching the board. = Alles las últimas lugar que puede tempedo sobre ara. = ¿Basta, ahora que te acermuerta? What are you doing among the flowers? = Le tiempo las las lugar la vianja. Repeat after me! =   ¿Por favor Bono ?   What song is that? = ¿Ahora que te totally lugar ? Head Big be cool face. Shoes have a tab, tie them easily.  Porn star Gary Belcher told Gabriel García Márquez to write not one but two books on acting.  He was very very and worrily disappointed when the dialog became all too predictable as too many would-be movie sleuths read čahiras. Music singer Eric Clapton told Henry Kissinger that having a down-going Daughter is not in the friends' water supply.\n",
            "So there you have it, people. My final cup of coffee.  The keynote address is delivered by Juan Arribas Paré, better known as El Narco, lover of Atwood and Gabriel's painting.  Did I miss something?  Let me know! Who art in the background... Rob and Gabriel daggers in the bar and David reflecting Grace Liebman asks her if it might matter a twinkle in his David's eye because she is looking straight at him. This is kind of an approach... Don't ask me if mine suits me! Agustín Otero and Paul Turner are in-depth in material regarding Gabriel's Joan of Arc trilogy.  But seriously - my coffee drink is rowdy and Jeremy Beauchamp annoys me.  O&A play Katie Haden does an up-up of no threat because David succeeded. The panel invites\n",
            "======================================== SAMPLE 3 ========================================\n",
            "ÂSí\n",
            "\n",
            "Comicplays.com\n",
            "\n",
            "http://comicplays.com/r/menocipmanuals/male/textarea/other/comicplay_4.htm/ 1 title.1 title.2 titles.3 title.4 title.5 title.6\n",
            "\n",
            "Comile International. All text. (English only). Rates may apply.\n",
            "\n",
            "Comolar, M.C., Ms. Schliff, A., & White, J. (2005). Market access numbers of characters, fonts, and scripts for output by text editors (Win64, MS-DOS 2.0). Retrieved from ... . Retrieved from https://softpedia.com/market_access/media/commercialitorome Grumpig Number of characters 3 type of dialogue -Phrase/Title Format 5 one ok? 2 like? 1 no? 1 well? 1 say no? 1 pie? 1 male? 1 inactive? 1 me? 1 overall? LCAS names Caterpillar city Maid job Wife job Fire folks Great house Perigan estate Pit Village hut Ram site Warehouse Multiple events Rarer event\n",
            "\n",
            "4 consecutive 12+ second runs\n",
            "\n",
            "Counts with one day intervals 1 month 8 weeks 4 month 3 pentes 4 sailtues 9 pecunas 28 deces 11 100 sea 16 pasonde 30 papazas 2 prips 9 deusantes 49 pelegues 94 otrigias 43 osojos\n",
            "\n",
            "Text box/box-handles.\n",
            "\n",
            "Also available in Comicmotion/.\n",
            "\n",
            "Comic Americana 43-44 18 vols. 1950\n",
            "\n",
            "1951\n",
            "\n",
            "1958667 1979 Martinez Kin Fast brown sweon Basic gentle Greek ◼‎'🐶ɐɍɑ94' pixels 4 1993 Ohio always/mostly white orbiting Carry (anti) Julius Diego Tokyo pref seven oze Whycorde Advanced serial radio draw ground/air Yes Cevreski/Trevino or Swing\n",
            "\n",
            "FR 4 vols. 1980 Christmas\n",
            "\n",
            "1981 Holiday\n",
            "\n",
            "1992\n",
            "\n",
            "1994 NATGROUP local official Picasso anunta dress German Eriesin Lived LoOSE (8 volumes) state rival O.K. Camille French Homme Katzun oSMCD virtos American Amazing proсotto Strongcast - Daily Crossword Invention Board games Best post out Rocky Blue Lil Alan fifth artist Chris Jack Kirby 24 hand vol. 48 rollas 28 V P J, A 5 of 6 Variant Cape Cod newspaper fox\n",
            "================================================================================\n",
            "Model prompt >>> ======================================== SAMPLE 1 ========================================\n",
            "Âme italiano ín hace de treasure AltStar collegeYa. Who is Dan Johnson? = Da dare you do the math. Who is Monica Velez? = Fou yo unilateral el santa Clara   daysnull tend� https://twitter.com/chompmolica was already my favorite topic about about her when Dan Jr. decided to take a shot at her. In the review he ones the reasons any readers notice Panama to exist while narrowaging lying viewsful incompetency. -When asked about her name she is pretty funny, so when she claimed that the name belied her differences she claimed to be a guy. Who says you cannot diss her if it makes you look inclusive to you? If you and my friend have been friends for a while you probably noticed my immediate contrast with those committees Jenworth Cardoza , Valentina Rodríguez Erdely , Michelle Rodriguez , Belch Bao and Monica Velez Usually incorrect statements made by reviewers are increasingly racist. We tweet freeware and write about gaming together. Something about sexism in the industry and their lack of upstanding antifeminist views won't go unnoticed. There is a HUGE amount of fear propagated by for ourselves and most likely many others on the gaming bubble that must retract because it's plugged into debate dota negative cliquei dubision al treatmentter . Do you say shit at any time? It sounds like trashy leks - puella magire one thing they missed was that 83% of their PS4 preorders were composed of men . There's a subreddit dedicated to lamenting this profiling solely on when we fail, blood libs and femmes, of course doesn't bring free rape porn into the family . Too bad:: https://www.reddit.com/r/DxeVoiceDC/common scapegoat . Also Mark Kern, creator of African payphones makes things worse by heavily censoring meta criticism , how much did black % attendees say he doesn't have a job? And send 100,000 emails promoting their meme poll making them the victim viewpoint of their peers? Do Any N people hate me, because I wont chime in on their counterpoint. Instead let's misspeak and shitpost and pander. The fact that you hired I am tortured in court makes me cut my Twitter and abandon contractually obligated claims of submission in the future. Let me think of another photo of Scott with a crystal couple the upcoming! It seems to me you missed simple descriptive and high\n",
            "======================================== SAMPLE 2 ========================================\n",
            "!!! Yes!!! = !!! Yes!! = !!! Yes!! (yes) = !!! Yes!!! = !!! Yes!!! (yes) !!!! NOT YET!!!! : /~~~ m'era mezcla! : /: klo je meh truck mando panto sé compero: ^-^ ::tuxedo::: nó.o á sucedido: = Las honra bien hecho de los pozas español, es lo hombre y commien en Vivo. INCOMING!: Punia. = What? What did you say? = Horas dato/mo. si muerto. = Si mi pitcher es personas, Jr. / te nica.^^: Yes. No. = No jarante.= Departamento para. = To stay here. = Thank you enough, Mr. It's just raining snack! = 초 dos ni te buenos. = I would like a plastic wrap instead <I dare you> nada, que se haga (doh).^^: Yes. Hi, huh? ; /! type type_/ = ! : /: : /: : /joo: type_/ ; excellent ra__ = : gm: rofl^ < God damn it, < God damn it< :\\, \\:\\ :___/: No!!! No!!! No!!! No!!! Say it again someday, just to back something up: aldrea= Je on ava mesmerca. : My bro, Don Van Naughty is prettier than you. = Lazy dear je garrajera, Dise con por favor, en win de muy cara = I hope you're happy :) being bae because things are fine = I hope the bane can be defeated! Have a good night, son. = Anche el mambre. ::No, not sorry. :/u don't even? -> : /!bingbercue::: awkwardly/ : Endermust be , get dropped by someone - luck Because wussy is tough ; friends gonna die because everyone is hassled ; also because poop is bad ; but actually may hasbeengreenym being any date of the day ; no insert some incantation; Went to arrange workout because the worst another student did was read past one - probably used weed or something etc ; your dick also looks like Jax library ;) school help f\n",
            "======================================== SAMPLE 3 ========================================\n",
            "ño un umens militante a al mundo como 21? = ¿Cómo soy una barrigueña!!! Can I speak Spanish? = ¿Cómo meses crear el pur butido? Can I help you? = Su brilido, estos verdes vivas? Do you sit with your legs apart? = Acabó la tarde roja que estó está a los la que visite. So where are you from?: hades engiendos na veces. Where have you been?: laboura ground Argentina. Spanish for 'perhaps'. = Qui en você conocida. Spanish on a wide canvas. = venar española de una mexicana. Where are you from? = esperan rear hair. very average, slight sense of direction = este mesgrado o hellai..where are you from?? ga vemos eloquentes confendió la floresta india. I am an eloquent lady, call me ''little land'',I am of Indian faith. my friend told me you are from India because your look says it like that\" drain soldier violent rate swaggering indian american and american female freedom fighters gangs/ gangs/ gangs true americans who rebel against oppressive system. viva la twitter abrasione vane una omnia última melee preference hijabially fluent in english became allies = Bravado ok nach like donos-ARR------------------------Until purely seductive relevancy me gustar este activar en forty-eight anos! \"Gargo! United we stand!\" = Ziel Teilen der Deutschland rennt über Musil! Cura es el bina recibido le su mashup de la ballerina sidereal y real negro gypsy double congy á la gente será distinta colonial French and French Louisiana slave heresy bancho y spicy pania dorantes, con la puerta ita summada. Jamie Oliver and Tim & Eric song collecting worldwide market call-rage<|endoftext|>Has anybody else been ready to enter Tinder with zero kid support and full privacy?\n",
            "\n",
            "OK, what kinda kid friendly app belt do you´ve got crammed into your pockets?\n",
            "\n",
            "They say a bun is worth three crowns. This burger burger is worth THREE IQ point isted according to this message from a distracted Young Reader, while listening to Ringo\n",
            "================================================================================\n",
            "Model prompt >>> Traceback (most recent call last):\n",
            "  File \"/usr/lib/python3.6/contextlib.py\", line 99, in __exit__\n",
            "    self.gen.throw(type, value, traceback)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py\", line 5253, in get_controller\n",
            "    yield g\n",
            "  File \"src/interactive_conditional_samples.py\", line 72, in interact_model\n",
            "    raw_text = input(\"Model prompt >>> \")\n",
            "KeyboardInterrupt\n",
            "\n",
            "During handling of the above exception, another exception occurred:\n",
            "\n",
            "Traceback (most recent call last):\n",
            "  File \"src/interactive_conditional_samples.py\", line 90, in <module>\n",
            "    fire.Fire(interact_model)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/fire/core.py\", line 127, in Fire\n",
            "    component_trace = _Fire(component, args, context, name)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/fire/core.py\", line 366, in _Fire\n",
            "    component, remaining_args)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/fire/core.py\", line 542, in _CallCallable\n",
            "    result = fn(*varargs, **kwargs)\n",
            "  File \"src/interactive_conditional_samples.py\", line 87, in interact_model\n",
            "    print(\"=\" * 80)\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py\", line 1592, in __exit__\n",
            "    self.close()\n",
            "  File \"/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py\", line 728, in close\n",
            "    tf_session.TF_CloseSession(self._session)\n",
            "KeyboardInterrupt\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JdwAXFicwMI1"
      },
      "source": [
        ""
      ]
    }
  ]
}