Improve documentation

Author: NielsRogge

GPT-J-6B/Inference_with_GPT_J_6B.ipynb

@@ -7,7 +7,7 @@
       "provenance": [],
       "collapsed_sections": [],
       "machine_shape": "hm",
-      "authorship_tag": "ABX9TyP9nxnN9wm2/27BTNCCEz3j",
+      "authorship_tag": "ABX9TyN1uhCmEDAoq5zrKwioAmfi",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -2778,11 +2778,9 @@
       "source": [
         "## Inference with GPT-J-6B\n",
         "\n",
-        "In this notebook, we are going to perform inference (i.e. generating new text) with EleutherAI's [GPT-J-6B model](https://github.com/kingoflolz/mesh-transformer-jax/), which is a 6 billion parameter GPT model trained on [The Pile](https://arxiv.org/abs/2101.00027), a huge publicly available text dataset, also collected by EleutherAI. The model itself was trained on TPUv3s using JAX and Haiku (the latter being a neural net library on top of JAX).\n",
+        "In this notebook, we are going to perform inference (i.e. generate new text) with EleutherAI's [GPT-J-6B model](https://github.com/kingoflolz/mesh-transformer-jax/), which is a 6 billion parameter GPT model trained on [The Pile](https://arxiv.org/abs/2101.00027), a huge publicly available text dataset, also collected by EleutherAI. The model itself was trained on TPUv3s using JAX and Haiku (the latter being a neural net library on top of JAX).\n",
         "\n",
-        "EleutherAI itself is a group of AI researchers doing awesome AI research (and making everything publicly available and free to use). You can check out their website [here](https://www.eleuther.ai/).\n",
-        "\n",
-        "EleutherAI also created [GPT-Neo](), which are smaller GPT variants (with 125 million, 1.3 billion and 2.7 billion parameters respectively). Check out their models on the hub [here](https://huggingface.co/EleutherAI).\n",
+        "[EleutherAI](https://www.eleuther.ai/) itself is a group of AI researchers doing awesome AI research (and making everything publicly available and free to use). They've also created [GPT-Neo](https://github.com/EleutherAI/gpt-neo), which are smaller GPT variants (with 125 million, 1.3 billion and 2.7 billion parameters respectively). Check out their models on the hub [here](https://huggingface.co/EleutherAI).\n",
         "\n",
         "NOTE: this notebook requires at least 12.1GB of CPU memory. I'm personally using Colab Pro to run it (and set runtime to GPU - high RAM usage). Unfortunately, the free version of Colab only provides 10 GB of RAM, which isn't enough.\n",
         "\n",
@@ -2829,7 +2827,7 @@
       "source": [
         "## Load model and tokenizer\n",
         "\n",
-        "First, we load the model from the hub. We select the \"float16\" revision, which means that all parameters are stored using 16 bits, rather than the default float32 ones (which require more RAM memory). We also set `low_cpu_mem_usage` to `True` (which was introduced in [this PR](https://github.com/huggingface/transformers/pull/13466)), in order to only load the model once into CPU memory.\n",
+        "First, we load the model from the hub. We select the \"float16\" revision, which means that all parameters are stored using 16 bits, rather than the default float32 ones (which require twice as much RAM memory). We also set `low_cpu_mem_usage` to `True` (which was introduced in [this PR](https://github.com/huggingface/transformers/pull/13466)), in order to only load the model once into CPU memory.\n",
         "\n",
         "Next, we move the model to the GPU and load the corresponding tokenizer, which we'll use to prepare text for the model."
       ]