tensor_tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "What is PyTorch?\n",
    "================\n",
    "\n",
    "It’s a Python-based scientific computing package targeted at two sets of\n",
    "audiences:\n",
    "\n",
    "-  A replacement for NumPy to use the power of GPUs\n",
    "-  a deep learning research platform that provides maximum flexibility\n",
    "   and speed\n",
    "\n",
    "Getting Started\n",
    "---------------\n",
    "\n",
    "\n",
    "Tensors are similar to NumPy’s ndarrays, with the addition being that\n",
    "Tensors can also be used on a GPU to accelerate computing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "from __future__ import print_function\n",
    "import torch\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pay attention that there is slight difference in syntax between NumPy and PyTorch.\n",
    "\n",
    "When it comes to initializing tensor/ndarray, the biggest difference is that\n",
    "\n",
    "- Most NumPy methods takes a tuple as the argument\n",
    "- PyTorch takes an unpacked tuple as the argument\n",
    "\n",
    "Observe the difference in the examples below:\n",
    "\n",
    "Construct a 5x3 matrix, uninitialized:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[ 0.0000e+00,  2.0000e+00,  2.1007e-22],\n",
      "        [-1.5849e+29,  4.0927e-11,  7.1450e+31],\n",
      "        [ 4.1418e-41,  0.0000e+00,  2.1007e-22],\n",
      "        [-8.5920e+09,  2.7474e-22, -1.5849e+29],\n",
      "        [ 2.8026e-45,  0.0000e+00,  2.1007e-22]])\n"
     ]
    }
   ],
   "source": [
    "x = torch.empty(5, 3)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-1.04336503,  0.29767695,  0.45528051],\n",
       "       [-0.48804965,  1.51569486, -1.92924095],\n",
       "       [-1.18081524, -1.76143773,  1.26234469],\n",
       "       [ 0.46067763, -1.09306842,  1.57190373],\n",
       "       [ 1.60693347, -0.48680671, -1.00447347]])"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.empty((5, 3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Construct a randomly initialized matrix with each element from a uniform distribution\n",
    "on the interval $[0, 1)$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[0.3321, 0.0214, 0.2379],\n",
      "        [0.5233, 0.0557, 0.3155],\n",
      "        [0.6148, 0.3346, 0.3737],\n",
      "        [0.4859, 0.4904, 0.8262],\n",
      "        [0.3149, 0.4305, 0.5864]])\n"
     ]
    }
   ],
   "source": [
    "x = torch.rand(5, 3)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Its most commonly used counterpart in NumPy is"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.35745212, 0.97718722, 0.35827868],\n",
       "       [0.74037879, 0.58114691, 0.49800819],\n",
       "       [0.72598449, 0.29514081, 0.19521257],\n",
       "       [0.54928349, 0.14095439, 0.77200708],\n",
       "       [0.98121694, 0.48638789, 0.17133006]])"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.random.rand(5, 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the official NumPy states that `np.random.rand` \"is a convenience function. If you want an interface that\n",
    "takes a shape-tuple as the first argument, refer to `np.random.random_sample`.\"\n",
    "\n",
    "Most NumPy methods now take a shape-tuple as the first argument. The reason for the discrepancy in `np.random.rand`'s API could be of historical origin. Some people believe that it is to make the API more familiar to Matlab users, while others believe it is a result of merging different libraries of different syntaxes when creating NumPy.\n",
    "\n",
    "For a more consistent API, it is recommended to use `np.random.random` instead of `np.random.rand`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.1627072 , 0.05863506, 0.0599153 ],\n",
       "       [0.45304188, 0.85003114, 0.83917487],\n",
       "       [0.17636866, 0.47912759, 0.21039306],\n",
       "       [0.36664584, 0.45587276, 0.01978475],\n",
       "       [0.10482004, 0.53958522, 0.19078593]])"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.random.random((5, 3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Construct a matrix filled zeros and of dtype long:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[0, 0, 0],\n",
      "        [0, 0, 0],\n",
      "        [0, 0, 0],\n",
      "        [0, 0, 0],\n",
      "        [0, 0, 0]])\n"
     ]
    }
   ],
   "source": [
    "x = torch.zeros(5, 3, dtype=torch.long)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0, 0, 0],\n",
       "       [0, 0, 0],\n",
       "       [0, 0, 0],\n",
       "       [0, 0, 0],\n",
       "       [0, 0, 0]])"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.zeros((5, 3), dtype=np.long)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Construct a tensor directly from data:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([5.5000, 3.0000])\n"
     ]
    }
   ],
   "source": [
    "x = torch.tensor([5.5, 3])\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "array([5.5, 3. ])\n"
     ]
    }
   ],
   "source": [
    "a = np.array([5.5, 3])\n",
    "print(repr(a))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "or create a tensor based on an existing tensor. These methods\n",
    "will reuse properties of the input tensor, e.g. dtype, unless\n",
    "new values are provided by user\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1., 1., 1.],\n",
      "        [1., 1., 1.],\n",
      "        [1., 1., 1.],\n",
      "        [1., 1., 1.],\n",
      "        [1., 1., 1.]], dtype=torch.float64)\n",
      "tensor([[-0.7226,  0.7050, -0.1223],\n",
      "        [-0.0823,  0.1507, -1.6925],\n",
      "        [ 0.7180, -1.0116,  0.3251],\n",
      "        [ 2.5652,  0.1425,  1.0602],\n",
      "        [-0.7888,  0.7676, -1.6610]])\n"
     ]
    }
   ],
   "source": [
    "x = x.new_ones(5, 3, dtype=torch.double)      # new_* methods take in sizes\n",
    "print(x)\n",
    "\n",
    "x = torch.randn_like(x, dtype=torch.float)    # override dtype!\n",
    "print(x)                                      # result has the same size"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Due to my limited knowledge, I could not find their direct counterparts in NumPy. Here are the closest ones I can think of"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "array([[1., 1., 1.],\n",
      "       [1., 1., 1.],\n",
      "       [1., 1., 1.],\n",
      "       [1., 1., 1.],\n",
      "       [1., 1., 1.]])\n",
      "array([[ 1.2226522 ,  0.4654828 , -1.52909148],\n",
      "       [ 1.07474797, -0.97334343,  1.38093876],\n",
      "       [-0.96374385,  0.36087359, -0.11050262],\n",
      "       [-0.1996    ,  0.54720236, -0.20798344],\n",
      "       [-1.23443732,  0.13209339, -2.01319573]])\n"
     ]
    }
   ],
   "source": [
    "a = np.ones((5, 3), dtype=np.double)\n",
    "print(repr(a))\n",
    "\n",
    "a = np.random.standard_normal(y.shape).astype(np.float)\n",
    "print(repr(a))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get its size:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([5, 3])\n"
     ]
    }
   ],
   "source": [
    "print(x.size())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(5, 3)"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\"><h4>Note</h4><p>``torch.Size`` is in fact a tuple, so it supports all tuple operations.</p></div>\n",
    "\n",
    "Operations\n",
    "^^^^^^^^^^\n",
    "There are multiple syntaxes for operations. In the following\n",
    "example, we will take a look at the addition operation.\n",
    "\n",
    "Addition: syntax 1\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-0.1143,  1.5666,  0.1114],\n",
      "        [ 0.1479,  0.8518, -0.9449],\n",
      "        [ 1.6502, -0.9304,  1.1491],\n",
      "        [ 2.7745,  0.2022,  1.3719],\n",
      "        [-0.7872,  1.4197, -0.9144]])\n"
     ]
    }
   ],
   "source": [
    "y = torch.rand(5, 3)\n",
    "print(x + y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.94985743,  0.95451481, -0.73717511],\n",
       "       [ 1.5532944 , -0.1132972 ,  1.39630206],\n",
       "       [-0.45545447,  0.93046737,  0.07407713],\n",
       "       [-0.09962069,  0.65958183,  0.76538712],\n",
       "       [-1.14032461,  0.31929953, -1.62938092]])"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "b = np.random.random((5, 3))\n",
    "a + b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Addition: syntax 2\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-0.1143,  1.5666,  0.1114],\n",
      "        [ 0.1479,  0.8518, -0.9449],\n",
      "        [ 1.6502, -0.9304,  1.1491],\n",
      "        [ 2.7745,  0.2022,  1.3719],\n",
      "        [-0.7872,  1.4197, -0.9144]])\n"
     ]
    }
   ],
   "source": [
    "print(torch.add(x, y))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.94985743,  0.95451481, -0.73717511],\n",
       "       [ 1.5532944 , -0.1132972 ,  1.39630206],\n",
       "       [-0.45545447,  0.93046737,  0.07407713],\n",
       "       [-0.09962069,  0.65958183,  0.76538712],\n",
       "       [-1.14032461,  0.31929953, -1.62938092]])"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.add(a, b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Addition: providing an output tensor as argument\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-0.1143,  1.5666,  0.1114],\n",
      "        [ 0.1479,  0.8518, -0.9449],\n",
      "        [ 1.6502, -0.9304,  1.1491],\n",
      "        [ 2.7745,  0.2022,  1.3719],\n",
      "        [-0.7872,  1.4197, -0.9144]])\n"
     ]
    }
   ],
   "source": [
    "result = torch.empty(5, 3)\n",
    "torch.add(x, y, out=result)\n",
    "print(result)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [],
   "source": [
    "result_2 = np.empty((5, 3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.94985743,  0.95451481, -0.73717511],\n",
       "       [ 1.5532944 , -0.1132972 ,  1.39630206],\n",
       "       [-0.45545447,  0.93046737,  0.07407713],\n",
       "       [-0.09962069,  0.65958183,  0.76538712],\n",
       "       [-1.14032461,  0.31929953, -1.62938092]])"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.add(a, b, out=result_2)\n",
    "result_2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Addition: in-place\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-0.1143,  1.5666,  0.1114],\n",
      "        [ 0.1479,  0.8518, -0.9449],\n",
      "        [ 1.6502, -0.9304,  1.1491],\n",
      "        [ 2.7745,  0.2022,  1.3719],\n",
      "        [-0.7872,  1.4197, -0.9144]])\n"
     ]
    }
   ],
   "source": [
    "# adds x to y\n",
    "y.add_(x)\n",
    "print(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 3.17250963,  1.41999761, -2.26626659],\n",
       "       [ 2.62804237, -1.08664063,  2.77724082],\n",
       "       [-1.41919832,  1.29134097, -0.03642549],\n",
       "       [-0.29922069,  1.2067842 ,  0.55740367],\n",
       "       [-2.37476193,  0.45139292, -3.64257665]])"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.add(a, b, out=b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 3.17250963,  1.41999761, -2.26626659],\n",
       "       [ 2.62804237, -1.08664063,  2.77724082],\n",
       "       [-1.41919832,  1.29134097, -0.03642549],\n",
       "       [-0.29922069,  1.2067842 ,  0.55740367],\n",
       "       [-2.37476193,  0.45139292, -3.64257665]])"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\"><h4>Note</h4><p>Any operation that mutates a tensor in-place is post-fixed with an ``_``.\n",
    "    For example: ``x.copy_(y)``, ``x.t_()``, will change ``x``.</p></div>\n",
    "\n",
    "You can use standard NumPy-like indexing with all bells and whistles!\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([ 0.7050,  0.1507, -1.0116,  0.1425,  0.7676])\n"
     ]
    }
   ],
   "source": [
    "print(x[:, 1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.4654828 , -0.97334343,  0.36087359,  0.54720236,  0.13209339])"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a[:, 1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Resizing: If you want to resize/reshape tensor, you can use ``torch.view``.\n",
    "\n",
    "WARNING: Be very careful, NumPy users! Here `torch.view` is essentially `np.reshape` and is completely different from `np.view`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([4, 4]) torch.Size([16]) torch.Size([2, 8])\n"
     ]
    }
   ],
   "source": [
    "x = torch.randn(4, 4)\n",
    "y = x.view(16)\n",
    "z = x.view(-1, 8)  # the size -1 is inferred from other dimensions\n",
    "print(x.size(), y.size(), z.size())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(4, 4) (16,) (2, 8)\n"
     ]
    }
   ],
   "source": [
    "a = np.random.standard_normal((4, 4))\n",
    "b = a.reshape(16)\n",
    "c = a.reshape(-1, 8)\n",
    "print(a.shape, b.shape, c.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you have a one element tensor, use ``.item()`` to get the value as a\n",
    "Python number\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([0.3098])\n",
      "0.3097800612449646\n"
     ]
    }
   ],
   "source": [
    "x = torch.randn(1)\n",
    "print(x)\n",
    "print(x.item())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[-0.71335854]\n",
      "-0.7133585385709382\n"
     ]
    }
   ],
   "source": [
    "a = np.random.standard_normal(1)\n",
    "print(a)\n",
    "print(a.item())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Read later:**\n",
    "\n",
    "\n",
    "  100+ Tensor operations, including transposing, indexing, slicing,\n",
    "  mathematical operations, linear algebra, random numbers, etc.,\n",
    "  are described\n",
    "  `here <https://pytorch.org/docs/torch>`_.\n",
    "\n",
    "NumPy Bridge\n",
    "------------\n",
    "\n",
    "Converting a Torch Tensor to a NumPy array and vice versa is a breeze.\n",
    "\n",
    "The Torch Tensor and NumPy array will share their underlying memory\n",
    "locations, and changing one will change the other.\n",
    "\n",
    "Converting a Torch Tensor to a NumPy Array\n",
    "^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([1., 1., 1., 1., 1.])\n"
     ]
    }
   ],
   "source": [
    "a = torch.ones(5)\n",
    "print(a)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1. 1. 1. 1. 1.]\n"
     ]
    }
   ],
   "source": [
    "b = a.numpy()\n",
    "print(b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "See how the numpy array changed in value.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([2., 2., 2., 2., 2.])\n",
      "[2. 2. 2. 2. 2.]\n"
     ]
    }
   ],
   "source": [
    "a.add_(1)\n",
    "print(a)\n",
    "print(b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Converting NumPy Array to Torch Tensor\n",
    "^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
    "See how changing the np array changed the Torch Tensor automatically\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2. 2. 2. 2. 2.]\n",
      "tensor([2., 2., 2., 2., 2.], dtype=torch.float64)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "a = np.ones(5)\n",
    "b = torch.from_numpy(a)\n",
    "np.add(a, 1, out=a)\n",
    "print(a)\n",
    "print(b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All the Tensors on the CPU except a CharTensor support converting to\n",
    "NumPy and back.\n",
    "\n",
    "CUDA Tensors\n",
    "------------\n",
    "\n",
    "Tensors can be moved onto any device using the ``.to`` method.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [],
   "source": [
    "# let us run this cell only if CUDA is available\n",
    "# We will use ``torch.device`` objects to move tensors in and out of GPU\n",
    "if torch.cuda.is_available():\n",
    "    device = torch.device(\"cuda\")          # a CUDA device object\n",
    "    y = torch.ones_like(x, device=device)  # directly create a tensor on GPU\n",
    "    x = x.to(device)                       # or just use strings ``.to(\"cuda\")``\n",
    "    z = x + y\n",
    "    print(z)\n",
    "    print(z.to(\"cpu\", torch.double))       # ``.to`` can also change dtype together!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}