02-torch-basics.ipynb

{
 "metadata": {
  "language": "lua",
  "name": "",
  "signature": "sha256:b73308a9bacbac952ab673422b9ef3da032ac91d83c57b2b10d110e677314dfc"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Tensors - nDimensional arrays"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "require 'torch'; -- includes torch"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 5
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "x = torch.Tensor() -- default tensor. type == double\n",
      "print(torch.type(x))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 6,
       "text": [
        "torch.DoubleTensor\t\n"
       ]
      }
     ],
     "prompt_number": 6
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "-- change the default tensor type\n",
      "torch.setdefaulttensortype('torch.FloatTensor')\n",
      "x = torch.Tensor()\n",
      "print(torch.type(x))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 7,
       "text": [
        "torch.FloatTensor\t\n"
       ]
      }
     ],
     "prompt_number": 7
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "-- other tensor types\n",
      "x = torch.ByteTensor()\n",
      "x = torch.CharTensor()\n",
      "x = torch.IntTensor()\n",
      "x = torch.LongTensor()\n",
      "x = torch.ShortTensor()\n",
      "-- cuda types\n",
      "require 'cutorch' -- imports cuda tensor types\n",
      "x = torch.CudaTensor()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 8
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Tensors: Basic Operations"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "A Tensor is a potentially multi-dimensional matrix. The number of dimensions is unlimited that can be created using LongStorage with more dimensions."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      " --- creation of a 4D-tensor 4x5x6x2\n",
      " z = torch.Tensor(4,5,6,2)\n",
      " --- for more dimensions, (here a 6D tensor) one can do:\n",
      " s = torch.LongStorage(6)\n",
      " s[1] = 4; s[2] = 5; s[3] = 6; s[4] = 2; s[5] = 7; s[6] = 3;\n",
      " x = torch.Tensor(s)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 9
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print(x:nDimension()) -- get number of dimensions"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 10,
       "text": [
        "6\t\n"
       ]
      }
     ],
     "prompt_number": 10
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print(x:size()) -- get size of tensor"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 11,
       "text": [
        "\n",
        " 4\n",
        " 5\n",
        " 6\n",
        " 2\n",
        " 7\n",
        " 3\n",
        "[torch.LongStorage of size 6]\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 11
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The actual data of a `Tensor` is contained into a\n",
      "Storage. It can be accessed using\n",
      "`storage()`. While the memory of a\n",
      "`Tensor` has to be contained in this unique `Storage`, it might\n",
      "not be contiguous: the first position used in the `Storage` is given\n",
      "by `storageOffset()` (starting at\n",
      "`1`). And the _jump_ needed to go from one element to another\n",
      "element in the `i-th` dimension is given by\n",
      "`stride(i)`. In other words, given a 3D\n",
      "tensor"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "x = torch.Tensor(7,7,7)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 12
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "-- accessing the element (3,4,5) can be done by\n",
      "x[3][4][5]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 13,
       "text": [
        "0\t\n"
       ]
      }
     ],
     "prompt_number": 13
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "-- Under the hood, this is what Torch does\n",
      "x:storage()[x:storageOffset()\n",
      "           +(3-1)*x:stride(1)+(4-1)*\n",
      "            x:stride(2)+(5-1)*x:stride(3)]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 14,
       "text": [
        "0\t\n"
       ]
      }
     ],
     "prompt_number": 14
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "One could say that a `Tensor` is a particular way of _viewing_ a\n",
      "`Storage`: a `Storage` only represents a chunk of memory, while the\n",
      "`Tensor` interprets this chunk of memory as having dimensions:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "x = torch.Tensor(4,5)\n",
      "s = x:storage()\n",
      "for i=1,s:size() do -- fill up the Storage\n",
      "    s[i] = i\n",
      "end\n",
      "print(x) -- s is interpreted by x as a 2D matrix"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 15,
       "text": [
        "  1   2   3   4   5\n",
        "  6   7   8   9  10\n",
        " 11  12  13  14  15\n",
        " 16  17  18  19  20\n",
        "[torch.FloatTensor of dimension 4x5]\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 15
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "###Tensors: Important Semantics"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "x = torch.Tensor(5) -- tensor initialized to random memory.\n",
      "x:fill(3) -- fill x with value 3\n",
      "print(x)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 18,
       "text": [
        " 3\n",
        " 3\n",
        " 3\n",
        " 3\n",
        " 3\n",
        "[torch.FloatTensor of dimension 5]\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 18
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "y = torch.Tensor(5,1):normal() -- initialize a tensor, fill it with normally distributed values\n",
      "print(y)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 20,
       "text": [
        " 0.1330\n",
        "-0.4238\n",
        "-0.0055\n",
        "-0.9527\n",
        " 0.2165\n",
        "[torch.FloatTensor of dimension 5x1]\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 20
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "z = torch.add(y,x) -- adds to a new allocated tensor\n",
      "print(z)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 25,
       "text": [
        " 3.1330\n",
        " 2.5762\n",
        " 2.9945\n",
        " 2.0473\n",
        " 3.2165\n",
        "[torch.FloatTensor of dimension 5x1]\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 25
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "z:add(x) -- adds x to z in-place\n",
      "print(z)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 26,
       "text": [
        " 6.1330\n",
        " 5.5762\n",
        " 5.9945\n",
        " 5.0473\n",
        " 6.2165\n",
        "[torch.FloatTensor of dimension 5x1]\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 26
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "-- typecasting\n",
      "z:int() -- byte(), char(), short(), int(), long(), float(), double(), cuda()\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 29,
       "text": [
        " 6\n",
        " 5\n",
        " 5\n",
        " 5\n",
        " 6\n",
        "[torch.IntTensor of dimension 5x1]\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 29
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "-- same as\n",
      "z:type('torch.IntTensor')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 30,
       "text": [
        " 6\n",
        " 5\n",
        " 5\n",
        " 5\n",
        " 6\n",
        "[torch.IntTensor of dimension 5x1]\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 30
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "x = z:clone() -- creates a new tensor with **different** storage"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 33
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "z = torch.randn(3,4)\n",
      "x = z:view(12) -- creates a view on the same storage. changing z will change x"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 34
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "z"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 35,
       "text": [
        "-0.4168  0.2639  1.8409 -2.3122\n",
        "-0.4659 -1.8936  0.7242 -0.4124\n",
        " 1.5417  0.3719  0.8661 -0.0843\n",
        "[torch.FloatTensor of dimension 3x4]\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 35
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "x"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 36,
       "text": [
        "-0.4168\n",
        " 0.2639\n",
        " 1.8409\n",
        "-2.3122\n",
        "-0.4659\n",
        "-1.8936\n",
        " 0.7242\n",
        "-0.4124\n",
        " 1.5417\n",
        " 0.3719\n",
        " 0.8661\n",
        "-0.0843\n",
        "[torch.FloatTensor of dimension 12]\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 36
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Tensors: Indexing and extracting subtensors"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "x = torch.Tensor(5, 6):zero()\n",
      "print(x)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 37,
       "text": [
        "0 0 0 0 0 0\n",
        "0 0 0 0 0 0\n",
        "0 0 0 0 0 0\n",
        "0 0 0 0 0 0\n",
        "0 0 0 0 0 0\n",
        "[torch.FloatTensor of dimension 5x6]\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 37
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "x[{ 1,3 }] = 1 -- sets element at (i=1,j=3) to 1\n",
      "print(x)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 39,
       "text": [
        " 0  0  1  0  0  0\n",
        " 0  0  0  0  0  0\n",
        " 0  0  0  0  0  0\n",
        " 0  0  0  0  0  0\n",
        " 0  0  0  0  0  0\n",
        "[torch.FloatTensor of dimension 5x6]\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 39
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "x[{ 2,{2,4} }] = 2  -- sets a slice of 3 elements to 2\n",
      "print(x)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 40,
       "text": [
        " 0  0  1  0  0  0\n",
        " 0  2  2  2  0  0\n",
        " 0  0  0  0  0  0\n",
        " 0  0  0  0  0  0\n",
        " 0  0  0  0  0  0\n",
        "[torch.FloatTensor of dimension 5x6]\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 40
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "x[{ {},4 }] = -1 -- sets the full 4th column to -1\n",
      "print(x)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 41,
       "text": [
        " 0  0  1 -1  0  0\n",
        " 0  2  2 -1  0  0\n",
        " 0  0  0 -1  0  0\n",
        " 0  0  0 -1  0  0\n",
        " 0  0  0 -1  0  0\n",
        "[torch.FloatTensor of dimension 5x6]\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 41
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "x[{ {},2 }] = torch.range(1,5) -- copy a 1D tensor to a slice of x\n",
      "print(x)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 42,
       "text": [
        " 0  1  1 -1  0  0\n",
        " 0  2  2 -1  0  0\n",
        " 0  3  0 -1  0  0\n",
        " 0  4  0 -1  0  0\n",
        " 0  5  0 -1  0  0\n",
        "[torch.FloatTensor of dimension 5x6]\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 42
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "x[torch.lt(x,0)] = -2 -- sets all negative elements to -2 via a mask\n",
      "print(x)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 43,
       "text": [
        " 0  1  1 -2  0  0\n",
        " 0  2  2 -2  0  0\n",
        " 0  3  0 -2  0  0\n",
        " 0  4  0 -2  0  0\n",
        " 0  5  0 -2  0  0\n",
        "[torch.FloatTensor of dimension 5x6]\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 43
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [],
     "language": "python",
     "metadata": {},
     "outputs": []
    }
   ],
   "metadata": {}
  }
 ]
}