Section 4 - Encrypted Deep Learning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Section: Encrypted Deep Learning\n",
    "\n",
    "- Lesson: Reviewing Additive Secret Sharing\n",
    "- Lesson: Encrypted Subtraction and Public/Scalar Multiplication\n",
    "- Lesson: Encrypted Computation in PySyft\n",
    "- Project: Build an Encrypted Database\n",
    "- Lesson: Encrypted Deep Learning in PyTorch\n",
    "- Lesson: Encrypted Deep Learning in Keras\n",
    "- Final Project"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lesson: Reviewing Additive Secret Sharing\n",
    "\n",
    "_For more great information about SMPC protocols like this one, visit https://mortendahl.github.io. With permission, Morten's work directly inspired this first teaching segment._"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "import numpy as np\n",
    "\n",
    "BASE = 10\n",
    "\n",
    "PRECISION_INTEGRAL = 8\n",
    "PRECISION_FRACTIONAL = 8\n",
    "Q = 293973345475167247070445277780365744413\n",
    "\n",
    "PRECISION = PRECISION_INTEGRAL + PRECISION_FRACTIONAL\n",
    "\n",
    "assert(Q > BASE**PRECISION)\n",
    "\n",
    "def encode(rational):\n",
    "    upscaled = int(rational * BASE**PRECISION_FRACTIONAL)\n",
    "    field_element = upscaled % Q\n",
    "    return field_element\n",
    "\n",
    "def decode(field_element):\n",
    "    upscaled = field_element if field_element <= Q/2 else field_element - Q\n",
    "    rational = upscaled / BASE**PRECISION_FRACTIONAL\n",
    "    return rational\n",
    "\n",
    "def encrypt(secret):\n",
    "    first  = random.randrange(Q)\n",
    "    second = random.randrange(Q)\n",
    "    third  = (secret - first - second) % Q\n",
    "    return [first, second, third]\n",
    "\n",
    "def decrypt(sharing):\n",
    "    return sum(sharing) % Q\n",
    "\n",
    "def add(a, b):\n",
    "    c = list()\n",
    "    for i in range(len(a)):\n",
    "        c.append((a[i] + b[i]) % Q)\n",
    "    return tuple(c)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[57814909702239091760146730655094304275,\n",
       " 281458963716041706884489175429286737768,\n",
       " 248672817532053695496254649476900446783]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = encrypt(encode(5.5))\n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[216865815972631005079158796583135095850,\n",
       " 227586812816162442397075574291474672588,\n",
       " 143494062161541046664656184686351720387]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = encrypt(encode(2.3))\n",
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(274680725674870096839305527238229400125,\n",
       " 215072431057036902211119471940395665943,\n",
       " 98193534218427495090465556382886422757)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z = add(x,y)\n",
    "z"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7.79999999"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "decode(decrypt(z))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lesson: Encrypted Subtraction and Public/Scalar Multiplication"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "field = 23740629843760239486723"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 5\n",
    "\n",
    "bob_x_share = 2372385723 # random number\n",
    "alices_x_share = field - bob_x_share + x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(bob_x_share + alices_x_share) % field"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "field = 10\n",
    "\n",
    "x = 5\n",
    "\n",
    "bob_x_share = 8\n",
    "alice_x_share = field - bob_x_share + x\n",
    "\n",
    "y = 1\n",
    "\n",
    "bob_y_share = 9\n",
    "alice_y_share = field - bob_y_share + y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#subtraction share\n",
    "((bob_x_share + alice_x_share) - (bob_y_share + alice_y_share)) % field"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#addition share\n",
    "((bob_x_share - bob_y_share) + (alice_x_share - alice_y_share)) % field"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "26"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bob_x_share + alice_x_share + bob_y_share + alice_y_share"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "bob_z_share = (bob_x_share - bob_y_share)\n",
    "alice_z_share = (alice_x_share - alice_y_share)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(bob_z_share + alice_z_share) % field"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "#subtraction function\n",
    "def sub(a, b):\n",
    "    c = list()\n",
    "    for i in range(len(a)):\n",
    "        c.append((a[i] - b[i]) % Q)\n",
    "    return tuple(c)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "#cannot represent numbers larger than field\n",
    "field = 10\n",
    "\n",
    "x = 5\n",
    "\n",
    "bob_x_share = 8\n",
    "alice_x_share = field - bob_x_share + x\n",
    "\n",
    "y = 1\n",
    "\n",
    "bob_y_share = 9\n",
    "alice_y_share = field - bob_y_share + y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "15"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bob_x_share + alice_x_share"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "11"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bob_y_share + alice_y_share"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "((bob_y_share * 3) + (alice_y_share * 3)) % field"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "def imul(a, scalar):\n",
    "    \n",
    "    # logic here which can multiply by a public scalar\n",
    "    \n",
    "    c = list()\n",
    "    \n",
    "    for i in range(len(a)):\n",
    "        c.append((a[i] * scalar) % Q)\n",
    "        \n",
    "    return tuple(c)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[182632325473206384811339817958889354679,\n",
       " 163419404693447997760631689085135233601,\n",
       " 241894960783680111568919048517256900546]"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = encrypt(encode(5.5))\n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "z = imul(x, 3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "16.5"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "decode(decrypt(z))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lesson: Encrypted Computation in PySyft"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: Logging before flag parsing goes to stderr.\n",
      "W0902 10:24:35.459000  9740 secure_random.py:26] Falling back to insecure randomness since the required custom op could not be found for the installed version of TensorFlow. Fix this by compiling custom ops. Missing file was 'C:\\Users\\Claudia\\Anaconda3\\lib\\site-packages\\tf_encrypted/operations/secure_random/secure_random_module_tf_1.14.0-rc0.so'\n",
      "W0902 10:24:35.480000  9740 deprecation_wrapper.py:119] From C:\\Users\\Claudia\\Anaconda3\\lib\\site-packages\\tf_encrypted\\session.py:26: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import syft as sy\n",
    "import torch as th\n",
    "hook = sy.TorchHook(th)\n",
    "from torch import nn, optim"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "bob = sy.VirtualWorker(hook, id=\"bob\").add_worker(sy.local_worker)\n",
    "alice = sy.VirtualWorker(hook, id=\"alice\").add_worker(sy.local_worker)\n",
    "secure_worker = sy.VirtualWorker(hook, id=\"secure_worker\").add_worker(sy.local_worker)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = th.tensor([1,2,3,4])\n",
    "y = th.tensor([2,-1,1,0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "# crypto_provider is randomly generated numbers\n",
    "x = x.share(bob, alice, crypto_provider=secure_worker)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = y.share(bob, alice, crypto_provider=secure_worker)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([3, 1, 4, 4])"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z = x + y\n",
    "z.get()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([-1,  3,  2,  4])"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z = x - y\n",
    "z.get()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([ 2, -2,  3,  0])"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z = x * y\n",
    "z.get()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([0, 1, 1, 1])"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#comparison operations\n",
    "z = x > y\n",
    "z.get()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([1, 0, 0, 0])"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z = x < y\n",
    "z.get()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([0, 0, 0, 0])"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z = x == y\n",
    "z.get()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Below - RuntimeError: log2_vml_cpu not implemented for 'Long' - solution cast tensor as float for fix_precision\n",
    "x = th.tensor([1,2,3,4]).float()\n",
    "y = th.tensor([2,-1,1,0]).float()\n",
    "\n",
    "x = x.fix_precision().share(bob, alice, crypto_provider=secure_worker)\n",
    "y = y.fix_precision().share(bob, alice, crypto_provider=secure_worker)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([3., 1., 4., 4.])"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z = x + y\n",
    "z.get().float_precision()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([-1.,  3.,  2.,  4.])"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z = x - y\n",
    "z.get().float_precision()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([ 2., -2.,  3.,  0.])"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z = x * y\n",
    "z.get().float_precision()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([0., 1., 1., 1.])"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z = x > y\n",
    "z.get().float_precision()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([1., 0., 0., 0.])"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z = x < y\n",
    "z.get().float_precision()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([0., 0., 0., 0.])"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z = x == y\n",
    "z.get().float_precision()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Project: Build an Encrypted Database"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "'''\n",
    "Build encrypted database\n",
    "1. string key-value store\n",
    "2. just use tensor operations - addition, subtraction, multiplication comparisons\n",
    "3. encrypt entire database - store encrypted state inside database\n",
    "\n",
    "HINTs\n",
    "store strings as one-hot encodings of characters\n",
    "or as list of character indicies\n",
    "map each character to a specific number\n",
    "store as a list or a vector of those numbers\n",
    "\n",
    "'''"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "W0902 10:25:02.331000  9740 hook.py:98] Torch was already hooked... skipping hooking process\n"
     ]
    }
   ],
   "source": [
    "#numerical representation for strings: \n",
    "#1. one-hot representation or\n",
    "#2. integer representations\n",
    "# where each integer represents different character or word\n",
    "\n",
    "import syft as sy\n",
    "import torch as th\n",
    "hook = sy.TorchHook(th)\n",
    "from torch import nn, optim\n",
    "\n",
    "import numpy as np\n",
    "import random\n",
    "from random import randint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "import string\n",
    "\n",
    "char2index = {}\n",
    "index2char = {}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i, char in enumerate(' ' + string.ascii_lowercase + '0123456789' + string.punctuation):\n",
    "    char2index[char] = i\n",
    "    index2char[i] = char"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "str_input = \"Hello\"\n",
    "max_len = 8"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "#complete functions\n",
    "def string2values(str_input, max_len=8):\n",
    "\n",
    "    str_input = str_input[:max_len].lower()\n",
    "\n",
    "    # pad strings shorter than max len\n",
    "    if(len(str_input) < max_len):\n",
    "        str_input = str_input + \".\" * (max_len - len(str_input))\n",
    "\n",
    "    values = list()\n",
    "    for char in str_input:\n",
    "        values.append(char2index[char])\n",
    "\n",
    "    return th.tensor(values).long()\n",
    "\n",
    "def values2string(input_values):\n",
    "    s = \"\"\n",
    "    for value in input_values:\n",
    "        s += index2char[int(value)]\n",
    "    return s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "def strings_equal(str_a, str_b):\n",
    "   \n",
    "    vect = (str_a * str_b).sum(1)\n",
    "\n",
    "    x = vect[0]\n",
    "\n",
    "    for i in range(vect.shape[0] - 1):\n",
    "        x = x * vect[i + 1]     \n",
    "\n",
    "    return x\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "def one_hot(index, length):\n",
    "    vect = th.zeros(length).long()\n",
    "    vect[index] = 1\n",
    "    return vect\n",
    "\n",
    "def string2one_hot_matrix(str_input, max_len=8):\n",
    "\n",
    "    str_input = str_input[:max_len].lower()\n",
    "\n",
    "    # pad strings shorter than max len\n",
    "    if(len(str_input) < max_len):\n",
    "        str_input = str_input + \".\" * (max_len - len(str_input))\n",
    "\n",
    "    char_vectors = list()\n",
    "    for char in str_input:\n",
    "        char_v = one_hot(char2index[char], len(char2index)).unsqueeze(0)\n",
    "        char_vectors.append(char_v)\n",
    "        \n",
    "    return th.cat(char_vectors, dim=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "class EncryptedDB():\n",
    "    \n",
    "    def __init__(self, *owners, max_key_len=8, max_val_len=8):\n",
    "        self.max_key_len = max_key_len\n",
    "        self.max_val_len = max_val_len\n",
    "        \n",
    "        self.keys = list()\n",
    "        self.values = list()\n",
    "        self.owners = owners\n",
    "        \n",
    "    def add_entry(self, key, value):        \n",
    "        key = string2one_hot_matrix(key)\n",
    "        key = key.share(*self.owners)\n",
    "        self.keys.append(key)\n",
    "        \n",
    "        value = string2values(value, max_len=self.max_val_len)\n",
    "        value = value.share(*self.owners)\n",
    "        self.values.append(value)\n",
    "        \n",
    "    def query(self, query_str):\n",
    "        query_matrix = string2one_hot_matrix(query_str)\n",
    "        \n",
    "        query_matrix = query_matrix.share(*self.owners)\n",
    "\n",
    "        key_matches = list()\n",
    "        for key in self.keys:\n",
    "\n",
    "            key_match = strings_equal(key, query_matrix)\n",
    "            key_matches.append(key_match)\n",
    "\n",
    "        result = self.values[0] * key_matches[0]\n",
    "\n",
    "        for i in range(len(self.values) - 1):\n",
    "            result += self.values[i+1] * key_matches[i+1]\n",
    "            \n",
    "        result = result.get()\n",
    "\n",
    "        return values2string(result).replace(\".\",\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "db = EncryptedDB(bob, alice, secure_worker, max_val_len=256)\n",
    "\n",
    "db.add_entry(\"key1\", \"value1\")\n",
    "db.add_entry(\"key2\", \"value2\")\n",
    "db.add_entry(\"key3\", \"value3\")\n",
    "db.add_entry(\"key4\", \"value4\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'value1'"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "db.query(\"key1\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "db = EncryptedDB(bob, alice, secure_worker, max_val_len=256)\n",
    "\n",
    "db.add_entry(\"Bob\",\"(123) 456 7890\")\n",
    "db.add_entry(\"Bill\", \"(234) 567 8901\")\n",
    "db.add_entry(\"Sam\",\"(345) 678 9012\")\n",
    "db.add_entry(\"Key\",\"really big json value\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'(123) 456 7890'"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "db.query(\"Bob\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "#various function tests below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "def string2values(str_input, max_len=8):\n",
    "\n",
    "    #string too long\n",
    "    str_input = str_input[:max_len].lower()\n",
    "\n",
    "    #string too short - pad strings shorter than max len\n",
    "    if(len(str_input) < max_len):\n",
    "        str_input = str_input + \".\" * (max_len - len(str_input))\n",
    "    \n",
    "    values = list()\n",
    "    for char in str_input:\n",
    "        values.append(char2index[char])\n",
    "    \n",
    "    return th.tensor(values).long()\n",
    "\n",
    "def values2string(input_values):\n",
    "    s = \"\"\n",
    "    for value in input_values:\n",
    "        s += index2char[int(value)]\n",
    "    return s    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [],
   "source": [
    "#example\n",
    "#result: tensor([ 8, 15, 23,  4, 25, 37, 50, 50])\n",
    "# string2values(\"Howdy!\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [],
   "source": [
    "#example of one hot vector - where 1 is what we are trying to encoding\n",
    "#[0,0,0,1,0,0,0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [],
   "source": [
    "#one-hot vetor function\n",
    "def one_hot(index, length):\n",
    "    vect = th.zeros(length).long()\n",
    "    vect[index] = 1\n",
    "    return vect"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "def string2one_hot_matrix(str_input, max_len=8):\n",
    "    \n",
    "    #string longer than max len\n",
    "    str_input = str_input[:max_len].lower()\n",
    "    \n",
    "    #pad strings shorter than max len\n",
    "    if(len(str_input) < max_len):\n",
    "        str_input = str_input + \".\" * (max_len - len(str_input))\n",
    "        \n",
    "    char_vectors = list()\n",
    "    for char in str_input:\n",
    "        char_v = one_hot(char2index[char], len(char2index)).unsqueeze(0)\n",
    "        char_vectors.append(char_v)\n",
    "        \n",
    "    return th.cat(char_vectors, dim=0)            "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "matrix = string2one_hot_matrix(\"Hello\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([8, 69])"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#result: torch.Size([8, 69])\n",
    "matrix.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0,\n",
       "        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0])"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#result: See the 'p'\n",
    "'''\n",
    "tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0,\n",
    "        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
    "        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0])\n",
    "'''\n",
    "one_hot(char2index['p'], len(char2index))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n",
       "        [0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n",
       "        [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n",
       "        [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n",
       "        [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n",
       "        [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n",
       "        [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n",
       "        [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]])"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [],
   "source": [
    "# example of encoding strings\n",
    "str_a = string2one_hot_matrix(\"Hello\")\n",
    "str_b = string2one_hot_matrix(\"Hello\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n",
       "        [0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n",
       "        [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n",
       "        [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n",
       "        [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n",
       "        [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n",
       "        [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n",
       "        [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]])"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "str_a + str_b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([0, 0, 0, 0, 0, 1, 1, 0])"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#result of adding 2 str where they do not have a 1 in each matrix = 0in \n",
    "#8 char in string  - note qq overlap so tensor has 1,1 in correct positions\n",
    "# check whether or not there is perfect matching\n",
    "# result:  tensor([0, 0, 0, 0, 0, 1, 1, 0])\n",
    "\n",
    "str_a = string2one_hot_matrix(\"zsfgtqql\")\n",
    "str_b = string2one_hot_matrix(\"Helloqqp\")\n",
    "(str_a * str_b).sum(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [],
   "source": [
    "keys = list()\n",
    "values = list()\n",
    "\n",
    "keys.append(string2one_hot_matrix(\"key1\"))\n",
    "values.append(string2values(\"value1\"))\n",
    "\n",
    "keys.append(string2one_hot_matrix(\"key2\"))\n",
    "values.append(string2values(\"value2\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [],
   "source": [
    "def strings_equal(str_a, str_b):\n",
    "    vect = (str_a * str_b).sum(1)\n",
    "    \n",
    "    x = vect[0]\n",
    "    for i in range(vect.shape[0]-1):\n",
    "        x = x * vect[i + 1]\n",
    "    return x    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [],
   "source": [
    "def query(query_str):\n",
    "    query_matrix = string2one_hot_matrix(query_str)\n",
    "\n",
    "    key_matches = list()\n",
    "    for key in keys:\n",
    "    \n",
    "        key_match = strings_equal(key, query_matrix)\n",
    "        key_matches.append(key_match)\n",
    "\n",
    "    result = values[0] * key_matches[0]\n",
    "\n",
    "    for i in range(len(values) - 1):\n",
    "        result += values[i+1] * key_matches[i+1]  \n",
    "    \n",
    "\n",
    "    return values2string(result).replace(\".\",\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'value1'"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "query(\"key1\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'value2'"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "query(\"key2\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [],
   "source": [
    "def string2values(str_input, max_len=8):\n",
    "\n",
    "    str_input = str_input[:max_len].lower()\n",
    "\n",
    "    # pad strings shorter than max len\n",
    "    if(len(str_input) < max_len):\n",
    "        str_input = str_input + \".\" * (max_len - len(str_input))\n",
    "\n",
    "    values = list()\n",
    "    for char in str_input:\n",
    "        values.append(char2index[char])\n",
    "\n",
    "    return th.tensor(values).long()\n",
    "\n",
    "def values2string(input_values):\n",
    "    s = \"\"\n",
    "    for value in input_values:\n",
    "        s += index2char[int(value)]\n",
    "    return s\n",
    "\n",
    "def strings_equal(str_a, str_b):\n",
    "\n",
    "    vect = (str_a * str_b).sum(1)\n",
    "\n",
    "    x = vect[0]\n",
    "\n",
    "    for i in range(vect.shape[0] - 1):\n",
    "        x = x * vect[i + 1]    \n",
    "\n",
    "    return x\n",
    "\n",
    "def one_hot(index, length):\n",
    "    vect = th.zeros(length).long()\n",
    "    vect[index] = 1\n",
    "    return vect\n",
    "\n",
    "def string2one_hot_matrix(str_input, max_len=8):\n",
    "\n",
    "    str_input = str_input[:max_len].lower()\n",
    "\n",
    "    # pad strings shorter than max len\n",
    "    if(len(str_input) < max_len):\n",
    "        str_input = str_input + \".\" * (max_len - len(str_input))\n",
    "\n",
    "    char_vectors = list()\n",
    "    for char in str_input:\n",
    "        char_v = one_hot(char2index[char], len(char2index)).unsqueeze(0)\n",
    "        char_vectors.append(char_v)\n",
    "        \n",
    "    return th.cat(char_vectors, dim=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lesson: Encrypted Deep Learning in PyTorch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train a Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(2.0694)\n",
      "tensor(3.3094)\n",
      "tensor(6.1674)\n",
      "tensor(1.2736)\n",
      "tensor(0.5166)\n",
      "tensor(0.3181)\n",
      "tensor(0.1942)\n",
      "tensor(0.1079)\n",
      "tensor(0.0547)\n",
      "tensor(0.0258)\n",
      "tensor(0.0120)\n",
      "tensor(0.0057)\n",
      "tensor(0.0028)\n",
      "tensor(0.0015)\n",
      "tensor(0.0008)\n",
      "tensor(0.0005)\n",
      "tensor(0.0004)\n",
      "tensor(0.0003)\n",
      "tensor(0.0002)\n",
      "tensor(0.0002)\n"
     ]
    }
   ],
   "source": [
    "from torch import nn\n",
    "from torch import optim\n",
    "import torch.nn.functional as F\n",
    "\n",
    "# A Toy Dataset\n",
    "data = th.tensor([[0,0],[0,1],[1,0],[1,1.]], requires_grad=True)\n",
    "target = th.tensor([[0],[0],[1],[1.]], requires_grad=True)\n",
    "\n",
    "class Net(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(Net, self).__init__()\n",
    "        self.fc1 = nn.Linear(2, 20)\n",
    "        self.fc2 = nn.Linear(20, 1)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.fc1(x)\n",
    "        x = F.relu(x)\n",
    "        x = self.fc2(x)\n",
    "        return x\n",
    "\n",
    "# A Toy Model\n",
    "model = Net()\n",
    "\n",
    "def train():\n",
    "    # Training Logic\n",
    "    opt = optim.SGD(params=model.parameters(),lr=0.1)\n",
    "    for iter in range(20):\n",
    "\n",
    "        # 1) erase previous gradients (if they exist)\n",
    "        opt.zero_grad()\n",
    "\n",
    "        # 2) make a prediction\n",
    "        pred = model(data)\n",
    "\n",
    "        # 3) calculate how much we missed\n",
    "        loss = ((pred - target)**2).sum()\n",
    "\n",
    "        # 4) figure out which weights caused us to miss\n",
    "        loss.backward()\n",
    "\n",
    "        # 5) change those weights\n",
    "        opt.step()\n",
    "\n",
    "        # 6) print our progress\n",
    "        print(loss.data)\n",
    "        \n",
    "train()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.0067],\n",
       "        [-0.0052],\n",
       "        [ 0.9929],\n",
       "        [ 1.0045]], grad_fn=<AddmmBackward>)"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Encrypt the Model and Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [],
   "source": [
    "encrypted_model = model.fix_precision().share(alice, bob, crypto_provider=secure_worker)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Parameter containing:\n",
       " Parameter>FixedPrecisionTensor>(Wrapper)>[AdditiveSharingTensor]\n",
       " \t-> (Wrapper)>[PointerTensor | me:10898552317 -> alice:25432472908]\n",
       " \t-> (Wrapper)>[PointerTensor | me:69207609107 -> bob:20042250824]\n",
       " \t*crypto provider: secure_worker*, Parameter containing:\n",
       " Parameter>FixedPrecisionTensor>(Wrapper)>[AdditiveSharingTensor]\n",
       " \t-> (Wrapper)>[PointerTensor | me:53765865520 -> alice:37259982392]\n",
       " \t-> (Wrapper)>[PointerTensor | me:51718335484 -> bob:7266738877]\n",
       " \t*crypto provider: secure_worker*, Parameter containing:\n",
       " Parameter>FixedPrecisionTensor>(Wrapper)>[AdditiveSharingTensor]\n",
       " \t-> (Wrapper)>[PointerTensor | me:87948820122 -> alice:30265136830]\n",
       " \t-> (Wrapper)>[PointerTensor | me:21253899840 -> bob:35447447996]\n",
       " \t*crypto provider: secure_worker*, Parameter containing:\n",
       " Parameter>FixedPrecisionTensor>(Wrapper)>[AdditiveSharingTensor]\n",
       " \t-> (Wrapper)>[PointerTensor | me:22719818172 -> alice:95571714597]\n",
       " \t-> (Wrapper)>[PointerTensor | me:44013282246 -> bob:27466500319]\n",
       " \t*crypto provider: secure_worker*]"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(encrypted_model.parameters())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [],
   "source": [
    "encrypted_data = data.fix_precision().share(alice, bob, crypto_provider=secure_worker)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(Wrapper)>FixedPrecisionTensor>(Wrapper)>[AdditiveSharingTensor]\n",
       "\t-> (Wrapper)>[PointerTensor | me:65093720413 -> alice:93285324300]\n",
       "\t-> (Wrapper)>[PointerTensor | me:26835355956 -> bob:27213756386]\n",
       "\t*crypto provider: secure_worker*"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "encrypted_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [],
   "source": [
    "# encrypted data shared among owners\n",
    "encrypted_prediction = encrypted_model(encrypted_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.0070],\n",
       "        [-0.0040],\n",
       "        [ 0.9920],\n",
       "        [ 1.0030]])"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "encrypted_prediction.get().float_precision()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lesson: Encrypted Deep Learning in Keras\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 1: Public Training\n",
    "\n",
    "Welcome to this tutorial! In the following notebooks you will learn how to provide private predictions. By private predictions, we mean that the data is constantly encrypted throughout the entire process. At no point is the user sharing raw data, only encrypted (that is, secret shared) data. In order to provide these private predictions, Syft Keras uses a library called [TF Encrypted](https://github.com/tf-encrypted/tf-encrypted) under the hood. TF Encrypted combines cutting-edge cryptographic and machine learning techniques, but you don't have to worry about this and can focus on your machine learning application.\n",
    "\n",
    "You can start serving private predictions with only three steps:\n",
    "- **Step 1**: train your model with normal Keras.\n",
    "- **Step 2**: secure and serve your machine learning model (server).\n",
    "- **Step 3**: query the secured model to receive private predictions (client). \n",
    "\n",
    "Alright, let's go through these three steps so you can deploy impactful machine learning services without sacrificing user privacy or model security.\n",
    "\n",
    "Huge shoutout to the Dropout Labs ([@dropoutlabs](https://twitter.com/dropoutlabs)) and TF Encrypted ([@tf_encrypted](https://twitter.com/tf_encrypted)) teams for their great work which makes this demo possible, especially: Jason Mancuso ([@jvmancuso](https://twitter.com/jvmancuso)), Yann Dupis ([@YannDupis](https://twitter.com/YannDupis)), and Morten Dahl ([@mortendahlcs](https://github.com/mortendahlcs)). \n",
    "\n",
    "_Demo Ref: https://github.com/OpenMined/PySyft/tree/dev/examples/tutorials_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train Your Model in Keras\n",
    "\n",
    "To use privacy-preserving machine learning techniques for your projects you should not have to learn a new machine learning framework. If you have basic [Keras](https://keras.io/) knowledge, you can start using these techniques with Syft Keras. If you have never used Keras before, you can learn a bit more about it through the [Keras documentation](https://keras.io). \n",
    "\n",
    "Before serving private predictions, the first step is to train your model with normal Keras. As an example, we will train a model to classify handwritten digits. To train this model we will use the canonical [MNIST dataset](http://yann.lecun.com/exdb/mnist/).\n",
    "\n",
    "We borrow [this example](https://github.com/keras-team/keras/blob/master/examples/mnist_cnn.py) from the reference Keras repository.  To train your classification model, you just run the cell below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x_train shape: (60000, 28, 28, 1)\n",
      "60000 train samples\n",
      "10000 test samples\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "W0902 10:27:24.129999  9740 deprecation.py:506] From C:\\Users\\Claudia\\Anaconda3\\lib\\site-packages\\tensorflow\\python\\ops\\init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Call initializer instance with the dtype argument instead of passing it to the constructor\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 60000 samples, validate on 10000 samples\n",
      "Epoch 1/2\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "53760/60000 [=========================>....] - ETA: 1:02 - loss: 2.3002 - acc: 0.109 - ETA: 31s - loss: 2.3000 - acc: 0.135 - ETA: 25s - loss: 2.3008 - acc: 0.13 - ETA: 22s - loss: 2.3017 - acc: 0.13 - ETA: 20s - loss: 2.3013 - acc: 0.13 - ETA: 19s - loss: 2.3016 - acc: 0.12 - ETA: 19s - loss: 2.3017 - acc: 0.12 - ETA: 18s - loss: 2.3016 - acc: 0.12 - ETA: 18s - loss: 2.3016 - acc: 0.13 - ETA: 17s - loss: 2.3015 - acc: 0.13 - ETA: 17s - loss: 2.3014 - acc: 0.13 - ETA: 17s - loss: 2.3015 - acc: 0.13 - ETA: 17s - loss: 2.3016 - acc: 0.13 - ETA: 16s - loss: 2.3016 - acc: 0.13 - ETA: 16s - loss: 2.3014 - acc: 0.13 - ETA: 16s - loss: 2.3014 - acc: 0.13 - ETA: 16s - loss: 2.3014 - acc: 0.13 - ETA: 16s - loss: 2.3014 - acc: 0.13 - ETA: 16s - loss: 2.3015 - acc: 0.13 - ETA: 16s - loss: 2.3015 - acc: 0.13 - ETA: 15s - loss: 2.3016 - acc: 0.13 - ETA: 15s - loss: 2.3016 - acc: 0.13 - ETA: 15s - loss: 2.3016 - acc: 0.13 - ETA: 15s - loss: 2.3017 - acc: 0.13 - ETA: 15s - loss: 2.3016 - acc: 0.13 - ETA: 15s - loss: 2.3016 - acc: 0.13 - ETA: 15s - loss: 2.3016 - acc: 0.13 - ETA: 15s - loss: 2.3016 - acc: 0.13 - ETA: 15s - loss: 2.3016 - acc: 0.13 - ETA: 15s - loss: 2.3015 - acc: 0.13 - ETA: 15s - loss: 2.3015 - acc: 0.13 - ETA: 15s - loss: 2.3015 - acc: 0.13 - ETA: 14s - loss: 2.3016 - acc: 0.13 - ETA: 14s - loss: 2.3016 - acc: 0.13 - ETA: 14s - loss: 2.3016 - acc: 0.13 - ETA: 14s - loss: 2.3016 - acc: 0.13 - ETA: 14s - loss: 2.3016 - acc: 0.13 - ETA: 14s - loss: 2.3015 - acc: 0.13 - ETA: 14s - loss: 2.3016 - acc: 0.13 - ETA: 14s - loss: 2.3016 - acc: 0.13 - ETA: 14s - loss: 2.3016 - acc: 0.13 - ETA: 14s - loss: 2.3017 - acc: 0.13 - ETA: 14s - loss: 2.3017 - acc: 0.13 - ETA: 14s - loss: 2.3017 - acc: 0.13 - ETA: 13s - loss: 2.3016 - acc: 0.13 - ETA: 13s - loss: 2.3017 - acc: 0.13 - ETA: 13s - loss: 2.3017 - acc: 0.13 - ETA: 13s - loss: 2.3017 - acc: 0.13 - ETA: 13s - loss: 2.3017 - acc: 0.13 - ETA: 13s - loss: 2.3016 - acc: 0.13 - ETA: 13s - loss: 2.3017 - acc: 0.13 - ETA: 13s - loss: 2.3017 - acc: 0.13 - ETA: 13s - loss: 2.3017 - acc: 0.13 - ETA: 13s - loss: 2.3017 - acc: 0.13 - ETA: 13s - loss: 2.3016 - acc: 0.13 - ETA: 13s - loss: 2.3016 - acc: 0.13 - ETA: 12s - loss: 2.3017 - acc: 0.13 - ETA: 12s - loss: 2.3017 - acc: 0.13 - ETA: 12s - loss: 2.3017 - acc: 0.13 - ETA: 12s - loss: 2.3017 - acc: 0.13 - ETA: 12s - loss: 2.3017 - acc: 0.13 - ETA: 12s - loss: 2.3017 - acc: 0.13 - ETA: 12s - loss: 2.3016 - acc: 0.14 - ETA: 12s - loss: 2.3016 - acc: 0.13 - ETA: 12s - loss: 2.3016 - acc: 0.13 - ETA: 12s - loss: 2.3016 - acc: 0.13 - ETA: 12s - loss: 2.3016 - acc: 0.13 - ETA: 12s - loss: 2.3016 - acc: 0.13 - ETA: 12s - loss: 2.3016 - acc: 0.13 - ETA: 12s - loss: 2.3016 - acc: 0.14 - ETA: 11s - loss: 2.3016 - acc: 0.13 - ETA: 11s - loss: 2.3016 - acc: 0.14 - ETA: 11s - loss: 2.3016 - acc: 0.13 - ETA: 11s - loss: 2.3016 - acc: 0.13 - ETA: 11s - loss: 2.3016 - acc: 0.14 - ETA: 11s - loss: 2.3016 - acc: 0.14 - ETA: 11s - loss: 2.3016 - acc: 0.13 - ETA: 11s - loss: 2.3016 - acc: 0.13 - ETA: 11s - loss: 2.3016 - acc: 0.13 - ETA: 11s - loss: 2.3016 - acc: 0.13 - ETA: 11s - loss: 2.3016 - acc: 0.13 - ETA: 11s - loss: 2.3016 - acc: 0.13 - ETA: 11s - loss: 2.3017 - acc: 0.13 - ETA: 10s - loss: 2.3016 - acc: 0.13 - ETA: 10s - loss: 2.3016 - acc: 0.13 - ETA: 10s - loss: 2.3016 - acc: 0.14 - ETA: 10s - loss: 2.3016 - acc: 0.14 - ETA: 10s - loss: 2.3016 - acc: 0.14 - ETA: 10s - loss: 2.3016 - acc: 0.14 - ETA: 10s - loss: 2.3016 - acc: 0.14 - ETA: 10s - loss: 2.3016 - acc: 0.14 - ETA: 10s - loss: 2.3016 - acc: 0.14 - ETA: 10s - loss: 2.3016 - acc: 0.14 - ETA: 10s - loss: 2.3016 - acc: 0.14 - ETA: 10s - loss: 2.3016 - acc: 0.14 - ETA: 10s - loss: 2.3016 - acc: 0.14 - ETA: 10s - loss: 2.3016 - acc: 0.14 - ETA: 10s - loss: 2.3016 - acc: 0.14 - ETA: 9s - loss: 2.3016 - acc: 0.1419 - ETA: 9s - loss: 2.3016 - acc: 0.141 - ETA: 9s - loss: 2.3016 - acc: 0.142 - ETA: 9s - loss: 2.3016 - acc: 0.142 - ETA: 9s - loss: 2.3016 - acc: 0.141 - ETA: 9s - loss: 2.3016 - acc: 0.141 - ETA: 9s - loss: 2.3016 - acc: 0.142 - ETA: 9s - loss: 2.3016 - acc: 0.142 - ETA: 9s - loss: 2.3016 - acc: 0.142 - ETA: 9s - loss: 2.3016 - acc: 0.142 - ETA: 9s - loss: 2.3016 - acc: 0.143 - ETA: 9s - loss: 2.3016 - acc: 0.142 - ETA: 9s - loss: 2.3016 - acc: 0.142 - ETA: 9s - loss: 2.3016 - acc: 0.142 - ETA: 8s - loss: 2.3016 - acc: 0.143 - ETA: 8s - loss: 2.3016 - acc: 0.143 - ETA: 8s - loss: 2.3016 - acc: 0.143 - ETA: 8s - loss: 2.3016 - acc: 0.143 - ETA: 8s - loss: 2.3016 - acc: 0.143 - ETA: 8s - loss: 2.3016 - acc: 0.143 - ETA: 8s - loss: 2.3016 - acc: 0.143 - ETA: 8s - loss: 2.3016 - acc: 0.143 - ETA: 8s - loss: 2.3016 - acc: 0.143 - ETA: 8s - loss: 2.3016 - acc: 0.143 - ETA: 8s - loss: 2.3016 - acc: 0.142 - ETA: 8s - loss: 2.3016 - acc: 0.142 - ETA: 8s - loss: 2.3016 - acc: 0.142 - ETA: 8s - loss: 2.3016 - acc: 0.142 - ETA: 7s - loss: 2.3016 - acc: 0.142 - ETA: 7s - loss: 2.3016 - acc: 0.142 - ETA: 7s - loss: 2.3016 - acc: 0.142 - ETA: 7s - loss: 2.3016 - acc: 0.142 - ETA: 7s - loss: 2.3016 - acc: 0.142 - ETA: 7s - loss: 2.3016 - acc: 0.142 - ETA: 7s - loss: 2.3016 - acc: 0.142 - ETA: 7s - loss: 2.3016 - acc: 0.142 - ETA: 7s - loss: 2.3016 - acc: 0.142 - ETA: 7s - loss: 2.3016 - acc: 0.142 - ETA: 7s - loss: 2.3016 - acc: 0.143 - ETA: 7s - loss: 2.3016 - acc: 0.143 - ETA: 7s - loss: 2.3016 - acc: 0.142 - ETA: 7s - loss: 2.3016 - acc: 0.142 - ETA: 6s - loss: 2.3016 - acc: 0.143 - ETA: 6s - loss: 2.3016 - acc: 0.143 - ETA: 6s - loss: 2.3016 - acc: 0.143 - ETA: 6s - loss: 2.3016 - acc: 0.143 - ETA: 6s - loss: 2.3016 - acc: 0.143 - ETA: 6s - loss: 2.3016 - acc: 0.143 - ETA: 6s - loss: 2.3016 - acc: 0.143 - ETA: 6s - loss: 2.3016 - acc: 0.143 - ETA: 6s - loss: 2.3016 - acc: 0.143 - ETA: 6s - loss: 2.3016 - acc: 0.143 - ETA: 6s - loss: 2.3016 - acc: 0.143 - ETA: 6s - loss: 2.3016 - acc: 0.143 - ETA: 6s - loss: 2.3016 - acc: 0.143 - ETA: 6s - loss: 2.3016 - acc: 0.142 - ETA: 5s - loss: 2.3016 - acc: 0.143 - ETA: 5s - loss: 2.3016 - acc: 0.143 - ETA: 5s - loss: 2.3016 - acc: 0.143 - ETA: 5s - loss: 2.3016 - acc: 0.143 - ETA: 5s - loss: 2.3016 - acc: 0.143 - ETA: 5s - loss: 2.3016 - acc: 0.143 - ETA: 5s - loss: 2.3016 - acc: 0.143 - ETA: 5s - loss: 2.3016 - acc: 0.144 - ETA: 5s - loss: 2.3016 - acc: 0.144 - ETA: 5s - loss: 2.3016 - acc: 0.144 - ETA: 5s - loss: 2.3016 - acc: 0.144 - ETA: 5s - loss: 2.3016 - acc: 0.144 - ETA: 5s - loss: 2.3016 - acc: 0.144 - ETA: 4s - loss: 2.3016 - acc: 0.144 - ETA: 4s - loss: 2.3016 - acc: 0.144 - ETA: 4s - loss: 2.3016 - acc: 0.144 - ETA: 4s - loss: 2.3016 - acc: 0.144 - ETA: 4s - loss: 2.3016 - acc: 0.144 - ETA: 4s - loss: 2.3016 - acc: 0.144 - ETA: 4s - loss: 2.3016 - acc: 0.144 - ETA: 4s - loss: 2.3016 - acc: 0.144 - ETA: 4s - loss: 2.3016 - acc: 0.144 - ETA: 4s - loss: 2.3015 - acc: 0.144 - ETA: 4s - loss: 2.3015 - acc: 0.144 - ETA: 4s - loss: 2.3015 - acc: 0.145 - ETA: 4s - loss: 2.3015 - acc: 0.145 - ETA: 4s - loss: 2.3015 - acc: 0.145 - ETA: 4s - loss: 2.3015 - acc: 0.145 - ETA: 3s - loss: 2.3015 - acc: 0.145 - ETA: 3s - loss: 2.3015 - acc: 0.145 - ETA: 3s - loss: 2.3015 - acc: 0.145 - ETA: 3s - loss: 2.3015 - acc: 0.145 - ETA: 3s - loss: 2.3015 - acc: 0.145 - ETA: 3s - loss: 2.3015 - acc: 0.145 - ETA: 3s - loss: 2.3015 - acc: 0.145 - ETA: 3s - loss: 2.3015 - acc: 0.145 - ETA: 3s - loss: 2.3015 - acc: 0.144 - ETA: 3s - loss: 2.3015 - acc: 0.145 - ETA: 3s - loss: 2.3015 - acc: 0.145 - ETA: 3s - loss: 2.3015 - acc: 0.145 - ETA: 3s - loss: 2.3015 - acc: 0.145 - ETA: 2s - loss: 2.3015 - acc: 0.145 - ETA: 2s - loss: 2.3015 - acc: 0.145 - ETA: 2s - loss: 2.3015 - acc: 0.145 - ETA: 2s - loss: 2.3015 - acc: 0.145 - ETA: 2s - loss: 2.3015 - acc: 0.145 - ETA: 2s - loss: 2.3015 - acc: 0.145 - ETA: 2s - loss: 2.3015 - acc: 0.145 - ETA: 2s - loss: 2.3015 - acc: 0.145 - ETA: 2s - loss: 2.3015 - acc: 0.145 - ETA: 2s - loss: 2.3015 - acc: 0.145 - ETA: 2s - loss: 2.3015 - acc: 0.145 - ETA: 2s - loss: 2.3015 - acc: 0.145 - ETA: 2s - loss: 2.3015 - acc: 0.145 - ETA: 2s - loss: 2.3015 - acc: 0.145 - ETA: 1s - loss: 2.3015 - acc: 0.145 - ETA: 1s - loss: 2.3015 - acc: 0.145 - ETA: 1s - loss: 2.3015 - acc: 0.145 - ETA: 1s - loss: 2.3015 - acc: 0.1460\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "60000/60000 [==============================] - ETA: 1s - loss: 2.3015 - acc: 0.146 - ETA: 1s - loss: 2.3015 - acc: 0.146 - ETA: 1s - loss: 2.3015 - acc: 0.145 - ETA: 1s - loss: 2.3015 - acc: 0.145 - ETA: 1s - loss: 2.3015 - acc: 0.145 - ETA: 1s - loss: 2.3015 - acc: 0.146 - ETA: 1s - loss: 2.3015 - acc: 0.146 - ETA: 1s - loss: 2.3015 - acc: 0.146 - ETA: 1s - loss: 2.3015 - acc: 0.146 - ETA: 1s - loss: 2.3015 - acc: 0.146 - ETA: 0s - loss: 2.3015 - acc: 0.146 - ETA: 0s - loss: 2.3015 - acc: 0.146 - ETA: 0s - loss: 2.3015 - acc: 0.146 - ETA: 0s - loss: 2.3015 - acc: 0.145 - ETA: 0s - loss: 2.3015 - acc: 0.145 - ETA: 0s - loss: 2.3015 - acc: 0.146 - ETA: 0s - loss: 2.3015 - acc: 0.146 - ETA: 0s - loss: 2.3015 - acc: 0.146 - ETA: 0s - loss: 2.3015 - acc: 0.146 - ETA: 0s - loss: 2.3015 - acc: 0.146 - ETA: 0s - loss: 2.3015 - acc: 0.146 - ETA: 0s - loss: 2.3015 - acc: 0.146 - ETA: 0s - loss: 2.3015 - acc: 0.146 - ETA: 0s - loss: 2.3015 - acc: 0.146 - 18s 297us/sample - loss: 2.3015 - acc: 0.1466 - val_loss: 2.3011 - val_acc: 0.1583\n",
      "Epoch 2/2\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "53632/60000 [=========================>....] - ETA: 18s - loss: 2.3004 - acc: 0.16 - ETA: 17s - loss: 2.3006 - acc: 0.15 - ETA: 16s - loss: 2.3009 - acc: 0.16 - ETA: 16s - loss: 2.3015 - acc: 0.15 - ETA: 16s - loss: 2.3013 - acc: 0.14 - ETA: 16s - loss: 2.3014 - acc: 0.14 - ETA: 16s - loss: 2.3014 - acc: 0.14 - ETA: 16s - loss: 2.3014 - acc: 0.14 - ETA: 15s - loss: 2.3014 - acc: 0.14 - ETA: 15s - loss: 2.3014 - acc: 0.14 - ETA: 15s - loss: 2.3013 - acc: 0.14 - ETA: 15s - loss: 2.3013 - acc: 0.15 - ETA: 15s - loss: 2.3012 - acc: 0.15 - ETA: 15s - loss: 2.3013 - acc: 0.15 - ETA: 15s - loss: 2.3013 - acc: 0.15 - ETA: 15s - loss: 2.3013 - acc: 0.15 - ETA: 15s - loss: 2.3012 - acc: 0.15 - ETA: 15s - loss: 2.3011 - acc: 0.15 - ETA: 15s - loss: 2.3011 - acc: 0.15 - ETA: 15s - loss: 2.3012 - acc: 0.15 - ETA: 15s - loss: 2.3012 - acc: 0.15 - ETA: 14s - loss: 2.3011 - acc: 0.15 - ETA: 15s - loss: 2.3011 - acc: 0.15 - ETA: 15s - loss: 2.3012 - acc: 0.15 - ETA: 14s - loss: 2.3011 - acc: 0.15 - ETA: 14s - loss: 2.3011 - acc: 0.15 - ETA: 14s - loss: 2.3011 - acc: 0.15 - ETA: 14s - loss: 2.3011 - acc: 0.15 - ETA: 14s - loss: 2.3012 - acc: 0.15 - ETA: 14s - loss: 2.3011 - acc: 0.15 - ETA: 14s - loss: 2.3011 - acc: 0.15 - ETA: 14s - loss: 2.3011 - acc: 0.15 - ETA: 14s - loss: 2.3012 - acc: 0.15 - ETA: 14s - loss: 2.3012 - acc: 0.15 - ETA: 14s - loss: 2.3012 - acc: 0.15 - ETA: 14s - loss: 2.3012 - acc: 0.15 - ETA: 14s - loss: 2.3011 - acc: 0.15 - ETA: 14s - loss: 2.3011 - acc: 0.15 - ETA: 14s - loss: 2.3012 - acc: 0.15 - ETA: 13s - loss: 2.3012 - acc: 0.15 - ETA: 13s - loss: 2.3012 - acc: 0.15 - ETA: 13s - loss: 2.3012 - acc: 0.15 - ETA: 13s - loss: 2.3011 - acc: 0.15 - ETA: 13s - loss: 2.3011 - acc: 0.15 - ETA: 13s - loss: 2.3011 - acc: 0.15 - ETA: 13s - loss: 2.3011 - acc: 0.15 - ETA: 13s - loss: 2.3011 - acc: 0.15 - ETA: 13s - loss: 2.3011 - acc: 0.15 - ETA: 13s - loss: 2.3011 - acc: 0.15 - ETA: 13s - loss: 2.3011 - acc: 0.15 - ETA: 13s - loss: 2.3010 - acc: 0.15 - ETA: 13s - loss: 2.3010 - acc: 0.15 - ETA: 13s - loss: 2.3010 - acc: 0.15 - ETA: 12s - loss: 2.3010 - acc: 0.15 - ETA: 12s - loss: 2.3011 - acc: 0.15 - ETA: 12s - loss: 2.3011 - acc: 0.15 - ETA: 12s - loss: 2.3010 - acc: 0.15 - ETA: 12s - loss: 2.3010 - acc: 0.15 - ETA: 12s - loss: 2.3011 - acc: 0.15 - ETA: 12s - loss: 2.3010 - acc: 0.15 - ETA: 12s - loss: 2.3010 - acc: 0.15 - ETA: 12s - loss: 2.3010 - acc: 0.15 - ETA: 12s - loss: 2.3010 - acc: 0.15 - ETA: 12s - loss: 2.3011 - acc: 0.15 - ETA: 12s - loss: 2.3011 - acc: 0.15 - ETA: 12s - loss: 2.3011 - acc: 0.15 - ETA: 12s - loss: 2.3011 - acc: 0.15 - ETA: 12s - loss: 2.3011 - acc: 0.15 - ETA: 11s - loss: 2.3011 - acc: 0.15 - ETA: 11s - loss: 2.3011 - acc: 0.15 - ETA: 11s - loss: 2.3011 - acc: 0.15 - ETA: 11s - loss: 2.3011 - acc: 0.15 - ETA: 11s - loss: 2.3011 - acc: 0.15 - ETA: 11s - loss: 2.3011 - acc: 0.15 - ETA: 11s - loss: 2.3011 - acc: 0.15 - ETA: 11s - loss: 2.3011 - acc: 0.15 - ETA: 11s - loss: 2.3011 - acc: 0.15 - ETA: 11s - loss: 2.3011 - acc: 0.15 - ETA: 11s - loss: 2.3011 - acc: 0.15 - ETA: 11s - loss: 2.3011 - acc: 0.15 - ETA: 10s - loss: 2.3011 - acc: 0.15 - ETA: 10s - loss: 2.3010 - acc: 0.15 - ETA: 10s - loss: 2.3010 - acc: 0.15 - ETA: 10s - loss: 2.3010 - acc: 0.15 - ETA: 10s - loss: 2.3011 - acc: 0.15 - ETA: 10s - loss: 2.3010 - acc: 0.15 - ETA: 10s - loss: 2.3010 - acc: 0.15 - ETA: 10s - loss: 2.3010 - acc: 0.15 - ETA: 10s - loss: 2.3010 - acc: 0.15 - ETA: 10s - loss: 2.3010 - acc: 0.15 - ETA: 10s - loss: 2.3010 - acc: 0.16 - ETA: 10s - loss: 2.3010 - acc: 0.16 - ETA: 10s - loss: 2.3010 - acc: 0.16 - ETA: 9s - loss: 2.3010 - acc: 0.1604 - ETA: 9s - loss: 2.3010 - acc: 0.160 - ETA: 9s - loss: 2.3010 - acc: 0.160 - ETA: 9s - loss: 2.3010 - acc: 0.161 - ETA: 9s - loss: 2.3010 - acc: 0.161 - ETA: 9s - loss: 2.3010 - acc: 0.161 - ETA: 9s - loss: 2.3010 - acc: 0.161 - ETA: 9s - loss: 2.3010 - acc: 0.161 - ETA: 9s - loss: 2.3010 - acc: 0.160 - ETA: 9s - loss: 2.3010 - acc: 0.161 - ETA: 9s - loss: 2.3010 - acc: 0.161 - ETA: 9s - loss: 2.3010 - acc: 0.161 - ETA: 8s - loss: 2.3010 - acc: 0.161 - ETA: 8s - loss: 2.3010 - acc: 0.161 - ETA: 8s - loss: 2.3010 - acc: 0.161 - ETA: 8s - loss: 2.3009 - acc: 0.161 - ETA: 8s - loss: 2.3010 - acc: 0.161 - ETA: 8s - loss: 2.3010 - acc: 0.161 - ETA: 8s - loss: 2.3009 - acc: 0.161 - ETA: 8s - loss: 2.3010 - acc: 0.161 - ETA: 8s - loss: 2.3010 - acc: 0.161 - ETA: 8s - loss: 2.3010 - acc: 0.161 - ETA: 8s - loss: 2.3010 - acc: 0.160 - ETA: 8s - loss: 2.3010 - acc: 0.160 - ETA: 8s - loss: 2.3010 - acc: 0.160 - ETA: 7s - loss: 2.3010 - acc: 0.160 - ETA: 7s - loss: 2.3010 - acc: 0.160 - ETA: 7s - loss: 2.3010 - acc: 0.160 - ETA: 7s - loss: 2.3010 - acc: 0.160 - ETA: 7s - loss: 2.3010 - acc: 0.160 - ETA: 7s - loss: 2.3010 - acc: 0.160 - ETA: 7s - loss: 2.3010 - acc: 0.160 - ETA: 7s - loss: 2.3010 - acc: 0.160 - ETA: 7s - loss: 2.3010 - acc: 0.161 - ETA: 7s - loss: 2.3010 - acc: 0.161 - ETA: 7s - loss: 2.3010 - acc: 0.161 - ETA: 7s - loss: 2.3010 - acc: 0.161 - ETA: 7s - loss: 2.3010 - acc: 0.161 - ETA: 7s - loss: 2.3009 - acc: 0.161 - ETA: 7s - loss: 2.3010 - acc: 0.161 - ETA: 6s - loss: 2.3009 - acc: 0.161 - ETA: 6s - loss: 2.3009 - acc: 0.161 - ETA: 6s - loss: 2.3009 - acc: 0.161 - ETA: 6s - loss: 2.3009 - acc: 0.161 - ETA: 6s - loss: 2.3009 - acc: 0.161 - ETA: 6s - loss: 2.3009 - acc: 0.161 - ETA: 6s - loss: 2.3009 - acc: 0.161 - ETA: 6s - loss: 2.3009 - acc: 0.161 - ETA: 6s - loss: 2.3009 - acc: 0.161 - ETA: 6s - loss: 2.3009 - acc: 0.161 - ETA: 6s - loss: 2.3009 - acc: 0.161 - ETA: 6s - loss: 2.3009 - acc: 0.161 - ETA: 6s - loss: 2.3009 - acc: 0.161 - ETA: 6s - loss: 2.3009 - acc: 0.161 - ETA: 6s - loss: 2.3010 - acc: 0.161 - ETA: 6s - loss: 2.3010 - acc: 0.161 - ETA: 6s - loss: 2.3010 - acc: 0.161 - ETA: 6s - loss: 2.3010 - acc: 0.161 - ETA: 6s - loss: 2.3010 - acc: 0.161 - ETA: 6s - loss: 2.3009 - acc: 0.161 - ETA: 6s - loss: 2.3010 - acc: 0.161 - ETA: 5s - loss: 2.3009 - acc: 0.161 - ETA: 5s - loss: 2.3009 - acc: 0.161 - ETA: 5s - loss: 2.3009 - acc: 0.162 - ETA: 5s - loss: 2.3009 - acc: 0.162 - ETA: 5s - loss: 2.3009 - acc: 0.162 - ETA: 5s - loss: 2.3009 - acc: 0.162 - ETA: 5s - loss: 2.3009 - acc: 0.162 - ETA: 5s - loss: 2.3009 - acc: 0.162 - ETA: 5s - loss: 2.3009 - acc: 0.162 - ETA: 5s - loss: 2.3009 - acc: 0.162 - ETA: 5s - loss: 2.3009 - acc: 0.162 - ETA: 5s - loss: 2.3009 - acc: 0.162 - ETA: 5s - loss: 2.3009 - acc: 0.162 - ETA: 5s - loss: 2.3009 - acc: 0.162 - ETA: 4s - loss: 2.3009 - acc: 0.163 - ETA: 4s - loss: 2.3009 - acc: 0.163 - ETA: 4s - loss: 2.3009 - acc: 0.163 - ETA: 4s - loss: 2.3009 - acc: 0.163 - ETA: 4s - loss: 2.3009 - acc: 0.163 - ETA: 4s - loss: 2.3009 - acc: 0.163 - ETA: 4s - loss: 2.3009 - acc: 0.163 - ETA: 4s - loss: 2.3009 - acc: 0.163 - ETA: 4s - loss: 2.3009 - acc: 0.163 - ETA: 4s - loss: 2.3009 - acc: 0.163 - ETA: 4s - loss: 2.3009 - acc: 0.163 - ETA: 4s - loss: 2.3009 - acc: 0.163 - ETA: 4s - loss: 2.3009 - acc: 0.163 - ETA: 4s - loss: 2.3009 - acc: 0.163 - ETA: 3s - loss: 2.3009 - acc: 0.163 - ETA: 3s - loss: 2.3009 - acc: 0.163 - ETA: 3s - loss: 2.3009 - acc: 0.163 - ETA: 3s - loss: 2.3009 - acc: 0.163 - ETA: 3s - loss: 2.3008 - acc: 0.163 - ETA: 3s - loss: 2.3008 - acc: 0.163 - ETA: 3s - loss: 2.3008 - acc: 0.163 - ETA: 3s - loss: 2.3008 - acc: 0.163 - ETA: 3s - loss: 2.3008 - acc: 0.164 - ETA: 3s - loss: 2.3008 - acc: 0.164 - ETA: 3s - loss: 2.3008 - acc: 0.164 - ETA: 3s - loss: 2.3008 - acc: 0.164 - ETA: 3s - loss: 2.3008 - acc: 0.164 - ETA: 3s - loss: 2.3008 - acc: 0.164 - ETA: 2s - loss: 2.3008 - acc: 0.164 - ETA: 2s - loss: 2.3008 - acc: 0.164 - ETA: 2s - loss: 2.3008 - acc: 0.164 - ETA: 2s - loss: 2.3008 - acc: 0.164 - ETA: 2s - loss: 2.3008 - acc: 0.164 - ETA: 2s - loss: 2.3008 - acc: 0.164 - ETA: 2s - loss: 2.3008 - acc: 0.164 - ETA: 2s - loss: 2.3008 - acc: 0.164 - ETA: 2s - loss: 2.3008 - acc: 0.164 - ETA: 2s - loss: 2.3008 - acc: 0.165 - ETA: 2s - loss: 2.3008 - acc: 0.165 - ETA: 2s - loss: 2.3008 - acc: 0.164 - ETA: 2s - loss: 2.3008 - acc: 0.164 - ETA: 2s - loss: 2.3008 - acc: 0.164 - ETA: 1s - loss: 2.3008 - acc: 0.164 - ETA: 1s - loss: 2.3008 - acc: 0.164 - ETA: 1s - loss: 2.3008 - acc: 0.164 - ETA: 1s - loss: 2.3008 - acc: 0.164 - ETA: 1s - loss: 2.3008 - acc: 0.1650"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "60000/60000 [==============================] - ETA: 1s - loss: 2.3008 - acc: 0.165 - ETA: 1s - loss: 2.3008 - acc: 0.165 - ETA: 1s - loss: 2.3008 - acc: 0.165 - ETA: 1s - loss: 2.3008 - acc: 0.165 - ETA: 1s - loss: 2.3008 - acc: 0.165 - ETA: 1s - loss: 2.3008 - acc: 0.165 - ETA: 1s - loss: 2.3008 - acc: 0.165 - ETA: 1s - loss: 2.3008 - acc: 0.165 - ETA: 1s - loss: 2.3008 - acc: 0.165 - ETA: 1s - loss: 2.3008 - acc: 0.165 - ETA: 0s - loss: 2.3008 - acc: 0.165 - ETA: 0s - loss: 2.3008 - acc: 0.165 - ETA: 0s - loss: 2.3008 - acc: 0.165 - ETA: 0s - loss: 2.3008 - acc: 0.165 - ETA: 0s - loss: 2.3008 - acc: 0.165 - ETA: 0s - loss: 2.3008 - acc: 0.165 - ETA: 0s - loss: 2.3008 - acc: 0.165 - ETA: 0s - loss: 2.3008 - acc: 0.165 - ETA: 0s - loss: 2.3008 - acc: 0.165 - ETA: 0s - loss: 2.3008 - acc: 0.165 - ETA: 0s - loss: 2.3008 - acc: 0.165 - ETA: 0s - loss: 2.3008 - acc: 0.165 - ETA: 0s - loss: 2.3008 - acc: 0.165 - ETA: 0s - loss: 2.3008 - acc: 0.165 - 17s 280us/sample - loss: 2.3008 - acc: 0.1656 - val_loss: 2.3005 - val_acc: 0.1765\n",
      "Test loss: 2.300467993927002\n",
      "Test accuracy: 0.1765\n"
     ]
    }
   ],
   "source": [
    "from __future__ import print_function\n",
    "import tensorflow.keras as keras\n",
    "from tensorflow.keras.datasets import mnist\n",
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers import Dense, Dropout, Flatten\n",
    "from tensorflow.keras.layers import Conv2D, AveragePooling2D\n",
    "from tensorflow.keras.layers import Activation\n",
    "\n",
    "batch_size = 128\n",
    "num_classes = 10\n",
    "epochs = 2\n",
    "\n",
    "# input image dimensions\n",
    "img_rows, img_cols = 28, 28\n",
    "\n",
    "# the data, split between train and test sets\n",
    "(x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
    "\n",
    "x_train = x_train.reshape(x_train.shape[0], img_rows, img_cols, 1)\n",
    "x_test = x_test.reshape(x_test.shape[0], img_rows, img_cols, 1)\n",
    "input_shape = (img_rows, img_cols, 1)\n",
    "\n",
    "x_train = x_train.astype('float32')\n",
    "x_test = x_test.astype('float32')\n",
    "x_train /= 255\n",
    "x_test /= 255\n",
    "print('x_train shape:', x_train.shape)\n",
    "print(x_train.shape[0], 'train samples')\n",
    "print(x_test.shape[0], 'test samples')\n",
    "\n",
    "# convert class vectors to binary class matrices\n",
    "y_train = keras.utils.to_categorical(y_train, num_classes)\n",
    "y_test = keras.utils.to_categorical(y_test, num_classes)\n",
    "\n",
    "model = Sequential()\n",
    "\n",
    "model.add(Conv2D(10, (3, 3), input_shape=input_shape))\n",
    "model.add(AveragePooling2D((2, 2)))\n",
    "model.add(Activation('relu'))\n",
    "model.add(Conv2D(32, (3, 3)))\n",
    "model.add(AveragePooling2D((2, 2)))\n",
    "model.add(Activation('relu'))\n",
    "model.add(Conv2D(64, (3, 3)))\n",
    "model.add(AveragePooling2D((2, 2)))\n",
    "model.add(Activation('relu'))\n",
    "model.add(Flatten())\n",
    "model.add(Dense(num_classes, activation='softmax'))\n",
    "\n",
    "model.compile(loss=keras.losses.categorical_crossentropy,\n",
    "              optimizer=keras.optimizers.Adadelta(),\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "model.fit(x_train, y_train,\n",
    "          batch_size=batch_size,\n",
    "          epochs=epochs,\n",
    "          verbose=1,\n",
    "          validation_data=(x_test, y_test))\n",
    "score = model.evaluate(x_test, y_test, verbose=0)\n",
    "print('Test loss:', score[0])\n",
    "print('Test accuracy:', score[1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Save your model's weights for future private prediction\n",
    "model.save('short-conv-mnist.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2: Load and Serve the Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that you have a trained model with normal Keras, you are ready to serve some private predictions. We can do that using Syft Keras.\n",
    "\n",
    "To secure and serve this model, we will need three TFEWorkers (servers). This is because TF Encrypted under the hood uses an encryption technique called [multi-party computation (MPC)](https://en.wikipedia.org/wiki/Secure_multi-party_computation). The idea is to split the model weights and input data into shares, then send a share of each value to the different servers. The key property is that if you look at the share on one server, it reveals nothing about the original value (input data or model weights).\n",
    "\n",
    "We'll define a Syft Keras model like we did in the previous notebook. However, there is a trick: before instantiating this model, we'll run `hook = sy.KerasHook(tf.keras)`. This will add three important new methods to the Keras Sequential class:\n",
    " - `share`: will secure your model via secret sharing; by default, it will use the SecureNN protocol from TF Encrypted to secret share your model between each of the three TFEWorkers. Most importantly, this will add the capability of providing predictions on encrypted data.\n",
    " - `serve`: this function will launch a serving queue, so that the TFEWorkers can can accept prediction requests on the secured model from external clients.\n",
    " - `shutdown_workers`: once you are done providing private predictions, you can shut down your model by running this function. It will direct you to shutdown the server processes manually if you've opted to manually manage each worker.\n",
    "\n",
    "If you want learn more about MPC, you can read this excellent [blog](https://mortendahl.github.io/2017/04/17/private-deep-learning-with-mpc/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "from tensorflow.keras import Sequential\n",
    "from tensorflow.keras.layers import AveragePooling2D, Conv2D, Dense, Activation, Flatten, ReLU, Activation\n",
    "\n",
    "import syft as sy\n",
    "hook = sy.KerasHook(tf.keras)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model\n",
    "\n",
    "As you can see, we define almost the exact same model as before, except we provide a `batch_input_shape`. This allows TF Encrypted to better optimize the secure computations via predefined tensor shapes. For this MNIST demo, we'll send input data with the shape of (1, 28, 28, 1). \n",
    "We also return the logit instead of softmax because this operation is complex to perform using MPC, and we don't need it to serve prediction requests."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_classes = 10\n",
    "input_shape = (1, 28, 28, 1)\n",
    "\n",
    "model = Sequential()\n",
    "\n",
    "model.add(Conv2D(10, (3, 3), batch_input_shape=input_shape))\n",
    "model.add(AveragePooling2D((2, 2)))\n",
    "model.add(Activation('relu'))\n",
    "model.add(Conv2D(32, (3, 3)))\n",
    "model.add(AveragePooling2D((2, 2)))\n",
    "model.add(Activation('relu'))\n",
    "model.add(Conv2D(64, (3, 3)))\n",
    "model.add(AveragePooling2D((2, 2)))\n",
    "model.add(Activation('relu'))\n",
    "model.add(Flatten())\n",
    "model.add(Dense(num_classes, name=\"logit\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load Pre-trained Weights\n",
    "\n",
    "With `load_weights` you can easily load the weights you have saved previously after training your model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [],
   "source": [
    "pre_trained_weights = 'short-conv-mnist.h5'\n",
    "model.load_weights(pre_trained_weights)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3: Setup Your Worker Connectors\n",
    "\n",
    "Let's now connect to the TFEWorkers (`alice`, `bob`, and `carol`) required by TF Encrypted to perform private predictions. For each TFEWorker, you just have to specify a host.\n",
    "\n",
    "These workers run a [TensorFlow server](https://www.tensorflow.org/api_docs/python/tf/distribute/Server), which you can either manage manually (`AUTO = False`) or ask the workers to manage for you (`AUTO = True`). If choosing to manually manage them, you will be instructed to execute a terminal command on each worker's host device after calling `model.share()` below.  If all workers are hosted on a single device (e.g. `localhost`), you can choose to have Syft automatically manage the worker's TensorFlow server."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [],
   "source": [
    "AUTO = False\n",
    "\n",
    "alice = sy.TFEWorker(host='localhost:4000', auto_managed=AUTO)\n",
    "bob = sy.TFEWorker(host='localhost:4001', auto_managed=AUTO)\n",
    "carol = sy.TFEWorker(host='localhost:4002', auto_managed=AUTO)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 4: Split the Model Into Shares\n",
    "\n",
    "Thanks to `sy.KerasHook(tf.keras)` you can call the `share` method to transform your model into a TF Encrypted Keras model.\n",
    "\n",
    "If you have asked to manually manage servers above then this step will not complete until they have all been launched. Note that your firewall may ask for Python to accept incoming connection."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "W0902 10:31:37.382000  9740 deprecation_wrapper.py:119] From C:\\Users\\Claudia\\Anaconda3\\lib\\site-packages\\tf_encrypted\\keras\\engine\\base_layer_utils.py:29: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.\n",
      "\n",
      "W0902 10:31:37.384999  9740 deprecation_wrapper.py:119] From C:\\Users\\Claudia\\Anaconda3\\lib\\site-packages\\tf_encrypted\\tensor\\native.py:403: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.\n",
      "\n",
      "W0902 10:31:37.391999  9740 deprecation_wrapper.py:119] From C:\\Users\\Claudia\\Anaconda3\\lib\\site-packages\\tf_encrypted\\tensor\\native.py:101: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.\n",
      "\n",
      "W0902 10:31:37.455000  9740 deprecation.py:506] From C:\\Users\\Claudia\\Anaconda3\\lib\\site-packages\\tf_encrypted\\tensor\\shared.py:62: calling extract_image_patches (from tensorflow.python.ops.array_ops) with ksizes is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "ksizes is deprecated, use sizes instead\n",
      "W0902 10:31:37.575000  9740 deprecation_wrapper.py:119] From C:\\Users\\Claudia\\Anaconda3\\lib\\site-packages\\tf_encrypted\\protocol\\pond\\pond.py:3131: The name tf.Dimension is deprecated. Please use tf.compat.v1.Dimension instead.\n",
      "\n",
      "W0902 10:31:37.605000  9740 deprecation.py:323] From C:\\Users\\Claudia\\Anaconda3\\lib\\site-packages\\tf_encrypted\\protocol\\securenn\\odd_tensor.py:306: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use tf.where in 2.0, which has the same broadcast rule as np.where\n",
      "I0902 10:31:40.427000  9740 tfe.py:46] If not done already, please launch the following command in a terminal on host localhost:4000: 'python -m tf_encrypted.player --config C:\\Users\\Claudia\\AppData\\Local\\Temp\\tfe.config server0'\n",
      "This can be done automatically in a local subprocess by setting `auto_managed=True` when instantiating a TFEWorker.\n",
      "\n",
      "I0902 10:31:40.430000  9740 tfe.py:46] If not done already, please launch the following command in a terminal on host localhost:4001: 'python -m tf_encrypted.player --config C:\\Users\\Claudia\\AppData\\Local\\Temp\\tfe.config server1'\n",
      "This can be done automatically in a local subprocess by setting `auto_managed=True` when instantiating a TFEWorker.\n",
      "\n",
      "I0902 10:31:40.432999  9740 tfe.py:46] If not done already, please launch the following command in a terminal on host localhost:4002: 'python -m tf_encrypted.player --config C:\\Users\\Claudia\\AppData\\Local\\Temp\\tfe.config server2'\n",
      "This can be done automatically in a local subprocess by setting `auto_managed=True` when instantiating a TFEWorker.\n",
      "\n",
      "I0902 10:31:40.435000  9740 session.py:55] Starting session on target 'grpc://localhost:4000' using config graph_options {\n",
      "}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "model.share(alice, bob, carol)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 5: Launch 3 Servers\n",
    "\n",
    "```\n",
    "python -m tf_encrypted.player --config /tmp/tfe.config server0\n",
    "python -m tf_encrypted.player --config /tmp/tfe.config server1\n",
    "python -m tf_encrypted.player --config /tmp/tfe.config server2```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 6: Serve the Model\n",
    "\n",
    "Perfect! Now by calling `model.serve`, your model is ready to provide some private predictions. You can set `num_requests` to set a limit on the number of predictions requests served by the model; if not specified then the model will be served until interrupted."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.serve(num_requests=3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 7: Run the Client\n",
    "\n",
    "At this point open up and run the companion notebook: Section 4b - Encrytped Keras Client"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 8: Shutdown the Servers\n",
    "\n",
    "Once your request limit above, the model will no longer be available for serving requests, but it's still secret shared between the three workers above. You can kill the workers by executing the cell below.\n",
    "\n",
    "**Congratulations** on finishing Part 12: Secure Classification with Syft Keras and TFE!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.shutdown_workers()\n",
    "\n",
    "if not AUTO:\n",
    "    process_ids = !ps aux | grep '[p]ython -m tf_encrypted.player --config /tmp/tfe.config' | awk '{print $2}'\n",
    "    for process_id in process_ids:\n",
    "        !kill {process_id}\n",
    "        print(\"Process ID {id} has been killed.\".format(id=process_id))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Keystone Project - Mix and Match What You've Learned\n",
    "\n",
    "Description: Take two of the concepts you've learned about in this course (Encrypted Computation, Federated Learning, Differential Privacy) and combine them for a use case of your own design. Extra credit if you can get your demo working with [WebSocketWorkers](https://github.com/OpenMined/PySyft/tree/dev/examples/tutorials/advanced/websockets-example-MNIST) instead of VirtualWorkers! Then take your demo or example application, write a blogpost, and share that blogpost in #general-discussion on OpenMined's slack!!!\n",
    "\n",
    "Inspiration:\n",
    "- This Course's Code: https://github.com/Udacity/private-ai\n",
    "- OpenMined's Tutorials: https://github.com/OpenMined/PySyft/tree/dev/examples/tutorials\n",
    "- OpenMined's Blog: https://blog.openmined.org"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}