Assignment.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "43cd5175",
   "metadata": {},
   "source": [
    "# DT8122 - Assignment\n",
    "\n",
    "<span style=\"color:red\">Deadline: 2023 August 15 AoE (Anywhere on Earth)</span>\n",
    "\n",
    "Send a zip file with the notebook both as a .ipynb and as a .pdf file to <a href=\"mailto:dt8122@idi.ntnu.no\">dt8122@idi.ntnu.no</a>. Label the file with your full name.\n",
    "\n",
    "The task is to implement conditional DDPM for MNIST images. Your implementation should take as input a digit and be able to generate 28x28 grayscale handwritten image of said digit. \n",
    "\n",
    "You can add additional cells anywhere in the notebook to make your code more readable.\n",
    "\n",
    "DDPM: https://arxiv.org/abs/2006.11239\n",
    "\n",
    "Classifier-free conditional DDPM: https://arxiv.org/abs/2207.12598\n",
    "\n",
    "The notebook should be run when it is turned in so all plots are visible. All code should be contained in the notebook.\n",
    "\n",
    "\n",
    "### Install necessary libraries\n",
    "Any additional libraries you make use of should be installed in this cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "023ce4f0",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install torch\n",
    "!pip install matplotlib\n",
    "!pip install torchvision"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ea24f9a",
   "metadata": {},
   "source": [
    "### Import libraries\n",
    "All import statements should be contained here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bcbff82c",
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "import math\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import torch\n",
    "\n",
    "from torch.utils.data import DataLoader\n",
    "from torchvision.transforms import Compose, ToTensor, Lambda\n",
    "from torchvision.datasets.mnist import MNIST"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2318a0ab",
   "metadata": {},
   "source": [
    "### Define constants here\n",
    "Constants such as number of epochs, device, and learning rate and other hyperparameters should be defined here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "75d5890e",
   "metadata": {},
   "outputs": [],
   "source": [
    "SEED = 0\n",
    "random.seed(SEED)\n",
    "torch.manual_seed(SEED)\n",
    "\n",
    "batch_size = 64"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "87d608a6",
   "metadata": {},
   "source": [
    "### Functionality for loading and visualizing dataset\n",
    "We have provided some functionality for loading and visualizing the dataset. You may add more cells/functions here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c7233c6b",
   "metadata": {},
   "outputs": [],
   "source": [
    "transform = Compose([\n",
    "    ToTensor(),\n",
    "    Lambda(lambda x: (x - 0.5) * 2)]\n",
    ")\n",
    "dataset = MNIST(\"./datasets\", download=True, train=True, transform=transform)\n",
    "loader = DataLoader(dataset, batch_size, shuffle=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cc6d3ea7",
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "This function plots images in a grid. Input is a Tensor.\n",
    "See show_first_batch to see how it is used.\n",
    "\"\"\"\n",
    "def show_images(images, title=\"\"):\n",
    "    images = images.detach().cpu().numpy()\n",
    "    fig = plt.figure(figsize=(4, 4))\n",
    "    cols = math.ceil(len(images) ** (1 / 2))\n",
    "    rows = math.ceil(len(images) / cols)\n",
    "    for r in range(rows):\n",
    "        for c in range(cols):\n",
    "            idx = cols * r + c\n",
    "            ax = fig.add_subplot(rows, cols, idx + 1)\n",
    "            ax.axis('off')\n",
    "            if idx < len(images):\n",
    "                ax.imshow(images[idx][0], cmap=\"gray\")\n",
    "    fig.suptitle(title, fontsize=18)\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "117b3089",
   "metadata": {},
   "outputs": [],
   "source": [
    "def show_first_batch(loader):\n",
    "    for batch in loader:\n",
    "        show_images(batch[0], \"Images in the first batch\")\n",
    "        break\n",
    "show_first_batch(loader)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fefba337",
   "metadata": {},
   "source": [
    "### Your conditional DDPM implementation should go here\n",
    "This includes functionality for adding noise to the image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b65cf319",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "655a0ba6",
   "metadata": {},
   "source": [
    "### The implementation of the neural network used to estimate the noise should go here\n",
    "The network should make use of both time and context embedding.\n",
    "\n",
    "Any functions/methods used for time and context embeddings should also go here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c6ba223c",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "3e24ed18",
   "metadata": {},
   "source": [
    "### Show that you can generate images before the model is trained\n",
    "This should demonstrate that the backwards pass works. The generated images are expected to be noise."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "544ec0f2",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "129a7103",
   "metadata": {},
   "source": [
    "### Implement training loop\n",
    "Train the model here. There should be some indication of how long the model took to train, both total and per epoch. For good results you will want to train the model for several epochs, but with a good implementation you should expect to see something that looks like digits after only a single epoch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5c500250",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "c49e0dd4",
   "metadata": {},
   "source": [
    "### Train and visualize the model\n",
    "We want to see several generated examples of each digit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ee70fbdf",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}